The benefits

"It's a nice culmination of what they learn about behavior change," says Rogers. Using Amazon Mechanical Turk, 2,000 online workers are recruited and randomly assigned to read different groups' messages. The results are analyzed and shared within 48 hours. "Students get to apply it, find out whether it worked, see how other groups applied the same material, all the while experiencing a fast-cycle randomized experiment."

The challenges

Facilitating class discussion requires some real-time work. In think-pair-share brainstorms, Rogers moves throughout the room, listening and coaching, so that he can frame the subsequent bigger group discussion around ideas that will appropriately reinforce the intended concepts: "It ensures credible, relevant applications that demonstrate students' distributed expertise. In other words, I try to avoid bringing the bill to the floor until I know I have the votes."

Takeaways and best practices

• Use specific prompts to yield better examples. 
  In the same think-pair-share exercise, Rogers used to ask students to apply one behavioral science principle to any issue they really cared about, but found that such breadth could lead students to miss the mark in their applications of the material. "It undermined other students' understanding, and sometimes caused some embarrassment." Instead, he now often formulates specific tasks, for example, "how does this apply to medication adherence?" rather than "how does this apply to healthcare?"
• Label exemplary work as such—not the norm. 
  In an effort to decrease the discouraged by peer excellence effect, Rogers is careful when sharing excellent student examples to stress that the work is exemplary, not representative. He is considering creating sample work at different grade levels (e.g., a B+ vs. an A- paper) for students' reference.  
• Facilitate a 'round of complimenting.' 
  At the conclusion of the randomized experiment, students present their work and receive notes on index cards from peers stating what they liked about their approach. The act incentivizes them to really listen and engage in other students' presentations. Hearing how each group applied the same material also "brings to life abstract principles that each of us can operationalize in different ways to different effects."

Bottom line 

The orchestration of these activities and discussions helps students learn the principles of human thinking and behavior change, and the important role of randomized experiments in making inferences about causal relationships.