A school-based intervention designed to provide youth with positive peer relationships that can promote positive behaviors, prevent bullying and victimization, and reduce emotional (mental health) problems and risky behaviors such as alcohol use.
Blueprints: Promising
Crime Solutions: Promising
OJJDP Model Programs: Promising
Mark J. Van Ryzin, Ph.D.
Research Professor at University of Oregon (markv@uoregon.edu)
Research Scientist at Oregon Research Institute (markv@ori.org)
Cary Roseth, Ph.D.
Professor at Michigan State University (croseth@msu.edu)
Dr. David Johnson and Dr. Roger Johnson
University of Minnesota
This program uses the cooperative learning (CL) approach described in Cooperation in the Classroom (Johnson et al., 2013), which is an umbrella term that includes reciprocal teaching, peer tutoring, jigsaw, and other methods where peers work together to maximize one another's learning. The cooperative learning approach ensures the establishment of positive interdependence in group-based learning activities, which can: (a) break down biases and prejudices among students that serve as barriers to social connection, and (b) provide a mechanism by which socially isolated students can establish positive relationships with peers. To achieve both of these ends, the approach asks teachers to incorporate opportunities for positive peer interaction through carefully structured group-based learning activities in school.
This program uses the cooperative learning (CL) approach described in Cooperation in the Classroom (Johnson et al., 2013), which is an umbrella term that includes reciprocal teaching, peer tutoring, jigsaw, and other group-based activities where peers work together to maximize one another's learning. Cooperative learning ensures the establishment of positive interdependence in group-based learning activities, along with individual accountability, explicit coaching in collaborative skills, a high degree of face-to-face interaction, and guided processing of group performance. These structured learning activities: (a) break down biases and prejudices among students that serve as barriers to social connection, and (b) provide a mechanism by which socially isolated students can establish positive relationships with peers. To achieve both of these ends, the approach asks teachers to incorporate opportunities for positive peer interaction through carefully structured group-based learning activities in school. Cooperative learning is essentially a framework within which teachers can apply the principle of positive interdependence to design their own group-based activities using existing curricula and learning materials. As such, this learning strategy may be used by teachers in any subject and as often as desired. In the study certified by Blueprints, students received the intervention over two years (during 7th and 8th grades).
Primary Evidence Base for Certification
Study 1
Compared to students in the control group, students in the treatment group showed:
For risk and protective factors, students in the treatment group relative to those in the control group reported:
Primary Evidence Base for Certification
Of the two studies Blueprints has reviewed, one (Study 1) meets Blueprints evidentiary standards (specificity, evaluation quality, impact, dissemination readiness). The study was done by the developer.
Study 1
Van Ryzin and Roseth (2017, 2018, 2019a, 2019b), Van Ryzin, Roseth, and Biglan (2020), and Van Ryzin, Roseth, and McClure (2020) conducted a cluster randomized controlled trial with 15 middle schools and 1,460-1,890 students to evaluate the program effect on alcohol use, bullying, peer relationships, and prosocial behavior. Outcome measures were assessed using online surveys at four time points (fall and spring of grade seven and fall and spring of grade eight). Using the same sample, Van Ryzin and Roseth (2019a, 2019b, 2021) and Van Ryzin, Roseth, and Biglan (2020) also conducted mediation analyses to determine indirect effects of the program.
Study 1
Van Ryzin, M. J., & Roseth, C. J. (2019a). Effects of cooperative learning on peer relations, empathy, and bullying in middle school. Aggressive Behavior, 5, 643-651. doi:10.1002/ab.21858
Van Ryzin, M. J., & Roseth, C. J. (2019b). Cooperative learning effects on peer relations and alcohol use in middle school. Journal of Applied Developmental Psychology, 64, 101059.
Van Ryzin, M. J., Roseth, C. J., & Biglan, A. (2020). Mediators of effects of cooperative learning on prosocial behavior in middle school. International Journal of Applied Positive Psychology, 5(1), 37-52.
Van Ryzin, M. J., & Roseth, C. J. (2021). The cascading effects of reducing student stress: Cooperative learning as a means to reduce emotional problems and promote academic engagement. Journal of Early Adolescence, 41(5), 700-724. doi:10.1177/0272431620950474
Individual: Favorable attitudes towards drug use, Stress
Peer: Interaction with antisocial peers
School: Low school commitment and attachment
Individual: Skills for social interaction
Peer: Interaction with prosocial peers
*
Risk/Protective Factor was significantly impacted by the program
See also: Cooperative Learning Logic Model (PDF)
Subgroup differences in program effects by race, ethnicity, or gender (coded in binary terms as male/female) or program effects for a sample of a specific race, ethnic, or gender group.
Study 1 (Van Ryzin & Roseth, 2021, 2022) tested for subgroup effects by gender and found equal benefits for boys and girls. Van Ryzin, Roseth, and McClure (2020) also tested for subgroup effects by race and ethnicity and found stronger effects intervention benefits for students of color than for white students.
Sample demographics including race, ethnicity, and gender for Blueprints-certified studies:
The Study 1 sample was 48.2% female and 76.4% White. Other racial-ethnic groups included Hispanic-Latino (14.3%), multiracial (4.2%), and American Indian-Alaska Native (3.5%). Asian, African American, and Native Hawaiian-Pacific Islander were less than 1%.
The training provides teachers with an in-depth understanding of the key design principles that contribute to the success of cooperative learning as an instructional strategy, including:
Every cooperative learning lesson must include all four of these key design principles. During the training, teachers are provided detailed information regarding each design principle, including implementation strategies and examples. Teacher training is conducted using cooperative learning, with teachers in the role of students. Participants experience what well-designed cooperative learning lessons look like, sound like, and feel like from the perspective of the student. These lessons support the learning of core design principles while providing an important experiential benchmark for teachers to support their own implementation in their classrooms.
This training typically is two full days or four half-days, either consecutive or spread across multiple weeks or months.
Source: Washington State Institute for Public Policy
All benefit-cost ratios are the most recent estimates published by The Washington State Institute for Public Policy for Blueprint programs implemented in Washington State. These ratios are based on a) meta-analysis estimates of effect size and b) monetized benefits and calculated costs for programs as delivered in the State of Washington. Caution is recommended in applying these estimates of the benefit-cost ratio to any other state or local area. They are provided as an illustration of the benefit-cost ratio found in one specific state. When feasible, local costs and monetized benefits should be used to calculate expected local benefit-cost ratios. The formula for this calculation can be found on the WSIPP website.
The cost for an initial 2-day in-person training for 25-30 teachers is $1,000/day plus trainer's travel expenses (approximately $1,000).
Training manuals for the teachers cost approximately $35 each.
Not applicable.
Implementation only requires that teachers receive professional development in cooperative learning. Teacher training can take place in any large room in the school (e.g., gymnasium, library, auditorium).
There are no ongoing curricula or materials expenses. Teachers trained in cooperative learning can adapt their own curricula and learning materials.
Middle school teachers incorporate cooperative learning into their regular classrooms. Teachers usually must spend some amount of time adapting their curricula and learning materials to be used with cooperative learning; the amount of time required varies as does the degree and frequency with which teachers use the learning strategy.
No information is available
Depending on how comfortable and confident that teachers become with cooperative learning, additional training can be scheduled at a cost of $1,000/day.
Technical assistance may be scheduled at a cost of $1,000/day.
Administrators can use an established, freely available observation protocol to assess individual teacher fidelity to the design principles of cooperative learning.
Not applicable.
No information is available
No information is available
In this example, a school district trains 25 teachers to implement cooperative learning with 1,250 7th grade students.
| Teacher training (2 days, onsite) | $2,000.00 |
| Trainer travel expenses | $1,000.00 |
| Teacher manuals (25 @ $35) | $875.00 |
| Total One Year Cost | $3,875.00 |
In Year 1, the total cost for implementation would be $3,875 with a per student cost of $3.10.
Cooperative learning is a teaching strategy that can be learned via training provided as part of regular teacher professional development. There is some minimal upfront cost for training and manuals but no required ongoing expenses.
Training in cooperative learning typically occurs during teacher professional development time, and schools typically already budget for teacher professional development. No additional funding streams are required.
Dr. David Johnson and Dr. Roger JohnsonUniversity of Minnesotadawojo40@gmail.com
A school-based intervention designed to provide youth with positive peer relationships that can promote positive behaviors, prevent bullying and victimization, and reduce emotional (mental health) problems and risky behaviors such as alcohol use.
Middle school students.
Subgroup differences in program effects by race, ethnicity, or gender (coded in binary terms as male/female) or program effects for a sample of a specific race, ethnic, or gender group.
Study 1 (Van Ryzin & Roseth, 2021, 2022) tested for subgroup effects by gender and found equal benefits for boys and girls. Van Ryzin, Roseth, and McClure (2020) also tested for subgroup effects by race and ethnicity and found stronger effects intervention benefits for students of color than for white students.
Sample demographics including race, ethnicity, and gender for Blueprints-certified studies:
The Study 1 sample was 48.2% female and 76.4% White. Other racial-ethnic groups included Hispanic-Latino (14.3%), multiracial (4.2%), and American Indian-Alaska Native (3.5%). Asian, African American, and Native Hawaiian-Pacific Islander were less than 1%.
Self-awareness, self-management, social awareness, decision making
Individual: Favorable attitudes towards drug use, Stress
Peer: Interaction with antisocial peers
School: Low school commitment and attachment
Individual: Skills for social interaction
Peer: Interaction with prosocial peers
*Risk/Protective Factor was significantly impacted by the program
This program uses the cooperative learning (CL) approach described in Cooperation in the Classroom (Johnson et al., 2013), which is an umbrella term that includes reciprocal teaching, peer tutoring, jigsaw, and other methods where peers work together to maximize one another's learning. The cooperative learning approach ensures the establishment of positive interdependence in group-based learning activities, which can: (a) break down biases and prejudices among students that serve as barriers to social connection, and (b) provide a mechanism by which socially isolated students can establish positive relationships with peers. To achieve both of these ends, the approach asks teachers to incorporate opportunities for positive peer interaction through carefully structured group-based learning activities in school.
This program uses the cooperative learning (CL) approach described in Cooperation in the Classroom (Johnson et al., 2013), which is an umbrella term that includes reciprocal teaching, peer tutoring, jigsaw, and other group-based activities where peers work together to maximize one another's learning. Cooperative learning ensures the establishment of positive interdependence in group-based learning activities, along with individual accountability, explicit coaching in collaborative skills, a high degree of face-to-face interaction, and guided processing of group performance. These structured learning activities: (a) break down biases and prejudices among students that serve as barriers to social connection, and (b) provide a mechanism by which socially isolated students can establish positive relationships with peers. To achieve both of these ends, the approach asks teachers to incorporate opportunities for positive peer interaction through carefully structured group-based learning activities in school. Cooperative learning is essentially a framework within which teachers can apply the principle of positive interdependence to design their own group-based activities using existing curricula and learning materials. As such, this learning strategy may be used by teachers in any subject and as often as desired. In the study certified by Blueprints, students received the intervention over two years (during 7th and 8th grades).
Increasing students' social contacts and reducing social alienation through cooperative and group-based learning activities puts at-risk youth in contact with low-risk or prosocial youth and interrupts the formation of deviant peer clusters. A key ingredient in cooperative learning is "positive interdependence," in which peers promote the success of one another through mutual assistance, emotional support, and sharing of resources, so that interpersonal attraction and acceptance is increased, peer rejection is reduced, and new friendships are supported that promote prosocial (rather than antisocial) behavioral norms.
Primary Evidence Base for Certification
Of the two studies Blueprints has reviewed, one (Study 1) meets Blueprints evidentiary standards (specificity, evaluation quality, impact, dissemination readiness). The study was done by the developer.
Study 1
Van Ryzin and Roseth (2017, 2018, 2019a, 2019b), Van Ryzin, Roseth, and Biglan (2020), and Van Ryzin, Roseth, and McClure (2020) conducted a cluster randomized controlled trial with 15 middle schools and 1,460-1,890 students to evaluate the program effect on alcohol use, bullying, peer relationships, and prosocial behavior. Outcome measures were assessed using online surveys at four time points (fall and spring of grade seven and fall and spring of grade eight). Using the same sample, Van Ryzin and Roseth (2019a, 2019b, 2021) and Van Ryzin, Roseth, and Biglan (2020) also conducted mediation analyses to determine indirect effects of the program.
Primary Evidence Base for Certification
Study 1
Treatment students, relative to control students, reported lower rates of alcohol use (Van Ryzin & Roseth, 2017, 2019b), emotional problems (Van Ryzin & Roseth, 2018, 2021), and bullying (Van Ryzin & Roseth, 2019a). In addition, students in the treatment group reported higher rates of relatedness (or close relationships with peers) (Van Ryzin & Roseth, 2018, 2019a, 2019b; Van Ryzin, Roseth, & Biglan, 2020) and prosocial behavior (Van Ryzin, Roseth, & Biglan, 2020) than students in the control group.
For risk and protective factors, students in the treatment group relative to those in the control group reported lower rates of willingness to use alcohol and deviant peer affiliation (Van Ryzin & Roseth, 2017, 2019b) and higher cognitive and emotional empathy (Van Ryzin & Roseth, 2019a).
Primary Evidence Base for Certification
Study 1
Compared to students in the control group, students in the treatment group showed:
For risk and protective factors, students in the treatment group relative to those in the control group reported:
Study 1 (Van Ryzin and Roseth, 2019a, 2019b) found significant indirect effects of the program on bullying through affective empathy and peer relatedness and significant indirect effects of the program on alcohol use through peer relatedness and deviant peer affiliation. Van Ryzin, Roseth, and Biglan (2020) found significant indirect effects of the program on prosocial behavior through peer relatedness. Van Ryzin and Roseth (2021) found significant indirect effects of the program on emotional problems and academic engagement through peer relatedness and reduced social stress.
In Study 1, standardized regression coefficients ranged widely but generally showed medium to large effects: .37 to .99 (Van Ryzin & Roseth, 2018), .61 to .69 (Van Ryzin & Roseth, 2017), .14 to .66 (Van Ryzin & Roseth, 2019a), .17 to .18 (Van Ryzin & Roseth, 2019b), and .08 to .51 (Van Ryzin, Roseth, & Biglan, 2020).
One study meets Blueprints standards for high-quality methods with strong evidence of program impact (i.e., "certified" by Blueprints): Study 1 (Van Ryzin and Roseth, 2017, 2018, 2019a, 2019b; Van Ryzin, Roseth, & Biglan, 2020; Van Ryzin, Roseth, & McClure, 2020). The study sample included seventh-grade students.
The study took place in 15 rural middle schools in the Pacific Northwest and compared the treatment group to a business-as-usual control group.
Additional Studies (not certified by Blueprints)
Study 2 (Low & Van Ryzin, 2023):
Low, S., & Van Ryzin, M. J. (2023). Student-centered instruction can build social-emotional skills and peer relations: Findings from a cluster-randomized trial of technology-supported cooperative learning. School Psychology. Advance online publication.
Blueprints: Promising
Crime Solutions: Promising
OJJDP Model Programs: Promising
Mark J. Van Ryzin, Ph.D.
Research Professor at University of Oregon (markv@uoregon.edu)
Research Scientist at Oregon Research Institute (markv@ori.org)
Cary Roseth, Ph.D.
Professor at Michigan State University (croseth@msu.edu)
Van Ryzin, M. J., & Roseth, C. J. (2017). Enlisting peer cooperation in the service of alcohol use prevention in middle school. Child development. Advance online publication. https://doi.org/10.1111/cdev.12981
Van Ryzin, M. J., & Roseth, C. J. (2018). Cooperative learning in middle school: A means to improve peer relations and reduce victimization, bullying, and related outcomes. Journal of Educational Psychology. Advance online publication. http://dx.doi.org/10.1037/edu0000265
Certified
Van Ryzin, M. J., & Roseth, C. J. (2019a). Effects of cooperative learning on peer relations, empathy, and bullying in middle school. Aggressive Behavior, 5, 643-651. doi:10.1002/ab.21858
Certified
Van Ryzin, M. J., & Roseth, C. J. (2019b). Cooperative learning effects on peer relations and alcohol use in middle school. Journal of Applied Developmental Psychology, 64, 101059.
Certified
Van Ryzin, M. J., Roseth, C. J., & Biglan, A. (2020). Mediators of effects of cooperative learning on prosocial behavior in middle school. International Journal of Applied Positive Psychology, 5(1), 37-52.
Certified
Van Ryzin, M. J., & Roseth, C. J. (2021). The cascading effects of reducing student stress: Cooperative learning as a means to reduce emotional problems and promote academic engagement. Journal of Early Adolescence, 41(5), 700-724. doi:10.1177/0272431620950474
Van Ryzin, M. J., & Roseth, C. J. (2022). The longitudinal relationship between peer relations and empathy and their conjoint contribution to reducing bullying in middle school: Findings from a randomized trial of cooperative learning. Journal of Prevention and Health Promotion, 1-19. doi:10.1177/26320770221094032
Van Ryzin, M. J., Roseth, C. J., & McClure, H. (2020). The effects of cooperative learning on peer relations, academic support, and engagement in learning among students of color. The Journal of Educational Research, 113(4), 283-291.
Low, S., & Van Ryzin, M. J. (2023). Student-centered instruction can build social-emotional skills and peer relations: Findings from a cluster-randomized trial of technology-supported cooperative learning. School Psychology. Advance online publication.
Summary
Van Ryzin and Roseth (2017, 2018, 2019a, 2019b), Van Ryzin, Roseth, and Biglan (2020), and Van Ryzin, Roseth, and McClure (2020) conducted a cluster randomized controlled trial with 15 middle schools and 1,460-1,890 students to evaluate the program effect on alcohol use, bullying, peer relationships, and prosocial behavior. Outcome measures were assessed using online surveys at four time points (fall and spring of grade seven and fall and spring of grade eight). Using the same sample, Van Ryzin and Roseth (2019a, 2019b, 2021) and Van Ryzin, Roseth, and Biglan (2020) also conducted mediation analyses to determine indirect effects of the program.
Compared to students in the control group, students in the treatment group showed:
For risk and protective factors, students in the treatment group relative to those in the control group reported:
Evaluation Methodology
Design:
Recruitment: All seventh graders in 15 rural middle schools in the Pacific Northwest were invited to participate in the study. In the initial reports (Van Ryzin & Roseth, 2017, 2018), of the 1,750 eligible seventh-grade students, 1460 (83%) received parental consent to participate in the study. The subsequent reports (Van Ryzin & Roseth, 2019a, 2019b; Van Ryzin, Roseth, & Biglan, 2020) included additional eighth graders who were not part of the original sample. The analysis sample rose to 1,890 students.
Assignment: Schools (n=15) were matched based on size and demographics (e.g., free and reduced-price lunch percentage) and randomly assigned to treatment (n = 7 schools, 668 students) or control (n = 8 schools, 792 students) conditions. [Note that numbers are reversed in Figure 1 of Van Ryzin and Roseth (2017).] The additional students included in Van Ryzin and Roseth (2019a, 2019b) and Van Ryzin, Roseth, and Biglan (2020) increased the numbers in both treatment and control schools, but the study did not provide specifics. Van Ryzin and Roseth (2021) reported having 875 students in the intervention schools and 1,015 students in the control schools.
Assessments/Attrition: The initial reports (Van Ryzin & Roseth, 2017, 2018) examined baseline data from the fall of grade seven and follow-up data from the spring of grade seven. At the follow-up, data were collected on a total of 1,325 students (attrition rate of 9%).
The subsequent reports (Van Ryzin & Roseth, 2019a, 2019b, 2021; Van Ryzin, Roseth, & Biglan, 2020) examined two more waves of data collection, one in the fall of grade eight and one in the spring of grade eight. The authors reported an analysis sample of 1,890 and more than 80% student participation in the data gathering. Van Ryzin and Roseth (2021) reported attrition of 10% at wave 2, 15% at wave 3, and 9% at wave 4. Given the length of the program (i.e., two years), the last follow-up corresponded to a posttest and the earlier follow-ups to interim assessments.
Sample:
Overall, the initial sample was 48.2% female and 76.4% White. Other racial-ethnic groups included Hispanic-Latino (14.3%), multiracial (4.2%), and American Indian-Alaska Native (3.5%). Asian, African American, and Native Hawaiian-Pacific Islander were less than 1%. Approximately 14% were reported as having special education status. Income data were not provided at the student level. The proportion of low-income students served at the school level ranged from 33% to 95%.
Measures:
Bullying and victimization: Van Ryzin and Roseth (2018, 2019a)
Subscales from the University of Illinois Bully Scale were used. Bullying was assessed using five items, including "I teased other students while we were in a group" and "I spread rumors about other students." Alpha reliability was .74-.83 across waves 1-4. Victimization was assessed using three items, including "Other students picked on me" and "Other students made fun of me." Alpha reliability was .93 and .94 at baseline and follow-up, respectively. For both subscales, students responded on a 5-point scale ranging from 0 (Never) to 4 (7 or more times), and items were averaged to arrive at the subscale scores.
Perceived stress: Van Ryzin and Roseth (2018)
Four items from the Perceived Stress Scale were used. Items included "In the last month, how often have you been upset because of something that happened unexpectedly?" and "In the last month, how often have you felt that things were going your way?" (reverse-scored). Students responded on a 5-point scale ranging from 0 (Never) to 4 (Very often). Alpha reliability was .59 at baseline and .63 at follow-up. Items were averaged to arrive at the scale score.
Emotional problems: Van Ryzin and Roseth (2018, 2021)
Three items from the Emotional Problems subscale of the Strengths and Difficulties Questionnaire were used. Items included "I worry a lot" and "I am often unhappy, depressed or tearful." Students responded on a 3-point scale ranging from 1 (Not true) to 3 (Certainly true). Alpha reliability was .71 at baseline and .75 at follow-up. Items were averaged to arrive at the scale score.
Peer Relatedness (or close relationships with peers): Van Ryzin and Roseth (2018, 2019a, 2019b) and Van Ryzin, Roseth, and Biglan (2020)
Four items from the Relatedness Scale were used. Items included "When I'm with my classmates, I feel accepted" and "When I'm with my classmates, I feel unimportant" (reverse-scored). Students responded on a 4-point scale ranging from 1 (Not at all true) to 4 (Very true). Alpha reliability was .71 at baseline and .79 at the first follow-up. Items were averaged to arrive at the scale score.
Engagement (or low school commitment and attachment): Van Ryzin and Roseth (2018)
Four items from the Behavioral Engagement subscale of the Engagement versus Disaffection with Learning Scale were used. Items included "I try hard to focus in class" and "In class, I do just enough to get by" (reverse-scored). Students responded on a 4-point scale ranging from 1 (Not at all true) to 4 (Very true). Alpha reliability was .75 at baseline. Items were averaged to arrive at the scale score.
Alcohol use: Van Ryzin and Roseth (2017, 2019b)
Students reported on their use of alcohol in the last month using the following scale: 1 = no use, 2 = occasionally (1-3 times), 3 = fairly often (4-6 times), 4= regularly (7-9 times), and 5 = all the time (10+ times).
Willingness to use alcohol: Van Ryzin and Roseth (2017)
Students reported on their willingness to use alcohol in the company of friends (e.g., "Suppose you were with a group of friends and there was some alcohol that you could have if you wanted. How willing would you be to have one drink?"). Students responded using the following scale: 1= not at all willing, 2= a little willing, 3= pretty willing, and 4= very willing.
Deviant peer affiliation: Van Ryzin and Roseth (2017, 2019b)
Students reported on the frequency in the last month with which they associated with other youth who engaged in delinquent activities, including "get in trouble a lot," "fight a lot," "take things that don't belong to them," and "skip school" (four items overall). Alpha reliability was .76-.84 across waves 1-3.
Empathy: Van Ryzin and Roseth (2019a)
A cognitive empathy scale used three items from the Basic Empathy Scale (e.g., "I can usually realize quickly when a friend is angry" and "I can often understand how people are feeling even before they tell me"). An affective empathy scale also used three items (e.g., "After being with a friend who is sad about something, I usually feel sad" and "I can often understand how people are feeling even before they tell me"). Alpha reliabilities were between 0.69 and 0.78.
Prosocial Behavior: Van Ryzin, Roseth, and Biglan (2020)
Three items from the Prosocial Behavior subscale of the Strengths and Difficulties Questionnaire were used. Items included "I usually share with others" and "I often offer to help others (parents, teachers, other students)." Alpha reliabilities ranged from .70 to .81 across waves 1-4.
Analysis:
Van Ryzin and Roseth (2017, 2018) used linear mixed multi-level models. In these models, student data (pretests, entered uncentered, and student demographics) were modeled at level 1 while school data (assignment) was modeled at level 2. All individual-level effects were fixed because the authors reported the cluster-level sample size was too small to model school-level random effects. Van Ryzin and Roseth (2017) reported that all models were fit using robust maximum likelihood (RML), which provides Huber-White standard errors. Willingness to use alcohol, which was different between treatment and control at baseline, was included as a covariate. Meanwhile, Van Ryzin and Roseth (2018) reported that interaction terms to represent moderation of intervention effects by baseline levels of marginalization (e.g., low engagement) were included at level 1. Baseline outcome measures were adjusted in each model.
For the analysis of all four waves of data, Van Ryzin and Roseth (2019a, 2019b) and Van Ryzin, Roseth, and Biglan (2020) used linear growth curve models. The Structural Equation Models estimated robust standard errors but given the small number of schools, did not use random effects for schools to adjust for clustering. The sample of 15 schools may not be large enough to accurately estimate robust standard errors, and the result may be to overstate the significance of the tests.
Intent-to-Treat: It appears that all reports included participants in their originally assigned condition regardless of participation in the treatment. Later reports gave only the final analysis sample size of 1,890 without details on completion rates for each assessment. However, it appears from the addition of new students in eighth grade and references to estimation "in the presence of missing data" that the maximum likelihood estimation included participants with partial data.
Outcomes
Implementation Fidelity:
Research staff blind to assignment observed teaching practices in intervention and control schools. Results indicated significantly higher levels of observed positive interdependence in intervention schools compared to control schools.
Baseline Equivalence:
Baseline equivalence was tested only on outcome measures, and there was a significant difference between treatment and control in 1 of the 5 outcomes reported in Van Ryzin and Roseth (2018) (emotional problems) and 1 of the 3 outcomes reported in Van Ryzin and Roseth (2017) (willingness to use alcohol), both in favor of treatment.
Later reports examined only the outcome measures used in the analysis. Van Ryzin and Roseth (2019a) found one significant baseline difference between conditions in four tests. Control students reported higher affective empathy, but the effect was very small (R-squared < .01). Van Ryzin and Roseth (2019b) found one significant baseline difference between conditions in three tests. Control students reported higher deviant peer affiliation, but the effect was small (R-squared < .01). Van Ryzin, Roseth, and Biglan (2020) found no significant baseline differences between conditions in two tests.
Differential Attrition:
Not examined.
Posttest:
Van Ryzin and Roseth (2017, 2018) tested 5 behavioral outcomes (bullying, victimization, emotional problems, close relationship with peers, and alcohol use) and found a significant main effect in favor of treatment on 3 of these outcomes. In addition, the authors assessed 4 risk and protective factors (perceived stress, deviant peer affiliation, engagement, and willingness to use alcohol) and found a significant main effect on 2 of these outcomes. Thus, the studies reported a main effect on 3 of 8 outcomes tested.
At the posttest, Van Ryzin and Roseth (2017) found that treatment students showed lower rates of alcohol use (beta = -.61) and deviant peer affiliation (beta = -.68) than control students. In addition, Van Ryzin and Roseth (2017) reported marginally significant results (p = .05) in favor of treatment for willingness to use alcohol (beta = -.69).
Van Ryzin and Roseth (2018) found that at the posttest, students in the treatment group reported lower rates of emotional problems (ES = .55) and higher rates of relatedness (or close relationship with peers; ES = .43) than students in the control group. In terms of moderation analyses, results showed that marginalized students in the treatment group reported lower rates of bullying, victimization, and perceived stress than marginalized students in the control group.
Two-Year Posttest: Van Ryzin and Roseth (2019a) found that the intervention group reported significantly better scores than the control group on all four outcomes: higher peer relatedness (wave 2), higher cognitive empathy and higher affective empathy (waves 2 and 3), and lower bullying (waves 2, 3, and 4). In addition, mediation tests confirmed the logic model by showing significant indirect effects of the program on bullying through affective empathy and peer relatedness.
Van Ryzin and Roseth (2019b) found that the intervention group reported significantly better scores than the control group on all three outcomes: higher peer relatedness (wave 2), lower deviant peer affiliation (waves 2 and 3), and lower alcohol use (waves 2, 3, and 4). In addition, mediation tests confirmed the logic model by showing significant indirect effects of the program on alcohol use through peer relatedness and deviant peer affiliation.
Van Ryzin, Roseth, and Biglan (2020) found that the intervention group reported significantly better scores than the control group on both outcomes: higher peer relatedness (wave 2) and more prosocial behavior (waves 2, 3, and 4). In addition, mediation tests confirmed the logic model by showing significant indirect effects of the program on prosocial behavior through peer relatedness.
Moderation: Van Ryzin, Roseth, and McClure (2020) presented tests for moderation by race and ethnicity that compared the intervention effects for white students with students of color (Latino. Native American, African American, and multi-racial). The results showed significantly stronger intervention effects on peer relatedness, academic support, and engagement for students of color than for white students.
Mediation: Van Ryzin and Roseth (2021) examined mediation using structural equation models with linear growth curves, fixed effects for schools, and robust standard errors to account for clustering. They first found that the program significantly increased the two outcomes of emotional problems and academic engagement. They further found that the program increased peer relatedness and reduced student stress across the first three waves, which in turn reduced emotional problems and promoted academic engagement in the last wave. The indirect effects of the program via increased peer relatedness and reduced student stress were small but significant for both emotional problems and academic engagement. The mediation was partial, as cooperative learning still had significant effects on emotional problems and academic engagement after controlling for the mediators.
Additional Results. Van Ryzin and Roseth (2022) examined the bidirectional relationship between peer relations and affective empathy but did not compare the intervention and control groups. They found that improved peer relations increased empathy at the same time increased empathy improved peer relations - meaning the two were mutually reinforcing. Together, changes in peer relations and empathy reduced bullying.
Supplemental Runs. In response to requests made by the Blueprints Advisory Board, Van Ryzin provided additional sets of results (May 25, 2023) that: 1) included only the students initially randomized to condition (and not those who entered the schools after randomization), 2) examined program effects on outcomes at wave 4, the end of eighth grade (and excluding data from interim waves at the end of seventh grade and the start of eighth grade), and 3) used binary measures for the highly skewed outcomes of alcohol, tobacco use, and marijuana use. In addition, Van Ryzin offered new details on ICCs for school-level clusters, which ranged from .02-.11, and on the statistical model (robust standard errors were not used).
To summarize the supplemental results, the sample based on originally assigned students showed baseline equivalence for the full randomized sample (one significant condition difference in 15 tests) and for the analysis sample (no significant condition differences in 15 tests). The latter tests indicated little in the way of differential attrition. Additional tests showed no relationship of gender or race/ethnicity to attrition. The statistical models for the sample of originally assigned students showed a significant intervention effect on outcomes at the end of eighth grade in 14 of 16 tests. The results thus demonstrated program efficacy for several risk and protective factors and for several key outcomes such as prosocial behavior (d = .52), anti-social behavior (d = .48), bullying (d = .89), emotional problems (d = .83), academic engagement (d = .52), tobacco use (OR = .44), alcohol use (OR = .56), and marijuana use (OR = .45).
Long-Term:
Not examined.
Study 2 (Low & Van Ryzin, 2023) leveraged an existing web-based software (https://PeerLearning.net) to support teachers in implementing Cooperative Learning with fidelity to essential design components. When considering lesson design, https://PeerLearning.net contains several lesson templates that possess built-in fidelity to key Cooperative Learning design principles. The lesson design process is implemented as a series of wizard-like screens that ask users to (a) enter/select lesson details (title, lesson type, subject, grade, and topic/domain), (b) upload the lesson content (i.e., curricular materials), (c) create prompts for student note-taking through https://PeerLearning.net, (d) add a follow-up activity to the lesson (if desired), (e) insert student info (i.e., a description of the learning goal for students, a plan for assessment, and any group-level bonuses that are offered to enhance positive interdependence), and (f) specify the length of each phase of the lesson to ensure that the lesson can finish on time. The ultimate goal of the software platform is to support teachers in adhering to best practices in Cooperative Learning while also reducing the implementation burden, allowing teachers to provide these learning experiences more frequently and with less time and effort. Training in https://PeerLearning.net for teachers began in the fall and consisted of two independent two-hour sessions roughly two months apart. Teachers in intervention schools were asked to use https://PeerLearning.net to deliver a lesson at least twice per month.
Summary
Low and Van Ryzin (2023) used a cluster randomized controlled trial to compare a sample of 813 students in 12 middle and high schools in the Pacific Northwest assigned to one of two conditions: (1) Cooperative Learning; or (2) control. Assessments at baseline and posttest measured social-emotional skills, prosocial behavior, and supportive peer relations.
Low and Van Ryzin (2023) found at posttest that students in intervention schools, relative to those in control schools, showed significantly greater improvements in:
Evaluation Methodology
Design:
Recruitment: The study took place during the 2021-2022 school year. Twelve middle and high schools in the Pacific Northwest volunteered to participate in the study. The authors noted that they intended to enroll eighth graders in middle schools and ninth graders in high schools, but two middle schools requested that they enroll sixth graders instead. Between 79% and 97% of eligible students in each school were successfully recruited, for an initial sample of 813 students. The authors did not provide information on consent rates by condition, however.
Assignment: Participating schools were matched based upon size, grade level, and demographics (i.e., free/reduced lunch percentage, race/ethnicity) into pairs. Using an online random number generator, one school from each pair was randomized to the intervention condition, with the remaining school being allocated to the control condition. Table 1 indicates that 389 and 424 students were allocated to the intervention and control groups, respectively. Control schools received no training and proceeded with their standard instructional practices.
Assessments/Attrition: Assessments were conducted at baseline and posttest (i.e., six months after baseline). Of the 813 students who responded to the baseline survey, 720 (88.6%) completed the posttest.
Sample:
The analytic sample was 50.2% female and 70.7% White. Other racial/ethnic groups included Hispanic/Latino (18.0%), multiracial (7.5%), and Asian, African American, American Indian/Alaska Native, and Native Hawaiian/Pacific Islander (all <3%).
Measures:
Outcome data came from student self-reports using online surveys. Two behavioral outcomes were computed for analysis: (1) prosocial behavior; and (2) supportive peer relations. The additional five outcomes of self-awareness, self-management, social awareness, relationship skills, and decision making represent risk and protective factors. Alpha reliabilities ranged from .59 to .90.
Analysis:
Analyses used two-level hierarchical linear models with maximum likelihood estimation to account for nesting of students within schools. However, the level-2 sample size of 12 schools may not be large enough to accurately estimate the standard errors, and the result may be to overstate the significance of the tests. Baseline outcomes were included as fixed effects in all models. Cross-level interaction terms were used to evaluate moderation by student characteristics.
Missing Data Methodology: The hierarchical linear models used FIML estimation to include participants with incomplete data.
Intent-to-Treat: All participants with available data (i.e., who completed at least one survey) were used in analyses.
Outcomes
Implementation Fidelity:
For schools in the intervention group, the number of lessons taught per school ranged from 31 to 108 (M = 39.3). Teachers in control schools indicated no use of Cooperative Learning.
Baseline Equivalence:
Tests for baseline equivalence indicated no significant differences between conditions on the seven outcome measures. Whether the tests used the initial or analytic sample is unclear, and no tests were reported for baseline sociodemographic characteristics.
Differential Attrition:
The authors reported that attritors "were not significantly different from the students who participated in Wave 2 in terms of sex, race/ethnicity, or intervention condition" (p. 5). No other information was provided.
Posttest:
Relative to students in schools assigned to the control group, those in intervention schools showed significantly greater improvements across all seven outcomes (ds = .34-.82). Results were unchanged when correcting for the family-wise error rate among the social-emotional learning subscales.
There was no evidence for moderation by sex, race/ethnicity, or grade level.
Long-Term:
Not examined.