Coping Power Universal

Study 1

In this pilot study, both Coping Power Program (CPP)-certified trained psychologists and teachers delivered the program. In later studies, teachers delivered the program with CPP staff support.

Summary

Muratori et al. (2015, 2016) randomized nine classrooms of first- and second-grade students (N=184) into intervention and control conditions and assessed measures of emotional, behavioral, and peer problems at pretest, posttest, and at 12-month follow-up.

Muratori et al. (2015, 2016) found that, relative to students in the control condition, students in the intervention condition showed significantly greater improvements in:

Posttest and 12-month Follow-up

• Overall stress (total problems)
• Hyperactivity/inattention
• Prosocial behavior

12-month Follow-up only

• School grades

Evaluation Methodology

Design:

Recruitment: Two primary schools (grades 1-5) from the Tuscany area of Italy agreed to participate in the study. Within the schools, nine teachers concerned about the lack of social skills and disruptive behavior of many children in their classrooms agreed to participate in the study. The nine classes of the teachers included 184 first- and second-grade students, and all parents consented to the participation and assessment of their children.

Assignment: The nine classrooms were randomly assigned to the intervention and control conditions. The intervention group comprised three first-grade classes and one second-grade class from the Capannori area and one second-grade class from the Lucca area. The control sample comprised two first-grade classes from the Lucca area and two second-grade classes from the Capannori area. The intervention classrooms added the weekly Coping Power Program sessions, while the control classrooms continued their academic curriculum as usual.

Assessments/Attrition: Assessments occurred at baseline, posttest immediately following program completion, and at 12-month follow-up. There was no attrition from baseline to posttest, but there was an 8% (N=22) loss from posttest to the follow-up assessment on behavioral measures and a 23% loss (N=42) at follow-up on academic records.

Sample: The average age of the first- and second-grade students was 7.5. There was a near even split between boys and girls, while 80-84% of the children were Italian. Based on the percentile distribution (e.g., 9.2% has scores over the 90th percentile), the sample appeared typical in terms of overall stress or total problems.

Measures:

All measures came from the teachers who helped deliver the program.

Five Behavioral Measures came from the Strengths and Difficulties Questionnaire (with the Italian translation and norms). The SDQ is a 25-item questionnaire used to assess conduct problems (e.g., bullying), hyperactivity (e.g., squirming), emotional symptoms (e.g., worrying), peer problems (e.g., disliked by other children), and prosocial behavior (e.g., helping). The SDQ-Overall Stress is the total score of the scale indicating general problematic behaviors. The mean Cronbach's alpha was .82.

Academic Performance was assessed using course grades for language arts and mathematics, as obtained from school records.

Analysis: To adjust for the nesting of individuals within classes and the randomization of classes, the analysis used linear mixed model estimation. Tests of the intervention came from coefficients for group-by-time interactions and implicitly controlled for baseline outcomes.

In the follow-up, a mixed factorial ANOVA (three time points x two conditions) was used to test the intervention effect, and a cross-lagged meditational model was used to test the intervention's indirect mechanism of behavioral improvements on academic performance.

Intent-to-Treat: The analysis likely used an intent-to-treat sample, although it did not state that explicitly.

Outcomes

Implementation Fidelity: Using videos of all the intervention sessions of four classes, a certified supervisor completed a checklist about variations from the manual during the three phases included in a session. The mean score of the intervention adherence standard checklist, based on a Likert scale from 1 to 4, was 2.90 though the follow-up lists the mean score as 3.60.

Baseline Equivalence: The study stated that there were no significant baseline differences between conditions on demographic factors or teacher-reported SDQ scales. The intervention group had three first-grade classes compared to two in the control group, but the mean age of the two groups did not differ significantly.

Differential Attrition: There was no attrition at posttest, but 8% of students dropped out between posttest and follow-up. Analyses revealed that dropouts had significantly lower baseline levels of prosocial behaviors. Course grades at 12-month follow-up could only be obtained from 77% of the baseline sample, with no assessment of whether this differed by condition or other sample characteristics.

Posttest: A significant group-by-time interaction emerged for 3 of 6 tests. The intervention group did significantly better than the control group on prosocial behaviors, hyperactivity/inattention problems, and overall stress (or total problems).

Long-Term: At 12-month follow-up, compared to the control group, the intervention group showed a significant improvement in overall stress, hyperactivity/inattention, prosocial behavior, and school grades. Mediation analyses revealed that the intervention's significant reduction of hyperactivity/attention problems accounted for the significant increases in academic performance.

Study 2

Summary

Muratori et al. (2017) randomized 40 classrooms of third- and fourth-grade students (N=901) into the intervention (20 classrooms, 488 students) or control condition (20 classrooms, 413 students) and assessed measures of conduct problems, hyperactivity, emotional symptoms, and overall stress at baseline and posttest.

Muratori et al. (2017) found that intervention students, relative to control students, significantly improved in:

• Conduct problems
• Emotional problems
• Hyperactivity

Evaluation Methodology

Design:

Recruitment: Pubic elementary schools in Lucca, Pisa, and Spoleto Italy were recruited to participate in the study. Six schools agreed to participate, and all third- and fourth-grade classes (n = 40 classes, 901 students) in these schools were selected.

Assignment: The 40 classes were randomly assigned to either treatment or control (20 classes and 488 students in treatment, 20 classes and 413 students in control). The treatment group included 10 third-grade classes and 10 fourth-grade classes, and though not explicitly stated it appeared the control group had the same breakdown.

Assessments/Attrition: Teachers completed assessments at baseline and posttest immediately following program completion. It was reported on p.757 that: "The students with complete data at both times were 841; attrition involved 6.72% of the students."

Sample: Sample characteristics were not reported.

Measures: Teachers delivered the program and also completed the outcome measures. The assessment included the Strengths and Difficulties Questionnaire (SDQ; Goodman, 1997; Tobia et al., 2011), which is a 25-item questionnaire designed to assess the occurrence of particular behaviors that have been associated with conduct problems, hyperactivity, emotional symptoms, peer problems, and prosocial behavior in children aged 4-16. In this study, SDQ-Overall Stress was measured, which is the total score of the four scales indicating general problematic behaviors: 1) conduct problems, 2) hyperactivity, 3) emotional symptoms, and 4) peer problems. In addition to assessing the total scale score, except for peer problems, each scale was also measured separately. Sample alphas were .83 for conduct problems, .86 for hyperactivity, and .84 for emotional symptoms.

Analysis: A 3-level model (which measured occasion within individuals, within classes, and within schools) was conducted with each student outcome using a random-intercept (fit to account for within classes and within schools). There were no controls for demographic variables.

Intent-to-Treat: It appeared that all participants were analyzed according to the condition in which they were assigned and all available data were utilized which is in line with intent-to-treat protocol.

Outcomes

Implementation Fidelity: Interventionists completed a measure of fidelity after completing each intervention session, rating whether they had covered each session objective "completely," "partially," or "not at all." The program leaders indicated that they "completely" (89% child group) or "partially" (11% child group) completed group intervention objectives, indicating a high rate of self-reported intervention fidelity. After each intervention session, teachers also completed a questionnaire that investigated the theoretical and practical preparation of the teachers on the daily session activities and principles. A certified Coping Power supervisor verified the percentage of correct answers: 87% of the answers were corrected in the current study.

Baseline Equivalence: Baseline equivalence was tested only on outcome measures, and it was unclear whether significance tests were conducted (see Table 2).

Differential Attrition: It was reported on p.757 that: "No significant differences were found between those who completed both data collections and those who completed only the baseline evaluation." Attrition-by-condition tests, however, were not conducted.

Posttest: At the posttest, compared to control group students, students in the treatment group were significantly less likely to experience conduct problems and less likely to exhibit hyperactive behaviors and emotional problems.

Long-Term: Not conducted.

Study 3

Summary

Muratori et al. (2019) randomized 70 classrooms of fourth- and fifth-grade students (N=1030) to the intervention condition (35 classrooms, 511 students) or a business-as-usual control (35 classrooms, 519 students). Teachers (who delivered the program) and parents (who were blind to condition) reported on youth conduct problems, hyperactivity, emotional symptoms, peer problems, and pro-social behavior at baseline and posttest.

Muratori et al. (2019) found that, relative to those assigned to the control condition, youth assigned to the intervention condition showed significantly greater improvements in:

• Overall behavioral problems (teacher- and parent-rated)
• Conduct problems (teacher- and parent-rated)
• Hyperactivity (teacher-rated)
• Peer problems (teacher-rated)
• Prosocial behaviors (teacher- and parent-rated)

Evaluation Methodology

Design:

Recruitment: Six schools in a disadvantaged area of an Italian city were selected for the study. The researchers contacted the principals of these schools and all agreed to participate. Inclusion criteria were: (1) the child had to be in fourth or fifth grade; and (2) the child's parents agreed to be involved. No parents asked to exclude their children from the study. The sample comprised 1,030 students attending 70 primary school classrooms in the six schools.

Assignment: Cluster randomization by classroom (35 classrooms assigned to treatment and 35 classrooms assigned to control) was completed by an independent researcher after all participants had completed the baseline assessment. Of the 1,030 youth in the six schools that agreed to participate, 511 attended classrooms assigned to the treatment condition and 519 attended classrooms assigned to the control condition that continued business as usual. The program lasted from November 2016 to April 2017.

Assessments/Attrition: Assessments occurred at baseline, in the fall of the school year, and at posttest, near the end of the school year and about one month after program completion. Complete data at both time points were obtained for 994 students, indicating minimal attrition (< 5%).

Sample: Students were in fourth or fifth grade and had an average age of 9.24 years. Approximately 11% of the sample identified as African American, with the remainder identifying as Caucasian. Thirty-five children had an intellectual disability and eight children had a sensory disability. Teachers involved in the project had a mean number of 10 years of experience with children.

Measures: Teachers (who delivered the program) and parents (who were blind to condition) reported on youth total problem behaviors, conduct problems, hyperactivity, emotional symptoms, peer problems, and prosocial behaviors using the Strengths and Difficulties Questionnaire (SDQ). Alpha reliabilities in the current sample ranged from α = 0.80 (parent-reported emotional symptoms) to α = 0.88 (teacher-reported prosocial behaviors).

Analysis: The researchers tested for intervention effects using four-level linear mixed-effects random-intercept models to account for within-subjects correlations, within-classroom correlations, and within-school correlations. Analyses were conducted using full maximum likelihood estimation, with measurement occasion (Level 1) nested within individuals (Level 2) nested within classrooms (Level 3) nested within schools (Level 4). The time measure included the baseline outcomes, and tests for program effects examined the time-by-experimental group interactions.

Intent-to-Treat: All available data were used in FIML analyses.

Outcomes

Implementation Fidelity: Teachers self-reported that they delivered 88% of the elements of the intervention sessions. They also showed a mean fidelity score of 62 out of 72 based on questions about their understanding of key program principles. Finally, the school psychologist observed recordings of 20% of the sessions, which showed that 84% of the reported levels of adherence were good or excellent.

Baseline Equivalence: The authors only stated that no significant differences were detected in baseline demographics or outcomes.

Differential Attrition: Minimal attrition (< 5%) occurred over the course of the study. In addition, the researchers stated that "no significant differences were found between children who completed evaluations at both time points and those who completed only the baseline evaluation" (p. 15-16).

Posttest: One month after the intervention, relative to control participants, children in the intervention condition showed significantly greater improvements in five of six non-independent teacher-rated outcomes (youth total problem behaviors, conduct problems, hyperactivity, peer problems, and prosocial behavior) and three of six independent parent-rated outcomes (youth total problem behaviors, conduct problems, prosocial behavior).

Long-Term: Not conducted.

Study 4

This study examined a middle school version of the universal program. It relied on the same theoretical model, principles, and techniques of the universal program for primary schools, but the activities were modified to be developmentally appropriate for the older age group.

Summary

Muratori et al. (2020) conducted a cluster randomized controlled trial to examine 40 middle-school classrooms with 839 students in Italian urban schools. The classrooms were assigned to those receiving the program and those receiving the usual school curriculum. A posttest assessment at the end of the school year measured externalizing, internalizing, and prosocial behavior.

Muratori et al. (2020) found that students in the intervention classrooms, relative to students in the control classrooms, showed significantly greater improvement in

• Parent-reported externalizing
• Parent-reported internalizing
• Parent-reported prosocial behavior
• Peacher-reported internalizing
• Teacher-reported prosocial behavior

Evaluation Methodology

Design:

Recruitment: The sample included 839 students attending 40 middle school classrooms located in Italian urban schools. The students were in seventh and eighth grade.

Assignment: An independent researcher randomized the 40 classrooms. Half received the program (n = 497 students) and half received the usual personal, social, and academic curriculum taught in Italian middle schools (n = 342). The randomization procedure was carried out after teachers and parents had completed the baseline evaluation.

Assessments/Attrition: The program lasted from November 2017 to April 2018, and assessments occurred at baseline in September 2017 and posttest in May 2018. According to Table 2, 95.9% of the students had complete data from teachers, while 87.0% of the students had complete data from parents.

Sample:

The sample consisted of 49% males and had an average age of 13.24 years. About 11% were Africans and 89% were Caucasian.

Measures:

Outcome measures came from classroom teachers, who delivered the program, and from parents, who were not informed of the randomly assigned condition of their children's classrooms. The six outcomes, three each from teachers and parents, included subscales of the Strengths and Difficulties Questionnaire, Italian validated version, relating to internalizing (alphas = .81 and .83), externalizing (alphas = .82 and .82), and prosocial behavior (alphas = .85 and .88).

Analysis:

The analysis used linear mixed-effects models with full-information maximum likelihood estimation. The models featured four levels (measurement occasion within individuals within classes within schools). The estimates adjusted for clustering within the 40 classrooms, the unit of assignment. Tests focused on condition differences in the change across time of the outcomes.

Missing Data Method: The analysis used FIML estimation, but it appears from Table 2 that only those with complete data were included.

Intent-to-Treat: The analysis used all participants with complete data.

Outcomes

Implementation Fidelity:

Teachers implementing the program completed a questionnaire after each session that assessed their knowledge of the principles of the lesson activities. Supervisors rated the questionnaires, giving teachers a mean rating of 66 out of 72.

Baseline Equivalence:

The authors stated the following: "Although the control group appears to have been less impaired (see Table 2), there are no significant differences between experimental and control groups in Strengths and Difficulties Questionnaire baseline-scores, children's age, teachers' age, number of females, and the number of children with disabilities." Table 2 reports figures for the analysis sample, thus implying that the tests were also done for the analysis sample.

Differential Attrition:

The study conducted no direct tests for attrition, but the tests for baseline equivalence using the analysis sample suggest little potential for bias.

Posttest:

For the non-independent measures from teachers, the results indicated significant interactions of group by time for internalizing (d = .17) and prosocial behavior (d = .36). For the independent measures from parents, the results indicated significant interactions for group by time for all three outcomes of internalizing (d = .23), externalizing (d = .14), and prosocial behaviors (d = .28).

Long-Term:

Not examined.

Study 5

Summary

Levantini et al. (2021) conducted a cluster randomized controlled trial to examine 316 third-grade students in 16 classrooms in two Italian schools. The classrooms were randomly assigned to an intervention condition or a school-as-usual condition. Assessments at posttest and one-year after posttest measured problem behavior, internalizing, externalizing, and prosocial behavior.

Levantini et al. (2021) found that students in the intervention classrooms, relative to students in the control classrooms, showed significantly greater improvement over time on non-independent teacher ratings of

• Problem behavior
• Internalizing
• Externalizing
• Prosocial behavior.

Evaluation Methodology

Design:

Recruitment: The authors reported only that 316 students in 16 third-grade classrooms in two public schools in Italy participated in the study.

Assignment: The study randomly assigned the 16 classrooms to either the program condition (n = 8) or the control condition (n = 8). The random allocation sequence was computer-generated. Control classrooms followed the standard curriculum activities provided in Italian schools.

Assessments/Attrition: Assessments occurred before the intervention (November 2017), after the intervention (May 2018), and approximately one year later (May 2019). The authors did not report attrition rates, and the 316 students listed as participants may be the analyzed sample rather than the randomized sample.

Sample:

The sample consisted of 54% females and averaged 8.45 years of age. About 87% of the students were Caucasian, 10% were Africans, and 3% were of other races.

Measures:

Teachers both delivered the program and rated students. Although teachers completing the follow-up one year after the program end may have been unaware of the original assignment, the authors stated the opposite: "the SDQ was completed by the same teachers who delivered the intervention." The study therefore did not appear to have any independent outcome measures.

The four teacher-rated outcomes came from the Italian version of the Strengths and Difficulties Questionnaire. The subscales included prosocial behavior, overall difficulties scores, internalizing problems (i.e., emotional difficulties, peer problems), and externalizing problems (i.e., conduct problems, hyperactivity). Cronbach's alphas were .80 for overall difficulties, .82 for internalizing, .81 for externalizing, and .85 for prosocial behavior.

Analysis:

The analysis used linear mixed-effects models with full-information maximum likelihood estimation and three levels (measurement occasion within individuals within classes). The random-intercept models thus accounted for within-subject correlations and within-class correlations. Tests for the interaction of condition-by-time adjusted for a heterogeneous first-order autoregressive covariance structure. A Bonferroni correction was applied to adjust for multiple comparisons. However, the level-2 sample size of 16 may not be large enough to accurately estimate the standard errors, and the result may be to overstate the significance of the tests.

Missing Data Method. The analysis used FIML, but the study lacked details on missing data and attrition.

Intent-to-Treat: Unclear given the lack of details on sample sizes at each time point.

Outcomes

Implementation Fidelity:

A school psychologist used a checklist to monitor whether the main lesson elements were delivered as intended. The checklists revealed that 87% of the elements of the intervention sessions were delivered.

Baseline Equivalence:

Table 1 lists outcome means by condition but without sample sizes or significance tests.

Differential Attrition:

No information.

Posttest and Long-Term:

All four of the outcomes had significant time-by-condition interaction terms that demonstrated greater improvement in the intervention condition relative to the control condition for total problems, prosocial behavior, internalizing, and externalizing. The analysis combined results from the posttest and follow-up, and additional checks compared changes from posttest to follow-up within conditions but not across conditions. The authors noted from the within-condition comparisons that intervention condition "improvements in internalizing and externalizing problems were not maintained at the follow-up."

Moderation analyses found that high baseline levels of teachers' occupational stress predicted poorer outcomes of the program.