Coping Power

Study 1

Summary

Lochman and Wells (2002a, 2003) and Lochman et al. (2013) used a randomized controlled trial to examine the effects of Coping Power in comparison to, and in combination with, a universal intervention program (Coping with Middle School Transitions). It randomized 245 high-risk students in 60 classrooms and 17 schools to a universal intervention or universal control condition and then to indicated intervention or indicated control groups. Assessments measuring substance use, self-regulation, delinquency, and aggression were conducted through a three-year post-intervention follow-up.

Lochman and Wells (2002a, 2003) and Lochman et al. (2013) found that, compared to the control group, children in the indicated intervention showed significantly:

• Larger reductions in parent-rated proactive aggressive behavior (posttest)
• Larger increases in teacher-rated behavioral improvement (posttest)
• More improvement in teacher-rated social skills (posttest)
• Lower self-reported delinquency rates (one-year follow-up)
• Less self-reported substance use (one-year follow-up)
• Lower teacher-rated aggression (three-year follow-up).

Evaluation Methodology

Design:

Recruitment: The sample came from 60 fifth-grade classrooms in 17 elementary schools in one Southeastern city. Fourth-grade teachers in the 17 schools were asked to rate all the children in their classes on a scale indicating aggressive behavior. Based on the ratings, the 31% most aggressive male and female children were identified, producing 473 children in the pool of potential risk subjects. Consent to participate in the study was obtained from the parents of 65% of the children in the pool, leaving a sample size of 245 moderate- to high-risk indicated subjects.

Assignment: This randomized controlled trial used two randomizations. First, classrooms were randomly assigned to either the Universal Intervention (n = 31) or Universal Comparison (n = 29) conditions. All parents with children in the Universal Intervention classrooms were invited to three parent sessions in fifth grade and a single parent session in sixth grade.

Within all 60 classrooms, the indicated children (approximately four per classroom) were further assigned randomly to the Indicated Intervention or Indicated Control conditions. The Indicated Intervention corresponds to the usual Coping Power Program.

The classroom and student randomizations produced four conditions:

• Universal Intervention classroom and Indicated Intervention student (n = 61),
• Universal Intervention classroom and Indicated Control student (n = 62),
• Universal Comparison classroom and Indicated Intervention student (n = 59), and
• Universal Comparison classroom and Indicated Control student (n = 63).

Assessments/Attrition: In Lochman and Wells (2002a, 2003), assessments occurred at baseline (Time 1), mid-intervention between the fifth and sixth grade (Time 2), post-intervention at the end of sixth grade (Time 3), and one-year follow-up at the end of seventh grade (Time 4). At posttest (Lochman & Wells, 2002a), the summer interview battery (child self-report and parent report measures) was administered to 214 participants (87.3% of the original sample) across all three time periods. Teacher measures collected at both mid- and post-intervention were available on 187 participants (76.3% of the original sample), and teacher measures collected as part of a full classroom assessment across the three time points were available on 125 participants (51.0% of the original sample). At one-year follow-up (Lochman and Wells, 2003), 82.0% of the students and 65.3% of teachers completed the assessment.

Sample Characteristics: The indicated sample was comprised of boys to girls in a 2-to-1 ratio. The ethnic distribution in the sample primarily consisted of African American (78%) and Caucasian (21%) children.

Measures: The outcome measures came from parents, teachers, and children and were individually administered to primary caretakers and children by research staff who were blind to the children's condition assignment. Parents helped deliver the parenting program to children. Teachers also helped deliver the Universal Intervention program, but the authors stated that "different teachers completed assessments at each time point, and the teachers at Time 3 (in sixth grade) had not been involved as active intervention delivery agents" (Lochman & Wells, 2002a, p. S44). It appears, however, that their involvement in the program would affect the Time 2 assessment.

Lochman and Wells (2002a) examined 23 outcome measures across seven domains (see Table 1): 1) student-reported substance use, 2) parent- and teacher-reported aggressive behavior, 3) student-reported internal-behavioral self-regulation, 4) teacher- and student-reported social behavior, 5) teacher- and student-reported social cognition, 6) parent- and student-reported perceived school bonding, and 7) parent-reported parenting practices. The measures came from validated scales, but some measures showed low reliabilities for the study sample.

Lochman and Wells (2003) examined three outcome measures: self-reported delinquency, self-reported substance use, and teacher-rated aggressive behavior. Teacher ratings were expected to be relatively free of bias as they were not informed of children's intervention status during the follow-up year. The measures came from validated scales but reliability information for the current sample was not provided.

Analysis: Lochman and Wells (2002a) first used multivariate analysis of variance with repeated measures to test for significant differences in group-by-time interaction terms across multiple outcome measures within the six domains having more than one outcome (i.e., all except substance use). The authors then used univariate analysis of variance with repeated measures and time-by-group interaction terms for outcomes showing trend-level or significant effects as part of the multivariate analysis. The repeated measures models included baseline controls for all but four of the outcomes, which were not assessed at baseline. The interaction terms examined differences in the time trend between the control group and 1) the indicated student intervention, 2) the universal classroom intervention, and 3) the combination of the indicated student and universal classroom intervention.

Lochman and Wells (2003) used analysis of variance across four time points for delinquency and three time points for substance use and aggression (because baseline measures were not available).

The analyses did not adjust for classroom clustering, despite assigning classrooms to conditions. Evaluating the classroom conditions would require some adjustment for clustering, but the authors used ANOVA models with complete cases and a control for the baseline outcome. However, when focusing only on the indicated intervention, which was based on individual assignment within classrooms, clustering should not be a major problem. The intervention and control children were part of the same classroom.

Missing Data Methods: The analysis of variance models in Lochman and Wells (2002a, 2003) used complete cases, without imputation or FIML.

Intent-to-Treat: The analysis in Lochman and Wells (2002a, 2003) used all participants with complete data at all three time points and excluded only those with missing data.

Outcomes

Implementation Fidelity:

For the Universal Intervention, overall teacher attendance at the Universal Teacher Meetings was 63%. Of the 672 parents randomly assigned to the Universal Parent Meeting Intervention, 141 (21%) attended at least one meeting. For the Indicated Intervention, attendance for the Coping Power Child Component sessions was 84% across the 16 months (85% in fifth grade, 83% in sixth grade). The Coping Power target children received a mean of 6.0 individual child contacts across the 16 months. Attendance for the Coping Power Parent Component was 26% (29% in fifth grade, 21% in sixth grade), with 62% of parents attending at least one of the sessions. In addition to their group contacts, the Coping Power target parents received a mean of 6.5 individual contacts from Coping Power staff over the 16-month period.

Baseline equivalence:

For the full randomized sample, Lochman and Wells (2003) reported no significant condition differences in aggressive behavior at screening, teacher-estimated cognitive functioning, gender, or race.

For the analysis sample, Lochman and Wells (2002a) reported tests for the baseline outcome measures. Of the 23 dependent variables, there was one significant baseline difference between conditions for lack of parental supportiveness, with parents in the combined indicated universal condition having higher baseline levels of non-supportiveness than the other three conditions. Because of the baseline differences for this variable, it was included as a covariate in the analysis of lack of parental supportiveness.

Differential Attrition:

Lochman and Wells (2002a) did not report attrition rates by condition but performed several tests. First, they used the main effects test with attrition (attrited, non-attrited) as the independent variable and four baseline measures (children's aggression score at screening, teachers' estimate of children's cognitive abilities at screening, gender, and race) as dependent variables. They stated that there were no significant differences between attrited and non-attrited participants on these baseline measures for (a) child self-report and parent report data; (b) the teacher ratings in Years 2 and 3; or (c) the teacher ratings of all students in their classrooms. Second, the tests for baseline equivalence across conditions for the analysis sample showed little evidence of condition differences in potential attrition bias.

Lochman and Wells (2003) also used the main effects tests for attrition at the one-year follow-up. They stated that there were no significant differences between attrited and non-attrited participants on the three outcomes for students or teachers.

Posttest:

Lochman and Wells (2002a): Of the tests for 23 outcomes and three contrasts (69 in total), six were significant for one of the three interaction effects (two had Universal-by-Time, two had Indicated-by-Time, and two had Universal-by-Indicated-by-Time). First, student-reported substance use and student-reported social competence were significantly improved for all three interventions combined compared to the control condition (i.e., Universal-by-Indicated-by-Time). Second, students in the indicated intervention had significantly better teacher-rated behavioral improvement and teacher-rated social skills than those in the control condition, regardless of classroom (i.e., Indicated-by-Time). Third, students in universal classrooms showed significantly smaller decreases in student-rated fear and larger reductions in teacher-rated social relationship problems relative to those in other classrooms, regardless of the indicated intervention (i.e., Universal-by-Time).

Long-Term

Lochman and Wells (2003): At the one-year follow-up, the intervention significantly lowered self-reported delinquency (d = .27), self-reported substance use (d = .58), and teacher-rated aggressive behavior (d = .35) relative to the control condition. Post hoc tests across the three intervention conditions generally showed greater benefits for the indicated intervention and the indicated intervention combined with the universal classroom intervention.

Moderation tests found that the Coping Power indicated intervention had stronger effects on substance use for older children and for children with moderate baseline aggression.

Three-Year Follow-Up (Lochman et al., 2013): Along with the additional two years of data it examined, this study of the indicated treatment sample added geocodes for residence and measures of neighborhood characteristics. The authors hypothesized that the Coping Power program would have stronger effects in less disadvantaged and more socially organized neighborhoods, where problems won't overwhelm the influence of the intervention.

The study did not examine the classroom program but instead collapsed the conditions into the Coping Power indicated intervention group (n = 120) and the control group (n = 125).

Attrition. Retention rates for parent and child assessments were 91% at Time 2, 88% at Time 3, 83% at Time 4, 82% at Time 5, and 86% at Time 6; retention rates for teacher assessments were 84% at Time 2, 69% at Time 3, 64% at Time 4, 65% at Time 5, 60% at Time 6, and 68% at Time 7.

Measures. The study focused on four outcome measures that partially overlap with measures used in earlier studies. Teachers providing ratings after the end of the program likely would be unaware of children's condition assignment. The measures, all with generally good reliabilities, included the:

• Teacher Observation of Classroom Adaption-Revised (TOCA-R) for aggression and academic behavior problems,
• Reduction of Aversive Treatment subscale of the Outcome Expectation Questionnaire, which measures expectations of the children that aggressive behavior will lead to good outcomes, and
• Lack of Support subscale from the Family Relations Scale, which measures low parental warmth and low positive involvement with children.

Two neighborhood measures included 1) a scale based on nine items from the Community and Neighborhood Measure that assesses parents' baseline perceptions of belonging to and support from the neighborhood, and 2) a disadvantage scale that combined census measures such as poverty, unemployment, and owner-occupied housing for the subjects' neighborhood of residence.

Analysis. Hierarchical Linear Modeling (HLM) was used with time as the first level, student as the second level, and neighborhood as the third level. The models included the condition indicator and five baseline covariates (i.e., family socioeconomic status, maternal depression, parental discipline avoidance, children's perceived deviant peer norms, children's school bonding) at level 2 and allowed the level-3 neighborhood measures to moderate the condition effect.

Missing Data Method: The analysis used FIML estimation with HLM to include participants with missing data.

Intent to Treat: The FIML estimation allowed for the analysis to include all but two participants who were dropped from analyses on the basis of living in census tracts isolated from the remainder of the sample.

Baseline Equivalence. Tests for baseline equivalence confirmed earlier reports by finding no condition differences in baseline aggressive behavior at screening, teacher-estimated cognitive functioning, gender, or race.

Differential Attrition. In logistic regressions with attrition (yes or no) as the outcome, only five of 65 tests of possible attrition were significant (i.e., tests were done for five child characteristics by six parent-child assessment points plus five child characteristics by seven teacher assessment points). The five reaching significance revealed no systematic pattern of bias across time.

Results. The Coping Power program significantly improved the growth curve of the intervention group relative to the control group for all four outcomes: teacher-rated aggression, teacher-rated academic behavior problems, self-reported expectations of benefits from aggression, and parent-reported supportiveness towards the child. The benefits held with controls for neighborhood disadvantage and neighborhood social organization.

Tests of moderation showed that neighborhood characteristics significantly interacted with the program in two of eight tests, but in the opposite direction. On one hand, the program did more to reduce the lack of parental support in better neighborhoods with low disadvantage. On the other hand, the program did more to reduce teacher-rated aggressiveness in worse neighborhoods with low social organization. The lack of a consistent pattern of moderation suggests that the program works across diverse neighborhoods.

Study 2

Summary

Lochman and Wells (2002b, 2004) used a randomized controlled trial with a one-year post-intervention follow-up to compare the effects of the Coping Power program to that of the Child Component alone. It randomized 183 boys in 11 schools, scoring in the top 22% on aggression ratings, to a control group, the child intervention (CI), or the child-parent intervention (CPI). There were 33 child sessions and 16 parent sessions and assessments measuring delinquency, substance use, and school behavior were completed at baseline, posttest, and one-year post-intervention follow-up.

Lochman and Wells (2002b, 2004) found that, compared to the control group, at the posttest, children in the child-parent intervention (CPI) showed significant:

• Reductions in self-reported delinquent behavior
• Reductions in parent-rated substance use
• Teacher-rated behavioral improvement at school.

Evaluation Methodology:

Design: This publication reports on an efficacy study conducted prior to Study 1 described above. Fourth- and fifth-grade boys were screened and selected for potential involvement in the study and were retained in the high-risk pool if they surpassed cutoffs on detailed behavioral measures on the Aggression subscale from the Achenbach Teacher Report Form, as rated by teachers and parents. Eighty-four fourth and fifth grade teachers in 11 elementary schools completed the screening process for Cohort 1 and 86 fourth and fifth grade teachers in 12 elementary schools completed the screening process for Cohort 2. The sample (N = 183) were in the top 22% of boys in teachers' ratings of children's aggressive and disruptive behaviors. Among the three risk components, there were no significant differences in ethnicity, grade level, cognitive ability, or screen score among those who consented to participate (59% of those approached) and those who did not. The 183 boys in both cohorts were randomly assigned to the child intervention only condition (CI) (N = 60), the child plus parent intervention condition (CPI) (N = 60), or the control condition (C) (N = 63). Both cohorts were assessed at three times: pre-testing (Time 1), post-intervention (Time 2), and a one-year follow-up (Time 3). Time 3 assessments were collected two summers after intervention (when boys had completed either sixth or seventh grade).

Intervention occurred in grades 5 and 6. Thirty-three structured group sessions were completed by 29 children groups across the 2 cohorts (fifth and sixth grades). Sixty parents from the 2 cohorts were assigned to 11 parent groups, and each of these groups met for 16 sessions over the 15 month intervention period. Attendance rates at child group sessions was 83% and attendance at parent group was 49%.

The child component consisted of a 1.25 year intervention conducted in the school setting. There were 33 weekly group sessions which lasted 40-60 minutes per session. Eight intervention sessions were held during the first intervention year and 25 were held in the second year, with sessions being somewhat shorter in the second year due to shorter class periods. Group sessions consisted of four to six boys and were co-led by a school-family program specialist and a school guidance counselor. Each boy also received approximately 12 individual ½ hour sessions.

The parent component consisted of 16 sessions, over the same 15 month intervention period. This intervention was delivered in groups of four to six single parents and/or couples and was led by two grant staff persons. Groups began with weekly meetings that were gradually reduced over the year to biweekly and then monthly sessions. Child care and transportation were provided to those in need. Parents also received stress management training in two of the sessions. The content of the child and parent components were synchronized so that the social-cognitive and problem-solving skills were introduced at the same time.

Sample Characteristics: Because funding was not sufficient to obtain a large enough sample to accurately analyze gender differences, only boys were selected for inclusion in the study. A normative sample of 63 boys was representative of the non-risk portion of the population. These boys were in the lower 78% assessed for aggression-disruptiveness. This sample was 64% fourth graders and 36% fifth graders and 47% African American and 53% White. 55% of those receiving one of the two intervention groups were fourth graders, 45% were fifth graders, and 38% were White.

Measures: Time 3 outcomes were multisource. Measures used in the analysis included self-report information (delinquency, via the National Youth Survey), and teacher and parent ratings (school behavior improvement and substance use, respectively). The substance use outcomes included cigarette, alcohol, and marijuana use and were measured using the National Youth Survey as well as parental reports. Teacher ratings of children's behavioral improvement covering the follow-up year were blind raters. There were four Time 3 outcome measures and nine Time 2 intervention change variables. There were two measures of social cognitive processes: the Attributional Measure and the Outcome Expectation Questionnaire, and two measures of schematic beliefs: the Multidimensional Locus of Control scale and the Object Representation inventory. Parenting processes were assessed with the Alabama Parenting Questionnaire, providing subscales for assessing Inconsistent Discipline and Maternal Involvement. The measures targeted intervention change variables that could serve as variables mediating intervention-outcome relations.

Analysis: Intervention effects were tested with ANOVAs and ANCOVAs using general linear models. Dependent variables were the Time 3 outcome variables. When baseline scores were available, they were used as a covariate in the ANCOVA; when baseline scores were not available, ANOVAs were conducted. The initial analysis for each dependent variable examined main effects for the four independent variables of Intervention, Ethnic Status, Grade level during intervention, and Screening Status Level, and the three interaction effects for Intervention X Ethnic Status, Intervention X Grade, and Intervention X Screening Status. If a significant or trend main effect for Intervention existed, then two planned comparisons were conducted to examine any differences between the Child Intervention (CI)and Control condition (C) and the Child plus Parent Intervention (CPI) to the Control condition (C).

Outcomes

There was 12.7% attrition in the Normative group by Time 3, with no significant differences between attrited and non-attrited participants on aggression, race, or cognitive abilities. There was 30%, 31%, and 27% attrition for intervention boys, parents, and teacher reports, respectively. More whites attrited than non-whites. Baseline equivalence analyses conducted for all individuals with Time 3 data, rather than the original sample, revealed no significant differences between conditions on cognitive abilities, ethnicity, grade level, and the dependent variables.

Delinquency: The results from the MANCOVA on self-reported covert delinquency revealed that the Intervention condition had greater reductions in delinquent behavior than did the Control group. When the planned Intervention cell comparisons were conducted, however, the CI condition was not significantly different from the C condition, while the CPI condition produced significantly greater reductions in delinquency in comparison to the C condition. There were no significant intervention effects on overt delinquency.

Substance Use: There were no intervention results for self-reported substance use. The ANOVA for parent-reported alcohol and marijuana use indicated that Intervention boys had lower rates of substance use at the Time 3 follow-up than did the Control boys. A significant interaction effect for Intervention X Ethnic Status was also significant, indicating that intervention white boys had lower rates of substance use than did white boys in the C condition. The substance use rates for intervention minority boys and control minority boys were relatively similar, and were both lower than the cells for white boys. A planned ANOVA using Income Level in place of Ethnic Status revealed a trend toward significance for the interaction effect for Intervention X Income Level with Intervention effects on substance use evident for higher income youth but not for lower income youth. When the planned Intervention cell comparisons were conducted, the CI condition was not significantly different from the C condition, while the CPI condition produced significantly greater reductions in parent-rated substance use than did the C condition.

School Behavior: The ANOVA for teacher-rated behavioral improvement at school indicated that Intervention boys demonstrated greater improvement during the Follow-Up year than did the Control boys. The interaction effect for Intervention X Ethnic Status was also significant, indicating that white boys in the CI condition had higher ratings of behavioral improvement at school than did white boys in the C condition. The ratings for minority boys in the CI and C conditions were relatively similar, and both were between the ratings for the white CI and C boys. When the planned Intervention cell comparisons were conducted, the CI condition had greater behavioral improvement than did the C condition, and the CPI condition had greater behavioral improvement than did the C condition.

Intervention Effects on the Targeted Intervention Change Variables at Post Intervention: These intervention effects were tested with ANCOVAs using general linear models with the Time 2 targeted intervention change variables as dependent variables.

Social-Cognitive Processes: The ANCOVA revealed that the Intervention boys tended to have greater reductions in attributional biases and anger than did the boys in the Control condition. When the planned Intervention cell comparisons were conducted, the CPI condition was not significantly different from the C condition, but the CI condition produced a greater reduction in attributional biases and anger than did the C condition. The interaction effect for Intervention X Grade was significant, indicating that boys in the Intervention in the fourth-fifth grades had greater changes in their expectations that aggression would not have good outcomes than did boys in the Control condition. The planned Intervention cell comparisons indicated that both the CI and the CPI conditions tended to have less optimistic expectations that aggression would lead to useful tangible outcomes in comparison to the C condition.

Schema: Intervention boys had greater increases in their internal locus of control for producing successful outcomes than did the Control boys. None of the interaction effects with Intervention were significant. The planned Intervention cell comparisons indicated that both the CI and the CPI conditions had greater increases in internal locus of control in comparison to the C condition. The Intervention boys demonstrated greater improvements in person perception than did the Control condition. Again, none of the interaction effects with Intervention were significant. The planned Intervention cell contrasts indicated that the CPI condition had greater improvements in person perception in comparison to the C condition, while the improvements for the CI condition were not significant.

Parenting Behaviors: No Intervention effects were obtained for parent or child reported parental involvement, or for parent reports of inconsistent discipline.

Normative Comparisons for Intervention Outcome Effects: In order to determine if the intervention children had moved to within a normative range at follow-up, two types of comparisons were made. First, ANOVAs were conducted to determine if aggressive children in either the intervention or control groups were significantly different from normative children on the outcomes that had been found to be significantly affected by intervention. Second, normative comparisons were conducted to determine if the normative and intervention means were close enough to be considered clinically equivalent. At the one-year follow-up, ANOVAs indicated that the aggressive control group had higher levels of delinquency and weaker levels of behavioral improvement at school in comparison to the normative group. ANOVAs for the aggressive boys in the intervention condition, however, indicated that these boys were not significantly different from the normative boys in their rates of self-reported delinquency, parent-reported alcohol and marijuana use, or in teacher-rated behavioral improvement at school. Thus, aggressive boys in the intervention group moved from a non-normal range to a normative range by the one-year followup. Equivalency testing, using one-tailed z-tests, was conducted using one standard deviation around the normative mean as the range of closeness. These tests indicated that the aggressive intervention boys were significantly within normal limits in their rates of delinquent behavior, their parent-reported levels of substance use, and their school behavioral problem improvements.

Test of the Theoretical Model (Lochman & Wells, 2002b): Path analyses were performed to test the hypothesis that the intervention effects on Time 3 outcomes would be mediated through changes from Time 1 to Time 2 in targeted intervention change variables. The analyses examined the two intervention cells combined together (N = 120) in comparison to the control condition (N = 63). Path analyses models examined the relationships between the exogenous variables and the endogenous variables (i.e., that the changes in Time 3 outcomes would be mediated through changes from Time 1 to Time 2 mediating variables).

Delinquency Outcome: Goodness of fit indices for the various models of delinquency outcomes indicated that the null model provided a poorer fit than either the base or the mediation models. The base model is one in which the intervention has direct effects on Time 3 delinquency and Time 2 mediating variables. The mediation model is one in which paths from each of the Time 2 mediating variables are added to the base model to determine if the addition of these mediation paths will produce a nonsignificant path from intervention to Time 3 delinquency. Consistent with the prior ANCOVA results, the base model intervention produced significant reductions in the delinquency outcome and had significant effects on outcome expectations, internal locus of control, and parental inconsistency; and tended to improve person perceptions. For the mediation model, the path from intervention to the Time 3 delinquency outcome changed from being significant in the base model to being nonsignificant in the mediation model, showing that the delinquency reductions were a result of changes in the theorized mediating mechanisms. In this model, Time 3 delinquency was significantly predicted by lower levels of Time 2 parental inconsistency and tended to be predicted by lower levels of hostile attributions.

Substance Use Outcome: Goodness of fit indices for the various models for the parent-rated substance use outcome indicated that the null model provided a significantly poorer fit than both the base model and the mediation model. In the base model, intervention tended to predict lower levels of Time 3 substance use, significantly predicted better outcome expectations and more internal locus of control, and tended to predict lower parental inconsistency. Although the mediation model did not provide a significantly better fit than the base model, the critical path for the mediation test indicated that the effect of intervention is at least partially mediated by the addition of the five Time 2 targeted intervention change variables despite the fact that none of the five Time 2 variables alone predicted the Time 3 substance use outcome.

School Behavior Outcome: Goodness of fit indices again indicated that in the case of the various models for the teacher rated school behavior outcome the null model provided a significantly poorer fit than both the base and mediation models. For the base model, intervention predicted significantly greater improvement in Time 3 school behavior during the follow-up year, predicted significantly more internal locus of control and less parental inconsistency, and tended to predict better outcome expectations and person perception. The mediation model did not provide a significantly greater fit to the data than the base model. With regard to the critical path for the mediation test, the effect of intervention appears to be at least partially mediated by the addition of the five Time 2 targeted intervention change variables in the mediation model. Although none of the five Time 2 variables alone significantly predicted the Time 3 school behavior outcome, lower levels of Time 2 hostile attributions tend to predict improved Time 3 school behavior.

Study 3

The study evaluated a version of the program that included the 34-session child component and the 16-session parent component but compared two types of training for program counselors: basic training and intensive feedback training. The Coping Power webpage indicates that practitioners can receive general certification, primary (basic) certification, or intensive certification. It thus remains unclear whether the typical implementation of the program will match the basic or intensive training conditions.

Summary

Lochman et al. (2009, 2012) randomized counselors in 57 schools to one of three conditions: Coping Power Training Plus Feedback, Coping Power Basic Training, or a comparison condition. Teachers screened at-risk children in the third grade. Based on ratings, the 30% most aggressive children (n=531) across all classes were selected for inclusion in the study. Posttest and follow-up assessments measured child externalizing, aggression, and academic skills.

Lochman et al. (2009, 2012) found that, compared to the control group, children in the Coping Power plus Training Feedback or Intensive Training treatment group - but not the Coping Power Basic Training treatment group - showed significantly:

• Lower rates of teacher-rated externalizing behaviors
• Lower rates of parent-reported externalizing problems
• Lower rates of self-reported assaultive behaviors
• Higher rates of teacher-rated positive social skills and academic behaviors
• Lower self-reported expectations that aggression would lead to positive outcomes
• Higher language arts grades

Evaluation Methodology

Design:

Recruitment: The sample came from 57 public schools located in five school systems across contiguous areas of north central Alabama. The schools were recruited in two annual cohorts, with 80% of those invited agreeing to participate. The student sample included the 30% most aggressive third-grade children as rated by teachers, except for those in the upper 2% who were more likely to already have psychiatric diagnoses and treatment. Of  3,838 children screened, 1,422 fell within the desired range of aggressive behavior, 670 were contacted, and 531 agreed to participate and were assessed at baseline.

Assignment: In a clustered randomized design, the 57 schools were randomly assigned to one of three levels of training conditions: Coping Power intensive training with feedback (n = 19 schools, 168 students), Coping Power basic training (n = 19 schools, 183 students), or a comparison group (n = 19 schools, 180 students). The assignment was stratified by the school system. However, the program was delivered by school counselors and 16 of the 57 schools shared a counselor. The eight pairs of schools were yoked for assignment so that schools sharing a counselor were assigned to the same condition. With the yoking of schools, the cluster sample of counselors was 49.

Assessments/Attrition: Assessments occurred at baseline (summer after third grade) and posttest (end of fifth grade). Parent and child assessment data were available for 94% of the children and parents and 88% of the teachers. In addition, school records were gathered at baseline (summer after third grade), interim (end of fourth grade), posttest (end of fifth grade), one-year follow-up (end of sixth grade), and two-year follow-up (end of seventh grade). The records were obtained for 66% of the five possible data points for each of the 531 participants.

Sample:

The sample included 65% boys, 84% African Americans, 14% Caucasians, and 2% of other race/ethnicity.

Measures:

Lochman et al. (2009) examined four behavioral outcomes that measured externalizing, minor assault, and social/academic performance. Two came from teachers, one from children, and one from parents. Parents helped deliver the program, and teachers in intervention schools knew of the program but did not help deliver it. Researchers collecting the data from parents and children were blind to condition. Three risk and protective factors measured social skills (parent-rated), outcome expectation (child-rated), and inconsistent discipline (parent-rated). The reported reliabilities came from other studies, and some scales had low values.

Note that the study used an instrument that contained 13 scales measuring self-reported delinquency, substance use, and behavior outcomes, but the authors selected only one, minor assault, as most appropriate.

Lochman et al. (2012) examined average language arts and mathematics teacher-reported grades that were obtained from school records for grades three to seven. The outcome of teacher-assigned grades appears largely independent. The grades may be affected by knowledge of intervention participants at posttest but are likely less affected by such knowledge than direct ratings of behavior. Moreover, grades at the one-year and two-year follow-ups are assigned by teachers who are very likely unaware of the program assignments.

Analysis:

Lochman et al. (2009) used hierarchical linear models, with students at level 1 and counselors at level 2. The models adjusted for the clustered randomization of schools/counselors and the nesting of students within schools/counselors. The models for the posttest outcomes included baseline scores for the dependent variables as covariates.

Lochman et al. (2012) used multilevel growth curves and hierarchical linear models to estimate changes in grades over five time points. The models nested time points (level 1) within students (level 2) and nested students within schools/counselors (level 3).

Missing Data Method: The analyses used full maximum likelihood estimation to include students with missing data.

Intent-to-Treat: The analyses included all participants with full maximum likelihood estimation.

Outcomes

Implementation Fidelity:

The two intervention groups scheduled 26-29 of the 34 child sessions and 12 of the 16 parent sessions. Attendance was 68-69% at the child sessions and 20-24% at the parent sessions. Based on the coding of audiotapes, the intervention group means for completing session objectives were 0.9-1.2 on a scale from 0 (not at all) to 2 (thoroughly). Also based on the coding of audiotapes, the intervention counselors had means of 2.5-2.9 on a scale ranging from 1 to five for quality of engagement with parents and children.

Baseline Equivalence:

Table 1 in Lochman et al. (2009) shows the baseline means by condition for the outcomes. The means appear similar, but neither the table nor the text mentions significance tests or the size of differences. The sample sizes listed in the table show that the tests were based on those with complete data at posttest.

Lochman et al. (2012) appeared to use the analysis sample for tests of baseline differences. They reported finding no significant differences across the three conditions for language arts grades but found significant differences for math grades. The basic training condition had higher baseline math grades than the control condition. There were no tests for sociodemographic measures, but the descriptive percentages indicate substantial differences. The three conditions - intensive training, basic training, and comparison - had 75%, 89%, and 87% African American participants and 60%, 63%, and 72% male participants.

Differential Attrition:

In Lochman et al. (2009), tests for baseline differences between the completers versus dropouts found no significant differences for the seven outcomes. In addition, tests for completer versus dropout differences within the three conditions found that three of 21 tests were significant. The authors noted that the significant results showed no clear pattern. The tests did not include sociodemographic measures. The Table 1 means for the baseline outcomes using the analysis sample also are relevant to differential attrition but lacked significance tests or effect sizes.

In Lochman et al. (2012), cases with missing data in three or more of the years (and thus had more than half of their data missing) were compared with those with no or lesser amounts of missing data. There were no significant differences in race, gender, baseline screening score, or condition assignment between participants with high attrition versus others. However, tests for baseline equivalence using the analysis showed some condition differences after attrition.

Posttest:

Lochman et al. (2009) found that the Basic Training condition did not do significantly better than the control group on any of the four behavioral outcomes or the three risk and protective factors. The Feedback Training condition relative to the control group had significant effects on all four behavioral outcomes and one of the risk and protective factors (outcome expectations). Effect sizes ranged from -.23 to -.41.

Long-Term:

Lochman et al. (2012) found that the intensive training condition (or training with feedback condition) significantly slowed the decline in language arts grades compared to the control group. There was no significant effect on math grades. The basic training condition did not have significantly different changes in grades than the control group. Moderation tests found that the influence of the intensive training program did not differ significantly by receipt of special education services.

Study 4

The Utrecht Coping Power Program adapted the Coping Power Program to work with more severely disturbed children and their parents. It contained all the essential elements of Coping Power, but the sessions were more varied, with proportionally fewer discussions and more activities to suit the short attention span of the children. The numbers of sessions, 15 for parents and 23 for children, were similar but not identical to a shortened version of the original program.

Summary

Van de Wiel et al. (2003, 2007) and Zonnevylle-Bender et al. (2007) conducted a randomized controlled trial with 77 Dutch youths ages 8-13 who met clinical criteria for disruptive behavior disorder. The study randomized the youths to the intervention group or a control group that received usual services such as family therapy or behavior therapy. Assessments at nine months, 15 months, and five years included measures of aggression, externalizing, substance use, and delinquency.

Van de Wiel et al. (2003, 2007) and Zonnevylle-Bender et al. (2007) found that the intervention group relative to the control group had significantly lower

• Parent-reported overt aggression at 15 months
• Last-month cigarette use at five years
• Lifetime marijuana use at five years.

Evaluation Methodology

Design:

Recruitment: Children were recruited at child psychiatric outpatient clinics or child mental health centers from October 1996 to August 1999. Children were included if they (a) were 8 to 13 years of age; (b) met the criteria for disruptive behavior disorder (oppositional-defiant disorder and/or conduct disorder in accordance with the DSM-IV); (c) were living within a family; and (d) scored at least 80 based on two intelligence subtests. A total of 77 children and parents participated.

Zonnevylle-Bender et al. (2007) also recruited a healthy control group (n = 61) that had similar sociodemographic characteristics as the original participants but without any indication of psychopathology.

Assignment: The study randomly assigned the 77 participants to either the intervention condition (n = 38) or the control condition (n = 39) after stratifying by sex and comorbidity with ADHD. The control subjects received mental health services such as family therapy and behavior therapy.

Assessments/Attrition: Assessments occurred at baseline, nine months later (posttest), 15 months later (six-month follow-up), and five years later. However, the six-month follow-up assessment served as the posttest for many. Subjects in both conditions received treatment from baseline to posttest, but four families in the intervention condition and 26 families in the control condition received treatment during the six-month follow-up period. Although not stated explicitly, there appears to be no attrition through the six-month follow-up (van de Wiel et al., 2003, 2007). For the five-year follow-up, Zonnevylle-Bender et al. (2007) had a sample of 61 (79.2% of the original 77 participants).

Van de Wiel et al. (2007) used a subset of control participants that received either family therapy or behavior therapy (n = 26). It was unclear if the control group was randomly or non-randomly assigned to the different treatment-as-usual therapies.

Sample:

The assigned sample included 88% boys and had a mean age of 10.1 years.

Measures: Van de Wiel et al. (2003, 2007) examined four main outcomes as reported by parents and teachers. Parents completed the Parent Daily Report measures of overt aggression (alpha = .77) and oppositional behavior (alpha = .85). They also completed the Child Behavior Checklist measure of externalizing behavior. Because parents helped deliver the parenting component of the program, their ratings may not be independent. Teachers completed the Teacher Report Form measure of externalizing behavior.

For the five-year follow-up, Zonnevylle-Bender et al. (2007) examined self-reported measures of last month and lifetime use of cigarettes, alcohol, and marijuana and a self-reported delinquency scale (alphas ranged from .76 to .85 across conditions).

Analysis:

Van de Wiel et al. (2003) did not describe their analysis methods. Van de Wiel et al. (2007) used repeated measures analyses of variance with interaction terms of treatment condition and time. They limited the sample to the 26 participants in the control group who received either family therapy or behavior therapy.

For the five-year follow-up, Zonnevylle-Bender et al. (2007) used analysis of variance to test for differences in three groups: the original intervention, the original control, and the added healthy control. The tests did not appear to control for baseline outcomes.

Missing Data Methods. Two analyses appeared to use all participants for the full sample (van de Wiel et al., 2003) or the subsample (van de Wiel et al., 2007). Zonnevylle-Bender et al. (2007) used participants with complete data at baseline and the five-year follow-up.

Intent-to-Treat: The studies appeared to use all participants with complete data regardless of whether or not they finished the treatment.

Outcomes

Implementation Fidelity:

Van de Wiel et al. (2003) reported that 68 of 77 participants (88%) completed the treatment. The number of switches in treatment method was lower in the intervention condition than in the control condition. Also, the number of treatments that ended without the mutual consent of therapist and family was lower in the intervention condition than in the control condition.

Baseline Equivalence:

Van de Wiel et al. (2003) reported that after random assignment, "the two groups did not differ in age, intelligence, comorbidity with ADHD, the dosage of medication related to this comorbidity, and the outcome measures at pretreatment." For the subsample, van de Wiel et al. (2007) similarly reported that the conditions did not differ significantly on age, sex, intelligence, educational level, ADHD comorbidity, and the outcome measures.

Differential Attrition:

Van de Wiel et al. (2003, 2007) appeared to have no attrition. Zonnevylle-Bender et al. (2007) had attrition (79% in both conditions). They reported that "there were no significant differences in age, gender, IQ, parents' employment, parents' education level, diagnosis, medication, and family composition at baseline between the patients who participated in the follow-up and those who did not." In addition, Table 1 showed no significant baseline condition differences for 10 sociodemographic measures using the analysis sample. For outcomes, the authors stated that there were no significant condition differences in the baseline substance use measures. Lastly, the time between the pretreatment measurement and the measurement five years after the start of treatment did not differ significantly between the two conditions. Based on the overall attrition rate and the condition difference in attrition rates, the study met both the What Works Clearinghouse cautious and optimistic standards.

Posttest:

Van de Wiel et al. (2003) first compared the intervention group participants to control group participants with low experience therapists. The authors reported that "There were no significant differences in the various outcome measures at posttreatment or at 6-month follow-up between these two groups." They next compared the intervention group to control-group participants with high experience therapists. The results "showed no significant differences in the various outcome variables at posttreatment or at 6-month follow-up."

Van de Wiel et al. (2007) first compared the intervention-group outcomes to the control-group subset receiving family therapy or behavior therapy. Of the four outcomes, one differed significantly over time. Parents in the intervention group reported less overt aggression than parents in the control group. The eight tests comparing the intervention group to family therapy participants and to behavior therapy participants showed one significant program effect. Parents in the intervention group reported less overt aggression than parents in the family therapy control group.

Long-Term:

At the five-year follow-up, Zonnevylle-Bender et al. (2007) reported that the three conditions (intervention, control, and healthy control) did not differ in the last month use of alcohol and marijuana, but they differed in the use of cigarettes. Significantly more control adolescents smoked cigarettes than the intervention adolescents and the healthy control adolescents.

For lifetime use, the three groups did not differ in their use of alcohol and cigarettes, but they differed in their use of marijuana, with significantly more control adolescents using marijuana than intervention or healthy control adolescents.

There were no significant differences among the three groups on the Delinquency Scale.

Study 5

The study evaluated the 24-session, one-year version of Coping Power. However, response was too low to hold the parent sessions.

Summary

Peterson et al. (2009) conducted a randomized controlled trial with 119 students ages 10-12 in five schools who were rated by teachers as at high risk for behavior problems. The study randomized the students to the intervention group or a control group that was eligible for usual school services. A 24-week posttest examined teacher-rated measures of externalizing, internalizing, social skills, and learning problems.

Peterson et al. (2009) found that, relative to the control students, the intervention students received significantly better teacher ratings for

• Depression
• Internalizing
• Behavioral symptoms
• Social skills
• Learning problems
• School problems
• Adaptive skills.

Evaluation Methodology

Design:

Recruitment: The sample of 119 students ages 10-12 came from five schools (two middle schools, one elementary school, and two combined elementary/middle schools) in four school districts. The students were at high risk for behavioral trouble at school, as rated by teachers. The sample appears to refer to those students who consented rather than to the randomized sample.

Assignment: The study randomly assigned students to either the intervention group or a control group that could receive any services or support normally provided by the school districts. Following random assignment, intervention participants met to learn about the program and give consent. Consent thus followed randomization for the intervention group, but the study did not present figures on non-consent. Also, it was unclear if consent was required for the control group. The unequal sample sizes after consent, 63 intervention students and 56 control students, add further to the ambiguities. 

Assessments/Attrition: Baseline and posttest assessments came at the time of the first and last program sessions, roughly 24 weeks apart. The study reported that teachers completed the baseline and posttest measures for 114 of the 119 consented participants (96%).

Sample:

The sample averaged 11.5 years of age, with about 61% males and 39% females. There was some ethnic diversity, with 53% White students, 43% Hispanic students, 3% Black students, and 1% other.

Measures:

Teachers completed the Teacher-Report form of the Behavior Assessment Scale for Children-2 for students in both conditions (self-report measures were obtained only for the intervention group). Measures for 13 subscales relating to externalizing, internalizing, social skills, and learning problems had alpha values ranging from .81 to .97. Teachers may have known of the intervention participants but had no involvement in the program.

Analysis:

The analysis used repeated-measures ANOVA with time-by-assignment terms that controlled for the baseline outcomes.

Missing Data Methods. The ANOVA used complete cases without imputation or FIML.

Intent-to-Treat: The study appeared to use all participants with complete baseline and posttest data.

Outcomes

Implementation Fidelity:

The parent sessions were not implemented. Otherwise, the only implementation information concerned the group leaders. An analysis found that intervention students showed greater behavior change if they had been in the groups led by a more experienced leader.

Baseline Equivalence:

Peterson et al. (2009) stated: "we used a one-way ANOVA of the Teacher-Report data to confirm that the students in the intervention and control groups were equivalent before we began data analysis." The tests did not refer to sociodemographics, and it appears that there were differences. All control students were in grade six while 19% of intervention students were in grade five and 81% were in grade six. Also, 49% of the intervention group was Hispanic compared to 36% of the control group.

Differential Attrition:

Attrition was minimal.

Posttest:

In Peterson et al. (2009), the time-by-assignment terms in Table 2 showed no significant intervention effects in five tests, but the terms in Table 3 showed seven significant intervention effects in eight tests. Teachers rated the intervention student as significantly more improved than control students on depression (d = .22), internalizing (d = .32), behavioral symptoms (d = .42), social skills (d = .45), learning problems (d = .20), school problems (d =. 26), and adaptive skills (d = .76). Additional analyses on self-report data obtained only for the intervention group (n = 63) did not show significant differences between the pre- and the postintervention scores.

Jurecska et al. (2011) similarly examined only the intervention group but in addition focused on a subgroup of 20 students who were identified as most responsive to the treatment. The analysis showed that this subgroup improved significantly more than other intervention participants.

Long-Term:

Not examined.

Study 6

This study of Pakistani boys examined a culturally adapted version of the abbreviated program (24 child sessions and 10 parent sessions implemented in one academic year). The cultural adaptation included adding religious and cultural beliefs, labeling feelings and emotions with culturally relevant language, and adding an extra session for expression of feelings. In addition, it appears that the parent sessions were not conducted because of low response (< 5%).

Summary

Mushtaq et al. (2017) conducted a randomized controlled trial to examine 112 fifth-grade boys in five schools in Pakistan who were identified by teachers as highly aggressive. The boys were individually randomized within the schools to the intervention group or a waitlist control group. Assessment at the end of the school year measured aggression and social competence.

Mushtaq et al. (2017) found that, relative to the control group, the intervention group had significantly

• Lower aggressive behavior as rated by parents and teachers
• Higher social competence and social support as self-reported by children
• Better social cognition as self-reported by children.

Evaluation Methodology

Design:

Recruitment: The study included boys who were screened for aggressive behavior in five public schools in Rawalpindi, Pakistan. From 15 classrooms with at least 70% of parents consenting to the initial screening, 859 boys ages 9 to 11 years old were recruited. Eligible youths needed to have intact and literate families with no history of psychopathology. Teacher ratings identified 174 severely aggressive boys.

Assignment: The study randomly assigned the 174 boys within schools to the intervention group (n = 87) or a waitlist control group (n = 87). However, consent followed randomization. In all, 31 (17.82%) parents and 10 (5.75%) boys refused to participate, and 21 (12.07%) boys did not meet the inclusion criteria. That left a sample of 112 (64.4%), with 51 (58.6%) in the intervention group and 61 (70.1%) in the control group.

Assessments/Attrition: The baseline assessment occurred in August 2011, and a posttest followed at the end of the one school-year intervention in April 2012. According to the CONSORT diagram, the participant loss occurred soon after randomization and resulted from non-consent and non-eligibility, and there was no attrition afterward.

Sample:

The sample included Pakistani boys who were in fifth grade during the program implementation. No other sociodemographic information was provided.

Measures:

Four trained psychology graduates who were blind to the boys' assigned condition collected the data. The measures came from teachers, parents, and boys. Parents helped deliver the program and their reports may not be independent. Teachers were not involved in delivering the program but may have been aware of assignment to the program. There were 15 measures in four categories: 1) proactive and reactive aggression from mothers and teachers (alpha values ranged from .68 to .73), 2) child-reported internal behavioral processes of fear and activity level (alpha values of .69 and .71), 3) child-reported social competence and social support (no alphas listed), and 4) child-reported social cognition (alpha values ranged from .76 to .90).

One other teacher-rated measure of social, cognitive, and behavioral improvement was collected only at the end of the school year and was available only for the intervention group.

Analysis:

The analysis used multilevel models that nested students within schools and included a time-by-group interaction term to capture changes from baseline.

Missing Data Methods. Based on information from the CONSORT diagram, the analysis used complete cases.

Intent-to-Treat: The analysis included all participants with complete data.

Outcomes

Implementation Fidelity:

The average child attendance rate was 89% in the intervention condition. Content fidelity was rated at 91%.

Baseline Equivalence:

The authors reported that tests for equivalence of outcome and demographic measures found no significant differences. Table 1 lists baseline means by condition for the outcomes but without significance tests.

Differential Attrition:

No tests using baseline measures were presented, even though consent after randomization differed substantially across conditions: 58.6% for the intervention group and 70.1% for the control group. Otherwise, there was no attrition.

Posttest:

For the 15 outcomes in Table 1, 14 reached statistical significance. The intervention group scored significantly lower on all four measures of aggressive behavior (as rated by both mothers and teachers) and on two measures of child-reported social competence and social support. Other significant risk and protective factors included measures of activity level and social cognition. The end-of-year ratings provided by teachers for the intervention group only showed improvement in multiple measures.

Long-Term:

Not examined.

Study 7

The study evaluated the program with 36 child sessions and 16 parent sessions. However, some activities were adjusted for the more severely disturbed children in the study. For example, the sessions had fewer discussions and more activities/play sessions to suit the short attention span of the children. The number and objectives of the sessions did not change from the original model. Both the manual and the homework forms of the program were translated into Italian with consultation of the authors of the treatment program.

Summary

Muratori et al. (2017) conducted a quasi-experimental study of 110 Italian children ages 8-9 with diagnosed conduct disorders. Using a sequential block approach, the study assigned the children and parents to three conditions: Coping Power implemented in the hospital (n = 33), a generic multi-component intervention implemented in the hospital (n = 37), or a treatment-as-usual group that received child therapy as part of community care services (n = 28). Assessments at posttest and 12 months later measured parent, child, and clinician reports on child behavior problems and callous-unemotional traits.

Muratori et al. (2017) found that, relative to the child therapy control group, the Coping Power group had significantly

• Lower parent-reported child aggressive behavior
• Lower parent-reported child callous-unemotional traits
• Lower family use of mental health services
• Lower inconsistent discipline,
• Lower negative parenting
• Lower child-reported callous-unemotional traits
• Higher clinician ratings of global improvement.

Evaluation Methodology

Design:

Recruitment: The sample included children ages 8-9 who were referred for behavioral problems to a Psychiatry clinic in Pisa, Italy. Screening at the clinic selected children with (1) a clinical diagnosis of Oppositional Defiant Disorder or Conduct Disorder; (2) an IQ score above 85; (3) a high externalizing score; and (4) a low global functioning score. A total of 110 children and families met the indicated inclusion criteria, completed pre-treatment assessments, and started the intervention.

Assignment: The quasi-experimental study used a non-equivalent control group design, with a sequential block approach to assigning children to three conditions: Coping Power implemented in the hospital (n = 33 after attrition), a generic multi-component intervention, named Beyond the Clouds, implemented in the hospital (n = 37 after attrition), or a treatment-as-usual group that received child therapy as part of community care services (n = 28 after attrition). The assignment sequence proceeded as follows: Participants were first assigned to the Coping Power group if spots were available; if not available, participants were next assigned to the generic intervention group if spots were available; and participants were lastly assigned to the control group when there were no available spots in groups for the other conditions.

Assessments/Attrition: Data were collected at baseline, the end of the 12-month treatment (posttest), and approximately 12 months after the end of the treatment (one-year follow-up). Of the 110 participants, 98 (89%) completed treatment and were given assessments (dropouts were not assessed).

Sample:

The sample was 87% male, 83% Caucasian, and 17% from Africa. About 23% of the families were classified as low SES.

Measures:

The study examined five child outcome measures, three reported by parents who helped deliver the program, one provided by the children, and one provided by clinicians. The parent-reported measures included 1) aggressive behavior from the externalizing subscales of the child behavior checklist (mean alpha = .75), 2) rule breaking from the externalizing subscales of the child behavior checklist (mean alpha = .75), and 3) callous-unemotional traits from an antisocial process screening device (mean alpha = .73). A fourth measure, the inventory of callous-unemotional traits, came from child responses (mean alpha = .72). A last measure came from two clinicians, who rated a child as a responder to the interventions when their Clinical Global Impression-Improvement score was very much improved or much improved (the rate of concordance between raters was .87).

The study also examined four parenting measures but only for the two intervention groups, not the control group. These measures included parental involvement (mean alpha = .78), positive parenting (mean alpha = .79), inconsistent discipline (mean alpha = .78), and use of harsh discipline (mean alpha = .76).

One last measure came from parents, who reported on the frequency with which they had used mental health services over the last year for behavioral or emotional problems.

Analysis:

The analysis used repeated measures models with interaction terms for time-by-group and the Tukey pairwise test to evaluate differences in effects across the three conditions. All analyses controlled for the use of medication during the treatment.

Missing Data Methods. The study appeared to use complete-case analysis without imputation or FIML.

Intent-to-Treat: The study used all participants with complete data, but only those who finished the treatments had complete data. The overlap between treatment completion and assessment completion may violate the intent-to-treat guideline.

Outcomes

Implementation Fidelity:

The average child and parent attendance rate at the sessions was 87%. Facilitator checklists indicated that over 80% of session objectives were delivered.

Baseline Equivalence:

Table 1 uses the analysis sample of 98 to test for 10 condition differences on baseline sociodemographic and clinical measures but not including the outcome measures. The table uses a .01 significance level, but age and the Children Global Assessment scale have t values above 2.0 that appear significantly different at the .05 probability level. In addition, the means show some large, though nonsignificant differences (e.g., 87% intact families in Coping Power versus 71% in the control group).

One other measure in the table, use of medication during treatment, differed significantly but did not involve a baseline difference.

Differential Attrition:

The study did not report any attrition tests or attrition by condition, but the limited tests for baseline equivalence using the analysis sample suggest possible bias, either from non-random assignment or attrition.

Posttest and Long-term:

The analyses focused on the changes over time and did not distinguish posttest from long-term results. For the first four child measures in Table 2, two parent-reported outcomes (aggressive behavior and callous-unemotional traits) and one child-reported measure (callous-unemotional traits) showed significant time-by-group terms. Pairwise comparisons further showed that Coping Power participants improved significantly more than both the alternate intervention and the control participants. Reported effect sizes were for within-condition changes rather than between conditions. At the follow-up assessment, significantly fewer Coping Power patients were evaluated by clinicians as non-responders (i.e., did not show much improvement).

For the four parenting measures in Table 3, the results only applied to Coping Power and the alternative intervention (not the control group). There were two significant time-by-group terms. The pairwise tests showed that only Coping Power group had a significant increase in positive parenting and a significant decrease in inconsistent discipline at post-treatment evaluation. The authors added, "No significant modifications in both groups resulted from posttreatment to the one-year follow-up evaluation on parenting practices."

At the follow-up assessment, significantly fewer Coping Power patients used mental health services.

Study 8

The study evaluated the abbreviated, one academic-year version of Coping Power with 16 sessions in the first semester and six booster sessions (eight total lessons) in the following semester.

Summary

McDaniel et al. (2018) conducted a quasi-experimental study that examined 33 students with elevated levels of disruptive behavior from one elementary school. The study assigned classrooms to one of three conditions: control (n = 13 students), Coping Power (n = 13 students), and an intervention named Check-In/Check-Out (n = 7 students). A posttest at the end of the school year measured teacher ratings of conduct problems and externalizing.

McDaniel et al. (2018) found that the intervention group, relative to the control group, had significantly reduced levels of teacher-rated

• Overall difficulties
• Externalizing
• Aggression.

Evaluation Methodology

Design:

Recruitment: The sample came from one urban public elementary school with 433 students and located in a small Southeastern city. The school was targeted due to poor academic performance and high rates of disciplinary problems. Student participants had to meet several inclusionary criteria: (1) Grades 3 through 5, (2) nominated by the classroom teacher as displaying elevated levels of disruptive behavior, and (3) scores in the elevated or high-risk categories on the Strengths and Difficulties Questionnaire in at least one deficit area. Of 62 students recruited for the study, a total of 33 students ranging in age from 8 to 11 years met all inclusion criteria and consented to participate.

Assignment: The study used a quasi-experimental design that the authors called "a functionally random procedure." All 12 classrooms for grades 3-5 were assigned to conditions by listing teacher names in alphabetic order and alternating selections into three conditions: control (n = 13 students), Coping Power (n = 13 students), and an intervention named Check-In/Check-Out (n = 7 students). Fewer participants were assigned to the Check-In/Check-Out group because fewer students were nominated from the classrooms assigned to the condition. The control students received the universal preventive supports available at the school.

It appears that each Coping Power group was jointly led by the same two interventionists, which may confound the program efficacy with the unique characteristics of the two leaders.

Assessments/Attrition: The three assessments occurred at baseline, the end of the first semester (interim), and the end of the second semester (posttest). The analysis sample included 33 students, implying either that there was no attrition or that the assigned sample had already excluded dropouts.

Sample:

All students were African American and attended mainstream classrooms, but 21% were identified for special education services for learning disabilities and 6% had documented attention-deficit/hyperactivity disorder diagnoses. Most students were male (88%). They were distributed by grade as follows: grade 3 (30%), grade 4 (49%), and grade 5 (21%).

Measures:

Teachers provided six measures from the Strengths and Difficulties Questionnaire (overall difficulties and five subscales) and four measures from the Behavior Assessment of Children in Schools-2 (overall externalizing and three subscales). The authors did not report reliabilities for the sample, but the measures are well validated.

The teachers who rated students were involved in the program. The authors noted that the interventionists who led the program sessions and classroom teachers communicated weekly about the behavior and problems of students participating in the program.

Analysis:

The analysis used repeated-measures mixed models with time (baseline, first semester, and second semester posttest), condition, and time-by-condition. Post hoc tests used a Bonferroni adjustment. The study assigned classrooms to the conditions but did not adjust for clustering.

Missing Data Methods: The analysis used complete cases.

Intent-to-Treat: The study appeared to use all available data, but it is possible that the reported assigned sample already excluded dropouts.

Outcomes

Implementation Fidelity:

The overall coverage of the objectives of each session was 92% on average, with a range of 83% to 100% across the sessions. The authors also concluded that overall, the program "is viewed as a socially valid and acceptable intervention." However, participation in the parent sessions was very low.

Baseline Equivalence:

The study reported only that the conditions did not differ significantly in the proportions of students categorized from the Strengths and Difficulties Questionnaire as normal, borderline, elevated risk, or high risk. Table 1 presents baseline means by condition for the 10 outcomes but lists no tests for differences.

Differential Attrition:

No apparent attrition.

Posttest:

Table 2 shows that four of 10 time-by-condition interaction terms, which included all three conditions, reached statistical significance. Table 3 focuses on within-group changes for the three conditions. It shows that three of the four significant interactions indicated significant improvement of the program group over time: overall difficulties, externalizing, and aggression. Table 4 shows that a greater proportion of students in the program condition decreased their risk category from the baseline to the posttest as compared with the control group and the alternative intervention group.

Long-Term:

Not examined.

Study 9

The program used by the study consisted of 34 weekly group sessions for children with a behavioral disorder. The program did not include the Coping Power parent sessions, but all participants completed a Cognitive Behavioral Parent Training Program before being included in the study.

Summary

Vanzin et al. (2018) conducted a quasi-experimental study that examined 55 Italian children ages 8-13 who had been diagnosed with ADHD. The study assigned the first 31 children to the treatment group and the next 24 to a waitlist control group. A posttest assessment at nine months after baseline measured parent and clinician ratings of child behavior problems.

Vanzin et al. (2018) found that, relative to the control group, the intervention group had significantly better

• Parent ratings of child social problems, attention problems, and rule-breaking
• Clinician ratings of ADHD symptom severity.

Evaluation Methodology

Design:

Recruitment: The sample came from children and adolescents who were referred to an Italian Child Psychopathology clinic for ADHD treatment from 2013 to 2015. Participants were ages 8-13, were diagnosed with ADHD, and had scores above 85 on an IQ test. Their parents were required to complete a Cognitive Behavioural Parent Training program before the start of the study. A total of 62 children met the eligibility requirements, but the exclusion of seven parents who did not complete the training reduced the sample to 55.

Assignment: The 55 participants were consecutively assigned to the treatment group (n = 31) or a waitlist, no-treatment control group (n = 24) according to the time of clinic admission.

Assessments/Attrition: Assessments occurred at baseline and posttest (nine months later). With the loss of two who dropped out and three who did not complete the posttest, the analysis sample included 50 participants, 91% of the final 55 (or 81% of the original 62).

Sample:

The mean age of the Italian sample was 9.94 years (ranging from 8 to 13 years). Boys represented 84% of the total group. The families of the participants had a medium socioeconomic status.

Measures:

The study examined 10 parent-rated outcomes from the Child Behavior Checklist (e.g., internalizing, externalizing, attention problems). As parents helped deliver the program, these measures are not independent. At baseline, Cronbach's alphas ranged from .609 to .841 for most scales, but three had alphas ranging from .200 to .549.

One additional measure, the Clinician Global Impression Severity, came from clinician ratings that ranged from (1) normal, not at all ill to (7) extremely ill. The authors noted that the measure has demonstrated acceptable reliability and validity in previous studies with Italian children. The authors also noted that the clinicians were not blind to the assessment design.

Analysis:

The analysis used multivariate MANOVA of changes from baseline to posttest for the 10 parent-rated outcomes and repeated measures ANOVA for the clinician-rated outcome.

Missing Data Methods. The analysis used complete cases without imputation or FIML.

Intent-to-Treat: The analysis used all participants who met eligibility requirements and provided posttest data.

Outcomes

Implementation Fidelity:

Not examined.

Baseline Equivalence:

Tests using the analysis sample (n = 50) found one significant difference for eight background measures (Table 2) and four significant differences for 12 baseline outcome measures (Table 3). The treatment children were older and had fewer problem behaviors than the control group.

Differential Attrition:

From the eligible sample of 55, the control group had no attrition, and the treatment group had an attrition rate of 16%. The overall attrition rate and the difference in attrition rates by condition do not meet the What Works Clearinghouse cautious or optimistic standards. In addition, the baseline equivalence tests for the analysis sample showed several condition differences that may in part be due to attrition.

Posttest:

In 10 tests for parent-rated outcomes, three reached statistical significance: social problems, attention problems, and rule-breaking. All three showed greater improvement in the treatment group than the control group.

For the clinician-rated measure, a significant group-by-time interaction indicated greater improvement in the intervention group.

Long-Term:

Not examined.

Study 10

This study evaluated a version of the program that was adapted to meet the unique needs of deaf and hard-of-hearing children. It translated the materials into sign language, used a greater variety of visual materials, and placed greater focus on learning basic social-cognitive skills. Teachers and caretakers received specialized training for their role in the program.

Summary

Lochman et al. (2019) conducted a randomized controlled trial to examine 49 students in a residential school for deaf and hard-of-hearing children who had been screened for high aggression. The study randomly assigned the students to an intervention group or a waitlisted control group. A one-year posttest measured conduct problems, aggression, and several mediators related to communication and problem-solving.

Lochman et al. (2019) found no effects on behavioral outcomes, but teachers rated the intervention students, compared to control students, as significantly more improved on

• Communicative competence
• Problem-solving skills.

Evaluation Methodology

Design:

Recruitment: The sample included full-time students in a residential school for the deaf who were identified as aggressive based on screening scores completed by teachers and staff. All children in the school from the fourth grade through high school were screened for aggressive behavior. The final sample of 49 students who were selected and consented to participate had moderate to severe aggressive behavior problems.

Assignment: The 49 children were randomly assigned to receive the group intervention in year one (intervention group, n = 26) or in year two (waitlisted control group, n = 23).

Four separate treatment groups ran during the first year. According to the authors, "each pair of co-leaders included a deaf leader and a hearing leader who was proficient in sign language." It appears that a different pair led each group, but it is possible that there was one pair of co-leaders, which would confound group leadership with the program.

Assessments/Attrition: The posttest followed the baseline assessment after one year. The posttest completion rate was 84%, with attrition due to withdrawal from the school, early graduation, and refusal to complete the assessment.

Sample:

The sample averaged 12.5 years of age, with a range from 9 to 16. About 67% of participants were male. Most (64%) were African American, while 32% were Caucasian, and 2% were Hispanic.

Measures:

The study examined three behavioral outcomes, all provided by teachers, who were involved in delivering the program (see p. 46, "group leader consultation with teachers occurred"). Two measures, one for aggression and one for conduct problems, came from the Behavioral Assessment System for Children (alphas = .83 and.71, respectively). A third measure, the Behavioral Improvement Rating, came from "the mean of two items indicating children's improvement in behavior problems and in their problem-solving and anger management."

The study examined 10 mediator variables, also provided by teachers, that include scales relating to communicative competence, social adjustment, self-concept, and conflict problem-solving skills. Reported alphas were in the acceptable range.

Analysis:

The study used an ANOVA model to compare condition differences in behavioral improvement across the intervention year. ANCOVA models controlling for baseline behavior were used to test condition differences in aggression and conduct problems at posttest.

Missing Data Method: The analysis used complete cases without imputation or FIML.

Intent-to-Treat: The analysis appeared to use all participants with complete data.

Outcomes

Implementation Fidelity:

Not assessed.

Baseline Equivalence:

Not reported.

Differential Attrition:

The study noted only that attrition rates in year one were similar across experimental groups (21% in the control group and 12% in the intervention group). The overall attrition rate and the difference in attrition rates between conditions do meet the What Works Clearinghouse cautious standard but meet the optimistic standard.

Posttest:

The intervention did not significantly affect the three behavioral outcome measures, though it did have a marginally positive impact on behavioral improvement (p = .09). For the 10 mediator measures, five reached statistical significance. Teachers rated the intervention students as significantly more improved on communicative competence and four measures of problem-solving skills than the control students.

The results further showed relationships between the mediators and the outcomes but did not include a full mediation analysis of the intervention impact on the behavioral outcomes.

Long-Term:

Not examined.

Study 11

This version of the program, designed for children with behavioral disorders, included 24 sessions with smaller groups than usual (about five children) and more activities than discussions. The parent sessions were reduced from 18 to 16 sessions.

Summary

Muratori et al. (2019) conducted a quasi-experimental study of 120 Italian children ages 8-9 with diagnosed conduct disorders. Using a sequential block approach, the study assigned the children and parents to the program or a control group receiving alternative treatment. Assessments at posttest, 12-month follow-up, and 60-month follow-up measured parent reports on child externalizing and child reports on callous-unemotional traits and substance use.

Muratori et al. (2019) found that, relative to the control group, the intervention group had significantly

• Lower scores on callous-unemotional traits
• Lower use of marijuana.

Evaluation Methodology

Design:

Recruitment: The sample included 120 children ages 9 to 10 who were recruited in a psychiatric outpatient clinic (Pisa, Italy) from October 2007 to October 2010. Eligible children met the criteria for Oppositional Defiant Disorder or Conduct Disorder and had an IQ score of at least 85.

Assignment: The study used a block sequential assignment procedure instead of a randomized design. Participants were assigned to the intervention when a place in a treatment group was available; if no places were available, participants were assigned to the control group.  No other clinical criteria were used for group assignment. The control group received a generic multi-component treatment model that used cognitive-behavioral principles and practices, including individual treatment for children and individual parent training. After attrition, the intervention group had 55 participants and the control group had 42 participants. Treatments for both conditions lasted 12 months.

Assessments/Attrition: Assessments occurred at baseline, posttest (one year after baseline), one-year follow-up (two years after baseline), and five-year follow-up (six years after baseline). Of the 120 participants, 23 were lost in the follow-up (10 patients interrupted the treatment and 13 did not participate in the follow-up post-treatment evaluations). The completion rate was 81%.

Sample:

The sample had 93% boys, and 32% of the families had low SES, and 50% had medium SES.

Measures:

First, externalizing from the Child Behavior checklist was completed by parents (alpha = .84 as averaged across the four time points). Second, the children used the self-reported Inventory of Callous Unemotional Traits (alpha = .74, averaged across the four time points). Third, child-reported measures of cigarette, marijuana, and alcohol use were obtained at the last assessment. The scores on these items were dichotomized into users and non-users.

Analysis:

The analysis used linear mixed-effects models with full-information maximum likelihood estimation, three levels (measurement occasion within individuals within treatment groups), and random intercepts for within-subjects correlations, within-treatment group correlations, and within-condition group correlations. Controls included pharmacological treatment and the levels of socio-economic condition. The fixed-effect portion of the model treated outcomes as a function of time, condition, and time interacting with condition.

Missing Data Method: The study used FIML estimation.

Intent-to-Treat: The analysis included all participants.

Outcomes

Implementation Fidelity:

The study reported only that the average child and parent attendance rate for the program was 86%.

Baseline Equivalence:

Tests used the analysis sample. Table 1 shows that there were no significant baseline differences between the two conditions for eight socio-demographic, two outcome, and three clinical variables.

Differential Attrition:

The authors stated that an "Attrition analysis showed no significant differences in age, gender, diagnosis ratio (ODD or CD) and clinical variables between those who dropped out and those who completed the study." In addition, tests for baseline equivalence using the analysis sample showed no significant differences.

Posttest and Long-Term:

One of the two conduct measures had a significant time-by-condition effect. The intervention group had a significantly greater decline over the six-year period for child-reported callous-unemotional traits. Of the three substance use measures, marijuana use was significantly lower at the last assessment for the intervention group than the control group.