Incredible Years – Teacher Classroom Management

Study 1

Evaluation Methodology

Design: A single-blind stratified cluster RCT design was used. An independent researcher paired classrooms according to school size, classroom size, and locality after pre-intervention measures were collected. Schools were randomly assigned to condition (intervention or control) and the paired school was automatically assigned to the control group in each case. Participants were teachers and their students from 12 classes (6 intervention and 6 control) from 11 primary schools in rural and urban areas of North West Wales (county of Gwynedd). Schools were recruited by a Gwynedd Education Service official. Class sizes ranged from 14 to 30 students, and students were between three and seven years old. Data was collected from eight full-time teachers and four job-share teachers (teachers who shared teaching duties with another teacher). Teachers completed the Teacher Strengths and Difficulties Questionnaire (TSDQ) to rate their students' behavioral, emotional, and social difficulties, using four subscales (hyperactivity problems, peer-problems, emotional problems, and conduct problems) to produce a Total Difficulties (TD) score. The 107 students targeted and recruited for assessment (index children) were chosen based on their TSDQ scores: ideally, those having the three highest, three lowest, and three mid-point scores, for a total of 9 children per class (one class only had 8 recruited students). In actuality, 80 children (75%) had scores in the low scoring range on the TSDQ, while 27 children (25%) had scores within the high scoring range. For data analysis, the target children were divided into two groups according to TSDQ guidelines, where there were 18 children designated to the high-scoring group, meaning their scores rated above the point of clinical concern.

Teachers in the intervention condition received one full day training session per month for five months, conducted by two trained leaders - a trained program leader and a certified program mentor. Nearly all (97%) of the teachers attended all five training sessions. Teachers in the control condition received no in-service or professional development opportunities during the study.

Sample Characteristics: Teacher characteristics: all teachers were female, with a mean age of 34 years and an average of nine years of teaching experience. Student characteristics: The mean percentage of students who were eligible for free school meals was 13% (slightly below the national average of 17%). Of the 107 participating children, 58 were boys and 49 were girls, with a mean age of 57.51 months. All children were Caucasian. All parents had to sign a consent for their children to be observed, and where consents were not given, the next highest scoring child was chosen. All participants received an incentive for participating in the study.

Measures: Outcome measures were collected at baseline and post-intervention. The Teacher-Pupil Observation Tool (T-POT) was used to measure the frequency counts of teacher and child behaviors through direct observation. Eight composite categories were used: teacher positives, teacher negatives, compliance, non-compliance, negative behaviors towards the teacher, prosocial behavior, deviance, and off-task behavior. This measure has good inter-rater reliability (78%). The T-POT measure was collected by three postgraduates with experience using similar observation measures. Observations were conducted during structured lessons, each child was observed for 15 minutes. At the same time, observers were coding teacher interactions with the class as a whole and class responses to the teacher and behavior generally. Coders were blind to the conditions they were observing.

Analysis: Independent t-tests were used to determine baseline equivalence. Analyses were intent-to-treat, using full maximum estimation likelihood for missing data. Analysis on outcome measures used hierarchical linear modeling to account for the clustered nature of the data, with post-intervention T-POT scores as dependent variables and intervention status as a classroom level random factor. Effect sizes were also calculated.

Outcomes

Fidelity Monitoring: The trainers for this study were certified to train the Teacher Classroom Management program. They also completed fidelity checklists (Teacher Workshop Checklist) following each training session. Participating teachers completed a satisfaction questionnaire at the end of implementation and teacher workshop evaluations at the end of each training workshop.

Baseline Equivalence and Differential Attrition: Observational data was available on all children at baseline and all but one student (from the control group) at post-intervention. All teachers were retained for the duration of the study. There were no significant baseline differences in child, teacher, or school demographics, nor were there differences in the number of children in the low and high TSDQ groups between conditions at pre-intervention.

Posttest:
Classroom-level findings: On teacher and classroom behavior observational measures, only one of eight measures was significant: classroom off-task behavior. Effect size (ES) for this outcome was large (.53).

Child-level findings: There were three significant program effects on teacher and index children's behavior, with moderate effect sizes: reductions on teacher negatives (ES = .36), child negatives to the teacher (ES = .42) and child off-task behavior (ES = .48). The intervention also had a significant effect on child compliance to commands (ES = .37). Further analysis on this measure showed a significant reduction in teacher commands to intervention group children (ES = .48), whereas teachers in the control group showed an increase in total number of commands. Also, children in the intervention group showed an increase from 68% to 81% in compliance to commands, with no change among control group children. When comparing the low and high Total Difficulties (TD)-scoring children, at post-intervention, there was a significant interaction effect on two measures: child negatives to teachers and child off-task behavior. The intervention was more beneficial to children who initially scored high on the TSDQ Total Difficulties, whereas children who scored low followed the same pattern from pre- to posttest as children in the control group (increases in child negatives to teacher and no change in child off-task behavior).

Study 2

Evaluation Methodology

Design: Twenty four community pre-schools in inner-city areas of Kingston, Jamaica were recruited to participate in the study, which took place from October, 2009 until June, 2010. Criteria for schools to be included in the study were: there were at least three to four classes of children with at least 20 students per class, schools needed to be situated in a specified geographical area and all teachers needed to consent to participate. Of the 50 schools that were initially approached, 24 met the criteria and were recruited to participate. Teachers completed an interviewer-administered screening questionnaire for each child (10 items measuring conduct disorder on the ICD-10 Classification of Mental and Behavioral Disorders), to determine which children were at high risk for subsequent externalizing problems. The three children from each class with the highest scores were enrolled to participate in the evaluation. Children were excluded from the study if they had attended school less than 70% of the time, had a developmental disability, lived in an institution, or were a sibling of an enrolled child. Based on this criteria, 24 children were excluded, and replaced with the next highest-scoring children. In total, 225 children from 73 classrooms were included in the analysis (113 intervention, 112 control).

Randomization occurred at the school level to either the intervention or control group. Each school contained one class per age group of 3-, 4-, and 5-year old children (one school had an additional class).

All intervention school teachers and principals received eight days of training for the Incredible Years Teacher Training Program, and training included adapting the program to be culturally appropriate. Adaptations included a) using culturally appropriate videos (Jamaican classrooms), b) training occurred over 8 days instead of 5-6 days, c) adding exercises and activities and emphasizing certain skills-building modules that were designed based on the experiences and concerns of the teachers, d) placing more emphasis on certain aspects of the intervention based on teacher feedback, and e) making handouts culturally appropriate. Adaptations were intended to increase teachers' self-confidence and their understanding and application of the skills introduced in the program and to enhance their ability to generalize these skills in their classes. Beyond training and to ensure fidelity, teachers received assistance with modeling, coaching, and other support and feedback each month for four months. Education materials were also provided to both intervention and control teachers on teaching strategies. Control group teachers also attended the regular in-service workshops provided by the Ministry of Education.

Sample Characteristics: The majority of the children in the study were male (59% in intervention, 63% in control), and the average age was 4.2 years. Most children lived with their mother, although less than half had a father living at home. Less than half of caregivers had completed high school. Most of the teachers in the study were female.

Measures: Data was collected at baseline and post-intervention. Researchers who collected the data were blind to condition. The primary outcome measure - directly observed in-class child behavior, was measured by observing each child for 5 minutes on a rotational basis (with 3 children per class) for a total of 15 minutes per day per child over 4 days and across different times of the school day, which resulted in 1 hour of observation. Event recording was used to count aggressive/destructive behaviors and friendship skills (in frequencies per hour). The behaviors were chosen based on the Dyadic Parent-Child Interaction Coding System and Multi-Option Observation System for Experimental Studies behavior categories. Observers rated the frequency of conduct problems, activity level, on-task behavior and follows rules/expectations of the classroom using seven-point scales.

Secondary outcome measures on child behavior were collected from teachers and parents. Teacher-reported measures included the Sutter-Eyberg Student Behavior Inventory (SESBI) to measure child conduct problems; the Conner's Global Index to measure hyperactivity and attention difficulties; the Strengths and Difficulties Questionnaire (SDQ) to measure behavior difficulties and prosocial skills, and the Preschool and Kindergarten Behavior Scales (PKBS): Social Skills Scale to measure social skills. Parent-reported measures included the Eyberg Child Behavior Inventory (ECBI) and the SDQ.

Also measured was child attendance (taken from school records) and parents' attitude to school, which was measured through a 10-item questionnaire. This measure was only collected at post-intervention.

Analysis: Analysis was conducted using multilevel multiple regression models on an intent-to-treat basis.

Outcomes

Baseline Equivalence and Differential Attrition: There were no significant differences between groups on child, family, classroom or school characteristics at baseline. At post-intervention, data from 15 children lost at follow-up was missing. The authors reported no significant differences between those who dropped out of the study and those who remained, but no statistics were presented.

Posttest: Significant outcomes were found on observed child behavior, including reductions in conduct problems and increased friendship skills. The intervention also had a significant effect on teacher-reported child behavior difficulties and social skills and parent-reported child difficulties. There were significant differences between groups on the proportion of children being rated in the clinical range for conduct problems, with fewer children in the intervention group being rated by teachers (but not parents) as in the clinical range at post-test, compared to control group children, and less likely to have pervasive conduct problems. Official records showed better child attendance for intervention children.

Study 3

Evaluation Methodology

Design: Eleven schools in or around the Limerick, Ireland area were recruited to participate in the study. Two teachers from each of the schools (n=22) agreed to participate. The teachers were randomly assigned to either the treatment or control condition on a 1:1 ratio for each school. At baseline, each teacher completed the Teacher Strengths and Difficulties Questionnaire (T-SDQ) for each child in their class (total n=445). Class sizes ranged from 11 to 29 students. From these results, a cross-section of index children (12 children per classroom) was determined for inclusion in the analysis. Based on the 'total difficulties' measure of the T-SDQ, the index children represented the four highest scoring, four lowest scoring, and the four middle-scoring children from each class.

Children were excluded from the study if they had a formal diagnosis of a developmental disorder, if parental consent forms were not returned, or if they were absent from school on more than one occasion during baseline assessment, yielding a baseline population of 234 index children. Seventeen children (7%) were lost to follow-up, so analyses for the study only included the remaining 217 (107 control group and 110 intervention) index children.

Sample: The majority (n=7) of the participating schools were designated as 'disadvantaged,' meaning they received a greater level of support in terms of student-teacher ratios, special grants, and extra support for students. Participant teachers in the study had been assigned either to a Junior or Senior Infant class. Most teachers (n=15) were between the ages of 25 and 34 years, had spent an average of 9 years teaching, with an average of almost 5 of those years teaching infant classes, and all but one were female. Of the total population (n=445) of children (ages 4-7) taught by the study teachers, teachers indicated that about 1/4 showed significant levels of social, emotional, and/or behavioral difficulties at baseline, and 42% were rated by teachers to be above the recommended cut-off indicating the presence of hyperactivity. The index population of children included 102 boys and 115 girls.

Measures: The Teacher Strengths and Difficulties Questionnaire (T-SDQ) and the Conners Abbreviated Teacher Rating Scale were used to measure the nature and severity of child emotional and behavioral problems, which included: emotional symptoms; conduct disorder; peer-relationship problems; and hyperactivity in the classroom. In addition, observational measures were captured to evaluate classroom management and child behavior outcomes using the Teacher-Pupil Observational Tool (T-POT). This tool measured teacher-pupil interactions in the classroom, and frequency counts of teacher behaviors and index child positive and negative behavior. Reliability of 73% was achieved and where possible, the research team was blind to the condition.

A self-reported teacher measure, the Teacher Strategies Questionnaire, was used to collect data relating to teachers' confidence in managing challenging behavior in the classroom, as well as their use of positive strategies, negative classroom management strategies, and strategies to improve home-school links. The questionnaire also captured teacher perceptions of the utility of the classroom management strategies. Intervention teachers also completed a Teacher Satisfaction Questionnaire at follow-up to assess their perceptions of the intervention.

Analysis: Changes in teachers' classroom strategies from baseline to follow-up were assessed by computing mean differences in teacher scores. Independent sample t-tests were then used to examine significance. ANCOVA was used to assess teachers' self-reported use of classroom management strategies. Analyses of child outcomes were conducted at the classroom level. Regression modeling was used to examine pre- and post-test differences in child behaviors, adjusting for clustering effects. Effect sizes were calculated for both teacher and child outcomes. Sub-analyses on child outcomes for the index group divided the children into two groups: 1) a 'high risk' group (children who scored a 12 or higher on the SDQ 'total difficulties' measure; n=63), and 2) a 'low risk' group (children who scored 11 or lower on the SDQ 'total difficulties' measure; n=154). Analysis of child outcome measures was conducted using robust regression analysis, controlling for clustering.

Outcomes

Posttest: At post-intervention, there was a significant reduction in observed incidence of teacher negatives in the intervention group, with intervention teachers using significantly fewer harsh and critical statements, as well as fewer negative commands in their interactions with students, when compared to control group teachers. There was a significant increase in giving more time to students to comply with instructions and questions among intervention teachers compared to control teachers. Measures approaching significance included use of positive classroom management strategies and use of direct commands among intervention condition teachers, compared to control condition teachers.

On self-reported use of classroom management strategies at post-intervention, there were significant between-group differences favoring the intervention group, indicating that teachers using the Incredible Years Teacher Classroom Management Program found the classroom management strategies both easy to implement and useful. The intervention group of teachers also reported significant decreases in frequency of use of inappropriate strategies for managing misbehavior, compared to control group teachers, who actually reported some increases in frequency and perceived usefulness of these strategies. Intervention group teachers were also generally satisfied with the IY program, indicating that the program was appropriate for managing classroom behavior, and 64% indicated they would recommend the program to other teachers. All of the intervention teachers also indicated that they felt more confident in their ability to manage classroom behavior problems.

There were a few child outcomes as reported by teachers to note. Intervention group children showed significant improvements in emotional symptoms (self-regulation and cooperation skills), compared to control group children. There were intervention group decreases in total difficulties score on the Strengths and Difficulties Questionnaire, and decreases in conduct problems approached significance, compared to scores from the control group. Hyperactivity, peer problems, and prosocial behavior did not differ.

In the sub-analysis of index children according to levels of behavioral need, there was a significant impact of the program on the high risk group of intervention children on total behavioral difficulties, peer problems (improved interpersonal skills), and distress and social impairment, compared to control group high risk children, as reported by teachers.

Outcomes from classroom observations of the index children were not statistically significant, but showed moderate effect sizes (described above), for intervention group children on measures of negative responses to teacher-initiated behaviors and non-compliant behavior.

Study 4

Evaluation Methodology

Design: For the first study, two Teacher Classroom Management (TCM) courses were conducted in 2003-04 with a total of 23 teachers (11 in one and 12 in the other). Teachers were nominated by their employers (two Authorities in North West Wales) to participate in the training, and attended either because they were interested, or their employer identified a need. Courses ran for one day per month for five months and teachers received feedback on classroom assignments in between course classes. The majority of teachers (65%) attended all training sessions.

The second study used 21 classrooms in three schools in Gwynedd. Schools were recruited to participate in the study. Observers were blind to condition, and each school had classrooms where teachers had and had not completed TCM training. Teacher and student behavior was coded continuously and simultaneously during 30-minute sessions. Each teacher was observed for one 30-minute session.

Sample Characteristics: All but one of the teachers was female and had an average age of 37 years, and teachers had been teaching an average of 17 years.

Measures: In the first study, data collection measures included the Teacher Satisfaction Questionnaire (TSQ), which measured four areas: usefulness of the program, confidence in using the ideas, difficulty/ease of putting the program into practice, and use of strategies for improving home-school links. A total satisfaction score was also calculated. TSQ data was collected from 20 teachers. Also collected was qualitative data, conducted via interview with 21 (91%) of the teachers once the course was completed and teachers had implemented the program in their classrooms. Two independent coders coded the transcripts with high inter-rater reliability (k = .82).

The observational component of the second study took place with 21 teachers from three primary schools in Gwynedd, where 10 teachers were trained in the TCM program and 11 were not. Measures were collected using the Teacher-Pupil Observation Tool (T-POT), which measured 12 teacher behaviors and 17 child behaviors. Scoring consisted of frequency counts of each behavior. The 29 items were then reduced to six global teacher and four child categories: Teacher measures - teacher positive, teacher negative, teacher praise, teacher indirect command, teacher direct command, and no opportunity given for compliance; Child measures - child positive behavior, child negative behavior, child compliance, and child non-compliance. Three coders were used to complete the observations. Observer scores were comprised of frequency counts of each behavior. Inter-rater reliability on the scoring was 70%.

Analysis: Mean scores for the first study were calculated with the TSQ data, ratings were on a scale of 1-5. In the second study, a Mann-Whitney U test was performed to determine differences between groups. Effect sizes were also calculated.

Outcomes

Posttest: In the first study, teachers indicated a high level of overall satisfaction with the program for all scales examined: usefulness of the program (4.5), confidence in delivering the program (4.6), ease of putting the program into practice (4.8), use of strategies to improve home-school links (3.5) and total satisfaction (4.2). The qualitative interview also revealed, overall, high satisfaction with the program, with teachers reporting benefits both to themselves as teachers (increased range of effective strategies and greater use of existing positive strategies) and their students (increased attention, more considerate of others and increased confidence). Teachers also indicated making use of the teaching strategies taught, with few barriers to implementation reported, and that the training helped them become better equipped to deal with difficult child behaviors. Most (91%) teachers indicated they would recommend the program to others.

Observational outcomes from study 2 indicate significant program effects on four of ten observed measures (2 teacher and 2 child). Specifically, teachers trained in the TCM program had significantly higher rates of using direct commands with children, and gave children significantly more time to comply with commands, compared to teachers who were not trained. For children, students in classrooms where the teacher was trained in TCM showed significantly less non-compliant behavior and significantly more positive behaviors, compared to students of untrained teachers. The effect sizes on the significant measures were large (direct commands = .94, no opportunity = .79, non-compliance = 1.13, child positives = .99).

Study 5

Evaluation Methodology

Design: This study was part of the Chicago School Readiness Project (CSRP), which provided teachers in Head Start classrooms in Chicago with weekly coaching support as part of their workforce development and adult training. Head Start sites that offered full day programming (2 or more classrooms) were recruited from seven urban neighborhoods in Chicago that had high rates of poverty. There were a total of 18 participating sites, which included 35 classrooms. Sites were randomized using a pair-wise matching procedure. The Incredible Years Teacher Classroom Management intervention was implemented with two cohorts of teachers and their classroom students, with cohort 1 implementation occurring from fall to spring 2004-05 and cohort 2 implementation occurring from fall to spring 2005-06. There were 94 participating teachers, and 65 (69%) consented to complete the teacher surveys. Four of the teachers either moved or quit during the school year, yielding a final dataset of 90 teachers. Intervention teachers were asked to participate in five six-hour long training sessions. Each teacher spent an average of 18 hours in training.

Each classroom receiving CSRP intervention services was assigned a weekly Mental Health Consultant (MHC) who attended all 5 teacher training sessions with the teaching staff and were available in the classrooms one morning a week (for an average of 4.54 hours per week). These consultants, or coaches, provided encouragement and feedback to teachers on their use of the classroom management strategies taught through the training sessions. Coaches were also available to provide stress management for the teachers. The MHCs held a master's in Social Work. Through CSRP, targeted children also received one-on-one child-focused mental health consultations. Control group classrooms were assigned a Teacher's Aide for the same amount of time as the MHCs (averaging 5.18 hours per week).

At baseline, there were 543 children attending Head Start and participating in the study. By the spring, there were 509 children participating (some children left while others entered the program and were added to the study).

Sample Characteristics: Demographic measures were only available for the 69% of teachers who completed surveys. Most teachers (97%) were female and the average age of teachers was 40. The majority of teachers were African-American (70%), followed by Latina (20%) and White (10%). Three quarters of the teachers held a post-secondary degree.

Participating children were those children for whom parental consent was obtained (M=91%). Classrooms consisted primarily of ethnic minority children (67% African American and 26% Latino). Classrooms averaged 15 to 16 children. Additional demographic information on the children was not provided.

Measures: Measures were collected at baseline and post-intervention. Data was collected using classroom observations. Twelve trained observers who were blind to condition collected classroom-level data using the Classroom Assessment Scoring System (CLASS) and the Early Childhood Environment Rating Scale, revised edition (ECERS-R). CLASS Observations were conducted one day a month in three sessions per day - breakfast, "circle time/free time" and lunch. The resultant data utilized the mean scores across the three observation sessions. The CLASS measure tested the impact of the intervention on classroom quality, using four scales: positive climate, negative climate, teacher sensitivity, and behavior management. Three quarters of the observations were double-coded by two observers to gauge inter-rater reliability, which ranged from .66 to .82. The ECERS-R was used to measure early childhood classroom quality. Items were scored on a scale from 1 to 7. ECERS-R data was also double-coded (43% of the time) to assess inter-rater reliability (.87).

Analysis: Analyses were intent-to-treat. Cohort membership was treated as a covariate in the analysis. Post-hoc repeated measures ANCOVA were conducted to yield covariate-adjusted estimates of means and standard deviations for treatment and control groups for the four dependent variables. Classrooms were nested within treatment and control groups, and Hierarchical Linear Modeling (HLM) was used to present class- and site-level outcomes. Effect sizes were also calculated.

Outcomes

Baseline Equivalence and Differential Attrition: There were no baseline differences between groups of teachers on any of the demographic or educational variables. There were also no baseline differences across sites on any of the teacher, child, or site measures assessed. Teacher attrition was low (5%) and child attrition reached 16%. Attrition analyses indicate that there were no statistically significant differences between the children who remained in the program compared to those who exited early. There were also no significant differences between the children who enrolled at baseline compared to those who enrolled later in the year, with the exception that those who were enrolled later were somewhat younger. There were also no significant differences between groups on remaining in the program compared to those exiting early, and only one significant difference between groups on those who began at baseline and those who were enrolled later, in that there were significantly more girls than boys who entered the control classrooms, compared to equal numbers of girls and boys entering the treatment classrooms late.

Posttest: Two of four classroom quality measures were significant: positive climate and negative climate, where intervention classrooms showed significant improvement compared to the control classrooms. Effect sizes were large (.89 for positive climate and .64 for negative climate).

Study 6

Evaluation Methodology

Design: Head Start centers were assigned to either the intervention or control condition. Assignment was not completely random, but based on staffing and transportation needs. Participants were screened and recruited through Head Start registrations over a two year period. Eligible families included 96 caregivers and their 103 preschool-aged children. Eligibility requirements were not described in the article. Fifty-nine children were assigned to the intervention group and 37 children were in the comparison group. All participants received monetary incentives for completing the assessments, and comparison group families received resources for mental health services.

Nine teachers participated in the study (6 intervention and 3 control). Intervention lasted a minimum of four months. Intervention teachers received one introductory training session on the intervention, and then participated in weekly individualized consultations that focused on classroom management strategies, increasing the use of effective and proactive discipline, and strengthening positive teacher-child relationships. Adaptations to the program model included the use of mental health consultants, who were two advanced graduate level students in clinical psychology. Teachers received support based on their particular classroom needs, and therefore, not all teachers received consultation on all areas of the Teacher Training Series.

Intervention families were all encouraged to participate in a 10-session Incredible Years Parent Training program, and 35% (n=21) chose to participate (i.e., attended at least 50% of the sessions). There were no group differences in terms of the severity of caregiver or teacher reports of child behaviors. However, significantly more caregivers who attended the sessions were older, married, and experienced less parental distress.

Sample Characteristics: Most of the participants were African-American (86-92%) and mean age of the children was 4.5 years. The majority of children were boys (68-72%) and lived in single parent homes (76-84%). The Total Hollingshead score was 25.8 for the intervention group and 24.6 for the comparison group.

Measures: Measures were collected at baseline, post-intervention, and one year post treatment. Attrition was somewhat high. Sixty-three (66%) participants had a complete data profile (both caregiver and teacher reports) for the baseline and posttest time points, with the remainder having only caregiver (10%) or teacher (19%) assessments at posttest or neither (5%). One year follow-up data was collected from 63% of caregivers.

Specific child behaviors (ADHD and Oppositional Defiant Disorder/ODD) were measured using the ADHD-IV Rating Scale, which was completed by both caregivers and teachers. The ADHD scale was amended to include additional symptoms of ODD and Conduct Disorder (CD). Broad externalizing behaviors were measured using the Behavior Assessment System for Children (BASC), also completed by both caregivers and teachers. Caregivers completed 3 additional measures: the Parenting Stress Index-Short Form on 4 scales of stress: child; caregiver-child relationship, caregiver, and total; the Parenting Scale, which contained three scales: laxness, over-reactivity, and verbosity; and the Child Behavior Management Questionnaire, which measured caregiver knowledge of behavioral principles as they apply to parenting. Teachers completed the Teacher Strategies Questionnaire (TSQ), which measured effective teaching strategies, both on difficulty and usefulness, on two scales: positive strategies and limit setting. All measurement tools were reported to have high reliability ratings.

Analysis: Hierarchical Linear Modeling was used to determine change over time on measures. For teacher measures, a two-level random intercept model was used to estimate the hierarchical/organizational structure of the data where children were nested within classrooms. For caregiver measures, a series of three-level random intercept and slope hierarchical models were used. Three waves of data for each of the three time points were nested within children and children were nested within classrooms. Analyses were intent-to-treat.

To measure clinical improvement in children's disruptive behavior, an examination was made of the differences in the number of children who evidenced at least 1 SD improvement (at baseline) in their score for at least one measure of disruptive behavior for the intervention versus the comparison group.

Outcomes

Baseline Equivalence and Differential Attrition: At baseline, there were no significant differences between the intervention and comparison groups on any of the demographic variables, nor on the severity of child behavior problems or eligibility requirements. There were also no differences between intervention and comparison group classrooms, in terms of the quality of classroom environment, teacher qualifications/experience, or teacher-child ratios. Attrition analyses revealed no significant differences between those who completed the post-intervention assessment and those who did not, with one exception. Children without teacher post data were rated by teachers as having higher initial levels of ADHD and ODD behaviors than children with teacher post data. These children were also rated by their caregivers as having higher initial levels of ADHD and externalizing behaviors overall. For one-year follow-up data, there were no differences in the rates of non-completion by group or in baseline statistics between participants with follow-up data and those without.

Posttest: Outcomes on child disruptive behavior were mixed, in that teacher reports showed significant differences favoring the intervention group, while parent reports did not. Teacher measures also indicated significant program benefits on use of effective teacher strategies, compared to control group teachers. Among caregiver-only measures, two of five measures were significant at posttest: caregivers in the intervention group reported significantly decreased verbosity and increased knowledge of behaviorally-based child management strategies (behavior management), compared to caregiver reports in the comparison group.

Clinical improvement in child disruptive behaviors showed that caregivers in the intervention group reported 64% of children improved at least 1 SD on one measure of disruptive behavior, compared to 33% of the children in the comparison group. Teachers in the intervention group reported that 55% of the children improved at least 1 SD on at least one measure compared to 30% of children in the comparison group.

One-year Follow-up: Child behavior management differences among caregivers remained significant at the one-year follow-up, but program effects were lost on parenting skills (verbosity).

Study 7

Evaluation Methodology

Design: Participant teachers were solicited via advertisement to complete an Incredible Years Teacher Training Course. Fifteen teachers from the Education Center in the Greater Dublin Area were recruited on a first come basis to complete the course from February to June 2007 (two consecutive afternoons a month for five months). The schools represented by these teachers were both suburban and rural.

Measures: Measures were collected at baseline, post-intervention, and at one-year follow-up (teacher measures only). Teachers completed the Incredible Years Teacher Strategies Questionnaire (TSQ) to measure the frequency of use and the usefulness of five teacher strategies: praise and effectiveness, proactive strategies, limit-setting, total positive approaches, and inappropriate strategies. Frequency of use of strategies that involved working with parents was also measured. Teachers also completed the Teacher Sense of Efficacy Scale to measure change in teacher perceived self-efficacy on four measures: student engagement, instructional strategies, classroom management, and total efficacy. Child behaviors were measured using the Strengths and Difficulties Questionnaire (SDQ) on six measures: emotional symptoms, conduct problems, hyperactivity, peer problems, total difficulties, and prosocial behavior. Impact of Difficulties (on self, peers and class) was also measured. The SDQ was completed per teacher on a number of children upon whom they chose to focus their teacher management skills. A total of 51 SDQs were completed (between 1 and 4 per teacher).

Qualitative measures were also completed by teachers to assess the quality of the course, using the Incredible Years Evaluation form.

Analysis: One-tailed tests were conducted on all measures. All quantitative measures were analyzed using the Wilcoxon Signed Ranks Test. Frequencies were used to analyze the qualitative measures.

Outcomes

Posttest: There were significant improvements on four of six child measures (SDQ): hyperactivity, peer problems, prosocial behaviors, and total difficulties. Effect sizes on these measures were medium to large (.46, .47, .52, and .62, respectively). Improvements on the remaining two measures (emotional difficulties and conduct problems) approached significance. There was also a significant improvement over time on the Impact of Difficulties measure in the proportion of children perceived as having no or minor difficulties as opposed to definite or severe difficulties. On teacher measures, three of four measures of teacher efficacy showed significant improvement, both at posttest and one-year follow-up: student engagement, classroom management, total teacher efficacy. There were no significant outcomes on any of the Teacher Strategies Questionnaire frequency measures, and only one of five perceived usefulness measures was significant: limit-setting.

Qualitative outcomes indicate that teachers were satisfied with the course and the program and felt that the strategies used were useful.

Study 8

Evaluation Methodology

Design: Participants were recruited from families requesting treatment for their child's conduct problems at the University of Washington Parenting Clinic. One-third of the families were self-referred, the others were referred by professionals (teachers, physicians) in the community. Families entered the study in three 50-55 family cohorts in the fall of 1995, 1996 and 1997. Families of 159 4- to 8-year-old children with oppositional defiant disorder were randomly assigned to one of six conditions: parent training (PT; n=31), child training (CT; n=30), parent training plus teacher training (PT+TT; n=24), child training plus teacher training (CT+TT; n=23), parent/child/teacher training (PT+CT+TT; n=25), and a waitlist control group (n=26). The waitlist control group received no treatment and had no contact with the research team or the team's therapists during the 8-9 month waitlist period. After the post-intervention assessment, control group families were offered the parent training program. Baseline assessments were conducted in early Fall, post-intervention assessments were conducted approximately six months later in the Spring, for all six conditions in each of the three cohorts. Follow-up assessments were conducted one year later in the Spring for five of the six conditions, because the control group was treated after post-intervention assessments.

Of the entire sample that completed baseline assessments, four families dropped out before beginning treatment and refused to participate in post-assessments. The rest of the sample completed post-assessments regardless of how many sessions attended. There was no significant difference in drop-out rate by treatment condition. No teachers dropped out of the TT condition.

Interventions: In the Child Training intervention, children were offered weekly two-hour sessions for 18 to 19 weeks for about six months with two therapists and six to seven children (Dinosaur School). The Parent Training intervention involved 10 to 12 parents meeting with 2 therapists for 2-hour sessions over the course of 22 to 24 weeks. Teacher Training involved 4 full days (32 hours) of group training sequenced throughout the school year.

Sample Characteristics: The children in the study were 90% boys, 79% white, with an average age of 5 years and 11 months. 74% of the parents were married.

Measures: Composite measures were calculated by adding multiple instruments in five areas:

Positive and Negative Parenting: Two parent-report measures (Parenting Practices Interview and Dyadic Parent-Child Interactive Coding System-Revised) and two observational measures (Coder Impressions Inventory and Parent Daily Discipline Inventory) were used to assess parenting styles and skills. The Parenting Practices Interview (PPI) has internal consistency alpha coefficients of .71 for harsh discipline and .66 for supportive parenting. The Dyadic Parent-Child Interactive Coding System-Revised (DPICS-R), which looks at positive parenting and critical statements, has a Cronbach's alpha of .78. The Coder Impressions Inventory-Parent (CII-P) is completed following a half-hour parent-child observation where parents are measured on 12 harsh-critical items (Cronbach's alpha = .89; interclass consistency coefficient = .54), 13 nurturing-supportive items (Cronbach's alpha = .88; interclass consistency coefficient = .67) and a rating on a 5-point scale of the observers perception of the degree to which a family needs help (interclass consistency coefficient = .64).

Child Conduct Problems at Home: Composite scores were calculated using one parent-report variable (the Eyberg Child Behavior Inventory - ECBI; Cronbach's alpha = .92) and four in-home observations including the DPICS-R coding system and the Coder Impressions Inventory for Children (CII-C).

Child Conduct Problems at School: The composite scores included two teacher report variables (scales from Teacher Assessment of School Behavior - TASB - Cronbach's alpha ranges between .62 to .91 ; and pertinent scales from the Teacher Rating of Perceived Competence Scale for Young Children - PCSC - reliability ranges between .70 to .90 for the subscales) and two summary scores from independent observations of classrooms.

Child Social Competence with Peers: The composite score includes two teacher-report variables (relevant scales from TASB and PCSC), one classroom observation (30-minute observations calculating conduct problems; Cronbach's alpha = .71) and one laboratory observation of the child with a peer (observation focused on Inappropriate Play scale from DPIS; internal consistency = .88).

Negative Classroom Management: A composite score of negative classroom management and atmosphere was computed via direct observation for each teacher using five variables: total teacher criticism; observation of classroom atmosphere (CAM, Cronbach's alpha ranging from .94 to .95) measured on a 10-item questionnaire; and three items from the Coder Impression Inventory - Teacher which rates harsh techniques, nurturing techniques and percentage of time teacher is inappropriate.

Analysis: The procedures were consistent with an intent-to-treat analysis. The four families who dropped out before treatment began were followed up, but they refused to complete post-assessments. Treatment effects for each measure were analyzed using six-group analysis of covariance with pretest scores as covariates for corresponding posttest scores. Planned comparisons contrasting each treatment condition with the control condition were conducted and then the addition of TT to CT and to PT was tested against CT alone and PT alone. Lastly, PT+CT+TT condition was tested against two factor conditions (CT+TT and PT+TT). Missing data were handled at two levels: an individual summary score was only computed if at least 60% of the items that made up the scale were present. Composite scores were also only computed if at least 60% of the summary scores in the composite were present. Cases were excluded from analysis on that composite if the composite score was missing at one of the time points. This resulted in sample sizes that fluctuated differently for different composite scores and treatment groups. The CT only condition had the most fluctuation at follow up for the father's positive parenting (n dropped from 30 to 22 at follow up).

Outcomes

Implementation Fidelity: Therapists conducting the parent or child group co-led their first group with a supervisor, followed a treatment manual for each session and documented fidelity to the program through a weekly protocol checklist of standards to be covered in each session. Group sessions were videotaped for feedback and analysis at weekly supervision meetings. Therapists received ongoing supervision, feedback and training throughout the study. Supervisors also randomly selected videotapes for integrity checks and the study reports that the analysis of checklists indicates high treatment integrity. Teacher training sessions were manualized, videotaped and the tapes were reviewed to ensure that training procedures did not vary across cohorts of teachers.

Attendance at the children and parent groups ranged between 90% and 100% for at least 15 sessions among the CT, CT+TT, PT, PT+TT and CT+PT+TT groups. All teachers attended the four days of training and all attended at least two meetings to work on individualized behavior plans for the child.

Baseline Equivalence: The study used ANOVA and chi-square analyses to compare baseline equivalence across all six conditions. No significant differences were found at baseline among all groups on demographic or family background variables. Further, there were no significant differences between conditions at baseline on any composite scores.

Differential Attrition: From the entire sample that completed baseline assessments, only four families dropped out prior to the beginning of the study and refused to participate in post-assessments. No information was provided about how these four families differed at baseline from those that continued in the study. At the two-year follow up, 9% of the families in the treatment groups (n=12 families) dropped out. Significantly more families in the CT condition dropped out. The children in these families who dropped out at two-year follow-up had fewer behavior problems at post-assessment.

Post-test (Webster-Stratton, Reid, and Hammond, 2004):
 Following the six-month intervention, all treatment conditions resulted in significantly fewer conduct problems, both at home and school, for children compared to controls. However, when compared with each other, there was little significant difference between treatment groups.

The following results at posttest specifically related to the teacher program in combination with either the child program or the parent program were found:

• reduction in conduct problems at home (with both mothers and fathers) and at school;
• teachers less negative;
• less negative parenting by mothers.

One-year post intervention (Maintenance effects, no control group):
Because the control group was treated after post-assessments, these follow up results were only focused on whether children and parents in the treatment conditions improved or maintained effects in the following eight areas: mother negative parenting; father negative parenting; mother positive parenting; father positive parenting; child conduct at home per father; child conduct at home per mother; child conduct at school; and, child social competence. Mixed design (Time x Condition) ANOVAs were computed for each composite score from post-assessment to follow up. Seven of the eight areas were not significant at one-year follow up. Only child conduct at school was significant at one-year follow up (F=3.45; p<.01). However, seven of the eight measures were maintained to the 1-year follow-up. School behavior of children in the PT+CT+TT deteriorated from post-assessment to follow-up.

Two-years post intervention (Reid et al., 2003):
At two-year follow-up, parents were asked if their children received additional services or treatment after completing the program, and what changes in medications their children received. Results indicated no significant differences for any of these variables among treatment conditions. On ratings of conduct problems at home, comparison among treatment groups showed that children in the PT+TT group had significantly better outcomes that in the PT alone condition (chi-sq=5.27; p<.02). On ratings of conduct at school, the number of children showing significant improvement according to teacher reports maintained or improved for all five treatment groups. However, there were no significant differences between treatment groups.

Study 9

Evaluation Methodology

Design: Using a randomized control design, 14 Head Start centers (36 classrooms) were randomly assigned to two groups:

• An experimental condition in which parents, teachers, and family service workers participated in the prevention programs (BASIC + ADVANCE + SCHOOL + TEACHER) (n=191, 23 classrooms from 9 centers)
• A CONTROL condition in which parents, teachers, and family service workers participated in the regular center-based Head Start program (n=81, 13 classrooms from 5 centers).

Centers were chosen based on their willingness to participate in the study and to be randomly assigned to either intervention or control. Parents were recruited into the experimental condition by Head Start family service workers and teachers during summer and fall orientation. Originally, 328 families indicated interest in participating in the study (225 experimental and 103 control). Thirty-four families (15%) from the experimental centers and 22 families (21%) from the control centers did not complete spring posts assessments. This left 272 families (191 experimental and 81 control).

The experimental condition included four components: the standard BASIC parent training program, an abbreviated version of the ADVANCE and academic skills training (SCHOOL) programs, and a teacher training program (TEACHER). The first three components were held for parents of children attending Head Start and consisted of 16 weeks of two-hour group training sessions led by Head Start family service workers. The twelve-week standard BASIC program was offered to parents in the Head Start year, and a four-week abbreviated version of ADVANCE and SCHOOL was offered in the kindergarten year. Whereas the ADVANCE component targeted parent problem-solving skills and strengthening their relationships with partners, the SCHOOL program promoted parents' increased communication with teachers; techniques for encouraging children's reading, academic and problem-solving skills; arranging child "play dates;" and coaching positive peer play skills. The final component, the TEACHER training program, focused on improving teachers' classroom management and discipline skills, building relationships with students and parents, and helping instructors promote students' social and emotional competence. All Head Start teachers and aides in the intervention group received six monthly workshops sequenced over the first year of the intervention. Those in the control centers received the usual Head Start services, which included parent education in stress management, nutrition, self-care, and dental care.

Sample Characteristics: This study included a sample of 272 Head Start mothers, 272 four-year old children and 61 teachers. As in Study 5, many of the families in this study faced multiple risk factors for child conduct problems, including low education, low income, parent substance abuse, depression, and spouse or child abuse. The sample of children was 54% male with an average age of 55 months. 63% of the families represented ethnic minority groups, including predominantly African American, Hispanic, and Asian American ethnicities, and 52% of the families were single-parent households.

Measures: Assessments were conducted at baseline, post intervention, one year, and two years post intervention (through grade one). They included parent reports of children's behavior at home (using the CBCL and ECBI); teacher reports of children's conduct problems at school (using the Social Competence and Behavior Evaluation, SCBE, to measure social competence, emotional expression and adjustment difficulties; and the teacher ADHD checklist to assess the presence of attention deficit hyperactivity disorder); and blinded observations at home and at school (assessing children's noncompliance and deviance at home, and conduct problems and the amount of time engaged in activities at school). Construct scores combining observational and report data were calculated for negative (harsh discipline, critical parenting) and positive (monitoring, praise, consistent discipline, positive affect) parenting styles; parent-teacher bonding (primarily assessing increased communication between parents and teachers and parent involvement in school activities); child conduct problems at home (noncompliant, aggressive, and disruptive behaviors) and at school (including children's aggression, noncompliance, ADHD symptoms, engagement and social competence); and observer reports of teacher classroom management style (including teachers' positive techniques and harsh discipline, and classroom atmosphere).

Analysis: Analyses of covariance on the post-treatment construct scores using the corresponding pretest construct score as a covariate, was conducted. When these ANCOVAs showed significant group effect for the construct, then adjusted posttreatment means were examined to determine if the experimental group improved significantly more than the control group.

OUTCOMES:

Fidelity Monitoring: Monthly supervision was provided for parent group leaders coupled with close monitoring, direct observations and detailed training sessions and training manuals. 100% of the group leaders reported discussing all videotaped vignettes assigned and assigning all homework sessions described. For teachers, workshop checklists, standardized handouts, and videotaping of all teacher training was used to assure the integrity of the intervention.

Baseline Equivalence and Attrition: Analysis compared baseline equivalence of the experimental and control groups of the sample who completed both pre and post assessments (n=272). Analyses revealed several differences between groups at baseline, with the experimental group reporting significantly more risk factors than control group members. Specifically, the intervention group contained fewer boys, more minority members, higher rates of mother depression and stressful life events, and lower annual incomes. There were also significant differences in two of the six construct domains. Intervention mothers had higher negative parenting scores and lower positive parenting and bonding scores. Because of the differences on the two construct scores, analyses of covariance on the post-treatment construct scores using the corresponding pretest construct score as a covariate, was used.

Attrition analyses done at the one-year follow-up indicated no significant differences in dropouts between groups in terms of demographic and baseline risk factors. Differences between drops and nondrops on risk, demographic, and construct scores were examined separately for intervention and control groups. There were no differences on any variable in the intervention group, and only one difference between drops and nondrops in the control group. More families from the control group who reported children with behavior problems dropped out between pre- and post-assessments.

Posttest: At post assessment, intervention mothers had significantly lower scores on the negative parenting construct and higher scores on the positive parenting construct compared to control mothers. Parent-teacher bonding was marginally significant in the full sample and significantly higher in a subsample of intervention mothers who attended six or more intervention sessions than for control mothers. In terms of children's behaviors, intervention children showed significant improvements on the conduct problems school construct (including lower reports of hyperactivity and antisocial behaviors and more social competence), marginally significant results on the home construct, and children of mothers who attended six or more sessions showed significant improvement on the conduct problems at home construct, compared with control children. In addition, children who were in the "highest risk" category (based on observations of high rates of non-compliant and aggressive behavior) at baseline showed greater clinically significant reductions in aggressive and noncompliant behaviors than high-risk children in the control group. Teachers' behavior also improved, with instructors in the intervention group having significantly higher scores on the classroom management construct than control teachers.

Long-term: When baseline scores were compared to the one-year follow-up measures, some intervention effects remained. Only four of the original six construct scores were examined (negative and positive parenting, parent-teacher bonding, and child behavior problems at home). Observer reports of teachers' classroom management and children's behavior at school were not available. There were marginally significant effects in the predicted direction for the Positive Parenting and the Child Conduct Problems at Home constructs. Parents who attended more than nine sessions continued to have lower scores on the negative parenting construct and higher scores on the positive parenting construct. Clinically significant findings on children's conduct problems found at post assessment continued to be present one year later. This difference was significant with the whole sample, as well as with the children of parents classified as "attenders". Parent-teacher bonding was significant at the one-year follow-up, but in the reverse direction, with intervention mothers showing less bonding with teachers than control mothers. Analyses of the two-year follow-up results are currently being conducted.

Study 10

Evaluation Methodology

Design:

Recruitment: Head Start centers that are part of a larger Head Start Cares program volunteered to participate in the program. To select a diverse sample, the study included centers in 10 states and 4 regions of the country. The study separately examined 3- and 4-year old children in 4-year old and mixed 3 and 4-year old classrooms. Among 4-year olds, 79% of parents consented at baseline to have their children participate, and 90% consented by the posttest.

Assignment: After being blocked on race/ethnic composition and part-day/full-day schedules, centers were randomly assigned (N = 104) to one of four conditions: Incredible Years Teacher Training, a control, and two other interventions. Analysis was then conducted with the separate interventions and, for 3-year olds, with a treatment group that combined all three interventions. The control group included 220 3-year-old children and the Incredible Years treatment group included 246 3-year-old children. For 4-year-old children, the control group consisted of 621 children and the Incredible Years treatment group consisted of 702 children.

Attrition: For the study of 3-year olds, the child pretest occurred in the fall and the posttest in the spring. The study reported approximately 12% attrition in the Incredible Years treatment group and 19% attrition in the control group among 3-year-olds. However, Table B6 (page 69) lists 211 control children and 214 Incredible Years children at baseline, while the analysis results in Table 8 (page 41) list the full sample of 220 control children and 246 Incredible Years children.

For the study of 4-year olds, the child pretest in the fall and posttest in the spring were supplemented by a 1-year follow-up in the spring of kindergarten. The design allowed for children who did not have consent at baseline to be added for the posttest and follow-up when parents gave consent late. Figure 3.1 thus shows 2,128 children who consented, were selected, and were in the center at baseline, and it shows 868 who were not consented or selected at baseline but were added later. The total equals 2,996. At posttest, the number remaining in the centers was 2,670 (89%). The study thus reported 11% attrition (p. 249). The study tracked 2,599 (87%) into kindergarten.

Note that the pretest occurred from September to December, possibly months after the program began.

Sample: Children in the 3-year-old sample were an average of 3.42 years old and roughly half female (50.44%). Children in the 4-year-old sample were an average of 4.38 years old and roughly half female (48.08%). No information is provided on race/ethnicity of the 3-year-old child sample. The sample of Head Start centers suggests socioeconomically disadvantaged participants.

Among 4-year-olds, the sample was 43% Hispanic, 33% Black, and 16% White. The sample shows evidence of being disadvantaged; 59% of participating children's families were eligible for the Supplemental Nutrition Assistance Program (SNAP) and the average income was $1,800 per month.

Measures: All measures of child outcomes reported for 3-year olds came from teacher reports. The study used 1) the Behavior Problems Index (BPI), which measures the frequency, range, and type of children's behavior problems, and had scales for total problems, externalizing, hyperactivity, and internalizing; 2) the Cooper-Farran Behavior Rating Scales for work related skills and interpersonal skills, 3) Social Skills Rating System, which measures children's' ability to regulate their own behavior, assert themselves to solve conflicts with peers, and ability to cooperate, 4) Student-Teacher Relationship Scale, which measures the teacher's perception of the quality of their relationship with individual children. Finally, the study examined pre-academic skills, using the Academic Rating Scale of early language and literacy, mathematical thinking, and general knowledge skills.

The study of 4-year olds supplemented preschool teacher reports with assessments done by "independent observers" and by kindergarten teachers who presumably did not know of group assignment. For child behavior outcomes, the study used Head-to-Toes, pencil tap, and the Cooper-Farran Behavioral Rating Scales (CFBS) to measure executive function and used the Behavior Problems Index (BPI) to measure behavior regulation, and learning behaviors. The study used the facial emotions identification task, emotions situation task, challenging situations task, the social skills rating system (SSRS), and the Cooper-Farran Behavioral Rating Scales (CFBS) to measure interpersonal skills. For pre-academic skills, the study used the Woodcock-Johnson III Letter-Word Identification test of pre-literacy and literacy skills and applied problems (math), the expressive one-word picture vocabulary test (EOWPVT) of vocabulary, and the Academic Rating Scale (ARS) for early language and literacy, math, and general knowledge skills.

To measure teacher outcomes among both 3-year-old and 4-year-old samples, the study used the Adapted Teaching Style Rating Scale (Adapted TSRS) to measure classroom management, social-emotional instruction, and scaffolding, in addition to the Classroom Assessment Scoring System (CLASS), which measured emotional support, classroom organization, instructional support, and literacy focus.

Analysis: The two studies analyzed two-level models for children within classroom and classrooms within centers. The models included fixed-effects dummy variables for the blocks in which centers were randomized, random effects for centers, and controls for baseline outcomes.

Intent-to-Treat: All participants with available data were included at each assessment point, but the study did not follow children who left the centers.

Outcomes

Implementation Fidelity: The study did not discuss implementation fidelity for 3-year-olds. For 4-year-olds, the study stated that implementation fidelity for Incredible Years exceeded "satisfactory." Mattera et al. (2013) reported that the intervention provided teachers with materials, training, coaching, monitoring and technical assistance. Coaches spent 90-minutes in each classroom observing and then meeting with teachers an average of 3 times per month. Monthly fidelity logs were completed by coaches where teachers received a fidelity rating on a 1 (low) to 5 (high) scale. Teachers on average received a rating of 3.69 from coaches on the fidelity scale. The study also reported that fidelity improved over time.

Baseline Equivalence: For the 3-year olds, tests for child age, gender, and outcome measures at baseline in Table B6 found no significant differences between the Incredible Years treatment group and the control group. However, the table includes only a subset of the cases. Tests for teacher and classroom characteristics revealed only two significant differences in race/ethnicity of teachers.

For the 4-year olds, only one significant child difference (p < .05) emerged from about 30 tests (Table B4). The Ns for the tests (541 for Incredible Years and 512) for the control group are again smaller than the full sample of randomized subjects. Tests for teacher and classroom differences (Tables B2 and B3) showed a few marginal differences and significantly lower scores in emotion coaching (p<.05).

For both 3- and 4-year olds, the occurrence of pretests after the program start may have affected the baseline equivalence tests.

Differential Attrition: The study of 3-year old children found lower rates of attrition among the Incredible Years treatment group than in the control group (12% versus 19%, p < .10) and reported (page 95) that higher-risk children were more likely to dropout. Specifically, children who left the centers had significantly higher levels of total behavior problems and externalizing and internalizing behavior problems than did those who stayed. An additional check examined whether differential attrition produced condition differences at baseline among those who remained at posttest. To quote page 97 for 3-year-olds, "Among children who did not leave the sample, there were no statistically significant differences between the Incredible Years group and the control group." The results suggest that attrition did not produce an unbalanced sample.

For 4-year olds, tests showed that children who left the centers were more likely to be male, be rated as having lower levels of social behavior, learning behavior, and pre-academic skills, and perform poorly on executive function and cognitive tests than those who stayed (Table Q.1.). Among Incredible Years subjects, attrition was 11% (significantly different at p < .10 than the 15% for control group). Table B5 tests for baseline differences among the analytic sample and therefore reflects the influence of attrition. It shows one significant difference (of 30 plus tests) on baseline social problem solving.

Posttest: For 3-year old children, the study found significant posttest differences for the Incredible Years classrooms as compared to the control group in 5 of the 12 teacher-reported social-emotional outcomes (social skills rating scale, assertion, cooperation, self-control, and closeness of the student-teacher relationship) and all 3 teacher-reported pre-academic skill outcomes (general knowledge, language, mathematical thinking).

Among 4-year-olds, the study reported no significant intervention effects for directly assessed pre-academic skills, but significant intervention effects for all 3 teacher-reported pre-academic skills: general knowledge, language and literacy, and mathematical thinking. The study also found a significant intervention effect for the teacher-reported work-related skills measure (p<.05), facial emotions and social problem-solving (p<.05), and teacher-reported social behaviors (p<.05).

For teacher and classroom measures, the study did not find any statistically significant posttest differences for 3-year-olds (p < .05) on 17 measures of teacher practice or 14 measures of classroom climate between the Incredible Years treatment group and the control group. Of 14 teacher-reported measures of teacher behavior for 4-year-olds, teachers in the Incredible Years treatment group reported significantly better scores for 3 of 6 measures of classroom management and 1 of 6 measures of social-emotional instruction. Of 11 teacher-reported measures of classroom climate among four-year-olds, teachers in the Incredibly Years treatment group reported improved on 1 of 4 measures of emotional support and 1 of 3 measures of classroom organization.

Mediation Analysis: The study did not conduct formal mediator analysis.

Long-Term: The study did not include a long-term follow-up for 3-year-olds and no significant effects were found at the one-year follow-up for 4-year-olds on measures of behavior and academics. However parents did report significantly greater receipt of special education services, which the authors interpret in positive terms as identifying child problems early.

Study 11

Evaluation Methodology

Design:

Recruitment: All children in 120 Seattle area Head Start programs and 14 elementary schools were invited to participate in the program over 4 years. Schools were selected based on a larger proportion of students receiving free or reduced price lunch. In total, 86% of the parents of children in the selected Head Start programs and 77% of the parents of students in the elementary schools (kindergarten and first grade) consented to participate.

Assignment: The study matched pairs of Head Start centers and schools based on size, geographic location, and demographics. The matched pairs were then randomly assigned to either the intervention or a waitlist control group. The sample size seems to vary between measures and the study is unclear about how many teachers were assigned to the treatment or intervention groups. For classroom and teacher measures, there were 120 classrooms with 153 teachers. For the observer measure of student conduct, there were 1,786 students nested in 160 classrooms and 119 teachers.

Attrition: A posttest assessment at the end of the school year followed a pretest assessment in the fall. For the teacher and classroom measures, the study reported that missing data reduced the number of classrooms from 120 to 115 (96%) and the number of teachers from 153 to 139 (91%). For the student measures, missing data reduced the number of students from 1,786 to 1,746 (98%).

Sample: Students in the study were, on average, 63.7 months of age and were 50% male. The sample was 18% Hispanic, 18% black, 20% Asian, 27% white, 8% African, and 9% other. Thirty one percent of the sample did not speak English as a first language.

Measures: The study measured social and emotional competencies, conduct problems, teacher competencies, teacher efforts to involve parents, and classroom environment through teacher reports and independent observations of teachers and students. The study also administered the Wally Problem-Solving test to a high-risk subsample. The measures showed good reliability.

Observers were blind to condition. To assess teachers, observers utilized the Multiple Option Observation System for Experimental Studies to examine positive reinforcement, critical statements, and amount of interaction with students. In addition, they used the Teacher Coder Impressions Inventory to evaluate teachers' styles and classroom management. The Teacher Coder Impressions Inventory rated teachers on harsh/critical, inconsistent/permissive, warm/affectionate, social/emotional, and effective discipline.

To assess child behavior, coders used the Multiple Option Observation System for Experimental Studies to assess total conduct problems, time disengaged or off-task, time in solitary place, child positive with teacher, child positive with peers, and time in peer involvement. They also used the School Readiness and Conduct Problems: Coder Observation of Adaptation-Revised to evaluate child school readiness based on emotional self-regulation, social skills, and conduct problems.

To assess classroom atmosphere, observers used the Classroom Atmosphere Measure to examine overall level of students' cooperation and problem solving, interest in subject matter, focus, responsiveness, on-task behavior, and classroom support.

The study piloted the Wally Problem Solving and Feelings Tests during the latter half of the study among students with more significant conduct problems, as reported by teachers, parents, or school counselors.

Teachers rated parent involvement using the INVOLVE-T questionnaire, which evaluates the extent to which parents participate in school activities, seem comfortable with the teacher and school environment, value education, support the teacher, and assist children with their homework.

Analysis: The study used pre-post ANCOVA multilevel models that allowed for both classroom- and teacher-level variation in intercepts. For the teacher outcome analyses, a three-level model nested repeated measures within teachers and teachers within classrooms. For the student outcome analyses, the four levels included repeated measures, students, classrooms, and teachers. The random effects could differ in each model, depending on tests for significance and convergence of the estimates.

Intent-to-Treat: The sources of missing data and the procedure used to deal with the missing data are unclear. Nearly all students were included (98%) in the student analyses and most teachers were included (90%) in the teacher analyses.

Outcomes

Implementation Fidelity: The study found that on average, 27 of the 30 required lessons were completed in the intervention classrooms and an average of 30 recommended vignettes and 25 small group activities were completed in the intervention classrooms.

Baseline Equivalence: The study reported no significant differences between conditions for schools, students, or teachers on demographic variables, but it did not discuss outcome variables at baseline.

Differential Attrition: The study did not discuss differential attrition, though missing data for students was low.

Posttest: For the observer rating of teacher styles, 4 of 5 constructs had significant main effects. As compared to control teachers, intervention teachers became less harsh/critical and inconsistent/permissive, more warm/affection, and placed more emphasis on social/emotional teaching. There was no significant main effect for effective discipline, however there was significant improvement among Head Start teachers only.

Observer ratings of teacher classroom management showed a significant effect for 1 of 3 measures. Teachers in the intervention condition became significantly less critical after the intervention as compared to control teachers. Neither teacher involvement nor teacher praise showed a significant effect.

Observer ratings of school readiness found a greater improvement in school readiness for intervention students as compared to control students. None of the 6 child constructs for conduct problems and disengagement had significant effects. However, 2 of the outcomes were significant for students with high baseline conduct problems.

Observers found a greater improvement in the classroom atmosphere in the intervention as compared to the control groups.

A subsample of children with conduct problems reported improvement in positive problem solving strategies and positive feelings they could identify. However, these findings are preliminary given a small sample size.

The teacher report of parent involvement found a significant effect for parent-teacher bonding, but no effect on three other measures of parent involvement.

Mediation Analysis: The study did not conduct formal mediator analysis.

Long-Term: The study did not include a long-term follow-up.

Study 11, continued
Thompson, et al. (2017)

Evaluation Methodology

Design:

Recruitment: Using the same data set as Webster-Stratton, Rei and Stoolmiller (2008) in which matched pairs of schools/Head Start centers were randomly assigned to condition, this study conducted an analysis on a sub-set of kindergarten teachers and their students entering kindergarten. Given the information provided in the text, it is not clear whether the school/center-level randomization is lost by examining just this subset of teachers.

Assignment: This study included 42 kindergarten teachers and 805 students entering kindergarten, with 19 teachers and 504 students in the treatment group and 23 teachers and 301 students in the control group.

Attrition: According to the Webster-Stratton, Rei & Stoolmiller (2008) study, teacher retention was 96%, but this study did not report attrition specifically for the subset of kindergarten teachers and their students.

Sample: A majority of teacher participants were female (97%) and white (75%), an additional 22% were African American. A majority of students were male (51%) and were 5.6 years old (on average). Nearly all students qualified for subsidized lunch, 39% of students' families identified as white, 20% as Asian, 13% as African American, and 17% as Hispanic and other.

Measures: The study created latent profile models using the Teacher-Parental Involvement Questionnaire (INVOLVE-T). In this questionnaire, teachers rated their perception of the extent to which parents participate in school activities, seem comfortable with the teacher and school environment, value education, support the teacher, and assist children with their homework. Three subscales were analyzed:

• Bonding, which gauged teacher perceptions of whether a teacher felt parents appeared comfortable and had a good relationship with a parent (sample α = .76).
• Parental involvement in education, which examined teacher perceptions of whether parents were involved in school activities or if they were supportive of educational goals (sample α = .91).
• Parental involvement with teacher, which assessed teacher perceptions of whether parents called, attended conferences or visited the school (sample α = .84).

Analysis: Latent class profile models for each of the three outcomes (bonding, parental involvement in education, and parental involvement with teacher) were estimated using FIML, and included a cluster to adjust for intraclass correlations.

Intent-to-Treat: The study used a Full Information Maximum Likelihood (FIML) procedure to account for missing data and the analysis examined outcomes for all cases with available data.

Outcomes

Implementation Fidelity: No significant departures from protocol or principles were noted. On average, teachers received 3.73 days of training and only four teachers attended less than four days of training. Checklists completed by researchers indicated that, on average, teachers and students were exposed to 87% of all planned intervention elements.

Baseline Equivalence: The treatment and control classrooms did not differ at baseline on two out of three indicators used to model the profiles. At baseline, teachers reported similar ratings of general parental involvement and bonding with parents, but teachers in the control group reported having better relationships with parents compared to treatment teachers.

Differential Attrition: The study reported that there was no relationship between missingness and observed student or teacher characteristics or treatment assignment. However, the study reported attrition for teachers overall, not for the kindergarten subsample, and did not provide the details of the attrition analysis.

Posttest: The study reported that there was a significant, but small, association between assignment and transition pattern (Cramer's V =.143). Teachers in the treatment group were more likely to transition to a more adaptive view of parental involvement (as defined by improved perceptions of bonding and involvement of parents at home and school) at follow-up compared to teachers in the control condition. Though the study refers to several tables and figures reporting results, these tables were not included in the report. Findings were therefore based on what was included in the authors' text.

Mediation Analysis: The study did not conduct a formal mediator analysis.

Long-Term: The study did not include a long-term follow-up.

Study 12

Evaluation Methodology

Design:

Recruitment: This study recruited nine urban schools in a Midwestern part of the United States that included a majority African American students. All the schools were implementing a Positive Behavior Supports and Interventions program with high fidelity, which may not be typical of other schools. Of all 109 eligible teachers, 96% enrolled and were randomized, and 84% of 2,168 eligible students consented to participate. Teachers were recruited and randomized across three sequential cohorts.

Assignment: The enrolled teacher participants were randomly assigned to either the intervention group or the control condition within each of the schools according to a blocked cluster randomized design. There were 53 teachers and 900 students allocated to the IY program group and 52 students and 917 students assigned to the control group. The three sets of two-day workshops were delivered in October, December, and February for the intervention group, while the control group classrooms continued their usual curriculum.

Attrition: Data were collected at three time points: beginning of the school year, immediately prior to intervention (October), and post-intervention (April and May). The CONSORT diagram showed 98% of the teachers and 92% of the students (1680 of 1817 randomized and consented students) completed the follow-up and were used in the analysis. Among those who completed the follow-up, the maximum rate of missing data in the final samples used for analyses was 1.8%.

Sample: The teacher participants were mostly female (97%) and white (75%), with 11.1 years of teaching experience on average. The student sample was 52% male, mostly African American (76%), and 61% qualified for free or reduced lunch.

Measures: Teachers (who helped deliver the program) filled out the 54-item Teacher Observation of Classroom Adaptation-Checklist to measure four child behaviors: concentration problems, disruptive behavior, prosocial behavior, and emotional dysregulation. Cronbach's alpha for these measures ranged from 0.77 to 0.96.

Teachers also filled out the Revised Social Competence Scale-Teacher version. Cronbach's alphas ranged from 0.93 to 0.96 for the social competence subscale and from 0.92 to 0.93 for the academic subscale.

Standardized tests of student academic performance in reading and math were obtained using the Woodcock-Johnson III Normative Update Tests of Achievement, and the test-retest reliability for this measure ranged from 0.80 to 0.95.

Analysis: Three-level hierarchical linear models were used to analyze the main effects of treatment on student behavior and academic performance. With students nested within teachers and teachers nested within schools, the models adjusted for random assignment of classrooms and associated clustering. Student-level covariates, including the pretest outcomes, were controlled at the first level.

Intent-to-Treat: The study lost 8% of the students who did not complete the posttest but otherwise used all subjects with data and imputed any missing items.

Outcomes

Implementation Fidelity: Almost all teachers attending all six workshops (attendance rates ranged from 94 -100%), and if teachers had to miss a workshop due to illness or other reasons they met with the coach to review the missed material.

Baseline Equivalence: The authors stated that the conditions were equivalent at baseline but did not report effect sizes or significance tests.

Differential Attrition: There were no tests for differential attrition of those lost to follow-up. A brief mention that the largest differential missing rates between intervention and control groups were 0.7 at posttest appears to refer to missing data among those completing the posttest.

Posttest: There were significant main effects of the program on teacher-rated prosocial behavior (d = 0.13, p = 0.03), emotional dysregulation (d = -0.14, p = 0.00), and overall social competence (d = 0.13, p = 0.02). The program failed to significantly improve the measures of academic outcomes, although the teacher-rated measure was marginally significant (p = .07). It had no effects on the two independent measures of reading and math achievement.

There was also a moderating effect of child pretest levels of social competence on posttest social competence reports (b = -0.09, p = 0.04), and a moderating effect of pretest academic competence scores on increases in academic competence over the course of the program, compared to controls (b = -0.08, p = 0.00).

Long-Term: Not examined.

Study 13

This study analyzed changes in teacher-reported perceptions of parental involvement in their children's education, but does not analyze a Blueprints behavioral outcome.

Evaluation Methodology

Design:

Recruitment: Teachers were recruited across three sequential, annual cohorts from nine urban schools serving primarily African American students. No more information was provided on how recruitment occurred. Teacher and parent participants provided written consent and students provided written assent.

Assignment: An experimental design was used in which teachers were randomly assigned to either the intervention or a wait-list control condition. Initially, 109 teachers and 2,168 students were deemed eligible for the study, but 4% of eligible teachers (n = 4) and 16% of eligible students (n = 350) declined to participate prior to randomization. A total of 105 teachers and 1,818 of their students were therefore randomized to condition. The intervention group included 53 teachers and 901 students, while the control group included 52 teachers and 917 students. Control group teachers continued their business-as-usual teaching and professional development opportunities.

Attrition: Baseline data for several measures not included in this write up (as they are not relevant to the experimental study) were collected in October and follow up data were collected in April of the same academic year. The study reported no teachers were lost to follow-up, while 63 students in the intervention condition (7%) and 57 students in the control condition (6%) moved out of the study schools. At follow-up, the sample therefore included 105 teachers and 1,698 students.

Sample: All students were in grades kindergarten through third grade. The majority of teachers were female (97%). Most (75%) were white, and an additional 22% were African American. Slightly over half the student sample was male (52%), and 76% of students were African American while 22% were white (with 2% identified as "other"). Half the student sample qualified for free or reduced price lunch.

Measures: The study measured teacher perceptions of parental involvement in their children's education using the Parent Involvement Measure-Teacher, which was completed for each student. Two subscales, Parent-Teacher Contact and Parent comfort, were included in the analysis examining differences in outcomes between treatment and control teachers. For the control group sample, test-retest reliability was .57 for the Parent-Teacher Contact subscale and .68 for the Parent Comfort subtest. Reliability was not reported for the treatment group or for the overall combined sample. Several other measures, including standardized achievement tests, classroom observations, and teacher ratings of parent and student behaviors, were included in the report but were not in the analysis examining differences between treatment and control groups. These measures were therefore excluded from this write up.

Analysis: The study used latent transition analyses, accounting for students clustered within classrooms, to examine changes in teacher perceived parental involvement from baseline to follow-up. Covariates included student socio-demographic characteristics (i.e., sex, race, and lunch status).

Intent-to-Treat: The study only excluded students missing all data. FIML procedures were used to account for missing data.

Outcomes

Implementation Fidelity: Attendance at workshops ranged from 94-100% for each of the six workshops, and coaches followed up with absent teachers to review the missed material. Teachers reported high satisfaction with the training workshops, and on average teachers received 6 hours of additional coaching between and after workshop sessions. Observations conducted in late April and May of the same academic year revealed that treatment teachers used higher rates of proactive teacher strategies compared to control teachers, though it is not clear who conducted these observations.

Baseline Equivalence: The study reported that baseline equivalence was demonstrated for outcome measures but no tests of baseline equivalence were reported for socio-demographic variables.

Differential Attrition: The study did not report analysis of differential attrition.

Posttest: Program outcomes included teacher perceptions of parent involvement patterns and the progression from a less adaptive to a more adaptive pattern. The study found that teachers in the control condition reported 7% of parents in the lower two adaptive classes at baseline transitioned to a more adaptive class at follow-up, whereas teachers in the treatment condition reported 39% of parents did the same. Of parents rated to have transitioned to the highest adaptive class at follow-up, 80% were in the treatment group. A chi-square test of transition types (those transitioning to higher classes, those staying the same, and those transitioning to lower classes) found that the differences between conditions were significant in favor of the treatment group. No effect sizes were reported.

Mediation Analysis: The study did not conduct a formal mediator analysis.

Long-Term: The study did not include a long-term follow-up.

Study 14

Evaluation Methodology

Design:

Recruitment: Schools that had previously implemented the program's parent training intervention were eligible to apply for participation, resulting in 21 intervention schools. Another 22 schools were recruited from rural and urban municipalities to act as waitlist control schools. No participating schools had implemented any other evidence-based school behavior intervention program for at least a year prior to baseline. Potentially participating children were identified by randomly selecting 7 children from each classroom (grades 1, 2, 3) for screening (n=1343). Only children who scored in the clinical range for externalizing behaviors were included in the study (n=83).

Assignment: Schools were non-randomly selected to serve in intervention (n=45 students) or waitlist control conditions (n=38 students).

Attrition: Total student attrition at posttest was 21%, due to school dropout, teacher dropout, and family/student dropout. The number of schools lost was not described.

Sample:

The student sample was split between grades 1 (33.7%), 2 (22.9%), and 3 (43.4%) and was predominantly male (78.3%). No other student demographic characteristics were described, but all children met diagnostic criteria for externalizing behaviors.

Measures:

Participants were assessed at baseline and posttest (approximately 9 months after baseline). All measures were rated by teachers, who also delivered the intervention.

Externalizing problems were measured with the Sutter-Eyberg Student Behavior Inventory-Revised, which consists of 7-point Likert items that evaluate frequency and severity of common behavior problems. These problems include temper tantrums, outing, acting defiant, difficulty staying on task, trouble paying attention, and failing to complete tasks or projects. Total scores are summed and range from 38 to 266, though a score above 144 is considered in the clinical range (α=.97).

Internalizing problems and academic performance were assessed using the Teacher Report Form of the Achenbach System of Empirically Based Assessment family of instruments. The internalizing instrument contains subscales of Anxiety/Depressed (α=.79), Withdrawn (α=.72), and Somatic Complaints (α=.53). Academic performance was assessed by informally comparing individual student scores to class averages across six different academic subjects.

Social skills were assessed using the Social Skills Rating System, which measures the occurrence and importance of specific social skills, academic competence, and behavioral problems. The instrument covers a wide range of behaviors, including making friends easily, temper control, getting along with others, and following directions. Internal consistency was high, α=.94.

Student-teacher relationship was assessed using the Student-Teacher Relationship Scale, which is comprised of two subscales, closeness and conflict. The closeness subscale (α=.81) measures the degree of affection and open communication between student and teacher, while the conflict subscale (α=.84) measures the extent to which that relationship is characterized by antagonistic, disharmonious interactions.

Analysis:

The effects of the intervention were evaluated using multilevel mixed-effects linear models, with measurements nested within individuals nested within schools.

Intent-to-Treat: Although most attrition occurred in the intervention schools, missing data were treated as missing at random within the individual, and all available data were used in the analysis.

Outcomes

Implementation Fidelity:

Fidelity was not assessed. The authors note that teachers "received either verbal or written feedback/guidance from the group leaders on classroom based practice of new skills," but do not mention the frequency of classroom observations nor any rating system.

Baseline Equivalence:

There were no significant baseline differences between groups on any demographic characteristics or outcome measures.

Differential Attrition:

There was a total attrition rate of 21%, with most of this (93%) occurring in the intervention group. No formal tests were used, but the authors imply that the difference in rates was significant.

Posttest:

At posttest, compared to the control group, participants in the intervention group showed significant improvement in teacher-reported student-teacher conflicts and academic performance. There were no significant group differences on any other outcomes.

Study 15

Evaluation Methodology

Design:

Recruitment:

The Incredible Years Norway program invited Norwegian municipalities where the Incredible Years Parenting Training program was already implemented to participate in the trial. A total of 3,161 students and 92 teachers were recruited for the study, 46 teachers (n=1,527 students) in the treatment group and 46 teachers (n=1,634 students) in the control group. Teachers in the intervention group received the program free of charge and a small economic compensation for the time spent in completing questionnaires. The comparison group consisted of teachers recruited from both rural and urban municipalities in which none of the Incredible Years programs were available. Interested kindergarten teachers applied to Incredible Years Norway to participate in the comparison group. They received economic compensation for time spent on administering assessments and were offered training in the Incredible Years Teacher Classroom Management program one year later. To be included in the study, none of the kindergarten staff could have attended an evidence-based kindergarten behavioral intervention or program in the course of the previous year. Parents who did not speak or understand Norwegian were excluded (though not explicitly stated, it is assumed this means the children of parents who did not speak or understand Norwegian were excluded), and if the kindergarten received the program, all children in the classroom indirectly received the intervention (whether or not they participated in the study).

Assignment:

Authors conducted a quasi-experimental study with no matching. A total of 86 teachers were assigned to a condition and completed a pretest assessment, though it was not clear why this number dropped from the 92 teachers originally recruited for the study. In each department (which is assumed to be a classroom, but was not clearly stated), the authors randomized seven children to participate in the trial. That is, 7 children from each classroom in the study were randomly selected by a statistician "blind to characteristics of the classroom" (p. 2218) to be assessed. For example, in a group of 21 children, a random number list from 1-21 was generated electronically, and the kindergarten teachers matched the 7 numbers with the children's alphabetical order. The authors claimed this was done to "reduce data dependency" (p. 2218). This resulted in 1,218 students assigned to condition for the purpose of the study, 581 children in treatment and 637 in control.

Attrition:

At the pretest, there were 1,049 students within 86 kindergarten classes that completed an assessment (for a student-level attrition rate of 14% and a teacher-level attrition rate of 7%). At the posttest, there were 90 teachers (though it was not clear how this number increased from 86 at the baseline assessment) and 908 students who completed an assessment, for a student-level attrition rate of 25%. Attrition rates at the teacher level could not be calculated given the numbers presented in the consort diagram (see Figure 1, page 2217).

Sample:

Demographic information at the student level was only reported by condition. Students' average age for treatment and condition was 4.3 years and 4.4 years (respectively). In addition, 50% of the treatment group and 48% of the control group were girls, while 12% of the treatment group and 9% of the control group were deemed high-risk.

Measures:

Participants were assessed at baseline and posttest after program implementation. All measures were rated by teachers, who also delivered the intervention. Measures included:

• Sutter-Eyberg Student Behavior Inventory-Revised: 38 items that measure the frequency (alpha = .94) and intensity (alpha = .97) of various types of problem behaviors.
• Child Teacher Report Form Internalization Scale: Four subscales which rate children behaviors on 99 items. Subscales include: (1) emotional reactive, (2) anxious depressed, (3) somatic complaints and (4) withdrawn. Though reliability and validity for this measure was not specifically reported, the authors stated that: "Test-retest reliability and validity on the C-TRF have been found to be high" (p. 2219).

Risk and protective factors were assessed using one measure:

• Social Competence and Behavior Evaluation for Teachers: A 40-item tool assessing social competence that summarizes the positive characteristics of a child. Test-retest reliabilities range from .74 to .87.

Analysis:

Linear mixed models were utilized to assess differences between groups in outcomes from pretest to posttest. In the analysis of the entire group, the data were hierarchically organized with observations (level 1) nested within children (level 2), and children were nested within teachers (level 3). Though the authors state the data were nested, they did not specifically report whether the analysis controlled for clustering. In addition, moderation analyses were conducted on a sub-group of children deemed "high-risk," as measured by those who scored equal to or above 138 on a baseline measure of the Sutter-Eyeberg Student Behavior Inventory Total Intensity score (n = 106).

Intent-to-Treat:

ITT was not explicitly discussed, nor was how missing data were handled, but it appeared that ITT protocols were followed.

Outcomes

Implementation Fidelity:

Not reported.

Baseline Equivalence:

Baseline equivalence was met for demographic and baseline outcome measures for all students who completed the pretest (n=1,049) with one exception - children in the treatment group were slightly younger than children in the control group. Baseline equivalence was not tested on the assigned sample.

Differential Attrition:

Children who attrited differed significantly from those with complete data on the behavior intensity measure at baseline. In terms of attrition by condition, none of the baseline child measures differed significantly between the intervention and comparison groups in the analytic sample, however tests involving demographic characteristics were not conducted.

Posttest:

At posttest, as compared to control, students in the treatment group showed greater improvement in intensity, problem and internalizing behaviors. In addition, compared to control, students in the treatment group showed greater social competence (a risk and protective factor).

Moderation analysis found that high-risk students in the treatment group showed improved social competence as compared to high-risk students in the control group.

Long-Term:

Not conducted.