ParentCorps

Study 1

Summary

Brotman et al. (2011) conducted a cluster randomized controlled trial in eight public schools in New York City assigned to intervention and control conditions. Two consecutive cohorts of pre-K students with English-speaking parents were recruited, yielding 171 participating families. Assessments of effective parenting practices and child behavior problems occurred at baseline (fall semester prior to program initiation) and posttest in late spring of the same academic year.

Brotman et al. (2011) found at posttest that, compared to the control group, participants in the intervention group scored significantly better on:

• Teacher-rated child behavior problems composite (internalizing, externalizing, and overall adaptive behavior)
• Self-rated effective parenting practices

Evaluation Methodology

Design:

Recruitment: Two consecutive cohorts of pre-kindergarten children from eight high-poverty public elementary schools with universal pre-K classes were recruited from one New York City school district based on racial/ethnic diversity and poverty as assessed by the percentage of the school eligible for free school lunch. All families with children enrolled in the participating schools were eligible for participation if they had a primary caregiver who spoke English (n=410). Only 171 consented.

Assignment: The eight schools were randomly assigned (using a matched pairs procedure based on number and type of prekindergarten classes and school-level student demographics) to either intervention (n=4; 118 children) or control conditions (n=4; 53 children). However, consent differed across conditions, with 50% consenting in the intervention group and 31% consenting in the control group. While the control group received services as usual, teachers in all schools received professional development on the content of the family intervention prior to randomization.

Also, the study stated the following: "Two consecutive cohorts of Pre-K students were recruited. In the 2nd year, one control school with two half-day classes discontinued its UPK program; therefore, the second cohort does not include children from that school. To compensate, enrollment was opened in one of the three remaining control schools to include children attending two half-day classes that were not designated specifically for lower-income children."

Attrition: Assessments occurred at baseline in the fall and at post-intervention in late spring of the same academic year. Between baseline and posttest, 5% of families were lost to attrition (n=162 at posttest). However, varying levels of available data were described for the individual measures, ranging from 50-88% across both assessment periods.

Sample:

Study children were an average of 4.14 years old at baseline, and 56% were girls. Thirty-nine percent identified as Black, 24% as Latino, 13% as White, 12% as Asian, and 12% as mixed race/ethnicity. Thirty-two percent were raised in single-parent families. The average age of primary caregivers was 33.8 years, and most primary caregivers (88%) were mothers. More than half (53%) of primary caregivers were born outside the United States. Families had an average of 2.26 children, including the child that participated in the intervention.

Measures:

Measures were collected from teachers, children, parents, and home observations of parent-child interactions. While observers were unaware of experimental conditions, teachers were aware of school assignments, helped deliver the program, and rated the children.

Effective parenting practices were assessed via parent self-reports of effective disciplinary practices using the Parenting Practices Interview (α=.61-.70); an author-developed test of knowledge of effective parent behavior management practices called the Effective Parenting Test (no validity reported); and observations of parenting effectiveness in semi-structured interactions in the home utilizing the Global Impressions of Parent-Child Interactions - revised tool, which displayed somewhat poor interrater reliability (ICC=.54).

Child behavior problems were assessed using the preschool version of the teacher-rated Behavior Assessment System for Children, which measures overall adaptive behaviors, as well as internalizing and externalizing problems (α=.83-.94). Behavior problems in the classroom were assessed using the New York Teacher Rating Scale, which focuses on diagnostic descriptors of oppositional defiant disorder and conduct disorder (α=.73). This scale was combined with the externalizing-problems scale.

Predictors of academic achievement including parent involvement and child school readiness skills. Parent involvement was assessed with teacher ratings using the Involvement in Education scale of the Involvement Questionnaire (α=.90). School readiness skills were evaluated with the Developmental Indicators for the Assessment of Learning-3, a standardized test that assesses motor, language, and conceptual skills related to school readiness (test-retest reliability > .70).

Analysis:

The primary outcome variables were evaluated using multilevel regression analyses similar to MANOVAs, controlling for baseline outcomes and gender. These analyses allow for the simultaneous estimation of treatment impacts over multiple outcomes within a domain (e.g., child behavior problems), thus testing whether the impact differed across the outcomes or remained roughly equivalent within a domain. Tests included adjustments for classrooms and schools using random-effects models since schools were the unit of randomization. However, the level-2 sample size of eight is small. On request from Blueprints, the authors sent information indicating that the reported results used degrees of freedom for the denominator based on the number of teachers nested within schools and did not apply the Kenward-Roger Correction for degrees of freedom in SAS Proc MIXED to adjust for the small number of randomized schools. The SAS code also indicated that the results did not use sandwich estimators for the standard errors, which are inappropriate for small samples.

Intent-to-Treat: There were differing attrition rates at Time 2 for the different sources of data, with 12-27% for the teacher, child, and parent data, and 50% for home observation data. Multiple imputation was used to account for missing data on all measures except for home observation, which had poor compliance at baseline and was subsequently dropped from the analysis. The imputation was done separately for the intervention and control groups.

Outcomes

Implementation Fidelity:

Fidelity was measured with content and process checklists completed by group leaders, which indicated a high level of fidelity (>90%) to intervention manuals for each parent and child group session. However, parents did not attend all sessions, with 71% of families in intervention schools attending at least one group session and 54% attending 5 or more out of 13.

Baseline Equivalence:

Baseline equivalence was established on all outcome and demographic variables, except for the gender of participating children, with significantly more boys attending control schools.

Differential Attrition:

Tests found no differences in attrition by intervention condition or baseline outcomes, and only gender was significant for demographic variables (17% for boys, 7% for girls), though baseline-by-condition attrition was not tested.

Posttest:

Compared to the control group, parents of children in intervention schools displayed significant improvements in effective parenting practices (d=.50) and children had significantly fewer teacher-rated behavioral problems (d=.56) at posttest. The number of parenting sessions attended was significantly associated with this increase. The program did not significantly improve the teacher-rated measure of parent involvement or the standardized test of school readiness.

Study 2

The program for this study involved 1) group sessions for pre-K students and parents and 2) professional development for pre-K and kindergarten teachers and paraprofessionals. The professional development component for kindergarten teachers extended the program to kindergarten.

Summary

Brotman et al. (2013, 2016) and Dawson-McClure et al. (2015) used a cluster randomized controlled trial in which ten public schools located in disadvantaged neighborhoods in New York City were assigned to intervention and control conditions. For the sample of 1,050 participating children recruited from four consecutive cohorts, assessments of parenting practices, child behavior problems and academic achievement occurred at baseline (fall semester prior to program initiation), the late spring of the same academic year (interim), the end of the kindergarten school year (posttest), and through the end of second grade (long-term follow-up).

Brotman et al. (2013, 2016) and Dawson-McClure et al. (2015) found that, compared to the control group, participants in the intervention group showed significant improvements in:

• Standardized test scores at the end of kindergarten
• Teacher-rated academic performance at the end of second grade
• Teacher-rated child internalizing and externalizing problems at the end of second grade
• Parent and teacher-rated effective parenting practices at the end of kindergarten

Evaluation Methodology

Design:

Recruitment: Ten high-poverty schools with pre-K programs were recruited from two school districts in New York City neighborhoods. The trial aimed to enroll all pre-K students in four consecutive years with the inclusion criterion of having an English-speaking caregiver (7% ineligible). Out of 1,280 potential participating children, 1,050 students were deemed eligible and consented.

Assignment: Schools were matched into pairs based on school size and then randomized into either an intervention condition (n=5) or an education-as-usual control (n=5) condition. Of the 1,050 students who were eligible and consented to participate, 561 were from intervention schools and 489 were from control schools.

Attrition: The five assessments for this pre-K program came at the beginning of pre-K (baseline), the end of pre-K (interim), the end of kindergarten (posttest), the end of first grade (one-year follow-up), and the end of second grade (two-year follow-up). Thus, children enrolled at age 4 and were followed to age 8. At posttest, Brotman et al. (2013) reported an overall attrition rate of 15.5% (n=162) due to withdrawal, transfers to other schools, and generally being lost to follow-up. Additional missing data on outcome measures, about 10% for the standardized test scores, came from absences during the day of testing. There was somewhat higher attrition for the second set of posttest phone-based family assessments (Dawson-McClure et al., 2015), which were completed by only 831 (79%) parents.

The two-year follow-up (Brotman et al., 2016) focused only on students (n=792) from the last three years of the study, as the first year was seen as needed for capacity building. On page 1151, the authors state: "Teacher ratings were missing for 16 (2.0%), 111 (14.0%), 270 (34.1%), and 334 (42.2%) cases at follow-up from pre-K through second grade, respectively; the KTEA score was missing for 169 children (21.3%) in kindergarten and 348 (43.9%) in second grade."

Sample:

At the school level, 71.9% were low-income (eligible for free lunch) and majority Black (90.7%). The participant-level data confirmed this: participating children were evenly split on gender, 85.4% Black (10.2% Latino), and 60.8% low-income. Just under half (44.7%) lived in a single-parent household, and 46.5% had a parent with a high school degree or less. Their neighborhoods were comprised of 67% single adults, had a 9.5% unemployment rate, contained 36.5% low-income households, and were 85.4% Black and 7.9% Latino.

Measures:

A variety of measures were collected by trained staff who were blind to condition, but several other measures came from teacher ratings or parent reports that were not independent. Ratings by first- and second-grade teachers at the follow-ups should be independent.

Reading, Writing, and Mathematics Achievement was assessed at the end of kindergarten and second grade using the Kaufman Test of Educational Achievement, a widely used and validated test. The analysis combined the three topic scores into a single measure. Pre-K baseline measures were taken from the more developmentally-appropriate Speed Diagnostic Indicators for the Assessment of Learning.

Global Academic Performance was assessed at the start and end of the pre-K and kindergarten academic years using the combination of teacher reports on 1) academic problems from a validated scale and 2) an academic progress measure developed for the study. The composite rating of performance was correlated positively (r = .55) with standardized test scores.

Internalizing and Externalizing Behaviors were assessed at each of five time points using teacher ratings from the Behavioral Assessment System for Children, Second Edition. Specifically, the internalizing scale included items on anxiety, depression, and somatization; the externalizing scale included items on conduct problems, aggression, and hyperactivity. The two scales were standardized on age and sex and averaged to form a composite of mental health.

Parent-Reported Child Behavior and Conduct Problems were assessed at posttest using a modified version of the New York Rating Scale, which displayed good reliability (α=.75-.87).

Effective Parenting Outcomes were assessed at posttest using parent self-reports across three domains: positive behavior support, behavior management, and involvement in early learning. Knowledge of positive behavior support and effective behavior management was measured with the Effective Practices Test (validity not reported). Positive Behavior Support was measured with the Parenting Practices Interview, Positive Reinforcement subscale, which had adequate reliability (α=.66-.71). Harsh and Inconsistent Behavior Management was measured with two subscales of the Parenting Practices interview, Harsh and Inconsistent Discipline (α=.66-.72). Parent Involvement in Early Learning was assessed via parent self-reports based on the Involve Interview and the Parent Perceptions of Parent Efficacy tool (α=.73-.82). As a check, parent involvement was also assessed via teacher reports using the school activities subscale of the Involve Interview (α=.74-.81).

Analysis:

Brotman et al (2013) analyzed the data at the student level using MANOVA-like techniques and linear mixed-effects models with adjustments for clustering in classes and schools. However, the level-2 sample size of 10 is small. On request from Blueprints, the authors sent information indicating that the reported results applied the Kenward-Roger Correction for degrees of freedom in SAS Proc MIXED to adjust for the small number of randomized schools. The SAS code also indicated that the results did not use sandwich estimators for the standard errors, which are inappropriate for small samples.

Multiple outcome scores were modeled simultaneously within broader domains (like child behavior problems or academic achievement), thus testing to see whether intervention impacts differed across outcomes or remained roughly equivalent within a domain. The models controlled for baseline school readiness - the pre-K equivalent of kindergarten academic performance - and implementation year. The teacher-rated outcomes, which were obtained over multiple years and defined growth trajectories, were modeled with time-by-condition interaction terms.

At the two-year follow-up, differences in academic performance and student behavior were assessed more typically using growth curve models over the post-program period with controls for baseline school readiness. However, the studies using this strategy (Dawson-McClure et al., 2015; Brotman et al., 2016) drew significance from the model's main effects of the intervention instead of the time-by-condition term, arguing that, although the initial gains made by the intervention group did not increase, they were maintained over time.

Intent-to-Treat: Despite some attrition, final analyses appeared to use all available data by incorporating multiple imputations rather than dropping cases with missing data. However, the first year of data was dropped altogether in the two-year follow-up.

Outcomes

Implementation Fidelity:

Fidelity was assessed in terms of adherence and facilitator competence. Multiple facilitators rated adherence to weekly protocols of key content and process elements; based on more than 300 sessions, adherence was 96% for parent group and 95% for child group sessions. While adherence was high on facilitator self-reports, attendance of weekly sessions was lower, with only 58% of families in intervention schools attending at least one group session and 39% attending 5 or more out of 13.

Baseline Equivalence:

Brotman et al. (2013) reported no significant differences between groups on any demographic or outcome variables at baseline in the posttest studies. Brotman et al. (2016) reported baseline differences in teacher-rated academic performance and mental health problems, which may have only been present in the subsample eligible for follow-up (i.e., the last three years of the four-year sample).

Differential Attrition:

At the kindergarten assessment, Brotman et al. (2013) reported that attrition did not differ by condition, outcomes, or demographic characteristics. The same held true for missing data on standardized tests. Despite a somewhat higher rate of attrition in the follow-up study (Brotman et al., 2016), children retained in second grade did not differ from those without data on demographics or outcomes. There were no tests in any of the studies for baseline-by-condition attrition.

Posttest:

At the kindergarten posttest, Brotman et al. (2013) found that, compared to the control schools, participants in the intervention schools scored significantly higher on a kindergarten achievement test (d=.18) and teacher-rated academic performance (d=.25). Baseline school readiness did not moderate the intervention impact on these outcomes. However, the effect on teacher-rated academic performance did not reach significance until the last of four assessments in the spring of kindergarten.

On posttest parent outcomes (Dawson-McClure et al., 2015), compared to the control group, intervention parents had significantly higher knowledge (d=.32), positive behavior support (d=.16), and teacher-rated parental involvement (d=.38). There was no effect of the intervention on parent-reported child conduct problems, but boys with low self-regulation at baseline showed significant improvement, down to levels typical of lower-risk children. There was a significant exposure effect showing much better outcomes for those who attended the intervention.

Long-Term:

The two-year follow-up assessment (Brotman et al., 2016) found that intervention children had significantly fewer teacher-rated mental health problems (a composite of internalizing and externalizing; d=.44) than children in the control condition.

For teacher-rated academic performance, the intervention-by-time interaction was not significant, but the intervention main effect was significant and positive. The authors interpreted the results to say that the positive impacts found on teacher-rated academic performance at the end of kindergarten were maintained through second grade. Tests across two time points for standardized test scores showed no significant intervention-by-time interaction, which the authors again interpreted to say that the kindergarten effects were maintained. However, they also stated that the main effects of the intervention were only marginally significant for reading and non-significant for math.

Study 3

This article described two separate studies, but the ParentCorps program was examined only in the second study. It used the same data as in Study 2 above but selected a subsample for analysis.

Brotman et al. (2012) began with the randomized clustered sample of 10 schools used in Study 2 but selected a subsample of 146 children with behavioral problems (106 in the intervention schools and 40 in the control schools). Measures of body weight, body size, obesity, physical activity, and sedentary behavior were obtained through second grade, two years after the program ended.

Brotman et al. (2012) found that, compared to the control group, participants in the intervention group showed significant reductions in:

• BMI, body size, and obesity

Evaluation Methodology

Design:

Recruitment: The study began with the same 10 schools, four cohorts, and 1,050 eligible and consented students as in Study 2. However, for the long-term follow-up in second grade, only the first two cohorts had data (N = 496). The study then selected a subsample of children (N = 146) from these two cohorts who had elevated behavior problems or physical aggression at age four based on two teacher-rated scales: the Behavioral Assessment Scale for Children and the New York Teacher Rating Scale. Children with behavioral problems have been found to have an elevated risk of obesity.

Assignment: The 10 schools were randomly assigned to the intervention condition (N = 5) or the control condition (N = 5). After selection of the subsample, the intervention schools contained 106 students and the control schools contained 40 students.

Assessments/Attrition: The primary outcome assessment was conducted in second grade, but some outcome measures were available at baseline and three follow-ups. Of the 146 students, only 60 (41%) completed the second-grade follow-up. Nearly all the missing data came from students who had left the sample schools - data were obtained for 98% of students remaining in the schools.

Sample:

The mean age was 4.38 years at baseline and 7.77 years at the last follow-up. About 87% of the children were black, 9% were Latino, and 4% were of other/mixed ethnicity. About 48% of parents had no more than a high school education; 16% had an annual household income below $15,000; 44% lived below the federal poverty guidelines; and 51% were single parents.

Measures:

The primary outcome measure, BMI, came from measurements of height (using a stadiometer) and weight (using an electronic scale) done by trained medical students or research assistants masked to intervention assignment. BMI z-scores were standardized by age and sex according to growth charts. A dichotomous indicator of obesity (BMI > 95th percentile) was used to evaluate clinical impact.

Because BMI measures were unavailable at baseline, the study used an additional measure of body size rating at baseline and three follow-ups. Research assistants used archived videotapes obtained during school assessments of child behavior to rate body size on a 9-point standardized scale.

Parents reported on their children's sedentary activity with two items assessing the amount of time children spent watching television and other screen time (e.g., video games, computers or the Internet) on a typical day. They also reported on the frequency with which their children engaged in five types of moderate-to-vigorous physical activities in the past week. Although parents helped deliver the program, the ratings on health behavior did not relate to program content and appear independent.

Analysis:

The analysis used linear and logistic regression models with the effect of school clustering taken "into account by allowing for correlation between outcomes of children from the same school" (p. e624). However, the level-2 sample size of ten is small. On request from Blueprints, the authors sent information indicating that the reported results applied the Kenward-Roger Correction for degrees of freedom in SAS Proc MIXED to adjust for the small number of randomized schools. The SAS code also indicated that the results did not use sandwich estimators for the standard errors, which are inappropriate for small samples.  

The models for BMI adjusted for the baseline body size rating but lacked an exact baseline outcome control. The growth models for the body size rating, which was available at three follow-up points, used generalized estimating equations and intervention-by-age interaction terms. The models for health behaviors did not include a baseline outcome control.

Intent-to-Treat: Multiple imputation was used for BMI and the body size rating but not for the health behavior outcomes.

Outcomes

Implementation Fidelity:

Not discussed.

Baseline Equivalence:

The conditions did not differ significantly on sociodemographic characteristics, although the authors noted a non-significant pattern for the baseline body size rating to be lower in the intervention condition than the control condition.

Differential Attrition:

Those who completed (n = 60) and did not complete the follow-up did not differ significantly with respect to age, ethnicity, or body size rating.

Posttest:

The trajectory results described below showed effects on body size rating at posttest, but most of the results focused on the long term.

Long-Term:

The baseline adjusted models of BMI and body size rating in Table 1 showed significant intervention effects in three of four tests (with p = .06 for the fourth). Analysis of developmental trajectories for body size rating found that condition differences emerged early and were maintained over time (Figure 1).

Table 2 for the models of health behaviors did not present main effects. Tests done separately by gender found no significant effects, but there was a marginally significant intervention effect on physical activity for boys (p = .06) and a marginally significant effect on sedentary behavior for girls (p =.09).