Family Check-Up – Toddler

Study 1

Summary

Shaw et al. (2006) conducted a randomized controlled trial in which 120 families (mothers and sons) participating in the WIC program were assigned to the treatment group or a business-as-usual control receiving standard WIC services. Data were collected at baseline and again at 12 and 24 months after baseline to measure maternal mental health, parenting, and child problem behaviors.

Shaw et al. (2006) found significant positive effects on child destructive behavior between the ages of 2 and 3 and maternal involvement when the child was age 4 compared to the control group.

Evaluation Methodology

Design: Mothers and their 17- to 27-month-old sons were originally recruited through the Women, Infant, and Children (WIC) Nutritional Supplement program in Pittsburgh in 2001. At 8 WIC sites, mothers were asked to complete a series of questionnaires about the "terrible twos." These surveys focused on the child's disruptive behavior, emotionality, parenting hassles, and maternal depression. Families who met criteria for Family Check-Up implementation were contacted to participate more intensively. Criteria for study inclusion included socioeconomic and family risks. Additionally, children needed to be above the normative mean on either the Intensity or Problem scales of the Eyberg Behavior Inventory.

Three hundred twenty-seven mothers were approached at the WIC sites, 271 were screened, and 124 qualified for participation (45.8%). Of these, 120 (96.8%) agreed to participate and were assessed at baseline with a home visitation consisting of behavioral observations of the mother and child and mother completion of questionnaires. Eligible parents (all mothers) and their children completed a 2.5-hour home visit. The child was instructed to play for 15 minutes with an assortment of age-appropriate toys while the mother completed questionnaires. Following the initial free play, mother and child participated in a clean-up task (5 minutes), followed by a delay of gratification task (5 minutes), three teaching tasks (3 minutes each), a second clean-up task (4 minutes), the presentation of two inhibition-inducing toys (2 minutes each), and a meal preparation and lunch task (20 minutes). During the second half of the visit, mothers completed a Five Minute Speech Sample and a series of questionnaires. Families were then randomized to a control or treatment condition (n = 60 in each group). After the home visitation session, parents would participate in a Get to Know You session, followed by a feedback session. In the feedback session, the Family Check-up provider summarized the results of the initial assessment and offered the Everyday Parenting program in six subsequent sessions. Assessments were conducted again 12 and 24 months after the initial visit. Families in the control condition had access to the same services provided by WIC but did not receive home visits or intervention from the Family Check-up providers.

Family Check-up providers used for the study were two master's-level therapists, who received 2.5-3 months of training in Family Check-Up. They used a combination of didactic instruction and role playing during Everyday Parenting sessions.

Sample Characteristics: Children were of a mean age of 24.1 months and mothers were of a mean age of 27.2 years. The average family income was $15,374 per year and the average number of family members per household was 4.49. Mothers had completed a mean level of 12.23 years of schooling, with 66.6% having a high school education or less. The ethnic breakdown of the sample was 48.3% African American, 40.0% Caucasian, and 11.7% biracial or other. Forty-five percent were married or living with a partner, 50% were single and never married, and 5% were separated, divorced, or widowed.

Measures: Demographic measures included family structure, parental education, parental income, parental criminal history, and areas of familial stress. The Beck Depression Inventory (BDI) was administered to measure mother's depressive states. Child inhibition was measured through behavioral observation and coded on the basis of the child's reactions to an approach by a stranger and to two novel objects (a tunnel and a robot). Behaviors were specifically coded for approach and proximity to mother, avoidance or wary response to examiner, examining or playing with free-play toys, and approach to novel objects. Internal consistency for the Inhibition factor was 0.63.

The primary outcome measure was the mother-reported Child Behavior Checklist 2-3 and 4-18 (CBCL), which measures internalizing and externalizing behaviors. Of particular interest were the Destructive and Aggression subscales. The secondary outcome measure was the Home Observation for Measurement of the Environment (HOME), in which observers blind to condition rated the quality of the home environment. Parent-centered measures included: parent keeps child in visual range, parent talks to child while doing housework, and parent structures child's play.

Analysis: An intention-to-treat analysis was conducted. One-tailed tests of significance were used, due to the relatively small sample size. ANCOVAs and hierarchical regressions were used to determine program effects.

Outcomes

Attrition: Fifty-three of the 60 (88%) treatment group participants completed treatment. Fifty-five controls and 57 treatment participants completed one-year follow-up while 56 controls and 53 treatment participants completed two-year follow-up. Data from 46 participants in each group were utilized in data analysis, for a 76.7% retention rate. There was no evidence of differential attrition at age 4 by group assignment (treatment or control), but no other tests were presented. Finally, while 7 families failed to complete treatment, 55 of the 60 (91.7%) families participated in the Get to Know You and feedback sessions. The average number of sessions was 3.26, including the Get to Know You and feedback sessions as two sessions. The number of sessions was not related to any of the three CBCL-based scales of disruptive problem behavior at either the one-year or two-year assessment.

Baseline Equivalence: At age 2, as measured at the first home visit, there were no significant differences between intervention and control groups on the three CBCL scales, the BDI, the HOME Involvement Scale, or on any sociodemographic factors.

Posttest:

Effects on Maternal Involvement: After accounting for the effects of ethnicity, results for the HOME scale showed no main effect for time but there was a main effect for treatment. By age 4, after controlling for baseline scores, the treatment group showed significantly higher scores than controls. There was no interaction between time and treatment group.

Effects on Child Disruptive Behavior: There were no significant effects on Aggression. There was a significant Time x Treatment effect for destructive behavior, based primarily on changes between ages 2 and 3, when those in the treatment group showed a significant decrease compared to those in the control group.

Effects on Children with Extreme Risk for Continued Conduct Problems: Hierarchical regression equations were significant for both aggression and destructive behavior. For destructive behavior, significant 2-way interactions were evident between treatment and maternal depression and between treatment and child inhibition. The treatment condition was associated with lower levels of destructive behavior at high levels and average levels of maternal depressive symptoms, but scores for destructive behavior were significantly higher for treatment, compared to controls, when maternal depressive symptoms were initially low. For interactions involving treatment group and child inhibition, the slopes indicate that treatment is associated with lower levels of destructive behavior at mean levels and low levels of child inhibition. The slope for the 3-way interaction was not significant, as treatment was more effective when either the parent or child risk factor was evident.

Study 2

Summary

Dishion et al. (2008) and six other reports examined 731 families in a randomized controlled trial in which the families were assigned to the intervention group or to a business-as-usual control group. Data measuring parenting, parent and child mental health, and child problem behaviors were collected at baseline, and again at 1, 2, 3, and 5.5 years after program initiation.

Dishion et al. (2008) and other articles found significant improvements for intervention participants, relative to controls, in:

• observer-rated positive behavior support provided by parents
• parent-reported child problem behaviors
• parent reported child externalizing
• parent and teacher-reported oppositional defiant behavior at the 5.5-year follow-up, particularly among those that engaged most in the program
• reports of child depressed/withdrawn symptoms at ages 7.5-8.5 via decreased maternal depression

Significant effects on risk and protective factors included:

• decreases in maternal depression
• increases in maternal involvement and positive parenting

Evaluation Methodology

Design: Participants were randomized into either the treatment or control group, with gender balanced to ensure an equal number of males and females in each group. Examiners were blind to the assignment. The assessments were done during home visits, beginning with baseline at age 2 and repeated at ages 3, 4, 5, and 7.5 for both control and intervention groups.

Families assigned to the FCU intervention met with a provider to receive the Family Check-Up three-session intervention. Parents were also offered follow-up sessions that focused on parenting practices, other family management concerns, and contextual issues. The Family Check-Ups were offered at ages 2, 3, and 4.

Participants were recruited in 2002 and 2003 from WIC centers in Pittsburgh, PA; Eugene, OR; and outside of Charlottesville, VA. Families were eligible if they had a child between the ages of 2 and 3, and also had socioeconomic, family, and/or child risk factors for future behavior problems. Of 1,666 families contacted, 879 were eligible.

Of the 731 families who participated at baseline, 659 (90%) were assessed at the 1-year (age 3) follow-up, 619 (85%) participated in the 2-year (age 4) follow-up, 621 (85%) completed measures at 3-year (age 5) follow-up, and 566 (77%) were retained at the 5.5-year (age 7.5) follow-up. However, due to lack of school consent, only 312 children (54% of families at the time) starting the school year of their 5.5-year follow-up were rated by their teachers (Dishion et al. 2014).

Sample: The mean age of the children was 29.9 months at baseline assessment. Half of the sample was female and primary caregivers self-identified as belonging to the following ethnic groups: 28% African American, 50% Caucasian, 13% biracial, and 9% other groups, with 13% identifying as Hispanic. More than two-thirds of families had an annual income under $20,000 and the average number of persons per household was 4.5. About one-third (36.2%) of participating parents were married, one-third (31.6%) were single, with the remainder living with a partner (19.8%), separated (7.7%), divorced (4%), or widowed (0.7%). Forty-one percent of participants had a high school diploma or GED equivalency, and an additional 32% had 1 to 2 years of post-high school training.

Measures: (Dishion et al., 2008; Shaw et al., 2009) A demographics questionnaire was administered at each assessment point to measure family structure, parental education and income, parental criminal history, and areas of family stress. The Epidemiological Studies on Depression Scale (CES-D) was used to assess maternal depressive symptomology at each follow-up (alpha = .76). Measuring early childhood problem behavior at each time period, the Child Behavior Checklist (CBCL) used maternal reports of internalizing and externalizing behaviors (alpha = .71, .75, .78, .80 at ages 2, 3, 4, 5), and the Eyberg Child Behavior Inventory Problem factor was used to assess the extent to which behaviors were a problem for the parent. In Rueben et al. (2015), the Child Behavior Checklist (parent or caregiver report) depressive symptom subscale was used to assess depressed/withdrawal symptoms in children (alpha = .77 and .76 at ages 7.5 and 8.5) while the Teacher Report Form and an adapted depressed/withdrawal symptom subscale was used to assess the children in school (alpha = .80 and .79 at ages 7.5 and 8.5).

The home visit assessment was videotaped and consisted of a series of timed activities which involved the child or parent and child together completing a variety of tasks (e.g., clean-up) and activities (e.g., free play sessions) (see Study 1 above for description). The home visits lasted 2.5 hours, during which time the parents also completed the written assessment measures. Videotaped home visit observations were coded using the Relationship Process Code, after which coders completed a coder impressions inventory regarding proactive and positive behavior support practices. All family interaction tasks were evaluated in the scoring of Positive Behavior Support practices. The following items were entered into the Positive Behavior Support scores: Parent Involvement, which was based on the home visitor's rating of the parents' involvement, using items from the Home Observation for Measurement of the Environment (HOME) inventory; Positive Behavior Support, which was based on caregiver's positive behavior (positive reinforcement, prompts and suggestions of positive activities, and positive structure); Engaged Parent-Child Interaction Time, which measured the average length of parent-child sequences that involved talking or physical interactions; and Proactive Parenting, which included tendency of the parent to anticipate potential problems and provide prompts or other structural changes to avoid child problem behavior. The same home visit and observation protocol was repeated at ages 3, 4, and 5 for both the control and intervention groups. In Dishion et al. (2014), the Adult-Child Relationship Scale was administered to help characterize parent-child relationships within the family (alpha = .60). Parents also completed the 15-item Confusion, Hubbub, and Order Scale in the 2014 study to determine the level of order present within the home (alpha = .74). Finally, neighborhood dangerousness was assessed in this same study by primary caregivers using the 19-item Me and My Neighborhood Questionnaire (alpha = .88). Families were compensated for their participation.

(Lunkenheimer et al., 2008) This evaluation utilized the measures collected through the HOME inventory to assess the Positive Behavior Support construct: parent involvement, positive reinforcement, engaged interaction, and proactive parenting. Language skills and self-regulation measures were collected to examine indicators of school readiness. To measure language skills, the Fluharty-2 Preschool Speech and Language Screening Test was used at two time points (ages 3 and 4). To measure behavioral self-regulation, the Children's Behavior Questionnaire (CBQ) was used at three time points (ages 2, 3, and 4).

(Connell et al., 2008) The demographics questionnaire and CBCL (internalizing and externalizing scales) were the measurement tools used in this assessment.

(Dishion et al., 2014) Teacher-reported and parent-reported measures assessed incidents of oppositional behavior in the classroom at age 7.5, coded using the 5-item DSM-oriented Oppositional Defiant Problems Scale from the Teacher Report Form (alpha = .90).

Moderator analysis (Gardner et al., 2009): In addition to the outcome measures of problem behavior and externalizing as described above, this study used several sociodemographic variables as moderators. These included: 1) single parenthood, 2) teen parent status, 3) low maternal education, 4) substance use problems, 5) maternal depression, 6) parenting daily hassles, and 7) partner relationship quality. Also, a cumulative risk index was generated from the seven indicators of sociodemographic risk and was examined as a moderator.

Analysis: (Dishion et al., 2008) Latent growth mixture modeling (LGMM) was used to examine heterogeneity in developmental trajectories of early problem behaviors and positive parenting. Mediator analyses were also conducted to examine the indirect effect of the intervention on the rate of change in problem behaviors through the effect of the intervention on maternal Positive Behavior Support at child age 3. This model examined whether the intervention was related to the change in Positive Behavior Support from child ages 2 to 3, and whether this change in Positive Behavior Support, in turn, predicted the rate of change in child behavior problems from ages 2 to 4, controlling for the direct effect of the intervention.

(Connell et al., 2008) Latent transition analysis (LTA) was used to identify classes (i.e., latent groups) of children exhibiting distinct profiles of internalizing and externalizing symptoms from ages 2 to 4. LTA was also used to examine transitions across classes from age 2 to age 3 and from age 3 to age 4.

(Lunkenheimer et al., 2008) To test the direct effects of the intervention on child language skills and inhibitory control, a repeated-measures ANOVA was used. To test the indirect effects of the intervention on the outcomes via Positive Behavior Support from parents, a longitudinal structural equation model was used. Child gender, child ethnic minority status, and parental education were included as covariates to control for potential differences by sociodemographic factors. All analyses were intent-to-treat.

(Shaw et al., 2009) Latent growth models (LGM) were used to evidence effects of the intervention on externalizing and internalizing behaviors, as well as perceptions of child behavior problems as reported by mothers. The models also examined whether assignment to the Family Check-Up was associated with reductions in maternal depressive symptoms from ages 2 to 3, and whether reductions in different problem behavior from ages 2 to 4 were mediated by reductions in maternal depressive symptoms from ages 2 to 3, controlling for the direct effect of the intervention. Analyses were intent-to-treat, using full information maximum likelihood estimation. For this study, only maternal reports of child problem behavior were used from the ages 3 and 4 assessments, and maternal reports of depression were used from the age 3 assessment.

(Gardner et al., 2009) The study tested intervention moderation with latent growth models in M-plus and outcomes measuring changes or linear growth slopes in child problem behavior and externalizing. The tests for moderation examined interaction effects between the intervention and covariates.

Dishion et al. (2014) first used Structural Equation Models to examine the intervention effect on the outcome for an intent-to-treat sample. The study then used complier average casual effect (CACE) structural equation modeling to investigate whether intervention effects on parent and teacher-reported problem behavior differed by degree of program engagement at 5.5-year follow-up. For the intervention group, engagement was defined and tested at three levels: 1) as having participated in at least 1 feedback session at any point across child ages 2-4, 2) as having participated in at least 2 feedback sessions, and 3) having participated in all 3 possible feedback sessions versus those that did not or would not have participated in the program. For the control group, covariates best predicting engagement in the intervention group were then used to identify a latent class of those that would have been likely to engage at these different levels, had they received the intervention. "Engagers" in both groups were then compared on the teacher- and parent-reported outcomes. Finally, the study also tested for indirect effects of the intervention on teacher-reported oppositional behavior through growth in parent-reported disruptive behaviors using the same strategy, with engagement as a moderator. Analyses accounted for baseline characteristics and outcomes.

Rueben et al. (2015) used structural equation modeling to examine the intervention effect on child depressed/withdrawal symptoms, and then whether maternal depressed symptoms at age 3 - controlling for maternal depressive symptoms at age 2 - had any effect. The analysis included reports from all three informants (parent, alternative guardian, and teacher) and controlled for income, education, site, and child gender.

The studies appear to adhere to the principles of intent-to-treat, using all available data and including those that were assigned to the intervention group but did not participate in program sessions.

Outcomes

Implementation Fidelity: The providers who led the FCU and follow-up parenting sessions were either PhD or master-level service workers who received 2.5 to 3 months of training in the Family Check-Up model. Consultants also attended annual parent meetings to update their training, discuss possible changes in the intervention model, and address any issues concerning the intervention.

Family intervention engagement could be interpreted as a proxy measure of implementation fidelity. Family intervention engagement (i.e., number of feedback sessions attended) was used as a moderator of the program's effects on problem behavior in Dishion et al. (2014). At age 2, 76% engaged in the feedback session, 72% in at least one follow-up session, and families attended an average of 3.4 follow-up sessions. At age 3 these values were 69%, 70%, and 3.1, and at age 4 they were 70%, 74%, and 3.5, respectively. Shaw et al. (2009) reported that, of the families that met with a Family Check-Up provider, the average number of sessions per family was 3.32 at age 2 and 2.83 at age 3. This number was uncorrelated with future levels of problem behavior after controlling for initial levels.

Baseline Equivalence and Differential Attrition: Analysis indicated no significant differences at baseline between groups on project site, children's race/ethnicity and gender, levels of maternal depression, or children's externalizing behaviors. There were also no significant differences found between participants who remained in the study through follow-up compared to those who dropped out. Dishion et al. (2014) further reported no differences at the 5.5-year follow-up between subjects with and without teacher-reported data.

Posttest: (Dishion et al., 2008): Tests of the hypotheses came in three steps. First, on the parent-reported Eyberg problem behavior scale, the intervention group experienced significantly reduced growth relative to the control group for those with initially elevated levels of problem behavior but not for those with lower levels. The effect size equaled .33 for the elevated group and .23 for both groups combined. On the parent-reported Child Behavior Checklist scale for externalizing, children in the intervention group showed significantly reduced growth relative to the control group. Intervention effects on both outcomes did not differ by gender or ethnicity. Second, the observer-based measure of parent Positive Behavior Support from ages 2 to 3 was significantly more improved among parents in the intervention group than the control group (d = .33). Third, tests of mediation examined the influence of parent Positive Behavior Support on child problem behavior and externalizing. For the Eyberg problem behavior scale and the Child Behavior Checklist externalizing scale, parent Positive Behavior Support significantly predicted less growth of these behaviors. For both outcomes, then, the indirect effects of the intervention via parent behavior were statistically significant but small (.03).

(Connell et al., 2008): The Latent Class Analysis identified four classes at each age (2, 3, and 4): normative, externalizing, internalizing, and comorbid. The analysis examined transitions from the latter three problematic classes to the normative class from ages 2 to 3 and from ages 3 to 4. The odds ratios for the intervention group relative to the control group lacked tests of statistical significance but showed that the intervention substantially increased the transition from comorbid to normative at ages 2 to 3 (OR = 60.4) and at ages 3 to 4 (OR = 167.8). The intervention also increased the transition from internalizing to normative at ages 3-4 (OR = 9.3) but not at ages 2-3 (OR = 1.0). The intervention had little influence on the transition from externalizing to normative at either age (OR = .89).

School Readiness (Lunkenheimer et al., 2008) Results of direct program effects on school readiness showed a marginal trend in favor of the intervention over the control group children in increasing self-regulation from age 2 to age 4, but no significant effect of the intervention on language development. In terms of indirect program effects, results showed a significant program effect on parents' Positive Behavior Support at age 3 (d = .24) for intervention parents as compared to control group parents. There were small but significant indirect effects of the intervention on the children's language skills and inhibitory control through Positive Behavior Support from parents.

Mediating Factors (Shaw et al., 2009): Results indicate significant reductions in both externalizing and internalizing behavior among children from ages 2 to 4 in the intervention group, compared to children in the control group. Perceptions of problem behavior among mothers of children in the intervention group remained stable, whereas mothers in the control group reported increases in problem behavior among their children. Mothers in the intervention group reported significantly greater decreases in depressive symptoms than control mothers. The indirect effect from the intervention to reduced maternal depressive symptoms to lower growth in externalizing and internalizing problems was statistically significant, although small in magnitude for both. These effects were also significant for perceived problem behavior, indicating that all three behaviors were mediated by changes in maternal depression from ages 2 to 3. Also, maternal depression continued to be a significant mediator of intervention effects for all three factors of child problem behavior when positive parenting was included as a dual mediator of intervention effects, with Positive Behavior Support continuing to serve as a significant mediator in the case of the Eyberg Problem Behavior Scale.

Moderator analysis (Gardner et al., 2009): The results showed three significant interactions out of 16 tests. First, there was a greater intervention effect on child problem behavior in families with less educated mothers (d = .68), whereas higher maternal education was associated with a smaller intervention effect (d = .18). Second, for the outcome measure of child externalizing, mother's education moderated the intervention effect in the same way. Third, intervention effects were stronger in two-parent families (d = .53), with smaller effects in families headed by single mothers (d = .04). There was no evidence of different program effects by the cumulative risk index.

Teacher-Reported Measures and Moderator Analysis at 5.5-year follow-up (Dishion et al. 2014): Parent reports of child oppositional defiant behaviors (d= .30) and teacher-reported oppositional behaviors (d= .26) were improved in the intervention group compared to the control group. Parent-rated child oppositional defiant behaviors at ages 2-5 mediated the intervention effect on teacher-rated oppositional behaviors at age 7.5.

For those that engaged at least once in the intervention or had similar baseline characteristics to engagers (in the control group), the CACE treatment-on-the-treated analysis found that the intervention group experienced less growth in oppositional behavior than did controls. This was true at each level of engagement, with larger effects for those that engaged more (1 session, d= .44; 2 sessions, d= .55; 3 sessions, d= .93). The indirect effects of the program on teacher-reported oppositional behavior followed the same general trend of larger effects at greater degrees of program engagement (1 session, d= .23; 2 sessions, d= .29; 3 sessions, d= .47), though the effect for the 3-session engagers was only marginally significant (p< .10).

Obesity (Smith et al., 2015). The trajectory of children's weight gain was also examined using path analysis with a latent growth model. This analysis revealed a significant indirect effect of the intervention on the trajectory of BMI later childhood. Assignment to the Family Check-Up was significantly related to Positive Behavior Support at age 3, controlling for baseline levels. Positive Behavior Supports was significantly associated with the nutritional quality of the meals served to the child during the in-home assessment from ages 2-5 years, which was significantly associated with a less steep increase in BMI.

Child Depressed/withdrawn symptoms (Reuben et al., 2015): There was no direct intervention effect on child depressed/withdrawn symptoms at 7.5 or 8.5 years old, but the effect of maternal depressive symptoms at age 3 was statistically significant in lowering child depressed/withdrawn symptoms. The indirect effects were very small.