Raising Healthy Children

Study 1

Summary

Catalano et al. (2003), Brown et al. (2005) and Haggerty et al. (2006) conducted a cluster randomized controlled trial with 10 schools and 938 first and second grade students. In the second year of the study, an additional 102 students were added to the sample. Schools were randomly assigned to the intervention or a treatment-as-usual control condition. Data measuring academic performance, problem behaviors, and drug use were collected at posttest, several additional time points through grade 10, and finally at 11 years after the study began.

Catalano et al. (2003) found at 18 months posttest, first- and second-grade students who started the program, relative to controls, showed significantly:

• Greater increases in teacher- and parent-reported academic performance and commitment to school.
• Greater increases in teacher-reported social competency and smaller increases in teacher-rated antisocial behaviors, but no significant difference by parent and self-report.

Brown et al. (2005) found during the middle to high school periods (with exposure to intervention materials/boosters through grade 10), intervention students, relative to controls, showed:

• Decline in the frequency of alcohol use, but no significant differences in alcohol use versus nonuse.
• Greater linear decline in the frequency of marijuana use, but no differences in marijuana use growth rates.

Haggerty et al. (2006) found in 11th or 12th grade, intervention students in the Safe Drivers Wanted program, relative to controls, showed:

• Lower likelihood to drive under the influence of alcohol.
• Lower likelihood to ride in a car with someone under the age of 21 who had been drinking.

Evaluation Methodology

Design: Three evaluations were performed drawing data from the same sample. The first evaluation measured the effectiveness of the intervention in affecting changes in academic and behavioral outcomes at 18 months post-intervention (end of 2nd or 3rd grade), while the second examined the effect of the intervention on substance use trajectory from grades 6-10, and the third examined the impact of the "Safe Drivers Wanted" component of RHC delivered in high school.

Participants included first and second graders in ten suburban public schools in the Pacific Northwest. The ten schools were paired on socioeconomic status and attendance patterns. One school in each pair was randomly assigned to receive either the Raising Healthy Children (RHC) intervention or to a treatment-as-usual control condition. To be considered eligible for participation in the study, children were required to have a parent who spoke English, Spanish, Vietnamese, or Korean and attended a regular first- or second-grade classroom. In the fall of Year One, active consent letters were sent to eligible families. After receiving the advance letter, each family was approached through a home visit and invited to participate in the project. Of the 1,239 eligible students, 938 (76%) consented to participate with 497 (53%) in program schools and 441 (47%) in control schools. Fifty-three percent of parents in the experimental group either attended a workshop or had a visit with the school-home coordinator by the end of Year Two.

Catalano et al., 2003 (Early Intervention): Data were collected four times during the 18-month period. Baseline data was collected prior to the intervention in December and January of Year One. The second set of data was collected during the spring of Year One, and the third and fourth sets were collected during the fall and spring of Year Two, respectively. Classroom teachers completed a student behavior checklist on each participating student in Year One at Baseline (December-January) and again at the end of the school year (May-June). Teachers also completed student behavior checklists in the fall and spring of Year Two when the students were in the second and third grades. Completion rates ranged from 95% to 99.8%. Only those participants with completed teacher-reported baseline checklists and follow-up data were used in the analysis (n = 908, 96.8%).

Data from parents were collected via a telephone interview lasting about 45 minutes in the winter of Year One as a baseline measure. Parents were again interviewed in the spring of Year One and in the spring of Year Two. Parents of participating students completed checklists containing questions about their child's behavior as well as family dynamics, parenting issues, and adult behaviors. Parents completed this form at baseline and then again in the spring of Year One and Year Two. One hundred percent of parents provided this information in Year One and 94.1% provided it in the spring of Year Two. A total of 938 parent reports were used for the analysis.

Surveys were read aloud to groups of students for independent completion in the spring of Year One and again in the spring of Year Two. A total of 938 students completed the self-report measure at Year One. A high rate of student retention was achieved: 100% in the spring of Year One and 98.1% in the spring of Year Two.

There was a significant difference between experimental and control participation by low-income families, with 44% of the control families and 33% of the experimental families qualifying for public assistance and/or free or reduced lunch. Significant gender differences were found from the parent-reported data at baseline for school commitment, with females having higher scores than males, except on academic performance and on the parent-reported social competency scale, with females scoring higher than males at baseline. Teacher-reported outcomes at baseline showed that females were rated significantly higher for commitment to school and social competency, respectively, while males were rated significantly higher on antisocial behavior. There were also gender differences on students' self-reported antisocial behavior scale, with males having significantly higher scores than females in Year One.

Brown et al., 2005 (Grades 6-10): In Year Two, the sample was augmented with an additional 102 students from a second eligible pool of 131 students who newly entered 1 of the 10 schools during second grade, thus yielding a total sample of 1,040 students for the drug use outcome measures. For the analysis sample in the drug use evaluation, 77 students were excluded because they were missing data for all substance use outcome measures during grades 6-10. An additional four students were excluded from the study based on questions regarding the validity of their self-reported responses of maximal levels of use for almost all types of substances during all measurement occasions. Thus, a final sample of 959 students (92%) was included in the drug use analyses.

In this evaluation, students in the treatment group participated in after-school tutoring sessions and study clubs during grades 4-6 and received individualized booster sessions, group-based workshops, and social skills booster retreats during the middle and high school years. Selected families in the treatment group participated in multiple-session parenting workshops and in-home services which were delivered during grades 1-8. During high school, booster sessions were delivered through in-home visits in which both parents and students completed assessments that covered specific developmental risk areas. Families who had moved outside the local geographic area had all intervention materials mailed to them with assessments completed through phone consultation.

Teachers in grades 1-7 received at least six staff development workshop sessions starting the year before they had contact with students in the study. Teachers were observed repeatedly in the classroom by independent raters to ensure fidelity to school intervention strategies. Twenty-seven percent of intervention students attended at least one study club, 40% attended at least one of the middle school retreats or workshops, and 51% attended at least one summer camp. A total of 51% of intervention families attended at least one group workshop, 35% received individual contacts including home-based services, and 77% received at least one middle or high school period booster workshop.

Student data collection in grades 6-9 consisted of both group and one-on-one in-school survey administration. Students who were not at school at the time of data collection were contacted at home and individually administered an in-person telephone or mail-in survey. In grades nine and ten, a one-on-one computer-assisted data collection technique was applied in which students entered their responses directly into a computer. Retention rates for student surveys during grades 6-10 were all greater than 88%.

Analysis of differential attrition revealed that a greater proportion of female students had missing outcome data than male students. No other significant differences were noted based on group assignment, first- versus second-grade cohorts, SES, or level of student antisocial behavior.

Haggerty et al., 2006 (High School Safe Drivers Evaluation): In the Safe Drivers Wanted evaluation, 924 students (89%) of the original sample of 1,040 were interviewed in the spring of 2004, 11 years after the study began, and were included in the analysis sample. Participants who did not complete the 11-year follow-up interview and were excluded from the analysis did not differ significantly from the analysis sample in terms of gender, whether they received free/reduced school lunch in the first two years of the project, whether they were in the intervention or control conditions, age, baseline rating of antisocial behavior by teachers, ethnicity (white or non-white), or grade cohort. Ninety-six percent of the eligible families with teens in the intervention condition were exposed to the first driving session, 56% received a home visit and 44% received a mailed packet with a telephone follow-up. Four percent were unavailable for the intervention. Only teens that had received their license (56%) were eligible for the second driving session. Eighty percent of licensed drivers who were in the intervention group and in the analysis sample were exposed to the second session, 59 through home visit and 41% through a mail-out packet with phone follow-up.

Sample: In the elementary sample, the experimental group was 55% male while the control group was 51.5%. The experimental group was 82% European American, 8.5% Asian, 4% African American, 3% Hispanic, and 2.4% Native American while the control group was 80.3% European American, 5.9% Asian, 5% African American, 5.2% Hispanic, and 3.6% Native American. For the drug use evaluation, the sample was 54% male, 82% European American, 4% Hispanic, and 3% Native American.

Measures: Teachers completed academic and behavioral measures on participating students. Commitment to school was measured using two items: "student tries hard in school" and "student wants to do well in school." Academic performance was measured using three items that focused on reading, language arts, and math with the student being compared to peers on these abilities. The academic performance items were not completed at baseline; however, these items were completed at each subsequent data collection period. Social competency was measured by nine items asking teachers about a student's ability to understand other people's feelings, cooperate with peers, accept responsibility for his/her actions, and share things. Antisocial behavior was measured using ten-items taken from the Teacher Observation of Classroom Adaptation-Revised (TOCA-R) and the Child Behavior Checklist-Teacher Report. Parents completed two measures, one rating their child's academic performance (using the same two scales used for teacher ratings) and the other their child's behavior. The academic performance scale for parents consisted of two items, one evaluating math and the other reading, and was not assessed during the fall of Year Two. Parent-rated social competency scale consisted of seven items that examined issues such as making friends with other children, resolving conflicts with peers or siblings, and controlling his/her temperament. The parent-rated antisocial scale was comprised of the same ten items that were completed by the teacher. The student survey was a 40-item self-report measure with a yes/no report scale targeting social competency and antisocial behavior.

Annual substance use measures were constructed from student self-reports of frequency of alcohol, marijuana, and cigarette use during both previous year and previous month time periods. A six-point Likert scale was created for alcohol and marijuana use (0 = no use in the previous year; 5 = 20 or more times in the past month), with a similar scale used for cigarette use (0 = no use in previous year; 5 = more than 40 cigarettes per day).

Measures of driving behavior were obtained from surveys of parents and students in the spring of 2004, when most students were in 11th or 12th grade. The annual survey of students was administered by trained interviewers using laptop computers to record answers. The youth survey took 50 minutes to complete. Interviews with parents were done over the telephone and took about 25 minutes. All outcome measures were dichotomous, reflecting a "yes" or "no" answer, or whether the behavior had occurred in the past 12 months. Session-specific outcomes regarding family rules about driving were measured by the following items: "Does your family have rules about driving?", "Did you help make the rule(s) (about driving)?", and "Have the family rules for driving been put in the form of a written contract?". Compliance with licensure laws was indicated by whether a participant had "driven without a license and not under the supervision of a parent, driving instructor, or other adult." Driving and substance use was assessed by whether the respondent indicated s/he had driven "while under the influence of alcohol" or "while under the influence of marijuana or other illegal drug." Participants were asked if they had ridden in a car driven by someone under age 21 "who had been drinking alcohol," if they had "received a ticket, or been stopped and warned for moving violations such as speeding, running a stop light, or improper passing" and whether the respondent had "been in a car accident while you were driving, whether or not you were responsible." In order to assess whether the intervention may have had an unintended consequence of earlier licensure, students were also asked if they had their driver's license at the time of the follow-up interview.

Analysis: Analyses were conducted based on the number of data points collected during the 18-month time period. Teacher-reported student outcomes (four data collection time points) were analyzed using growth curve analysis with hierarchical linear models (HLM) in terms of the growth rate and level (at the end of 18 months) of each child adjusted for baseline. In this analysis, two models of change were run. The initial model estimated the slope and intercept for each child, provided a test of whether the variance in slope and intercept estimates across individuals was due to sampling error, and provided estimates of the precision of the parameters. The second model examined whether treatment condition predicted the individual slope and intercept after controlling for baseline, gender, and low income. Both models included time in Level One as a predictor of outcome. Thus, level differences reported are those at the end of the 18-month intervention. Parent-reported outcomes (three data collection time points) and child self-reported data (two data collection time points) were analyzed using regression analysis. For analyses of parent-reported data, the effects of treatment condition were modeled after controlling for low income, gender, and baseline. These variables were also controlled for in the child self-reported analyses, however, baseline ratings were comprised of parent-reported data of their child at baseline because child self-reported data at baseline were not collected. Gender was controlled for in each of the analyses.

For the drug use analyses, a two-part latent growth model was used, dividing the original distribution of substance use outcomes into two parts, each modeled by separate, but correlated, growth functions. In Part 1 of the model, nonuse was isolated from any level of use in the past year. Use-versus-nonuse outcome variables for each substance were analyzed as a random-effects logistic growth model with the log odds of use regressed on growth factors. In Part 2 of the model, frequency of use outcomes were modeled as a latent growth model with growth factors of nonzero substance use regressed on intervention status and background variables following traditional latent growth modeling techniques for normally distributed substance use measures. Substance nonuse within each time period was treated as missing data for frequency of use. Change in use-versus-nonuse frequency-of-use outcomes were modeled as linear, quadratic, or piecewise growth.

In order to assess intervention versus control differences attributable to the Safe Drivers Wanted visits, covariates measured in the spring of grade 8 were included in the analysis models to adjust for salient pre-driving session behaviors of parenting and youth substance use. Eighth grade covariates included socioeconomic status, parenting behaviors, and student substance use. Socioeconomic status was indicated by parent report of whether the family received food stamps, Temporary Assistance for Needy Families (TANF), or free/reduced price school lunch for the student. Parenting behaviors were assessed with three scales that measure key targets of the RHC family intervention and were derived from student surveys: monitoring; guidelines; and appropriate consequences. Substance use was indicated by student report of use of cigarettes, alcohol, and marijuana in the prior year. Gender was also included as a covariate. Based on outcome measures by experimental condition, prevalence was examined using contingency tables. In order to assess the statistical significance of these differences in a manner consistent with the group-randomized design of the study, two-level multi-level models were used with students grouped by the elementary school in which they were recruited into the study. The Level 1 (individual level) sampling model was Bernoulli and a logit link was used, with the intercept of the model allowed to vary across the 10 schools (Level 2). Experimental condition was then treated as a characteristic of these school groups, and used to explain between-school variations in the intercept of the logistic regression model. Eighth-grade measures and gender were included as Level 1 covariates with their effects fixed across schools. Only those intervention students who had obtained their driver's licenses received the second transition-to-driving session. In the second analysis, parallel procedures were used to examine experimental versus control comparisons on outcome measures for the students who had their driver's license at the time of the spring 2004 survey.

In order to avoid the bias introduced by listwise deletion of cases, NORM software was used to impute 10 data sets with complete data for all 927 subjects, augmented with random error. The multilevel models were run using HLM 6.0, with results averaged across analyses on the 10 imputations in order to estimate appropriate standard errors.

Outcomes

Posttest: At the 18-month post-test, after controlling for gender, low income, and baseline, the main effects for treatment on slope and intercept were significant. In addition, there were no significant Gender X Intervention or Income X Intervention interactions.

Academic Outcomes: Students receiving the intervention had significantly higher levels of commitment to school and academic performance than students in the control group, based on teacher-reported data. Results from parent-reported data with gender, low income, and baseline scores controlled indicated that receiving the intervention was significantly predictive of academic performance, with intervention students having higher academic performance than control students. For commitment to school, the intervention condition showed a significant effect with program students showing a higher parent-rating of commitment to school than control students.

Behavioral Outcomes: Teacher-reported social competency data indicated that students who received the intervention had a significantly increasing growth rate, while control students decreased significantly in their level of social competency. Similar results were found regarding teacher-reported antisocial behavior, with a significant difference in level and growth between intervention and control students. Control students had a higher level of antisocial behavior and an increasing growth rate, while students receiving the intervention had a lower level of antisocial behavior and a decreasing growth rate. The parent ratings of social competency and antisocial behavior (controlling for gender, low income, and baseline), however, indicated that program condition was not a significant predictor of social competency or antisocial behavior at 18 months posttest. The child self-report data were limited to two outcomes: social competency and antisocial behavior. Both regression equations used parent' ratings of social competency and antisocial behavior, respectively, as substitute baseline measures because child self-report data were not collected at baseline. With low income, gender, and the baseline measure controlled for, program condition was not a significant predictor for social competency or for antisocial behavior. Analyses of effects by gender indicated females receiving the intervention did show significantly higher increases in prosocial skills as compared to males. In addition, the program effects in antisocial behavior did not differ across gender.

Drug Use Outcomes: Prevalence rates for alcohol, marijuana, and cigarette use increased generally during grades 6-10. Rates of substance use in the treatment group during grade 10 were similar to population-based rates from students in the state of Washington. Mean frequency of alcohol and marijuana use peaked at eighth grade and declined thereafter and the mean frequency of cigarette use increased throughout grades 7-10. No significant difference was found between students in the intervention group and controls for change in alcohol use versus nonuse. Students in the intervention group demonstrated a significantly greater rate of decline in the frequency of alcohol use during grades 8-10, relative to students in the control group. In addition, students in the first-grade cohort demonstrated a significantly greater decline in the growth in frequency of alcohol use during grades 8-10, relative to the second-grade cohort.

No significant differences were found in marijuana use growth rates between intervention students and controls. However, for frequency of marijuana use, results indicated a significant intervention effect, with students in the intervention group exhibiting greater linear decline in the frequency of marijuana use than students in the control group. The only significant effect for frequency of cigarette use was for baseline antisocial behavior, with higher levels related significantly to more cigarette smoking in the spring of grade 8. No other variables were associated with change in either cigarette use-versus-nonuse or frequency of cigarette use outcomes.

Safe Drivers Wanted Outcomes: Although intervention students were no more likely to report that their families had driving rules, they were significantly more likely to report having a rule in the form of a written contract than students in the control group. Parents in the intervention group were also significantly more likely to report having a rule in the form of a written contract, and intervention students were significantly more likely to report they had input in making those rules than their control group counterparts. Intervention participants were significantly less likely to drive under the influence of alcohol or to ride in a car with someone under the age of 21 who had been drinking compared to the control group. There were no significant differences found between intervention and control students with respect to receiving traffic tickets or getting into accidents. The intervention was not associated with students being more or less likely to get their driver's license, with the same percentage of students in each condition having received their license at follow-up.

Among the subsample of students who had obtained a license by spring 2004, the differences in unsafe driving were slightly stronger than in the sample as a whole. These students were significantly more likely to report that their family had a written driving contract, that they helped to make the family driving rules, and significantly less likely to report driving under the influence of alcohol or riding in a car with someone under the age of 21 who had been drinking than their control group counterparts. Parents of these students were significantly more likely to report that their family had a written driving contract than their control group counterparts.