Strengthening Families 10-14

Study 1

Summary

Spoth, Redmond et al. (1999), Spoth et al. (2000), and 13 other reports used a cluster randomized trial to examine 33 Iowa public schools and 667 students. Participants were assigned to three conditions: Strengthening Families 10-14 (formerly Iowa Strengthening Families), Preparing for the Drug Free Years, and a minimal contact control condition. A variety of substance use and related measures were obtained at posttest, 18, 30, 48, and 72-month and 10-year follow-ups.

Spoth, Redmond et al. (1999), Spoth et al. (2000), and 13 other reports found that, relative to the control group, the Iowa Strengthening Families program produced the following significant findings:

• Lower rates of initiation in each of the three alcohol ever-use measures at the one-, two-, and four-year follow-ups.
• Lower transitions to substance use at the two-year follow-up.
• Lower proportion reporting initiation of each of the five substance use behaviors (use of alcohol, alcohol without parental permission, drunkenness, cigarettes, and marijuana) at the four-year follow-up (10th grade).
• Lower frequency and proportion reporting past month alcohol use and past month cigarette use in 10th grade.
• Lower composite index for both alcohol and tobacco at 10th grade.
• Lower scores for observer ratings and adolescent report of aggressive and hostile behaviors in the 10th grade.
• Slower overall growth in lifetime use of alcohol, lifetime cigarette use, and lifetime use of marijuana at the six-year follow-up (Grade 12).
• Delayed growth rates for initiation of alcohol use without parental permission, drunkenness, cigarette use, and the Alcohol Use Composite Index shown by Grade 12.
•  Fewer youth scoring at or above borderline range of the CBCL-YSR anxious/depressed index by 12th grade.
• Slower rate of increase in polysubstance use over time (6th to 12th grade), significantly more for girls than for boys, although overall levels of use were lower in the intervention group for both genders.
• At the 12th grade follow-up, less methamphetamine use, although the total number of adolescents reporting methamphetamine use in the past 12 months was extremely small.
• In young adulthood, lower rates of drunkenness frequency and on a polysubstance use index.
• In young adulthood, lower self-reported lifetime sexually transmitted diseases and substance use during sex.

Significant Program Effects on Risk and Protective Factors:

• Improvements in intervention-targeted parenting behaviors, which, in turn, had significant effects on both parent-child affective quality and general child management at both posttest and one year following posttest.

Evaluation Methodology

Design:

Recruitment/Sample Size: Participants in the study were families of sixth graders enrolled in 33 rural schools in 19 contiguous counties in a Midwestern state. Schools were selected for participation if 15% or more families in the school district were eligible for free or reduced-cost school lunches and community size was 8,500 or fewer. All families of sixth graders in participating schools (1,309) were recruited for participation. Of these families, 667 (51%) enrolled in the project and completed pre-testing in the fall of 1993. These families were similar to the eligible families on sociodemographic characteristics.

Study type/Randomization/Intervention: The schools were randomly assigned to one of three experimental groups: a minimal contact control condition, or to one of the following universal family-focused preventive intervention programs: the Iowa Strengthening Families Program (ISFP, now known as Strengthening Families 10-14) or Preparing for the Drug Free Years (PDFY, now called Guiding Good Choices). A randomized block design was used, in which schools were blocked on the proportion of students who resided in lower income households and on school size. Each group included 11 schools, and pretested families included 238 in the ISFP group, 221 in the PDFY group, and 208 in the control group. Refusal rates were similar across conditions: 54% for the ISFP group and 51% for the control group.

Assessment/Attrition: After completion of the 7-week intervention, families were assessed approximately 6, 18, 30, 48, and 72 months following the pretest (when the students were in the sixth, seventh, eighth, tenth, and twelfth grades, respectively). An additional follow-up was conducted with the target children when they had entered young adulthood, at the approximate age of 21. These assessments constitute seven waves. From wave 2 (posttest) to wave 7 (young adult follow-up), overall attrition rates were 17.4%, 29.2%, 34.3%, 33%, 31.5%, and 27.4%.

Sample Characteristics:

The majority (over 98%) of participants was Caucasian and came from dual parent families (85%). Of these dual parent families, 64% included both of the target child's biological parents. In just more than half of these families, the target child was a girl. The mean ages of mothers and fathers were 37.0 and 39.6 years, respectively, and the large majority of both mothers and fathers (97% and 96%, respectively) had completed high school. The mean age of the target child was 11.3 years at the beginning of the study, and 51% were female.

Measures:

The measures came from multiple sources, including independent ratings of in-home videotapes of families in structured family interaction tasks and in-home interviews with parents and children. The measures relied on standardized instruments and commonly used items from the National Survey of Delinquency and Drug Use.

Parenting. Parent-child affective quality was assessed through self-reports (e.g., "How often did you let this child know you care about him/her" or "How often did you yell, insult or swear at the child during disagreements"), and observation (e.g., levels of warmth, hostility, contempt) of positive and negative affect. General child management assessed the self-reported frequencies of standard setting, child monitoring, consistent discipline, as well as the observed standard setting (listening, communication, reasoning) and consistent discipline. Targeted behaviors were developed from self-reports and assessed substance use rules and consequences, anger and emotion management, involving children in activities and decisions, and communication of parent intentions and values (targeted only by ISFP).

Four indicators of a proximal Parenting Competency outcome were developed from 13 self-report questionnaire items. These indicators measured: (1) Rules: parents' explanation of substance use rules and of consequences to their child when violations occur; (2) Involvement: parental efforts to involve their child in family activities and decisions; (3) Anger Management: parental management of anger and strong emotion in the parent-child relationship; and (4) Communication: parental activities to communicate understanding of children's feelings and goals as well as parental intentions and values. Five-point Likert-type items ranging from 1 (strongly agree) to 5 (strongly disagree) were used to construct each of the indicators.

Substance Use. Measures of alcohol initiation and use were obtained through self-reports, and analyses were conducted on both individual items and on the sum of these as an index. The former included questions such as "Have you ever drunk beer, wine, wine coolers, whiskey, gin, or other liquor?" Substance use was measured according to lifetime cigarette use ("Have you ever smoked cigarettes?"), lifetime alcohol use ("Have you ever drunk beer, wine, wine coolers, whiskey, gin, or other liquor?"), and advanced substance use (which included several items due to low base rates, such as past month use of cigarettes and alcohol, lifetime drunkenness, and lifetime use of illicit substances). A collapsed measure had five statuses: no use, alcohol experimentation, tobacco experimentation, experimentation with both alcohol and tobacco, and more advanced use. Five measures of specific lifetime use behavior (alcohol, alcohol without parental permission, drunkenness, cigarettes, and marijuana) were also evaluated individually.

Past month use of alcohol, cigarettes, smokeless tobacco, marijuana, inhalants, and other illicit drugs summed to create a scale ranging from "0" (no past month use) to "6" (past month use of all substance categories). Adolescent methamphetamine use was assessed with a single item: These items were added at the 12th grade assessment. Prescription drug misuse was measured beginning in grade 10 by questions about past year and lifetime misuse of narcotics, barbiturates, quaaludes, tranquilizers and amphetamines.

Student substance-related risk was measured by three indicators, based on participant self-report questionnaire items. The first was the alcohol initiation index described above, the second consisted of a single item assessing the participant's attitude toward alcohol use: "How wrong do you think it is for someone your age to do any of the following things: drink beer, wine, wine coolers, or liquor?" The third indicator also consisted of a single item assessing the participant's potential response to peer pressure for alcohol use: "If you were at a party and one of your friends offered you an alcoholic drink, how likely would you be to--... Drink it."

Aggression. Aggressive and hostile behavior was measured using a multi-informant, multimethod format and included independent observer ratings of adolescent aggressive and hostile behaviors in adolescent-parent interactions, family member reports of aggressive and hostile behaviors in those interactions, and adolescent self-report of aggressive and destructive conduct across settings. Each measure represented an index, calculated as the number of individual behaviors serious or frequent enough to be considered currently problematic or indicative of progression towards serious aggressive or violent behaviors.

Internalizing. Internalizing symptoms were assessed with the Anxiety-Depression index from the Child Behavior Checklist - Youth Report (CBCL-YSR), as the average of 15 items (e.g., "How true is each of these statements for you now or in the past 6 months: "I feel lonely," "I am nervous or tense," and "I am unhappy, fearful or depressed") rated on a three-point scale.

Academics. Academic success was measured using multiple reports from mother, father and student response to the question, "Which of the following is closest to the grades 'your child' (or 'you' for student interviews) usually gets in school?" The responses were scored on a 9-point scale.

School engagement was measured using an exploratory factor analysis of 12 items scored on a Likert-type scale. Three indicators assessing an affective, cognitive, and behavioral component of school engagement were created. Four items (e.g., "In general, I like school a lot") comprised the affective indicator. The cognitive indicator consisted of two items (e.g., "I know how to study and how to pay attention in class so that I will do well in school"). The third indicator of school engagement, observable behavior, consisted of three items and included statements such as "I usually finish my homework."

Wave 7 (young adulthood):

Measures of substance use frequency included

• drunkenness frequency (i.e., "How often do you usually get drunk?");
• alcohol-related problems (using a modified form of the Rutgers Alcohol Problems Index);
• past year cigarette frequency (i.e., "During the past 12 months how often did you smoke cigarettes?");
• past year illicit drug frequency (using nine open-ended items asking "How many times in the past 12 months did you use [specific substance]?");
• a polysubstance use index created using the three measures of substance use, dichotomizing each variable to indicate use (1) or no use (0) of substances, and summing the three dichotomous items to form an index with values ranging from 0 (indicating no use of any substance) to 3 (indicating at least some occurrence of all three substance use behaviors).

Measures of health-risking sexual behavior included

• self-reports of the number of sexual partners in the past year;
• condom use in the past year (with five responses ranging from "none of the time" to "always" and including a "don't know" option);
• substance use and sex to include two questions asking "When you have sex, how often have you been drinking alcohol" or "using drugs other than alcohol?" (including the same response options as for condom use);
• Sexually Transmitted diseases with participants asked "Have you ever been diagnosed with a sexually transmitted disease (STD or VD) other than HIV/AIDS, such as gonorrhea, genital warts, chlamydia, trich, herpes, or syphilis?"

Adolescent Alcohol, Tobacco, and Illicit Drug Initiation Index:

A substance use initiation index was calculated using data from waves 1 to 6. This index is the sum of the five individual substance initiation measures, each scored so that "Yes" = 1 and "No" = 0 (with measures corrected for consistency so that when "Yes" was reported for initiation of any substance at any wave, responses were scored "Yes" for each subsequent wave). Scores ranged from 0, indicating no initiation, to 5, indicating the initiation of alcohol use (without parental permission), drunkenness, tobacco, marijuana, and other illicit drugs. Internal consistency for this index, as assessed by Cronbach's alpha, averaged .60 across waves.

Analysis:

Across the numerous articles, the types of analyses included ANOVAs, log-linear models, structural equation models, multilevel or hierarchical models, and latent growth curves. The analyses consistently adjusted for clustering and non-independence of observations within schools, the unit of assignment. The few exceptions occurred for rarely reported outcomes, such as meth use and prescription drug misuse, where the low rates for the outcome and limited within-school dependence led to unstable multilevel estimates. The analyses included baseline outcomes as predictors whenever possible.

Intent to Treat: Most analyses used a form of maximum likelihood estimation or full information maximum likelihood estimation that included as many participants as possible, regardless of their participation in the programs.

Outcomes:

Implementation Fidelity:

Approximately 50% of pretested families attended at least one session with 94% of these families represented by a family member in five or more sessions, 88% attending six or seven sessions, and 62% attending all seven (Spoth, Clair, and Trudeau, 2014). Each team of intervention implementers was observed three or four times to assess whether the teams covered all key program content in ISFP. Coverage of the component tasks or activities described in the intervention manual showed an average coverage of 87% in the family session segments, 83% in the parent session segments, and 89% in the youth session segments (Spoth, Guyll, Trudeau, and Goldberg-Lillehoj, 2002).

Baseline Equivalence:

As demonstrated in multiple articles, intervention and control group participants were equivalent on family sociodemographic characteristics, reported substance use, and all outcome measures collected at pretest.

Differential Attrition:

Across the multiple waves, tests for differential attrition examined the relationships of each baseline measure with condition, attrition, and condition-by-attrition. The tests thus determined if differential attrition existed and if it differed by condition. Most articles reported that there were no significant main or interaction effects in the differential attrition models. One exception, Guyll et al. (2004), reported that families failing to complete all five assessments had younger parents, parents with less education, and children who reported more alcohol use at pretest when compared to those who completed measures at all assessment points. However, these relationships were similar in both conditions.

Posttest and Long-term:

Waves 2 and 3: Posttest and One Year Follow-up (Sixth and Seventh Grades)

Direct and Indirect Effects of Parenting Outcomes (Spoth et al., 1998)

Using posttest data, the effects of the intervention on one proximal and two distal parenting outcomes was tested. ISFP vs. control group comparisons: All hypothesized structural effects were statistically significant at the .01 level, and the indirect effects were significant for general child management and parent-child affective quality. The intervention effect size on targeted behaviors was .51. Extending this test to one year following posttest, results indicate that statistically significant effects on parenting outcomes were sustained (Redmond, Spoth, Shin, and Lepper, 1999).

Implementation Adherence (Spoth, Guyll, Trudeau, and Goldberg-Lillehoj, 2002)

Each team of intervention implementers was observed three or four times to assess whether the teams covered all key program content in ISFP. Coverage of the component tasks or activities described in the intervention manual showed an average coverage of 87% in the family session segments, 83% in the parent session segments, and 89% in the youth session segments. Eight schools achieved a level of adherence that allowed them to be classified as "higher adherence." "Lower adherence" schools were those in which at least one of the three session components was lower than 80% adherence and at least one other component was lower than 85% adherence. Three of the 11 schools were classified as lower adherence. Average adherence scores demonstrated a significant difference between groups.

At post-test, the ISFP intervention produced significant positive effects on mean differences in the total sample, and in both the higher- and lower-adherence schools, on intervention-targeted parenting behaviors compared to the control group. These results were largely sustained at 1.5 years post-baseline, with the exception that the positive difference between the lower-adherence and the control condition schools did not attain significance. There were no significant mean differences between the intervention and control conditions for the substance refusal and resistance skill outcomes at either post-test or at 1.5 years post-baseline.

Waves 3 and 4: One- and Two-year Follow-ups (Seventh and Eighth Grades)

Spoth, Redmond, and Lepper (1999)

Results indicated significantly lower Alcohol Initiation Index (AII) scores for intervention group compared to control group adolescents at both follow-ups, with effect sizes in the medium and large range: .26 for the one-year follow-up and .39 at two-years. ISFP adolescents showed lower rates of initiation in each of the three alcohol ever-use measures at both follow-ups, relative to the control group: relative reductions for ISFP vs. control at one-year were 31.5% for alcohol use, 60.5% for using without permission, and 29.2% for ever being drunk; at two years, they were 45.1%, 56%, and 55.6% respectively.

Latent Transition and Loglinear Analyses: (Spoth, Reyes, Redmond, and Shin, 1999)

Loglinear analyses showed the absence of a significant experiment group x outcome interaction at the end of one year. This was confirmed with Z tests which showed that the estimated probability of a positive outcome at the one-year follow-up was higher, but not significantly so, for intervention group adolescents in all instances. At the two-year follow-up, there was a significant experiment group x outcome interaction effect, and Z tests confirmed that three of four tests were statistically significant (both tests for PDFY and one for ISFP). ISFP adolescents in the no use status at the one-year follow-up were more likely to remain in that status at the two-year follow-up, compared to control youth. Those who had already initiated substance use at the one-year follow-up were more likely to remain in this status compared to the control group, but this finding was only marginally significant at the .10 level. Thus, while substance use rates increased among all groups over the course of the study, transitions to substance use at the two-year follow-up were significantly lower among intervention group adolescents.

Wave 5: Four-year Follow-up (Tenth Grade)

Substance Use Outcomes (Spoth, Redmond, and Shin, 2001)

At the end of tenth grade, the ISFP group relative to controls showed significantly lower new user proportions across five lifetime substance use measures (ever drank alcohol, ever drank without parent permission, ever been drunk, ever smoked cigarettes, ever used marijuana). The differences were highest for drunkenness and marijuana use, with relative reduction rates (i.e., the difference in the proportions of new users) for ISFP adolescents of 40.1% for having ever been drunk and 55.7% for having ever used marijuana.

Differences in the proportion of adolescents using alcohol and tobacco in the past month, and marijuana in the past year were also examined at the tenth grade follow-up. Significantly lower proportions of ISFP youth, compared to the control group, reported past month alcohol use (with a relative reduction of 30%) and past month cigarette use (with a relative reduction of 46%). Thus, the findings reveal both primary prevention (delayed initiation) and secondary prevention (delayed progression) effects for adolescents receiving the ISFP intervention.

The frequency of past month alcohol and cigarette use, as well as the alcohol composite and tobacco composite index scores, were lower at 10th grade for ISFP participants relative to controls. The marijuana use variable was too skewed for appropriate analysis.

Aggressive and hostile behaviors at 10th Grade (Spoth, Redmond, and Shin, 2000)

For the measures of observer ratings of aggressive and hostile behaviors in interactions, multilevel ANCOVA's showed significantly lower scores on the observer rated index of aggressive and hostile behaviors in the ISFP group, compared to the control group, at the 10th grade assessment. When these interactions were analyzed separately by the sex of the parent, there was a significant experimental group difference in the aggressive and hostile behaviors exhibited towards mothers, with lower levels of aggression and hostility in the intervention group. No significant group differences were found with fathers.

Although the observed aggressive and hostile behavior score was lower for the experimental group, analyses failed to show significant intervention control-group differences in the family member report of aggressive and hostile behaviors in parent-adolescent interactions at the 10th grade assessment. Supplemental analyses of outcomes using individual family reports demonstrate a significant experimental group difference in the aggressive and hostile behaviors exhibited towards mothers, with lower levels of aggression and hostility in the intervention group. As with the findings on the observer rated index, no significant group differences were found with fathers.

The ISFP group demonstrated a significantly lower score than the control group on the adolescent report of aggressive and destructive conduct at the 10th grade follow up assessment. Multilevel ANCOVA's demonstrate a low-medium effect with 1 in 4 of all control group adolescents (24.5%) reporting one or more aggressive or destructive behaviors, compared to 1 in 7 (14.6%) for the treatment group.

Parental risk moderation of substance use trajectories (Guyll, Spoth, Chao, Wickrama, and Russell, 2004)

This analysis investigated whether the family risk factor of parental social emotional maladjustment moderated the effects of ISFP and PDFY. With regard to main effects over a 4-year period, relative to control group adolescents, ISFP adolescents demonstrated both lower final levels of alcohol use and slower rates of increasing use across the time frame of the study. In contrast, the PDFY intervention did not significantly affect final levels of use but did significantly reduce the rates of increasing alcohol use. As with the results for alcohol use, the ISFP youth demonstrated lower levels of final tobacco use and slower rates of increasing use when compared to youth in the control group. No significant intervention effects for tobacco use were noted in comparisons between the PDFY and control adolescents.

There was no indication that family risk moderated the effects of ISFP or PDFY on lower final levels of alcohol use or slower rates of increasing alcohol use over time. Results also revealed no tendency for family risk to moderate the effect of either ISFP or PDY on final levels of tobacco use or rates of increasing use over time.

Wave 6: Six-year Follow-up (Twelfth Grade)

Growth curve analyses (Spoth, Redmond, Shin, and Azevedo, 2004)

At the six-year follow-up (six years following baseline, grade 12), nonlinear growth curve analysis demonstrated that adolescents in the ISFP condition had a slower overall growth in lifetime use of alcohol, lifetime cigarette use, and lifetime use of marijuana relative to controls. In addition, a significant pretest difference was observed for lifetime use of alcohol without parental permission. Within the ISFP group at baseline, a lower level of alcohol use without parental permission relative to the control group was noted, followed by a growth rate similar to controls, yielding a growth rate lagging behind that of the control group over the course of the study. Finally, there were significant differences in times to inflection points for three outcomes, the alcohol composite use index, lifetime drunkenness, and lifetime cigarette use. In all instances, control group growth rates reached their maximum values at an earlier point in time than in the ISFP group.

Four outcomes showed significantly delayed growth rates to specific use levels (levels approximately half that of control group 12th-grade rates). This delayed growth was evident for initiation of alcohol use without parental permission, drunkenness, cigarette use, and the alcohol use composite index. The growth in substance use among PDFY adolescents lagged behind those of control group adolescents for the two tobacco outcomes (the tobacco composite use index and lifetime cigarette use).

Internalizing symptoms and polysubstance use (Trudeau, Spoth, Randall, and Azevedo, 2007)

The ISFP condition adolescents demonstrated a lower rate of increase across time on internalizing symptoms than control condition adolescents; however, ISFP did not have a significant effect on overall levels of internalizing across the time frame studies. It should be noted that approximately 18% of the adolescents surveyed scored in the borderline to clinical range of the CBCL-YSR anxious/depressed index on average across Waves 2-6. Supplemental analyses found that the intervention clearly had a positive impact on clinically significant levels of internalizing. By 12th grade, significantly fewer ISFP group adolescents scored at or above borderline range compared to the adolescents in the control group.

ISFP adolescents demonstrated a lower overall level and a lower rate of increase in monthly polysubstance use across time (sixth through the 12th grade) compared with the control adolescents. The intervention slowed the rate of increase in polysubstance use over time significantly more for girls than for boys, although overall levels of use were lower in the intervention group for both genders. There were significant associations between internalizing and polysubstance use on both overall levels and rates of change of polysubstance use for girls, but not for boys. Girls demonstrated a higher overall level of internalizing, a greater rate of increase and a greater rate of deceleration over time than boys. Both boys and girls in the ISFP demonstrated a lower rate of increase in internalizing than control group adolescents.

Methamphetamine Use (Spoth, Clair, Shin, and Redmond, 2006)

At the 12th grade follow-up (the only data point at which information on methamphetamine use was collected), none of the 148 participants in the ISFP intervention condition reported using methamphetamines in the past year; among the 156 participants in the control condition, 5 (3.21%) reported use, resulting in a statistically significant difference. Among the PDFY condition participants, 5 (3.57%) reported using methamphetamines in the past 12 months - a rate similar to that in the control group.

Mediation (Spoth, Randall et al., 2008)

This study first examined the intervention effects on parenting competency and substance use risk in grade six. The significant results replicated previous studies. In addition, a mediation analysis showed that the intervention improved school engagement in grade eight and academic success in grade 12 indirectly through improvements in parenting competency and substance abuse risk.

Wave 7: Long-term Follow-up in Young Adulthood (approximately age 21)

Substance Use Frequency (Spoth, Trudeau, Guyll, Shin, and Redmond, 2009)

For all models, the ISFP condition significantly predicted the slope for the adolescent substance initiation index, indicating slower growth in the index among ISFP subjects.

The direct effects models found significant ISFP effects on drunkenness frequency and the polysubstance use index, a marginally significant direct effect on cigarette frequency (p<.10), and insignificant effects on alcohol-related problems and illicit drug frequency.

Significant indirect effects of the ISFP intervention were observed for all outcome measures, with lower values reported by the intervention group when compared to the control group. The models including both direct and indirect effects found no improvements in model fit (when compared to the indirect effects models) for any of the outcome variables, or a significant direct intervention effect.

The analysis of relative reduction rates found significant effects (p<.01, one-tailed) for the ISFP intervention on all dichotomized young adult outcome measures: drunkenness frequency, alcohol-related problems, cigarette frequency, illicit drug frequency, and on the polysubstance use index.

Prescription Drug Misuse (Spoth, Trudeau et al., 2008)

The analysis showed that the ISFP condition had significantly lower lifetime narcotic and barbiturate misuse in young adulthood than the control group.

Health-risking Sexual Behaviors and Sexually Transmitted Diseases (Spoth, Clair, and Trudeau, 2014)

For all models, the ISFP condition predicted the slope for the adolescent substance initiation index (p<.01, one-tailed tests), indicating less change for the ISFP subjects.

The direct effects models found significant ISFP program effects on lifetime STDs and substance use during sex.

Significant indirect effects of the ISFP intervention (p<.01, one-tailed tests) were observed for lifetime sexually transmitted diseases, past year number of partners, and substance use during sex (but not condom use) with lower reported values among the intervention group when compared to the minimal contact control group. For lifetime STDs only, the model including direct and indirect effects showed a significant improvement in model

Study 2

Summary

Spoth et al. (2002) evaluated the effects of combining two programs - Strengthening Families 10-14 and LifeSkills Training (a middle school-based intervention) - with a cluster randomized controlled trial of 36 Iowa middle schools. A variety of substance use and related measures were obtained at posttest, 1-year follow-up, and 11^th and 12^th grade follow-ups.

Spoth et al. (2002) found that, relative to the control group, the combined LifeSkills Training and SFP 10-14 condition demonstrated:

• lower alcohol use
• lower marijuana initiation
• past year and lifetime methamphetamine use

Evaluation Methodology

Design: Participants in the study were seventh graders enrolled in 36 randomly selected rural schools in 22 contiguous counties in a midwestern state. Criteria for the selection of the initial pool of schools were: 20% or more of households in the school district within 185% of the federal poverty level; community size (school district enrollment under 1,200); and all middle-school grades (6-8) taught at one location. A randomized block design guided the assignment of the 36 schools to one of three experimental conditions: 1) a combined Life Skills Training (LST) and Strengthening Families Program 10-14 (SFP 10-14) group, 2) an LST only group, and a no-treatment control group. Students in the combined LST and SFP 10-14 group received both curriculums, including booster sessions (n = 4 booster sessions for the SFP 10-14 and n = 5 booster sessions for LST), while students in the LST only group received the LST curriculum including 5 booster sessions. After schools were matched and randomly assigned to conditions, school officials were contacted and informed of the experimental condition to which their school had been assigned. All seventh grade students in participating schools were recruited for participation.

On average, 46 students in each school completed the pre-test (n = 1,664 total), with 549 in the combined LST and SFP 10-14 group, 621 in the LST only group, and 494 in the control group. A total of 1,563 students completed the post-test (n = 517 in the combined LST and SFP 10-14 group, n = 583 in the LST only group, and n = 463 in the control group), while 1,372 students completed the one year follow-up (n = 453 in the LST and SFP 10-14 group, n = 503 in the LST only group, and n = 416 in the control group). At the 11th grade follow up (4.5 years post-baseline), a total of 558 families participated (n = 190 in the combined LST and SFP 10-14 group, n = 202 in the LST only group, and n = 196 in the control group), and at the 12th grade follow-up (5.5 years post-baseline), a total of 597 families participated (n = 191 in the combined LST and SFP 10-14 group, n = 209 in the LST only group, and n = 197 in the control group). For the evaluation of modeling factors influencing enrollment in the intervention, a total of 730 families were eligible for participation because they had previously completed a prospective telephone survey after the pre-test was administered.

Data collection in the form of student surveys were completed in classrooms at pre-test, post-test (one month after completion of the intervention), at the one-year follow-up (one year after completion of the intervention), at the 11th grade follow-up and at the 12th grade follow-up. In addition, a bogus pipeline procedure was performed in order to promote honesty in answering smoking related questions. The sample was analyzed for pre-test equivalence on sociodemographic and outcome measures; the only significant difference discovered was that the control group contained more dual-parent families than the two intervention groups. This variable was included as a control variable in the subsequent outcome analyses. Analysis of differential attrition revealed no significant dropout by condition interactions from pre- to post-test or from post-test to follow-up for any outcome or sociodemographic variable. The only data reported from the 11th and 12th grade follow-ups was on past year and lifetime methamphetamine use.

Sample: The sample was 53% male and predominantly Caucasian (96%).

Measures: Self-reported use of alcohol, cigarettes, and marijuana was obtained from the classroom-administered questionnaire. Individual items included (a) "Have you ever had a drink of alcohol?", (b) "Have you ever smoked a cigarette?", (c) "Have you ever smoked marijuana or hashish?" All three items were answered using a yes/no format. Inconsistent reports in lifetime substance use were corrected. Lifetime use measures were adjusted to control for baseline use, with these adjusted lifetime use measures (new-user rates) indicating whether use was initiated since baseline. Three lifetime use items were individually examined and summed to form the substance initiation index (SII). Adolescent past year methamphetamine use was assessed using the single item: "In the past 12 months, how many times have you used methamphetamines (meth)?" Adolescent lifetime use of methamphetamines was assessed using the single time "Have you ever used methamphetamine (meth)?" These items were then dichotomized and recoded into 0 for respondents who did not use methamphetamines and 1 for those who did. These items were not included in the pre-test surveys, but were added at later waves, starting at the 7th grade spring semester data collection point (post-test), and at all subsequent waves of data collection.

Implementation adherence was measured using a detailed checklist on which an observer rated whether or not the program implementer covered each aspect of each activity. A total of 25 to 50% of the total number of observed sessions for each of the three ISFP components was observed by a second observer to calculate interrater reliability. LST implementation forms allowed observers to record whether each lesson had been implemented as originally designed. Classroom observers were asked to evaluate a series of lesson objectives and activities and to indicate whether or not the program content was covered when the lesson was taught.

As a part of the predictors of parental inclination to enroll in preventive interventions, three family sociodemographic factors were included: (a) respondent educational attainment, (b) number of children, and (c) household income. Respondent education was coded from 1-8 with 1 representing less than a high school education and 8 representing an advanced degree. Household income was coded on a 7-point scale, with 1 representing incomes of less than $5,000 and 7 representing incomes of greater than $75,000 annually.

The parenting resource use measure was constructed as the mean of six 3-point items assessing the use of particular resources for parenting support during the previous two years. These resources included reading newspaper or magazine articles, talking with friends and relatives, talking to a religious leader, talking to a family counselor, use of support groups for parents, and attendance in skills-building programs. The response format included "not at all," "occasionally," or "regularly," except for the item on skills-building programs, which was originally scored 0-5 or more, then rescaled to have a 3-point range.

A standardized measure of child behavior problems was modified to accommodate time constraints on the telephone interview. Nine problem behaviors (e.g., argues a lot, is disobedient at home, has temper tantrums or a hot temper) were assessed and scores were averaged into an overall measure of problematic behavior. Each behavior was rated by the parent as being "not true," "somewhat true," or "very true" of their 6th grader during the past 6 months.

Analysis:

A multilevel (mixed model) analysis of covariance (ANCOVA) using SAS Proc Mixed with restricted maximum likelihood estimation and listwise deletion of missing data was used to test for intervention effects on the SII. Because assignment to treatment conditions was made at the school level, school was incorporated as a random effect in the analyses. In addition, new-user analyses for specific substances were conducted at the school level, based on the proportions of new users in each school.

Descriptive analyses of observed implementation adherence measures were conducted and interrater agreement was assessed. After determining the classification criteria for higher- versus low-adherence intervention implementation groups, the differences between the two groups were assessed using t -tests. Analyses of covariance were used to test for intervention outcome effects in the total sample, along with the higher-adherence and the lower-adherence subgroups. The analysis also included two planned contrasts: once comparing the LST-only condition with the control group, and one comparing the LST and SFP 10-14 combined group with the control group.

Tests of statistical significance were conducted for methamphetamine use data only for those waves in which at least 5 participants reported methamphetamine use in at least one of the three conditions. Because of the small cell sizes, the Fisher exact test was used to assess differences in methamphetamine use between the intervention and control conditions in each study.

After conducting initial descriptive and correlational analyses, structural equation modeling was conducted with LSREL 8. Weights were applied to the sample data to adjust for differences in the sampling rates across the 36 schools (all families were selected in small schools; families were randomly selected from larger schools). Modeling analyses were based on a correlation matrix incorporating polychoric correlations. The overall fit of the model was determined through the likelihood ratio chi-square statistic, the Goodness-of-Fit Index (GFI), and Adjusted Goodness-of-Fit Index (AGFI), the Normed Fit Indwx (NFI), and Hoelter's Critical N (CN). T-tests were used to assess the statistical significance of individual model parameters. In addition, modification indices were examined for evidence of important parameter omissions. Finally, supplemental modeling analyses also were conducted using robust standard errors and full information techniques to assess possible biases associated with multivariate non-normality and missing data, respectively.

Outcomes

Because only initiation measures were applied in the outcome analysis, the post-test was considered to be the baseline time point (since the analyses examined differences in substance initiation after delivery of the interventions).

The substance initiation index (SII) score at one year after intervention posttest was lowest for the LST and SFP 10-14 combined condition, while the LST-only group had the next lowest SII score and the control group had the highest SII score. The LST and SFP 10-14 combined group scored significantly lower on the SII than the control group, but the difference between the combined group and the LST-only group was non-significant. The LST-only group also scored significantly lower on the SII than the control group.

New User Rates: The LST and SFP 10-14 combined condition demonstrated the lowest new user rate (for alcohol and marijuana) compared to the LST-only and control groups. The relative reduction rate (the percentage difference in the proportion of new users in the intervention group relative to the control group) for the combined condition was 30% for alcohol initiation, while the same rate for the LST-only condition was 4.1%. There were no significant findings associated with cigarette initiation.

Implementation Adherence: Each LST classroom was observed either two or three times. As in Study 1, observers confirmed that a high level of program content was delivered; the average adherence level in the classroom implementation of LST was 85%. Higher-adherence schools were defined as those demonstrating that at least 80% of program content was delivered in all classrooms. Fifteen of 24 schools achieved that rate while eight were classified as lower-adherence. Mean LST intervention implementation adherence scores differed between the higher- and lower-adherence schools.

Proximal outcome variables included substance-related knowledge, perceived substance use among peers and adults, and substance refusal and resistance skills. In both experimental conditions, the intervention tended to increase substance-related knowledge (smoking, drinking, and marijuana use knowledge) in both the total sample and in the higher-adherence schools at post-test. Compared with the higher-adherence schools, substance-use knowledge scores in the lower-adherence group of schools were influenced only in the LST condition, and not in the LST and SFP 10-14 combined condition. At the 1.5 year follow-up, positive intervention effects on substance-related knowledge remained, though fewer significant effects emerged. Effects on smoking knowledge were significant in the LST and SFP 10-14 combined condition for the total sample and higher-adherence schools, whereas positive effects emerged in the LST-only condition in the lower-adherence group of schools; drinking knowledge attained significance only for the LST-only condition in the lower-adherence group of schools. Students in both the LST and LST and SFP 10-14 combined schools viewed adult substance use to be less prevalent than did their counterparts in the control condition, for both the total sample and the higher-adherence group of schools. Overall, the pattern of means for perceptions of adult use remained stable from post-test to the 1.5 year follow-up, with significant effects still evident in the total sample and higher-adherence schools for both treatment groups, and positive effects in the lower-adherence schools becoming significant for the LST-only group.

Long Term

At the 11th grade follow-up, only 1 adolescent of the 187 in the combined LST and SFP 10-14 group reported using methamphetamines in the past year; among the 193 participants in the control condition, 8 reported use, resulting in a statistically significant difference. Of the 199 LST-only condition participants, 5 reported using methamphetamines in the past year, which was not statistically different from the control group. Of the 187 participants in the LST and SFP 10-14 group, only 1 reported lifetime methamphetamine use, while 10 of the 193 participants in the control condition reported lifetime use, a significant difference. Of the 199 LST-only participants, 5 reported lifetime methamphetamine use, which was not significantly different from the control group.

At the 12th grade follow-up, 4 of the 189 participants in the combined LST and SFP 10-14 group reported past year methamphetamine use; 9 of the 196 participants in the control condition reported past year methamphetamine use, a non-significant difference. Of the 208 participants in the LST-only group, 3 reported past year methamphetamine use, a difference from the control group that approached statistical significance. A total of 5 of the 190 participants in the LST and SFP 10-14 combined group reported lifetime methamphetamine use, while 15 of the 197 participants in the control condition reported lifetime use, a significant difference. Of the 208 LST-only condition participants, 5 reported lifetime methamphetamine use, which was significantly different from the control group.

Study 3

Summary

Coombes et al. (2012) employed a quasi-experimental design to explore effects with 37 families in the United Kingdom. A variety of substance use and related measures were obtained at posttest, and 3-month follow-up.

Coombes et al. (2012) found no significant intervention effects on substance use, aggressive/destructive behavior, school absence, parenting behavior or family life.

Evaluation Methodology

Design: This exploratory, quasi-experimental study was designed to evaluate the effectiveness of an adapted version of the Strengthening Families Program 10-14 (UK) on a sample of children in the United Kingdom. A total of 53 parents/caregivers as well as 69 youth from 37 families were used in the evaluation. All youth were from three schools in England. Data were collected at baseline, post-test and at 3-month follow-up.

Families were allocated to treatment or control conditions at the judgment of the researchers, who attempted to match the two groups on demographic characteristics. The intervention group included 26 parents/guardians and 34 youth. The control group included 27 parents/guardians and 35 youth. The program was delivered in schools for two of the locations and a community center for the third location.

The evaluation used materials of the Strengthening Families program that were adapted for the UK. Here, the program was used as a universal intervention to target a whole population group that had not been identified on a basis of individual risk. Families with youth age 10-14 years were recruited from advertisements in three schools from three different geographical regions of the UK.

• Location 1 was a city in England with approximately 77,040 people and 15 schools. The intervention was delivered in one school with children age 11-16 years. The school had a total of 323 students.
• Location 2 was a former industrial town with a population of 71,599 and 14 schools. This location served as the control school, and contained children age 11-16. The population of this school was 956.
• Location 3 was a city in south Whales with a population of 341,054 and 20 schools. Here, the school served children age 11-19 and had a total population of 737 students.

Attrition: Of the 1,352 eligible families, 3% were recruited and completed baseline assessment. This included 53 parents/guardians and 69 youth. A total of 100% of the sample was retained at the 3-month follow-up.

Sample: The percentage of population considered "White British" averaged 93% across the three locations (96% in Location 1, 91% in Location 2 and 92% in location 3). The mean age of parents/guardians was 38.2 years with a mean education of 13.3 years. A total of 43% of the families were dual-parent, and the mean number of children in the families was 3.2. The youth had a mean age of 11.2 years and 52% were female.

Measures: All measures were self-report. Few details of the measures were provided, other than they were incorporated from validated measures used in previous SFP10-14 evaluations in the US.

Youth measures included:

• Substance use: Youth were asked about their alcohol and drug initiation and use.
• Aggressive and destructive behavior: Youth were asked about their conduct and interpersonal relationships.
• School absence: Youth were asked about school attendance.

Parent measures included:

• Parenting behavior: Parents were asked about discipline practices, setting standards and child-monitoring.
• Family life: Parents were asked about family cohesion, expressiveness and conflict.

Analysis: A quantitative analysis was conducted, but no details of the analysis are provided. However, a qualitative analysis of a small, purposive sample of parents/guardians (n = 14) was performed. Here, a content analysis of focus group discussions was analyzed for meaningful themes.

Outcomes:

Implementation fidelity: A total of three, 5-person teams conducted the intervention in the three locations. All team members were experienced facilitators who had undergone a 3-day training. Group size ranged from 6-7 families. A total of 98% of families were present at 5 or more of the sessions, 86% attended 6-7 sessions and 66% attended all 7 sessions.

Baseline equivalence: There was no significant difference between baseline and control groups on sociodemographic variables, but differences in outcome variables were not examined.

Differential attrition: The full sample was retained throughout the analysis.

Post-test: There were no statistically significant effects. Qualitative analysis of treatment parents revealed that they listened more to their child when they were upset, worked together with their children to solve problems and had improved family functioning. Qualitative analysis of youth revealed the program was successful in teaching peer resistance skills and developing positive friendships. Youth also reported feeling an improved relationship with their family.

Study 4

Summary

Rulison et al. (2015) examined diffusion effects on friends of participants in the program, using data from students who were eligible for the program in Study 1 but chose not to participate. A variety of substance use measures were collected at posttest, and 1-, 2-, and 3-year follow-ups.

Rulison et al. (2015) found that the cumulative proportion of friends attending SFP 10-14 significantly reduced:

• drunkenness
• cigarette use

Evaluation Methodology

Design:

Recruitment: All non-exempt students who did not participate in the previous trial of SF10-14 (described in Study 1) were used for their post-intervention data. The participating 6th-grade students came from the 13 intervention communities only.

Assignment: The participating students were originally assigned to the intervention and completed surveys that named their friends but chose not to participate in the program. Rather than using assignment, the study compared subjects with few and many friends who participated in the program. A majority consistently reported having no friends who attended SF10-14 (54-66%), while the rest reported between one and three friends attending SF10-14. No more than 1.2% ever reported having more than three friends attending the program.

Attrition: Subjects completed a baseline survey at school in the fall of sixth grade and post-intervention follow-up surveys in spring of sixth, seventh, eighth, and ninth grade. Attrition is unclear. The text says only that, of the 5,449 students in the final analytic sample, 61.1% completed surveys at all four waves, 28.7% completed surveys at two or three waves, and 10.2% completed a survey at one wave.

Sample:

The sample was split evenly across gender (50.5% female), but was predominantly white in all four waves (~82%). There was a fairly high percentage receiving free or reduced-price lunches (~32-24%), and a majority of participants came from two-parent households (75-77%).

Measures:

To assess substance use, participants were asked about whether they had been drunk or smoked cigarettes in the last month. Anti-substance use attitudes were assessed on four standardized subscales of moral attitudes, expectations, refusal intentions, and refusal efficacy. Friends' characteristics were collected to assess how many had friends participating in SF10-14 and how often they spent time with them doing "unstructured socializing." Using the data collected on SF10-14 participants in previous studies/publications, researchers used these friends' self-reports of substance use, parent - youth relationships, and parental discipline style.

Analysis:

The diffusion effects of SF10-14 were evaluated with multilevel models, nesting time within students within school cohorts. The models controlled for sociodemographic background factors but did not control for the baseline outcome (perhaps because levels were near zero at the start of 6th grade).

Intent-to-Treat: The study appears to have used data from all participants, even if they completed only one wave of data collection.

Outcomes

Implementation Fidelity:

Not applicable.

Baseline Equivalence:

Without conditions, the study examined selection by number of friends who participated in the program. It correlated this key measure with 15 baseline measures. Only one significant correlation emerged: frequency of attending religious services was significantly, although very weakly, correlated with mean proportion of SFP-attending friends across waves (r = .03, p = .044).

A related issue concerns selection into non-participation in the program. The study compared those who participated in the program (and were not studied here) with those who did not participate (and made up the sample). Differences between nonparticipants and participants before program delivery would indicate selection bias. Of the 15 variables tested, three significant differences emerged: participants had higher grades, were more likely to come from a two-parent family, and attended religious services more often.

Differential Attrition:

Attrition was not discussed.

Posttest and Long-term:

In posttest and follow-up, the cumulative proportion of friends attending SF10-14 significantly reduced the self-reported likelihood of drunkenness and cigarette use. Unstructured, unsupervised time with friends also increased the odds of drunkenness and substance abuse and mediated the influence of the cumulative proportion of friends attending SF10-14.

Study 5

Summary

Baldus et al. (2016) used a randomized controlled trial that included low socioeconomic districts across four cities in Germany. Parent and youth assessment measured substance use and other behavior problems at posttest, and 6- and 18-month follow-ups.

Baldus et al. (2016) found no significant differences between conditions when using the complete intent-to-treat sample with multiple imputation estimates for missing data.

Evaluation Methodology

Design:

Recruitment: Staff from addiction and welfare organizations recruited families from low socioeconomic districts across four cities in Germany. They spoke about the program and invited families to participate in school meetings for parents of 7th-graders. Eligible families had at least one child between 10 and 14 years of age who had not been diagnosed with a substance use disorder before the study began and whose parents did not report "severe behavioral problems documented in school records." One child and one parent from each participating family enrolled in the study.

Assignment: The researchers matched family pairs by school level, gender, and age, and then randomly assigned one family from each pair to the intervention (n=153) and the control (n=149) conditions. Parents in the control condition attended a single meeting at which they received general information and a leaflet on parenting.

Assessments/Attrition: Parents and youth completed structured interviews at baseline, after program delivery, and at 6- and 18-month follow-ups. Of the 302 assigned families, baseline data were collected for 292 families. Data from the additional surveys included n=276 (posttest), n=268 (6-month follow-up), and n=262 families (18-month follow-up). Authors state that at the 18-month follow-up, data were obtained for 86.3% of the intervention sample and 85.6% of control sample; these figures are slightly lower than those indicated in Figure 1.

Sample:

The sample included families from low socioeconomic districts from the cities of Hamburg, Munich, Schwerin, and Hanover in Germany.

Measures:

Measures were collected immediately before and after the intervention, and at six and 18-month follow-ups. The primary outcomes were lifetime substance use of tobacco, alcohol, and cannabis, measured as separate binary variables from participant self-report and validated using urine samples. Secondary outcomes included the proportion of participants who reported substance use at each assessment period, the prevalence of substance use over the prior 30 days, and self- and parent-reported behavior problems. All measures were collected during structured interviews by condition-blind assessors. Measures were used and validated in prior studies, but the researchers did not provide reliability and validity information for the current sample.

Analysis:

All analyses except the analysis of new users were baseline adjusted. Binary substance use outcomes were analyzed using logistic regression models predicting substance use from condition while controlling for baseline substance use and including school-level, gender, and age as covariates. Problem behavior outcomes were analyzed using mixed models with an auto-regressive covariance structure, predicting change from baseline from the fixed effects of time of assessment, intervention condition, school-level, age, gender, and community, the random effect of subject, and the interaction between condition and time. Multiple imputation with the Markov chain Monte Carlo method were used to handle missing data.

Intent-to-Treat: The researchers used multiple imputation to include all participants with baseline data.

Outcomes

Implementation Fidelity:

Trained raters evaluated 94 videotaped sessions, and judged that on average 85.5% of the prescribed content was delivered during the sessions. Implementation fidelity ratings did not differ significantly by location, session type, time in the session that the video tape was made, or session number.

Baseline Equivalence:

The researchers tested for baseline equivalence on ten measures, including demographic characteristics and outcome variables, and found no significant differences.

Differential Attrition:

Families with baseline adolescent tobacco use were more likely to drop out relative to families with no baseline adolescent tobacco use, which may explain the difference in results between the completer data set and the imputed data set. This was the only significant difference found on nine tests of demographics and outcome variables. The authors stated in supplemental materials that they tested for a differential attrition effect of adolescent tobacco use by intervention condition at the 18-month follow-up but found no relationship. It appears that, given the insignificant differences between completers and dropouts, tests for differential effects by condition were not conducted for other measures.

Posttest:

The researchers only measured long-term effects for the substance use variables. There were no significant effects on problem behavior at post-test.

Long-Term:

At the 18-month follow-up, the primary outcome of lifetime prevalence of tobacco use was significantly lower in the intervention condition than in the control condition in the completer data set, but not in the data set with imputed missing information. There were no significant effects on lifetime alcohol or cannabis use, or on problem behavior. For the secondary outcomes, the supplementary tables presented the effects of the intervention on the 30-day prevalence of tobacco, alcohol, and cannabis use and on new users of tobacco, alcohol, and cannabis. None of the tests demonstrated significant differences between conditions.

Study 6

Summary

Okulicz-Kozaryn & Foxcroft (2012) and Foxcroft et al. (2016) used a cluster randomized controlled trial that included twenty communities in Poland. A variety of substance use and related measures were obtained at baseline and 12- and 24-month follow-ups.

Okulicz-Kozaryn & Foxcroft (2012) and Foxcroft et al. (2016) found no significant effects on behavioral outcomes or risk and protective factors.

Evaluation Methodology

Design:

Recruitment: Eligible participants included families with 10-14 year old children from community settings across Poland. Information about the program was distributed through "conferences, journal articles, information bulletins and personal contact" and those that expressed interest had the opportunity to enroll in the study. Twenty communities enrolled in total, and within each of these communities, volunteer community workers recruited 36 families through "community agencies, schools and via information leaflets and personal contact." Recruitment of families continued for the first year of the study. In each recruited family, one parent and one child had to participate. If a family in the intervention condition had two eligible children, both parents were asked to participate.

Assignment: The researchers used a parallel group cluster randomized control trial to randomly assign communities using concealed allocation to the intervention or control conditions with a 2:1 ratio, including twice as many communities in the intervention condition. Four communities could not be randomly assigned for logistical reasons and were therefore evenly distributed across the intervention and control conditions. The initial sample included 7 communities, 219 families and 247 children in the control condition and 13 communities, 292 families, and 367 children in the intervention condition.

Attrition: Attrition was high overall. Of the initial randomized sample of 614 children, 182 (30%) did not complete the 12-month follow-up survey, and 280 (46%) did not complete the 24-month follow-up.

Sample:

The initial sample included parents and children aged 10-14 from twenty communities across Poland. The mean age of children in the intervention condition was 12.1 and the mean age in the control condition was 11.6. The intervention condition was 36.6% female while the control condition was 44.5% female. Most children lived in dual-parent homes and had relatively low income.

Measures:

Assessments were conducted at baseline and followed-up at 12- and 24-months. Primary outcome measures included:

• Self-reports of alcohol, cigarette, and other drug use including the age of first use, use in the past 30 days, and 12-month prevalence
• Alcohol use without parent permission
• Drunkenness and binge drinking in the previous 30 days.

Secondary outcome measures included the following scales:

• General Child Management
• Parent-Child Affective Quality
• Aggressive and Hostile Behaviors in Interactions
• Aggressive and Destructive Conduct
• Externalizing Behaviors subscale of the Strengths and Difficulties Questionnaire

Reliability and validity information for the scales were not provided.

Analysis:

For the primary substance use outcomes, the researchers used Bayesian analysis with a weakly informative student t prior in a binomial model with a logit link function. The models included community as a random effect and experimental group, gender, age, and baseline outcome levels as fixed effects.

Analyses were conducted multiple ways to account for missing data and baseline non-equivalence, including complete-case analyses in which missing data were removed with listwise deletion, multiple imputation analyses in which partial data were used to estimate missing data, and a propensity-score matched dataset that matched participants on age, parent binge drinking, parent child management score, parent education, and family disposable income.

Intent-to-Treat: The researchers stated that they used intent-to-treat analyses.

Outcomes

Implementation Fidelity:

No quantitative tests of implementation fidelity were performed.

Baseline Equivalence:

The researchers used propensity score matching to correct for suspected baseline non-equivalence, though no tests of baseline equivalence were reported.

Differential Attrition:

Despite high overall attrition, the researchers did not report tests for differential attrition. Of the initial randomized sample of 367 children in the intervention condition, 131 (36%) did not complete the 12-month follow-up, while only 51 (21%) of children in the control condition did not complete the 12-month follow-up. At the 24-month follow-up, 190 (52%) of children in the intervention condition did not complete the survey, and only 90 (36%) of children in the control condition did not complete the survey. The researchers also report that attrition was higher for families with full-time-employed and better-educated parents, but statistics for these tests were not reported.

Posttest:

The intervention had no effect on behavioral outcome measures or risk and protective factors at the 12-month follow-up.

Long-Term:

The intervention had no effect on behavioral outcome measures or risk and protective factors at the 24-month follow-up.

Study 7

Summary

Riesch et al. (2012) randomly assigned 16 schools in Madison, Wisconsin, and Indianapolis, Indiana, to intervention or control conditions. The family environment was assessed at baseline, immediately post-intervention, and six months post-intervention.

Riesch et al. (2012) found no significant effects on behavioral outcomes.

Evaluation Methodology

Design:

Recruitment: A total of 17 randomly-selected public elementary schools with a fifth grade in Madison, Wisconsin and Indianapolis, Indiana were approached to serve as recruitment sites. Schools were stratified by minority enrollment as high or low (above or below 60%) to ensure representation among groups. All but one of the approached schools agreed to serve as recruitment sites, and adult-youth dyads were recruited from the remaining 16 schools. Dyads were eligible to participate if they spoke English and intended to live in the area for at least eight months after enrollment. Of the 916 dyads that were invited, a total of 214 indicated an interest in participating, of which 197 were eligible. Of the eligible dyads, 167 eligible families consented, yielding an overall enrollment rate of 18% across schools and cities.

Assignment: Adult-youth dyads were randomly assigned to either the intervention or control condition by school. Families in the control condition received no programming, and simply completed outcome measures.

Attrition: Between baseline and the six-month follow-up, 35 families (21%) withdrew from the study.

Sample:

The sample included fifth-graders and their parents from Madison, Wisconsin and Indianapolis, Indiana. More than half of the families were African American, and about a third were Euro-American. About half of the families reported incomes below $25,000, and a third reported incomes above $50,000. Parents included in the study were mostly female. The average age of youth participants was 10.8 years (SD = 0.69) and about half were male.

Measures:

The family environment was assessed using established measures administered by interview and surveys in family homes at baseline, immediately post-intervention, and 6-months post-intervention. These measures yielded the following nine outcomes:

• Youth perception of family cohesion
• Parent perception of family cohesion
• Youth perception of communication with mother
• Youth perception of communication with father
• Mother perception of communication with the youth
• Youth perception of family involvement
• Parent perception of family involvement
• Youth perception of family supervision
• Adult perception of family supervision

Alpha reliabilities for parents and youth in the current sample were reported for all measures except the Health Risk Behavior Survey, and ranged from .62 to .85.

Analysis:

Change was assessed using a two-level multilevel model for fixed-occasion repeated outcomes to account for clustering by school, which was the unit of randomization. The models included baseline outcome levels as covariates. Intercepts were modeled as random effects.

Intent-to-Treat: All available data were used in the primary analyses. A second series of models were estimated that included a dichotomized treatment dosage variable.

Outcomes

Implementation Fidelity:

In the intervention condition, 42 of 86 (49%) families received at least five of the seven sessions.

Baseline Equivalence:

The researchers found no significant differences in demographic characteristics between conditions at baseline.

Differential Attrition:

The researchers found no significant differences in attrition by demographics, but did not test for differential attrition by condition.

Posttest:

The ITT analyses yielded no significant effects on behavioral outcomes. One significant effect on a protective factor was found: parents in the intervention condition reported better perceptions of family supervision, compared to the control condition, immediately following the intervention, though this difference was not maintained at the six-month follow-up.

Long-Term:

Not examined.

Study 8

This study reports on intervention effects for a culturally adapted version of the Strengthening Families 10-14 program implemented in a sample of African American families.

Summary

Spoth et al. (2003) used a randomized controlled trial that assigned 85 families to intervention or waitlist control conditions. Caregivers and adolescents reported on outcomes relating to family communication/interactions, parent-child relationships, affect, and child peer resistance skills at baseline and posttest (approximately 6-10 weeks after the baseline assessment).

Spoth et al. (2003) found that, relative to the waitlist control group, the intervention group exhibited significantly greater improvements at posttest in:

• Child participation in family meetings
• Intervention-targeted child behaviors (e.g., relationship and communication with parents)

Evaluation Methodology

Design:

Recruitment: Families were randomly selected from a larger pool of African American families who were participating in a separate longitudinal study and had completed the first wave of assessments for that study. As part of the longitudinal study, researchers identified block group areas in Des Moines, Iowa, in which African American families made up 10% or more of the population and in which 20% or more of families with children lived below the poverty line. Of the 503 eligible families, 390 (78%) agreed to participate in the longitudinal study and 348 (69%) completed the first wave of data collection. Of the 348 families, 200 (57%) were randomly selected and targeted for recruitment in the current study. Of the 200 families who were randomly selected and targeted, 151 (76%) were contacted and 119 (60%) consented to participate in the study.

Assignment: The 200 families were randomly assigned to either the intervention or waitlist control group (individual ns not provided). Consent occurred after participant assignment to condition, but the study did not present consent rates separately by condition. The intervention condition consisted of the culturally adapted version of the Strengthening Families 10-14 program, whereas the control condition continued business as usual and were offered the intervention at a later time. 

Assessments/Attrition: Assessments occurred at baseline and post-test (approximately 6-10 weeks after the baseline assessment). Of the 119 families who consented to participate in the study, 110 (92%) participated to some degree but only 85 (71%) provided sufficient data for inclusion in analyses (see Figure 1, p. 442). Counting losses from both non-consent and incomplete data, attrition reached 58%.

Sample:

The final sample consisted of adolescents 10-12 years of age (M = 10.5 years) and their primary caregivers. Most primary caregivers (82.7%) were biological mothers, 4.5% were biological fathers, 4.5% were grandmothers, 6.4% were foster or adoptive parents, and < 1% were other biological relatives, stepparents, or nonrelatives. Approximately 94% of the primary caregivers were female. Caregivers were 25-62 years of age (M = 38.4 years). For caregiver education, 6.3% reported having less than a high school diploma, 38.5% had a high school diploma or GED, 40.6% had some postsecondary education or training, 11.5% had a bachelor's degree, and 3.5% had a master's degree. On average, primary caregivers had a total of 3.5 children, of whom 2.5 were still living at home. The median gross household income was $26,600.

Measures:

Adolescents and caregivers reported on measures of family communication/interactions, affect, and child peer resistance skills at baseline and post-test. Eight outcomes were computed for analysis, five of which were caregiver-reported (intervention-targeted parenting behaviors, number of family meetings in the past month, child participation in family meetings, parent-child affective quality, and child alcohol-related peer resistance) and three of which were child-reported (intervention-targeted child behaviors, general peer resistance skills, and alcohol refusal skills). The measures of parent-child affective quality and intervention-targeted child behaviors (e.g., "I try to see things from my parent's/parents' point of view") may be viewed as measures of close relationships with parents. In the current sample, alpha reliabilities for all measures across both assessments ranged from low (α = .59) to excellent (α = .95).

Analysis:

Analyses used repeated measures analysis of variance (ANOVA) with one between-subjects factor (condition: control, intervention) and one within-subjects factor (time: baseline, post-test). Significant time-by-condition interaction effects were further probed post hoc with paired-samples t-tests of condition means.

Intent-to-Treat: The authors imputed missing baseline data for 22 participants who did not complete the baseline assessment. The method used predicted values generated by regressing baseline scores on post-test scores using data from participants who had completed both assessments. The authors thus stated that their analyses "were conducted using an intent-to-treat approach" (p. 453). Although the analyses could not include the 81 participants who were assigned to condition but did not consent to participate and did not impute missing posttest data, they appeared to use all available data regardless of participation in the program.

Outcomes

Implementation Fidelity:

Trained observers tracked the extent to which program facilitators completed the specific activities required by the intervention protocol. Program adherence was 91.5% for caregiver sessions (22% of sessions were observed), 84.4% for youth sessions (32% of sessions were observed), and 86.9% for joint family sessions (41% of sessions were observed).

Baseline Equivalence:

Tests for baseline equivalence were conducted on the analysis sample (n = 85) only. Of eight tests, the groups differed with respect to one variable: as stated by the authors, "the intervention group evidenced a smaller average score for the intervention-targeted child behaviors variable than did the waitlist control group" (p. 454). Tests for baseline equivalence on demographic variables were not reported.

Differential Attrition:

In analyses comparing dropouts to those completing the program, the authors state only that there were "no significant differences between the two groups" (p. 454). However, it is unclear as to whether these analyses were performed for both demographic and outcomes variables. Also, tests for baseline equivalence using the analysis sample indicated one condition difference that may relate to attrition. Perhaps most importantly, the conditions differed in attrition rates. Due to the lack of information on the number of families assigned to either condition, it is not possible to calculate attrition rates for the two groups. Assuming a 1:1 allocation ratio and combining losses from non-consent and incomplete data, attrition was substantially higher in the intervention condition (66%) than the control condition (49%).

Posttest:

At post-test, relative to participants in the control condition, participants in the intervention condition exhibited significantly greater improvements in two of eight outcomes tested: child participation in family meetings (a risk and protective factor) and intervention-targeted child behaviors (a behavioral outcome relating to close relationships with parents). However, the latter outcome differed across conditions at baseline.

Long-Term:

Not examined.

Study 9

This study used a culturally adapted version of the program for families in Sweden. The program consisted of two parts with seven and five sessions, respectively, held separately for youths and parents, except that each part finished with a joint family session (see Skärstrand et al., 2008). Youth received the training during the day from class teachers with assistance from group leaders. Parent participation in evening sessions was voluntary. Part 2 of the program consisted of the original booster sessions and added additional material on alcohol and drugs.

Summary

Skärstrand et al. (2014a) conducted a cluster randomized controlled trial with 22 elementary schools in Stockholm, Sweden in 2003-2006. A variety of student-report questions assessing alcohol, tobacco and illicit drug use and related measures were obtained at pre-test and one-, two-, and three-year follow-ups.

Skärstrand et al. (2014a) found no significant long-term effects (effects at posttest were not tested).

Evaluation Methodology

Design:

Recruitment: Inclusion criteria were: (1) having grade 6-9 in the same school, and (2) not having age-integrated classes. Sixty of the 226 elementary schools in Stockholm were eligible and 22 applied to participate in the study. No other information on recruitment was provided.

Assignment: A total of 22 schools were randomized to the intervention or control condition (11 in each) within two strata: high (n=12) or low (n=10) socio-economic position. The total number of students within these 22 schools was not reported. Control schools received business-as-usual; a survey sent to all principals of schools in the control group revealed that while no schools had implemented a structured manual-based program, all had used some sort of concomitant alcohol, tobacco, and other drug (ATOD) prevention activity (e.g., an invited lecturer).  

Assessments/Attrition: One intervention school and two control schools declined participation after randomization, for a school-level attrition rate of 14%. However, student-level data were only collected on the 19 remaining schools (n=707 students). All 19 schools remained in the study throughout the follow-up period. Student-level attrition was tracked at allocation (n=707), informed consent (n=587), baseline (n=521), one-year follow-up (n=508), two-year follow-up (n=465), and three-year follow-up (n=447) (see Figure 1). Using informed consent as the starting point to examine attrition, student-level attrition rates were 13% (one-year FU), 21% (two-year FU), and 24% (three-year FU). However, attrition rates would be higher if the calculations included the total number of students within the 22 schools initially randomized.  

Sample: The sample was evenly divided in terms of gender (50% boys, 50% girls). The majority (82%) attended a school located in a district with "low" social load (defined as the distribution of extra resources to city districts). In addition, most students (83%) were born in Sweden. Parent participation in the program was low, with 47% and 27% of intervention youth being represented by at least one parent in parts one and two, respectively. Thirty-four adults delivered the program (20 classroom teachers and 14 assistants); however, instructor demographics were not reported.

Measures: Student self-report measures administered at all four time points included: (1) four single-item yes/no questions indicating alcohol (drunkenness past 30 days, drunkenness lifetime), tobacco, and illicit drug (lifetime) use; and (2) an index of "other norm-breaking behaviors" (ranging from minor misdemeanors like cheating on a school test to more severe behaviors like robbery or burglary). Cronbach's alpha for this index was 0.86, and the measure was also analyzed as a binary variable ("Any norm-breaking behavior lifetime" versus "None").

Analysis: Two-level mixed-effects logistic regression models (with students nested within schools) were run at each follow-up time point, controlling for pre-test outcomes. However, the level-2 sample size of 19 is likely not large enough to accurately estimate the standard errors, and the result may be to overstate the significance of the tests. Moderation analyses included gender, age and highest education of the responding parent, ethnicity of the child and social load (i.e., distribution of extra resources) of the area.

Intent-to-Treat: The authors stated that they followed an intent-to-treat protocol. It was reported that the maximum number of missing values on the outcomes varied between 18% at baseline and 26% at the last follow-up (p. 583). As a sensitivity analysis, imputation was performed separately for each group and time of measurement using the Statistical Analysis Software (Version 9.1.3) multiple imputation procedure with the Markov chain Monte Carlo method. The authors analyzed program effects under the extreme assumptions that missing values were either all negative (best case scenario) or all positive (worst case scenario).

Outcomes

Implementation Fidelity: The only information on fidelity reported was that all treatment youth received the program, 47% of which were represented by at least one parent in part one of the program and 27% in part two. A separate article (Skärstrand et al., 2009) found that having a low score on a scale measuring parents' emotional warmth and having a more restrictive attitude towards youth and alcohol predicted program recruitment, whereas retention was associated with being born in Sweden and having a low score on the scale measuring warmth. Skärstrand et al. (2009), however, found no other predictors of participation and retention in the Swedish version of the SFP, leading the authors to conclude that the program seems to attract all types of parents in the general population.

Baseline Equivalence: There was baseline equivalence between groups for seven of eight socio-demographic characteristics (parent's education was significantly higher in the intervention group, Table 1) and all five baseline pretests (Table 2).

Differential Attrition: On page 581, the authors explained: "We performed an in-depth analysis of the missingness mechanisms, using logistic regression to model the odds of missingness on the basis of several characteristics of the students. We also formally tested for differential missing pattern between treatment groups by including in the logistic models a cross-product interaction term between intervention condition and students' characteristics and baseline behaviours." There were two sociodemographic differences (out of eight) in completers vs. attritors: The students with missing values of the outcomes at the last follow-up were significantly more likely to be male and to have higher baseline prevalence of drunkenness compared with students who had answered the questionnaire (it is unclear if these analyses examined whether having pretest data predicted missingness). However, "the interaction terms between intervention condition and baseline characteristics of the students were not statistically significant for any variable (data not shown), indicating that patterns of missingness were similar across experimental conditions" (p. 583).

Posttest and Long-Term: Smoking, alcohol and illicit drug use increased over time, but there were no significant effects on any of the five outcomes across all three time points (for a total of 15 models). In addition, "all analyses performed on imputed data confirmed results of the available case analysis, indicating no effectiveness of the prevention programme (data not shown)" (p. 583). Skärstrand et al. (2014b) discuss possible explanations for the lack of effects of the Swedish version of SFP 10-14 (originally developed and tested in the United States).

In terms of moderation analyses, for the outcome drunkenness lifetime, the treatment showed a negative effect on boys at the two-year follow-up, but this effect was not confirmed at the three-year follow-up. A negative effect among boys in the treatment group was also found for smoking, but only at the three-year follow-up. There were no other significant moderation effects.

Study 10

Summary

Segrott et al. (2022) conducted a cluster randomized trial of 715 families in Wales, UK. Assessments occurred at baseline, 9-month and 15-month follow-ups, with primary youth substance use outcomes measured at the 24-month follow-up.

Segrott et al. (2022) found no significant effects on behavioral outcomes or risk and protective factors at the 24-month follow-up.

Evaluation Methodology

Design:

Recruitment: Participants were families recruited from seven counties (town or rural district) in Wales, UK between February 2010 and June 2012. Families were self-referred or referred by education, health, or social services agencies that identified families who may benefit from the program. Eligibility criteria included English-speaking families with at least one child aged 10-14 and one parent/caregiver who could travel to a community setting location where the program was being offered. Families that did not live together or had very high needs/challenges (e.g., serious substance misuse problems) were excluded. Of 849 families who were referred, 728 (86%) were screened and deemed eligible, and 715 (98%) of the eligible families (including 919 parents and 931 youth) consented to participate in the study.

Assignment: Families were randomized to the intervention condition (n=361, including 461 adults and 477 youth) or control condition (n=354, including 457 adults and 454 youth). Randomization occurred within seven geographical area strata and among three variables (family category type, age of children, number of children in family). The intervention condition consisted of the SFP 10-14 UK plus usual care, whereas the control condition was usual care only.

Attrition: Assessments occurred at baseline and at 9-month, 15-month, and 24-month post-randomization follow-ups, with the 9- and 15-month follow-ups as intermediate assessment points. The flowchart in Figure 1 was complex, but it appears as though one control family (< 1%) withdrew after allocation, 25 families (3.5%; 21 control families, four intervention families) withdrew at the 9-month follow-up, 16 (2.2%; seven control families, nine intervention families) withdrew at the 15-month follow-up, and 69 (9.7%; 41 control families, 28 intervention families) did not complete the 24-month follow-up. 

Sample: The sample included parents and children from Wales, UK. On average, parents were aged 37 years and mostly male (77%) and White (about 99%). The average child age was about 11.8 years and just over half were female (53.7%). Average family affluency was 19.7% low, 50.4% medium, and 29.9% high.

Measures: Youth and parent reports of baseline and long-term 24-month follow-up measures consisted of substance use, youth externalizing and internalizing behaviors and emotional well-being, health status, family functioning, and parenting. At 9- and 15-month follow-ups, measures were focused on intermediate outcomes (e.g., parenting, parent-child bonding, and service utilization).

The study had two primary, youth self-reported outcomes at the 24-month follow-up: number of occasions having consumed alcohol in the last 30 days and drunkenness during the last 30 days. Youth self-reported secondary outcomes were ever use of cannabis, age of alcohol use initiation, frequency of drinking more than five drinks in a row in the last 30 days, frequency of different types of alcoholic drinks, alcohol-related problems, and academic performance at age 15/16. Weekly smoking was validated by a salivary measure. Measures were generally from well-known sources, but alphas were not presented.

Several tertiary outcomes and risk and protective factors were also assessed (e.g., child well-being and stress, child sleeping patterns, self-efficacy, school bonding, interaction with anti-social peers, attachment, involvement, and cost effectiveness).

Analysis: Multilevel logistic regressions were used to account for children and parents nested within families. Analyses were first conducted for primary outcomes (youth drinking and drunkenness) and then repeated for secondary outcomes. Covariates included three stratified randomization variables (family type, average age of child, geographic area) and baseline outcomes. Despite two differences in baseline characteristics (child gender and parent cigarette smoker), those variables were not controlled for in analyses. Primary outcome analyses used a p-value of between < .025 and < .05 for statistical significance. Cox proportional hazards models were conducted for the secondary outcome of time to initiation of alcohol, smoking, and drugs.

Missing Data Method: The authors stated, "The primary analyses were conducted on a modified intention to treat population (participants who provided outcome data analyzed in the arm to which they were randomized)" (p. 5). This indicates they used listwise deletion or complete case analysis. Further, the authors stated that multiple imputation was used in sensitivity analyses, but not for the primary analyses, which included 746 youth (80% of randomized sample) for alcohol consumption and 732 youth (79% of randomized sample) for drunkenness.

Intent-to-Treat: The authors stated that all participants with complete data were analyzed in the arm to which they were randomized. This strategy meets the common definition of intent-to-treat. In addition, the authors conducted a separate, supplemental analysis of a non-intent-to-treat sample in which families had to attend at least five sessions to be included in the analysis.

Outcomes

Implementation Fidelity: A total of 218 (60%) families in the intervention group received five or more sessions. Facilitators rated participant engagement high in 94% of activities. There was no evidence that engagement differed by group size. Data were available for 50 of 56 programs, and facilitators rated 96% of individual activities as mostly/fully covered, whereas observers rated 77% of observed activities as mostly/fully covered.

Baseline Equivalence: According to Table 2, the authors noted that there were "small differences" between treatment and control conditions in two of 15 tests at baseline: relative to the control group, the intervention group had more female youth and more parents who had ever smoked a cigarette. However, no significance tests were reported.

Differential Attrition: The authors stated only that completion rates were similar across trial arms. Completion figures listed in the CONSORT Diagram (Figure 1) show that the attrition meets the What Works Clearinghouse differential attrition standard under the optimistic but not the cautious threshold (e.g., at the 24-month follow-up, the attrition rate overall was 15.5% and the difference in attrition rates between conditions was 8.4%). This evidence suggests some potential for bias from attrition. However, the use of multiple imputation in the sensitivity analyses adjusts for attrition and may counter concerns over bias.

Posttest: Not examined.

Long-Term: At 24-months post-randomization, there were no significant effects on any behavioral outcomes for either primary or secondary outcomes. Further, there were no significant effects on tertiary outcomes/risk and protective factors.  

In exploratory moderation analyses with several baseline variables, there was no evidence of differential intervention effects for any of the subgroups for either primary outcome.

Study 11

This study tested an adapted version of SFP 10-14 in Brazil, called Famílias Fortes (Strong Families). On page 2 of the article, the authors noted that the culturally adapted program upholds the core components of the original program while making surface-level, culturally-relevant changes (e.g., replacing "rugby" and "basketball" with "football" to better align with the Brazilian context). The changes as described suggest keeping this adapted version under the main program rather than creating a new program.

Summary

Sanchez et al. (2024) conducted a cluster randomized trial of a culturally adapted version of the program in Brazil. A total of 60 social centers were randomized, which included 805 families assigned to the intervention group (n=371) or waitlist control group (n=434). Assessments occurred at baseline and six months after baseline, and primary measures were youth-reported substance use outcomes.

Sanchez et al. (2024) found that there were no significant differences between intervention and control groups on youth substance use outcomes.

Evaluation Methodology

Design:

Recruitment: Participants were families recruited from 60 Social Assistance Reference Centers in 12 municipalities located in Brazil. The social centers provided programs and resources to low-income families and individuals in the communities. Each center invited 15 families to participate in the study, based on the list of families that frequented the centers. Inclusion criteria were families with at least one youth between 10 and 14 years old; adult and youth living in the same house; availability to attend the seven program meetings; and residing within one km from the social centers. Exclusion criteria were families with very high needs or challenges such as drug addiction and family breakdown. If a social center had more than 15 families interested, the first 15 who volunteered were included in the study. A total of 805 families (n=1,610 total participants, one parent/guardian and one youth from each family) were eligible and agreed to participate in the study.

Assignment: In a cluster randomized trial, 60 social centers were randomly assigned within cities to the intervention condition (n=30 clusters, 371 families) or waitlist control condition (n=30 clusters, 434 families). Families in social centers assigned to the control group were put on a waiting list to participate in the program after the study end.

Assessments/Attrition: Assessments occurred at baseline (November/December 2021) and six months after baseline (May/June 2022). The program length was seven weeks. There was no cluster attrition, but family attrition was 12.4% at the 6-month follow-up.

Sample: The sample of youth had a mean age of 12.6 years and included 51.5% females. The sample of parents/guardians had a mean age of 39.5, was mostly adult females (91.8%), and was mainly from lower socioeconomic status (73.5%). At baseline, the substance most used by adolescents was alcohol (13.3%), and marijuana was the drug with the highest perceived risk (75.3%).

Measures: Youth self-reports and non-independent parent reports of baseline and 6-month follow-up measures consisted of youth lifetime substance use behavioral outcomes (alcohol, binge drinking, tobacco, marijuana, inhalants) and several risk and protective factors (i.e., youth perceived substance use risk, parenting styles, parental discipline, children's exposure to parental substance use). Measures were drawn from international well-known sources that were translated and validated for Brazilian Portuguese and instruments used in other Brazilian substance use prevention programs. However, no reliability information was presented for risk and protective factor scale measures.

Analysis:

Intervention effects were analyzed using multilevel mixed-effects modeling with repeated measures. Three-level mixed-effects models included repeated time observations nested within participants at level 1, participants clustered within social centers at level 2, and social centers at level 3. Models were estimated using social centers and participants as random effects, and explanatory variables (study condition, time of assessment, the interaction between group and time) and covariates (gender, age, socioeconomic status) as fixed effects.

Missing Data Method: On pages 5, the authors stated, "In addition, mixed-effects models deal with missing data using maximum-likelihood estimation analyzing all available outcome data, regardless of whether an individual has complete data, making these models consistent with an ITT analysis." Data were assumed to be missing at random.

Intent-to-Treat: The authors conducted intent-to-treat analyses, which included all families who started the study regardless of the number of meetings they attended or whether they completed the questionnaire at follow-up. The authors also conducted a separate supplemental analysis of a non-intent-to-treat sample (per protocol) in which families had to attend at least five sessions and respond to both the baseline and follow-up questionnaires to be included in the analysis.

Outcomes

Implementation Fidelity: Of the 324 families assigned to the intervention group, 89% received five or more doses of the intervention (i.e., they participated in at least five of the seven program meetings), and 23.3% did not participate in all of the program meetings.

Baseline Equivalence: According to Table 1, there were differences between intervention and control groups in six of 20 tests at baseline (i.e., socioeconomic status, baseline parent drinking behaviors, and parental discipline measures) for the randomized sample.

Differential Attrition: On page 6, the authors noted that their attrition analysis showed few statistically significant differences on sociodemographics and baseline outcomes for families who completed vs. attrited at the 6-month follow-up. Table 4 shows differential attrition for three of 20 baseline variables for completers vs. attritors: families lost at the 6-month follow-up had youth that were younger and indicated higher inhalant use at the baseline, and they had parents/guardians who were more forgiving and less negligent. Additionally, the authors stated that attrition was not differential by intervention condition, though no significance tests were reported. Attrition rates were 12.7% for the intervention group and 12.2% for the control group, which meet the What Works Clearinghouse differential attrition standard under both cautious and optimistic thresholds.

Posttest: There were no significant program effects on any of the four adolescent substance use outcomes. However, there were significant effects on three of 13 risk and protective factors. The program reduced the chance of parents/guardians being classified as negligent; decreased the chance of adults exposing adolescents to their drunken episodes; and increased the use of nonviolent discipline by caregivers. Table 3 indicates that ITT and per-protocol analyses showed similar results.

Long-Term: Not tested.