LifeSkills Training (LST)

Study 1

Summary

Botvin et al. (1990, 1995, 2000) and Griffin et al. (2004, 2006) conducted a cluster randomized trial to examine 56 New York State schools with 5,954 seventh-grade student participants. The schools were assigned to two LST groups (one with in-person training and one with video training) and a control group. Students in the schools were followed through 10 years to assess self-reported substance use and HIV risk behaviors.

Botvin et al. (1990, 1995, 2000) and Griffin et al. (2004, 2006) found that, relative to the control group, the two intervention groups reported significantly lower

• Marijuana use at posttest
• Tobacco, alcohol, and marijuana use at the three-year follow-up
• Driving violations at the three-year follow-up
• Illicit drug use at the 3.5-year follow-up.
• HIV risk behaviors at the 10-year follow-up.

Evaluation Methodology

Design:

Recruitment: The study recruited 56 schools in three geographic areas of New York State that contained 5,954 seventh-grade students. Griffin et al. (2006) reported that 5,569 of the students completed the baseline assessment.

Assignment: The schools were randomized within strata based on the level of reported cigarette use to one of three conditions: (1) an intervention with a formal one-day training workshop and implementation feedback by project staff (n = 18), (2) an intervention with training provided by videotape and no implementation feedback (n = 16), and (3) a treatment-as-usual control group (n = 22). Botvin et al. (1995) reported the loss of one school after random assignment but before the study began. That, plus the intentional oversampling of control schools, contributed to the uneven number of schools across conditions.

Assessments/Attrition: Initial assessments in Botvin et al. (1990) occurred at baseline (fall 1985) and at the end of each year of the three-year intervention (year one at the end of 1985/86, year two at the end of 1986/87, and year three at the end of 1987/88). The year-three assessment served as the posttest. Botvin et al. (1990) reported that, of the original 5,954 students eligible for the study, 4,466 (75%) provided data at both the pretest and posttest.

The assessment six years after baseline or three years after posttest in Botvin et al. (1995) came at the end of grade 12. It included 3,597 (60.4%) of the original 5,954 students. Griffin et al. (2004) also examined data six years after baseline but reported a sample size of 2,042 (34%), which oddly is the same sample size as at the 10-year follow-up.

The 6.5-year assessment or 3.5-year follow-up in Botvin et al. (2000) came after the end of grade 12. The subsample included only 447 (7.5%) participants who completed a mail questionnaire related to illicit drug use.

The assessment 10 years after baseline or seven years after posttest in Griffin et al. (2006) included 2,042 youth (34% of the students at baseline).

Sample: The initial sample was approximately half (52%) male and predominantly (91%) white. Participating students attended a mixture of suburban and rural schools, 83% lived with both parents, and 58% of the fathers attended at least one year of college. At the 10-year follow-up, participants had an average age of 25.

Measures: The measures came from student reports collected by research staff rather than the teachers who implemented the program or from mail or phone questionnaires. The exceptions are two measures based on records from the Department of Motor Vehicles.

At posttest (Botvin et al., 1990), the five behavioral outcomes included smoking frequency, drinking frequency, drinking amount, drunkenness, and marijuana use frequency. The study provided no information on the reliability or validity of the substance use variables. However, to enhance the validity of drug use self-reports, breath samples were collected at each data point. The breathalyzer data were not examined but were said to increase the validity of self-reports. Also, unique IDs helped ensure confidentiality. Other mediating outcome measures of knowledge, attitudes and beliefs, skills, self-efficacy, self-esteem, and social anxiety had, with two exceptions, acceptable alpha reliability values. A two-thirds subset of the sample was randomly selected to complete the mediating measures. 

At the three-year follow-up, Botvin et al. (1995) examined nine smoking, alcohol, and marijuana use measures and eight polydrug use measures based on the combination of tobacco, alcohol, and marijuana use. Students provided data through surveys completed in classrooms, by mail, or by phone. Although originally obtained as ordinal scales, the substance use outcomes were treated as dichotomies. The authors noted that carbon monoxide levels in expired air were significantly correlated with self-reported cigarette smoking (r = 0.35, p<.001) across groups, "providing presumptive evidence for the validity of the self-report data."

At the three-year follow-up, Griffin et al. (2004) matched student names to records at the Department of Motor Vehicles. Measures included two dichotomous scores - one indicating the presence of any violations on one's driving record and the other indicating the presence of any points.

At the 3.5-year follow-up, Botvin et al. (2000) examined eight outcomes measuring the frequency of lifetime use of illicit drugs. The measures summed individual items to create composite scores for marijuana, cocaine, inhalants, nonmedical pill use, heroin and other narcotics, hallucinogens, total illicit drug use, and total illicit drug use other than marijuana.

At the 10-year follow-up, Griffin et al. (2006) examined three measures of HIV risk behavior: number of sex partners in the last year, having sex when drunk or high, and high-risk substance use (i.e., alcohol or marijuana intoxication or use of other illicit substances). If participants reported all three, they received a score of one on the HIV risk index; otherwise, they received a zero.

Analysis:

The initial analysis (Botvin et al., 1990) used MANCOVA with pretest scores as covariates. Significant multivariate results were followed by univariate general linear models with preplanned comparisons of covariate-adjusted means. The models did not adjust for clustering within schools, the unit of assignment. More appropriately, given the cluster randomized assignment, the additional analysis reported in Footnote 3 used schools as the unit of analysis and thereby avoided the problem of inflated statistical significance.

The follow-up analysis (Botvin et al., 1995) treated schools as the unit of analysis. It employed ordinary least-squares regression for school means with baseline means, assignment blocks, and experimental groups as predictors. One-tailed significant tests were used, however.

Botvin et al. (2000) used generalized estimating equations to adjust for school clustering. The models controlled for gender, age, grades, and drinking frequency at baseline. Griffin et al. (2004, 2006) also used generalized estimating equations to adjust for school clustering.

Missing Data Strategy: Botvin et al. (1990, 1995, 2000) and Griffin et al. (2004, 2006) used complete cases without imputation of FIML.

Intent-to-Treat: Botvin et al. (1990) did not conduct an intent-to-treat analysis. They examined only the 3,684 students who received at least 60% of the prevention program. This high-fidelity sample included 82% of students with complete data and included 28 of the 34 randomized intervention schools but all 22 of the randomized control schools.

Botvin et al. (1995, 2000) and Griffin et al. (2004, 2006) conducted both intent-to-treat analyses using all participants with baseline and follow-up data as well as non-intent-to-treat analyses of participants experiencing high-fidelity implementation. Although using all available data, the intent-to-treat analyses were limited by very high attrition.

Outcomes

Implementation Fidelity:

The quantitative measure of the amount of intervention material covered ranged from a low of 27% to a high of 97% with a mean of 68%. The level of program implementation was roughly comparable for the two intervention conditions (67% and 68%). About 75% of the students in the prevention conditions were exposed to 60% or more of the program material.

Baseline Equivalence:

All reports used the analysis sample for tests of baseline equivalence.

• Botvin et al. (1990) reported that an overall MANOVA for the five primary behavioral outcomes at baseline showed no significant differences across the three conditions.
• Botvin et al (1995) tested for baseline equivalence of sociodemographic and outcome measures and reported that there were no significant differences.
• Botvin et al. (2000) reported that the conditions "were similar at baseline on most of the drug use variables, including smoking and drinking quantity, and drunkenness and marijuana use frequency." However, students in the control group reported significantly higher levels of drinking frequency than intervention students.
• Griffin et al. (2004) reported "no differences between the experimental and control groups at baseline in terms of gender composition or the alcohol use index."
• Griffin et al. (2006) reported no pretest differences "between the experimental and control groups in terms of any of the demographic variables (gender, minority status, or percent from two parent families), or in terms of rates of substance use or grades received in school in the 7th grade." Also, "there were no differences across conditions in terms of percent married or cohabitating, percent of college graduates, or percent with incomes of $15,000 per year or less."

Differential Attrition:

All reports tested for differential attrition by using the analysis samples for tests of baseline equivalence. In addition, each report included the following:

• Botvin et al. (1990) reported on a series of two-way ANOVAs for each of the primary behavioral outcomes. Main effects analyses found that baseline smokers, drinkers, and marijuana users had higher attrition rates than nonusers. Interaction effects analyses found greater attrition among the baseline marijuana users in the control condition. The authors thus noted that the net effect of the attrition was to provide for a more conservative test of the interventions. Otherwise, Botvin et al. (1990) reported that the demographic characteristics of the analysis samples "were the same as those of the original sample[s]."
• Attrition was high (40%) at the three-year follow-up (Botvin et al., 1995). With attrition as the outcome, the study tested for the main effects of condition and the interaction effects of condition and baseline drug use status. The authors reported that "No differential attrition effects were found for any of the drug use variables in either the full sample or the high fidelity sample."
• The attrition rate of 66% at the three-year follow-up in Griffin et al. (2004) was high, and the authors presented no tests other than the baseline equivalence comparisons.
• The attrition rate of 92.5% at the 3.5-year follow-up was exceedingly high (Botvin et al., 2000), but the authors presented no tests other than the baseline equivalence comparisons.
• The attrition rate of 66% at the seven-year follow-up was high (Griffin et al., 2006). The authors noted that the rate "was similar across conditions." Main effect analyses indicated that those who reported smoking, drinking, or marijuana use at baseline were significantly more likely to drop out of the study. Also, males and minorities dropped out of the study at a higher rate compared to females and non-minorities. However, interaction analysis indicated that the rate of attrition of substance users and demographic factors did not differ across experimental conditions.

Posttest:

Botvin et al. (1990) examined posttest results at the end of the three-year program. The individual-level MANCOVA analysis without adjustment for clustering found significant condition differences overall across the five substance use outcomes. Individual-level ANOVA tests then showed significant condition differences for three of the five outcomes: smoking, drunkenness, and marijuana use. The aggregate analysis reported in Footnote 3 accounted for clusters but used one-tailed tests for the three significant outcome variables. The one-tailed tests replicated the significant results for all three outcomes, but a two-tailed test showed only one significant outcome: marijuana use. Finally, 23 individual-level tests for risk and protective factors showed significant condition differences favoring the intervention groups (or 12 adjusting for multiple tests).

Long-Term:

At the six-year assessment or three-year follow-up, Botvin et al. (1995) reported 34 tests in Tables 2 and 3 (two conditions by nine substance use outcomes and eight polydrug use outcomes). Counting only those significant at .01 in the one-tailed tests, the results showed five significant outcomes. The intervention group without implementation feedback had significantly lower scores than the control group on monthly cigarette smoking, drunkenness, weekly poly cigarette and alcohol use, and weekly poly cigarette, alcohol, and marijuana use. The intervention group with implementation feedback had significantly lower scores than the control group on weekly poly cigarette, alcohol, and marijuana use.

Also, at six years, Griffin et al. (2004) found that, after controlling for 12th-grade alcohol use, the intervention group had lower odds of having driver's license violations (OR = .75) and lower odds of having driver's license points (OR = .75) than the control group. The authors noted that the effects remained significant without the control for alcohol use. Further tests suggested that stronger anti-drinking attitudes in 10th grade mediated the intervention effect on driver's license violations.

At the 6.5-year assessment or 3.5-year follow-up, four of eight tests reached statistical significance. The combined intervention groups relative to the control group had significantly lower use of heroin and other narcotics, hallucinogens, total illicit substances, and total illicit substances other than marijuana.

At the 10-year assessment or seven-year follow-up (Griffin et al., 2006), the intervention group had significantly lower HIV risk scores than the control group (OR = .70). However, the effect was not significant in estimates using structural equation models. Also, there was no intervention effect on condom use. Mediation tests suggested that the program reduced HIV risk behavior in young adulthood by reducing alcohol and marijuana intoxication during junior and senior high school, but the analysis used only a non-intent-to-treat high-fidelity sample.

Study 2

Study 3

Summary

Smith et al. (2004) and Vicary et al. (2004, 2006) used a cluster randomized trial to examine 732 seventh-grade students in nine Pennsylvania middle schools. The schools were randomly assigned to LST, infused LST that delivered the program as part of the regular curriculum, or a control group. Assessments through the end of ninth grade examined substance use outcomes.

Smith et al. (2004) found no effects of LST compared to the control group for males and found effects in grade seven for females that disappeared by grade eight. Vicary et al. (2004, 2006) found no effects on substance use in grades eight or nine.

Evaluation Methodology

Design:

Recruitment: The sample came from nine middle schools in nine rural Pennsylvania school districts. The schools were of low socioeconomic status (one third of the student body in the school district qualifying to receive free or reduced lunch) and had relatively small enrollment (less than 1,000). A total of 732 consented youths who began grade seven in 1999-2000 participated in the study.

Assignment: The cluster randomized design randomly assigned the nine middle schools to three conditions, each with three schools: LST (n = 234 students), infused LST that delivered the program as part of the regular curriculum (n = 297 students), and a control group that received neither program (n = 201 students).

Assessments/Attrition: The four assessments occurred at baseline (the beginning of grade seven) and the end of grades seven, eight, and nine. Students received the main program in grades seven and eight and received booster sessions in grade nine. Thus, the first two post-baseline assessments occurred while the three-year LST program was ongoing, and the last served as a posttest. Smith et al. (2004) reported retention rates of 97% at baseline, 96% at the grade seven assessment, and 90% at the grade eight assessment. Vicary et al. (2006) reported that 79% of the students completed all four assessments.

Sample:

The sample was 54% male and 97% white.

Measures:

The six substance use outcome measures came from student self-reports and included frequency of cigarette use, alcohol use, drunkenness, binge drinking, marijuana use, and inhalant use. To maintain confidentiality, ID codes were used in place of names, and teachers were not present. However, 19 students were removed at the third assessment for suspicious or inconsistent answering patterns. Vicary et al. (2004) used log or double log transformations of these substance use variables to better approximate a normal distribution.

Vicary et al. (2004, 2006) examined numerous other self-reported risk and protective measures related to substance use, including attitudes, normative beliefs, knowledge, decision-making, communication skills, refusal skills, media awareness, assertiveness, and coping with anxiety. The scales all had good reliabilities.

Analysis:

The analysis used regression models with fixed effects for schools, controls for baseline outcomes and other covariates, and robust standard errors for non-normally distributed outcomes. There were too few schools to use multilevel models, but checks found "negligible" ICCs. The analysis also used multiple imputation with academic performance and religious participation serving as auxiliary predictors in the imputation model. The analysis did not examine main effects, as the authors stated that they made the decision a priori to analyze the outcomes separately for males and females.

Missing Data Methods: The analysis used multiple imputation.

Intent-to-Treat: The analysis included all participants.

Outcomes

Implementation Fidelity:

For the LST condition, the teachers delivered 90% of the lessons, and student attendance averaged 93%. The authors noted informally that program teachers would have benefitted from refresher training.

Baseline Equivalence:

Smith et al. (2004) noted condition differences for three variables (free lunch eligibility, substance use, and problem behavior) but gave no other details on the number of tests or the size of the differences. The models controlled for these three baseline measures. Vicary et al. (2004) stated that "Pre-intervention comparisons of the sample revealed no significant differences by treatment condition for any of the substance use variables or the variables used to construct risk status."

Differential Attrition:

Vicary et al. (2004) stated that completion rates for students were similar across the conditions (68% for LST versus 72% for the control group). Vicary et al. (2006) reported that there was "no differential attrition across the three conditions" but provided no details.

Posttest:

Seventh- and Eighth-Grade Results

Smith et al. (2004) presented separate results for males and females. For males, none of the 12 outcome tests (six at the seventh-grade assessment and six at the eighth-grade assessment) showed a significant difference in substance use between the LST group and the control group. For females, four of the outcomes at the end of seventh grade (alcohol use, binge drinking, marijuana use, and inhalant use) were significantly lower for the LST group than the control group, but all four differences declined to non-significance by the end of eighth grade.

Vicary et al. (2004) examined females only and presented separate results for low-risk and high-risk groups. Although the program significantly reduced substance use among both low-risk and high-risk girls at the end of grade seven, all the effects fell to non-significance by the end of grade eight. Tests for effects on the risk and protective factors showed a significant program effect at the end of grade eight only for knowledge among low-risk girls and assertiveness skills among high-risk girls. Overall, tests for moderation by risk status showed stronger treatment effects for the high-risk females than low-risk females.

Ninth-Grade Results

Vicary et al. (2006) presented separate analyses for females and males. They found no significant effects of LST relative to the control group for any of the substance use outcomes at the end of grade nine. For the risk and protective factors, they found that the LST groups did better than the control group on coping and communication for girls but found no effects for boys.

Long-Term:

Not examined.

Study 4

The intervention consisted of 15 sessions held in seventh grade and 10 booster sessions held in eighth grade, all taught by classroom teachers who had attended a one-day training. The authors noted that the program had been adopted for minority youth but without changing the "underlying prevention strategy."

Summary

Botvin et al. (2001a, 2001b) and Griffin et al. (2003) used a cluster randomized trial to examine 29 New York City public schools with 5,222 seventh-grade student participants. The schools were assigned to an LST group or a control group that received the standard curriculum. Students in the schools were followed for one year after the program end to assess self-reported substance use.

Botvin et al. (2001a, 2001b) and Griffin et al. (2003) found that, relative to the control group, the LST group reported significantly lower

• Alcohol use and polydrug use at the interim
• Tobacco use, alcohol use, and inhalant use at posttest
• Binge drinking at the one-year follow-up.

Evaluation Methodology

Design:

Recruitment: The sample included 5,222 seventh-grade students in 29 New York City public schools.

Assignment: After stratifying by the amount of cigarette use among students, the study randomized the 29 schools to either the intervention group (16 schools, 1,713-2,144 students after attrition) or control group (13 schools, 1,328-1,477 students after attrition). Control youth received the standard curriculum in place in NYC schools.

Assessment/Attrition: An interim assessment occurred at the end of grade seven, about three months after the initial intervention sessions. A posttest assessment occurred at the end of grade eight, after the booster sessions (Botvin et al., 2001a). The authors did not mention any school attrition. Of the 5,222 students, 3,621 (69%) provided data at all three assessments and comprised the analysis sample. A follow-up assessment occurred about one year after the posttest (Botvin et al., 2001b). Of the 5,222 students, 3,041 (58%) provided data at baseline, posttest, and follow-up assessments

Sample: Demographic characteristics of the sample included approximately half (47%) male and predominantly minority (61% African American, 22% Hispanic, 6% Asian, 6% White, and 5% other or mixed background). Sixty-two percent of participants were eligible for the free lunch program. Approximately half (54%) of students lived in dual-parent households, and 36% lived in mother-only households.

Measures: Students completed questionnaires on self-reported drug use. A team of data collectors of the same ethnic backgrounds as the participants administered the questionnaires. Prior to implementation, breath samples (to measure carbon monoxide levels) were collected from all participants in order to increase the validity of the self-report data. Two measures of cigarette smoking were obtained: frequency (9-item scale) and quantity (11-item scale). Two measures of alcohol consumption were obtained: frequency (9-item scale) and amount consumed per drinking occasion (6-item scale). Dichotomous measures of binge drinking were added to the posttest and follow-up assessments. Frequencies of smoking marijuana, getting "high" from marijuana, and using inhalants were also measured with a 9-item scale. Polydrug use measures were based on the responses to single drug use frequencies and examined in terms of lifetime use and current use. Also included in the analysis of this study were assessments of behavioral intentions, drug attitudes and knowledge, and social and personal competence.

Analysis: Botvin et al. (2001a) used generalized linear models ANCOVA but, more importantly, also used generalized estimating equations to adjust for clustering within schools, the units of analysis. The models adjusted for pretest scores, but the tables reported one-tailed significance tests. Botvin et al. (2001b) used only generalized estimating equations with controls for pretest scores and appeared to use two-tailed significance tests.

Missing Data Methods: The complete-case analyses examined individuals who provided pretest, posttest, and follow-up data, without imputation or FIML.

Intent to Treat: The analyses included all participants with complete data.

Outcomes

Implementation Fidelity:

The mean level of program points covered was 48%, suggesting a lower level of fidelity than found in previous research.

Baseline Equivalence

Tests using the analysis sample indicated no significant baseline differences between conditions for any of the 10 substance-use outcomes. However, tests for demographic measures found three significant condition differences for percent Black (more in the intervention condition), percent Hispanic (more in the control condition), and percent receiving free lunches (more in the intervention group). These variables were added as controls in the analyses.

Differential Attrition:

All reports examined baseline equivalence among the analysis samples. In addition, they included main effect and interaction tests for differential attrition.

At posttest, Botvin et al. (2001a) reported that main effects tests found attrition to be higher among smokers, drinkers, and marijuana users. The study also conducted interaction tests for attrition (i.e., pretest use status x condition) and reported one significant result. Baseline marijuana users in the control group were more likely to drop out than their counterparts in the intervention group.

At the one-year follow-up, Botvin et al. (2001b) reported higher attrition among males and baseline drinkers but indicated that the rate of attrition of males and drinkers did not differ across experimental conditions. They also stated that "Attrition rates were similar across both experimental and control conditions" but offered no details.

Posttest:

Interim. At three months after the 15 year-one sessions, the results from the generalized estimating equations showed significant effects on drunkenness, drinking quantity, drinking knowledge, and normative expectations for smoking and drinking.

Posttest. After the initial and the booster sessions (Botvin et al., 2001a), the results from the generalized estimating equations in Table 2 showed eight significant intervention effects on measures of smoking, drinking, inhalants, and polydrug use. All effects remained significant when using two-tailed tests rather than one-tailed tests. Further tests showed intervention effects on measures of knowledge, intentions, attitudes, perceived peer expectations, perceived adult expectations, and risk-taking. Mediation analyses indicated that prevention effects on some drug use outcomes were mediated in part by risk-taking, behavioral intentions, and peer normative expectations regarding drug use. Botvin et al. (2001b) added that the intervention significantly lowered binge drinking at the posttest (OR = .41).

High-Risk Subsample: Griffin et al. (2003) examined 21% of the original sample that was classified as high risk for substance use initiation based on having poor grades and peers who used substances (n = 802). Tests at posttest for the subsample controlled for pretest scores and used generalized estimating equations to adjust for school clustering. The text reported one-tailed probability levels but provided information to infer two-tailed significance levels. The (two-tailed) results showed that the LST group had significantly lower substance use means than the control group for smoking (d = .22), drinking (d = .22), and polydrug use (d = .21). In addition, the study reported no significant baseline differences between conditions and no differences across conditions in the determinants of attrition.

Long-Term:

At the one-year follow-up (Botvin et al., 2001b), the intervention significantly reduced binge drinking (OR = .42). There were also several significant program effects on proximal drinking variables, including drinking knowledge, pro-drinking attitudes, and peer drinking norms.

Study 5

Summary

Zollinger et al. (2003) used a quasi-experimental design to examine 16 Indiana middle schools, 12 of which self-selected to implement LST. Sixth-grade students (n = 1,598) were followed through eighth grade to assess cigarette use and attitudes and knowledge about cigarette use.

Zollinger et al. (2003) found that, relative to the control group, the LST group reported significantly lower

• Smoking prevalence
• Smoking knowledge and self-efficacy.

Evaluation Methodology

Design:

Recruitment: The sample included 15 middle schools from the Indianapolis Public School system. Students enrolled in the system from 1997-2000 (n = 27,865) were included in the study.

Assignment: The Life Skills Training curriculum was implemented in 12 of the 16 schools. Intervention schools were not randomly chosen, but the specifics surrounding their selection were not discussed in the study. Results were presented for students with no exposure to the program (26.9%), exposure during one school year (32.9%), and exposure during two school years (40.2%).

Assessments/Attrition: Baseline data were collected in 1997 on sixth-grade students. Surveys followed in 1998 on sixth- and seventh-grade students and in 1999 and 2000 on sixth-, seventh-, and eighth-grade students. The study combined two cohorts, those who participated in all of the 1997, 1998, and 1999 surveys, and those who participated in all of the 1998, 1999, and 2000 surveys. The study noted that 29% declined to participate, 2% were dropped because of partial or unserious responses, and 69% completed the surveys. The final reported analysis sample size was 1,598.

Sample: Approximately 56% of the participants were female, 59% were African American, and 31% were White.

Measures: Self-report surveys were administered by the school system. Survey administrators were instructed on guidelines and protocols for administering the survey, including issues regarding confidentiality, consistency, and logistics. Student identification numbers assigned by IPS were used to match the surveys completed by students in the sixth, seventh, and eighth grades.

Responses to tobacco use items were used to classify students as frequent smokers, current smokers, those who tried smoking cigarettes, and non-smokers. Current smokers had smoked in the past 30 days. Non-smoking students had never smoked a cigarette, not even a puff or two. All other students were classified as having tried smoking.

A self-administered survey collected data about middle school students' knowledge, attitudes, beliefs, self-efficacy, decision-making ability, and behavior toward tobacco use and related issues. 

Analysis: Responses were compared using the z -test for proportions to determine statistical significance. The analysis did not adjust for clustering within schools or control for baseline outcomes.

Missing Data Method: The complete-case analysis did not use imputation or FIML.

Intent to Treat: The analysis included participants with complete baseline and posttest data.

Outcomes

Implementation Fidelity:

Not examined.

Baseline Equivalence:

The study reported that tobacco-related behavior and attitudes of these students at baseline did not differ significantly from nonintervention schools.

Differential Attrition:

Not examined

Posttest:

The results in Table 2 showed significantly fewer current smokers among students exposed to one year or two years of the program than among students unexposed to the program. The effects were significant for both males and females and for whites who were exposed to two years of the program. There were also significant effects of the program on smoking intentions, self-efficacy, smoking knowledge, and attitudes toward smoking.

Study 6

Treatment condition students received the 15-session program. The program was revised to be more appropriate for the targeted population. Modifications included an adjusted reading level, more relevant illustrative examples, and revised situations for behavioral exercises. Treatment teachers attended a one-day training workshop.

Summary

Botvin et al. (1997) used a cluster randomized trial to examine seven junior high schools in New York City with 833 student participants. The schools were assigned to an LST group or a control group that received the standard curriculum. Students in the schools were assessed on substance use at baseline and posttest.

Botvin et al. (1997) found that, relative to the control group, the LST group reported significantly lower

• Cigarette smoking at posttest
• Drinking at posttest
• Marijuana use at posttest
• Polydrug use at posttest.

Evaluation Methodology

Design:

Recruitment: Seven junior high schools in New York City with 833 students participated in the study.

Assignment: The study randomly assigned the seven schools to the intervention or to a standard care control condition.

Assessments/Attrition: The study conducted assessments at pretest and posttest (about three months later). Of the 833 participating students at pretest, 721 (87%) completed posttest measures.

Sample Characteristics: The majority of students were girls (53%) and the mean age of the students was 12.6 years. The ethnic-racial composition of the sample was 25.8% African-American, 69.6% Hispanic, .7% White, 1.4% Asian, 1.5% Native American, and 1.0% Other. Most of the sample lived with their mother-only (37.3%) or both parents (35%). The majority of students (78.6%) qualified for free or reduced lunch.

Measures: Students completed two measures: a questionnaire and a carbon monoxide (CO) breath sample to enhance the validity of the self-reported data. Students completed the questionnaire during class and answered questions about current drug use (five behavioral measures: smoking, drinking, drinking amount, drunkenness, marijuana use; two multiple substance measures: ever use and current use) and intentions for drug use in the future. Also assessed were behavioral intention (for drug use), normative expectations, attitudes towards drug use, and social competence (decision-making, advertising influences, anxiety reduction, and communication).

Analysis: The study used analysis of covariance with adjustments for pretest outcomes and one-tailed significance tests. However, the study did not adjust for clustering within schools, the units of assignment.

Missing Data Methods: The analysis used data provided by students who completed both the pretest and posttest.

Intent to Treat: The analysis used all participants with complete data.

Outcomes

Implementation Fidelity:

Not examined.

Baseline Equivalence:

Tests found no significant differences between conditions for gender, free lunch, family structure, or "the primary behavioral outcomes." There were differences between conditions on race/ethnicity, with a lower proportion of Hispanic students and a higher proportion of African American students in the control condition compared to the treatment condition.

Differential Attrition:

Not examined.

Posttest:

In Table 1, tests for seven behavioral outcomes produced six significant effects (using the p values to define two-tailed tests). The intervention students reported lower smoking, drinking, marijuana use, and multiple substance use. The results also showed significant intervention effects on substance use intentions and normative expectations.

Long-term:

Not examined.

Study 7

The study evaluated the 15-week version of the program (without booster sessions after the first year).

Summary

Botvin et al. (2006) used a cluster randomized trial to examine 41 New York City public and parochial schools with 4,858 sixth-grade student participants. The schools were randomly assigned to an intervention group or a control group that received the standard curriculum. Students in the schools were assessed on measures of verbal and physical aggression, fighting, and delinquency at pretest and posttest.

Botvin et al. (2006) found that, relative to the control group, the intervention group reported significantly lower

• Delinquency at posttest
• Frequent fighting at posttest.

Evaluation Methodology

Design:

Recruitment: The sample consisted of 4,858 sixth-grade students from 41 NYC public and parochial schools.

Assignment: The study randomly assigned the 41 schools to the intervention condition (n = 20 schools, 2,374 students) or a control condition (n = 21 schools, 2,484 students) that received the standard health education curriculum.

Assessments/Attrition: Pretest assessment occurred in sixth grade and posttest approximately three months later, after the intervention. The authors did not report on attrition or assessment completion rates. The student sample of 4,858 refers to those completing both the pretest and posttest.

Sample: The sample was 51% boys and 49% girls; 39% African American, 33% Hispanic, 10% White, 6% Asian, 2% Native American, and 10% of other or mixed ethnicity.

Measures: Measures came from student reports and assessed verbal and physical aggression, fighting, and delinquency. Reliabilities for the scales were sufficiently high.

Analysis: The analyses used generalized estimating equations to account for the within-cluster correlation and included pretest values of each outcome, plus gender, percent black, percent Hispanic, grades, and implementation score were used as covariates.

Missing Data Method: The analysis used complete cases without imputation or FIML

Intent to Treat: The analysis used all participants with complete data.

Outcomes

Implementation Fidelity:

The mean number of program points covered across all observations was 45.5%, and the range was 21% to 86%.

Baseline Equivalence:

The two conditions did not differ at baseline on any of the violence or delinquency scales, or on gender or academic performance. The intervention group had significantly more Hispanic students (36.7%) than the control group (30.2%), while the control group had significantly more Black students (43.8%) than the intervention group (33.1%).

Differential Attrition:

Not examined.

Posttest:

For the full sample (Table 1), the intervention reduced any delinquency in the past year, frequent fighting in the past year, and frequent delinquency in the past year. Five other outcomes were not significantly different, including past month and high frequency verbal and physical aggression, as well as fighting in the past year.

Long-Term:

Not examined.

Study 8

Summary

Crowley et al. (2014) used a quasi-experimental design with propensity score weighting that examined seventh-grade students in 28 school districts in rural Iowa and Pennsylvania. Schools in the treatment districts implemented LST or one of several other programs. Nonmedical prescription opioid use served as the outcome.

Crowley et al. (2014) found that, relative to students in the control group, students in the LST group reported significantly

• Lower use of nonmedical prescription opioids.

Evaluation Methodology

Design:

Recruitment: The sample came from 28 rural communities in Iowa and Pennsylvania that met several eligibility requirements: (1) school district enrollment between 1301 and 5200 students, (2) at least 15% of families eligible for reduced-cost lunch, (3) maximum of 50% of the adult population employed at or attending a college or university, and (4) not involved in other university-affiliated, youth-focused prevention initiatives.

Assignment: After being matched on geographic location and size, one of each pair of districts was randomized to the intervention condition (n = 14 districts) and one to the control condition (n = 14 districts, 5,292 students). Families in the intervention districts were first offered a choice of implementing or not a family-based program in the sixth grade, and then schools were offered a choice of three different school-based programs in the seventh grade. Only a subset of schools chose to implement LifeSkills Training. A total of 1,166 students received LifeSkills Training either alone or in combination with the family-based program, but the authors did not report the sample sizes of the solo or combination subgroups. Because the families and schools selected their own programs, the LifeSkills Training group was not assigned randomly within the intervention condition.

To adjust for the non-random assignment within the intervention condition, the study used propensity score models to estimate the probability of receiving each of seven possible assignments. The models included 43 covariates covering four levels: individual, family, school, and research team. The covariates included student-level variables (prescription opioid use, gender, alcohol use, ever been intoxicated, level of alcohol use, inhalant use, hard drug use, tobacco use, youth substance use expectations, school attitude, problem-solving capacity, school adjustment, school attendance, refusal efficacy, refusal intentions, stress management, substance use norms, and future use), school-level variables (school uses a structured curriculum, percentage of free lunch, parent outreach, community pressure, teacher resistance, involvement of agency, school attitude towards prevention, district attitude towards prevention, and number of teachers in school), and variables related to the research team (extension reputation, team size, time for parent recruitment, schools' prevention attitudes, and success of community coalition).

Attrition: Assessments occurred in sixth grade (pretest) and at the end of each year through twelfth grade. No districts dropped from the sample, and Figure 1 shows that (based on the average cluster size) the completion rate in grade 12 was 76% for the control group and 68% for all intervention groups combined. In addition, the supplementary materials noted that "average missingness of an item was about 10.0%."

Sample:

About 50% of the students were female, with a mean age of 11.8 years overall. Students in the intervention and control groups had an average family income of $50,174 and $52,704, respectively. Around half of the sample came from dual-parent households.

Measures:

The single outcome was measured by asking each participant whether they had ever used prescription opioids (Vicodin, Codeine, Percocet, or OxyContin) for nonmedical purposes.

Analysis:

After estimating the participants' propensity to receive different programs, the study transformed the propensity scores into inverse probability weights. The weights balanced the group comparisons on the propensity scores and, ideally, on the covariates used to estimate the propensity scores. Then, the analysis used multilevel logistic models to account for clustering.

Missing Data Method: The supplementary materials reported that multiple imputation accounted for any missing data and "allowed for complete data analysis for both the propensity models and the outcome analysis." It is uncertain if the imputation was used for all participants or participants who completed the follow-up assessments but had some missing data.

Intent-to-Treat: The analysis appeared to include either all participants or all participants with complete follow-up data.

Outcomes

Implementation Fidelity:

The authors stated only that "program adherence was high."

Baseline Equivalence:

The study compared the standardized mean differences (SMD) between the control and treatment groups without and with the propensity score weighting. The supplementary materials noted that "weighting generally lowered or maintained the SMDs of each confounder and no confounders had an absolute SMD above .2 when weighted, which is generally considered to be small."

Differential Attrition:

The supplementary materials noted only that "Previous evaluations of PROSPER explored the study's missingness and found no evidence of threats to internal validity from differential sample attrition at grade 12." Note also that the propensity score weighting would adjust for attrition-based differences between conditions.

Posttest and Long-Term:

The propensity score weighting results in Table 2 show significantly lower misuse of prescription opioids for the LifeSkills-only condition compared to the control condition across all assessments from the sixth to twelfth grade.

Study 9

Students in the combined LST and SFP 10-14 group received both curricula in seventh grade, as well as nine booster sessions in eighth grade (four for SFP 10-14 and five for LST). Students in the LST-only group received the LST curriculum in seventh grade as well as five booster sessions in eighth grade.

Summary

Spoth et al. (2002, 2006, 2014), Spoth, Randall et al. (2008), Spoth, Trudeau et al. (2008) and Trudeau et al. (2003) found that, relative to the control group, the LST group reported significantly

Spoth et al. (2002, 2006, 2014), Spoth, Randall et al. (2008), Spoth, Trudeau et al. (2008) and Trudeau et al. (2003) used a cluster randomized trial that assigned 36 middle schools with 1,664 seventh-grade students to three conditions: LST, LST plus Strengthening Families 10-14, or a control group. Assessments of substance use continued through age 22.

• Lower substance use initiation at posttest
• Lower cigarette initiation at the 3.5-year follow-up
• Slower growth rates in cigarette initiation and drunkenness at the 3.5-year follow-up.

Evaluation Methodology

Design:

Recruitment: Participants in the study were seventh graders enrolled in 36 randomly selected rural schools in 22 contiguous counties in a Midwestern state. Criteria for the selection of the initial pool of schools were: 20% or more of households in the school district within 185% of the federal poverty level; community size (school district enrollment under 1,200); and all middle-school grades (6-8) taught at one location. A total of 1,673 students completed the pretest, but 9-18 were dropped because they moved from a school in one condition to a school in another condition. The study thus used baseline samples of 1,655-1,664. Trudeau et al. (2003) reported that about 90% of the students completed the pretest.

Assignment: A randomized block design guided the assignment of the 36 schools to one of three experimental conditions: 1) a combined Life Skills Training (LST) and Strengthening Families Program 10-14 (SFP 10-14) group (n = 12 schools, 549 students), 2) an LST only group (n = 12 schools, 621 students), and 3) a no-treatment control group (n = 12 schools, 494 students). Trudeau et al. (2003) examined only two conditions (LST and control, n = 1,115).

Assessments/Attrition: Assessments occurred at baseline, six months later following the first-year intervention, and then yearly through the 12th grade. When counting the second-year booster sessions as part of the program, the first assessment occurred at an interim point, the grade eight assessment occurred at posttest, the grade 11 assessment occurred 2.5 years after the program end, and the grade 12 assessment occurred 3.5 after the program end.

• Spoth et al. (2003) reported that, of the 1,664 students who completed the pretest, 1,563 (94%) completed the interim assessment, and 1,372 (82%) completed the posttest.
• Trudeau et al. (2003) examined the same three assessments but only two conditions and included 847 students (76%) in 24 schools who had no missing data and had not changed schools.
• Spoth et al. (2006) examined the results from grade 11 and grade 12 assessments, 4.5-5.5 years after baseline and 2.5-3.5 years after the program end. They examined a randomly selected subsample of participants who completed an in-home assessment. A total of 691 families completed the pretest (67%), 12 were dropped because they moved to a school in another condition, 588 completed the grade 11 assessment (85% of those with a pretest), and 597 completed the grade 11 assessment (86% of those with a pretest).
• Spoth, Trudeau et al. (2008) also examined the results for grade 11 and grade 12 assessments, 4.5-5.5 years after baseline and 2.5-3.5 years after the program end. They used the full sample rather than the home-assessment sample and reported sample sizes of 1,443 in grade 11 (87%) and 1,212 in grade 12 (73%).
• Spoth, Randall et al. (2008) reported sample sizes of 1,472 in grade 11 (88%) and 1,237 in grade 12 (74%).
• Spoth et al. (2014) presented results for 11 assessments overall, with four of the assessments coming after high school, at ages 19, 20, 21, and 22. The authors noted that their study eliminated 18 students "who changed conditions (i.e., moved from a school district in one condition into one in a different condition)." According to Figure 1, about 72% of the baseline students provided data at age 22. Of students eligible for the young adult follow-ups -  those who participated in the 11th or 12th-grade assessments - 84% provided data at age 22.

Sample: For the baseline sample, slightly over half (53%) were male, and the majority of participants (96%) were Caucasian. About 77% were living with biological parents, and 21% qualified for free or reduced-price lunch.

Measures: The outcome measures came from self-reports. The behavioral measures relied on single questions about substance use, and the scales for risk and protective factors had good reliabilities.

• Spoth et al. (2002) examined lifetime use of alcohol, cigarettes, and marijuana, with corrections for inconsistent reports of lifetime substance (e.g., reporting never used after reporting used in an earlier wave). Prior to implementation, breath samples (to measure carbon monoxide levels) were collected from all participants in order to increase the validity of the self-report data collected.
• Trudeau et al. (2003) also examined substance use expectancies, refusal intentions, and substance initiation. Three dichotomous measures were summed to create an initiation scale (after correcting for inconsistencies across waves).
• Spoth et al. (2006) examined 12-month and lifetime methamphetamine use.
• Spoth, Trudeau et al. (2008) examined lifetime prescription drug use (i.e., drugs or medications that were prescribed by a doctor to someone else).
• Spoth, Randall, et al. (2008) examined multiple measures of substance use initiation as well as more serious use of substances (e.g., frequency, polydrug use).
• Spoth et al. (2014) examined measures of substance misuse, including drunkenness, alcohol-related problems, cigarette use, and illicit substance use.

Analysis: Spoth et al. (2002) used multilevel (mixed) models with restricted maximum likelihood estimation and random effects to adjust for clustering. The models for lifetime substance use controlled for baseline lifetime use to measure initiation since baseline. In addition, new-user analyses for specific substances were conducted at the school level, based on the proportions of new users in each school from the end of the program in year 1 through the end of the booster sessions in year 2. The tests used one-tailed probabilities.

Trudeau et al. (2003) used latent growth and structural equation models with restricted maximum likelihood estimation and adjustments for within-school clustering. The growth models incorporated baseline outcomes in the intercept and slope estimates.

Spoth et al. (2006) and Spoth, Trudeau et al. (2008) did not adjust for randomization of schools. They used Fisher's exact test due to the small number of participants who used methamphetamines or misused prescription drugs and the lack of within-school dependence in the outcomes.

Spoth, Randall et al. (2008) used mixed models with FIML and controls for pretest scores but conducted school-level analyses for some outcomes. They stated, "To ensure that measures exhibited reasonably normal distributions, the dichotomous initiation outcome measures . . . were aggregated to the school level prior to analysis." They reported significance levels for one-tailed tests. In order to "reduce the likelihood of biased parameter estimates," only adolescents with complete information on the study variables for at least three of the seven assessments were included in the analyses.

Spoth et al. (2014) used latent growth models with adjustments for clustering within schools and FIML estimation to include all available data. Rather than examining condition differences in outcomes during young adulthood, the models estimated the indirect effects of the interventions on substance use outcomes via adolescent initiation. That is, the intervention affected adolescent initiation of substance use, which in turn affected young adult substance use.

Missing Data Method: Most reports used mixed models and structural equation models to include all participants who had baseline data and remained at the same school. Spoth et al. (2006) and Spoth, Trudeau et al. (2008) used complete case analyses. Spoth, Randall et al. (2008) used FIML but included students with pretest, posttest, and at least one additional wave of data rather than all students with baseline data.

Intent to Treat: Most reports used FIML estimation to include nearly all students with baseline data, but they dropped a small number who moved from schools in one condition to schools in another condition, and they did not follow students moving to schools outside the study sample. Others used complete case analysis but also dropped students. Spoth, Randall et al. (2008) dropped students who did not complete at least three surveys.

Outcomes

Implementation Fidelity:

Spoth et al. (2002) reported in Table 1 that 94% received the main intervention, and 75% received the booster sessions.

Baseline Equivalence:

Spoth et al. (2002) tested for baseline equivalence on sociodemographic and outcome measures; the only significant difference was that the control group contained more dual-parent families than the two intervention groups. This variable was included as a control in the outcome analyses.

Differential Attrition:

Overall, the study used strong tests for differential attrition and found little evidence of attrition bias.

• Spoth et al. (2002) reported no significant dropout by condition interactions from pretest to posttest or from posttest to follow-up for any outcome or sociodemographic variable. In addition, the overall attrition rate combined with the difference in attrition rates by condition met the WWC's optimistic and cautious standards.
• Trudeau et al. (2003) reported that students who dropped out were at higher risk of substance use but that "no differences were found between the intervention and control conditions on dropout status for any of the variables."
• Spoth et al. (2006) and Spoth, Trudeau et al. (2008) reported the results of condition-by-attrition interaction tests. Although high-risk students were more likely to drop out than lower-risk students, the dropout rates were similar across conditions.
• Spoth, Randall et al. (2008) found no significant condition-by-dropout status interactions on the outcome variables for each wave (assessed from each wave to each succeeding wave).
• Spoth et al. (2014) reported that those who remained in the study had a lower level of substance use at pretest than those who dropped out. However, they found one instance of differential attrition. There was a lower rate of attrition among control group participants from dual biological parent families at the 19-year-old assessment point. "No other significant pretest or differential attrition effects were found" (p. 951).

Posttest:

The summary below adjusts for the one-tailed tests reported by the authors in many of the analyses.

Over the period from baseline to posttest, the results in Spoth et al. (2002) showed that the LST-only group was marginally significantly lower on the Substance Initiation Index than the control group. Tests for new users found that the LST-only group had a significantly lower marijuana initiation rate than the control group but did not differ significantly from the control group on alcohol or cigarette initiation.

Over the same period but for only two conditions, Trudeau et al. (2003) found significant slope-by-condition effects on refusal intentions and substance initiation. The results indicated a slower increase across assessments for the intervention group than the control group on substance initiation and a slower decrease in refusal intentions. Additional moderation tests found stronger intervention effects for girls than boys.

Long-Term

2.5-3.5 Year Follow-Up (Grades 11 and 12):

• Spoth et al. (2006) found a significant difference in lifetime methamphetamine use between LST-only and control conditions in grade 12 but not in grade 11.
• Spoth, Trudeau et al. (2008) found no significant difference in prescription drug misuse between LST-only and control conditions at either the 11th or 12th-grade follow-ups.
• After adjusting for two-tailed tests, Spoth, Randall et al. (2008) found significant effects on the substance initiation index and cigarette initiation at the end of grade 12 and on drunkenness initiation and cigarette initiation across all waves. Tests for moderation found significantly stronger effects on all outcomes for high-risk students than for low-risk students.

7.5 Year Follow-Up (Ages 19, 20, 21, and 22)

Spoth et al. (2014) used latent growth models to examine the indirect effects of LST versus the control group through adolescent initiation of substance use. The indirect effects produced lower levels in the LST group than the control group on four outcomes at ages 19-22: drunkenness frequency, alcohol-related problems during the past year, cigarette use frequency, and illicit drug use frequency. Only one of the outcomes, alcohol-related problems, showed an indirect effect of LST on the slope or rate of change, suggesting that the LST advantage declined over the young adult years. With controls for the indirect effects, LST had little direct influence on the outcomes. The results support arguments that the program influences adult substance use by delaying adolescent imitation, which then moderates substance use in young adulthood.

Study 10

Following the pre-test, students in the experimental condition participated in a 10-session smoking prevention program. Although sessions were conducted weekly, the time interval between the beginning and the end of the program was 12 weeks due to school holidays. The program was administered by an outside specialist and utilized a combination of group discussion and special skills training.

Summary

Botvin & Eng (1980) and Botvin et al. (1980) used a cluster randomized trial to examine 281 students in grades 8-10 who were attending two New York City schools. The two schools were randomly assigned to an LST or control group. Student participants were assessed on smoking, smoking knowledge, and social relations at baseline, posttest, and three-month follow-up.

Botvin & Eng (1980) and Botvin et al. (1980) found that, relative to the control group, the LST group reported significantly

• Fewer smokers
• Higher smoking knowledge.

Evaluation Methodology

Design:

Recruitment: A sample of 281 students was drawn in an unspecified manner from a population of eighth-, ninth-, and tenth-grade science and health education students in two suburban New York City schools. Both schools were generally comparable with respect to socioeconomic status and the prevalence of cigarette smoking and were predominantly middle class.

Assignment: The two schools were randomly assigned to either the experimental (n = 121 students) or the control (n = 160) students condition. Potential confounding exists with the assignment of only one school to each condition. Unique characteristics of one or both, rather than the intervention, may produce different outcomes.

Assessments/Attrition: All participants completed a pretest, a posttest, and a three-month follow-up.  Data for the three-month follow-up were collected for roughly 77% of the posttest sample.

Sample: No specific information regarding the gender or racial composition of the sample was provided.

Measures: The self-reported measures included smoking status, smoking knowledge, psychosocial knowledge, locus of control, self-image, social anxiety, influenceability, and the need for group acceptance. The authors stated that "test-retest reliability for the questionnaire was .65 over a three-month interval."

Analysis: All pretest smokers were eliminated from the analysis of smoking status, permitting the comparison of the experimental and control groups in terms of the number of new smokers. The individual-level analysis used chi-square tests or ANOVA and did not adjust for clustering within schools, the unit of assignment.

Missing Data Method: The analysis used complete cases.

Intent to Treat: The analysis dropped non-smokers in examining the smoking outcome, but otherwise used all participants with complete data.

Outcomes

Implementation Fidelity:

Not examined.

Baseline Equivalence:

The study presented no tests for individuals. It noted only that both schools had approximately the same baseline smoking rate (31% and 29%).

Differential Attrition:

The study presented no tests. It noted that only 80% of the experimental group and 74% of the control group completed the follow-up.

Posttest:

At the posttest, Botvin et al. (1980) found that significantly fewer students in the intervention group began smoking during the course of the study when compared to students in the control group. The benefits were strongest among eighth graders and weakest among the tenth graders. In addition, the intervention group had a significantly greater increase in smoking knowledge than the control group, and there was a significant two-way interaction between sex and treatment condition for social anxiety, with the males in the experimental condition showing the greatest decrease in social anxiety.

At the three-month follow-up for the subsample of pretest nonsmokers, Botvin & Eng (1980) found significantly fewer new smokers in the intervention group than the control group. For risk and protective factors, the results for the intervention group relative to the control group showed significantly greater decreases in the need for group acceptance, social anxiety and significantly greater increases in smoking knowledge Among tenth graders, as with the eighth graders, there was a significant interaction between sex and treatment condition for social anxiety, with the males showing the greatest decrease.

Study 11

Summary

Aviles (2019) used a quasi-experimental design with propensity score matching to examine sixth-grade students in 13 schools that had implemented LST and 10 schools that had not. Self-reported substance use served as the outcome.

Aviles (2019) found that, relative to the control group, the LST group had significantly

• Fewer smokers.

Evaluation Methodology

Design:

Recruitment: The study examined sixth-grade students in 13 schools located in the mid-Atlantic region that had implemented LST.

Assignment: The quasi-experimental design matched the 13 LST schools (n = 1,052 students) to 10 control schools not utilizing the treatment (n = 494 students). The propensity scores used to match the schools resulted from the following predictors: gender distribution, socioeconomic status (i.e., percentage of low income students in the school as reflected by the percentage of youth receiving free or reduced-price lunch), locality (e.g., urban, suburban, or rural), aggregated community risk factor scale scores for the school (e.g., perceived availability of drugs), aggregated family risk factor scale scores for the school (e.g., poor family management, parental attitudes favorable to substance use), and aggregated school-based risk factor scale scores for the school (e.g., poor academic performance).

Assessments/Attrition: The cross-sectional design did not include baseline data. It instead measured the outcomes in the fall of sixth grade, presumably while the program was ongoing.

Sample:

The student sample was 89% Caucasian with 52% females.

Measures:

Data came from the Pennsylvania Youth Survey, administered in the fall every two years to students in grade six. The survey included 11 self-reported substance use measures and 12 measures of risk and protective factors. To maintain reliability of the substance use measures, students who provided inconsistent answers or reported using a fake drug were treated as having missing data. Reliabilities for the scales were as low as .42 and .61.

Analysis:

The analysis used two-level random effects models with linear, Poisson, inflated Poisson, binomial, and negative binomial link functions. Low ICC values of .002 to .014 indicated little clustering.

Missing Data Method: The analysis used Full Information Maximum Likelihood estimation.

Intent-to-Treat: All participants were included in the analysis except those with missing data on the gender, race, or age covariates.

Outcomes

Implementation Fidelity:

Not examined.

Baseline Equivalence:

Not examined.

Differential Attrition:

Not examined.

Posttest:

Of the 11 substance use measures, two showed a significant effect. The program partly impacted 30-day cigarette use by significantly reducing whether a person smoked but not the amount a person smoked. The program also had a significant iatrogenic effect on lifetime prescription drug use. There were no significant effects on the 12 risk and protective measures. Tests for moderation suggested stronger effects for boys than girls.

Long-Term:

Not examined.

Study 12

Summary

Velasco et al. (2017) used a quasi-experimental design that examined 55 high schools in Italy that had already implemented or not implemented the LST program. The LST schools had 1,350 students, while the control schools with similar characteristics had 1,014 students. Student assessments of substance use came at baseline and posttest.

Velasco et al. (2017) found that, relative to the control group, the LST group reported significantly

• Lower rates of smoking initiation at posttest
• Greater substance use knowledge at posttest
• Less positive attitudes toward smoking at posttest
• Lower normative expectations about smoking and drinking at posttest.

Evaluation Methodology

Design:

Recruitment: Velasco et al. (2017) conducted a quasi-experimental design with crude matching that involved 55 schools in the Lombardy region of Italy. Data were collected before (baseline) and after (posttest) implementation, and again one- and two-years following program completion.

Assignment: All students who participated in the study were enrolled in one of 55 high schools distributed across Italy (n=3,048 students, 138 classrooms). Thirty-one intervention schools (n = 1,350 students) were randomly selected from the set of schools participating in a larger implementation of the program in Italy after being stratified by geographic area. For the comparison group, schools with similar characteristics (e.g., municipality size, school size, and students' demographics) as the intervention schools were selected, resulting in twenty-four control schools (n=1,014 students). Within each treatment and control school, 3 classes (if available) were randomly selected to participate in the study. During the first year of middle school, students in the intervention schools received 15 sessions of the treatment followed by 10 booster sessions implemented during the second year of middle school and nine during the third year. All comparison schools were not involved in other drug prevention interventions.

Attrition: A total of 1,586 students from 106 classes and 48 schools completed a posttest after the final year of program implementation (i.e., at the two-year follow-up after implementation of the 19 booster sessions). The attrition rates at the final assessment were 48% for students, 23% for classrooms, and 13% for schools.

Sample:

Students were 51% female for the intervention and comparison and students had a mean age of 11 years. The majority of students in the intervention (92%) and comparison (90%) were Italian.

Measures:

Three forms of substance use were assessed, smoking, alcohol use, and drunkenness, each of which were measured using a 9-point frequency response scale anchored by 1 (never) to 9 (more than once a day).

Thirteen measures assessing risk and protective factors were collected, including:

• Life skills - assertiveness (sample α = .73); social skills (sample α = .68); decision-making (sample α = .80); advertising resistance skills (sample α = .69); and anxiety reduction skills (sample α = .67).
• Psychosocial outcomes - students' distress (sample α = .81); well-being (sample α = .81); and risk-taking (sample α = .68).
• Beliefs about substance use - knowledge of the physiological effects of substance use (no alpha reported); beliefs about myths/misconceptions about drugs (no alpha reported); attitudes about smoking (sample α = .78) and alcohol (sample α = .74); normative expectations related to tobacco and alcohol use in terms of perceived prevalence of drug use among adults (no sample alpha).

Analysis:

Analyses reported here examined the effects of the intervention at the 2-year follow-up (i.e., after the booster sessions). While the text states that covariates included pretest scores, gender and age, Table 4 (which reports results after the two-year follow up when the full program was completed) states that the covariates included just gender and age. Thus, it was not clear whether analyses controlled for baseline pretests. Analyses used GEE to account for the clustering of students within schools. Scores on the behavioral measures (smoking, alcohol use, and drunkenness) were recorded to identify students who initiated alcohol or tobacco use (never vs. more than once in lifetime), and those who transitioned from less than weekly use to weekly use in order to verify the effectiveness of the program in preventing the initiation or the regular use of these substances.

Missing Data Method: Students with missing data at the posttest and 2-year follow up were dropped from the analysis.

Intent-to-Treat: It appeared that all students with complete were analyzed according to the condition in which they were assigned.

Outcomes

Implementation Fidelity:

Not reported.

Baseline Equivalence:

There were no significant baseline differences between groups in terms of school characteristics (municipality size and school size) and school-level demographics (gender, age, and nationality). There were also no school-level baseline differences in substance use (tobacco, alcohol or drunkenness). Baseline equivalence tests, however, were not conducted at the classroom- or student-level.

Differential Attrition:

Differential attrition tests (i.e., assessing the baseline and demographic differences between completers and attritors) were not conducted. There were no significant differences in attrition found between groups with regards to baseline outcomes (i.e., well-being or substance use), but these attrition-by-condition tests did not assess demographic variables.

Posttest:

At the two-year follow-up (when the program, including all 19 booster sessions, was completed), results showed that compared to control, treatment students self-reported:

• Lower rates of initiating smoking (behavioral outcome)
• Higher rates of skills awareness and anxiety reduction skills (R&P factor)
• Greater knowledge of the physiological effects of substance use (R&P factor)
• More accurate believes and fewer misconceptions about drugs (R&P factor)
• Less positive attitudes toward smoking (R&P factor)
• Lower normative expectations about adults' smoking and drinking (R&P factors).

Out of the 13 R&P measures, however there was one negative effect (i.e., at the posttest, lower assertive skills were reported in the treatment group compared to the control group).

Long-Term:

Not conducted.