A delivery system that attempts to foster implementation of evidence-based youth and family interventions, complete with ongoing needs assessments, monitoring of implementation quality and partnership functions, and evaluation of intervention outcomes.
Blueprints: Promising
Greg Pliler
PROSPER Program Manager
Iowa State University
Email: gpliler@iastate.edu
Website: http://www.prosper.ppsi.iastate.edu/
Richard Spoth
Partnerships in Prevention Science Institute, Iowa State University
Once formed, the local team is tasked to select evidenced-based, universal-level family-focused and school-based programs to implement with middle school youth and their families in the local school district.
(Spoth et al., 2007; Spoth & Greenberg, 2005): PROSPER is a practitioner-scientist partnership model that evolved out of a series of partnership-based prevention projects grounded in the Land Grant University-based Extension system and the elementary/secondary public school system. As a delivery system rather than substantive program, PROSPER attempts to foster implementation of evidence-based youth and family interventions, complete with ongoing needs assessments, monitoring of implementation quality and partnership functions, and evaluation of intervention outcomes. The program is best characterized by a school, community, and university partnership. The partnership includes (1) state-level university researchers and Extension-based program directors, (2) a prevention coordinator team typically based in the Cooperative Extension System (CES), and (3) local community strategic teams, consisting of a Cooperative Extension System team leader, a representative from the public elementary/secondary school systems who serves as a co-leader, representatives of local human service agencies and other relevant service providers, and other community stakeholders, such as youths and parents. As PROSPER teams develop, they should involve other stakeholders who can positively influence program recruitment, program implementation, and sustainability (such as individuals from various church groups, parent groups, businesses, law enforcement agencies, and/or the media). The local strategic teams receive technical support from the university-level and CES prevention coordinator team members, who attend the local team meetings. This technical assistance is proactive, meaning contact is made with local team members frequently (weekly or biweekly) in order to actively engage in collaborative problem solving.
Once formed, the local team is tasked to select evidenced-based, universal-level family-focused and school-based programs to implement with middle school youth and their families in the local school district. In the implementations described below, the local teams chose to implement the Strengthening Families Program 10-14, as the family-focused program, and chose LifeSkills Training, Project ALERT, or All Stars for the school-based program. The Strengthening Families Program: For Parents and Youth 10-14 is a program designed to enhance a variety of parenting skills, such as nurturing, limit setting, and communication. It also seeks to enhance youths' prosocial and peer resistance skills. The program is delivered in seven, two-hour sessions, in which parents and youths are seen separately in the first hour and then together in the second hour.
The school-based programs are implemented during class periods and the three programs from which local teams choose are all geared towards substance abuse prevention. LifeSkills Training (LST) consists of 15 lessons, guided by social learning theory, that promote substance abuse awareness and peer-resistance skill building. Project ALERT is an 11-session program grounded in the social influence model of prevention. It attempts to (1) build resistance skills, (2) teach students to identify peer pressures and critically examine messages from peers, the media, parents, and others that might pressure them, and (3) change children's beliefs about substance use norms and the social, emotional, and physical consequences of substance use. The All Stars program's 13 sessions are grounded in social learning theory and problem behavior theory, and target violent behaviors in addition to substance use. The program attempts to (a) influence perceptions about substance use and violence, (b) increase the accuracy of students' beliefs about peer norms of the use of violence, drugs, and alcohol, (c) have students commit to avoid these types of behavior, and (d) foster bonding to school. It should be noted that PROSPER teams were also given a choice of three family-based programs, but all teams and all sites chose the Strengthening Families Program: For Parents and Youth 10-14.
Primary Evidence Base for Certification
Study 1
(Spoth et al., 2007):
The following substance use outcomes significantly improved at 3-year follow-up (10th grade) or over time from the baseline to the 3-year follow-up (Spoth et al., 2011):
The following substance use outcomes significantly improved at the 4-year follow-up (11th grade), 5-year follow-up (12th grade), or over time from the baseline to the 5-year follow-up (Spoth, Redmond et al., 2013):
Compared to the control group, intervention participants significantly improved on these 5-year follow-up outcomes (Spoth, Trudeau et al., 2013):
The intervention group improved on the following outcome at each of the five follow-ups (1-year, 2-year, 3-year, 4-year, and 5-year; Spoth et al., 2015):
The following protective factors showed significant improvement at posttest, the 1-year, or 2-year follow-up (Redmond et al., 2009):
(Spoth et al., 2017): At the age-19 follow-up, the intervention group, relative to the control group, reported significantly lower:
Primary Evidence Base for Certification
Blueprints has reviewed one study of PROSPER and it meets Blueprints evidentiary standards (specificity, evaluation quality, impact, dissemination readiness). The study was done by the developer.
Study 1
Spoth et al. (2007, 2015), Redmond et al. (2009) and six additional studies randomized 28 schools to treatment and control conditions in a cohort sequential design which began with 12,022 sixth graders at pretest. Youth self-report data was collected at pretest, the end of 6th grade and annually through the end of 12th grade, and finally when youth were age 19 (7.5 years after baseline and 6 years after program end). Measures included drug use, other problem behaviors, and family relations.
Study 1
Redmond, C., Spoth, R. L., Shin, C., Schainker, L. M., Greenberg, M. T., & Feinberg, M. (2009). Long-term protective factor outcomes of evidence-based interventions implemented by community teams through a community-university partnership. Journal of Primary Prevention, 30, 513-530.
Spoth, R. L., Trudeau, L. S., Redmond, C. R., Shin, C., Greenberg, M. T., Feinberg, M. E., & Hyun, G. (2015). PROSPER partnership delivery system: Effects on adolescent conduct problem behavior outcomes through 6.5 years past baseline. Journal of Adolescence, 45, 44-55.
Spoth, R., Redmond, C., Shin, C., Greenberg, M., Clair, S., & Feinberg, M. (2007). Substance-use outcomes at 18 months past baseline: The PROSPER community-university partnership trial. American Journal of Preventive Medicine, 32(5), 395-402.
Individual: Favorable attitudes towards drug use*
Peer: Interaction with antisocial peers*, Peer rewards for antisocial behavior
Family: Poor family management*
Individual: Perceived risk of drug use*, Problem solving skills*, Prosocial involvement, Refusal skills
Peer: Interaction with prosocial peers*
Family: Attachment to parents*, Opportunities for prosocial involvement with parents*, Parent social support*
*
Risk/Protective Factor was significantly impacted by the program
See also: PROSPER Logic Model (PDF)
Sample demographics including race, ethnicity, and gender for Blueprints-certified studies:
The sample for Study 1 (Redmond et al., 2009) was mostly White (85%), followed by Hispanic/Latino (5%) and African American (3%); roughly half of the students were female (51%).
There are three segments of training that are introduced over time as teams organize and become focused on program selection and implementation.
Currently, there is no train the trainers' model. However, the PROSPER Network will work with states to bring new staff on-board without going through the three units in a formal setting. For example, once a state has begun implementing the model and established the required infrastructure, the State Coordinator can work with Network staff (virtually) to train new sites.
Program Benefits (per individual):
$271
Program Costs (per individual):
$359
Net Present Value (Benefits minus Costs, per individual):
($87)
Measured Risk (odds of a positive Net Present Value):
39%
Source: Washington State Institute for Public Policy
All benefit-cost ratios are the most recent estimates published by The Washington State Institute for Public Policy for Blueprint programs implemented in Washington State. These ratios are based on a) meta-analysis estimates of effect size and b) monetized benefits and calculated costs for programs as delivered in the State of Washington. Caution is recommended in applying these estimates of the benefit-cost ratio to any other state or local area. They are provided as an illustration of the benefit-cost ratio found in one specific state. When feasible, local costs and monetized benefits should be used to calculate expected local benefit-cost ratios. The formula for this calculation can be found on the WSIPP website.
Initial training for PROSPER includes three units of training priced at $13,500 plus trainer travel expense as follows:
The costs of the curriculum are included in the initial training costs. (Note: Costs for family-focused and school-based program curricula are not included.)
There are no additional licensing costs.
Development of an implementation plan for 2-4 communities costs approximately $4,960. These costs cover Network technical assistance and support provided to state extension systems related to identification of community sites and staff that will optimize the likelihood of successful model implementation. The plan includes detailed tasks and timeframes along with the necessary information and tools to complete the tasks before the Unit 1 training.
Costs will vary depending on the degree of staff turnover.
Key Staffing Roles
(Note: Costs for program observations are not included.)
None.
No information is available
This example assumes a state is implementing the PROSPER delivery system with three community sites. The costs cover the PROSPER infrastructure and do not include programming.
| Implementation Plan for 3 communities | $4,960.00 |
| Training - Unit 1 | $5,500.00 |
| Training - Unit 2 | $4,000.00 |
| Trainer Travel - 2 trips | $3,000.00 |
| Team Operation - $1,000 x 3 communities | $3,000.00 |
| State Partnership Director - .20 FTE | $13,000.00 |
| Prevention Coordinators - .15 FTE per community | $32,000.00 |
| Community Team Leaders - $12,000 X 3 communities | $36,000.00 |
| Evaluator - .10 FTE | $7,000.00 |
| PROSPER Network Management | $15,000.00 |
| PROSPER National Network Team T/A | $15,000.00 |
| Access to PROSPER Data System | $2,500.00 |
| PROSPER National Network Management of Data Collection | $4,000.00 |
| PROSPER National Network Data Analysis and Reporting | $4,000.00 |
| Total One Year Cost | $148,960.00 |
With three community sites, costs average to approximately $50,000 per community for year one. It is not possible to estimate a cost per individual served in the first year because communities focus on developing the PROSPER delivery system and they don't fully implement selected programs during that year. With the scale of the above example, the PROSPER developers estimate 1,130 participants per community would be served through the year following the final (Unit 3) training session, with and average annual estimtated cost per participant of $123.
PROSPER was designed as a delivery system that could be built into and help direct collaborative work between Cooperative Extension systems and public school systems. Implementation of PROSPER is typically funded through a combination of in-kind support in the form of staffing from state and county cooperative extension systems and local community fundraising.
PROSPER helps to ensure that public funds supporting Cooperative Extension systems and local school districts are used effectively by helping to direct the work of staff in these systems toward sustained high quality implementation of evidence-based programs.
States that have established state funding streams dedicated to supporting prevention or evidence-based programs may have administrative or quality dollars that can help support the PROSPER statewide infrastructure.
Formula Funds: Key formula grants focused on education and prevention of juvenile delinquency and substance abuse may be able to support staffing for the infrastructure or the implementation of selected evidence-based programs.
Discretionary Grants: Federal research grants from the National Institutes of Health (NIH) and the Center for Disease Control and Prevention (CDC) have largely funded the initial development of the PROSPER system. This funding has been supplemented by the Annie E. Casey Foundation. Discretionary grants from the Department of Health and Human Services, the Department of Education, or the Department of Justice that support the delivery of evidence-based programs to promote social emotional development and prevent substance abuse and juvenile delinquency may also be relevant. Examples include: School Climate Tranformation Grants (ED); Project Prevent (ED); and Investing in Innovation (ED).
PROSPER state teams typically work with their university development officer or university foundation to raise dollars from foundations and individuals.
At the local community level, teams typically raise a range of in-kind donations and small grants and donations from local foundations, civic organizations, and businesses.
Richard SpothPartnerships in Prevention Science Institute, Iowa State University2625 North Loop Drive, Suite 2400ISU Research Park, Building 2Ames, IA 50010U.S.A.515-294-5383rlspoth@iastate.edu http://www.prosper.ppsi.iastate.edu/
A delivery system that attempts to foster implementation of evidence-based youth and family interventions, complete with ongoing needs assessments, monitoring of implementation quality and partnership functions, and evaluation of intervention outcomes.
Research findings are based on implementation in general population public school children in the sixth and seventh grades in rural towns and small cities in Iowa and Pennsylvania. The first program year also targeted parents of these children for a family-focused intervention program.
Sample demographics including race, ethnicity, and gender for Blueprints-certified studies:
The sample for Study 1 (Redmond et al., 2009) was mostly White (85%), followed by Hispanic/Latino (5%) and African American (3%); roughly half of the students were female (51%).
Individual: Favorable attitudes towards drug use*
Peer: Interaction with antisocial peers*, Peer rewards for antisocial behavior
Family: Poor family management*
Individual: Perceived risk of drug use*, Problem solving skills*, Prosocial involvement, Refusal skills
Peer: Interaction with prosocial peers*
Family: Attachment to parents*, Opportunities for prosocial involvement with parents*, Parent social support*
*Risk/Protective Factor was significantly impacted by the program
Once formed, the local team is tasked to select evidenced-based, universal-level family-focused and school-based programs to implement with middle school youth and their families in the local school district.
Spoth & Greenberg, 2005: While a number of the specific programs delivered through PROSPER are based on social learning theory, the fundamental component of the PROSPER model is the practitioner-scientist collaboration. There are a number of theoretical frameworks upon which to build a partnership-focused program such as PROSPER. Such theoretical models focus on organizational learning, social organizations, and health services delivery. PROSPER seeks to expand the use of partnership-based interventions, and primarily relies on Rogers' diffusion of innovation theory to discuss how best to scale-up the partnerships between practitioners and scientists, as these partnerships are of fundamental importance to the PROSPER model. The diffusion of innovation theory identifies four main elements of importance: innovation, communication channels, time, and social systems. To focus on and develop the social system (defined as "a set of interrelated units engaged in joint problem-solving to accomplish a common goal about youth competency-building or prevention"), there must be patterned relationships and communications as well as opinion leaders and change agents. To partner effectively and deliver evidence-based interventions, "external resource" agents must link the state and the community with "internal capacity" agents in public schools. Finally, PROSPER emphasizes comprehensiveness of services - services which focus on more than one type of youth problem and target competency-building and positive youth development.
Primary Evidence Base for Certification
Blueprints has reviewed one study of PROSPER and it meets Blueprints evidentiary standards (specificity, evaluation quality, impact, dissemination readiness). The study was done by the developer.
Study 1
Spoth et al. (2007, 2015), Redmond et al. (2009) and six additional studies randomized 28 schools to treatment and control conditions in a cohort sequential design which began with 12,022 sixth graders at pretest. Youth self-report data was collected at pretest, the end of 6th grade and annually through the end of 12th grade, and finally when youth were age 19 (7.5 years after baseline and 6 years after program end). Measures included drug use, other problem behaviors, and family relations.
Primary Evidence Base for Certification
Study 1
(Spoth et al., 2007): There were significant program effects after both the family- and school-focused interventions were delivered, for PROSPER youth relative to controls, on lifetime use of gateway (cigarettes, alcohol, marijuana) and illicit drugs (methamphetamine, ecstasy, marijuana, prescription medications), on past-year use of marijuana and inhalants, and on initiation of marijuana, inhalant, methamphetamine and ecstasy use. Marginally significant effects were found for new user rates of drunkenness and cigarette use, past month cigarette use, and past year drunkenness. Effects on lifetime use of both gateway and illicit substances, and past-month cigarette use were moderated by risk level, meaning that effects were stronger on these measures for youths who had already initiated use of gateway drugs at baseline.
(Redmond et al., 2009): Of 63 comparisons of protective factor outcomes (21 outcomes measured at each of three time points: posttest, 1-year follow-up, and 2-year follow-up), 29 improved significantly, 13 showed marginal significance, and 21 did not approach significance. Most significant improvements were at posttest and the 1-year follow-up. Problem solving, substance use expectancies, and association with antisocial peers were significantly improved at all three assessments. Five outcomes, including general child management, harsh discipline, child-to-mother affective quality, parent-child activities, and family environment improved at posttest and the 1-year follow-up. Other outcomes significant only at posttest included child-to-father affective quality, substance refusal intentions, attitude toward substance use, and assertiveness. Child monitoring, inductive reasoning, parent-child affective quality, and mother-to-child affective quality improved only at the 1-year follow-up.
(Spoth et al., 2011): Of 15 substance use outcomes examined at the 3-year follow-up, seven showed significant and five showed marginally significant improvement for the intervention group at the follow-up. Significantly slower growth over time was observed for 13 of the 15 outcomes; one additional outcome showed marginally significant effects. Initiations into marijuana, inhalant, methamphetamine, and ecstasy, Illicit Substance Use Index, past-year marijuana use, and past-year methamphetamine use were significantly improved for both modeling approaches. Additional outcomes that were significant only for models looking across time included initiations into drunkenness and cigarettes, Gateway Substance Initiation Index, past-month alcohol use, past-month cigarette use, and past-year drunkenness.
(Spoth, Redmond et al., 2013): Of 20 comparisons (ten substance use outcomes at both the 4-year follow-up, or 11th grade, and the 5-year follow-up, or 12th grade), seven showed significant improvements for the intervention group, five were marginally significant, and eight did not differ. Of nine substance use outcomes measured across the study period, condition-by-time interactions showed significance for three outcomes, marginal significance for three outcomes, and no significance for two outcomes. Across the two follow-ups and over time, lifetime illicit substance use, past-year methamphetamine use, and frequency of marijuana use showed improvement for the intervention group. Past-year marijuana use was significantly reduced at the 4-year follow-up, but not at the 5-year follow-up or over the study period.
(Spoth, Trudeau et al., 2013): Compared to the control participants, significantly fewer intervention participants reported lifetime prescription opioid misuse or prescription drug misuse at the 5-year follow-up.
(Spoth et al., 2015): At each of the five follow-ups, the intervention group reported significantly fewer conduct problem behaviors compared to the control group.
(Spoth et al., 2017): At the age-19 follow-up, the intervention group, relative to the control group, reported significantly lower lifetime use of numerous illicit drugs and significantly lower current frequency of cigarette use, frequency of marijuana use, and drug-related problems.
Primary Evidence Base for Certification
Study 1
(Spoth et al., 2007):
The following substance use outcomes significantly improved at 3-year follow-up (10th grade) or over time from the baseline to the 3-year follow-up (Spoth et al., 2011):
The following substance use outcomes significantly improved at the 4-year follow-up (11th grade), 5-year follow-up (12th grade), or over time from the baseline to the 5-year follow-up (Spoth, Redmond et al., 2013):
Compared to the control group, intervention participants significantly improved on these 5-year follow-up outcomes (Spoth, Trudeau et al., 2013):
The intervention group improved on the following outcome at each of the five follow-ups (1-year, 2-year, 3-year, 4-year, and 5-year; Spoth et al., 2015):
The following protective factors showed significant improvement at posttest, the 1-year, or 2-year follow-up (Redmond et al., 2009):
(Spoth et al., 2017): At the age-19 follow-up, the intervention group, relative to the control group, reported significantly lower:
In Study 1, individual-level effect sizes for significant protective factors at posttest, 1-year follow-up, and 2-year follow-up were small, ranging from .10 to .15. Effect sizes at the community level for these factors were larger (.43 to .81). Substance use outcomes at the 3-year follow-up showed small to medium-large effect sizes (.13 to .77), with community level effects ranging from .44 to .74. The study did not provide standardized effect sizes for substance use outcomes at the 4- and 5-year follow-ups but reported relative reduction rates of 3.3% to 31.4%. Relative reduction rates for conduct problem behaviors at the follow-ups ranged from 3.6% to 15.2%. Spoth et al. (2015) reported small relative risk effect sizes on conduct problem behaviors at long-term follow up. Spoth et al. (2017) reported small effect sizes ranging from .13-.15.
One study meets Blueprints standards for high quality methods with strong evidence of program impact (i.e., "certified" by Blueprints): Study 1 (Spoth et al., 2007, 2015; Redmond et al., 2009; and additional reports). The study took place in rural towns and small cities in Iowa and Pennsylvania and compared the treatment to a control condition.
This program is a preventive intervention targeting misuse of illicit drugs including opioids. This program is not a treatment for substance use disorder. As an upstream preventive intervention, this program targets and reduces problem behaviors that are associated with increased risk of developing substance use disorder or opioid use disorder later in life.
Additional articles reported on PROSPER, but are not included in this write-up because they did not describe program effects. Two articles described the PROSPER model of delivery, and two articles focused on implementation fidelity. Spoth et al. (2004) presented a model to guide capacity-building in state public education systems for delivery of evidence-based programs, and Spoth and Greenberg (2011) demonstrated how PROSPER responded to challenges to successful community practice of evidence-based programs. Spoth, Clair et al. (2007) indicated that an average attendance rate for family program activities of 17% was higher than other community-based recruitment rates. Spoth, Guyll et al. (2007) reported high rates of implementation adherence and other indicators of implementation quality for both cohorts of the study. In examining implementation quality across as many as six cohorts, Spoth et al. (2011) also found consistently positive implementation results.
Spoth, R., Clair, S., Greenberg, M., Redmond, C., & Shin, C. (2007). Toward dissemination of evidence-based family interventions: Maintenance of community-based partnership recruitment results and associated factors. Journal of Family Psychology, 21 (2), 137-146.
Spoth, R. & Greenberg, M. (2011). Impact challenges in community science-with-practice: Lessons from PROSPER on transformative practitioner-scientist partnerships and prevention infrastructure development. American Journal of Community Psychology, 48, 106-119.
Spoth, R., Greenberg, M., Bierman, K., & Redmond, C. (2004). PROSPER community-university partnership model for public education systems: Capacity-building for evidence-based, competence-building prevention. Prevention Science, 5 (1), 31-39.
Spoth, R., Guyll, M., Lillehoj, C. J., Redmond, C., & Greenberg, M. (2007). PROSPER study of evidence-based intervention implementation quality by community-university partnerships. Journal of Community Psychology, 35 (8), 981-999.
Spoth, R., Guyll, M., Redmond, C., Greenberg, M., & Feinberg, M. (2011). Six-year sustainability of evidence-based intervention implementation quality by community-university partnerships: The PROSPER study. American Journal of Community Psychology, 48: 412-425.
Blueprints: Promising
Greg Pliler
PROSPER Program Manager
Iowa State University
Email: gpliler@iastate.edu
Website: http://www.prosper.ppsi.iastate.edu/
Osgood, D. W., Feinberg, M. E., Gest, S. D., Moody, J., Ragan, D. T., Spoth, R., . . . Redmond, C. (2013). Effects of PROSPER on the influence potential of prosocial versus antisocial youth in adolescent friendship networks. Journal of Adolescent Health, 53, 174-179.
Certified Redmond, C., Spoth, R. L., Shin, C., Schainker, L. M., Greenberg, M. T., & Feinberg, M. (2009). Long-term protective factor outcomes of evidence-based interventions implemented by community teams through a community-university partnership. Journal of Primary Prevention, 30, 513-530.
Spoth, R. L., & Greenberg, M. T. (2005). Toward a comprehensive strategy for effective practitioner-scientist partnerships and larger-scale community health and well-being. American Journal of Community Psychology, 35(3/4), 107-126.
Certified Spoth, R. L., Trudeau, L. S., Redmond, C. R., Shin, C., Greenberg, M. T., Feinberg, M. E., & Hyun, G. (2015). PROSPER partnership delivery system: Effects on adolescent conduct problem behavior outcomes through 6.5 years past baseline. Journal of Adolescence, 45, 44-55.
Spoth, R., Redmond, C., Clair, S., Shin, C., Greenberg, M., & Feinberg, M. (2011). Preventing substance misuse through community-university partnerships: Randomized controlled trial outcomes 4½ years past baseline. American Journal of Preventive Medicine, 40(4), 440-447.
Certified Spoth, R., Redmond, C., Shin, C., Greenberg, M., Clair, S., & Feinberg, M. (2007). Substance-use outcomes at 18 months past baseline: The PROSPER community-university partnership trial. American Journal of Preventive Medicine, 32(5), 395-402.
Spoth, R., Redmond, C., Shin, C., Greenberg, M., Feinberg, M., & Schainker, L. (2013). PROSPER community-university partnership delivery system effects on substance misuse through 6½ years past baseline from a cluster randomized controlled intervention trial. Preventive Medicine, 56, 190-196.
Spoth, R., Trudeau, L., Shin, C., Ralston, E., Redmond, C., Greenberg, M., & Feinberg, M. (2013). Longitudinal effects of universal preventive intervention on prescription drug misuse: Three randomized controlled trials with late adolescents and young adults. American Journal of Public Health, 103(4), 665-672.
Spoth, R., Redmond, C., Shin, C., Greenberg, M. T., Feinberg, M. E., & Trudeau, L. (2017). PROSPER delivery of universal preventive interventions with young adolescents: Long-term effects on emerging adult substance misuse and associated risk behaviors. Psychological Medicine, 47, 2246-2259. doi:10.1017/S0033291717000691
Summary
Spoth et al. (2007, 2015), Redmond et al. (2009) and six additional studies randomized 28 schools to treatment and control conditions in a cohort sequential design which began with 12,021 sixth graders at pretest. Youth self-report data was collected at pretest, the end of 6th grade and annually through the end of 12th grade, and finally when youth were age 19 (7.5 years after baseline and 6 years after program end). Measures included drug use, other problem behaviors, and family relations.
Across the multiple studies and time points, compared to the control group, youth in the intervention group had significant improvements on:
Evaluation Methodology
Design (Spoth et al., 2007): PROSPER recruited 28 school districts from Iowa and Pennsylvania to participate in a randomized, cohort sequential design involving two cohorts. School districts were eligible if they enrolled between 1300 and 5200 students and if at least 15% of the students were eligible for free or reduced-price school lunches. Communities were blocked on school district size and geographic location and then randomly assigned to either the treatment or control condition. Five schools declined to participate. There were 14 schools in both the treatment and control conditions. The family intervention was delivered in the 6th grade year, while the school-based intervention was delivered in the seventh grade year. Baseline assessments were conducted prior to intervention activities. A mid-program assessment was administered at the end of 6th grade, after the family-focused program activities but before the school-based intervention. The intervention lasted for 1.5 years. The posttest was conducted in 7th grade and follow-ups were administered at 1 year (8th grade, or 2.5 years after baseline), and each year after that, up to the 5-year follow-up (12th grade or 6.5 years after baseline).
As noted in Spoth et al. (2007), two communities withdrew after randomization and were replaced. However, the studies offer few details about the replacement.
All 28 school districts participated in all waves. For students within the school districts, the study reported slightly different response rates and sample sizes in different articles. The program designer reported that discrepancies are attributable to students changing conditions or transferring schools, varying requirements of specific samples used in analyses, and differences in how samples were defined.
Spoth et al. (2007) indicated that 12,022 students participated at baseline, and 10,781 students took the posttest. Redmond et al. (2009) reported that 11,931 students completed pretest surveys, with 10,706 (90%) completing posttest assessments, 10,170 (85%) completing 1-year follow-ups in 8th grade, and 9,438 (79%) assessed at the 2-year follow-up in 9th grade. Spoth et al. (2011) counted 11,960 baseline participants, 10,737 students at posttest, 10,209 at 1-year, 9,474 at 2-year, and 8,655 at 3-year follow-ups. Spoth, Redmond et al. (2013) reported a baseline sample of 11,960, with students changing conditions omitted, and did not indicate sample sizes at other waves, noting that across eight assessments, an average of 86% of all eligible students completed the surveys "with slightly higher rates of participation at earlier data collection points." Spoth et al. (2015) reported a baseline sample of 10,849 students, 10,320 at posttest, 11,008 at 1-year follow-up (7th grade), 10,927 at 2-year follow-up (8th grade), 10,785 at 3-year follow-up (9th grade), 9,616 at 4-year follow-up (10th grade), 8,677 at 5-year follow-up (11th grade), and 7,774 or 71.7% at 6-year follow-up (12th grade).
Spoth et al. (2017): The age 19 follow-up assessment, which came 7.5 years after baseline and six years after the program end, examined a randomly selected subsample of participants who had completed the 6th-grade baseline assessment and had been enrolled in their baseline school district in 9th grade. The random selection was stratified by school district, gender, and risk status (based on substance use, conduct problems, poverty, etc.), and high-risk participants were oversampled. A total of 1,985 subsample participants (18% of the baseline sample) completed the assessment. In addition, some measures had substantial missing data (e.g., n = 1445 for drinking frequency, or 73% of the subsample). Although the Consort diagram listed the number of participants who completed the survey, it did not list the number of randomly selected participants who did not complete the survey.
Sample Characteristics: The majority of students were white (85%), followed by Hispanic/Latino (5%), and African-American (3%). Roughly half of students were female (51%). Most (64%) students lived with both of their biological parents, and less than one-third (31%) reported receiving free or reduced price lunch (Redmond et al., 2009).
Measures: Analyses relied on self-reports of substance use. Assessments consisted of the Substance Initiation Index-Gateway scale, which asks students if they ever drank alcohol, smoked a cigarette, or smoked marijuana, and the Substance Initiation Index-Illicit scale (also called Illicit Substance Use Index and Lifetime Illicit Substance Use), which asked students about lifetime use of methamphetamines, ecstasy, marijuana, prescription drugs and medications, and Vicodin, Percocet and Oxycontin (prescription opioids). New user rates were calculated to control for baseline rates, based on those students who did not report lifetime use of a substance at baseline. New-user rates were computed for drinking alcohol, drunkenness, cigarette use, marijuana use, inhalant use, methamphetamine use, and ecstasy use. Past-month rates were computed for alcohol, cigarettes and drunkenness and past-year rates were computed for drunkenness, marijuana, inhalant and methamphetamine use. Due to low past-month rates, only past-year rates were analyzed in Spoth et al. (2007), excluding methamphetamine use, which also had a very low prevalence rate.
The following student-reported protective factors were collected at posttest, the 1-year follow-up, and the 2-year follow-up (Redmond et al., 2009):
Additional substance use items were self-reported (Spoth, Redmond et al., 2013):
The following problem behavior outcome was collected at each of seven assessments (baseline, posttest, and 1-year, 2-year, 3-year, 4-year, and 5-year follow-ups; Spoth et al., 2015).
The following measure was collected at baseline, mid-program, posttest, and 1-year and 2-year follow-ups (Osgood et al., 2013):
Spoth et al. (2017): The age 19 follow-up examined 24 outcome measures (listed in Table 2) that covered six domains: lifetime illicit and non-prescribed drug use, current substance use, frequency of substance use, drug/alcohol-related problems, health-risking sexual behaviors/STIs, and antisocial/delinquent behaviors. All measures came from participant self-reports. The measures had evidence of validity from other studies but consisted of dichotomous indicators and counts that were not suitable for calculating reliabilities.
Analysis: (Spoth et al., 2007): Intent-to-treat analyses were conducted using ANCOVAs, with state, cohort, block, and risk status included as factors at the school level. An individual-level covariate, general child management, was also included, having been assessed with 13 items on the student questionnaires that measured monitoring and consistent discipline.
(Redmond et al., 2009): Multilevel ANCOVA models compared outcomes across conditions. Models controlled for factors associated with the study design (intervention condition, state, cohort, and block), baseline outcome levels, and whether the child lived with both biological parents. Multilevel analysis accounted for the nested design of the study. The study reported using an intent-to-treat strategy, though information on sources of attrition and sample sizes used in analysis were not provided.
(Spoth et al., 2011): For the point-in-time analysis, multilevel ANCOVA models compared outcomes across conditions at the 3-year follow-up. Models controlled for factors associated with the study design (intervention condition, state, cohort, and block), baseline outcome levels, and a general child management score. Restricted maximum likelihood estimated variance components in the model.
For change-in-time analysis, condition-by-time interactions in repeated measures ANCOVAs determined whether trajectories of substance use differed significantly across schools. Rather than use multilevel models, all data were aggregated to the school level, and five waves of data from baseline (6th grade) to the 3-year follow-up (10th grade) were included. Models included factors for state, cohort, condition, block, and risk status. An auto-regressive model was employed to describe the variance-covariance structure of each outcome across waves of data collected.
The study reported following intent-to-treat by including "all youth for whom usable data were collected", but did not indicate the numbers of students included in models. The study reported using full-information maximum likelihood estimation to handle missing data.
(Spoth, Redmond et al., 2013): Multilevel longitudinal models analyzed outcomes at 4- and 5-year follow-ups (11th and 12th grades) and across time (from baseline to the 5-year follow-up) using growth trajectories. The study identified the level 2 unit as both school and school district, and the program designer noted that school district is the accurate term except for later high school waves when there was only one school per district. Models controlled for state, cohort, condition, and block at level 2, and time, risk status, and baseline outcomes at level 1. All models included baseline outcomes as a covariate; growth analyses included seven waves of data, starting with the mid-program assessment and continuing through 12th grade. To account for correlations over time in the repeated measures, auto-regressive covariance with heterogeneous variances was employed. The non-iterative minimum variance quadratic unbiased estimator option estimated variance components, which the study argued provided more conservative, larger standard errors than iterative methods.
The study reported following intent-to-treat. However, it did not indicate how many students were included in models and omitted students who transferred schools and changed conditions. In a letter to Blueprints, the first author noted that less than 1% were omitted because they changed conditions and that the study included other subjects regardless of participation in the program. Otherwise, students were included if they completed three of eight survey waves and missing data were accounted for using full-information maximum likelihood estimation.
(Spoth, Trudeau, Shin et al., 2013): Multilevel models compared lifetime misuse of prescription opioids and any prescription drugs (opioids or any other prescription drugs) across conditions at the 5-year follow-up. No other analytic details were given. Baseline outcome controls would have been inappropriate given the outcome of lifetime misuse. Presumably, no individuals reported misuse at pretest.
The study also used multilevel models to examine moderated program effects for higher risk individuals (defined as initiation of alcohol, cigarettes, or marijuana at baseline).
(Spoth et al., 2015): Multilevel zero-inflated Poisson models analyzed conduct problem behaviors. Students were clustered in schools and school intercepts were allowed to vary randomly. Results indicated program effects at each of the five follow-ups (1-year, 2-year, 3-year, 4-year, and 5-year). Relative reduction rates were calculated using a cutoff of three or more problem behaviors. To transform trajectories into meaningful units for comparison, additional models examined the difference between control and intervention conditions in terms of the time (in months) to reach 1.70 problem behaviors (the 9th grade mean score). The re-parameterization of the zero-inflated Poisson model parameters could then yield the estimator and standard error of the difference across conditions in the length of time from baseline to reach this level.
Moderation analysis interacted condition status with an indicator of higher risk defined as lifetime alcohol, cigarette, or marijuana use at baseline.
All cases with pretest and at least 2 additional assessments were included. Missing data for these cases were filled in with multiple imputation. These procedures resulted in an analyzed sample size of 9,287 (85.6%).
(Osgood et al., 2013): Multilevel regression models determined program effects on antisocial influence potential. Networks (level 1) were nested within schools (level 2), which were nested within school districts (level 3), which were nested within random assignment pairs (level 4). In addition to random intercepts at each level, cohorts were allowed to vary through a random coefficient at the district level. Models controlled for wave through a categorical variable. Models appear to have included baseline and mid-program assessments in the analysis without distinguishing effects only for post-intervention outcomes.
(Spoth et al., 2017): The analysis used multilevel ANCOVA models, either binary or linear depending on the outcome. The multilevel models adjusted standard errors for clustering within school districts (i.e., the unit of assignment), and they controlled for the design factors (school district size and location), cohort, state, and risk status (also used as a moderator). The models did not control for baseline outcomes, likely because the age-19 outcomes were not measured at the sixth-grade baseline assessment. Note that the level-2 sample size of 28 may not be large enough to accurately estimate the standard errors, and the result may be to overstate the significance of the tests.
The authors stated that "An intent-to-treat approach was applied for all analyses." Although the analysis did not use multiple imputation, it appeared to use all participants with complete data. The sample sizes listed in Table 2 varied because of missing responses for particular outcomes, but there was no indication that participants were dropped because of non-participation in the program.
Outcomes
Implementation Fidelity (Spoth et al., 2007): A total of 1064 families attended at least one session of the family intervention, representing only 17% of all eligible families in the 14 schools. Strategies and incentives used to boost treatment engagement included a promotional video and informational displays used at parent-teacher conferences, phone and mail invitations to individual families, classroom presentations, announcements in school newsletters, community distributions of promotional items, and participation incentives ($10 family gift, door prizes, and a $3 youth gift for each session attended). Group facilitators were trained by SFP 10-14 personnel and each team of program facilitators was observed two to three times to assess adherence to intervention protocol. Adherence rates ranged from 92% for family sessions, 88% for parent sessions, and 91% for youth sessions. Of the families who did attend, 90% attended at least 4 sessions, while 63% attended six or more sessions.
Trained observers also monitored selected classroom lessons of the school-based programs. Adherence rates for LST, Project ALERT, and All Stars were 89%, 89% and 91%, respectively.
Baseline Equivalence: Groups were equivalent at pretest on all sociodemographic measures (child gender, age, grades, school absence, race, free school lunch, biological parents) and all fourteen outcome measures (Spoth et al., 2007). There were also no significant baseline differences across conditions on conduct problem behaviors (Spoth et al., 2015). The study reported that intervention and control groups were similar on baseline antisocial influence potential, with only 1 of 48 specifications differing significantly (Osgood et al., 2013).
For the age-19 follow-up, Spoth et al. (2017) reported from tests for baseline equivalence that "there were no significant differences on any sociodemographic measure (e.g., gender, age, race, school lunch status) or on any outcomes."
Differential Attrition: Attrition after one program year (the family intervention year) was 4.4% for controls and 11.2% for treatment youth. Attrition after two program years, from pretest, was 11% for controls and 9.7% for treatment youth. There was no evidence of differential attrition at posttest (the end-of-second-year assessment; Spoth et al., 2007).
The study reported that there were no significant condition-by-attrition interaction effects for sociodemographic variables or lifetime marijuana use at the 5-year follow-up (Spoth, Trudeau, Shin et al., 2013).
Two factor analyses of variance showed that there was a significant condition-by-attrition interaction for conduct problem behavior at posttest and the 1- and 2-year follow-up, but not for the 3-, 4-, or 5-year follow-ups. The significant interactions showed that individuals with a higher level of conduct problem at baseline were more likely to be retained in the intervention, compared to the control condition (Spoth et al., 2015).
For the age-19 follow-up, Spoth et al. (2017) tested for "Differential representation . . . by examining whether the two-way interaction of Condition × Outcome pretest score predicted participation at age 19." The tests revealed no significant interactions. It was unclear if the tests examined participation relative to the baseline sample, the eligible sample with 9th-grade data, or the random subsample selected for the age-19 follow-up. Additionally, tests for differential attrition may be incomplete, as there was no report of tests of sociodemographic characteristics by condition.
Posttest and follow-ups:
(Spoth et al., 2007): There were significant program effects for PROSPER youth, relative to controls, on both lifetime substance use indices at posttest. The program was also significantly more effective in preventing onset of marijuana, inhalant, methamphetamine, and ecstasy use for PROSPER youth, relative to controls. There were marginally significant effects on prevention of onset for drunkenness and cigarette use, for past-month cigarette use, and for past-year drunkenness. Finally, there were significant program effects on use of both marijuana and inhalants in the past year, relative to control youth.
Effects were also examined by risk status, which was determined by pretest scores on the Substance Initiation Index-Gateway scale. In other words, those who had initiated use of cigarettes, alcohol, or marijuana at pretest were designated as higher-risk, while those who had not were designated as lower-risk. Risk moderation effects did not reach statistical significance for any individual new-user rates. There were significantly stronger intervention effects for the higher-risk subsample, however, on both lifetime use of gateway drugs and illicit drugs. Finally, risk-level also moderated the program effect on past-month cigarette use.
(Redmond et al., 2009): Of 63 comparisons (21 outcomes measured at posttest, a 1-year follow-up, and a 2-year follow-up), 29 improved significantly (p<.05 for two-tailed tests) and 13 showed marginal significance (p<.05 for one-tailed tests) for the intervention group; 21 outcomes did not differ across conditions. Most significant improvements were at posttest and the 1-year follow-up.
Problem solving, substance use expectancies, and association with antisocial peers improved significantly at all assessments. General child management, harsh discipline, child-to-mother affective quality, parent-child activities, and family environment were each significant at the posttest and 1-year follow-ups. Perceived substance use norms were marginally significant at the posttest and significant at the 1- and 2-year follow-ups. Assertiveness was significant at posttest and marginally significant at the 1- and 2-year follow-ups. Substance refusal intention, significant at the posttest, was marginally significant at the 2-year follow-up. Child-to-father affective quality was significant at posttest and marginally significant at the 1-year follow-up. Attitude toward substance use improved significantly at posttest, but did not differ at other waves. Both parent-child affective quality and mother-to-child affective quality showed marginal significance at posttest, significance at the 1-year follow-up and nonsignificance at the 2-year follow-up. Child monitoring, inductive reasoning, overall parent-child affective quality, and mother-to-child affective quality were marginally significant at posttest and significant at the 1-year follow-up.
Consistent discipline and substance use plans were marginally significant at posttest and nonsignificant at later assessments. Substance refusal efficacy and father-to-child affective quality did not show improvement at any wave.
(Spoth et al., 2011): Of 15 substance use outcomes measured at the 3-year follow-up, seven significantly improved for the intervention group (p < .05, two-tailed), five were marginally significant (p < .05, one-tailed), and three did not differ across conditions. Significant reductions were reported for initiation into marijuana, inhalant, methamphetamine, and ecstasy use, the Illicit Substance Use Index, past-year marijuana use, and past-year methamphetamine use. Drinking alcohol, drunkenness, cigarette use, Gateway Substance Use Initiation Index, and Inhalant use showed marginal significance. Past-month alcohol and cigarette use and past-year drunkenness did not demonstrate any differences.
Of 15 substance use outcomes measured across time, longitudinal analysis showed significantly slower growth for 13 outcomes (p < .05, two-tailed), marginally significant decreases over time for one outcome (p < .05, one-tailed), and no difference over time for one outcome. Initiation into drunkenness, cigarettes, marijuana, inhalant, methamphetamine, and ecstasy, Gateway Substance Initiation Index, Illicit Substance Use, past-month alcohol use, past-month cigarette use, and past-year drunkenness, marijuana, inhalant, and methamphetamine use all showed significant condition-by-time interactions. Initiation into drinking alcohol showed marginal significance, and past-year inhalant use did not differ significantly.
Though the study reported significance levels from one-tailed tests, this write-up doubled those values to get two-tailed significance levels.
(Spoth, Redmond et al., 2013): Of 20 comparisons (10 substance use outcomes at the 4-year follow-up, or 11th grade, and 10 outcomes at the 5-year follow-up, or 12th grade), seven showed significant improvements for the intervention group (p < .05, two-tailed), five were marginally significant (p<.05, one-tailed), and eight did not differ. Lifetime illicit substance use, past-year methamphetamine use, and frequency of marijuana use improved significantly at both follow-ups, and past-year marijuana use also showed significance at 11th grade and marginal significance at 12th grade. Past-month cigarette use was marginally significant at both follow-ups and 11th grade frequency of drunkenness and past-year inhalant use at 12th grade were also marginally significant. No differences were reported at the 11th or 12th grade for past-month drunkenness, past-year driving after drinking, or frequency of driving after drinking; nor were there differences for past-year inhalants at 11th grade or frequency of drunkenness at 12th grade.
Of eight substance use measures looked at across the study period (baseline to 5-year follow-up), significant condition-by-time interactions (p < .05, two-tailed) were reported for four outcomes (lifetime illicit substance use, past-year marijuana, past-year methamphetamine, and frequency of marijuana), marginally significant interactions were noted for two outcomes (past-month cigarettes, and frequency of drunkenness) and two outcomes showed no differences (past-month drunkenness and past-year inhalants).
Moderation analysis indicated stronger program effects on some outcomes for individuals indicating any lifetime use of alcohol, cigarettes, or marijuana at baseline. Lifetime illicit substance use at 11th grade, 12th grade, and over the study period dropped significantly more for this higher risk group. Past-year marijuana use also showed significant moderation effects at 11th and 12th grade and was marginally significant for the growth models. Similarly, frequency of marijuana use had significant moderation effects for 11th grade and was marginally significant at 12th grade and over time. Significantly stronger program effects for high risk participants also emerged for frequency of driving after drinking, past-year driving after drinking (marginal), and frequency of drunkenness (marginal).
Though the study reported significance levels from one-tailed tests, this write-up doubled those values to get two-tailed significance levels.
(Spoth, Trudeau, Shin et al., 2013): Compared to the control group, significantly fewer intervention participants reported lifetime prescription opioid misuse (one-tailed p=.019; relative risk reduction (RRR) =21%) or prescription drug misuse (one-tailed p=.016; RRR=20%). The study also reported that there was no evidence of differential program effects for higher risk participants, but did not provide details on the moderation analysis.
(Spoth et al., 2015): At each of the five follow-ups from 8th grade to 12th grade, the intervention group reported significantly improved conduct problem behaviors compared to the control group. Results showed that across the 9th and 10th grades, relative risk reductions for intervention group adolescents reporting 3 or more conduct problem behaviors were 13.7% and 14.5%, respectively. Following 10th grade, the frequency of conduct problem behaviors began to level off and diminish.
(Osgood et al., 2013): Across five waves (pretest to 2-year follow-up), intervention group networks showed significantly lower antisocial influence potential, indicating that antisocial youths were less central than other youth in intervention networks compared to control networks. These findings held for different specifications for network centrality and antisocial attitudes and behavior, with 26 of 48 combinations showing significant associations in the expected direction.
(Spoth et al., 2017): At the age-19 follow-up, tests showed significant program effects for 11 of the 24 outcomes. Eight of the significant effects occurred in the lifetime drug use domain (e.g., marijuana, cocaine, non-prescription drugs), and three of the significant effects occurred for current behavior: frequency of cigarette use (d = .15), frequency of marijuana use (d = .13), and drug-related problems (d = .14). There were no significant effects for current use measures, the health-risking sexual behavior/STI measures, or the antisocial behavior measure. Also, post hoc tests for changes since high school indicated no significant condition differences on any of the outcomes. The significant outcomes thus appear to have sustained previous improvements rather than resulted from new improvements.
Moderation tests found only one significant interaction for drug-related problems, with high-risk participants benefitting more from the intervention than low-risk participants. Generally, however, risk-related moderation effects were non-significant.