Tom – Page 2 – Blueprints for Healthy Youth Development

Issue No. 19

Welcome to the Blueprints Bulletin

Leadership Letter:

In our last newsletter, we explained how we apply our standards of evidence (Steeger et al., 2021) in rating interventions with an evidence base that is strong enough to be recommended by Blueprints for large-scale adoption (please click here for a summary of our review process).

Obtaining desired outcomes from scaling up Blueprints-certified interventions, however, requires multiple layers of support, including (for example): (1) adequate and appropriately structured funding; (2) partnerships with stakeholders at every level to ensure necessary support for program implementation and continuing service provision; and (3) community engagement to support local service delivery.

Several effective frameworks for broad dissemination have been established to ensure the impact of Blueprints-certified programs on a large scale.

The focus of this newsletter is highlighting examples of these efforts, with the goal of providing a “blueprint” for national, state, and/or local initiatives that might want to adopt a similar approach.

Statewide Public/Private Initiatives to Implement a Portfolio of Blueprints-Certified Interventions

In Colorado (home to the Prevention Science Program at the Institute of Behavioral Science at CU Boulder, which houses Blueprints), examples include:

Invest in Kids (IIK), a nonprofit organization that for over two decades has worked alongside Colorado communities to adopt, implement, and successfully scale proven programs that have the greatest long-term impact on young children and families experiencing poverty. These Blueprints programs, which IIK sustains via federal, state, and local grants and investments from private foundations and individual donors, include: 1) Nurse-Family Partnership® (Model), a relationship-based program, delivered in the home, that partners highly-trained registered nurses with first-time mothers and their babies and is designed to improve prenatal and child rearing practices and parent self-sufficiency through the child’s second birthday; 2) The Incredible Years® (Promising), a prevention program designed to increase a child’s success at school and at home by promoting positive parent, teacher, and child relationships; and most recently, 3) Child First® (Promising), a home-based two-generation mental health-focused program which works to heal and protect young children and their families from the devastating effects of chronic stress and trauma.
Colorado Equitable Economic Mobility Initiative (CEEMI), a newly formed independent nonprofit collaborating with public and private partners to create a more equitable economic ecosystem by blending public, private, and philanthropic funds to enroll Coloradans facing poverty in effective employment programs. CEEMI is working to scale up two Blueprints-certified programs: 1) Accelerated Study in Associate Programs (Model Plus), a post-secondary program addressing potential barriers to academic success and promoting credit accumulation and associate degree completion through comprehensive advisement and career and tutoring services provided by dedicated advisers; and 2) Year Up (Promising), a training and internship program that helps young people with limited post-secondary education get high-quality jobs by learning to work with technology, developing employment skills, and obtaining internships. Year Up is also one of the workforce programs backed by strong evidence of effectiveness identified by the National Conference of State Legislatures, and their new Center for Results-Driven Governing, for policymakers to consider as states leverage the American Rescue Plan (ARP) in seeking ways to re-employ workers and build a workforce that will help lead their economic recovery in this post-coronavirus economy.
Safe Communities Safe Schools (SCSS), an initiative within the Center for the Study and Prevention of Violence at the University of Colorado Boulder that follows a research-based model to build a school’s unique capacity to reduce violence by promoting social, emotional, and physical safety. Included in the SCSS Model is support that allows schools to (1) measure and strengthen their readiness to adopt evidence-based interventions, including those listed on the Blueprints registry, (2) build high-functioning multi-disciplinary teams to champion the work, and (3) receive tailored tools, processes, and technical assistance to support high-quality implementation.
Communities That Care (CTC), a framework which aims to reduce adolescent substance misuse, violence, and depression by implementing from a menu of strategies particular programs, practices and/or policies—including Blueprints-certified interventions—that have demonstrated effectiveness in reducing risk factors and enhancing protective factors. The Colorado Department of Public Health and Environment (CDPHE) adopted CTC in 2016 as a statewide prevention model. CDPHE currently funds 30 CTC communities across the state.

National Philanthropic Foundation/Community Partnerships Helping Scale Blueprints-Certified Interventions

Two examples of efforts to scale evidence-based interventions, including many on the Blueprints registry, funded by national philanthropic organizations include:

Evidence2Success®, which is an initiative of the Annie E. Casey Foundation that brings together public-system leaders and community members to understand how children are doing with the help of data; select evidence-based programs, including those from Blueprints to enhance strengths and address needs; and develop financing and action plans to support the ongoing use of those proven programs. The initiative began with a pilot site in Providence, RI in 2012 and has since expanded to Mobile, Alabama; Selma, Alabama; Kearns Township, Salt Lake County, Utah; Memphis, Tennessee; and Miami, Florida. While scale is sometimes more quickly achieved through a focus on system-wide reform and legislative change, Evidence2Success is built at the intersection of place-based change and system change – with a strong value on system change informed by community input and leadership. As part of the Evidence2Success framework, public systems and schools commit up front to redirecting a portion of their combined resources for children. These partners work with a team of finance and administration professionals to identify funding sources within their agency budgets and coordinate funding to invest in programs that respond to the outcomes prioritized by the partnership. In recent years, the Foundation has carefully tracked the resources leveraged by sites in support of Blueprints programs and supporting infrastructure. Since 2017, sites have leveraged more than $6 million across child welfare, juvenile justice, school district, public health, social service, city, county, and substance abuse and mental health systems. The addition of related activities, such as clinical services and trauma trainings in Providence, raises the total amount leveraged to $15 million.
Arnold Ventures’ Moving the Needle initiative, which seeks to spur expanded implementation of programs with credible evidence of meaningful positive effects in order to make significant headway against U.S. social problems. Specifically, the initiative is designed to encourage state or local jurisdictions, or other entities, to: (1) Adopt social programs shown in well-conducted randomized controlled trials (RCTs) to produce large, sustained effects on important life outcomes (see attachment 1 in the RFP for the list of eligible programs; many are listed on the Blueprints registry); (2) Implement these programs on a sizable scale with close adherence to their key features; and (3) Determine, through a replication RCT, whether the large effects found in prior research are successfully reproduced so as to move the needle on important social problems. Arnold Ventures (AV) has provided more than $38 million in funding over the past five years to help scale up and conduct replication trials of these interventions. Click here to listen to a 10-minute interview with David Anderson, VP of AV’s Evidence-Based Policy Initiative, for insights about building credible evidence in social policy.

Reallocation of Criminal Justice Dollars to Prevention

The Virginia Department of Juvenile Justice (Virginia DJJ) built a more robust continuum of services, with a focus on implementing evidence-based and evidence-informed programs and services statewide, thereby reducing the overuse of juvenile correctional centers and developing alternatives to incarceration. The Blueprints registry is identified within the Virginia DJJ’s guidance document as a resource in selecting effective prevention programs ready for large-scale adoption. As a result of Virginia DJJ’s commitment to evidence-based programs and high-quality implementation, youth and families who are involved with the juvenile justice system anywhere in the Commonwealth now have access to one or more Model/Model Plus programs. Click here for a summary report on Virginia DJJ’s Transformation over the past five years.

Conclusion

Attempts to scale inconclusive, ineffective, or harmful programs are a waste of time, money, and opportunity. As such, we advocate for identifying and implementing interventions that are known to work, that can be implemented at scale with fidelity, and that are cost effective. Blueprints provides a trusted source of accessible information on experimentally proven interventions recommended for large-scale implementation that meet a high evidentiary standard based upon a rigorous systematic review of all the research evidence. This newsletter provides just a few models for scaling social programs that produce important improvements in people’s lives.

If you are aware of additional models or frameworks for large-scale dissemination of evidence-based interventions, including those on the Blueprints registry, we would be interested in profiling your work. Please contact us at blueprints@colorado.edu.

And as always, thank you for your continued interest in and support of Blueprints.

Happy Holidays!

Sincerely,

Pamela Buckley, PhD
Principal Investigator
Institute of Behavioral Science
University of Colorado Boulder
Pamela.Buckley@Colorado.edu

Karl G. Hill, PhD
Co-Principal Investigator
Chair of the Blueprints Board
Institute of Behavioral Science
University of Colorado Boulder
Karl.Hill@Colorado.edu

Blueprints for Healthy Youth Development is hosted by the University of Colorado Boulder, Institute of Behavioral Science, with current support from Arnold Ventures and former funding from the Annie E. Casey Foundation. Each intervention included in the Blueprints database has been reviewed carefully by an independent advisory panel that looked at research on the intervention’s impact, practical focus and potential for implementation in public systems.

2021 in Review
By the numbers

Below is a summary of our year in review.

Interventions Reviewed & Certified in the Past Year

• Number of Unique Interventions Reviewed = 47
• Number of Individuals Articles or Reports Reviewed = 191
• Number of Interventions Certified = 5

Coping Power Universal (Promising)
Added: Feb. 18, 2021
Learn more

YouthBuild (Promising)
Added: Mar. 24, 2021
Learn more

EveryDay Intervention (Promising)
Added: Jun. 4, 2021
Learn more

Cannabis eCHECKUP TO GO (Promising)
Added Jul. 7, 2021
Learn more

ASSISTments (Promising)
Added Nov. 22, 2021
Learn more

Blueprints Virtual Talks

Our PI Dr. Pamela Buckley gave a talk to the Prevention Technology Transfer Center Network, which serves to improve implementation and delivery of effective substance abuse prevention programs. The webinar first described how an evidence base for an intervention is built. It then presented an overview of Blueprints, followed by a demonstration on navigating the registry’s website and tailoring searches to locate effective interventions that fit local communities’ unique circumstances. It concluded with a discussion on best practices in making adaptations to effective interventions to increase cultural appropriateness without modifying core components, or the essential intervention activities deemed necessary to produce desired outcomes. Watch the webinar by clicking here.
Drs. Pamela Buckley and Karl Hill gave a talk at the Prevention Science Methodology Group Virtual Grand Rounds on Addressing Health Equity and Social Justice within Prevention Registries. The webinar first presented an overview of the Blueprints registry, and then discussed concerns regarding adaptation/cultural relevance that registry staff and users encounter with increasing frequency. Next, an overview of a recently funded project was presented that begins to address these concerns within Blueprints. While lack of representation of youth of color in health-related research studies has been well documented, a critical evaluation of this omission has not been undertaken to substantiate this claim. This new project involves reviewing the scope of extant research on representation of ethnic minority groups in preventive intervention research, thus serving as a vehicle for decision-making regarding the generalizability of evidence-based interventions listed on clearinghouse websites (such as Blueprints). Go here to download slides and view the webinar.

Blueprints Interventions in the News
Relevant articles and helpful resources

In case you have missed them, here are a few articles and web postings that discuss Blueprints and/or feature some of our Blueprints’ Model/Model Plus and Promising Programs:

MDRC (a nonprofit, nonpartisan education and social policy research organization) has released a report for state agencies and academic institutions on how and why to implement evidence-based programs that support college completion, and the timing couldn’t be better: Such programs could get federal support as part of the $3.5 trillion Build Back Better Act. The report highlights ASAP (a Blueprints Model Plus program) as an example of an evidence-based, multifaceted support program that – while expensive – can reduce the cost per degree. Accelerated Study in Associate Programs (ASAP) is a post-secondary college-based prevention program that aims to address potential barriers to academic success and promote credit accumulation and associate degree completion in college students through comprehensive advisement and career and tutoring services provided by dedicated advisers. Click here to read the report.
Blueprints was recommended by the United Nations Office on Drug and Crime (UNODC) as they updated international standards for drug and crime prevention. The UNODC (United Nations Office on Drugs and Crime & World Health Organization, 2018) suggested that national standards globally enforce a requirement of implementing evidence-based strategies only by utilizing registries such as Blueprints, citing us by name. Read more here, including page 42 where Blueprints is cited.
Family Foundations, certified as a Promising program based on an experimental study conducted by Feinberg et al. (2014), is a a universal prevention program to improve mother, child, and birth outcomes through promoting coparenting quality among couples who are expecting their first child. Watch this 3-minute video to learn more.

Our mailing address is:
University of Colorado Boulder | Institute of Behavioral Science
483 UCB, Boulder, CO 80309

Issue No. 18

Welcome to the Blueprints Bulletin

Leadership Letter:

We are excited to share with you an important milestone Blueprints recently reached. But first, a quick summary of our history will help frame the discussion.

Blueprints History

The Blueprints for Healthy Youth Development mission is to provide a comprehensive registry of scientifically proven and scalable interventions that prevent or reduce the likelihood of antisocial behavior and promote a healthy course of youth development and adult maturity. Those interventions are rated as either Promising or Model/Model Plus.

Originally started as Blueprints for Violence Prevention, Blueprints was one of the earliest efforts to establish a clear scientific standard for evaluating the evidence of an intervention’s effectiveness, implementing a rigorous expert review process and certifying those interventions that met this standard. Launched in 1996 by internationally renowned researcher Dr. Delbert S. Elliott (who also founded the Center for the Study and Prevention of Violence in 1992), the early years of Blueprints focused on identifying programs that were effective in addressing violence and drug use outcomes. Since its inception, Blueprints has expanded its scope to include mental and physical health, self-regulation, educational achievement, and other positive developmental outcomes.

Celebrating an Important Milestone: 100 Certified Interventions!

In August 2021, Blueprints reached an important milestone by certifying its 100^th intervention!

Of these, 18 are rated Model/Model Plus and 82 are Promising.

The Blueprints website provides an interactive search feature to identify Blueprints-certified interventions based on specific criteria that allows users to browse through a wide range of interventions that match those criteria.

Model and Model Plus interventions are listed separately from Promising interventions.

This is because Model and Model Plus interventions have demonstrated efficacy for changing outcomes over time and are recommended by Blueprints for large-scale implementation.

Promising interventions show promise of efficacy but require follow-up research before being recommended for large-scale adoption.

How does an Intervention receive a Rating of Model or Model Plus?

The Blueprints registry standard for a Model rating requires:

A theoretical rationale/logic model
One or more high quality randomized control trials
An experimental replication
Sustainability of effects for at least one-year post intervention
No evidence of negative or harmful effects
The organizational capacity to provide materials, training, and information for new users to adopt and implement the model intervention with fidelity

Blueprints also has a Model Plus rating, which requires at least one trial by an evaluation team independent of the program developer and his or her colleagues. In sum, the logic is that to consider any program as ready to be relied on in practice or for scaling up, there should be a clearly specified theoretical rationale, reliable evidence of positive, sustained impact based on a sound experimental research design, and at least one replication of positive findings.

A high-quality quasi-experimental design (QED) can meet Blueprints standards, but to earn a Model or Model Plus rating, effective interventions must have demonstrated positive effects in at least one high-quality RCT.

The Blueprints website provides more information about our standards and review process.

Testing Promising Interventions for Scale-up

While most certified interventions listed on the Blueprints website are rated Promising (82 out of 100), several are currently being tested for replication.

Under the Research Grants Focused on Systematic Replication (84.305R) competition, the Institute of Education Sciences (IES), which is the research arm of the US Department of Education, will support systematic replication studies of interventions that have produced beneficial effects on education outcomes in one or more rigorous causal-impact studies. We are aware of two replications of Blueprints Promising programs that IES is currently funding:

Targeted Reading Instruction (TRI, formerly called Targeted Reading Intervention) is a teacher professional development program designed to help classroom teachers acquire key diagnostic reading instruction strategies differentiated to meet the exact needs of individual students who are struggling with early reading. Blueprints certified the program based on an RCT conducted by Amendum et al. (2011). Dr. Robin Wisniewski from RTI International received a 5-year grant (07/01/2021 – 06/30/2026) that will follow first grade students and teachers for an additional 2 years beyond the intervention to test TRI’s long-term impacts on reading achievement.
Whole Number Foundations Level K (Roots) is a 50-session small group mathematics intervention delivered as a pull-out program during the regular school day. The aim is to help strengthen whole number concepts and operations skills in students at risk for developing long-term mathematics difficulties. Blueprints certified the intervention based on an RCT conducted by Clarke et al. (2016). Dr. Ben Clarke from the University of Oregon received a 5-year grant (07/01/2020 – 06/30/2025) to conduct a replication study of ROOTS for students at risk for mathematics learning disabilities across school types and student populations intentionally selected to differ from previous efficacy studies.

Arnold Ventures, a philanthropy dedicated to tackling some of the most pressing problems in the United States, has provided support for RCTs of programs across the spectrum of social policy whose prior evidence shows potential for sizable effects on education, earnings, crime, and other important outcomes. Not only has Arnold Ventures provided support for the Blueprints registry, but it has also supported replications of several interventions rated as Promising on Blueprints:

Big Brothers Big Sisters of America (BBBSA) is a community mentoring program that matches a volunteer adult mentor to an at-risk child or adolescent to delay or reduce antisocial behaviors and improve academic success. The program was certified based on an RCT conducted by Grossman & Tierney (1998). Dr. David DuBois from the University of Illinois at Chicago and Dr. Carla Herrera (an independent consultant) are conducting a multi-site RCT with a sample of 20 BBBSA agencies nationwide (the grant term runs from 2016-2024).
Career Academies, which are school-based programs designed to provide students with the credentials and skills needed to make successful transitions to post-secondary education and careers, was certified by Blueprints based on a multi-site RCT conducted in a diverse set of urban and small-city high schools in California, Texas, and several cities on the east coast (Kemple & Snipes, 2000; Kemple, 2004; Kemple & Wilner, 2008). MDRC is conducting a replication of Career Academies in California high schools. The term of the grant is from 2017-2032.
Child First, a two-generation home visitation program which works to heal and protect young children and their families from the devastating effects of chronic stress and trauma, was certified based on an RCT conducted by Lowell et al. (2011). With support from Arnold Ventures and The Duke Endowment, MDRC is leading an experimental study that aims to replicate the findings from the earlier study in multiple locations across Connecticut and North Carolina.
EAAA (Enhanced Assess, Acknowledge, Act) Sexual Assault Resistance Education is a four-unit program to help first-year college women resist acquaintance sexual assault by providing them with information and resistance training. The intervention was certified based on an RCT conducted at three universities in Canada by Senn et al. (2015). Dr. Koss from the University of Arizona is leading a multi-site RCT that includes a one-year follow-up across five U.S. universities.
Rochester Forensic Assertive Community Treatment (R-FACT) is an outpatient treatment program to reduce recidivism and promote recovery among justice-involved adults with a serious mental illness that was certified based on an RCT conducted in New York by Lamberti et al. (2017). The University of Rochester is conducting an RCT to evaluate the R-FACT model in Minnesota. The grant runs from 2018 to 2025.
Year Up is a training and internship program that helps young people with limited post-secondary education get high-quality jobs by learning to work with technology, developing employment skills, and obtaining internships. The program was certified based on an RCT conducted by Fein & Hamadyk (2018) in eight sites across the US. Dr. David Fein of Abt Associates is leading long-term follow-ups to two RCTs of adaptations of the Year Up model, and the grant runs from 2019-2022.

Nominate an Intervention

If you are aware of an intervention Blueprints has rated as Promising that is being replicated and you would like us to review the study, please send all evaluation articles/papers to: blueprints@colorado.edu.

And if you are a funding agency awarding grants to replicate intervention programs designed to promote healthy youth development, please contact us as well. We would be interested in profiling your work and any interventions from the Blueprints registry you are funding to replicate.

More information on nominating an intervention for Blueprints review can be found on our website by clicking here.

As always, thank you for your continued interest in and support of Blueprints.

Sincerely,

Pamela Buckley, PhD
Principal Investigator
Institute of Behavioral Science
University of Colorado Boulder
Pamela.Buckley@Colorado.edu

Karl G. Hill, PhD
Co-Principal Investigator
Chair of the Blueprints Board
Institute of Behavioral Science
University of Colorado Boulder
Karl.Hill@Colorado.edu

RCTs Headed by Blueprints Staff (with support from Arnold Ventures):

RCT evaluation of LifeSkills Training to prevent substance abuse in high school students. This project, headed by Drs. Karl Hill and Christine Steeger, will evaluate whether the Botvin LifeSkills Training (LST) Middle School Program – a Blueprints Model Plus program that has been shown in rigorous evaluations to substantially reduce youth substance use – can produce similar impacts when provided directly to high school students in a more abbreviated and potentially more scalable format. Blueprints PI Dr. Pamela Buckley is also consulting on the project. Click here to read more about the study.

RCT evaluation of Functional Family Therapy-Gangs (FFT-G). This study, headed by Dr. David Pyrooz (Associate Professor of Sociology at the University of Colorado Boulder) and Blueprints PI Dr. Pamela Buckley, is an RCT of Functional Family Therapy–Gangs (FFT–G), a therapeutic intervention for justice-involved youth and their families. A previous, well-conducted RCT of FFT-G (i.e., certified by Blueprints) with a sample of 129 juveniles on probation and at risk of gang involvement in Philadelphia found that FFT–G led to statistically significant reductions in the percent of youth with drug charges and the percent of youth adjudicated for any offense during the 18-month period following random assignment (Gottfredson et al., 2018). Under this current project, the researchers will replicate FFT–G in two judicial districts in the Denver, CO area with a sample of 400 juveniles on probation determined to be gang-embedded. Click here to read more about the study.

Our mailing address is:
University of Colorado Boulder | Institute of Behavioral Science
483 UCB, Boulder, CO 80309

Issue No. 17

Announcement:

We are excited to announce that Dr. Pamela Buckley is the new Principal Investigator (PI) of Blueprints for Healthy Youth Development. Dr. Buckley joined the Blueprints team in 2016 and has been serving as Co-PI and Director. Dr. Karl Hill is still involved in the vision and scientific integrity of Blueprints and will assume the role of Co-PI and will remain Chair of the Blueprints Advisory Board. Read more about Drs. Buckley and Hill’s experience here.

Leadership Letter:

This newsletter features results from an online survey conducted in collaboration with our friends at Evidence-Based Associates on Model/Model Plus and Promising Programs’ COVID-19 (COVID) response.

In May 2020, Blueprints self-funded a survey that was administered to contacts listed for the Blueprints certified programs on how evidence-based interventions had responded to, and begun to plan for, the aftermath of the COVID pandemic. The 2020 survey contained eight questions and 58 of the 94 programs surveyed responded, for a 62 percent response rate.

This one-year follow-up, administered between May and June 2021, had a 58% response rate (57 of the 99 certified programs responded).

Results show most programs have experienced a small (or no) impact in delivering services as a direct result of COVID. However, a wide range of modifications to service delivery and implementation strategies were reported – findings that remained relatively stable over time.

Go here to review the May 2021 findings.

Impact of COVID-19 on dissemination/implementation

Most programs reported either no significant impact in service delivery from COVID, or a small impact causing some difficulties in maintaining dissemination but overall stability. In addition, more than one-quarter reported a positive impact, with COVID providing new opportunities for development. There was, however, a small percentage of programs that experienced a great impact, leading to discontinuation or serious difficulties in sustaining dissemination of their Blueprints-certified program.

Status of the dissemination/implementation

The data show the majority of programs that completed our survey had received requests for changes to the delivery modality and training/support of their intervention. These findings were relatively consistent across the initial (May 2020) and follow-up (May 2021) surveys. As of May 2020, just under half reported experiencing new requests for the adoption of their model – a number that increased by ten percentage points, as measured by the May 2021 responses. Meanwhile, none had discontinued implementation in the wake of COVID as of May 2020 (a percentage that slightly increased in May 2021), but some reported they temporarily suspended their program (a percentage that slightly decreased from the initial to the follow-up survey).

Modifications to service delivery

We conducted our May 2020 survey as many states were experiencing varying levels of stay-at-home orders, schools had been abruptly closed, and most business and childcare facilities were either shut down or beginning to phase in opening plans. As a result, programs were quickly faced with important decisions to ensure the safe continuity of programming while following public health guidelines. Just under one-quarter had not modified the intervention or its delivery due to COVID at the time of our May 2020 survey. The majority, however, provided online resources, turned to tele-sessions and/or video conferencing, and/or offered online training or lessons to support service delivery. A small percentage started a blog as a forum for conversation.

By May 2021, as the United States and some countries around the world had begun to ease restrictions with the availability of the COVID-19 vaccine, most programs continued to modify their service delivery by providing online resources and training and turning to tele-sessions/video conferencing – however, these numbers consistently increased by roughly 10 percentage points. Meanwhile, those blogging to offer a forum for conversation remained a small percentage (and this number dropped in half from May 2020 to May 2021).

In a new question posed with the follow-up survey, we found that an overwhelming majority had implemented changes to their delivery modality. An even greater majority also had implemented changes to their training/support model.

Impact of modified service delivery on intervention outcomes

More than one-third of survey respondents are either collecting data or plan to collect data soon on the relationship between modifications made due to COVID and intervention outcomes. This finding was consistent across both surveys. However, just over one-quarter reported in May 2020 that they do not have the resources to collect these data – a number that increased to over one-third in May 2021.

What was learned about implementing evidence-based interventions in the context of COVID?

We included an open-ended item asking about lessons learned. Responses are summarized here:

The pandemic highlighted the importance of evidence-based programs for youth as needs increased (e.g. increased levels of behavioral and mental health problems) and virtual programming revealed the basic needs of families supporting these youths (e.g. food insecurity, housing, mental and basic health resources).
Though developing virtual formats while maintaining fidelity to essential program elements is a lot of work, many programs found various levels of success in providing online/virtual training and/or implementation.
There is a need for in-person/online hybrid models. Virtual delivery encounters challenges – e.g., lack of access to or difficulties among participants with incorporating technology, inability to fully conduct interactive program activities, and lower levels of engagement and community-building. Virtual delivery also provides opportunities – e. g., increased engagement by removing barriers to access, reaching a wider audience, and reducing costs of training and delivery.
For school-based programs, hard copy curricula must take advantage of the modern classroom and technology so that schools will continue to adopt these programs. Teachers have learned to use technology more effectively, and will continue to use technology platforms to assist with teaching.
Flexibility is key for any program.

On March 11, 2020, the coronavirus outbreak was declared a pandemic by the World Health Organization. Since then, COVID has touched all aspects of life, from health care and mental health services to education, the environment, and the economy. Blueprints seeks to assist evidence-based interventions in sharing our survey data so that we can work together in supporting families and communities through the complexities of these challenging times. We hope the resources provided by survey respondents and shared in the next section of this newsletter offer helpful tips and ideas.

Sincerely,

Pamela Buckley, PhD
Principal Investigator
Institute of Behavioral Science
University of Colorado Boulder
Pamela.Buckley@Colorado.edu

Karl G. Hill, PhD
Co-Principal Investigator
Chair of the Blueprints Board
Institute of Behavioral Science
University of Colorado Boulder
Karl.Hill@Colorado.edu

Blueprints for Healthy Youth Development is hosted by the University of Colorado Boulder, Institute of Behavioral Science, with current funding from Arnold Ventures and former funding from the Annie E. Casey Foundation. Each intervention included in the Blueprints database has been reviewed carefully by an independent advisory panel that looked at research on the intervention’s impact, practical focus and potential for implementation in public systems.

Blueprints News & Resources
COVID-19 Response Resources

In lieu of featuring one Model/Model Plus and one Promising program as we have historically done in each newsletter, below are links to resources provided from survey responses that support evidence-based interventions in offering services during COVID. Several links are updates from what was provided along with results from the May 2020 survey. However, some new links have been added that were reported with the May 2021 follow-up survey results.

Model/Model Plus Programs

Body Project (a multi-session group intervention designed to prevent the onset of eating disorders such as anorexia, bulimia, and binge eating among female high school and college students with body image concerns).
A version for delivery by videoconference rather than in-person is available at no cost by contacting the program developer: Dr. Paul Rohde (paulr@ori.org).

Botvin LifeSkills Training (a classroom-based program designed to prevent teenage drug and alcohol abuse, tobacco use, violence and other risk behaviors by teaching students self-management skills, social skills, and drug awareness and resistance skills).
For COVID-related resources, go to:
LST COVID-19 Update
e-Learning Version of LST Middle School 1 (e-LST)

Early College High School Model (a high school model designed to increase students’ access to a postsecondary credential, particularly for underrepresented students).
The following guidance on using federal relief money applies to all college programs offered in high school, which includes early colleges: Early College COVID-19 Guidance

Multisystemic Therapy® (a juvenile crime prevention program designed to improve the real-world functioning of youth by changing their natural settings – home, school, and neighborhood – in ways that promote prosocial behavior while decreasing antisocial behavior).
For COVID-related resources, go to: MST COVID-19 Statement

Nurse-Family Partnership (a nurse home visiting program for first-time pregnant mothers designed to improve prenatal and child rearing practices through the child’s second birthday).
The links below provide COVID-related resources:
NFP Response to COVID-19
NFP COVID-19 Update for Moms

Project Towards No Drug Abuse (classroom-based drug prevention program designed for at-risk youth that aims to prevent teen drinking, smoking, marijuana, and other hard drug use).
Online resources can be found here: Transition to Online Implementation

ParentCorps (a multi-level intervention designed for pre-kindergarten children living in low-income neighborhoods to create safe, nurturing, and predictable environments at home and in the classroom and improve relationships and communication between parents and teachers).
For COVID-related resources, go to: Tools for Educators and Families

Promoting Alternative Thinking Strategies (a classroom-based social emotional learning program for elementary school students to reduce aggression and behavior problems).
For COVID-related resources, go to:
PATHS Online Free Resources COVID-19
Additional Resources for Remote Teaching

Promising Programs

Child First (a two-generation home visitation program that provides psycho-therapeutic services and intensive care coordination, while building adult reflective and executive capacity, to prevent or diminish serious emotional disturbance, developmental and learning disabilities, and abuse and neglect among young children).
The links below provide COVID-related resources:
Podcast on how Child First adapted their Home-Visiting Model During the Pandemic
Webinar: Child First Telehealth Case Presentation
Child First COVID-19 Resources
Resources for Home Visiting Models
Practice Spotlight: Dyadic Play via Telehealth
Practice Spotlight: Creative Ways to Engage Families

The Good Behavior Game (classroom management approach that rewards children for displaying appropriate on-task behaviors during instructional times).
For COVID-related resources, go to: How Positive Reinforcement Can Keep Students Engaged in a Virtual Setting

Head Start REDI (an enrichment program for low-income preschoolers and their families that supplements the Head Start preschool program).
Free online training and coaching modules (small charge to get credits for completing the modules):
Better Kid Care

Incredible Years®

Parent (a group-based program that strengthens parent competencies to promote young children’s social, emotional, and academic competence and prevent the development of conduct problems).
Teacher Classroom Management (a program that provides teachers of children ages 3-8 years with classroom management strategies to manage difficult behavior while promoting social, emotional, and academic competence).
Child Treatment (a child treatment program used by counselors and therapists to treat children ages 3-8 years with conduct problems, ADHD, and internalizing problems).

For COVID-related resources, go to each of these links: 1) Incredible Years Online Workshops and Video Streaming; and 2) Incredible Years Resources for Group Leaders

Parent-Child Interaction Therapy (a program designed for children and their parents that focuses on decreasing child behavior problems, increasing positive parent behaviors, and improving the quality of the parent-child relationship).
For COVID-related resources, go to:
PCIT COVID-19 Therapist and Trainer Resources
Resources for Parents

Promoting First Relationships (a training program for promoting secure and healthy relationships between caregivers and young children birth to three years of age).
The links below provide COVID-related telehealth resources:
Department of Health and Human Services
Michigan Association for Infant Mental Health
ACEs Connection

Promoting Health Among Teens! (Abstinence Only) (a program to to reduce risky sexual behavior for African American youth by providing information on sexual risks and helping teens build negotiation, refusal, and problem solving skills for practicing abstinence).
For COVID-related resources, go to: Virtual Training of Educators

SPORT and InShape Prevention Plus Wellness (a health promotion program that highlights the positive image benefits of an active lifestyle to reduce the use of alcohol, tobacco, and drug use by high school students).
For COVID-related resources, go to: Prevention Plus Wellness Remote Implementation

Strengthening Families Program: For Parents and Youth 10-14 (a group program that aims to promote good parenting skills and positive family relationships; reduce aggressive, hostile behavior, and substance abuse in adolescence; and improve family relationships).
For COVID-related resources, go to: Strengthening Families COVID-19 Resources

Tools of the Mind (an early childhood program designed to promote academic learning and prosocial behaviors by supporting the development of executive function and other regulation-related skills).
The links below provide COVID-related resources:
Tools COVID-19 Resource Center
Tools @ Home for Parents
Preparing for Virtual Workshops & Technical Assistance (TA)

Triple P System (a public health approach to reach all parents in a community to enhance parental competence and prevent or alter dysfunctional parenting practices, thereby reducing family risk factors both for child maltreatment and for children’s behavioral and emotional problems.)
Triple P America delivered several webinars on delivering Triple P remotely and using Triple P Online. Recordings of these webinars are available by emailing: contact.us@triplep.net. Also, these links provide free resources developed in response to COVID:
Parenting During COVID-19
COVID-19 Module

Year Up (a training and internship program that helps young people with limited post-secondary education get high-quality jobs by learning to work with technology, developing employment skills, and obtaining internships).
Online COVID-related articles:
Forbes: Year Up’s Shift To Virtual Operations
New York Times: Gaining Skills Virtually to Close the Inequality Gap

Blueprints Virtual Talks

Drs. Pamela Buckley and Karl Hill gave a talk at the Prevention Science Methodology Group Virtual Grand Rounds on Addressing Health Equity and Social Justice within Prevention Registries. The webinar first presented an overview of the Blueprints registry, and then discussed concerns regarding adaptation/cultural relevance that registry staff and users encounter with increasing frequency. Next, an overview of a recently funded project was presented that begins to address these concerns within Blueprints. While lack of representation of youth of color in health-related research studies has been well-documented, a critical evaluation of this omission has not been undertaken to substantiate this claim. This new project involves reviewing the scope of extant research on representation of ethnic minority groups in preventive intervention research, thus serving as a vehicle for decision-making regarding the generalizability of evidence-based interventions listed on clearinghouse websites (such as Blueprints). Go here to download slides and view the webinar.

Dr. Pamela Buckley participated in a panel at the 36th Annual San Diego International Conference on Child and Family Maltreatment presented by the Chadwick Center for Children and Families at Rady Children’s Hospital-San Diego. The virtual conference was designed to equip and support professionals in the child welfare field to effectively identify and address child maltreatment, trauma, and violence. The presentation, titled Child Welfare And Evidence Registries: Comparative Analysis, explained the Title IV-E Prevention Services Clearinghouse, which was developed for the Family First Prevention Services Act, and compared it to the California Evidence-Based Clearinghouse (CEBC) and Blueprints for Healthy Youth Development. The purpose was to discuss how registries can inform decision-making in the adoption of evidence-based interventions specific to the child welfare system. The panel discussion can be viewed online by clicking here.

Blueprints Publications

In case you have missed them, here are a few papers published by staff and Advisory Board members of the Blueprints team:

Combs et al. (2021) conducted a study that explored whether teachers trained online have similar levels of fidelity of implementation compared to teachers trained in-person on the Botvin LifeSkills Training (LST) middle school program (a Blueprints Model Plus intervention). Findings indicate that online training builds competencies important for school-based EBI implementation, while also highlighting potential shortcomings related to quality of delivery. The authors conclude that ensuring the inclusion of experiential learning activities (e.g., practice delivering content, receiving feedback on delivery) may be key to quality of delivery as online trainings for facilitators of school-based EBIs evolve. Go here to read the full study (published as an open-access article).

Blueprints promotes only those interventions with the strongest scientific support. Blueprints staff recently published a paper that describes the common methodological problems in randomized control trials that prevent interventions from being certified by Blueprints (Steeger, Buckley et al., 2021). Read more here.

Blueprints staff and board members published a paper that examined the prevalence of transparent research practices for studies reviewed by Blueprints from 2018 through 2019 (Buckley et al., 2021). Specifically, the study examined the rate of 1) public availability of data, code and research materials used to conduct confirmatory research, 2) prospective registration or registration before data analysis, and 3) discrepancies between confirmatory research reported in the trial registration (i.e., registered primary outcomes) and those included in articles (i.e., published primary outcomes). Findings suggest that: 1) preventive intervention research needs to be more transparent, and 2) since clearinghouses rely on robust findings to make well-informed decisions and researchers are incentivized to meet clearinghouse standards, clearinghouses should consider policies that encourage transparency to improve the credibility of evidence-based interventions. Go here to read the paper. The preprint (full copy report) can be read by clicking here.

A full list of Blueprints’ publications can be found on our website by clicking here.

Our mailing address is:
University of Colorado Boulder | Institute of Behavioral Science
483 UCB, Boulder, CO 80309

Drs. Pamela Buckley and Karl Hill (Blueprints PI and Co-PI) gave a talk at the Prevention Science Methodology Group Virtual Grand Rounds on Addressing Health Equity and Social Justice within Prevention Registries

The webinar first presented an overview of the Blueprints registry, and then discussed concerns regarding adaptation/cultural relevance that registry staff and users encounter with increasing frequency. Next, an overview of a recently funded project was presented that begins to address these concerns within Blueprints. While lack of representation of youth of color in health-related research studies has been well-documented, a critical evaluation of this omission has not been undertaken to substantiate this claim. This new project involves reviewing the scope of extant research on the representation of ethnic minority groups in preventive intervention research, thus serving as a vehicle for decision-making regarding the generalizability of evidence-based interventions listed on clearinghouse websites (such as Blueprints). Go here to download slides and view the webinar: http://cepim.northwestern.edu/calendar-events/2021-05-18-buckley-hill

Issue No. 16

Leadership Letter:

The Importance of Dissemination Readiness

Did you know that Blueprints for Healthy Youth Development lists dissemination readiness as one of four formal requirements for meeting its standard of evidence? This means that, before an intervention is listed as Blueprints-certified, we verify that there is a clear description of its activities and explicit implementation materials or procedures. Blueprints’ other standards are intervention specificity, evaluation quality, and intervention impact; the dissemination readiness dimension is evaluated only for interventions that satisfy these first three requirements.

According to Blueprints’ dissemination readiness criteria, well-evaluated interventions must have materials or instructions that specify the intervention content and guide the implementation of the intervention, such as manuals, training, and technical assistance, and (where available) specify costs associated with implementation (such as those for start-up, implementation, and support) and staff resources (for example, staff qualifications and time commitments) needed to deliver the intervention with fidelity.

Thus, interventions that meet the Blueprints standard of evidence must be available for use.

Blueprints certifies programs that meet all four of its standards (intervention specificity, evaluation quality, intervention impact, and dissemination readiness) as Model/Model Plus or Promising.

We recently conducted a study to determine whether online clearinghouses rate and provide information about the degree to which well-evaluated programs can be well-implemented. Our goal was to fill a knowledge gap by examining if and how the dissemination readiness of evidence-based interventions has been defined, evaluated, and communicated to potential users. To do so, we compared 11 different U.S.-based online private and federally funded clearinghouses on (1) the degree to which they use dissemination readiness as a criterion for inclusion/exclusion of evidence-based interventions, and (2) the extent of information and support they provide about dissemination readiness to facilitate real-world implementation.

We found wide variability, with few having standards about dissemination readiness or making information about implementation of evidence-based interventions easily accessible to users. Across the 11 clearinghouses we examined, only Blueprints has both a requirement for dissemination readiness and provides relatively complete information relevant to dissemination readiness.

Our findings indicate the need for online clearinghouses to (1) do more to assess dissemination readiness and (2) offer more complete information on dissemination readiness and implementation support to users. Doing so, however, requires a commitment from funders (who have finite resources) to invest in the dissemination readiness of evidence-based interventions. That is, funding is needed for developers to provide robust dissemination materials and time is needed for clearinghouse staff to collect this information. We therefore want to recognize the Annie E. Casey Foundation, which provided funding to Blueprints to develop the tools we currently use to gather information on dissemination readiness and historically paid contractors to collect this information as part of its Evidence2Success initiative.

Why Conduct this Study?

This is important work to share because we want to convey to users our process for listing on our website interventions rated as Model/Model Plus or Promising. If an intervention does not meet most or any of Blueprints dissemination readiness criteria but has a high-quality design, that intervention will receive a “not dissemination ready” rating – meaning it has met criteria for evaluation quality (as determined by the Blueprints Advisory Board) but has not yet met the dissemination readiness criteria. This could be one reason why certain interventions are identified with an effective rating on other clearinghouses but not listed as a Blueprints-certified intervention on our website.

If you are interested in reading more, the full study was published in Evaluation Review, a peer-reviewed academic journal that aims to advance the practice of evaluation and to publish the results of high-quality evaluations. The abstract and full text have been published as an open-access article and can be accessed through the publications page of the Blueprints website as well as on PubMed.gov.

Recognizing These Challenging Times

In closing, we cannot ignore that many communities, families, and individuals continue to struggle. With a global pandemic, protests and riots in parts of the United States and world, divisive elections, online learning and remote work, and – for some parts of the US – extreme weather conditions, this past year will go down as one of the most devastating in modern history. As we all continue to cope with and adapt to these realities, Blueprints will persist in working on our mission of promoting effective preventive interventions and helping governmental agencies, foundations, and practitioners use strong evidence to make informed decisions when identifying interventions that provide a high probability of success when taken to scale in communities. It is this commitment that helps us persevere through these challenging times. We thank each one of you for your part in our community and for your involvement in this important effort.

Sincerely,

Pamela Buckley, PhD
Director and Co-Principal Investigator
Blueprints Initiative
Institute of Behavioral Science
University of Colorado Boulder
Pamela.Buckley@Colorado.edu

Karl G. Hill, PhD
Principal Investigator
Blueprints Initiative
Institute of Behavioral Science
University of Colorado Boulder
Karl.Hill@Colorado.edu

Reference: Buckley, P. R., Fagan, A., Pampel, F. & Hill, K. (2020). Making evidence-based interventions relevant for users: A comparison of requirements for dissemination readiness across program registries. Evaluation Review, 44(1), 51-83. https://doi.org/10.1177/0193841X20933776

Featured Model Plus Program
GenerationPMTO

Blueprints Certified: 2020 (Moved from Model to Model Plus)

Ages Served: Early Childhood (3-4) – Preschool, Late Childhood (5-11) – K/Elementary, Early Adolescence (12-14) – Middle School, Late Adolescence (15-18) – High School

Program Outcomes: Antisocial-aggressive Behavior, Conduct Problems, Delinquency and Criminal Behavior, Emotion Regulation, Externalizing, Internalizing, Positive Social/Prosocial Behavior.

A family training program that aims to teach effective family management skills in order to reduce antisocial and problematic behavior in children through trainings implemented in a variety of formats and settings.

Learn more > >

Featured Promising Program
Bounce Back

Blueprints Certified: 2020

Ages Served: Late Childhood (5-11) – K/Elementary

Program Outcomes: Anxiety, Emotional Regulation, Post Traumatic Stress Disorder (PTSD).

Bounce Back is a school- and group-based program designed to improve symptoms of posttraumatic stress, depression, and anxiety among children with posttraumatic stress symptoms.

Learn more > >

Blueprints Interventions in the News
In case you have missed them, here are a few newspaper articles and web postings that feature activities of some Blueprints-certified interventions:

Child First (a Blueprints promising program) is a home visitation program that works to heal and protect young children and their families from the devastating effects of chronic stress and trauma. With a $700,000 grant awarded in 2018, The Duke Endowment is supporting a randomized controlled trial of Child First in eastern North Carolina. Arnold Ventures is providing funding to study Child First in Connecticut. MDRC is conducting research in both states and Meghan McCormick (an MDRC research associate) explains more about evaluating Child First and about finding a silver lining in COVID-19. Go here to read the full interview.
Big Brothers Big Sisters of America (BBBSA) is a Blueprints promising program that matches an adult volunteer mentor with an at-risk child or adolescent to delay or reduce antisocial behaviors and improve academic success. Emily Johnson, CEO of BBBS of Southwest Idaho, says their chapter is matching kids with adult mentors based on compatibility. Go here to learn more about the procedures BBBS of Southwest Idaho is following to help mentors and mentees interact while ensuring the safe continuity of programming during the COVID-19 outbreak.
The Communities That Care (CTC) chapter of West Chester (a township in Pennsylvania) is providing resources for parents to help them deal with the challenges the COVID-19 pandemic has presented. CTC (a Blueprints promising program) is a prevention system designed to reduce levels of adolescent delinquency and substance use through the selection and use of effective preventative interventions. The CTC chapter of West Chester has dedicated its 2021 Parent Speaker series to the topic of emotional health to keep parents connected, and to offer concrete solutions concerning today’s parenting challenges. To learn more about the speaker series and the efforts being made by the West Chester CTC chapter, go here.
The Pace Center for Girls in Alachua (a county in Florida) recently received a $5,000 Golden Grant award from the North Florida McDonald’s Operators Association. Pace Center for Girls (a Blueprints Promising program) is a set of gender-responsive prevention and early intervention programs for girls with multiple risk factors for juvenile justice system involvement, which uses a holistic approach to re-engage girls with learning, improve academic performance, and address underlying trauma. Natalya Bannister, the executive director for Pace Center for Girls Alachua, says the program was vulnerable due to the COVID-19 pandemic and that the funds will go towards supporting their reading initiatives. Go here to learn more.

Our mailing address is:
University of Colorado Boulder | Institute of Behavioral Science
483 UCB, Boulder, CO 80309

Bounce Back

A school- and group-based program designed to improve symptoms of posttraumatic stress, depression, and anxiety among children with posttraumatic stress symptoms.

Bounce Back is comprised of 10 one-hour group sessions of four to six students, two to three individual sessions, and one to three parent education sessions that span a 3-month period. Typically held during school hours, group sessions cover a range of topics, such as relaxation training, cognitive restructuring, social problem solving, positive activities, trauma-focused intervention strategies, and emotional regulation and coping skills. These topics and methods derive from established successful interventions for children with PTSD, including a gradual approach of anxiety-provoking situations and a modified trauma narratives approach.

Program elements designed specifically to function with participants aged 5-11 include identifying feelings and their links to thoughts and actions, using published storybooks to relate concepts and connect engagement activities, and creating personal storybooks as an age appropriate concrete trauma narrative. Student participation is encouraged with games and activities specific to age groups and with “courage cards” tailored to each student. Group sessions are structured, and include agenda setting; review of activity assignments; introduction of new topics through games, stories, and experiential activities; and assigning activities for the next group meeting.

Blueprints has certified two studies evaluating Bounce Back.

In Study 1, Langley et al. (2015) randomly assigned 74 students within four different schools to either immediate intervention or a waitlist control group. Parent- and child-reports of posttraumatic stress and depression, and child reports of anxiety symptoms, were assessed at baseline and three months after baseline (posttest). Compared to the control group at the posttest, treatment students significantly improved on posttraumatic stress symptoms (parent and child reported), anxiety symptoms (child reported), and emotional regulation as well as emotional/behavioral problems (both parent reported). Additionally, a protective factor, self-reported social adjustment, significantly improved for children who received the intervention as compared to children in the control group.

In Study 2, Santiago et al. (2018) conducted a cluster randomized controlled trial in which eight schools and 52 students were assigned to immediate intervention or a waitlist control group. Measures assessing PTSD symptoms, anxiety, depression, coping skills and classroom behavior were collected before the intervention (pretest) and three months after baseline (posttest). Study 2 reported that at the posttest, compared to the control group, students in the treatment group showed improvements in posttraumatic stress symptoms (child reported). In terms of risk and protective factors, the study reported that treatment students, as compared to control students, showed improvements in coping (parent reported).

References:

Langley, A. K., Gonzalez, A., Sugar, C. A., Solis, D., & Jaycox, L. (2015). Bounce Back: Effectiveness of an elementary school-based intervention for multicultural children exposed to traumatic events. Journal of Consulting and Clinical Psychology, 83(5), 853-865.

Santiago, C. D., Raviv, T., Ros, A. M., Brewer, S. K., Distel, L. M., Torres, S. A., . . . Langley, A. K. (2018). Implementing the Bounce Back trauma intervention in urban elementary schools: A real-world replication trial. School Psychology Quarterly, 33(1), 1.

Read the Program Fact Sheet

GenerationPMTO

GenerationPMTO is a group of theory-based parent training interventions that can be implemented in a variety of family contexts. The program aims to teach effective family management skills in order to reduce antisocial and problematic behavior in children ranging in age from 3-16 years. GenerationPMTO is delivered in group- and individual-family formats, in diverse settings (e.g., clinics, homes, schools, community centers, homeless shelters), and over varied lengths of time depending on families’ needs. Typically, sessions are one week apart to optimize the opportunity for learning and rehearsing new practices. The number of sessions provided in parent groups ranges from 6 to 14; in clinical samples the mean number of individual treatment sessions is 25.

The central role of the GenerationPMTO therapist is to teach and coach parents in the use of effective parenting strategies, namely skill encouragement, setting limits or effective discipline, monitoring, problem solving, and positive involvement. In addition to these core parenting practices, parents are taught to identify and regulate emotions, enhance communication, give clear directions, and track behavior. Finally, the program weaves the promotion of school success throughout relevant components.

GenerationPTMO has experienced widespread implementation across several states and international locations. The Blueprints certification of GenerationPMTO is based on four studies – two conducted in the United States, one in Norway, and one in Iceland. An independent research team (not associated with the program developers) conducted the most recently certified study; with this last evaluation, the program meets the requirements for “Model Plus” status on the Blueprints registry.

Forgatch & DeGarmo (1999) randomized 238 recently separated mothers and their sons in grades 1-3 to the intervention (n=153) or to a non-intervention control group (n=85). The participants lived in a moderate-sized city in the Northwest. Results show that compared to the control group at posttest, intervention boys experienced a significant decrease in noncompliance; their mothers experienced reductions in coercive parenting and negative reinforcement and increases in positive parenting, effective parenting practices and adaptive functioning. At 30 months, child internalizing and externalizing behaviors decreased as did maternal depression. At the last follow-up, the intervention significantly reduced the 9-year average and rate of growth in teacher-reported delinquency. Additionally, assignment to the intervention was associated with reduction in average levels of deviant peer associations from baseline to 8 years. Intervention boys and their mothers experienced lower arrest rates and boys experienced delayed age at first arrest compared to the control group.

Bjørknes & Manger (2012) and Bjørknes et al. (2012) conducted a randomized control trial among 96 immigrant families in Norway. Eligible families had a child with or at risk of developing conduct problems. Mothers were randomly assigned to either the intervention (n=50) or to a waitlist control group (n=46) with assessments occurring at baseline and post-intervention. The program improved positive parenting practices and child conduct problems for intervention families as compared to control families.

Sigmarsdóttir et al. (2014) recruited participants from five municipalities in Iceland. Eligible families were referred by schools, educational services or social services and had a child displaying behavioral problems at home and/or in school. Families were randomly assigned to the intervention (n=51) or to a services-as-usual control condition (n=51) in which subjects received a variety of community services normally provided for children with behavioral problems. Parents, teachers, and children completed assessments at baseline and end of treatment (approximately one year after baseline). A significant reduction was found for child adjustment problems (construct comprised of behavior problems, social skills, and depressive symptoms) among children whose parents received the treatment, compared to children in the control condition.

A team of independent evaluators, Akin et al. (2016, 2018), conducted a randomized controlled trial in a Midwestern state with families of children who were in foster care and had a serious emotional disturbance. In this study, 918 families were randomized into intervention (n=461) and control (n=457) groups. Assessments occurred at baseline, posttest, and 6-month follow-up (12 months after baseline). Compared to control group participants at posttest and follow-up, children in the intervention group significantly improved on social emotional functioning, problem behaviors and social skills.

In terms of cost-benefit analysis, Washington State Institute for Public Policy (December 2019) reports $1.87 in measured benefits per $1 spent in implementing GenerationPMTO.

References:

Akin, B. A., Lang, K., McDonald, T. P., Yan, Y., & Little, T. (2016). Randomized trial of PMTO in foster care: Six-month child well-being outcomes. Research on Social Work Practices, 29(2), 206-222.

Akin, B. A., Lang, K., Yan, Y., & McDonald, T. P. (2018). Randomized trial of PMTO in foster care: 12-month child well-being, parenting, and caregiver functioning outcomes. Children and Youth Services Review, 95, 49-63.

Bjørknes, R., & Manger, T. (2012). Can parent training alter parent practices and reduce conduct problems in ethnic minority children? A randomized controlled trial. Prevention Science, 14(1), 52-63.

Bjørknes, R., Kjøbli, J., Manger, T., & Jakobsen, R. (2012). Parent training among ethnic minorities: Parenting practices as mediators of change in child conduct problems. Family Relations, 61, 101-114.

DeGarmo, D. S., Patterson, G. R., & Forgatch, M. S. (2004). How do outcomes in a specified parent training intervention maintain or wane over time? Prevention Science, 5, 73-89.

Forgatch, M., & DeGarmo, D. (1999). Parenting Through Change: An effective prevention program for single mothers. Journal of Consulting and Clinical Psychology, 67(5), 711-724.

Forgatch, M. S., Patterson, G. R., DeGarmo, D. S., & Beldavs, Z. (2009). Testing the Oregon delinquency model with 9-year follow-up of the Oregon Divorce Study. Development and Psychopathology, 21(5), 637-660.

Martinez, C., & Forgatch, M. (2001). Preventing problems with boys’ noncompliance: Effects of a parent training intervention for divorcing mothers. Journal of Consulting and Clinical Psychology, 69(3), 416-428.

Sigmarsdóttir, M., Thorlacius, O., Guõmundsdóttir, E. V., & DeGarmo, D. S. (2014). Treatment effectiveness of PMTO for children’s behavior problems in Iceland: Child outcome in a nationwide randomized controlled trial. Family Process, 54, 498-517.

Read the Program Fact Sheet

Blueprints Director Dr. Pamela Buckley presented at the International Conference on Child and Family Maltreatment virtual conference

Dr. Pamela Buckley (Director of Blueprints) participated in a panel at the 36th Annual San Diego International Conference on Child and Family Maltreatment presented by the Chadwick Center for Children and Families at Rady Children’s Hospital-San Diego. The virtual conference was held January 23-29, 2021 and was designed to equip and support professionals in the child welfare field to effectively identify and address child maltreatment, trauma, and violence. The presentation, titled “Child Welfare And Evidence Registries: Comparative analysis”, explained the Title IV-E Prevention Services Clearinghouse, which was developed for the Family First Prevention Services Act, and compared it to the California Evidence-Based Clearinghouse (CEBC) and Blueprints for Healthy Youth Development. The purpose was to discuss how registries can inform decision-making in the adoption of evidence-based interventions specific to the child welfare system. In addition to Dr. Buckley, the panel discussion included Dr. Suzanne Kerns and Dr. Sandra Jo Wilson from the Prevention Services Clearinghouse and Jennifer Rolls-Reutz from the CEBC. It started with an overview of the Prevention Services Clearinghouse, followed by a comparative analysis of each registry’s standards and review processes. The panelists then facilitated a discussion around the role of registries in supporting advancements in implementation science specific to child welfare involved families. The panel discussion was available on demand throughout the conference, and can be viewed online by clicking this link.

Issue No. 15

Welcome to the Blueprints Bulletin

Leadership Letter:

On March 11, 2020, the coronavirus outbreak was declared a pandemic by the World Health Organization. As noted in our last newsletter and in publications and experiences around the world, COVID-19 has touched all aspects of life and caused many disruptions, including the cancellation of the 2020 Blueprints Conference (which is held every other year and brings together evidence-based program professionals and advocates from around the world). While we continue to navigate the uncertainty of planning large events during these troubling times, we remain committed to identifying interventions that have the highest standards for promoting education, good behavior, emotional well-being, physical health, and positive relationships.

A debate on evidence-based approaches for guiding juvenile justice programming

One of the tracks planned for the 2020 Blueprints Conference was titled “successful scale-up frameworks,” in which sessions would focus on broad dissemination to ensure the impact of Blueprints-certified interventions on a large scale. Though unable to host these conversations to date, this newsletter summarizes a related debate featured in a recent issue of Criminology & Public Policy (CPP). As CPP is a peer-reviewed academic journal covering criminology and its implications for public policy, the debate focuses on evidence-based approaches for guiding juvenile justice programming and assessing the system-level effects on recidivism as reforms are implemented and evidence-based interventions are adopted.

Featured first is an essay led by Blueprints founder and board member, Dr. Del Elliott, in which several members of the Blueprints leadership team serve as co-authors. Titled “Evidence-based juvenile justice programs and practices: A critical review,” the full-text article can be read here: Elliott, Buckley, Gottfredson, Hawkins & Tolan (2020). The focus is on interventions rated as “model,” an evidence-level that Blueprints specifically notes is required for scale-up. The Blueprints model standard requires: 1) documentation of the intervention activities targeting risk or protective factors theoretically linked to change a behavioral youth outcome, 2) one or more high quality randomized controlled designs (RCTs), 3) an experimental replication, 4) sustainability of effects for a minimum of one year post intervention, 5) no evidence from high-quality evaluations of harmful effects, and 6) the organizational capacity to provide information for potential users to adopt and implement the intervention with fidelity. While less consensus exists regarding the definition of evidence-based practices, they typically involve generic types of strategies often informed by meta-analysis that share core components but do not involve the same detailed package of prescribed activities as with Blueprints-rated “model” interventions. Elliott and colleagues contend that we should invest more heavily in model interventions and treat generic practices as a complementary approach in need of further rigorous evaluation.

Dr. Mark Lipsey of Vanderbilt University authored the second essay on this debate. Titled “Effective use of the large body of research on the effectiveness of programs for juvenile offenders and the failure of the model programs approach”, the abstract is located here (Lipsey, 2020) and the full-text article can be read by paid subscribers to CPP. In his essay, Lipsey explains the development of the Standardized Program Evaluation Protocol (SPEP), which he promotes as an evidence-based practice intervention. Lipsey describes SPEP as an assessment based on a meta-analysis he conducted in 2009 involving four program characteristics treated as core components in an evidence-based intervention that are predictive of favorable recidivism effects. SPEP is offered to juvenile justice systems as a set of practice guidelines for improving their programming, with the claim that increasing compliance levels across these four characteristics (i.e., their SPEP score) will increase effectiveness.

Both essays agree that juvenile justice interventions should be supported by credible evidence of effectiveness.

The third essay is written by Dr. Brandon Welsh of Northeastern University, who cites that a key criticism of evidence-based interventions (whether they are model programs or generic practices) is the limited research demonstrating their ability to achieve population impacts. In this paper, titled “The case for rigorous comparative research and population impacts in a new era of evidence-based interventions for juvenile offenders” (the abstract can be read here), Dr. Welsh notes that greater attention to both approaches is necessary in order to have lasting impacts in reducing recidivism rates. He also cites the importance of frameworks such as the Communities That Care (CTC) model, which aims to reduce delinquency and later offending by implementing particular interventions—from a menu of strategies—that have demonstrated effectiveness in reducing risk factors and enhancing protective factors. Though not mentioned in Welsh’s essay, CTC is rated as “promising” on the Blueprints registry (click here to review the Blueprints write-up of CTC).

We hope that you enjoy this lively debate. We also wish you safety and health during this holiday season.

Sincerely,

Pamela Buckley, PhD
Director and Co-Principal Investigator
Blueprints Initiative
Institute of Behavioral Science
University of Colorado Boulder
Pamela.Buckley@Colorado.edu

Karl G. Hill, PhD
Principal Investigator
Blueprints Initiative
Institute of Behavioral Science
University of Colorado Boulder
Karl.Hill@Colorado.edu

Blueprints for Healthy Youth Development is hosted by the University of Colorado Boulder, Institute of Behavioral Science, with current funding from Arnold Ventures and former funding from the Annie E. Casey Foundation. Each intervention included in the Blueprints database has been reviewed carefully by an independent advisory panel that looked at research on the intervention’s impact, practical focus, and potential for implementation in public systems.

Blueprints Year in Review
We accomplished a lot in 2020!

(1) We are proud to report that we reviewed 51 unique interventions, 142 individual articles or reports and certified seven interventions. Many of the articles we reviewed in 2020 came to our attention through published scientific studies that we found in comprehensive searches of the evaluation literature, databases, and journals. In addition, we searched blogs, the internet, and other registries to locate evaluation studies. We also reviewed interventions by a request from developers or evaluators (click here to nominate an intervention). (2) We expanded our social media presence from Facebook and Twitter to also include Instagram. (3) We planned two meetings for the Blueprints Advisory Board, a distinguished panel of methodological experts with a variety of content expertise (one Board meeting in 2020 was canceled due to COVID-19). (4) We launched features on our website to (a) assist with downloading and saving searches on Blueprints-certified interventions, and (b) allow users to export searches of interventions reviewed by Blueprints that did not meet our certification standards (click here). (5) We conducted an online survey in collaboration with our friends at Evidence-Based Associates on Model/Model Plus and Promising Programs’ COVID-19 response (the full survey and aggregated responses can be downloaded by clicking here). (6) We published 2 academic, peer-reviewed papers – one of which is highlighted in this newsletter (see Blueprints publications listed on our website here). And (7) we planned for the 2020 Blueprints biennial conference (cancelled due to COVID-19).

In addition, members of the Blueprints advisory board accomplished a lot in 2020! (1) Dr. Karl Hill led a webinar titled “Why Use Evidence and Where to Find It: Blueprints for Healthy Youth Development” (hosted by the Northwest Prevention Technology Transfer Center). The webinar discussed the importance of employing evidence and prevention science in substance misuse prevention work. Click here to view the recorded webinar. (2) Dr. Frances Gardner collaborated with the World Health Organization, UNICEF, the Global Partnership to End Violence Against Children, the Internet of Good Things, USAID and Centers for Disease Control and Prevention to develop a website with open-access online parenting resources during COVID-19. Go here to learn more. (3) Dr. Larry Hedges published a working paper with a colleague from Northwestern University titled “Addressing the Challenges to Educational Research Posed by COVID-19,” which discussed how the COVID-19 pandemic has impacted the conduct of ongoing research, especially randomized field trials. The paper seeks to identify some of the problems that may arise because of this disruption and identify possible responses to the disruption. Read the full working paper here. (4) Dr. Abigail Fagan was elected to the board of the Society for Prevention Research (SPR), an organization dedicated to conducting and translating prevention research that promotes health and well-being. She will serve a one-year term (June, 2020 through June, 2021) as President-Elect, after which she will serve a two-year term (June, 2021 through June, 2023) as President. (5) Dr. Velma McBride Murry, Dr. Patrick Tolan, and Dr. Abigail Fagan were recognized by SPR for making a significant impact in the field of prevention science. Dr. Murry and Dr. Tolan both received the “Advances in Culture and Diversity in Prevention Science Award” and Dr. Fagan received the “Service to SPR” Award. And (6) Dr. Velma Murry was elected to the National Academy of Medicine, which provides trusted scientific advice.

2020 in Review
By the numbers

Below is a summary of our year in review.

Interventions Reviewed & Certified in the Past Year

• Number of Unique Interventions Reviewed = 51
• Number of Individuals Articles or Reports Reviewed = 142
• Number of Interventions Certified = 7

Learning Together (Promising)
Added Jan. 9, 2020
Learn more

Year Up (Promising)
Added Feb.14, 2020
Learn more

Tools of the Mind (Promising)
Added: May 19, 2020
Learn more

Bounce Back (Promising)
Added: Jul. 22, 2020
Learn more

Pace Center for Girls (Promising)
Added: Sep. 24, 2020
Learn more

GenerationPMTO (Moved from Model to Model Plus)
Added: Sep. 24, 2020
Learn more

Accelerated Study in Associate Programs (Model Plus)
Added: Dec. 2, 2020
Learn more

Featured Model Program
Accelerated Study in Associate Programs (ASAP)

Blueprints Certified: 2020
Ages Served: Early Adulthood (19-22) – The program targets low-income community college students.
Program Outcomes: Academic Performance, Post-secondary Education

A post-secondary college-based prevention program that aims to address potential barriers to academic success and promote credit accumulation and associate degree completion in college students through comprehensive advisement and career and tutoring services provided by dedicated advisers.

Learn more > >

Featured Promising Program
Pace Center for Girls

Blueprints Certified: 2020
Ages Served: Early Adolescence (12-14) – Middle School; Late Adolescence (15-18) – High School
Program Outcomes: Academic Performance, Truancy – School Attendance

Learn more > >

Blueprints Interventions in the News
In case you have missed them, here are a few newspaper articles and web postings that feature efforts of some of our Blueprint’s Model/Model Plus and Promising Programs in helping communities facing challenges stemming from the COVID-19 pandemic:

The Nurse-Family Partnership Program (a Blueprints Model program) continues to help new mothers navigate pregnancy and parenthood amid #COVID19. NFP is a nurse home visiting program for first-time pregnant mothers designed to improve prenatal and child rearing practices through the child’s second birthday. Go here to learn more.
Big Brothers Big Sisters of America (BBBSA) is a Blueprints Promising program that matches adult volunteer mentors with an at-risk child or adolescent to reduce antisocial behavior and improve academic success. The CEO of BBBS of East Tennessee says their mission is critical amid #COVID19; a recent news feature reported more than 100 children on their wait list. Learn more here.
The Education Policy Institute (EPI) says the learning gap between rich and poor students of primary school age in England has widened for the first time since 2007, suggesting #COVID19 has had a major impact on the education system. To help combat the disruption, the UK Department for Education created a “COVID-19 Catch-Up Premium,” which includes funding for the Nuffield Early Language Intervention (a Blueprints Promising program). Go here to learn more.

Our mailing address is:
University of Colorado Boulder | Institute of Behavioral Science
483 UCB, Boulder, CO 80309

Pace Center for Girls

A set of gender-responsive prevention and early intervention programs and services for girls with multiple risk factors for juvenile justice system involvement, which uses a holistic approach to re-engage girls with learning, improve academic performance, and address the underlying trauma that contributes to female delinquency.

Pace Center for Girls (Pace) encompasses a set of gender-responsive prevention and early intervention programs and services for girls with multiple risk factors for juvenile justice system involvement including academic failure, chronic truancy, and dropping out of school. Pace uses a balanced, holistic approach to re-engage girls with learning, improve academic performance, and to address the underlying trauma that contributes to female delinquency. The Pace model provides the full academic school day and social service interventions in a safe, trauma-informed, strengths-based environment that reflects an understanding of the lives of girls and responds to their unique needs and challenges.

During a typical day at Pace, girls attend daily core academic classes, including language arts, math, social studies, life skills, and science. Every other week, or as needed, girls attend individual and group counseling sessions that focus on fostering positive behavioral change. Weekly, or as needed, girls also attend academic advising sessions to plan and monitor academic progress.

Blueprints has certified one study evaluating Pace Center for Girls.

Millenky et al. (2019) conducted a randomized control trial involving 14 Pace Centers in the state of Florida. A total of 1,125 girls who applied to the program were randomized within center to either a treatment group that received Pace services or to a control group whose members received referrals to other community services. At the end of one year (the typical length of the program), significant positive impacts were found on enrollment, attendance, credits earned and suspensions for the treatment as compared to the control girls.

References:

Millenky, M., Treskon, L., Freedman, L., & Mage, C. (2019). Focusing on girls’ futures: Results from the evaluation of PACE Center for Girls. MDRC.

Read the Program Fact Sheet

Author: Tom

Drs. Pamela Buckley and Karl Hill (Blueprints PI and Co-PI) gave a talk at the Prevention Science Methodology Group Virtual Grand Rounds on Addressing Health Equity and Social Justice within Prevention Registries

Bounce Back

GenerationPMTO

Blueprints Director Dr. Pamela Buckley presented at the International Conference on Child and Family Maltreatment virtual conference

Pace Center for Girls

Contact

Sign up for Newsletter