A cognitive behavioral multi-component training program designed to reduce antisocial behavior and/or police contact among boys at risk for such engagement by decreasing the factors that make children susceptible for continued delinquency and strengthening the protective factors of the parents, the child, and the family structure.
Blueprints: Promising
Crime Solutions: Effective
OJJDP Model Programs: Effective
Child Development Institute (CDI)
197 Euclid Ave.
Toronto, Ontario
M6J 2J8, CANADA
Phone: 416-603-1827
SNAP@childdevelop.ca
www.childdevelop.ca
www.stopnowandplan.com
Leena Augimeri, Ph.D.
Child Development Institute (CDI)
SNAP (Stop Now And Plan) was originally called the Under 12 Outreach Project (ORP), a non-gender-specific program established in 1985 to meet a service gap for children in conflict with the law. Today, SNAP is an evidence-based children's mental health and crime prevention model consisting of two gender-specific programs: SNAP® Boys (discussed here), and SNAP® Girls. The SNAP® Boys program is a 13-week, multi-component treatment program for boys aged 6-11 years who have had police contact as a result of delinquency or who are at risk of police contact. The focus is on increasing emotion regulation, self-control and problem-solving skills that help to keep high-risk boys in school and out of trouble by making better choices 'in the moment,' working with parents/caregivers to support their children and enhancing their parent management skills. The program aims to decrease the factors that place children at risk for continued delinquency and strengthen the protective factors of the parents, the child, and the family structure. The intervention is aimed at the child, the family, and the child-in-the-community, which have been found to yield the most positive long-term preventative effects.
The SNAP® Boys program is a 13-week, multi-component treatment program for boys in the middle years (aged 6-11 years) who have had police contact as a result of delinquency or who are at risk of police contact. SNAP® Boys is based on a cognitive behavioral model and focuses on the development and treatment of aggression and antisocial behavior and consists of five core components adapted from established interventions for children with conduct problem behavior: (1) boys problem solving and self-control skills training group; (2) parent training group; (3) family counseling; (4) individual befriending; and (5) school support and reading tutoring. The primary objective of SNAP® Boys is to reduce police contact among a population which is at risk for engaging in criminal activity by decreasing the factors that place children at risk for continued delinquency and strengthening the protective factors of the parents, the child, and the family structure. The intervention is aimed at the child, the family, and the child-in-the-community, which have been found to yield the most positive long-term preventative effects.
Boys meet in structured group sessions lasting 90 minutes, once a week for thirteen weeks. Sessions are organized around the following topics: (a) Joining In/Goal Setting, (b) Stopping Stealing, (c) Rewarding Yourself, (d) Fair Play, (e) Dealing with Angry Feelings, (f) Group Pressure, (g) Avoiding Trouble, (h) When You're Not So Sure, (i) Apologizing, and (j) Dealing with Accusations/Being Blamed. Groups activities include guided discussions, modeling, role-playing, relaxation training, and a self-control technique called "SNAP", which stands for "Stop Now And Plan."
Parents are encouraged to participate in the thirteen week 90-minute sessions addressing social learning approaches to family management. Parenting skills covered in these sessions include: (a) stopping stealing, (b) routines, (c) listening, (d) monitoring, (e) self-control and problem-solving, (f) time-out, and (g) encouraging and rewarding. In addition, parents can access crisis intervention and individual family intervention, as well as phone calls or visitations by a Parent Group Leader to parents who did not attend the parent group. The Parent Group Leader reviews the content of the parent group with the parent and provides a written summary of the social skills and family management issues covered. Lastly, services such as counseling or attendance at the hosting center's twice monthly Parent Association Group are available after the parent group ends.
The portion of the intervention that addresses the child-in-the-community includes "individual befriending" by involving a SNAP® Boys staff member or volunteer in a mentor type of relationship (such as that of Big Brothers/Big Sisters) and linking the participant to resources in the community. Additionally, children may access academic tutoring and parents may receive help with school advocacy on an as-needed basis to ensure that their children receive the best possible education.
Primary Evidence Base for Certification
Study 3
Burke and Loeber (2014, 2016) found 12 months after program completion, relative to controls, SNAP Boys participants had a lower number of criminal charges reported in official criminal records and significantly lower scores on parent-ratings of:
Additionally, intervention boys had significantly higher scores on:
Primary Evidence Base for Certification
Of the three studies Blueprints has reviewed, one study (Study 3) meets Blueprints evidentiary standards (specificity, evaluation quality, impact, dissemination readiness). In addition, Study 3 was done by the developer.
Study 3
Burke and Loeber (2014, 2016) conducted a randomized controlled trial in which 252 boys were randomized to intervention or control groups. Assessments were completed at baseline, upon program completion, and six and 12 months after program completion (though some participants continued to receive SNAP services through the follow-up periods).
Study 3
Burke, J. D., & Loeber, R. (2014). The effectiveness of the Stop Now and Plan (SNAP) program for boys at risk for violence and delinquency. Prevention Science, 16, 242-253.
Burke, J. D., & Loeber, R. (2016). Mechanisms of behavioral and affective treatment outcomes in a cognitive behavioral intervention for boys. Journal of Abnormal Child Psychology, 44 (1), 179-189.
Individual: Antisocial/aggressive behavior*, Bullies others, Early initiation of antisocial behavior*, Favorable attitudes towards antisocial behavior, Gang involvement, Physical violence, Stress
Family: Neglectful parenting, Parental attitudes favorable to antisocial behavior, Parent stress*, Poor family management*, Psychological aggression/discipline, Violent discipline
School: Low school commitment and attachment
Individual: Coping Skills, Problem solving skills*, Prosocial behavior*, Prosocial involvement
Peer: Interaction with prosocial peers
Family: Attachment to parents, Nonviolent Discipline, Parental involvement in education, Parent social support, Rewards for prosocial involvement with parents
School: Rewards for prosocial involvement in school
*
Risk/Protective Factor was significantly impacted by the program
See also: SNAP® (Stop Now And Plan) Boys Logic Model (PDF)
Subgroup differences in program effects by race, ethnicity, or gender (coded in binary terms as male/female) or program effects for a sample of a specific racial, ethnic, or gender group:
Study 3 (Burke & Loeber, 2014, 2016) tested for subgroup effects by using a homogenous sample with 75% or more of African American males.
Sample demographics including race, ethnicity, and gender for Blueprints-certified studies:
SNAP Lead Staff Training
The lead staff training is conducted in person with Directors, Managers and Supervisors of the prospective SNAP Affiliate site and lasts for approximately 2-3 days. This training is designed to ensure that agency leadership and direct SNAP program Managers/Supervisors can support the implementation of the program within their agency and provide ongoing guidance and supervision of the process. At the end of this training a plan is confirmed with each agency for on-boarding staff, conducting core SNAP training, providing all required materials and equipment and to prepare to promote the program in their community and agency.
Training Outline
Module 1:
Module 2:
Module 3:
SNAP Core Staff Training
This training consists of up to 9 days of SNAP training for the staff team responsible for delivering the SNAP clinical program (SNAP Boys) along with the Manager/Supervisor responsible for this team. This training can be delivered in the community in its entirety or 5 days are delivered in community and the other days delivered online depending on the preference and needs of the SNAP Affiliate. In this training the participants learn about the principles and theories of SNAP, SNAP and other therapeutic skills, all SNAP service components, topics of the child and caregiver group sessions, how to use the SNAPiT online platform, and how to deliver the SNAP groups and the program with fidelity while continuing to build staff competencies and skills.
Training Outline
Day 1:
Day 2:
Day 3:
Day 4:
Day 5:
Day 6:
Day 7:
Day 8 & 9
Program Benefits (per individual):
$17,234
Program Costs (per individual):
$4,288
Net Present Value (Benefits minus Costs, per individual):
$12,947
Measured Risk (odds of a positive Net Present Value):
86%
Source: Washington State Institute for Public Policy
All benefit-cost ratios are the most recent estimates published by The Washington State Institute for Public Policy for Blueprint programs implemented in Washington State. These ratios are based on a) meta-analysis estimates of effect size and b) monetized benefits and calculated costs for programs as delivered in the State of Washington. Caution is recommended in applying these estimates of the benefit-cost ratio to any other state or local area. They are provided as an illustration of the benefit-cost ratio found in one specific state. When feasible, local costs and monetized benefits should be used to calculate expected local benefit-cost ratios. The formula for this calculation can be found on the WSIPP website.
Implementation Training for newly Licensed Affiliates is $22,000. Scheduled to be completed within a 3-month period, it includes:
Additional travel costs for training will depend on the SNAP Affiliate's proximity to Toronto, Canada.
Included with license fee (initial start-up materials such as manuals, guides, posters).
See below.
Equipment such as laptop/tablet hybrids, cameras, camcorders may need to be purchased if the SNAP Affiliate does not already have such items. Recommended budget is $5,000 - $10,000 for equipment, as costs will vary by region.
Appropriate space for training and program delivery is required. In addition to staff offices and client meeting rooms, three group spaces are required for SNAP as 3 groups run simultaneously (child, parent and sibling activity groups).
No information is available
Generally, there should be 5 dedicated SNAP service delivery staff and 1 SNAP Clinical Manager/Supervisor for full implementation (serving 56 newly admitted clients and 28 continued care children annually). There is flexibility for smaller team size (~4 SNAP staff and 1 SNAP Supervisor/Manager) to accommodate smaller communities.
Administrative costs range from 10-20% of implementation costs.
Transportation costs may range up to $2,000 for clients to attend SNAP groups and staff to meet clients. This will vary depending on the community's transportation network.
Program expenses such as refreshments, incentives (prize box) and supplies may cost up to $3,000. A SNAP program budget template is provided to each potential SNAP Affiliate to help estimate costs.
Primary support is covered under the annual SNAP Licensing Fee. This includes access to intellectual property, consultation and fidelity monitoring, training for new staff, package of resource materials, SNAP Community of Practice and access to SNAPiT (SNAP Implementation Tool).
Optional services and items include:
Fidelity monitoring and evaluation support is provided as part of the annual SNAP Licensing agreement.
Annual SNAP License and Implementation Fee of $6,500 includes:
* Training and support to web-based SNAPiT system that supports video monitoring, evaluation, implementation, client information, outcome and fidelity monitoring.
** Fidelity/integrity activities include onsite group observation or offsite video reviews, file audits, implementation issue discussions, clinical case reviews, program documentation, staffing issues, community mobilization and other general support. The cost will decrease each year as the SNAP Affiliate's implementation progresses.
No information is available
All prices are subject to change by the Institute. The Institute may also increase prices based on a percentage corresponding to the percentage increase in the global Consumer Price Index for the province of Ontario up to a maximum of five percent (5%). This schedule is designed to provide organizations with a comprehensive understanding of the activities required for successful implementation. Each organization/community is unique; therefore local capacity, resources and needs will vary and influence the implementation process as well as the timing of activities and associated costs.
The Child Development Institute has experience with larger implementations such as state-wide or national initiatives which involve a SNAP train-the-trainer model that requires a specialized costing budget.
In this example, existing staff members of a community agency, including 5 service delivery personnel and 1 clinical Manager/Supervisor, implement the program with 56 children and their families in Year 1. It is assumed that the agency has sufficient space for client meetings and child, parent and sibling activity groups. Note that salaries will vary by locale.
| Implementation training (one-time cost) | $22,000.00 |
| Annual SNAP license and implementation support | $6,500.00 |
| Transportation to lead staff training (one-time cost) | $3,000.00 |
| Equipment - computer and video (one-time cost) | $7,500.00 |
| Transportation for clients and staff | $2,000.00 |
| Refreshments, incentives, other supplies | $3,000.00 |
| Staff salaries - 5 staff and 1 Manager/Supervisor | $461,000.00 |
| Administrative - 10-15% of all costs above | $50,500.00 |
| Total One Year Cost | $555,500.00 |
The number of clients a SNAP program provides services to in Year 1 typically tends to be lower than in subsequent years until the program is established in the community and referrals begin to flow.
The Year 1 cost to implement and begin delivery of SNAP Boys services to children and their families, including the one-time start-up costs noted above (e.g., training, materials, equipment), is estimated at $555,500. The cost per boy for 56 new admissions is $9,920.
In Year 2 onward, based on typical program service utilization rates of 84 SNAP children/year and their families (56 new admissions plus 28 clients carried over into the next fiscal year as part of continued case management), the total ongoing yearly cost is estimated at $519,750. The cost per boy is estimated at $6,187.
No information is available
The Provincial Government of Ontario endorsed SNAP as a model program and committed to full implementation and program funding for a dozen communities/organizations across Ontario. Given SNAP's robust outcomes/evidence and detailed implementation support, this funding model can be replicated in other provinces, states, and regions. SNAP fits under various funding opportunities such as crime prevention, mental health, education, and gender-based violence. SNAP can provide support for your community funding proposal.
Public Safety Canada, a Canadian federal government department, provided SNAP with a grant to offer Canadian SNAP Affiliates with pro-bono implementation and fidelity monitoring support for the first two years of implementation between January 2017 and December 2021.
SNAP Affiliates should investigate any potential federal entitlement funding based on their jurisdiction.
Many SNAP Affiliates have successfully applied for foundation grants to support SNAP implementation. However, given that many foundations focus on new project funding, ongoing SNAP program funding is less likely to come from foundations.
An example of a successful public-private partnership supporting SNAP implementation and programming comes from a SNAP Affiliate in Pittsburgh. Initial funding was secured by a local philanthropist on the condition that the local government commit ongoing funding if the pilot resulted in statistically significant change among clients, which it did, and therefore, ongoing government funding was secured.
In many cases, several community-based organizations and schools unite to form a community team dedicated to supporting the SNAP program. Each partner provides different supports and resources to ensure SNAP's success and sustainability in the community.
Non-profit SNAP Affiliates may use a fee for service model on the conditions that there is a sliding scale for families that cannot afford program fees and that any profit is reinvested into the SNAP Affiliates' SNAP implementation and evaluation activities.
Leena Augimeri, Ph.D.DirectorChild Development Institute (CDI)197 Euclid Ave.Toronto, Ontario M6J 2J8Canada416-603-1827 extension 3112416-576-1867laugimeri@childdevelop.ca
A cognitive behavioral multi-component training program designed to reduce antisocial behavior and/or police contact among boys at risk for such engagement by decreasing the factors that make children susceptible for continued delinquency and strengthening the protective factors of the parents, the child, and the family structure.
Families with boys aged 6-11 years, referred for police contact or aggressive behaviors.
Subgroup differences in program effects by race, ethnicity, or gender (coded in binary terms as male/female) or program effects for a sample of a specific racial, ethnic, or gender group:
Study 3 (Burke & Loeber, 2014, 2016) tested for subgroup effects by using a homogenous sample with 75% or more of African American males.
Sample demographics including race, ethnicity, and gender for Blueprints-certified studies:
Individual: Antisocial/aggressive behavior*, Bullies others, Early initiation of antisocial behavior*, Favorable attitudes towards antisocial behavior, Gang involvement, Physical violence, Stress
Family: Neglectful parenting, Parental attitudes favorable to antisocial behavior, Parent stress*, Poor family management*, Psychological aggression/discipline, Violent discipline
School: Low school commitment and attachment
Individual: Coping Skills, Problem solving skills*, Prosocial behavior*, Prosocial involvement
Peer: Interaction with prosocial peers
Family: Attachment to parents, Nonviolent Discipline, Parental involvement in education, Parent social support, Rewards for prosocial involvement with parents
School: Rewards for prosocial involvement in school
*Risk/Protective Factor was significantly impacted by the program
SNAP (Stop Now And Plan) was originally called the Under 12 Outreach Project (ORP), a non-gender-specific program established in 1985 to meet a service gap for children in conflict with the law. Today, SNAP is an evidence-based children's mental health and crime prevention model consisting of two gender-specific programs: SNAP® Boys (discussed here), and SNAP® Girls. The SNAP® Boys program is a 13-week, multi-component treatment program for boys aged 6-11 years who have had police contact as a result of delinquency or who are at risk of police contact. The focus is on increasing emotion regulation, self-control and problem-solving skills that help to keep high-risk boys in school and out of trouble by making better choices 'in the moment,' working with parents/caregivers to support their children and enhancing their parent management skills. The program aims to decrease the factors that place children at risk for continued delinquency and strengthen the protective factors of the parents, the child, and the family structure. The intervention is aimed at the child, the family, and the child-in-the-community, which have been found to yield the most positive long-term preventative effects.
The SNAP® Boys program is a 13-week, multi-component treatment program for boys in the middle years (aged 6-11 years) who have had police contact as a result of delinquency or who are at risk of police contact. SNAP® Boys is based on a cognitive behavioral model and focuses on the development and treatment of aggression and antisocial behavior and consists of five core components adapted from established interventions for children with conduct problem behavior: (1) boys problem solving and self-control skills training group; (2) parent training group; (3) family counseling; (4) individual befriending; and (5) school support and reading tutoring. The primary objective of SNAP® Boys is to reduce police contact among a population which is at risk for engaging in criminal activity by decreasing the factors that place children at risk for continued delinquency and strengthening the protective factors of the parents, the child, and the family structure. The intervention is aimed at the child, the family, and the child-in-the-community, which have been found to yield the most positive long-term preventative effects.
Boys meet in structured group sessions lasting 90 minutes, once a week for thirteen weeks. Sessions are organized around the following topics: (a) Joining In/Goal Setting, (b) Stopping Stealing, (c) Rewarding Yourself, (d) Fair Play, (e) Dealing with Angry Feelings, (f) Group Pressure, (g) Avoiding Trouble, (h) When You're Not So Sure, (i) Apologizing, and (j) Dealing with Accusations/Being Blamed. Groups activities include guided discussions, modeling, role-playing, relaxation training, and a self-control technique called "SNAP", which stands for "Stop Now And Plan."
Parents are encouraged to participate in the thirteen week 90-minute sessions addressing social learning approaches to family management. Parenting skills covered in these sessions include: (a) stopping stealing, (b) routines, (c) listening, (d) monitoring, (e) self-control and problem-solving, (f) time-out, and (g) encouraging and rewarding. In addition, parents can access crisis intervention and individual family intervention, as well as phone calls or visitations by a Parent Group Leader to parents who did not attend the parent group. The Parent Group Leader reviews the content of the parent group with the parent and provides a written summary of the social skills and family management issues covered. Lastly, services such as counseling or attendance at the hosting center's twice monthly Parent Association Group are available after the parent group ends.
The portion of the intervention that addresses the child-in-the-community includes "individual befriending" by involving a SNAP® Boys staff member or volunteer in a mentor type of relationship (such as that of Big Brothers/Big Sisters) and linking the participant to resources in the community. Additionally, children may access academic tutoring and parents may receive help with school advocacy on an as-needed basis to ensure that their children receive the best possible education.
SNAP is a cognitive behavioral-based psychosocial intervention that aims to improve various child (emotional regulation, social competence, prosocial behavior, peer associations, involvement in community) and parent (emotional regulation, effective child management, positive support systems, involvement in community, problem-solving and coping skills, prosocial values and conduct) factors. Improvement in these areas will lead to decreased involvement with the criminal justice system in the long term. These child and parent factors, though, are mediated by more specific, proximal factors which, for children, include self-control, problem-solving, antisocial attitudes, cognitive distortions, low empathy, poor social skills, school failure, learning disabilities, poor coping ability, unstructured free time, and a susceptibility to peer pressure. Proximal parental factors include poor emotional regulation, punitive or laissez faire discipline techniques, inconsistency, poor monitoring and supervision, and strained parent-child relationships. Distal factors include a lack of relational and community supports, stressors such as sibling rivalry, parental psychopathology, and spousal conflict, poverty, parental antisocial values and conduct, and substance abuse.
Primary Evidence Base for Certification
Of the three studies Blueprints has reviewed, one study (Study 3) meets Blueprints evidentiary standards (specificity, evaluation quality, impact, dissemination readiness). In addition, Study 3 was done by the developer.
Study 3
Burke and Loeber (2014, 2016) conducted a randomized controlled trial in which 252 boys were randomized to intervention or control groups. Assessments were completed at baseline, upon program completion, and six and 12 months after program completion (though some participants continued to receive SNAP services through the follow-up periods).
Primary Evidence Base for Certification
Study 3
Burke and Loeber (2014, 2016) reported that from pretest to 12-month follow-up (with some participants still receiving SNAP services at 12-month follow-up), the SNAP group participants, relative to controls, showed significantly lower scores on parent-ratings of: aggressive behavior, conduct problems, externalizing behavior, internalizing behavior, withdrawn-depressed behavior, and anxious-depressed behavior. Intervention participants showed significantly higher scores on prosocial behaviors, emotional regulation skills, and problem-solving skills. Additionally, SNAP group participants had significantly lower parent-rated symptom counts for: attention deficit hyperactivity disorder; oppositional defiant disorder; depression; and separation anxiety. Among youth for whom official criminal records were available, the number of charges reported was significantly lower in the intervention group relative to controls.
Primary Evidence Base for Certification
Study 3
Burke and Loeber (2014, 2016) found 12 months after program completion, relative to controls, SNAP Boys participants had a lower number of criminal charges reported in official criminal records and significantly lower scores on parent-ratings of:
Additionally, intervention boys had significantly higher scores on:
In Study 3, Burke and Loeber (2016) found several significant mediating effects. The treatment significantly improved parent ratings of child prosocial behavior, child emotional regulation skills, positive parent behavior, and parent stress related to child difficulties. These measures in turn significantly improved parent-rated scores on child aggression and thus significantly mediated the influence of the treatment on aggression.
One study meets Blueprints standards for high quality methods with strong evidence of program impact (i.e., "certified" by Blueprints): Study 3 (Burke & Loeber, 2014, 2016). The sample included boys with behavioral problems. The study was conducted in Pittsburgh, PA and compared the treatment to standard services.
Additional Studies (not certified by Blueprints)
Study 1 (Day & Hrynkiw-Augimeri, 1996; Augimeri et al., 2007)
Study 2 (Lipman et al., 2007)
No studies of SNAP® Girls have been certified by Blueprints.
Blueprints: Promising
Crime Solutions: Effective
OJJDP Model Programs: Effective
Sample of SNAP Affiliate Sites with a strong history of implementing SNAP in their communities:
Florida Network of Youth and Family Services
Key contact: Stacy Gromatski (President/CEO)
2850 Pablo Avenue
Tallahassee, FL 32308
Main: 850-922-4324
Auberle Social Service Agency
Key Contact: Darla Poole (COO)
1101 Hartman Street
McKeesport, PA 15132
Main: 412-673-5800
Child Development Institute (CDI)
197 Euclid Ave.
Toronto, Ontario
M6J 2J8, CANADA
Phone: 416-603-1827
SNAP@childdevelop.ca
www.childdevelop.ca
www.stopnowandplan.com
Augimeri, L. K., Farrington, D. P., Koegl, C. J., & Day, D. M. (2007). The SNAP Under 12 Outreach project: Effects of a community based program for children with conduct problems. Journal of Child and Family Studies, 16, 799-807.
Day, D. M., & Hrynkiw-Augimeri, L. (1996). Serving children at risk for juvenile delinquency: An evaluation of the Earlscourt Under 12 Outreach Project (ORP). Final report submitted to the Department of Justice.
Lipman, E., Kenny, M., & Snideman, C. (2007). Banyan community service Under 12 Outreach Program: Final evaluation report. Hamilton, ON, Canada: Offord Centre for Child Studies.
Certified
Burke, J. D., & Loeber, R. (2014). The effectiveness of the Stop Now and Plan (SNAP) program for boys at risk for violence and delinquency. Prevention Science, 16, 242-253.
Certified
Burke, J. D., & Loeber, R. (2016). Mechanisms of behavioral and affective treatment outcomes in a cognitive behavioral intervention for boys. Journal of Abnormal Child Psychology, 44 (1), 179-189.
This evaluation reports on the program as originally designed, prior to the separation into gender specific programs.
Summary
Day and Hrynkiw-Augimeri (1996) randomized matched pairs of children (total n=32) to an immediate treatment group (ITG) or a delayed treatment group (DTG); the latter group began the program five months later than the initial group. Assessments were conducted at Pretreatment, Posttreatment 1 (after the ITG group received the intervention), Posttreatment 2 (after the DTG received the intervention), Follow-up 1 (6 months posttest), and Follow-up 2 (10 months posttest). Therefore, there was a true control group only at Posttreatment 1.
Day and Hrynkiw-Augimeri (1996) reported that compared with the waitlist controls:
Evaluation Methodology
Design:
Recruitment: Sixty-one youths were referred to the study during the summer of 1994 by parents and police and were arranged by the Youth Bureau of the Metropolitan Toronto Police Force. The main referral criterion was police contact within the past 6 months for commission of an act which could have resulted in a criminal charge, had the child been over the age of 11. Children who were referred only by parents, who had no police contact, must have been parent-rated in the clinical range on the Delinquent scale of the Child Behavior Checklist. Of those referred, 32 (52.5%) met these eligibility criteria and entered the study.
Assignment: The 32 eligible children were matched on age, sex, and severity of delinquency and then randomly assigned to either the Immediate Treatment Group (ITG), which received the intervention between September and December of 1994, or to the Delayed Treatment Group (DTG), who received a recreational, nonclinical intervention until February 1995. At this time, the DTG received the intervention and the ITG switched to the recreational component through May of 1995.
Assessments: Assessments were conducted at Pretreatment, Posttreatment 1 (after the ITG group received the intervention), Posttreatment 2 (after the DTG received the intervention), Follow-up 1 (6 months posttest), and Follow-up 2 (10 months posttest). Therefore, there was a true control group only at Posttreatment 1.
Sample: The sample was 75% male; the mean age of the ITG group was 8.6 years, compared to 9.1 years for the DTG group. 81.3% of ITG and 93.5% of DTG youth had police contact. There was no information on the ethnic composition of either sample.
Measures: Parents and teachers completed the Child Behavior Checklist which primarily assesses externalizing and internalizing behaviors. Teachers also completed the ADD-H Comprehensive Teacher's Rating Scale (ACTeRS), assessing attention, hyperactivity, social skills, and oppositional behavior.
Parents, additionally, completed the Parent Report of Child's Antisocial Activity (SRA-P), which assesses status offenses, substance abuse, violence, theft, and property damage. Parent self-reports were assessed with several additional tools. The Beck Depression Inventory (BDI) measures depression. The Parenting Stress Index - Short Form (PSI-SF) measures parental distress, parent-child dysfunctional interaction, and difficult child. The Family Adaptability and Cohesion Evaluation Scale (FACES-II) measures adaptability and family cohesion. The Parenting Dimensions Inventory (PDI) measures aspects of parenting behaviors, including attitudes, family organization, disciplinary style, and monitoring. The Perceived Ineffectiveness Index (PII) measures a parent's sense of effectiveness in managing their child's behavioral problems. Parents also completed the Consumer Satisfaction Questionnaire (CSQ-Parent), an extensive demographic questionnaire (the Earlscourt Family Information Form) and a more general, unstructured interview at Follow-up 2.
Participating children completed the Self-report Antisocial Behavior Scale (SRA-C) which measures involvement in delinquent behaviors over the past 6 months, antisocial attitudes, association with delinquent peers, and depressive symptoms (by inclusion of the Mood and Feeling Questionnaire - MFQ). Children also completed the Self-Control Scale (Kids Like Me) to measure self-control, the Children's Personality Questionnaire (CPQ) to measure personality structure at pretreatment and follow-up 2, the Young Children's Social Desirability Scale (What I Do) to measure the degree to which the child responds in a socially desirable manner as a form of defensive denial, and the Consumer Satisfaction Questionnaire - Child (CSQ-Child).
Analysis: Analysis was conducted in two stages. First, a series of 2 X 3 MANOVAs were conducted with two conditions (ITG and DTG) and three time periods (Pretreatment, Posttest 1, Posttest 2). Significant effects were followed-up with a priori Dunn comparison tests. A second series of 2 X 3 MANOVAs were conducted with the same two conditions and different time periods: Pretreatment, Follow-up 1 and Follow-up 2. These significant effects were also examined with a priori Dunn comparison tests.
Outcomes
Implementation Fidelity: Fall Session (through Posttest 1) - The mean attendance rate for the children's group was 87%. All but one child attended at least 67% of the sessions. Most parents (87%) parents attended at least one session of the parents' group, but the mean attendance rate was 47%. About half of the parents attended at least half of the sessions. All parents received at least one session of family counseling and the median number of sessions was 2 sessions. Seventy-five percent of children received academic tutoring with a mean number of 6.1 sessions (median of 4).
Winter Session (Posttest 1 through Posttest 2) - The mean attendance rate for the children's group, after treatment conditions switched programs, was 82%. All children attended at least 66% of the sessions. Forty-two percent of parents attended at least one session of the parents' group and the mean rate was 44%. Half attended at least one session of family counseling (mean number of 3.4 sessions) and 67% of the children received academic tutoring (mean number of 3.3 sessions, median of 1).
Baseline Equivalence: Groups were equivalent at baseline on demographic characteristics (gender, age, class placement, reason for referral, parental marital status and income level, government assistance status) and on many outcome measures, including parent-reported delinquency, externalizing, and internalizing, parenting stress index, parental depression, perceived ineffectiveness index, family functioning (adaptability and cohesion), rate of police contact, teacher-rated social skills, child-reported self-control, and most of the child-reported delinquent behaviors (attitude towards delinquency, likelihood of getting caught, perception of delinquent acts, negative peer influence, delinquency of peers). Groups differed significantly on parenting measures (ITG parents scored significantly higher on responsiveness and significantly lower on nonrestrictive attitude and monitoring), teacher-reported internalizing (DTG had a significantly higher mean score), and child-reported delinquency and mood and feeling (scores significantly higher for ITG youth) and social desirability (scores significantly higher for DTG youth).
Differential Attrition: Four cases (12.5%) attrited from the sample (all from the DTG), as they attended less than three program sessions. Examination of pretreatment data indicated that the attrited group was rated by parents to be significantly more delinquent than the non-attriters, though there was no other evidence of differential attrition. Only 26 children, however, had complete data for follow-up analyses (18.8% attrition). These cases differed from the original 32 cases on parenting attitudes (those included had significantly more positive attitudes and monitored their child significantly more closely than the original group) and on child-perceived likelihood of getting caught engaging in delinquent acts (the included group perceived this likelihood to be significantly lower than the initial group).
Posttest 1 through Follow-up 2: Multivariate analyses revealed significant Group X Time interactions for parent- and child-reported delinquency from pretreatment through posttest 2 and, from pretreatment to follow-up 1 to follow-up 2, for parent-reported CBCL delinquency and externalizing, parental nurturing, child-reported delinquency, delinquency of peers, perceptions towards delinquency, and perceived parental ineffectiveness. In all of these cases, improvements favored the Immediate Treatment Group. It should be noted, though, that the Delayed Treatment Group received the program between posttests 1 and 2. Posttest 1, then, is the only time at which a true comparison group exists. Additionally, there were no significant within-group effects of treatment on DTG youth, indicating that the second implementation of the SNAP ORP program was not effective and effects of multivariate analysis should be interpreted with this in mind.
These within-group effects were calculated with a priori Dunn comparison tests. There were no significant changes from pretreatment to any assessment point for the DTG group, except for SRA-P measurements of delinquency, for which there were significant improvements for both groups at all assessment points. Within the ITG group, there were significant effects from pre-posttest 1 on CBCL-rated delinquency, externalizing, parental stress, and child-reported delinquency. These effects were maintained at each assessment point through follow-up 2. Additional significant effects for the ITG group were found at posttest 2 on internalizing behaviors and on child-reported negative peer influences and child-reported peer delinquency at follow-up 1; all of these effects were maintained through follow-up 2 as well. There were significant effects for the ITG group found for parental nonrestrictive attitudes at posttest 2, parental nurturing at follow-up 2, and perceived parental ineffectiveness at posttest 2 and follow-up 2. It should be noted that there were no effects on family functioning or on any teacher-reported outcomes.
Over half (64.3%) of ITG children who were in the clinical range on CBCL externalizing moved into the nonclinical range by follow-up 2, compared to none of the 6 DTG children in the clinical range. Additionally, 85.7% of ITG youth moved from the clinical to the nonclinical range on the CBCL Delinquent scale from pretreatment to follow-up 2, significantly more than the 44.9% of DTG who moved out of clinical status. In total, 42.9% of the 28 children fell in the clinical range on CBCL Externalizing at Follow-up 2. Children's pretreatment externalizing scores and the interaction effect of treatment condition X externalizing score were both significant predictors of clinical status at follow-up 2.
15 months posttest: Augimeri et al. (2007) reported on the same sample as above but with an additional assessment point at 15 months posttest. Treatment effects on the delinquency and externalizing subscales of the CBCL maintained significance. This study also examined program effects on subsequent official criminal involvement, but there were no significant differences between groups on total number of convictions or the average number of convictions by offense type.
Summary
Lipman et al. (2007) used a quasi-experimental design that compared 299 boys who participated in the program to 116 boys retrospectively selected from the program wait list. Children and their families were assessed at pretest and 6, 12, 24, 36, and 48 months after program completion. The boys on the wait list began receiving the intervention 6 months after the initial program group.
Lipman et al. (2007) found, compared to controls at posttest, significant improvements for treatment boys on:
Evaluation Methodology
Design:
Recruitment: Participants were recruited for the program through community advertisement (newspaper, radio, television), police, child welfare, school personnel, and children's mental health services. Eligible boys were 6-11 years old, lived in Hamilton, Ontario, and had a police contact or were considered to be at risk of police contact. Boys with developmental delay or who were non-English speaking were excluded. This evaluation reports on 14 cohorts of the program implemented with 299 boys between January 2002 to April 2006. Between 17 and 25 boys participated in each program session (cohort).
There were so many referrals to the program that boys who had to wait at least six months before beginning in a SNAP group completed their baseline questionnaires and were eligible to complete a second set of baseline questionnaires that represented any change in their functioning while on the waiting list. This allowed for a comparison sample (n = 116), though actual collection of comparison group data didn't begin until three years into the project.
Assessments/Attrition: Assessments were conducted at pretest and 6, 12, 24, 36, and 48 months after program entry, while cohorts were ongoing; therefore, there was pretest data for all 14 cohorts, 6 month data for cohorts 1-12, 12 month data for cohorts 1-11, 24 month data for cohorts 1-9, 36 month data for cohorts 1-5, and 48 month data for cohorts 1 and 2. Sample sizes for each assessment point decreased dramatically over time and also differed by instrument. For example, there were fewer youths in each cohort with complete teacher-reported data than parent-reported data.
Collection of the second set of baseline questionnaires from boys waiting for the SNAP group began at the start of Session 11, which occurred in April 2005 (over 3 years into the project). However, the amount of time between the first and second baselines for comparison boys is quite variable. For intervention boys, the mean number of days between baseline and 6-month post-baseline was 226 for parent-reports (range: 16 - 555 days) and 230 for teacher-reports (range 30 - 625 days), or about 7 - 8 months. For comparison boys, the mean number of days between the first and second baseline was 230 for parent-reports (range: 44 - 583 days) and 198 for teacher-reports (range: 51 - 566 days), or about 6 to 8 months.
Sample: Boys were of an average age of 9.51 years and the mean age of the primary caretaker for each child was 36.7 years. 82% of the primary parents were mothers. About half of the boys lived in a single parent family and about 21% lived in a family where there was no natural parent living in the home. Participants had an average of 1.5 siblings living in their home. There is no information on the ethnic composition of this sample, but all are assumed to be Canadian. Over half (62.4%) of the boys were referred and included in the study due to clinical scores on the CBCL Rule Breaking Behavior scale while 23.4% were included due to police contact.
Measures: Offending behavior was measured with the Child Behavior Checklist (CBCL) and the Teacher's Report Form (TRF) for rule-breaking behavior, aggressive behavior, and conduct problems. Social competence was assessed with teacher-reports on the Adaptive Functioning Scale, parent- and teacher-reported emotional and behavioral functioning (summing items such as anxious/depressed, withdrawn/depressed, somatic complaints, social problems, thought problems, attention problems, rule-breaking behavior, aggressive behavior, internal problems, external problems, and total problems), and parent-reported Total Competency (summing items regarding school, social, and recreational competencies).
The EARL-20B, developed by the former Earlscourt Child and Family Center, was also used to assess risk for future antisocial behavior. Twenty items assess risk and are divided into Family, Child, and Responsivity categories and completed by staff members involved with program participants. The Family domain assesses household circumstances, caregiver continuity, supports, stressors, parenting style, and antisocial values and conduct, while the Child domain assesses developmental problems, onset of behavioral difficulties, abuse/neglect/trauma, hyperactivity/impulsivity/attention deficits, likability, peer socialization, academic performance, neighborhood, authority contact, antisocial attitudes, antisocial behavior, and coping ability. Various summations of these items yield critical risk scores, overall clinical judgment scores, and a total score.
Court records were obtained for program graduates who, at the time of this report, were about 12 years of age.
Analysis: Baseline comparisons were calculated with t-tests and chi-square tests. Paired t-tests or ANOVAs were used to compare youth with a short wait list time versus those with long wait list times (earlier sessions versus later sessions) and intervention youth versus comparison youth. Comparisons from pretest to 6-month post-baseline were made using paired t-tests. Growth curve analysis was used to analyze effects of treatment (with no comparisons made) throughout long-term follow-up assessment points.
Outcomes:
Baseline Equivalence: At baseline, there were no significant differences between program and comparison boys on any parent- or teacher-rated outcomes, except parent-rated rule breaking behaviors, for which intervention boys had significantly worse scores.
Attrition: Attrition was high. Sample sizes for each assessment point decreased dramatically over time and also differed by instrument. There were 299 youth with parent-reports and 267 with teacher-reports at pretest. When assessments were conducted at or around 6 months later, they were conducted only with members of cohorts 1 through 12. Therefore, parents and teachers of 247 youth were targeted for assessment at this time. Only 151 parent-reports and 130 teacher-reports were obtained at 6 months, however, for attrition rates of 49% and 51.3%, respectively, from the pretest sample. Attrition from the targeted sample of cohorts 1-12 was 38.9% for parent-reports and 47.4% for teacher-reports. Within each cohort, attrition ranged from 6% to 62% from pretest and from 16% to 76% from the targeted sample. Of those with pretest parent-reports, 79 (26.4%) were retained at 12-months, 32 (10.7%) were retained at 24-months, 16 (5%) at 36-months, and 2 at 48-months. Of those with pretest teacher-reports, 69 (25.8%) were retained at 12-months, 26 (9.7%) at 24-months, 7 (2.6%) at 36-months, and 1 at 48-months. Attrition was high for the comparison group as well. Of 116 comparison youth, 33.6% (39) were lost from the parent-report sample and 43.1% (50) were lost from the teacher-report sample. There was no analysis of differential attrition.
Court records were available for only 32 (10.7%) youth.
Program Effects for Treatment versus Comparison Youth at 6-month Post-Baseline: This analysis was conducted on the assessment point that occurred just as comparison youth were starting the program and 6 months after the treatment youth began treatment. Treatment boys showed significant improvements on all parent-rated outcomes (rule-breaking behavior, aggressive behavior, conduct problems, emotional/behavioral problems, and total competence) and on teacher-rated adaptive functioning. Comparison boys had made significant improvements between their first and second pretests on all parent-rated outcomes except total competence and on teacher-rated adaptive functioning. After accounting for age and time between pre- and 6-month assessments (as some youth waited longer than others to start the program), there were significant differences between groups. Treatment boys improved significantly more than comparison boys on parent-rated aggressive behaviors, rule breaking, conduct problems, and emotional and behavioral problems. There were no significant differences between groups on any teacher-rated outcomes.
Posttest Outcomes within the Treatment Group: There were significant improvements between baseline and 6-month postbaseline for treatment youth on parent-reported rule-breaking, aggressive behavior, and conduct problems. There was also a significant effect over time on teacher-rated conduct problems (though not on teacher-rated aggression or rule-breaking). Effect sizes were large for these parent-reported effects (ranging from 0.80 to 0.87). There were also significant improvements on parent-rated emotional and behavioral problems and competence but no significant effects on teacher-rated emotional and behavioral problems or on adaptive functioning.
Controlling for age and time between baseline and post-test evaluations, measured in days (mean days since baseline for CBCL = 226; mean days since baseline for TRF = 277), effects on all parent-rated outcomes except competence remained significant and no effects on teacher-rated outcomes remained. Further analyses controlled for additional variables, including attendance rates for both child and parent, low income, being on public assistance, being a single parent, and number of siblings. After controlling for these variables, the only remaining significant effects were for parent-rated aggressive behaviors and parent-rated emotional and behavioral problems.
Interactive effects were tested for the following variables: ADHD, current or past child welfare involvement, waiting list time (greater or less than 6 months), police involvement, wardship status, low-income families. There were no significant interactions on any outcomes for ADHD, waiting list time, police contact, wardship status, or low-income status. However, boys with current or past child welfare involvement show significantly more improvement on parent-rated conduct problems than those without child welfare involvement, accounting for age and time between assessments.
Long-term Follow-up: There were significant improvements over time for all parent-rated outcomes and effect sizes were large for rule-breaking, aggressive behavior, and conduct problems and were moderate for emotional problems and competence. There were no significant effects over time on teacher-rated outcomes. Effects on the EARL-20B variables and on parent management strategies were not analyzed statistically.
Court records: Of the 32 boys who returned consent forms for court records, only two made an appearance in court. One was charged with multiple offenses. Data from court records were not analyzed any further than this, as very few of those who gave consent for researchers to review their court records were actually court-involved.
Summary
Burke and Loeber (2014, 2016) conducted a randomized controlled trial in which 252 boys were randomized to intervention or control groups. Assessments were completed at baseline, upon program completion, and six and 12 months after program completion (though some participants continued to receive SNAP services through the follow-up periods).
Burke and Loeber (2014, 2016) found 12 months after program completion, relative to controls, SNAP Boys participants had a lower number of criminal charges reported in official criminal records and significantly lower scores on parent-ratings of:
Additionally, intervention boys had significantly higher scores on:
Evaluation Methodology
Design:
Recruitment: Participants were recruited from new referrals to two SNAP program agencies in Pittsburgh, PA. Parents were referred to the program by police or teachers or were made aware of the service through local advertisements. Of 481 parents informed about the study, 70.1% (n=337) expressed interest in participation, and their sons were screened for eligibility. Participating boys had to have an IQ greater than 70 and a qualifying behavioral score via parent report or teacher report of aggressive behavior, rule breaking, conduct problems, or externalizing behavior. To accommodate participants in a standard services control condition (including referrals to a program typically paid for by Medical Assistance (MA)), participants had to have, or be eligible for, MA. Participants were excluded if they were already engaged in SNAP or the standard services program. Of 337 participants screened, 252 boys (74.8%) met the inclusion criteria and consented to participate.
Assignment: Because SNAP has a parent component that needed to include siblings, participating parents rather than youth were randomly assigned to a SNAP intervention group (n=130 boys) or a standard services control group (n=122 boys). Control group participants received assistance in their efforts to engage in a diverse set of services, including initial referrals to one high-intensity assistance program and other wraparound services.
Assessment/Attrition: Assessments, to include parent self-administered reports, interviews, and the collection of official criminal records, were completed in four waves: at baseline and at three-month, nine-month, and 15-month follow-ups. Of the randomized sample, attrition rates at follow-ups were 10.7% at three months, 18.3% at nine months, and 16.3% at fifteen months. Since the initial 3-month SNAP treatment was followed by other services, the assessments do not provide a long-term follow-up.
Sample: The mean age of participating boys was 8.5 years. Based on parental reports, 76% identified as African American, 13% as White, and 10% reported more than one racial category. The average IQ was 91.6, and 14.7% had parent-reported police contact due to the boy's behavior. Families were disproportionately of lower income with 50% of the sample reporting a household income below $15,000, and 14% reporting an income above $33,201.
Measures:
The Child Behavior Checklist (CBCL), completed by parents, was used to assess 1) externalizing behavior, 2) aggression, 3) rule breaking, 4) conduct problems, 5) internalizing behavior, 6) withdrawn-depressed behavior, 7) anxious-depressed behavior, and 8) somatic complaints. Test-retest reliability alphas for these subscales ranged from 0.82 to 0.93. The aggression and anxious-depressed subscales were chosen for use in the mediation analysis.
The Child Symptom Inventory-4: Parent Checklist (CSI-4) was used to provide symptom count scores for 1) attention deficit hyperactivity disorder, 2) conduct disorder, 3) oppositional defiant disorder, 4) depression, 5) separation anxiety, and 6) general anxiety. Test-retest reliability alphas for these subscales ranged from 0.53 to 0.78.
Official criminal records were obtained from the county department of juvenile probation for 150 of the 252 subjects. Records were obtained for the youth who were at the age of criminal responsibility (age 10) at the time of request, and for whom parents provided consent for the collection of these records.
For the mediation analyses, Burke & Loeber (2016) added 12 measures obtained from five instruments.
Analysis: Multilevel mixed regression modeling with maximum likelihood estimation was used with data nested within condition, youth nested within sibling clusters, and observations by wave nested within individuals. Fixed effects for the covariates of race, age, income, IQ score, and police contact were incorporated in the models through a process of stepwise removal. Because treatment group clustering was found to be non-significant, treatment group clusters were excluded from the models. Analyses in Burke and Loeber (2014) excluded observations at baseline, testing group differences at the 3-, 9-, and 15-month follow-up time points, while the mediation analysis in Burke and Loeber (2016) included the baseline outcomes. Subsequent analyses submitted at the request of Blueprints did include baseline measures in models reported by Burke and Loeber (2014).
Intent-to-Treat: Regardless of level of program participation, once assigned to condition, participants remained in the study. The mixed models used all available data, and tests using multiple imputation showed no notable differences for one outcome.
Outcomes:
Implementation fidelity: Fidelity checks comprising observations by study team members included ratings of program elements during treatment segments. Adherence to SNAP treatment protocols during these observations was 92% or higher. During the first three-month period, children attended on average 6.25 of the 12 child sessions, and parents attended an average of 5.02 of the 12 parenting sessions. Of the 130 children assigned to SNAP, 23.1% of the children attended no child SNAP groups, 28.5% of the parents attended no parent SNAP groups, and in 22.3% of the parent-child dyads, neither attended any groups.
Baseline Equivalence: No significant differences were found for demographic characteristics, IQ, or outcome measures at baseline. There was a marginally significant group difference (p<.10) for parent-reported conduct disorder symptoms at baseline, with higher symptom scores among control group participants.
Differential Attrition: Tests for differential attrition are not reported in the articles, but a special analysis submitted to Blueprints tested for the effects of baseline outcomes-by-treatment condition on a measure of missing data. Tests for 12 outcomes showed one significant (baseline aggression) interaction term and one marginally significant (generalized anxiety) interaction term. In addition, tests found one marginally significant instance of differential attrition in the criminal records measures.
Posttest: At waves two through four (from 3- to 15-month follow-up), SNAP group participants, relative to controls, showed significantly lower scores on six of eight subscales of the parent-completed Child Behavior Checklist (CBCL): 1) aggressive behavior, 2) conduct problems, 3) externalizing behavior, 4) internalizing behavior, 5) withdrawn-depressed behavior, and 6) anxious-depressed behavior. Effect sizes ranged from small (d=.25, conduct problems) to small-medium (d=.31, externalizing behavior).
A subgroup analysis including only those participants with clinical level scores on the rule breaking, aggressive behavior, and conduct problems subscales at baseline (n=169) found that SNAP group participants, relative to controls, had significantly lower scores on all three subscales and on the externalizing behavior subscale.
At waves two through four, SNAP group participants, relative to controls, had significantly lower symptom counts on four of six subscales of the parent-completed Child Symptom Inventory (CSI-4): 1) attention deficit hyperactivity disorder, 2) oppositional defiant disorder, 3) depression, and 4) separation anxiety. A marginally significant difference was observed with SNAP group participants having lower counts on the conduct disorder subscale (p=.08).
When modeling outcomes at waves three through four (9- and 15-month follow-ups) only, two outcomes from the CBCL were reduced to non-significance: conduct problems and externalizing behavior. Similarly, the separation anxiety and general anxiety subscales of the CSI-4 were reduced to non-significance.
Among the 150 youth over the age of criminal responsibility, there were no significant group differences in the number of participants who had any charges against them, but the number of charges was significantly lower in the intervention group. The article states that some parents opted out of data collection, but the number is never stated.
Mediation: Burke and Loeber (2016) first examined the effects of the treatment on the 12 mediators, 6 of which were significant and 1 of which was marginally significant. The treatment improved three child problem-solving skills, parent-rated prosocial behavior, parent-rated emotional regulation skills, self-reported positive parenting, and self-reported parent stress related to child difficulties.
Next, the analysis found that four of the seven measures affected by the treatment also significantly affected aggression and therefore mediated the treatment effects. The four significant mediators included prosocial behavior, emotional regulation skills, positive parent behavior, and parent stress related to child difficulties. For the anxious-depressed subscale, none of the mediators proved significant.