A six-session group-based intervention, facilitated by paraprofessionals, for youth in foster care as they transition to middle school. The program aims to prevent internalizing and externalizing problems that may lead to more serious longer term outcomes such as delinquency, substance use, and high-risk sexual behavior. Foster parents also attend a six-session program.
Blueprints: Promising
Patricia Chamberlain
Oregon Social Learning Center
10 Shelton McMurphey Boulevard
Eugene, OR 97401-4928
Patricia Chamberlain and Leslie Leve
Oregon Social Learning Center
The KEEP SAFE (Middle School) program begins during the summer prior to middle school entry and consists of two parallel components (both led by paraprofessionals): a six-session group-based intervention for the foster-care youth and a six-session, group-based intervention for the foster parents. The groups meet twice weekly for 3 weeks during the summer. Facilitators teach youth about setting goals, establishing positive relationships with peers and adults, building confidence, and developing decision-making. Problem-solving skills and opportunities are also provided so they can practice positive behaviors. A ceremony is conducted at the end of the summer in which the youth proclaim their goals and commitments to each other and their foster parents. Sessions for foster parents focus on maintaining stability in the home, preparing the youth for middle school, and developing behavioral reinforcement techniques and realistic expectations. Parents are given homework assignments designed to encourage them to practice their new skills at home. Continued sessions (i.e., ongoing training and support) are provided to foster parents (group-based) and youth (one-on-one sessions) once a week for two hours during the first year of middle school.
The KEEP SAFE (Middle School) program begins during the summer prior to middle school entry and consists of two parallel components (both led by paraprofessionals): a six-session group-based intervention for the foster-care youth and a six-session, group-based intervention for the foster parents. The groups meet twice weekly for 3 weeks, with approximately 7 people per group.
The parent sessions are led by a facilitator and a co-facilitator, each of whom are experienced foster parents with bachelor's degrees. Sessions are also often led by child welfare case workers and case work supervisors in "real world" implementation formats. The youth groups are led by a facilitator and three assistants. These individuals are supervised by a masters or doctoral-level clinician. Using paraprofessionals as the primary facilitators reduces the costs of the program dramatically.
Continued sessions (i.e. ongoing training and support) are provided to foster parents and youth (one-on-one sessions) once a week for two hours during the first year of middle school.
The content of the program is as follows:
Focus for youth: Facilitators teach youth about setting goals, establishing positive relationships with peers and adults, building confidence, and developing decision-making skills. Problem-solving skills and opportunities are also provided so they can practice positive behaviors. A ceremony is conducted at the end of the summer in which the youth proclaim their goals and commitments to each other and their foster parents.
Focus for foster parents: These sessions are primarily concerned with enabling foster parents to maintain stability in the home, prepare the youth for middle school, and develop behavioral reinforcement techniques and realistic expectations. Parents are given homework assignments designed to encourage them to practice their new skills at home.
Primary Evidence Base for Certification
Study 1
Smith et al. (2011), Kim and Leve (2011), and Kim et al. (2013) found that intervention girls, relative to control girls, demonstrated significant effects for:
Primary Evidence Base for Certification
The one study Blueprints has reviewed (Study 1) meets Blueprints evidentiary standards (specificity, evaluation quality, impact, dissemination readiness). The study was done by the developer.
Study 1
Smith et al. (2011), Kim and Leve (2011), and Kim et al. (2013) conducted a randomized controlled trial that examined 100 girls in foster care from two counties (one urban, one rural) in Oregon. The study randomized participants to an intervention group or a control group that received standard services from local child welfare organizations. Assessments occurred five to seven years after posttest and measured internalizing, externalizing, prosocial behavior, health-risking sexual behavior, foster-care placement stability, and substance.
Study 1
Kim, H. K., Pears, K. C., Leve, L. D., Chamberlain, P. C., & Smith, D. K. (2013). Intervention effects on health-risking sexual behavior among foster care girls: The role of placement disruption and substance use. Journal of Child and Adolescent Substance Abuse, 22(5), 370-387.
Kim, H., & Leve, L. (2011). Substance use and delinquency among middle school girls in foster care: A three-year follow-up of a randomized controlled trial. Journal of Consulting and Clinical Psychology, 79(6), 740-750.
Individual: Stress
Peer: Interaction with antisocial peers
Family: Parent stress, Poor family management
Individual: Problem solving skills, Prosocial involvement, Rewards for prosocial involvement, Skills for social interaction
Peer: Interaction with prosocial peers
Family: Opportunities for prosocial involvement with parents, Rewards for prosocial involvement with parents
*
Risk/Protective Factor was significantly impacted by the program
Subgroup differences in program effects by race, ethnicity, or gender (coded in binary terms as male/female) or program effects for a sample of a specific racial, ethnic, or gender group:
Study 1 (Smith et al., 2011; Kim & Leve, 2011; Kim et al., 2013) found subgroup effects by using a homogenous sample of all females.
Sample demographics including race, ethnicity, and gender for Blueprints-certified studies:
The all-female Study 1 sample (Smith et al., 2011; Kim & Leve, 2011; Kim et al., 2013) was 63% White, 10% Latino, 9% African American, 4% Native American, and 14% multiracial.
The training model for KEEP SAFE facilitators closely parallels the services that the facilitator will provide to the foster parents. The initial 5-day training is experiential, and the trainees conduct a KEEP SAFE group though all of the sessions with colleagues. In this way the content of the KEEP SAFE curriculum is reviewed along with discussions about group process, logistics, and supervision.
Following the initial training, each KEEP SAFE group meeting is recorded and uploaded to a secure website. Weekly consultations with the facilitator are used to highlight issues with program fidelity, to support the delivery of the program content with accuracy, and to assist with questions specific to the current group of foster parents. We have found that by the conclusion of a third group, most facilitators feel comfortable with the material and the delivery of the material to a group of foster parents. It is following the third group that facilitators may opt to apply for certification. If the fidelity benchmarks are achieved, the facilitator may continue to hold KEEP SAFE without weekly consultations. A certified facilitator will have quarterly checks of program fidelity, with support plans developed only if difficulties are noted.
Certified facilitators are eligible to received further training in the delivery of the 5-day initial training course and consultation as a trained trainer. Such trainers also receive support in their first training, and in the coding of recorded sessions for intervention fidelity. Such trainers can enhance the sustainability of the practice by training others within their organization.
Source: Washington State Institute for Public Policy
All benefit-cost ratios are the most recent estimates published by The Washington State Institute for Public Policy for Blueprint programs implemented in Washington State. These ratios are based on a) meta-analysis estimates of effect size and b) monetized benefits and calculated costs for programs as delivered in the State of Washington. Caution is recommended in applying these estimates of the benefit-cost ratio to any other state or local area. They are provided as an illustration of the benefit-cost ratio found in one specific state. When feasible, local costs and monetized benefits should be used to calculate expected local benefit-cost ratios. The formula for this calculation can be found on the WSIPP website.
$40,000 for a five-day, on-site training for a facilitator and co-facilitator; execution of a contract along with site readiness activities; weekly recording and upload of KEEP SAFE group sessions via a HIPAA-compliant system; weekly consultations with a KEEP SAFE consultant; and fidelity review for certification after the facilitator has competed three KEEP SAFE groups. Up to four additional teams (facilitator and co-facilitator) can be added to the training for an additional $24,000 per team. These costs do not include travel and related expenses (airfare, hotel, per diem) for the trainers.
The costs of the curriculum are included in the initial training costs.
Materials Available in Other Language: KEEP is available in Danish and Spanish. Contact developers for costs as they are not billed in any consistent manner.
There are no additional licensing costs.
$1,950 for initial planning consultation, plus travel costs. Many teams find a one-day stakeholders meeting to be helpful in establishing a referral base and galvanizing community interest, support and understanding. Holding a stakeholders meeting is not a requirement to receive training or support.
Running KEEP SAFE groups requires sufficient space for a group of up to 12 participants. The space should be readily accessible to the foster parents. Most providers find that they are able to use agency and community resources.
Facilitators are required to record and upload each KEEP SAFE session to a secure website. This requires a Windows-based computer and internet access. It is also helpful to have an external microphone to make the recordings clear. The use of computers dedicated to recording and uploading of session recordings helps to facilitate this process.
Qualifications: Facilitators and co-facilitators are child welfare professionals.
Ratios: 1 facilitator and 1 co-facilitator lead a group of 12 participants.
Time to Deliver Intervention: 90 minute weekly group sessions for 20 weeks.
It is helpful, but not required to have an on-site supervisor who is involved in the training and support processes. While there will be weekly consultation regarding the KEEP SAFE intervention and fidelity, this consultation does not replace local knowledge about foster parent certification, policies, and legal requirements. Trained supervisors are also able to provide coverage and support for facilitators.
Once facilitators have run three KEEP SAFE groups, they are eligible to be certified as KEEP SAFE facilitators. This certification permits the facilitator to continue to run KEEP SAFE groups without the need or cost of weekly consultation. Certified facilitators are also eligible (with additional selection criteria) for a subsequent three-day training which would permit them to train others within their organization. This on-site training costs $12,900 (with travel costs additional) and includes the three-day training, shadowing of the trainee's first training, and support in achieving reliability of the fidelity tracking system.
KEEP SAFE has a strong commitment to sustaining the participation of foster parents who start the groups. In the randomized controlled trials, participating foster parents were given training credits, incentives for participation (as reimbursement for travel expenses) and make-up sessions in their homes when they were unable to meet with the group. These practices resulted in a high percentage of foster parents receiving all of the KEEP SAFE material, and reinforced the status of the foster parents as partners in the care of the young people in their homes.
There are no additional or ongoing trainings required. If a facilitator finds that he/she is having difficulty maintaining intervention fidelity, additional training can be outlined.
$3,000 per year. Once a facilitator is certified, the individual must run at least one KEEP SAFE group each year, and then continue to upload sessions to the website. Access to the site costs $3,000 per year and includes a quarterly review of available sessions for fidelity.
None.
No information is available
Staff turnover is an important cost consideration. Certifications are held by the individual, rather than the organization. Making plans for sustainability and potential staff turnover are an important part of the site readiness and initial planning.
This example assumes that a community-based organization would offer the KEEP SAFE program to 3 groups of 12 parents, each with 1 facilitator and 1 co-facilitator, for 20 sessions, with three cohorts of parent groups per year.
| Facilitator and co-facilitator initial training costs | $40,000.00 |
| Estimated trainer travel costs and expenses for 1 trainer | $1,500.00 |
| Initial planning consultation | $1,950.00 |
| Fidelity monitoring and evaluation | $3,000.00 |
| Facilitator and co-facilitator's salary | $120,000.00 |
| Fringe at 30% | $36,000.00 |
| Total One Year Cost | $202,450.00 |
With 108 foster parents participating, the initial cost of the program is approximately $1,874/parent.
As a program designed for foster or kinship caregivers with young people aged 12-17 years, KEEP SAFE is designed to increase the parenting skills of foster and kinship caregivers, decrease the number of placement disruptions, improve child outcomes and increase the number of positive placement changes. KEEP SAFE can potentially be supported by child welfare funding streams, Medicaid and fees for service.
The child welfare system is a potential source of funding for KEEP SAFE. Funds from system-wide savings accrued from stabilizing placements, reducing foster parent attrition, and maintaining youth with caregivers upon their return from foster care can be reinvested in the KEEP SAFE program.
State and local child welfare funding sources are key sources of support for the KEEP SAFE program. States that are building KEEP SAFE into their ongoing training of foster parents have built support for KEEP SAFE into their child welfare budgets.
Entitlement Funds:
Formula Funds: Title IV-B is the primary federal block grant supporting child welfare preservation and prevention services. Title IV-B can be used fairly flexibly to support a range of child welfare services and could potentially be used to pay for initial training and/or ongoing costs of KEEP SAFE training.
Discretionary Grants: Federal discretionary grants from the Department of Health and Human Services, Administration for Children and Families (DHHS, ACF) can be used to cover the costs of initial training, consultations, certification and training of trainers.
Foundations, especially those with a stated interest in the well-being of vulnerable children in foster care, can provide funding for initial training and staffing, as well as meeting space.
Patricia Chamberlain and Leslie LeveOregon Social Learning Center10 Shelton McMurphey BoulevardEugene, OR 97401-4928US(541) 485-2711pattic@oslc.orglesliel@oslc.org
A six-session group-based intervention, facilitated by paraprofessionals, for youth in foster care as they transition to middle school. The program aims to prevent internalizing and externalizing problems that may lead to more serious longer term outcomes such as delinquency, substance use, and high-risk sexual behavior. Foster parents also attend a six-session program.
KEEP SAFE is intended for middle school youth (ages 11-14) in foster care. The program has been evaluated with girls in foster care who have just left elementary school and are beginning middle school in the coming months.
Subgroup differences in program effects by race, ethnicity, or gender (coded in binary terms as male/female) or program effects for a sample of a specific racial, ethnic, or gender group:
Study 1 (Smith et al., 2011; Kim & Leve, 2011; Kim et al., 2013) found subgroup effects by using a homogenous sample of all females.
Sample demographics including race, ethnicity, and gender for Blueprints-certified studies:
The all-female Study 1 sample (Smith et al., 2011; Kim & Leve, 2011; Kim et al., 2013) was 63% White, 10% Latino, 9% African American, 4% Native American, and 14% multiracial.
Risk factors for developing internalizing and externalizing problems are especially acute for children with foster care histories because they are more likely to have been subject to childhood maltreatment. In addition, foster care girls are at increased risk compared to foster care boys because they have significantly higher rates of childhood abuse than boys and high rates of co-occurring physical abuse. Girls in foster care typically come from families with higher levels of stress and criminality than boys in foster care.
Compared to non-foster girls, foster girls are more likely to experience an unstable rearing environment, which is associated with early pubertal onset, and early pubescence is associated with internalizing and externalizing behavior problems.
Individual: Stress
Peer: Interaction with antisocial peers
Family: Parent stress, Poor family management
Individual: Problem solving skills, Prosocial involvement, Rewards for prosocial involvement, Skills for social interaction
Peer: Interaction with prosocial peers
Family: Opportunities for prosocial involvement with parents, Rewards for prosocial involvement with parents
*Risk/Protective Factor was significantly impacted by the program
The KEEP SAFE (Middle School) program begins during the summer prior to middle school entry and consists of two parallel components (both led by paraprofessionals): a six-session group-based intervention for the foster-care youth and a six-session, group-based intervention for the foster parents. The groups meet twice weekly for 3 weeks during the summer. Facilitators teach youth about setting goals, establishing positive relationships with peers and adults, building confidence, and developing decision-making. Problem-solving skills and opportunities are also provided so they can practice positive behaviors. A ceremony is conducted at the end of the summer in which the youth proclaim their goals and commitments to each other and their foster parents. Sessions for foster parents focus on maintaining stability in the home, preparing the youth for middle school, and developing behavioral reinforcement techniques and realistic expectations. Parents are given homework assignments designed to encourage them to practice their new skills at home. Continued sessions (i.e., ongoing training and support) are provided to foster parents (group-based) and youth (one-on-one sessions) once a week for two hours during the first year of middle school.
The KEEP SAFE (Middle School) program begins during the summer prior to middle school entry and consists of two parallel components (both led by paraprofessionals): a six-session group-based intervention for the foster-care youth and a six-session, group-based intervention for the foster parents. The groups meet twice weekly for 3 weeks, with approximately 7 people per group.
The parent sessions are led by a facilitator and a co-facilitator, each of whom are experienced foster parents with bachelor's degrees. Sessions are also often led by child welfare case workers and case work supervisors in "real world" implementation formats. The youth groups are led by a facilitator and three assistants. These individuals are supervised by a masters or doctoral-level clinician. Using paraprofessionals as the primary facilitators reduces the costs of the program dramatically.
Continued sessions (i.e. ongoing training and support) are provided to foster parents and youth (one-on-one sessions) once a week for two hours during the first year of middle school.
The content of the program is as follows:
Focus for youth: Facilitators teach youth about setting goals, establishing positive relationships with peers and adults, building confidence, and developing decision-making skills. Problem-solving skills and opportunities are also provided so they can practice positive behaviors. A ceremony is conducted at the end of the summer in which the youth proclaim their goals and commitments to each other and their foster parents.
Focus for foster parents: These sessions are primarily concerned with enabling foster parents to maintain stability in the home, prepare the youth for middle school, and develop behavioral reinforcement techniques and realistic expectations. Parents are given homework assignments designed to encourage them to practice their new skills at home.
The developers based the program on "developmental theories and intervention work with at-risk youth" and uses a behavioral model of change, emphasizing reinforcement and contingency management.
Primary Evidence Base for Certification
The one study Blueprints has reviewed (Study 1) meets Blueprints evidentiary standards (specificity, evaluation quality, impact, dissemination readiness). The study was done by the developer.
Study 1
Smith et al. (2011), Kim and Leve (2011), and Kim et al. (2013) conducted a randomized controlled trial that examined 100 girls in foster care from two counties (one urban, one rural) in Oregon. The study randomized participants to an intervention group or a control group that received standard services from local child welfare organizations. Assessments occurred five to seven years after posttest and measured internalizing, externalizing, prosocial behavior, health-risking sexual behavior, foster-care placement stability, and substance.
Primary Evidence Base for Certification
Study 1
Smith et al. (2011), Kim and Leve (2011), and Kim et al. (2013) found that, relative to the control group, the intervention group showed significant improvements in internalizing and externalizing problem behaviors, prosocial behavior, levels of substance use, and health-risking sexual behavior.
Primary Evidence Base for Certification
Study 1
Smith et al. (2011), Kim and Leve (2011), and Kim et al. (2013) found that intervention girls, relative to control girls, demonstrated significant effects for:
Study 1 (Smith et al., 2011; Kim & Leve, 2011; Kim et al., 2013) found that there were small-to-moderate mediating effects of prosocial, internalizing, and externalizing behaviors on the long-term outcomes of substance use (beta = -.04) and delinquency (beta = -.07), and small-to-moderate mediating effects of tobacco and marijuana use on health-risking sexual behavior (beta = -.12).
Study 1 (Smith et al., 2011; Kim & Leve, 2011; Kim et al., 2013) effect sizes were generally moderate at the two-year follow-up, with Cohen's d ranging from .45 to .50 for significant outcomes and with correlations of the intervention and outcomes ranging from -.22 to -.28.
One study meets Blueprints standards for high-quality methods with strong evidence of program impact (i.e., "certified" by Blueprints): Study 1 (Smith et al., 2011; Kim & Leve, 2011; Kim et al., 2013). The sample included girls in foster care. The study took place in two counties (one urban, one rural) in Oregon and compared the treatment group to a services-as-usual control group.
As an upstream preventive intervention, this program targets and reduces problem behaviors that are associated with increased risk of developing substance use disorder or opioid use disorder later in life.
Blueprints: Promising
Britany Binkowski
Assistant to the Commissioner
Office of Child Welfare Reform
UBS Tower, 10th Floor
315 Deaderick St., Nashville, TN 37243
o. 615-741-1405
c. 615-708-9084
Britany.Binkowski@tn.gov
Sylvia Rowlands
Senior Vice-President
Evidence-Based Programs
590 Avenue of the Americas
New York, New York 10011
(212) 660-1342
sylvia.rowlands@NYFoundling.org
Patricia Chamberlain
Oregon Social Learning Center
10 Shelton McMurphey Boulevard
Eugene, OR 97401-4928
Certified Kim, H. K., Pears, K. C., Leve, L. D., Chamberlain, P. C., & Smith, D. K. (2013). Intervention effects on health-risking sexual behavior among foster care girls: The role of placement disruption and substance use. Journal of Child and Adolescent Substance Abuse, 22(5), 370-387.
Certified Kim, H., & Leve, L. (2011). Substance use and delinquency among middle school girls in foster care: A three-year follow-up of a randomized controlled trial. Journal of Consulting and Clinical Psychology, 79(6), 740-750.
Smith, D., Leve, L., & Chamberlain, P. (2011). Preventing internalizing and externalizing problems in girls in foster care as they enter middle school: Immediate impact of an intervention. Prevention Science, 12(3), 269-277.
Summary
Smith et al. (2011), Kim and Leve (2011), and Kim et al. (2013) conducted a randomized controlled trial that examined 100 girls in foster care from two counties (one urban, one rural) in Oregon. The study randomized participants to an intervention group or a control group that received standard services from local child welfare organizations. Assessments occurred one to two years after posttest and measured internalizing, externalizing, prosocial behavior, health-risking sexual behavior, foster-care placement stability, and substance use.
Smith et al. (2011), Kim and Leve (2011), and Kim et al. (2013) found that intervention girls, relative to control girls, demonstrated significant effects for:
Evaluation Methodology
Design: This randomized controlled trial of the KEEP SAFE (Middle School) program was conducted with girls in foster care from two counties (one urban, one rural) in Oregon. Girls were eligible if they were finishing elementary school between 2004 and 2007 and were referred to foster care through the local child welfare system. Researchers recruited girls and their foster parents from the pool of eligible participants (N=145). Eligibility criteria required that girls not yet be reunited with their original family and that the foster parents provide consent. Recruitment occurred on a rolling basis and stopped when enrollment reached 100. The participants were randomly assigned to the intervention (n=48) and the control condition (n=52) which consisted of the usual services provided by the local child welfare organizations.
After randomization the girls and their foster parents were assessed at baseline (T1) and at 6 months post baseline (T2). The girls were assessed again at 12-, 24-, and 36-months post baseline, which represent the periods shortly after the full program ended (T3), one year after the program ended (T4), and two years after program ended (T5), respectively. The baseline assessment consisted of a standardized interview and questionnaires for each girl and foster parent, an interview with the girl's caseworker, and the collection of child welfare records. The interviews lasted approximately two hours and were conducted in person by assessors who were blind to group assignments. The data collection instruments were designed to measure child and family characteristics, child behaviors, and parenting practices.
The summer component of the program was implemented in the summer before commencement of middle school. The mean time between baseline (T1) and the 6-month follow-up (T2) was 147 days (s.d.= 45.6). When changes in placement/caregiver occurred between baseline and T2 (20% of sample), the girl was followed to her new placement and the girl's new caregiver was recruited to participate in the study. The authors did not report when the transitions to new caregivers occurred.
Booster sessions took place during the first year of middle school. On average, parents attended ten weekly meetings between the completion of the summer session and the T2 assessment. The authors did not report on the specific frequency of the girls' attendance in follow-up sessions, but "participation rates mirrored those of their caregivers" (Kim and Leve, 2011).
The baseline (T1) sample size for the intervention group was 48 and subsequent sample sizes were 48, 47, 44, and 45, respectively, for T2, T3, T4, and T5. Of the three subjects that were lost, one could not be located and two refused to participate. The baseline (T1) sample size for the control group was 52 and subsequent sample sizes were 50, 50, 48, 45, respectively, for T2, T3, T4, and T5. Of the seven subjects that were lost, three could not be located and four refused to participate. Thus, attrition was about 10% percent from baseline to the 2-year post-intervention follow-up.
Sample Characteristics: The average age of the girls at baseline was 11.5 years. The ethnic distribution was 63% European American, 10% Latino, 9% African American, 4% Native American, and 14% multiracial. At baseline, the girls had been in foster care an average of 2.9 years and had experienced an average of 1.4 out-of-home placements. In this sample, 56% of the girls had at least one documented incident of physical abuse, 67% had at least one documented incident of sexual abuse, and 82% had at least one documented incident of physical or sexual abuse.
Measures: Measures were divided into categories of short-term outcomes, long-term outcomes, and predictors.
For short-term outcomes, the Parent Daily Report Checklist (PDR) was used to measure internalizing problems, externalizing problems, and prosocial behavior. This 34-item measure of child behavior problems was administered by telephone individually to foster parents and girls on three consecutive or closely spaced days. In each call, subjects were asked about the preceding 24-hours to improve reliability of the measure. The PDR has been shown to be both valid and reliable.
Placement changes in foster care were measured from welfare system records as the sum from the start of the study through posttest. The number of placement changes ranged from 0 to 7 during this period, and the mean was .56.
Long-term outcomes included substance use, delinquency, and health-risking sexual behavior.
Predictor measures included:
Analysis: For the 6-month assessment, Smith et al. (2011) ran separate stepwise hierarchical linear regression models, one for each T2 outcome (internalizing, externalizing, prosocial behaviors). The first step consisted of the general predictor variables (baseline age, maltreatment history, pubertal development, and behavior) and the second step included the intervention dummy. The study reported beta coefficients, but did not state that the coefficients were standardized (authors state in personal correspondence that coefficients were standardized).
The two-year posttest follow-ups (Kim & Leve, 2011; Kim et al., 2013) conducted structural equation modeling. To use the full intent-to-treat sample (n=100), full information maximum likelihood (FIML) estimation was used. This method has been shown to provide unbiased estimates when data are missing at random. Little's MCAR test indicated that the data were, in fact, missing completely at random. The composite tobacco and marijuana use variable was log transformed and the number of placement changes was square root transformed to more closely resemble normal distributions. Baseline outcomes of substance use, delinquency, and health-risking sexual behavior were not measured at baseline or used as predictors.
Outcomes
Fidelity: The interventionists were supervised weekly, and would regularly be shown videotapes of sessions and provided with feedback regarding adherence to the clinical model. The authors did not report how well the interventionists adhered to these guidelines. Caregivers attended 5.62 of the 6 summer sessions, and 20 of the 40 follow-up sessions during the year offered, on average. Participation rates of girls in the summer sessions mirrored those of their caregivers. Of the 40 follow-up sessions offered to youth, average attendance was 56.4%.
Baseline equivalence: Kim and Leve (2011) found no significant differences on demographic characteristics, age at first placement, number of placements, foster care type, history of delinquency, history of special services, internalizing behavior, or externalizing behavior. Only the baseline measure of severity of neglect differed significantly, but it had no influence in any of the models.
Differential attrition: The authors did not report on the characteristics of attriters vs. non-attriters, yet Little's MCAR test indicated that data were missing completely at random.
Posttest: Smith et al. (2011) examined internalizing problems, externalizing problems, and prosocial behavior at about six months after baseline (conclusion of the summer component, but still during booster sessions occurring in the first middle school year). The program was significantly related to a decrease in internalizing (beta = -.28, p<.01) and externalizing (beta = -.21, p<.01) problems, but not significantly related to prosocial behavior.
Kim and Leve (2011) examined placement changes, internalizing behavior, externalizing behavior, and prosocial behavior at 6-month and posttest combined. Girls in the intervention had significantly fewer placement changes (Cohen's d = .50) and significantly greater prosocial behavior (Cohen's d = .46). There were no significant differences on externalizing and internalizing symptoms (Cohen's d = .02).
Long-term: In Kim and Leve (2011), the intervention group reported moderately lower levels of substance use (composite of tobacco, alcohol, and marijuana) than girls in the control group (Cohen's d = .47) at the two-year posttest follow-up. When the substances were examined separately, the program did not have a significant effect on alcohol use but did significantly affect tobacco use (Cohen's d = .45) and marijuana use (Cohen's d = .57). Delinquency (a composite of the girl's own delinquency and her association with delinquent peers) was marginally significantly lower for intervention girls.
In Kim et al. (2013), girls from the intervention showed significantly lower levels health-risking sexual behavior (Cohen's d = .48) than the control group. Post-hoc analysis indicated that only 4.4% of the girls in the program, as opposed to 17.8% of the girls in the control condition, reported having sexual intercourse in the past year (chi-square = 4.05, df = 1, p = .04).
Mediation: First, the program had significant indirect effects (beta=-.04) on substance use through increased positive prosocial behavior at T2 and T3 and reduced internalizing and externalizing behaviors at T3 and T4. Second, the program did not have a direct effect on delinquency but did have an indirect effect (b=-.07) on delinquency through increased prosocial behavior at T2 and T3. Third, the program had a significant indirect effect (beta=-.12) on health-risking sexual behavior through reduced tobacco and marijuana use.