A community-based program that assists young people with histories of foster care or juvenile justice involvement in making a successful transition to adulthood by providing intensive, individualized, and clinically focused case management, support, and counseling. Specific goals include improved family relationships, increased high school graduation, decreased illicit substance use, and decreased intimate partner violence.
Blueprints: Promising
Katja Russell
Youth Villages
3320 Brother Blvd.
Memphis, TN 38133
(901) 251-4811
Katja.Russell@youthvillages.org
www.youthvillages.org
Timothy Goldsmith, Ph.D.
Youth Villages
LifeSet, formerly known as Transitional Living, which is operated by the social services organization Youth Villages, is an independent living program for youth in need (e.g., transitioning from foster care or juvenile justice custody). The program lasts 9 months for most youth who successfully complete the program and involves intensive, individualized, and clinically focused case management, support, and counseling. At entrance, each person receives an assessment and individualized treatment plan. The bulk of the services are then provided during hour-long, weekly sessions with a case manager, who typically serves only eight youth at a time.
Youth Villages is a non-profit social service organization based in Memphis, Tennessee, but operating community-based programs, such as LifeSet, in multiple states. LifeSet provides community-based case management, support, and counseling to young adults who were formerly in foster care or juvenile justice custody or otherwise opportunity youth transitioning into adulthood. After an individualized assessment, most services are then provided during hour-long, weekly sessions with a case manager, who typically serves only eight youth at a time.
The specific curriculum focuses on preparing adolescents for young adulthood with life-skills training on topics like money management, securing safe and appropriate housing, educational enrollment, and job-seeking skills. In addition, the program uses evidence-based and/or research-informed practices for behavioral treatment for alcohol and other substance abuse. The counseling involves talk about particular issues in the participant's life from both the past and present. Youth take part in action-oriented activities, such as opening a bank account or gathering information about classes at a community college.
Other resources for participating youth include trauma-focused cognitive behavioral therapy, a 12- to 20-week course of therapy provided by specially trained Youth Villages staff; referral to community services, such as General Educational Development classes or housing services; access to some flexible funds for expenses such as purchasing appropriate clothing for interviews or an apartment application fee; skill development; cultivation of social support, both formal and informal; and work for youth seeking postsecondary education, vocational training, or employment opportunities.
Primary Evidence Base for Certification
Study 1
Valentine et al. (2015) found, relative to a control group, the program improved:
Skemer and Valentine (2016) found in the long-term follow-up, relative to the control group, the treatment group reported greater likelihood of:
Primary Evidence Base for Certification
One study has been reviewed by Blueprints and meets Blueprints evidentiary standards (specificity, evaluation quality, impact, dissemination readiness). In addition, this study was conducted by independent evaluators.
Study 1
Valentine et al. (2015) and Skemer and Valentine (2016) conducted a randomized controlled trial in which 1,322 young adults who had been in foster care or juvenile justice custody were randomly assigned to a treatment group or a control group receiving a list of social service resources available in the community. Posttest data came from a survey and administrative data on college enrollment, both obtained 12-14 months after random assignment. Additional administrative records were collected two years after random assignment. Data measured educational attainment, employment and earnings, housing instability, economic hardship, social support, mental and physical health, and criminal involvement.
Study 1
Valentine, E. J., Skemer, M., & Courtney, M. E. (2015). Becoming adults: One-year impact findings from the Youth Villages Transitional Living evaluation. MDRC. http://www.mdrc.org/sites/default/files/Becoming_Adults_FR.pdf
Individual: Favorable attitudes towards drug use, Gang involvement, Substance use, Youth employment*
Peer: Interaction with antisocial peers, Romantic partner violence*
School: Low school commitment and attachment, Poor academic performance, Repeated a grade
Individual: Academic self-efficacy, Coping Skills, Problem solving skills, Prosocial behavior, Prosocial involvement, Refusal skills, Skills for social interaction
Peer: Interaction with prosocial peers
Family: Attachment to parents
*
Risk/Protective Factor was significantly impacted by the program
See also: LifeSet Logic Model (PDF)
Subgroup differences in program effects by race, ethnicity, or gender (coded in binary terms as male/female) or program effects for a sample of a specific race, ethnic, or gender group.
Study 1 (Valentine et al., 2015; Skemer & Valentine, 2016) tested for subgroup effects by location and found equal benefits for urban and non-urban residents.
Sample demographics including race, ethnicity, and gender for Blueprints-certified studies:
The sample for Study 1 (Valentine et al., 2015) was mostly White/non-Hispanic (51%) and Black/non-Hispanic (37%), and slightly over half of participants were male (52%).
A highly structured two-week staff training-and-development process includes an initial 4-day clinical training, clinical supervisor training, on-the-job training, and training staff on utilizing various evidence based practices, or EBPs, that are commonly employed throughout treatment. Ongoing training support includes quarterly clinical booster trainings, weekly clinical consultation, weekly team supervision, weekly individual supervision and development, and field supervision. All of the supervision and consultation components are specified in the LifeSet Program Model.
Week One:
The first week includes a four-day clinical partner training dedicated to LifeSet. Staff will gain knowledge about the admission and eligibility criteria of LifeSet youth. These criteria are based on the LifeSet program model utilized during the Randomized Controlled Trial. Staff will also be trained on the Pre-Admission Assessment/Psychosocial document, which is the initial screening tool used to gather a wide range of information regarding a young person at the onset of treatment. The remainder of the week consists of various counseling techniques that are often used with young people such as identification of behavioral drivers, selection of appropriate clinical interventions to target these behaviors, and creation of comprehensive service plans. Staff will gain access to the on-line clinical intervention portal and will learn to use these interventions and resources with young people.
Week Two:
The second week of training begins with a one-day individual clinical supervisor group supervision training. Group supervision is the process by which service plans and intervention direction are reviewed and is a collaborative process for the team of specialists, led by the supervisor. In this training, supervisors will learn how to review service plans, understand what type of feedback to give, and how to create a collaborative atmosphere for group supervision in order to facilitate team engagement. The remainder of week two consists of various on-the-job training activities to include multiple modules containing practical information, such as how to document appropriately, safety in the community, and further instruction on various aspects of the LifeSet program. Finally, instruction is provided on utilizing various evidence based practices, or EBPs, that are commonly employed throughout treatment.
On-going Training:
Beyond the initial 2 weeks of training, staff receive training in the form of clinical quarterly one-day on-site boosters, designed to increase staff understanding in various skills and ideas related to working with young people.
Program Benefits (per individual):
($4,063)
Program Costs (per individual):
$10,312
Net Present Value (Benefits minus Costs, per individual):
($14,375)
Measured Risk (odds of a positive Net Present Value):
7%
Source: Washington State Institute for Public Policy
All benefit-cost ratios are the most recent estimates published by The Washington State Institute for Public Policy for Blueprint programs implemented in Washington State. These ratios are based on a) meta-analysis estimates of effect size and b) monetized benefits and calculated costs for programs as delivered in the State of Washington. Caution is recommended in applying these estimates of the benefit-cost ratio to any other state or local area. They are provided as an illustration of the benefit-cost ratio found in one specific state. When feasible, local costs and monetized benefits should be used to calculate expected local benefit-cost ratios. The formula for this calculation can be found on the WSIPP website.
The cost for initial orientation training is $5,500, including travel. A program start-up support fee of $7,500 covers technical assistance to support program start-up, referral and census management, and data collection and integration. All of these costs associated with start-up are combined into a Start-up Training and Support Fee.
Costs for training materials and access to the online clinical manual are included in costs above.
An annual agency certification fee of $5,000 per organization covers licensing and re-licensing activities, and is included in the Annual Model Implementation Fee.
During the start-up phase, staff salaries, costs for equipment and occupancy, and communications will be incurred prior to service delivery. Philanthropic support could be utilized to cover start-up costs.
Even though this intervention is primarily home- and community-based, staff need to have access to meeting and training space throughout service delivery.
Costs for materials are included in ongoing training costs.
LifeSet specialists and supervisors must have a Bachelor's degree in a social services or a related field and have at least one year of experience at a minimum; a Master's degree in a social services field is preferred. One LifeSet supervisor supports four to five LifeSet specialists. One LifeSet specialist provides services to 8-10 young people at a time with one face-to-face contact per week, at minimum, per young person.
A large portion of the program implementation costs are based on staff salaries which vary widely across geographies.
Administrative overhead can be projected between 10-30% depending on the size and scope of the program and the partner organization.
Reimbursement for staff travel should be considered as a cost of implementing this model and can vary widely across geographies.
Costs for occupancy and equipment also vary based on geography and will depend on the availability of existing space and equipment for staff to use when implementing this model.
Ongoing training costs are $8,000 annually, including travel, and cover quarterly on-site booster trainings. Additional on-site orientation trainings can be added as requested by the partner organization.
The Program Implementation and Development Fee of $25,000 per team covers the cost of clinical consultation and ongoing technical assistance. In addition, a network access fee of $5,000 per agency covers access to the partner data portal, communication materials, and network activities. All of these implementation support and training costs are combined into an Annual Model Implementation Fee. Once partner organizations have trained an internal consultant to deliver weekly consultation and training activities, annual fees decrease to $10,000 and consist of the network access fee and the agency certification fee.
A partner organization can choose to train its own internal clinical consultant after 2 years of high fidelity model implementation. A one-time fee of $16,500 for Consultant Development and Coaching covers the year-long process of the consultant's training, development, and coaching.
An annual agency certification fee of $5,000 covers ongoing fidelity assessments and monitoring (included in the Annual Model Implementation Fee).
No information is available
No information is available
No information is available
This example assumes an agency implements the program using two teams, each with four specialists and one supervisor. The specialists have caseloads of eight youth during ongoing program implementation. As a whole, the two teams can serve approximately 100 youth during the first year. Average episode of treatment is 215 days.
Salaries, fringe, and non-FTE costs as well as agency overhead will vary by geography and implementing provider, thus the costs provided here should be used as a guide only.
Implementing providers may incur additional costs of allocating administrative staff and/or operating expenses for communications, equipment and supplies to the LifeSet program. These costs are not included in the example.
| Start-up Training and Support Fee | $13,000.00 |
| Annual Model Implementation Fee | $68,000.00 |
| LifeSet Specialist (Bachelors-level) - 4 FTE @ $43,000 | $172,000.00 |
| LifeSet Specialist (Masters-level) - 4 FTE @ $48,375 | $193,500.00 |
| Clinical Supervisor - 2 FTE @ $55,000 | $110,000.00 |
| Fringe @ 25% | $118,875.00 |
| Travel | $92,000.00 |
| Overhead @ 13% | $99,759.00 |
| Total One Year Cost | $867,134.00 |
For an agency with eight specialists collectively serving 64 youth on any given day, during Year One, about 100 youth would receive the LifeSet Program. The average cost per youth would be $8,671 (approximately $40/day).
As a program that provides services to youth aging out of child welfare custody and with demonstrated impacts in the areas of housing stability, employment, mental health, and exposure to domestic violence, any local, state, or federal funding targeted at this population or the outcomes that the program achieves should be explored for funding options.
Historically, there has not been a pre-existing government funding stream that is a perfect match for LifeSet, so philanthropic funds have been raised to support program start-up and match government contributions through public-private partnerships. Implementing organizations continually assess new funding opportunities, actively encourage federal funding for this vulnerable population, and advocate for changes in existing funding such as the Fostering Connections to Success Act which allows Title IV-E funds toward placement, independent living allowances and case management. Outcome- and performance-based contracts may also be utilized in future funding arrangements.
State and local funds targeted at opportunity youth, youth in child welfare custody, and youth at risk of homelessness could be used to fund LifeSet. State general funds, legislative appropriations, and special tax levies could provide state-specific funding. Existing funds from multiple state agencies (including child welfare, mental health, juvenile justice, workforce development) can be braided to fund LifeSet.
Formula Funds:
Discretionary Grants: Grants from the Administration for Children and Families (ACF) and Substance Abuse and Mental Health Services Administration (SAMHSA) at the U.S. Department of Health and Human Services could be utilized for funding.
Entitlements: In some jurisdictions, Medicaid funds have been utilized to partially fund LifeSet services. In North Carolina, a B-3 transitional living definition under the 1915(b)(c) waiver provides funding for some LifeSet participants. In other states, Medicaid state-plan community support and case management definitions may be utilized for partial funding of LifeSet.
LifeSet has been awarded numerous grants from local, state and national foundations to cover costs of the program when state funding is unavailable or for young adults who otherwise would not qualify for funding to receive services.
When expanding the LifeSet program into new geographies, philanthropic support has been utilized in the following ways:
Timothy Goldsmith, Ph.D.Chief Clinical OfficerYouth Villages3320 Brother Blvd.Memphis, TN 38133USA901-251-5000 youthvillages.org
A community-based program that assists young people with histories of foster care or juvenile justice involvement in making a successful transition to adulthood by providing intensive, individualized, and clinically focused case management, support, and counseling. Specific goals include improved family relationships, increased high school graduation, decreased illicit substance use, and decreased intimate partner violence.
The program serves youth ages 18 to 24 who are aging out of foster care or juvenile justice custody and need help transitioning to adulthood.
Subgroup differences in program effects by race, ethnicity, or gender (coded in binary terms as male/female) or program effects for a sample of a specific race, ethnic, or gender group.
Study 1 (Valentine et al., 2015; Skemer & Valentine, 2016) tested for subgroup effects by location and found equal benefits for urban and non-urban residents.
Sample demographics including race, ethnicity, and gender for Blueprints-certified studies:
The sample for Study 1 (Valentine et al., 2015) was mostly White/non-Hispanic (51%) and Black/non-Hispanic (37%), and slightly over half of participants were male (52%).
Problem solving skills, life skills, trauma.
Individual: Favorable attitudes towards drug use, Gang involvement, Substance use, Youth employment*
Peer: Interaction with antisocial peers, Romantic partner violence*
School: Low school commitment and attachment, Poor academic performance, Repeated a grade
Individual: Academic self-efficacy, Coping Skills, Problem solving skills, Prosocial behavior, Prosocial involvement, Refusal skills, Skills for social interaction
Peer: Interaction with prosocial peers
Family: Attachment to parents
*Risk/Protective Factor was significantly impacted by the program
LifeSet, formerly known as Transitional Living, which is operated by the social services organization Youth Villages, is an independent living program for youth in need (e.g., transitioning from foster care or juvenile justice custody). The program lasts 9 months for most youth who successfully complete the program and involves intensive, individualized, and clinically focused case management, support, and counseling. At entrance, each person receives an assessment and individualized treatment plan. The bulk of the services are then provided during hour-long, weekly sessions with a case manager, who typically serves only eight youth at a time.
Youth Villages is a non-profit social service organization based in Memphis, Tennessee, but operating community-based programs, such as LifeSet, in multiple states. LifeSet provides community-based case management, support, and counseling to young adults who were formerly in foster care or juvenile justice custody or otherwise opportunity youth transitioning into adulthood. After an individualized assessment, most services are then provided during hour-long, weekly sessions with a case manager, who typically serves only eight youth at a time.
The specific curriculum focuses on preparing adolescents for young adulthood with life-skills training on topics like money management, securing safe and appropriate housing, educational enrollment, and job-seeking skills. In addition, the program uses evidence-based and/or research-informed practices for behavioral treatment for alcohol and other substance abuse. The counseling involves talk about particular issues in the participant's life from both the past and present. Youth take part in action-oriented activities, such as opening a bank account or gathering information about classes at a community college.
Other resources for participating youth include trauma-focused cognitive behavioral therapy, a 12- to 20-week course of therapy provided by specially trained Youth Villages staff; referral to community services, such as General Educational Development classes or housing services; access to some flexible funds for expenses such as purchasing appropriate clothing for interviews or an apartment application fee; skill development; cultivation of social support, both formal and informal; and work for youth seeking postsecondary education, vocational training, or employment opportunities.
The program incorporates cognitive-behavioral theory in working with youth.
Primary Evidence Base for Certification
One study has been reviewed by Blueprints and meets Blueprints evidentiary standards (specificity, evaluation quality, impact, dissemination readiness). In addition, this study was conducted by independent evaluators.
Study 1
Valentine et al. (2015) and Skemer and Valentine (2016) conducted a randomized controlled trial in which 1,322 young adults who had been in foster care or juvenile justice custody were randomly assigned to a treatment group or a control group receiving a list of social service resources available in the community. Posttest data came from a survey and administrative data on college enrollment, both obtained 12-14 months after random assignment. Additional administrative records were collected two years after random assignment. Data measured educational attainment, employment and earnings, housing instability, economic hardship, social support, mental and physical health, and criminal involvement.
Primary Evidence Base for Certification
Study 1
Valentine et al. (2015) found significant program effects, relative to the control group, for 5 of 16 tests for primary outcomes and 7 of 41 tests for secondary outcomes 12-14 months after assignment. The significant primary outcomes included earnings, housing instability, economic hardship, mental health, and intimate partner violence. Among the outcomes not significantly improved by the program were education, social support, physical health, medical care, substance use, alcohol use, condom use, and criminal involvement.
Skemer and Valentine (2016) reported no significant program effects on education or recidivism in the two years after assignment. There was also no significant effect on overall earnings, however treatment group participants were more likely to be earning $2,500 or more after year 1 and $2,500 or more after year 2.
Primary Evidence Base for Certification
Study 1
Valentine et al. (2015) found, relative to a control group, the program improved:
Skemer and Valentine (2016) found in the long-term follow-up, relative to the control group, the treatment group reported greater likelihood of:
In Study 1, Valentine et al. (2015) reported small effect sizes for all significant outcomes (Cohen's d = .10 - .20).
One study meets Blueprints standards for high quality methods with strong evidence of program impact (i.e., "certified" by Blueprints): Study 1 (Valentine et al., 2015; Skemer & Valentine, 2016). The sample for the study included youth aged 18-24 from across the state of Tennessee who were formerly in foster care or juvenile justice custody and were interested in the program and able to live independently with appropriate support. The treatment group was compared to a control group that received a list of social service resources available in the community.
An earlier report provided a detailed description of the Transitional Living program model and assessed its implementation. See Michelle Manno, Erin Jacobs, Julianna Alson, and Melanie Skemer. (2014). Moving Into Adulthood: Implementation Findings from the Youth Villages Transitional Living Evaluation. New York: MDRC.
A later summary report covered some of the findings that were presented in more detail in the Study 1 reference, but it added no new findings and is not otherwise included in the text. See Valentine, E. J., Skemer, M., & Courtney, M. E. (2018). Making their way: Summary report on the Youth Villages transitional living evaluation. New York: MDRC.
Blueprints: Promising
Katja Russell
Youth Villages
3320 Brother Blvd.
Memphis, TN 38133
(901) 251-4811
Katja.Russell@youthvillages.org
www.youthvillages.org
Skemer, M. & Valentine, E. J. (2016). Striving for independence: Two-year impact findings from the Youth Villages transitional living evaluation. New York: MDRC.
Certified Valentine, E. J., Skemer, M., & Courtney, M. E. (2015). Becoming adults: One-year impact findings from the Youth Villages Transitional Living evaluation. MDRC. http://www.mdrc.org/sites/default/files/Becoming_Adults_FR.pdf
Summary
Valentine et al. (2015) and Skemer and Valentine (2016) conducted a randomized controlled trial in which 1,322 young adults who had been in foster care or juvenile justice custody were randomly assigned to a treatment group or a control group receiving a list of social service resources available in the community. Posttest data came from a survey and administrative data on college enrollment, both obtained 12-14 months after random assignment. Additional administrative records were collected two years after random assignment. Data measured educational attainment, employment and earnings, housing instability, economic hardship, social support, mental and physical health, and criminal involvement.
Valentine et al. (2015) found, relative to a control group, the program improved:
Skemer and Valentine (2016) found in the long-term follow-up, relative to the control group, the treatment group reported greater likelihood of:
Evaluation Methodology
Design:
Recruitment: The study recruited youth ages 18 to 24 years of age who had been in foster care or juvenile justice custody (both part of the Department of Child Services) in the state of Tennessee for at least 365 days (not necessarily continuously) after age 14 or for at least one day after age 17. Additionally, youth needed to be interested in program services, appropriate for the program (that is, no histories of severe violence, mental health problems, drug use, or developmental delays), and able to live independently with appropriate supports. Between October 2010 and October 2012, 1,322 young people enrolled in the evaluation, which represented about 25% of those evaluated for eligibility. As the authors note (p. 14), "It is likely that the study sample included a somewhat higher-functioning group, with stronger social connections, than is representative of the general population of foster care or juvenile justice youth."
Assignment: Sixty percent of the sample members were assigned at random to the program group (N = 788) and 40 percent were assigned at random to the control group (N = 534). The control group received a list of social service resources available in the community.
Assessment and Attrition: Pretest data were collected on sociodemographic background, including information on arrest history, receipt of mental health counseling or alcohol and drug use treatment, and state custody history. Data came from a survey and from administrative data on college enrollment, recidivism, and earnings. The survey was taken 12-14 months after random assignment; administrative data were collected both one year and two years after assignment. The survey was completed by 1,114 of the 1,322 sample members (83.6% of the program group, and 85.2% of the control group). The administrative records were obtained for all participants.
Sample: The sample ranged in age from 18-24 years but about 90% were 18 or 19 years old. There were slightly more males than females (52% versus 48%). The sample was racially diverse, with a majority being white/non-Hispanic (51%), and a significant minority being black/non-Hispanic (37%).
Sample members struggled with employment and education. Only 19% held jobs at baseline. Only 29% had earned a high school diploma or GED certificate, and only 14% were enrolled in postsecondary schooling. Moreover, 43% had repeated a grade, 81% had been suspended, and over 25% had been in special education at some point.
About 25% of the sample had no contact at all with their mothers, 66% reported having more than one foster placement, 61% reported having been in custody because they had been neglected, abused, or adjudicated as unruly, and 52% reported having been in custody for delinquency. Sixty-four percent had ever been arrested - more than double the rate found among the general population.
Measures: All measures except college enrollment, earnings, and recidivism came from subject self-reports in the posttest survey.
Education. Three primary education outcomes included high school diploma receipt, GED certificate receipt, and vocational training participation. Additional measures of enrollment in college, a 4-year college, and a 2-year college came from administrative records for the year 1 and year 2 follow-up..
Earnings and Employment. Earnings from formal work in the 12 months before the posttest survey as reported by the subject served as the primary measure and summarized time employed, average hours, and average wages. Secondary measures included ever employed, ever employed full time, and ever employed part time. In addition, after year 1 and year 2, administrative records reported total earnings, ever employed, and number of quarters employed.
Housing Instability. The primary outcome equaled the number of four indicators of housing instability that a youth experienced in the year before the posttest survey, including homelessness, couch surfing, the inability to pay rent, and loss of housing because of the inability to pay rent. Secondary outcomes consisted of the four separate components of the scale.
Economic Hardship. The economic hardship scale equaled the number of five indicators experienced in the year before the survey, including not having necessary clothing or shoes, inability to pay a utility bill, having one's gas or electricity shut off because of an inability to pay the bill, having one's phone service shut off because of an inability to pay the bill, and delaying paying a bill in order to buy food. Secondary outcomes included the five component measures plus measures of savings, debt, and receipt of five types of public assistance.
Social Support. The first primary outcome equaled the mean number of people whom a youth could ask for various types of help, based on responses to seven questions (e.g., "How many different people can you go to when you need someone to listen to your problems when you're feeling low?"). The second primary outcome indicated whether or not a young person reported feeling very close to at least one familial or non-familial adult. Secondary outcomes included 6 measures relating to closeness to a non-family member, family closeness, and family members and friends who would offer a place to stay.
Health. The sole primary outcome was a mental health problems scale (21-question Depression Anxiety Stress Scale). This scale is composed of three individual subscales that measure the severity of symptoms of depression, anxiety, and stress. Secondary measures included self-rated physical health, access to medical and dental care, and health care coverage.
Safety. The five primary outcomes included measures of binge drinking, illegal drug use, condom use, victimization, and partner violence.
Criminal Involvement. The first primary measure, the criminal behavior scale, combined participation in different crimes into one main crime indicator. The second primary outcome, contact with the criminal justice system, measured spending a night in jail or prison during the year before the survey. Secondary outcomes measured having an arrest, a conviction, or involvement in five specific crimes. Additional administrative data collected after year 1 and 2 included measures of arrests, convictions, conviction class, and conviction categories.
Analysis:
The studies did not provide details on model estimation. Notes below the tables in Valentine et al. (2015) and Skemer and Valentine (2016) reported only that the results are regression-adjusted, controlling for pre-random assignment characteristics. A footnote on page 52 in Valentine et al. (2015) adds that for education outcomes, the impact estimates were adjusted for baseline measures, including whether individuals had already earned a high school diploma or GED certificate.
Intent-to-Treat: The study dropped only those subjects who, at posttest, could not be located, missed appointments or calls for the survey, refused to participate, were deceased, or were not interviewed for other reasons. Analyses of administrative data included all assigned participants.
Outcomes
Implementation Fidelity: The studies did not provide detail on fidelity of program delivery. However, the overall rate of program participation was high. About two-thirds of intervention subjects participated in YVLifeSet services for at least 5 months, and about half were still participating 9 months after random assignment. In total, program group members averaged about 26 YVLifeSet sessions during the 12 months after random assignment.
In comparison to the control group, which was eligible for a variety of services outside the program, the intervention group was more likely to have received help across multiple service areas.
Baseline Equivalence: Table 1.1 in Valentine et al. (2015) presents tests for 15 baseline sociodemographic measures, with one significantly different (p < .05). The intervention group had more contact with other relatives (90.2%) than the control group (85.8%). Additional tests for four baseline measures of custody history in Table 1.2 showed one significant difference. The intervention group entered custody at younger ages (25.5% versus 19.8% at ages 11-14).
Differential Attrition: Attrition rates did not differ significantly between the intervention and control groups. Appendix Table A.1 in Valentine et al. (2015) shows that survey respondents were significantly more likely than non-respondents to be female, have had regular contact with biological mothers (p < .10) and with other relatives at baseline, have attended substance abuse treatment in the year before the study, and have exited state custody for the final time at an earlier age. However, the differences were generally small.
Appendix Table A.2 tests for baseline equivalence among the analysis sample (excluding dropouts). Two significant differences emerged (p < .05): frequency of contact with biological mothers and age at first entry into state custody. These differences were similar to those for the randomized sample. Further, tests using administrative data for the full randomized sample and the completer subsample did not differ (p. 115).
The analysis of administrative data was complete for the randomized sample.
Posttest: A count shows significant program effects for 5 of 16 tests for primary outcomes and 7 of 41 tests for secondary outcomes (Valentine et al. 2015).
Education. None of the three primary or three secondary education measures was significantly improved by the intervention relative to the control group.
Earnings and Employment. The intervention significantly increased earnings (after 10 outliers were dropped), with an effect size of .12. It had marginal effects (p < .10) on the secondary outcome measures of ever employed and ever employed part time but not on ever employed full time.
Housing Instability. The intervention group experienced significantly less housing instability than did the control group (d = -.16). Two of the four secondary components were significant - homelessness (d = .14) and couch surfing (d = -.17).
Economic Hardship. The intervention group experienced significantly fewer forms of economic hardship than did the control group (d = -.13). For the secondary outcomes, 3 of 12 reached statistical significance: did not have necessary clothing or shoes, delayed paying a bill to buy food, and received benefits from the Supplemental Nutrition Assistance Program.
Social Support. None of the primary or secondary outcome measures of social support was significantly improved by the intervention.
Health. The intervention group reported fewer mental health problems than the control group (d = 0.13). Only 1 of 6 secondary outcomes was significant - did not receive medical care when needed.
Safety. Intervention subjects reported improved primary outcomes in 1 of 5 tests for safety: They were significantly less likely to be involved in a violent intimate relationship.
Criminal Involvement. The intervention group did not significantly differ from the control group on the criminal behavior scale or contact with the criminal justice system. For secondary outcomes, 1 of 7 tests was significant, with the intervention group having lower reported receipt of cash or goods for sex (2.2% versus 4.3%), but the authors noted that this small difference was likely due to chance.
Subgroup Analysis. The authors state (p. 97), "The results provide almost no evidence of variation in impacts across different subgroups of youth defined by history of juvenile justice custody, by geographic setting, by receipt of EFC Services at baseline, or by latent class assignment."
Two-Year Follow-Up. Skemer and Valentine (2016) found that there were no significant effects on either education or criminal involvement. There was also no program effect on total earnings. After year 1 and year 2, treatment group participants were more likely to make $2,500 or more, as compared to the control group. Overall, only 2 of 20 tests reached statistical significance at the .05 level.
Long-Term: Skemer and Valentine (2016) noted the following on page 29: "two years after participants enrolled in the study, well after the vast majority of program group members had left the Transitional Living program." This statement implies that some participants remained in the program at 2 years and had no long-term follow-up. It also implies that other participants who finished recently lacked a full year follow-up.