Functional Family Therapy (FFT)

Study 1

Summary

Alexander & Parsons (1973) evaluated the program in a randomized controlled trial in which 99 delinquent adolescents and their families were randomized to the intervention group (n=46) or to one of three comparison groups: a no-treatment control group (n=10), a client-centered family therapy comparison group (n=19), or a psychodynamic family therapy comparison group (n=11). Administrative records were used to collect data on recidivism, defined as any new court referral, over an observation period lasting between 6 and 18 months after program completion. An analysis in Klein, Alexander, & Parsons (1977) examined siblings of the initial youth sample at 2.5-3.5 months after program completion.

Alexander & Parsons (1973) reported that compared to all comparison groups, both intervention group participants and their siblings had a significantly lower

• Recidivism rates

Evaluation Methodology

Design:

Recruitment: A total of 99 delinquent adolescents were referred by the Salt Lake County Juvenile Court to the Family Clinic at the University of Utah from October 1970 to January 1972. All eligible participants had committed a behavioral or status offense including running away, being "declared ungovernable," truancy, shoplifting, and possession of alcohol, tobacco, or "soft drugs." The study also examined the siblings of the sample youth.

Assignment: Though a total of 99 families were randomized, the authors report only the numbers of the 86 families for whom records could be found. Thus, these 86 families were randomly assigned to either the intervention group (n=46) or to one of three comparison groups: (1) Client-centered family groups program (n =19); (2) Psychodynamic family program (n =11); or (3) No treatment control (n =10). However, the text noted that randomized occurred "with minor exceptions due to program availability," which suggests deviations from full random assignment. Also, the follow-up periods varied for individual families. The period length will affect the likelihood of recidivism, but the authors stated that "across groups the period was comparable."

An additional post hoc control group (n=46) was created by random selection from several hundred eligible court cases that were not initially assigned to any group. The second additional comparison group included 2,800 cases seen countywide during 1971. Because these two groups were examined on a post-hoc basis, they were not included in the statistical analysis.

Assessments/Attrition: Data on recidivism were collected anywhere between 6 to 18 months following program completion for the eligible youth and from 2.5-3.5 years for siblings of the eligible youth. In terms of available data, approximately 13% of the randomized participants could not be linked to the court records used in the study.

Sample:

Participating adolescents were majority female (55.8%) and ranged from 13 to 16 years old.

Measures:

Data on adolescent recidivism were collected from court records. Recidivism was defined as a new court referral, but the geographic coverage of the courts was unclear. In a follow-up, Klein, Alexander, & Parsons (1977) collected data on court referrals for the siblings of original participants for 2.5-3.5 years after program completion.

Analysis:

The effects of the program were evaluated using chi-square tests to assess differences in recidivism by group.

Intent-to-Treat: Intent-to-treat analysis was utilized, since participants who dropped out of treatment were included.

Outcomes

Implementation Fidelity:

The study reported that the sessions for the treatment group had, as desired, significantly more equality in talk time, less silence, and more interruptions than the sessions for the control group. Otherwise, the study noted that 12 of 46 cases (26%) of the treatment group dropped the program before completion and that therapists were regularly supervised throughout program delivery.

Baseline Equivalence:

The authors stated that subjects were compared on demographic variables (i.e., age, SES, sex), prior recidivism rates, and pretest scores on the three interaction measures, and no differences were found among the groups. However, these figures were not presented.

Differential Attrition:

There was an overall loss of approximately 13% of subjects who could not be matched with state records. There were no tests for those missing records.

Posttest:

Over the posttest observation period, which lasted between 6 and 18 months depending on participant, tests showed significant overall differences across the four conditions. The intervention group participants had a lower rate of recidivism incidents (26%) compared to participants in the no-treatment control group (50%), the client-centered family therapy comparison group (47%), and the psychodynamic family therapy comparison group (73%). The intervention group recidivism rate was also lower than the post-hoc randomized comparison group (48%) and the county-wide recidivism rate (51%).

Long-Term:

There was no long-term follow-up for the original sample, because part of the sample was followed for less than 12 months after completion of the program. However, the study of siblings (Klein, Alexander, and Parsons, 1977) examined a period of 2.5 to 3.5 years after intervention. It found that significantly fewer siblings of the identified delinquents whose families received FFT had juvenile court records (20%) compared to families receiving no or other interventions (40% to 63%).

Study 2

Summary

Barton et al. (1985) evaluated the program in a quasi-experimental design to compare 74 delinquent youth from a Utah juvenile facility to a comparison group constructed using simple matching. Data on frequency and severity of recidivism offenses over a 15-month observation period. In two additional studies, Barton et al. (1985) also examined the effectiveness of the FFT model when performed by undergraduate paraprofessionals and when performed by state workers and probation officers.

Barton et al. (1985) reported that at the end of the 15-month posttreatment follow-up period, significantly fewer youth in the FFT group had been charged with

• Committing an offense

Seriously Delinquent Adolescents
Study 1 of 3

Design: Seventy-four seriously delinquent youth who were incarcerated in a Utah juvenile facility for serious and repeated offenses were the subjects of the study. The youth were divided into two groups, the FFT group and a group who were given traditional services. The two groups were equivalent in frequency and severity of offenses during the 30-month period before incarceration. A presumption of the sampling procedure was that the youth who were judged to be "equivalently noxious to the community," or who had equivalent severity of offenses were likely to be incarcerated for similar lengths of time. Youth were yoked by time of entry and exit into the incarceration and parole systems in order to homogenize sources of variance, such as police vigilance or judicial sentencing rigor. Three weeks before release from the state institution, both groups began interventions. FFT interventions included family therapy and support services such as remedial education, job training and placement and school placement. The services continued for an average of 30 hours of contracted services for each family.

Sample: Seventy-four delinquent adolescents incarcerated in a Utah state training school were subjects of this study. The 30 youth selected for FFT had been referred by state personnel and were deemed able to return to a community living arrangement (which included at least one adult who had served them as a parent for more than three years). Alternative treatment subjects included 44 youth who were similar to the FFT youth with respect to severity of offenses, living arrangements, ethnicity, age, SES, educational level and number. Demographic information specific to race and sex was not included in the study.

Measures: Two primary dependent measures were used: number (frequency) of offenses and recidivism. Severity of offenses was also examined using the State of Utah's code which assigns quantitative rankings to types of offenses committed.

Analysis: A 2X2 ANOVA was used to compare the two groups for recidivism and frequency of offense.

Outcomes: At the end of the 15-month posttreatment follow-up period, 60% of the FFT group had been charged with committing an offense, whereas 93% of the alternative treatment group had been charged with an offense. The frequency of offenses in the FFT group was lower than in the alternative treatment group (mean frequency = .20 vs.47, respectively). With non-recidivists excluded from both groups, the results were similar (mean frequency = .34 vs.51).

Paraprofessional Therapists
Study 2 of 3

Design: The goal of this study was to evaluate the effectiveness of the FFT model when performed by undergraduate paraprofessionals compared with the types of providers used in prior FFT evaluations, therapists with substantial education, training, and experience. Therapy outcomes and measures of family process were evaluated.

Subjects: 27 status delinquents (adolescents with offenses including runaway, truancy, sexual promiscuity, possession of alcohol, and ungovernability) who were referred by probation workers. Eight undergraduates were selected for therapist training in the FFT model and received 32 hours of training. An average of 10 sessions was provided by the paraprofessionals to the status delinquents in a clinic.

Outcomes: Results obtained by the paraprofessionals were equivalent to those obtained by senior/graduate level therapists in earlier studies. Recidivism rates (court referrals after treatment) were 26% for the FFT group, compared with 24-25% with graduate students trained in FFT and rates obtained with more experienced professional therapists. A one-sample binomial test showed that this rate was also lower than the population base rate of this juvenile court district as a whole (51%). Changes in the family processes characteristic of families with delinquent youth, specifically, decreases in family defensiveness, were seen with this sample, and were reported to be similar to those decreases in defensiveness in prior FFT studies (statistics were not provided).

Foster Placement
Study 3 of 3

Design: The goal of this study was to evaluate the effectiveness of the FFT model when performed by state workers and probation officers compared with prior FFT evaluations that utilized therapists with substantial education, training, and experience. Therapy outcomes and measures of family process were evaluated.

Subjects: 109 children and adolescents at risk for foster care placement were referred by workers who investigate cases for protective or alternative custody. Reasons included status delinquent offenses, school problems, and custody issues/ineffective parenting. Therapists were from the State Division of Family Services Social Workers. All 22 were given a week-long training in FFT after they had worked for Family Services for some time. However, there was no state mandate to use FFT, and only 2 therapists used FFT (one had an undergraduate degree in psychology and the other had a MSW).

Measures: The trained FFT therapists' rates of referrals for foster care placement (72 hours or more prior to FFT training and after FFT training were completed) were compared with those of their co-workers and to their own pre-training rates.

Outcomes: Comparisons of cases treated by the trained workers before and after their training showed significant decreases in rates of referrals (from 48% to 11%). Comparisons of cases treated by the trained workers following their training (11% referred to foster care) were significantly lower than the 216 cases seen by co-workers during the same time period (49% referred to foster care).

Study 3

Unlike the usual program, the FFT treatment used in this study occurred in the home rather than the clinic, and it did not have a delimited time period.

Summary

Gordon et al. (1988, 1995) and Gordon (1995) examined FFT with 54 rural, low SES, Caucasian youth in Ohio using a quasi-experimental design with non-random assignment to groups (matched assignment with more severe cases assigned to FFT). Administrative data on recidivism were collected at 28- and 60-month follow-ups.

Gordon et al. (1988, 1995) and Gordon (1995) reported that compared to the usual care comparison group, intervention group participants had significantly reduced:

• Recidivism (posttest and follow-up)

Evaluation Methodology

Design:

Recruitment and Assignment: This study used a quasi-experimental design with non-random assignment and limited matching. For the intervention group, 28 juveniles from a rural Southeastern Ohio county were court ordered to attend FFT. According to Gordon et al. (1988), youths assigned to FFT experienced serious family conflict and were deemed by the court as likely to recidivate.

The comparison group came from additional juveniles who were adjudicated delinquents and status offenders and came before the court over the same time period. The study randomly selected 27 who received regular probation services and little or no counseling from mental health professionals. The two conditions were matched on schools, with equal proportions coming from the same four school systems. However, the court viewed the juveniles in the intervention group as higher risk than the comparison juveniles.

Assessments and Attrition: The initial follow-up period reported in Gordon et al. (1988) occurred, on average, 28 months after assignment. One intervention participant who failed to begin therapy was dropped, leaving 27 in the intervention group and a 1.8% attrition rate. The follow-up period in Gordon et al. (1995) was 32 months later (i.e., 60 months after assignment), when the original youths were adults. After 60 months, 23 remained in the intervention group and 22 in the comparison group (18% attrition). Both follow-ups combined posttest and long-term results.

Sample:

All participants lived in an economically depressed community, and 60% lived in single-parent households. The youths averaged about 15 years of age and were all non-Hispanic white. About 70% were male and 30% female. Only three had parents who attended college.

Measures:

Gordon et al. (1988) defined recidivism as a charge or complaint that resulted in a court hearing. Because the periods of follow-up differed (on average, 27.8 months for the intervention group and 31.5 months for the comparison group), the authors annualized the recidivism measure. Gordon, Graves, & Arbuthnot (1995) defined recidivism as conviction in adult or juvenile court for anything other than traffic offenses.

Analyses:

Chi-square analyses were used to compare recidivism rates between conditions.

Intent to Treat: One intervention participant who failed to begin therapy was dropped.

Outcomes

Implementation Fidelity:

The median number of 1.5-hour family sessions was 16 and extended over an average of 5.5 months. About 82% completed the therapy sessions. Gordon et al. (1988) stated that the model fidelity was high, as the FFT therapists were responsive to feedback, but the study provided no quantitative measures.

Baseline Equivalence:

The intervention youths had higher offense rates than the comparison youths at the baseline but were said to not differ in socioeconomic status or single-parent households.

Differential Attrition:

No tests for the 18% attrition at the 60-month follow-up.

Posttest and Long-term:

28-month follow-up (Gordon et al., 1988; Gordon, 1995): The FFT group was found to have a significantly lower annualized recidivism rate of 5% compared to the 25% rate for the comparison group. Overall, the FFT group had 11% total recidivism compared to 67% in the regular services comparison group.

60-month follow-up (Gordon, Graves, & Arbuthnot, 1995): When felony and misdemeanor recidivism rates were combined, the overall rate for the FFT group (8.7%) was significantly lower than for the comparison group (40.9%). For misdemeanors only, there was a trend toward significance (p < .10), with a 4.3% recidivism rate for the FFT group and 27.3% for the comparison group. For felonies, the 4.3% (FFT) and 13.6% (comparison) difference was not statistically significant.

Study 4

Summary

Hansson et al. (2000) tested the program in a randomized controlled trial where 89 delinquent Swedish youth and their families were assigned to either the intervention condition (n=49) or a usual treatment control group (n=40). Data on recidivism were collected at 12 and 24 months after program completion. Data on youth internalizing and externalizing behaviors were collected at pretest and a 2-year follow-up.

Hansson et al. (2000) reported that compared to the usual care comparison group, intervention group participants showed significant reductions in:

• Recidivism (posttest and follow-up)

Evaluation Methodology

Design:

Recruitment: The study screened all young people taken into police custody in Lund, Sweden between June 1993 and June 1995. After excluding young people reported to Social Services and young people already undergoing treatment provided by social services or Child Psychiatry, there was a total sample of 89 young people and their families.

Assignment: The study stated that random assignment was used, however no details on the method of assignment were provided. Also, it appears that consent followed randomization, with no information provided on consent rates: The treatment group included 49 participants and the business as usual control group included 40 participants.

Assessments/Attrition: The study conducted a survey at pretest and 2-year follow-up, and it also collected recidivism information at the one- and two-year follow-ups. Recidivism information was obtained from administrative records and was available for all participants, however only 19 (47.5%) of the 40 comparison group families were interviewed before treatment and 17 (42.5%) were interviewed after treatment. The study also reported additional attrition for individual questions or measures. Among the treatment group, 41 participants (84%) completed the follow-up survey.

Sample: Participants were mostly boys (87%) and approximately 15 years of age.

Measures: The study used administrative records of rearrest, defined as being taken into custody by the police or being referred to Social Services again, however offered few additional details. In addition, participants completed survey measures: 1) the Symptom Check List describing psychological and emotional symptoms (alpha .79), 2) the Child Behavior Checklist on child competence and behaviors, completed by parents, 3) the Sense of Coherence questionnaire, measuring attitudes to life that can increase resistance to stress and are supportive of good health, and 4) the Self-Reported Antisociality questionnaire for rate of criminal and/or antisocial behavior.

Analysis: The study compared recidivism rates among participants one year and two years after the start of the intervention using a chi-square analysis.

Intent-to-Treat: The study used all available data for the recidivism measure, but in analyzing the survey data, it excluded the control group due to attrition and excluded treatment youth at sub-clinical levels at pretest.

Outcomes

Implementation Fidelity: The study did not include any measures of fidelity, noting only that all but one of the 49 treatment families competed the therapy.

Baseline Equivalence: The study found no statistical difference between conditions at baseline for gender or age. The study also collected information on the behaviors of participants at baseline (including truancy, running away from home, theft, assault, etc) but for only 47 (95.9%) of the treatment group and 19 (47.5%) of the control group. It reported two differences between conditions, but did not report what those differences were.

Differential Attrition: For the survey measures with attrition, the study stated that there is evidence that those families in the control group who did complete the survey at the 2-year follow-up were more likely to be those families whose children did not recidivate. As a result, the study only included survey results from the treatment group at the 2-year follow-up.

Posttest: The study found a significantly lower proportion of participants in the treatment group recidivated one year after the start of the intervention, as compared to the control group.

Long-Term: Two years after the start of the intervention, a significantly lower proportion of treatment group participants recidivated, as compared to the control group.

The study reported that more than half of treatment group participants who reported clinical levels of emotional symptoms and problem behaviors at the pretest had normal clinical values at the follow-up, however it did not report if this change was significant, nor did it compare results in the control group. The study also reported more mothers in the treatment group had fewer emotional symptoms at the two-year follow-up, as compared to mothers in the control group. However, the study also did not include significance tests and only 38% of control group mothers were included.

Study 5

Summary

Waldron et al. (2001) and French et al. (2008) tested the program in a randomized controlled trial, where 120 delinquent New Mexico youths were randomly assigned to the FFT intervention group (n=30), an individual cognitive-behavioral therapy (CBT) comparison group (n=31), a combined FFT and CBT group (n=29), or a psychoeducational comparison group (n=30). Outcomes were the frequency and intensity of marijuana use measured at posttest and 3-month follow-up.

Waldron et al. (2001) and French et al. (2008) reported that at posttest, but not at follow-up, intervention group participants showed significant reduction compared to the control group in:

• Marijuana use (frequency)

Evaluation Methodology

Design:

Recruitment: The study reported the recruitment of 120 adolescent boys and girls ages 13-17 who were referred to the University of New Mexico Center for Family and Adolescent Research for drug-abuse treatment. A court order mandated treatment for most of the youths. Eligibility requirements included meeting diagnostic criteria for a substance abuse disorder relating to illicit drug use (mostly marijuana for this sample). As noted below, it appears that 127 were actually recruited but seven were dropped for not attending a single therapy session.

Assignment: After initial assessment, adolescents were randomly assigned to one of four conditions: FFT (N = 30), individual community-based therapy (CBT; N = 31), a combination of FFT and CBT (joint; N = 29), or a psychoeducational control group (N = 30). Adolescents were offered 12 weeks of therapy provided at the University of New Mexico.

Assessments/Attrition: Assessments were conducted at four months (i.e., end of treatment) and seven months (i.e., three months post treatment). Only six participants (5%) did not complete either of the follow-up assessments, but an additional 5.6% were dropped for non-participation.

Subjects: Subjects were primarily white and Hispanic/Latino, though some were Native American and of other ethnicities. The average age of the sample was 15.6 years, and boys made up 76-84% of the sample.

Measures: The primary substance use outcome measures came from self-reports using the Timeline follow-back interview: (1) percentage of days marijuana was used and (2) minimal marijuana use. Other measures based on collateral reports from parents and siblings, urine drug screenings, other drug use measures, and externalizing problems demonstrated convergent validity. French et al. (2008) examined a delinquency subscale of the Child Behavior Checklist Youth Self Report.

Analyses: Waldron et al. (2001) used repeated-measures ANOVAs with four groups and three time points (including the baseline outcome scores). Most of the analyses focused on within-group change, but a few tests made comparisons across conditions. French et al. (2008) estimated regressions with iteratively reweighted least squares and robust standard errors.

Intent to Treat: Waldron et al. (2001, p. 806) stated that "seven families agreed to participate in treatment but did not attend a single therapy session" and "were dropped from the research sample." This implies a randomized sample of 127 rather than 120 and exclusion of 5.6% for non-participation - a possible violation of the intent-to-treat criterion. Otherwise, 11 participants left treatment prematurely, but the 10 who completed the assessment were included in the analysis. Missing data were imputed for incomplete responses to questions. French et al. (2008) similarly excluded the seven families who did not attend a single therapy session.

Outcomes

Implementation Fidelity:

Ratings of one therapy session for half the sample used a scale ranging from 1 (least adherence) to 10 (greatest adherence). The FFT mean was 9.09. About 81% of the FFT therapy sessions were completed.

Baseline Equivalence:

Waldron et al. (2001, Tables 1 and 2) reported no significant differences on 11 baseline sociodemographic and outcome measures. French et al. (2008) reported on tests for nine baseline sociodemographic and outcome measures. Two tests emerged significant at a less stringent probability level (p < .10): FFT had a lower age at first drug use and fewer single-parent families (38% versus 62% for the control group).

Differential Attrition:

Waldron et al. (2001) examined the 18 participants who attended two or fewer sessions (i.e., the seven excluded from the sample for attending no sessions plus the 11 who left treatment prematurely). However, 10 of those 18 were included in the analysis. The authors stated that there were no significant differences between the 18 and the others on seven measures (condition, ethnicity, gender, single-parent families, age, family income, and substance use at baseline).

Posttest:

Waldron et al. (2001) found a significant main effect on the days using marijuana at four months but not at seven months. A Bonferroni adjusted comparison indicated that the FFT condition had significantly fewer days than did the control group at four months. Tests for clinical reductions to a minimal marijuana level did not examine FFT separately. There were no significant differences across conditions on the urine screens, internalizing, externalizing, or adolescent or caregiver family conflict reports.

Like Waldron et al. (2001), French et al. (2008, Table 3) found in the regression results that the FFT group reported significantly fewer days of marijuana use than the control group at four months (but not at seven months). The intervention did not significantly influence delinquency.

Long-term:

Not examined.

Study 6

Summary

Barnoski (2004) evaluated the program in a randomized controlled trial where delinquent youth in Washington State were randomly assigned to the FFT intervention group (n=427) or a wait-list control group (n=323). Data on recidivism (all, felony only, violent felony) were collected at 6, 12, and 18 months after program completion.

Barnoski (2004) reported that after 18 months, compared to the control group, intervention group participants demonstrated a significant reduction in:

• Felony recidivism (high adherence subgroup only)

Evaluation Methodology

Design and Subjects: In 1997, the Washington State Legislature passed the Community Justice Accountability Act (CJAA), with the primary goal of reducing juvenile crime in a cost-effective manner by establishing research-based programs in the state's juvenile courts. FFT was one of the four programs selected for meeting two criteria (sufficient research evidence of the ability to lower recidivism and able to be implemented by Washington's juvenile courts). Recidivism rates were compared between those youth randomly assigned to FFT or the control group. The program began with 427 families (and 323 wait-list controls) and 36 therapists. At 18-month follow-up, there were 387 families in the FFT group and 313 control families. Further, 16 of the 33 remaining therapists were rated by FFT, Inc. as competent or highly competent, and 17 were rated as not competent or borderline.

Measures: Recidivism rates (misdemeanor and felony, felony, and violent felony).

Analyses: Not specified. Likely chi square.

Outcomes

At 18-month follow-up, FFT participants (n =387, compared to control participants n =313), showed no statistically significant differences for the three types of recidivism. However, when FFT was delivered competently (i.e., with fidelity), the program reduced felony recidivism by 30%. Youth seen by the competent therapists had a 17 percent felony recidivism rate compared with 27 percent for the control group, a statistically significant reduction of 38 percent. For violent felony recidivism, the competent therapist group had a 3 percent rate compared with 6 percent for the control group, a 50 percent reduction that was marginally statistically significant at the p =.115 probability level.

Stability of Results: 6-month, 12-month, and 18-month adjusted felony recidivism rates were compared. The reduction in felony recidivism between the control and competent therapist groups at 12 months was 40 percent compared with 38 percent at 18 months, indicating that FFT's suppression effect on felony recidivism was relatively constant.

Cost-Benefit Analysis: When FFT was delivered by competent therapists, it generated $10.69 in benefits (avoided crime costs) for each dollar spent on the program. When not competently delivered, FFT cost the taxpayer $4.18. Averaging these results for all youth receiving FFT, regardless of therapist competence, resulted in a net savings of $2.77 per dollar of costs.

Study 7

Summary

Sexton and Turner (2010) evaluated the program in a randomized controlled trial. Eligible delinquent youth from 917 families were stratified by county and then randomly assigned in equal proportions to the FFT intervention group or a probation-as-usual comparison group. The authors tested the moderating effects of therapist adherence and family risk/protective factors on felony and misdemeanor recidivism, with data collected for two years after program completion.

Sexton and Turner (2010) reported that compared to the control group, intervention group participants demonstrated a significant reduction in:

• Felony recidivism (high adherence subgroup only)

Evaluation Methodology

Design: The project was designed to build on the Barnoski (2002) statewide evaluation by including information on client risk and protective factors, a more comprehensive assessment of treatment adherence, and an expanded subject pool. The project involved 38 therapists and 917 families in 14 different counties that represented both rural and urban settings. After being sentenced to probation, youth who scored moderate to high risk on the Washington State Juvenile Court Administrative Risk Assessment (WSJCR-RA) were assigned to FFT or control conditions. A stratified randomization procedure was used at the county level according to the guidelines developed and mandated by the State Juvenile Justice system using a 1:1 assignment. Eligible adolescents were assigned in the same 1:1 random manner throughout the study as caseload openings permitted. Participants in the treatment-as-usual condition received traditional probation services in their local county. Each of the therapists received systematic training and supervision in FFT.

Subjects: The project involved 38 therapists and 917 families in 14 different counties that represented both urban and rural settings. Adolescents entered the study because they had been adjudicated for a crime and were sentenced to probation. Subjects were 79% male, ranged in age from 13-17 years, and were 78% white, 10% African American, 5% Asian, 3% Native American, and 4% unidentified. The subjects were drug involved (85.4%), used/abused alcohol (80.47%) and exhibited other mental health or behavioral problems (27%). Most had committed felony crime (56.2%), and many had committed misdemeanors (41.5%). Problem behaviors committed by the subjects included weapons crimes (10.4%), gang involvement (16.1%), out of home placements (10.5%), a history of running away from home (14.1%), and school dropout (46.3%).

The FFT therapists were also diverse in demographics and prior professional backgrounds, which allowed for the systematic study of the role of therapist characteristics. Of the therapists, 79% were female and 74% White, 4% African American, 4% Asian, 4% Mexican American, and 4% multiracial. Though none of the therapists had prior experience with FFT, their training (Master's and Bachelor degree clinicians, most with licenses or certification) and experience (ranged from 1 to 40 years in family therapy, counseling, and both) were otherwise diverse.

Measures: After being sentenced to probation, all youth were administered the preliminary screening version of the Washington State Juvenile Court Administration Risk Assessment (WSJCA). Those youth scoring moderate to high risk were assessed using the full WSJCR (WSJCR-RA). The WSJCA is a 100-item structured interview that is conducted with the youth and their families to assess multiple risk and protective factors in ten domains, including criminal history, school participation, use of free time, employment, peer relationships, family, alcohol and drug history, mental health, attitudes (deviant or prosocial), and social skills. Preliminary analysis of the data indicated that a composite risk index could be formed from the school, leisure, family, mental health, (deviant) attitudes, and (poor) social skills domains. An antisocial peer association domain was independent of this composite risk index, and a protective factors index could be formed from the leisure time, work, prosocial peer influence, positive family influence, prosocial attitudes, and positive social skills domains. The school protective domain was only assessed for the half of the sample that was in school and was thus not included in the composite index.

Treatment adherence ratings were conducted according to the adherence protocol in use during the project. The rating by the clinical supervisors was based on the degree to which the therapist described the case in terms of the core principles of FFT and the degree to which they reported following the manual's specified goals for each phase of the clinical intervention. Therapists' adherence ratings were gathered over a 2-year period, although some of the cases in the first year were not rated. Likert scale ratings were aggregated into a 4-category system ranging from non-adherence, borderline adherence, adherence, and high adherence. The primary outcome measure was the youth's adjudicated post treatment criminal behaviors. Criminal recidivism was obtained from official state juvenile justice records. Crimes were classified as misdemeanor, felony, or violent crime.

Analysis: First, a one-way multivariate analysis of variance assessed possible pretreatment differences in the three study samples (control, non-adherent therapists, adherent therapists) to assess possible confounds for interpreting comparisons among the samples. Second, a random regression (HLM) was conducted with the therapist factor as the independent variable to assess possible nesting efforts, resulting from the fact that each therapist treated multiple families. Third, to test the main hypothesis, planned contrasts within a logistic regression analysis were conducted to examine differences between the three study samples on the 12-month, adjudicated recidivism measures of misdemeanor, felony, and violent recidivism. The logistic regression analysis controlled for theoretically specified covariates assessed at pretreatment. Fourth, a 2 x 2 logistic regression analysis was conducted to examine the possible interaction effects of pretreatment family risk and protective factors on the therapist model adherence independent variable, with the presence or absence of criminal recidivism as the dependent variable. Fifth, the possible contribution of therapists' case experience with FFT on recidivism rates of the youth they treated was examined, using a 2 (therapist adherence) by 3 (therapist case experience) repeated measures analysis of covariance (therapist adherence was the between participant factor and therapist experience was the within factor).

Outcomes

Overall, the findings suggest that FFT was effective in reducing youth behavioral problems, although only when the therapists adhered to the treatment model.

Pretreatment Condition Differences: Results indicated a significant effect of Study Sample on the Age, Criminal History, and the Relationship Risk dependent variables. For Age, the control group sample was older than the High Adherent FFT sample. For Criminal History, the control sample had a higher level than the low adherent sample. For Peer Risk, the Low Adherent FFT sample had a higher level than the control sample. These findings led the researchers to control for possible confounding effects of age, criminal history, and relationship risk for comparisons between the three study samples.

Therapist Nesting Effects within Adherence Levels: The results of the logistic regression did not provide evidence of nesting effects for the High Adherence group nor the Low Adherence group on the misdemeanor or felony dependent variables. Subsequent analyses omitted the therapist-nesting variable as an independent variable.

Test of Primary Hypothesis: Effects of Treatment on Adolescent Recidivism. Felony: As hypothesized, the logistic regression analysis for separate independent variables indicated that the comparison of High Adherence versus control sample in felony recidivism rates was statistically significant (i.e., higher recidivism for controls). The control sample was not lower in felony recidivism than the Low Adherence sample. Post hoc comparisons also indicated that the Low Adherence sample had a higher recidivism rate than the High Adherence sample.

Misdemeanor and Violent Recidivism: Analyses were conducted to determine the impact of FFT on misdemeanor and violent felony rates. Results for the misdemeanor measure indicated that the Low Adherence FFT sample was higher than the High Adherence FFT sample rate and not different from the control sample rate. Criminal history, male gender, family risk, relationship risk, and younger ages were also associated with greater risk of misdemeanor recidivism. Comparisons on violent felony recidivism measures indicated that the Low Adherence FFT sample had higher recidivism rates than the High Adherence sample rate or the control group rate.

Further, Sexton and Turner (2010) compared the interactive effects of therapist adherence and family risk or protective factors on outcomes, and found that High Adherence FFT therapists had lower recidivism rates than the Low Adherence FFT therapists within the High Family Risk samples and the Low Family Risk samples (18% and 32%, 12% and 28%, respectively). However, for peer risk, the binary logistic regression indicated that the High Adherence Therapist had a significantly reduced level of recidivism only in the High Relationships Risk group (20% and 33%) but not in the Low Relationship Risk group (15% and 20%, not significant), meaning therapist adherence to the model had an impact when the adolescent in treatment was also exposed to high-risk peers, but adherence was not statistically significant for low risk peers.

Effects of Therapist Adherence and Case Experience on Recidivism Rates: The results of the 2 x 3 repeated measures ANOVA revealed a significant main effect for the Therapist Adherence independent variable, with the Low Adherence FFT sample having a higher composite recidivism score than the High Adherence FFT sample. These results suggested that the Therapist Adherence effect on recidivism emerges early and persists across case experience.

Study 8

Summary

Aos et al. (2011) and Lee et al. (2015) conducted meta-analyses of up to eight evaluations of FFT.

Aos et al. (2011) and Lee et al. (2015) conducted meta-analyses of up to eight evaluations of FFT.

Meta-analysis

Aos et al. (2011) conducted a meta-analysis using eight studies: (1) Alexander & Parsons, 1973; (2) Barnoski, 2004; (3) Barton, et al., 1985; (4) Gordon, et al., 1995; (5) Gordon, 1995; (6) Klein, et al., 1977; (7) Sexton & Turner, 2010; and (8) Alexander et al., 1998. Lee et al. (2012) updated the meta-analysis through April 2012, also with eight studies (dropping Alexander et al., 1998, and adding Hansson, 1998). Lee et al. (2015) used two studies: Barton et al. (1985) and Gordon et al. (1995).

The meta-analyses examined crime outcomes. Aos et al. (2011) reported an adjusted effect size for FFT of -0.32 with a standard error of 0.15 for age 16, and -0.32 with a standard error of 0.29 for age 26. Lee et al. (2012) reported the same figures. Lee et al. (2015) reported an adjusted effect of -.976 with a standard error of .245 for age 18.

Study 9

Summary

Friedman (1989) evaluated the program in a randomized controlled trial of adolescent drug abusers who were randomized to the FFT intervention group (n=85) or an alternate parent-focused comparison group (n=50). Outcomes included child internalizing/externalizing behaviors and behavioral problems, assessed at baseline and posttest (9 months after program completion).

Friedman (1989) reported no main program effects.

Evaluation Methodology

Design: For this evaluation, Functional Family Therapy (FFT) was compared to a parent group on numerous measures of adolescent substance abuse obtained from both the adolescent clients and their mothers. All families included parents and adolescent drug abusers who were recruited from six outpatient drug-therapy programs. Families were randomly assigned to either the FFT method (n = 85) or the parent group method (n = 50). For the purposes of this evaluation, both programs were delivered in 24 weekly sessions. Post-test data were collected 15 months after initiation of treatment, or approximately 9 months after termination of the 6-month treatment. For the dropout families and early terminators, there was a longer period between termination of treatment and the evaluation.

The FFT model focused on improving family communication, fostering trust and providing feedback to families about their functional dynamics. The parent group method taught active listening skills, constructive confrontation and negotiation of power. The parent group method was based on several existing programs including Parent Effectiveness Training (PET), the Parent Communication Project of the Canadian Addiction Research Foundation, and the Parent Assertiveness Training Program. Both groups were provided with access to individual drug counseling.

Attrition: Of the 169 families that started treatment, 135 (80%) were retained for evaluation 15 months later.

Sample: The sample was predominantly male (60%) and white (89%). The adolescents had a mean age of 17.9 years and a mean education of 9.3 years (which is significantly below the expected education level for individuals of similar age). A total of 44% of the adolescents admitted to having sold drugs, and 40% had been arrested at least once. Substance use among clients was high: In the previous year, 95% of the adolescents had used alcohol, 94% had used marijuana, 69% had used amphetamines and 41% had used cocaine. Of the parents, about 51% lived together, and the remaining were divorced (37%) or separated (8%).

Measures: Both adolescents and their parents were administered a variety of survey instruments.

Client Assessment Battery

A comprehensive, structured interview was administered to each adolescent. The interview gathered information on their personal history, behavior, relationships and attitudes. Additionally, several standardized assessments were administered:

• Rosenberg Self-Esteem Scale (11 items).
• Brief Symptom Inventory (53 items).
• Family Roles Task Scale (25 items).
• Parent-Adolescent Communication Form (20 items).
• Family Environment Scale (90 items).

Parent Assessment Battery

The mother of the adolescent also received a structured interview on family history, membership and structure, demographics, SES, living arrangements, how and why the adolescent entered treatment, problems related to drug use and other emotional or behavioral problems of the client. Additionally, several standardized assessments were administered:

• Family Role Task Behavior Scale (25 items).
• Parent-Child Relationship Problems Scale (32 items).
• Emotional/Psychological Problems Inventory (35 items).

Drug Severity Index

A summary score of drug severity was also used as a key outcome variable. The score was based on the sum of multiple frequencies and risk levels for various substances, including opiates, hallucinogens, inhalants, alcohol, marijuana and over-the-counter drugs.

A factor analysis reduced some 65 separate measures to 15 outcome measures: six based on factor scores for the adolescent's data, and nine based on factor scores for the mother data.

Adolescent measures

1. How well do you get along with your mother?
2. Negative self-image.
3. Number of positive role behavior in the family.
4. Degree of conflict within the family.
5. Drug severity index.
6. Adolescent father communication.

Mother measures

1. Number of adolescent's negative or problem behaviors.
2. How often have family relations been tense?
3. Adolescent's delinquency/problems with the law.
4. How frequently has the adolescent been using drugs in the past 3 months?
5. Number of mother's negative types of reactions to the adolescent (sub-divided into 19 items).
6. To what degree do you approve of your adolescent's behavior/attitude during the past months?
7. How frequently has your adolescent been using alcohol in the past 3 months?
8. Adolescent has problems in school.
9. Is the adolescent in conflict with the parent(s)?

Analysis: Multiple regression analysis was used to determine whether either group showed a significantly greater degree of change on any of the 15 factor-score outcome criteria. For the 15 regression equations, classes of independent variables were entered as follows: (1) Those variables on which the two treatment groups to be compared differed significantly or showed a trend towards significant difference in status at admission were entered at step 1 as control variables. (2) The characteristics of the adolescents or families at baseline that correlated to a significant degree with the amount of change in the dependent variable from baseline to follow-up (the change score) were entered as step 2. (3) The value (score) obtained by each subject in the two treatment groups at baseline on the outcome criterion value was entered as step 3. (4) The dichotomous variable designating the assignment of the subject to either of the two treatment groups being compared as the key independent predictor variable in the regression equation was entered as step 4. The dependent variable in each equation was the value (score) obtained by the subject at post-test on the particular outcome criterion variable being analyzed.

Outcomes

Implementation fidelity: All therapists were experienced family counselors. For this study, therapists received instruction in the FFT approach in a two-day workshop, as well as in biweekly case reviews. Additionally, a monitoring procedure was implemented to ensure that therapists adhered to the standardized therapy model. This analysis revealed that only 3% of the sessions were inconsistent with the goals of FFT. There were no significant differences between the two groups in terms of the number of sessions attended. However, families assigned to FFT had significantly more involvement from both parents, when compared to the parent group. There were no significant differences between the two groups of adolescents on either the number of individual or the number of group sessions they attended.

Baseline equivalence: Clients in the parent group were significantly younger and had significantly less education than the FFT group.

Differential attrition: In the parent group, a disproportionate number of parents did not show for any of the therapy sessions. Otherwise, the study did not report on tests for differential attrition.

Post-test: There was no significant difference between the family therapy or parent groups on any of the 15 analyses performed. Although family therapy proved no better than the parent group, adolescents and their parents from both groups reported several significant improvements including: (1) reductions in adolescent substance use/abuse, (2) decrease in adolescent psychiatric symptomology, (3) decrease in adolescent negative family role task behavior and increase in positive behavior within the family and (4) improvement in adolescent communication with mother and father.

Study 10

Slesnick and Prestopnik (2004) examined treatment attendance for two treatment groups but did not have a services-as-usual control group or analyze outcomes. Slesnick and Prestopnik (2009) included a control group and behavioral outcomes.

Summary

Slesnick and Prestopnick (2004, 2009) tested the program in a randomized controlled trial of runaway adolescents with alcohol problems in New Mexico. Participants were randomized to one of three conditions: an ecologically-based family therapy intervention condition (n=37), an office-based FFT intervention condition (n=40), or a usual care comparison group (n=42). Primary outcomes included substance use and psychological functioning, measured at baseline, posttest and two follow-ups occurring 6 and 15 months after baseline.

Slesnick and Prestopnik (2004, 2009) reported that at posttest, compared to the usual care control group, intervention group participants reported significantly reduced:

• Days of drug or alcohol use

Evaluation Methodology

Design:

Recruitment: This evaluation examined the effectiveness of two family-based treatments in runaway adolescents with alcohol problems. All participants were recruited through two runaway shelters in Albuquerque, NM. The sample was one of convenience, and youths were approached while at the shelter and were not otherwise seeking treatment. To be eligible, adolescents had to have a primary alcohol problem (e.g., alcohol dependence), be between 12-17 years old, reside within 60 miles of the research site, not reside with foster families, and not be wards of the state.

Assignment: At baseline, 119 participants were randomized to one of three study conditions: (1) ecologically-based family therapy (EBFT; n = 37), (2) office-based Functional Family Therapy (N = 40) and (3) a service-as-usual control group (N = 42). Both EBFT and FFT were offered for 16 50-minute sessions, about one each week for a total of 3-4 months. The control condition consisted of informal meetings and case management that the shelter staff provided. Eligible subjects were randomly assigned to the three conditions using a procedure called urn randomization that successively adjusted probabilities of assignment according to the distribution of age, gender, ethnicity, and other variables.

Assessments/Attrition: Assessments were conducted at baseline and three, nine, and 15 months post-baseline. Of the 119 participants at baseline, a total of 75 (63%) completed all assessments. This included 23 (62%) in the EBFT condition, 40 (65%) in the FFT condition and 26 (62%) in the control condition.

Sample: The sample was 55% female and all participants were between 12 and 17 years old, with a mean of 15.1 years. Race/ethnicity was 5% African American, 29% Anglo, 52% Hispanic and 22% other. All participants were primary alcohol-using and 106 (89%) met DSM-IV criteria for alcohol abuse or dependence. Additionally, 66% of the youth had a diagnosis of marijuana abuse/dependence and 22% had a diagnosis of other substance abuse/dependence. Of the 13 youth not meeting criteria for alcohol abuse or dependence, all showed patterns of problem alcohol use.

Measures:

Substance use: The Form 90, Problem Oriented Screening Instrument for Teenagers (POSIT) and Adolescent Drinking Index (ADI) were used to assess alcohol and drug use patterns. Additionally, urine toxicology screens were collected at baseline and post-test.

Psychological functioning: The Youth Self-Report of the Child Behavior Checklist, Beck Depression Inventory, National Youth Survey Delinquency Scale (NYSDS) and Shaffer's CDISC were all used to assess self-reported delinquency, aggression, depression, and anxiety.

Family functioning: The Family Environment Scale (FES), Conflict Tactic Scale (CTS) and Parental Bonding Instrument were used to assess conflict resolution, family functioning, and parent-child relationships.

Analysis: Treatment differences were assessed using a series of 3 (treatment modality) X 4 (time) repeated-measures ANOVAs. The number of treatment sessions was used as a covariate for all analyses. Since the control group did not meet with project therapists, the number of outside treatment sessions was taken from a self-report measure. For the other study conditions, the number of sessions was taken from therapist records, and data from outside therapy sessions was taken from self-report.

Intent to Treat: Analyses were completed with all study participants, although the authors conducted a separate analysis for the treated group.

Outcomes

Implementation fidelity:

There were differences among groups in the number of sessions attended, with those in EBFT attending more sessions than those in FFT (Slesnick & Prestopnik, 2004). Additionally, EBFT therapy engaged more participants (defined as more than five sessions attended) than FFT.

Baseline equivalence:

The control group reported higher scores on the delinquency scale than either the FFT or EBFT group. No other significant differences appeared among groups on demographic characteristics or main variables (p < .10).

Differential attrition:

There were no significant differences between those who completed all assessments and those who did not on any dependent variable or demographic characteristic (p <.10).

Posttest:

In the intent-to-treat analysis, only 1 of 19 outcomes differed significantly across groups. The percentage of days of drug or alcohol use significantly decreased for both the ecologically based family therapy and the FFT groups. All other outcomes failed to show significant differences.

Study 11

Summary

Celinska et al. (2013) tested the intervention using a quasi-experimental design comparing youth with aggressive behavioral problems who were referred to either the FFT intervention group (n=36) or a usual care comparison group (n=36). Primary outcomes included behavioral problems and child internalizing/externalizing behaviors, measured at baseline and posttest.

Celinska et al. (2013) found that, compared to the control group, intervention group participants showed significant improvement in

• Life domain functioning related to family, school, and vocation
• Child behavioral/emotional needs (impulsivity, depression, anxiety, anger control, substance abuse)
• Child risk behaviors (suicide risk, self-mutilation, danger to others, sexual aggression, running away, delinquency, fire setting)

Evaluation Methodology

Design:

Recruitment /Sample size: The sample included 72 adolescents ages 11-17 who lived with a parent or guardian and had a history of aggressive behavior, destruction of property, or chronic truancy. Youth with serious criminal behavior, drug or alcohol use, or mental health problems were not eligible. The subjects came from a single county in New Jersey and were referred for help by a diverse set of agencies and service providers in the county.

Study type/Randomization/Intervention: The study used a quasi-experimental design that assigned subjects to FFT (n = 36) or a program of individual therapy or mentoring (n = 36). Subjects in the FFT condition received treatment from the Children at Risk Resources and Interventions - Youth Intensive Intervention Program, while subjects in the comparison condition went first to Youth Case Management, which then referred clients to treatment providers across the county. Subjects in the two conditions were referred to the programs from mostly different sources. FFT subjects came from Probation, the Family Crisis Intervention United, Family Court, and the Divisions of Youth and Family Services (81% of the cases were mandated to participate in FFT); comparison subjects came from various sources, including Children Mobile Response and Stabilization Services, the Division of Youth and Family Services, and parents.

The authors said that the FFT subjects were compared with matched youth, but subjects assigned to each condition were largely referred by different agencies and treated by different organizations. The matching appears to refer to selection of the comparison group by case managers and the agency supervisor who had been trained by research staff in identifying appropriate cases for the study (p. 27). No other information on matching was provided.

Assessment/Attrition: All 72 subjects completed the initial assessment before the intervention and a posttest at discharge from the intervention. The duration of the interventions varied across subjects and families, but the FFT group had a mean of 3.4 months and the comparison group had a mean of 4.5 months.

Sample characteristics: The authors noted that the sample was more diverse than in other FFT evaluations.Most youth were male (61-69%). The sample had a mean age of 15.1-15.5, and consisted of African Americans (36-44%), Latinos (33-36%), whites (14-19%), and others (8%). The study stated that the group was also diverse in terms of reasons for referral to the program.

Measures:

Outcome measures came from the Strengths and Needs Assessment, an instrument constructed specifically for this evaluation. The study stated that that the assessment exhibits face, construct, and predictive validity and also shows good interrater and auditor reliability, but it listed references to previous studies rather than specifics for this data set.

In consultation with each client and family, FFT therapists who delivered the interventions also scored the measures. For the comparison group, case managers scored the measures. Both therapists and case managers received training and ongoing guidance in using the assessment instrument.

The general instrument contains seven dimensions:

• life domain functioning related to family, school, and vocation,
• child strengths in areas of family life, personal achievements, and community involvement,
• acculturation in language and culture,
• caregiver strengths (involvement with child and stability at home),
• caregiver needs (mental and physical health problems)
• child behavioral/emotional needs (impulsivity, depression, anxiety, anger control, substance abuse), and
• child risk behaviors (suicide risk, self-mutilation, danger to others, sexual aggression, running away, delinquency, fire setting).

Analysis: The analysis used ANCOVA to compare differences across conditions in the change from pretest to posttest.

Outcomes

Implementation fidelity: Each therapist had to complete FFT Site Certification Training and was monitored via a web-based system. Additional support came from calls with a national FFT consultant and from onsite contact with a certified supervisor. However, no fidelity measures were reported.

Baseline Equivalence: Tests showed no significant differences between the conditions on gender, race/ethnicity, length of time in the program, and the seven outcome measures.

Differential attrition: All subjects completed both pretest and posttest.

Post-test: Most of the results examined the significance of changes within conditions, showing that both conditions generally improved the outcomes, but Table 5 tested for differences in changes across conditions. These results showed significantly greater improvement in the FFT group for three of the seven outcomes: life domain functioning, child behavioral/socioemotional need, and child risk behaviors.

Long-term effects:

The study did not collect long-term, follow-up data and therefore was not able to demonstrate sustained effects.

Study 12

Summary

Hansson et al (2004) evaluated the program using a quasi-experimental design in one Swedish county. Participants were delinquent youth referred to the FFT intervention group (n=45) and a comparison group (n=43) constructed using simple matching. Administrative data on recidivism were collected for 18 months after baseline.

Hansson et al. (2004) reported that at 18 months after first program contact, compared to participants in the matched comparison group, intervention group participants had a significantly

•  Lower rate of recidivism

Evaluation Methodology

Design:

Recruitment: The study included 45 families referred by Social Services in one county in Sweden. A social worker referred young people suspected of crimes by the local police, based on his or her clinical judgment. No additional inclusion criteria were provided.

Assignment: The treatment group (N = 45) was matched with 17 comparison individuals, who were also suspected of crime and referred by the police, based on borough, gender, age, type of crime, and proximity of timing of the charge. The study matched the 45 treatment group participants with 43 comparison group individuals, however the analysis only included 17 who were recommended some form of treatment that was comparable with FFT. It is not clear why the other comparison group individuals did not receive the treatment. Note also that most of the analyses focused only on the FFT group, without a comparison group.

Assessments/Attrition: Recidivism was measured for the 18 months after first contact with the study; all other measures were collected before and after treatment. The study had recidivism data on all participants. In the follow-up survey of FFT participants, 36 families (80%) responded.

Sample: The most common type of crimes among participants were theft (40%), assault/robbery/intimidation (26%), and shoplifting (23%). Approximately half of participating young people lived with both parents, 90% of participants were male, and the average age was 15.

Measures: The study used administrative records of recidivism or relapse, but did not define what constituted recidivism. For the FFT treatment group only, participants completed survey measures: 1) the Family Climate questionnaire, used to measure closeness, distance, spontaneity, and chaos, 2) the Family Relationship Scale, measuring relational disturbances in the family. The study stated that these measures had good reliability and validity, but did not report measures, and were self-reported measures, but it was not clear if the adolescent completed the questionnaire or the parent. The study also used 3) the Child Behavior Checklist on child competence and behaviors, elsewhere described as a parent-report, 4) the Youth Self Report, also describing internalizing and externalizing behaviors, and 5) the Symptom Checklist for parent symptoms. The study also reported these measures to have good validity and reliability, without measures. The study also reported results from the Sense of Coherence questionnaire and Stegen's Ladder of Life measure, but did not provide information on these measures.

Analysis: The study compared recidivism rates among participants 18 months after first contact with the program using a chi-square analysis and no controls. Analyses of the other measures examined change over time within the treatment group and without a control group

Intent-to-Treat: The study used all available data for the recidivism measure, but in analyzing the survey data, it excluded the control group.

Outcomes

Implementation Fidelity: The study did not include any measures of fidelity.

Baseline Equivalence: The study stated that there was no statistical difference between the treatment and comparison group "as regards which official measures were taken by the police," which may refer to the type of crime for which the subjects were charged.

Differential Attrition: The study reported that there were no significant differences in recidivism among treatment group individuals who participated in the follow-up and those who did not. The study did not report any other tests of differential attrition.

Posttest: The study found a significantly lower proportion of participants in the treatment group recidivated 18 months after first contact, as compared to the control group.

Results limited to the treatment group and without a control group showed improvements from pretest to posttest for somatic complaints, anxiety, social problems, attention problems, aggression, internalized symptoms, externalized symptoms, and a total problem behavior score. There was also improvement in a second measure of withdrawal, anxiety, social problems, cognitive problems, attention problems, criminal behavior, aggression, internalized symptoms, externalized symptoms, and a total measure. In addition, there was improvement in the Stegen Ladder of Life, which measures participant optimism about the future in both current outlook and perception of one year in the future.

Risk and Protective: At the posttest, again for the treatment group only, the results showed higher scores for family closeness, lower distance, and lower chaos, as compared to pretest. For measures of family function, the treatment group reported improvements in attribution, over-involvement, and the total family function score from pretest to posttest. From pretest to posttest, the treatment group reported improvements in parent psychiatric symptoms: somatic, depression, anxiety, anger, paranoia, other, and a total measure.

Long-Term: The follow-up for the recidivism measure is 18 months from first contact with the program, it was unclear if this constituted a long-term follow-up.

Study 13

Summary

Gottfredson et al. (2018) and Thornberry et al. (2018) evaluated the program in a randomized controlled trial that recruited participant families from one Philadelphia juvenile court. Participants were 129 delinquent adolescent males and their families, randomized into the FFT intervention group (n=66) or a treatment-as-usual family therapy comparison group (n=63). Primary outcomes of interest were delinquency/recidivism, substance use, and gang involvement. Self-report assessments occurred at baseline and 6 months post-randomization, while data on recidivism were collected from official sources for 18 months after randomization.

Gottfredson et al (2018) and Thornberry et al. (2018) reported that after 18 months, compared to the control group, intervention group participants had significantly fewer:

• New drug charges
• Delinquent adjudications

Evaluation Methodology

Design:

Recruitment: Participants were the families of adolescent males (aged 11-17 years) who were ordered to family services as a condition of probation by one Philadelphia Family Court judge between 15 September 2013 and 4 February 2016. Given the presence of gangs in the city, the youth were considered at high risk for gang involvement. Youth were ineligible for study participation if they had been referred to FFT services in the past year. Out of the 158 families assessed for eligibility, a total of 129 families consented to participation in the study.

Assignment: Participating families (n=129) were randomly assigned to the FFT intervention condition (n=66) or a treatment-as-usual control group (n=63), which received the Family Therapy Treatment Program (FTTP). Both groups received a wide variety of other services besides FFT and FTTP. Also, the authors pointed out a problem of contamination: 21% of control subjects received FFT, while 20% of the intervention group did not receive FFT.

Attrition: Assessments occurred at baseline, 6 months post-randomization, and 18 months post-randomization. The 6-month assessment appears to serve as a posttest, and the 18-month assessment includes 12 months after program end. There was approximately 8% attrition at the 6-month self-report assessment, but the 18-month assessment used official data and there was no attrition.

Sample:

Youth participants were exclusively male and predominantly Black (80%), with an average age of 15 years. Approximately 18% reporting living with both parents; virtually all participants lived with at least one parent. Parent participants were majority female (79%) and Black (80%) with an average age of 41 years. Most reported having a high school diploma or equivalent (78%). Most were currently receiving public assistance (83%).

Measures:

Recidivism was measured at 18 months post-randomization, with data on arrests, prevalence of felony charges, dispositions, percentage adjudicated delinquent, and residential placements collected from official sources. Two researchers coding the measures showed high agreement (94-100%). Delinquency was measured at baseline and 6 months post-randomization using adolescent self-reports on delinquency variety (α=.68-.81). Alcohol and substance abuse were measured at 6 months post-randomization with adolescent self-reports of substance variety (α=.59-.62), alcohol frequency (α=.89-.90), and marijuana frequency (single item), as well as with parent reports of youth substance variety (α=.47-.53) and marijuana frequency (single item). Gang involvement was measured at baseline and 6 months post-randomization with adolescent self-report on a single-item response.

Risk factors:

Gang risk was measured at baseline through adolescent self-reports of current or former gang membership as well as current or former gang membership of family members and close friends. Participants who responded "yes" to any of these questions were considered high risk.

Attitudes towards delinquency/drug abuse were measured at baseline and 6 months post-randomization with parent reports (α =.84-.94) and adolescent self-reports (α=.90).

Analysis:

The effects of the program were evaluated by comparing posttest adjusted means, which were calculated using response-appropriate regression models that controlled for baseline outcomes whenever possible and covariates for pretest predictors that differed significantly across conditions. When baseline outcomes were not available, covariates included all pretest measures that significantly predicted the outcome. The text used a .10 significance level in its interpretation but reported probabilities to allow for use of a .05 level.

Intent-to-Treat: All available data were used.

Outcomes

Implementation Fidelity:

Eighty percent of intervention participants received at least one program session, with 53% completing the program. Fidelity was reviewed by the program consultant in weekly meetings, with the average fidelity rating a 4.1 out of 6 for all clients who began the program.

Baseline Equivalence:

There were significant baseline differences between conditions on four (out of 65) tests: delinquency, violent delinquency, drug use variety, and past residential placements.

Differential Attrition:

Tests for baseline-by-condition attrition examined the interaction of condition and attrition status for each of the pretest variables. There were no significant interactions for the youth, and only one significant interaction for parents. Footnote 5 states, "The control attritors were younger than their post-tested counterparts (35 vs 41) while the treatment attritors were older than their post-tested counterparts (45 vs 41)." There was no attrition at the 18-month follow-up assessment.

Posttest:

At 6 months, 27 tests for outcomes showed no significant effects and 15 tests for risk and protective factors showed no significant effects.

When stratified by baseline gang risk level, at 6 months post-randomization, high gang risk treatment youth had significantly lower self-reports of general delinquency and alcohol frequency, as well as a marginally significant reduction in recent residential placements compared to the high gang risk control group youth.

Long-Term:

The observation period covered at least one year after program completion but results for recidivism mixed the 6-month period of program implementation with the additional 12 months after program end. Over the full 18 months after randomization, compared to the control group, youth in the treatment group had significantly fewer new drug charges and delinquent adjudications. There was a marginally significant reduction in new property charges.

Compared to high risk control group youth over the 18 months after randomization, high gang risk treatment group youth had a significantly lower percentage facing person or property charges (0-18 months), percent and number of arrests (7-18 months), and percent facing felony charges (7-18 months). Low gang risk treatment group youth had significantly fewer drug charges through 18 months post-randomization compared to low gang risk control group participants.

Study 14

Summary

Humayun et al. (2017) used a randomized controlled trial to examine youth criminal offenders ages 10-17 and their parents. A total of 111 families were randomly assigned to a treatment group receiving FFT plus management-as-usual services (N = 65) or to a control group receiving management-as-usual services only (N = 46). Youth were assessed at posttest (six months after assignment) and at one-year follow-up (18 months after assignment). Outcome measures included self-reported delinquency, official conviction, conduct disorder, oppositional defiant disorder, negative and positive behavior toward parents, and parenting quality.

Humayun et al. (2017) found no significant beneficial program effects.

Evaluation Methodology

Design:

Recruitment: Youth and parents were recruited from 2008 to 2011 through Offender Services, Support Services, and other crime-prevention agencies in two counties in England. The youth, ages 10-18, had been sentenced for offending or were receiving agency intervention following contact with the police. The study excluded youth not living at home, having a sibling in the study, not speaking fluent English, or showing a severe developmental delay; it excluded parents receiving a parenting program or not speaking fluent English. The total eligible sample was 111 families.

Assignment: After baseline assessment, families were independently randomized to FFT (N = 65) or a control group (N = 46) using constrained adaptive randomization. The randomization ratio was varied to ensure adequate caseloads for FFT therapists and ranged from 3:1 FFT cases to control cases during the early period to 1:3 FFT to control cases at the end.

The management-as-usual control group worked with a case manager who provided counseling and variety of support services to help with youth problems. The control participants did not receive family therapy but were substantially involved with the youth justice agencies in the community, including participation in reparation and victim awareness programs. The intervention group (referred to as FFT-MAU) received both family therapy - from three therapists - and management as usual.

Assessments/Attrition: Assessments came six and 18 months after randomization. With three to six months of FFT, the six-month assessment served as a posttest and the 18-month assessment as a one-year follow-up. The completion rates were 81% at the posttest and 80% at the one-year follow-up.

Sample:

Youth were predominantly male (70%) and White British (90%), with below average IQ (mean = 84). Most lived with single (55%), unemployed (57%) caregivers, 85% of whom were the youth's biological mother; 60% of caregivers had no education beyond the age of 16.

Measures:

The primary outcome of self-reported delinquency came from questions about 19 criminal acts committed over the last six months (at posttest) or the last 12 months (at follow-up). The measure summed the frequency of each act (alpha = .87).

There were ten secondary outcomes. One measure of convicted offenses came from records in the national computer database. The offenses included community sentences, custodial sentences, and police cautions for minor offenses.

Four measures of symptoms and diagnoses of oppositional defiant disorder and conduct disorder came from parent interviews. Coders used a semi-structured diagnostic interview, with high intercoder reliability (ICCs ranged from .95 to .99). The supplementary document noted that coders did not know of condition assignment.

Four outcomes came from observations of parent-youth interactions. Trained coders rated youth positive behaviors, youth negative behaviors, positive parenting, and negative parenting. Coders did not know of condition assignment, and intercoder reliability was high (ICCs ranged from .74 to .86).

One parent-reported measure of poor child monitoring included items such as not knowing the child's friends. It had an alpha reliability of .74.

Analysis:

The analysis used linear mixed models with repeated measures, interaction terms for condition-by-time, individual random intercepts, and a control for the baseline outcome. Consistent with randomization, the family (i.e., either youth or parent) served as the unit of analysis. To deal with attrition, the models included any baseline variable associated with missing outcome data as a predictor and, as a result, the covariates varied depending on the outcome.

Intent-to-Treat: The mixed models included all available data, including those with partial completion of the three assessments, although the analysis did not impute missing data.

Outcomes

Implementation Fidelity:

An FFT consultant monitored the session notes of the therapists, and the study reported that fidelity to the FFT model was adequate or better for 77% of cases. About 95% of the FFT group started therapy and 59% completed all five phases of treatment.

Baseline Equivalence:

Table 1 lists condition means at baseline, which appear similar, but it did not include significance tests.

Differential Attrition:

The study tested for relationships of baseline characteristics with missingness and reported (p. 1027) that gender, age, and referral agency were significantly related to missing data for one or more of the outcomes. Other than noting that the models controlled for these measures, the text provided no further information.

Posttest:

For the primary outcome of self-reported delinquency, the program failed to have a significant effect at posttest. The program similarly had no effect on any of the ten secondary outcomes.

Long-Term:

At the one-year follow-up, the program had no effect on the primary outcome of self-reported delinquency. The only significant effect for the ten secondary measures was possibly iatrogenic - the control group had higher observed positive youth behavior.

Study 15

Summary

Celinska et al. (2019) used a quasi-experimental design to compare delinquent or at-risk youth non-randomly assigned to receive FFT (N = 107) with similar youth receiving case management (N = 48). The sample came from one New Jersey county. Measures of needs and strengths were obtained at posttest and measures of recidivism were obtained at one year after the program ended.

Celinska et al. (2019) found at the one-year follow-up that, relative to the comparison group, the intervention group had significantly lower

• Recidivism for drug offenses, property offenses, and technical violations.

Evaluation Methodology

Design:

Recruitment and Assignment: The sample of 155 came from 1) 107 youths enrolled in the Children at Risk Resources and Interventions - Youth Intensive Intervention Program and 2) 48 youths enrolled in the Youth Case Management program. The youths in the first group received FFT and served as the intervention group. They were selected by case managers using the following eligibility criteria: age (between 11 and 17); residing in the Middlesex County, New Jersey; involvement with probation, detention, youth and family services, or family crisis programs; a history of being at risk for delinquency behavior; one involved parent or guardian; no serious mental health issues or substance abuse.

The second group of 48 served as the comparison group. It was "selected by the case managers" and "matched with those who received FFT." The specifics of the matching procedure were not discussed. These youths were part of a case management program that did not offer services but made referrals to the community service providers. They received various types of therapy or mentoring.

The intervention group began with 141 youths who enrolled in FFT, but excluding 28 who did not complete FFT and six who had incomplete data reduced the final sample to 107. The comparison group began with 60 youths, but excluding 10 who did not complete their interventions and two with missing data reduced the final sample to 48.

Assessments/Attrition: Assessments occurred at baseline and posttest for the strengths and needs outcomes and at one year after program completion for the recidivism outcomes. Data were collected between 2006 and 2011. Attrition due to incomplete data was low (3-4%), but exclusion due to program non-completion was larger (17-20%).

Sample:

The sample averaged 15 years of age and was mostly white (54-62%) and male (56%). About half were receiving mental health treatment, and the number of prior convictions ranged from 1.73-2.27.

Measures:

The Strengths and Needs Assessment, a therapist-rated tool used to decide on appropriate treatment and to later evaluate success of the treatment, has seven domains: life domain functioning (13 items), child strengths (9 items), acculturation (3 items), caregiver strengths (6 items), caregiver needs (5 items), child behavioral/emotional needs (9 items), and child risk behaviors (10 items). At baseline, the reliabilities for the seven domains ranged from only .30-.65.

Recidivism measures came from the Family Automated Case Tracking System. Dichotomous variables captured whether the subject was sanctioned for technical violations, reconvicted for a new offense, or re-institutionalized for a new offense within one year of program completion. The recidivism measures also distinguished reoffending for violent, property, and drug crimes.

Analysis:

For the strengths and needs outcomes, the analysis used t-tests to evaluate differences in changes from baseline to posttest across the two conditions. For the recidivism outcomes, logistic regression models controlled for background measures, include prior offending.

Intent-to-Treat: The study violated the intent-to-treat requirement by excluding all participants who did not complete the program.

Outcomes

Implementation Fidelity:

Not discussed.

Baseline Equivalence:

Three of 24 tests showed significant differences between conditions at baseline. The three differences were for prior drug offenses, prior violent offenses, and prior technical violations. In addition, some of the non-significant differences were large (e.g., 52% female in the comparison group versus 40% in the intervention group). Other demographic, criminal history, and strength and needs measures did not differ significantly.

Differential Attrition:

Attrition due to incomplete data was low, but no tests were presented for those excluded because they did not complete the programs.

Posttest:

No significant differences across conditions emerged in t-tests for the seven domains of the strengths and needs assessment. Both conditions showed similar improvements.

Long-Term:

Logistic regression models for recidivism showed three significant effects of the intervention in six tests: drug offenses (OR = .13), property offenses (OR = .07), and technical violations (OR = .33). Two of the three significant outcomes (drug offenses and technical violations) differed significantly at baseline.

Study 16

Regas (1983) used an eight-week version of FFT.

Summary

Regas (1983) used a randomized controlled trial (N = 20) to examine youths in grades 8-9 who teachers rated as hyperactive. After random assignment to three conditions - FFT, group therapy, and a waitlist control group - the youths and their families were followed through an eight-week posttest with measures of family relationships, hyperactivity, and inappropriate behavior.

Regas (1983) reported at posttest that, relative to the control group, the FFT group had significantly lower:

• Hyperactivity and aggressive behavior as rated by teachers and parents

Evaluation Methodology

Design:

Recruitment: Using a population of eighth and ninth graders from three junior high schools in Indiana, the study had staff identify students with potential hyperactivity problems, and two teachers then rated the potentially hyperactive students on the Conners Teacher Rating Scale. Those rated as hyperactive by both teachers were invited with their parents to participate. Of the 46 students identified by the teachers (6% of those in the schools), 20 families agreed to participate.

Assignment: The 20 families and the hyperactive adolescents were first matched into male and female pairs and then randomly assigned to FFT (n = 6, four therapists), group therapy (n =7, two leaders), or a waitlisted control group (n = 7). The group therapy sessions included youth only, while FFT sessions included the youth and family members.

Assessments/Attrition: Assessments occurred at pretest and posttest (end of the eight-week program) for all three conditions. A follow-up occurred five months after the posttest for the two treatment groups but not the waitlisted control group. Two of the 20 families dropped out (10%), one from group therapy and one from the control group.

Sample:

The youth sample included eight females and 10 males with a mean age of 13.83. The 32 parents (18 mothers and 14 fathers) had a mean age of 39.19, with 75% having attended or graduated from high school, 16% having attended or graduated from college, and 9% having attended graduate school. Most (81%) of the parents were employed outside the home. Although four of the mothers participated in the study without their spouses, only two (6%) of the parents were presently separated or divorced. At some point, however, 54% of the sample had been divorced. Half had been married once, 38% had been married twice, and 13% had been married three times.

Measures:

The study relied on six instruments with 27 scales or subscales that family members, parents, youth, or teachers completed. The author reported reliability and validity results from other studies but not for the study sample. Most of the reliability and validity information applied to the full scales rather than to the subscales used in the analysis.

The study author collected all the data, but the measures appeared to minimize researcher subjectivity. The parents and teachers knew which children were treated, possibly influencing their responses, but they were not involved in delivering the program

• The Family Concept Inventory consisted of 80 items with 12 subscales that all family members completed. It measured healthy values, feelings, and attitudes about the family.
• The Kvebaek Family Sculpture consisted of seven subscales that all family members completed. It measured perceptions of family closeness/distance.
• The Connors Teacher Rating Scale consisted of 39 items and five subscales that teachers competed. It measured hyperactivity.
• The Werry-Weiss-Peters Activity Scale consisted of 31 items and a single global score that parents completed. It measured hyperactivity or inappropriate activity.
• The Parent Checklist of Child's Behavior consisted of 24 items and a single global score that parents completed. It measured hyperactivity and aggression.
• The Tennessee Self Concept Scale Adolescent consisted of 100 items and a single global score that youth completed. It measured positive self-concept or self-esteem.

Analysis:

Most analyses used analysis of covariance with the pretest score as a covariate and Newman-Keuls multiple comparison tests of adjusted means. For the follow-up of the two treatment groups only, the repeated measures analysis of covariance focused on the change from posttest to follow-up.

Intent-to-Treat: The analyses used all 18 subjects with posttest data. However, it appears that no effort was made to follow up with the two families who dropped out.

Outcomes

Implementation Fidelity:

No quantitative measures were presented.

Baseline Equivalence:

Tables 4-7 compared 12 baseline outcomes and demographic characteristics across the three conditions using 18 of the 20 randomized families. The tests showed two significant differences, one on the number of times married and one on the number of times divorced. For both, the FFT group scored lowest. Five additional tests described on page 53 indicated a significant difference in income.

Differential Attrition:

Not examined.

Posttest:

To briefly summarize, FFT showed consistent significant effects on family relationship measures but more limited effects on youth behavior. For the seven measures of youth hyperactivity at posttest, the FFT group did better than the control group on four. However, the FFT group did not differ significantly from the group therapy group at follow-up.

Family Concept: Of the 12 subscales measuring healthy family relationships, 11 showed significant ANCOVA results at posttest (Table 10), with the FFT group having significantly higher scores than both the control group and the group therapy group (Table 11). At follow-up involving only the two treatment groups, the FFT group scored significantly higher than the group therapy group on three of 12 subscales (Table 12).

Kvebaek Family Sculpture: Of the seven subscales measuring distance between family members, five showed significant ANCOVA results at posttest (Table 16), with the FFT group having significantly better scores than both the control group and group therapy group (Table 17). The follow-up analyses involving only the two treatment groups showed no significant differences (Table 18).

Connors Teacher Rating Scale: Of the five subscales measuring hyperactive and problematic behavior at school, three showed significant ANCOVA results at posttest (Table 22), with the FFT group having significantly lower scores than the control group but not the group therapy group (Table 23). The follow-up analyses involving only the two treatment groups showed no significant differences (Table 24).

Werry-Weiss-Peters Activity Scale: The parent-rated scale measuring hyperactive and inappropriate behavior at home showed a significant ANCOVA result at posttest (Table 28), with the FFT group scoring significantly lower than the control group but not the group therapy group (Table 29). The follow-up analyses involving only the two treatment groups showed no significant differences (Table 30).

Parent Checklist of Child's Behavior: The global scale of hyperactive and aggressive behavior at home showed no significant condition differences at posttest or follow-up (Tables 34-35).

Tennessee Self Concept Scale: The global scale of positive self-concept showed no significant condition differences at posttest or follow-up (Tables 39-40).

Long-Term:

Not examined.

Study 17

Summary

Smith-Hollimon (2005) used a partially randomized controlled trial (N = 86) to examine youths adjudicated through a local juvenile probation department. The three conditions - an FFT group, a probation-as-usual control group, and a non-randomly assigned group receiving other mental health services - were followed through the end of probation with measures of youth functioning, parent functioning, family functioning, and youth recidivism.

Smith-Hollimon (2005) found no significant program benefits.

Evaluation Methodology

Design:

Recruitment: The sample consisted of the population of youth adjudicated through a local juvenile probation department starting in October 2001 (N =86). This population resided in small Midwestern towns and rural communities. The youths and their families were excluded from the eligibility for FFT or the control group if the adolescent was involved in inpatient or other counseling services, needed crisis care, or had no family involvement. These adolescents and families comprised a separate probation group called "other mental health services."

Assignment: The chief juvenile probation officer determined whether the adolescent was eligible for random assignment based on a checklist, and if so, the officer randomly assigned the adolescent and family to receive either FFT (N = 27) or a supervised probation control group (N = 30). FFT received the supervised probation services plus the program therapy. A third group ineligible for FFT was non-randomly allocated to other mental health services (N = 29).

The study noted that adolescents and families randomly assigned to receive the FFT treatment completed an informed consent form. With no information on the consent rates and no requirement for consent among the control group, it is possible that consent following assignment may have compromised the randomization. In addition, the analysis may have further compromised the randomization by combining the non-randomly assigned third group with the two other randomly assigned groups.

Assessments/Attrition: The posttest assessment occurred at the final meeting of the adolescents with their probation officer upon discharge. The author provided no information on the length of the follow-up period or on differences in the period across conditions. At pretest, Tables 25-27 show sample sizes ranging from 14 to 54, with the lowest Ns for father reports (16% of the sample of 86) and the highest Ns for youth reports (63% of the sample of 86). At posttest, Tables 31-33 show sample sizes ranging from 10 to 52. Compared to the pretest, attrition was low, but compared to the assigned sample of 86, attrition was high.

Sample:

This population was primarily lower SES (57.3% below $34,999), and many families relied on government assistance for medical insurance. The sample was 33% female, 67% male, 93% Caucasian, 4.7% African American, and 2.3% Hispanic.

Measures:

The study evaluated four composite measures. The author reported adequate reliability alphas or, in the case of behavioral functioning, an association with recidivism. For the study data, Tables 13-18 show component correlations among the subscales used to create the measures. The author stated only that the positive and significant correlations justify the creation of the composite measures used in the analysis.

Youth Functioning. The youth-reported measure is a composite of subscales from the Problem Oriented Screening Instrument for Teenagers (POSIT) and the Problem Oriented Screening Instrument for Parents (POSIP). The subscales measured substance abuse, mental health, peer relations, aggressive behaviors, school performance, and social skills.

Parent Functioning. The parent-reported Outcome Questionnaire-45.2 measured parent functioning and covered 1) symptom distress with items on depression and anxiety symptoms; 2) interpersonal relationships with items on satisfaction and problems with family and work; and 3) social roles with items on dissatisfaction, conflict, distress, and inadequacy in tasks related to their employment, family, and leisure.

Family Functioning. Youth and parents completed the Family Assessment Measure III, the family relations subscale of the Problem Oriented Screening Instrument for Teenagers (POSIT), and the Problem Oriented Screening Instrument for Parents (POSIP). The FAM-III subscales measured task accomplishment, role performance, affective expression, involvement, control, and values and norms. The POSIT and POSIP measures included items on substance use and abuse, mental health, family relations, peer relations, aggressive behavior, and delinquency.

Behavioral Functioning. The probation officer completed the Washington State Juvenile Court Prescreen, which measured criminal history and social history and, based on the two scores, provided an overall level of risk.

Analysis:

The analysis used a repeated measures (time) MANOVA factorial design with three groups and four outcomes (youth, parent, family, and behavioral functioning). The models used baseline outcomes as covariates if they differed across conditions and included a time-by-services interaction term.

Intent-to-Treat: The study did not make the sources of missing data clear.

Outcomes

Implementation Fidelity:

In terms of attendance, 67% of families completed treatment, and 33% of families dropped out of treatment. Families completed an average of 12 sessions. Table 41 summarizes results in which therapist adherence was rated in four categories. The figures show 12% were rated as low, 18% as low average, 29% as high average, and 41% as high.

Baseline Equivalence:

Tables 19-30 examined baseline differences in about 20 sociodemographic and outcome measures. Only family income showed a significant difference, with the control group having lower levels.

Differential Attrition:

Not examined.

Posttest:

The key results came from tests for time-by-treatment interactions in Tables 34-40. Of eight tests, only one, family functioning as rated by the mother, reached statistical significance, but comparison of posttest differences on the two component scales showed no significant differences (pp. 116-117).

Long-Term:

Not examined.

Study 18

Summary

Erickson (2008) used a quasi-experimental design (N = 80) to examine youths who had been adjudicated for a sexual offense in Washington State. After non-random assignment into two conditions - FFT and parole treatment as usual - based on the region of adjudication, the study surveyed families about youth and parent problems at 3-4 months after the program start and examined recidivism two years after the program start.

Erickson (2008) found no significant program benefits.

Evaluation Methodology

Design:

Recruitment: Participants included 80 adolescents living in Washington State who had been adjudicated for a felony sexual offense, been released from residential care, and were starting their parole. The Juvenile Rehabilitation Administration chose the participants based on case data and risk assessment over the study period of 2000-2007. Only those with an involved parent and designated Level I or Level II sex offenders were eligible. Based on data from other populations, the author suggested that the sample was relatively low-risk for juvenile sexual offenders.

Assignment: The quasi-experimental design non-randomly assigned participants into two groups based on the place of adjudication. The 41 participants assigned to the 3-4 month FFT intervention resided in state regions 3 and 4, while the 39 participants assigned to the treatment-as-usual control group resided in state regions 5 and 6. The control condition involved a multi-modal cognitive-behavioral approach and an emphasis on psychoeducation that lasted over the two-year parole period.

Assessments/Attrition: Assessments occurred at baseline (before the start of parole), 3-4 months after the parole start date (posttest), and two years after the parole start date (long-term follow-up given the 3- or 4-month program). Only two participants were lost (2.5%), one because the family attended only two FFT sessions and the mother was reportedly intoxicated during these sessions, and one because the control participant died while on parole.

Sample:

The sample was 99% male, with a mean age of 15.3 years. Most youths were white (78%), followed by black (9%), Asian/Pacific Islander (6%), Hispanic (5%), and Native American (1%). The mean age of the youth's first criminal offense was 13 years. Most youths had only sexual felony adjudications (86%), whereas the remainder had a criminal history involving both sexual and nonsexual felony adjudications (14%).

Measures:

Data came from self-reports of the youth and parents and from existing youth records available from the Juvenile Rehabilitation Administration and the Washington State Institute for Public Policy. Measures were obtained at the same time for both conditions, but the periods of treatment differed. As a result, the two-year follow-up period for recidivism included many post-treatment months for the FFT group, but the control group remained on parole for the full two years.

Sexual recidivism was measured as the number of adjudications for a misdemeanor or felony sex crime for the two years following release from a residential institution.

Nonsexual recidivism was measured as the number of adjudications for a misdemeanor or felony nonsexual crime for the two years following release from a residential institution. Felony nonsexual adjudications were further classified as overall or violent.

Individual symptomatology was measured as emotional and behavioral symptoms and problems reported by the youth and parent. The measure came from the Outcome Questionnaire-45. The author cited other studies for reliability and validity.

Parent report of youth's problems was measured by the parent's description of their child's troublesome situations, behaviors, and moods. The measure came from the Youth Outcome Questionnaire. The author cited other studies for reliability and validity.

Analysis:

For recidivism, logistic regression was used to control for criminal history. For symptomatology and youth problems, a mixed-model factorial repeated-measures analysis of variance was used to compare change over time between pre-treatment and post-treatment mean scores.

Intent-to-Treat: One participant was dropped for not attending the intervention but represented only a very small part of the sample.

Outcomes

Implementation Fidelity:

On average, program participants attended 15.75 sessions and 3.60 booster sessions. The mean length of the treatment was 4.68 months without including booster sessions and 11.14 months with booster sessions. The program cost was substantially lower for FFT than the usual two-year treatment.

Baseline Equivalence:

A MANOVA tested for differences between the conditions on five baseline measures. The results indicated no significant differences.

Differential Attrition:

Attrition was minimal.

Posttest:

Individual Symptomatology. The intervention showed no significant effects (Tables 18-20).

Parent Reports on Youth Problems. The intervention showed no significant effects (Tables 21-22).

Long-Term:

Sexual Recidivism. There were no significant condition differences, as no one in either condition recidivated.

Non-sexual Recidivism. Tables 16-17 showed one significant effect, but it was iatrogenic. The intervention group had significantly higher misdemeanor recidivism. There was also some evidence of an iatrogenic moderation effect such that the intervention group had higher recidivism than the control group when the age of first crime was older.

Study 19

Summary

Baglivio et al. (2014) used a retrospective quasi-experimental design with propensity score matching in a study of juvenile offenders in Florida. The design compared an FFT treatment group with a Multisystemic Therapy treatment group (but without a treatment-as-usual control group). Of the initial sample of 2,302, the study matched 628 in each group on baseline measures. The recidivism outcomes were assessed during treatment and 12 months after the end of treatment.

Baglivio et al. (2014) found that, relative to a group receiving a different therapy, the FFT group had significant fewer:

• Offenses during treatment but not thereafter

Evaluation Methodology

Design:

Recruitment: The study included all juvenile offenders under the care of the Florida Department of Juvenile Justice who were referred to FFT or Multisystemic Therapy (MST) between July 1, 2009, and June 30, 2011 (N = 2,312 youth). The sample was reduced to 2,203 after dropping 109 youths who did not have risk-assessment information, moved out of the area during service provision, had either been hospitalized or died before or during service provision, were from out of state, or were deemed as not meeting the criteria to receive treatment services.

Assignment: The study used a retrospective quasi-experimental design with propensity score matching. Of the 2,203 youth, 629 located at 10 sites were referred by probation officers to Multisystemic Therapy and 1,574 located at 11 sites were referred to FFT. The matching identified 628 in each group for analysis. There was no treatment-as-usual control group. The authors said that probation offers used the same eligibility criteria in assigning youth to both treatments, noting that "MST and FFT were very rarely available at the same time to the same sites." They also said that "There was no decision making that occurred for deciding one youth must attend MST whereas another youth is more appropriate for FFT." However, statements indicate that matching was done on youth residing across different sites.

The propensity score model used 13 covariates to predict the type of treatment referral: gender, race, region of the state, age at admission to service, overall risk to re-offend, criminal history, a global social risk measure, most serious prior offense, age at first offense, antisocial peer association, alcohol use, drug use, and parental authority. These measures were said to include the hypothesized reasons a probation officer would refer a youth to the treatments. The matching used a nearest-neighbor algorithm without replacement and the standard caliper of .05 (a caliper of .01 gave similar substantive results). The study conducted separate matchings for the full sample, males, females, whites, non-whites, low-risk youth, and moderate-high to high-risk youth.

Assessments/Attrition: The outcomes were measured both during the intervention and 12 months after the intervention ended. The retrospective design using administrative records meant there was no attrition.

Sample:

The sample was 72% male and 46% non-white. The criminal history score, which ranged from 0% to 74% of the maximum of 100, averaged 26% for the sample.

Measures:

Data for this study came from juvenile and adult records of the Florida Department of Juvenile Justice.

Offense during service. This dichotomous measure captured any new criminal offense referral (arrest) or violation of probation that was filed during the time period the youth was receiving or enrolled in one of the treatments.

Recidivism. This dichotomous measure captured whether the youth was adjudicated or convicted for an offense that occurred within 12 months of termination of service.

Analysis:

The analysis used t-tests for condition differences at the follow-up assessments.

Intent-to-Treat: The analysis included youth who both did and did not successfully complete the services.

Outcomes

Implementation Fidelity:

FFT had a completion rate of 76.5% and an average length of service of 95 days. The authors noted that FFT was substantially less costly than Multisystemic Therapy.

Baseline Equivalence:

The two conditions differed significantly on eight of 13 baseline covariates before matching but showed no significant differences after matching. The authors further reported that none of the covariates differed by an effect size greater than 0.25. The same results showed in the separate tests of baseline equivalence for the matched samples of males, females, whites, non-whites, low-risk youth, and moderate-high to high-risk youth.

Differential Attrition:

No attrition.

Posttest:

For the full matched sample, the outcome measure of offenses during treatment was significantly lower for the FFT group than the Multisystemic Therapy group. For the six subgroups, FFT did significantly better in two - females and low-risk youth. With the Bonferroni correction for multiple tests, the effects remained significant for the two subgroups but not overall.

Long-Term:

For the full matched sample, the outcome measure of recidivism during the 12 months after the end of treatment did not differ significantly across conditions. For the six subgroup tests, FFT did significantly better for one, moderate-high to high-risk youth. The effect did not remain significant with the Bonferroni correction for multiple tests.

Study 20

Summary

Darnell and Schuler (2015) used a retrospective quasi-experimental design with propensity score matching to study youths ages 11-18 who were being released from a court-ordered out-of-home placement. The design compared 524 youths receiving FFT to 7,434 youths receiving standard probation. The key outcome measured return to out-of-home placement over the 36 months after initial release.

Darnell and Schuler (2015) found that, relative to the comparison group, the FFT group had significantly lower:

• Out-of-home placements but only in the first few months and not thereafter

Evaluation Methodology

Along with FFT, the study also examined an adaptation of FFT called Functional Family Parole (FFP) that combined FFT with supervised probation and thus differed from the original program.

Design:

Recruitment: The sample consisted of youths ages 11-18 who were eligible to receive FFT and FFP after being released from a court-ordered out-of-home placement, as long as they did not pose a significant risk to the community. Out-of-home placements included foster care, group homes, and psychiatric hospitals. The youths were beginning probation supervision in a large city, were primarily African American and Hispanic, and were returning home to live with their families. The study covered the period from July 1, 2007, to January 1, 2012.

Assignment: This retrospective quasi-experimental design used propensity score matching. It assigned the youths into four conditions: 1) FFT with standard probation (N = 524), 2) FFP instead of standard probation (N = 216), 3) both FFT and FFP (N = 539), and 4) a standard-probation comparison group (N = 7,434). The comparison group either did not receive FFT or FFP during the study period or were released during the previous fiscal year, before implementation of the interventions (N=1,442). Referrals to FFP were determined by provider availability and caseload. Referrals to FFT were based on provider availability and the probation supervisors' judgment that the family was capable of engaging in the intervention. The comparison group met face-to-face with the probation officers once a month and received referrals to support services as needed.

The propensity score matching used a weighting approach to equalize conditions on: gender, race/ethnicity, age at study start, age at first arrest, age at first felony, age at first out-of-home placement, prior arrests, prior out-of-home placements, geographic divisions of the service area, and prior petitions of various types (i.e., battery, assault, burglary, petty theft, robbery, and vandalism). Four propensity score models were fit, one for each condition; this meant that the FFT group was matched with all other conditions rather than with the comparison group only. The authors noted that a referral to FFT depended on unmeasured and unmatched judgments that officers made about the family's ability to engage in services.

Assessments/Attrition: The period of observation began with the start of parole, which immediately followed release from out-of-home placement, and ran through October 2012. The maximum period lasted over seven years, but the study restricted the analysis to 36 months after the start of parole.

Sample:

The sample was 78% male, 60% Latino, nearly 30% African American, and less than 10% white. The average age at release was 16.5 years, and the youth had on average 2.1 prior arrests.

Measures:

The measures came from administrative data of the juvenile justice and child welfare departments. The outcome was a subsequent out-of-home placement, including placements in residential camps, juvenile justice secured facilities, relative or non-relative foster care, group homes, and psychiatric hospitals.

Analysis:

Logistic regression with propensity score weighting was used to model new out-of-home placements, with time treated as both polynomials and discrete monthly periods. The models controlled for gender and race/ethnicity (two measures of substantive importance) and age at current release (the least well-balanced baseline measure after weighting).

Intent-to-Treat: The retrospective study used all available data and imputed missing data for covariates in the propensity score model.

Outcomes

Implementation Fidelity:

The study lacked data on fidelity but mentioned that the average number of FFT sessions was 9.1, and the average time to completion was 4.2 months.

Baseline Equivalence:

The study presented no significance tests but computed the averaged standardized difference of each condition with the overall mean. The maximum effect sizes were .42 before weighting and .19 after weighting.

Differential Attrition:

No attrition.

Posttest:

At the end of the 36-month observation period, the FFT group did not differ significantly from the comparison group on cumulative out-of-home placements.

The group effects in Table 2 showed significantly lower out-of-home placements for the FFT group than the comparison group during the first month after the program start. However, the group-by-time, time-squared, and time-cubed effects indicated that the FFT advantage declined thereafter and converged with the comparison group after 16 months.

The group effects in Table 3 covered the first 12 months only. They indicated that the FFT group had significantly lower out-of-home placements in months 1-2, but this difference fell to insignificance by month 3.

Long-Term:

The study did not examine posttest and long-term results separately, but the dissipation of program effects after the first few months indicated that there were no long-term effects.

Study 21

Summary

Peterson (2017) used a retrospective quasi-experimental design with propensity score matching to study youths ages 11-18 who fell under juvenile justice probation supervision in Washington State between January 1, 2010, and September 30, 2012. The design compared 441 youths receiving FFT to 441 matched youths receiving standard treatment. The key outcome measured offending recidivism for at least 18 months after the program start.

Peterson (2017) found no significant program effects.

Evaluation Methodology

Design:

Recruitment: The study examined youth ages 11-18 who fell under juvenile justice probation supervision in Washington State between January 1, 2010, and September 30, 2012. The sample began with 1,764 participants.

Assignment: The retrospective QED used propensity score matching to identify the two conditions. The intervention group included youths who started in the program within six months of their eligibility date (N = 586). The control group consisted of those who were eligible for the program during that same period but did not start the program (N = 1,178). Besides the treatment, there appeared to be two other differences between conditions. First, only those without any prior participation in an evidence-based program were selected for the control group, but the same criterion may not have applied to the intervention group. Second, the recidivism clock began at the time of program eligibility for the control group but began for the intervention group at the time of the program start, which came up to six months after the program eligibility date.

After propensity score matching on the eight domains of the Positive Achievement Change Tool risk-assessment instrument, gender, age, and county of residence, each condition had 441 participants.

Assessments/Attrition: The study followed participants for 30 months after their program eligibility date or program start date (18 months for an offense plus 12 months for a conviction). Given the usual treatment period for FFT, the follow-up period included both posttest and long-term results. The retrospective design meant there was no attrition.

Sample:

The original and matched samples of youths were primarily white (62-66%), with 11-12% identifying as black and 17-20% as Hispanic. Most youths were ages 15-16, and 67-74% were male. About 50-58% were rated as high risk for recidivism.

Measures:

The three outcome measures included misdemeanor recidivism, felony recidivism, and violent felony recidivism. The measures defined recidivism as an offense that occurred within 18 months of the start date and that resulted in a conviction within 12 months of the offense date.

Analysis:

The analysis compared means without controls, presumably with t-tests or chi-square tests.

Intent-to-Treat:

The study noted its reliance on an intent-to-treat design that used eligible participants regardless of their program completion.

Outcomes

Implementation Fidelity:

About 80% of program participants had ratings of therapist adherence. The ratings of not competent (9%), competent (74%), and highly competent (17%) indicated good program fidelity.

Baseline Equivalence:

The means after matching in Table 1 appear similar, but the table did not present significance tests or standardized mean differences.

Differential Attrition:

No attrition after matching.

Posttest and Long-Term:

There were no significant differences between conditions for any of the recidivism measures. In 13 tests for moderation (see Table on page 9), one showed a statistically significant program effect for those with moderate risk levels and one showed a statistically significant program effect for unmatched program completers.

Study 22

Summary

Eeren et al. (2018) used a quasi-experimental design with propensity score matching to compare youths who received Multisystemic Therapy (N = 1,074) or FFT (N = 640) to deal with behavioral problems. The key outcome measured caregiver- and youth-reported externalizing at the end of treatment.

Eeren et al. (2018) found no significant effects showing better outcomes for FFT than the comparison MST group.

Evaluation Methodology

Design:

Recruitment: The sample of 1,714 adolescents came from families bringing their children to treatment at an institute for personality disorders and behavioral problems in the Netherlands. The participants started either FFT or MST between October 2009 and June 2014.

Assignment: The quasi-experimental design used propensity scores to equalize the two treatment conditions. First, clinicians assigned the participants to FFT (N = 640) unless there were indicators that the more intensive MST treatment (N = 1074) was suitable (i.e., when the youth had a high risk of delinquent behavior). Those assigned to MST were more likely to have a court order for treatment and to have more serious problems. The authors noted, however, that the model of assignment left room for interpretation and chance such that the MST and FFT samples showed substantial overlap.

Second, a propensity score model used the large number of covariates listed in Table 1, including the outcomes and sociodemographic characteristics of the youth and caregiver. To keep the full sample of participants, the propensity scores were used as weights, with the MST group weighted as 1, and the FFT group weighted by the odds of their propensity scores. The propensity score matching included all baseline data by using missing data indicators in the model.

Assessments/Attrition: The outcome measures were gathered at the end of treatment, which on average came about six months after the treatment start. Only 697 of the 1,714 participants (40.7%) completed the outcome measures.

Sample:

For the FFT sample, 54% of the adolescents were male and 96% were born in the Netherlands. For the MST sample, 67% were male and 83% were born in the Netherlands. The adolescents averaged 16 years of age and 51-61% lived with two parents.

Measures:

The primary outcome, externalizing problem behavior, was measured with the Child Behavior Checklist completed by the primary caregiver (alpha = .94) and with the Youth Self Report completed by the child (alpha = .88). The three secondary outcomes measured whether or not the child 1) was living at home (rather than being placed out of home), 2) was engaged in school or work for at least 20 hours per week, and 3) had new police contact due to inappropriate or illegal behavior. The measures were gathered routinely by therapists after treatment and in consultation with the primary caregiver. According to the authors, the quality assurance systems used with both programs ensured that these outcomes were monitored by the therapist, the team supervisor, and the team consultant.

Analysis:

The analyses used linear or logistic regression models with the propensity score weights. The 95% confidence intervals of the final treatment effects were estimated using simple bootstrapping. Moderation tests examined models for two subsamples of youth, those with and without a court order for treatment.

Intent-to-Treat: The study did not list reasons for the high level of attrition, but the reliance on data obtained at the end of treatment suggests that only those completing the treatment provided data. That the treatment completion rates for the two conditions were the same as the outcome completion rates indicates that the study did not follow program dropouts. Further, the authors said they estimated "the average treatment effect in the treated."

Outcomes

Implementation Fidelity:

Only 422 (39%) completed MST and only 275 (43%) completed FFT. The mean treatment duration was 196.2 days for FFT, which was above the expected duration (i.e., 90 to 150 days according to the FFT quality assurance system). The mean FFT adherence score as assigned by supervisors and independent consultants was 3.97, which was above the FFT target of 3.0.

Baseline Equivalence:

Tests for baseline equivalence used the analysis sample of 697. Table 1 showed many differences before using the matching weights, but after weighting, all covariates were balanced except for the total score of behavioral problems. Table 1 does not present significance tests for the matched comparisons, but the standardized mean differences after weighting were all lower than .25. Table 2 presents variance ratios and percentile scores to further show that the distributions of the covariates, as well as the means, were similar after weighting.

Differential Attrition:

Attrition rates were similar across conditions (43% for FFT and 39% for MST). However, FFT adolescents who completed the posttest assessment differed significantly from those who did not with regard to their country of birth, living situation, and whether or not they had a court order before treatment (see Table 1 in Online Supplemental Material). Also, MST adolescents who completed the posttest assessment differed significantly from those who did not with regard to their country of birth, living situation, engagement in school or work, whether or not they had a court order before treatment, as well as the country of birth, level of education, and employment status of their primary caregiver, and whether or not this primary caregiver had a partner. Although the study did not provide direct comparisons across conditions, the baseline measures were related more often to attrition in the MST group than the FFT group.

Posttest:

The primary outcome of externalizing did not differ significantly across conditions for either caregiver or youth reports. One of the three secondary outcomes differed significantly, with the MST group showing more engagement with school or work than the FFT group.

Separate analyses of youth without a court-ordered treatment showed significantly less caregiver- and youth-reported externalizing for the MST group than the FFT group (d = .32-.34) but no differences for the three secondary outcomes. Youth with a court-ordered treatment were not analyzed because balance on the baseline measures was not achieved.

Long-Term:

Not examined.

Study 23

Summary

Aultman-Bettridge (2007) used a quasi-experimental design to study recidivism of female delinquents in Washington State. After matching FFT (N = 70) and MST participants (N = 77) with a comparison group (N = 203) on age, gender, and risk of recidivism, the study examined recidivism over a 6- to 18-month period.

Aultman-Bettridge (2007) found no significant program effects.

Evaluation Methodology

Design:

Recruitment: The study began with 173 girls in King County, Washington, who were referred to juvenile court and then to either FFT or Multisystemic Therapy between January 2004 and June 2006. However, 16 could not be linked to data sources, leaving 157 girls in the two treatment groups. A higher proportion of white girls were referred to the programs than in the overall population of court referrals.

Assignment: The numbers don't add up, but the authors stated, "During the 30 months of the study, 157 girls who had filings in King County Juvenile Court were referred to either the FFT {n= 70) or MST (n=77) program." The study then selected a comparison group of 203 girls who had court filings in the same time period but were not referred to the FFT or Multisystemic Therapy treatments. The selections were done to obtain equivalence on race, age, and risk profiles for reoffending.

Assessments/Attrition: The participants were followed from the program start (or assessment date for the comparison group) to six months, 12 months or 18 months later. The follow-up period depended on the starting date, with six-month data available for all youths, 12-month data available for youths entering the program prior to December 2005, and 18-month data available for youths entering the program prior to July 2005. Missing baseline or follow-up data reduced the treatment sample to 102 (65% of the initial 157).

Sample:

As shown in Table 4.3, the all-female sample was primarily white (41-54%) and African American (30-39%). Most (60-66%) were rated as having a high risk for recidivism.

Measures:

The two major outcomes came from court records but, in one case, depended on ratings provided by probation officers. First, measures of the changes from initial assessment to post assessment covered six risk-factor domains: family processes, positive social relationships, substance abuse, mental health symptoms, attitudes toward delinquency, and social skills. The post-assessment data came from reassessments or final assessments completed by the youth's probation counselor on completion of either the program or probation.

Second, recidivism equaled any new filing in King County Juvenile Court that occurred from the program or referral start date to a maximum of 18 months later. The measure was restricted to King County court and therefore missed offenses adjudicated elsewhere in the state, but analyses suggested that the measure provided an adequate estimate of juvenile recidivism. Recidivism took three forms: a dichotomous measure of any re-offense, the number of new filings reported (six months, 12 months, and 18 months, where possible), and the number of days between program start and the first new offense.

Analysis:

The analysis used paired t-tests or chi-square analyses. However, most tests did not directly compare the FFT group with the comparison group.

Intent-to-Treat: The recidivism analysis used all available data, but the analysis of the risk factors focused on FFT completers.

Outcomes

Implementation Fidelity:

The author noted that FFT had a relatively high rate of program dropouts, and service delivery took longer than recommended. Program implementation therefore was not at the level of fidelity that was needed to produce more pronounced positive results.

Baseline Equivalence:

As noted on page 84 and in Table 4.3, the comparison group did not significantly differ from the program group (both FFT and Multisystemic Therapy combined) on three measures (race, age, or risk for re-offending). However, comparison of FFT with Multisystemic Therapy showed that significantly fewer girls at high risk of reoffending were referred to FFT (page 94).

Differential Attrition:

Tests comparing youths who had complete data with those who did not (page 74) found no significant differences on race, program status (completion versus non-completion), specific program (FFT vs. Multisystem Therapy), program site, or the initial risk level (high, moderate, or low).

Posttest and Long-term:

The results did not distinguish between posttest and long-term findings. Overall, the program group - FFT and Multisystemic Therapy combined - did not show improvement on any of the recidivism measures over the comparison group at six months, 12 months, or 18 months. Comparison of FFT with Multisystemic Therapy at 12 months (Table 5.35) found no significant differences between the two programs in 12-month recidivism rates.

Additional analyses of risk factors for the full sample compared FFT and Multisystemic Therapy combined with the comparison group. The results showed no significant differences. The only comparisons of FFT separated from Multisystemic Therapy focused on completers only.

Study 24

Summary

Dunham (2009) used a quasi-experimental design to study recidivism of adjudicated juveniles in Miami, Florida. After matching FFT participants to a group of youths receiving non-family services, the study examined recidivism for 12 months after the program end.

Dunham (2009) found no significant program effects.

Evaluation Methodology

Design:

Recruitment: The study began with youths ages 10-18 who had non-adjudicated criminal histories and, as part of the diversion program, had been referred between the years 2004-2006 to FFT treatment at the Miami-Dade Unit of the Institute for Child and Family Health Subjects. The final FFT sample excluded those who (1) did not attend at least one session (i.e., start services), (2) worked with a therapist who provided FFT services for less than 50 total cases or had less than 12 months of FFT practice experience, (3) lacked data on program completion, criminal history, or follow-up offending, or (4) had pre-treatment or post-treatment charges exceeding three standard deviations above the total sample mean. The resulting FFT condition included 111 youths and their families.

Assignment: The study used a matched comparison group design. The comparison group began with youths ages 10-18 who were referred during the years 2004-2006 to non-family-based probation treatments. The study randomly selected participants from the Juvenile Services Division database using a stratified sampling procedure designed to find matches for the FFT group. The final sample excluded those who (1) did not attend at least one session (i.e., start services), (2) received family-based or family components in the treatment, (3) lacked data on pre-treatment and follow-up criminal history charges, or (4) had total pre-treatment or follow-up charges exceeding three standard deviations above the total sample mean. Of the 232 youths who met the eligibility criteria, 111 were randomly selected to equal the number of FFT youths.

Assessments/Attrition: The follow-up period for recidivism was 12 months after the program end date. The study excluded cases that were missing data from eligibility and therefore did not have formal attrition.

Sample:

The sample was predominately male (69%) and had a mean age at the beginning of treatment of 15.13 years. Most youths were Hispanic (49%) or African American (39%), with a mix of other ethnicities (12%). Most had only a misdemeanor charge prior to treatment (77%), with the remainder having a criminal history involving felony charges (23%).

Measures:

The Miami-Dade County Juvenile Services Department provided information regarding each youth's criminal history, including offenses occurring before, during, and after treatment. Recidivism referred to the commission of a crime (as recorded in the system). Crime severity was measured as the number of crimes committed by the youth multiplied by the severity of the type of crime committed. Several other measures were available only for the FFT group.

Analysis:

The between-condition analyses used logistical regression and linear regression models with controls for the pretreatment crime severity. The analyses of within-group changes included only the FFT condition and lacked a control group.

Intent-to-Treat: The study dropped participants who did not attend at least one session (though it did so for both conditions). In addition, it dropped participants who were treated by less experienced FFT therapists but did nothing similar for the comparison group.

Outcomes

Implementation Fidelity:

Table 11 shows that 90% of the FFT group that started services successfully completed the program.

Baseline Equivalence:

In four tests, the conditions differed significantly and substantially on ethnicity, gender, and pre-treatment crime severity scores. Only age was statistically the same across conditions.

Differential Attrition:

There was no formal attrition after the sample selection, but the study presented no information on differences across conditions due to initial exclusions.

Posttest and Long-term:

The analyses found no significant condition differences for either of the two outcomes, recidivism or crime severity.

Study 25

Many of the results presented in this study combined standard FFT with FFT plus contingency management procedures, but the comparison treatment similarly had two versions without and with contingency management and some results listed separate figures for the two FFT conditions.

Summary

Robbins et al. (2020) used a randomized controlled trial with a sample of 121 youth and parents who were assigned to four conditions: FFT, FFT with contingency management, a motivational treatment, and a motivational treatment with contingency management. The study followed the youth through 12 months from baseline with measures of substance use, externalizing, and family conflict.

Robbins et al. (2020) found, relative to the two non-FFT conditions, significantly lower

• Marijuana use for both FFT conditions combined
• Externalizing for the FFT with contingency management condition

Evaluation Methodology

Design:

Recruitment: Adolescents were referred primarily from the juvenile justice system. Inclusion criteria for adolescents were: (1) 13-17 years of age (2) meets DSM-IV diagnostic criteria for substance abuse or dependence, (3) living at home with the participating parent, and (4) sufficient residential stability to permit probable contact at follow-up. Exclusion criteria for adolescents were (1) evidence of psychotic or organic state of sufficient severity to interfere with the understanding of study instruments and procedures, (2) deemed dangerous to self or others, (3) services other than outpatient treatment were required for the youth (e.g., inpatient, detoxification), and (4) reported substance use of less than 13% of the previous 84 days. A total of 126 participants joined the study.

Assignment: All 126 adolescents were randomly assigned to one of four conditions, but five were lost soon after randomization. The four conditions included:

• Motivational Enhancement Therapy/Cognitive Behavior Therapy (MET/CBT, n =27),
• Motivational Enhancement Therapy/Cognitive Behavior Therapy with Contingency Management (MET/CBT-CM, n = 28),
• Functional Family Therapy (FFT, n = 32), and
• Functional Family Therapy with Contingency Management (FFT-CM, n = 34).

Randomization was done in blocks so that all conditions could begin treatment within a 30-day window. Adolescents in MET/CBT and MET/CBT-CM received group-based sessions over 14 weeks, while those in FFT and FFT-CM received family therapy sessions over 14 weeks from four therapists. The Contingency Management versions used immediate, tangible reinforcers upon evidence of abstinence from substance use or other desirable behavior.

Assessments/Attrition: Along with weekly drug tests during treatment, assessments occurred at baseline, and two, four, eight, and 12 months after randomization. The four-month assessment came at the expected end of the therapy services in all four conditions; the 12-month follow thus came eight months after the program end. Among the 121 participants, completion rates listed separately by assessment and condition in Figure 1 ranged from 61% (n missing = 11) to 91% (n  missing = 3). The five excluded cases were dropped because two violated protocol (one youth was not living at home, one family was already receiving an evidence-based family therapy), one did not like the randomization outcome, and two did to respond after randomization.

Sample:

Adolescents were predominately male (76%). Most indicated that they were Hispanic (77.0%), while the remaining reported being White, non-Hispanic (13.5%), Native American (7.1%), African American (1.6%) or Asian American (0.08%). Baseline urine drug tests for substance use were positive for 77% of youths. The median family income was $33,344. Caregiver's education level included: no high school diploma (28%), high school diploma or graduate (21%), some college or vocational (32%), or a college graduate/post graduate studies (19%).

Measures:

Substance Use. The outcome measures focused on abstinence. For the last seven weeks of the 14-week treatment period, measures of one-week, two-week, and three-week abstinence came from weekly urine and saliva tests, youth reports, and parent reports. For the full study period, the self-reported Timeline Followback interview was used to establish the percent days of  marijuana use abstinence during the preceding 12 weeks. The authors noted (p. 27) that the low usage rate of other substances (they tested for 11) led to the focus on abstinence from marijuana (the primary drug of choice for the sample). They further noted that the measures of abstinence had an excess number of zeros that skewed the distribution.

Externalizing problems. Youth externalizing was measured using the Youth Self Report (alpha = .90 across all assessments).

Family functioning. Family conflict was measured using the Family Environment Scale for both parents (alpha = .760) and adolescents (alpha = .728).

Analysis:

The analyses used repeated-measures MANOVA or ANOVA models to test for the effects of incentive (contingency-management versus none), treatment modality (FFT versus MET/CBT), and the interaction of incentive by modality. In assessing the efficacy of FFT, the treatment modality effect was crucial (although it showed the effect of both FFT and FFT-CM together). With significant interactions, the analyses compared the effects of each of the four conditions separately. The models used baseline abstinence measures and gender as covariates. Pairwise comparisons of means used Bonferroni adjusted probability levels.

To include the full sample, missing data were imputed using 10 data sets estimated separately within each of the four conditions. The imputation used baseline indices for days of use, gender, age, as well as the parent and youth reports of externalizing behavior.

Intent-to-Treat: The analyses used all participants except five by imputing missing data.

Outcomes

Implementation Fidelity:

The mean number of sessions attended was about 13 for all four conditions, which the authors say (p. 13) demonstrated "strong support for the effectiveness of all four treatments in retaining adolescents in treatment." Experienced therapists reviewed recordings of treatment sessions for fidelity and provided feedback. However, a formal quantitative assessment or rating of therapist fidelity per session was not completed.

Baseline Equivalence:

Tables 1 and 2 present baseline means for the full sample and all four conditions but no significance tests or effect sizes. To test for significant differences, Blueprints staff used the means and standard deviations for the four conditions to perform one-way ANOVAs. One of the 23 baseline means in Tables 1 and 2 differed significantly across the conditions (p = .037). It appears that one condition (MET/CBT-CM) had substantially lower alcohol use (mean = 1.57) than the other groups (means ranged from 4.18-7.23).

Differential Attrition:

The CONSORT diagram (Figure 1) reveals differences in completion rates across conditions. At the four-month assessment, the MET/CBT-CM condition had a rate of 68% compared to 85-91% for the three other conditions. At the eight-month assessment, the same condition had a completion rate of 61% compared to 81-85% for the other conditions.  Using the WWC criteria for overall attrition and differences in attrition across groups at both the eight-month and 12-month follow-ups, the comparison of attrition in the FFT-only group with the FFT-CM group and the MET/CBT group met the cautious and optimistic standards. However, the comparison of the FFT-only group with the MET/CBT-CM group met neither the optimistic nor cautious standard.

The authors stated that analyses did not show a relationship between baseline variables and completion of the 12-month assessment, but that youth/families attending fewer sessions were less likely to complete the final assessment.

In addition, the use of multiple imputation may moderate the potential for bias from differential attrition.

Posttest:

Weekly Substance Use Abstinence During Treatment: For the three outcomes of one-, two-, and three-weeks abstinence, the two FFT conditions combined had significantly better outcomes for two- and three-weeks abstinence than the other two conditions combined (eta-squared of .058 and .064 respectively).

Adolescent Daily Marijuana Abstinence During Treatment: The conditions with contingency management differed significantly from the conditions without, but there were no differences between the combined FFT treatments and the other treatments.

Adolescent Daily Marijuana Abstinence Post-Treatment: The combined FFT conditions had significant more days of abstinence than the other conditions (eta-squared = 0.034). As the incentive effect was not significant, it suggests that the FFT condition did better than its MET/CBT condition.

Adolescent Externalizing Behaviors Post-Treatment. There was a marginally significant (p = .08) time-by-modality-by-incentive interaction. Pairwise comparisons at four months found that FFT-CM was significantly lower than FFT and MET/CBT-CM and marginally lower than MET/CBT. Pairwise comparison at eight months found that the FFT-CM condition was significantly lower than the FFT and MET/CBT conditions and marginally lower than the MET/CBT-CM condition (p = .062). No condition differences were observed at the 12-month assessment.

For a measure of exceeding the clinical threshold for externalizing, Figure 2 presented changes over time for the four groups but without between-condition tests for differences. Although all four conditions experienced relatively large effect size reductions from baseline, it appears that the reductions for both FFT conditions (d = .55 and .82 at 12 months) were larger than for both MET/CBT conditions (d = .40 and .41 at 12 months).

Family Functioning (Conflict): There was a significant modality-by-incentive interaction, but the authors did not report any of the pairwise comparison. They stated in the discussion that "With respect to family functioning, the analyses did not support an effect of treatment on changes in family conflict."

Long-Term:

Not examined.

Study 26

Gan et al. (2018) offered details on problems implementing FFT in a non-Western culture, but the information on the program evaluation came fully from Gan et al. (2021).

Summary

Gan et al. (2018, 2021) conducted a randomized controlled trial with a sample of 120 youths in Singapore who had been placed on community-based probation and had moderate to severe risk of recidivism. After random assignment of the youths to an intervention group receiving FFT with treatment-as-usual or a control group receiving only treatment-as-usual, assessments of emotional and behavioral problems occurred at the end of the program and the end of probation.

Gan et al. (2018, 2021) found that, relative to the control group, the intervention group had significantly

• lower scores on a scale of emotional and behavioral problems
• greater likelihood of successful completion of probation.

Evaluation Methodology

Design:

Recruitment: The sample targeted youth offenders aged 13-18 years who had been placed on community-based probation in Singapore between December 2014 and March 2018. Participants needed to have a moderate or higher baseline risk severity of recidivism, at least eight months remaining on their probation order, and a stable living arrangement with their caregiver(s). Those already receiving services or having low intellectual functioning, active psychotic symptoms, high risk of suicidal or self-injurious behaviors, or sexualized behaviors were excluded. Of the 369 youth who were screened for eligibility, 120 met the criteria and joined the study.

Assignment: The 120 participants were randomly assigned with an online random number generator to an FFT with treatment-as-usual intervention group (n = 63) or a treatment-as-usual-only control group (n = 57). Participants were blind to the treatment group allocation. Those in the intervention group received FFT as their first program, in addition to standard probation services as prescribed by their probation officers, such as attending programs that addressed offense- or family-related needs.

Assessments/Attrition: Assessments occurred at baseline (time 1), after the first program (time 2), and at the end of probation (time 3). The timing of the assessments was not standardized but varied with the length of probation. The average follow-up duration from the end of the first program to the end of probation was 9.82 months - too short to qualify as a long-term follow-up. According to the CONSORT diagram, 88%, 65%, and 55% completed the first, second, and third assessments, respectively.

Sample:

The sample had a mean age of 16.2 years at the start of probation. Most youths were male (89.2%), and close to half came from low-income families that were eligible for government-funded social assistance (45.8%).

Measures:

The key behavioral outcome, obtained using the Youth Outcome Questionnaire Self-Report Version 2.0, measured emotional and behavioral problems (alpha = .94) and came from youth self-reports.

Two risk and protective factors were examined. Family functioning was measured with the General Functioning subscale of the McMaster Family Assessment Device (alpha = .91) and came from youth self-reports. A measure of successful probation completion during the probation period was obtained from official case closure reports.

Analysis:

For continuous outcomes, the analysis used repeated-measures analyses of variance with group-by-time interaction terms. Tests showing violation of sphericity required adjustment of the degrees of freedom. For the binary outcome, the analysis used logistic regression with controls for baseline recidivism risk and gender.

Missing Data Method: For participants with missing data at the last two assessments, the last observation was carried forward from the previous available time point. It appears that only those missing data at the first assessment were dropped.

Intent-to-Treat: The authors said that "Study attrition at time 1 and time 2 were due to treatment dropout." This implies some overlap between attrition and program completion, but dropouts occurred in both conditions, and the imputation of missing data supports the claim that the analysis was intent to treat.

Outcomes

Implementation Fidelity:

Ten of the 61 youths who commenced FFT did not successfully complete the program, resulting in a dropout rate of 16.4%. Youths who completed FFT spent an average of 4.7 months in the program and underwent an average of 12 therapy sessions. Over the first two years of implementation, 100% of all dissemination adherence and fidelity ratings recorded were above the prescribed cutoff scores. In the third year, 100% of all dissemination adherence ratings and 90% of fidelity ratings were in the target range.

Baseline Equivalence:

Table 1 presents tests for three sociodemographic, two screening, and two outcome measures. None showed a significant difference.

Differential Attrition:

The high attrition rate of 45% and a difference in condition attrition rates of 8% at the end of probation fails to meet either the What Works Clearinghouse cautious or optimistic standard for bias-free attrition. The authors conducted chi-square tests of independence and independent samples t-tests to explore possible differences between program completers and dropouts. They stated, "There were also no differences between trial completers and trial dropouts on the aforementioned variables [sociodemographic characteristics, recidivism risk profiles, and psychometric outcome scores]. This suggests that attrition from the study was approximately random."

Posttest:

The behavioral outcome measure, the emotional and behavioral problems scale, showed a significant group-by-time interaction effect (eta-squared = .043). Post hoc tests by time examined changes within the intervention group rather than across conditions. However, condition differences appear similarly large at both time 2 and time 3 in Figure 2.

The measure of family functioning did not show a significant group-by-time interaction term. However, the intervention group showed significantly greater improvement than the control group on binary measures of reliable change and clinical recovery.

The intervention group successfully completed probation significantly more often than the control group (OR = 3.99).

Long-Term:

Not examined.