Multisystemic Therapy® (MST®)

Study 1

Summary

The study used a randomized controlled trial that assigned 84 youths at risk of out-of-home placement for serious criminal activity to an MST intervention group or a control group receiving usual youth services. Assessments at posttest and 2.4-year follow-up measured aggression, criminal activity, and arrests.

The study reported that, relative to the control group, the MST intervention group showed significantly:

• Lower self-reported delinquency at posttest
• Less aggression with peers at posttest
• Greater family cohesion at posttest
• Fewer arrests and incarcerations at 59 weeks
• Longer time to rearrest through 2.4 years.

Evaluation Methodology

Design: A pretest-posttest control group design, with random assignment to conditions and follow-up for arrest and incarceration measures was used to compare the effectiveness of family preservation using MST versus the usual services delivered by the Department of Youth Services (DYS). Eligible youths were referred by the DYS in yoked pairs, with one youth randomly selected to receive MST (n=43) and the other to receive the usual services (n=41). The study excluded six youths assigned to MST because they did not participate in the program, likely violating the intent-to-treat criterion. Assessments were administered to both groups at approximately the same time.

Therapists with caseloads of 4 delivered the MST program to the treatment group. On average, the treatment duration was 13.4 weeks encompassing 33 hours of direct contact, with 24-hour-a-day case coverage. Depending on the stage of treatment, sessions were held as often as every day or as infrequently as once a week usually in the family's home and ranged in duration from 15 to 90 minutes.

Youths in the usual service condition met monthly with a probation officer who monitored their compliance with court ordered stipulations (e.g., curfew, school attendance, participation with other agencies). Although youths and families were often referred for mental health services, few substantive services were actually delivered.

Pretreatment and posttreatment assessments were completed by 77% of the families in the MST condition (n=33) and 56% of families in the control condition. Criminal histories and demographics of the participants completing both assessments are essentially the same as the larger sample. Tests of attrition showed families who completed both assessments, compared to those who did not, were more likely to be African-American, to have participated in the MST condition, and to have reported higher maternal symptomatology, more structured family relations, and greater social competence in the youth. Attrition analyses on premature treatment terminators were not appropriate for this study because it was not possible to terminate usual services.

Sample: Out of 96 youths who were originally referred by DYS staff because they were at-risk for out-of-home placement due to involvement in serious criminal activity, 84 participated in the study (n=43 in MST and n=41 usual-services). Over half of the youths in the sample had been convicted of a serious violent offense and the sample averaged 3.5 previous arrests. The mean age of the sample was 15.2 years, 77% were male, 56% African American, 42% were Caucasian, and 2% were Hispanic American. The MST and usual services groups did not differ significantly on demographic variables or criminal history.

Measures: A multimethod, multifocus measurement battery was used to assess variables related to the ultimate and instrumental goals. Ultimate outcomes included decreases in criminal activity and incarceration. Instrumental goals included improved family relations, peer relations, and social competence and decreased symptomatology in youths and parents.

Archival records from the time of referral to the MST or control condition were evaluated for postreferral arrests and postreferral incarceration. In addition, the full-scale score of the Self-Report Delinquency scale (SRD) assessed youths' reports of criminal behavior during the previous 4 months. The Family Adaptability and Cohesion Evaluation scales (FACES) were used to assess parental and youth perceptions of family cohesion and adaptability. The Missouri Peer Relations Inventory (MPRI) evaluated parental and youth perceptions of the adolescent's friendships on the dimensions of emotional bonding, aggression, and social maturity. Adolescent symptomatology was assessed through maternal reports on the Revised Behavior Problem Checklist (RBPC). Parental symptomatology was assessed with the self-report Symptom Checklist-90-Revised (SCL-90-R) and adolescent social competence was assessed with the Social Competence scale of the Child Behavior Checklist (SCS-CBC).

Analysis: One-way analyses of variance (ANOVA) were used to evaluate between-groups differences for arrests and incarceration following referral to the project (n=84), and one way analyses of covariance were used to evaluate between groups differences on the SRD and the psychosocial measures at the posttreatment assessment, with the corresponding pretreatment score serving as the covariate (n=56).

Outcomes

Posttest: Shortly after treatment ended, self-reported offenses were lower for MST youth compared to youth receiving usual services (Mean=2.9 vs. 8.6). At 59 weeks postreferral, MST youths had more than half as many arrests (Mean=0.87 vs. 1.52) as youths receiving usual services (58% of MST youths experienced no rearrests vs. 38% of usual services group). MST youths also spent an average of 73 fewer days incarcerated in DYS facilities (Mean=5.8 vs. 16.2) than usual services youth (20% of MST youths vs. 68% of usual services youth experienced incarceration). Aggression with peers decreased significantly for MST youth, while remaining the same for youth receiving usual services. Families receiving MST also reported more cohesion, whereas family cohesion decreased in the usual services condition.

Long-term: A 2.4-year follow-up (Henggeler, Melton, Smith, Schoenwald, and Hanley, 1993) showed that MST doubled the percentage of youth not rearrested, in comparison with the usual services group.

Study 2

Summary

The study used a randomized controlled trial that assigned 47 juvenile offenders to an MST intervention group or a control group receiving usual youth services. Assessments at posttest measured substance abuse and delinquency.

The study found that, relative to the control group, the MST intervention group reported significantly:

• Less soft drug use at posttest.

Evaluation Methodology

Design: A pretest-posttest control group design, with random assignment to conditions and follow-up for arrest and incarceration measures was used to compare the effectiveness of family preservation using MST versus the usual services (US) delivered by the Department of Youth Services (DYS). At the time of this study, assessment data had been obtained for 47 youths, 28 in the MST condition and 19 in the US condition. In the MST condition, 89% of the referred families participated in treatment.

Therapists with caseloads of 4 delivered the MST program to the treatment group. On average, the treatment duration was 13.4 weeks encompassing 33 hours of direct contact, with 24-hour-a-day case coverage. Depending on the stage of treatment, sessions were held as often as every day or as infrequently as once a week, usually in the family's home, and ranged in duration from 15 to 90 minutes.

Youths in the usual service condition met monthly with a probation officer who monitored their compliance with court ordered stipulations (i.e., curfew, school attendance, participation with other agencies). Although youths and families were often referred for mental health services, few substantive services were actually delivered.

Sample: The mean age of the sample was 15.1 years, 72% were male, 74% African American, and 26% were Caucasian. The MST and usual services groups did not differ significantly on demographic variables or criminal history.

Measures: Self-reported substance abuse was measured with the soft drug use and hard drug use subscales of the self-report delinquency scale in the National Youth Survey.

Analysis: One-way analyses of variance (ANOVA) were used to evaluate between-groups differences for self-reported drug use.

Outcomes

Post-test: Self-reported soft drug use was significantly lower at posttreatment for the youths in the MST condition than for the youths in the Usual Services condition. Differences between samples for hard drug use could not be tested due to a very low base rate (i.e., only three youths reported such use).

Study 3

Summary

The study used a randomized controlled trial that assigned 176 juvenile offenders ages 12-17 to an MST intervention group or an individual therapy control group. Assessments at posttest through a 25-year follow-up measured problem behavior, arrests, and family relations.

The study found that, relative to the control group, the MST intervention group showed significantly:

• Decreased parent-reported youth behavior problems
• Improved family relations
• Fewer arrests (through 21.9 years from treatment end)
• Fewer arrests of siblings.

Evaluation Methodology

Design: A pretest-posttest control group design, with random assignment to conditions and multiple follow-ups for arrests, was used to compare the effectiveness of MST with that of individual therapy (IT). Participants were 200 twelve- to seventeen- year old juvenile offenders and their families who were referred to the project by juvenile court personnel and agreed to participate in a pretreatment assessment session; five other families were referred but did not agree to participate. Twenty-four (12%) of the families refused service. The remaining 176 families were randomly assigned to receive either MST (n=92) or IT (n=84). Of these, 140 (79.5%) completed treatment, and 36 (21.5%) dropped out. Dropout rates were not significantly different for the treatment or control group. Statistical tests showed no between-group differences in the criminal histories or demographic characteristics for the IT dropouts or the MST treatment refusers. Pre and posttreatment assessments were available for 126 families. The average treatment for MST involved 23.9 hours and 28.6 hours for the IT completers. These means were significantly different.

Sample: Juvenile offenders and their families with (a) at least two arrests (b) currently living with at least one parent figure, and (c) showed no evidence of psychosis were included in the study. The juvenile offenders were involved in extensive criminal activity as evidenced by their average 4.2 previous arrests and the fact that 63% had been incarcerated. Their average age was 14.8 years; 67% were male; 70% White, and 30% African American; 65% were from families characterized by low socioeconomic class; and 53% lived with two parental figures.

Measures: Individual adjustment included measures of psychiatric symptomatology in mothers, fathers, and adolescents and a measure of adolescent behavior problems assessed through mothers' reports. Family relations were measured by parental and adolescent perceptions of family functioning and video-recorded observed family interactions. Maternal and teacher perceptions of peer relations were evaluated with the 13-item Missouri Peer Relations Inventory (MPRI). Juvenile court, local police, and state police records, collected an average of 3.95 years, were used to obtain data on post-probation arrests. Substance abuse was measured as an arrest for a substance-related offense.

Analysis: Repeated measures multivariate analyses of variance (MANOVAs) and ANOVAs were used to evaluate whether significant changes pre- to post assessment were experienced by the 70 MST youths and families or the 56 IT youths and families who completed pretreatment and posttreatment assessments. Survival analysis was employed to determine the proportion of participants not arrested in each group by the length of time from release from probation. Additional analyses examined the number of arrests and the seriousness of those arrests among recidivists in the MST and IT groups. Hierarchical multiple regression analysis was used to evaluate the effect of treatment on violent offending and to evaluate the effects of potential moderators (age, race, social class, gender, pretreatment arrests) of MST effectiveness.

Outcomes

Post-test: At posttest (within one week of treatment completion), MST resulted in decreased symptomatology in parents (self-reports), compared to an increase in the IT families (mothers) and no change (fathers); and decreased behavior problems in MST youth (parent reports), whereas mothers of IT youth reported an increase in behavior problems. MST families experienced favorable effects on perceived family relations (increased cohesion and adaptability) whereas cohesion and adaptability decreased in the IT condition. Observed family interactions among the MST families improved (increased supportiveness and decreased conflict-hostility across family dyads), compared to IT families in which dyadic relations deteriorated (mother-adolescent supportiveness), conflict-hostility increased (father-adolescent), or no change indicated (on measures of supportiveness and conflict-hostility).

4 Year Follow-up: By the end of 4 years, 26.1% of youth in the MST group had been arrested at least once, compared to 71.4% of the IT group. Specifically, MST completers had lower recidivism rates (n=77, 22.1%) than MST dropouts (n=15, 46.6%), IT completers (n=63, 71.4%), IT dropouts (n=21, 71.4%), and treatment refusers (n=24, 87.5%). Additionally, recidivists in the MST group had been arrested less often and for less serious crimes than IT youth. MST youth were less likely to be arrested for violent crimes (e.g., rape, attempted rape, sexual assault, aggravated assault, assault/battery) following treatment than IT youth. MST youth also had a significantly lower rate of substance-related arrests than IT youth (4% vs. 16%) (Henggeler et al., 1991). MST was shown to be equally effective with youths of different gender and ethnic backgrounds.

13.7 Years Post Treatment (Schaeffer and Borduin, 2005): This long-term follow-up sample consists of 87 MST and 78 IT (usual treatment) individuals. Participants were on average 28.8 years old. Both juvenile and adult substantiated (i.e., charges that were dismissed at trial were excluded) criminal records were obtained. Survival analysis to time of first arrest for various types of offenses was conducted. MST participants were at lower risk of rearrest (i.e., more likely to survive) during follow-up than were IT participants. By the end of 13.7 years, 81% of the IT group had been arrested at least once, compared with 50% of the MST group. With regard to specific offenses, MST participants were at lower risk of arrest for violent offenses, nonviolent offenses, and drug offenses during follow-up than IT participants. Additionally, MST participants had 54% fewer arrests, were sentenced to 61% fewer days of confinement in adult detention facilities and 37% fewer days of probation as adults than were comparison counterparts.

21.9-Year Follow-Up (Sawyer and Borduin, 2011): This study began with the same 176 subjects examined in earlier studies and, like the other follow-ups, it gathered data on arrests from Missouri official records. After 21.9 years from the end of the MST program, subjects had reached an average age of 37.3 years. A search of juvenile and adult criminal arrest records in the state identified arrests that occurred after the program and that led to conviction. Those without recorded arrests and with a Missouri driver's license were counted as having lived in the state without an arrest and conviction. Note that the records search was done for subjects who completed the program as well as for subjects who had dropped out.

The records search determined that 84.1% of the sample (n = 148) had lived in the state since the prior follow-up. The remaining 15.9% of the sample for whom residency could not be verified were considered lost to follow-up. Attrition rates did not differ significantly across the treatment and control groups, and there were no differences in the pretreatment criminal histories or demographic characteristics of participants included in the follow-up versus participants considered lost to follow-up. The authors concluded that the loss of subjects leaving the state and unavailable for follow-up did not bias the findings.

Outcome measures included arrests with convictions for felonies, violent felonies, nonviolent felonies, and misdemeanors (excluding traffic). The outcome measures also included civil suits involving family instability or financial problems. Since the study sampled subjects after an arrest, the new arrests indicated recidivism.

The results revealed clear long-term benefits of MST. The treatment group had significantly better outcomes for four of the six measures: felonies, violent felonies, nonviolent felonies, and civil suits involving family instability. For example, 54.8% of control participants versus 34.8% of MST participants had been rearrested at least once for a felony offense by the end of the 21.9-year follow-up period. The odds ratio of 2.27 for control subjects relative to intervention participants indicates a medium-sized effect. The largest group difference occurred for violent felonies: The odds ratio of 4.08 for arrest for a violent felony translates into a large effect.

Survival analyses, which model the time to arrest or censoring rather than just the occurrence of an arrest, confirmed the descriptive results. The hazard ratio for all felonies of .616 suggests a weak to medium effect of the program. Further, zero-inflated Poisson regression models examined program effects on the number of rearrests and length of sentences. The results proved less strong for these measures: The program significantly reduced only the number of arrests for misdemeanors. Tests for moderation by background factors (age, socioeconomic status, pretreatment arrests) in the zero-inflated Poisson regression showed that the results proved similar across subjects from divergent backgrounds.

Columbia, Missouri, 25 Year Sibling Follow-Up (Wagner et al., 2014):

To examine the general effects of MST beyond the youth participants, the 25-year follow-up measured arrest rates and incarceration times for the closest sibling of those targeted in the original MST study.

Evaluation Methodology

Design: Of the 176 randomized juveniles included in the original study, 129 had siblings in the home during the period of intervention. These 129 closest-in-age siblings, rather than the original sample, served as subjects in the 25-year follow-up.

Assignment: The original subjects, but not the sibling subjects, were randomized to the treatment and control groups. Among the 129 siblings participating, 67 participants (51.9%) belonged to the treatment group and 62 participants (48.1%) to the control group.

Attrition: The study obtained publicly available Missouri court records of the closest sibling to the original participants in the study. Of the 129 available participants, 19 (14.73%) appeared to have left the state, lacked data on court records, and were designated as lost to follow-up.

Sample: Of the selected participants, 60% were the younger sibling of the MST participant, while 40% were older siblings. Half of the siblings included in the study are male. The majority of participants are white (86.4%).

Measures: The study used publicly available adult (age 17+) criminal records. Researchers coded crime classification (felony or misdemeanor) and date of arrest. In addition, the study used the number of days sentenced to incarceration or probation. The inability to obtain data for states other than Missouri might limit the measures, as might the inability to obtain data on juvenile offenses.

Analysis: Based on the publicly available criminal records, the study analyzed the relative odds of arrest of control and treatment group participants. In addition, the study reported the cumulative survival function for arrests. Finally, the number of arrests and years sentenced were analyzed using a Zero Inflated Poisson Regression, which evaluates the impact of treatment condition on the number of posttreatment arrests and years sentenced to incarceration or probation.

Intent-to-Treat: The analysis excluded the 19 individuals who were lost to follow-up because of residence outside the state but otherwise used all available data.

Outcomes:

Baseline Equivalence: A comparison of the treatment and control siblings for the analytic sample of 110 subjects showed no significant differences on demographic variables, but the study lacked baseline measures for other variables.

Differential Attrition: No baseline differences on condition or demographic variables were reported between those located and those lost to follow-up, but the study lacked baseline measures for other variables.

Long-Term:

• Siblings in the control group were significantly more likely to have been arrested at least once as compared to siblings in the treatment group (OR = 3.36).
• A log-rank test of the survival functions for the treatment and control group showed that the treatment group was at a significantly lower risk of arrest (p=.004) and misdemeanor arrest (p=.02) during the follow-up period than the control group.
• The Zero Inflated Poisson Regression analysis included the odds estimate for any arrests and rate estimates for number of arrests. The treatment significantly reduced odds estimate of arrest for any crime, any felony, and any misdemeanor but did not influence the number of arrests.
• The Zero Inflated Poisson Regression also examined adult sentencing, finding that the treatment significantly reduced the odds of incarceration and probation and the years of the sentence for incarceration and probation.

Study 4

Summary

The study used a randomized controlled trial that assigned 155 juvenile offenders to an MST intervention group or a usual-services control group. Assessments at posttest and 1.7-year follow-up measured re-offending and family relations.

The study found that, relative to the control group, the MST intervention group showed significantly:

• Better emotional-behavioral functioning
• Fewer days incarcerated.

Evaluation Methodology

Design: This study followed a 2 x 2 x 2 Condition (MST vs. usual services [US] x Time (pretest vs. posttest) x Site (Site 1 [S1] vs Site 2 [S2]) mixed factorial design, with random assignment. Eighty-two families were randomly assigned to MST treatment conditions and 73 to usual services. Families from each group were paired to further control for historical and related threats to validity. Nine of the families could not be paired. Between pretest and posttest, 9.7% (n=15) of the families dropped out of the study. Mothers from the dropout group were slightly better educated than mothers from the completers. Two sites, covering a three county area, were chosen to achieve representation of racial group and urban and rural settings. S1 encompassed urban and rural areas and had a majority Caucasian population, while S2 was mostly rural and majority African American.

The average treatment for the MST groups involved 122.6 days for S1 and 116.6 for S2. Youths in the US condition were placed on probation for a minimum of 6 months and typically seen by his or her probation officer at least once per month. School attendance was monitored and referrals were made to other social service agencies for help in particular problem areas (e.g., alcohol and drug abuse programming, vocational counseling or training).

Sample: To be included in the study, adolescents had to (a) be between 11 and 17 years (b) have committed a serious criminal offense or have at least three prior criminal offenses other than status offenses, and (c) be at imminent risk of being placed outside the home because of serious criminal involvement. The juvenile offenders averaged 3.07 previous arrests and 59% had been incarcerated at least once. At the time of referral their average age was 15.22 years; 81.9% were male; 19.4% were White, and 80.6% were African American; approximately 50% were from two parent households and the median family income was between $5,000 and $10,000 per year.

Measures: Individual emotional adjustment and adolescent behavior problems, criminal activity, family relations, parental monitoring, peer relations, and MST treatment adherence were all measured in this study. Primary caregiver and adolescent psychological distress were assessed by the Global Severity Index. Adolescent behavior was measured through caregiver reports. A self-report delinquency scale was utilized to assess criminal activity during the previous 3 months. Arrest and incarceration histories were collected approximately 1.7 years from the end of the project and included offense, arrest, adjudication, and incarceration histories. Assessments of family functioning, parental monitoring, and peer relations were provided by reports from the primary caregiver and the adolescent. The MST Adherence Measure was completed by the parents, adolescents, and therapists after randomly selected therapy sessions.

Analysis: A series of 2 x 2 x 2 Time (pretreatment vs. posttreatment) x Treatment Condition (MST vs. US) x Site (S1 vs. S2) ANOVAs were conducted on the psychosocial measures collected during the pretreatment and posttreatment assessments. In addition, 2 x 2 (Treatment Condition x Site) ANOVAs were used to evaluate longer term outcomes for rearrest and incarceration through the 1.7 year follow-up. Hierarchical multiple regression analyses were conducted to test whether outcomes in the MST condition were associated with treatment adherence.

Outcomes

Youths in the MST condition reported substantially reduced psychiatric symptomatology, whereas their US counterparts reported slightly increased symptomatology. There were no significant treatment effects for youth reports on the self-reported delinquency assessment. The annualized rate of rearrest and the average seriousness of rearrests did not differ significantly between groups. On the basis of archival incarceration records, the annualized rate of days incarcerated was 47% lower for youths in the MST condition (33.2 days per year per youth) than their US counterparts (70.4 days per year per youth). No treatment effects were observed on the family relations measures or for the peer relations measure.

Results in this study show that MST program fidelity is significantly associated with many of the outcome measures. High adherence based on parent, adolescent, and therapist reports predicted favorable outcomes, and low adherence predicted poor outcomes on the following measures: adolescent symptomatology, rates of parental and emotional distress, adolescent self-reports of index offenses, rates of rearrest, and rates of incarceration.

Mediation: (Huey, et al., 2000): Across two independent samples, the study above and the Charleston County, SC, Drug Study (Study 6 in this writeup), results supported a family-centered mediation model. The model results demonstrated that MST improved family relations (i.e., quality of family functioning, family cohesion, and parent monitoring), and improved family relations in turn predicted decreased delinquent peer affiliation and subsequent delinquent behavior.

Study 5

Summary

The study used a quasi-experimental design with matching that examined 155 youths who were non-randomly assigned to three groups: an MST intervention group for juvenile offenders, an alternate family intervention for juvenile offenders, or a usual-services control group of non-offenders. Assessments at posttest measured conduct problems, anxiety, and aggression.

The study found that, relative to the comparison group, the MST intervention group showed significant:

• Decreases in behavior problems at posttest
• Improvements in family relations at posttest.

Evaluation Methodology

Design: A 3 X 2 mixed factorial design was used with three groups of adolescents and their families (family-ecological treatment, alternative treatment families, and normal controls). Offenders were not randomly assigned to treatment groups, but the groups were matched on demographic variables and pretreatment arrest histories. One hundred sixteen families of juvenile offenders were referred to the family-ecological treatment between June 1978 and June 1982. Alternative treatment families were the delinquent control group and consisted of 40 juvenile offenders and their families, who were referred to other mental health agencies for services. To control for developmental maturation and to provide a frame of reference, 50 nonpathological adolescents and their families who matched the demographic characteristics of the treatment families were recruited from local high schools for participation in the study. Pre- and post-treatment (3 weeks after treatment terminated) data was gathered for 57 treatment families, 23 alternate treatment families, and for 44 control families. The style and quantity of the family-ecological treatment was based on the family's needs and varied widely (from 2 to 47 hours, M=20 hours). Alternative treatment hours averaged 24 hours of intervention over an approximately 3-month period.

Sample: Specific characteristics of the sample, beyond the information listed above, were not provided in this study.

Measures: Child psychopathology was measured through youths self-reports, family member's self-reports of their perceptions of family relations, and observational measures of family relations based on the audio-recording of members' discussions.

Analysis: To determine an appraisal of any differential changes experienced by the three groups during the treatment period, a test of significance was used to show whether there was a significant multivariate effect for the set of dependent measures. If a significant effect was found, a 3 x 2 mixed factorial ANOVA was performed on each measure in the set. If the ANOVA revealed a significant interaction effect, the Scheffe test was used to evaluate the pre-post changes for each group on that measure.

Outcomes

Results showed that adolescents in the family-ecological condition had a decrease in behavior problems on each of the subscales: conduct problem, anxiety-withdrawal, immaturity, and socialized aggression. Alternative treatment and normal control adolescents showed no change. No significant multivariate interaction effects emerged in self-reported family relations. However, observational ratings indicated significant positive change among the family-ecological group (e.g., the mother-adolescent and marital dyads were warmer and more affectionate, and the adolescent was more actively involved in family discussions after treatment). Such changes were not observed for the alternate treatment group, and some family relations had deteriorated in this group (the marital relationship and the father-adolescent relationship in these families showed decreased warmth and affection following treatment).

Study 6

Summary

The study used a randomized controlled trial that assigned 118 youths with diagnosed substance abuse to an MST intervention group or a usual-services control group receiving outpatient substance use treatment. Assessments from posttest to a four-year follow-up measured drug use, criminal activity, and out-of-home placement.

The study found that, relative to the control group, the MST intervention group showed significantly:

• Lower substance use at posttest but not six-month follow-up
• Better school attendance at six months
• Fewer aggressive crimes at four years.

Evaluation Methodology

Design: The study followed a 2 (treatment type: MST vs. usual services) x 3 (time: pretreatment [T1], posttreatment [T2], and 6-month follow-up [T3]) mixed factorial design, with random assignment of participating families to the treatment condition (N=58) and usual services control condition (N=60). Out of 140 screened adolescents, 84% (N=118) agreed to participate in the study. Fifty-seven out of the 58 (98%) of the families assigned to the MST condition completed a full course of treatment, while 47 (78%) of the families assigned to the usual services condition completed treatment.

The amount of MST therapeutic services each family received was based on clinical need. Families in the MST condition received services for an average of 130 days, with an average of 40 hours direct contact. Youths in the comparison condition were referred by their probation officer to receive outpatient substance abuse services from the local office of the state substance abuse commission. Youths in this condition received few substance abuse or mental health services during the first 5 months following recruitment into the project. In fact, 78% of these families received no treatment.

An examination of the baseline comparability between youths and families in the treatment and control groups revealed youths in the MST condition reported higher rates of drug use prior to treatment.

Sample: Participants were 118 twelve to seventeen year-old adolescents recruited from the Department of Juvenile Justice in Charleston County, South Carolina. To be included in the study, youth met diagnostic criteria for substance abuse or dependence, had formal or informal probationary status, and lived with at least one parent figure. The average age of the sample was 15.7 years at the time of referral with 79% male, 50% African American, 47% Caucasian, 1% Asian, 1% Hispanic American, and 1% Native American. Based on socioeconomic measures, the sample is relatively disadvantaged.

Measures: A multimethod (self-report, parent report, biological, and archival) strategy was used to examine the following three outcomes: drug use, criminal activity, and out-of-home placement. Adolescent drug use was assessed through adolescent self-reports and urine toxicology screenings. Adolescent criminal activity was measured through youth self-reports on the Self-Report Delinquency Scale (SRD) and computerized arrest records. Days in out-of-home placements were documented through the monthly service utilization survey. MST program adherence was assessed using the 26-item MST Adherence Measure, which was administered to the primary caregivers and youths in the MST condition and completed by the project's three therapists following randomly selected therapy sessions. Three measures were used to obtain school attendance data: a school data form (SDF), the Child Behavior Checklist, and the monthly service utilization survey completed by the MST therapist (Brown, Henggeler, Schoenwald, Brondino, and Pickrel, 1999).

Analysis: To determine the effects of self-report drug use and delinquency, comparisons were conducted contrasting the behavior between the MST and usual services groups over the three time intervals. Analyses were conducted within the context of 2 (treatment condition) x 3 (time: T1, T2, and T3) mixed model analyses of variance (ANOVA), with one between- and one within-subjects factor. Due to the between group differences between reported drug use at baseline, analyses of covariance were also conducted on the drug use measures. One-way ANOVAs were conducted to measure arrests, out-of-home placements, and the rates of positive drug testing during treatment. Regression analyses were conducted to test whether adherence scores were associated with key outcomes. A 2 (condition: MST vs. US) x 3 (Time: pretreatment, posttreatment, 6-month follow-up) repeated measures ANOVA was conducted to examine changes in school participation across the treatment conditions over time.

Outcomes

Post-test: There were no significant treatment effects on the drug use measures (using ANCOVA), self-reported criminal activity, or arrest records. Although more youth in the MST group were incarcerated (19), compared with 16 youth in the US group, the MST youths were incarcerated for substantially shorter durations (569 days vs. 1051 days). MST fidelity outcomes showed modest results: decreased drug use at T2 but not at T3; a 50% decrease in days in out-of-home placement; and no effects for self-reported offending or the urine screens.

Comparisons of MST adherence across three studies, showed the current study on adolescent drug abuse were the worst (e.g., lowest adherence and highest non-productive sessions). Overall, adherence scores were marginally associated, in the expected direction, with criminal activity and out-of-home placement. The findings are contradictory in the observed associations between treatment adherence and adolescent drug use. Some results demonstrate higher fidelity is related to improved youth outcomes (e.g., adolescent reports of high treatment adherence were associated with reduced probability of using drugs other than alcohol and marijuana at T3). On the other hand, several findings showed the opposite effects (e.g., caregiver ratings of high adherence were associated with increased adolescent alcohol and marijuana use at T2).

MST effects were moderated in two instances. For females, MST was effective in decreasing alcohol and marijuana use from T1 to T2 in comparison to the US group, but between T2 and T3 female alcohol and marijuana use significantly increased for females in the MST condition, whereas counterparts in the US condition improved. A similar finding was revealed for age. For younger adolescents (age 15 and below), MST was effective in decreasing alcohol and marijuana use from T1 to T2 in comparison to the US group who increased, but between T2 and T3 younger adolescents in the MST group significantly increased their alcohol and marijuana use, whereas counterparts in the US condition decreased their use.

Participants in the MST condition showed a sustained increase in the percentage of adolescents in school through the 6-month follow-up. In contrast, the percentage of adolescents in school in the US condition decreased at posttreatment but then increased at the 6-month follow-up (Brown, Henggeler, Schoenwald, Brondino, and Pickrel, 1999).

Notes: Previous studies support the potential viability of MST for substance abusing adolescents. The modest results in this study may be due to the difficulty in transporting MST from the program developers to the supervisors and therapists implementing the program. Although the findings were not entirely consistent, high fidelity in this study tended to be associated with improved clinical outcomes.

Cost or Cost-Benefit: (Schoenwald, Ward, Henggeler, Pickrel, and Patel, 1996)

A study was completed to determine the monetary costs of MST during the 11-month period from referral into the program to the six-month follow-up. The total cost of services in the US condition during the 11-month period was $198,729. The total cost of seeing youth in the MST condition was $298,724 (this includes MST program costs + costs of mental health and substance abuse services). Costs per youth were $5,063 for youth in the MST condition and $3,369 for youth in the US condition. However, based on the clinical outcomes in this study, a 46% reduction in incarceration days and a 64% reduction in hospitalization/residential treatment, the costs of implementing the MST program are nearly offset by the savings in incarceration costs. Since the average incarceration cost is $100/day, the decrease in days incarcerated resulted in a $48,200 savings. In addition, the cost of MST compares favorably to the estimated lifetime costs of untreated delinquency and drug abuse which range from $309,000 to $1 million.

Four-Year Follow-up of Drug Outcomes - Charleston County, South Carolina (Henggeler, Clingempeel, Brondino, and Pickrel, 2002)

Evaluation Methodology

Design: This analysis examines the 4-year outcomes from the randomized clinical trial of MST with 118 juvenile offenders meeting formal diagnostic criteria for substance abuse or dependence. Eighty (43 MST and 37 usual services) of the 118 adolescents who participated in the original study completed the follow-up assessment. Between-group differences were determined at time 5 (T5) by comparing mean scores or percentages of the participants in the MST and usual services conditions on each of the dependent measures. Attrition analyses showed that study dropouts and completers did not differ significantly on the measures assessed. Comparisons between the MST and usual services participants at baseline showed two significant differences. MST participants were older and reported more frequent use of marijuana.

Sample: The average age of the participants at follow-up was 19.6 years; 76% male, 60% African American, and 40% were Caucasian. Approximately 48% had not obtained a high-school education or GED and 12% had completed some college or technical school beyond high school.

Measures: Adult criminal activity was measured through youth self-reports on the Self-Report Delinquency Scale (SRD) and computerized arrest records. Drug use was assessed through self-report and urine and head hair samples. Psychiatric symptoms were measured by the Externalizing and Internalizing scales of the Young Adult Self-Report (YAS).

Analysis: Multivariate analyses of covariance (MANCOVAs) were conducted on measures of (1) aggressive criminal behavior, (2) property crimes, (3) self-reported illicit drug use, (4) psychiatric symptoms to determine the differences between the MST and usual services groups at T5. Chi-square analyses were conducted on the biological drug use indicators (i.e., no positive test versus any positive test).

Outcomes

MST participants showed a 75% reduction in convictions for aggressive crimes since the age of 17 years and reported committing significantly fewer aggressive crimes during the past 12 months, compared with the usual services group. There were no significant differences in self-reported drug use between the two groups. However, the biological measures showed young adults in the MST condition had significantly higher rates of marijuana abstinence (55% versus 28%) than did their usual services counterparts. There were no significant differences in psychiatric symptoms between the two groups.

Moderator analyses showed that the impact of treatment did not vary as a function of demographic characteristics, comorbid psychopathology, or initial (T1) levels of drug use and criminal behavior.

Study 7

Summary

The study used a randomized controlled trial that assigned 100 Norwegian youths with serious antisocial behavior to an MST intervention group or a control group receiving child welfare services. Assessments at posttest and two-year follow-up measured problem behavior, delinquency, and out-of-home placement.

The study found that the MST intervention group, relative to the usual-services control group, showed significantly:

• Decreased youth internalizing symptoms at posttest
• Increased social competence at posttest
• Decreased out-of-home placements at posttest
• Lower delinquency at the two-year follow-up
• Fewer behavior problems, internalizing, and externalizing at the two-year follow-up.

Evaluation Methodology

Design: This study followed a 2 (treatment type: MST vs. Child Welfare Services (CS)) x 2 (time: pretreatment, post-treatment) x 4 (county municipality) mixed factorial design, with random assignment to treatment conditions. One hundred adolescents and their families from four counties in Norway, referred to treatment for antisocial behavior, participated in this study. A weighted randomization procedure was utilized to assign families to the treatment or control condition, with each family having a 6/10 chance of receiving MST and a 4/10 chance of receiving the usual CS services. This resulted in 62 families being assigned to MST and 38 families to CS. Assessments were conducted at study entry and after termination of MST treatment (approximately 6 months after intake). Comparison of the two groups at baseline showed CS caregivers were more likely than MST caregivers to be divorced, and MST caregivers were more likely to be married to someone other than the child's biological parent. These differences had no moderating effect on the outcomes.

Four families dropped out of the MST program early in the treatment, but were replaced with new families. In addition, 4 families withdrew from the study prior to the post-treatment assessment, resulting in a 96% retention rate. Analysis of attrition was not possible because the government-approved informed consent allowed dropouts to have their data expunged from research records.

MST for youths with serious antisocial behavior was implemented as detailed in the MST treatment manual with no major modifications to the original intervention model. Treatment options for the CS group included foster care and institutional placement. If out-of-home placement was not warranted, less intensive services such as home-based treatment or social work was provided. In this study, 14 youths received long-term institutional placement, 5 were placed in a crisis institution for assessment, 6 were supervised by a social worker in their homes, 7 were given other home-based treatment, and 6 refused the services offered.

Sample: To be included in the study, adolescents had to (a) be between 12 and 17 years, (b) exhibit serious problem behavior, and (c) have parents that were sufficiently involved and motivated to start MST. The sample consisted of 63 boys and 37 girls, who averaged 14.95 years of age. Ninety-five percent of the caregivers had a Norwegian background. Thirty-nine percent of the sample had been previously placed out of the home, 54% had a history of running away from home, 30% had been suspended from school, and 90% had a history of school truancy.

Measures: Because Norway's criminal justice system does not make arrests and convictions for youth under 15 years, and offenders under the age of 18 are usually sent to Child Welfare Services rather than prosecuted, archival arrest data is not available for this study. Instead, behaviors for the main outcomes were assessed using multiple informants (parents, teachers, and youths). Youth symptomatology and social competence was measured by caregiver, adolescent, and teacher ratings on the Child Behavior Checklist (CBCL). The Self-Report Delinquency Scale (SRD) was utilized to measure delinquent behavior. Social competence with peers was assessed from adolescent, caregiver, and teacher reports on the Social Competence with Peers Questionnaire (SCPQ). The Social Skills Ratings System (SSRS) was utilized to measure a broad array of social skills. The Family Adaptability and Cohesion Evaluation Scales-III (FACES-III) was completed by both caregivers and adolescents. This instrument measured two key family constructs: cohesion and adaptability. This study also provided assessed out-of-home placement and family satisfaction with the services they received.

Analysis: ANOVAs and chi-square analyses were conducted to evaluate between and within group differences on the measures collected during the pretreatment and post-treatment assessments.

Outcomes

Based on combined caregiver, youth, and teacher reports, youths in the MST condition demonstrated a significant decrease in internalizing behavior and a marginally significant decrease in externalizing behavior (p=.07) at post-treatment, compared with youths in the CS condition. Likewise, MST youths demonstrated a significant increase in social competence, compared to CS youths. There were no significant between group differences on family functioning measures. However, family cohesion increased significantly over time within the treatment condition. MST youths were maintained in the home significantly more often than CS youths. Of those at home during the pretreatment assessment (n=84), 90.6% of MST youths were also at home during the post-treatment assessment, compared with 58.1% of CS youth.

Follow-up Two Years After Intake (Ogden and Hagen, 2006)

The follow-up sample consisted of 75 participants. One of the four sites was dropped from the analysis because of poor fidelity, thus this is not an intent to treat analysis. The specific goals of the long-term follow-up were to investigate whether MST was successful in preventing out of home placement and examine reductions in behavior problems, such as delinquency. MST participants were less likely to have been placed out of the home than were their RS (received regular child welfare services) counterparts. This was especially true for boys and older MST participants. At the two-year follow-up, MST youths self-reported less delinquency than RS youths, parents reported fewer behavioral problems on the CBCL total problem scale and less internalizing, and teachers reported fewer behavior problems, less internalizing and less externalizing problems.

Non-significant differences were reported by youth on the total problem scale of the YSR and the YSR Externalizing and Internalizing scales, as well as the parent reports on the CBCL Externalizing.

Quasi-Experimental Comparisons with New Intervention Group (Ogden et al., 2007)

This study used 50 participants from two of the four sites in the original year 1 sample. It also examined 55 new participants at the two sites who were all assigned to the MST condition during year 2 of the program implementation. The design compared the non-randomized MST participants in year 2 to the randomized MST and control participants in year 1. The study lacked randomization in year 2 and made comparisons across two different years. The results demonstrated sustained effectiveness of the program over time. Compared to the control participants in year 1, the intervention participants in year 2 reported significantly better outcomes for preventing out-of-home placement, youth-reported delinquency and internalizing, and teacher-reported internalizing and externalizing. Further, comparisons of intervention participants showed similar improvements in year 2 as in year 1.

Notes: The community services available to the control group, to which the MST group was compared, are much more comprehensive and treatment-oriented than the services typically available to juvenile offenders in previous MST research. The more modest treatment effects obtained in this study, compared to other evaluations, may be attributable to the nature of the usual services condition.

Gender differences in MST outcomes (Ogden and Hagen, 2009)

This analysis uses students from the previous program evaluation to look at gender differences in treatment outcomes.

Sample: This study used data from 2 previous studies in Norway. The first was an RCT in which 100 adolescents were randomly assigned to either MST (n=62) or regular services (n=38). An additional group of 55 non-randomized adolescents received MST and participated in another study during the second year of program operation and were added for the purpose of studying the sustainability of program effectiveness. The current study included the 117 youth who received MST treatment in the program's first and second year of operation. Of these, 41 were girls and 76 were boys (65%). Adolescents ranged in age from 12-17 years. While somewhat fewer of the girls' caregivers were married or cohabiting (54%) compared to boys' caregivers (62%), the difference was not statistically significant. There were no significant differences on girls' and boys' caregivers' mean age (41.26 years) and family income. At intake to treatment, 91.3% of the adolescents had been living at home with their caregivers for the last 6 months. No gender differences appeared regarding living situation at intake.

Measures: Adolescents' behavior problems were assessed with multi-informant ratings from caregivers, teachers and adolescents using the CBCL, the Teacher's Report Form (TRF) and the Youth Self-Report (YSR). Youth alcohol and drug use was measured by the Personal Experience Inventory, which was completed by the youth themselves, as was the self-report delinquency scale. A subscale from the Family Adaptability and Cohesion Evaluation Scale was used to assess family cohesion. Caregivers' satisfaction with the MST treatment was measured using the Family Satisfaction Survey. Finally, the youth's place of living was reported by both parents and intake teams at intake and again at treatment termination.

Analysis: To test for gender differences at intake, chi-square tests and one-way ANOVAS were run. Four MANOVA models were run to test for gender differences in behavioral improvement following treatment.

Outcomes
At intake, girls and boys looked far more similar than different, although some differences were apparent at posttest. Girls were rated by their parents as presenting fewer externalizing problems than boys at post-treatment, after controlling for their wave 1 scores. This suggested that girls improved more than did boys on this measure. There were no gender differences on parent reports of internalizing or attention problems. Results revealed that girls rated themselves as having significantly more internalizing problems than did boys at the post-assessment, after controlling for their pretreatment scores. Boys, however, scored significantly higher on the self-report delinquency scale than did girls. This suggests that girls improved more regarding self-report delinquency, whereas boys reported greater reductions in self-assessed internalizing problems. No gender differences were found in terms of changes following treatment on the externalizing and attention problem scales of the YSR, on self-reported drug use or out of home placement. According to teachers, boys and girls showed similar levels of behavioral change following treatment.

Overall, although girls present a problem profile different than boys, and risk factors are somewhat different, MST is effective regardless of gender.

Study 8

Summary

The study used a randomized controlled trial that assigned 411 Canadian youths at high risk for criminal offending to an MST intervention group or a usual-services control group. Assessments from posttest to a three-year follow-up measured family relations and criminal convictions.

The study found that, relative to the control group, the MST intervention group was significantly:

• Less likely to be sentenced to a term of secure custody
• Improved on parent reports of family adaptability
• Improved on youth externalizing behavior and internalizing symptoms.

Evaluation Methodology

Design: This study employed an experimental design whereby each of the 411 referred youth from four Canadian communities were randomly assigned to either the MST group or the usual services group, which functioned as a control group. Two categories of criteria were used to determine eligibility for MST. The first assessed the appropriateness of a family preservation intervention for the youth and the second verified that the presenting issues of the youth were among those for which MST has been empirically validated.

Pretesting was administered once the family gave consent and before random assignment was made. Data was collected from youth, parents, and teachers. However for the teacher information there was a low response rate of 57% at intake and 35% at discharge. While the response rate was the same for the two groups, there were also some significant differences between the aggregate scores of the MST and the control groups. Among the 57% of cases for which the teacher information was available, the members of the control group had higher levels of externalizing behavior problems at pretest. The instruments from the intake battery were readministered when an MST case was closed. If the MST case closed prematurely, post-testing was not re-administered. The members of the control group were contacted five months after intake and asked to complete the testing again.

In total, post-testing for the youth is available in only 62% of the cases and was more likely to be available for the MST group. There was no post-testing available for either a parent or a youth in 49% of control cases and for 28% of the MST group. The members of the MST group had a significantly higher response rate at discharge (71%), compared with the members of the usual services group (52%). Nineteen percent of the MST group dropped out prior to concluding the treatment, however, dropouts did not differ significantly from those who stayed on any of the intake testing. The authors' report no response bias could be identified. Analyses at post-test comparing MST dropouts and MST completers show that MST drop outs performed poorly compared with both the MST completers and the usual services group. The researchers were unable to identify the usual services drop outs.

All youths were tracked for three years to gauge offending and levels of correctional service utilization at 6, 12, 24, and 36 months after the case was closed. The case was closed for the MST group at the last session with the family. For the control group, the case was considered closed six months after the family signed the consent to participate in the study. At this report 380 youths were tracked at least six month post-discharge, 323 were reached one-year post discharge, 192 were two years post-discharge, and 82 reached three years.

Sample: To be included in the study, youth had had to be at high risk for committing a criminal offense and meet the eligibility criteria. One of the sites included youth under the age of 12 (27 youth). The average age was calculated for each of the four sites and ranged from 13.9 to 15.3, with an overall average of 14.6 years. Twenty-six percent of the sample was female. Thirteen percent self-identified as aboriginal. Family SES varied in the sample, about one-third were welfare dependent with poor educational achievement and low SES. Also, about 30% of the sample was middle class families with good educations and high incomes.

In all, 64% of the youth were referred to the MST project by probation officers. Although there had to be evidence of past criminal behavior to qualify for MST, about one third had no record of prior criminal convictions at referral. Thirty percent had been sentenced to at least one sentenced custody stay prior to referral. On average, youth with prior convictions had served 47 days in sentenced custody.

A clinical profile was drawn from psychometric testing completed by the youth, caregivers, and teachers. According to parent ratings, 84% of youth were over the clinical cutoff for conduct problems and half were over the cutoff for depression. Youth self-reports were only slightly lower, with 61% placing themselves over the cutoff for conduct and 48% for depression. One third of parents placed themselves over the cutoff for caregiver depression and poor family functioning. Teachers rated the youths as low on academic competence and social skills, placing almost all of them at or below the tenth percentile.

Measures: The following instruments were used as baseline information and to assess the impact of MST on the youth: (1) Standard Client Information System (SCIS); (2) Beliefs and Attitudes Scale; (3) Family Adaptability and Cohesion Scale - II (FACES) - this is a 30-item scale that measures family adaptability (negotiation style, roles, assertiveness, leadership, discipline, child control, rules) and family cohesion (emotional bonding, coalitions, space, family boundaries, shared time/friends, decision-making, and shared activities); (4) Social Skills Rating System; (5) Parental Supervision Index.

Follow-up data on recidivism and correctional service utilization are collected using a "CR" check of the Canadian Police Information Centre (CPIC). Records are checked at six months and after one, two, and three years post-discharge.

Analysis: The analyses included a comparison of group means and a survival curve analyses at four follow-up time periods for rates of conviction.

Outcomes

Post-test: There were no significant differences between groups in rates of convictions when the means are compared. Likewise, the survival curve did not demonstrate significant group differences regarding convictions. There was no difference in the rate at which the two groups were sentenced to custody during the follow-up or in the length of time before being convicted after being in treatment. Members of the MST group were significantly more likely to be sentenced to a term of open custody and significantly less likely to be sentenced to a term of secure custody, compared to the usual services group. This pattern was observed in three sites. There were no significant differences between groups for the length of time youths spent in custody or for the total number of offenses for which youth were prosecuted.

MST youth improved significantly on some problems as measured by psychological testing. Compared to the control group, the MST youth improved significantly on parent reports of family adaptability, caregiver depression, and youths' externalizing behavior. The MST group also improved significantly on youth report of internalizing symptoms when compared to the control group. While there were no significant differences between groups regarding parental supervision, these data need to be interpreted with caution because there were significant group differences at intake and because pre/post data are only available for only half the sample.

Notes: The Canadian MST program was implemented with a lower budget and less supervision by MST Inc. than previous MST studies. This may have undermined the fidelity of program implementation. Another possible factor that could account for the lack of results in the Canadian study, compared with studies in the U.S., is that the usual services that the control group received were of higher quality than the services youth receive in the U.S.

The results reported here are from an interim report and does not represent the final report.

Study 9

Summary

The study used a randomized controlled trial that assigned 164 youths enrolled in behavior intervention classrooms to an MST intervention group or a no additional treatment control group. Assessments from posttest to a one-year follow-up measured conduct problems, externalizing, and criminal offending.

The study found that, relative to the control group, the MST intervention group showed significantly lower:

• Parent- and adolescent-reported externalizing problems
• Absenteeism in school.

Evaluation Methodology

Design:

Recruitment: The authors contacted public junior and high schools to recruit adolescents between the ages of 11 and 18 who were enrolled in self-contained behavior intervention classrooms that served as restrictive settings designed for adolescents with conduct problems. Of 213 families with children in classrooms, 164 (77%) provided consent to participate in the study. The sample scored highly on baseline measures of externalizing and internalizing.

Assignment: After baseline measures were collected, 84 adolescents were randomly assigned to the treatment group and 80 were randomly assigned to a treatment-as-usual control group that did not receive the treatment.

Assessments/Attrition: Outcomes were assessed at baseline, three months (mid-program), six months (posttest), and 18 months (one-year follow-up). By the one-year follow-up, 7% of the families withdrew from the study or could not be located.

Sample:

Participants were enrolled in the 7th through 11th grades at baseline, with a mean age of 14.6 years. The vast majority of adolescents (83%) were male, 60% were black, and 40% were white. Parents or guardians averaged 40.8 years of age, 71% had at least completed high school, and the median reported family income was $17,500.

Measures:

Weiss et al. (2013): All measures were obtained from separate interviews for parents and adolescents at a location of their choosing, while teachers were given the assessment materials to complete on their own time. Parents both participated in the program to help their child and provided many outcome measures. However, an additional analysis found that adjusting for a measure of positive impression among parents did not change the results.

The primary outcomes were adolescent conduct problems as assessed by parent, adolescent, and teacher reports on the Child Behavior Checklist (CBCL) and criminal charges obtained from court records. The CBCL is used to obtain a broad measure of child behavioral and emotional problems, and its 118 items produce separate scales for externalizing and internalizing problems. The authors report that these scales have average 1-week test-retest reliability of .89. Criminal records were obtained with parental and child assent from juvenile court records from 1 year prior to baseline through 2.5 years post-baseline. Charges were coded as status offenses, misdemeanors, or felonies, with the analysis focusing on incidents wherein the adolescent was charged with a felony.

Secondary outcomes included several areas of conduct problems such as delinquency and drug use that were measured using the Self Report Delinquency Scale (validity and reliability were not reported). School functioning data was also included, and consisted of the student's average grade across all core academic subjects, school attendance, and the number of days suspended during the study period.

The authors also detailed the measurement of nine factors related to the study's risk and protective factors. To assess family relationships, parents and youth completed the Family Adaptability and Cohesion Evaluation Scales III; to assess authoritarian, authoritative, and permissive parenting behavior, the adolescent's primary caregiver completed the Parental Authority Questionnaire; and to assess parent mental health problems, parents completed the measures of externalizing and internalizing from the Personality Assessment Inventory.

Weiss et al. (2015): Three externalizing scales from the parent-reported Child Behavior Checklist, the Youth Self-Report form, and the Teacher Report Form served as the outcomes (alpha values were .89, .85, and .93, respectively).

Analysis:

The analysis used hierarchical linear models for repeated measures with time nested within persons. The models specified time as a random effect with linear and quadratic terms. These models inherently adjusted for baseline outcomes. However, the models did not adjust for clustering within the unspecified number of schools.

Cox proportional hazards models were used to analyze the court data, and estimated whether time to first felony arrest differed by treatment group, adjusting for whether the adolescent had been arrested in the previous year.

Intent-to-Treat: The study adhered to the principles of intent-to-treat, using all available data for all respondents, regardless of program completion.

Outcomes:

Implementation Fidelity: Quantitative measures indicated that therapists' overall adherence to Multisystemic Therapy principles was moderately high to high, and consistent with the other studies. With the average treatment lasting 5.19 months, 88% of cases were considered to have been closed successfully, and 8% were considered to have been closed partially successfully.

Baseline Equivalence: Of 15 demographic and baseline primary outcome measures, one differed significantly between treatment groups. The treatment group had a higher proportion of parents who had graduated from high school. The authors state that this variable was not related to treatment outcomes, so it was not included in any of the outcome analyses. Also, the study did not appear to test for differences in the other 14 secondary and risk/protective outcomes.

Differential Attrition: Over the course of the study, 3 families in the treatment group (3.6%) and 8 families (10%) in the control group withdrew from the study or could not be located by the 1-year follow-up assessment The authors did not assess differential attrition.

Posttest and Long-Term: In Weiss et al. (2013), two of four primary outcome measures were improved relative to the control group, with intervention-group parents and adolescents reporting greater decreases in externalizing problems; however, teacher reports and arrest data did not show significant treatment effects. Only one of five secondary outcomes differed significantly between groups, with a quadratic treatment effect for the number of days absent in favor of the treatment group. Among nine risk and protective factors, two showed significant improvement in the treatment group: decreased permissive parenting behavior and decreased parent internalizing mental health problems.

Weiss et al. (2015) briefly summarized the main effect results, with significantly greater declines for the treatment than the control group for parent-reported externalizing and youth-reported externalizing but not for teacher-reported externalizing. The major part of the analysis examined moderation of intervention effects by parenting measures. The results showed that families sometimes gained from MST when parents reported maladaptive processes (e.g., low baseline levels of adaptive child discipline skills) but more often gained from MST when parents reported strengths that facilitated the treatment (e.g., high levels of adaptive functioning).

Study 10

Summary

The study used a quasi-experimental design that examined 1,137 juvenile offenders from Los Angeles County who were assigned to MST treatment. Those participating in the treatment served as the intervention group and those not participating served as the control group. Assessments after six months measured arrests and incarcerations.

The study found no overall beneficial effects of MST relative to the comparison group but showed improvements in:

• Rates of rearrest, incarceration, and completion of community service among Hispanic youth only.

Evaluation Methodology

Design:

The program was evaluated using a QED without randomization of 1,137 juvenile offenders that qualified for MST participation based on the Los Angeles County Juvenile Justice and Crime Prevention Act's (JJCPA) eligibility criteria. The intervention group consisted of 757 individuals who were accepted into and participated in Los Angeles' MST program between January 2003 and December 2010, and the comparison group contained 380 youth who were eligible but did not participate in MST between January 2001 and December 2010, most often because of lack of Medicaid coverage. There is little information on the kind of treatment that the comparison group received.

Data from all participants were tracked for 6 months after qualifying for (in the comparison group) or entering the program (in the intervention group) using automated databases maintained by the Los Angeles County Probation Department, but not all measures applied to all participants. The full sample (100%) had arrest and incarceration data, with fewer having data on completing probation (N= 724, 96% of treatment group; N= 353, 93% of comparison group), probation violations (N= 722, 95% of treatment group; N= 353, 93% of comparison group), completing restitution (N= 477, 63% of treatment group; N= 255, 67% of comparison group), and completing community service (N= 363, 48% of treatment group; N= 153, 40.3% of comparison group). Risk and protective factors targeted by the intervention were only collected from the treatment group beginning in 2004, resulting in 508 participants (67% of treatment group) with data in this area.

Sample:

The mean age of program participants was 15.3 at baseline, and almost all youth (96%) had at least one prior arrest. The majority of participating youth were Male (77%) and Hispanic/Latino (about 74%), with a smaller number identifying as African American (about 20%) and very few as White or some other ethnicity (about 7%). Most participants were arrested for violent (30%) or other crimes (43%) that led to their qualifying for MST, with fewer arrested for property (21%) or drug crimes (5%).

Measures:

Juvenile justice outcomes, all measured as zero for no and one for yes, included 1) arrests, 2) incarcerations, 3) successful completion of probation, 4) successful completion of restitution, 5) successful completion of community service, and 6) probation violations. All were measured continuously for 6 months after qualifying for MST and were obtained from an automated database maintained by the Los Angeles County Probation Department.

Five risk and protective factors targeted by the program were measured among the treatment group at baseline and discharge from the program, beginning in 2004. The factors included parenting skills, family relations, network of social supports, success in educational or vocational settings, and involvement with prosocial peers. Performance in each area was rated as satisfactory or unsatisfactory according to criteria specified by MST caseworkers; for example, improvement in parenting skill required parents to demonstrate at least two of the following: 1) increased limit setting, 2) established and enforced consequences, 3) increased monitoring. MST caseworkers who delivered the therapy provided the scores.

Analysis:

Logistic regression was used to evaluate all juvenile justice outcomes with race/ethnicity, treatment group, age, gender, and type of offense at recent arrest included as covariates. Race-stratified analyses were also performed, comparing the effects of MST treatment for African American and Hispanic participants. McNemar's test (paired chi-square) was used to evaluate the significance of pre- to post- changes in risk and protective factors and did not include adjustments for sociodemographic or criminogenic factors.

The study adhered to the principles of intent-to-treat, analyzing all participants with relevant data in their original condition.

Outcomes:

Implementation Fidelity: No measure of implementation fidelity was reported.

Baseline Equivalence: The treatment and comparison groups were similar at baseline, significantly differing only on ethnicity (MST participants were more likely to be Hispanic). While baseline equivalence was not examined for the study outcomes, similar measures such as having a prior arrest and offense type at last arrest did not differ by group. However, the groups likely differed on unmeasured socioeconomic characteristics that partly determined assignment.

Differential Attrition: While the study did not examine differential attrition, all subjects were retained for the analysis of arrests and incarcerations. The smaller sample sizes for other outcomes are likely a result of variation in the sentences given to participants at trial.

Posttest: After controlling for race, age, gender, and type of offense, the treatment group showed significant improvement in 3 of 6 outcomes (arrests, incarcerations, and completion of community service) relative to the comparison group. Without the controls, the intervention group had significantly more probation violations than the comparison group.

Race-specific analysis of these outcomes revealed that the program's positive effects were observed for Hispanic youth only. While Hispanic MST youth had significantly lower rates of arrest, lower rates of incarceration, and greater odds of completing probation than Hispanic comparison youth, African American MST youth had higher rates of arrest than the comparison group and did not differ significantly on any other outcomes.

Of the 6 functional areas targeted by the program, all (parenting skills, family relations, network of social supports, educational/vocational success, involvement with prosocial peers) were significantly improved at posttest within the treatment group (though with no comparison to the non-MST group).

Study 11

Summary

The study used a randomized controlled trial that assigned 256 Dutch adolescents with antisocial behavior to an MST intervention group or a treatment-as-usual control group. Assessments from posttest to a three-year follow-up measured a variety of outcomes relating to antisocial behavior.

The study found that, relative to the control group, the MST intervention group showed significant reductions in:

• Externalizing
• Oppositional defiant disorder
• Conduct disorder
• Property offenses

Evaluation Methodology

Design:

Recruitment: Adolescents with various types of serious antisocial behavior were referred to the study from multiple community sources across the Netherlands and during the years 2006 to 2010. Of 318 assessed for eligibility, 256 met inclusion criteria and consented to participate.

Assignment: The study randomly assigned the 256 participants to MST (N = 147) or a treatment-as-usual control group (N = 109). The randomization was adjusted using a 1:2 ratio in favor of MST for a six-month period due to a low number of referrals.

Assessments/Attrition: Assessments took place at baseline before the start of the program, about 6 months later at posttest, and at 1.08 years after pretest (roughly 6 months postintervention). Official recidivism data were collected at 3.06 years (mean length, called a 2-year follow-up in the paper) after pretest. They also occurred monthly during the intervention period. About 13% of the participants were lost at posttest and 23% were lost at follow-up.

Sample Characteristics: Most of the subjects had a Dutch background (55%). Many belonged to ethnic minorities (34% Moroccan and 32% Surinamese). Half lived in single-parent homes, and 50% of the mothers and 36% of the fathers were unemployed. More than half the families lived below minimum income levels. Judicial records showed that 71% had been arrested at least once.

Measures: Baseline and posttest assessments took place in the homes of the subjects, and "the majority" of research assistants doing the assessments were blind to the assigned condition. Measures came from parent reports, adolescent reports, and observations and showed acceptable reliability. A brief list summarizes the numerous measures:

• Cognitions based on parent-reported sense of competence and adolescent-reported self-esteem, personal failure, and hostility
• Parenting based on parent-, adolescent-, and observer-reported positive discipline, inept discipline, and relationship quality
• Deviant peers and prosocial peers reported by adolescents
• Externalizing behaviors from the parent-reported Child Behavior Checklist
• Oppositional defiant disorder and conduct disorder from the parent-reported Defiant Behavior Disorders rating scales
• Externalizing behaviors from the adolescent-reported Youth Self Report
• Violent offending and property offending from the adolescent-reported Self-Report Delinquency scale
• Official recidivism data from the Judicial Registration System, Dutch Ministry of Justice

Assessments completed within the intervention period were done by phone with an abbreviated set of items for each scale.

Analysis: Using an intent-to-treat strategy, the study employed three different statistical approaches. First, the analysis examined posttest outcomes for the groups with ANOVA and controls for baseline outcomes. Second, the analysis examined within-intervention trajectories using latent growth models. Third, structural equation models tested for mediation effects.

Missing Data Method: An email to Blueprints from the lead author noted that the expectation maximum likelihood algorithm in LISREL replaced missing data for sociodemographics and relevant baseline outcomes.

Intent to Treat: Given evidence of data missing completely at random, the analysis used all cases with the expectation maximum likelihood algorithm in LISREL.

Outcomes

Implementation Fidelity: A Therapist Adherence Measure that the investigators collected monthly from the parents assessed adherence to the nine principles of MST. The mean of 4.36 on a scale from 1 to 5 showed acceptable adherence.

Baseline Equivalence: No significant baseline differences were found on any of the demographic or outcome variables.

Differential Attrition: Participants lost to postintervention assessment did not differ significantly on any of the assessed variables from those that remained in the study, based on the the additive test for completers vs. attritors (Asscher et al., 2013; 2014; Dekovic et al., 2012). The authors stated that Multiple Imputation was carried out by the expectation maximum likelihood algorithm. Additionally, Dekovic et al. (2012) used the Little's test to provide evidence that data were missing completely at random.

Posttest: Deković et al. (2012) presented results on 1) intervention effects at posttest, 2) within-intervention change, and 3) mediated effects. First, the program significantly improved four of the five posttest measures compared to the control group: parental sense of competence, positive discipline, relationship quality, and externalizing problems. Only inept discipline did not improve. Second, as shown by differences across groups in the mean slopes, the program significantly increased the within-intervention rates of growth in the same four outcomes as above. Effect sizes similarly increased over time. By the end of the sixth month, effect sizes for the four significant outcomes ranged from about .2 to .5. Third, mediation models showed that participation in MST was significantly related to a greater increase in maternal sense of competence, which in turn predicted an increase in positive discipline. These changes then predicted a decrease in adolescent externalizing. However, a similar model using relationship quality rather than positive discipline did not significantly mediate between the program and externalizing behavior.

Asscher et al. (2013) found significant improvement in the treatment group relative to the control group for 5 of 6 primary measures: parent-reported externalizing, opposition defiant disorder, and conduct problems, and adolescent-reported externalizing and property offending. Effect sizes ranged from .25-.36. They also found significant program improvement for 9 of 15 secondary measures. The significant outcomes included measures of parent-reported sense of competency, youth-reported hostility, parent-, adolescent-, and observed-reported positive parenting, parent- and observer-reported quality of relationship, observer-reported inept discipline, and adolescent-reported prosocial peers. However, one significant iatrogenic effect emerged for adolescent-reported sense of personal failure. Effect sizes for the secondary measures ranged from .26-.47.

Asscher et al. (2013) also tested for moderation effects by ethnicity and age, with little evidence of differences across groups. Tests for moderation by gender found that for adolescent cognitions, the treatment had larger (and more positive) effects for boys than for girls.

Manders et al. (2013) tested for additional moderation effects. They found that MST was more effective than the control in decreasing externalizing problems for the "lower callous/unemotional" and "lower narcissism" group, but not for the "high callous/unemotional" and "high narcissism" group.

Long-term (Asscher et al., 2014): There were no differences between the conditions in frequency or number of arrests, time to re-arrest, or type of arrest, at the six-month and 2-year follow-ups. At the end of the final follow-up period, 63% of the MST group and 53% of the TAU group had been rearrested at least once, but these differences were not significant.

Moderation (Asscher et al., 2018): The study examined moderation tests for differences in program effects for extremely violent and not extremely violent youth. The results showed no significant moderation at posttest for externalizing or relationship quality. However, additional analyses of monthly data from pretest to posttest demonstrated different patterns of change. The extremely violent youth worsened initially but then showed large improvements in both externalizing and relationship quality.

Study 12

Summary

The study used a randomized controlled trial that assigned 93 juvenile offenders to an MST intervention group or a treatment-as-usual control group. Assessments through 18-month follow-up measured reoffending and youth functioning in eight domains.

The study found that, relative to the control group, the MST intervention group showed significantly:

• Lower rates of reoffending and rearrest
• Greater improvements in four areas of functioning (home, school, community, and moods and emotions)

Evaluation Methodology

Design:

Recruitment: Participants were 93 youth who appeared before a family county court in a Midwestern State between October 1998 and April 2001. Youth were recruited for participation if they met the following inclusion criteria: (a) a felony conviction, (b) a suspended commitment to the Department of Youth Services incarcerating facility, and (c) parents' consent to participate. All youth were either on probation at the time of the study or had been on probation previously.

Parents or legal guardians of youth meeting the study inclusion criteria were asked to consent to random assignment to either MST or TAU. The court agreed to randomize (into the MST or TAU conditions) families in which caregivers and youths agreed to participate in the study. Both parents and youth provided informed consent, which was obtained either by court personnel or by the MST supervisor. Overall, 89% of eligible participants who met inclusion criteria agreed to participate in the study (n=105). If parents or guardians did not agree but were eligible for MST, it was at the discretion of the court whether to assign them to MST without study participation.

Assignment: Randomization was accomplished by having the court administrator flip a coin. 48 youths were randomly assigned to the MST condition and 45 youth were assigned to the TAU condition.

Assessments/Attrition: 89% of participants completed the study (i.e., 11% of study participants dropped out resulting in a final analysis sample of 93 participants). Recidivism was tracked through an 18-month post-treatment follow-up.

Sample: The mean age of all youths was 14.1 years at the time of enrollment in the study. Twenty-two percent of the participants were female, and seventy-eight percent were male. The racial composition of the sample was as follows: 15.5% African American, 77.5% European-American, 4.2% American Hispanic and 2.8% biracial.

Measures:

Official charge data: The county family court keeps detailed information regarding juvenile arrests. The recidivism analyses in this study were based on those charges for which the youth was formally arraigned following discharge from treatment (for the MST group) or at 6 months post-recruitment (for the TAU group). Charge data were examined through 24-month post-recruitment for both groups. Despite the level of detail in the court record concerning rearrests, few details were available on the specific type of new charge; however, each charge was designated as either a misdemeanor or a felony.

CAFAS: Ratings for the Child and Adolescent Functional Assessment Scale (CAFAS) were made corresponding with the beginning of treatment, discharge, and 6 months following discharge for MST youth. For TAU youth, ratings approximated timing of administration for MST youth (i.e., baseline, 6-month post-recruitment, and 12-month post-recruitment). The CAFAS measures youth functioning in eight areas: school and work, home, community, behavior, substance use, and thinking. For each subscale, the child receives a score of 0, 10, 20, or 30. A score of zero indicates no or minimal impairment, 10 indicates mild impairment, 20 indicates moderate impairment, and 30 indicates severe impairment. A total score based on the summation of the subscales reflects overall youth functioning. The range of possible scores on the CAFAS is 0 to 240. The higher the CAPAS score, the greater the functional impairment.

Brief subscale descriptions provided in the current edition of the CAFAS manual are as follows: (a) School/Work: ability to function satisfactorily in a group education environment, (b) Home: extent to which youth observes reasonable rules and performs age appropriate tasks, (c) Community: respect for the rights of others and their property and conformity to laws, (d) Behavior Toward Others: appropriateness of youth's daily behavior, (e) Moods/Emotions: modulation of the youth's emotional life, (f), Self-Harmful Behavior: extent to which the youth can cope without resorting to harmful behavior or verbalizations, (g) Substance Use: youth's substance use and the extent to which it is disruptive, and (h) Thinking: ability of youth to use rational thought processes.

To provide a standardized method of CAFAS assessment, research assistants used court records to rate MST and TAU youth at baseline, at immediate posttreatment (for MST youth) or 6-month post-recruitment (for TAU), and at 6-month post-treatment (for MST youth) or 12-month post-recruitment (for TAU youth).

Analysis: A 2 (treatment condition: MST vs. TAU) x 3 (time: pretreatment, immediate post-treatment, and 6-month post-treatment follow-up) mixed factorial design was used to evaluate functioning. Relative odds ratios were calculated for the likelihood of rearrest. Binary logistic regression was conducted to compute the relative risk of rearrest in the TAU versus the MST group. The survival analysis was conducted using a Fleming-Harrington test that weighted offenses later in time more heavily than offenses earlier in time. The rationale for using this weighting scheme with post-treatment arrest data derives primarily from knowing that rates of reoffending typically increase over time as active supervision of youths' activities decreases.

General linear modeling repeated measures were used to examine changes in six CAFAS subscale scores over time. The CAFAS total score is not presented as there is limited evidence that this score presents information separate from that already covered in the subscales. To minimize the effect of multiple tests made on the data, alpha was set at .008. Using the adjusted alpha, time by interaction effects were tested.

Missing Data Methodology: The study used complete case analysis without imputation or FIML estimation.

Intent-to-Treat:  The study referred to including participants who "completed the treatment and completed the study." The wording implies that only program completers were kept for the analysis.

Outcomes

Implementation Fidelity:

Baseline Equivalence: There were no statistically significant between-group differences with respect to race or sex. There were also no statistically significant between-group differences on court-related variables such as age at first offense, number of pre-treatment offenses, number of pre-treatment misdemeanors, and number of pre-treatment felonies. However, these results appeared to use the analysis sample rather than the randomized sample. Further, tests using the analysis sample in Table 2 show several significant differences between the conditions at baseline (i.e., time 1).

Differential Attrition:

The study did not refer to tests for attrition or show condition differences in attrition rates, but the tests for baseline equivalence of conditions using the analysis offer evidence of differential attrition (see Table 2 for time 1).

Posttest and Long-term:

At the 18-month post-treatment follow-up, the recidivism rate for the MST group (66.7%) was significantly lower than the overall recidivism rate for the TAU group (86.7%). Youth in the MST group were also arrested and arraigned for significantly fewer offenses. There were no significant between-group differences in the percentage of felonies versus misdemeanors. Youths in the TAU group were 3.2 times more likely than youths in the MST group to be rearrested. For youth with at least one rearrest, the average time to first arrest was 135 days for youths in the MST group and 117 days for youths in the TAU group, a nonsignificant difference.

With regards to youth functioning, both groups evidenced improvement in functioning over time, with the MST CAFAS scores significantly better on four of six subscales: home, school, community, and moods and emotions. The difference between the groups was nonsignificant for substance use and behavior towards others.

Study 13

The main goal of this study was to evaluate whether Multisystemic Therapy (MST) is more effective in reducing youth offending than an equally comprehensive management protocol called Youth Offending Teams (YOTs). This study can be seen as true independent evaluation since it was conducted without the program developers close involvement. Whereas most other MST evaluations were conducted in the U.S., this study was fielded in the United Kingdom.

Summary

The study used a randomized controlled trial that assigned 108 juvenile offenders in London, England, to an MST intervention group or a treatment-as-usual control group. Assessments from posttest to an 18-month follow-up measured reoffending and problem behavior.

The study found that, relative to the control group, the MST intervention group showed significant decreases in:

• Aggression, delinquency, and psychopathic traits at posttest
• Nonviolent offenses at the 18-month follow-up assessment.

Evaluation Methodology

Design:

Recruitment /Sample size/Attrition:
For the recruitment process, this study used referrals from two local youth offending services in North London. Young persons were included in the study if they were between 13 and 17 years of age; living in the home of and being brought up by a parent or principal caretaker; and being on a court referral order for treatment, supervision, or following imprisonment. Youth were excluded if they were a sex offender; presented only with substance misuse; were diagnosed with a psychotic illness; or posed a risk to research personnel. A total of 478 young persons were referred to the study team, of whom 370 (77%) were excluded because they could not be contacted, refused to consent to assessment, or did not meet the inclusion criteria. The remaining 108 participants were randomized into Multisystemic Therapy (MST) treatment group (n = 56) or Youth Offending Teams (YOT) control group (n = 52). For primary outcome measures, one participant was excluded from the MST group "due to lack of evidence of offence at intake" (p. 1222) (attrition of < 1%). For secondary outcome measures, 3 participants in the MST group and 1 participant in the YOT group did not complete the questionnaires (attrition of 4%).

Study type/Randomization/Intervention:
The study employed a randomized control trial design. The MST team comprised three therapists and a supervisor. The therapists had low caseloads, usually visited the families at least 3 times per week, and were available by telephone to support them 24 hours per day and 7 days per week. The lengths of the interventions ranged from 11 to 30 weeks (mean = 20.4 weeks). Students in the YOT control group received the usual services. As in MST, YOT interventions are extensive and multi-component and included helping the young person to re-engage in education; help with substance misuse problems and anger management; training in social problem-solving skills, and programs for crime awareness. The YOT treatments were delivered by professional social workers, therapists, or probation officers. During the intervention period, participants in the YOT group received approximately 21 professional appointments. There are two key differences between MST and YOT. 1) In contrast to YOT, MST is delivered in a family context by a single person. 2) There is no overarching model or set of principles that governs the selection of treatments in YOT, which could be compared to those in MST. Thus, YOT interventions are offered on an "as needed" basis by specialist agencies to which the young person is referred.

Assessment:
The primary outcome measure (offending behavior) was measured at 6-monthly intervals: for the 6 months before randomization, for the 6 months covering the intervention period, and then every 6 months until 12-months after posttest (to which the authors refer as an 18-month follow-up assessment). All secondary outcome measures were obtained at baseline and at posttest, after the MST treatment was completed (6 months after randomization).

Sample Characteristics: The majority of participants were male (82%) and were, on average, 15 years of age. The sample was racially diverse with 34% white, 32% black, 5% Asian, and 24% classified as Mixed/Other. Participants had an average of more than two offenses at intake with more than half the convictions constituting violent offenses. Only a small minority of individuals were living with two parents; more than two-thirds lived with their mothers but not their fathers, and less than 10% with their fathers but not their mothers. Only one-third were in mainstream education. Of the parents, 31% had left school with no academic qualifications; 40% had no vocational qualifications; and 54% were without income. In sum, almost all subjects lived in socioeconomically disadvantaged families.

Measures:
Validity of measurements:
All measures were borrowed from prior questionnaires for which validity and reliability had been demonstrated. Alpha values of the employed scales are not reported.

Primary outcome measures:

• The number of records of offending behavior (count data) was obtained from the National Young Offender Information System (YOIS) database. In addition, a dichotomous variable measured the occurrence of 6-month periods free of any offending behavior.

Secondary outcome measures:
Self- and parent-rated symptoms of antisocial behavior, delinquency-linked cognitions, personality functioning, and parenting variables were measured using the following items/scales:

• Self-Report of Youth Behavior (SRYB): Based on youth's self-report, this scale measured antisocial behavior such as vandalism, theft, burglary, and fraud.
• Youth Self-Report (YSR) questionnaire: The delinquency and aggression subscales of the YSR were employed.
• Child Behavior Checklist (CBCL): Was completed by parents or guardians.
• Antisocial Beliefs and Attitudes Scale (ABAS): Based on youth's self-reports, this scale measured beliefs and attitudes toward standards of acceptable behavior in social and family contexts.
• Positive Parenting and Disciplinary Practices (PP) measure: This questionnaire was completed by the primary caregiver.
• Subjective Family Image Test (SFIT): Completed by the family, this questionnaire assessed the quality of the emotional bond between adolescent and parent.
• Antisocial Process Screening Device (APSD): This parent-completed questionnaire measured youth psychopathic traits.
• Youth's Involvement with Delinquent Peers (IDP) scale: This parent-completed 16 items scale was adapted from the Youth in Transition Study.

In addition, a demographic data form was used to gather information regarding participants' ethnicity and socioeconomic background.

Analysis: The authors employed multilevel models to account for the hierarchical data structure (measurement points are nested within persons). Both intercepts and slopes of time were allowed to vary randomly across individuals. Depending on the investigated outcome measure, linear, logistic, or Poisson models were used. All models implicitly control for baseline characteristics.

Intention-to-treat: The study followed the intent-to-treat principle.

Outcomes

Implementation fidelity: Therapists that delivered the intervention received thorough training in MST programs and procedures as part of the study. In addition, therapists received weekly supervision from the MST supervisor, as well as weekly 1-hour consultation (via telephone) with an MST outside expert, on-site booster training sessions four times per year, and twice-yearly implementation reviews. The research team closely followed the MST Organizational Manual. In addition, the MST Therapist Adherence Measure (TAM) was used to assess adherence to the nine MST treatment principles.

Other than to note that YOT youth attended a significantly greater number of appointments than the MST youth, the study does not provide measures of fidelity.

Baseline Equivalence: The intervention and control group did not differ significantly at baseline on any measured variable.

Differential attrition: No test for differential attrition was performed. The authors point out that "missing values were not a significant problem in the analysis of the data set for primary outcomes (< 5%) and data for all participants were used for secondary outcomes, although three individuals in the MST and one in the YOT group provided no self-report information" (p. 1225).

Post-test/Long-term: Due to the reporting of time x group interaction, it is not possible to disentangle post-test and long-term effects, except for instances in which the authors provide these details in the text.

Primary outcomes:
Significant group x time interactions suggest a greater decrease in recorded offenses for youths in the MST group compared to youths in the control group. The number of offenses between the two study groups did not differ at post-test, and 6-month follow-up but became significant (p<.001) at the 12-month follow-up assessment. In addition, a significant group x time interaction was found for participants with 6-month periods free of offenses (p<.001). This effect was more pronounced in the MST group than in the YOT group. A sub-category analysis showed that the group x time interaction for both outcome measures was significant for nonviolent offenses (6-month period free of offenses, p<.005; No. of recorded offenses, p<.02) but not for violent offenses. For example, at the 12 months follow-up assessment only 8% in the MST group compared with 34% in the control group had one record or more of a nonviolent offense during the past 6 months (p<.001).

Secondary outcomes:
Out of 21 tests for secondary outcome measures, 5 (24%) were significant. Significant program effects were observed for measures of aggression and delinquency (p<.05). For all significant findings, the group x time interaction suggested that the problem behavior declined stronger for the MST compared to the YOT group. In addition, a number of significant group differences were found for personality, relational, and cognitive measures. The measure for parent reported psychopathic traits (ASPD) declined substantially more over time in the intervention compared to the control group (p<.02). In addition, positive parenting increased in the MST group but decreased in the control group, resulting in a significant (p<.05) group difference.

Effect size: The study reported that the effect size of the change over time for the MST group was medium for both aggression (ES = 0.42) and delinquency (ES = 0.64), while effect sizes were small for the YOT group on aggression (ES = 0.09) and delinquency (ES = 0.25). However, the study does not report effect sizes for differences between groups across time.

Mediating effects: A mediator analysis demonstrated that positive parenting did not account for program related changes in offending behavior among youths. Similarly, adherence to MST standards (based on parent's independent reports about their therapy) did not mediate the group differences on the primary outcome measure of offense frequency.

Study 14

Summary

The study used a randomized controlled trial that assigned 156 Swedish youths with a diagnosed conduct disorder to an MST intervention group or a treatment-as-usual control group. Assessments at posttest and two-year follow-up measured delinquency, alcohol and drug use, and school attendance.

The study found no effects of MST on parenting, mental health, delinquency, social competence, alcohol use, or drug use.

Evaluation Methodology

Using a clinical sample of youths diagnosed with conduct disorder, Sundell et al. (2008) examined the seven-month posttest results, while Lofhölm et al. (2009) examined the 17-month follow-up results.

Design:

Recruitment: The sample consisted of youths ages 12-17 years who met the criteria for a clinical diagnosis of conduct disorder. Exclusion criteria were (a) on-going treatment by another provider; (b) substance abuse without other antisocial behavior; (c) sexual offending; (d) autism, acute psychosis, or imminent risk of suicide; and (e) the presence of the youth in the home constituting a serious risk to the youth or the family. All youths were referred to the study from March 2004 through February 2005 by child welfare services in 27 local authorities (municipalities) from Sweden's three largest cities and one town. A total of 168 families were asked to participate in the study, and 156 (93%) accepted.

Assignment: Randomization of youths to the intervention group (n = 79) or control group (n = 77) immediately followed initial data collection, when research staff opened a sealed and numbered envelope that contained the result of computer-generated randomization. The randomization used the six MST sites as a blocking variable. The control group youths received treatment as usual, which consisted of social services such as individual counseling, family therapy, mentorship, out-of-home care, and residential care.

Assessments/Attrition: The posttest came about seven months after randomization, and a follow-up came 24 months after randomization or about 17 months after posttest. At the posttest, 14 of the 156 families lacked data (9%). Seven had withdrawn from the evaluation, and seven refused to complete the questionnaire. For specific posttest measures, missing data varied between 9-12% for youths and between 8-12% for guardians, with the exception of the guardian's mental health, where missing data reached 19%. At the follow-up, 16 young people and 15 parents did not complete the questionnaire. For specific measures, missing data varied between 10-11% for young people and between 10-16% for guardians.

Sample:

The sample consisted of 61% boys and 39% girls with a mean age of 15.0 years. At study intake, 67% of the youths had been arrested at least once and 32% had been placed outside of the home at some point during the six months before the study intake. Almost half (47%) of the families spoke a language other than Swedish in the home. A majority of the youths (67%) lived in a single-parent home. Of the mothers, 18% had a college education and 51% were unemployed. Of the families, 61% lived entirely or in part on social welfare grants.

Measures:

Youth Psychiatric Symptoms. Six measures from both caregiver and child ratings of total, internalizing, and externalizing symptoms came from the Child Behavior Checklist and Youth Self-Report. Internal consistency was high (.88-.96).

Sense of Coherence. One youth-reported measure from the Sense of Coherence scale reflected the youth rating of life as comprehensible, manageable, and meaningful. Internal consistency was acceptable (.78).

Self-Report Delinquency. Youths answered 40 questions designed to measure antisocial behavior pertaining to violence, general delinquency, and status offenses. Internal consistency was high (.92). In addition, social workers and parents reported on the number of police reports involving the youth.

Alcohol and drug consumption. Youths reported on the frequency and amount of their consumption of five types of alcoholic beverages (e.g., beer and wine) during the six months before measurement. Youths also reported on their use of six types of drugs (e.g., cannabis and cocaine) during the six months before measurement. The Alcohol Use Disorder Identification Test assessed risky alcohol consumption, dependency, and alcohol-related harm. Internal consistency was high (.88). The Drug Use Disorder Identification Test measured current drug-related problems, excluding alcohol. Internal consistency was high (.93).

Anti-Social Peers. One subscale from the Pittsburgh Youth Study measured relationships with antisocial peers. This measure had moderate internal consistency (.71).

Social Competence with Peers. Youth social competence was measured through caregiver and adolescent ratings of social interactions with peers. Internal consistency was .77 for youths and .84 for guardians.

Social Skills. A scale was adapted from the Social Skills Rating System. Internal consistency was acceptable (.72).

School attendance. Information on school attendance came from the school authorities and was based on the 95% of students who continued on to high school after completing compulsory education at Grade 9.

Out-of-Home Care and Social Services. The two measures came from case-file reviews. Social workers also reported on the type and extent of services received by families before referral.

Parenting Skills. Caregivers as well as youths described parenting skills, with a focus on parental knowledge, parental monitoring, parental soliciting, youth disclosure, and family decision making. Internal consistency was high for youth reports (.88) and parent reports (.89).

Mother's Mental Health. Mothers completed the Symptom Checklist-90 with nine symptom constructs and one global index. Internal consistency was high (.98).

Analysis:

The analyses used repeated measures ANOVA for outcomes with both a pretest and posttest and used one-way ANOVA for outcomes with no baseline measure. The key coefficients came from the treatment-by-time interaction terms.

Intent-to-Treat: The analyses used all cases in their original assignment. Missing data at the posttest were imputed by carrying the baseline measure forward, while missing data at follow-up were imputed with multiple imputation.

Outcomes

Implementation Fidelity:

About 73% of the MST clients completed the program on the basis of the mutual agreement of the primary caregiver(s) and the MST team. This was similar to the average treatment completion rate (74%) for MST programs worldwide. Fidelity scores for 160 interviews with 60 families (alpha = .86) had a mean of 4.00 (range of 1.73 to 5.00), approximately one standard deviation lower than that reported in a U.S. study.

Baseline Equivalence:

Intervention parents exhibited significantly more mental health symptoms than control parents, and control parents reported more children with early-onset behavior problems than intervention parents. No other significant differences existed between the treatment and control groups on demographic or psychosocial variables.

Differential Attrition:

At posttest, the 14 youths who did not complete the questionnaire did not differ significantly from the 142 youths who did on either demographic or psychosocial characteristics. At follow-up, the 16 young people and 15 parents who did not complete the questionnaire did not differ significantly on any of the baseline measures from the 140 young people and 141 parents who did.

Posttest:

Tests reported by Sundell et al. (2008) for 20 outcomes revealed only one significant result: The intervention youth received more days of service than the control youth. Numerous tests for moderation found few subgroup differences in program effects.

Long-Term:

Tests reported by Lofhölm et al. (2009) for 20 outcomes revealed no significant results. Numerous tests for moderation found few subgroup differences in program effects.

Study 15

Summary

The study used a randomized controlled trial that assigned 684 English youths with antisocial behavior to an MST intervention group or a treatment-as-usual control group. Assessments through five years measured out-of-home placement and criminal offending.

The study found several posttest effects of the MST intervention, but the effects disappeared after one year and the program showed no long-term effects.

Evaluation Methodology

Design:

Recruitment: The sample came from nine sites across England. Young people ages 11-17 with moderate-to-severe antisocial behavior were recruited from local service organizations. Those eligible had at least three severity criteria indicating past difficulties across several settings and one of five general inclusion criteria for antisocial behavior: persistent (weekly) and enduring (≥6 months) violent and aggressive interpersonal behavior; at least one conviction plus three additional warnings, reprimands, or convictions; conduct disorder diagnosed according to DSM-IV criteria and not responding to treatment; permanent school exclusion for antisocial behavior; and significant risk of harm to others or self. Between February 4, 2010, and September 1, 2012, 1,076 families of young people were referred as eligible, and 684 families consented to baseline assessment.

Assignment: Eligible families were randomly assigned (1:1) by an independent researcher to the management-as-usual control group (n = 342) or to three to five months of Multisystemic Therapy followed by management as usual (n = 342). The randomization stratified for treatment site, sex, age at study enrolment (<15 years or ≥15 years), and age at onset of antisocial behavior. Management as usual involved the best available local services for young people, as identified by a multi-agency referral panel.

Assessments/Attrition: Outcomes were assessed at baseline and six, 12, 18, 24, 36, 48, and 60 months after randomization. With the program lasting three to five months, the assessments included a posttest and six-month follow-up plus long-term follow-ups ranging from one year to 4.5 years after treatment end. For the primary outcome, which was measured at two time points, 99% of the participants had data at 18 months and 89% at 60 months. For secondary outcomes, which were measured at six time points, 85% of the 684 randomized families remained in the study at six months, 76% at 12 months, 72% at 18 months, 70% at 24 months, 63% at 36 months, and 51% at 48 months.

Sample:

More than 80% of the sample youths met DSM-IV criteria for a conduct disorder (including oppositional defiant disorder). Their mean age was 13.8 years, with 63.5% being male. About 75-78% of the families had low income, and most were white (76-80%).

Measures:

Investigators and research assistants were masked to treatment allocation and were located separately to avoid contamination. The primary outcome in Fonagy et al. (2018) was the proportion of participants in out-of-home placement at 18 months. Secondary outcomes included time to first criminal offense and the number of offenses (total, non-violent, and violent), based on official police records. In addition, Fonagy et al. (2018) examined about 39 other secondary outcomes (see Tables 4-7 for a list). Many relating to conduct problems, delinquency, ADHD, and strengths and difficulties came from questionnaires completed by young people, parents, and teachers. Parents rated their children as well as their own parenting skills and mental health. Because parents and youths participated jointly in the program, parent ratings of the youths can be considered independent. Measures of school attendance and exclusions came from educational records. Psychiatric disorders were assessed by clinicians at baseline and 12 months using the Development and Well-Being Assessment instrument. Appendix Table A.6 shows good reliabilities for nearly all scales.

The primary outcome in Fonagy et al. (2020a, 2020b) was the proportion of young people with any criminal conviction at 18 months and 60 months. The secondary outcomes included the total number of recorded offenses with convictions, time to conviction, and whether the conviction resulted in a custodial sentence, community sentence, or a caution. The numerous self-reported secondary outcomes completed by the young person and their parent or caregiver measured antisocial behavior and attitudes, parenting, family functioning, and the use of healthcare, social care, education, and criminal justice services (see Table 2 in Fonagy et al., 2020b, pp. 10-11). With a few exceptions, the measures had acceptable reliability coefficients (see Appendix 1, Table 20 in Fonagy et al., 2020b, p. 59).

Analysis:

The analysis of out-of-home placement at 18 months in Fonagy et al. (2018) used a mixed-effects logistic regression model, and the analysis of the time-to-event outcomes for first criminal offense used Cox regression. Random effects adjusted for clustering by therapist. The logistic regression model included covariates for site, number of past convictions, sex, and age at onset of criminal behavior. The analysis of secondary outcomes used linear mixed models, with adjustment for baseline values, and Poisson mixed models for count variables.

The analyses in Fonagy et al. (2020a) used mixed-effects logistic regression models for dichotomous outcomes, Cox proportional hazards models for time-to-event outcomes, Poisson mixed models for count outcomes, and linear mixed-effects models for continuous outcomes. All models included the following covariates: prior criminal convictions, sex, age at onset of criminal behavior, site as a random effect, and the baseline outcome. The models used Full Information Maximum Likelihood estimation under the assumption of data missing at random. In addition, the analysis used multiple imputation for all baseline and outcome variables. For the numerous secondary outcomes, the Benjamini-Hochberg multiple testing procedure adjusted for multiple comparisons.

Intent-to-Treat: The analyses included all randomized participants with available data. A sensitivity analysis used multiple imputation but found only minor differences for the results (see the Appendix in Fonagy et al., 2018).

Outcomes

Implementation Fidelity:

The study did not present quantitative analyses, but the authors stated that eight of nine sites performed well above the standards expected by the developers. Detailed cost analyses showed no savings from MST.

Baseline Equivalence:

As shown in Table 1 (Fonagy et al., 2018), the two conditions were similar at baseline, but the study lacked significance tests or d values. The authors noted that slightly more young people with attention deficit hyperactivity disorder were in the intervention group (33%) than in the control group (27%).

Differential Attrition:

At 18 months, the control group had fewer families available for assessment (68%) than the intervention group (75%). Three direct observational points were available for 86% of the intervention families and 81% of the control families. Appendix ii (Fonagy et al., 2018, p. 18) compared the baseline scores of participants who completed the 12-month assessment with those who did not. Although there were no significant differences between these groups at p < .01, there were four significant differences at p < .05 in 42 tests.

Fonagy et al. (2020a) noted that "missing data due to loss to follow-up was not associated with any baseline characteristics, corroborating the representativeness of the sample."

Posttest:

At six months (posttest) or 12 months (six-month follow-up), the program did not significantly affect the number of total crimes, violent crimes, or non-violent crimes.

At six months, Tables 4-6 in Fonagy et al. (2018) show significant program effects for 17 of 33 tests, while at 12 months, the tables show significant program effects for eight of 33 tests. The intervention group did better than the control group on total strength and difficulties (youth reported), emotional problems (youth reported), mood and feelings (youth reported), family satisfaction (parent reported), total strengths and difficulties (parent-reported), emotional problems (parent reported), hyperactivity (parent reported), and parent mental health (parent-reported). In addition, school attendance and exclusion did not differ significantly across conditions, and clinician-rated mental health at 12 months showed no significant effects in six tests (Table 7).

Long-Term:

At 18 months (one-year follow-up), Fonagy et al. (2018) reported no significant program effect on the primary outcome of out-of-home placement or on the secondary outcome of time to first offense (Table 2). At 18 months, there was a significant iatrogenic effect on the mean number of crimes (Table 3). Moderation tests showed three significant subgroup differences in program effects in 19 tests, but all three indicated harmful effects of the program for high-risk youth.

Also at 18 months in Fonagy et al. (2018), Tables 4-6 list significant program effects for two of 33 tests: youth callous and unemotional traits and parent mental health (labeled as General Health Questionnaire in Table 6). By the long-term assessment, none of the earlier significant youth effects persisted, and among the parent risk and protective measures, only self-rated mental health remained significant.

Fonagy et al. (2020a) found no significant differences between conditions at the 60-month follow-up for the primary outcome of a criminal conviction. For the secondary outcomes related to offending, the MST group did significantly worse than the control group at 24 and 48 months, but the difference fell to non-significance by 60 months. For the secondary outcomes at 24, 36, and 48 months, the results showed no condition differences in youth- or parent-reported outcomes after adjusting for multiple tests (see Appendix 3, Tables 22-35 and Appendix 4, Tables 36-45 in Fonagy et al., 2020b).

Study 16

Summary

The study used a quasi-experimental design that examined 320 youths with serious emotional disorders. The participants were non-randomly assigned to an MST-only group, a wraparound-only group, or a wraparound plus MST group that received both interventions. Measures of behavioral problems and global functioning were obtained over an 18-month period.

The study found that, relative to the wraparound-only condition, the MST-only condition showed significantly greater reductions in:

• Parent-rated youth behavioral problems.

Evaluation Methodology

Design:

Recruitment: The sample consisted of 320 children and adolescents ranging in age from 4 to 17.5 years. Participants came from a care center covering a rural and frontier area of Nebraska and targeting youth with serious emotional disorders. Referrals came from schools, welfare agencies, justice organizations, and other child-serving agencies. To qualify for wraparound services, the youths must have demonstrated a diagnosable mental health disorder, functional impairment, and high risk for restrictive placement, dropping out of school, or involvement in the juvenile justice system. Addition criteria for MST services included physical aggression, delinquency, and substance use.

Assignment: In a quasi-experimental design that used no matching, the study examined three groups of youth: a wraparound-only group (n = 213), an MST-only group (n = 54), and a wraparound plus MST group (n = 53) that received both interventions, sometimes at the same time and sometimes in sequence. Assignment appeared to be based on the extent of problems of the youth, as wraparound and MST had some different requirements for treatment and wraparound plus MST was provided to youth who did not respond to their original treatment. The authors noted (p. 159) that "Propensity score matching was also considered to account for baseline differences; however, the sample size was insufficient to allow for reliable estimates."

Assessments/Attrition: Assessments occurred at baseline and three consecutive six-month follow-ups over 18 months. Attrition rates were 11% at six months, 28% at 12 months, and 37% at 18 months. The final analysis sample, which allowed for use of participants without data for all four time points, included 298 participants (93%). Average treatment lengths ranged from 5.5 to 15 months, which means the assessments did not include a long-term follow-up.

Sample:

The sample averaged 12 years of age, and 73% were male. The racial distribution of the sample was 90% white, 4% American Indian, and 6% other. About 11% of the participants were of Hispanic ethnicity. Most families (57%) reported a gross household income of $25,000 or less.

Measures:

The study examined two outcome measures, one reported by parents (total behavioral problems from the Child Behavior Checklist that combined internalizing and externalizing subscales) and one reported by caregivers or clinicians (global functioning across eight life domains from the Child and Adolescent Functional Assessment Scale). Parents participated in but did not deliver the intervention and therefore provided independent assessments of their children. Both instruments are commonly used and well validated, though the authors did not report reliability and validity information for the study sample.

Analysis:

The analysis used repeated measures mixed models with maximum likelihood estimation and an unstructured covariance structure, random coefficients, and random slopes. The models controlled for baseline outcomes, gender, age, minority status, family income, and number of living placements reported at each six-month follow-up. Condition-by-wave interaction terms tested for program effects.

Intent-to-Treat: Models were estimated with imputed and non-imputed data, but because the results did not differ, the study reported results from the non-imputed data.

Outcomes

Implementation Fidelity:

MST-only youth spent 5.5 months in therapy and reported higher participation in family preservation and family therapy than the wraparound-only youths.

Baseline Equivalence:

Table 1 showed eight significant baseline condition differences in 11 tests. The MST-only group was older, included more males, and had fewer problems than the other groups. Youth who received both wraparound and MST had more severe problems than those of the other two groups, likely because they did not respond to the first assigned intervention.

Differential Attrition:

The authors reported that attrition at 18 months did not differ by treatment group, gender, age, minority status, family income, or the baseline outcomes.

Posttest:

The mixed models found, as indicated by the group-by-wave interactions, significantly greater improvements for the MST-only group than the wraparound-only group for one of two outcomes - total behavioral problems from the parent-rated Child Behavior Checklist but not global functioning from the Child and Adolescent Functional Assessment Scale.

Long-Term:

Not examined.

Study 17

Summary

The study used a randomized controlled trial that assigned 674 youths with serious emotional disorders in rural Tennessee to an MST condition or a usual-services condition. Measures of behavioral problems and out-of-home placements were obtained over an 18-month period and used to examine MST effectiveness in counties with and without a countywide service-facilitation program.

The study found that, relative to the usual-services condition, the MST condition showed significantly fewer:

• Out-of-home placements
• Behavioral problems but only in counties with a countywide service-facilitation program

Evaluation Methodology

The study focused on testing the benefits of an organizational intervention called ARC (Availability, Responsiveness, Continuity) that aims to improve the delivery of community-based mental health services. In so doing, the study examined MST alone as well as MST in combination with ARC.

Design:

Recruitment: The sample came from 14 of the poorest, least populated counties in the rural Appalachian region of eastern Tennessee. Between October 2003 and September 2007, youths referred to juvenile court in each of the 14 counties were recruited for the study if they met a list of 11 eligibility criteria, including ages 9-17, referral for a status offense or delinquent behavior, at risk of out-of-home placement, and a diagnosis of serious functioning problems or psychiatric symptoms. A total of 674 youth and their caregivers met the eligibility criteria and agreed to participate.

Assignment: First, randomization at the county level assigned six counties to the ARC intervention and six to no ARC intervention. Second, delinquent youth were randomly assigned within each county to either the MST program or to the usual services program. The second assignment used predetermined, concealed randomization of sequence numbers based on the order of recruitment. Youth assigned to usual services received a variety of inpatient and outpatient mental health services. The double randomization created four conditions that allowed for 1) the comparison of youth receiving MST (n = 349) with youth receiving usual services (n = 325) and 2) the county-level ARC intervention to moderate MST effects.

Assessments/Attrition: The four assessments occurred at baseline, six months, 12 months, and 18 months. Completion rates were 77% at six months, 69% at 12 months, and 52% at 18 months. According to the CONSORT diagram, however, the analysis included 84-88% of the participants who had data for at least two time points. With the MST intervention lasting an average of about 3.5 months, the 18-month assessment defined a long-term follow-up.

Sample: The youth sample averaged 14.9 years of age, was 69% male, and reflected the racial characteristics of the rural Appalachian region of eastern Tennessee (91% white). Over half (53%) of the youths had two or more mental health diagnoses.

Measures: The outcomes included two measures. Total youth behavior problems, a combination of internalizing and externalizing, was measured by the caregiver-rated Child Behavior Checklist (alphas ranged between .94 and .95 across waves). Youth out-of-home placements in state custody were measured by interviews with the youth's caregiver. As caregivers participated in but did not deliver the program, the measures appeared to be independent.

Analysis: The analysis used multilevel random-effects models with youth nested within counties, linear functions for the continuous outcome, and a logit link function for the dichotomous outcome. The equations for the models on pages 545-546 included county as a level-2 factor with random effects that adjusted for within-county clustering. Although not stated explicitly, the models imply the use of FIML estimation.

Missing Data Methodology: The analysis excluded only those participants with data for less than two time points and used mixed models with FIML to include others with incomplete data.

Intent-to-Treat: The analysis did not include all participants but included many with incomplete data.

Outcomes

Implementation Fidelity: Detailed analyses compared MST fidelity with and without the ARC. The findings indicated that the inter-organizational components of ARC facilitated greater therapist sensitivity to, and improvement in relations between, the family, community, and service system.

Baseline Equivalence: The authors reported only that there were no significant differences between ARC and non-ARC county means on total population, per capita income, and percentage of children in poverty. No figures or tests were presented for baseline differences in youth characteristics.

Differential Attrition: The authors reported that "a pattern-mixture model for the analysis of missing data found no differences in the effects of ARC and MST between youth with and without data at each wave" and that "no differences were found in the effects of ARC and MST between youth with and without data at each wave." Pattern mixture models typically use available data to identify missing data patterns and then include indicators of the patterns as covariates. Also, the use of imputation may moderate potential attrition bias.

Calculations based on the CONSORT diagram indicated attrition of 13-18% in the MST condition versus attrition of 10-14% in the control condition. Based on the overall attrition rate and the difference in attrition rates between conditions, the study met both the WWC cautious and optimistic standards at all assessments.

Posttest and Long-term: Over the 18-month follow-up period, out-of-home placements were significantly lower for the youths who received MST than for youths who received usual services (OR = .472). ARC did not significantly moderate the effect of MST.

Over the first six months and the next 12 months (18 months total), the measure of behavioral problems did not differ significantly between the MST group and the usual services group. However, moderation tests found that MST was significantly more effective than usual services in ARC countries in the first but not the second period.

Study 18

Summary

The study used a quasi-experimental design that examined 215 youths with serious behavioral problems in Washington State who had been referred to receive MST services. These youths were then matched to comparison youths who had an encounter with the public mental health system. Measures were obtained for criminal convictions and mental health services over one year.

The study found that, relative to a matched comparison group, the MST group showed significantly higher:

• Utilization of mental health services.

Evaluation Methodology

Design:

Recruitment: The study began with adolescents in Thurston and Mason counties of Washington State who met the following criteria: ages 12 to 17 years, exhibited behavioral challenges and significantly interrupted functioning across multiple domains, and faced high risk of being placed out of home. Referrals came mostly from the juvenile justice or mental health systems (71%). Of 219 referrals made from April 2007 through June 2010, 215 enrolled in the study and 183 remained after removing duplicates and those without matches in the administrative database.

Assignment: In this quasi-experimental design, the 183 youths referred to MST served as the intervention group. Comparison youths were ages 9-17 and had an encounter in the public mental health system over the same study period used for the intervention youths. The study identified matched pairs of youths who were similar in age, gender, ethnicity, primary diagnosis, level of functioning, prior mental health service utilization, and criminal history. It was unclear if the comparison youths came from the same counties. For crime outcomes, 101 participants were successfully matched, and for mental health service utilization outcomes, 126 participants were successfully matched.

Assessments/Attrition: The study assessed outcomes over a one-year follow-up period. As it included the treatment time, the one-year assessment period did not include a long-term follow-up. The retrospective study used a sample of youths with existing data, and there was no attrition.

Sample:

The intervention group included 60% males, averaged 14 years of age (with a range from under 10 to 17), and was 83% white, 9% Hispanic, 5% Native American, 2% Asian and Pacific Islander, and 1% African American. About 68% had a misdemeanor or felony conviction in the previous year, and the majority were diagnosed with behavioral and disruptive disorders.

Measures:

Outcome measures came from administrative data gathered by multiple state agencies. The eight measures included criminal convictions (total, misdemeanor, felony, violent) and mental health services utilization (inpatient admission, outpatient treatment, support services, and crises services). The study provided no details on the reliability or validity of the measures.

Analysis:

The analysis used logistic regression with controls for age, gender, race, primary mental health diagnoses, level of functioning, and prior measures for the outcome of interest.

Intent-to-Treat: The analysis used all available data.

Outcomes

Implementation Fidelity:

The study stated that "MST . . . has met adherence standards throughout the project," but also that the program served some youths who did not fit the eligibility criteria.

Baseline Equivalence:

Before matching, MST youths were more likely than comparison youths to be involved in the criminal justice system, use intensive inpatient and other mental health services, be diagnosed with behavioral or attention deficit disorders, and have experienced multiple out-of-home placements. After matching (Exhibit A6.1, p. 7), the conditions still differed on five of 15 baseline measures.

Differential Attrition:

No attrition.

Posttest:

Tests showed no significant condition differences over the one-year follow-up period on any of the four criminal conviction measures (although the percentages favored the MST youth and the effect size for any conviction was -.10). Additional tests for use of mental health services found one significant intervention effect: the MST group had higher utilization rates of mental health support services. However, the author noted that increased use of support services could be characterized as a good outcome (MST promotes access to services) or a negative outcome (MST is associated with increased dependence on public services).

Long-Term:

Not examined.

Study 19

Summary

The study had no control group, instead offering a comparison of MST with Functional Family Therapy. It used a quasi-experimental design that examined 1,256 youths under the care of the Florida Department of Juvenile Justice who were referred to MST or Functional Family Therapy for treatment. The study matched participants in the two interventions using propensity scores (628 in each) and measured criminal offending during the intervention and over the next 12 months.

The study found that, after statistical correction for multiple tests, the MST group did not differ significantly from the matched Functional Family Therapy group on offending during the treatment period or during the 12 months after the treatment.

Baglivio et al. (2014) found no significant effects favoring the MST group compared to the matched Functional Family Therapy group.

Evaluation Methodology

Design:

Recruitment: The sample included all 2,312 juvenile offenders under the care of the Florida Department of Juvenile Justice who were referred to MST or FFT between July 1, 2009, and June 30, 2011. Excluding 109 youth who turned out to not qualify for the programs left 2,203 for the study. The statewide sample came from 10 MST sites and 11 FFT sites within 50 of Florida's 67 counties (both rural and urban for each service).

Assignment: The quasi-experimental design began with 629 youths referred to MST and 1,574 youths referred to FFT. Hundreds of probation officers made the referrals over the two-year period. With MST and FFT rarely available at the same time and the same sites, officers seldom had a choice between the two programs. Officers at all sites used the same form and the same inclusionary/exclusionary criteria for both programs. However, officers in sites with MST may have used different selection criteria for the program than officers in sites with FFT.

The study used propensity score matching to adjust for observed differences across the conditions with nearest-neighbor matching, no replacement, and a caliper of .05 (all analyses were replicated using a more restrictive caliper of .01 and nearest-neighbor matching without replacement). Predictors included gender, race, region of the state, age at admission to service, overall risk to reoffend, criminal history, a global social risk measure, most serious prior offense, age at first offense, antisocial peer association, alcohol use, drug use, and parental authority. Separate matching was done for the full sample, for males, females, whites, non-whites, low-risk youth, and moderate-high to high-risk youth.

The final matched sample included 628 youths in each condition (1,256 total).

Assessments/Attrition: The study examined data for the period of service and for the 12 months after the end of service. Service lasted about four months for MST and about three months for FFT. The retrospective data had no attrition.

Sample:

The matched sample was 69% male and 48% non-white.

Measures:

The two outcome measures came from state administrative records. First, a dichotomous measure of offending during service included any new criminal offense referral (arrest) or violation of probation while receiving or enrolled in MST or FFT. The longer time of service for the MST group could affect the offending measure.

Second, a dichotomous measure of recidivism captured whether the youth was adjudicated or convicted for an offense that occurred within 12 months of termination of MST or FFT (termination of service could be either successful or unsuccessful completion). The measure included data from adult records for youths who turned 18 years of age during the follow-up period. The recidivism measure, unlike the offending measure, used adjudicated/convicted offenses rather than arrests.

Analysis:

The analysis used t-tests with a Bonferroni correction for two outcomes (.05/2 = .025) in comparing the two matched conditions.

Intent-to-Treat: The analysis used all youth referred to the services, even if they did not successfully complete the program.

Outcomes

Implementation Fidelity:

The study noted only that the completion rate for MST of 71.9% was significantly lower than for FFT (76.5%).

Baseline Equivalence:

Many significant condition differences existed before matching, with MST having lower scores on percent male, risk, criminal history, and seriousness of priors. However, after matching, there were no significant differences in tests for 13 matching variables. Effect sizes for the differences were all below .25.

Differential Attrition:

No attrition.

Posttest:

With the usual significance tests, the FFT group had significantly lower offending during services than the MST group, but the effects fell to non-significance with a Bonferroni correction and with matching using a .01 caliper. Some evidence suggests that the significant effects favoring FFT were strongest for young women and low-risk youth.

Long-Term:

Recidivism for the 12 months after the end of treatment did not differ significantly across the two conditions.

Study 20

Summary

The study had no control group, instead offering a comparison of MST with Functional Family Therapy. It used a quasi-experimental design that examined 697 youths from a Dutch institute for behavioral problems who were referred to MST or Functional Family Therapy for treatment. The study matched participants in the two interventions using propensity scores and measured posttest externalizing as the primary outcome.

The study found that the MST group did significantly better than the matched Functional Family Therapy group on:

• Engagement in school or work at posttest.

Evaluation Methodology

Design:

Recruitment: The sample consisted of 1,714 adolescents and their families who started either Functional Family Therapy (FFT) or MST at an institute for personality disorders and behavioral problems in the Netherland between October 2009 and June 2014. About 45% of the youths had a court order for treatment and were considered high risk. Of the 1,714 youths undergoing treatment, 697 completed the treatment and comprised the final sample.

Assignment: Adolescents with more serious problems were assigned to MST (n = 1074) and those with less serious problems to FFT (n = 640). After dropping those not completing the treatment, 422 youths (39%) remained in MST and 275 youths (43%) remained in FFT. To adjust for differences, the study used weights based on propensity scores. The covariates used in the propensity model (Table 1) included measures of sociodemographic characteristics, family relationships, and the baseline outcomes. Use of covariate missing values as predictors allowed for inclusion of all participants in the propensity score model.

The MST weight was 1, while the FFT weight was an odds ratio (i.e., the propensity score divided by one minus the propensity score). The propensity scores were estimated for the full sample and also for the subsamples of youths with and without court orders.

Assessments/Attrition: The posttest assessment occurred at the end of treatment. With data only on participants who finished treatment, 697 (40.7%) completed the primary outcome measures.

Sample:

Of the adolescents who completed MST, about 65% were male and about 81% were born in the Netherlands. They had an average age of 15.7 years.

Measures:

Externalizing as reported by both parents (Child Behavior Checklist) and youths (Youth Self Report) served as the primary outcome, with the measures having alphas of .94 and .88, respectively. The three secondary outcomes included 1) living at home (i.e., no out-of-home placement); 2) engagement in school or work for at least 20 hours per week at the end of treatment; and 3) new police contact due to inappropriate or illegal behavior during the treatment period. The therapists reported the three secondary outcomes as part of a standard scoring procedure that supervisors and consultants monitored.

Analysis:

The analysis used linear regression (with the propensity score weights) for continuous outcomes and logistic regression for binary outcomes. Tests of significance used 5,000 bootstrap samples to obtain non-parametric 95% confidence intervals.

Intent-to-Treat: Using only participants who completed treatment, the study examined a treatment-of-treated sample rather than an ITT sample.

Outcomes

Implementation Fidelity:

The mean adherence score within MST was .53, which was in accordance with the norm of ≥0.50 provided by MST Institute.

Baseline Equivalence:

Before matching, tests showed numerous significant differences and large standardized differences (d > .25) between the conditions (Table 1). After matching, the standardized differences all fell to below .25. Also after matching, the variance of key measures was similar across conditions. However, the authors noted that eight participants from the MST group and 12 participants from the FFT group were removed from the baseline tests because "there was no overlap" in the propensity scores.

Differential Attrition:

Those completing the treatment and having outcome data differed significantly from those who did not with regard to their country of birth, living situation, and whether or not they had a court order before treatment, as well as the level of education and employment status of their primary caregiver, and whether or not this primary caregiver had a partner.

Posttest:

One of five outcomes showed significant condition differences at posttest: A significantly higher proportion of adolescents who had completed MST versus FFT were engaged in school or work after treatment (d = .19). Tests for subgroup effects found that among adolescents without a court order, MST did significantly more than FFT to reduce both parent- and child-reported externalizing problems.

Long-Term:

Not examined.

Study 21

Summary

The study had no control group, instead offering a comparison of MST with an adapted version of MST for adolescents with intellectual disabilities (MST-ID). It used a quasi-experimental design that examined 63 Dutch adolescents with intellectual disabilities who were referred to MST or an adaptation of MST for adolescents with intellectual disabilities. The study matched participants in the two interventions using propensity scores and measured living arrangements, school/work, and police contacts at posttest and six-month follow-up as the primary outcomes.

The study found no significant effects favoring the MST-only group.

Evaluation Methodology

Design:

Recruitment: The sample, obtained in the Netherlands between March 2014 and October 2015, included families of adolescents with antisocial or delinquent behavior and with intellectual disabilities (i.e., an IQ score of 50 to 85). In addition, eligible participants were at least age 12 and were receiving no other treatments. Of the 214 families who met the inclusion criteria, 128 families (60%) consented to participate.

Assignment: Dutch agencies referred families to standard MST (n = 73) or MST‐ID (n = 55), a program that adapted MST for adolescents with intellectual disabilities. The alternate treatment was provided by two teams from one organization specializing in care for people with intellectual disabilities, while the standard MST was offered by 24 teams from seven Dutch organizations.

The study used propensity score weights based on 27 predictors to adjust for condition differences. All observed baseline variables, as well as missing indicators for all baseline variables with missing data, were included as predictors in the propensity score model. The missing data indicators allowed all families to be included in the propensity score estimation. The resulting propensity scores were used as weights equalizing condition differences. However, limiting the matched participants sample to those with overlapping propensity scores reduced the sample size from 128 to 63. The differences for 40 families in the MST group and 25 in the alternative group were too great to include in the analysis (i.e., scores did not fall in the range of scores observed in the other treatment group).

Assessments/Attrition: Assessments occurred at baseline, posttest, and six-month follow-up. Six months after treatment, 87 families (68%) provided data. As noted above, additional losses came from non-matching propensity scores, which reduced the sample to 63 families (49%).

Sample:

Across the two conditions, the percent female in the sample ranged from 36-44%. The average ages were 14.9 and 15.2. About 95% of the sample was born in the Netherlands.

Measures:

The outcome measures came from the reports of therapists and parents at baseline and posttest and from parents at follow-up. The three primary outcomes included if the adolescent was living at home, attending school or working for at least 20 hours a week, and free of police involvement since the start of treatment. At posttest, therapist reports were monitored by the team supervisor and the MST consultant.

The study also examined eight secondary outcomes at posttest relating to parenting, family relations, and youth externalizing. At the follow-up, only the rule-breaking component of externalizing was examined. The study reported no information on reliability or validity for the secondary measures.

Analysis:

The analysis used logistic regression for dichotomous outcomes and linear regression for continuous outcomes. Bootstrapping was used to calculate 90% confidence intervals for all outcome measures, with 5,000 bootstrap samples drawn from the weighted sample.

Intent-to-Treat: The study did not drop participants for not completing the treatment, though many subjects were lost in the matching.

Outcomes

Implementation Fidelity:

On a scale ranging from 1 (not at all) to 5 (very much), the average therapist adherence score was 4.38 for standard MST.

Baseline Equivalence:

Using the full sample of 128, there were five significant condition differences before matching. After matching, significance tests were not reported, but all baseline variables as well as the missing indicators used in the propensity score estimation had standardized mean differences lower than 0.25. This result came from excluding the 65 families with non‐overlapping propensity scores and using the analysis sample of 68.

Differential Attrition:

The study did not test for differences between those with and without posttest and follow-up data.

Posttest:

All significant effects favored the alternative treatment. Based on a 90% confidence interval, none of the three primary outcomes differed significantly at posttest, while one, living at home, favored the alternate treatment at six-month follow-up. Several secondary outcomes were significantly better for the alternate treatment group.

Long-Term:

Not examined.

Study 22

Summary

The study used a quasi-experimental design that examined 740 youths referred to MST by the state of Rhode Island. The intervention group completed the treatment, while the comparison group did not complete the treatment for reasons unrelated to case progress. After using propensity scores to match the two conditions, the study examined delinquency outcomes over the next six years.

The study found that the intervention group relative to the comparison group showed significantly lower risks for

• Out-of-home placements
• Adjudications
• Juvenile training school.

Evaluation Methodology

Design:

Recruitment: The sample came from five MST providers that were active in Rhode Island from 2008-2011. A total of 772 youths ages 11-17 were referred to MST by either Child Welfare Services or Juvenile Correctional Services in Rhode Island, primarily for reasons of behavioral problems, disobedience, truancy, and delinquency. The youth who received an out-of-home placement prior to initial program contact were excluded, leaving a final sample of 740.

Assignment: The treatment condition included all youth referred to MST who either completed treatment or had the opportunity to do so. The latter group included youth who were unable to complete treatment due to either lack of engagement or a placement occurring while involved in treatment. Of the total sample of 740, 78% (n = 577) met criteria for inclusion in the treatment condition, with 90% (n = 522) completing treatment, 8% (n = 46) discharged due to placement, and 2% (n = 9) discharged due to lack of engagement. The median length of stay in the program for this treatment group was 4.4 months. The comparison group included the remaining 163 youth (22%) who were referred to MST but did not have the opportunity to complete treatment as a result of reasons unrelated to case progress. Of these, 94% (n = 154) were removed from treatment due to administrative or funding/referral reasons and 6% (n = 9) moved out of the program's service area. The median time for the comparison group from initial program contact to program discharge was 1.45 months.

The study used propensity score matching to balance pre-treatment characteristics of youth across treatment conditions. After comparing multiple methods of matching, the authors settled on the Kernel-based matching approach with a bandwidth of 0.08 and a common support restriction. The covariates included age, gender, race, number of prior episodes of out-of-home care, number of placements, history of maltreatment, history of adjudication, history of juvenile training school placement, and case assignment prior to or during MST treatment. The matched sample lost 20 participants from outside the common support region.

Assessments/Attrition: The study outcomes were examined over a six-year period after the initial program contact. The retrospective use of pre-existing data meant there was no attrition.

Sample:

The majority (57%) of youth were male, and the average age was 14.9 years at the time of referral. For self-reported race and ethnicity, 49% were White, 14% Black, 29% Hispanic, and 8% other or multiracial. In terms of case history, 36% of youth had a history of maltreatment, and more than half (57%) had a history of out-of-home placement. More than one-quarter of youth had previously been adjudicated (26%) and placed in a juvenile training school (27%).

Measures:

The data came from administrative records of the MST providers and the Rhode Island Department of Children, Youth, and Families. The three outcomes measured the time-to-event for youth of out-of-home placement (i.e., removal from parental custody due to reasons such as child behavior or disability, parental inability to cope, abuse or neglect), adjudication (i.e., court determination that the youth committed a crime or delinquent offense), and placement in a juvenile training school.

Analysis:

The analysis used Cox proportional hazards regression models to examine the risk of removal and delinquency outcomes following initial contact with the program. Youth who did not experience the outcome were censored at the end of the study observation period. For youth who experienced a subsequent MST referral or treatment prior to the end of the study observation period (n = 48), the outcomes were censored at the beginning of the second MST referral for treatment. In addition, youth who turned 18 prior to event occurrence or the end of the study period were censored on their 18th birthday. The covariates included prior history for the outcomes.

Intent-to-Treat: The study used all referred participants who began the program, though 4% were excluded because they experienced the outcome before the program began.

Outcomes

Implementation Fidelity:

No quantitative information.

Baseline Equivalence:

Prior to matching, significant differences were found between the treatment and comparison groups on race/ethnicity, the number of removals from parental custody, and the number of out-of-home placement settings in the first removal episode. Relative to the comparison group, the treatment group had fewer racial and ethnic minority youth, experienced fewer out-of-home care episodes, and had fewer placement transitions in their first episode of care. After matching, no significant differences were found between the treatment and comparison groups on these covariates.

Differential Attrition:

No attrition.

Posttest and Long-Term:

The Cox survival models showed significantly lower risks for all three outcomes - out-of-home placement, adjudication, and juvenile training school - in the intervention group than the comparison group. The hazard ratios ranged from .57 to .61.

Study 23

Summary

The intervention examined in this study adapted MST principles and techniques for youth with an autism spectrum disorder. The study used a randomized controlled trial that assigned 15 youths with an autism spectrum disorder and severe disruptive behavior to an MST condition or a usual-services condition. Measures of conduct problems, internalizing, and aggression were obtained at six months and 12 months after recruitment.

The study found that the MST intervention group showed significantly better family adaptability at six- and 12-months post-recruitment than the usual-services control group.

Wagner et al. (2019) found that, relative to the usual-services control group, the MST intervention group showed significantly better:

• Family adaptability at six- and 12-months post-recruitment.

Evaluation Methodology

Design:

Recruitment: The sample came from families and youths registered at an interdisciplinary academic medical center specializing in the diagnosis and treatment of youths with an autism spectrum disorder. Eligible youths had a confirmed autism spectrum disorder, had recently evidenced severe disruptive behavior, and were 10 to 17 years of age. Of 42 families screened for eligibility, 17 were actually eligible and consented to participate and two withdrew during the assessment phase (due to identified factors that affected their eligibility).

Assignment: The 15 participating families were randomized (with a 1:1 ratio using a computerized random number generator) to the MST (n = 8) or usual community services (n = 7) conditions. The control group received a variety of services, but all focused on the individual youth rather than on the systems in which the youth was embedded.

Assessments/Attrition: Assessments occurred at baseline, six months, and 12 months after recruitment. The completion rates were 87% (n = 13) at six months and 73% (n = 11) at 12 months. The 12-month assessment referred to the period after recruitment rather than after program completion and therefore did not define a long-term follow-up.

Sample:

The sample youths had a mean age of 13.8 years. They were 87.5% boys, 75% white, 6.25% African American, 12.5% Asian American, and 6.25% Hispanic/Latino. About 62.5% lived with two caregivers (e.g., biological parents, stepparents).

Measures:

The outcomes, all from caregiver reports, included measures of youth conduct problems, internalizing, family relations, and peer relations. Reliabilities for the measures were high but came from reports of other studies rather than from the sample.

Analysis:

The analysis used analyses of covariance models with a time-by-group term to test for condition differences in changes from baseline to posttest and follow-up. Baseline outcomes were included as part of the time variable and sometimes as covariates as well. Supplemental analyses replaced missing data by carrying the last observation forward.

Intent-to-Treat: The four families that did not complete all three assessments were excluded from primary analyses, but supplemental analyses estimated program effects for the full sample by imputing missing data.

Outcomes

Implementation Fidelity:

Not examined.

Baseline Equivalence:

Tests showed no differences in demographic characteristics. However, caregivers in the MST condition reported higher levels of caregiver stress, lower levels of family adaptability, and more youth conduct problems than did caregivers in the control condition.

Differential Attrition:

Retention was greater in the MST group than the control group at both follow-ups: 100% versus 71.4% at six months and 87.5% versus 57.1% at 12 months. However, the authors reported that "no between-condition differences on demographic or psychosocial measures were found."

Posttest:

One significant effect emerged in 13 tests: The MST group scored better than the control group on family adaptability. Several marginally significant effects also favored the MST group: youth emotional bonding with peers, intensity of youth aggression, and family cohesion. As reported by the authors, the results remained unchanged when imputing missing data and using the full sample.

Long-Term:

Not examined.