GenerationPMTO

Study 1

Summary

Forgatch & DeGarmo (1999) and related studies utilized an experimental longitudinal design, which included random assignment into the PMT group or a non-intervention control group. The participants were 238 recently separated mothers and their sons in grades 1-3. Mothers in the experimental group (n = 153) received the PMT intervention and mothers in the control group (n = 85) received no intervention. Participants received extensive multiple-method assessments at baseline, 6 months, 12 months, 18 months, 30 months (DeGarmo et al., 2004; Forgatch & DeGarmo, 2007; Martinez & Forgatch, 2001), and 6, 7, 8, & 9 years (Forgatch et al., 2009; Patterson et al., 2010).

In a nine-year study of divorced parents, the PMTO group compared to the control group experienced significant:

• Reductions in coercive parenting and negative reinforcement at posttest (Forgatch & DeGarmo, 1999)
• Increases in positive parenting, effective parenting practices, and adaptive functioning at posttest
• Decreases in boys' noncompliance at posttest
• Reduced maternal depression and child internalizing and externalizing at 30-month follow-up (DeGarmo et al., 2004; Martinez & Forgatch, 2001)
• Reductions in poverty and greater rise out of poverty at 30-month follow-up (Forgatch & DeGarmo, 2007)
• Lower levels and lower growth in teacher-rated delinquency at nine-year follow-up (Forgatch et al., 2009)
• Reductions in average levels (but not growth) of deviant peer association
• Lower rates of arrest and delayed age at first arrest at nine-year follow-up (Forgatch et al., 2009)
• Lower rates of police arrests among mothers at nine-year follow-up (Patterson et al., 2010)
• Increases in socioeconomic status levels among mothers at nine-year follow-up (Patterson et al., 2010, Forgatch & DeGarmo, 2007).

Evaluation Methodology

Design: The study used an experimental longitudinal design, which included random assignment into the PMT group or a non-intervention control group. The participants were 238 recently separated mothers and their sons in grades 1-3. The participants were all residing in a medium-sized city in the Pacific Northwest; they were recruited through media advertisement, flyers and divorce court records. Mothers in the experimental group (n = 153) were randomly assigned to receive the PMTO intervention and the mothers in the control group (n = 85) received no intervention. The unequal assignment to group conditions was done to provide sufficient sample size within the experimental group to examine potential full implementation effects of the intervention. Families received extensive multiple-method, multiple-setting and multiple-agent assessment at baseline, 6 months and 12 months. The first four parent groups participated in a 16-session program; the program was then condensed into 14 sessions. Of the 13 parent groups, 31% were exposed to the 16-session program and 69% to the 14-session program. By 12 months, 28 families of the 153 assigned to the experimental condition did not participate, for an attrition rate of 18%. In the control condition, 15 of the 85 families did not participate (18%). Differential attrition analyses revealed no differences in the rate of attrition between the experimental and control conditions. There was a main effect discovered for SES and negative reinforcement. Those mothers who did not participate in the study at 12 months scored higher on baseline negative reinforcement than did those who continued participation and they were of lower SES. There were no differences on any of the outcome variables by condition. The analyses of pretest group differences revealed one difference between experimental and control mothers. Mothers in the experimental group had higher levels of negative reciprocity at baseline than did mothers in the control group.

Sample: The sample was restricted to boys because research suggests boys are more likely than girls to exhibit adverse effects of divorce as preadolescents. At baseline, mothers had been separated for an average of 9.2 months. Mothers' mean age was 34.8 years and sample boys' mean age was 7.8 years. The ethnic composition of the boys in the sample was 86% White, 1% African American, 2% Latino, 2% Native American, and 9% from "other" racial/ethnic groups including those who were identified as belonging to more than one group. The mean annual family income was $14,900 and 76% of the families were receiving public assistance. The majority of mothers (76%) had some academic or vocational training beyond high school, although only 17% had completed a 4-year college degree or higher. Approximately 20% of the women completed their education with high school graduation; 4% had not completed high school. Most mothers were classified within the lower and working-class ranges in terms of occupation: 32% unskilled, 21% semiskilled, 23% clerical/skilled, 22% minor professional to medium business, and 3% major business/major professional.

Measures: The measures utilized in this evaluation included: parenting practices measures (negative reinforcement, negative reciprocity, positive involvement, skills encouragement, monitoring, and problem-solving outcomes); child-rated adjustment variables (depressed mood, peer adjustment, deviant peer association); mother-rated child adjustment variables (anxiety, depressed mood and externalizing) and teacher report (externalizing, delinquency, prosocial behavior and adaptive functioning); police arrests for mothers/boys; maternal depression; and maternal standard of living.

One study (Reed et al., 2013) created three constructs: 1) poor discipline from separate measures of negative reinforcement, negative reciprocity, and bad discipline; 2) positive parenting from separate measures of skill encouragement, problem solving, and positive involvement; and 3) monitoring.

Analysis: Intervention effects in most studies were tested using a repeated measure ANOVA or using path analyses (to test for predictors of child adjustment), structural equation modeling (SEM), and latent growth curve modeling (LGM). One study of parenting (Reed et al., 2013) used linear growth models to compare changes in outcomes from baseline to the 30-month assessment and used cross-lagged structural equation models to examine the bi-directional relationships over time between measures of parenting.

Outcomes

Posttest:

Parenting Practices: The intervention group experienced reductions in observed coercive parenting, reductions in negative reinforcement, reductions in negative reciprocity, prevented decay (evidenced in higher scores of parental positive involvement) in positive parenting and generally improved effective parenting practices in comparison to mothers in the control group. Improved parenting practices were correlated significantly with improvements in teacher-reported school adjustment, child-reported maladjustment and mother-reported maladjustment. There were some effects in the opposite direction: Reductions were found for positive parenting practices and a sample-wide reduction was discovered for positive parent involvement and skill encouragement (however, follow-up analyses described below showed greater improvement in parenting practices in the intervention group). There were no significant differences between groups for skill involvement, parent problem solving or child problem solving.

Teacher-Rated Child Adjustment: The analyses revealed no significant differences between groups for externalizing and prosocial behavioral measures. There was a marginally significant effect found for adaptive functioning, indicating that the experimental group was rated slightly higher (better functioning) than controls on this measure.

Child- and Mother-Rated Child Adjustment: The analyses revealed no differences between the experimental and control groups on any of the child (peer adjustment and depressed mood) and mother (anxiety, depressed mood and externalizing) rated adjustment measures. The intervention did not produce direct effects on child outcomes, in mother or child-rated domains of child adjustment. There was a significant indirect effect on child rated adjustment through improved parenting practices.

Mediational Analyses: (DeGarmo, Patterson & Forgatch, 2004)

In this study with a 30-month mediational analysis, the same 238 families in the experimental longitudinal design received extensive multiple-method, -setting, and -agent assessment five times: baseline, 6 months, 12 months, 18 months, and 30 months. Effect sizes indicated that parenting changed first within 12 months, followed by changes in boy behaviors and finally changes in maternal depression within 30 months. Follow-up findings indicated that intervention effects on reductions in maternal depression were mediated by reductions in boy externalizing; growth reduction in externalizing behavior was mediated by growth reduction in boys' internalizing behaviors. PMTO effects on internalizing were direct and indirect, partially mediated by parenting practices. The findings were consistent with prior findings testing the coercion model, which indicated that effective parenting practices predicted reductions in child behavior problems. Child externalizing served as a mediator for intervention effects on maternal depression.

2-Year Follow-up: (Martinez & Forgatch, 2001; Reed et al., 2013)

In this study with a 30-month mediational analysis, the same 238 families in the experimental longitudinal design received extensive multiple-method, -setting, and -agent assessment five times: baseline, 6 months, 12 months, 18 months, and 30 months. The intervention produced enduring benefits to coercive discipline (effect was significant at 12 months and weakened at 30 months), positive parenting (significant at 12 and 30 months) and boys' noncompliance (significant at 30 months).

In Martinez and Forgatch (2001), the results indicated that the intervention protected the experimental group from the increases in noncompliance and coercive discipline and decreases in positive parenting that were experienced by the control group. However, the intervention did not produce improvements in noncompliance, coercive discipline or positive parenting. Mothers and sons in the experimental group maintained stable outcome trajectories, whereas those in the control group deteriorated. The intervention's impact on boys' noncompliance was mediated independently by its impact on coercive discipline and positive parenting. Change in positive parenting was more strongly associated with change in noncompliance than was change in coercive discipline.

In Reed et al. (2013), the intervention increased positive parenting and prevented deterioration in discipline and monitoring over the 30-month period. Over time, the improvements reinforced one another: Improved positive parenting supported better subsequent monitoring, and improved positive parenting and monitoring supported subsequent effective discipline.

9-Year Follow-up: (Forgatch et al., 2009; Patterson et al., 2010)

In Forgatch et al. (2009), the intervention significantly reduced the 9-year average of teacher-reported delinquency and the 9-year rate of growth in teacher-reported delinquency. The direct effects on delinquency were tested in mediational models with longitudinal sequencing of the predictors. Assignment to the experimental group was associated with improvements in parenting from baseline to 12 months, which in turn was associated with reductions in average levels of delinquency and individual variance in growth rates in delinquency. Assignment to the experimental group was also associated with reductions in average levels of deviant peer association from baseline to 8 years, but not growth rates in deviant peer association. Both average levels and growth of deviant peer association over 8 years were in turn significantly associated with average levels of delinquency.

For arrests, the study found more mixed support for the mediation hypotheses. The program significantly reduced the average number of arrests over time but not the growth rate in frequency of arrests. In addition, the intervention indirectly influenced average arrest frequency (but not growth rate in arrests) through average deviant peers.

Survival analyses showed a 37% reduced risk of earlier timing of first arrest for the intervention group in comparison to the control group. Change in effective parenting from baseline to 12 months reduced the size of the intervention effect, thus demonstrating the mediating influence of parenting.

In Patterson et al. (2010), mothers experienced benefits as measured by standard of living (i.e., income, occupation, education, and financial stress) and frequency of police arrest over nine years. In terms of direct effects, linear growth models showed a higher average effect of the intervention than the control group, but the linear and quadratic change over time was not significantly different for the two groups. In terms of mediated effects, structural equation models indicated that the program reduced coercive parenting over one year; reduction in coercive parenting in 12 months mediated the intervention's effect on growth in positive parenting over 2.5 years, which then mediated the direct effect of the intervention on the 9-year average standard of living. This long chain of relationships produced an indirect effect of .04. The structural equation model for mother total arrest probability indicated a similar indirect effect.

Study 2

Summary

Bank et al. (1991) used a randomized trial of Parent Training (PT), which included 60 boys referred by the Lane County (Oregon) Juvenile Court in the 1970s. Participants were assigned to either the Oregon Social Learning Center OSLC) PT Intervention (n = 28) or Community Control (n = 27) treatment conditions. Offense data were collected for the year prior to intake, the treatment year, and the three years following treatment.

Bank et al. (1991) found that, compared to the control group, the PMTO group had significantly

• Faster decreases in rates and prevalence of juvenile arrests than community controls.

Evaluation Methodology

Design: The referred youth met the following criteria: repeat offender (minimum of two recorded offenses, at least one of which was a nonstatus offense), less than 16 years old at intake (mean age of 14), and living within 20 miles of the Oregon Social Learning Center (OSLC). The participants' families were extremely distressed, most suffering external crises, marital discord, and parent depression or antisocial behavior.

Participants were randomly assigned to either the OSLC or Community Control (CC) treatment conditions. Five subjects who were incarcerated for 11 consecutive months or longer (after assignment but before treatment) were dropped from the original sample for a final total of 55. Treatment began six months after intake. The OSLC treatment condition (n = 28) was originally designed for youth up to age 12, and was somewhat modified for the older subjects. Therapy sessions were conducted with each family individually and incorporated manuals and videos. Parents were trained to identify prosocial and antisocial behaviors including class attendance, defiance, homework, hanging out with kids in trouble, curfew violations, and drug use. The CC treatment condition (n = 27) included intensive family therapy (weekly 90-minute sessions combining behavioral and family systems counseling), group therapy (2 hours per week, focused on drug use, which half the subjects attended), and monitoring of school attendance and performance by the family therapist or probation officer. Therapy lasted 5 months and averaged 50 hours of direct treatment plus optional follow-up sessions.

Measures: Delinquency was measured using offense records from the Lane County Juvenile Court and from other juvenile courts for those who moved during the follow-up years; if multiple offenses occurred at the same time, only the most serious was counted. Three offense types were tracked: total offenses, status offenses, and nonstatus offenses (felonies and misdemeanors). Offense data was collected for the year prior to intake (BL), the treatment year (TX), and the three years following treatment (FU1, FU2, FU3). Data on time spent in incarceration was collected for the BL, TX, FU1, and FU2 years.

Observation data from the home and brief daily telephone interviews (PDR) with the parents were taken for the OSLC treatment group only. The former was measured using the Family Interaction Coding System to capture positive, negative, and neutral behaviors of family members and focused primarily on boys' Total Aversive Behavior (TAB, e.g. negative commands, disapproval, destructiveness, negativism, humiliation, and whining) and parents' "Abusive Cluster" (AC, e.g. threats, humiliation, physical punishment and yelling). The PDR interviews recorded the number of child problem behaviors during the past 24 hours and were conducted during the week prior to treatment and in the week before termination.

Outcomes

A MANOVA analysis of groups by years revealed a significant difference in nonstatus offenses between the two groups during treatment time in the expected direction, but there were no significant differences in status or nonstatus offense rates at any other time. Overall offense rates for youth in both groups declined significantly after the onset of treatment, and reductions were produced significantly faster by the OSLC treatment. Prevalence rates showed a significant decrease over the 5 years for both groups and no significant differences between groups, except at FU3 for nonstatus offenses in the expected direction. OSLC youth spent significantly less time incarcerated during TX (28.5 days per year vs. 45.4) and FU1 (33.9 vs. 69.3 days), and less time during FU2 (p<.068, 34.4 vs. 37.7 days).

For OSLC youth in which family data were collected, there were no significant changes in TAB observations from intake to termination, but PDR data indicated that delinquent behaviors reported by parents were significantly reduced at termination (especially for stealing, in which no stealing incidents were reported).

Summary: Overall, the results show significant decreases in rates and prevalence of juvenile arrests for both groups, but the OSLC treatment produced quicker results that were at least as strong as those produced by CC treatment and were obtained with one-third less reliance on incarceration.

Study 3

Summary

Patterson et al. (1982) utilized a randomized design in which 46 families were referred to OSLC by pediatricians, school or mental health personnel, or parents. Nineteen families that were not excluded and did not drop from the study were randomly assigned to experimental treatment (n = 10) or a comparison group (n = 9). Observations were conducted during baseline (2-week duration) and when experimental families were terminated (after an average of 17 hours of therapy time).

Patterson et al. (1982) found that, compared to the control group, the PMTO group showed significant reductions in

• Deviant behavior among treatment children, compared to control group children (63% vs. 19%)
• Observations of aversive behaviors between groups, with 70% of treatment children testing within the normal range post-intervention, compared to 33% of control group children.

Evaluation Methodology

Design: The study was a randomized design in which 46 families were referred to the Oregon Social Learning Center (OSLC) by pediatricians, school or mental health personnel, or parents. Participants were included in the study if they met the following criteria: age (3-12 years); residing within a 25-mile radius of the Center (located in Eugene, OR); the primary referral problem was aggression (children had to rate above the 90th percentile on the Total Aversive Behavior scale); parents agreed to home visitations, telephone interviews, and attendance at therapy sessions; and neither parents nor children had previously been diagnosed as psychotic, severely retarded, or autistic.

Of the 46 family referred to the OSLC, 10 dropped out and 17 of the referred subjects had low rates of observed aggression, leaving 19 families who were randomly assigned to experimental treatment (n=10) or a comparison group (n=9). Analyses of pretest group equivalence revealed no differences between experimental and control groups at baseline. There is no report of differential attrition analyses.

The families assigned to the control group received mixed treatment including: an eclectic approach (3); behavior modification training (2); an Adlerian approach (1); structural family systems approach (1), and a combination of relaxation and physical exercise (1). One family, although referred for treatment, received none. The average treatment time was 11.75 sessions. Observations for both groups were conducted during baseline (2 week duration) and at termination (after an average of 17 hours of therapy time, experimental families were terminated) of treatment.

Sample Characteristics: The children were described as having a mean age of 6.80 years (experimental group) and 6.78 years (control group). Gender was described as 60% male (experimental) and 77% male (control). The mean unemployment for heads of household was 11% (experimental) and 25% (control). The mean monthly income was $931 (experimental) and $570 (control). The father was absent approximately 52.5% of the time in both groups and the percentage of parents reporting school problems at intake was 80% (experimental) and 55% (control).

Measures: The Family Interaction Coding System (FICS), completed by staff observing family interactions at baseline and termination; the Total Aversive Behavior (TAB), which included a summed score of 14 noxious behaviors; the Parent Daily Report (PDR), completed by parents and assessing the frequency of 34 problem behaviors during the prior 24-hour period, and administered by telephone five times weekly for 2 weeks during baseline and three times weekly for 2 weeks at termination.

Analysis: A two-way repeated measures ANOVA was conducted on outcome measures (TAB scores). TAB scores were based on observations in the home during baseline and termination; the rates for the 14 deviant behavior codes were summed to form the composite TAB score. Frequency analyses were utilized for the parent report data.

Posttest: An evaluation conducted at termination demonstrated a 63% reduction in the intervention children's mean rate of deviant behavior (from .92 to .32), compared to a 17% reduction (from .89 to .74) for control children. An analysis of TAB scores revealed that 70% of the experimental children tested within the normal range at termination (although none had done so at baseline), compared to 33% of the control subjects.

Analysis of PDR data assessing problem behaviors showed significant decreases from baseline to termination for both groups: the experimental mean decreased from 3.19 per day to 1.66 per day, compared to 3.20 and 1.96, respectively, for the control group. When parents rated their satisfaction with treatment at termination, 90% of the experimental group's parents rated the treatment as "very effective," compared to 25% of the parents of control children.

Study 4

Summary

Ogden & Hagen (2008) and Hagen et al. (2011) utilized a randomized design in which 59 Norwegian families were assigned to the PMTO group and 53 families were assigned to a regular services comparison group. Data were collected at intake (baseline) and at post-treatment (approximately 11 to 12 months later), and one-year follow-up.

Ogden & Hagen (2008) and Hagen et al. (2011) found that, compared to the control group, the PMTO group showed significant improvements in

• Parent-rated externalizing behavior problems and total problems
• Teacher-rated social competence
• Parent use of observer-rated effective discipline
• Total aversive behavior in two-parent families.

Evaluation Methodology

Design: Participants were 112 children and their parents recruited to the study via existing child services agencies. The recruitment period lasted from January 2001 to April 2005. After completion of the intake battery, families were randomly assigned to either the PMTO group or the regular services (RS) comparison group. The randomization was pairwise and was carried out locally by therapists with the use of sealed envelopes. The randomization procedure resulted in 59 families assigned to PMTO and 53 families assigned to the RS comparison group. Twelve families who were in the pipeline at the time therapists were instructed to stop the recruitment process were included in the study. This resulted in a slightly unequal sample size in the two treatment conditions as therapists were told to go through with the pending family, but not to wait for the second family of the pair. As such, the last 12 families recruited to the study were randomly assigned, but not in a pairwise fashion.

Families eligible for participation in the project had contacted the agencies because of child conduct problems, including any behavior listed in the DSM-IV as a symptom of oppositional defiant disorder, conduct disorder, or a problem description consistent or synonymous with these symptoms, such as aggression, delinquency, or disruptive classroom behavior. Children were deemed ineligible for participation if they were autistic, had severe mental retardation, had documented sexual abuse, or had custodial parents with severe mental retardation or psychopathology. No child was actually excluded from the study on the basis of these restrictions, however. Families were recruited using the regular referral procedures of the existing service agencies. The standard procedure for acceptance was based on clinical judgments of the therapists rather than formal screening, and the same procedure was used to include children and families in all agencies in the study. Written informed consent was obtained from parents prior to inclusion in the study.

Post-treatment assessments were conducted between October 2001 and May 2006 (approximately 11 to 12 months after the intake assessment battery was administered) and follow-up assessments were conducted between January 2003 and July 2007 (approximately a year after posttest). Of the 112 children and families who completed the intake assessment battery, 97 (87%) participated in the post-treatment assessment and 75 (67%) participated in the one-year follow-up. There were no significant differences on any of the main outcome variables or on characteristics such as age, gender, recruitment site, or parent demographics between families who completed the post-treatment assessment and those who were lost. Children in the PMTO group were on average significantly older than were children in the RS group, and PMTO children were also significantly more likely to have an older sibling. There were significantly more single-parent households in the RS group than the PMTO group. There were no other significant baseline differences between treatment conditions on main outcome variables, demographics, or other child characteristics.

Manuals from two Oregon Social Learning Center preventive interventions were used to train the PMTO therapists: Parenting Through Change and Marriage and Parenting in Stepfamilies. These manuals were translated by the Norwegian Implementation Team. PMTO candidates underwent 18 months of training and had to complete 3 to 5 full-scale PMTO therapy cases during their training periods. PMTO candidates also participated in regular booster sessions and regular meetings with their supervisors in which they discussed their performance, treatment challenges, and clinical outcomes. A total of 33 PMTO therapists from children's service agencies representing all regions of Norway contributed to the study.

For cases with partial data on a given measure, expectation maximization procedure was carried out to predict the overall score on that variable. A missing-completely-at-random test was conducted for each expectation maximization. If this failed, or if the solution did not converge, regression was used to predict single values that would be used in the computation of the total score.

Sample Characteristics: The sample was 80.4% male and 19.6% female with ages ranging from 4 to 12 years. Children in the sample demonstrated serious behavioral problems according to their T -scores on the Child Behavior Checklist (CBCL) and the Teacher Report Form (TRF) at intake. The mean family income was the equivalent of approximately $57,380, which represents middle to lower income level in Norway. The mean age of the primary caregiver was 39.07 years, and 43.3% of the families were married or cohabiting with the other biological parent of the child, 40.4% were single parents, and 16.3% were married or cohabiting with another adult. Information was missing for 8 families on civil status. Information about ethnic background was provided by 67 respondents, and of these, 94% were Norwegian and 6% were from other western European countries.

Measures: The instruments administered in this study included the Child Behavior Checklist (CBCL), the Teacher Report Form (TRF), the Social Skills Rating System (SSRS), the Family Satisfaction Survey, the Parent Daily Report (PDR), the Fidelity of Implementation (FIMP) system, the Coders Impression (CI), the Structural Interaction Tasks (SITs), and its coding manual, the Family and Peer Process Code (FPPC).

Observed assessment of child-initiated negative interactions (chains): Parents and children participated in a series of videotaped structured interaction tasks (SITs) for the purpose of assessing parenting practices and child behavior. The SITs lasted 30 minutes for children older than age 8 and 25 minutes for children younger than age 8. Both age groups included (a) a problem-solving task in which the family was asked to resolve an identified source of conflict in the family and (b) an evaluation task in which the family discussed how well they had cooperated during the lab procedure. Families with children younger than 8 years also participated in a free-play and clean-up task and a waiting task. Families with children older than 8 years spent more time in the problem-solving task; they were also asked to plan a fun family activity for the upcoming week. Parent-child interactions were coded using the FPPC, which numerically records interpersonal exchanges in real time using duration and sequence. The code scores the initiator, the recipient, and the verbal and behavioral content, and valence. Negative child-initiated engagement behavior had to be reciprocated with negative behavior by the parent within a 6-second window for it to qualify as a chain. The chain was terminated once a non-negative engagement episode occurred. At Wave 1, the PMTO group had 6.8 SITs missing, while the RS group had 1.9% missing; at Wave 2, the PMTO group had 6.8% missing, and the RS group had 20.8% missing. The follow-up study focused on a composite score derived from the FPPC focusing on total aversive behavior (TAB) among target children and parents. Separate TAB scores were calculated for two-parent and single-parent families.

The CI measure was a questionnaire completed by coders immediately after scoring the SIT videotapes. Each of the five parenting dimensions (discipline, problem-solving, monitoring, positive involvement, and skill encouragement) was scored with subscales, as was the level of child compliance. Skill encouragement was found to be psychometrically unreliable for this sample, however, and it was thus excluded from further analysis.

The Externalizing Problems and Internalizing Problems scales, along with the Total Problems scale of the CBCL and the TRF were used in this evaluation. Items were rated on a 3-point Likert scale. The Total Problems scale included all behavioral items on the CBCL/TRF and covered externalizing and internalizing problems, thought problems, attention difficulties, and social problems.

The SSRS is a standardized, multirater, and multifactorial instrument that assesses social skills and academic competence in children and youths. The original response choices were modified from a 3-point to a 4-point Likert scale. The parent version has 38 items assessing cooperation, assertion, responsibility, and self-control domains. The teacher version has 30 items measuring cooperation, self-control, and assertion. Academic competence was measured by teacher ratings on the nine-item scale of the SSRS, which assesses general academic performance, reading and mathematics, motivation, and cognitive functioning. Items were rated on a 5-point scale according to the teacher's rank order of the student (1 = lowest 10%, 2 = next lowest 20%, 3 = middle 40%, 4 = next highest 20%, and 5 = highest 10%). The academic competence scale was missing for 44% of the sample, but this was primarily due to the fact that the scale is only applicable to children age 6 and older attending school.

The PDR is an index of observable child behavior problems in the U.S. that was translated into 34 culturally appropriate and age sensitive items. Parents were asked to answer yes or no to whether particular behaviors on the part of the child had taken place within the past 24 hours. The index includes both serious transgressions and less extreme behaviors. The instrument was administered by phone to the primary caregiver on 3 consecutive days at baseline and again at post-treatment.

Parents completed the Family Satisfaction Survey, a 12-item questionnaire, at treatment termination. Caregivers were asked to rate treatment effectiveness, the quality of their interaction with the therapists, and whether they would recommend the treatment to others on a 4-point scale.

Competent adherence to the PMTO treatment protocol was assessed from evaluations of videotaped therapy sessions using the FIMP system. The FIMP system measures treatment fidelity and evaluates the PMTO therapist on the following 5 dimensions: PMTO knowledge, structuring, teaching practices, process skills and overall quality. Raters use a 9-point scale on which the therapeutic skills central to PMTO treatments are evaluated. Ten-minute segments of four different sessions were coded (introduction to and troubleshooting sessions on both discipline and encouragement). Scores were averaged across the 4 sessions.

Analysis: A series of analyses of covariance were used to test treatment effects in the intention-to-treat design. Dosage, (combining parents' and children's hours in treatment) pre-treatment scores, age level, and gender were entered as covariates. The three main externalizing behavior measures, the CBCL externalizing scale, the TRF externalizing scale, and the PDR were analyzed separately in the analysis of covariance models using T scores and raw scores. A path model using maximum likelihood estimation, testing both specific indirect and individual effects with bootstrapped standard errors was also tested. In the model, treatment condition predicted effective discipline, and effective discipline predicted three key child outcomes: child compliance, child-initiated negative chain, and an externalizing composite variable. The composite variable included z scores of the externalizing subscales of the CBCL and TRF and the summed 3-day PDR score. The standardization was carried out separately for each assessment point. Pre-treatment score for all child outcomes were entered as control variables.

Treatment effects on outcome variables using the intent-to-treat design were analyzed using regressions with bootstrapped standard errors. Baseline scores on outcome measures, age, gender, dosage and time between posttest and follow-up were used as controls. A completer's analysis was also conducted but not reported in the write-up.

Outcomes

Posttest:

Child adjustment: Children in the PMTO group scored significantly lower on parent-rated externalizing behavior problems at the end of treatment than did the children in the RS group. PMTO children also received significantly lower scores on the CBCL total problem scale than did RS children. Scores on the PDR across 3 days indicated that significantly fewer problems were reported for children younger than 8 in the PMTO group compared with their RS group counterparts. There was no significant difference between the two treatment conditions for children older than 8 on the PDR across three days.

There was no main effect of treatment on the TRF externalizing problem scale at the post-treatment assessment, but a significant Treatment X Age interaction emerged. Younger children in the PMTO group scored significantly lower on the TRF externalizing scale than did younger children in the RS group. There was no significant simple effect for the older children. Younger children in the PMTO condition also scored significantly lower on the TRF total problem scale than did their RS counterparts. There was no significant group difference for the older children.

Children of families assigned to the PMTO group were rated as significantly more socially competent by their teachers at the end of treatment than were children of families assigned to the RS group. There was no main effect of treatment on the parent-reported social competence measure, but a Treatment X Gender interaction effect emerged. When pursuing this interaction effect, however, it was found that neither of the simple main effects was significant, although the difference in SSRS scores appeared to be greatest between the PMTO and RS girls. No age effect was found for this measure.

Finally, no direct treatment effect was found for the variables of observed child-initiated negative chain, internalizing symptoms, or academic competence.

Parenting skills: Parents who received PMTO scored significantly better than the RS group on one of four observed parenting outcomes (effective discipline) at the end of treatment. There was a simple main effect of treatment on parental monitoring in which PMTO parents with younger children scored significantly higher than did the parents of younger children in the RS group. There was no significant difference between PMTO and RS parents of older children on this measure. RS parents of older children scored higher on problem-solving than did parents of older children in the PMTO group, a negative effect. There was no significant difference between groups on the parenting dimension of positive involvement.

Family satisfaction survey: A one-way ANOVA showed that parents in the PMTO group were significantly more satisfied with the treatment they received than were parents in the RS group. In families in which a second parent informant was present, the second parent informants in the PMTO group were more satisfied with their treatment than were their RS group counterparts. There was no significant difference in therapists' scores between the two conditions on their reported satisfaction with treatment.

Treatment attendance (dosage): PMTO parents received more hours of treatment than did RS parents. RS children, however, received more hours of treatment than did PMTO children. Thus, dosage was defined as the combined parent and child hours of treatment. Reports on treatment attendance were available for 89 families.

Test of indirect effects model: A path model was tested in order to investigate whether the effects of group assignment on measures of child functioning were indirectly affected by improved parenting skills. The model estimated individual and specific indirect effects with bootstrapped standard errors. Discipline was chosen as the parenting dimension to be entered into the model as this is considered the most important parenting skill in reducing externalizing behavior and non-compliance in children with conduct problems. The child externalizing composite, frequency of child-initiated negative chain during the SIT, and child compliance were entered as the dependent variable in the model. These were three key child outcomes, and they represented three different assessment methods. Age level, SIT condition, and pre-treatment scores on parental discipline and on the child variables were entered as control variables. The model showed excellent fit. Treatment condition was found to be significantly associated with effective discipline in favor of parents in the PMTO group. Moreover, better parental disciplinary skills predicted greater child compliance, fewer child-initiated negative chains, and lower child externalizing scores. The specific indirect effects of all predictors on the three child outcomes, via parental discipline, with bootstrapped standard errors were also tested. A significant indirect effect emerged for treatment condition on child compliance, via parental discipline.

Long-term:

Only one of the 21 measures examined was significantly different between groups using data at posttest and one-year follow-up. The composite measure of total aversive behavior (TAB) for two-parent families was significantly lower (p<.01) for families in the PMTO group.

Differential attrition: No significant main effect of retention was found (i.e., none of baseline measures differed significantly between completers and attriters). However, there was an interaction between condition and retention. Families with children with higher delinquency scores and lower teacher-rated social skills at baseline were more likely to be retained in the control group than the PMTO group.

Mediation analysis: Effective discipline and family cohesion were examined as mediators of child behavior. The program significantly improved effective discipline at posttest, which in turn significantly predicted less child aggression, opposition and aversive behavior and greater parent-reported social skills at one-year follow-up. The program significantly improved family cohesion at posttest, which significantly predicted less child delinquency and externalizing behavior and greater teacher-reported social skills at one-year follow-up.

Study 5

Summary

Forgatch et al. (2005) and related studies (Bullard et al., 2010; DeGarmo & Forgatch, 2007; Wachlarowicz et al., 2012) examined the effects of the PMTO program to prevent conduct problems in children in 110 families whose mother was recently married using randomized controlled assignment. Added components to the program included material addressing stepfamily issues and measures were collected at 6-, 12-, and 24-months post-baseline.

Forgatch et al. (2005), DeGarmo & Forgatch (2007), Bullard et al. (2010), and Wachlarowicz et al. (2012) found that, compared to the control group, the stepparent families receiving PTMO showed a significant

• Increase in positive parenting and step-fathering practices
• Decline in child behavior problems and depression
• Improvement in marital interaction and mothers' marital satisfaction (but not fathers')
• Decline in coercive parenting.

Evaluation Methodology

Design: Mothers and stepfathers had to be married within the previous two years, cohabitating, and have a 5- to 10-year-old biological child of the mother. The focal child had to reside in the stepfamily at least 50% of the time and had to display five or more mother-reported conduct problems. Of the 121 respondents to the advertisement, 113 met study criteria, and three of these families agreed to participate in a planned intensive single subject design and were not used in this study, leaving 110 families agreeing to participate in the study.

Of the 110 participating families, 67 (61%) were randomly assigned to PTMO. Although more subjects were assigned to the intervention group to increase the power of the tests, the assignment was random. Intervention families received the standard, manualized version of the PMTO program as well as a stepparent component that added material addressing stepfamily issues (e.g., presenting a united parenting front and the role of stepparents). The remaining 43 families were assigned to the non-intervention control condition. These families received no PMTO but received a resource guide of family services available in the community.

Effects of the intervention were assessed via measurements taken at baseline, and at 6-, 12- and 24-months after baseline. With the program lasting approximately 27 weeks, the 24-month assessment occurs more than one year after the end of the program. Attrition occurred across the assessments. About 91% of baseline subjects completed the 6-month assessment, 91% completed the 12-month assessment, and 82% completed the 24-month assessment.

Sample: Couples had been married an average of 15.58 months. Most mothers and stepfathers had some education beyond high school. The average gross annual income was $39,432, and the average age for mothers and stepfathers was 31.3 and 32.7, respectively.

Measures: Measures of parenting practices, child externalizing problems, marital relationship processes, and marital satisfaction were collected at baseline (prior to intervention), and at 6, 12, and 24-months after baseline.

Measures of parenting practices were assessed using data derived from videotaped parent-child interactions and coded using the Family and Peer Process code (FPP). Observers rating the child-parent interactions were blind to condition, and blind reliability checks on tapes of 15% of the sessions showed acceptable levels of agreement. Scales developed from the observations included positive parenting (defined by skill encouragement, positive involvement, problem solving and monitoring) and coercive parenting (defined as frequency of negative reinforcement, negative reciprocity and inept discipline). A special construct for stepfathers combined measures of positive involvement, problem solving, monitoring and supervision, coercive discipline, negative reciprocity, negative engagement, and negative reinforcement.

Measures of child externalizing behavior problems were assessed using mother and stepfather-reported scores of the Child Behavior Checklist as well as using teacher reports. Both child aggression and delinquent behavior measures were used to assess overall child externalizing behavior. A measure of child depression used symptoms reported by the child on the Child Depression Inventory.

Measures of marital relationship process were derived from observers' ratings of couples' problem solving in regards to parenting. This scale included measures of quality and outcome of problem solving as well as overall relationship quality.

Measures of marital satisfaction were derived from the Dyadic Adjustment Scale (DAS) as reported by mothers and stepfathers. The DAS scale is a 32-item scale that reflects satisfaction, cohesion, consensus and affection.

Analysis: Intent-to-treat analyses included data from all families assigned to PMTO regardless of whether they attended the PMTO sessions. Hypotheses were tested using structural equation modeling with the full information maximum likelihood (FIML) estimation method that uses data on all available assessments of subjects and, under the assumption of random missing data, produces optimally efficient estimates of standard errors.

In Forgatch et al. (2005), structural equation models showed the effect of the intervention on latent factors for the outcomes with controls for the latent factors at baseline.

In DeGarmo & Forgatch (2007), regressions were used to test for program effects on the change in outcomes at 12 months and 24 months. The change over time implicitly controlled for baseline outcome values.

In Bullard et al. (2010), tests of the program effects came from comparing the average change over time (i.e., mean slope) for the intervention group to the treatment group. The change over time implicitly controlled for baseline outcome values.

In Wachlarowicz (2012), intent to treat growth models tested the degree to which parent antisocial characteristics modered the effects of PMTO.

Outcomes:

Implementation Fidelity: The families in the treatment condition had the opportunity to attend PMTO sessions over an approximate 27-week period. Of the 67 families in the treatment condition, 11 attended no sessions, 7 attended 1 to 5 sessions, 10 attended 6 to 10 sessions, 22 attended 11 to 15 sessions, and 17 attended more than 15 sessions.

Using an observational measure of the therapy sessions called the Fidelity of Implementation Code (FIMP), Forgatch et al. (2005) evaluated adherence to the training. The measure predicted changes in observed parenting outcomes, suggesting its validity as a measure of adherence, but no other information was presented.

Baseline Equivalence: Baseline equivalence was not discussed in any of the studies, although results for group differences in the mean intercepts in Bullard et al. (2010) suggest that one outcome differed at baseline: Marital satisfaction reported by the mother was significantly higher at baseline for the intervention group.

Differential Attrition: The articles reported that data were missing completely at random, and two of the three articles used full information maximum likelihood (FIML) estimation that adjusts estimates for missing data. The article not using FIML, DeGarmo & Forgatch (2007), said that attrition did not differ significantly across conditions and that there were no significant differences on predictors or outcomes when comparing families retained and families not assessed at follow-up.

Posttest: Forgatch et al. (2005) found that the intervention significantly improved couple parenting (i.e., a latent factor measuring positive and non-coercive parenting of stepfather and stepmother) at posttest with controls for the baseline latent factor. The eta-squared measure of the intervention of .14 indicated a medium-to-large effect size. The intervention did not directly influence measures of child home and school problems, but a mediation model showed indirect effects. The intervention improved couple parenting at one year, which in turn reduced noncompliance and home problems of children at one year.

DeGarmo & Forgatch (2007) focused on stepfathering rather than couple parenting. The intervention significantly improved a composite measure of stepfather parenting but on average did not improve child depression or noncompliance to mother at posttest (12 months). However, the program showed benefits for child depression and child noncompliance among stepfathers who showed improved parenting skills at 6 months.

Bullard et al. (2010) showed significant differences across the intervention and control groups in the improvement over time for 4 of 8 outcomes: coercive parenting, marital relationship processes, mother's marital satisfaction, and teacher rating of child externalizing problems. Changes in parenting practices, stepfather's marital satisfaction, mother's rating of child externalizing problems, and father's rating of child externalizing problems did not differ across groups. In addition, the program effect on child externalizing behavior was mediated by parenting practices. Improved parenting practices were also related to improved marital relationships and marital satisfaction.

Wachlarowicz et al (2012) found that PMTO was reliably related to growth in positive parenting and to decreases in coercive parenting. Parent antisocial characteristics moderated the effect of PMTO on coercive but not on positive parenting practices. PMTO resulted in greater reductions in coercive parenting as parent antisocial histories were more extensive, and this moderator effect was found for both mothers and stepfathers.

Long-Term: Although the studies generally did not separate the posttest results (one year after baseline) from follow-up results (two years after baseline), some evidence suggests that the intervention effects fade. Forgatch et al. (2005) found a significant indirect effect of the program on home problems at posttest but not on school problems at follow-up. DeGarmo & Forgatch (2007) showed significant improvement in stepparenting at posttest but not follow-up.

Study 6

Summary

Bjørknes et al. (2012) and Bjørknes & Manger (2012) used a randomized controlled trial to examine the effect of PMTO on parent practices as mediators of change on child conduct problems among 96 Pakistani and Somali immigrant families in Norway. Cultural adaptations (translations) were made to the program to accommodate the needs of the targeted population.

Bjørknes et al. (2012) and Bjørknes & Manger (2012) found that among immigrant mothers from Pakistan and Somalia living in Norway, the PMTO group compared to the control group showed significantly improved

• Positive parenting practices
• Child conduct problems.

Evaluation Methodology

Design: Study participants were 96 Somali and Pakistani immigrant families in Norway. These two nationalities were chosen because they constitute the two largest immigrant groups in Norway and because of a known need for services for these families. Participants were recruited through professional referrals (school, child care, or child welfare system), through community information meetings, or by staff from the study recruitment team in 2007-2008. Participation in the study was determined by families having a child with or at risk of developing conduct problems. Mother-reported scores on the Eyberg Child Behavior Inventory at baseline indicated 30% of children within the clinical range. A total of 118 mothers were assessed for eligibility, and 96 enrolled in the study. Mothers were randomly assigned to either a Parent Management Training Oregon Model (PMTO) treatment group (n=50) or a waitlist (WL) control group (n=46). Randomization was stratified by community and ethnicity.

Treatment mothers received the intervention in various community-care centers in Oslo, and attended an average of 10.75 sessions (out of 18). The sessions were group sessions, and were ethnically homogeneous, comprised of 8-12 mothers per group. Sessions were adapted to be culturally appropriate, and were taught by two Norwegian PMTO therapists in coordination with link workers, who were trained to work as bilingual PMTO assistants. All therapists were trained psychologists or social workers with postgraduate training in PMTO. Fidelity checklists were completed to monitor implementation adherence.

At posttest, 13 of the 96 mothers (13%) were lost to attrition.

Sample Characteristics: The majority of the families were from Pakistan (59%, compared to 41% from Somalia). All were Muslim, and had lived in Norway for an average of 11.69 years. The majority of the study children were male (63%) and the children had a mean age of 5.9 years. The mean age of the mothers in the study was 33.6 years. Among parents, 7% had a college or university degree, 38% had completed high school, 32% had completed only elementary school, and 23% had received no school education. The majority of mothers (77%) were married, and 20% of the families received public financial support (compared to 2.3% nationally). About 23% of the subjects were single mothers.

Measures: Measures were collected at baseline and post-intervention, largely in the form of self-reports. Assessments were completed by bilingual interviewers, and assessment tools were in Norwegian and Urdu/Somali. Outcome measures on child conduct problems included the Intensity Score on the Eyberg Child Behavior Inventory and the Parent Daily Report (PDR). The PDR was conducted by telephone on three consecutive days, and scores were summed to a final score. Both measures were completed by the mothers. Reliability on both measures at both time points was .93 and higher.

Mediating measures of maternal parenting practices (harsh discipline and positive parenting) were assessed using the Parent Practices Interview questionnaire, also completed by the mothers. Reliability scores pre/post were high (harsh discipline scale = .81-.84; positive parenting scale = .69-.74).

Teacher reports assessed child behavior to measure total problems and externalizing. Teachers also rated social skills of the children (cooperation, assertion, self-control). Alpha coefficients all exceeded .90.

Analysis: Bjørknes & Manger (2012) used an intent-to-treat analysis with baseline outcomes treated as covariates in analysis of covariance. The estimates adjusted for missing data by using the statistical method of expectation maximization or by using the last-observation-carried forward approach.

Bjørknes et al. (2012) also used an intent-to-treat analysis. Participants lost to follow-up were included in the analysis using the full-information maximum likelihood (FIML) approach. Change scores on outcome variables were calculated as post-assessment sum scores minus pre-assessment sum scores. Path analysis was used to determine the effect of the predictor variable (group assignment) on the outcome variable (changes in child conduct problems) by two mediators (changes in maternal harsh discipline and positive parenting). A multiple group comparison test was also conducted to determine potential differences in the mediational pathways in the two ethnic groups.

Outcomes

Baseline Equivalence: Bjørknes & Manger (2012) found no significant baseline differences on sociodemographic characteristics but found one significant difference on baseline outcomes: the parenting practice measure of harsh discipline for age. Bjørknes et al. (2012) reported no differences between the intervention and control groups at baseline on the three latent outcome scales of harsh discipline, positive parenting, and child conduct problems.

Differential Attrition: Bjørknes & Manger (2012) reported that there were no significant differences on baseline outcomes between attending and dropout mothers. They also reported that there were no significant differences between subjects with and without complete teacher data. Bjørknes et al. (2012) reported that missing data, including that due to respondent attrition, was missing completely at random. That implies no differential attrition. Further, FIML estimation included all subjects, including those with missing data.

Posttest: Bjørknes & Manger (2012) found that the intervention significantly improved 2 of 4 parenting practices and 2 of 5 child behaviors. For parenting practices, use of harsh discipline declined and positive parenting improved relative to the control group. For child behavior, the conduct problem composite and Eyberg Child Behavior Inventory improved relative to the control group. Effect sizes for the significant outcomes ranged from .27 to .54. Also, effect sizes proved larger for women who attended at least half the group sessions.

However, the benefits at home did not extend to school. The intervention did not significantly influence the teacher-reported total scale, externalizing behavior scales, or Social Skills Rating System total scale.

Bjørknes et al. (2012) found significant correlations between group assignment and the three outcome variables (child conduct problems, harsh discipline, and positive parenting), where intervention group mothers showed significantly increased positive parenting and reduced harsh discipline, and their children showed reduced conduct problems, compared to control group participants.

Results of the path analysis in Bjørknes et al. (2012) showed a significant direct effect of group assignment on change in child conduct problems, where intervention children had significantly reduced child conduct problems, compared to their waitlist control counterparts. The overall path model showed significant path coefficients from group assignment to changes in harsh discipline and positive parenting, where positive outcomes significantly predicted changes in child conduct problems. When comparing the overall model to a model in which the direct path from the group assignment to changes in the children's conduct problems was not constrained to zero, results were not significant, indicating that the direct path was not crucial and that the effect of group assignment on child outcomes was mediated by the two parenting measures. Additionally, both paths were tested separately and testing revealed significant indirect effects of both parenting measures on child outcomes, indicating that the intervention effects on child conduct problems were fully mediated by the changes in harsh discipline and positive parenting. There were no significant differences between ethnic groups.

Long-term: Not examined.

Study 7

Summary

Sigmarsdóttir et al. (2013, 2014) used a randomized controlled trial to examine 102 families in Iceland. Cultural adaptations (translations) were made to the program to accommodate the needs of the targeted population. Results for treatment families were compared to a group that received usual services for children with behavioral problems.

Sigmarsdóttir et al. (2013, 2014) found that, compared to the control group, the PMTO group in Iceland showed significantly fewer

• Child adjustment problems.

Evaluation Methodology

Design: Participants for this study were recruited from five municipalities throughout Iceland in 2007 and referred to the study by professionals in the community (schools, educational services, social services). Criteria for study participation included a child displaying behavioral problems at home and/or in school, having no history of sexual abuse, and not meeting diagnostic criteria for autism. All 102 families who were referred agreed to participate.

Families were randomly assigned to either the treatment (PMTO; n=51) or a Services as Usual (SAU; n=51) control condition. Subjects in the control condition received a variety of community services normally provided for children with behavioral problems, including diagnosis and/or counseling from a psychologist or school counselor, and services offered by social services or the health care system. In addition, 60% of the children attended schools where Positive Behavioral Support was being implemented (63% of the treatment and 59% of the SAU group).

The posttest occurred at the end of the treatment, approximately one year after baseline. A follow-up assessment is planned but not reported in this study. Overall, 95% of the sample completed the posttest (98% in the intervention group and 92% in the control group).

Sample Characteristics: Children in the sample had high scores on measures of problem behavior and externalizing. They ranged in age from 5 to 12 (mean age was 8 years). Of the 102 participant children, 52% lived with both biological parents, while 21% were from combined families, and 27% were from single parent households (this matched national demographic data). Among parents, 28% had a college or higher university degree, 43% had completed high school or occupational school, and 28% had only completed elementary or junior high school. The majority of the families in the sample (77%) had a socioeconomic status below the national average (U.S. equivalent of $70,000 a year).

Measures: The main outcome measure of child adjustment in Sigmarsdóttir et al. (2014) came from parents, teachers, and children. However, independent teacher ratings had the weakest loading on the latent construct of child adjustment, while non-independent parent ratings had the strongest loadings. Problem behavior was measured using the Child Behavior Checklist - Parent Report (the CBCL 1-5 version was used for families whose children were in kindergarten). Reliability scores for this measure were above .90 on both pre- and post-treatment measures. Social Skills was measured using the Social Skills Rating System (SSRS), which measured children's problem behaviors and children's adaptive functioning in social situations, and was completed by both parents and teachers. Reliability scores were high (.87 and .79 for pre- and post-treatment parent measures, respectively, and .90 for teacher measures at both time periods). Child depression was measured using the Children's Depression Inventory, completed by the children (reliability = .93).

A measure of parenting practices in Sigmarsdóttir et al. (2013) focused on mothers only and came from direct observations of interaction that were coded by researchers unaware of condition. The observations defined a latent variable construct based on four components: skill encouragement, discipline, problem solving, and positive involvement. Researchers coded family interactions during five or six tasks (5-10 minutes each). The tool has been used in previous research and reportedly has good predictive validity. To check on reliability, 20% of randomly selected videos of family behavior showed acceptable inter-rater agreement (0.80 to 0.85 at pretreatment and 0.82 to 0.88 at post treatment). Cronbach's alphas for each of the four scales used in the latent construct exceeded .80

Analysis: Structural Equation Modeling was used to create latent constructs for child adjustment and parenting practices at pretest and posttest and then examine the effect of condition on the posttest latent factors while controlling for the pretest latent factors.

Intent to Treat: The SEM analysis used Full Information Maximum Likelihood estimation that employed available information from all cases.

Outcomes

Implementation Fidelity. The PMTO manuals and parent materials were translated into Icelandic and adapted to be culturally appropriate. Parents in the treatment condition attended an average of 22.63 weekly sessions of 38 possible. An observation-based rating system scored therapists in training with a mean of 7.2 on a scale ranging from 6 to 9.

Baseline Equivalence: There were no baseline differences between groups on any of the demographic variables, nor on baseline child outcome indicators.

Differential Attrition: There were also no baseline differences between those retained at and not retained at posttest. The Little test further showed that data were missing completely at random.

Posttest: Sigmarsdóttir et al. (2014) found a significant reduction in child adjustment problems (construct comprised of behavior problems, social skills, and depressive symptoms) among children whose parents received the PMTO treatment, compared to the Services as Usual children. The program had weak-to-moderate effect sizes of .31 and .54.

Sigmarsdóttir et al. (2013) failed to find a significant treatment effect on the outcome measure of change in mothers' parenting. However, moderation tests indicated that the program buffered the harm for parenting practices of high depressive symptoms among mothers.

Long-term: Not examined.

Study 8

Summary

Kjobli & Ogden (2012), Kjøbli & Bjørnebekk (2013), and Kjøbli et al. (2018) conducted a randomized trial of brief parent training (3-5 sessions) in primary care settings among 216 Norwegian families whose children were exhibiting signs of problem behaviors.

Kjøbli & Ogden (2012), Kjøbli & Bjørnebekk (2013), and Kjøbli et al. (2018) found that, compared to the control group, the Brief Parent Training intervention group showed

• Fewer parent-reported behavioral problems, lower externalizing, callous-unemotional traits, and anxiety/depression, and improved social competence.
• More positive parenting practices
• Less harsh and inconsistent discipline.

Evaluation Methodology

Design: The study used an individual-level randomized control design with data gathered at baseline, within two weeks post-intervention, and six-month follow-up. Families were recruited for participation in the study if they had children between the ages of 3 and 12 who exhibited signs of problem behavior at home, daycare, or school and had contacted, or been contacted by, a primary care agency due to these behavioral problems. A total of 228 families were assessed for eligibility and 216 families were randomly assigned to Brief Parent Training (n=108) or regular services (n=108). Of the 12 families excluded, 10 refused to participate and 2 did not meet criteria due to age or existing autism diagnosis.

Brief Parent Training (BPT) was delivered to families individually over 3-5 hours and was tailored to match the needs of the family and the behavior problems of the child.

At post-assessment 95 intervention families (88% retention) and 92 control families (85% retention) participated. At follow-up, 90 intervention families (83% retention) and 83 control families (77% retention) participated.

Sample characteristics: The average age of the children in the study was 7.28 years; 31.9% of the children were girls, 51% lived with both biological parents, and 36.6% lived with single parents. Parents had an average age of 35.3 years; 39.4% had a college or higher degree, 93.5% reported a Norwegian background and their average income was $88,815 which represents an upper-middle-income level.

Measures: A total of 14 variables were measured across three areas: parent-reported child outcomes (5 variables); parent-reported parenting practices (6 variables plus one additional measure of parent distress); and teacher-reported child outcomes (3 variables).

Parent-reported child outcomes (5 variables) were gathered with the Eyberg Child Behavior Inventory (ECBI) using the Intensity (pretest alpha=.91; posttest alpha=.93) and Problem scales (pretest alpha=.87; posttest alpha=.90), the Home and Community Social Behavior Scales (HCSBS) to measure externalizing behavior (pretest alpha=.93; posttest alpha=.94) and social competence (pretest alpha=.93; posttest alpha=.95); and the Child Behavior Checklist (CBCL) to measure anxiety/depression (pretest alpha=.83; posttest alpha=.88).

Parent-reported parenting practices (6 variables plus one additional measure of parent distress) were gathered using six scales from the Parenting Practices Interview: Harsh for Age (pretest alpha=.70; posttest alpha=.77), Harsh Discipline (pretest alpha=.78; posttest alpha=.80), Inconsistent Discipline (pretest alpha=.74; posttest alpha=.64), Appropriate Discipline (pretest alpha=.79; posttest alpha=.83), Positive Parenting (pretest alpha=.69; posttest alpha=.73), and Clear Expectations (pretest alpha=.61; posttest alpha=.64). An additional measure of parental distress was assessed using Symptom Checklist-5 (pretest alpha=.88; posttest alpha=.85).

Teacher-reported outcomes (3 variables) were gathered using the School Social Behavior Scales for externalizing behavior (pretest and posttest alpha=.97) and social competence (pretest and posttest alpha=.97) and the Teacher Report Form to measure anxiety/depression in children (pretest alpha=.85; posttest alpha=.87).

All measures and scales have been used in prior published work and reliability and validity have been established. Parents who helped deliver the program also rated child behaviors, but teacher ratings were independent.

Analysis: For the posttest analysis, Kjøbli & Ogden (2012) presented three multivariate analyses of variance (MANCOVA) to examine the intervention effects across the three areas: parent-reported child outcomes; parent-reported parenting practices; and teacher-reported child outcomes. If the composite scores were found to be significant, posthoc analyses were conducted to examine which variables produced an effect. An ANCOVA was run to examine intervention effects on parent mental distress.

For cases with completely missing data, MANCOVAs were run with the entire sample and were analyzed with the last observation carried forward (pretest scores were inserted at posttest). For missing data at the item level, imputation used the expectation maximization (EM) procedure. Little's MCAR test was conducted for each EM, which found that data were missing completely at random. Further analyses were conducted without imputations and yielded similar results.

For the six-month follow-up analysis, Kjøbli & Bjørnebekk (2013) used linear mixed models with group-by-time interaction terms that included baseline values and captured condition differences in the rate of change in the outcomes. Maximum likelihood estimation allowed for use of all cases in the analysis.

Intention-to-treat: The study complied with the intent-to-treat principle by analyzing all participants randomized to the condition regardless of the dose received.

Outcomes

Implementation fidelity: Interventionists who delivered the sessions received a standardized 9-day training, a manual, and supervision/instruction by trained therapists. They also received group-based supervision meetings every 6-months.

Reports of practitioner fidelity to the model were gathered from parents using a 32-item survey (alpha=.96) asking parents to indicate the degree to which the interventionists covered topics and core components of the intervention. Fidelity was high with a mean score of 4.49 on a scale from 1 to 5.

Baseline Equivalence: Based on parent and teacher reports, one demographic variable was significantly different between groups at baseline: families in the intervention group had a higher level of education than families in the control group and therefore education level was used as a control variable in the analysis. Further, based on teacher-report, the intervention group scored lower on teacher-reported social competence (p=0.008), higher on externalizing problems (p=0.007), and higher on anxiety/depression (p=0.04). These variables were also controlled for in all teacher-reported main analyses.

Differential attrition: At posttest (Kjøbli & Ogden, 2012), no differences were found between those who completed pre- and posttests on all outcome variables and most demographic variables except that parents who completed both pre- and posttests had a higher level of education (p=.02) than those parents who did not complete posttests. At follow-up (Kjøbli & Bjørnebekk, 2013), attrition rates did not differ significantly by condition, and tests found only one baseline measure (parent education) that predicted attrition.

Posttest: At immediate posttest (Kjøbli & Ogden, 2012), the multivariate tests revealed that two of the three areas examined (parent-reported child outcomes, p=.04 and parent-reported parenting practices, p=.00) were significantly different between the control and intervention groups. No significant differences were found for teacher-reported child outcomes, neither for the area as a whole nor for the three variables used to measure it. Of the 15 variables examined, 9 were significantly different between intervention and control groups: all five variables (p's<.041) of parent-reported child outcomes and four variables (harsh for age, harsh discipline, inconsistent discipline, positive parenting; all p's<.021) of parent-reported parenting practices were significant (measures that were not significant were: appropriate discipline and clear expectations). The parent distress measure was not significantly different between groups.

At the six-month follow-up (Kjøbli & Bjørnebekk, 2013), significant group-by-time coefficients emerged for five of 14 outcomes. Three of the outcomes relating to child problem behavior showed significantly greater reductions for the intervention group than the control group (d = .27-.33), and two of the outcomes relating to positive parenting showed significantly more improvement for the intervention group than the control group (d = .34-.53). None of the teacher-reported measures differed across conditions. Moderation tests indicated no difference in program effects by the presence of clinical-level conduct problems at baseline.

Kjøbli et al. (2018) focused on a single outcome measure of callous-unemotional traits (which had good reliability). Relative to the control group, children in the Brief Parent Training group showed significantly lower scores on the callous-unemotional scale at posttest (d = .32) but not at follow-up.

Long-term effects: The study did not collect long-term, follow-up data and therefore was not able to demonstrate sustained effects.

Study 9

Summary

Kjøbli et al. (2013) and Kjøbli et al. (2018) used a randomized controlled trial of group-based PMTO with 137 Norwegian families assigned to PMTO or a comparison group. Twelve 2.5-hour group sessions were provided to parents in the PMTO group to promote parenting skills in families whose children exhibited conduct problems.

Kjøbli et al. (2013) and Kjøbli et al. (2018) found that the PTMO group relative to the control group reported significantly improved

• Child outcomes (intensity, callous-unemotional traits, and social competence)
• Reported parenting practices (harsh discipline) at posttest and 6-month follow-up
• Parent distress at posttest
• Teacher-report child social competence at posttest but not at 6-month follow-up.

Evaluation Methodology

Design: Using a randomized controlled design, researchers sought to evaluate the program's effectiveness at improving conduct problems and social competence of children and parenting practices and mental health of parents. Families with children between the ages of 2 and 12 who exhibited problem behavior or had developed conduct problems were recruited after they had contacted one of 11 agencies across Norway. Of the 153 families assessed for eligibility, 137 (90%) agreed to participate and were randomized to the PMTO intervention (n=72) and control conditions (n=65). The control group was offered PMTO after study termination and was free to seek and receive any available intervention offered in regular services except for closely related interventions based on the same principles. Because some families chose to wait and receive PMTO after the study period and because some agencies did not offer any other types of services or interventions, 33 (51%) families in the control group did not receive any intervention. Five families (7%) in the PMTO group refused the intervention after randomization.

Assessments were done at baseline, posttest, and 6-month follow-up. From the intervention group, 92% of families completed the posttest and 89% completed the 6-month follow-up. In the control group, 92% of families completed the posttest and 91% completed the 6-month follow-up.

Sample Characteristics: The average age of the 137 children was 8.56 years, 36.5% were girls, 48.2% lived with both biological parents and 36.5% lived with single parents. The average age of the participating parent was 37.42 years, 92% had a Norwegian background and 60.6% had finished high school.

Measures: A total of fourteen variables were measured: parent-reported child outcomes (5 variables); parent-reported parenting practices (5 variables); parental mental distress (1 variable); and, teacher-reported child outcomes (3 variables).

Parent-reported child outcomes (5 variables) were gathered with the Eyberg Child Behavior Inventory (ECBI) using the Intensity (alphas=..92-.93) and Problem scales (alphas=.88-.90), the Home and Community Social Behavior Scales (HCSBS) to measure externalizing behavior (alphas=.94) and social competence (alphas=.95-.96); and the Child Behavior Checklist (CBCL) to measure anxiety/depression (alphas=.82-.85).

Parent-reported parenting practices (5 variables) were gathered using five scales from the Parenting Practices Interview: Harsh Discipline, Inconsistent Discipline, Appropriate Discipline, Positive Parenting, and Clear Expectations (all alphas ranged between .67 and .89).

An additional measure of parental distress was assessed using Symptom Checklist-5 (alphas=.87-.90).

Teacher-reported outcomes (3 variables) were gathered using the School Social Behavior Scales for externalizing behavior and social competence and the Teacher Report Form to measure anxiety/depression in children (all alphas ranged between .87 and .96).

All measures and scales have been used in prior published work and reliability and validity have been established.

It was unclear if the researchers who gathered the data were blind to the participant condition, but the measures from the parents, who in effect deliver the program, lacked independence.

Analysis: Linear mixed models were used to analyze the effects of the intervention using an intent-to-treat analysis. The models allowed use of all subjects, including those with incomplete data, and adjusted for overtime correlations within subjects. Tests for group-by-time interaction terms implicitly included controls for baseline outcomes.

Because the parents in the intervention condition were nested within groups, further analysis was conducted to examine the effect of group nesting on outcomes. Group nesting influenced the outcome of one of the measures and therefore the interaction term of group nesting-by-time was included in an analysis but did little to change the results.

Effect sizes were calculated based on t-tests and were reported as Cohen's d.

Outcomes

Implementation Fidelity: Parents completed an evaluation of the degree to which the therapists covered the topics and the core components of the intervention (e.g., "We have practiced how to give my child good directions" and "We practiced how to use timeout"). Adherence was high with a mean score of 4.84 of 5. Therapists were provided a two-day training in the intervention and received group supervision every six months.

Baseline Equivalence: The authors provided no figures but stated that no significant differences were found between groups at baseline on demographic or outcome variables.

Differential Attrition: The authors provided no figures but stated that no differences were found between completers and non-completers on demographic or outcome variables at baseline.

Posttest and 6-month follow-up: Eight of the 14 outcomes were significantly different between the intervention and control groups at immediate posttest. Three of the five variables of parent-reported child outcomes (intensity, problem behavior and social competence; p<.05) and three variables of parent-reported parenting practices (harsh discipline, positive parenting and clear expectations; p<.01) were significantly improved in the intervention group compared to the control group. The parent distress measure was significantly different between groups (p=.03) at posttest. One of the three teacher-report measures (social competence; p=.01) was significantly improved in the intervention group compared to the control group.

Seven of the 14 outcomes were significantly different between the intervention and control groups at 6-month follow-up. Three of the five variables of parent-reported child outcomes (intensity, externalizing and social competence; p<.03) and three variables of parent-reported parenting practices (harsh discipline, inconsistent parenting and positive parenting; p<.05) were significantly improved in the intervention group compared to the control group. The parent distress measure was significantly different between groups (p=.04). None of the three teacher-report measures was significantly different between groups at 6-month follow-up.

Analysis of interaction by time effects from baseline to 6-month follow-up revealed that 6 of the 14 outcomes were significant. Results were similar to the posttest results reported above with two exceptions - problem behavior and parent distress measures were not significantly different between groups overall despite significant differences at either the posttest or follow-up.

Kjøbli et al. (2018) focused on a single outcome measure (which had good reliability) of callous-unemotional traits. Relative to the control group, children in the PMTO group showed significantly lower scores on the callous-unemotional scale at posttest (d = .39) and follow-up (d = .48).

Long-term effects: The study did not collect long-term, follow-up data and therefore was not able to demonstrate sustained effects.

Effect size

Cohen's d effect sizes ranged between .34 (for parent-reported child problem behavior) and .88 (for parent-reported positive parenting) at posttest and between .34 (for parent-reported inconsistent discipline) and .88 (for parent-reported positive parenting) at 6-month follow-up.

Study 10

This study involved boys aged 8-12 with Oppositional Defiant Disorder or diagnosed Conduct Problems in the Netherlands.

Summary

Schoorl et al. (2017) used a non-matched quasi-experimental design in the Netherlands to test the intervention's effects on aggression in boys with oppositional defiant disorder/conduct disorder. Participants were recruited from medical clinics, special education schools, and regular elementary schools. There were a total of 64 participants assigned to the intervention group (n=22) or the control group (n=42). Primary outcome measures included parent- and teacher-reported child aggression. This study also included measures of salivary cortisol reactivity as a measure of neurobiology.

Schoorl et al., (2017) found that compared to the control group, intervention group participants had significant reductions in:

• Frequency of parent-reported aggression.

Evaluation Methodology

Design:

Recruitment: Participants were boys aged 8-12 recruited from clinical health centers, special education schools, and regular elementary schools in the Netherlands. Inclusion criteria included an IQ of above 70 and a diagnosis of either Oppositional Defiant Disorder (ODD) or Conduct Disorder (CD). A total of 65 boys were recruited and consented to participate.

Assignment: Participants were assigned to either the clinical intervention condition (n=22) or a control group (n=43). The method of treatment assignment was not discussed.

Attrition: There was only one drop-out, making the total retention rate 98%.

Sample:

The sample was exclusively male and predominantly white (62%), averaging approximately 10 years old at intake. Comorbidity of ADHD was approximately 70% and anxiety was approximately 59%, while depression ranged from 9% to 17%, and conduct disorder from 18% in the intervention group to 41% in the control group. Psychostimulant medication usage was approximately 38% for the entire sample.

Measures:

Assessments occurred at baseline, post-intervention (6 months after baseline), and a follow-up 12 months after baseline and 6 months after program completion. Primary outcomes include ­aggression, measured by both the Parent Daily Report and the Teacher Report Form (aggressive behavior subscale), and neurobiology, assessed with resting heart rate, cortisol reactivity, and cortisol recovery. Resting heart rate was measured for 3 minutes while participants were seated comfortably and watching a relaxing video. Salivary cortisol was gathered from participants' saliva using a .5ml tube and was measured during and after a psychosocial stressor, in this case being led to believe they were losing a computer game competition (with a highly coveted prize for winning) against another boy. Negative parenting practices were assessed with the Alabama Parenting Questionnaire, subscales of supervision and monitoring, inconsistent discipline, and corporal punishment. Internal consistency was not directly presented here but was reported to be adequate.

Analysis:

Preliminary analyses used repeated measures ANOVAs and paired t-tests to examine differences in reported aggression over time, and stepwise regression was used to examine the relationship between parental factors and neurobiology. Models controlled for baseline aggression scores but not for other baseline characteristics even if they differed between groups at baseline.

Intent-to-Treat: All available data were used in the analysis.

Outcomes

Implementation Fidelity:

Fidelity was monitored by checking video samples of recorded sessions, though no quantitative measure was presented.

Baseline Equivalence:

There were significant baseline differences in IQ and parent-reported frequency of aggression, and a moderately significant difference in comorbidity of conduct disorder.

Differential Attrition:

There was only one attritor.

Posttest:

There was a significant reduction in the treatment group on parent-reported aggression over time, relative to the control group; however, there was no impact on the independently measured teacher rating of aggression. Group-by-cortisol interactions showed that parent-rated aggression declined significantly more for intervention group participants with high cortisol reactivity than for those with low activity.

Long-Term:

Follow-up occurred 6 months after program completion.

Study 11

Summary

Several reports (Akin et al., 2016; Akin, Lang, McDonald et al., 2018; Akin, Lang, Yan, & McDonald, 2018; Akin & McDonald 2018; Yan & De Luca, 2021) evaluated the intervention in a randomized controlled trial using post-randomized consent for families of children who were in foster care and had a serious emotional disturbance. A total of 918 (55%) of the 1,652 randomized subjects who met eligibility criteria were approached for post-randomized consent for participation (461 in the intervention group and 457 in the usual care comparison group). Assessments occurred at baseline, posttest (6 months after baseline), and a 6-month follow-up (12 months after baseline). Primary outcomes included child socio-emotional functioning, child problem behaviors, child social skills, and child reunification with parents. Parent outcomes/risk factors included parenting skills and caregiver functioning.

The foster-parent studies (Akin et al., 2016; Akin, Lang, McDonald et al., 2018; Akin, Lang, Yan, & McDonald, 2018; Akin & McDonald 2018; Yan & De Luca, 2021) found that compared to control group participants, intervention group children had significantly improved:

• Socio-emotional functioning (caseworker-rated, posttest and follow-up)
• Problem behaviors (parent-report, posttest, and follow-up)
• Social skills (parent-report, posttest, and follow-up).

Evaluation Methodology

Design:

Recruitment: The study used a post-randomized consent design. Participants were parents and children who were aged 3-16, had documented behavioral problems, and were under foster care in the state of Kansas during the study period. Further eligibility criteria included the following: child's case plan goal of reunification with family; parent residence in the service area; parent not incarcerated for more than 3 months; and parent legally allowed to resume or maintain contact with the child. A total of 6,657 children were assessed for eligibility, and 1,652 met child-level eligibility criteria and were randomized. Then 918 of the randomized children/parent dyads were approached for participation. Consent followed randomization and being approached, with 664 dyads of 918 (72%) consenting to participate.

Assignment: Participants were randomized prior to consent. Initially 1,652 dyads were randomized to the intervention (n=855) or comparison group (n=797) though only 918 were approached for participation due to timing and location of service openings. Of those approached, 664 of the parents consented to participate, with more than half in the intervention group (n=360), and less than half in the usual-care comparison group (n=304). Consent rates differed across conditions: 78.1% for the intervention group and 66.5% for the comparison group. Note, however, that one of the child outcome measures for socio-emotional functioning was available for non-consented parents. Similarly, the child reunification measure was available for non-consented parents and utilized the approached sample of 918 dyads (intervention=461 and comparison=457).

Assessments/Attrition: Assessments occurred at baseline, posttest (6 months after baseline), and a 6-month follow-up (12 months after baseline). Attrition was measure-specific and depended on the treatment of unconsented and unapproached cases. For child measures at the 12-month follow-up (Figure 1 in Akin, Lang, Yan, and McDonald, 2018), 412-635 subjects with data on the two outcome measures represented 25-38% of the randomized sample, 45-69% of the approached sample, and 62-96%% of the consented and approached sample. Child reunification was measured from 24-60 months after the beginning of the intervention and, due to the use of administrative records, was available for full approached sample of 918.

Sample:

Participating children were majority male (53.5%) and white (77.2%) with an average age of approximately 12 years old. Approximately 54% of children had a diagnosed disability and 21.5% had prior removals to foster care. Parents had an average age of 38 years at first removal and were 52.2% single mothers. The most common reasons for removal were neglect (37%) and parent substance abuse (21.4%).

Measures:

Assessments occurred at baseline, posttest, and a 6-month follow-up. Socio-emotional functioning was measured for children ages 6-16 using the caseworker-administered Child and Adolescent Functioning Assessment Scale (α=.62), which provides an overall score of functioning based on eight subscales (school, home, community, behavior towards others, moods/emotions, thinking problems, self-harm, and substance use). Children aged 3-5 were assessed with the caseworker-administered Preschool and Early Childhood Functional Scale (α=.94). Caseworkers providing the ratings were aware of condition but did not deliver the program.

Child problem behaviors and prosocial skills were measured with the parent-report Social Skills Improvement System-Rating Scales (α=.82-.91), which measures problem behavior with five subscales (externalizing, bullying, hyperactivity/inattention, internalizing, and Autism Spectrum) and prosocial skills with seven subscales (communication, cooperation, assertion, responsibility, empathy, engagement, and self-control). This measure was only administered if the parent had visited with the child within the previous 60 days. Parents doing the ratings also helped deliver the program to their children.

Child Reunification was defined as legal release from foster care to a parent. It may be viewed as an indicator of the absence of child maltreatment. The state's public child welfare agency provided state administrative data for up to approximately 60 months after the intervention. The authors noted, however, that case managers and judges, who were aware of participation in the program, may have partly based their reunification decisions on that participation.

Risk factors:

Effective parenting was measured using the Family Interaction Task, an independently-rated observation-based assessment that requires the parent and child work cooperatively on age-appropriate tasks for approximately 30 minutes (α=.50-.63). Caregiver functioning was measured using four scales from the North Carolina Family Assessment Scale (parent mental health, parent substance use, parent use of social supports, and readiness for reunification) and was completed by foster care case managers (α=.66-.83).

Analysis:

Effects of the intervention were evaluated using repeated-measures ANCOVA and logistic regression models that controlled for time, baseline outcomes, and other covariates. Missing data were imputed using a principal component auxiliary variable technique.

Family reunification was analyzed using survival analysis models with Cox regressions to examine the time to reunification.

Intent-to-Treat: The imputation allowed for the use of all available data (N = 918) in analysis, with no cases dropped due to nonresponse or nonconsent. However, of the full randomized sample, those not approached for participation in the study (N = 734) were excluded. The family reunification analyses presented both ITT and per protocol samples separately.

Outcomes

Implementation Fidelity:

All PMTO sessions were video recorded and eligible to be randomly selected for fidelity review. Program deliverers were rated for fidelity on a quarterly basis until they were certified at which point fidelity was rated annually. On a 9-point scale, last available fidelity scores averaged 6.1 for noncertified practitioners (n=30) and 6.6 for certified practitioners (n=16).

Baseline Equivalence:

At baseline for the approached sample of 918, Table 1 in Akin et al. (2016) shows one significant group difference (parents married) and two marginally significant differences (out of 16 total measures). In comparing children approached and not approached after randomization (p. 215), there were two small but significant differences in child age and child race.

Differential Attrition:

Group condition and several baseline characteristics were associated with attrition. In addition, tests for baseline-by-condition attrition showed some differences. Measures from caseworkers and family reunification records were available for the full approached sample of 918.

Posttest:

At the 6-month posttest, intervention group participants had significantly improved caseworker-rated socio-emotional functioning (d=.31), parent-reported child problem behaviors (d=.09), and parent-reported prosocial skills (d=.09) compared to comparison group participants. These differences were still significant at follow-up. The socio-emotional functioning measure was independently rated and available for all approached subjects.

Risk factors:

At posttest, compared to the comparison group, intervention group parents had significantly improved parent mental health (OR=2.01), parent substance use (OR=1.67), parent use of social supports (OR=2.37), and readiness for reunification (OR=1.63). Although the intervention group generally maintained a positive trajectory, these differences were not significant at follow-up.

Long-Term:

With the exception of family reunification, the final follow-up assessment occurred 6 months after posttest. For family reunification, the ITT survival analysis over 24-60 months showed only a marginally significant effect (p = .083) for the program condition relative to the control condition (hazard ratio = 1.16).

Yan & De Luca (2021) created latent classes based on parent functioning and child well-being at both baseline and posttest. The latent classes divided participants into groups such as high risk, medium-level functioning, and high-level functioning. The analysis of main effects indicated that PMTO participants, controlling for initial latent class, had a higher likelihood of entering into a higher functioning latent class at posttest than control participants. The analysis of moderation effects, however, found mixed evidence of PMTO benefits.

Study 12

Summary

Fossum et al. (2014) used a quasi-experimental design to compare two non-randomly assigned groups of Norwegian children ages 3-9 with clinical-level aggression (n = 236). One group received PMTO (n = 147) and one group received the Incredible Years (n = 89). Measures of parent-rated conduct problems were obtained at posttest.

Fossum et al. (2014) found no effects of the PMTO program on conduct problems relative to an Incredible Years alternative.

Evaluation Methodology

Design:

Recruitment: The participants were young children in Norway who had been referred to outpatient treatment for disruptive and aggressive behaviors as reported by their parents. Eligible children were ages 3.9-9.0 years old, had scores within the clinical range of the Child Behavior Checklist aggression subscale and had no debilitating physical impairment. The samples came from two other studies and participants in a national-level parent training program. A total of 236 children and their families were included in the sample.

Assignment: In the quasi-experimental design, the two conditions came from separate studies of two programs implemented at the national level: PMTO (n = 147) and Incredible Years (n = 89). The study provided no information on how families came to participate in either of the two programs, but the authors referred to the conditions as being matched in that all children met the broad eligibility criteria of age and baseline severity of aggression. The eligibility exclusions differed by condition: 47 from PMTO and 12 from the Incredible Years did not meet the eligibility criteria for clinical-level aggression.

Assessments/Attrition: The study examined the sample at baseline and posttest (though without specifying the exact timing of the posttest for each condition). The completion rate at posttest was 76%.

Sample:

The sample had an average age of 7.1 years and consisted of 75% boys. The maternal age at study intake was 35.0 years and 21.7% had not completed high school. In total, 36% of the children lived in single-parent families. All but 3.4% of the participants were native Norwegians.

Measures:

All outcome measures came from parent ratings of four subscales of child conduct problems from the Child Behavior Checklist: internalizing, attention, social problems, and externalizing. No other information was provided.

Analysis:

Analysis of covariance was used to estimate condition differences on the pre- to post-change scores, controlling for child characteristics (pre-treatment internalizing, social problems, attention problems, and child age) and/or maternal characteristics (age, single parent, and level of education).

Intent-to-Treat: The study provided no details but appeared to use all available data.

Outcomes

Implementation Fidelity:

The study noted only that treatment dosages were the same across conditions.

Baseline Equivalence:

Table 1 shows the 13 variables for which tests of baseline equivalence were conducted. Both mean child and maternal age differed significantly between conditions, with both the children and the mothers in PMTO being older. The other 11 measures did not differ significantly across conditions.

Differential Attrition:

The attrition rate of 33% in the PMTO group was considerably larger than the 8% attrition rate in the Incredible Years group. Otherwise, the authors noted that the non-completers and the completers did not differ significantly on baseline child externalizing, parent marital status, or parent level of education.

Posttest:

The multivariate analysis with covariates found no condition differences on any of the four outcomes.

Long-Term:

Not examined.

Study 13

Summary

Akin et al. (2015) used a randomized controlled trial to examine 121 children ages 3-16 who were in foster care and had a serious emotional disturbance. The randomly assigned PMTO group (n = 78) was compared to a services-as-usual control group (n = 43) on measures of child social-emotional well-being and placement stability at a six-month posttest.

Akin et al. (2015) found moderation effects but did not present results for the main effects of the intervention.

Evaluation Methodology

Design:

Recruitment: Eligible participants included children from all regions of Kansas who were between the ages of 3 and 16, were in foster care, and had a serious emotional disturbance within six months of entering foster care. Participating families also had: 1) a case plan goal of reunification; 2) caregivers who resided in the service area and had not been incarcerated for more than three months at the time of study enrollment; and 3) caregivers who did not have an order of "no contact" from the court. The sample consisted of 121 children and parents or caregivers seeking to reunify with the child.

Assignment: The study randomly assigned participants to the PMTO intervention group (n = 78) or a services-as-usual comparison group (n = 43). However, the "design assigns subjects to intervention and comparison arms beforehand so they are aware of their assignment group prior to agreeing to study participation" (p. 257). The authors did not present information on consent rates. Although random assignment into pools occurred at a 1:1 ratio, intervention cases were allocated from the pools at an approximately 2:1 ratio because "intervention practitioners were working toward PMTO certification and required a minimum caseload."

Assessments/Attrition: Children were assessed at baseline and six months later (posttest). The authors referred to using statistical methods that account for missing data but made no mention of the extent of attrition or missing data.

Sample:

The mean age of the children was 11.7 years. About 56% were female, 79% were white, 19% were African American, and 3% were Native American.

Measures:

The outcome measures came from child and parent questionnaires administered by caseworkers and from administrative data. Three key measures of child social-emotional well-being outcomes included 1) child-reported social-emotional functioning, 2) problem behavior from non-independent parent ratings, and 3) social skills from non-independent parent ratings. The parent reports appeared to have come from visits with the child. Cronbach's alphas exceeded .80 with one exception (.63 for baseline social-emotional functioning). One other measure from administrative data, annual placement instability, equaled the annualized rate of placement settings.

Analysis:

The analysis estimated structural equation models separately for the intervention group and control group and then tested for condition differences in the model coefficients. The models included baseline outcomes but did not directly test for the main effects of the intervention. Although Table 2 presents posttest means of the outcomes for both conditions, it lacks significance tests and effect sizes.

Intent-to-Treat: The lack of information on attrition and missing data makes the use of an intent-to-treat sample unclear.

Outcomes

Implementation Fidelity:

Not examined.

Baseline Equivalence:

The baseline condition means reported in Tables 1 and 2 lack significance tests and effect sizes, and some differences were apparent. For example, the intervention group included 49% males versus 35% for the control group.

Differential Attrition:

There appeared to be no attrition.

Posttest:

The main analysis showed consistent moderation effects in that all the structural equation model coefficients differed across the conditions. For example, only the intervention group demonstrated significant effects of placement stability on posttest child social skills. The models did not include main effects. The outcome means in Table 2 show higher values for the intervention group than the control group but there were no significance tests reported.

Long-Term:

Not examined.

Study 14

Summary

Maaskant et al. (2016, 2017) used a randomized controlled trial to examine 88 foster parents and foster children with severe problem behavior. The randomly assigned PMTO group (n = 47) was compared at posttest and four-month follow-up to a care-as-usual control group (n = 41) on measures of parent stress, parenting, and child problem behavior, externalizing, and internalizing.

Maaskant et al. (2016, 2017) found that, relative to the control group, the intervention group parents reported significantly:

• Lower parent stress at posttest (but not at follow-up)
• Greater parenting warmth for mothers at posttest (but not at follow-up).

Evaluation Methodology

Design:

Recruitment: The sample included Dutch foster parents of children aged 4-12 years old who were in foster care for at least one year and had severe externalizing behavior problems. Assessments for eligibility used two screenings, one for child scores above the clinical cutoff on the Strengths and Difficulties Questionnaire and one for parent reports of five or more daily child behavior problems. Of 606 families assessed, 88 foster parents and children met the eligibility criteria and agreed to participate.

Assignment: One of the researchers randomized participants to the intervention group (n = 47) or a care-as-usual control group (n = 41). All staff and counselors of foster care organizations were blind to the randomization process. Two families - one in each condition - dropped out immediately after randomization (without baseline assessment), leaving a sample with one or both foster parents of 86 foster children.

Assessments/Attrition: The posttest assessment occurred approximately 10 months from baseline, and a follow-up occurred four months after the posttest. About 72% of the foster parents completed the posttest and follow-up assessments. Completion rates were slightly higher for foster mothers than foster fathers and teachers.

Sample:

The children in the sample had a mean age of 7.7 years and included a mix of genders, with 48% boys and 52% girls. The children averaged about four years in foster care. About 30% were non-Dutch, but no other information on race or ethnicity was presented.

Measures:

The 15 outcome measures examined in the study included scales for parenting stress, parenting behavior, and child behavior problems (total, externalizing, and internalizing). Both foster mothers and fathers provided ratings of their stress and parenting as well as non-independent ratings of the behavior of their foster children. Teachers completed independent measures of child behavior. Cronbach's alphas were generally but not always acceptable. They ranged from .59 to .83 for parenting behavior, .67 to .94 for parenting stress, and .78 to .99 for child behavior.

Analysis:

The analysis used multilevel models that included repeated time measures nested within participants and one or both (if present) foster parents. The models adjusted for clustering between foster mothers and fathers and within individuals. Tests for program effects came from group-by-time interaction terms that contained the baseline time point. Moderation tests corrected for the Benjamini-Hochberg False Discovery Rate.

Intent-to-Treat: The analysis used all participants with baseline data, even those with missing posttest and follow-up data.

Outcomes

Implementation Fidelity:

Ratings of therapist adherence done during training and before the study showed a mean score of 7.18 on a scale from 1-9, where 7-9 indicated good adherence. However, only 30 of 47 (64%) participants assigned to PMTO received the treatment.

Baseline Equivalence:

Tests of 12 baseline demographic measures in Table 1 found one significant difference (p < .05), and tests of 15 outcome measures at baseline in Table 3 found no significant differences.

Differential Attrition:

Attrition rates varied by condition: 36% and 38% for the intervention group and 20% and 17% for the control group at the posttest and follow-up, respectively. The authors reported one baseline difference between posttest completers and drop-outs: foster parents who dropped out had significantly fewer years of foster experience.

Posttest:

For the 15 outcomes at posttest (Maaskant et al., 2017), the intervention group relative to the control group had significantly greater improvement among mothers for total parent stress, parent domain stress, child domain stress, and parenting warmth (effect sizes from .34 to .54). Fathers showed the same results for stress but not for parenting warmth. The intervention did not significantly affect any of the child behavior measures from parents or teachers. There was also no significant moderation of program effects.

At the follow-up (Maaskant et al., 2016), none of the outcomes remained significant.

Long-Term:

Not examined.

Study 15

Summary

Thijssen et al. (2017) used a quasi-experimental design with partial randomization to examine 155 Dutch children ages 4-11 with externalizing problems who were assigned to PMTO (n = 94) or a care-as-usual control group (n = 61). Assessments of parenting and parent- and teacher-rated child behavioral problems occurred at baseline and six, 12, and 18 months after baseline.

Thijssen et al. (2017) found no effects of the PMTO program on child behavior, parenting, or parenting stress relative to a care-as-usual control group.

Evaluation Methodology

Design:

Recruitment: The sample included children aged 4-11 and their parents, who were recruited at five child service agencies across the Netherlands between June 2009 and January 2014. The children had to have a clinical-level externalizing score on the Child Behavior Checklist. A total of 155 families met the eligibility criteria and consented to participate.

Assignment: Randomization occurred in only three of the five child service sites, making the study quasi-experimental. One site recruited families only for the PMTO condition, and one site recruited only control families. Given differences in recruitment success of the last two sites, the sample sizes differed for PMTO (n = 94) and the care-as-usual control group (n = 61). Baseline data collection followed assignment, with two lost from the PMTO group (2%) and seven from the control group (11%).

Assessments/Attrition: Assessments occurred at baseline and six, 12, and 18 months after baseline. With the treatment lasting from 15 to 25 weeks, the last follow-up came roughly one year after the program end. The authors reported that 27 participants (17%) dropped out, of which nine (6%) provided no baseline data. In addition, missing values reached approximately 50% for some outcomes.

Sample:

The sample consisted of 71% boys and had a mean age of 7.13 years. Three quarters met DSM diagnostic criteria for ADHD, 67.3% for ODD, and 11.6% for CD.

Measures:

Of the seven child outcome measures, three came from the parent-reported Child Behavior Checklist (internalizing, externalizing, total problems), three came from the Teacher Report Form (internalizing, externalizing, total problems), and one came from parent reports on the number of daily problem behaviors. Reliability measures were all .80 or higher. Only the three teacher ratings met the independence standard, however.

Measures of parenting practices came from two sources, parent self-reports and coding of structured observations. However, only two of the parent-reported measures had reliabilities high enough to include in the analysis: discouraging undesirable behavior and the total parenting score. The 16 measures obtained from the structured observations, which were coded by researchers blind to condition, included separate items for the main caregiver, non-main caregiver, and child. Most of the measures of inter-rater reliability were fair to excellent, but two were poor (ICC < .40).

Four measures of self-reported parent stress and psychological symptoms had high reliabilities (alphas ≥ .93).

Analysis:

The analysis used repeated-measures ANOVA, with time (from baseline through the three follow-ups) as the within-subjects factor and treatment condition as the between-subjects factor. Tests for program effects came from condition-by-time interaction terms. Multiple imputation was used for missing data among participants with baseline data.

Intent-to-Treat: The analysis had to exclude the 6% of the sample without baseline data but used multiple imputation to include all participants with baseline data.

Outcomes

Implementation Fidelity:

The study reported only that the PMTO participants received on average 23.85 treatment sessions.

Baseline Equivalence:

Table 1 shows no significant condition differences for 12 demographic characteristics, though some differences were substantial (e.g., 22% of the PMTO group and 37% of the control group had a high school education). For the seven outcome measures, one significant difference emerged between conditions: children from the control group displayed higher scores on the daily problems report than children in the PMTO group.

Differential Attrition:

The study did not present any tests. It noted only that attrition rates were similar across conditions (18% in the PMTO group and 16% in the control group).

Posttest and Long-Term:

The over-time analysis combining posttest and long-term assessments revealed no significant condition differences on any of the seven child behavior measures, two self-reported parenting measures, 15 observed parenting behaviors, or four parent stress and psychological symptoms measures. Only one observed parenting behavior, interpersonal atmosphere for the non-primary caregiver, showed significantly greater improvement for the PMTO group than the control group. Tests for moderation found little evidence of subgroups that benefitted more from PMTO than care as usual.

Study 16

Summary

Scavenius et al. (2020) used a randomized controlled trial to examine 132 Danish children with behavioral problems and their caregivers. The randomly assigned PMTO group (n = 68) was compared at posttest and 18-20-month follow-up to a services-as-usual control group (n = 62) on measures of parent stress, parenting, and child problem behavior, externalizing, internalizing, and prosocial behavior.

Scavenius et al. (2020) found no effects of the PMTO program on child behavior, parenting, or parenting stress relative to a services-as-usual control group.

Evaluation Methodology

Design:

Recruitment: The sample came from 11 municipalities in Denmark that offered both PMTO and family treatments. Eligible families had children aged three to 13 years who were referred to municipal treatment because of child behavioral problems. In total, 123 families with 132 children joined the study (i.e., nine families participated with two children). All families received treatment between January 2013 and July 2016.

Assignment: Families were randomized within sites either to the PMTO intervention (n = 68) or a services-as-usual control group (n = 62). Allocation ratios varied between sites and were adjusted according to the site's treatment capacity. The sum of the samples across the two conditions (n = 130) indicates the loss of two randomized participants. In addition, the PMTO group lost four before the baseline assessment, while the control group lost none.

Assessments/Attrition: Families were assessed at three time points: baseline, posttest (about 7-8 months after baseline), and follow-up (about 18-20 months after baseline and 10-13 months after posttest). Of the 132 children, 95% completed the baseline assessment, 94% completed the posttest, and 85% completed the follow-up.

Sample:

Most participating children were boys (71%), and the youth mean age at baseline was 8.0 years. The vast majority of respondents were mothers (91%) and employed (61%).

Measures:

All outcome measures came from parents. Although assessments were conducted by trained interviewers who were not involved in the treatment, the parent-reported measures of child behavior do not meet standards of independence for parenting programs. The eight child outcomes came from the Strengths and Difficulties Questionnaire and included measures such as total problems, externalizing, internalizing, and prosocial behavior. Cronbach's alphas for the total score were 0.68, 0.73, and 0.74 across the three time points, but there were no details for the subscales used in the analysis. The eight parent measures relating to efficacy, sense of competence, life satisfaction, sense of coherence, stress, and depression had good reliabilities (alphas ≥ .80).

Analysis:

The analysis used regression models with fixed effects for child to control for unmeasured between-child differences and time-by-treatment interaction terms to test for program effects. As some families had multiple children, cluster-robust standard errors adjusted for clustering at the family level. The authors stated that the fixed-effects regression, by controlling for time-invariant child characteristics, also controlled for potential attrition bias. Supplementary information presented results using mixed-effects models to control for the non-independence of observations.

Intent-to-Treat: The analysis used all randomized participants with baseline data, regardless of completion of the program or the subsequent assessments.

Outcomes

Implementation Fidelity:

A separate study demonstrated that the fidelity ratings collected during the implementation of PMTO in Denmark were considered in the "good work range."

Baseline Equivalence:

As shown in Table 1, one significant difference between conditions emerged in 11 tests for sociodemographic characteristics: Control caregivers had more advanced education than caregivers in PMTO. The study did not report tests for the outcomes.

Differential Attrition:

The authors noted that dropout rates were not significantly different between conditions (10% in the PMTO group versus 16% in the control group). The study offered no other information on differential attrition, but the authors argued that fixed effects controlled for many determinants of attrition.

Posttest:

The results revealed no significant condition differences in the change over time for any of the child or parent measures at posttest.

Long-Term:

The results revealed no significant condition differences in the change over time for any of the child or parent measures at follow-up (10-13 months after the program end).