A training program for service providers in the use of effective strategies for promoting secure and healthy relationships between caregivers and young children birth to three years of age.
Blueprints: Promising
Jennifer Rees, Program and Training Manager
University of Washington
CHDD South Building Room 212
Box 357920
Seattle, WA 98195
rees@uw.edu
206-616-5380
Monica Oxford, MSW, Ph.D.
University of Washington
Promoting First Relationships® trains workers in early intervention, community mental health, home visiting and early care and education settings to deliver a home visiting program based on infant mental health principles. Promoting First Relationships® is strengths based. It uses joint observation and reflection on videotaped caregiver-child interactions to increase caregivers' confidence and competence. Providers support caregivers' ability to read their child's nonverbal cues, empathize with and provide comfort when their child is distressed, and understand that their child's difficult behavior may reflect underlying social and emotional needs. The program is delivered in the home to caregivers and children (birth to 3 years of age) in 10 weekly sessions of 60-75 minutes. The program model requires that providers receive regular reflective consultation.
Promoting First Relationships® is an attachment-based, strengths-based mental health training program for workers in early intervention, community mental health, home visiting and early care and education settings. It is designed to increase caregiver sensitivity and responsivity by helping caregivers identify possible "miscues," empathize with the child's underlying distress, and understand the child's behavior as reflecting an unmet need. A better understanding of cues is expected to then lead to more responsive, nurturing care. In addition, the program addresses the fact that infants and toddlers in child welfare may give behavioral signals that lead even nurturing caregivers to provide non-nurturing care. The program is delivered in the home to caregivers and children (birth to 3 years of age) in 10 weekly sessions of 60-75 minutes. Videotaping the dyad and reflective observations of the videotaped sessions occur on alternate weeks, for a total of five video reflective sessions. Promoting First Relationships® uses reflective practice principals with video feedback to focus on the deeper emotional feelings and needs underlying difficulties in the parent and child relationship and to help caregivers think about their child's developing mind. Learning materials include handouts, worksheets, and 'Thoughts for the Week' that are used in a flexible, adaptive manner to fit the needs of the dyad. These cover topics such as strategies for calming ourselves and our children, how to meet the socioemotional needs of young children, understanding and responding to challenging behavior, and recognizing the need for young children to feel safe and secure in their relationships with their caregivers.
Primary Evidence Base for Certification
Study 2
Oxford, Spieker et al. (2016) found significant improvement in the treatment group, as compared to the control group for:
Primary Evidence Base for Certification
Of the five studies Blueprints has reviewed, one (Study 2) meets Blueprints evidentiary standards (specificity, evaluation quality, impact, dissemination readiness). The study was done by the developer.
Study 2
Oxford, Spieker et al. (2016) randomly assigned infant-caregiver dyads to either the Promoting First Relationships treatment program or to a control group receiving phone consultations and information on resources. All participating parents had been reported to child protective services for child maltreatment. The study assessed child and caregiver behaviors at baseline, posttest, 3-month follow-up, and 6-month follow-up.
Study 2
Oxford, M. L., Spieker, S. J., Lohr, M. J., & Fleming, C. B. (2016). Promoting First Relationships: Randomized trial of a 10-week home visiting program with families referred to child protective services. Child Maltreatment, 21, 267-277.
Family: Parent history of mental health difficulties*, Parent stress*
*
Risk/Protective Factor was significantly impacted by the program
Subgroup differences in program effects by race, ethnicity, or gender (coded in binary terms as male/female) or program effects for a sample of a specific race, ethnic, or gender group.
Study 2 (Oxford, Spieker et al., 2016) found subgroup effects by using a homogenous sample with 75% or more of families classified as economically disadvantaged (i.e., receiving food stamps).
Sample demographics including race, ethnicity, and gender for Blueprints-certified studies:
A majority of parents in Study 2 (Oxford, Spieker et al., 2016) were mothers (91%). In terms of race, most parents were white (77%), while 3% of the sample parents were American Indian or Alaska Native, 4% were Asian, 5% were African American, and 11% were of a mixed race or "other." Around 20% of parents were Hispanic. Most of the sample received food stamps (79%).
Promoting First Relationships® (PFR) includes 2 levels of training. Level 1 is a prerequisite for Level 2. The studies certified by Blueprints included service providers with Level 2 training.
Level 1-- Knowledge Building:
This two-day workshop is designed to give service providers knowledge about using Promoting First Relationships® within one's own practice. The workshop is taught by PFR Master Trainers and includes the curriculum, parent handouts, and training in the following:
Level 2 -- Skill Building (Provider Level):
In the mentored distance learning, participants work individually or in pairs with a PFR Master Trainer.
Level 3 -- Agency Trainer Level Training (by invitation only):
Individuals who have successfully completed the Level 2 training, may participate in an Agency Trainer Level Training (Level 3), by invitation only. This training certifies individuals to provide Level 2 trainings to service providers within their own agencies per the Agency Trainer Guidelines.
Program Benefits (per individual):
$725
Program Costs (per individual):
$1,380
Net Present Value (Benefits minus Costs, per individual):
($655)
Measured Risk (odds of a positive Net Present Value):
47%
Source: Washington State Institute for Public Policy
All benefit-cost ratios are the most recent estimates published by The Washington State Institute for Public Policy for Blueprint programs implemented in Washington State. These ratios are based on a) meta-analysis estimates of effect size and b) monetized benefits and calculated costs for programs as delivered in the State of Washington. Caution is recommended in applying these estimates of the benefit-cost ratio to any other state or local area. They are provided as an illustration of the benefit-cost ratio found in one specific state. When feasible, local costs and monetized benefits should be used to calculate expected local benefit-cost ratios. The formula for this calculation can be found on the WSIPP website.
Level 1 (Learner's Workshop): $425 per person for 2-day workshop in Seattle, WA.
Level 2 (Distance Learning): $2,150 per person when training as part of a pair group. $3,000 per person if trainee is a stand-alone individual (not part of a pair group). Conducted over 15 weeks and includes 16 contact hours (online) with a master trainer and the initial certification fee. Prerequisite is Level 1.
Note: The Blueprints certified studies included service providers with Level 2 training.
None.
None (though it may be necessary to replace toys periodically).
Delivered as a home visit. Average home visit is one hour plus travel time and prep time. Estimated 2 hours per family per week over 10 weeks-20 hours FTE per family. Varies depending on time for travel.
None.
$125 per year to reassess fidelity. Videos are sent to the program for coding and review. If the provider does not reach fidelity, he/she can get additional support.
None.
No information is available
No information is available
In this example, an agency uses four providers to deliver Promoting First Relationships® to client families. Each provider serves 20 families in the first year. At two hours per family, on average, and 20 families per year per provider, the average FTE for each provider required to serve 20 families is 0.20. The agency uses its own office space for training and video review. Equipment and travel expenses (for in-home visits) are not included here as they may vary substantially by location.
| Level 1 Training: 4 x $425 | $1,700.00 |
| Travel to Level 1 Training: 4 x $1,000 | $4,000.00 |
| Level 2 Training: 4 x $2,150 | $8,600.00 |
| BabyCues video and cards, PFR parent/caregiver video and social/emotional cards: 4 x $115 | $460.00 |
| Annual Fidelity Monitoring: 4 x $125 | $500.00 |
| Total One Year Cost | $15,260.00 |
The four providers collectively deliver the program to 80 families in the first year. Thus, the Year 1 per family expense is $190.75 (does not incude costs for travel or toys--toys are optional). The costs for subsequent years would be substantially reduced as most of the expense involves training.
Promoting First Relationships® (PFR) is a home visiting program that offers early intervention services to promote healthy attachment and development for infants who have had a change in their primary caregiver. Federal funding streams aimed at promoting healthy development of young children and preventing child abuse, including Medicaid, Title V, IDEA Part C, and Title IV-B Child Welfare Services, can support Promoting First Relationships®. State funds allocated for child welfare services can also support Promoting First Relationships. Finally, the Affordable Care Act made an historic investment in home visiting, allocating $1.5 billion to support states in implementing evidence-based home visiting programs through the Maternal, infant and Early Childhood Home Visiting Program.
In addition to commitments of state general fund dollars for Medicaid match, many states allocate state and county funds to support preventive child welfare services. These dollars are an important source of support for Promoting First Relationships®.
Entitlements:
Medicaid: If the intervention is delivered by Medicaid-eligible providers, the program can potentially be billed as a Medicaid therapeutic service, or through targeted case management services under Medicaid. Whether Medicaid billing is an option depends on the state Medicaid plan.
Formula Funds:
Discretionary Grants: There are many federal discretionary grants supporting child welfare and early care and education that can potentially support PFR, including programs within SAMHSA, the Department of Education and the Children's Bureau within DHHS.
Foundations that prioritize investments in child welfare services and healthy early childhood development are good targets for investing in PFR. Local United Ways that invest in home visiting programs and child welfare have also supported PFR.
Many states have Children's Trust Funds that are supported with dedicated revenue streams from license plates, commemorative documents and tax form check-offs. Children's Trust Funds support child abuse prevention services and are a potential source of support for PFR.
Monica Oxford, MSW, Ph.D.Research Professor, Family and Child NursingUniversity of WashingtonBOX 357920Seattle, Washington 206-685-6107mloxford@uw.edu
A training program for service providers in the use of effective strategies for promoting secure and healthy relationships between caregivers and young children birth to three years of age.
The program targets infants and toddlers (birth to three) who are at elevated risk for child maltreatment or who have experienced a court-ordered change in caregiver.
Subgroup differences in program effects by race, ethnicity, or gender (coded in binary terms as male/female) or program effects for a sample of a specific race, ethnic, or gender group.
Study 2 (Oxford, Spieker et al., 2016) found subgroup effects by using a homogenous sample with 75% or more of families classified as economically disadvantaged (i.e., receiving food stamps).
Sample demographics including race, ethnicity, and gender for Blueprints-certified studies:
A majority of parents in Study 2 (Oxford, Spieker et al., 2016) were mothers (91%). In terms of race, most parents were white (77%), while 3% of the sample parents were American Indian or Alaska Native, 4% were Asian, 5% were African American, and 11% were of a mixed race or "other." Around 20% of parents were Hispanic. Most of the sample received food stamps (79%).
Caregivers who exhibit sensitivity to children, offer support, express commitment, understand children, and experience less stress are more likely to have positive relationships with children. Caregiver knowledge of children's social-emotional needs.
Family: Parent history of mental health difficulties*, Parent stress*
*Risk/Protective Factor was significantly impacted by the program
Promoting First Relationships® trains workers in early intervention, community mental health, home visiting and early care and education settings to deliver a home visiting program based on infant mental health principles. Promoting First Relationships® is strengths based. It uses joint observation and reflection on videotaped caregiver-child interactions to increase caregivers' confidence and competence. Providers support caregivers' ability to read their child's nonverbal cues, empathize with and provide comfort when their child is distressed, and understand that their child's difficult behavior may reflect underlying social and emotional needs. The program is delivered in the home to caregivers and children (birth to 3 years of age) in 10 weekly sessions of 60-75 minutes. The program model requires that providers receive regular reflective consultation.
Promoting First Relationships® is an attachment-based, strengths-based mental health training program for workers in early intervention, community mental health, home visiting and early care and education settings. It is designed to increase caregiver sensitivity and responsivity by helping caregivers identify possible "miscues," empathize with the child's underlying distress, and understand the child's behavior as reflecting an unmet need. A better understanding of cues is expected to then lead to more responsive, nurturing care. In addition, the program addresses the fact that infants and toddlers in child welfare may give behavioral signals that lead even nurturing caregivers to provide non-nurturing care. The program is delivered in the home to caregivers and children (birth to 3 years of age) in 10 weekly sessions of 60-75 minutes. Videotaping the dyad and reflective observations of the videotaped sessions occur on alternate weeks, for a total of five video reflective sessions. Promoting First Relationships® uses reflective practice principals with video feedback to focus on the deeper emotional feelings and needs underlying difficulties in the parent and child relationship and to help caregivers think about their child's developing mind. Learning materials include handouts, worksheets, and 'Thoughts for the Week' that are used in a flexible, adaptive manner to fit the needs of the dyad. These cover topics such as strategies for calming ourselves and our children, how to meet the socioemotional needs of young children, understanding and responding to challenging behavior, and recognizing the need for young children to feel safe and secure in their relationships with their caregivers.
The primary theoretical framework for Promoting First Relationships (PFR) was Bowlby's attachment theory, which stresses that the child's attachment to the primary caregiver can keep the child safe, secure, and protected while providing a secure base for exploration. Attachment formation is vital to child development and may be difficult for children who have experienced a change in primary caregiver; a lack of caregiver attachment can lead to problem behaviors and mental health issues.
Booth-Laforce et al. (2020): "We believed that PFR would align well with the cultural beliefs and practices of our tribal partner, and we worked to adapt the program in ways that would enhance its acceptability and feasibility in the community while retaining the core elements of the PFR intervention" (p. 99). The initial trial (Booth-Laforce et al. , 2020) was conducted in collaboration with a rural American Indian tribe on a reservation in the Northwest region (as defined by the Indian Health Service) of the USA. Together, the researchers and tribal leaders adapted aspects of the Promoting First Relationships (PFR) intervention so that they aligned more closely with the tribe's cultural practices and preferences. Researchers hired and trained Native members of the tribal community who lived on the reservation to assist in implementing the study: a PFR provider to deliver the PFR intervention and a Research Visitor to collect assessment data.
Context for adapting Promoting First Relationships for native populations: American Indian populations have been resilient in the face of attempts to terminate their tribes and eradicate their cultures. However, the resulting negative impact on physical and mental health has been profound. American Indian health disparities have persisted and for some factors are even increasing. Thus, given their own challenges, American Indian parents may struggle to provide an environment for their children that optimizes growth and development, even though children are highly valued and considered an important part of American Indian communities. Although prior studies have highlighted specific parent-child and family issues and their impact on children's development, it is equally important to consider the effects of historical factors, especially historical trauma and cultural oppression. The bitter legacy of historical trauma includes past governmental policies that forcibly removed American Indian children from their families and placed them in boarding schools that punished expression of Native culture. The resultant traumatization and separation from their primary caregiver, as well as the loss of parenting skills, parenting models, and cultural values, likely contribute to current burdens of substance use and domestic violence in American Indian populations. Efforts to implement programs that support American Indian parents and their capacity to support their children must overcome these endemic barriers, as well as community resistance to outside interference born of longstanding mistrust of the federal government. Because American Indian children are a vulnerable group, they deserve intensive, culturally sensitive efforts to develop and disseminate preventive interventions designed to meet their unique needs in their own tribal communities.
Primary Evidence Base for Certification
Of the five studies Blueprints has reviewed, one (Study 2) meets Blueprints evidentiary standards (specificity, evaluation quality, impact, dissemination readiness). The study was done by the developer.
Study 2
Oxford, Spieker et al. (2016) randomly assigned infant-caregiver dyads to either the Promoting First Relationships treatment program or to a control group receiving phone consultations and information on resources. All participating parents had been reported to child protective services for child maltreatment. The study assessed child and caregiver behaviors at baseline, posttest, 3-month follow-up, and 6-month follow-up.
Primary Evidence Base for Certification
Study 2
Oxford, Spieker et al. (2016) found an effect for child atypical affective communication (a risk factor). In addition, there were significant effects on 2 of 4 caregiver outcomes, both of which measured reciprocal parent-child warmth: 1) parent understanding of toddlers (assessed only at posttest and 6-month follow-up); and 2) parent sensitivity (only at 6-month follow up). In addition, within one-year post intervention, chances of removal from the birth parent home were greater for children in the control condition than children in the treatment condition at any given time.
Primary Evidence Base for Certification
Study 2
Oxford, Spieker et al. (2016) found significant improvement in the treatment group, as compared to the control group for:
For Study 2, Oxford, Spieker et al. (2016) reported a small effect size (d = .20) for the parent sensitivity outcome.
One study meets Blueprints standards for high-quality methods with strong evidence of program impact (i.e., "certified" by Blueprints): Study 2 (Oxford, Spieker et al., 2016). The sample for the study included parents who had been reported to Child Protective Services for child maltreatment.
Study 2 took place in Washington States and compared the treatment group to a services-as-usual control group.
Additional Studies (not certified by Blueprints)
Study 1 (Nelson & Spieker, 2013; Oxford et al., 2013, 2016; Pasalich et al., 2016; Spieker et al., 2012, 2014)
Nelson, E. M., & Spieker, S. J. (2013). Intervention effects on morning and stimulated cortisol responses among toddlers in foster care. Infant Mental Health Journal, 34, 211-221.
Oxford, M. L., Fleming, C. B., Nelson, E. M., Kelly, J. F., & Spieker, S. J. (2013). Randomized trial of Promoting First Relationships: Effects on maltreated toddlers' separation distress and sleep regulation after reunification. Children and Youth Services Review, 35, 1988-1992.
Oxford, M. L., Marcenko, M., Fleming, C. B., Lohr, M. J., & Spieker, S. J. (2016). Promoting birth parents' relationships with their toddlers upon reunification: Results from Promoting First Relationships home visiting program. Children and Youth Services, 61, 109-116.
Pasalich, D. S., Fleming, C. B., Oxford, M. L., Zheng, Y., & Spieker, S. J. (2016). Can parenting intervention prevent cascading effects from placement instability to insecure attachment to externalizing problems in maltreated toddlers? Child Maltreatment, 21, 175-185.
Spieker, S. J. Oxford, M. L., & Fleming, C. B. (2014). Permanency outcomes for toddlers in child welfare two years after a randomized trial of a parenting intervention. Children and Youth Services Review, 44, 201-206.
Spieker, S. J., Oxford, M. L., Kelly, J. F., Nelson, E. M., & Fleming, C. B. (2012). Promoting First Relationships: Randomized trial of a relationship-based intervention for toddlers in child welfare. Child Maltreatment, 17, 271-286.
Study 3 (Booth-LaForce et al., 2020)
Booth-LaForce, C., Oxford, M. L., Barbosa-Leiker, C., Burduli, E., & Buchwald, D. S. (2020). Randomized Controlled Trial of the Promoting First Relationships® Preventive Intervention for Primary Caregivers and Toddlers in an American Indian Community. Prevention Science, 21(1), 98-108.
Study 4 (Oxford et al., 2020)
Oxford, M., Hash, J., Lohr. J. M., Bleil, M., Fleming, C., Unützer, J., & Spieker, S. (2020). Randomized trial of Promoting First Relationships® for new mothers who received community mental health services in pregnancy. Manuscript submitted for publication.
Study 5 (Booth-LaForce et al., 2023)
Booth-LaForce, C., Oxford, M. L., O'Leary, R., & Buchwald, D. S. (2023). Promoting First Relationships® for primary caregivers and toddlers in a native community: A randomized controlled trial. Prevention Science, 24, 39-49. https://doi.org/10.1007/s11121-022-01415-y
Spieker et al. (2012; Study 1) conducted the original experimental study in which infants and toddlers in foster care and their caregivers were randomly assigned to condition. Using the same sample, Spieker et al. (2014) analyzed caregiver stability and permanency outcomes of children 2 years after enrollment in the study and Pasalich et al. (2016) conducted additional moderation analyses. Meanwhile, Nelson & Spieker (2013) used a subsample of the dyads randomized in the Spieker et al. (2012) study to examine patterns of cortisol activity by condition in assessing responses to stress from pre- to post-intervention. Similarly, Oxford et al. (2013) used a subsample of the sample reported in Spieker et al. (2012) to assess differences between treatment and control in toddler sleep as well as the effect of the treatment on those children who were being reunified with their birth parents after a foster care placement (see Oxford, Marcenko et al., 2016). Thus, all these studies are grouped under Study 1.
For Studies 3 and 5, Booth-LaForce et al. (2020, 2023) tested the program in American Indian families living on a rural reservation. Authors adapted the program based on focus groups (Study 3, Booth-LaForce et al., 2020) with community members and staff input to align with the tribe's cultural practices and preferences and hired and trained Native members of the tribal community who lived on the reservation to assist with implementation. Only those aspects of the program were adapted that could increase cultural relevance while maintaining core principles and components. This included: (1) creating a unique name for the study, (2) commissioning an appropriate study logo by a Native artist, (3) lengthening each home visit to include more time for conversation, (4) providing a small gift for the child at each research visit, and (5) adding one handout about caregiver-child transitions and separations.
Study 5 (Booth-LaForce et al., 2023) was preregistered with ClinicalTrials.gov (NCT02139332).
Blueprints: Promising
Jennifer Rees, Program and Training Manager
University of Washington
CHDD South Building Room 212
Box 357920
Seattle, WA 98195
rees@uw.edu
206-616-5380
Nelson, E. M., & Spieker, S. J. (2013). Intervention effects on morning and stimulated cortisol responses among toddlers in foster care. Infant Mental Health Journal, 34, 211-221.
Oxford, M. L., Fleming, C. B., Nelson, E. M., Kelly, J. F., & Spieker, S. J. (2013). Randomized trial of Promoting First Relationships: Effects on maltreated toddlers' separation distress and sleep regulation after reunification. Children and Youth Services Review, 35, 1988-1992.
Oxford, M. L., Marcenko, M., Fleming, C. B., Lohr, M. J., & Spieker, S. J. (2016). Promoting birth parents' relationships with their toddlers upon reunification: Results from Promoting First Relationships home visiting program. Children and Youth Services, 61, 109-116.
Pasalich, D. S., Fleming, C. B., Oxford, M. L., Zheng, Y., & Spieker, S. J. (2016). Can parenting intervention prevent cascading effects from placement instability to insecure attachment to externalizing problems in maltreated toddlers? Child Maltreatment, 21, 175-185.
Spieker, S. J. Oxford, M. L., & Fleming, C. B. (2014). Permanency outcomes for toddlers in child welfare two years after a randomized trial of a parenting intervention. Children and Youth Services Review, 44, 201-206.
Spieker, S. J., Oxford, M. L., Kelly, J. F., Nelson, E. M., & Fleming, C. B. (2012). Promoting First Relationships: Randomized trial of a relationship-based intervention for toddlers in child welfare. Child Maltreatment, 17, 271-286.
Certified Oxford, M. L., Spieker, S. J., Lohr, M. J., & Fleming, C. B. (2016). Promoting First Relationships: Randomized trial of a 10-week home visiting program with families referred to child protective services. Child Maltreatment, 21, 267-277.
Booth-LaForce, C., Oxford, M. L., Barbosa-Leiker, C., Burduli, E., & Buchwald, D. S. (2020). Randomized Controlled Trial of the Promoting First Relationships® Preventive Intervention for Primary Caregivers and Toddlers in an American Indian Community. Prevention Science, 21(1), 98-108.
Oxford, M., Hash, J., Lohr. J. M., Bleil, M., Fleming, C., Unützer, J., & Spieker, S. (2020). Randomized trial of Promoting First Relationships® for new mothers who received community mental health services in pregnancy. Manuscript submitted for publication.
Booth-LaForce, C., Oxford, M. L., O'Leary, R., & Buchwald, D. S. (2023). Promoting First Relationships® for primary caregivers and toddlers in a native community: A randomized controlled trial. Prevention Science, 24, 39-49. https://doi.org/10.1007/s11121-022-01415-y
Summary
Spieker et al. (2012) randomly assigned infant-caregiver dyads to either the Promoting First Relationships treatment program or a control group receiving another home visitation program. All participating infants had experienced a court-ordered change in caregiver and were referred by a state agency. The study assessed child behaviors and security and caregiver sensitivity and support at baseline, posttest, and 6 months after completion of the program. Using the same sample, Spieker et al. (2014) analyzed caregiver stability and permanency outcomes of children 2 years after they had enrolled in the study, and Pasalich et al. (2016) conducted additional moderation analyses. Meanwhile, two articles (Nelson & Spieker, 2013; Oxford et al., 2013) used a subsample of the dyads randomized in the Spieker et al. (2012, Study 1) to examine additional outcomes, including infant/toddler stress and sleep problems.
Spieker et al. (2012) showed significant improvement among children in the treatment group as compared to the control group at posttest in:
Evaluation Methodology
Design:
Recruitment: Researchers in the Spieker et al. (2012) study contacted caregivers of infants in one county between the ages of 10 and 24 months who had experienced a court-ordered placement that resulted in a change in primary caregiver within the prior 7 weeks, based on Department of Social and Health Services records. Initially, 427 caregivers were contacted and 280 cases were deemed eligible for the study (i.e., caregivers had to speak English and could be foster parents, biological parents or adult kin), of which 219 enrolled and 61 declined. After enrollment, an additional 9 dyads were declared ineligible, so the sample size at randomization was 210 caregiver/children dyads.
Assignment: Spieker et al. (2012) randomly assigned participating parent-child dyads, blocked by caregiver type (foster parents, biological parents or adult kin), to either the treatment or a control group. Control group participants received The Early Education Support program, which consisted of three monthly 90-minute home visits to promote growth and development and information on available resources. Of the 210 dyads, 105 were each randomly assigned to the treatment or control conditions.
Attrition: Of the 210 dyads randomized in Spieker et al. (2012), 175 (or 83%) completed the posttest with their original caregiver and thus were included in the analysis. At 6 months after the intervention, the sample included 129 dyads who completed the assessment with their original caregiver (70% of those randomized).
Spieker et al. (2014) also examined permanency and stability at 2-year follow-up. Presumably, although the authors did not discuss it, there was no attrition as the measures employed were obtained through state records. Pasalich et al. (2016) used the same sample and the same measures as Spieker et al. (2012).
Sample: Over half of infants in the Spieker et al. (2012) sample were male (55%) and approximately 55% of infants were white, 20% were mixed race, 15% were black, 10% were Hispanic, 7% were Native American or Alaskan native, and 4% were other races or were not identified. The average age of infants at the beginning of the study was 36 months and on average infants had experienced over 2 caregiver changes since birth. Approximately 27% of households in the study earned an income of less than $20,000 per year. On average, caregivers were around 36 years of age and had 13 years of education.
Measures: Spieker at al. (2012) and Pasalich et al. (2016) included 2 pre- and posttest measures of child behavioral outcomes and 3 child behavioral outcome measures administered at baseline and 6 months after the intervention (without posttest assessment). In addition, the authors used 5 measures of risk and protective factors and 6 assessing caregiver outcomes.
Child behavioral outcomes included:
Risk & protective factor child outcome measures included:
For caregiver outcomes assessing reciprocal parent-child warmth, Spieker et al. (2012) and Pasalich et al. (2016) included the Nursing Child Assessment Teaching Scale to report parent sensitivity, which was independently coded by researchers using a videotaped session (sample α ranged from .71 to .79). Parent sensitivity items represented aspects of positive interaction and indicators of mutuality (e.g., contingency, gaze, and positive affect), caregiver verbal and nonverbal support of child, and sensitive instruction during the teaching task. The Indicator of Parent-Child Interaction, also scored by observers, reported support (sample α ranged from .76 to .84), which included items such as "acceptance/warmth," "descriptive language," and "follows child's lead." Commitment to child, which assessed the caregiver's desire to parent the child as long as the child remains in care or is benefitting from care and the caregiver's ability to fully attach to the child without withholding feeling or putting up barriers to limit the extent of the attachment, was rated based on interviews with caregivers using the This Is My Baby questionnaire (interrater agreement was r = .89). Caregivers reported understanding of toddlers (which measured caregivers' knowledge of infant and toddler social-emotional needs and developmentally appropriate expectations) with the Raising a Baby survey (sample α ranged from .73 to .77). Finally, the caregivers rated their perception of having a difficult child (Stress - Difficult Child) and having a dysfunctional parent-child relationship (Stress - Dysfunctional Interaction) with the Parenting Stress Index (sample α ranged from .87 to .89). All six of these measures were administered at baseline, posttest and 6-month follow-up.
Two years after initial enrollment in the Spieker et al. (2012) study, Spieker et al. (2014) used state child welfare administrative data to measure whether a child had remained with the study caregiver since randomization (stability) and whether the child remained with the same caregiver with a legal discharge to that caregiver (permanency).
Analysis: Spieker et al. (2012) used analysis of covariance models to assess differences by experimental condition in caregiver and child outcomes at pretest, posttest, and follow-up. For posttest and 6-month follow-up, the baseline score on the given measure (when available) was included as a covariate, in addition to other demographic variables including whether the child experienced multiple removals from the birth home, caregiver type, and age of child. Pasalich et al. (2016) conducted an additional moderation analysis to analyze whether there was a differential effect of the treatment on a cascade from placement instability to insecure attachment to externalizing problems in toddlers.
At the two-year follow-up, Spieker et al. (2014) used logistic regression to predict stability and permanency, with intervention group and various demographic variables (i.e., age of child at randomization, time in child welfare at time of randomization, number of placement changes prior to randomization, multiple removals - whether the child experienced one or more failed reunifications with the birth parent prior to randomization, and commitment to child) included as covariates. Baseline outcomes for the Spieker et al. (2012) study were not used in the Spieker et al. (2014) analysis.
Intent-to-Treat: Spieker et al. (2012) reported using an intent-to-treat protocol and treated infant-caregiver dyads that did not remain intact at follow-up time points as missing or attrition. To analyze intervention effects on outcomes they used both ANCOVA models with a listwise deletion approach to missing data and growth models that used maximum likelihood estimates and included data on all participants, including those with missing data at follow-up time points. Pasalich et al. (2016) reported they used FIML to handle missing data. Using the same sample as Spieker et al. (2012), Spieker et al. (2014) included all 210 children, consistent with an intent-to-treat model.
Outcomes
Implementation Fidelity: Spieker et al. (2012) reported that three video feedback sessions per family were conducted during the study. Independent observers assigned a global rating on a scale of 1-5 assessing level of fidelity to each treatment segment observed, and reported an average score of 4.04 (SD = .76). In addition, 97% of activities were completed.
Baseline Equivalence: Spieker et al. (2012) reported no differences between conditions in outcome measures at baseline. Infants in the treatment condition were significantly more likely to have experienced multiple removals from their birth families, but it appears there were no other demographic differences at baseline.
Differential Attrition: Spieker et al. (2012) reported no differences between completers and attritors for condition, gender, caregiver type, multiple removals, or number of caregiver changes before enrollment. At posttest, treatment group participants who did not have a change in caregiver included younger children and were more likely to have completed all sessions of the intervention. However, differential attrition for dyads in the comparison was unclear, and the study did not analyze condition by outcome for attrition.
Posttest: At posttest, Spieker et al. (2012) found a significant improvement in caretakers' perception of child competence among treatment group infants and toddlers as compared to control group infants and toddlers. However, this significant difference did not remain at the 6-month follow-up. Spieker et al. (2012) also found a significant improvement in parent sensitivity and understanding of infants and toddlers among treatment caregivers as compared to infants and toddlers among control caregivers at posttest, but these differences also did not remain at 6-month follow-up.
Pasalich et al. (2016) found several moderator effects (no main effects were tested in this study). That is, among children with more than 4 placement changes (27% of the sample), those in the control group exhibited lower levels of attachment security as compared to children in the treatment group at posttest. In addition, the study found that placement changes were significantly and inversely associated with attachment security and lower levels of attachment security were associated with higher scores on externalizing problems at a later assessment.
Long-Term: Two years after initial enrollment in the study, Spieker et al. (2014) found no overall effect of intervention condition on stability (i.e., placement with no interruptions or disruptions) or permanency (i.e., a stable placement ending with a legal discharge to the study caregiver); a significant interaction effect between type of caregiver and condition on permanency was detected. Compared to controls, receiving treatment increased the likelihood of permanency among children with foster/kin caregivers than among children with birth parents, and the size of this effect was large (OR = 9.67).
Study 1 Cont'd
Evaluation Methodology
Design:
Recruitment: The Nelson & Spieker (2013) study invited 57 of the 210 caregiver-child dyads randomized in the Spieker et al. (2012) study to participate in additional analyses of child cortisol levels. It is not clear how participants were selected from the full sample and if the randomization achieved in the full sample was maintained. Drawing from the same overall sample, researchers identified 56 biological parents and their recently reunified toddlers to conduct additional analyses comparing differences by condition in sleep problems (Oxford et al. 2013) and emotion regulation (Oxford, Marcenko et al., 2016).
Assignment: Of the 57 dyads invited to participate in the Nelson & Spieker (2013) study, 54 agreed and provided baseline saliva samples. The study states that 21 dyads were assigned to the treatment group and 25 were assigned to the control group. However, this only accounts for 46 participants, not the 48 who provided data at both time points or the 54 originally selected. For the Oxford et al. (2013) and Oxford, Marcenko et al. (2016) studies, 43 (treatment = 18 and control = 25) of the 56 dyads remained intact from enrollment to 6-month follow up and were therefore included in the analysis.
Attrition: Of the 54 participants who participated in the Nelson & Spieker (2013) study and provided baseline measurements, 2 caregivers dropped out and 4 children experienced a placement change by the postintervention measure. In addition, baseline morning cortisol samples were collected on 43 children and this data point was used as a covariate in the overall analysis. Of the 56 dyads identified for the Oxford et al. (2013) and Oxford, Marcenko et al. (2016) studies, 43 dyads remained intact and were included in the analysis.
Sample: Ages in the subsample reported in Nelson & Spieker (2013) ranged from 10 to 25 months, at baseline. In addition, 38% of children were living with a foster parent, 38% had been returned to a birth parent, and 25% were living with a family member. A majority of participants were white (68.8%), 17% were black, 13% were American Indian, and 2% were Hawaiian Native. The majority of the infants in the subsample in Oxford et al. (2013) and Oxford, Marcenko et al. (2016) were white (67%), female (53%), had a female caregiver (88%) and came from a household that earned less than $20,000 a year (60%).
Measures: In the Nelson & Spieker (2013) study, at both time points, authors collected five saliva samples: on arrival, just before the brief separation from caregiver, 30 minutes after the return of the caregiver, 45 minutes after the return of the caregiver, and the following morning. Cortisol levels were analyzed in each sample. Researchers established a conservative cut point of .05 ig/dL based on average variability within the sample to allow for capture of meaningful changes in cortisol level across collection points. The average ¼ SD for collection points ranged from .051 to .018. In this study, increasing cortisol levels in theoretically stressful conditions were considered a more adaptive behavior, while flat cortisol levels were considered potentially deleterious.
Oxford et al. (2013) pulled items from the Child Behavior Checklist and the Brief Infant Toddler Social and Emotional Assessment to create a 6-item measure of sleep problems that was completed by caregivers in treatment and control at baseline and 6-month follow up (sample α = .87). Separation distress was measured using the Toddler Attachment Sort-45 (inter-rater reliability, r = .92).
Measures used in the Oxford, Marcenko et al. (2016) study were the same as those used in the Spieker et al. (2012) study.
Analysis: Nelson & Spieker (2013) conducted multinomial logistic regression to analyze the association between posttest cortisol pattern and intervention group. The categorical outcome variable was increasing, decreasing, or flat cortisol levels during the course of the research visit. Analyses controlled for condition, time of day, child's age, baseline morning cortisol level, and flat or not flat cortisol pattern observed at baseline. OLS regression was used in the Oxford et al. (2013) study, which included sleep problems at baseline, age in months at enrollment, whether the child had experienced multiple removals from the biological parent's home, and time between baseline and 6-month follow up as covariates.
Oxford, Marcenko at al. (2016) conducted the same analysis and used the same covariates as Spieker et al. (2012).
Intent-to-Treat: Nelson & Spieker (2013) excluded baseline data from the 2 participants who dropped out of the study and had missing data for morning cortisol for an additional 5 participants. Meanwhile, Oxford et al. (2013) and Oxford, Marcenko et al. (2016) both dropped 13 dyads that did not remain intact during the study period. As no information was provided on handling of missing data, it is not clear if these studies followed intent-to-treat protocol.
Outcomes
Baseline Equivalence: Nelson & Spieker (2013) found no significant differences between the treatment and control groups for morning cortisol levels, however there were no statistical tests conducted on the demographic variables of this subsample. Oxford et al. (2013) found no demographic differences between groups at baseline, but did not test for baseline outcome differences. Oxford, Marcenko et al. (2016) reported no significant group differences in scores on measures at baseline, but did not clarify whether demographic and outcome measures were both tested.
Differential Attrition: None of the studies discussed differential attrition among the subsample.
Posttest: At posttest, Nelson & Spieker (2013) found that children in the treatment group were significantly more likely to have increasing postintervention cortisol levels. In addition, moderator analyses found that older children were more likely to have increasing postintervention cortisol levels. The study was not found to affect morning cortisol levels. Oxford et al. (2013) found that treatment predicted fewer sleep problems. Meanwhile, Oxford, Marcenko et al. (2016) found no program effects at posttest but a significant effect on observed parent support at the 6-month follow-up in favor of the treatment group.
Summary
Oxford, Spieker et al. (2016) randomly assigned infant-caregiver dyads to either the Promoting First Relationships treatment program or to a control group receiving phone consultations and information on resources. All participating parents had been reported to child protective services for child maltreatment. The study assessed child and caregiver behaviors at baseline, posttest, 3-month follow-up, and 6-month follow-up.
Oxford, Spieker et al. (2016) found significant improvement in the treatment group, as compared to the control group for:
Evaluation Methodology
Design:
Recruitment: Participants were eligible if they were conversant in English, had housing, lived in a certain region within Washington state, and had a child between the ages of 10-24 months and an open child welfare case with an allegation of maltreatment. To recruit participants, a Department of Child and Family Services volunteer contacted families who had recently been reported to Child Protective Services (N=1,070). Of the total 1,070 families contacted, 504 could not be reached, 172 were not eligible, and 133 declined to participate. A total of 251 families agreed to a home visitation, during which an additional 4 were deemed ineligible.
Assignment: A total of 247 families were randomly assigned to the treatment (n=124) or the control (n=123) group in racial/ethnic blocks. The control condition received 3 phone consultations providing a needs assessment, mailed packet of personalized information, and two 10-minute follow-up check-in calls.
Attrition: Participants were assessed at four time points: 1) pretest (n = 247); 2) posttest (n = 225, 91% of the randomized sample); 3) 3-month follow-up (n = 215, 87%); and, 4) 6-month follow-up (n = 211, 86%).
Sample: A majority of parents in the study were mothers (91%). In terms of race, most parents were white (77%), while 3% of the sample parents were American Indian or Alaska Native, 4% were Asian, 5% were African American, and 11% were of a mixed race or "other." Around 20% of parents were Hispanic. Parent education included high school graduation (45%), GED (31%), and neither high school graduation nor GED (24%). Less than one third (31%) were employed full or part time, 28% were unemployed and looking, and 41% were not employed (i.e., were homemakers, students, retired, on disability, other). Over half of parents were never married (56%), 24% were married at the time of the study, and 22% were separated or divorced. Most of the sample received food stamps (79%) and average household income in the past year was $21,883, indicating lower socioeconomic status. Just over half of children in the study were male (54%) and were, on average, 16.37 months at baseline.
Measures: The study included 2 pre- and posttest measures of child behavioral outcomes and 1 child behavioral outcome measure administered at baseline and 3 months after the intervention (without posttest assessment). In addition, the authors used 3 measures of risk and protective factors for children, and 4 assessing caregiver outcomes.
Child behavioral outcomes included:
Measures of risk & protective child outcomes included:
Measures of caregiver outcomes assessing reciprocal parent-child warmth included:
In addition, the study gathered official records on additional allegations of child maltreatment and removals from the birth parent home. For each child in the study, records were obtained of new allegations and removals that occurred between baseline and 12 months after the child's parent completed the intervention. For cases where the parent did not start or did not fully complete the intervention, information was obtained on allegations and removals that occurred within 12 months of when the parent would have completed the intervention if the parent had done so on schedule.
Analysis: The study used mixed model regressions to predict outcome variables, with intervention condition, age of child at baseline, and months between baseline and post-intervention assessment (which varied between conditions) as covariates. For measures assessed at only one posttest time point, the study used regression models predicting the posttest outcome measure. For measures with multiple posttest scores, 2-level random effects models were conducted in which posttest scores were nested within dyads. Survival analysis was conducted for models involving child maltreatment outcomes (i.e., new allegations and removal from birth home, as measured by child welfare administrative data).
Intent-to-Treat: For the child welfare outcomes, all study children were included in the analysis, consistent with an intent-to-treat model. For the data collected in the home, the study used an intent-to-treat protocol and employed Full Information Maximum Likelihood (FIML) protocols to account for missing data; however, infants who experienced a change in caregiver during the course of the study were excluded (which could be a violation of intent-to-treat protocol).
Outcomes
Implementation Fidelity: To ensure fidelity, the study used video-taped sessions to review providers' interactions with families. However, the study did not provide quantitative measures of these reviews. In the treatment condition, 86% of participants received all 10 sessions and 89% of control participants received all 3 telephone consultations.
Baseline Equivalence: The study reported there were no differences between conditions on any demographic variables at baseline. There was one difference in caregiver outcome at baseline in which the control group had significantly higher scores on parenting sensitivity.
Differential Attrition: There were no differences by completion status on demographic variables, but tests of differential attrition by outcomes were unclear.
Posttest: The study found no significant effects on the three behavioral outcomes (social-emotional competence, behavior problems and emotion regulation). Findings, however, showed a significant effect in favor of the treatment group on 1 of 3 child risk and protective outcomes at posttest: child atypical affective communication. In addition, within one-year post intervention, chances of removal from the birth parent home were greater for children in the control condition than children in the treatment condition at any given time.
For caregiver outcomes, the study found a significant effect on parent understanding of toddlers, which was only measured at posttest and 6-month follow-up, and parent sensitivity, measured at all four time points.
Long-Term: The study did not report long-term findings.
Summary
Booth-LaForce et al. (2020) randomly assigned 34 primary caregivers living on or near an American Indian reservation with their toddlers aged 10-30 months to either a treatment or waitlist control group. Measures of child and caregiver behaviors were collected at baseline and immediately after the intervention.
Booth-LaForce et al. (2020) found that, at the immediate posttest, compared to the control group, participants in the treatment group showed improved:
Evaluation Methodology
Design:
Recruitment: Researchers recruited American Indian and Alaska Native families from the tribal health clinic, but also posted flyers in the community, publicized the study on Facebook, and provided information to Head Start teachers. To be eligible, participants had to have at least one parent or guardian who 1) was at least 18 years old; 2) spoke English; 3) was the primary caregiver for an Native American child aged 10-30 months; 4) had telephone access; 5) was not in a treatment facility or shelter; 6) was not hospitalized or imprisoned; 7) was willing to have researchers come to their home; and 8) lived on or near the reservation. If the household included multiple children in the target age range, the child with whom the caregiver wished to work was selected to participate in the study.
Assignment: Participants were 34 toddlers (10-30 months old) and their primary caregivers. Families were randomized to a treatment (n = 17) or waitlist control (n = 17) group. Researchers used an algorithm to randomize participants, blocking on male or female caregiver, in equal numbers to the treatment and control group. Waitlisted families in the control group had the opportunity to receive the intervention after the study assessments.
A problem with the design stems from the reliance on a single provider to deliver the program to all intervention participants. Page 11 states, "We trained the PFR Provider, who was a tribal member living on the reservation." The characteristics of the single PFR Provider were thus confounded with the program.
Assessments/Attrition: Families in both groups completed the follow-up assessment after the intervention period concluded. In the treatment group, the median time between baseline and follow-up assessments was 15 weeks (M = 18.00, SD = 5.29). In the control group, the median time between assessments was 17 weeks (M = 19.14, SD = 9.45). Of the 34 parent-child dyads who completed baseline assessments, 23 dyads completed a posttest (see consort diagram in Figure 1), for an overall attrition rate of 32%.
Sample: Of the total sample, 73% of the primary caregivers identified as American Indian (AI) or Alaska Native (AN), 15% as white, and 12% as mixed. All were living on the reservation, all of the non-Native caregivers had AI/AN partners or spouses, and all had an AI/AN child between 10 and 30 months old.
Measures:
The primary outcome measure assessed caregiver-child interaction (i.e., Reciprocal Parent-Child Warmth) and included the total score for Caregiver/Child Interaction on the Nursing Child Assessment Teaching Scale (NCATS), coded based on video recordings of the teaching task. A certified NCATS coder viewed the video recordings and scored them for caregiver sensitivity, stimulation of the child, and emotional responsiveness. The coder was blind to treatment group and time of assessment (baseline versus follow-up) and reviewed the videos in a randomized order provided by the investigators. The coder evaluated the caregiver's verbal and non-verbal support and the child's clarity of cues and response to the caregiver by responding. Researchers state that the NCATS has excellent psychometric properties, though no information on reliability or validity was reported.
The following measures were completed by caregivers (and thus are considered non-independent outcome measures):
Analysis: Researchers used multiple regression models, adjusting for caregiver race, time elapsed between assessments, and the baseline score on the given measure.
Missing Data Method: Missing data were addressed using multiple imputation in SPSS (version 24) and STATA (version 14).
Intent-to-Treat: The authors stated they conducted "intent-to-treat" analyses.
Outcomes
Implementation Fidelity: After initial certification, the intervention group leader submitted one video recording of an intervention session per family to enable a master trainer to monitor ongoing fidelity by rating the quality and completeness of intervention delivery on a 5-point scale (M = 4.21, SD = 0.70 in the present study). Nine of the 17 caregivers in the treatment group completed all 10 visits.
Baseline Equivalence: T-tests found no significant baseline differences on any variables except the ITSEA Competence score. For this measure, the treatment had higher baseline scores compared to the control group. In addition, Table 1 shows some large non-significant differences between conditions (e.g., 65% female child in the intervention group versus 41% in the control group).
Differential Attrition: No tests of differential attrition were conducted. However, according to the What Works Clearinghouse standard, based on the overall attrition rate and the condition difference in attrition rates, the potential for attrition bias exists under both the cautious and optimistic thresholds. Attrition rates were 18% for the control group and 47% for the treatment group for a differential attrition rate of 29%. The use of multiple imputation may partly counter concerns over attrition bias.
In a separate document provided to Blueprints, the authors performed tests for baseline equivalence using data from participants who completed the posttest assessment. When compared to tests for the full randomized sample, these tests check for distortions of equivalence due to attrition. In 12 tests for condition differences in sociodemographic and outcome variables, 3 measures were significantly different - caregiver race, and caregiver reports of child externalizing and competence. Disproportionately fewer American Indian/Alaska Native primary caregivers were retained in the intervention group (44%), compared to the control group (93%). On the child measures, the intervention group had lower levels of baseline externalizing (d = 1.06) and higher levels of competence (d = 1.15).
Posttest: Relative to the control group, families in the treatment group showed significantly greater improvements in 3 of 5 behavioral outcomes. At the posttest, compared to the control group, participants in the treatment group earned a higher Caregiver/Child total score (d = 1.02) - which measures caregiver's verbal and non-verbal support and the child's clarity of cues and response to the caregiver - and Caregiver/Child Contingency total score (d = 1.21), which assesses the contingent responsiveness of the caregiver to the child, or vice versa.
In addition, significant findings in favor of the treatment group were detected in 1 of 2 risk and protective factors. Caregivers in the treatment group reported higher scores for Raising a Baby (d = 0.58), which measured caregivers' knowledge of the social-emotional needs of infants and toddlers as well as caregivers' level of developmentally appropriate expectations of their children.
Long-Term: Not reported.
Summary
Oxford et al. (2020) randomly assigned 252 mothers and their infants to either the treatment or control group. Measures of infant and mother behaviors were collected at baseline, 6 months after baseline (i.e., the posttest, since the intervention took place between baseline and the 6-month assessment), and 12 months after baseline (thus considered a 6-month follow-up).
Oxford et al. (2020) found that, compared to control participants, intervention participants demonstrated significantly better
Evaluation Methodology
Design:
Recruitment: Researchers recruited participants through participating health centers. To be eligible, mothers needed to have received mental health treatment during pregnancy, be conversant in English or Spanish, have an infant under three months of age, have access to a telephone, planning to stay in the area until the child's first birthday, and to not have received the intervention previously. Each month, these health centers created a list of eligible mothers who were then contacted about the study.
Assignment: Participants consisted of 252 pairs of mothers and infants. These pairs were randomly assigned, blocked with preferred language (English or Spanish), to either receive the intervention or to a control group. The control group received a packet of resources available in the area, not including the intervention.
Assessments/Attrition: Overall attrition was less than 5% at both time points. That is, of those who completed the baseline assessment, 97% completed the 6 month assessment (posttest) and 96% completed the 12-month assessment (6-month follow-up).
Sample: Mothers ages ranged from 18 to 42 years. They were 66% White, 18% Black, and 16% other races. Forty-seven percent identified as Hispanic, and 33% preferred to read and speak in Spanish. Median family annual income was less than $20,000. To be eligible, mothers needed to have received mental health treatment during pregnancy. At baseline, 51% had mild to severe depression symptoms, 54% had mild to severe anxiety, and 35% had PTSD.
Measures:
Measures consisted of observational assessments of mothers interacting with their infants as well as some self-report measures completed by mothers.
Observational Measures
Parenting sensitivity was measured using the Nursing Child Assessment Teaching Scale (NCATS). For this task, mothers are instructed to attempt an activity with their infant that they had not yet demonstrated (e.g., transferring an object from one hand to another). A coder, blind to condition, coded the interaction on 50 possible behaviors that demonstrate parental sensitivity. Behaviors were scored as having occurred (1) or not (0) and summed to create a sensitivity score. A single bilingual and bicultural coder, blind to intervention condition, was trained to reliability by a certified NCATS instructor and passed biannual reliability checks with a minimum of 85% inter-rater agreement with a master trainer.
The Infant CARE-Index assessed parent-child interactions during play. A 3-5 minute play session was recorded and assessed by coders. The index produces a score for parent-child synchrony (dyadic synchrony), three measures of parent behavior (sensitivity, control, unresponsiveness) and four measures of child behavior (cooperation, compulsivity, difficultness, passivity). Coders had to meet a standard for inter-rater reliability of r = .80 on the measure of dyadic synchrony and an average of r = .70 across all scales. As a further check of reliability, over the course of coding approximately 20% of the videos were randomly selected to be coded by a reliability coder. Inter-rater correlations for dyadic synchrony between pairs of coders averaged r = .67. Meanwhile, inter-rater correlations for difficultness on the 20% of videos selected for coding by an additional coder averaged r = .49. The researchers only reported on dyadic synchrony and child difficultness.
Self-Report Measures
Mothers completed the Raising a Baby Scale, a measure of caregiver knowledge and understanding of infant and toddler social-emotional needs and developmental expectations (αs ranged from .68 to .71 across the three time points), and the Maternal Confidence Questionnaire, which measures the perceptions mothers have of their ability to care for and understand their infants with higher scores indicating more confidence (αs ranged from .66 to .76 across the three time points). At the final visit (infant age 12 months), mothers also completed the Infant Toddler Social Emotional Assessment (ITSEA). This measure has mothers rate their infants on an extensive list of behaviors. These behaviors produce composite scores of infant externalizing, internalizing, and dysregulation. This measure was only given at the final visit because it is inappropriate for infants younger than 12 months. The ITSEA has been standardized and normed on a nationally representative sample that was stratified to match the 2002 United States Census. In the study sample, alpha reliabilities for the domains were α = .81 for externalizing, α = .68 for internalizing, and α =.80 for dysregulation.
Analysis: Treatment effects were assessed at each time point using multiple linear regression controlling for preferred language (English or Spanish) and the baseline assessment of that outcome (for all outcomes except those assessed by the ITSEA).
Intent-to-Treat: The researchers followed an intent to treat approach. Missing covariate data were modeled using maximum likelihood estimation.
Outcomes
Implementation Fidelity: Providers were certified in the program before conducting home visits. In addition, providers were required to submit a videotaped recording of one session with each family that was assessed for program fidelity. The majority of videos (94%) met standards for fidelity; providers who did not received additional training. In addition, the majority of mothers in the treatment condition received the full program (82%).
Baseline Equivalence: Table 1 provides descriptive information on demographics by condition at baseline and Table 2 provides descriptive information about outcomes by condition at baseline. However, the researchers do not present significance tests of differences between conditions on any of these measures.
Differential Attrition: The researchers state that there were no significant differences in attrition by condition nor significant baseline-by-condition attrition effects. The researchers also state that "Mothers with incomplete data were more likely to be separated, divorced, or widowed, had lower incomes, and had more children at baseline" (page 8). No statistics are presented with this statement, so it is unclear as to whether or not these are statistically significant differential attrition effects. Also, overall attrition was less than 5%.
Posttest: Results showed significant differences in favor of the treatment group on 1 of 8 child behavioral outcomes, and on 2 of 5 risk and protective factors. Specifically, mothers in the treatment condition, compared to the control condition, reported significantly less externalizing behavior by their infants at 12 months of age (d = .28). In terms of risk and protective factors, mothers in the treatment condition demonstrated significantly better parental sensitivity than mothers in the control condition, both when their infants were 6 months old (d = .25) and when their infants were 12 months old (d = .26). Finally, mothers in the treatment condition reported significantly greater understanding of toddlers than mothers in the control condition, both when their infants were 6 months old (d = .21) and when their infants were 12 months old (d = .45).
Long-Term: Not examined.
Summary
Booth-LaForce et al. (2023) randomly assigned 162 Native caregiver-child dyads living on a rural reservation to the intervention group or control group. Measures of caregiver-child interaction quality, child externalizing behavior, caregiver knowledge of children's social-emotional needs, and caregiver depressive symptoms were assessed at baseline, immediate posttest, and three months after intervention end.
Booth-LaForce et al. (2023) found that, relative to participants in the control group, participants in the intervention group showed significantly:
Evaluation Methodology
Design:
Recruitment: The study was conducted from 2017 to 2019. Researchers recruited Native caregiver-child dyads living on a rural reservation in the Northern Plains of the United States. Families were recruited at community events, from the Special Supplemental Nutrition Program for Women, Infants and Children (WIC), and via flyers/brochures, word of mouth, Facebook, and community programs (e.g., Head Start). Interested individuals were contacted by a Native Research Visitor, who provided detailed information on the study, determined eligibility, and then scheduled a home visit to obtain informed consent and collect baseline data. Family eligibility criteria included at least one parent or guardian who (1) was at least 18 years old and lived on or near the reservation; (2) spoke English; (3) was the primary caregiver for a child aged 10-31 months reported to be Native by their caregiver; (4) had telephone access; (5) was not living in a treatment facility or shelter, nor hospitalized or imprisoned; and (6) was willing to have study staff come to their home. If multiple children in the target age range lived in the household, the caregiver was asked to select one child to participate. A total of 458 families were assessed for eligibility, and 162 agreed to participate.
Assignment: Caregiver-child dyads were randomly assigned to the intervention group (n = 81) or a Resource and Referral (RR) control group (n = 81) after baseline data collection. An algorithm was used to randomize participants, blocking on male or female caregiver, in equal numbers to study conditions. The intervention group received an adapted version of the Promoting First Relationships® program for Native families that, similar to the original version, was delivered across 10-weekly home visits (with an average completion time of 14 weeks). The authors noted that the adaptations (e.g., changing the study name and logo to align with the Tribe's culture and language) did not change the basic format, structure, or principles of the program. The control group service consisted of three phone contacts between a local Native staff member and primary caregivers, and staff also provided families with tailored follow-up information on local services for their needs.
Assessments/Attrition: Assessments were at baseline, immediate posttest, and three months after the intervention end. At both posttest and follow-up time points, overall attrition was 32.7% (n=53).
Sample: Participants were primary caregivers (93% female) and their toddlers (10-31 months old; M = 20.10; 50% female). The majority of the primary caregivers were biological mothers of the study children (75%). Other primary caregivers were a female relative (16%), the biological father (6%), a male relative (<1%), other female (2%), or other male (<1%). Almost all caregivers (96%) self-reported as American Indian or Alaska Native, 62% were married or partnered, and 45% had a high school education or below. All of the children were reported to be American Indian or Alaska Native.
Measures: Study outcomes included independent observational assessments of caregiver-child interaction quality (primary outcome) and non-independent survey measures completed by caregivers (secondary outcomes). Caregiver measures were reports of child externalizing behavior and two risk and protective factors (knowledge of children's social-emotional needs and caregiver depressive symptoms). The measures came from well-known sources, and the authors reported acceptable reliability coefficients for the current sample.
Analysis: Repeated-measures multivariate analyses of covariance (MANCOVAs) were used to test for intervention effects, controlling for baseline outcomes. The authors conducted three sets of outcome analyses to examine overall post-intervention effects, as well as separate effects at posttest and 3-month follow-up.
Missing Data Strategy: The study excluded participants without follow up data (i.e., listwise deletion or complete case analysis) and did not use missing data strategies.
Intent-to-Treat: The researchers conducted an intent-to-treat analysis.
Outcomes
Implementation Fidelity: The authors stated that intervention providers passed every fidelity test, with a mean global rating of 4.88 out of 5 points. All caregivers in the intervention group agreed or strongly agreed that they were satisfied with the program (M = 3.90), and all agreed or strongly agreed that they would recommend the intervention to other caregivers (M = 3.90). In the intervention group, 51% of the participants completed all 10 visits, 45% withdrew from the study before starting the program or receiving only one visit, and 4% dropped out of the study after three or more visits. In the control group, 88% completed all of the calls, and 12% withdrew from the study before or during the implementation period.
Baseline Equivalence: The authors stated that there were no statistically significant differences between the intervention and control groups on any demographic variables at baseline, though no significance tests were presented. There was no mention of tests for baseline outcomes but the means for the intervention and control groups in Table 2 (p. 45) appear similar.
In a separate document provided to Blueprints, the authors performed tests for baseline equivalence. Tests confirmed that there were no significant differences on any of the seven demographic variables; however, one of four tests on outcomes measures indicated that intervention children had significantly higher baseline levels of externalizing than control children (d = .33).
Differential Attrition: The study did not test for differential attrition. However, according to the What Works Clearinghouse standard, based on the overall attrition rate and the condition difference in attrition rates, the potential for attrition bias exists under both the cautious and optimistic thresholds. Attrition rates by condition were 49% for the intervention group and 16% for the control group for a differential rate of 33%.
Again, in the separate document provided to Blueprints, tests for baseline equivalence using data from participants who completed the posttest and follow-up assessments indicated no significant differences other than the higher levels of externalizing for intervention children that existed for the baseline sample (d = .53).
Posttest: After controlling for baseline outcomes, analyses indicated an overall post-intervention effect on observed caregiver-child interaction quality (η2 = .07) but not for child externalizing behaviors. At specific follow-up time points, caregiver-child interaction quality was significantly higher at 3-month follow-up (but not posttest) for the intervention group (η2 = .06) compared to the control group. There were no significant intervention effects on child externalizing behaviors at either posttest or 3-month follow-up.
For risk and protective factors, results indicated a significant overall post-intervention effect on caregiver knowledge of children's social-emotional needs but not for caregiver depressive symptoms. Findings at specific follow-up time points showed that caregivers in the intervention group, compared to those in the control group, had significantly higher knowledge of children's social-emotional needs and less severe depressive symptoms at both posttest and 3-month follow-up.
Exploratory moderation tests showed that the effect of the intervention on caregiver-child interaction quality at 3-month follow-up was stronger for caregivers with low levels of baseline depressive symptoms.
Long-Term: Not examined.