A school-based intervention that aims to reduce drug use and school failure among high-risk adolescents through small group meetings focused on enhancing school attendance, promptness, achievement, and discipline.
Blueprints: Promising
Brenna H. Bry, Ph.D.
Rutgers University
Grad. Sch. of Applied & Professional Psychology
152 Frelinghuysen Road
Piscataway, NJ 08854
bbry@rci.rutgers.edu
Brenna H. Bry, Ph.D.
Rutgers University
Achievement Mentoring (formerly Behavioral Monitoring and Reinforcement Program) is a school-based intervention designed to change the negative school behavior of middle school adolescents. Students meet in small groups and systematically work through behavior change. The intervention consists of four components: (1) Collecting up-to-date information about each student's school-related behavior; (2) Providing systematic feedback to the student and/or the parents about the student's behavior; (3) Attaching point values to the student's behavior to earn incentives; and (4) Helping the student figure out how he/she can earn more points. The program lasts for two years.
Blueprints has not certified the high school adaptation of the program.
Achievement Mentoring is a two-year school-based intervention designed to change the negative school behavior of middle school adolescents. Students meet in small groups and systematically work through behavior change. The intervention consists of four components: (1) Collecting up-to-date information about each student's school-related behavior (from daily attendance, tardiness, and disciplinary action records, and interviews with teachers); (2) Providing systematic feedback to the student and/or the parents about the student's behavior; (3) Attaching point values to the student's behavior (e.g., points for coming to school, not being tardy, receiving no disciplinary action, etc.), with points earned towards an extra school trip; and (4) Helping the student figure out how he/she can earn more points.
Primary Evidence Base for Certification
Study 2
Bry & George (1980) found that, relative to the control group, the intervention group improved significantly more on
Study 3
Bry (1982) found that the intervention group relative to the control group showed significantly
Primary Evidence Base for Certification
Of the three studies Blueprints has reviewed, two (Studies 2 and 3) meet Blueprints evidentiary standards (specificity, evaluation quality, impact, dissemination readiness). Both studies were done by the developer.
Study 2
Bry & George (1980) used a randomized controlled trial to examine 40 seventh-grade students in an urban middle school who were at high risk of school failure. The study randomly assigned the students to the intervention group or a no-treatment control group and assessed the students on academic performance at the end of seventh and eighth grades.
Study 3
Bry (1982) used a randomized controlled trial to examine 58-60 seventh-grade students recruited as part of Studies 1 and 2 (Bry & George, 1979, 1980). The randomly assigned students were assessed at one year and five years after the main program to measure school problems, unemployment, substance use, and delinquency.
Study 2
Bry, B. H., & George, F. E. (1980). The preventive effects of early intervention on the attendance and grades of urban adolescents. Professional Psychology, 11, 252-260.
Study 3
Bry, B. H. (1982). Reducing the incidence of adolescent problems through preventive intervention: One- and five-year follow-up. American Journal of Community Psychology, 10(3), 265-276.
Individual: Antisocial/aggressive behavior, Favorable attitudes towards antisocial behavior
School: Low school commitment and attachment, Poor academic performance*
Individual: Clear standards for behavior, Prosocial behavior
Family: Attachment to parents
School: Rewards for prosocial involvement in school
*
Risk/Protective Factor was significantly impacted by the program
Sample demographics including race, ethnicity, and gender for Blueprints-certified studies:
The Study 2 sample (Bry & George, 1980) consisted of 68% Black students, 32% white students, 60% male students, and 40% female students. The Study 3 sample (Bry, 1982) included 67% male, 33% female, 42% African American, and 58% White students.
Sites interested in implementing the Achievement Mentoring Program purchase a package of services from the Princeton Center for Leadership Training (PCLT) that includes training, curriculum, ongoing technical support, and fidelity monitoring. Specifically, the following services are included:
NOTE: Mentors must participate in the initial mentor implementation training in order to be eligible to provide mentoring to students.
The following includes a more detailed description of the training that is provided:
Stakeholder Meetings with District/Agency Administrators
Initial and follow-up (3-hour) stakeholder meetings with the district/agency administrators and site-based coordinator focus on planning and preparing for program implementation and ensuring that the necessary systems and processes are in place to ensure the mentors are able to participate in mentor training, have time in their schedule to meet with mentees, and are able to access student school records. Another focus of these meetings includes developing a plan to introduce the program to the school faculty and other student support personnel who will play a role in providing the mentors with information about the mentees. In addition, these stakeholder meetings focus on facilitating communication among district/agency administrators, the program/site-based coordinator, and the mentors, as well as assessing program progress, troubleshooting obstacles, and developing a program sustainability plan.
Training for Program Delivery Staff
An initial 2-day implementation training is for the program/site-based coordinator and the adult mentors and focuses on the specific components of the program model, its theoretical basis, and developing the skills and competencies needed to be an effective mentor. It is a program requirement that any individual who intends to mentor students participate in this initial training.
Location of Meetings/Training
The stakeholder meetings and initial implementation training take place at the implementation site or a nearby location. The program developer, Dr. Brenna Bry, or trainers from the Princeton Center for Leadership Training, Dr. Bry's training arm, travel to the implementation site to provide the initial training.
Program Benefits (per individual):
$9,441
Program Costs (per individual):
$1,342
Net Present Value (Benefits minus Costs, per individual):
$8,098
Measured Risk (odds of a positive Net Present Value):
64%
Source: Washington State Institute for Public Policy
All benefit-cost ratios are the most recent estimates published by The Washington State Institute for Public Policy for Blueprint programs implemented in Washington State. These ratios are based on a) meta-analysis estimates of effect size and b) monetized benefits and calculated costs for programs as delivered in the State of Washington. Caution is recommended in applying these estimates of the benefit-cost ratio to any other state or local area. They are provided as an illustration of the benefit-cost ratio found in one specific state. When feasible, local costs and monetized benefits should be used to calculate expected local benefit-cost ratios. The formula for this calculation can be found on the WSIPP website.
A one-time start-up fee for the Achievement Mentoring program is $20,000, plus trainer travel. This includes three days of on-site training for up to 5 mentors, program manuals for 5 mentors, reporting forms, and monthly consultation with mentors for two academic years.
Curriculum and materials are included in the start-up fee listed under initial training and technical assistance.
Materials Available in Other Language: A program brochure and parent letter are available in Spanish, at no additional cost. No other program materials have been translated.
None.
None.
Minimal photocopying of reporting forms required.
When offered by a school, mentors are usually existing school staff members who volunteer their time as mentors. Mentoring usually occurs during the school day. In some instances mentors have been compensated for their time, although this is unusual.
Costs for optional in-school AM program launch luncheon and year-end AM program awards luncheon. In a school-based program, there is no cost for space or transportation. Costs for space and transportation would need to be considered for implementation at a non-school site. Sites may also want to consider providing mentee and mentor incentives.
These costs are included in the start-up fee listed above.
The start-up fee includes fidelity monitoring and program evaluation (mentor and mentee surveys) for the first two years.
None.
No information is available
No information is available
The following example is for a school to initiate the Achievement Mentoring program with 5 mentors and 20 students (4 per mentor).
| Start-up Fee (includes training, materials, on-going consultation, fidelity monitoring and program evaluation) | $20,000.00 |
| Travel - estimated cost varies depending on site location | $3,500.00 |
| Total One Year Cost | $23,500.00 |
Fee also covers second school year of AM program.
For a school to initiate the Achievement Mentoring program with 5 mentors and 20 mentees, the Year 1 cost would be $23,500. This consists of the initial start-up fee and trainer travel. The cost per mentee would be $1175 for the two school years of the program.
Achievement Mentoring would rely on grants and fundraising for financial support. In this case, funds would be needed primarily to pay the one-time start-up cost.
Grant funding from departments of education and juvenile justice agencies at the local and state levels should be explored.
Discretionary Grants: Discretionary grants from a variety of federal agencies should be considered in support of the Achievement Mentoring Program. Department of Education grant opportunities are an obvious source. With its focus on youth behavior, Department of Justice grants should be explored, in particular those issued by the Office of Juvenile Justice and Delinquency Prevention.
Foundation grants, particularly from foundations with a focus on educational achievement, juvenile justice and mentoring, would be excellent sources of funding for Behavioral Monitoring.
Fundraising by a PTA as well as corporate giving should be considered as sources of start-up funding.
All information comes from the responses to a questionnaire submitted by the purveyors of Achievement Mentoring at the Princeton Center for Leadership Training to the Annie E. Casey Foundation.
Brenna H. Bry, Ph.D.Rutgers UniversityGraduate School of Applied Professional Psychology152 Frelinghuysen RoadPiscataway, NJ 8854bbry@rci.rutgers.edu
A school-based intervention that aims to reduce drug use and school failure among high-risk adolescents through small group meetings focused on enhancing school attendance, promptness, achievement, and discipline.
The program targets middle and junior high school students at high risk for increasing school failure experiences. Teens are considered to be at-risk if they meet at least two of three criteria: (1) low academic motivation, (2) family problems, i.e., feeling of distance from the family, and (3) frequent or serious discipline referrals.
Sample demographics including race, ethnicity, and gender for Blueprints-certified studies:
The Study 2 sample (Bry & George, 1980) consisted of 68% Black students, 32% white students, 60% male students, and 40% female students. The Study 3 sample (Bry, 1982) included 67% male, 33% female, 42% African American, and 58% White students.
Risk Factors: low academic motivation, a disregard for rules, and a feeling of distance from the family.
Protective Factors: natural community of reinforcers in society which maintain positive behavior, belief that desired consequences are gained through the subjects' own actions.
Individual: Antisocial/aggressive behavior, Favorable attitudes towards antisocial behavior
School: Low school commitment and attachment, Poor academic performance*
Individual: Clear standards for behavior, Prosocial behavior
Family: Attachment to parents
School: Rewards for prosocial involvement in school
*Risk/Protective Factor was significantly impacted by the program
Achievement Mentoring (formerly Behavioral Monitoring and Reinforcement Program) is a school-based intervention designed to change the negative school behavior of middle school adolescents. Students meet in small groups and systematically work through behavior change. The intervention consists of four components: (1) Collecting up-to-date information about each student's school-related behavior; (2) Providing systematic feedback to the student and/or the parents about the student's behavior; (3) Attaching point values to the student's behavior to earn incentives; and (4) Helping the student figure out how he/she can earn more points. The program lasts for two years.
Blueprints has not certified the high school adaptation of the program.
Achievement Mentoring is a two-year school-based intervention designed to change the negative school behavior of middle school adolescents. Students meet in small groups and systematically work through behavior change. The intervention consists of four components: (1) Collecting up-to-date information about each student's school-related behavior (from daily attendance, tardiness, and disciplinary action records, and interviews with teachers); (2) Providing systematic feedback to the student and/or the parents about the student's behavior; (3) Attaching point values to the student's behavior (e.g., points for coming to school, not being tardy, receiving no disciplinary action, etc.), with points earned towards an extra school trip; and (4) Helping the student figure out how he/she can earn more points.
The program is based on behavior modification theory. It is based on the findings of various research studies indicating that problem behaviors in adolescents are preceded by increases in cynicism about the predictability of the world and decreases in the sense of competence to deal with it. Additionally, this cynicism can be prevented and self-efficacy increased through repeated exposure to environments in which it is clear that desired consequences are gained through the subject's own actions.
Primary Evidence Base for Certification
Of the three studies Blueprints has reviewed, two (Studies 2 and 3) meet Blueprints evidentiary standards (specificity, evaluation quality, impact, dissemination readiness). Both studies were done by the developer.
Study 2
Bry & George (1980) used a randomized controlled trial to examine 40 seventh-grade students in an urban middle school who were at high risk of school failure. The study randomly assigned the students to the intervention group or a no-treatment control group and assessed the students on academic performance at the end of seventh and eighth grades.
Study 3
Bry (1982) used a randomized controlled trial to examine 58-60 seventh-grade students recruited as part of Studies 1 and 2 (Bry & George, 1979, 1980). The randomly assigned students were assessed at one year and five years after the main program to measure school problems, unemployment, substance use, and delinquency.
Primary Evidence Base for Certification
Study 2
Bry & George (1980) found no significant differences on any of the outcomes after one year. After two years, change scores were significantly better for the intervention group than the control group for grades and attendance but not for tardiness or disciplinary actions.
Study 3
Bry (1982) found that at the one-year follow-up, the intervention group relative to the control group showed significantly fewer school problems and lower unemployment. At the five-year follow-up, there were no significant condition differences.
Primary Evidence Base for Certification
Study 2
Bry & George (1980) found that, relative to the control group, the intervention group improved significantly more on
Study 3
Bry (1982) found that the intervention group relative to the control group showed significantly
Two studies meet Blueprints standards for high-quality methods with strong evidence of program impact (i.e., "certified" by Blueprints): Study 2 (Bry & George, 1980), and Study 3 (Bry, 1982). The samples for these studies included middle school students.
Additional Studies (not certified by Blueprints)
Study 1 (Bry & George, 1979)
Bry, B. H., & George, F. E. (1979). Evaluating and improving prevention programs: A strategy from drug abuse. Evaluation and Program Planning, 2, 127-136.
Blueprints: Promising
Brenna H. Bry, Ph.D.
Rutgers University
Grad. Sch. of Applied & Professional Psychology
152 Frelinghuysen Road
Piscataway, NJ 08854
bbry@rci.rutgers.edu
Bry, B. H., & George, F. E. (1979). Evaluating and improving prevention programs: A strategy from drug abuse. Evaluation and Program Planning, 2, 127-136.
Certified
Bry, B. H., & George, F. E. (1980). The preventive effects of early intervention on the attendance and grades of urban adolescents. Professional Psychology, 11, 252-260.
Certified
Bry, B. H. (1982). Reducing the incidence of adolescent problems through preventive intervention: One- and five-year follow-up. American Journal of Community Psychology, 10(3), 265-276.
Summary
Bry & George (1979) used a randomized controlled trial to examine 40 seventh-grade students in two New Jersey middle schools who were at high risk of school failure. The study randomly assigned the students to the intervention group or a no-treatment control group and assessed the students on academic performance at the end of seventh and eighth grades.
Bry & George (1979) found that, relative to the control group, the intervention group improved significantly more on
Evaluation Methodology
Design:
Recruitment: The sample came from seventh-grade students in two New Jersey middle schools. Participating students needed to meet two of three selection criteria as rated by school staff: (1) low academic motivation, (2) family problems, i.e., feeling of distance from the family, or (3) frequent or serious discipline referrals. The cutoffs used to define the selection thresholds differed between the two schools, and 26 students out of a class of 250 in each school met the criteria. The students were matched on class, teacher, and achievement. When the process matched three students, the two most similar were selected, leaving 10 matched pairs in each school and a total sample of 40 students.
Assignment: Each pair member was randomly assigned to the intervention group or a no-treatment control group. Consent followed randomization, with the three non-consenting program students replaced in the pair and a new randomization done within the new pair. The same replacement procedure was used for a program student who was transferred in the first year to a class for emotionally disturbed children. The control group received no special program but could access typical school resources.
Attrition: The first assessment occurred at the end of seventh grade, only partway through the two-year program. This assessment included all randomized students. The second assessment occurred at the end of eighth grade and after a second year of the program (noting, however, that the program changed in one school). The analysis was limited to only one of the schools, which lost one program student. After also dropping the matched pair, the analysis included 18 of the 20 assigned students (90%).
Sample: The sample consisted of 65% males and 83% whites. Most students came from families in the lower-middle and upper-lower classes.
Measures: Measures came from school records, and they included absences, tardy days, grades, and disciplinary actions. Teachers who assigned grades assisted in delivering the program.
Analysis: The analysis examined the median change from baseline to the one-year assessment separately for the two schools. It then examined the change for one of the schools.
Intent-to-Treat: Analyses followed an intent-to-treat approach using all available data.
Outcomes
Implementation Fidelity:
During the first year, the low involvement of teachers and parents at one school warranted some changes. Staff members became more involved in working with teachers and parents. Parent attendance at the bi-yearly meetings was low.
Baseline Equivalence:
Statistical tests showed no differences between the program and control groups on the baseline measures in either school.
Differential Attrition:
Not examined, though attrition occurred in only one school and, given attrition for one student, both the program and control students in a pair were dropped.
Posttest:
In the first year, the tests for changes in the medians showed significant intervention benefits for two of four outcomes in school A (days in school and GPA) but no significant benefits and one significant iatrogenic effect on GPA in school B. Tests for the second year were limited to school B and, after the program was revised to better connect program staff with teachers and parents, the tests showed a significant effect on days in school and no significant iatrogenic effect.
Long-term:
Not examined.
Summary
Bry & George (1980) used a randomized controlled trial to examine 40 seventh-grade students in an urban middle school who were at high risk of school failure. The study randomly assigned the students to the intervention group or a no-treatment control group and assessed the students on academic performance at the end of seventh and eighth grades.
Bry & George (1980) found that, relative to the control group, the intervention group improved significantly more on
Evaluation Methodology
Design:
Recruitment: The study selected 40 seventh graders from a class of 555 students in a large, urban, racially mixed junior high school. Based on consultation with school personnel and examination of the guidance folders of every seventh grader, the research team selected students meeting at least two of the three criteria: (a) low academic motivation, (b) family problems, and (c) frequent or serious discipline referrals. These students were judged to be at risk for school failure.
Assignment: Random assignment of 40 students occurred after matching 20 pairs on classroom assignment, teachers, academic track, sixth-grade attendance, and sixth-grade grades. One member of each matched pair was randomly assigned to the intervention group and the other to the control group, which received no special program but could access other resources ordinarily available within the school system.
Attrition: Outcome assessments of the two-year program occurred at the end of the first year (interim) and the end of the second year (posttest). Five program students transferred to other school systems during the second year of the program. After dropping both students in the matched pair, 15 pairs and 30 students remained (75%) for the analysis.
Sample:
The sample consisted of 68% Black students, 32% white students, 60% male students, and 40% female students. Most students came from upper lower-class families. The selected students had been absent from school for an average of 22 days during sixth grade and had a D+ grade point average.
Measures:
The study examined four outcome measures that were obtained from school records. Teachers who assigned grades assisted in delivering the program.
Each measure was treated as a change score from the baseline (sixth grade) measure, except for times disciplined, which was measured as a change score from seventh to eighth grade because of unavailable sixth-grade data.
Analysis: The analysis used Wilcoxon matched-pairs signed-ranks tests to examine one-year and two-year results.
Intent-to-Treat: Analyses followed an intent-to-treat approach using all randomized pairs with complete data.
Outcomes
Implementation Fidelity:
Not examined.
Baseline Equivalence:
The authors stated that "Statistical analyses of the students' sixth-grade grades and attendance showed no significant differences between the program and control groups prior to the start of the treatment."
Differential Attrition:
The study did not examine differential attrition, but potential attrition bias was minimized by excluding both students in a pair when one student dropped out.
Posttest:
After one year, no significant differences in any of the outcomes were found. After two years, the change scores were significantly better for the intervention group than the control group for grades and attendance but not for tardiness or disciplinary actions.
Long-term:
Not examined.
Summary
Bry (1982) used a randomized controlled trial to examine 58-60 seventh-grade students recruited as part of Studies 1 and 2 (Bry & George, 1979, 1980). The randomly assigned students were assessed at one year and five years after the main program to measure school problems, unemployment, substance use, and delinquency.
Bry (1982) found that the intervention group relative to the control group showed significantly
Evaluation Methodology
Design:
Recruitment: The study used data initially gathered in Studies 1 and 2 (Bry & George, 1979, 1980). The baseline sample thus included 80 seventh graders from two separate program evaluations, one from a low-income, inner-city school (n = 40) and one from two middle-class, suburban schools (n = 40). The students met at least two of three selection criteria: (1) low academic motivation, (2) family problems, i.e., feeling of distance from the family, or (3) frequent or serious discipline referrals.
Assignment: Subjects from the same schools and classrooms with similar sixth-grade records were matched into pairs, and one member of the pair was randomly assigned to the intervention group and the other member to the control group. The control group received no special program but could access the typical resources available within the school system.
Attrition: Post-baseline assessments occurred at one year and five years after the end of the two-year program. However, booster sessions were offered during the third year, making the one-year follow-up a posttest and the five-year follow-up a four-year follow-up. The one-year follow-up assessed 66 of the original 80 randomized students (83%) and obtained outcome measures for 58 (73%). The five-year follow-up obtained data for 60 of the original 80 randomized students (75%). To minimize attrition bias, the study dropped the matched pair of those who had moved outside the school system or refused interviews.
Sample: The one-year sample included 67% males and 33% females (mean age = 15.5 years). More students (55%) came from the suburban schools than the urban school (45%). By race, 42% were African American and 58% were white.
Measures:
For the one-year follow-up, records were used to code the extent of school problems (i.e., suspension, absenteeism, tardiness, and academic failure), and students were interviewed about unemployment, drug use, alcohol use, and criminal behavior (i.e., vandalism, car theft, grand theft, and robbery). Interviewers were unaware of condition, and their ratings of the subject's honesty in reporting showed no significant difference between conditions. However, the authors presented no information on the reliability or validity of the scales they used.
For the five-year follow-up, criminal records were obtained from the county probation department to examine intervention effects on delinquency and drug-related arrests. The authors stated that "the appearance of a young person's name in the county files indicates a serious or chronic involvement with the criminal justice system."
Analysis: The analyses examined condition differences at follow-up without baseline controls and used Wilcoxon matched-pairs signed-ranks tests, chi-square tests, and a Mann-Whitney U test.
Intent-to-Treat: Analyses followed an intent-to-treat approach using all randomized pairs with complete data.
Outcomes
Implementation Fidelity:
Not examined.
Baseline Equivalence:
No statistical differences were found between the conditions in sixth-grade grades or attendance, age, sex, race, or number of contacts with the county court.
Differential Attrition:
The study did not examine differential attrition, but potential attrition bias was minimized by excluding both students in a pair when one student dropped out or refused to be interviewed.
Posttest:
At the one-year follow-up, which actually came after booster sessions and represented a posttest, the intervention group relative to the control group showed significantly (p < .05) fewer school problems and lower unemployment.
Long-Term:
At the five-year follow-up, the intervention failed to significantly affect (p < .05) either drug-related arrests or delinquency.