Fraction Face-Off!
A 12-week math tutoring program to improve the understanding of fractions for at-risk 4th graders through increased instruction on measurement interpretation of fractions.
Program Outcomes
- Academic Performance
Program Type
- Academic Services
- Mentoring - Tutoring
Program Setting
- School
Continuum of Intervention
- Selective Prevention
Age
- Late Childhood (5-11) - K/Elementary
Gender
- Both
Race/Ethnicity
- All
Endorsements
Blueprints: Promising
Program Information Contact
Vanderbilt University
Attn: Fuchs Research Group
230 Appleton Place, PMB #228
110 Magnolia Circle, Suite 418
Nashville, TN 37203-5721
Phone: (615) 343-4782
Email: frg@vanderbilt.edu
Website: https://frg.vkcsites.org/
Program Developer/Owner
Lynn S. Fuchs
Vanderbilt University
Brief Description of the Program
This program seeks to improve the understanding of fractions for at-risk 4th graders through increased instruction on measurement interpretation of fractions. The 12-week program consists of 36 30-minute lessons that take place during the math block, math center, or intervention time. Tutors instruct three students on a sequence of content that focuses primarily on representing, comparing, ordering, and placing fractions on a 0 to 1 number line, but also includes attention to part-whole interpretation and fair shares representation. Four activities comprise each lesson: introduction of concepts and skills, group work, a speed game to build fluency, and individual work. Tutors also work with students to improve task-oriented behavior, teaching students what on-task behavior means, monitoring behavior, and providing students with prizes for on-task behavior.
This program seeks to improve the understanding of fractions for at-risk 4th graders through increased instruction on measurement interpretation of fractions. The 12-week program consists of 36 30-minute lessons that take place during the math block, math center, or intervention time. Tutors instruct three students on a sequence of content that focuses primarily on representing, comparing, ordering, and placing fractions on a 0 to 1 number line, but also includes attention to part-whole interpretation and fair shares representation. Four activities comprise each lesson: introduction of concepts or skills, group work, a speed game to build fluency, and individual work. Tutors also work with students to improve task-oriented behavior, teaching students what on-task behavior means, monitoring behavior, and providing students with prizes for on-task behavior. Full-time or part-time graduate-student employees serve as tutors and are trained to implement the manualized program in a 2-day workshop. Biweekly 1-hr meetings update tutors on upcoming topics and provide time to discuss problems.
Variations on the basic program involve special attention to 1) fluency practice using speed tests for flashcards, or 2) conceptual practice requiring students to explain their reasoning about fractions to the group.
Outcomes
Primary Evidence Base for Certification
Study 1
Fuchs et al. (2013) found at the posttest, compared to the control group, the intervention group improved significantly on:
- compare fractions
- fraction number line
- fraction items
- part-whole interpretation
- measurement interpretation
- fraction calculations
Study 2
Fuchs et al. (2014) found at the posttest, compared to the control group, the intervention group improved significantly on:
- fraction number line
- NAEP (standardized test) total
- fraction calculations
Brief Evaluation Methodology
Primary Evidence Base for Certification
Two studies have been reviewed by Blueprints and both (Study 1 and Study 2) meet Blueprints evidentiary standards (specificity, evaluation quality, impact, dissemination readiness). These studies were conducted by the developer.
Study 1
Fuchs et al. (2013) conducted a randomized controlled trial with 290 4th grade students at risk of having problems with fractions. Students were randomly assigned to an intervention group or to a control group that received usual classroom instruction and remediation classes. Data were collected at baseline and posttest to measure skills with fractions.
Study 2
Fuchs et al. (2014) conducted a randomized controlled trial with 277 4th grade students at risk of having problems with fractions. Students were randomly assigned to three conditions: 1) an intervention group with fluency activities, 2) an intervention group with conceptual activities, or 3) a control group that received usual classroom instruction and remediation classes. Data were collected at baseline and posttest to measure skills with fractions.
Study 1
Fuchs, L. S., Schumacher, R. F., Long, J., Namkung, J., Hamlett, C. L., Cirino, P. T., . . . Changas, P. (2013). Improving at-risk learners' understanding of fractions. Journal of Educational Psychology, 105(3), 683-700.
Study 2
Fuchs, L. S., Schumacher, R. F., Sterba, S. K., Long, J., Namkung, J., Malone, A., . . . Changas, P. (2014). Does working memory moderate the effects of fraction intervention? An aptitude-treatment interaction. Journal of Educational Psychology, 106(2), 499-514. doi:10.1037/a0034341
Risk Factors
School: Poor academic performance
Protective Factors
School: Instructional Practice
*
Risk/Protective Factor was significantly impacted by the program
See also: Fraction Face-Off! Logic Model (PDF)
Subgroup Analysis Details
Subgroup differences in program effects by race, ethnicity, or gender (coded in binary terms as male/female) or program effects for a sample of a specific race, ethnic, or gender group.
Studies 1 (Fuchs et al., 2013) and 2 (Fuchs et al., 2014) found subgroup effects by using a homogenous sample with 75% or more of economically disadvantaged students who were eligible for free- and reduced-price lunches.
Sample demographics including race, ethnicity, and gender for Blueprints-certified studies:
Studies 1 (Fuchs et al., 2013) and 2 (Fuchs et al., 2014) included 4th graders who were at risk of having problems learning fractions.
- The sample for Study 1 was approximately 52% African American, 25% White, 19% Hispanic, and 52% male.
- The sample for Study 2 was approximately 59% African American, 23% Hispanic, 16% White, and 61% female.
Tutor training takes place in a 1-day workshop (typically 7 hours) at the district or school site. Appropriate workshop length may be discussed with the trainer at the time of scheduling. In the one-day training workshop for tutors, (a) an overview of the tutoring program, goals, and topics is presented, and (b) tutoring procedures are modeled and practiced for each activity in the first set of tutoring topics. Following demonstration by the trainer, tutors practice techniques and activities in pairs and receive feedback. Additional consultation with the trainer is available by email or phone following training. Instructors may be certified teachers or paraprofessionals.
The training agenda includes the following topics and activities:
- Fraction Face-Off! Program and RTI Overview
- Walk-Through of Manuals, Materials, Program Sequence
- Use of Behavior Management and Point System
- Implementation Fidelity
- Trainer Modeling of Lesson Content and Activities
- Tutor Practice of Lesson Content and Activities, with Feedback Provided by Trainer
- Availability of Technical Support
Source: Washington State Institute for Public Policy
All benefit-cost ratios are the most recent estimates published by The Washington State Institute for Public Policy for Blueprint programs implemented in Washington State. These ratios are based on a) meta-analysis estimates of effect size and b) monetized benefits and calculated costs for programs as delivered in the State of Washington. Caution is recommended in applying these estimates of the benefit-cost ratio to any other state or local area. They are provided as an illustration of the benefit-cost ratio found in one specific state. When feasible, local costs and monetized benefits should be used to calculate expected local benefit-cost ratios. The formula for this calculation can be found on the WSIPP website.
Start-Up Costs
Initial Training and Technical Assistance
Training of tutors is provided through the American Institutes for Research (AIR). Visit the Fuchs Tutoring Training Request page to sign-up. You will need to indicate the type of training (e.g., reading or math), the grade level requested, the number of people to be trained, and your location. You will be contacted by a trainer and can negotiate dates. More information about training can be found here. Cost are negotiated directly with AIR.
Curriculum and Materials
Scripted manuals are required for program implementation. Fraction Face-Off! Manuals are available for $75-85. Implementation Ready Packs (IRPs) can be purchased separately for $75. These include materials for 12 students with templates available in the manual to make additional copies.
$25 estimated cost per student to photocopy supporting materials.
$15 per tutor for supplemental materials (Fraction Tiles).
$10 per student for prizes to stock "fractions store", where students apply fraction knowledge to purchase prizes.
Note: Prizes are up to the tutor/school. In evaluation studies, popular prizes included sports-themed pens, pencils and erasers, card games (e.g., Old Maid and Hearts), small beach balls, play jewelry, and jump ropes.
Licensing
None.
Other Start-Up Costs
No information is available
Intervention Implementation Costs
Ongoing Curriculum and Materials
$25 estimated cost per student for photocopies of supporting materials.
$10 per student for prizes to stock "fractions store", where students apply fraction knowledge to purchase prizes.
Staffing
School-based tutoring may be provided by student instructional support services personnel. If school personnel are not available, salaries for tutors would be needed.
Qualifications: Program generally is delivered by teachers or paraprofessionals who are supervised by teachers. Schools may fill these positions with student instructional support staff, substitute teachers, bachelor's level paraprofessionals, graduate students (typically education/social sciences), or school volunteers. Classroom teachers may implement the program in small groups outside of mathematics instructional time.
Ratios: The program is delivered in small groups of 2-3 students per tutor. A typical scenario may be a school with 4 first-grade classrooms and 4-5 small tutoring groups that are covered by 2 tutors.
Other Implementation Costs
No information is available
Implementation Support and Fidelity Monitoring Costs
Ongoing Training and Technical Assistance
Initial training, delivered by the American Institutes for Research, prepares tutors to implement the program as designed. It is recommended that sites include back-up tutors in the initial training if tutor turnover is anticipated. Training costs would be incurred if new tutors are hired following the initial training workshop for the site.
Fidelity Monitoring and Evaluation
Fidelity monitoring may be completed by school or district personnel.
Ongoing License Fees
None.
Other Implementation Support and Fidelity Monitoring Costs
No information is available
Other Cost Considerations
No information is available
Year One Cost Example
In this example, Fraction Face-off! is implemented in a school district with four elementary schools each with four fourth grade classrooms. Three students in each classroom are identified as at-risk for mathematics difficulty and placed in the program. Two tutors per school (8 in all) serve 48 students district wide. It is assumed that the tutors are part of the school-based instructional staff.
Costs provided do not include training of tutors. Contact AIR to determine training costs.
| Implementation manuals @ $85/tutor | $680.00 |
| Implementation ready packs @ $75/school | $300.00 |
| Photocopies of supporting materials @ $25/student | $1,200.00 |
| Supplemental materials @ $15/tutor | $120.00 |
| Prizes for fractions store @ $10/student | $480.00 |
| Total One Year Cost | $2,780.00 |
With 48 students in the Fraction Face-Off! program, the Year 1 cost per student is $57.92 plus the additional cost of training 8 tutors through AIR.
Funding Overview
No information is available
Maximizing Federal Funds
Formula Funds:
- The Individuals with Disabilities Education Act (IDEA) 2004 provides funding to school districts for early intervening services (EIS) that assist students who need increased academic and behavioral support to succeed in a general education environment. As such, IDEA 2004 is a potential source of funding for Fraction Face-off!
- Title I funds may be used for Fraction Face-off! Schools with at least a 40 percent poverty rate may choose to operate a schoolwide program that allows Title I funds to be combined with other federal, state and local funds to upgrade the school's overall instructional program. All other schools must operate targeted assistance programs for the most at-risk students; these programs must supplement the regular education program normally provided by state and local educational agencies.
Foundation Grants and Public-Private Partnerships
The National Council of Teachers of Mathematics offers funding to "increase the breadth and depth of mathematics content knowledge" of elementary school teachers committed to mathematics teaching and learning. If a school or district wishes to train teachers in Fraction Face-off! tutoring, this may be a potential source of funding.
President Obama's 2015 budget includes investments to improve science, technology, engineering, and mathematics (STEM) education in K-12 schools, with a priority on excellent teachers, rigorous courses, and regional partnerships that enable school districts to partner with local employers, museums, universities, and others to transform STEM teaching and learning by developing coordinated plans to promote student inspiration, achievement, and preparation in STEM subjects.
Program Developer/Owner
Lynn S. FuchsVanderbilt University230 Appleton Place, PMB 228Nashville, TN 37203U.S.A.lynn.fuchs@vanderbilt.edu
Program Outcomes
- Academic Performance
Program Specifics
Program Type
- Academic Services
- Mentoring - Tutoring
Program Setting
- School
Continuum of Intervention
- Selective Prevention
Program Goals
A 12-week math tutoring program to improve the understanding of fractions for at-risk 4th graders through increased instruction on measurement interpretation of fractions.
Population Demographics
Targets 4th graders with low scores on a broad-based calculations assessment.
Target Population
Age
- Late Childhood (5-11) - K/Elementary
Gender
- Both
Race/Ethnicity
- All
Subgroup Analysis Details
Subgroup differences in program effects by race, ethnicity, or gender (coded in binary terms as male/female) or program effects for a sample of a specific race, ethnic, or gender group.
Studies 1 (Fuchs et al., 2013) and 2 (Fuchs et al., 2014) found subgroup effects by using a homogenous sample with 75% or more of economically disadvantaged students who were eligible for free- and reduced-price lunches.
Sample demographics including race, ethnicity, and gender for Blueprints-certified studies:
Studies 1 (Fuchs et al., 2013) and 2 (Fuchs et al., 2014) included 4th graders who were at risk of having problems learning fractions.
- The sample for Study 1 was approximately 52% African American, 25% White, 19% Hispanic, and 52% male.
- The sample for Study 2 was approximately 59% African American, 23% Hispanic, 16% White, and 61% female.
Other Risk and Protective Factors
Risk factor: low fraction proficiency
Risk/Protective Factor Domain
- Individual
Risk/Protective Factors
Risk Factors
School: Poor academic performance
Protective Factors
School: Instructional Practice
*Risk/Protective Factor was significantly impacted by the program
See also: Fraction Face-Off! Logic Model (PDF)
Brief Description of the Program
This program seeks to improve the understanding of fractions for at-risk 4th graders through increased instruction on measurement interpretation of fractions. The 12-week program consists of 36 30-minute lessons that take place during the math block, math center, or intervention time. Tutors instruct three students on a sequence of content that focuses primarily on representing, comparing, ordering, and placing fractions on a 0 to 1 number line, but also includes attention to part-whole interpretation and fair shares representation. Four activities comprise each lesson: introduction of concepts and skills, group work, a speed game to build fluency, and individual work. Tutors also work with students to improve task-oriented behavior, teaching students what on-task behavior means, monitoring behavior, and providing students with prizes for on-task behavior.
Description of the Program
This program seeks to improve the understanding of fractions for at-risk 4th graders through increased instruction on measurement interpretation of fractions. The 12-week program consists of 36 30-minute lessons that take place during the math block, math center, or intervention time. Tutors instruct three students on a sequence of content that focuses primarily on representing, comparing, ordering, and placing fractions on a 0 to 1 number line, but also includes attention to part-whole interpretation and fair shares representation. Four activities comprise each lesson: introduction of concepts or skills, group work, a speed game to build fluency, and individual work. Tutors also work with students to improve task-oriented behavior, teaching students what on-task behavior means, monitoring behavior, and providing students with prizes for on-task behavior. Full-time or part-time graduate-student employees serve as tutors and are trained to implement the manualized program in a 2-day workshop. Biweekly 1-hr meetings update tutors on upcoming topics and provide time to discuss problems.
Variations on the basic program involve special attention to 1) fluency practice using speed tests for flashcards, or 2) conceptual practice requiring students to explain their reasoning about fractions to the group.
Theoretical Rationale
This program is based on a model of mathematics learning developed by others, and centers on conceptual understanding with the major focus on the measurement interpretation of fractions.
Theoretical Orientation
- Skill Oriented
Brief Evaluation Methodology
Primary Evidence Base for Certification
Two studies have been reviewed by Blueprints and both (Study 1 and Study 2) meet Blueprints evidentiary standards (specificity, evaluation quality, impact, dissemination readiness). These studies were conducted by the developer.
Study 1
Fuchs et al. (2013) conducted a randomized controlled trial with 290 4th grade students at risk of having problems with fractions. Students were randomly assigned to an intervention group or to a control group that received usual classroom instruction and remediation classes. Data were collected at baseline and posttest to measure skills with fractions.
Study 2
Fuchs et al. (2014) conducted a randomized controlled trial with 277 4th grade students at risk of having problems with fractions. Students were randomly assigned to three conditions: 1) an intervention group with fluency activities, 2) an intervention group with conceptual activities, or 3) a control group that received usual classroom instruction and remediation classes. Data were collected at baseline and posttest to measure skills with fractions.
Outcomes (Brief, over all studies)
Primary Evidence Base for Certification
Study 1
Fuchs et al. (2013) reported significant program effects on each of six fraction outcomes at posttest (compare fractions, fraction number line, fraction items, part-whole interpretation, measurement interpretation, and fraction calculations) for the intervention group, compared to the control group.
Study 2
Fuchs et al. (2014) reported significant program effects on three outcomes at posttest (fraction number line, NAEP [standardized test] total, and fraction calculations).
Outcomes
Primary Evidence Base for Certification
Study 1
Fuchs et al. (2013) found at the posttest, compared to the control group, the intervention group improved significantly on:
- compare fractions
- fraction number line
- fraction items
- part-whole interpretation
- measurement interpretation
- fraction calculations
Study 2
Fuchs et al. (2014) found at the posttest, compared to the control group, the intervention group improved significantly on:
- fraction number line
- NAEP (standardized test) total
- fraction calculations
Mediating Effects
In Study 1, Fuchs et al. (2013) reported significant mediation for two of the three theoretically specified pathways. The study concluded that improvement in measurement interpretation partially mediated the intervention effect on fraction items, and improvement in measurement completely mediated the program effect on part-whole interpretation. Part-whole interpretation improvement did not mediate measurement interpretation.
In Study 2, Fuchs et al. (2014) found that improvement in fraction number line scores mediated the effect of both the fluency and conceptual intervention on the outcome of NAEP total score. The indirect effect of .12 for the fluency condition (compared to the control condition) was significant, and the indirect effect of .19 for the conceptual condition (compared to the control condition) was significant.
Effect Size
Effect sizes were small to large in Study 1 (Fuchs et al., 2013), ranging from .29 to 2.50, and were medium to large in Study 2 (Fuchs et al., 2014), ranging from .60 to 1.13.
Generalizability
Two studies meet Blueprints standards for high quality methods with strong evidence of program impact (i.e., "certified" by Blueprints): Study 1 (Fuchs et al., 2013) and Study 2 (Fuchs et al., 2014). The samples for these studies included 4th grade students who were at risk of having problems learning fractions. The Study 1 participants were enrolled in 13 schools and the Study 2 participants were enrolled in 14 schools; location was not reported for either study. In both studies, the treatment group was compared to a control group receiving usual classroom instruction.
Endorsements
Blueprints: Promising
Peer Implementation Sites
Jenna Noonan Davis
4th-Grade Teacher
Glenview Elementary
1020 Patricia Dr.
Nashville, TN 37217
Jenna.Noonan@mnps.org
615-360-2906
Program Information Contact
Vanderbilt University
Attn: Fuchs Research Group
230 Appleton Place, PMB #228
110 Magnolia Circle, Suite 418
Nashville, TN 37203-5721
Phone: (615) 343-4782
Email: frg@vanderbilt.edu
Website: https://frg.vkcsites.org/
References
Study 1
Certified Fuchs, L. S., Schumacher, R. F., Long, J., Namkung, J., Hamlett, C. L., Cirino, P. T., . . . Changas, P. (2013). Improving at-risk learners' understanding of fractions. Journal of Educational Psychology, 105(3), 683-700.
Study 2
Certified Fuchs, L. S., Schumacher, R. F., Sterba, S. K., Long, J., Namkung, J., Malone, A., . . . Changas, P. (2014). Does working memory moderate the effects of fraction intervention? An aptitude-treatment interaction. Journal of Educational Psychology, 106(2), 499-514. doi:10.1037/a0034341
Study 1
Summary
Fuchs et al. (2013) conducted a randomized controlled trial with 290 4th grade students at risk of having problems with fractions. Students were randomly assigned to an intervention group or to a control group that received usual classroom instruction and remediation classes. Data were collected at baseline and posttest to measure skills with fractions.
Fuchs et al. (2013) found at the posttest, compared to the control group, the intervention group improved significantly on:
- compare fractions
- fraction number line
- fraction items
- part-whole interpretation
- measurement interpretation
- fraction calculations
Evaluation Methodology
Design: This randomized controlled trial determined effects of a tutoring program for 4th grade children at risk of having problems learning fractions, defined as scoring below the 35th percentile on a broad-based calculations screening assessment. The study reported sampling 2-8 students per classroom, stratified by whether or not the student scored below the 15th percentile on the screening assessment. Eighteen students scoring below the 9th percentile on both subtests of the Wechsler Abbreviated Scales of Intelligence were excluded, resulting in a sample of 290 students. Presumably the study sampled all classrooms in participating schools, but only specified that the 290 students came from 53 fourth grade classrooms in 13 schools. Control group students received the usual classroom instruction and remediation classes, and did not receive any small-group tutoring. Intervention lessons were provided during instructional periods such that math instructional time was similar for control and treatment students.
Of the original 290 students, 31 (10.7%) either moved before the end of the study or had missing data. Screening was conducted in August and September, and pretests were given in September and October. The intervention took place from late October to late March, and posttests were administered in early April (within 2 weeks of intervention ending).
Sample Characteristics: The sample consisted of 4th grade children who scored below the 35th percentile on a broad-based calculations assessment. The sample was mostly African American (51-54%), followed by white (24-26%), Hispanic (19%), and other race/ethnicity (3-4%). A large proportion of students (81-83%) received subsidized lunch, and between 9 to 12% of students were English language learners. Half (50-54%) the sample was male. A small number (5%) of children received special education.
Measures: The study used the following outcome measures, collected at pretest and posttest:
- Comparing fractions, from the 2010 Fraction Battery, developed by others. This measure assesses magnitude understanding with 15 items (alpha=.84).
- Fraction number line, developed by others. This measure assesses measurement interpretation of fractions by requiring students to place fractions on a number line. Low scores indicate stronger performance. Test-retest reliability was .79.
- NAEP total fraction items, taken from fraction items from the 1990-2009 NAEP, a standardized test developed by others. This 18-item outcome had an alpha of .72.
- Part-whole interpretation, taken from eight items in the 1990-2009 NAEP, a standardized test developed by others (alpha=.60).
- Measurement interpretation, taken from eight items in the 1990-2009 NAEP, a standardized test developed by others (alpha=.62).
- Fraction calculations, a combination of Fraction Addition and Fraction Subtraction problems in the 2010 Fraction Battery, developed by others. This 20-item measure had an alpha of .90.
The study collected the following potential moderators at pretest, except attentive behavior, which was collected 6 weeks into intervention:
- Listening Recall, taken from the Working Memory Test Battery for Children, developed by others. Test-retest reliability ranged from .84 to .93.
- Counting Recall, taken from the Working Memory Test Battery for Children, developed by others. Test-retest reliability ranged from .84 to .93.
- Attentive behavior, taken from the inattentive subscale of the Strength and Weaknesses of ADHD-Symptoms and Normal-Behavior, developed by others (alpha=.96). This measure was reported by teachers.
- Processing speed, taken from the Woodcock-Johnson III Cross Out measures, developed by others (alpha=.91).
- Listening comprehension, taken from the Woodcock Diagnostic Reading Battery-Listening Comprehension, developed by others (alpha=.80).
- Whole number calculation skills, from the WRAT-4-Arithmetic, developed by others (alpha=.85).
Analysis: Ordinary least squares path analytical framework estimated intervention effects. Models controlled for baseline outcome levels, and baseline whole-number calculation skill score. For four outcomes (comparing fractions, fraction number line, part-whole interpretation, and fraction calculations) not meeting the homogeneity of regression assumption, an interaction between intervention condition and baseline outcomes was also included as a control. The study specified the reasoning behind testing homogeneity (p.690): "because we relied on a residualized change approach to analyze intervention effects (i.e., covarying pretest scores to reduce within-group error variance), we assessed the homogeneity of regression assumption".
The theoretically specified moderation analyses used the controls described above, but also included a variable for the moderator (listening recall, counting recall, attentive behavior, processing speed, or listening comprehension) and for the interaction between the moderator and condition status. When more than one moderator was significant for a given fraction outcome, the study re-examined each moderator while controlling for performance on the other significant moderator variable. It appears that the study checked models combining all moderators and outcomes, with only significant results reported.
The study reported the coefficients for intervention as main effects, but did not detail whether interaction terms including condition status should change interpretation of main effects.
The study also examined three theoretically specified mediation pathways; it determined whether improvements on fraction number line, measurement interpretation, or part-whole interpretation mediated NAEP total, part-whole interpretation, or measurement interpretation outcomes, respectively. This analysis used two steps: (1) effects of intervention on the mediator, controlling for the covariate(s), (2) effects of intervention on the outcome, controlling for the same covariates(s) and the mediator. All models controlled for the baseline outcome and the mediation model looking at NAEP total as the outcome also used fraction calculations as a covariate. For the indirect (mediation) effect, the study used bootstrapping estimation with 5,000 draws to estimate standard errors and 95% confidence intervals.
The study appears to follow intent-to-treat, as all students who did not move and who had complete data were included in the analyses. It does not appear, however, that the study attempted to follow movers.
Randomization and analysis were at the individual level. Students were clustered in classrooms, and the study determined that accounting for this clustering did not alter the results. Intra-class correlations (0 to .063) indicated that little to no proportion of variance was due to classrooms. The study reported that supplemental analyses using multilevel models demonstrated the same results as the single-level models, and multilevel results are available from the first author.
Outcomes
Implementation Fidelity: Each session was audiotaped, and research assistants listened to a random 20% of these tapes. The mean percentage of points addressed during those sessions was 97.69, suggesting high fidelity.
Baseline Equivalence: The intervention and control groups did not significantly differ on the broad-based calculations assessment, or any of the demographic measures (sex, English language learners, subsidized lunch status, special education, and race). The study reported that pretest outcome scores across conditions were comparable and provided means that looked similar, but it did not indicate results of a significance test.
Differential Attrition: The study stated that the 31 students who moved before the end of the study or had missing data did not significantly differ on pretest measures from the other 259 students.
Posttest: The study reported significant program effects for each of the six fraction outcomes at posttest (compare fractions, fraction number line, NAEP total, part-whole interpretation, measurement interpretation, and fraction calculations). Effect sizes ranged from .29 to 2.50.
Moderation: Models looking at main effects also provided information on moderating effects. First, interaction terms for whole-number calculation skill scores were nonsignificant for all six outcomes. Second, interactions with baseline levels were significant for the four outcomes that included the terms. The study did not interpret these results, but the interaction coefficients in the table indicate that intervention effects were weaker for those with higher pretest scores on compare fractions, part-whole measurement, and fraction calculations, but stronger for those with higher pretest scores for fraction number line.
Third, additional moderation analyses examined five moderators for six outcomes (30 models), and six significant effects emerged. Program effects were stronger for the compare fractions outcome among those with higher listening recall or attentive behavior and for the NAEP total outcome among those with higher attentive behavior. Conversely, the effect of the intervention was weaker on fraction number line for participants with higher counting recall and on fraction calculations for those with higher listening comprehension or processing speed. Table 5 (p. 693) reported listening recall as a moderator for compare fractions and count recall for number line, but this appears to be a mistake, as the text reports, in multiple places, that count recall is a moderator for compare fractions and listen recall for number line.
Mediation: Significant mediation was reported for two of the three pathways examined. The study concluded that improvement in measurement interpretation partially mediated the intervention effect on NAEP total, and improvement in measurement completely mediated the program effect on part-whole interpretation. Part-whole interpretation improvement did not mediate measurement interpretation.
Study 2
Summary
Fuchs et al. (2014) conducted a randomized controlled trial with 277 4th grade students at risk of having problems with fractions. Students were randomly assigned to three conditions: 1) an intervention group with fluency activities, 2) an intervention group with conceptual activities, or 3) a control group that received usual classroom instruction and remediation classes. Data were collected at baseline and posttest to measure skills with fractions.
Fuchs et al. (2014) found at the posttest, compared to the control group, the intervention group improved significantly on:
- fraction number line
- NAEP (standardized test) total
- fraction calculations
Evaluation Methodology
Design: This randomized controlled trial determined effects of a tutoring program for 4th grade children at risk of having problems learning fractions, defined as scoring below the 35th percentile on a broad-based calculations screening assessment. The study reported sampling 2-8 students per classroom, stratified by whether or not the student scored below the 15th percentile on the screening assessment. Eighteen students scoring below the 9th percentile on both subtests of the Wechsler Abbreviated Scales of Intelligence were excluded, resulting in a sample of 277 students from 49 classrooms in 14 schools. Control group students received the usual classroom instruction and remediation classes, and did not receive any small-group tutoring. Intervention lessons were provided during instructional periods such that math instructional time was similar for control and treatment students.
Students were randomly assigned at the individual level to three conditions: 1) treatment with fluency activities, 2) treatment with conceptual activities, and 3) control. Another 281 low-risk classmates (> 34th percentile) were randomly sampled to serve as a comparison group but were not used to test program efficacy.
Of the original 277 students, 34 (12.3%) moved before the end of the study. The analytic sample thus had data on 243 students (84 fluency practice; 79 conceptual practice; and 80 control). Screening was conducted in August and September and pretests were given in September and October. The intervention took place from late October to late March, and posttests were administered in early April (within 2 weeks of intervention ending).
Sample Characteristics: The sample consisted of 4th grade children who scored below the 35th percentile on a broad-based calculations assessment. The sample was mostly African American (58-61%), followed by Hispanic (22-24%) and white (14-17%). A large proportion of students (86-95%) received subsidized lunch, and 5-14% of students were English language learners. There were more girls (59-63%) than boys. A small portion of children (8-12%) received special education.
Measures:
Screening Measures
- WRAT-4 Arithmetic. Used to screen students below the 35th percentile, this measure has students complete calculations of increasing difficulty and had an alpha of .85 for the sample.
- Wechsler Abbreviated Scale of Intelligence. This measure assessed vocabulary and matrix reasoning and had an alpha exceeding .92.
Moderator Measures
- Working Memory. The measure comes from listening recall tasks, with more correct responses indicating greater working memory. The test-retest reliabilities ranged from .84 to .93.
- Attentive Behavior. The measure comes from the inattentive subscale of the Strength and Weaknesses of ADHD-Symptoms and Normal-Behavior, developed by others (alpha=.96). This measure was reported by teachers.
- Processing Speed. Students are given tasks to complete in 3 minutes; the measure has a reliability of .91.
- Language. This measure of vocabulary has a split-half reliability of .86-.87 and correlates highly (r = .72) with the Wechsler Intelligence Test.
Outcome Fraction Measures
- Fraction Number Line. This measure assesses measurement interpretation of fractions by requiring students to place fractions on a number line. Low scores indicate stronger performance. Test-retest reliability was .80.
- NAEP Total Fraction Items. The fraction items come from the 1990-2009 NAEP, a standardized test developed by others. This 18-item outcome had an alpha of .81.
- Fraction Calculations. The measure combines Fraction Addition and Fraction Subtraction problems in the 2010 Fraction Battery, developed by others. This 20-item measure had an alpha of .93.
The study collected the moderators at pretest, except attentive behavior, which was collected 6 weeks into intervention. Testers were blind to condition when administering and scoring the tests.
Analysis . The study estimated cross-classified partially nested multilevel models for each of the three outcomes. The models accounted for nesting at the classroom level for all three conditions and at the small-group tutoring level for the two intervention conditions. That is, students (Level 1 units) were partially nested and cross classified in small groups (Level 2a unit, occurring only in the treatment arms) and classrooms (Level 2b unit). Classroom ICCs ranged from .04 to .07, while small-group ICCs for treatment subjects ranged from 0 to .22. Restricted maximum likelihood estimates with adjustments of standard errors for clustering came from PROC MIXED in SAS. However, the paper also states that tests for moderation used an ordinary least squares framework.
The models controlled for baseline outcomes.
The study appears to follow intent-to-treat, as all students who did not move were included in the analyses. It does not appear, however, that the study attempted to follow movers.
Outcomes
Implementation Fidelity: Each session was audiotaped, and research assistants listened to a randomly selected 20% of these tapes. The mean percentage of points addressed during those sessions was 97.3% in the fluency condition and 97.0% in the conceptual condition.
Baseline Equivalence: The three conditions did not differ significantly (p > .05) on the Wechsler Intelligence Test or on sociodemographic characteristics, including gender, English learners, subsidized lunch, special education, and race.
The study noted that the three groups were comparable at baseline on the three outcome measures. Differences in scores for fraction number line, NAEP, and fraction calculations were not significant (p >.05). Effect sizes for the three measures for pretest comparisons were under .17 with one exception - the fluency condition had higher scores than the conceptual condition on fraction calculations (d = .31).
Differential Attrition: The study stated that the 34 students who moved before the end of the study did not differ statistically from the remaining 243 students on pretest measures or on pretest measure as a function of condition.
Posttest: For all three outcome variables - fraction number line, NAEP total, and fraction calculations - the two intervention groups scored significantly better than the control group, even after a Bonferroni adjustment (p < .001). Effect sizes calculated to take account of the partial nested design ranged from .60 to 1.12 for the fluency condition and from .63 to 1.13 for the conceptual condition. The fluency and conceptual conditions did not differ significantly from one another.
Mediation: The results showed that improvement in fraction number line scores mediated the effects of both the fluency and conceptual interventions on the outcome of NAEP total. The indirect effect of .12 for the fluency condition (compared to the control condition) was significant, and the indirect effect of the conceptual condition (compared to the control condition) of .19 was also significant.
Moderation: The model used fraction number line score as the outcome but appears to use only subjects in the two intervention conditions (not the control condition) (N = 163). The models focused on the interaction between working memory (WM) and the fluency intervention relative to the conceptual intervention. The interaction results showed that "for students with very low WM, effects favored the conceptual condition; however, for students with more adequate WM, effects favored the fluency condition." Interactions involving attentive behavior, processing speed, and language were not statistically significant. The results were in the opposite direction of those predicted by the hypothesis - fluency tasks would work better for those with poorer working memory.