Peer Assisted Learning Strategies (PALS)

Study 1

Summary

Fuchs et al. (1999) conducted a cluster randomized controlled trial in which 24 teachers were assigned to the standard intervention (PALS), the intervention with preparation in help-giving strategies (PALS-HG), or to a control group with no collaborative reading activities. Teachers delivered their respective treatments to the entire class; however, only three students per class - one at risk, one average achieving, and one high achieving - were included in the evaluation. Reading skills were measured at baseline and posttest.

Fuchs et al. (1999) found, in comparison to the control group at posttest, students in the intervention group had significantly better:

• Reading skills (grades 2 and 3 of the PALS group)
• Reading skills (grade 4 of the PALS-HG group)

Evaluation Methodology

Design:

Recruitment:
It is unclear from which pool the 24 teachers were chosen (1 or more schools) or how the recruitment was conducted.
Study type/Randomization:
This study constitutes a randomized control trial. Randomization was conducted at the classroom-level by randomly assigning teachers to intervention vs. control group.
Teachers: Participants were 15 general education teachers at primary grades 2 and 3 and 9 general education teachers at intermediate grade 4 (all teachers N=24). Stratified by grade level, teachers were randomly assigned to two treatments: collaborative reading activities, operationalized with peer-assisted learning strategies (PALS) (N=10 at grades 2-3; N=6 at grades 4) or a control group with no collaborative reading activities (N=5 at grades 2-3; N=3 at grade 4). Then half the PALS teachers were randomly assigned to a treatment that included preparation in help-giving strategies (PALS-HG). Thus, two experimental groups (PALS and PALS-HG) and one control group was established.
Students: Teachers implemented their respective treatments with all students in their reading classes. However, only for a selection of students, treatment effects were assessed. Each teacher identified three students: (a) one at risk (AR) student who, as judged by the teacher, was manifesting social behavior difficulties that permeated the school day, including reading instruction, (b) one average achieving (AA) student who demonstrated no chronic behavior problems and whose reading performance, as judged by the teacher, was near the middle of the class, and (c) one high achieving (HA) student who also demonstrated no chronic behavior problems and whose reading performance as judged by the teacher, was near the top of the class.

Intervention/Attrition:
PALS sessions comprised three activities: partner reading, summarization, and prediction. For 21 weeks teachers incorporated 3 PALS sessions (35 minutes) into their regular teaching activities. Teachers taught the PALS lessons to and conducted PALS with all children in their naturally constituted classes. Compared to PALS, PALS-HG teachers taught their students specific strategies designed to help partners figure out correct responses on their own, rather than directly correcting the partner. These elaborated help-giving (HG) strategies include, for example, helping reader to use context, helping reader use the vowel sounds, provide additional assistance in the summarizing exercise through asking targeted questions, etc. (see p.209, table 4).

Assessment:
All assessments were conducted by trained research assistants. Reading achievement tests were administered immediately before and after treatment. Observations to characterize the nature of help students provided to peers during PALS and PALS-HG were conducted during weeks 22 and 23 of the study.

Sample characteristics:
Sample characteristics are not described in the text.

Measures:
Validity of measurements:
Inter-rater reliability was assessed. The agreement between independent observers was usually high (around 80%).

Primary outcomes:
The study employed the reading comprehension subtest of the third edition of the Stanford Diagnostic Reading Test (SDRT). For grade 2 children, the SDRT included 28 items that require sentence reading and 20 items that involved paragraph comprehension (overall Kuder-Richardson reliability=.93). For grades 3 and 4, the measures included 24 literal and 24 inferential comprehension items (overall Kuder-Richardson reliability=.94).

In addition, a structured observation system was used to assess the nature of help students provided to peers. RAs counted the errors corrected and the frequency that tutors relied on HG (help giving) strategies.

Analysis:
The unit of analysis was classroom, not students. The study uses analysis of variance to contrast, at the between-classroom level, the three study groups (PALS-HG vs. PALS vs. control) and to evaluate within-classrooms differences between the three student types (AR vs. AA vs. HA). To follow up significant effects, the authors used the Fisher LSD post hoc procedure to evaluate pairwise comparisons. Since changes in scores were evaluated, baseline controls are included in the models.

Intention-to-treat: The study setup does not allow judgement of whether the intent-to-treat principle was followed, but it appears that the three students selected at baseline for each classroom were used at posttest.

Outcomes

Implementation fidelity:
A research assistant (RA) was assigned to each teacher, who served as consultant to the teacher during the study. Teachers participated in a full-day workshop, conducted separately for PALS and PALS-HG teachers. In addition, different RAs (who were not assigned as consultants) observed and rated teachers' accuracy of implementing PALS. For PALS and PALS-HG, the percentage of correctly implemented program elements was 98.2 and 97.5 respectively.

Baseline Equivalence:
Chi-square tests indicated no relation between treatment and students' race, free/reduced-price lunch status, or gender. In addition, the three treatment groups did not differ at baseline on any outcome measure for the investigated student types (AR, AA, HA).

Differential attrition:
Attrition is not mentioned and a test for differential attrition was not performed.

Post-test:
For grades 2-3, the results showed that comparing the improvement in reading skills between pretest and posttest, PALS students outperformed PALS-HG (d=1.05) and the control group (d=.76). In this comparison, PALS-HG and the control group showed comparable improvements. In contrasts, for grade 4, PALS-HG students outperformed PALS (d=.72) and control group students (d=.98), while PALS and control group students showed comparable improvements in performance.
Student Help: On average, PALS-HG students corrected peers more frequently (p<.01) than PALS students. Also, PALS-HG students employed more frequent help giving strategies than PALS students. However, no difference in the intervention effect was found comparing the two grade groups (grade 2-3 vs. grade 4) using an interaction term (intervention x grade).

Long-term effects:
The study did not investigate long term effects.

Study 2

Summary

Fuchs et al. (2001) conducted a cluster randomized controlled trial in which 33 teachers (in 8 schools) were assigned to phonological training using "Ladders," phonological training with "Ladders" in combination with PALS, or to a control group with no phonological training. Teachers delivered their respective treatments to the entire class; however, only 14 students per class - six low achieving, four average achieving, and four high achieving - were included in the evaluation. Reading skills were measured at baseline, posttest and five-month follow-up.

Fuchs et al. (2001) found, in comparison to the control group at posttest, students in the combined intervention groups (Ladders and Ladders + PALS) had significantly better:

• Phonological awareness (maintained at follow-up)
• Alphabetic skills

Evaluation Methodology

Design:
Recruitment:
The authors recruited 4 Title 1 (high poverty) and 4 non-Title I schools in the Metro-Nashville Public Schools system. The researchers discussed the study with principals and teachers of 10 schools, 1 declined and out of the remaining 9 schools the first 8 that agreed to participate were chosen.

Sample size/Attrition:
Among the eight study schools, 38 of 43 kindergarten teachers volunteered to participate. The researchers chose 33 (no rationale is given for selection criteria), ensuring that 18 taught in Title 1 schools and that 15 taught in non-Title 1 schools. In each classroom, children were selected for study participation based on the Rapid Letter Naming (RLN) Test and teacher judgment. Children with the lowest six scores in each classroom were assigned by research assistants to a low achiever (LA) group; four students with scores in the middle of the distribution were assigned average-achiever (AA) status; the four students with the highest scores were labeled high achievers (HA). In this manner, 379 children were assigned to a LA, AA, or HA category. The study does not report attrition. However, a simple calculation (33 x 14 = 462) reveals that only 82% of the assigned students were used in the final evaluation. In addition, 25 children were identified as special-education students and were also rated LA, AA, or HA, resulting in a final study sample of 404 children. The 404 children were assessed at pre- and posttest but only 312 (77%) were assessed at the follow-up test.

Study type/Randomization/Intervention:
This study uses a randomized control trial design. The 33 teachers were assigned to one of three study groups by means of stratified (school type) randomization. Eleven teachers (N=6 Title 1; N=5 non-Title 1) were assigned to each study group. The study compared three groups: 1. Phonological training using "Ladders" (based on the Ladders-To-Literacy workbook) activities; 2. Phonological training (Ladders) in combination with peer assisted learning strategies (PALS); 3. Control group for which no phonological training was provided. The Ladders intervention was based on 15 activities, designed to enhance phonological skills. Ladder activities were teacher led, directed to the whole class. The intervention "Ladders + PALS" required children to work in dyads with same-age peers in addition to the Ladder activities. In the control group, teachers continued their regular reading/language arts instructions.

Assessment:
Research staff administered pretest (fall of kindergarten school year 1), posttest (spring of school year 1) and follow-up assessments 5 months after posttest (fall of school year 2). While the pre- and posttests were administered in 2 sessions, the follow-up assessment was administered in 1 session.

Sample characteristics:
Median percentages of African American kindergarten children in the Title 1 and non-Title 1 schools were 52% and 21% respectively. For the study classrooms only, the median proportion of children receiving free or reduced lunch was 81% in Title 1 schools and 29% in non-Title 1 schools. The average price of a home in the communities surrounding the four Title 1 schools was $53,923 vs. $136,309 in the non-Title 1 school communities.

Measures:
Validity of measurements:
All measures have been used in published studies and thus validity and reliability can be assumed. However, research staff that did the testing appears to be not blind to the treatment conditions.

Primary outcomes:
A number of tests were used to evaluate children's improvement in reading skills:

• Rapid Letter Naming (RLN) Test
• Rapid Letter Sound (RLS) Test
• Segmentation - assesses children's ability to deconstruct words into component sounds
• Word Attack Subtest of the Woodcock Reading Mastery Test - Revised Form G - assesses ability to pronounce pseudowords
• Word Identification Subtest (Word ID) of the Woodcock Reading Mastery Test - Revised Form G
• Blending - child is required to put sounds together in order to form a word
• Spelling Subtest of the Wechsler Individual Achievement Test

Analysis:
The study used teachers (classrooms) as unit of analysis and thus averaged the scores of each teacher for all three (LA, AA, and HA) student groups. To examine treatment effects, the study used multivariate analyses of variance, combining conceptually similar and highly correlated outcome-measures into two constructs: phonological awareness (segmenting, blending) and alphabetic (RLN, RLS, Word ID, Word Attack, Spelling measures). Two planned contrasts compared 1) Ladders and Ladders + PALS vs. control, and 2) Ladders vs. Ladders + PALS. It is not clear whether the models include baseline controls.

Intention-to-treat: The study may not have followed the intent-to-treat principle. Even though theoretically 462 students were chosen to participate in the study, the analysis is based on 379 students (plus 25 special-education students). It is not clear why the 83 students were not included in the evaluation and whether investigators made an effort to obtain data for these potential attritors. However, for the follow-up assessment researchers tested students who were now located in numerous different schools and classrooms which displays a substantial effort to apply the intent-to-treat principle.

Outcomes
Implementation fidelity:
All teachers received special training for their respective intervention. Research staff monitored implementation fidelity through classroom visits and rating of teachers' quality of delivering Ladder and PALS exercises (lesson clarity, lesson presentation, student involvement). Overall, the implementation fidelity was high. For example, for PALS the accuracy of implementation ranged from 77% to 87%.

Baseline Equivalence:
There were no statistically significant differences among the three study groups in the number of hours per week teachers reported devoting to language arts and reading. No significant difference was observed in teachers' class size, years of teaching experience, age, gender, education, or race. Statistical tests show that students were comparable across study groups on characteristics such as number of school absences, age, and report card grades. In addition, the three groups did not differ on the outcome measures at baseline for any of the student subgroups (LA, AA, HA).

Differential attrition:
No test of differential attrition was performed even though a substantial number of students could not be tested at the follow-up assessments.

Posttest:
Phonological awareness (segmentation, blending): The results showed that for all student categories, the combined intervention groups (Ladders and Ladders + PALS) did outperform the control group (LA: F=5.73, p<.01; AA: F=10.86, p<.001; HA: F=5.40, p<.01). No significant difference was observed comparing the two intervention groups (Ladders vs. Ladders + PALS).

Alphabetic (RLN, RLS, Word ID, Word Attack, Spelling measures): Comparing the combined intervention groups (Ladders and Ladders + PALS) to the control group showed significant improvements among LA students (F=6.85, p<.001) and for the AA category (F=2.75, p<.05) but not for the HA category. No significant difference was observed comparing the two intervention groups (Ladders vs. Ladders + PALS).

Long-term effects:
Phonological awareness: Even though the phonological improvements observed at posttest decreased at the 5-month follow-up assessment, the effects remained significant comparing the combined intervention groups (Ladders and Ladders + PALS) to the control group (LA: F=3.40, p<.05; AA: F=4.78, p<.05; HA: F=5.54, p<.05).

Alphabetic: No significant findings were observed for any contrast.

Effect size:
On phonological awareness, the PALS intervention (which included also Ladder exercises) produced largely strong effects, compared to a control group, depending on the specific test and students category (LA: segmentation d=.45; blending d=1.27; AA: segmentation d=2.10; blending d=1.97; HA: segmentation d=1.07; blending d=1.21). On alphabetic performance (RLN, RLS, Word ID, Word Attack, Spelling measures), medium to strong size effects were observed, comparing Ladders + PALS to the control group. Stronger effects were observed among LA (d=.28-1.28) and AA (d=.73-1.42) students compared to HA students (d=.08-.90).

Study 3

Summary

Stein et al. (2008) conducted a cluster randomized controlled trial in which 279 kindergarten teachers (in 97 schools) were assigned to one of four conditions: 1) 1-day workshop before K-PALS implementation; 2) workshop plus a "booster" training workshop; 3) workshop, booster, and a class-room helper (a research assistant knowledgeable in K-PALS); and 4) a control group that did not implemented K-PALS. Teachers delivered their respective treatments to the entire class; however, only 12 students - four from each category (low, average and high achieving) - were included in the evaluation. Reading achievement was measured at baseline and posttest.

Stein et al. (2008) found, in comparison to the control group at posttest, students in all intervention groups had significantly higher gains on:

• Sound identification

Evaluation Methodology

Design:
Recruitment/Sample size/Attrition:
The authors recruited schools at three sites in Tennessee (Nashville), Minnesota (Minneapolis, St. Paul, and Bloomington), and South Texas. In year 1, 48 schools agreed to participate in the study (10 in Nashville, 19 in Minnesota, and 17 in South Texas). In year 2, 49 schools participated (14 in Nashville, 21 in Minnesota, and 14 in South Texas). Once schools agreed to participate, project staff met with teachers to explain study participation requirements and procedures. The study recruited 145 teachers in Year 1 (52 in Nashville, 42 in Minnesota, and 51 in South Texas) and 134 teachers in Year 2 (54 in Nashville, 40 in Minnesota, and 40 in South Texas).

To all kindergartners, for which parental consent was obtained, the Rapid Letter Naming and Rapid Letter Sounds (RLS) test was administered. On the basis of students' performance, 4 children for each category (low achiever LA, average achiever AA, high achiever HA) were identified to participate in the study. The authors also included all children with identified disabilities. Pretest and posttest measures were administered to 1,674 kindergarteners in Year 1 and 1,555 in Year 2. The study does not report on attrition.

Study type/Randomization/Intervention:
This study used a randomized control trial design. Within each school, kindergarten teachers were randomly assigned to one of four study conditions: 1) 1-day workshop before K-PALS implementation (workshop group); 2) workshop plus a "booster" training workshop (booster group); 3) workshop, booster, and a class-room helper (a research assistant knowledgeable in K-PALS) (helper group); and 4) a control group that did not implemented K-PALS.

In K-PALS, same-age children work in dyads through 72 structured lessons. Based on subjective judgment of students' skills, teachers pair the strongest learners with the weakest learners. Each student in a pair takes a turn as a reader (tutee) and coach (tutor). Pairs remain together for 4 to 6 weeks. K-Pals was conducted by study teachers four times per week in 35-minutes sessions for 21 weeks. For each of the 72 peer-mediated lessons, children engage in the same four activities: 1) What Sound (learning letter-sound correspondence), Sound Boxes (learning decodable words), Sight Words (learning sight words), and Reading Sentences.

Assessment:
Program effects were assessed through comparison of gains in reading ability between pretests and posttest approximately 20 weeks after the administration of the pretest. The testing was conducted by trained research assistants.

Sample characteristics:
For the Nashville schools, non-Hispanic Whites, African Americans, Hispanics, and "Others" constituted 37%, 42%, 18%, and 3%, respectively, of the student community. Sixteen percent of students received special education services. The student population at the Minnesota site constituted of 45% African Americans, 14% Asians, 11% Hispanics, 4% Native Americans, and 26% non-Hispanic Whites. More than half of the students (57%) received subsidized lunch, 14% got special education services, and 24% were English language learners. The South Texas site involved six school districts in Hidalgo County, which ranks among the poorest counties in the nation, with 36% of its population living in poverty. The school children population was 80% Hispanic, of which many children showed limited English proficiency.

Measures:
Validity of measurements:
The used measures have been used by prior studies. To ensure a high level of data quality, double-scored data were entered in separate databases and went through a series of electronic and hand checks by a data manager. Research assistants were specifically trained to administer the tests correctly. Inter-rater agreement in Nashville and Minnesota was 96%. In South Texas, pre- and posttest reliability was 99%.

Primary outcomes:
Student reading achievement gains were calculated as the pre- to post-test gain on the Rapid Letter Naming and Rapid Letter Sounds (RLS) test.

Secondary measures:
Student-level measures:
The study included dummy variables for whether student was an English language learner, received free or reduced-price lunch, or had an individualized education plan. Additional dummies for race/ethnicity and gender were used.

Teacher-level variables:

• Teacher-Directed What Sound fidelity rubric
• Measures of teacher's perception of school context
• Instructional Coherence (8 items, alpha=.80)
• Teacher Community (5 items, alpha=.85)
• Principal Leadership (15 items, alpha=.93)
• Teacher Efficacy (efforts and ability, attitudes and habits, school resources)
• K-PALS Instructional Coherence (3 items, alpha=.63)
• K-PALS Principal Leadership (2 items, alpha=.84)

School-level variables:
Dummy variables for site location and study year were used.

Analysis:
The study used a multilevel modeling approach. In the employed 3-Level, random intercept models, students are nested in teachers' classrooms, while classrooms are nested in schools. The analysis controls for baseline scores on the outcome measure.

Intention-to-treat: It appears that the study gathered and used data on all subjects who were randomized and completed the surveys. However, the lack in reporting of attrition prevents a final conclusion regarding the study's compliance to the intent-to-treat principle.

Outcomes

Implementation fidelity:
All teachers that implemented K-PALS were provided with a comprehensive manual that detailed the intervention and provided guidelines for its implementation. Teachers received training and supervision to different degrees, based on intervention group assignment. To evaluate implementation fidelity, the teacher and the student pairs were observed and evaluated.

Baseline Equivalence:
No test for baseline equivalence was performed.

Differential attrition:
Attrition was not reported and no test for differential attrition was performed.

Posttest:
All three program groups (workshop group, booster group, helper group) showed higher average gains in sound identification (RLS) than the control group, confirming the effectiveness of PALS regardless of teachers' training and support. Strongest improvements were obtained if teachers had obtained the workshop and the booster training (booster group: b=18.67, p<.01), followed by teachers who were assisted by helpers (helper group: b=16.11, p<.01), and teachers who had only received the workshop training (workshop group: b=11.10, p<.01).

However, once a measure for fidelity of implementation was introduced into the models, the program group effects became insignificant. This indicated that the effects of the K-PALS program are substantially mediated by the fidelity with which teachers implement the K-PALS program. Implementation fidelity, varied by teachers' support with teachers assisted by research assistants (helper group) showing the highest levels of implementation fidelity.

Effect size:
Compared to a control group, PALS resulted in strong effect sizes (d=1.02-1.18) that varied slightly as a function of different levels of teachers' support.

Long-term effects:
No long-term effects were investigated in this study.

Study 4

Summary

Calhoon et al. (2007) conducted a cluster randomized controlled trial in which six teachers (94 students, three schools) were assigned to the intervention or to a control group with no peer-mediated reading activities. Reading skills were assessed at baseline (fall), midway (winter) and immediately following the intervention (spring).

Calhoon et al. (2007) found, in comparison to the control group at posttest, students in the intervention group had significantly higher gains on:

• Phoneme segmentation
• Nonsense word fluency
• Oral reading fluency

Evaluation Methodology

Design:
Recruitment/Sample size/Attrition:
Three Title 1 elementary schools in a south-western school district, located near the border of Mexico were recruited for the study. Six first-grade teachers were selected based on their interest in and willingness to participate in the project. All teachers utilized the Two Way Bilingual Immersion (TWBI) program offered by their schools, providing approximately equal amounts of English and Spanish instruction throughout the school day. Students from each teachers' classroom (N=94) were selected for evaluation of program effectiveness. Of the 94 students, 18 (19%) were lost over the study period due to moving or transferring to another school, reducing the effective sample size to 76 students. The PALS group comprised 43 students (English Proficient (EP): N=30; English Language Learners (ELLs): N=13) and the control group comprised 33 students (EP: N=22; ELLs: N=11).

Study type/Randomization/Intervention:
The study constitutes a randomized control trial. Teachers were randomly assigned to either the PALS (N=3) or control group (i.e., no peer-mediated reading activities) (N=3). PALS was implemented according to the published teacher training manual and training protocol developed by Fuchs et al. (2000). PALS sessions followed a structured instructional routine as follows: (a) the teacher briefly presents the activities of the day, (b) students practice the PALS activities for about 15 minutes while the teacher circulates around the room to supervise, and (c) the teacher instructs students to switch to "Story Sharing," a partner reading activity that also lasts about 15 minutes. Pairs are assigned to teams and work together for about four weeks. Students participated in 60 PALS sessions that lasted 30-35 minutes three times per week for 20 weeks. PALS was implemented during the English portion of reading instruction.

Assessment:
Students' reading skills were assessed two weeks before (Fall), midway through (Winter), and immediately following the intervention (Spring). Assessments were conducted by trained research assistants. It is not clear whether research assistants were blind to the group assignments.

Sample characteristics:
More than 80% of children in all three schools received free or reduced-cost lunch, indicating the low socio-economic status of the student population. Of the 76 students, 79% were Hispanic. Twenty-four students were determined by their schools to have limited English proficiency (ELLs) while 52 students were considered English Proficient (EP).

Measures:
The study used four Dynamic Indicators of Early Literacy Skills (DIBELS) reading subtests:

• Letter Naming Fluency (LNF)
• Phoneme Segmentation (PSF)
• Nonsense Word Fluency (NWF)
• Oral Reading Fluency (ORF)

Analysis:
Statistical methods/baseline control:
A repeated-measures ANOVA was conducted with time (Fall, Winter, Spring) and condition (PALS vs. contrast) for each reading subtest. As such, the analysis controls for baseline scores.

Intention-to-treat: It is unclear whether the study complied with the intent-to-treat principle.

Outcomes

Implementation fidelity:
Teachers for the treatment conditions attended a one-day workshop that provided explicit training on PALS. A research assistant was assigned to each teacher, who visited the classroom and observed implementation fidelity and provided support for the teacher. The fidelity scores for PALS implementation ranged from 94.9% to 98.7% and thus demonstrate high implementation fidelity.

Baseline Equivalence:
No significant difference was observed between groups on gender, ethnicity, and years retained. However, the PALS condition had a higher number of female students and lower number of students with special education labels. In addition, statistical tests show no differences between intervention and control group at baseline on three of the four outcome measures. Only for the NWF measure a significant difference between EP intervention and EP control group was observed.

Differential attrition:
Even though attrition was substantial (19%), no test for differential attrition was performed.

Posttest:
Primary finding: Results show significant time x treatment interactions favoring PALS for three out of four outcome measures (Phoneme Segmentation (PSF) F=4.38, p=.01; Nonsense Word Fluency (NWF) F=4.30, p=.01; and Oral Reading Fluency (ORF): F=63.73, p=.001). No significant time x treatment interaction was found for Letter Naming Fluency (LNF).

Secondary finding: The study further investigated treatment effects for the subgroups of English Language Learners (ELLs) and English Proficient (EP) students. For ELLs students, significant time x treatment interactions were demonstrated for two out of four measures (LNF: F=4.48, p<.05; and NWF: F=5.38, p<.01). Similarly, for EP students, two out of four time x treatment interactions were found to be significant favoring PALS (PSF: F=6.20, p<.01 and ORF: F=59.53, p<.001).

Effect size:
PALS significantly improved reading skills compared to a control group with moderate effect sizes ranging from .50 to .53 depending on the outcome measure.

Long-term effects:
The study did not investigate long term effects of PALS.

Study 5

Summary

Calhoon et al. (2006) conducted a cluster randomized controlled trial in which six teachers (96 students, three schools) were assigned to the intervention or to a control group with regular core reading instruction. Reading skills were assessed at baseline (fall), midway (winter) and immediately following the intervention (spring).

Calhoon et al. (2006) found, in comparison to the control group at posttest, students in the intervention group had significantly higher gains on:

• Phoneme segmentation
• Nonsense word fluency

Evaluation Methodology

Design:
Recruitment/Sample size/Attrition:
The study was fielded in three Title 1 elementary schools in a New Mexico border town. Even though bilingual education was available at these schools (e.g., Spanish immersion programs, Two-way bilingual programs), the students studied were all placed in English Language-only first-grade classrooms. The study started with 96 first-grade students, in 6 classrooms of which 18 students (19%) moved, resulting in a final sample of 78 students. No statistical investigation of differential attrition was performed.

Study type/Randomization/Intervention:
This study constitutes a randomized control trial. The 6 participating teachers (and their respective classrooms) were randomly assigned to either the PALS treatment or a control condition. The program was implemented in accordance with the PALS training manual (Fuchs et al. 2000). As usual, teachers created dyads by pairing one high- and one low-performing reader and then trained all students in the classes to follow procedures for first-grade PALS. Pairs worked together for about 4 weeks. Students participated in 60 PALS sessions that lasted 30-35 minutes three times per week for 20 weeks. Sessions included two types of activities, Sounds and Words and Story Sharing. In contrast, teachers in the control condition provided the regular core reading instruction that their district and school provided to all first-grade students.

Assessment:
The assessments were done by trained research assistants. Students were tested in Fall, prior to intervention, in Winter during the 10th week of PALS, and in Spring immediately following the 20-weeks of intervention. It is not clear whether research assistants were blind to students' group assignment.

Sample characteristics:
In the three schools, over 75% of students received free and reduced lunch. The majority of the students studied were of Hispanic descent. Of the 78 students, 68% were Hispanic bilingual students, while 32% were Anglo English-monolingual students. Even though children were bilingual to varying degrees, they did not require English as a second language instruction.

Measures:
Validity of measurements:
All measures have been used in prior studies and high levels of reliability have been reported (e.g., test-retest reliability ranging from .92-.97).

Primary outcomes:
Three subtests of the Dynamic Indicators of Early Literacy Skills (DIBELS) measures were used:

• Phoneme Segmentation Fluency (PSF)
• Nonsense Word Fluency (NWF)
• Oral Reading Fluency (ORF) - this measure was only administered in Winter and Spring.

Analysis:
The study employed repeated measure ANOVAs that control for baseline scores but analyzed individuals rather than classrooms.

Intention-to-treat: It is unclear whether the study complied with the intent-to-treat principle.

Outcomes

Implementation fidelity:
During a one-day workshop, teachers were trained to conduct PALS. A research assistant was assigned to each teacher to provide support and guidance during the implementation process. Research assistants also assessed implementation fidelity through direct observation (40-item checklist). The fidelity scores for PALS implementation ranged from 90.28% to 97.34% with a mean of 93.75%.

Baseline Equivalence:
Statistical tests did not show much variation in demographic characteristics of children between intervention and control group. The only difference was observed for special education with higher numbers of students in the control group receiving this service. There were no significant differences in the pretreatment reading performance between students in the PALS and control conditions on any of the used measures.

Differential attrition:
Even though attrition was substantial, a test for differential attrition was not performed.

Posttest:
Compared to the control group, PALS was able to significantly improve reading performance on 2 out of 3 tests. More specifically, a significant interaction effect for time x treatment was present for the measures of Phoneme Segmentation Fluency (PSF), F(2, 148) = 12.79, p < .000, and Nonsense Word Fluency (NWF), F(2,148) = 5.32, p < .01, but not for Oral Reading Fluency (ORF), F(1,74) = 110.10, p>.05.

Furthermore, the three-way interaction of time x ethnicity x treatment was not significant for either PSF, F(1, 148) = 2.42, p = .09, NWF, F(1,148) = .996, p > .05, or ORF F(1,74) = .14, p >.05, suggesting that the program is able to improve reading skills regardless of race/ethnic background of the students.

Focusing on at-risk students, the study further demonstrated that consistently higher percentages of PALS than control students initially at risk achieved grade level performance at the end of the school year. Similarly, among PALS students, there was less slippage from performance on grade level (at the beginning of the school year) to at risk (at the end of the school year). These findings held across both Hispanic and non-Hispanic subgroups.

Effect size:
Large to moderate effect sizes ranging from .58 to .93 were found for the PALS treatment for 2 out of 3 measures on reading skills (Phoneme Segmentation Fluency, Nonsense Word Fluency).

Long-term effects:
The study did not investigate long-term program effects.

Study 6

Summary

Calhoon et al. (2005) conducted a cluster randomized controlled trial in which four special education teachers (38 students, 2 middle schools) were assigned to Linguistic Skills Training in combination with PALS or to a control group with no peer mediated activities. Reading skills were assessed at baseline and posttest.

Calhoon et al. (2005) found, in comparison to the control group at posttest, students in the intervention group scored significantly higher on:

• Letter Word Identification
• Word Attack
• Passage Comprehension

Evaluation Methodology

Design:
Recruitment/Sample size/Attrition:
Four special education teachers from two middle schools in a southwestern school district were recruited for the study. All teachers were selected based on their interest and willingness to participate. A requirement was that teachers had students with chronic reading difficulties in their classrooms. The study population comprised 38 students with reading disabilities in the classrooms of the four teachers. No attrition occurred.

Study type/Randomization/Intervention:
This study constitutes a randomized control trial. Teachers (and their respective classrooms) were randomly assigned to one of two treatment conditions: Linguistic Skills Training (LST) in combination with PALS (N=2) or a control group (N=2).

The LST phonological skill program was designed specifically to accompany the PALS reading comprehension program. Each LST lesson begins with a scripted teacher-led lesson, followed by teacher-directed practice. Students then practice in peer tutoring pairs closely monitored by the teacher. LST systematically and explicitly teaches phonological skills through letter-sound correspondence and phonological awareness exercises. The PALS program incorporates three essential reading activities: partner reading, paragraph shrinking, and prediction relay. LST was implemented 3 days per week (51 hours across study period) while PALS was implemented 2 days a week (34 hours across study period). The control group received reading instructions using a widely implemented remedial reading program (Saxon Phonics Intervention), which, however, has no peer-mediated activities. Saxon Phonics Intervention follows a more traditional format by integrating explicit teaching of decoding skills, spelling, vocabulary, reading fluency, and reading comprehension. The Saxon Phonics Intervention program was implemented 3 days a week (51 hours across study period). Saxon Phonics Intervention was coupled with the SRA Skill Acquisition program, which aims at improving reading comprehension. The SRA (Science Research Associates) Skill Acquisition program was implemented 2 days a week (34 hours across study period). Each session lasted between 35 and 40 minutes.

Assessment:
Assessments were conducted 2 weeks before intervention (pretest) and after 31-weeks of intervention (posttest) by trained research assistants. However, it is unclear whether the research assistants were blind to the group assignment of students.

Sample characteristics:
Student participants consisted of 32 sixth, 5 seventh, and 1 eighth grader (N=38) from four special education, self-contained language arts classrooms. All students had a learning disability and were reading at least three grade levels below their current grade placement. The study population comprised more boys (66%) than girls (34%). The majority of students were of Hispanic descent (66%), followed by whites (29%) and blacks (5%). On average, students were 12 years of age and had an IQ around 90.

Measures:
Validity of measurements:
The Woodcock-Johnson Test of Achievement-3 is a widely used, norm-referenced diagnostic achievement test with good evidence for internal consistency (r exceeds .80 for all subtests).

Primary outcomes:
The following four reading subtests of the Woodcock-Johnson Test of Achievement-3 (WJ-3) were used.

• Letter Word Identification
• Word Attack
• Reading Fluency
• Passage Comprehension

Analysis:
The authors conducted two statistical tests. First, they used univariate ANOVAs to compare posttest scores between groups for each reading subtest. As such, this test does not control for baseline controls. Second, the authors used univariate ANOVAs to assess growth (pretest scores minus posttest scores), which includes baseline values.

Intention-to-treat: The study gathered and used data on all subjects who were randomized and tested.

Outcomes
Implementation fidelity:
Teachers attended a 1-day workshop that provided explicit training on their individual programs. A research assistant observed the program implementation and provided ongoing support and feedback. Overall, a high implementation fidelity was reported (PALS=90.3, LST=98.7, Saxon Phonics Intervention=96.6, SRA Skill Acquisition=91.7).

Baseline Equivalence:
No significant differences were found between treatment groups for gender, race, grade level, or IQ. However, a significant difference was observed for student age with the LST/PALS group being slightly older than the control group (12.11 vs. 11.71). No statistically significant difference was observed between treatment groups on any of the four outcome measures at baseline.

Differential attrition:
Because no attrition occurred, no test of differential attrition was performed.

Posttest:
The results showed statistically significant differences between groups at posttest, with the LST/PALS condition outperforming the control condition on Letter-Word identification (F=14.32, p=.001), Word Attack (F=10.03, p=.01), and Passage Comprehension (F=11.35, p=.01). However, no significant difference between conditions was found for Reading Fluency.

The results for assessing growth (pretest scores minus posttest scores) produced similar results. Significant growth for the LST/PALS treatment group over the control group was detected for Letter-Word Identification (F=14.32, p=.001), Word Attack (F=13.19, p=.001), and Passage Comprehension (F=11.35, p=.01). No significant differences were found for growth on Reading Fluency.

Effect size:
This study reports generally strong effect sizes, for the three of four significant outcome measures (Letter Word Identification: d=1.10; Word Attack: d=.99; Passage Comprehension: d=.94).

Long-term effects:
The study did not assess long-term effects of the program.

Study 7

Summary

Mathes and Babyak (2001) conducted a cluster randomized controlled trial in which 30 first grade teachers (38 students, 5 schools) were assigned to standard PALS, PALS + skills-focused mini-lessons, or to a control group. From each class five students were selected for evaluation - three low achieving, one average achieving, and one high achieving. Reading skills were assessed at baseline and posttest.

Mathes and Babyak (2001) found using one-tailed significance tests (p<.05), in comparison to the control group at posttest, students in both intervention groups scored higher on:

• Word Attack, Word Identification, Basic Skills (low achievers)
• Word Attack, Word Identification, Basic Skills, Passage Comprehension (average achievers)

Evaluation Methodology

Design:
Recruitment/Sample size/Attrition:
Thirty 1st grade teachers in 5 schools in a southeastern medium-sized school district were recruited. Over the course of the study, two 1st grade teachers (7%) assigned to the PALS+ML group withdrew from the study (these teachers considered conducting PALS and the mini-lectures too demanding). From within each participating classroom, the highest-achieving (HA) child, one average achieving (AA) child, and the 3 lowest-achieving (LA) children were selected for assessment, based on teacher's judgment (reassessed by the investigators).

Study type/Randomization/Intervention:
This study used a randomized control trial design. Of the 30 teachers (who volunteered to participate), 10 were assigned to 1st grade PALS, 10 were assigned to 1st grade PALS + skills-focused mini-lessons (ML), and 10 were assigned to a control group. PALS was conducted with the entire class during three 35-minutes sessions per week for 14 weeks. As usual, teachers paired students based on reading strength (stronger + weaker reader). During PALS, pairs worked through two 15-minute routines which the authors termed "Sounds and Words" and "Story Sharing." Sounds and Words provided for systematic, explicit instruction in synthetic phonics (Letter-Sounds, Hearing Sounds, Sounding-Out, Passage Reading); Story Sharing used shared reading and rereading of authentic text (Pretend-Read, Read-Aloud, Retell).

In 8 classrooms, skills-focused mini-lessons (ML) were implemented in addition to PALS. These additional sessions were conducted with the 3 low achieving (LA) children in small groups 3 times per week for 15 to 20 minutes each session. The mini-lessons mirrored exactly the content of the Sounds and Words portion of 1st-Grade PALS but, unlike the children coaches, teachers provided instruction according to student needs. Each mini-lesson was conducted prior to the children conducting PALS. Teachers in the control group conducted reading instruction in their typical manner.

Assessment:
Students reading skills were assessed by means of a pretest and a posttest, administered by trained researchers. In addition, for continuous progress monitoring (CPM) the researchers administered probes of (1) oral reading fluency of connected 1st grade text and (2) phonological awareness segmenting skills each week of the study duration. It is not clear whether the individuals administering the tests were blind to children's group assignment.

Sample characteristics:
Teachers: The teachers were mostly Caucasian (about 80%) and African American (20%), had on average 15 years teaching experience, and around 20 students in their reading instruction classes. Students: Students were between 6 and 7 years of age, about 50% were male, and almost all had attended kindergarten. The majority of students were Caucasian with around 30% African Americans and small proportion (<5%) of other racial groups. Among the low achieving sub-group, about 70% qualified for special education due to some form of language/speech disability.

Measures:
Validity of measurements:
The validity of the used measures has been demonstrated in prior studies.

Primary outcomes:
Reading achievement was assessed using the following measures of the Woodcock Reading Mastery Test - Revised edition:

• Word Identification
• Word Attack
• Basic Skills
• Passage Comprehension

Analysis:
Multivariate analysis of variance (MANOVA) was performed to examine change from pre to posttest scores. In addition, univariate ANOVAs were performed for individual items if the MANOVA showed a significant main effect. The authors determined statistical significance using 1-tailed tests.

Intention-to-treat: It is unclear whether the study followed the intent to treat principle.

Outcomes

Implementation fidelity:
All participating teachers attended a one-day training session, where they received a detailed manual and learned how to conduct the procedures. Project staff provided support during the implementation as needed. In addition, classroom observations were conducted and checklists were used to quantify implementation fidelity. Overall, teachers conducted PALS with 92.59% and ML with 89.53% accuracy.

Baseline Equivalence:
No statistical difference in teachers' characteristics and teaching style across study groups was observed. Statistical tests indicated no significant differences between groups on characteristics such as students' age, gender, race, prior retention, or qualification for special education service. ANOVAs revealed no statistically significant differences between LA, AA, or HA participant groups on pretest measures.

Differential attrition:
Attrition was not reported, and no analysis of differential attrition was performed.

Posttest:
Low Achiever: For low achievers (LA) a significant main effect for treatment was observed using MANOVA (F=1.19, p<.05). Univariate analyses indicate that for the Word Attack, Word Identification, and Basic Skills subtests, both PALS and PALS+ML significantly improved reading skills compared to a control group. However, the results for PALS and PALS+ML did not differ significantly from each other. Nevertheless, effect sizes suggested that PALS+ML produced stronger improvements than PALS alone. No statistically significant difference was found for the measure of passage comprehension. Thus, out of 4 tests, 3 were significant in favor of the PALS intervention groups.

Average achievers (AA) and high achievers (HA): The MANOVA indicated significant main effects for treatment condition (p<.05). Univariate ANOVAs for average achievers (AA) yielded statistically significant differences, in favor of PALS, for all four outcome variables (Word Attack, Word Identification, Basic Skills, and Passage Comprehension). In contrast, among high achievers (HA) no statistically significant findings were detected for any measure of reading skills.

Effect size:
Medium to strong effect sizes were observed for significant variables among LA students (ES=.61 to 1.00) and AA students (ES=.83-1.66). No significant results were observed for HA students.

Long term follow-up:
Long-term program effects were not investigated.

Study 8

Summary

Mathes et al. (2001) used a quasi-experimental design to evaluate the program with 36 teachers (in 8 schools) assigned to PALS, PALS + Computer Assisted Instructions, or a control group. From each class six students were selected for evaluation - four low achieving, one average achieving, and one high achieving. Reading skills were assessed at baseline and posttest.

Mathes et al. (2001) found, in comparison to the control group at posttest, low-achieving students in both intervention groups scored significantly higher on:

• Nonword Efficiency
• Word Attack
• Basic Skills
• Passage Comprehension

Evaluation Methodology

Design:
Recruitment/Sample size/Attrition:
Thirty-six first-grade teachers in eight schools in a southeastern medium-sized school district were recruited to participate in the study. Of the 36 teachers, 12 were assigned to conduct PALS only, 12 were assigned to conduct PALS + Computer Assisted Instructions (CAI), and 12 were assigned to a control group. Six of the classrooms included both kindergarten and first-grade students and were distributed evenly across treatment groups (two teachers of multiage students in each condition). The remaining 30 classrooms had only first-grade students.

The results from a 1-min oral reading probe of all children in the 36 classrooms were used to rank-order children within each classroom. One high achieving (HA) child, an average-achieving (AA) child, and the four lowest achieving (LA) children were selected for evaluation. The study started out with 33 HA, 35 AA, and 130 LA students. Throughout the year, 1 HA, 2 AA, and 9 LA students moved out of the area and 3 LA students were removed from the study because of excessive absences (remaining sample N=183 or 92% of baseline sample). The study did not investigate differential attrition.

Study type/Randomization/Intervention:
This study uses a quasi-experimental design. Some of the teachers (N=6) who had participated in a prior study (Mathes and Babyak 2001), requested continued participation in the PALS condition, and were assigned randomly to either the PALS or PALS+CAI intervention. The researchers then identified control group matches (N=3) from within the same school. The remaining teachers (N=27) were recruited to participate in the study and randomly assigned to PALS (N=9), PALS+CAI (N=9), or control group (N=9).

The PALS intervention was implemented as usual for a total of 16 weeks. Students in the PALS+CAI condition received additional phonological awareness instruction using the software packages Daisy Quest and Daisy's Castle. CAI was implemented only with LA subjects for a total of 8 weeks (4 weeks prior and 4 weeks during PALS). Each week, children completed three 20- to 30-min CAI sessions. Teachers in the control group conducted reading instructions in their typical manner.

Assessment:
Trained research assistants conducted all assessments at pretest and posttest. However, it is not clear whether research assistants were blind to the group assignment.

Sample characteristics:
Teachers: Teachers were mostly Caucasians (about 85%) with a small number of African American teachers (about 15%). Teachers had around 17 years of teaching experience, of which about 8 years were spent teaching first grade. Students: On average, students were 6 years of age, all had attended kindergarten and between 5% and 27% had been prior retained in grade depending on the particular group. The majority of students were Caucasians, with a substantial fraction of African Americans (around 30%). About 20% of the low achieving children had been referred to special education.

Measures:
Reading achievement was measured using the following tests:

• Woodcock Reading Mastery Test - Revised (WRMT-R); Subtests included Word Identification, Word Attack, Passage Comprehension (alpha= .68-.99; concurrent validity ranges from .85 to .91)
• Test of Early Reading - 2 (TERA-2); 46 items that measure a child's concepts of print (alpha=.70-.98)
• The Test of Word Reading Efficiency (TOWRE); 2 items, administered only to LA children (alpha=.95-.96)
• Comprehensive Test of Phonological Processes (CTOPP), 2 items (alpha=.87-.95).

Note: Even though alpha values are reported, most subtests were not combined to form a scale (except for TERA-2), but rather were included as multiple outcomes in a MANOVA.

Analysis:
Statistical methods/baseline control/correct unit of analysis:
The study used multivariate analysis of variance (MANOVA) and follow-up univariate analyzes of variance, controlling for baseline scores. However, researchers decided to determine statistical significance using a one-tailed test. Group assignment was conducted at the teacher (classroom) level while the statistical analysis was performed with individual level data.

Intention-to-treat: The study did not comply with the intent-to-treat principle. The researchers purposefully removed 3 LA students from the study "because of excessive absences."

Outcomes

Implementation fidelity:
To ensure that teachers did not differ systematically in their teaching approaches, information on teaching methodology and reading instructional time allocation was collected. No significant differences in reading approaches were observed across groups. Teachers attended an all-day workshop where they received a detailed manual and learned how to conduct the procedures. A research assistant provided assistance during the implementation process and evaluated implementation fidelity. Overall, teachers conducted PALS with 93.83% accuracy.

Baseline Equivalence:
ANOVAs revealed no statistically significant differences between study groups on most of the teacher's characteristics, except for age and total years teaching experience. Groups did not differ on student's characteristics such as gender, age, race, prior attendance of kindergarten, prior retention, and qualification for special education service. A statistically significant difference among LA participant groups was found at baseline on all four components of the WRMT-R test, with lower values for children in the PALS+CAI treatment group.

Differential attrition:
No test of differential attrition was performed.

Posttest:
LA students: To summarize the findings, PALS+CAI did no better than PALS, but both PALS groups did better than the control group. More specifically, MANOVAs found significant main effects for treatment on all 3 test groups with more than one outcome measure (WRMT-R: F=2.42, p<.05, TOWRE: F=5.68, p<.001, CTOP: F=2.77, p<.05). Follow-up univariate ANOVAs showed that both the PALS and PALS+CAI group scored significantly higher (one-tailed test) than the control group on 8 out of 9 sub-tests (no significant difference was observed for the TOWRE subtest of Word Efficiency). However, only 4 out of the 9 sub-tests are significant at the .01 level and thus would be significant at .05 in a two tailed test. A statistical comparison between the effects for the PALS and PALS+CAI groups revealed no significant differences.

AA and HA students: Results from MANOVA indicated no significant main effect for treatment. Follow-up univariate ANOVAs indicated a statistically significant difference for AA students for the word Attack subtest of the WRMT-R (thus 1 out of 5 tests was significant). No other statistically significant findings were detected for any other measure for either AA or HA students.

Effect sizes:
For significant variables (8 out of 9 tests), PALS showed moderate effect sizes (ES=.43-.74) for LA students. For the one significant measure of the AA category a moderate effect size was reported (ES=.69).

Long-term effects:
The study did not investigate long-term effects.

Study 9

Summary

Mathes et al. (1998) used a quasi-experimental design to evaluate the program with 20 teachers (in 6 schools) assigned to the intervention or a control group. From each class, five students were selected for evaluation - three low achieving, one average achieving, and one high achieving. Reading skills were assessed at baseline and posttest.

Mathes et al. (1998) found, in comparison to the control group at posttest, students in the intervention group scored significantly higher on:

• Woodcock Reading Mastery
• Word Identification, Word Attack (low achievers)
• Passage Comprehension (average achievers)

Evaluation Methodology

Design:
Recruitment/Sample size/Attrition:
Twenty teachers in 6 schools in a southeastern U.S. urban school district were recruited to participate in the study, 10 to conduct PALS and 10 to serve as a control group. Within each of the 20 participating first-grade classrooms, the research team identified 5 students as participants based on performance on a standardized test (Curriculum-Based Measurement). The 3 lowest achieving (LA) students, one average-achieving (AA) student, and one high-achieving (HA) student were selected for study participation (total N=100). Four LA students (4%) moved during the course of the year. Despite the prevalence of attrition, no analysis of differential attrition was performed.

Study type/Randomization/Intervention:
The study used a quasi-experimental design. Four teachers had been involved in pilot testing the PALS program and requested continued participation. Control group matches for these 4 teachers were recruited from among teachers who had similar teaching profiles (e.g., years of experience, degree held, self-reported approach to teaching). The remaining 12 teachers were recruited to participate in the project, and then randomly assigned to either the PALS or control group.

PALS was conducted as usual with the entire class, three times per week for 35-minute sessions for 16 weeks. One difference from the original study was that within pairs of students the role assignment was fixed. Thus, stronger students acted always as the tutor while the weaker students always performed the role of the tutee. Teachers in the control condition conducted reading instruction in their normal fashion.

Assessment:
Students' reading skills were assessed pretest (beginning of school year) and posttest after 16 weeks of intervention. Assessment was conducted by trained research assistants. However, it is not clear whether research assistants were blind to students' group assignments.

Sample characteristics:
Teachers: Teachers were 85% Caucasian and 15% African American. The majority of teachers were in the age range of 30 to 39 years, had 11 years of teaching experience, and around 40% were holding a masters degree. Students: All classrooms were first-grade classrooms in which the average age of students was 6 years. Among low achieving students, 25% in the intervention and 4% in the control group showed a learning or language disability. Also, among low achievers 11% of children in the intervention group and 4% in the control group had been prior retained. In both the intervention and control group, 56% were Caucasian and 44% were African American.

Measures:
Reading achievement was assessed using three tests, which have been widely used and validated in prior studies:

• Woodcock Reading Mastery Tests-Revised (WRMT-R); subtests include Word Identification, Word Attack, Passage Comprehension; reliability coefficients ranged from .68 to .99.
• Test of Early Reading Ability-2 (TERA-2), 46 items that yield one score, reliability coefficients ranged from .70-.98. Note: TERA-2 was administered only to LA students.
• Comprehensive Reading Assessment Battery - Revised (CRAB-R), used two tests, Reading Fluency and Comprehension, reliability of this measure was not assessed.

Analysis:
Because data on multiple dependent sub-tests was collected on the WRMT-R and the CRAB-R, multivariate analysis of variance (MANOVA) was performed initially. Since the authors examined change from pre- to post-treatment standard scores, they included baseline controls. A significant main effect on the MANOVA was followed-up by univariate ANOVAs for each sub-test separately. The analysis used individual-level information while the group assignment was done at the classroom-level.

Intention-to-treat: It is unclear whether the study complied with the intent-to-treat principle. No attempt was made to collect data on the 4 students who moved away.

Outcomes

Implementation fidelity:
Teachers attended an all-day workshop in which they were trained in the correct implementation of PALS and received a detailed manual explaining all aspects of the program. Through observations the researchers measured implementation fidelity. Data indicated that teachers conducted PALS with 87.67% accuracy, while student implementation was at 81.17% accuracy.

Baseline Equivalence:
ANOVAs revealed no statistically significant differences between groups on demographic characteristics of participating teachers. However, few statistically significant differences on students' demographics were observed across the intervention groups. More HA students in the PALS group were Caucasian and more LA students in the PALS group were special education students. In addition, ANOVAs indicated a statistically significant difference between PALS and control group for LA students on the CRAB-R Word Reading test results prior to intervention.

Differential attrition:
No test for differential attrition was performed.

Posttest:
MANOVAs showed a significant main effect for treatment on the WRMT-R tests (F=5.55, p<.05) but not for the CRAB-R tests. Follow-up univariate ANOVAs of WRMT-R change scores revealed a statistically significant difference for LA students between the intervention and control group in favor of PALS on two out of three subtests (Word Identification: F=6.84, p<.05; Word Attack: F=8.47, p<.01). For AA students, statistically significant differences were detected between groups on one of three subtests (Passage Comprehension: F=6.30, p<.05), while for HA students no statistically significant differences were observed.

The TERA-2 was analyzed using ANOVA since it yields only one score. The TERA-2 was given only to LA students. A significant effect was observed in which LA students who participated in PALS experienced statistically significantly higher improvements in reading skills compared to students in the control group (F=4.05, p<.05).

In summary, for LA students 3 out of 6 tests (50%) were significant, while only 1 out of 6 tests (16%) was significant for AA students and none was significant for HA students.

Effect size:
For significant variables among LA students a moderate effect size was reported (Word identification: ES=.70; Word attack: ES=.78), while for the one significant variable among AA students a strong effect was detected (Passage comprehension: ES=1.12).

Long-term effects:
No long-term effects were investigated.

Study 10

Summary

Mathes et al. (2003) used a quasi-experimental design to evaluate the program with 26 teachers (in 6 schools) assigned to the PALS intervention, teacher-directed small-group sounds and word lessons, or a control group. Only low achieving students in each classroom were selected for evaluation. Reading skills were assessed at baseline and posttest.

Mathes et al. (2003) found, in comparison to the control group at posttest, students in the PALS group scored significantly higher on:

• Word Attack
• Nonword Efficiency
• Phonetic Segmentation

Evaluation Methodology

Design:
Recruitment/Sample size/Attrition:
Twenty-two first-grade teachers in six schools (selected based on size, mean reading skills, and student's SES) in a medium-sized southeastern school district were recruited to participate in the research. Seven conducted PALS, seven conducted teacher-directed small-group sounds and words lessons (TDI), and eight served as the control group. Only low achieving students, at risk of reading failure, were selected based on reading test scores. A selection criterion for participation was that students read below 8 words per minute. The goal was to select 5 low achieving students per class (theoretical N=110). However, several classrooms did not have five students who met the selection criterion and the researchers did not obtain parental consent for all qualifying students reducing the sample to N=100. Finally, across the year, two PALS, five TDI, and four contrast students moved out of participating classrooms, further reducing the sample to N=89. No test for differential attrition was performed.

Study type/Randomization/Intervention:
This study used a quasi-experimental design. A number of teachers (N=8) had conducted PALS in prior studies and had requested continued participation in the experimental condition. These teachers were randomly assigned to the PALS or TDI intervention group. The researchers then identified control group matches (N=4) from among teachers with similar teaching styles. The remaining teachers (N=10) were recruited to participate in the project and then randomly assigned to PALS, TDI, or the control group.

PALS was conducted as usual in 35-minute sessions, 3 times per week for 16 weeks. Students were paired so that in each group there was a stronger and a weaker reader. The tutor and tutee roles were reciprocal so that throughout the PALS session, each child performed both roles. In the small-group teacher-directed instruction (TDI) condition, small groups of four to five children were formed. In these groups, teachers provided additional training three times per week for 30 minutes each session. The TDI lessons began at the same time as the first-grade PALS intervention and mirrored exactly the content of PALS. Teachers in the control group conducted reading instruction in their typical manner.

Assessment:
Students' reading skills were assessed pretest (beginning of school year) and posttest after 16 weeks of intervention (end of school year). Assessment was conducted by trained research assistants. However, it is not clear whether research assistants were blind to students' group assignments.

Sample characteristics:
Teachers: Teachers were 85% Caucasian and 15% African American. Teachers had on average 17 years of teaching experience, and around 50% were holding a masters degree. Students: All classrooms were first-grade classrooms in which the average age of students was 6 years. About 50% of the students were Caucasian, 40% were African American, and 10% were categorized as "other." Since all children were low achievers, a substantial fraction (60%) received additional tutoring. Around 25% of the children qualified, or were in the process of qualifying, for special education.

Measures:
Reading achievement was assessed using three tests and related sub-tests as described in the following. Prior research has demonstrated adequate reliability and concurrent validity for each measure.

• Woodcock Reading Mastery Tests-Revised (WRMT-R); sub-tests include Word Identification, Word Attack, Passage Comprehension
• Test of Word Reading Efficiency (TOWRE); sub-tests include phonemic decoding efficiency test and sight word efficiency test
• Comprehensive Test of Phonological Processes (CTOPP); sub-test used was the phoneme segmentation test

Analysis:
Because data on multiple dependent sub-tests were collected on the WRMT-R and the TOWRE, multivariate analysis of variance (MANOVA) was performed initially. Since the authors examined change from pre- to post-treatment scores, baseline controls are implicitly included. A significant main effect on the MANOVA was followed-up by univariate ANOVAs for each sub-test separately. The analysis used individual-level information while the group assignment was done at the classroom-level.

Intention-to-treat: It is unclear whether the study complied with the intent-to-treat principle. No attempt was made to collect data on the 11 students who moved away.

Outcomes

Implementation fidelity:
Teachers attended an all-day workshop in which they were trained in the correct implementation of their respective intervention and received a detailed manual explaining all aspects of the program. A staff member of the research team was assigned to each teacher and provided assistance if needed. Through observations and checklists, implementation fidelity was assessed. Data indicated that teachers conducted PALS with 89.22% and TDI with 87.43% accuracy.

Baseline Equivalence:
ANOVAs revealed statistically significant differences between groups on teacher's age and total years teaching experience. Teachers did not differ in self-reported reading approach or time allocated for certain reading activities. However, teachers in the intervention groups incorporated phonics instruction more frequently in their general reading arts classes. No significant difference was observed on student characteristics across the three study groups. In addition, no statistically significant difference was observed among participant groups on any outcome measure.

Differential attrition:
No test for differential attrition was performed.

Posttest:
Using MANOVA to examine changes from pre- to post-test performance indicated a main effect for treatment on WRMT-R (F=8.19, p<.001) and on the TOWRE (F=4.38, p<.05). Follow-up univariate ANOVAs revealed that both PALS and TDI groups were significantly different from the control group on the following measures: CTOPP segmentation, TOWRE nonword efficiency, and WRMT-R word attack. The TDI group was significantly different from the control group on three additional tests, the WRMT-R word identification and passage comprehension subtests and the TOWRE word efficiency subtest. Thus, for PALS 3 out of 6 tests (50%) were significant, while for TDI all 6 tests (100%) were significant. Even though TDI shows stronger effects than PALS, no statistically significant difference between these two interventions was observed.

Effect size:
Among low achieving students, PALS showed moderate to strong effect sizes for the three significant sub-tests (CTOPP-segmentation: ES=.87; TOWRE-nonword efficiency: ES=.57; WRMT-R-word attack: ES=.59). On most measures, stronger effects were observed for TDI (CTOPP-segmentation: ES=.50; TOWRE-nonword efficiency: ES=.73; WRMT-R-word attack: ES=.81).

Long-term effects:
The study did not investigate long-term effects.

Study 11

Summary

McMaster et al. (2005) conducted a cluster randomized controlled trial in which 33 teachers were assigned to standard PALS, PALS + Fluency, or a control group. Reading skills were assessed at baseline and posttest.

McMaster et al. (2005) did not find significant between-group differences on any of the outcome measures.

Evaluation Methodology

Design:
Randomization/Sample size/Attrition:
This study took place in eight Metropolitan Nashville schools participating in a large-scale investigation of the First-Grade PALS reading program from October 2000 to April 2001. Four of the schools were high-poverty Title I schools, and four were middle-class Non-Title I schools. Thirty-three first-grade teachers who volunteered to participate were stratified by school type (Title I or Non-Title I) and assigned randomly within school to one of three conditions (Standard PALS: n=11; PALS + Fluency: n=11; control: n=11). However, the evaluation of nonresponders was only conducted with the Standard PALS and PALS + Fluency classrooms (n=22). A three-step process guided selection of the nonresponders:

• Selecting students at risk for unresponsiveness: Students were rank ordered based on scores of a Rapid Letter Naming (RLN) test. From within each classroom, the 8 lowest-performing students and 4 average-performing students were selected. These lowest-performing students were considered at risk for unresponsiveness to PALS (n=176), while the average-performing students served as a comparison group (n=88).
• Monitoring these students' progress: From October to December 2000, the at-risk and average-performing students' reading progress was monitored weekly.
• Identifying nonresponders among the at-risk group: Nonresponders were identified in January 2001, after students participated in PALS for three 35-min sessions per week for 7 weeks. After 7 weeks, complete monitoring data were available for 166 at-risk and 87 average students. This reflects an attrition rate of 6% and 1% for at-risk and average students, respectively. Among the at-risk students, those scoring more than .50 standard deviations below the average readers within a class on the Curriculum-Based Measurements (CBM) were identified as nonresponders (n=66).

Within the Standard PALS and PALS + Fluency classes, the 66 nonresponders were stratified by low (n=28) and very low (n=38) status based on CBM levels. Then, 22 students were assigned randomly to PALS; 22 to Modified PALS, and 22 to tutoring. Because of attrition, at the end of the study, there were 21, 15, and 20 students in the PALS, Modified PALS, and Tutoring treatment, respectively for a total of 56 nonresponders (15% attrition).

Study type /Intervention:
This study used a randomized control trial design. In the large-scale investigation that provided the context for this study, Standard PALS was compared to PALS + Fluency, which was designed to promote reading fluency. Standard PALS and PALS + Fluency were implemented 3 times per week for approximately 35 min per session for which teachers paired higher with lower performing readers to conduct the PALS activities in pairs (reciprocal roles of coach and reader). Standard PALS activities included letter-sound recognition, decoding, sight word recognition, and reading short stories. PALS + Fluency included the same activities, with two modifications. The sight words were presented in phrases rather than in isolation, and the short stories were read in a repeated reading, "Speed Game" format. Modified PALS incorporated three important alterations: First, fewer sounds and words were introduced at one time; Second, the coach modeled the sounds and words for the reader; Third, greater emphasis was placed on phonological awareness and decoding skills. Tutoring took place three times per week for 35 min per session and substituted for PALS. Tutoring included similar activities as in PALS but was designed to more closely resemble a more individualized special education-like pull-out approach in which a specially trained research assistant provided the one-to-one tutoring.

Assessment:
A battery of measures was individually administered to all study participants prior to (October 2000) and immediately following (April 2001) the treatment period. Staff did not test students whom they had tutored, to avoid examiner bias.

Sample characteristics:
Socio-demographic characteristics of the study sample are not described.

Measures:
All measures had been used in prior published research. Research assistants administering the tests were trained until an interrater agreement of >90% was achieved on all measures.

Primary outcomes:
The measures used in this study included tests of rapid naming, phonological awareness, reading words, and spelling.

• Rapid Naming: Two measures were employed, the Rapid Letter Naming (RLN) test and the Rapid Letter Sound (RLS) test.
• Phonological Awareness: Two tests measured the number of phonemes expressed correctly in 1 min as well as the ability to blend phonemes into words.
• Reading Words: The Word Identification (Word ID) and Word Attach subtests of the Woodcock Reading Mastery Test-Revised were used to measure word recognition and decoding skills (internal consistency >.80).
• Spelling: The Wechsler Individual Achievement Test spelling subtest was administered (test-retest reliability coefficient =.94).

Monitoring Measures:

• Chapter tests: A cumulative, untimed test that recorded the percentage of sounds and words read correctly in a PALS session.
• Dolch Probes: This test measures the number of words read correctly in 1 min of a pool of 126 high-frequency words.
• Nonword Fluency Probes: The test measures the total number of phonemes in nonwords pronounced correctly in 1 min (alternate form reliability=.83).

Analysis:
A three-factor nested design was used to analyze change scores, comparing pre- and posttreatment group differences. Treatment (PALS vs. Modified PALS vs. Tutoring) was nested within PALS program (Standard PALS vs. PALS + Fluency). Nonresponder status (low vs. very low) was nested within treatment. The posttreatment data were analyzed with 2 x 3 x 2 (PALS program x treatment x nonresponder status) analyses of covariance (ANCOVAs), controlling for December Dolch measurement levels.

Intention-to-treat: It is unclear whether the study followed the intent-to-treat principle. No explanation was provided for the attrition and no reported attempt was made to obtain measurements for those who dropped out during the study period.

Outcomes

Implementation fidelity:
All Standard PALS and PALS + Fluency teachers attended a full-day workshop in which the PALS procedures were described, demonstrated, and practiced. Also, research staff members attended a full-day workshop to learn the Modified PALS and Tutoring procedures. Each staff member was assigned tutoring and/or classroom support roles. Staff members provided weekly technical assistance to classroom teachers. Implementation fidelity was measured using a checklist. On average, PALS was implemented with 92% fidelity, Modified PALS with 86% fidelity, and Tutoring with 97% fidelity.

Baseline Equivalence:
Analyses of variance (ANOVAs) and Chi-square tests showed no statistically significant difference at baseline across the three study groups (PALS, modified PALS, tutoring) on any outcome measure or demographic characteristic (sex, ethnicity, English Language Learner status, Title I status, and special education status).

Differential attrition:
Attrition occurred (15%) but a test for differential attrition was not conducted.

Posttest:
No statistically significant between-group differences were found on any outcome measure. However, a power analysis confirmed that the lack of significant results can be attributed to the small number of cases in each group (PALS n=21, Modified PALS n=15, Tutoring n=20). Thus, the authors calculated effect sizes as another means to explore important between group differences. Small-to-moderate effects were found favoring (a) Tutoring over PALS on Word ID (ES=.43), Word Attack (ES=.38), blending (ES=.44), and comprehension (ES=.32); (b) Tutoring over Modified PALS on Word ID (ES=.44), blending (ES=.49), and spelling (ES=.31); and (c) Modified PALS over PALS on Rapid Letter Sound (ES=.40), Word Attack (ES=.33), and comprehension (ES=.34).

Finally, investigating the proportion of students who were identified as persistent nonresponders (scoring .50 standard deviations below average performers on CBM) at posttest was significantly (p<.05) different comparing Tutoring and PALS (50% vs. 81%) in favor of Tutoring.

Long-term effects:
The study did not conduct any long-term follow-up and therefore was not able to demonstrate sustained effects.

Study 12

Summary

Calhoon and Fuchs (2003) conducted a cluster randomized controlled trial in which 10 mathematics resource classes were randomly assigned to a hybrid version of PALS (combined with curriculum-based measurements training) or a control group. The study included three teachers who each taught both intervention and control classes. Math skills were assessed at baseline and posttest.

Calhoon and Fuchs (2003) found students in the intervention group outperformed the control group at posttest on:

• Mathematical computation scores

Evaluation Methodology
Three teachers from three high schools in a southeastern urban school district participated. These three teachers taught a total of 10 self-containing mathematics resource classes. These 10 classes were randomly assigned to an intervention and a control group and thus each teacher was teaching both intervention and control classes. The study population comprised 92 9th through 12th graders identified as having a math disability. The study used a hybrid intervention program combining PALS and curriculum-based measurements (CBM) training. PALS occurred 2 days a week for approximately 30 minutes per session while CBM training occurred once a week for 50 minutes. The PALS/CBM treatment was implemented for 15 weeks. In the control group no PALS or CBM was used.

Student's progress was measured at the beginning (pretest) and end (posttest) of the academic year using The Math Operations Test-Revised (MOT-R), the Math Concepts and Applications Test (MCAT), and the mathematics portion of the Tennessee Comprehensive Achievement Test (TCAP).

Outcomes
Using ANOVAs, a significant difference was observed between groups, with the PALS/CBM group outperforming the control group on mathematical computation scores (MOT-R) (F=7.96, p<.05). However, no difference was found on the test for mathematical concepts/applications (MCAT) and the TCAP.

Study 13

Summary

Fuchs, Fuchs, and Karns (2001) conducted a cluster randomized controlled trial in which 20 kindergarten teachers were randomly assigned to the intervention or a control group. To estimate treatment effects, a subset of students (n=168) from low, average and high achieving groups was selected. Math skills were assessed at baseline and posttest.

Fuchs, Fuchs, and Karns (2001) found compared to the control group at posttest, students in the intervention group had significantly improved:

• Mathematical skills

Evaluation Methodology
Participants were 20 kindergarten teachers in three Title 1 and two non-Title 1 schools in a southeastern metropolitan public school system. Teachers with their classrooms were randomly assigned to PALS or a control group. To estimate treatment effects, a subset (N=168) of kindergarteners (LA: N=21, AA: N=101, HA: N=31) was selected from each classroom based on performance on the mathematical section of a standardized test (SESAT). In addition, the authors separated the group of children (N=15) who had been referred for special education (DIS). No attrition seems to have occurred and thus the study complied with the intent-to-treat principle. Teachers in the control group followed the district's core curriculum while teachers in the intervention group amended the core curriculum by implementing PALS twice weekly for 15 weeks, each time for 20 minutes. To ensure fidelity, teachers received training in PALS and program implementation was monitored by research assistants.

Children completed pre- and posttests using the mathematics portion of the SESAT and posttested on the mathematics portion of the Primary 1 level of the Stanford Achievement Test. The study used ANOVAs controlling for baseline scores to estimate program effects.

Outcomes
Baseline equivalence was demonstrated by the authors. A three-way ANOVA on the pre- and posttest SESAT scores revealed a statistically significant time x condition interaction (F=4.78, p<.05), indicating that growth in mathematical skills for all PALS students (all student categories combined) exceeded that of the control group students, however, with a trivial effect size (ES=.24). Investigating the results for each student category separately, strongest effect sizes were observed for AA students (ES=.53), followed by LA students (ES=.46) and DIS students (ES=.41). A non significant, negative effect occurred for HA students (ES=-.20). However, no significant main effect for treatment was observed for the Primary Stanford posttest scores.

Study 14

Summary

Fuchs et al. (2002) conducted a cluster randomized controlled trial with 20 teachers and 360 students. Teachers were randomly assigned to the intervention or a control group implementing the district's standard curriculum. Math skills were assessed at baseline and posttest.

Fuchs et al. (2002) found, in comparison to the control group at posttest, the intervention group scored significantly higher on:

• Math skills that were aligned with the PALS curriculum

Evaluation Methodology

Participants were 20 first-grade teachers in a southeastern metropolitan public school system. Teachers with their respective classrooms were assigned randomly within each school to PALS or a control group. Students were classified as low achievers (LA: N=82), average achievers (AA: N=170), high achievers (HA: N=93), or as learning disabled (DIS: N=18). The authors analyzed all students who were present at pre- and posttest (327 [86%] of 380 students). No analysis of differential attrition was performed. The control group teachers followed the district's core curriculum while the intervention group teachers implemented PALS three times each week for 16 weeks, each time for 30 minutes. Children completed pre- and posttests on the mathematics portion of the Primary 1 level and Primary 2 level of the Stanford Achievement Test. The authors separated the 94 items of these tests into those aligned (72 items) and unaligned (22 items) with the PALS curriculum. The study used ANOVAs, controlling for baseline scores to assess treatment effects.

Outcomes
Implementation fidelity was assured through teachers training workshop and through observation and rating of the accuracy of implementation (mean accuracy of implementation was 96%). Statistical tests indicated baseline equivalence for the used outcome measures.

A three-way ANOVA (treatment, student category, time) showed a significant treatment x time interaction (F=6.52, p=.011). On the aligned portions of the Stanford Achievement Test, the improvement of PALS students exceeded that of the control group (F=5.66, p<.018) with an effect size of .31. Effect sizes for the LA, AA, and HA students were .34, .33, .31, respectively. By contrast, on the unaligned portions of the Stanford Achievement Test, the improvement of PALS students did not differ significantly from the control group (F=.09, p=.760).

Similarly, for the separate analysis of treatment effects for the subgroup of students with disabilities, a significant difference between PALS and the control group in math skills improvement was observed for the aligned portion of the Stanford Achievement Test (ES=.55), but no significant differences were observed for the unaligned scores (ES=-.13).

Study 15

Summary

Fuchs, Fuchs, Phillips et al. (1995) conducted a cluster randomized controlled trial with 40 teachers who were assigned to teacher-mediated instruction with PALS or a control group using teacher-directed instruction without PALS. Three students with low to average achievement were selected from each class to be included in the evaluation. Math skills were assessed at baseline and posttest.

Fuchs, Fuchs, Phillips et al. (1995) found, in comparison to the control group at posttest, the intervention group scored significantly higher on:

• Math acquisition and transfer skills

Evaluation Methodology

Design:
Recruitment/Sample size:
The study was conducted in nine schools in a southeastern, urban school district. From these schools, 40 general educators teaching grades 2-4 were recruited to participate in the study. To be eligible for participation, teachers had to include in their mainstream mathematics instruction at least one student with an identified learning disability. The program was administered to all students in the classroom, but only a selection of students was used for the assessments.

Study type/Randomization/Intervention:
The study employed a randomized controlled trial design. Stratified by grade level, teachers were randomly assigned to two treatments: teacher-mediated instruction with PALS (n=20) or a control group (i.e., teacher-directed instruction without PALS, n = 20). Each teacher identified three students, based on personal judgment, with different learning histories: (a) one student who was chronically low achieving in mathematics and had been classified as learning disabled (LD student) according to state regulations; (b) one student who was chronically low achieving but had never been referred for special education assessment (LA student); and (c) one student whose mathematics performance was near the middle of the class (AA student). As such, it can be assumed that the sample consisted of 120 students (40 teachers x 3 students), although this information is not explicitly stated. PALS occurred twice weekly for 25 weeks with each session lasting 25-30 minutes. Students were assigned to work in teams on a mathematic operation with one student functioning as tutor and the other being the tutee. Every 2 weeks, tutoring assignments changed to allow every student to function as tutor and tutee. Teachers in the control group implemented their usual teaching routine.

Assessment/Attrition:
Achievement tests were conducted immediately before and after the treatment in small groups administered by research assistants. It is unclear whether those conducting the measurements were blinded to students' group assignment.

The authors reported that "no students withdrew from the 25-week study," and thus attrition was of no concern.

Sample characteristics:
Data on student's age, gender, race, and socio-economic background was not provided. The demographic information displayed in Table 2 provides only limited information on teachers' characteristics. It appears that teachers were predominantly white females, responsible for classes of an average size of 24 students, had about 13 years teaching experience, and were mostly between 30 and 50 years of age.

Measures:
Validity of measurements:
All measures and scales have been used in prior published work and reliability and validity has been established.

Primary outcomes:

• Math Operations Test-Revised: This test was used to assess the level of acquisition of math skills (alpha = .86). The scores equal the number of correct answers out of 8 problems.
• Mathematics Concepts and Applications Test: This test was used as a measure of transfer skills (alpha = .90). The scores equal the number of correct answers out of 8 problems.

Secondary outcomes:

• Teacher planning: Instructional planning was measured through (1) criteria based evaluation of instructional plan sheets and (2) through the Teacher Planning Scale (alpha = .64).

Analysis:
For the analysis of group differences in students' math skills, ANOVAs were conducted with values for pre- to posttest growth. Thus, the analyses implicitly included baseline values.

Intention-to-treat: The study followed the intent-to-treat principle.

Outcomes

Implementation fidelity:
Teachers were trained in the correct implementation of PALS during a one-day workshop. In addition, a research assistant (RA) was assigned to each teacher to serve as a consultant and provided help and feedback. RAs met with teachers once every 1-2 weeks for 5 to 10 minutes. RAs also observed teachers correct implementation of the program. The percentage of correctly implemented elements was 99.57 for weekly assessment, 97.53 for biweekly student feedback, and 98.30 for tutoring. As such, a high level of implementation fidelity was demonstrated.

Baseline Equivalence:
A test for baseline equivalence showed no significant differences on any characteristics for both students and teachers across groups with one exception: Among students with learning disability (LD), the IQ test revealed significantly lower levels of IQ within the PALS group compared to the control group.

Differential attrition:
Attrition did not occur and therefore no test for differential attrition was needed.

Posttest:
Primary outcomes:
For an ANOVA on the pre- to posttest growth, the treatment main effect (d=.40; p<.01) revealed that across types of learners (LD, LA, and AA students) and types of measures (mathematics acquisition and transfer skills), students in the PALS treatment outperformed those in the control group. This pattern held for each of the student types (LD, LA, and AA students) and for both measures (mathematics acquisition [d=.62] and transfer skills [d=.40]). Although not statistically significant, some discernible differences in the effect size emerged across the two variables for students with different learning histories. Comparing PALS to the control group, effect sizes for math skills acquisition were .30, .95, and .32 for LD, LA, and AA students, respectively. Effect sizes for transfer skills were .22, .07, and .34 for the same order of student types.

Secondary outcomes:
Although the two groups of teachers provided comparable amounts of mathematics instructions, PALS teachers differed from control group teachers in their teaching plans. PALS teachers reported teaching more operations skills (p<.05), devoting more time to one-to-one instruction (p<.01), less time to independent seatwork (p<.05), and more frequently used peer-mediated instruction (p<.01). Results for the Teacher Planning Scale indicated that PALS teachers scored higher on efficient classroom organization (p < .05) and responsiveness to individual needs (p < .05).

Long-term effects:
The study did not collect long-term, follow-up data and therefore was not able to demonstrate sustained effects.

Study 16

This study compared an enhanced version of PALS that included support for specialized adaptation when student progress was inadequate to the standard PALS curriculum.

Summary

Fuchs, Fuchs, Hamlett et al. (1995) conducted a cluster randomized controlled trial with 20 teachers who were assigned to PALS with specialized adaptation or PALS without adaptation. Within each class, one student who was chronically low achieving in mathematics and classified as learning disabled was selected to be included in the evaluation. Math skills were assessed weekly during the entire study period.

Fuchs, Fuchs, Hamlett et al. (1995) found no significant between-group effects on students' math skills.

Evaluation Methodology

Design:
Recruitment/Sample size:
General educators (n=20) teaching grades 2 to 4 were recruited from four schools in a southeastern, urban school district. To be eligible for participation, teachers had to include in their mainstream mathematics instruction at least one student with an identified learning disability.

Study type/Randomization/Intervention:
The study employed a randomized controlled trial design. Stratified by grade level, teachers were randomly assigned to PALS with specialized adaptation (ADAPT; n=10) and PALS without adaptation (noADAPT; n=10). Teachers in the ADAPT group used routine and specialized teaching adaptation. Students' progress was monitored using curriculum-based measurement (CBM) and when students' progress was inadequate, support to implement adaptations for PALS was provided based on a decision rule. Teachers in the no-ADAPT group also received information about students' progress but without decision rules for implementing adaptations to PALS.

Although the program was implemented among all students, only a selection was analyzed. From each classroom, a student was selected that (a) was chronically low achieving in mathematics, as judged by the teacher, and (b) had been classified as learning disabled according to state regulations. Peer-mediated instruction occurred twice weekly with each session lasting 20 to 30 minutes. Every 2 weeks tutoring assignments changed so that all students acted as both tutors and tutees over the course of the semester. Teachers implemented routine adaptation for 25 school weeks, while specialized adaptation lasted 12 weeks for 3rd and 4th grade teachers and 6 weeks for 2nd grade teachers.

Assessment/Attrition:
Students' progress in mathematical skills was assessed using Curriculum Based Measurements (CBM) every week for the entire study period (25 weeks). Teachers administered the weekly assessments in whole-class format.

Although no information on attrition was provided, it appears that all students selected for monitoring were retained in the study.

Sample characteristics:
Teachers: Teachers in the sample taught classes with an average size of 25 students. About 35% of teachers had a Master's degree and 65% were holding a Bachelor's degree. All teachers were female and racially homogeneous (90% Caucasian, 10% African American).

Students: Students were on average 9.4 years of age and their learning disability had been identified about 2 years ago. Students were predominantly males (75%) and African American (60%) with an average IQ of 89 points.

Measures:
Primary outcome:

• CBM slope: Achievement was assessed by estimating a regression line for the Curriculum Based Measurements (CBM) digits scores across time for each student. Each CBM assessment comprises 25 mathematical problems. At grades 2, 3, and 4, respectively, students had 1.5, 2, and 3 min to complete the assessment. Performance was scored as the total number of correct digits.

Analysis:
The authors conducted one-way ANOVAs and ANCOVAs with scores for the CBM slopes to investigate group differences. Using slope measures implicitly controls for baseline values.

Intention-to-treat: Due to a lack of information on attrition, it remains unclear whether the study followed the intent-to-treat principle, but with no attrition and analysis of all subjects, this would not be a problem.

Outcomes

Implementation fidelity:
A research assistant (RA) was assigned to each teacher and served as a consultant. All teachers participated in a full-day workshop that explained the program components and their implementation. The accuracy with which teachers implemented the treatment was assessed by RAs through direct observation. In the ADAPT condition, the percentage of correctly implemented elements was 100 for CBM, 98.35 for graph and skills profile feedback, 97.81 for peer-mediated instruction, and 82.46 for specialized adaptation. In the no-ADAPT condition, the percentages of correctly implemented elements, on the first three dimensions, respectively, were 99.15, 96.70, and 98.86. As such, a high level of implementation fidelity was guaranteed.

Baseline Equivalence:
One-way analyses of variance (ANOVAs) revealed no significant differences between groups on teacher characteristics (e.g., grade taught, class size, degree earned, race, number of students with learning disability, overall classroom achievement) and student characteristics (e.g., age, grade level functioning in math, IQ, years since identified for special education).

Differential attrition:
No information on attrition was provided and no test for differential attrition was performed. However, it appears that attrition did not occur.

Posttest:
The one-way ANOVA conducted on slope values (CBM measurement points across time) indicated no significant difference for students with learning disabilities in the PALS group (no-ADAPT) compared to the PALS with special adaptation group (ADAPT). Even if the models controlled for IQ, no statistical group difference was observed.

Long-term effects:
The study did not collect long-term, follow-up data and therefore was not able to demonstrate sustained effects.

Study 17

Summary

Fuchs et al. (1997) conducted a cluster randomized controlled trial with 40 special education teachers who were assigned to PALS with training in how to offer and receive elaborated help, PALS with training in elaborated help and in methods for providing conceptual mathematical explanations, or a non-treatment control group. Within each class, four students ranging from low to high achievers in mathematics were selected to be included in the evaluation. Math skills were assessed at baseline and posttest.

Fuchs et al. (1997) found, in comparison to the control group at posttest, students in both treatment groups score higher on:

• Mathematical operations
• Mathematical concepts/applications

Evaluation Methodology

Design:
Recruitment/Sample size:
The authors recruited 40 general education teachers of grades 2-4. The location and number of schools from which teachers were recruited was not disclosed. To be eligible to participate, teachers had to include students with identified learning disabilities in their mainstream mathematics instruction.

Study type/Randomization/Intervention:
This study employed a randomized controlled trial design. Stratified by grade level, teachers were assigned to (1) peer-mediated instruction (PMI, an earlier term for PALS) with training in how to offer and receive elaborated help (PMI-E), (2) PMI with training in elaborated help and in methods for providing conceptual mathematical explanations (PMI-E + Conceptual), and (3) a non-treatment control group.

PMI was implemented in two 35-minute sessions per week for 18 weeks. Both students in a dyad served the roles of tutor and tutee within each class session. Dyads were formed so that pairs constituted of one student who required assistance while the other one was able to provide help. In the elaborated help group (PMI-E) students were specially trained to improve the PMI learning experience through methods such as explaining how to find the answer instead of just giving it, using different ways to provide an explanation, and ways to ensure an explanation was understood. In the PMI-E + Conceptual group (PMI-EC), students received additional training in how to contextualize problem situations (e.g., real-life examples), and how to represent quantities with visual images or physical materials.

Although the program was implemented with all students, each teacher identified four students for whom treatment effects were assessed: (a) one student who was chronically low achieving in mathematics and had been classified as learning disabled (LD), (b) one student who was chronically low achieving (LA), (c) one student whose mathematics performance was near the middle of the class (AA), and (d) one student whose mathematics performance was near the top of the class (HA). To verify the appropriateness of teachers' nominations of students as LA, AA, and HA, the authors collected pretest measures for all students in a class. For eight (6.7%) of the 120 nominated students, class rank did not conform to nominated status and teachers were requested to substitute these students. The 40 LD students were identified through a formal assessment rather than a nomination process.

Assessment/Attrition:
Assessments were administered immediately before (pretest) and after treatment (posttest), in small groups by research assistants (RAs), who were trained in test administration.

No information on attrition was provided.

Sample characteristics:
Teachers: Teachers were between 30 and 50 years of age, predominantly female (98%), and 23% indicated minority as racial/ethnic group. About half of the teachers (55%) were holding Bachelor's degrees while the remaining 45% indicated having Master's degrees.

Students: Students were on average 9 years of age and 70% were male. Of the study population, 63% belonged to a racial ethnic minority group. Students attended grades 2, 3, and 4. No information on socioeconomic background was provided.

Measures:
Validity of measurements:
The employed scales have been used in prior published work. However, the authors did not report validity measures (e.g., Cronbach's alpha) for the employed scales.

Primary outcomes:

• Mathematical operations: The operation subtest of the Comprehensive Mathematics Test was employed to test students' skills in addition, subtraction, multiplication, and division of whole numbers, decimals, and fractions. Performance was scored as number of correct problems.
• Concepts/applications: The concepts and applications subtest of the Comprehensive Mathematics Test used story problems and graphic scenarios to investigate students' ability to apply their mathematical skills to tangible problems. Performance was scored as number of correct problems.

Analysis:
The authors performed ANOVAs to investigate group differences. Student type (LD, LA, AA, HA) was treated as within-classroom factor, which allowed using "classroom as the unit of analysis," to estimate the effect of the program. For the post hoc analyses, Fisher least significant difference (LSD) tests were employed to evaluate pairwise comparisons for the treatment main effect.

Intention-to-treat: Due to a lack of information on attrition, it remains unclear whether the study followed the intent-to-treat principle.

Outcomes

Implementation fidelity:
A research assistant (RA) was assigned to each teacher and served as a consultant. Teachers participated in a one-day workshop in which they learned about the concepts and correct implementation of the PMI procedures. RAs assessed the accuracy with which teachers implemented PMI. For PMI-Elaborated and PMI-Elaborated + Conceptual, respectively, the percentages of correctly implemented elements were 98 and 97.

Baseline Equivalence:
One-way ANOVAs and Chi-square tests revealed no significant differences among the three groups on teachers' age level, years teaching, class size, gender, race, or highest degree earned. Tests for students demonstrated statistical equivalence for age, gender, race, grade-level functioning, IQ, and years since identified for special education.

Differential attrition:
No information on attrition was provided and no test for differential attrition was conducted.

Posttest:
Investigating growth from pretest to posttest yielded a significant effect for treatment (p<.001) for both outcome measures, although growth was greater on the concepts/application than on the operations subtest. Post hoc tests showed that the growth of the Peer Mediated Instructions with Elaborated help training (PMI-E) + Conceptual group exceeded that of both the PMI-E (ES=.32; p<.05) and the control group (ES=.73; p<.001). The growth of the PMI-E group, in turn, exceeded that of the control group (ES=.42; p<.01).

An insignificant interaction between treatment and student type (p = .07) suggested that the program was similarly effective for students of varying learning histories. Despite this insignificant finding, the authors tentatively explored differences in program effectiveness for the various student types. They found that the PMI-E + Conceptual treatment was more effective than the control treatment for LA (ES=1.15; p<.001) and HA (ES=.63; p=.001) students but not for LD or AA students. Similarly, the PMI-E + Conceptual treatment was more effective than the PMI-E treatment for LA (ES=.54; p=.049) and HA (ES=.63; p=.031) students but not for LD or AA students.

Long-term effects:
The study did not collect long-term, follow-up data and therefore was not able to demonstrate sustained effects.

Study 18

Summary

Fuchs, Fuchs, Karns et al. (2001) conducted a cluster randomized controlled trial with 20 kindergarten teachers who were assigned to the intervention or a business-as-usual control group. Within each class, a subset of students (total n=162) ranging from low to high achievers in mathematics were selected to be included in the evaluation. Math skills were assessed at baseline and posttest.

Fuchs, Fuchs, Karns et al. (2001) found, in comparison to the control group at posttest, students in the treatment group score higher on:

• Mathematics standardized test

Evaluation Methodology

Design:
Recruitment/Sample size:
Kindergarten teachers (n=20) were recruited from 3 Title 1 and 2 non-Title 1 schools in Nashville. To be included in the study, teachers had to agree to be assigned randomly to the K-PALS intervention or control group.

Study type/Randomization/Intervention:
The study employed a randomized controlled trial design. Kindergarten teachers (n=20) were randomly assigned to the K-PALS intervention group or the control group (information on group size was not provided). The control group employed the same basal math instructions but without using peer assisted learning strategies. In the intervention group, teachers implemented K-PALS with all students. As usual, only a subset of students per classroom was selected for analysis: (a) students with learning disabilities (LD, intervention group children: n=8; control group children: n=7); (b) low achievers (LA, intervention group children: n=8; control group children: n=7); (c) average achievers (AA, intervention group children: n=49; control group children: n=52); high achievers (HA, intervention group children: n=14; control group children: n=17). The program was implemented for 16 weeks with two K-PALS sessions, 20 minutes each, occurring every week.

In this study, students were paired by first ranking the children based on their mathematical competence and then pairing the highest-performing child with the lowest-performing student, the second highest-performing child with the next-to-the-lowest-performing child, and so forth. Every third week, however, teachers used an alternative strategy, in which each student was paired with the next highest-performing student. However, tutoring roles were reciprocal so that both students in each pair functioned as tutor and tutee within every session.

Assessment/Attrition:
Assessments were conducted prior to program implementation (pretest) and after the program was completed (posttest). It is unclear who conducted the assessments and whether these individuals were blind to students' group assignment.

No information on attrition was provided.

Sample characteristics:
No information on sample characteristics for teachers and students was provided.

Measures:
Validity of measurements:
The employed scale has been used in prior published work. However, reliability and validity measures (e.g., Cronbach's alpha) were not reported.

Primary outcomes: (aside from mentioning the names of the used scales no information was provided)

• The mathematics portion of the Primary 1 level of the Stanford Achievement Test was employed to evaluate changes in children's mathematical skills.

Analysis:
The statistical methods employed for this analysis were not reported. However, based on other works by Fuchs et al. it seems likely that ANOVAs were employed as a commonly used methodology.

Intention-to-treat: Due to a lack of information on attrition, it is unclear whether the study followed the intent-to-treat principle.

Outcomes

Implementation fidelity:
The authors report that "with the exception of one teacher, K-PALS was conducted accurately: Teachers implemented lessons well, and children worked on K-PALS gameboards in the manner in which the activities had been designed." However, no measures to quantify implementation fidelity were reported.

Baseline Equivalence:
A test for baseline equivalence was not conducted.

Differential attrition:
No data on attrition was reported and a test for differential attrition was not performed.

Posttest:
The results from the mathematics standardized achievement test demonstrated a stronger improvement in mathematical skills among students in the K-PALS intervention group compared to the control group (significance levels were not reported). This difference was evident across all student types (e.g., LD, LA, AA, HA).

Long-term effects:
The authors mention that they also tested "as many children as [they] could find again in the subsequent fall." At that time, when children were beginning first grade, K-PALS students continued to outperform their counterparts of the control group.