A school-based program designed to improve reading ability among elementary school students by providing teachers with training, professional development, and a research-based curriculum.
Blueprints: Promising
What Works Clearinghouse: Meets Standards Without Reservations - Positive Effect
Delonda Morton
McGraw-Hill Education
National Director for Professional Development
8787 Orion Place
Columbus, OH 43240
(815) 258-1010
delonda.morton@mheducation.com
For McGraw-Hill Education Learning Specialist:
www.mheducation.com/prek-12/explore/open-court/product.html
McGraw-Hill Education
Open Court Reading (OCR) is a phonic-based K-3 curriculum. It includes age-appropriate materials for students, training in pedagogy for teachers, and workshops for professional development of teachers. The OCR curriculum includes three components: Foundational Skills, Reading and Responding, and Language Arts.
Open Court Reading (OCR) is a phonic-based K-3 curriculum. It includes age-appropriate materials for students, training in pedagogy for teachers, and workshops for professional development of teachers. The OCR curriculum includes three components: Foundational Skills, Reading and Responding, and Language Arts. Foundational Skills activities, depending on the grade level, build skills in phonemic awareness, sounds and letters, phonics, fluency, and word knowledge. Reading and Responding focuses instruction on building background, thinking about text prior to reading, developing vocabulary, reading from the student anthology, and emphasizing reading for understanding through complex texts. The Language Arts section emphasizes the writing process; spelling; grammar usage and mechanics; additional vocabulary; penmanship; and listening, speaking, and viewing.
Primary Evidence Base for Certification
Study 1
Borman et al. (2008) found, compared to the control group, students in the treatment classrooms had significant improvements in:
Primary Evidence Base for Certification
Of the three studies Blueprints has reviewed, one study (Study 1) meets Blueprints evidentiary standards (specificity, evaluation quality, impact, dissemination readiness) and was conducted by independent evaluators.
Study 1
Borman et al. (2008) conducted a multisite cluster-randomized controlled trial in which 57 elementary school classrooms with a total of 1,099 children from grades 1 through 5 were randomly assigned to the treatment condition or a control condition in which teachers continued with instruction as usual. Assessments completed at pretest and posttest measured reading and vocabulary skills.
Study 1
Borman, G. D., Dowling, N. M., & Schneck, C. (2008). A multisite cluster randomized field trial of Open Court Reading. Educational Evaluation and Policy Analysis, 30, 389-407.
School: Poor academic performance*
School: Instructional Practice
*
Risk/Protective Factor was significantly impacted by the program
Sample demographics including race, ethnicity, and gender for Blueprints-certified studies:
In Study 1 (Borman et al., 2008), treatment classrooms were composed of 71% minority students and control classrooms 74% minority students. No information was provided on gender.
Initial Teacher Training
McGraw-Hill Education recommends initial implementation training for each teacher. The purpose of this one day training will cover program materials, convey the structure and management of Open Court Reading, and outline the most effective teaching strategies for the curriculum.
The initial one-day training is negotiated with the product order but it could be up to $2500.
Onsite Support
Following the initial training, McGraw-Hill Education can provide support to schools through site visits designed to ensure Open Court Reading is implemented effectively. Curriculum specialists can provide classroom demonstrations for individuals or groups of teachers. These are often followed by debriefing sessions to discuss the content, teaching techniques, and observations. This partnership between teacher and curriculum specialists ensures the most effective teaching strategies are achieved for the students. Grade level and faculty meetings are used to address specific content needs of the staff.
Once McGraw-Hill Education receives information on the grades purchased and number of participants we will work with your District to determine the number of in-kind professional development days. At any time during the adoption if your District feels there is an identified need for additional support, McGraw-Hill Education will provide additional support at the cost of $2,500/day/curriculum specialist.
Source: Washington State Institute for Public Policy
All benefit-cost ratios are the most recent estimates published by The Washington State Institute for Public Policy for Blueprint programs implemented in Washington State. These ratios are based on a) meta-analysis estimates of effect size and b) monetized benefits and calculated costs for programs as delivered in the State of Washington. Caution is recommended in applying these estimates of the benefit-cost ratio to any other state or local area. They are provided as an illustration of the benefit-cost ratio found in one specific state. When feasible, local costs and monetized benefits should be used to calculate expected local benefit-cost ratios. The formula for this calculation can be found on the WSIPP website.
The curriculum price ranges from $108-150 per student and the price for materials ranges from $597-1200 per classroom.
If customers purchase the student materials in a package, they will receive 6-year student licenses to access the online student material. Licenses of different lengths are available for purchase.
The ideal classroom for Open Court Reading would have the ability to have a separate space for a workshop or be able to transform the classroom into a workshop area. The earlier grades should have a group reading area, typically a carpeted area, to utilize the Big Books in a meaningful way.
If the school decides to purchase student workbooks, they need to purchase new workbooks annually. The Skills Practice and Lesson/Unit Assessment come in workbook format. Workbooks currently cost approximately $10 per student. Teachers will receive blackline masters and if they prefer to only use those, they can.
Qualifications: Lead teachers and teachers delivering the intervention are typically certified teachers.
Ratios: Open Court Reading is typically delivered in a regular classroom setting so would be implemented at the student to teacher ratios in the District.
None.
The Online Professional Learning Environment provides free 24/7 support throughout the implementation. If needed, Districts can purchase additional support from McGraw-Hill Education at the cost of $2,500/day/curriculum specialist.
There is no required fidelity monitoring and evaluation fee, but if sites want support for fidelity monitoring, they can purchase support at the cost of $2,500 per day.
The teacher package and the student packages come with a 6-year license to access the online Open Court Reading assets. If the school decides to purchase a shorter subscription or the 6 years is up, they will have to renew their subscription.
No information is available
None.
The following example is for a District implementing Open Court Reading in 4 elementary schools for grades K - 3, with 640 students and 32 teachers. The cost of teacher staff time is not included in the estimate as it is assumed that the District is implementing the program as part of its core curriculum during the school day.
| Teacher Training (2 trainings x $2500 per day) | $5,000.00 |
| Travel reimbursement for Trainers | $2,400.00 |
| Curricula ($130 X 640 students) | $83,200.00 |
| Classroom Materials ($900 X 32 classrooms) | $28,800.00 |
| Site Visit from Curriculum Specialist ($2500 x 4 schools) | $10,000.00 |
| Travel Reimbursement for Curriculum Specialist | $1,200.00 |
| Total One Year Cost | $130,600.00 |
With 640 students served in the first year, the cost per student would be $204 per student. Note that the cost of the intervention would decline significantly beyond the first year of implementation after the initial curricula and training costs have been paid.
Open Court Reading is a comprehensive reading, writing and language arts curriculum for grades K - 3. It is delivered by teachers in the classroom and is generally supported by core professional development, curricula and staffing resources within schools. The recent passage of the Every Student Succeeds Act, which reauthorized the federal Elementary and Secondary Education Act, places new requirements on the use of evidence-based programs in federally funded education programs. State and local education agencies are now required to use evidence-based programs, particularly in schools in need of improvement.
The critical resource commitment required to implement Open Court Reading is the allocation of teacher time for training and delivering the intervention. Open Court Reading can be utilized as the part of the core language arts curriculum, with teachers delivering the programs as part of their regular school day.
State education funds allocated to local school systems as well as locally-appropriated public school funding can support Open Court Reading. Professional development funds can also be used for teacher training.
Formula Funds:
Discretionary Grants: Federal discretionary grants from the Department of Education can be used to fund the initial training and materials. Discretionary funds targeting literacy as well as improvement for struggling schools can be used.
McGraw-Hill EducationCustomer ServicePO Box 182605Columbus, Ohio 43218USA(800) 338-3987(609) 308-4480customer.service@mheducation.com www.OpenCourtReading.com
A school-based program designed to improve reading ability among elementary school students by providing teachers with training, professional development, and a research-based curriculum.
The program targets children in early elementary grades.
Sample demographics including race, ethnicity, and gender for Blueprints-certified studies:
In Study 1 (Borman et al., 2008), treatment classrooms were composed of 71% minority students and control classrooms 74% minority students. No information was provided on gender.
The program intends to improve later school outcomes by improving the quality of reading instruction among early learners.
School: Poor academic performance*
School: Instructional Practice
*Risk/Protective Factor was significantly impacted by the program
Open Court Reading (OCR) is a phonic-based K-3 curriculum. It includes age-appropriate materials for students, training in pedagogy for teachers, and workshops for professional development of teachers. The OCR curriculum includes three components: Foundational Skills, Reading and Responding, and Language Arts.
Open Court Reading (OCR) is a phonic-based K-3 curriculum. It includes age-appropriate materials for students, training in pedagogy for teachers, and workshops for professional development of teachers. The OCR curriculum includes three components: Foundational Skills, Reading and Responding, and Language Arts. Foundational Skills activities, depending on the grade level, build skills in phonemic awareness, sounds and letters, phonics, fluency, and word knowledge. Reading and Responding focuses instruction on building background, thinking about text prior to reading, developing vocabulary, reading from the student anthology, and emphasizing reading for understanding through complex texts. The Language Arts section emphasizes the writing process; spelling; grammar usage and mechanics; additional vocabulary; penmanship; and listening, speaking, and viewing.
Many of the inequalities in school and society can be traced to the first few years of formal schooling and children's initial experiences learning to read. The Open Court Reading curriculum is intended to improve reading outcomes among elementary school students.
Primary Evidence Base for Certification
Of the three studies Blueprints has reviewed, one study (Study 1) meets Blueprints evidentiary standards (specificity, evaluation quality, impact, dissemination readiness) and was conducted by independent evaluators.
Study 1
Borman et al. (2008) conducted a multisite cluster-randomized controlled trial in which 57 elementary school classrooms with a total of 1,099 children from grades 1 through 5 were randomly assigned to the treatment condition or a control condition in which teachers continued with instruction as usual. Assessments completed at pretest and posttest measured reading and vocabulary skills.
Primary Evidence Base for Certification
Study 1
Borman et al. (2008) found, compared to the control group, the treatment group showed significant improvements in an aggregate measure across all grade levels for all three outcomes: a reading composite score, reading vocabulary, and reading comprehension.
Primary Evidence Base for Certification
Study 1
Borman et al. (2008) found, compared to the control group, students in the treatment classrooms had significant improvements in:
In Study 1, Borman et al. (2008) found very small effect sizes for reading composite (d=0.16), reading vocabulary (d=0.19), and reading comprehension (d=0.12).
One study meets Blueprints standards for high quality methods with strong evidence of program impact (i.e., "certified" by Blueprints): Study 1 (Borman et al., 2008). The study took place in rural, suburban, and urban schools in North Carolina, Georgia, Florida, Indiana, Texas, and Idaho. The treatment group was compared to an instruction-as-usual control group.
Additional Studies (not certified by Blueprints)
Study 2 Skindrud and Gersten (2006)
Skindrud, K., & Gersten, R. (2006). An evaluation of two contrasting approaches for improving reading achievement in a large urban district. The Elementary School Journal, 106,389-408.
Study 3 Vaden-Kiernan et al. (2018)
Vaden-Kiernan, M., Borman, G., Caverly, S., Bell, N., Sullivan, K., Ruiz de Castilla, V., . . . Jones, D. H. (2018). Findings from a multiyear scale-up effectiveness trial of Open Court Reading. Journal of Research on Educational Effectiveness, 11(1), 109-132.
Blueprints: Promising
What Works Clearinghouse: Meets Standards Without Reservations - Positive Effect
Delonda Morton
McGraw-Hill Education
National Director for Professional Development
8787 Orion Place
Columbus, OH 43240
(815) 258-1010
delonda.morton@mheducation.com
For McGraw-Hill Education Learning Specialist:
www.mheducation.com/prek-12/explore/open-court/product.html
Certified Borman, G. D., Dowling, N. M., & Schneck, C. (2008). A multisite cluster randomized field trial of Open Court Reading. Educational Evaluation and Policy Analysis, 30, 389-407.
Skindrud, K., & Gersten, R. (2006). An evaluation of two contrasting approaches for improving reading achievement in a large urban district. The Elementary School Journal, 106, 389-408.
Vaden-Kiernan, M., Borman, G., Caverly, S., Bell, N., Sullivan, K., Ruiz de Castilla, V., . . . Jones, D. H. (2018). Findings from a multiyear scale-up effectiveness trial of Open Court Reading. Journal of Research on Educational Effectiveness, 11(1), 109-132.
Summary
Borman et al. (2008) conducted a multisite cluster-randomized controlled trial in which 57 elementary school classrooms with a total of 1,099 children from grades 1 through 5 were randomly assigned to the treatment condition or a control condition in which teachers continued with instruction as usual. Assessments completed at pretest and posttest measured reading and vocabulary skills.
Borman et al. (2008) found, compared to the control group, students in the treatment classrooms had significant improvements in:
Evaluation Methodology
Design:
Recruitment: The study offered free materials, training, and support to schools interested in implementing the Open Court Reading program in exchange for participation in the study. The study required participating schools to distribute materials and training only to those classrooms randomly assigned to the treatment and to abide by the data collection schedule. The study selected 6 schools.
Assignment: The study used a block randomized plan for assignment of classrooms within the 6 selected schools. Each grade level, 1 through 5, represented the blocks. Within the blocks, 57 classrooms were randomly assigned to either the control or treatment group. The treatment group received the Open Court Reading curriculum, whereas the control classrooms continued with the curriculum they had been using previously.
Attrition: To preserve anonymity, individual students were not linked at pretest and posttest. Of the originally randomized students (n=1,099), the number of students who participated in the final assessments ranged from 917 to 923, including the loss of one school, which did not want to complete posttesting, and individual students within other schools who were unavailable for testing. The analytic sample had five schools and 49 classrooms. Attrition was stated as up to 15.49% of control students and 15.78% of treatment students.
Sample: Treatment classrooms were, on average, composed of 71% minority students and 77% of students who received free or reduced-price lunches. Similarly, the composition of control classrooms was 74% minority and 76% of students who received free or reduced-price lunches.
Measures: The study used the CTBS/5, Terra Nova Reading Comprehension and Vocabulary tests at pretest and posttest. It treated the reading comprehension and vocabulary scores separately, in addition to a reading composite score. Independent assessors blind to condition gave the tests. No information on validity or reliability was presented, but the tests appear to be commonly used and likely well validated.
Analysis: Given the randomization of classrooms within schools, the study used random effects hierarchical linear models with three levels for individuals, classrooms, and school/blocks. The mean classroom pretest scores served as a fixed covariate to control for baseline outcomes, as the models could not link individual pretest and posttest scores.
Intent-to-Treat: The study included all schools and students who completed the posttest.
Outcomes
Implementation Fidelity: The study does not provide quantitative measures of implementation fidelity, however observation of classrooms found good fidelity to the training.
Baseline Equivalence: The study reported that the percentages of minorities, special education students, free-lunch participants, English as Second Language students and the mean pretest outcomes were statistically equivalent across treatments and control classrooms.
Differential Attrition: The study provided analysis of differential attrition by condition and found no significant difference (p=.87). However, without linking pretest and posttest scores, it could not test for individual differences between completers and dropouts on baseline measures.
Posttest: Models were presented for all five grades combined. The treatment variable revealed a statistically significant classroom-level effect of assignment to OCR on all three outcomes. The study expressed these estimated impacts as effect sizes. For the reading composite (d=0.16), reading vocabulary (d=0.19), and reading comprehension (d=0.12) scores, the effect sizes were small.
Long-Term: The study did not conduct a long-term follow-up on student outcomes.
Summary
Skindrud and Gersten (2006) conducted a quasi-experimental design study with 936 second and third graders. The study selected eight schools implementing the Open Court Reading curriculum that matched four schools implementing Success for All on demographic characteristics and reading scores. Assessments at pretest and posttest measured reading and language skills.
Skindrud and Gersten (2006) found that 2nd grade students in Open Court Reading schools showed significantly greater improvement than students in Success for All schools on:
Evaluation Methodology
Design:
Recruitment: The study selected 8 schools in the Sacramento City Unified School District that were implementing Open Court Reading and these were matched with 4 schools of similar socioeconomic profiles that were implementing Success for All. Little additional information is provided about recruitment of classrooms or schools. The study took place after California legislation required schools to implement one of the two programs; the 12 selected schools had all chosen to implement one of these programs during the year of the study.
Assignment: Assignment was based on school self-selection into the Open Court Reading or Success for All programs. It appears that 4 of 5 Success for All schools were used and 8 of about 54 Open Court Reading schools were then chosen as matches. Matching was based on Title I poverty criteria. For academic outcomes, 2nd and 3rd grade students (n=1614) were studied.
Attrition: Attrition over two years was stated as 41% for each of the Open Court Reading and Success for All groups. After also dropping grade-retained students, the analysis included 936 of the original 1614 students (57% completion).
Sample: The sample included 2nd and 3rd grade students. The selected schools ranged from 58% to 86% on district poverty criteria.
Measures: The study used the Stanford Achievement Test 9th edition reading and language subtests for both posttests, and the Iowa Test of Basic Skills as a pretest. No information on validity or reliability was presented, but the tests appear to be commonly used and likely well validated. Teachers gave the tests but likely had little influence over the objective test results.
Analysis: The study conducted two-by-two analyses of covariance for individuals with one between-subjects factor (reading programs) and one within-subjects factor (year of posttest). The models for individual students did not adjust for assignment of schools to the conditions. The pretest scores from the Iowa Test of Basic Skills served as a covariate. Models were estimated separately for classrooms starting the program in the 2nd grade and classrooms starting the program in the 3rd grade.
Intent-to-Treat: The study included only those students continuously enrolled for the two study years. Students who entered and/or left; were grade-retained, or absent during testing were dropped. These exclusions may violate intent-to-treat.
Outcomes
Implementation Fidelity: The study does not provide quantitative measures of implementation fidelity for Open Court Reading; however schools reported few implementation problems by the end of year 2. In Success for All classrooms, teacher observation readings were 99 on average (comparable to the average of 100 nationally).
Baseline Equivalence: Mean pretest scores were significantly different between conditions, 35.9 for Success for All schools and 30.1 for Open Court Reading schools (d = .30). The schools appeared similar on other student and teacher characteristics.
Differential Attrition: Analysis of attrition showed that there were no significant differences between rates for the two conditions or between the reading scores of dropouts in the two conditions. However after removing attriters, the mean pretest score for the sample was significantly lower. Also, tests for attrition do not appear to have included students who were excluded because they were retained in grade.
Posttest: Posttest analysis showed that in classrooms that implemented the treatment starting in 2nd grade, Open Court Reading classrooms had significantly better reading scores at posttest (p<.001) than did Success for All classrooms. Similarly, these classrooms had better language test scores (p<.001). However, for classrooms that began implementing the program in the 3rd grade, there were significant but weak differences between conditions on reading and language at only the second posttest.
Tests among students in the bottom quintile of readers showed stronger effects of the program. Additional analyses examined demand for special education services and teacher responses to surveys but did not make comparisons across groups.
Long-Term: The study did not conduct a long-term follow-up on student outcomes.
Summary
Vaden-Kiernan et al. (2018) conducted a multisite cluster randomized trial in 7 school districts across the U.S. A total of 49 elementary schools were randomized within each district to the intervention group or a control group using the standard reading curriculum. The period of observation was two years, with four classrooms (two in each age cohort) in each school randomly assessed on reading ability in the spring.
Vaden-Kiernan et al. (2018) reported no significant positive program effects.
Evaluation Methodology
Design:
Recruitment: Participating schools were recruited by district across the U.S. over a 3-year period. All districts had at least four elementary schools (K-5) with at least 44 students enrolled at each grade level, and no participating districts had implemented the intervention in the previous 3-5 years. A total of 49 elementary schools from 7 school districts participated in the study.
Assignment: Schools were randomized within districts to either the intervention group (n=25) or the control group (n=24), which implemented the standard reading curriculum for each school. Within each school, two classrooms from each grade were randomly selected to complete assessments. If the randomly selected classroom did not have at least 44 students, students from other classrooms in that grade were randomly selected until the minimum was met.
Attrition: At the school level, there was 4% attrition after two years. There was approximately 12% student attrition after one year and 37% attrition after two years.
Sample:
Approximately half of participating schools were located in the Midwest (51%) with the remainder in the South (41%) or West (8%). Approximately 37% of schools were in an urban location, and 31% were rural. Schools had an average of 30 full-time teachers with an average teacher to student ratio of 15/75. A majority of students in these schools were white (67%) and eligible for free or reduced-price lunches (59%).
Measures:
Data were collected in spring of both implementation years, from (possibly) different samples of students. Reading ability was assessed using the Group Reading Assessment and Diagnostic Evaluation, a standardized tool designed to assess four areas of reading ability (reading readiness, vocabulary, reading comprehension, and oral language) for students in grades K-6 (α=.95-.99). Demographic data on students were collected from school rosters.
Analysis:
The effects of the intervention were assessed using hierarchical linear models that controlled for schools' baseline reading scores and included cross-level interactions for subgroup analyses.
Intent-to-Treat: Two schools were dropped from analysis when they were combined into a new school.
Outcomes
Implementation Fidelity:
Fidelity was acceptable overall, with teachers averaging 3.88 (4 point scale) for quality of delivery based on three recorded lessons over the course of the school year.
Baseline Equivalence:
There were significant differences between conditions on student race/ethnicity and free/reduced price lunch status.
Differential Attrition:
No tests, but the authors report less than 3% differential attrition of students between conditions at the end of year 2.
Posttest:
While there were within-group improvements in both years, there were no significant positive impacts on standardized reading test scores compared to control group schools. There was a significant improvement in reading scores for Hispanic children, but there were also iatrogenic effects for other subgroups including first graders, females, non-English language learners, and students who did not qualify for free or reduced-price lunches.