An early literacy tutoring program designed to strengthen students' early reading and comprehension skills through daily tutoring sessions.
Blueprints: Promising
Dr. Claire Hagen Alvarado, Director
Literacy First
The University of Texas at Austin
3925 W. Braker Ln, Suite 3.801
Austin, TX 78759
(512) 471-6190
cha@austin.texas.edu
https://www.literacyfirst.org/
Dr. Claire Hagen Alvarado
Literacy First
Literacy First, a program within the Dana Center of University of Texas (UT) Austin, trains paraprofessionals to teach students in kindergarten through Grade 2 how to read in English or Spanish. Literacy First's early literacy tutoring program is designed to strengthen students' early reading and comprehension skills (e.g., phonemic awareness, letter sound identification) through daily, 30-minute tutoring sessions. The program tracks student data, including benchmark assessments, weekly progress monitoring, attendance, and demographics. The intervention identifies students reading below grade level each fall and provides tutoring across the school year.
Literacy First, a program within the Dana Center of University of Texas (UT) Austin, trains paraprofessionals to teach students in kindergarten through Grade 2 how to read in English or Spanish. Literacy First's early literacy tutoring program is designed to strengthen students' early reading and comprehension skills (e.g., phonemic awareness, letter sound identification) through daily, 30-minute tutoring sessions. The program tracks student data, including benchmark assessments, weekly progress monitoring, attendance, and demographics. The intervention identifies students reading below grade level each fall and provides tutoring across the school year.
Literacy First is currently implemented in five school districts in Central Texas, serving more than 1,600 Grades K-2 students annually. Fifty percent of Literacy First's tutors are bilingual, allowing the program to provide support to both Spanish- and English-speaking students. Literacy First has established itself as a program that supports highly trained volunteers to implement an intensive, tailored tutoring intervention that relies on data to guide instruction.
The program model evaluated with experimental design included AmeriCorps members as paraprofessional tutors. In addition to the AmeriCorps model, Literacy First has begun to use a district capacity building model in which Literacy First staff train school district staff (full-time tutors and instructional coaches) to deliver the program. This model is essentially identical to the AmeriCorps version; however, it has not yet been evaluated with an experimental design.
Primary Evidence Base for Certification
Study 1
Caverly and Stoker (2021) found that the intervention students in Grades K-2, relative to control students, had significantly higher:
Additionally, in the underpowered sample of Grade 2 students followed to Grade 3, relative to controls, intervention students scored higher (approaching significance) on:
Primary Evidence Base for Certification
The one study Blueprints has reviewed (Study 1) meets Blueprints evidentiary standards (specificity, evaluation quality, impact, dissemination readiness). The study was done by the developer.
Study 1
Caverly and Stoker (2021) used a multisite randomized controlled trial across 22 elementary schools in Texas to estimate the impact of Literacy First on Grades K-2 students' literacy outcomes and longer-term Grade 3 reading and math outcomes for a subsample of Grade 2 students. A total of 520 kindergarten students, 504 Grade 1 students, and 798 Grade 2 students were randomly assigned within schools and grades to intervention or business-as-usual control conditions. Assessments occurred at the beginning and end of school year for Grades K-2 and at the end of the school year in Grade 3.
Study 1
Caverly, S., & Stoker, G. (2021). Literacy First: Evaluation Summary Report. American Institutes for Research.
School: Instructional Practice
*
Risk/Protective Factor was significantly impacted by the program
See also: Literacy First Logic Model (PDF)
Subgroup differences in program effects by race, ethnicity, or gender (coded in binary terms as male/female) or program effects for a sample of a specific race, ethnic, or gender group.
Study 1 (Caverly and Stoker, 2021) found subgroup effects by using a homogenous sample with 75% or more of Hispanic students classified as economically disadvantaged.
Sample demographics including race, ethnicity, and gender for Blueprints-certified studies:
Most of the sample in Study 1 (Caverly & Stoker, 2021) was Hispanic (86%).
Pre-service training includes 60 hours of professional development that teaches the foundations of literacy and an introduction to Literacy First assessments and curriculum. Additionally, tutors build relationships with their campus teams, their Literacy First Coaches, and Literacy First program staff. They also receive support for behavior management, trauma-informed care, school safety, and how to navigate their school campus. The training is differentiated by language when appropriate, including specific instruction in Spanish for bilingual tutors. Pre-service training may take place in an in-person or virtual setting.
After pre-service training, tutors enter their campuses and begin assessing all students within a grade level. They receive careful support from their coaches, including a required observation that monitors the fidelity of assessment administration. They then build their caseloads and begin teaching daily 1:1 lessons. Coaches complete observations using fidelity checklists for lessons and spend at least a half a day on each campus working side-by-side with the tutors as they learn the program and become more independent.
Source: Washington State Institute for Public Policy
All benefit-cost ratios are the most recent estimates published by The Washington State Institute for Public Policy for Blueprint programs implemented in Washington State. These ratios are based on a) meta-analysis estimates of effect size and b) monetized benefits and calculated costs for programs as delivered in the State of Washington. Caution is recommended in applying these estimates of the benefit-cost ratio to any other state or local area. They are provided as an illustration of the benefit-cost ratio found in one specific state. When feasible, local costs and monetized benefits should be used to calculate expected local benefit-cost ratios. The formula for this calculation can be found on the WSIPP website.
The initial training cost of $1,364 for one campus of 2-4 tutors includes funds for meetings space, food, and materials. The cost would vary with hybrid and remote training.
Curriculum cost for 2-4 tutors is $291.
An annual license fee of $60 provides access to Literacy First data services platform.
Most Literacy First costs are based on salaries for the expert literacy coaches. The start-up costs listed above do not include the expense of paying tutors.
Literacy First publishes the curriculum within the University of Texas, and costs are minimal. Schools may need updated books occasionally, but the cost would be $100 or less.
The model works best when teams of 2-4 tutors work full-time on one campus, with each tutor serving approximately 20 students.
UT Literacy First experts provide 60 hours of pre-service training, weekly coaching, and 3 data analysis meetings. Cost is $26,312. Additionally, estimated local travel for the UT Literacy First experts to visit campuses weekly is $1,800 (price will vary for virtual and hybrid coaching).
An indirect cost of $2,673 for programs operating within the University of Texas includes equipment, utilities, general administration, and accounting.
Schools need a dedicated, quiet space for tutoring. The best option is a classroom that is close in proximity to the K-2 classrooms.
Items do not include the cost of paying tutors. Within the AmeriCorps grant, tutors are provided to schools in Central Texas; tutors cannot be provided beyond the current grant parameters to other geographic regions. As an alternative, Literacy First has implemented the same program with paraprofessionals that are paid as employees of school districts. The model works best when teams of 2-4 paraprofessionals work as full-time tutors on one campus.
Every week throughout the school year, instructional coaches on the Literacy First staff monitor assessment and instruction, providing guidance and job-embedded training. Staff are on-site with tutors every week, observing lessons, modelling lessons, coaching, and supporting. Coaches provide caseload-review sessions with each tutor every 3 to 4 weeks to review all of the data collected with each student (weekly progress-monitoring data, anecdotal notes, lesson notes, etc.), and to provide coaching around individualizing lessons for each student based on the needs evident in the data. This weekly job-embedded support ensures that tutors are always receiving the guidance and advice from expert staff that helps them to improve their practice for as long as they are tutors.
The goal in working with districts is to build the capacity of the paraprofessionals, campus, and district staff to orchestrate the Literacy First program with minimal support. With consistent staffing and district/campus buy-in and training, costs can be decreased by about $10,000 per campus per year. The flagship district has used this gradual release model for five years and is now operating independently with a sustaining cost of $2,000 per campus for data services and curriculum costs.
Fidelity checking is provided with a rubric and feedback form for both assessment and instruction observations. Every tutor is observed (using a fidelity checklist) several times per year administering assessments to ensure alignment and fidelity across the program. Additionally tutors are observed (again, using a fidelity checklist) throughout the year - more intensively at the beginning of the year, but even as the year unfolds Coaches follow a rubric while observing lessons because the rubric is built into the feedback form.
An annual license fee of $60 provides access to Literacy First data services platform.
No information is available
No information is available
This example provides the first year costs due to the University of Texas for one elementary school with four tutors delivering the Literacy First program to Grade K-2 students. Each tutor works with 20 students throughout the year. Costs do not include payments to tutors. In the research evaluation certified by Blueprints, AmeriCorps members provided tutoring. More recently, paraprofessionals already working within the schools have been trained to deliver the program.
| Pre-service Training, Coaching, and Data Analysis Meetings | $26,312.00 |
| Travel for UT Literacy First experts | $1,800.00 |
| Initial Training - space, food, materials | $1,364.00 |
| Curriculum | $291.00 |
| Annual License | $60.00 |
| UT Indirect Costs | $2,673.00 |
| Total One Year Cost | $32,500.00 |
For one school delivering Literacy First to 80 students, the total Year 1 expense would be $32,500 (excluding tutor payment) and the per student expense would be $406.25.
No information is available
Schools and districts have leveraged various federal formula funds in the past, including ESSER (Elementary and Secondary School Emergency Relief) funds through the CARES (Coronavirus Aid Relief and Economic Security) Act to hire tutors.
Literacy First often seeks out philanthropic support to offset the costs of the program.
Dr. Claire Hagen AlvaradoDirectorLiteracy First3925 W. Braker Ln, Suite 3.801Austin, TX 78759(512) 232-2286cha@austin.texas.edu
An early literacy tutoring program designed to strengthen students' early reading and comprehension skills through daily tutoring sessions.
Students in Grades K-2 reading below grade level each fall.
Subgroup differences in program effects by race, ethnicity, or gender (coded in binary terms as male/female) or program effects for a sample of a specific race, ethnic, or gender group.
Study 1 (Caverly and Stoker, 2021) found subgroup effects by using a homogenous sample with 75% or more of Hispanic students classified as economically disadvantaged.
Sample demographics including race, ethnicity, and gender for Blueprints-certified studies:
Most of the sample in Study 1 (Caverly & Stoker, 2021) was Hispanic (86%).
Early literacy gaps
School: Instructional Practice
*Risk/Protective Factor was significantly impacted by the program
Literacy First, a program within the Dana Center of University of Texas (UT) Austin, trains paraprofessionals to teach students in kindergarten through Grade 2 how to read in English or Spanish. Literacy First's early literacy tutoring program is designed to strengthen students' early reading and comprehension skills (e.g., phonemic awareness, letter sound identification) through daily, 30-minute tutoring sessions. The program tracks student data, including benchmark assessments, weekly progress monitoring, attendance, and demographics. The intervention identifies students reading below grade level each fall and provides tutoring across the school year.
Literacy First, a program within the Dana Center of University of Texas (UT) Austin, trains paraprofessionals to teach students in kindergarten through Grade 2 how to read in English or Spanish. Literacy First's early literacy tutoring program is designed to strengthen students' early reading and comprehension skills (e.g., phonemic awareness, letter sound identification) through daily, 30-minute tutoring sessions. The program tracks student data, including benchmark assessments, weekly progress monitoring, attendance, and demographics. The intervention identifies students reading below grade level each fall and provides tutoring across the school year.
Literacy First is currently implemented in five school districts in Central Texas, serving more than 1,600 Grades K-2 students annually. Fifty percent of Literacy First's tutors are bilingual, allowing the program to provide support to both Spanish- and English-speaking students. Literacy First has established itself as a program that supports highly trained volunteers to implement an intensive, tailored tutoring intervention that relies on data to guide instruction.
The program model evaluated with experimental design included AmeriCorps members as paraprofessional tutors. In addition to the AmeriCorps model, Literacy First has begun to use a district capacity building model in which Literacy First staff train school district staff (full-time tutors and instructional coaches) to deliver the program. This model is essentially identical to the AmeriCorps version; however, it has not yet been evaluated with an experimental design.
Literacy First addresses the early reading gap which is growing in U.S. education systems. Students who do not read well by third grade are more likely to drop out of high school while students who graduate high school are more likely to be employed and more fully contribute to their communities. The program uses intensive full-year tutoring in Grades K-2 to bring students up to grade-level reading by third grade.
Primary Evidence Base for Certification
The one study Blueprints has reviewed (Study 1) meets Blueprints evidentiary standards (specificity, evaluation quality, impact, dissemination readiness). The study was done by the developer.
Study 1
Caverly and Stoker (2021) used a multisite randomized controlled trial across 22 elementary schools in Texas to estimate the impact of Literacy First on Grades K-2 students' literacy outcomes and longer-term Grade 3 reading and math outcomes for a subsample of Grade 2 students. A total of 520 kindergarten students, 504 Grade 1 students, and 798 Grade 2 students were randomly assigned within schools and grades to intervention or business-as-usual control conditions. Assessments occurred at the beginning and end of school year for Grades K-2 and at the end of the school year in Grade 3.
Primary Evidence Base for Certification
Study 1
Caverly and Stoker (2021) found that the intervention students in Grades K-2, relative to control students, had significantly higher letter sound fluency, decoding fluency, nonsense word fluency, oral reading fluency, and whole word reading. Nearing significance, intervention students followed to Grade 3 scored higher on reading and mathematics tests than control students.
Primary Evidence Base for Certification
Study 1
Caverly and Stoker (2021) found that the intervention students in Grades K-2, relative to control students, had significantly higher:
Additionally, in the underpowered sample of Grade 2 students followed to Grade 3, relative to controls, intervention students scored higher (approaching significance) on:
Study 1 (Caverly & Stoker, 2021) presented Hedges' g standardized effect sizes, ranging from .19-.83 (small to large).
One study meets Blueprints standards for high-quality methods with strong evidence of program impact (i.e., "certified" by Blueprints): Study 1 (Caverly & Stoker, 2021). The sample included elementary school students in grades K-2 with below-level reading skills. The study took place in Austin, Texas, and compared the treatment group to a business-as-usual control group.
Blueprints: Promising
Dr. Claire Hagen Alvarado, Director
Literacy First
The University of Texas at Austin
3925 W. Braker Ln, Suite 3.801
Austin, TX 78759
(512) 471-6190
cha@austin.texas.edu
https://www.literacyfirst.org/
Certified
Caverly, S., & Stoker, G. (2021). Literacy First: Evaluation Summary Report. American Institutes for Research.
Summary
Caverly and Stoker (2021) used a multisite randomized controlled trial across 22 elementary schools in Texas to estimate the impact of Literacy First on Grades K-2 students' literacy outcomes and longer-term Grade 3 reading and math outcomes for a subsample of Grade 2 students. A total of 520 kindergarten students, 504 Grade 1 students, and 798 Grade 2 students were randomly assigned within schools and grades to intervention or business-as-usual control conditions. Assessments occurred at the beginning and end of school year for Grades K-2 and at the end of the school year in Grade 3.
Caverly and Stoker (2021) found that the intervention students in Grades K-2, relative to control students, had significantly higher:
Additionally, in the underpowered sample of Grade 2 students followed to Grade 3, relative to controls, intervention students scored higher (approaching significance) on:
Evaluation Methodology
Design:
Recruitment: A total of 22 elementary schools in the Austin Independent School District in Texas participated in the study. All Grades K-2 students who scored at the "Tier 2 level" on the beginning-of-the-year assessment (i.e., below grade level) were eligible to receive program services, which included 520 kindergarten students and 504 Grade 1 students. Initially, 1,046 Grade 2 students were eligible across two cohorts, but due to exceeding school tutoring capacities, a total of 798 Grade 2 students were randomly selected for the study and had parental consent.
The study was conducted with one cohort of kindergarten and Grade 1 students and two cohorts of Grade 2 students (in 2018 and 2019). Grade 2 students in Cohort 1 were followed through Grade 3. The original study design followed Cohort 2 through Grade 3, but due to COVID-19, students did not complete the State of Texas Assessments of Academic Readiness (STAAR) in the 2019-20 school year. Therefore, Grade 3 students were Cohort 1 of the initial Grade 2 sample.
Assignment: In a multisite randomized trial, students in grades K-2 were randomly assigned within grade at each school, to either a treatment group that received Literacy First tutoring (n=260 in kindergarten; n=252 in Grade 1; n=397 in Grade 2) or a business-as-usual control group (n=260 in kindergarten, n=252 in Grade 1; n=401 in Grade 2). Students in the control condition were eligible for other types of assistance typically available to students in the participating schools (e.g., literacy coach support).
Assessments/Attrition: Pretest and posttest assessments were completed at the beginning and end of the school year. Attrition was presented separately for each of the three grades. In kindergarten and Grade 1 samples, less than 1% of the randomized student samples (n=4-5 students) attrited between random assignment and pretest. Overall attrition rates between randomization and posttest were about 16% (n=81) for kindergarten students, 14% (n=71) for Grade 1 students, 21% (n=169) for Grade 2 students (across cohorts 1 and 2), and 39% (n=205) for Grade 2 students followed into Grade 3 (i.e., cohort 1 of initial Grade 2 sample).
Sample:
Overall, the kindergarten students were primarily Hispanic (85%) and economically disadvantaged (94.5%), had limited English proficiency (57.5%), and did not participate in special education (89.5%). Grade 1 students were primarily Hispanic (84%) and economically disadvantaged (92%), had limited English proficiency (52%), and did not participate in special education (88.5%). Most of the Grade 2 students who participated in the study were also Hispanic (88%), and over half the students were female (54%). About 69% of the students in the sample were identified as limited English proficient, and about 10% of the students were receiving special education services. Almost all students in the study were identified as economically disadvantaged (93%).
Measures:
The study used existing administrative data from the Austin Independent School District and primary data collected from students. Administrative data from the school district included STAAR Grade 3 Reading and Mathematics data. Grades K-1 early literacy skills data and Grade 2 oral reading fluency data were collected by the intervention program staff. Grade 2 reading comprehension data were collected by a partner evaluator (American Institutes of Research) staff. Students completed either the English or Spanish versions of the assessments depending on their English proficiency level.
Specific shorter-term outcomes in each grade were letter sound fluency and decoding fluency in kindergarten; nonsense word fluency, oral reading fluency, and whole words reading in Grade 1; and oral reading fluency and reading comprehension in Grade 2. Longer-term reading and math outcomes were assessed a year later in Grade 3.
Analysis:
Two-level hierarchical linear models were used for analyses, with students at level 1 and schools at level 2. Level-1 covariates were student-level background characteristics (race/ethnicity, gender, special education status, and economic disadvantage), students' beginning-of-the-year assessment scores, indicators for prior Literacy First tutoring (Grades 1 and 2 only), an indicator for students who completed the assessments in Spanish, and an indicator for students in cohort 2 (Grade 2 students only). To measure assessment scores on the same scales, students' test scores were converted to standardized z-scores using sample-based means and standard deviations on each of the assessments, separately by English and Spanish version language. Analyses were conducted separately for each grade, K-2, and for the longer-term reading and math outcomes in Grade 3.
Intent-to-Treat: The authors stated that students who took the pretest and posttest assessments in different languages did not have comparable scores and were not included in the analyses. The numbers were small in grades K-1 (less than 1%) and grade 2 (4%) and came similarly from both conditions. However, the long-term outcome analyses through grade 3 dropped 15% because of tests taken in different languages (13% from the treatment, and 16% from the control).
Outcomes
Implementation Fidelity:
Not discussed.
Baseline Equivalence:
Tests for student baseline equivalence used the analysis samples for each of the four grades K-3 (Exhibits/Tables 6-10). For kindergarten, Grade 2 and Grade 3 students, there were no statistically significant differences between students in the treatment and control groups on any of the student characteristics or outcomes. For Grade 1 students, one of 17 tests showed differences at baseline, with more White students in the treatment compared to control group.
Differential Attrition:
The authors reported differential attrition rates between study conditions for students in each of the three grades (pgs. 9-10). Between treatment and control groups, differential attrition rates were 2.9% for kindergarten students, 1.1% for Grade 1 students, and 4% (for the oral fluency outcome) and 5.7% (for the reading comprehension outcome) for Grade 2 cohort 1 and 2 students. Finally, students followed into Grade 3 (cohort 1 of initial Grade 2 sample) had a 0.2% differential attrition rate between conditions (Exhibit/Table 5). No significance tests were reported, and the authors noted that the combination of overall attrition and condition differences did not indicate potential bias according to What Works Clearinghouse standards. Note that the authors' reports of attrition and differential attrition rates do not include those few kindergarten (n=5) and Grade 1 (n=4) students who attrited between randomization and pretest.
In addition, the tests for baseline equivalence in the analysis samples suggested that differential attrition was not a serious problem but lacked a direct comparison to baseline equivalence for the randomized sample.
Posttest:
For all seven short-term (pretest to end-of-school-year posttest) literacy outcomes using separate kindergarten, Grade 1, and Grade 2 samples, students in the intervention group performed better than students in the control group. Specifically, kindergarten intervention students showed higher letter sound fluency (Hedges' g = .83) and decoding fluency (Hedges' g = .53); Grade 1 intervention students scored higher on nonsense word fluency (Hedges' g = .33), oral reading fluency (Hedges' g = .38), and whole word reading (Hedges' g = .24); and Grade 2 intervention students had higher letter sound fluency (Hedges' g = .33) and decoding fluency (Hedges' g = .19).
Long-Term:
Students who received the intervention in Grade 2 scored higher on reading and math in Grade 3 than control group students; however, these effects were not statistically significant. Given that sample was limited to Cohort 1 students, these analyses were somewhat underpowered.