A literacy-focused curriculum and support system designed for preschool children ages 3, 4, and 5 years old. The program is designed to enhance existing classroom curricula by specifically focusing on improving children's early literacy skills and knowledge.
Blueprints: Promising
Dr. Madelaine Cosgrove (mcosgrov@unf.edu)
or Howaida Mousa (h.mousa@unf.edu)
Florida Institute of Education at the University of North Florida
Adam W. Herbert University Center
12000 Alumni Drive
Jacksonville, FL 32224
(904) 620-2496
Curriculum information and materials available at: https://www.unf.edu/fie/ellm/plus/
Dr. Madelaine Cosgrove
Florida Institute of Education at the University of North Florida
The Early Literacy and Learning Model (ELLM) is a literacy-focused curriculum and support system targeting young children from low-income families. The program is designed to enhance existing classroom curricula by specifically focusing on children's early literacy skills and knowledge. The program is designed to be implemented year-round or during the academic year and supplements the daily activities of the classroom. Teacher support and family involvement opportunities also occur regularly throughout the year.
The ELLM program components include the following:
The Early Literacy and Learning Model (ELLM) is a literacy-focused curriculum and support system targeting preschool children ages 3, 4, and 5 years old, particularly those from low-income neighborhoods. The program is designed to enhance existing classroom curricula by specifically focusing on improving children's early literacy skills and knowledge. The program is designed to be implemented year-round or during the academic year and supplements the daily activities of the classroom. Teacher support and family involvement opportunities also occur regularly throughout the year.
The ELLM program components include the following:
The ELLM curriculum materials include a set of literacy performance standards; monthly literacy packets; targeted instructional strategies; resource guides for teachers; a classroom book-lending library; family and teacher tip sheets; and literacy calendars.
One hour of daily literacy instruction is provided to implement the ELLM literacy building blocks. Trained literacy coaches provide instructional support to preschool teachers who use the curriculum.
The ELLM program also contains a family involvement action plan. Families have access to many resources, including a classroom book-lending library that enables children to take books home daily to share with their parents. Parents receive monthly family tip sheets and calendars with suggestions for literacy activities they can engage in with their children. Parents also have the opportunity to engage in preschool site-based family activities during the school year.
Primary Evidence Base for Certification
Study 2
Cosgrove et al. (2006), found that in comparison to controls, students in the intervention group showed:
Primary Evidence Base for Certification
Of the two studies Blueprints has reviewed, one study (Study 2) meets Blueprints evidentiary standards (specificity, evaluation quality, impact, dissemination readiness). This study was done by the developer.
Study 2
Cosgrove et al. (2006) conducted a cluster-randomized trial with 48 classrooms and 466 preschool children, in which classrooms (one per school) were assigned to either the intervention or a waitlist control condition. Measures of alphabet recognition and early literacy skills were collected at baseline and posttest.
Study 2
Cosgrove, M., Fountain, C., Wehry, S., Wood, J., & Kasten, K. (2006, April). Randomized field trial of an early literacy curriculum and instructional support system. Paper presented at the annual meeting of the American Educational Research Association, San Francisco, CA.
School: Poor academic performance
Family: Parental involvement in education
School: Instructional Practice
*
Risk/Protective Factor was significantly impacted by the program
See also: Early Literacy and Learning Model Logic Model (PDF)
Sample demographics including race, ethnicity, and gender for Blueprints-certified studies:
ELLM/Plus Introduction Training (2-day): $375 per participant (with a minimum of ten participants) for training at program site plus meeting costs (food, A/V) and travel costs for trainers. Training provides not only an overview of the curriculum, but also opportunities to engage in hands-on experiences with curriculum implementation through modeling and practice activities, and using supplemental resources to enhance daily instruction. Content includes:
Strategic Coach Institute (5-day): $2,650 per participant plus travel to training and administrator or coach time. Florida Institute of Education recommends that a program administrator or a coach participate in this training that equips them to provide ongoing fidelity monitoring and training to staff.
Customized training is also available. Customized training is designed to address needs identified by administrators and/or participants overseeing implementation. Lengths of training and follow-up activities are decided on a case-by-case basis to meet implementation needs of participants.
Source: Washington State Institute for Public Policy
All benefit-cost ratios are the most recent estimates published by The Washington State Institute for Public Policy for Blueprint programs implemented in Washington State. These ratios are based on a) meta-analysis estimates of effect size and b) monetized benefits and calculated costs for programs as delivered in the State of Washington. Caution is recommended in applying these estimates of the benefit-cost ratio to any other state or local area. They are provided as an illustration of the benefit-cost ratio found in one specific state. When feasible, local costs and monetized benefits should be used to calculate expected local benefit-cost ratios. The formula for this calculation can be found on the WSIPP website.
None.
Each classroom needs sufficient space to set up a minimum of 5 functioning learning centers, a small group teaching table, word walls, and space to display children's work.
Ongoing material costs associated with ELLM/Plus implementation varies depending on replacement needs. Estimate yearly costs for replacement of books for the classroom lending library at $200 per class and replacement of classroom manipulatives at $100 per class. Recommend replacing the curriculum at least every five years as curriculum is updated every 2 to 3 years.
Qualifications: No specific requirements - intervention typically delivered by early care and education teachers.
Ratios: No specific requirements - intervention implemented in early care settings governed by ratio requirements in state licensing standards.
Time to Deliver Intervention: The ELLM/Plus model is a comprehensive curriculum that can be integrated into half or full day early learning schedules.
None.
Periodic technical assistance through email and telephone is available at no cost. The cost for on-site technical assistance designed to answer implementation questions, address site-based needs, assist in monitoring progress, and provide additional support varies depending on other needs of the site, and is billed at a rate of $100 per hour plus travel costs.
Fidelity monitoring tools and training on how to use them are included in the start-up training. Trained coaches or administrators are responsible for fidelity monitoring on an ongoing basis, including walk-throughs and completion of checklists.
None.
No information is available
The high level of turnover in the early care and education field means that new staff may need to complete start-up training on an ongoing basis. Building local capacity for ongoing training and fidelity monitoring is an efficient and cost effective strategy to sustain ongoing implementation with fidelity. FIE researchers have worked with local sites to train site coordinators and/or a cadre of coaches who then take on the responsibility for implementation support and fidelity monitoring. This strategy is particularly relevant for sites that have local early care and education quality specialists or trainers who work with local programs.
This example assumes that an early care and education program would offer the Early Literacy and Learning Model Plus in 10 classrooms operating 10 months per year with two teachers per classroom. Costs included are the start-up costs for materials and training recommended by the publishers. This model does not include ongoing costs of teachers and facilities.
| Teacher Initial Training Costs: $375 x 22 (20 teachers, 2 admin) | $8,250.00 |
| Trainer Travel: $1,000 x 2 | $2,000.00 |
| Strategic Coach Institute Training | $2,650.00 |
| Coach Travel | $1,000.00 |
| ELLM/Plus 10-Month Kit (curriculum, materials): $1,532 x 10 classrooms | $15,320.00 |
| Lending Library: $600 x 10 classrooms | $6,000.00 |
| Manipulatives: $200 x 10 classrooms | $2,000.00 |
| Total One Year Cost | $37,220.00 |
With 200 students in 10 classrooms, the additional cost to the program (beyond their existing staffing and operations cost) is approximately $186/child.
The Early Learning and Literacy Model Plus is generally integrated into the ongoing operation of early learning and education programs, so the implementation costs are supported by the funding sources that support the ongoing operations of early care programs. These include parent fees and government child-care and preschool funding. The start-up costs can potentially be covered through portions of government child-care and preschool funding that are designated to support quality improvement or administration. Public-private partnerships in which foundation partners help to support initial training and curriculum costs are another promising strategy for supporting ELLM/Plus.
Many states have created state funded preschool programs that fund school and community-based pre-kindergarten programs. States invest varying levels of resources in supporting the quality of these efforts, but there is potential for states to allocate funds or invest some portion of their existing quality or administrative funds in ELLM/Plus training, curriculum, and coaching.
Formula Funds: The core formula funds supporting early care and education can potentially support ELLM/Plus.
Discretionary Grants: Federal discretionary grants supporting early care and education and early literacy programs can potentially support ELLM/Plus. Most relevant federal discretionary grants are administered by The Child Care Bureau or The Head Start Bureau within the Department of Health and Human Services, or the Department of Education.
Foundation grants and public-private partnerships are an important financing strategy for ELLM/Plus. Early Care and Education Programs that find it difficult to identify dollars to support the initial training and curriculum purchases can potentially partner with a local foundation or corporation to support these costs.
A final category of strategies to consider are mechanisms to generate new revenue and to set-aside funding for specific populations or sets of services. Several states and localities generate funding for early care and education services through the imposition of fees on products that can result in social costs. The most popular of these "sin taxes" generally target tobacco and alcohol use. New tax and fee strategies are generally difficult to put in place, and the mechanism for establishing them depends on the state. In some states new revenue mechanisms can be put in place through ballot initiatives, while in others they require state legislation.
Survey and interview with purveyor, Florida Institute of Education.
Dr. Madelaine CosgroveFlorida Institute of Education at the University of North FloridaAdam W. Herbert University Center12000 Alumni DriveJacksonville, FL 32224USA(904) 620-2496mcosgrov@unf.eduh.mousa@unf.edu
A literacy-focused curriculum and support system designed for preschool children ages 3, 4, and 5 years old. The program is designed to enhance existing classroom curricula by specifically focusing on improving children's early literacy skills and knowledge.
The program targets preschool aged children (age 3 to 5 years,) living in low-income neighborhoods. The program has been tested with samples that are primarily African American with smaller percentages of White and Hispanic children living in urban and rural areas. There was an equal mix of male and female children who were an average of 4-5 years old.
Sample demographics including race, ethnicity, and gender for Blueprints-certified studies:
The program targets children who reside in low-income neighborhoods and are likely to come to school underprepared for literacy success. It provides an enriched learning environment and supports family involvement in learning. While the program is delivered classroom-wide, additional services are targeted to children assessed with low levels of early literacy skills.
School: Poor academic performance
Family: Parental involvement in education
School: Instructional Practice
*Risk/Protective Factor was significantly impacted by the program
The Early Literacy and Learning Model (ELLM) is a literacy-focused curriculum and support system targeting young children from low-income families. The program is designed to enhance existing classroom curricula by specifically focusing on children's early literacy skills and knowledge. The program is designed to be implemented year-round or during the academic year and supplements the daily activities of the classroom. Teacher support and family involvement opportunities also occur regularly throughout the year.
The ELLM program components include the following:
The Early Literacy and Learning Model (ELLM) is a literacy-focused curriculum and support system targeting preschool children ages 3, 4, and 5 years old, particularly those from low-income neighborhoods. The program is designed to enhance existing classroom curricula by specifically focusing on improving children's early literacy skills and knowledge. The program is designed to be implemented year-round or during the academic year and supplements the daily activities of the classroom. Teacher support and family involvement opportunities also occur regularly throughout the year.
The ELLM program components include the following:
The ELLM curriculum materials include a set of literacy performance standards; monthly literacy packets; targeted instructional strategies; resource guides for teachers; a classroom book-lending library; family and teacher tip sheets; and literacy calendars.
One hour of daily literacy instruction is provided to implement the ELLM literacy building blocks. Trained literacy coaches provide instructional support to preschool teachers who use the curriculum.
The ELLM program also contains a family involvement action plan. Families have access to many resources, including a classroom book-lending library that enables children to take books home daily to share with their parents. Parents receive monthly family tip sheets and calendars with suggestions for literacy activities they can engage in with their children. Parents also have the opportunity to engage in preschool site-based family activities during the school year.
If preschool children (especially those at-risk) regularly engage in intensive, explicit, and intentional early literacy instruction; embedded in quality programs using standards- and research-based curricula delivered with fidelity by practitioners who are provided ongoing job-embedded support; then, participating children will acquire the knowledge and skills needed to become successful readers.
Primary Evidence Base for Certification
Of the two studies Blueprints has reviewed, one study (Study 2) meets Blueprints evidentiary standards (specificity, evaluation quality, impact, dissemination readiness). This study was done by the developer.
Study 2
Cosgrove et al. (2006) conducted a cluster-randomized trial with 48 classrooms and 466 preschool children, in which classrooms (one per school) were assigned to either the intervention or a waitlist control condition. Measures of alphabet recognition and early literacy skills were collected at baseline and posttest.
Primary Evidence Base for Certification
Study 2
Cosgrove et al. (2006) found that the intervention students scored significantly higher on all measures - three early literacy scales and one alphabet recognition scale - than the students in the control group.
Primary Evidence Base for Certification
Study 2
Cosgrove et al. (2006), found that in comparison to controls, students in the intervention group showed:
In Study 2 (Cosgrove et al., 2006), at the end of the intervention year, with the exception of the differences in the TERA-3 Conventions of Print scale adjusted mean posttest scores, resulting effect sizes were 25 percent of a standard deviation or larger.
One study meets Blueprints standards for high quality methods with strong evidence of program impact (i.e., "certified" by Blueprints): Study 2 (Cosgrove et al., 2006). The study took place within three lower-income areas in Florida, in which the treatment was compared with a waitlist control condition.
Additional Studies (not certified by Blueprints)
Study 1 PCERC (2008)
Preschool Curriculum Evaluation Research Consortium (PCERC) (2008). Effects of Preschool Curriculum Programs on School Readiness (NCER 2008-2009) (pp. 99-108, C-17, C-18, D-17, D-18). National Center for Education Research, Institute of Education Sciences, U.S. Department of Education. Washington, D.C.: Government Printing Office.
The PCERC (2008) was not as detailed about the methodology used as Cosgrove et al. (2006), so some information on the randomization process as described in the PCERC (2008) study actually came from Cosgrove et al. (2006).
Blueprints: Promising
Dr. Madelaine Cosgrove (mcosgrov@unf.edu)
or Howaida Mousa (h.mousa@unf.edu)
Florida Institute of Education at the University of North Florida
Adam W. Herbert University Center
12000 Alumni Drive
Jacksonville, FL 32224
(904) 620-2496
Curriculum information and materials available at: https://www.unf.edu/fie/ellm/plus/
Preschool Curriculum Evaluation Research Consortium (PCERC) (2008). Effects of Preschool Curriculum Programs on School Readiness (NCER 2008-2009) (pp. 99-108, C-17, C-18, D-17, D-18). National Center for Education Research, Institute of Education Sciences, U.S. Department of Education. Washington, D.C.: Government Printing Office.
Certified Cosgrove, M., Fountain, C., Wehry, S., Wood, J., & Kasten, K. (2006, April). Randomized field trial of an early literacy curriculum and instructional support system. Paper presented at the annual meeting of the American Educational Research Association, San Francisco, CA.
Summary
PCERC (2008) examined 28 preschool classrooms and 244 children. Equal numbers of classrooms were assigned to the treatment and waitlist control conditions; however, as part of a pilot study (see Study 2), the treatment classrooms were randomized a year earlier than the control classrooms. Data were collected at baseline, posttest (end of preschool), and follow-up (end of kindergarten), and included measures of literacy and math skills, behavior, and teacher-child interaction.
PCERC (2008) found:
Evaluation Methodology
Design
The Early Literacy and Learning Model (ELLM) was evaluated in a randomized controlled trial the included classrooms from 28 preschools located in three geographic areas of Florida. The sampled preschools included full-day Head Start, subsidized, faith based, and public school-based early intervention programs. Randomization was done during an earlier national pilot study (2002-2003). However, two preschools dropped out during the pilot year and were replaced for the national study year (2003-2004). A total of 30 classrooms and teachers were randomized, but two dropped out leaving 28 classrooms and teachers for the analysis. The intervention teachers were randomized in 2002 and participated in the pilot study (Study 2) as well as the current study discussed here, while a new pool of classrooms was recruited in 2003 and teachers were randomized to the control condition from this pool.
A total of 297 children and 294 parents were recruited for participation in the national study. The final sample included 244 children (137 treatment, 107 control) and 243 parents. Data were collected on 243 children and 204 parents at the time of the baseline data collection. In the follow-up year of the national evaluation, the sample of schools went from 28 preschools to 119 schools with kindergarten classrooms. The sample of classrooms went from 28 preschool classrooms to 175 kindergarten classrooms. The kindergarten sample included 237 children and 236 parents from original sample of participants. Follow up data were collected on 218 children and 177 parents.
Appendix B (page B-9) reports on completion rates at pretest, posttest (end of preschool), and follow-up (end of kindergarten). For children, the rates were 100%, 92%, and 89%. It is not clear to what extent the national research team attempted to collect post-test and/or follow up data from children who left the study prematurely.
Regarding the counterfactual condition, a number of curricula were used in the control classrooms to include Creative Curriculum (Dodge, Colker and Heroman 2002), & Beyond Centers and Circletime (Phelps 2002), High Reach Learning Pre-K (High Reach Learning 1997) and High/Scope (Hohmann and Weikart 2002).
Sample
The research team recruited preschool programs from three distinct geographic locations within the state. The research team first identified elementary school neighborhoods in each geographic location (Counties A, B, and C) with low-performing schools. Using the Florida Department of Education's school grading report card system the research team identified grade D and F elementary schools in each of the three counties. It was expected that children from the preschool programs in these low-performing elementary school neighborhoods would transition into these grade D and F elementary schools during the kindergarten year of the study. As such, preschool programs within the low performing elementary school neighborhoods were randomly selected for inclusion in the sampling pool of preschool programs.
The average age of children was 4.6 years at the time of the baseline data collection and half (50%) was male. The overall sample was primarily African American (71%) with smaller percentages of White (14%) and Hispanic (8%) children. Approximately 13% of the children were reported by their parents as having a disability.
The average age of the primary caregiver (usually the mother) was 31 years. Almost half (40%) of the primary caregivers were never married; 37% were married at the time of the fall assessment data collection. More than one-third of the primary caregivers reported having had some college (36%) or had graduated from college (6%); 37% had a high school diploma or GED; and 22% had not finished high school. More than half (54%) of the primary caregivers were employed full-time; 11% were employed part-time; and 33% were unemployed. At baseline, a higher percentage of parents in the treatment group had completed some post-high school education relative to those assigned to the control group (41% vs. 29%, p < .01).
There were 28 teachers who participated in the national preschool year intervention study. All were female. The majority identified themselves as African American (64%) or White (21%). The preschool teachers had on average 11 years of teaching experience, with an average of seven years teaching preschool. Fifty percent of the teachers had a high school diploma or GED and 21% had a bachelor's degree. Many of the teachers reported having a state-awarded preschool certificate (52%); a teaching license or certificate (46%); or a Child Development Associate (CDA) credential (46%). Eighteen percent reported having no teacher certification credentials. At baseline, teachers in the treatment group had more years of experience teaching in a preschool setting relative to those assigned to the control group (9 years vs. 4 years, p < .01).
Measures
Data were collected by the national research team from parents, teachers, and schools and through direct assessments of the children. No reliability or validity information was provided on the 23 measures used; however, most are commonly used measures in educational intervention research.
Mathematics
Reading
Phonological Awareness
Language
Behavior
Overall classroom environment (baseline & posttest only)
Teacher-child interactions (baseline & posttest only)
Classroom Instruction (posttest only)
Analysis
Analysis was conducted at the individual child-level (for child specific measures) and the classroom-level (for teacher specific measures). Randomization was done at the preschool classroom level. Appendix B (page B-10) describes the use of hierarchical linear models to account for nesting of children within classrooms. Appendix B (page B-8) also states that the analyses used an intent-to-treat logic by including all randomized subjects, even if they were lost to follow-up.
The unadjusted mean scores, adjusted mean difference scores with standard errors, and effect sizes were reported. Effect sizes were calculated using repeat measures (RM) analysis or analysis of covariance (ANCOVA) depending on the data available at each time point.
Adjustments of child-level measures included the following covariates: (a) child's age, (b) gender, (c) race/ethnicity, (d) disability status (as reported by parent), and (e) mother's education. For all analysis of classroom-level measures adjustments included the following covariates: (a) teacher has a BA degree, (b) previous teaching experience, (c) child/adult ratio in classroom, (d) average class size, (e) city size, and (f) geographic site.
RM analysis was used to detect effects on the phonological awareness and social behavioral measures at baseline and post-test. RM analysis was also used to detect effects on the three mathematics outcomes, the three reading outcomes, the overall classroom environment and teacher-child interactions at all three data collection time points.
ANCOVAs were used to assess the long-term effects on phonological awareness and the three social behavioral measures; the covariates, for which, included the baseline score of the measure plus those listed above. ANCOVAs were also conducted for the measures of classroom instruction using the same covariates as listed above; however, classroom instruction data was only collected at post-test, thus baseline data was not included as a covariate.
OUTCOMES
Implementation Fidelity
The research team collected videotaped data to measure the fidelity of ELLM curriculum implementation. Teachers were videotaped twice during the preschool year. The videotapes were segmented and coded to capture the presence or absence of the ELLM critical elements. Scores ranged from 0 to 147. A score of 118 or more was considered high implementation fidelity, 89 to 117 as medium fidelity, and 88 or below as low implementation fidelity.
Even though 11 of the 14 teachers were in their second year of implementing the ELLM, none were rated as high implementers. The intervention teachers were all rated as low to medium implementers. All but one of the control teachers were rated as low implementers (the one exception was rated as a medium implementer). Overall the intervention teachers were given a global 'medium' rating on implementation fidelity.
Baseline Equivalence
At baseline, the groups were equivalent at on all measured outcomes (as listed below), on all child demographics and on most parent and teacher demographics. A higher percentage of parents in the treatment group had completed some post-high school education relative to those assigned to the control group (41% vs. 29%, p < .01). Additionally, teachers in the treatment group had more years of experience teaching in a preschool setting relative to those assigned to the control group (9 years vs. 4 years, p < .01).
Differential Attrition
One control and one treatment preschool dropped out of the study during the preschool year. There is no discussion of how these two preschools differed from the 28 who remained. For individuals, Appendix B (page B-8) states in general that there was no evidence of different rates of attrition across experimental and control groups but does not present specific analyses of differential attrition for this study.
Posttest
Of the 23 measures assessed at post-test (end of pre-k), none showed a statistically significant effect of the program the ELLM children's outcomes relative to the children in the control condition.
Long-term
At the long-term follow-up (1 year after the program ended i.e. end of K), the two child-level measures of language development showed statistically significant effects. They had small to medium effect sizes (PPVT: ES = .34, p < .05; TOLD Grammatical Understanding: ES = .44, p < .05). The other 10 outcome measures were not significantly influenced by the intervention.
Summary
Cosgrove et al. (2006) conducted a cluster-randomized trial with 48 classrooms and 466 preschool children, in which classrooms (one per school) were assigned to either the intervention or a waitlist control condition. Measures of alphabet recognition and early literacy skills were collected at baseline and posttest.
Cosgrove et al. (2006), found that in comparison to controls, students in the intervention group showed:
Evaluation Methodology
Design
As described above, low-performing public elementary schools housing at least one early intervention pre-kindergarten class were identified in each of the three locations and randomly assigned to either ELLM or wait-list control status. Additionally, two Head Start and two subsidized sites were randomly selected in the neighborhood of each elementary school. One class from each site was randomly selected to participate. Treatment group classes implemented the Early Literacy and Learning Model curriculum in conjunction with existing curriculum. Control group classes only implemented existing curriculum. (Prevalent existing curricula were Creative Curriculum, High/Scope, and High Reach.) The intervention lasted for a year with measures applied at the start and end of the year.
Sample
Four hundred sixty-six four-year-old children attending preschools in low-income neighborhoods served as the study sample for this investigation. These children were drawn from 48 pre-school classes, from three different geographic locations in Florida. These three locations represented differing degrees of urbanicity. The children were 12% White, 71% Black, 8% Hispanic, and 9% of other racial background.
Measures
Data were collected on the children's ability to recognize the 52 upper- and lowercase letters of the alphabet, and their early literacy abilities were measured by the Test of Early Reading Ability-Third Edition, Form A (TERA-3).
The TERA-3 is composed of three scales measuring unique but related early literacy skills: 1) the Alphabet scale measures graphophomenic knowledge; 2) the Conventions of Print scale measures knowledge of conventions of English print; and 3) the Meaning scale measures ability to comprehend meaning of print. Cronbach's Alpha coefficients of internal consistency for 4-year-old children for the Reading Quotient and the Alphabet, Conventions, and Meaning subtests are .97, .94, .88, and .94 respectively. Alpha coefficients for the 5-year-old children were comparable.
The Alphabet Letter Recognition Inventory (ALRI) is a locally developed inventory of children's ability to recognize the upper- and lowercase letters of the alphabet. Trained assessors presented uppercase letter flashcards, arranged in a fixed non-alphabetic order, to each child. The child was asked to name the letter. Following presentation of the uppercase letters, lowercase letter flashcards were presented in a similar fashion. The children's responses were recorded on scannable forms and computer scored.
Analysis
Hierarchical linear modeling (HLM) was used to test the statistical significance of these observed differences because children experienced ELLM together in classes rather than in one-on-one settings. Child-level variables included in the analyses were the TERA-3 pretest standardized scores, the ALRI pretest scores, the child's age in months on September 1, 2002, and gender. All continuous variables were grand-mean centered. Class-level variables included in the analyses were class assignment as ELLM or wait-list control, urbanicity, and educational attainment of the teacher (coded as either having completed a bachelor's degree or not).
Baseline Equivalence and Differential Attrition
At baseline, and again, near the end of the school year, children were assessed on their emerging literacy abilities and on their ability to recognize the 52 upper- and lowercase letters of the alphabet. Treatment children did not differ significantly from control children on these measures at baseline.
Attrition occurred at both the site and child levels. One wait-list control and one ELLM classroom in the largest city withdrew from the study and were not included in the analysis. In the 48 remaining classrooms, child-level attrition occurred, and 21% of the remaining children with pretest scores were unavailable for posttest. Comparison of the TERA-3 mean pretest scores of children with both pretest and posttest scores to those of the children with only pretest scores provided no evidence of systematic attrition.
Posttest
Examination of the TERA-3 and ALRI posttest scores indicated that those of the ELLM children were higher than those of the wait-list control children on all measures.
The child-level variables were statistically significant predictors in all of the analyses except for the child's gender in the Analysis of the Meaning scale. In all cases, older children scored at lower posttest levels than younger children, and, where gender was significant, boys achieved at lower levels than girls. These findings were the same for children in the ELLM and wait-list control classrooms.
At the class level, only assignment to ELLM or wait-list control classes was a statistically significant predictor in all analyses. The educational attainment of the classroom teacher was a statistically significant predictor only in the analysis of the Conventions of Print scale, and for this analysis, children who were taught by bachelor-degreed teachers achieved at higher levels in both the ELLM and wait-list control classrooms.
Analyses indicated that ELLM was more effective than traditional approaches in raising the emergent literacy achievement of children. As measured by all TERA-3 scales, the children experiencing the ELLM literacy curriculum and instruction support system achieved higher adjusted mean posttest levels of emergent literacy skills than children in the wait-list control classrooms. With the exception of the differences in the TERA-3 Conventions of Print scale adjusted mean posttest scores, resulting effect sizes were 25 percent of a standard deviation or larger (ARLI ES=.25, TERA-3 Reading ES=.28, TERA-3 Alphabet=.28, TERA-3 Conventions of Print=.17, and TERA-3 Meaning ES= .29).