as CREP Little Rock School District Center far Research in Educational Policy Center for Research in Educational Policy The University of Memphis 325 Browning Hall Memphis, Tennessee 38152 Toll Free: 1-866-670-6147 Evaluation of the Prekindergarten Literacy Program 2005 - 2006 RECEIVED Final Report DEC 1 3 2006 OFFICE OF DESEGREGATION MONITORING CREP Little Rock School District Center for Research in Educational Policy Center for Research in Educational Policy The University of Memphis 325 Browning Hall Memphis, Tennessee 38152 Toll Free: 1-866-670-6147 Evaluation of the Prekindergarten Literacy Program 2005 - 2006 December 2006 Anna Grehan Lynn Harrison Deborah Slawson Center for Research in Educational Policy Shana Pribesh John Nunnery Old Dominion UniversityEXECUTIVE SUMMARY The purpose of this report is to summarize and present the results of the Little Rock School Districts (LRSD) Prekindergarten Literacy Program. The primary goal of this evaluation was to determine the impact the Prekindergarten Literacy program had on improving and remediating the academic achievement of African American students. Additionally, this study was designed to determine program implementation, perceptions of strengths and weaknesses of the program by the relevant stakeholders, effectiveness of the program in screening and monitoring students progress, and the level of participation of African Americans in the preschool program. RESEARCH QUESTIONS This evaluation was structured around the following primary and supplementary research questions: Primary Evaluation Question Has the prekindergarten program been effective in improving and remediating the academic achievement of African American students? Supplemental (Qualitative/Step 2) Evaluation Questions What are the quality of instruction and level of implementation of prekindergarten programs in classroom environments at the 28 schools implementing in 2005-2006? What is the level of participation in the prekindergarten program by African American children relative to other ethnic groups at the school? To what extent does the prekindergarten program provide screening assessments and other appropriate measures to help identify and monitor the progress of African American children who may be at-risk for academic failure? What are the perceptions of prekindergarten teachers and paraprofessionals regarding the prekindergarten program implementation, impacts, strengths and weaknesses? What are the perceptions of the principal, kindergarten teachers, and first grade teachers regarding prekindergarten program implementation, impacts, strengths, and weaknesses? What are the perceptions of parents/guardians of prekindergarten children regarding program impacts, strengths, and weaknesses? 2005 -2006 Evaluation of the Prekindergarten Literacy Program 1EVALUATION DESIGN AND MEASURES Participants. The LRSD identified 28 preschool sites to he included in this study. From those 28 sites, there were 1,316 four-year old children in 69 preschool classrooms. Demographic data indicated 69.0% were African American, 19.0% were Caucasian, and 6% were Hispanic
approximately two-thirds (66.0%) qualified for free or reduced price lunch. There were 69 prekindergarten teachers and 69 paraprofessionals at the 28 sites, and 214 kindergarten or first grade teachers. Design. A mixed methods design was employed to address all of the research questions. Both quantitative and qualitative data were collected either by trained CREP researchers or through the acquisition and analyses of student achievement results provided by LRSD. Instrumentation. Student achievement results for 2005-2006 prekindergarten students were analyzed to determine gains throughout the school year, and achievement results for kindergarten students in 2005-2006 were used with comparisons made between those that attended LRSD prekindergarten programs versus those that had not. In addition, a ten year longitudinal analysis was conducted in which achievement data were compared between cohorts that had attended LRSD prekindergarten classrooms with those that had not. In addition to the student achievement results, two classroom observation tools were utilized, four survey questionnaires were administered, and a phone interview was conducted. Following are descriptions of the instruments and achievement measures used in this study. Early Literacy Observation Tool (E-LOT^). The E-LOT is the preschool version of the Literacy Observation Tool (LOT) both of which were developed by researchers at CREP as an instrument for observing literacy instruction. (Smith, Ross, &Grehan, 2002). The LOT has been aligned to the National Reading Panel and National Research Council findings and captures all five essential components of the Reading First program. The E-LOT has been customized to accurately reflect pre-school activities, especially as they relate to literacy activities. Early Language & Literacy Classroom Observation (ELLCO). The ELLCO is a toolkit that provides researchers and practitioners with guidelines for describing the degree to which a classroom supports literacy activities. Primarily an environmental checklist, the ELLCO contains three assessment tools: a Literacy Environment Checklist, protocols to conduct classroom observations and administer teacher interviews, and a Literacy Activities Rating Scale. The total time to complete the observation is approximately one to one and one-half hours. The ELLCO is used for research purposes in over 150 preschool classrooms and has a reported statistical reliability of 90% or better. Principal Phone Interview. Ten randomly selected principals from schools with prekindergarten programs participated in a 30-45 minute phone interview with CREP researchers. An interview protocol was developed for this study that included questions regarding the principals perceptions of the prekindergarten program at their school and the impact of the prekindergarten program on student achievement, with special note of the perceived impact on African American achievement. 2005 -2006 Evaluation of the Prekindergarten Literacy Program 2Prekindergarten Teacher Questionnaire (PKTQ). Prekindergarten teachers in all 28 schools with prekindergarten programs were asked to complete the PKTQ. The PKTQ was designed to capture prekindergarten teachers attitudes and perceptions of their schools prekindergarten program and their experiences as a prekindergarten teacher. Prekindergarten Paraprofessional Teacher Questionnaire (PKPTQ). Prekindergarten paraprofessional teachers in all 28 schools with prekindergarten programs were asked to complete the PKPTQ. The PKPTQ was designed to capture paraprofessional teachers attitudes and perceptions of their schools prekindergarten program and their experiences as a paraprofessional teacher. Classroom Teacher Questionnaire (CTQ). Kindergarten and first grade teachers in all schools with prekindergarten programs were asked to complete the CTQ. The CTQ was designed to capture these teachers attitudes and perceptions of their schools prekindergarten program. Parent/Guardian Questionnaire (PQ). All parents/guardians of prekindergarten students were asked to complete the PQ. The PQ was designed to capture attitudes and perceptions of their childs prekindergarten program. Early Screening Inventory (ESI). The Early Screening Inventory is a developmental screening instrument that was revised and standardized in 1997. Published by Pearson Early Learning, ESI allows the childs prekindergarten teacher to quickly screen and assess children in the areas of motor development, language development, and cognition. In this evaluation, the ESI database for 2005-2006 was analyzed. Work Sampling System (WSS). Developed in 1995, WSS is a performance assessment system, rather than a battery of tests, that assesses and documents childrens skills, knowledge, behavior, and accomplishments over a spectrum of educational areas and throughout a variety of occasions. WSS is a method in which teacher observations are organized around a set of criteria and procedures. For the purposes of this study, WSS fall 2005 prekindergarten student results were collected and compared with WSS spring 2006 prekindergarten students results as a measure of student growth throughout the year. Dynamic Indicators of Basic Early Literacy Skills (DIBELS). DIBELS is a tool for early identification of children with potential literacy problems and an assessment of response to instruction. The DIBELS assessment is designed to enable educators to modify their approach if a student is not on course to achieve reading goals. The Early Childhood Research Institute on Measuring Growth and Development at the University of Oregon constructed DIBELS. The measures were developed upon the essential early literacy domains discussed in both the National Reading Panel (2000) and National Research Council (1998) reports to assess student development of phonological awareness, alphabetic understanding, and automaticity and fluency with the code. Not all students in the LRSD were assessed with DIBELS. Primarily DIBELS was administered to those kindergarten students in Reading First schools or those students who did not do well on portions of the Qualls Early Learning Inventory. 2005 -2006 Evaluation of the Prekindergarten Literacy Program 3Qualls Early Learning Inventory (QELI). The QELI is an assessment tool primarily for kindergarten students. The QELI identifies student development as either developed,' developing, or delayed in six areas. Arkansas requires all incoming kindergarten students to be tested with the QELI. For the purposes of this study, fall 2005 QELI results for all incoming kindergarteners were collected for comparisons between those who had prekindergarten the prior year and those who had not. Iowa Test of Basic Skills (ITBS). The ITBS is a norm-referenced group administered test that measures the skills and achievement of students in grades K-8. Developed at the University of Iowa, the ITBS provides an in-depth assessment of students achievement of important educational objectives. PROCEDURES Data for the evaluation were collected April-May 2006 for the 2005-2006 school year. During the week of April 17*, 12 randomly selected preschool sites were visited and observed using the E-LOT and the ELLCO. All observations were conducted by trained CREP researchers. Principals dispersed questiormaires to all teachers in April, and prekindergarten teachers were responsible for disseminating the parent questionnaires. Principal phone interviews were conducted during May. Student achievement data were provided electronically by LRSD throughout the summer and fall of 2006. RESULTS Qualitative: Questionnaires, Interviews, Observations, and Screening Questionnaires. Questionnaires distributed to prekindergarten teachers, paraprofessional teachers, kindergarten and first grade teachers, and parents/guardians of preschool students were instrumental in gathering perceptions of the preschool program. Prekindergarten Teacher Questionnaire: Teachers were overwhelmingly positive about the prekindergarten program and believed it was having a positive impact on all students and was bridging the achievement gap for Afiican American students. They reported having a strong understanding of the program in general, and the literacy components in particular. Prekindergarten teachers indicated a great deal of support from the principal and the other faculty. The most negative responses were in the areas of parental support and time for teacher collaboration. Prekindergarten Paraprofessional Questionnaire: Paraprofessionals were also very positive about the preschool program and believed their work as paraprofessionals to be very important. Paraprofessionals felt supported by the teacher, the principal and other faculty and staff. They also indicated the primary strength of the prekindergarten program in improving student achievement and preparing students for kindergarten. Like the prekindergarten teachers, the most negative responses were for questions related to parental support and teacher collaboration. Classroom Teacher Questionnaire: Kindergarten and first grade teachers were also positive regarding the preschool programs at their schools and most agreed it helped 2005 -2006 Evaluation of the Prekindergarten Literacy Program 4 prepare the children for kindergarten. Teachers expressed the need for more information about the preschool program and the curriculum. Parent/guardian Questionnaire: Parents or guardians of all preschool students were asked for their opinion regarding the program and a large majority responded positively. Parents/guardians indicated the prekindergarten program was highly engaging for their children, was preparing them well for kindergarten, and through the literacy program, their child was becoming reading ready. Interviews. Principals responses during phone interviews indicated a high level of support and enthusiasm for the prekindergarten programs. Principals were eager to affirm the positive impacts of the prekindergarten program, the ease with which it complemented broader school initiatives, and the overall positive support the program received from all staff. Principals all strongly agreed that the prekindergarten program was having a positive impact on African American achievement in particular and overall student achievement as well. Observations. Observations with the ELLCO and the E-LOT indicated overall positive preschool classroom environments with knowledgeable teachers and actively engaged students. Although the ELLCO checklists reflected ample literacy curriculum and materials, class libraries, and other varieties of print and writing materi^s throughout the classrooms, observations with the E-LOT suggested literacy activities were not the predominant focus of many prekindergarten classrooms. Instruction that incorporated phonological awareness, emergent writing, oral language and comprehension activities were not fully developed. Screening. Initial results from the ESI data suggested that although African American students scored lower on the ESI than other races, the differences were not statistically significant. Averages were adjusted to accoimt for the influence of gender, socio-economic status, and English language limitations, and no statistically significant differences were found. Quantitative: Student Achievement The impact on student achievement was analyzed at three levels: prekindegarten achievement gains, kindergarten achievement gains, and longitudinal gains realized over students ten year school career. Analysis A: Prekindergarten Achievement Results. Student results based on WSS were positive and suggested the prekindergarten program improved or remediated the academic achievement of African American students. African American students consistently performed higher on the WSS assessments at Time 3 than at Time 1 in all seven domains: Personal and Social Development, Language and literacy. Mathematical Thinking, Scientific Thinking, Social Studies, The Arts and Physical Development and Health. Analysis B: Kindergarten Achievement Results. Students who attended a LRSD preschool program had higher achievement levels on incoming assessments than those who did not attend LRSD preschool program. Achievement results at the end of the kindergarten year were also higher for those who attended LRSD preschool programs than those who had not. In addition, African American students who attended a preschool program in the LRSD consistently out scored African Americans who did not attend preschool on both fall kindergarten assessments and spring kindergarten assessments a 2005 -2006 Evaluation of the Prekindergarten Literacy Program 5 Analysis C: f through Grade Achievement Results. Results from the longitudinal analysis suggested that African American students who participated in a prekindergarten program were statistically significantly less likely to need special education services as they continued their school careers and were more likely to remain on grade level than those African Americans who did not attend a preschool program. Results from this analysis also indicated that there were long term impacts for students who had attended a prekindergarten program in the district, even ten years after participation. For African American students this, impact was dramatic and more pronounced than for the population in general. FINDINGS 1. Has the prekindergarten program been effective in improving and remediating the academic achievement of African American students? In order to gain a full understanding of the impact of the prekindergarten program on African American student achievement, analyses of both cmrent and former prekindergarten students were conducted. Immediate impacts were gauged according to achievement results during the prekindergarten year and achievement results as students entered kindergarten. Long term impacts of the prekindergarten program were assessed through analyzing student achievement data over a ten year period. Findings from this three-tiered approach are described below, but all suggested the prekindergarten program had immediate and long-range impacts on all students, with long term effects especially dramatic for the African American population. Analysis A: Prekindergarten Achievement For the first analysis, test scores on the Work Sampling System by Pearson were examined for 2005-2006 prekindergarten students. Initial fall 2005 scores were compared with spring 2006 student scores. The database from Pearson had 1,300 student scores and comparisons were made between fall 2005 scores and spring 2006 scores. Results from the WSS data suggested that the prekindergarten program was effective in improving or remediating the academic achievement of AIncan American students. African American students consistently performed higher on the WSS assessments at Time 3 than at Time 1 in all seven domains: Personal and Social Development, Language and literacy. Mathematical Thinking, Scientific Thinking, Social Studies, The Arts and Physical Development and Health. Most students reached proficiency (the child can reliably demonstrate the skills, knowledge, behaviors or accomplishments) by Time 3 on all 55 indicators. The pre/post test design employed to assess prekindergarten effectiveness was not rigorous and was open to multiple threats to internal and external validity. Although all students who participated in the state-funded preschool program in Little Rock, Arkansas in 2005-2006 were assessed in this study and most displayed evidence of skill growth, it can not be stated definitively that other factors, such as enrichment programs or parental intervention, did not contribute to the gains. 2005 -2006 Evaluation of the Prekindergarten Literacy Program 6Analysis B: Kindergarten Achievement The evidence from this analysis suggested the preschool program had an overall positive impact on kindergarten student achievement and was effectively improving or remediating the academic achievement of African American students. Relative to students who had not attended a LRSD prekindergarten program, kindergarten students who had attended a Little Rock preschool consistently performed higher on QELI, DIBELS and ITBS subtests. Likewise, African American students who were enrolled in the Little Rock preschool program, consistently performed higher on QELI, DIBELS and ITBS subtests than those who did not attend a preschool program. The design employed to assess effectiveness of the prekindergarten program was rigorous, yet prone to threats of internal and external validity. Students were not randomly assigned to treatment (prekindergarten program) and control (no prekindergarten program), thus, the sample is influenced by selection biases with parents of children who volxmtarily selected to participate in prekindergarten programs more likely to be different than those who chose not to participate. Likewise, this study does not capture information regarding other types of preschool services kindergarten students may have received. The students that were not enrolled in a LRSD prekindergarten program may have been enrolled in private preschool programs or involved in high-quality child care that proximated preschool. Therefore, these students may have received an alternate, yet similar, treatment to that being examined in this study. The sample was representative of Little Rock kindergarten students and appropriate for this evaluation. However, attempts to generalize the findings from this study to other school districts must be carefully consideration based on the above mentioned limitations. Notwithstanding, there does appear to be strong evidence supporting the relationship between Little Rock preschool attendance and positive academic performance. The results were positive across all three achievement tests and 15 subtests. The effect sizes were small to moderate, but all were statistically significant. Therefore, there is strong indication that preschool attendance positively impacted kindergarten academic performance. Analysis C: L* through 10*^ Grade Student Achievement For the longitudinal component of the study, ITBS data for the past ten years were analyzed to evaluate potential long-term benefits of LRSDs prekindergarten program. Scores of students who had previously attended a LRSD prekindergarten program were compared with scores of students who had not. The results of this analysis were very positive. Students who had participated in a LRSD prekindergarten program were more likely to remain on grade level throughout their school career and were less likely to received special educational services than those students who had not. For African American students, those that attended a prekindergarten program had significantly higher ITBS scores than all other cohorts. This longitudinal analysis answered these specific items: What is the relationship between African American students participation in the prekindergarten program and subsequent receipt of special education services? Afncan American students who participated in a prekindergarten program were statistically significantly less likely to need special education services later in their school careers. In fact, African American students who did not attend prekindergarten were almost 2005 -2006 Evaluation of the Prekindergarten Literacy Program 7twice as likely to need special education services. Overall, 8.9% of African American PK students received special education services in 2006, versus 14.8% of African American comparison students. Are African American students who participated in prekindergarten more likely to remain on grade level throughout their school career? For nine out of ten cohorts examined, African American students who attended prekindergarten were more likely to remain on grade level than those who did not. Across cohorts, 87% of African American prekindergarten students were enrolled in the appropriate grade for their age versus 79% of African American comparison students. Is there a relationship between prekindergarten participation and subsequent performance on the ITBS, and does this relationship differ for African American students versus other students? Prekindergarten participation had a statistically significant, positive effect on the performance of all students, even ten years after program participation. However, participation had a much more pronounced positive effect on the achievement of African American students, with typical effect size estimates of about +0.20 across cohorts and subtests. 2. What are the quality of instruction and level of implementation of prekindergarten programs in classroom environments at the 28 schools implementing in 2005-2005? The prekindergarten program appears to be well implemented with positive classroom environments. An overwhelming majority of prekindergarten teachers, both African American and non-African American, indicated a thorough understanding of the program, valuable training and support from the district and the principal, and an adequate supply of materials for program implementation. Prekindergarten teachers agreed children were excited about learning and most indicated the environment was preparing students for kindergarten. Paraprofessional teachers were also positive regarding the program and responded favorably regarding student enthusiasm, teacher and principal support, materials that supported the program, and the degree to which the program was preparing children for kindergarten. Kindergarten and P grade teachers at the 28 sites were generally positive as well, with an overwhelming majority expressing confidence that the prekindergarten program prepared the children for kindergarten and helped insure that they were reading ready. Through phone interviews, principals reflected positively about the prekindergarten program and deemed it an integral part of their elementary schools. Observations indicated well-implemented programs and positive classroom environments. Site researchers found most of the classrooms were well-equipped with print-rich environments
however opportunities to capitalize on the literacy curriculum, books and themed materials were often missed or not fully explored. In addition, although most teachers were effective classroom managers, observations did not reflect a variety of instructional techniques to address all necessary components of literacy instruction. Learning centers were a predominant orientation of instruction, yet they were often not used to complement literacy instruction. 3. What is the level of participation in prekindergarten by African American students relative to other ethnic groups at the school? 200S -2006 Evaluation of the Prekindergarten Literacy Program 8African American students represented the majority of the preschool population during the 2005-2006 school year. According to demographic information provided by the Little Rock School District, in the 69 preschool classrooms that were part of this study, the African American population was 69.0%. The Caucasian population represented 19%, 6.0% were Hispanic, and 6.0% described their ethnicity as other. 4. To what extent does the prekindergarten program provide screening assessments and other appropriate measures to help identify and monitor the progress of African American children who may be at-risk for academic failure? Analysis of ESI data indicated that preschool teachers had screened all eligible students. The majority of those screened were African American, which is not surprising given the majority of students in the LRSD are African American. Although African American students scored slightly lower on the ESI, the results were not significant from those of other ethnicities. Over 80% of all preschool teachers, African American and others, agreed that the Prekindergarten Literacy program was useful as a screening tool for assessing at-risk African American students. For paraprofessionals, 72.7% of the non-African American paraprofessionals agreed that the prekindergarten program was useful in monitoring progress of African American students
for African Americans this percentage fell to 63.4%. All principals agreed that their well-experienced prekindergarten teachers were able to distinguish problems that needed to be addressed and that through screening and appropriate follow-up checks, there were adequate opportunities for early identification of those at-risk of failing to make academic progress. 5. What are the perceptions of prekindergarten teachers and paraprofessional teachers regarding prekindergarten program implementation, impacts, strengths, and weaknesses? Prekindergarten teachers and paraprofessional teachers were very positive regarding the prekindergarten program. For both teachers and paraprofessionals and across ethnic groups, the majority of items on the questionnaires were answered positively by over 80% of the respondents. Most teachers and paraprofessionals understood their programs, felt supported in their work, and agreed that students were being positively impacted, both academically and socially. Questions that allowed teachers and paraprofessionals to respond in their own words were helpful in understanding the perceptions of strengths and weaknesses of the prekindergarten program. For African American teachers and paraprofessionals, the most popular strength listed was the socialization and peer interaction that children experienced in the classroom. While a popular response for non-African American teachers and paraprofessionals, their most popular response was the literacy component. Responses on the questionnaire indicated that the biggest area of concern for prekindergarten teachers and paraprofessionals was the lack of parental involvement. When 2005 -2006 Evaluation of the Prekindergarten Literacy Program 9 asked to list the least effective components of the prekindergarten program, the most common answer across all ethnic groups was none or n/a, indicating a high level of overall satisfaction. All responding prekindergarten teachers and paraprofessionals agreed that the prekindergarten program should be maintained. The most frequent reason given for keeping the program was the preparation it provided for kindergarten. 6. What are the perceptions of principal, kindergarten, and first grade classroom teachers regarding the prekindergarten program implementation, impacts, strengths, and weaknesses? All principals who were interviewed were very positive about the preschool program, and most felt it had a significant impact on the long-term achievement of their students. Principals were very complimentary about the prekindergarten teachers and indicated great confidence in their abilities. The literacy aspect of the prekindergarten program was also well thought of by principals, and most understood it to provide the basic foundations of future reading success. Principals reported that the prekindergarten program was well-integrated into the school and enjoyed school-wide support from the rest of the faculty. Principals indicated that the achievement gap historically experienced by African American students was being closed through the prekindergarten program. Concerns articulated by principals included transportation issues and the desire to reach more of the four-year old population. Kindergarten and first grade teachers were also very positive about the prekindergarten program. The majority of both African American and non-African American teachers felt that students were positively impacted because of the program and that the prekindergarten program helped prepare students for kindergarten as well as future reading success. 7. What are the perceptions of parents/guardians of prekindergarten children regarding program impacts, strengths, and weaknesses? Parents who responded to the questionnaire were overwhelmingly supportive of the preschool program. This was true for parents/guardians of African American children, as well as for those who represented other ethnic groups. Of the 14 items respondents were asked to evaluate, over 90% of all parents/guardians responded positively to half of them (7). The childrens excitement for preschool, learning, and teachers was evident to those parents and guardians who completed the questionnaires. The majority of all parents/guardians agreed that preschool was preparing their child for kindergarten. In general, for most items, responses between parents of African American students verses those that were not African American were similar and positivewith two exceptions. Those respondents with non-African American children were more likely to respond positively regarding their childs reading readiness due to the prekindergarten program (82.1% agreed) than those who were parents/guardians of African American children (71% in agreement). Also, when asked about the school their child attended, 93% of the parents/guardians of non-African children responded favorably, compared with 84.7% of the African American parent/guardians. 2005 -2006 Evaluation of the Prekindergarten Literacy Program 10The most popular prekindergarten strengths listed by parents/guardians, regardless of the childs ethnicity, were: high quality, caring teachers and the socialization and interaction with peers that occurred in the classrooms. When asked to list changes that should be made to the program, the most popular response for both African American and non-Affican parent/guardians was no change. This is further indication of a high level of support from the parents/guardians of students who participated in the prekindergarten program. RECOMMENDED PROGRAM MODIFICATIONS The Little Rock School District has been recognized as having a strong prekindergarten program. In addition to the positive support expressed by all stakeholders, achievement results reflect a program that is having a positive impact on African American student achievement. Recommendations to further capitalize on the programs success are grouped within the four broad categories listed below: I. Instructional Effectiveness A renewed focus on literacy instruction would strengthen students pre- reading skills and further prepare preschoolers for kindergarten. Preschools in the district benefit from first-rate materials, curriculum, and knowledgeable support staff
yet many opportunities to connect literacy instruction with other classroom activities were missed in the classrooms observed. Teachers and paraprofessionals would benefit from coaching and modeling that focuses on preschool literacy instruction, especially concepts of print, fluency activities, oral language, early writing skills, and phonemic awareness, with an emphasis on incorporating these instructional techniques in small group instruction, learning center activities, and other hands-on activities. Mentoring and support from district professionals could aid teachers in providing instructional activities that capitalize on students natural enthusiasm for learning. It is recommended that the district should investigate the feasibility of investing additional resources in the necessary support staff to strengthen the program. It is recommended that a district wide program monitoring system to assess fidelity of literacy curriculum implementation and instructional effectiveness be implemented. This could be a valuable source of formative data and an effective feedback mechanism for project implementation. IL Use of student assessment data It is recommended that a member of the district research and evaluation office be identified as a specialist in preschool assessments. This individual would understand assessment results and help teachers interpret assessment data, guide the district in making data driven decisions, monitor and make recommendations on district performance goals and benchmarks to improve student achievement. 2005 -2006 Evaluation of the Prekindergarten Literacy Program 11 The district should investigate the current mix of assessment tools with an eye toward identifying more user friendly instruments. The district should explore the usage of instruments that provide immediate access to data for analysis and classroom reports for teachers through a web-based interface. III. Expansion of preschool program Given the demonstrated benefits of preschool to at-risk children, to the school, and to the commimity at large, thefe is clearly a need for more preschool classrooms. The district has made an impressive step in this direction with the dedication of the renovated former elementary school. Fair Park, as a preschool only site. This commitment and concentration of resources should serve as a model for future planning. Exploring a public transportation system for preschool students would demonstrate a commitment to inclusiveness that would help to reach the children that could benefit most from the preschool program. IV. Parent Involvement A more concerted, planned effort to communicate with parents and increase their involvement should be undertaken. Because parents must physically bring their children to the classroom and pick them up, preschool teachers have the opportimity to connect with parents on a daily basis. I EXPECTATIONS OF PROGRAM MODIFICATIONS Many of the valuable pieces to an outstanding preschool program are in place in the Little Rock School District. With the incorporation of the recommended program modifications, the district and community could expect: An increased commitment of teachers to improve their instructional practices and to align the program components with current research. A coordinated preschool assessment program which would enhance student achievement. A broadened commitment to inclusiveness that would further enhance and strengthen the program. Expanded preschool communities committed to reaching and preparing the most at- risk students for kindergarten. An increased number of students who enter kindergarten with the necessary social skills to thrive in an organized environment. An increased number of students entering kindergarten who have the skills necessary to learn to read. A ripple effect with long-term gains on student achievement tests being realized through high school. Increased parental involvement and better preschool attendance. More tailored instruction to meet students individual needs. I 2005 -2006 Evaluation of the Prekindergarten Literacy Program 12 The ability to identify special needs students earlier and provide the help they need. A decrease in supplemental services needed. 2005 -2006 Evaluation of the Prekindergarten Literacy Program 13Evaluation of the Prekindergarten Literacy Program in the Little Rock School District 2005 - 2006 INTRODUCTION Little Rock School Districts (LRSD) Early Childhood program dates back to 1969 with the opening of the Center for Early Development and Education, directed by Dr. Bettye Caldwell at the University of Arkansas at Little Rock. Originally housed at the Kramer School, the Center was moved to Rockefeller Elementary, and the school was designed as a center for early childhood and for grades 4 through 6. In 1987, Rockefeller became a K-5 elementary school and was renamed Rockefeller Elementary and Early Childhood Magnet School. Currently, Rockefeller serves children as young as six weeks old through 5* grade. In 1991, LRSD received an Arkansas Better Chance (ABC) grant that provided additional funding for preschool education for children in poverty. The original grant provided funding for 78 children in six classrooms in three elementary schools. Over the past years, the preschool program has expanded into every school except three, with funding from poverty index funds, desegregation funds, and district funds. In spring 2004, additional funding for preschool classrooms came from the state of Arkansas through a grant, Arkansas Better Chance for School Success (ABCSS), a successor to the former ABC grant. Initially 25 of the LRSDs elementary schools were approved for ABCSS funding. These funds required a 40% match from the district. During FY 2004-2005, ABCSS and district funds were used to support the preschool program and to expand the number of classrooms to 65. During 2005-2006, Fair Park Early Childhood Center, a renovated elementary school devoted entirely to preschool education, opened and the total number of four-year old preschool classrooms in the LRSD rose to 69. In addition to the increase in the number of preschool classrooms, there were program changes and additions during the 2005-2006 school year to further increase students readiness for kindergarten. The LRSD implemented a comprehensive curriculum system, the Map for Pre- K Literacy, which provided content guides with strategies and resources, concrete monthly benchmarks, and formative assessments for planning. The structure specifically addressed critical early learning skills such as oral language, listening comprehension, vocabulary, phonological awareness, print awareness, and alphabet knowledge and early mathematics skills. 2005 -2006 Evaluation of the Prekindergarten Literacy Program 14Instructional orientations that included group activities and learning centers with a variety of theme-oriented literature and hands-on activities were promoted. The curriculum also called for formative monthly assessments. In conjunction with the Map for Pre-K Literacy, the Houghton Mifflin PRE-K cuniculum was adopted to provide daily lesson structure that is scientifically-based and would complement and strengthen the curriculum map in addressing the critical areas of early literacy instruction. A complete description of the prekindergarten program was provided by the LRSD Early Education Department and is included in Appendix E. The benefits of a strong preschool program are well-researched and documented. According to most long-term studies, Prekindergarten students have been three times more likely to make better grades and significantly more likely to graduate from high school on time. (Southern Education Foundation, 2006, p.9) Consistently, research indicates that high quality ?9 preschool programs are imperative to childrens academic and cognitive development, especially as such programs address growth in emergent literacy. Emergent literacy refers to a continuous period of young childrens development and includes all of their attempts to interpret or communicate using symbols, regardless of whether the symbols are print, scribbles or pictures (Dixon-Krauss, 1996). Children vary widely in their exposure to print, knowledge of letters, and phonemic awareness, and these differences in emergent literacy skills before kindergarten may continue to impact their success in subsequent years. As children progress through school, initial differences in emergent literacy skills and in reading ability become more pronounced (Butler, Newman & Dickinson, 2006). However, preschool attendance, in and of itself, does not necessarily impact a childs later school successthe quality of the program determines its longterm impacts. Evidence continually shows that for there to be a lasting impact on achievement, programs must focus on oral language, emergent literacy, and social development. (Schweinhart, Barnes, Weikart, Barnett, & Epstein, 1993). Nationwide, Arkansas is ranked highly for its preschool programs. The State of Preschool: 2005 State Preschool Yearbook, the third in a series of works by the National Institute for Early Education Research (NIEER), states that Arkansas was the only state of the 38 states with preschool programs to meet all 10 of the NIEER recommended benchmarks. These benchmarks include comprehensive early learning standards, teacher degree and professional development requirements, class size and student teacher ratio, support services, and site visits. Although Arkansas was ranked 18'* among the 38 states with preschools in terms of four-year- 2005 -2006 Evaluation of the Prekindergarten Literacy Program 15 old preschool accessibility, from sy2001-2002 to sy2004-2005, enrollment of four-year-olds throughout the state has risen over 100% (Barnett, Hustedt, Robin, Schulman, 2006). RESEARCH QUESTIONS The primary focus of this research study was to evaluate the effectiveness in improving and remediating the academic achievement of African American students in the Little Rock School District (LRSD). Additionally, this study addressed other'issues pertinent to understanding the Prekindergarten Literacy program and evaluating its impact on achievement. These other issues included the quality and level of implementation of the prekindergarten literacy programs, the level of participation of African Americans in these prekindergarten programs, the extent to which screening assessments were utilized to determine those at-risk for failure, and the perceptions of prekindergarten teachers, paraprofessionals, and parents/guardians of student attending prekindergarten, as well as kindergarten and first grade teachers and principals in whose schools where the preschool programs were examined. A mixed methods design was employed to address all of the research questions. Student achievement results for 2005-2006 prekindergarten students were analyzed to determine gains throughout the school year, and achievement results for kindergarten students in 2005-2006 were used with comparisons made between those who attended LRSD prekindergarten programs versus those who had not. In addition, a ten year longitudinal analysis was conducted in which achievement data were analyzed between cohorts who had attended LRSD prekindergarten classrooms with those who had not. In addition to the student achievement results, two observation tools were utilized, four survey questionnaires were administered, and a phone interview was conducted. The evaluation of the LRSD Prekindergarten Literacy program was structured around the following research questions: Primary Evaluation Question Has the prekindergarten program been effective in improving and remediating the academic achievement of Afiican American students? Supplemental (Qualitative/Step 2) Evaluation Questions What are the quality of instruction and level of implementation of prekindergarten programs in classroom environments at the 28 schools implementing in 2005-2006? 2005 -2006 Evaluation of the Prekindergarten Literacy Program 16 What is the level of participation in the prekindergarten program by African American children relative to other ethnic groups at the school? To what extent does the prekindergarten program provide screening assessments and other appropriate measures to help identify and monitor the progress of African American children who may be at-risk for academic failure? What are the perceptions of prekindergarten teachers and paraprofessionals regarding the prekindergarten program implementations, impacts, strengths and weaknesses? What are the perceptions of the principals, kindergarten teachers, and first grade teachers regarding prekindergarten program implementations, impacts, strengths, and weaknesses? What are the perceptions of parents/guardians of prekindergarten children regarding program impacts, strengths, and weaknesses? 2005 -2006 Evaluation of the Prekindergarten Literacy Program 11 EVALUATION DESIGN AND MEASURES The evaluation period extended from April 2006 through May 2006. The evaluation design was based on both quantitative and qualitative data collected from observations of prekindergarten classrooms and literacy instruction, phone interviews with principals, parent surveys, prekindergarten teacher and paraprofessional surveys, and questionnaires completed by kindergarten and first grade teachers who taught in schools with prekindergarten classrooms. Student level achievement data included Work Sampling System (WSS) results, Qualls Early Learning Inventory (QELI), Dynamic Indicators of Basic Early Literacy Skills (DIBELS), and Iowa Test of Basic Skills (ITBS). The results of the Early Screening Inventory (ESI) were also used to provide insight regarding prekindergarten screening measures. The student level achievement data and screening assessment data were received from the district in the summer and fall of 2006. Site researchers from the Center for Research in Educational Policy (CREP) conducted observations of the prekindergarten classrooms and phone interviews with principals. Principals were responsible for dissemination of teacher questionnaires, and prekindergarten teachers for distribution of the parent/guardian questionnaires. Participants The Little Rock School District is the largest school district in the state of Arkansas and served 26, 524 students during the 2005-2006 school year. Approximately 1,316 four-year-old students, roughly 5% of the total student population, attended preschool classes in one of the 28 designated state-funded prekindergarten sites. Of those 28 sites, 27 were elementary (k-5) schools and one site. Fair Park, was an early childhood center comprised entirely of prekindergarten classes. Almost 70 percent (69.0%) of those attending prekindergarten classes were African American, 19.0% were Caucasian, and 6.0% were Hispanic. Approximately two- thirds of the students (66.0%) qualified for free and reduced price lunch. Table 1 summarizes the total school and preschool populations for 2005-2006. 2005 2006 Evaluation of the Prekindergarten Literacy Program 18Tablet. Total School and Preschool Populations, 2005-2006 School Bale Baseline Brady Carver Chicot Dodd Fair Park Forest Park Franklin Fulbright Geyer Springs Jefferson M.L. King Mabelvale McDermott Meadowcliff Otter Creek Pulaski Heights Rockerfeller Romine Stephens Terry Wakefield Washington Watson Western Hills Wilson Woodruff Total Students 340 299 378 520 591 309 137 415 446 651 337 451 616 377 453 383 593 334 493 375 576 644 504 641 517 294 288 236 % African American 83.2 72.9 83.3 54.2 71.9 56.0 62.8 21.4 96.2 24.7 88.4 33.3 65.9 78.8 64.0 77.3 65.1 44.9 76.5 81.1 95.7 56.7 75.0 81.6 87.6 76.9 86.1 91.9 % White 4.1 4.7 6.3 36.7 4.6 21.4 29.9 73.5 2.2 66.5 3.0 63.9 25.0 10.6 20.8 9.9 21.4 47.9 17.0 2.4 1.0 25/0 2.6 6.6 1.5 15.3 3.1 5.9 % Hispanic 8.2 19.4 3.4 3.1 21.5 17.8 .7 1.2 .4 2.2 7.4 .4 1.0 7.2 7.5 10.2 7.1 1.8 2.0 11.7 1.6 9.0 21.4 4.5 8.3 6.1 7.3 .8 % other 4.4 3.0 6.9 6.0 2.0 4.9 6.6 3.9 1.1 6.6 1.2 2.4 8.1 3.4 7.7 2.6 6.4 5.4 4.5 4.8 1.7 9.3 1.0 7.3 2.5 1.7 3.5 1.3 % Free & Reduced Lunch 82.9 94.3 86.2 52.9 88.0 70.9 54.0 17.8 96.4 16.9 86.9 28.2 60.2 86.7 67.3 88.0 57.8 44.6 72.4 81.9 91.3 53.4 91.3 80.3 95.2 77.6 90.6 88.1 ^"Vno.^- Students " African Piskinderaarten <g American 38 *5 82.0 75.0 70.0 , White . 50 0.0 " 5.0 s 30.0 Hispanic Other 8.0 18.0 5.0 0.0 3.0 .^18.0 0.0 8: -de. 38 ' 137 39 58 40 37 , -I?'' 70.0 8.0 - - 20.0 55.0 f 4'26.0 _ 11.0 63.0'''' 0-.- 23.0 J<740 100.0 13.0 Sih 0.0 ^'"40 ' -at 92.0 . ST 18.o 0.0 70.0 3.0 3.0 I! 77 71 , 37 Jf 40 60 38 40 80 38 20 V 34 1316 - . 62.0 70.0 51.0 78.0 62.0 - 26.0 83.0 <,.27.0 14.0 32.0 10.0 23.0 58.0 5.0 0.0 0.0 10.0 5.0 130 0,0 1.0 8.0 7.0 3.0 0.0 15.0 0.0 6.0 11.0 0.0 5-0 .ivi- lO-O 0.0 ^160 69.0 87.0 94.0 66.0 73.0 84.0 92.0 70.0 88.0 85.0 69.0 18.0 2.0 0.0 24.0 i 5.0 5.0 0.0 20.0 6.0 6.0 " 19.0 3.0 - 8.0 3.0 8.0 20.0 0.0 5.0 . 10.0 y 6.0 3.0 6.0 10.0 30 3.0.$^. 3.0 * 3.0 11.0 3.0 0.0 0.0 6.0 6.0 Lunch 69.0 92.0' 68.0 ' 55 0'" 92.0 61.0 54.0 13.0 980 8.0 92.0 J*', 10.0 52.0:^ 83.0 83.0 f 44.0 16.0 54.0 68.0 87.0 . < 50.0 1 90.0 < 79.0 87.0 70.0 if 88.0 88.0 66.0 W w -<? A' % >- 39 & P 39 .M 39 3 - 7< 30.0 /-^-../.O . iX 0.0 J Source: Little Rock School District Each preschool classroom had one preschool teacher and one paraprofessional teacher leading classroom instruction for a total of 69 preschool teachers and 69 paraprofessional teachers. Within the 27 elementary schools that participated in the study, there were 1,966 kindergarten students in 115 kindergarten classrooms, and 1,899 first-grade students in 99 first grade classrooms. Table 2 summarizes the breakdown of early education classes per school. 2005 -2006 Evaluation of the Prekindergarten Literacy Program 19Table 2. Early Education Classes by School, 2005-2006 No. School Bale Baseline Brady Carver Chicot Dodd Fair Park Forest Park Franklin Fulbright Geyer Springs Jefferson M.L. King Mabelvale McDermott Meadowcliff Otter Creek Pulaski Heights Rockerfeller Romine Stephens Terry Wakefield Washington Watson Western Hills Wilson Woodruff Total Students 340 299 378 520 591 309 137 415 446 651 337 451 616 377 453 383 593 334 493 375 576 644 504 641 517 294 288 236 No. Prekindergarten XArVl>k'.r ' ' . ^Students/^^*^ Classes/ j -.'Jrekindetgarten'' Teachers .Prekindetgarten' ' s>- Q I' 39 -a -.38 40 39 > 58 - ->'^40 \^2 40 V 8 . 71 37 \ 40 39 39 60 70 38 40 - ' 80 38 > 20 17 - 34 1316 Source: Little Rock School District 'a f 5*'' '4e-^ 4 2 2 4 - 2 ' 1 2 69 No. Students, K 62 41 80 76 101 57 N/A 61 84 102 55 61 98 41 76 73 100 60 59 59 98 120 78 99 100 40 46 39 1966 No. K Classes/ Teachers 4 3 5 4 7 4 N/A 3 5 6 3 4 6 3 4 4 5 3 3 4 6 7 4 5 5 2 4 2 115 Students, I** grade 50 * 35 N/A " A 7V. L . 65 120 50 " IZ : s 86 54" 70 73 95 50 64 64\ 87 115 97 100 92 48 48 , 34 1899 No. I" Grade Classes/ Teachers? 2 N/A^ &3 o -sr, 3 4 2 3/ 4 6 6 4 - . 6 4 7 2 :f.-4 rl
2 99. ?' J 2 1 . 1^ r s**^5 137* > WS w: 2 2 3 6 2 2 2 3 4 ,- , a 2 . 3 INSTRUMENTATION A variety of instruments were used to collect qualitative data. Site researchers employed two observation tools for use in viewing literacy instruction in prekindergarten classrooms. A principal protocol was designed to determine principals understanding of the Prekindergarten Literacy program at their school. CREP also developed four questioimaires that were disseminated to: (a) prekindergarten teachers, (b) prekindergarten paraprofessionals, (c) kindergarten and first grade teachers, and (d) parents/guardians of the prekindergarten students. All questionnaires were designed to gather perceptions and attitudes of the Prekindergarten Literacy program. Finally, a data from a prekindergarten screening inventory were analyzed. A detailed description of each follows. Classroom Observation Measures 2005 -2006 Evaluation of the Prekindergarten Literacy Program 20In April 2006,4 site researchers from CREP visited 17 randomly selected prekindergarten classrooms in 12 different schools. The following tools were used to capture literacy instruction and record observations: Early Literacy Observation Tool (E-LOT^). The E-LOT is the preschool version of the Literacy Observation Tool (LOT) both of which were developed by researchers at CREP as an instrument for observing in elementary classrooms where teachers are engaged in teaching reading and other literacy practices (Smith, Ross, &Grehan, 2002). The LOT has been aligned to the National Reading Panel and National Research Council findings and captures all five essential components of the Reading First program. The E-LOT has been customized to accurately reflect preschool activities, especially as they relate to literacy. Individual observations and the summary are organized around the following categories: Instructional Orientation, Instructional Components, Learning Centers (types and activities). Student Activities and Teacher Interactions during the Learning Centers, Classroom Environment and Climate, and Visible Print Environment. The subcategories of Instructional Components include the five essential components of reading including: Concepts of Print, Alphabetic and Phonological Awareness, Fluency, and Vocabulary and Oral Language Development. The E- LOT captures explicit instruction in these five component areas. Quantitative data yielded by the observations reflect the degree to which targeted components are occurring in classrooms. The rubric ratings range from 0 = Not observed, to 1 = Rarely, 2 = Occasionally, 3 = Frequently, and 4 = Extensively observed. There are two forms for recording observations of literacy instruction on the LOT. The Literacy Observation Tool Notes form is completed for each teacher observed. At the conclusion of the observation period, the individual notes forms are then synthesized and summarized on the Literacy Observation Tool Data Summary form. Whereas the standard LOT procedure involves a trained observer visiting 7-9 randomly selected classrooms, for 10 minutes each, during the typical PA to 2 hour reading/ literacy block, the E-LOT can be structured differently. As in the case of this study, E-LOT observations are typically targeted observations. Rather than observing multiple classrooms for 70 to 90 minutes, targeted observations require the observer to remain in one classroom for this period of time. This type of observation allows the site researcher to view the full spectrum of activities, including both direct instruction and learning center instruction. All observations in this study were targeted E- LOTs. 2005 -2006 Evaluation of the Prekindergarten Literacy Program 21A recent reliability study using Generalizability Theory (Sterbinsky & Ross, 2003) indicated that LOT observations have a reliability of .75 when at least five LOTs are completed in a school. Additionally, the validity of the LOT was assessed via a content validation process in the development phase, including the use of a panel of subject matter experts, both researchers and practitioners, from areas such as the University of Memphis, the Memphis City Schools, and the state departments of education in Tennessee, Louisiana, and Illinois. To ensure the reliability of data, observers receive a manual that describes and operationally defines reading variables that comprise the E-LOT and participate in formal training to ensure that the identifying and coding of reading variables occurs in a consistent manner to guarantee the reporting of reliable and valid data. After receiving the manual and instruction in a group training session, each observer participates in sufficient practice exercises and an inter-rater reliability consensus rating process to ensure that his/her data are comparable with those of experienced observers. The LOT was piloted in the Memphis City Schools and is used in multiple schools across the United States, including 70 Reading Excellence Act (REA) and 75 Reading First (RF) schools in the state of Tennessee. The REA and RF evaluations and research studies include rigorous matched control quasi-experimental designs for which the LOT is the primary outcome measure. The E-LOT has been used in two Early Reading First projects, in Tennessee and Oklahoma, and will be prominent in a federally funded, longitudinal study of preschools in the Memphis City Schools system. A copy of the E-LOT can be found in Appendix A. Early Language & Literacy Classroom Observation (ELLCO). The ELLCO is a toolkit that provides researchers and practitioners with guidelines for describing the degree to which a classroom supports literacy activities. Primarily an environmental checklist, the ELLCO contains three assessment tools: a Literacy Environment Checklist, protocols to conduct classroom observation and administer teacher interviews, and a Literacy Activities Rating Scale. The total time to complete the observation is approximately one to one and one-half hours. The ELLCO is used for research purposes in over 150 preschool classrooms and has an established statistical reliability of 90% or better. Interviews During May, 2006, 10 randomly selected principals from schools with prekindergarten programs participated in a 30-45 minute phone interview with CREP researchers. An interview I 7 )
i. 2005 -2006 Evaluation of the Prekindergarten Literacy Program 22 (' I 1,'^ S'protocol was developed for this study and included questions regarding the principals perceptions of the prekindergarten program at their school and the impact of the program on student achievement, with special note of the impact on African American achievement. Principals were also asked questions regarding the impact the prekindergarten program had on literacy instruction, parental involvement and professional development. Principals were asked to reflect on the perception of other teachers and personnel at the school regarding the program. A copy of the principal interview protocol is included in Appendix B. Surveys Prekindergarten Teacher Questionnaire (PKTQ) Prekindergarten teachers in all 28 schools examined in this study were asked to complete the PKTQ. The PKTQ was designed to capture teachers attitudes and perceptions of their schools prekindergarten program and their experiences as prekindergarten teachers. The PKTQ contained 3 sections. Section 1 consisted of 26 items teachers were asked to evaluate using a 5- point Likert-type scale ranging from (1) Strongly Disagree to (5) Strongly Agree. Items assessed included teachers understanding of the prekindergarten program, principal and staff support of the program, the impact the program had on student achievement and parental involvement, and professional development experiences. The second section contained 7 items which collected demographic information
and the final section allowed teachers to record their own responses to 4 open-ended discussion questions. These questions were: What are the most effective aspects of the prekindergarten program? What are the least effective? Do you think the prekindergarten program should be continued? Why or Why not? What type of professional development have you received as a preschool teacher? Prekindergarten Paraprofessional Teacher Questionnaire (PKPTQ) Prekindergarten paraprofessional teachers in all 28 schools with prekindergarten programs were asked to complete the PKPTQ. The PKPTQ was designed to capture paraprofessional teachers attitudes and perceptions of their schools preschool program and their experiences as paraprofessional teachers. Like the PKTQ, the questionnaire for paraprofessionals contained 3 sections. Section 1 consisted of 18 items paraprofessional teachers were asked to evaluate using a 5-point Likert-type scale ranging from (1) Strongly Disagree to (5) Strongly Agree. Items assessed included paraprofessional teachers imderstanding of the prekindergarten program, principal and staff support of the program, collaboration with their prekindergarten 2005 -2006 Evaluation of the Prekindergarten Literacy Program 23I ,1 teacher, the impact the program had on student achievement and parental involvement, and professional development experiences. The second section contained 7 items which collected demographic information
and the final section allowed paraprofessionals to record their own responses to 4 open-ended discussion questions. These questions were: What are the most effective aspects of the prekindergarten program? What are the least effective? Do you think the prekindergarten program should be continued? Why or Why not? What type of professional development have you received as a preschool paraprofessional teacher? Classroom Teacher Questionnaire (CTQ) Kindergarten and first grade teachers in all 28 schools with prekindergarten programs were asked to complete the CTQ. The CTQ was designed to capture these teachers attitudes and perceptions of their schools prekindergarten program. This questionnaire also contained 3 sections. Section 1 consisted of 12 items in which teachers were asked to evaluate using a 5- point Likert-type scale ranging from (1) Strongly Disagree to (5) Strongly Agree. Items assessed included teachers understanding of the prekindergarten program, support of the program, effectiveness of the program, and impact of the prekindergarten on student achievement and parental involvement. The second section contained 7 items which collected demographic information
and the final section allowed teachers to record their own responses to 3 open-ended discussion questions. I 'M tl I These questions were: What are the most effective aspects of the prekindergarten program? What are the least effective? Do you think the prekindergarten program should be continued? Why or Why not? Parent/Guardian Questionnaire (PQ) All parents/guardians of prekindergarten students were asked to complete the PQ. The PQ was designed to capture attitudes and perceptions of their childs prekindergarten program. This questionnaire consisted of 12 items in which parents or guardians were asked to evaluate using a 5-point Likert-type scale ranging from (1) Strongly Disagree to (5) Strongly Agree. Items assessed included their understanding of the prekindergarten program, the impact the program had on their child, and their overall perceptions of the teacher and the school. Respondents were also asked to indicate their childs ethnicity, age, and gender. The final section allowed parents/guardians to record their own responses to 2 open-ended discussion questions. These questions were: What are the best things about your childs preschool experience? What changes would you like to see in the preschool program at this school? fl p I Jj !! r'i, r r 1 2005 -2006 Evaluation of the Prekindergarten Literacy Program 24 1A copy of all four surveys: PKTQ, PKPTQ, CTQ, and PQ can be found in Appendix C. 2005 -2006 Evaluation of the Prekindergarten Literacy Program 257 SCREENING ASSESSMENTS Early Screening Inventory (ESI). The Early Screening Inventory is a developmental screening instrument that was revised and standardized in 1997. Published by Pearson Early Learning, ESI allows the childs teacher to quickly screen and assess children in the areas of motor development, language development, and cognition. ESI is considered highly reliable and valid, with 9 out of 10 children at risk for failure to be adequately identified. ESI is a continuous protocol divided into two sections: ESI-P is for 3-41/2 year olds and ESI-K is for ages 4.5 years to age 6. The 2005-2006 school year was the first year for prekindergarten teachers to use the Early Screening Inventory to assess their students. STUDENT ACHIEVEMENT RESULTS In addition to the screening assessments, interviews, questionnaires, and observation tools, student achievement data were derived from student scores on the Work Sampling System (WSS), the Dynamic Indicators of Basic Early Literacy Skills (DIBELS), the Qualls Early Learning Inventory (QELI), and the Iowa Tests of Basic Skills (ITBS). A description of each assessment follows. Work Sampling System (WSS). Developed in 1995, WSS is a performance assessment system, rather than a battery of tests, that assesses and documents childrens skills, knowledge, behavior, and accomplishments over a spectrum of educational areas and throughout a variety of occasions. WSS is a method in which teacher observations are organized around a set of criteria and procedures. There are 3 primary components to WSS: (1) Development Guidelines and Checklists, (2) Portfolios, and (3) Summary Reports. The checklists are designed for teachers to complete without formally testing their students. Each checklist covers seven areas including personal and social development, language and literacy, mathematical thinking, scientific thinking, social studies, the arts, and physical development. Portfolios are used to collect meaningful representations of childrens work
finally, summary reports are designed to be completed three times a year and are a way of translating the information from the checklists and portfolios into an easily understood document for parents and teachers. For the purposes of this study, WSS fall 2005 prekindergarten students results were collected and compared with WSS spring 2006 prekindergarten students results as a measure of student growth throughout the year. 2005 -2006 Evaluation of the Prekindergarten Literacy Program 26 I
VDynamic Indicators of Basic Early Literacy Skills (DIBELS/ DIBELS is a tool for early identification of children with potential literacy problems and an assessment of response to instruction. The DIBELS assessment is designed to enable educators to modify their approach if a student is not on course to achieve reading goals. The Early Childhood Research Institute on Measuring Growth and Development at the University of Oregon constructed DIBELS. The measures were developed upon the essential early literacy domains discussed in both the National Reading Panel (2000) and National Research Council (1998) reports to assess student development of phonological awareness, alphabetic understanding, and automaticity and fluency with the code. Each measure has been thoroughly researched and demonstrated to be reliable and valid indicators of early literacy development and predictive of later reading proficiency to aid in the early identification of students who are not progressing as expected. When used as recommended, the results can be used to evaluate individual student development as well as provide grade-level feedback toward validated instructional objectives. The Institute has validated the instruments ability to predict outcomes and has tested its reliability with young children across the country. Not all students in the LRSD were assessed with DIBELS. Primarily DIBELS was administered to those kindergarten students in Reading First schools or those students who did not do well on portions of the QELI. Qualls Early Learning Inventory (QELI). The QELI is an assessment tool primarily for kindergarten students. The QELI identifies student development as either developed,' developing, or delayed in six areas: general knowledge, oral communication, written language, math concepts, work habits, and attentive behavior. The QELI is an inventory, not a test, which teachers complete on each child based on recent observations of the students behavior. In 2000, a national standardization study was conducted and norms were established to allow for a variety of interpretations of the scores. The state of Arkansas requires all incoming kindergarten students to be inventoried with the QELI. For the purposes of this study, fall 2005 QELI results for all incoming kindergarteners were collected for comparisons between those who had attended prekindergarten the prior year and those who had not. Iowa Test of Basic Skills (ITBS). The ITBS is a norm-referenced group administered test that measures the skills and achievement of students in grades K-8 and was developed at the 2005 -2006 Evaluation of the Prekindergarten Literacy Program 27 I i S! University of Iowa. The ITBS provides an in-depth assessment of students achievement of important educational objectives. Tests in Reading, Language Arts, Mathematics, Social Studies, Science, and Source of Information yield reliable and comprehensive information. For analysis of kindergarten students, spring 2006 ITBS achievement scores were used. For the longitudinal analysis of the prekindergarten program, 10 years of ITBS data were compiled in which comparisons between those cohorts who attended preschool in the LRSD were compared with those students who did not attend preschool in the LRSD. Table 3 sununarizes each evaluation question, the relevant participants, and the measurement used for analyses. I.-
I I I b' '.i III J 5 2005 -2006 Evaluation of the Prekindergarten Literacy Program 28 1Table 3. Summary of Data Sources and Participants by Evaluation Question Evaluation Question Participants Data Sources Methodology Primary Question 1. Has the prekindergarten program been effective in improving and remediating the academic achievement of African American students? Xt
2005-2006 Prekindergarten students in the 28 state-funded LRSD preschool sites Prekindergarten student program data - Work Sampling System scores (2005-2006 data) Fall scores compared to spring scores for notable gains. B: 2005-2006 LRSD kindergarten students C: Qualls Early Learning Inventory (QELI) or Dynamic Indicators of Basic Early Literacy Skills (DIBELS) Fall 2005 data
DIBELS in Reading First schools only Spring 2006 ITBS kindergarten scores Kindergarten Fall QELI or DIBELS scores: comparisons of 2004-2005 prekindergarten students vs. non-prekindergarten students. Supplemental Questions 1. What are the quality of instruction and level of implementation of prekindergarten programs in classroom environments at the 28 schools implementing in 2005-2006? 2. What is the level of participation in prekindergarten by African American students relative to other ethnic groups at the school? All LRSD students in 1- 10 grade All prekindergarten teachers All prekindergarten paraprofessionals All K and 1 grade classroom teachers at schools with prekindergarten programs Principals at prekindergarten schools Randomly selected pre kindergarten classrooms All prekindergarten schools ITBS District longitudinal achievement data PKTQ PKPTQ CTQ Phone interviews of 10 randomly selected principals ELLCO and E-LOT School records/archival data ITBS scores compared between previous prekindergarten students vs. non- prekindergarten students. Comparison of those identified as former prekindergarten participants vs. those who did not attend LRSD piekindergarlen All surveys collected participants attitudes and perceptions regarding prekindergarten program. Phone intenriews gathered principals' attitudes and perceptions regarding prekindergarten management. Classroom environments and literacy instruction were evaluated using obsen/ation tools Comparisons of African American prekindergarten population relative to other ethnic groups 2005 -2006 Evaluation of the Prekindergarten Literacy Program 29Table 3, continued. Evaluation Question Participants Data Sources Methodology 3. To what extent does the prekindergarten program provide screening assessments and other appropriate measures to help identify and monitor the progress of African American children who may be at-risk for academic failure? All prekindergarten teachers and paraprofessionals PKTQ&PKPTQ Teachers attitudes and perceptions regarding screening collected 2005-2006 prekindergarten student data Early Screening Inventory (ESI) ' Database analyzed for screening records Principals 4. What are the perceptions of prekindergarten teachers and paraprofessional teachers regarding the prekindergarten program implementation, impacts, strengths, and weaknesses? 5. What are the perceptions of principal, K and 1* grade classroom teachers regarding the prekindergarten program Implementation, impacts, strengths, and weaknesses? 6. What are the perceptions of parents/guardians of prekindergarten children regarding program impacts, strengths, and weaknesses? Phone interviews of 10 randomly selected principals Principals knowledge and perception of screening collected All prekindergarten teachers and paraprofessionals PKTQ&PKPTQ Teachers attitudes and perceptions regarding the prekindergarten program collected All K & 1 grade classroom teachers 10 randomly selected principals Parents/guardians of all 2005-2006 LRSD prekindergarten students CTQ Principal phone interview PQ Teachers attitudes and perceptions regarding the prekindergarten program collected Principals attitudes and perceptions regarding the prekindergarten program collected. Parents/guardians attitudes and perceptions regarding the prekindergarten program collected PROCEDURE Data for the evaluation were collected April-May 2006 for the 2005-2006 school year. During the week of April 17^, 12 randomly selected preschool sites were visited and observed using the E-LOT and the ELLCO. All observations were conducted by trained CREP researchers. Also in April, all teacher questionnaires were disseminated during school faculty meetings. Prekindergarten teachers were responsible for distributing parent questionnaires to preschool parents. In May, 10 randomly selected principals were interviewed via phone. Student achievement data were received from the district during the summer and fall of 2006. METHODS - EARLY SCREENING INVENTORY Sample. In 2005-2006, the Little Rock School District reported that the Early Screening Inventory (ESI) was administered to 1,206 preschool students. Due to duplicate records, missing student identification numbers, and missing inventory scores, the final sample consisted of 1,105 students enrolled in the Little Rock preschool program. The sample was drawn from 28 LRSD preschool sites included all eligible students who were enrolled in preschool in 2005-2006. The number of students from each of the schools appeared to be reflective of preschool enrollment. 2005 -2006 Evaluation of the Prekindergarten Literacy Program 30 IIn the previous year, 1,068 students were enrolled in Little Rock preschool programs. Students per school ranged from 15 to 113 students (See Table 1). Seventy-one percent of the preschool students in the sample were African Americans. Most students were English speaking (95%), eligible for free or reduced price lunch (68%), and did not have a formal lEP (100%). There were slightly more females (51%) than males in the sample. Tables 4 and 5 summarize the sample population. 2005 -2006 Evaluation of the Prekindergarten Literacy Program 3]Table 4. ESI Sample by School Whole Sample African Americans School Name N Percent N Percent Bale Baseline Brady Carver Chicot Dodd Fair Park Forest Park Franklin Fulbright Geyer Springs Jefferson King Mabelvale McDermott Meadowclitf Otter Creek Pulaski Heights Rockefeller Romine Stephens Terry Wakefield Washington Watson Western Hills Wilson Woodruff Total 35 30 37 15 50 31 3.2 2.7 3.3 1.4 113 37 50 18 30 20 4.5 2.8 10.2 3.3 30 23 28 12 36 17 71 62 32 40 35 18 31 46 66 35 35 68 32 19 15 34 1105 4.5 1.6 2.7 1.8 6.4 5.6 2.9 3.6 3.2 1.6 2.8 4.2 6.0 3.2 3.2 6.2 2.9 1.7 1.4 3.1 100.0 75 9 50 1 28 4 43 42 16 31 20 5 20 40 62 24 28 56 30 13 13 27 783 3.8 2.9 3.6 1.5 4.6 2.2 9.6 1.1 6.4 0.1 3.6 0.5 5.5 5.4 2.0 4.0 2.6 0.6 2.6 5.1 7.9 3.1 3.6 7.2 3.8 1.7 1.7 3.4 100.0 2005 -2006 Evaluation of the Prekindergarten Literacy Program 32Table 5. Preschool Demographics Whole Sample African Americans Characteristic Race or Ethnicity White African American Hispanic Asian/Pacitic Islander Native American Other Total Gender Female Male Total Free and Reduced Lunch Eligibility Free Full Pay Reduced Total Limited English Proficiency No Yes N 198 783 63 26 2 33 1105 560 545 1105 645 358 102 1105 1045 60 1105 Percent 17.9 70.9 5.7 2.4 0.2 3.0 100.0 50.7 49.3 100.0 58.4 32.4 9.2 100.0 94.6 5.4 100.0 N 404 379 783 554 151 78 783 779 4 783 Percent 51.6 48.4 100.0 70.8 19.3 10.0 100.0 99.5 0.5 100.0 Special Education No 1105 1105 100.0 100.0 783 783 100.0 100.0 Measures. The early screening analysis utilized demographic data and one standardized achievement measures. The demographic characteristics were used for descriptive statistics and as covariates. The achievement measure served as the dependent variable. Demographics. The following student demographic characteristics were provided by Little Rock School District: Race/Ethnicity. Race/Ethnicity was reported in six categories: White, African American, Hispanic, Asian/pacific islander, Native American, and Other. For purposes of this analysis, which centers on remediation of African American students, dummy variables for African American (1) and all Other Races (0) were created. 2005 -2006 Evaluation of the Prekindergarten Literacy Program 33Gender. Gender was reported in two categories: Female and male. In this analysis, female was coded as (1) and male as (0). Free and Reduced Lunch Eligibility. Students were coded as being either eligible for free lunch, reduced price lunch or full-pay. Dummy variables were created that condensed free and reduced price lunch eligibility into one category (1) and full-pay into a second (0). This variable served as an imperfect proxy for socioeconomic status. Limited English Proficiency. Students who had limited English proficiency in kindergarten were coded as 1
those who were English proficient were coded as 0. Achievement. This report utilized data from the Early Screening Inventory (ESI). The Early Screening Inventory is a developmental screening instrument that was revised and standardized in 1997. Published by Pearson Early Learning, ESI allows the childs pre-k teacher to quickly screen and assess children in the areas of motor development, language development, and cognition. The 2005-2006 school year was the first year for pre-k teachers to use the Early Screening Inventory to assess their students. Design. To describe the extent to which the prekindergarten program used screening assessments and other appropriate measures to identify African-American children at-risk for academic failure, a cross-sectional design that examined one group of students (preschoolers) at one point in time (Fall 2005) was utilized. The reader should note that although this is a common design for social science research, this is not a design suited to draw causal inference. Instead, this design describes who received screening and illustrate conelational relationships between demographics and screening scores. 2005 -2006 Evaluation of the Prekindergarten Literacy Program 34 bMETHODS - STUDENT ACHIEVEMENT Students achievement was analyzed using a three-tiered approach in which both immediate and long term results were studied. Methodology for Analysis A, Analysis B, and Analysis C is described below. Analysis A
Prekindergarten Achievement Sample. For this analysis, the sample was restricted to students enrolled in public school programs and were in a center-based preschool program (n=1342). Furthermore, the sample was restricted to those students who were enrolled in ABC State funded preschool programs. This resulted in a final analysis sample of 1,300 students from 28 schools.* Table 6 illustrates the original number of prekindergarten students in the study and the number of students included in Analysis A. Minor changes in population were expected, due to the dynamic nature of student populations. * This analysis includes 136 students originally omitted due to a coding error. All available student scores are now included in the analysis. 200S -2006 Evaluation of the Prekindergarten Literacy Program 35 I1 Table 6. Sample Population: Analysis A 1I I School * Number of Students, Bale Baseline Brady Carver Chicot Dodd Fair Park Forest Park Franklin Fulbright Geyer Springs Jefferson M.L. King Mabelvale McDermott Meadowcliff otter Creek Pulaski Heights Rockerfeller Romine Stephens Terry Wakefield Washington Watson Western Hills Wilson Woodruff , Prekindergarten 38
40 - 2O.'?...i5.. 71: LiS: 40
37 60 70 38 40 39 J 19 39 37:W'K--Si 40 58S 40 I Si ijts 80 . 38 , 20 - 17 . 34 1,316 Number of Students, Analysis A 39 45 39 20 75 38 132 39 65 39 39 39 73 72 39 40 40 19 40 55 67 1 41 81 37 20 25 41 1,300 Percent of Sample, Analysis A 3.0 3.5 3.0 1.5 5.8 2.9 10.2 3.0 5.0 3.0 3.0 3.0 5.6 5.5 3.0 3.1 3.1 1.5 3.1 4.2 5.2 .1 3.2 6.2 2.8 1.5 1.9 3.2 X 100.0 w 39 t Seventy-five percent of the students in the sample were African American (See Table 7). Most students were English speaking (95%), did not have a formal lEP (99%), and were bom in I 2000 or 2001 (99%). The sample was evenly split between male and female students. I I I 2005 -2006 Evaluation of the Prekindergarten Literacy Program 36 i Table 7. Demographics of Analysis A sample (n = 1,164) n % Ethnicity American Indian, Alaskan Native Asian Black, African American Hispanic/Latino Other White 2 26 915 72 42 243 .2 2.0 70.4 5.5 3.2 18.7 Gender lEP Language Ages Female Male Yes English Other Spanish Born in 1999 Born in 2000 Born in 2001 Born in 2005 657 643 50.5 49.5 12 1232 17 51 4 429 865 2 .9 94.8 1.3 3.9 .3 33.0 66.5 .2 Measures. The Work Sampling System is a valid, research-based performance assessment that utilizes guidelines and checklists, portfolios, and summary reports. Teachers document childrens skills, knowledge, behavior, and academic accomplishments using age- appropriate instruments. The Work Sampling System (WSS) is published by Pearson Early Learning, has been utilized since 1991 and is purportedly a curriculum-embedded assessment based on national and state standards. Efforts have been made to measure both the validity and reliability of this instrument. In a validity study conducted in 17 urban Title I classrooms, WSS ratings were compared to scores on the nationally-normed, standardized Woodcock Johnson Psychoeducational Battery-Revised and were found to be highly correlated and a stronger predictor variable of test scores than demographic variables (Meisels et al. 2001). Meisels et al. (1995) found high internal and moderately high interrater reliability when they studied the instrument with 100 kindergarten aged children and controlled for gender, maturation, and initial ability. Teachers repeatedly assess childrens growth in the following seven domains: Personal and Social Development 2005 -2006 Evaluation of the Prekindergarten Literacy Program 31 Language and Literacy Mathematical Thinking Scientific Thinking Social Studies The Arts and Physical Development and Health Students are scored on an ordinal scale with three categories: Not Yet (1) - indicating that the child cannot perform the indicator In Process (2) - the skills, knowledge, behaviors or accomplishments are intermittent or emergent but are not demonstrated reliably or consistently. Proficient (3) - the child can reliably demonstrate the skills, knowledge, behaviors or accomplishments. Design. To determine Afiican American preschool students progress on the domains measured by the WSS, a pre-and post-test design was employed for the sample. In the fall, 2005, teachers used the WSS to determine the baseline status of preschool students in the seven domains, for Time 1 results. Teachers again assessed students midyear. Time 2 results, and again at the end of the year, for Time 3 results. An advantage to this design is that all preschool aged children enrolled in a LRSD public preschool program were assessed. However, this meant there was no comparison group. Therefore, it was problematic to determine if students enrolled in Little Rock preschool programs made better progress (based on the WSS measure) than students who were not enrolled. Additionally, the design was prone to ceiling and floor effects. Students who tested at the highest proficiency stage at Time 1 had no place to move upward - the measurement capped growth for the highest achievers. Likewise, those that scored in the Not Yet category had no option for showing that they regressed. Analysis B
Kindergarten Achievement Sample. For this analysis, achievement results for kindergarten students in the LRSD for 2005-2006 were studied. In 2005-2006, Little Rock School District reported that 2,234 students enrolled in kindergarten
however, 70 students had duplicate records. When the duplicates were removed, a sample size of 2,164 kindergarten students was established Because the purpose of this analysis was to compare students progress in kindergarten for those who attended Little Rock preschool programs with those who did not, a close 2005 -2006 Evaluation of the Prekindergarten Literacy Program 38 I , I, Jexamination of preschool enrollment was conducted to ensure that students who were counted in the preschool enrollment were fully served. Fully served was defined as enrolling in a LRSD preschool program prior to October 1,2004. Students enrolling by this date were assumed to have received most of a full year of service. Using this definition, we found 83 students identified as having attended preschool programs did not meet the definition of fully served (2 students enrolled in 2006, 19 enrolled in 2005, and 62 enrolled in 2004 but after the cut-off deadline of October These 83 students were reclassified as not having preschool services. Thus, the final analysis sample consisted of 1,098 students who received preschool services from the Little Rock School District and 1,066 kindergarteners who did not. Table 8 summarizes the kindergarten population relative to the prior years prekindergarten attendance. The sample was drawn from 31 Little Rock elementary schools and should include all eligible students who were enrolled in kindergarten in 2005-2006. The number of students from each of the elementary schools was reflective of kindergarten enrollment and ranged from 35 to 108 students per school. Table 9 illustrates the kindergarten population by school. Sixty-six percent of the kindergarten students in the sample were African American. Most students were English speaking (93%), eligible for free or reduced price limch (69%), and did not have a formal lEP (91%). There were slightly more males (52%) than females in the sample. Caucasian students were significantly less likely to have been enrolled in prekindergarten programs than African American students (17 % versus 73%). Students who were determined in kindergarten to have limited English proficiency (LEP) were much less likely to have attended the prekindergarten program. Likewise, kindergarten students who were flagged as needing special education were significantly less likely to have attended a Little Rock preschool program. Table 10 further describes the demographics of the student population. Table 8. Description of PreKindergarten Enrollment Preschool Participants Non-participants N 1098 1066 Whole Percent 50.7 49.3 African Americans N 805 614 Percent 56.7 43.3 N 293 452 Other Races Percent I 39.3 : 60.7 Total 2164 100.0 1419 100.0 745 100.0 2005 -2006 Evaluation of the Prekindergarten Literacy Program 39Table 9. Description of Enrollment by School Whoie Sample African American other Races School Code 006 017 018 020 021 022 024 025 027 028 029 030 032 033 035 036 037 038 039 040 041 042 043 Preschool Participants Non-participants Preschool Participants Non-participants Preschool Participants Non participants N 43 29 37 42 20 21 37 38 14 53 29 41 34 35 65 34 28 28 20 33 54 65 18 Percent 3.9 2.6 3.4 3.8 1.8 1.9 3.4 3.5 1.3 4.8 2.6 3.7 3.1 3.2 5.9 3.1 2.6 2.6 1.8 3.0 4.9 5.9 1.6 N 54 29 37 34 56 20 25 42 25 48 10 21 22 39 38 24 29 31 17 26 42 36 40 Percent 5.1 2.7 3.5 3.2 5.3 1.9 2.3 3.9 2.3 4.5 0.9 2.0 2.1 3.7 3.6 2.3 2.7 2.9 1.6 2.4 3.9 3.4 3.8 N 31 27 35 29 14 19 3 36 10 42 28 8 20 29 46 26 25 10 18 29 51 57 12 Percent 3.9 3.4 4.3 3.6 1.7 2.4 0.4 4.5 1.2 5.2 3.5 1.0 2.5 3.6 5.7 3.2 3.1 1.2 2.2 3.6 6.3 7.1 1.5 N 24 22 30 21 23 12 1 39 10 31 7 4 15 23 28 20 22 5 15 20 41 30 14 Percent 3.9 3.6 4.9 3.4 3.7 2.0 0.2 6.4 1.6 5.0 1.1 0.7 2.4 3.7 4.6 3.3 3.6 0.8 2.4 3.3 6.7 4.9 2.3 2005 -2006 Evaluation of the Prekindergarten Literacy Program 40 N 12 2 2 13 6 2 34 2 4 11 1 33 14 6 19 8 3 18 2 4 3 8 6 Percent 4.1 0.7 0.7 4.4 2.0 0.7 11.6 0.7 1.4 3.8 0.3 11.3 4.8 2.0 6.5 2.7 1.0 6.1 0.7 1.4 1.0 2.7 2.0 N 30 7 7 13 33 8 24 3 15 17 3 17 7 16 10 4 7 26 2 6 1 6 26 Percent 6.6 1.5 1.5 2.9 7.3 1.8 5.3 0.7 3.3 3.8 0.7 3.8 1.5 3.5 2.2 0.9 1.5 5.8 0.4 1.3 0.2 1.3 5.8Table 9, continued. Whole Sample Preschool Participants African American Other Races School Code N Percent 044 20 1.8 045 20 1.8 046 20 1.8 047 38 3.5 048 40 3.6 050 50 4.6 051 46 4.2 052 46 4.2 Total 1098 100.0 Non-participants Preschool Participants Non-participants Preschool Participants Non participants N 29 15 19 70 60 49 31 48 1066 Percent 2.7 1.4 1.8 6.6 5.6 4.6 2.9 4.5 100.0 N 18 20 18 25 9 33 34 , 43 805 Percent 2.2 2.5 2.2 3.1 1.1 4.1 4.2 5.3 100.0 N 23 15 12 26 6 18 21 36 614 Percent 3.7 2.4 2.0 4.2 1.0 2.9 3.4 5.9 100.0 2005 -2006 Evaluation of the Prekindergarten Literacy Program 41 N 2 na 2 13 31 17 12 3 293 Percent 0.7 0.7 4.4 10.6 5.8 4.1 1.0 100.0 N 6 na 7 44 54 31 10 12 452 Percent 1.3 1.5 9.7 11.9 6.9 2.2 2.7 100.0Table 10. Kindergarten Demographics Whoie Sample African Americans other Races Preschool Participants N Percent N Non-participants Percent Preschool Participants Non-participants Preschool Participants N Percent N Percent N Percent N Non-participants Percent Ethnicity White Black Hispanic Asian/Pacific Islander Native American Other Total 191 805 56 15 2 29 1098 17.4 73.3 5.1 1.4 0.2 2.6 100.0 282 614 101 17 11 41 1066 26.5 57.6 9.5 1.6 1.0 3.8 100.0 Gender Female Male 533 565 1098 48.5 51.5 100.0 497 569 1066 46.6 53.4 100.0 398 407 805 49.4 50.6 100.0 288 326 614 46.9 53.1 100.0 135 158 293 46.1 53.9 100.0 209 243 452 46.2 53.8 100.0 Free and Reduced Lunch Eligibility Free Full Pay Reduced 662 334 102 1098 60.3 30.4 9.3 100.0 662 339 65 1066 62.1 31.8 6.1 100.0 576 146 83 805 71.6 18.1 10.3 100.0 491 89 34 614 80.0 14.5 5.5 100.0 86 188 19 293 29.4 64.2 6.5 100.0 171 250 31 452 37.8 55.3 6.9 100.0 Limited English Proficiency No Yes 1040 58 1098 94.7 5.3 100.0 973 93 1066 91.3 8.7 100.0 804 1 805 99.9 0.1 100.0 614 614 100.0 100.0 236 57 293 80.5 19.5 100.0 359 93 452 79.4 20.6 100.0 2005 -2006 Evaluation of the Prekindergarten Literacy Program 42Table 10, continued. Whole Sample African Americans Other Races Preschool Participants N Percent N Non-particIpants Percent Preschool Participants N Percent N Non-particIpants Percent Preschool Participants N Percent N Non-partlcIpants Percent Special Education No Yes 1035 63 1098 94.3 5.7 100.0 945 121 1066 88.6 11.4 100.0 759 46 805 94.3 5.7 100.0 534 80 614 87.0 13.0 100.0 276 17 293 94.2 5.8 100.0 411 41 452 90.9 9.1 100.0 Enrollment in prekindergarten programs was significantly different, Independent samples t test, p<.05 2005 -2006 Evaluation of the Prekindergarten Literacy Program 43Measures. Analysis B drew upon six demographic and three standardized achievement measures. The demographic characteristics were used for descriptive statistics and as covariates. The achievement measures served as the dependent variable. Demographics. Student demographic characteristics were provided by Little Rock School District and are described below. Prekindergarten Program Enrollment. Each kindergarten student was tagged with an indicator of whether he/she attended a Little Rock prekindergarten program. This field, along with data about the timing of enrollment, was used to construct a variable that defined students as having participated in a full-year of preschool in the Little Rock system (1) or not (0). Race/Ethnicity. Race/Ethnicity was reported in six categories: Caucasian, African American, Hispanic, Asian/Pacific Islander, native American, and other. For the purpose of this analysis, which centers on remediation of African American students, a dummy variable was created for African American (1) and all Other Races (0). Gender. Gender was reported in two categories
Female (1) and male (2). Free and Reduced Price Lunch Eligibility. Students were coded as being either eligible for free lunch, reduced price lunch or full-pay lunch. Dummy variables were created that condensed free and reduced price lunch eligibility into one category (1) and full-pay into a second (0). This variable served as an imperfect proxy for socioeconomic status. Limited English Proficiency. Students who had limited English proficiency in kindergarten were coded as 1
those who were English proficient were coded as 0. Special Education. Students who had an Individual Education Plan (lEP) in kindergarten were identified as needing special education (1). Those who did not have an lEP were coded as 0. Achievement. This report utilized data from three student achievement measures
Qualls Early Learning Inventory, Dynamic Indicators of Early Literary, and Iowa Tests of Basic Skills. Qualls Early Learning Inventory. The Qualls Early Learning Inventory (QELI) (previously known as the Iowa Early Learning Inventory) is designed to measure six behavioral areas and is intended to be administered in conjunction with a measure of achievement such as the Iowa Test of Basic Skills (ITBS). The primary purpose of the QELI is to provide observational information for kindergarten and first graders in the following areas: (a) General Knowledge, (b) Oral Communication, (c) Written Language, (d) Math Concepts, (e) Work 2005 -2006 Evaluation of the Prekindergarten Literacy Program 44 ftHabits, and (f) Attentive Behavior. The instrument is completed by students classroom teachers and takes about 10 minutes to complete. There are 44 items distributed across the six behavioral areas which are scored on a 3 or 4 point scale ranging from never to often. National norms are used to generate a developmental description of each behavioral area. The three developmental levels that are used for reporting are delayed (0), developing (1), or developed (2). For these analyses, two types of scores were reported for each behavioral area: percent correct (0 to 100) and classification (delayed, developing, developed). Total scores were converted into percent correct so that behavioral areas with different numbers of items could be compared with a similar rating rubric. The QELI has been vetted for validity and reliability. The inventory includes behaviors that: (a) are related to achievement, (b) can be further developed as a result of experience, (c) are not considered a disability if delayed in development, and (d) can be observed in the classroom. In April 1999, kindergarten and first grade teachers from various regions of the U.S. reviewed items and used them in their classes. Early childhood specialists reviewed the final form for content relevance, appropriateness of items, and potential bias. The empirical evidence of validity included correlations among the six QELI behavior scores and correlations between six QELI scores and scores from the subtests of the Iowa Test of Basic Skills (ITBS). The correlations are fairly high among the six QELI scores ranging in value from a low of .594 to a high of .865. The correlations between the six QELI scores and ITBS subtest scores demonstrate low to moderate correlations ranging in value from .178 to .570. In general, the cognitive skills on the QELI were more highly correlated with the scores on the ITBS than the two behavioral skills. Internal-consistency reliability using coefficient alphas for each domain ranged from .80 to .93, which are adequate. The normative data were collected in the spring of 2000. The 2,108 students in the standardization were from 392 kindergarten classes in 47 states. Roughly, the findings indicated that 7% of students were in the "Delayed" category, 30% were in the "Developing" category, and 63% were in the "Developed" category for each of the six domains. The standardization study was conducted concurrently with the national standardization of the Iowa Test of Basic Skills (Eastman Lukin, 2003). Dynamic Indicators of Basic Early Literacy Skills. The Dynamic Indicators of Basic Early Literacy Skills, 6* Edition (DIBELS) assessments are designed to identify and monitor the 2005 -2006 Evaluation of the Prekindergarten Literacy Program 45 progress of students who are unlikely to meet state reading standards in third grade. DIBELS is a teacher administered battery of early literacy tests designed for use as benchmark or monitoring assessments so teachers can tailor instruction to meet students needs. The benchmarks are given three times a year, but the tests are quite brief and can be administered in 1 to 3 minutes. Most of the tests have 20 alternate forms allowing teachers to evaluate a struggling student frequently. In this study, the following DIBELS subtests were examined: Letter Naming Fluency, Phoneme Segmentation Fluency, and Nonsense Word Fluency. Results are reported as total scores. Because each domain has a different possible total score which can change over the three administrations, comparisons can be difficult. Rules are provided to translate scores into At Risk, Some Risk, and Low Risk categories. The intent is that students who are At Risk will be targeted for intensive intervention. For this study, the Risk categorizations were used to determine proportion of students in each category versus the expected proportions. The Little Rock students did not take the full complement of tests (beginning, middle, and end), therefore, growth can not be measured. Instead, analyses concentrated on the end score. The reliability and validity evidence for DIBELS is robust. The Letter Naming Fluency (alternate form: .87) showed the most evidence of reliability. Similarly, these tests had strong predictive and concurrent validity evidence when compared to the Woodcock-Johnson Reading Tests and other measures. The average concurrent validity coefficients (correlations with other measures taken at the same time) were .58 for Nonsense Word Fluency and .44 for Phoneme Segmentation Fluency. The predictive validity coefficients were .47 for Phoneme Segmentation Fluency, and .68 for Nonsense Word Fluency. No data were found concerning the reliability or validity of the instructional risk classifications (Shanahan, 2003
Brunsman 2003). Iowa Tests of Basic Skills. The Iowa Test of Basic Skills (ITBS) is designed to assess basic skills necessary for students to progress through school. The teacher administered tests are available in Levels, roughly corresponding to the ages of children who will take them (Levels 5 and 6 correspond to kindergarten). At this level, the ITBS takes no more than 30-minutes to complete. For this study, six skill areas were examined: Vocabulary, Word Analysis, Listening, Revised Writing, Concepts Problems, and Math Total (with computation). In addition, two summary scores that combined the six core skill areas were used: Core Total (with no computation) and Core Total (with computation). 2005 -2006 Evaluation of the Prekindergarten Literacy Program 46Domain scores were reported as normal curve equivalents (NCE) on a scale of 1-99. A normal curve equivalent score is a type of norm-referenced score. It differs from percentile rank score in that it allows meaningful comparison between different test sections within a test. For example, if a student receives NCE scores of 53 on the Reading test and 45 on the Mathematics test, you can correctly say that the Reading score is eight points higher than the Mathematics score. Unlike percentile rank scores, the interval between scores is equal. This means that you can average NCE scores to compare groups of students or schools. Normal Curve Equivalent scores can only be used for students who are similar in age or grade to those in the norming sample. The ITBS tests were normed in 2000 and 2005 based on a national sample of over 100,000 students including kindergarteners. The ITBS tests have been scrutinized in terms of validity and reliability. Content for the ITBS was based on the careful studying of texts and other curriculum materials, the recommendations of professional societies, and the practices of school districts. The authors created subtest intercorrelations and correlations with the Cognitive Abilities Test (CogAT), a test of problem solving and reasoning
correlations with future grades and future test performance
studies of cognitive processes students use for the test, especially for problem solving
bias studies
equating studies related to score meaning
and studies of interpretation and understanding of parents and teachers. The overwhelming evidence is positive, that is, the ITBS scores provide valid measures of basic academic skills, if defined and used in the manner intended. Most subtest reliabilities are in the .80s and .90s across Forms K, L, and M
in general. Levels 5-8 have lower reliabilities (around .80), Core Total and Composite reliabilities are all above .90. Listening Assessment reliabilities range from .67 to .79 (Brookhart 1996). Design. To determine the impact of participation in a Little Rock prekindergarten program on kindergarten performance, a quasi-experimental, post-test only research design was employed. Students who participated in the Little Rock preschool program were compared to students who did not participate in the preschool program, on performance indicators of student achievement. Demographic characteristics were included as covariates to account for the relative influence of personal characteristics on student achievement. Although this was a robust design for social science research, this was not a design suited to draw causal inference. Instead, the design illustrated correlational relationships between participation and later student achievement. 2005 -2006 Evaluation of the Prekindergarten Literacy Program 47Analysis C: 1** through 10*** Grade Student Achievement Sample. For this analysis, district enrollment records that contained 19,919 students in first through tenth grades in 2006 were utilized. The analysis sample was limited to students who initially enrolled in Little Rock Public Schools in either prekindergarten or kindergarten. Cohorts were defined by the 2006 grade level the students were expected to attain if they were not retained in grade. For example, cohort nine was comprised of students who initially enrolled in either prekindergarten in 1995 or kindergarten in 1996. Thus, the final sample for analysis included 12,852 students who initially enrolled in a Little Rock school and remained enrolled in Little Rock schools throughout their school career. Of these, a total of 4,664 (36.3%) had been enrolled in a prekindergarten program. Overall, students who had been enrolled in a prekindergarten program were somewhat more likely to be African American (76% versus 72% of comparison students) and less likely to be male (48% versus 51% of comparison students
see Table). Prekindergarten and comparison students were equally likely to receive free or reduced- price lunch (64% each). Table 11 provides a summary of these demographic variables by cohort and prekindergarten status. Measures. Special education status and current grade level were extracted from LRSD enrollment records to permit an examination of special education refenal rates and retention in grade for African American prekindergarten versus African American comparison students. Four ITBS normal curve equivalent (NCE) scores from the 2005-2006 school year were used as student achievement outcome variables: Total Reading, Spelling, Revised Writing, and Math Core Total - Computation. These were selected because they were the only subtests for which scores were available for all grade levels considered. Analyses. Mean proportions of African American students who received special education status and who were enrolled at the expected grade level were computed for each cohort. Within each cohort, analyses were performed to determine whether there was a statistically significant relationship between African American students prekindergarten status and their remaining at the appropriate grade level or receiving special education status. To control for the large number of comparisons, a sequential Bonferroni procedure was used, and the critical value of alpha set alp .005. For ITBS NCE scores, a 2 (PK vs. Comparison) X 2 (African American vs. Other) X 10 (Cohorts) X 4 (ITBS subtests) multivariate analysis of covariance was performed, with free or reduced price lunch serving as a covariate. Wilks 2005 -2006 Evaluation of the Prekindergarten Literacy Program 48 lambda was employed as the criterion of multivariate significance. Where multivariate results indicated significant main or interaction effects, follow-up univariate analyses of covariance were performed. Effect size estimates were computed by subtracting the comparison group adjusted mean from the prekindergarten adjusted mean, and dividing by 21.06 (the national norm NCE standard deviation). 2005 -2006 Evaluation of the Prekindergarten Literacy Program 49Table 11. Proportion of Students who were African American, Received Free or Reduced-Price Lunch, or were Male by Cohort and Prekindergarten Participation Status Cohort 1 2 3 4 5 6 7 8 9 10 Total Prekindergarten Comparison Prekindergarten Comparison Prekindergarten Comparison Prekindergarten Comparison Prekindergarten Comparison Prekindergarten Comparison Prekindergarten Comparison Prekindergarten Comparison Prekindergarten Comparison Prekindergarten Comparison Prekindergarten Comparison Prekindergarten Black .62 .73 .68 .72 .68 .75 .69 .74 .72 .77 .76 .77 .76 .78 .77 .76 .76 .85 .76 .81 .72 .76 Free or Reduced- Price Lunch .66 .69 .68 .70 .70 .63 .69 .66 .66 .68 .66 .63 .62 .60 .63 .60 .57 .61 .51 .50 .64 .64 Male .50 .48 .53 .48 .52 .51 .54 .50 .50 .51 .51 .45 .50 .41 .52 .41 .50 .51 .49 .44 .51 .48 N 920 659 "922 615 W 616 753 610 528 '875"" 405 "sir 312 788" 292 311 797 316 4664 2005 -2006 Evaluation of the Prekindergarten Literacy Program 50DATA COLLECTION Table 12 provides the type of measure, instrument, brief time line, and number and description of collection process. Table 12. Data Collection Summary Type of Measure Site Visits Surveys Interviews Data Analysis and Reporting Instrument ELLCO & E-LOT Timeline April 2006 Number Collected 17 Prekindergarten teacher Paraprofessional K& 1Teacher Parent/Guardian Principal Phone Interview WSS.QELI, ESI DIBELS, ITBS Spring 2006 Spring 2006 Spring 2006 Spring 2006 May 2006 Received in Summer and Fall 2006 64 58 144 686 10 Description Four trained site researchers from CREP observed 17 classrooms at 12 sites
Literacy activities including direct instruction and learning centers were observed. Each observation lasted approximately 90 minutes and included a brief teacher interview. Distributed by principals: all prekindergarten teachers were asked to complete
response rate: 92.8% Distributed by principals
all paraprofessionals were asked to complete
response rate: 84.1% Distributed by principals
all teachers asked to complete
response rate: 67.3% Distributed by prekindergarten teachers
all parents/guardians of prekindergarten students asked to complete
response rate: 52.1%_______________________________ Conducted by CREP researcher
randomly selected principals were interviewed via phone for 30-45 minutes Data was sent at various times from LRSD. RESULTS When applicable, the results of this study are summarized and presented with distinctions or comparisons between African Americans and those of other races, or of non-African American ethnicity. This format was utilized due to the nature of the study and the primary and supplementary goals of the research to explore distinctions in student achievement results, as well as in program implementations, attitudes, and perceptions of the prekindergarten programs. Prekindergarten Teacher Questionnaire Descriptive results. During the spring of 2006, all 69 prekindergarten teachers in the LRSD were asked to complete a questionnaire regarding their perceptions of the prekindergarten program. The Prekindergarten Teacher Questionnaire (PKTQ) contained 26 items teachers were asked to evaluate using a 5-point Likert scale. Also included in the questionnaire were 7 2005 -2006 Evaluation of the Prekindergarten Literacy Program 51 demographic items and 4 open-ended questions. There were 64 prekindergarten teachers who completed the questionnaire20 who were African American, 40 who were other ethnicities, and 4 who did not indicate ethnicity. A complete summary of the 26 items can be found in Table 13. Overall responses from both African American teachers and teachers of other ethnicity were very positive regarding the prekindergarten program, with 19 items in which over 80% of all teachers expressed either strong agreement or agreement. These items were: Teachers had a thorough understanding of the schools Prekindergarten Literacy program
There was adequate initial and ongoing professional development/training for implementation of Prekindergarten Literacy program
The professional development provided by the district was valuable
The principal was an effective instructional leader
Student achievement was positively impacted by the Prekindergarten Literacy program
The program was valuable for improving the achievement of African American students
There were adequate materials to implement the program
The content areas in the curriculum map were appropriate for the Prekindergarten Literacy program
The monthly benchmark goals were realistic for Prekindergarten Literacy programs
The preschool teachers were generally supportive of the Prekindergarten Literacy r .1 program
The teachers were encouraged to communicate concerns questions, and ideas regarding the Prekindergarten Literacy program
The Prekindergarten Literacy program was useful as an opportunity for early identification of at-risk African American students
The program engendered enthusiasm for learning
The program was valuable in preparing children for kindergarten
Instructional elements of the program were based on scientifically-based reading research
I'll I I 2005 -2006 Evaluation of the Prekindergarten Literacy Program 52 1 Because of the Prekindergarten Literacy program, systematic and explicit instruction in phonemic awareness, oral language, vocabulary, concepts of print, and comprehension occurred daily in the schools classrooms
The literacy and language components were effective for reading readiness
Group activities engaged the students and complemented instruction and, Learning centers engaged the students and complemented instruction There were two items to which African American and other teachers had comparable negative responses: because of the preschool program, more parents were involved in the school
and that they had time to collaborate with other prekindergarten teachers. For African American teachers, 45.0% responded favorably regarding parental involvement and 70.0% responded favorably regarding collaboration with other prekindergarten teachers. For those who were of other ethnicities, just over half of the respondents (52.5%) agreed that more parents were involved because of the prekindergarten program, and 70.0% responded favorably regarding teacher collaboration. There were 5 items in which there were notable differences between African American and other respondents. These items included the following: Teachers had sufficient planning time to implement the Prekindergarten Literacy program: 85% of the African American teachers responded favorably to this item, while 72.5% of others responded favorably
Interim and anchor assessments in the prekindergarten literacy curriculum are useful in assessing growth and progress: 95% of the African American teachers agreed with this item, while only 72.5% of others were in agreement. Teachers had a thorough understanding of the prekindergarten curriculum map: 80% of the African American teachers agreed, while for others, the percentage was much higher at 95.0%. Teachers in the school who were not part of the preschool program were supportive of the Prekindergarten Literacy program: for Afncan American teachers, 75% agreed with this statement
whereas, 90% of other ethnicity teachers responded favorably. The Prekindergarten Literacy program was useful in monitoring progress of Afncan American students: 75/o of the African American teachers responded favorably
the level of agreement for this item rose to 85% for other ethnicities. 2005 -2006 Evaluation of the Prekindergarten Literacy Program 53Demographic data. Over half (65.0%) of the African American teachers who responded to the demographic items reported that they had 5 years or fewer experience as a prekindergarten teacher
for other teachers, 45% indicated 5 years or fewer experience. Only 5% of the African American teachers indicated prekindergarten teacher experience that was more than 16 years, while 15% of the teachers of other ethnicity had more than 16 years experience. Regarding educational level, 35% of African American respondents had a bachelors degree, 30% indicated they had a masters degree, and 15% reported a degree beyond a masters. For non-African American respondents, half reported they had a bachelors degree, 37.5% a masters degree, and 5% indicated they had a degree beyond a masters. Respondents ages were well distributed among all the age groups. At the two extremes, of those that responded, 20% of both African American teachers and other teachers indicated they were 29 years old or less
while 10% of the African American teachers responded they were 60 years old or older, and 2.5% of the other ethnic groups indicated this age. All teachers who responded said they were female
and 60.9 indicated they were Caucasian, 31.3% African American, and 1.6% Pacific Islander. Table 14 contains all demographic responses. Open-Ended Responses. Teachers were asked several open-ended questions in order to further understand their perceptions of the prekindergarten program. A verbatim list of all prekindergarten teachers responses to all open-ended questions can be found in Appendix D. Teachers were first asked to list the most effective aspects of the program. There were 15 African Americans teachers who responded to this question, and 34 distinct aspects were named. For those teachers of other ethnicities, 33 responded to the question and 78 responses were given. Table 15 categorizes and compares the responses. For African American teachers, the most often named aspect was socialization and interaction with other children, which was mentioned 5 times and comprised 14.7% of the responses. Teachers who were not African American also listed this item in 6 responses, with a frequency of 7.7%. The most popular aspect of the prekindergarten program for other ethnicity teachers was the literacy curriculum, which appeared 21 times for a frequency of 26.9%. Other popular responses for both groups included kindergarten preparation, oral language development and exposure to classroom settings. Teachers were also asked to list the least effective aspects of the prekindergarten program. There were 7 African American teachers who responded to this question and 8 I 2005 -2006 Evaluation of the Prekindergarten Literacy Program 54 responses were given. Twenty-five non-African American teachers responded to this question for a total of 28 responses. One-fourth (25%) of the African American teachers responded either none or n/a when asked about least effective aspects
for other teachers this appeared in 17.9% of the responses. Another popular response for African American teachers was too much paperwork, which appeared in two of eight responses. For other teachers, the frequency of this response was 7.1%. Table 16 has complete results for this question. Teachers were also asked whether they felt the prekindergarten program should be continued and all responded favorably. When asked to list reasons for its continuance, the most common response for both groups of teachers was that it helped prepare the children for kindergarten. See Table 17 for a complete list. Teachers were also asked to list the professional development they had received and responses varied greatly. The most common responses can be found in Table 18. 2005 -2006 Evaluation of the Prekindergarten Literacy Program 55Table 13. PREKINDERGARTEN TEACHER QUESTIONNAIRE Total Teacher Responses: 64 Responses of African American Teachers
20 Responses of non African American Teachers: 40 I have a thorough understanding of my schools Prekindergarten literacy program. Teachers, Total African American teachers Non African American teachers I have received adequate initial and ongoing professional development/training for implementation of the Prekindergarten literacy program. Teachers, Total African American teachers Non African American teachers Professional development provided by the district has been valuable. Teachers, Total African American teachers Non Ahican American teachers % Strongly Agree . & Agree 98.4 100.0 100.0 95.3 90.0 100.0 93.8 90.0 97.5 The principal is an effective instructional leader. Teachers, Total African American teachers Non African American teachers Teachers are given sufficient planning time to implement the Prekindergarten literacy program. 79.7 80.0 82.5 % Neutral 0.0 0.0 0.0 1.6 5.0 0.0 6.3 10.0 2.5 12.5 10.0 12.5 % Strongly Disagree & Disagree 1.6 0.0 0.0 1.6 0.0 0.0 0.0 0.0 0.0 4.7 0.0 5.0 Teachers, Total African American teachers Non African American teachers Student achievement has been positively impacted by the Prekindergarten literacy program. 76.6 85.0 72.5 9.4 10.0 10.0 14.1 5.0 17.5 Teachers, Total African American teachers Non African American teachers Overall, this program is valuable for improving the achievement of African American students. 95.3 90.0 97.5 1.6 5.0 0.0 0.0 0.0 0.0 Teachers, Total African American teachers Non African American teachers I have time to collaborate with other Prekindergarten teachers. Teachers, Total African American teachers Non African American teachers I have adequate materials to implement the program. Teachers, Total African American teachers Non African American teachers The content areas presented in the curriculum map are appropriate for the Prekindergarten literacy instruction. 93.8 90.0 95.0 70.3 70.0 70.0 92.2 100.0 90.0 1.6 5.0 0.0 0.0 0.0 0.0 14.1 15.0 12.5 12.5 5.0 17.5 3.1 0.0 5.0 4.7 0.0 5.0 Teachers, Total African American teachers Non African American teachers The monthly benchmark goals are realistic for Prekindergarten literacy programs. Teachers, Total African American teachers Non African American teachers The interim and anchor assessments in the Prekindergarten literacy curriculum are useful in assessing growth and progress. 922 90.0 92.5 95.3 95.0 95.0 6.3 5.0 7.5 0.0 0.0 0.0 4.7 5.0 5.0 0.0 0.0 0.0 Teachers, Total African American teachers Non African American teachers I have a thorough understanding of the Prekindergarten curriculum map. Teachers, Total African American teachers Non African American teachers Because of the Prekindergarten literacy program, more parents are involved in the school. Teachers, Total African American teachers Non African American teachers 79.7 95.0 72.5 89.1 80.0 95.0 50.0 45.0 52.5 15.6 0.0 22.5 4.7 5.0 5.0 4.7 5.0 2.5 1.6 5.0 0.0 31.3 35.0 30.0 15.6 10.0 17.5 2005 -2006 Evaluation of the Prekindergarten Literacy Program 56Table 13, continued. Total Teacher Responses: 64 Responsesof African American Teachers: 20 Responses of non African American Teachers: 40_____________________________________ Preschool teachers in the school are generally supportive of the Prekindergarten literacy program. % Strongly Agree & Agree % Neutral , Strongly Disagree & Disagree Teachers, Total African American teachers Non African American teachers Teachers in the school (not preschool teachers) are generally supportive of the Prekindergarten literacy program. 90.6 90.0 95.0 7.8 10.0 5.0 1.6 0.0 0.0 Teachers, Total African American teachers Non African American teachers Teachers ate encouraged to communicate concerns, questions, and ideas regarding the Prekindergarten literacy program. 81.3 75.0 90.0 15.6 25.0 10.0 3.1 0.0 0.0 Teachers, Total African American teachers Non African American teachers The Prekindergarten literacy program is useful in monitoring progress of African American students. 90.6 95.0 92.5 7.8 5.0 7.5 1.6 0.0 0.0 Teachers, Total African American teachers Non African American teachers The Prekindergarten literacy program is useful as a screening tool for assessing at-risk African American students. 81.3 75.0 85.0 14.1 20.0 12.5 3.1 o:o 2.5 Teachers, Total African American teachers Non African American teachers With the Prekindergarten literacy program, children are excited about learning. Teachers, Total African American teachers Non African American teachers The Prekindergarten literacy program is valuable in preparing children for kindergarten. Teachers, Total African American teachers Non African American teachers Instructional elements of the Prekindergarten literacy programassessments, programs, materialsare based on scientifically-based reading research. Teachers, Total African American teachers Non African American teachers Because of the Prekindergarten literacy program, systematic and explicit instruction in phonemic awareness, oral language, vocabulary, concepts of print, and comprehension occurs daily in our schools classrooms. 82.8 80.0 87.5 10.9 10.0 10.0 4.7 5.0 2.5 98.4 100.0 100.0 96.9 90.0 100.0 92.2 90.0 92.5 0.0 0.0 0.0 1.6 5.0 0.0 4.7 0.0 7.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 Teachers, Total African American teachers Non African American teachers The literacy and language components are effective for reading readiness. Teachers, Total African American teachers Non African American teachers Group activities engage the students and complement instruction. Teachers, Total African American teachers Non African American teachers Learning centers engage the students and complement instruction. Teachers, Total African American teachers Non African American teachers 95.3 95.0 95.0 96.9 95.0 97.5 100.0 100.0 100.0 93.8 95.0 95.0 Note: Item percentages may not total 100% because of missing input from some respondents. 3.1 0.0 5.0 1.6 0.0 2.5 0.0 0.0 0.0 1.6 0.0 2.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.1 0.0 2.5 2005 -2006 Evaluation of the Prekindergarten Literacy Program 57 JTable 14. PREKINDERGARTEN TEACHER QUESTIONNIARE DEMOGRAPHIC INFORMATION 5 years or fewer 6-10 years 11-15 years 16-20 years How many years experience do you have as a Prekindergarten teacher? Teachers, Total African American teachers Non African American teachers 51.6 65.0 45.0 15.6 20.0 15.0 15.6 10.0 20.0 9.4 5.0 12.5 20 or more years 1.6 0.0 2.5 How many years experience do you have as an employee in any school? Tea
This project was supported in part by a Digitizing Hidden Special Collections and Archives project grant from The Andrew W. Mellon Foundation and Council on Library and Information Resoources.

<dcterms_creator>Center for Research in Educational Policy, University of Memphis, Memphis, Tennessee</dcterms_creator>