Data Hams Membership Corporal Punishment Suspenslen-Out of School Eipulsion } Mental Retardation Serious Emotional Disturbance Specific Learning Disability H$ Diploma 1 HS Certincete ot Attendance er Completion In Need et LEP i Enrolled In LEP G<ned and Talented AP Maihemalics I J AP Science American IndlanZAIasIcan Native Number % AsiarUPacIflc Islander Number % ! i Male Female Total Male Female Total Male Female Total Male Female Total Male Female Total Male Female Total Male Female Total Male Female Total Male Female Total Male Female Total Male * Female Total Male Female Total Maie Female Total Male Female Total 261.403 250.365 511.040 4.178 952 5.120 25.133 12.060 37,213 1.114 321 1.435 3.739 2.025 6.564 4,095 1,161 5,256 25,724 12.429 36.153 10 128 10.766 20.892 214 250 472 39.085 35.920 75.005 33.561 30.060 64.421 11.966 12890 24.656 743 793 1536 951" 1.012 1.963 057 0 54 1.11 1 14 0 26 1 40 0 79 0 36 1 17 1 26 0 37 1 64 0 59 0 45 1 04 096 0 27 1 23 0 93 0 4$ 1 38 042 0 44 006 0 40 0 49 0 09 1 20 1 10 2 30 1 17 108 2 25 0 42 0 45 0 07 0 24 026 OSO 031 0 33 0 65 967.626 909.064 1.678.690 545 . 141 686 45.637 15.148 60,765 1.538 302 1.840 6.846 5.112 11.958 3.884 1.235 4.919 29.132 12.651 41.783 58.826 58.460 115.266 984 979 1,963 247.466 209.657 457 123 223.080 169.212 412.292 90.541 96.706 167.247 19.406 Department of Education, Office for Civil Rights 1998 Elementary and Secondary School Civil Rights Compliance Report Projected Values for the Nation Hispanic Number % 2 to 3.540.501 199 3.356.212 4 00 6.904 713 0 IS 0 04 019 1 43 0 40 1 91 1 76 0 35 2 11 109 061 1 90 0 67 0 29 1 16 1 05 0 46 1 51 2 33 2 40 4 73 1 06 1 65 3 70 7 57 6 42 13 99 7 78 660 14 30 3 17 3 39 657 6 31 19.092 5.058 ____24i1^ 333.065 129 646 462.711 11 392 2.769 14.161 36.177 27.007 63.184 29.787 7 963 37.750 295.745 140.891 444,436 120.063 132.244 252.307 3 251 3.133 ____6 384 1.278.115 1.169.690 2.467813 1.109 860 1.036.566 2.146.446 117.954 128 239 246.193 10.353 10.445 6 00 ------10.403 37.853 19.496 18.705 38.201 1231 641 6 15 12 57 20.036 10.475 11.827 22 302 Black (non-Hlspsnic) Number % White (non-Hltpanlc) Number % Total Number % Disabled Number % Limited-Engllsh Pronclent 21-W%.TI(n >60%-Tlm Number % Number % Number % Number 7 71 3.990.703 730 3.007.526 15 01 7.606.229 0 69 14.037.124 0 45 13.907.401 5 23 1 39 682 1045 4 07 14 52 13 05 317 16 22 5 75 4 29 10 04 700 1 87 887 .10 67 537 16 04 4 92 5 42 10 35 6 13 5 91 12 04 39 11 36 40 75 51 38 72 36 16 2153. 4 14 4 50 0 63 3 37 341 6 77 3 45 3 89 7 34 99.358 36.165 135.523 600.299 359.923 1.040 222 19.710 7.314 27.024 124.536 03.292 207.629 86.794 25.600 114.602 342.575 169,372 511.947 140.056 178.280 316,316 0.913 0.551 17.464 40.088 35.480 75,566 35.990 32.204 68,194 104.717 134 999 239.716 9.123 12.192 21.315 7.884 11.493 19.377 17 14 20.024.605 3225 23.813.437 30 40 22.390.647 62 60 46.004,004 27 22 991 37 12 21 35 11 30 32 65 22 50 6 36 30 96 1960 13 24 33 04 20 86 606 26 92 12 36 6 11 10 40 574 7 23 1297 16 60 16 12 32 93 1.23 109 231 1,26 1 12 230 3 87 4 73 840 2 97 390 693 2 59 3 70 0 37 167.753 31,619 199.572 1.168.326 416.481 1,564.769 33.368 9.469 42.837 195,492 144.044 339.536 204.534 56 649 263,183 1.106,047 548,156 1.734,203 653.218 679.936 1,733.152 14,937 11.619 26,756 103.429 89247 192,670 93.794 01.259 175,053 1.050,065 1,095,279 2.154,144 110.741 107.287 226.008 110.875 111,490 222,165 45 95 8 72 54 67 36 67 13 07 49 75 3622 10 65 4907 31.08 22 90 S3 97 48 05 13.78 61 62 42 61 19 79 62 59 35 00 30 09 7109 28 16 22 20 5045 316 2 73 5.90 3 27 2 63 6 11 37 13 30 40 75 S3 38 01 34 66 73 49 3641 36 67 73 06 290.923 74.135 365,058 2.252.463 933.256 3,165,721 87,122 20.178 67,298 366.790 262.281 629.071 330.894 94.015 425,709 1.079.223 091.299 2.770.522 1.160.268 1.257.667 2.437,955 20.299 24.739 53.030 1,700.101 1.560.002 3.266,183 1,498.305 1.370,100 2.066.405 1.304.044 1.460.114 2.052,150 158,369 149.180 307.549 149.481 154.527 304,008 5133 48 67 100.00 79.69 20 31 100 00 70 70 29 30 100 00 76 89 23 11 100 00 56 31 41 69 100 00 77 73 22 27 100 00 67 83 32 17 100 00 48 41 51 59 100 00 53 36 46 64 100 00 52 27 47 73 100 00 52 20 47 60 10OO0 46 53 51 47 100 00 51 49 48 51 100 00 49 17 50 83 100 00 5.592.517 1.708.196 1.560,013 12 16 3.268,209 4.380 910 ____________5j27O__ 93.269 30.117 _________123.386 2.394 525 2.919 15.633 11.965 __________27,598 6.301 1.740 ___________SjO41_ 92,408 54.314 . 146,722 3,71 339 7.10 1.19 025 1.44 293 095 387 2 74 060 334 249 1 90 4,39 1 48 041 1 69 334 1 96 87.397 47.666 115,065 96.684 30.670 127.360 620.610 396.443 Use these numbers with caution due to large stalislical uncertainty In the estimate because the standard For eiample, if the eslimale Is 100 students and the standard error Is 28, the estimate will be flagged Non-response rate it more than 10% but less than 20%. Non-response rata is mere than 20% but less than 30%. U Non-response rate Is more than 30%. 93.910 56.811 150.729 12.335 7.271 19,606 145,539 70.066 223,605 119.096 64.076 183,974 30.020 15.122 45,142 1.758 1.171 2.929 1.953 1.432 3,385 3 05 2 33 0 18 23 26 13 71 36 97 445 2 39 0 84 4.16 2 24 042 1 05 0 53 1 50 0 57 0 30 095 064 047 111 23.279 22.762 46,041 5.30 1,217,053 10.71 758 18.29 22 71 7,21 29,92 29 62 14 31 43,93 109.488 03.987 193,475 06.953 27.239 114,192 742.051 347.538 1,009,589 17 40 190.509 1335 131,409 30.76 321.990 2043 147.037 6 40 37.200 26 02 105,117 2678 315.795 12 54 146.006 3933 462,601 error ol the estimate eiceeds 25 percent of the estimate. 0 62 080 1 61 30 20 51 34 8 43 5 16 Pt>e I i sExhibit No. 119: LI230-90 1998 Elementary & Secondary School Civil Rights Compliance Report Projected Values for the Nation-OCR 4 1 I. I !)5A ,? V- i '} % i 5 It S'/"/ Illi '.a i* V e9 'ft. Exhibit No. 120: LI230-90 1998 Elementary & Secondary School Civil Rights Compliance Report Projected Values for AR-OCR ^Illl s syl II I I ft 1 I 3 II Dat9 Items Membership Corporal Punishment Suspension-Out of School Expulsion Mental Retardation Serious Emotional Disturbance Specific Learning Disability HS Diploma HS Certificale ot Attendance or Completion In Need of LEP Enrolled in LEP Gifted and Talented AP Mathematics AP Science Male Female Total Male Female Total Male Female Total Male Female Total Male Female Total Male Female Total Male Female Total Male Female Total Male Female Total Male Female Total Male . Female Total Male Female Total Male Female Total Male Female Total American Indian/Alaskan Native Number 1,124 979 2.103 50 . 2 52 44 12 56 5 0 5 18 12 30 0 2 2 48 21 69 43 35 78 0 0 0 10 2 12 8 9 62 73 135 2 6 2 3 % 0 25 0 22 047 0 12 000 0 13 0 14 0 04 0 16 0 58 000 0 58 0 15 010 0 25 000 041 041 0 23 0 10 0 33 0 16 0 13 0 29 000 000 000 0 13 0 03 0 IS 0 11 001 013 012 0 15 0 27 0 08 015 0 23 0 03 006 009 AslanZPaciftc Islander Number 1.723 1.707 3,430 36 45 128 34 162 6 0 6 22 % Department of Education, Office for Civil Rights 1998 Elementary and Secondary School Civil Rights Compliance Report Projected Values for the State of Arkansas Hispanic Number % Black (non-Hispanic) Number % White (non-Hispanic) Number % Total Number % Disabled Number % LimHed-Engilsh Proficient Number % <21%-Time Number % 2V6O%-TimB Number % >60%-Time Number *_l 0 39 0 39 0 77 0 09 0 02 0 11 041 oil 0 52 0 69 000 069 009 009 0 18 1 021 0 000 21 6 27 99 128 225 5 11 16 459 435 694 436 424 660 265 264 529 33 39 72 39 37 76 14 0 21 0 10 0 03 0 13 0 37 047 0 84 1 69 3 73 5 42 5 75 5 44 11 19 6 07 590 11 97 0 53 0 53 1 08 1 27 1 50 2 77 1 17 1.11 2 26 6 485 5.853 12 338 462 78 538 389 69 478 25 0 25 64 37 __221_ 8 5 13 255 134 __389_ 160 142 302 5 6 13 3.577 3 194 8.771 3.244 2.872 8 116 354 . 371 . 725 6 13 10 7 17 1 46 1 32 2 78 1 13 0 19 1 32 1 25 0 29 1 54 2 89 000 2 69 0 69 030 099 1 66 1 04 2 69 1 20 0 63 1 84 080 0 53 1 12 1 69 2 71 4 41 44 77 39 98 64 75 45 16 39 98 85 13 071 0 74 1 45 0 23 0 27 050 030 0 21 0 51 50.905 50,158 101.061 7,350 3,070 10.420 8.505 4.079 12,584 300 77 377 3.235 2.111 5,346 11 49 11 32 2280 1801 7 52 25 53 27 38 13 13 40 50 34 64 8 89 43 53 28 55 17 32 43 67 166.672 157.373 324.245 24.937 4,819 29,756 13.681 4.108 17.789 358 94 452 3.900 2.767 6.667 37 65 35 51 73 16 61 10 11 81 72 91 44 04 13 22 57 26 41 34 10 85 52 19 32 01 22 71 54 71 84 4 17 39 32 4 6 63 116 4 24 02 3,278 1.372 4,650 2,586 3,273 5.859 19 15 34 63 49 112 31 31 ____6^ 4.694 5,673 10,387 248 336 586 282 445 727 1547 8 47 21 94 9 63 12 19 21 62 6 44 508 11.53 0 79 061 1 40 0 43 043 088 941 11 36 20 79 9 53 1299 2251 8 47 13 37 2184 Use these numbers wHh caution due to large atatlstical uncertainty in the estimate because the 247 4 51 14 102 4 21 12 349 4 72 26 227,110 218.067 443,177 32.636 7.975 40.611 22.747 8,321 31.086 695 171 668 7.246 4.937 12,165 341 142 5125 48 75 100 00 80 46 19 54 100 00 73 22 26 78 100 00 60 25 19 75 100 00 59 48 40 52 100 00 70 60 29 40 50,154 11 32 4,229 3.782 7,991 220 47 267 179 40 219 8 5 13 0 95 0 85 1 80 054 012 0.65 058 013 0.70 092 0 58 1 50 107 b 0 88 11.328 4 733 16,061 10.078 10.312 20.390 111 120 231 118 82 200 73 64 137 18 146 19.960 38.106 973 954 1.927 1.257 1,250 2.507 53 45 22 33 75 78 37 53 38 40 7593 37 63 4066 7831 1 48 103 250 102 0 69 191 36 39 4003 76 42 37 38 36 65 7403 37 77 37 56 75 33 483 4 10000 14.930 6.265 21,195 12.965 13.867 26.852 141 154 295 4,227 3,782 7,969 3,792 3,392 7,184 23,521 26,341 49,882 1.281 1.342 2403 1,589 1.739 3,326 70 44 29 56 100 00 46 26 51 72 100 00 47 80 52 20 10000 52 91 47 09 10000 52 78 47 22 1O0O0 47 17 52 83 100 00 48 44 51 58 10000 47 75 52 25 10000 64 8 171 9 8 0 8 9 8 163 8 84 8 247 8 0 53 1 40 1 86 000 168 0 77 0 40 1 17 1.185 758 1,943 9 73 6 22 1595 3.902 2.816 6,720 81 4 1877 40 4 6 28 121 4 25 05 6.023 2.855 8.678 2842 12 53 40,94 32 02 23 13 55.15 120 4 24 84 66 4 1368 188 4 36 51 2.192 1.372 3.564 1799 11 26 29 25 149 4 30.85 38 7 87 953 473 1.428 31 21 52 189 122 311 184 115 299 482 344 826 38 18 56 9 23 3 55 1 78 5 31 10 51 7 12 17 03 2 37 1 53 3 89 256 1 60 4 16 0 97 0 69 166 1 e6 0 69 2 15 0 42 0 27 089 283 275 538 0 53 0 55 106 FAra.amnU u Ik. i. J Z----------------------........u.e standard error of the estimate exceeds 25 percent of the estimate. For example, if (he estimate is 100 students and the standard error is 28. (he es(lma(e will be flaooed la Ik... >M< u..* 1________ 4 Non-response rale is more than 10% but less than 20%, Non-response rate is more then 20% but less than 30%. O Non-response rale Is mors (han 30%. 7.962 3.207 11,169 37 66 15 13 52 79 187 4 38 72 965 407 1.392 4 65 1 92 6 57 PageExhibit No. 121: Year 2 Evaluation The Effectiveness of the PreK-2 Literacy Program in the LRSD 1999-2000 and 2000-01. LI230-9 IT. I 5 I 1 I) I 5H, f -Ul, (I ill II I ll Year 2 Evaluation: The Effectiveness of the I PreK-2 Literacy Program in the Little Rock School District 1999-2000 and 2000-2001 Executive Summary Presented to the Board of Education Little Rock School District October 2001 Prepared by Dr. Bonnie A. Lesley Dr. Ed Williams Patricia Price Pat Busbea Ann Freeman Ken Savage Anita Gilliam Sharon KiilsgaardYear 2 Evaluation: The Effectiveness of the PreK-2 Literacy Program in the Little Rock School District 1999-2000 and 2000-2001 Executive Summary This report is one in a series of at least seven reports to the Board of Education over the past three years relating to the improvement of student achievement in PreK-2 literacy. The study is organized into ten sections, as follows: 1. 2. 3. 4. 5. 6. 7. 8. 9. Section I includes the Introduction, as well as a delineation of the Research Questions for the study and a description of the methodologies employed. Section II provides background information on the program design and its relationship to the Strategic Plan and the Revised Desegregation and Education Plan. Section III describes the selection of appropriate assessments for grades K-2 and the processes by which readiness standards were established for each grade level for the Developmental Reading Assessment. It also includes information on national and local validation studies of the Observation Study and the Developmental Reading Assessment, as compared to the Achievement Level Test. The literacy plans design in relationship to the findings in national research studies on early literacy is described in Section IV. This section also includes an alignment of the research with the assessments selected by the District. Three major sections on data analysis follow. Section V is a description of each of the tables that was constructed from the data reports to assist the writers of this report and its readers in analyzing the results on the eight measurements: the five sub-tests on the Observation Survey (OS) the Developmental Reading Assessment (DRA) and the reading and language usage sub-tests of the Achievement Level Tests (ALTs). Section VI is a detailed analysis of the data in each table and a comparison of 1999-2000 and 2000-01 data, by race. Additional data are provided in Section VII on the achievement gap among schools and on some statistical studies that were conducted relating to program effectiveness and the relationship between teacher participation in professional development and the achievement of their students. Following the data analysis is Section VIII that summarizes the program strengths and weaknesses and specifies the implications for instruction, with specific recommendations for improvements in 2001-02. Section IX is the Bibliography for the study. 10. Section X includes 87 tables of school-level data. Those interested in individual school performance or comparisons are encouraged to use the model in this report for data analysis at the District level to conduct similar analyses at the school level. 1The appendices A-E include copies of previous evaluation reports on the PreK-2 literacy program. In addition, a draft of a PreK-2 study was presented to the Board of Education for information in summer 2000 by the Plarming, Research, and Evaluation Department. The complete text of Section VIII, which includes the analysis of all available data and a set of recommendations for improvement in 2001-02, follows: VIII. Program Evaluation Findings and Recommendations for Improvement The research questions posed for this program evaluation are as follows: 1. Are the new curriculum standards/benchmarks, instructional strategies, and materials effective in teaching primary grade students how to read independently and understand words on a page? (See Section 5.2.1a of the Revised Desegregation and Education Plan and Strategy 2 of the Strategic Plan.) 2. Is the new program effective in improving and remediating the academic achievement of African American students? (See Section 2.7 of the Revised Desegregation and Education Plan.) 3. 4. 5. 6. Is there a relationship between teacher participation in professional development and student achievement? (See Policy IL expectation to examine cost effectiveness and Strategy 7 of the Strategic Plan.) Is there evidence of success in each of the four literacy models in use Early Literacy Learning in Arkansas (ELLA) only ELLA and Reading Recovery Success for All and Direct Instruction? (See Section 2.7 of the Revised Desegregation and Education Plan.) What are the programs strengths and weaknesses? (See Policy IL.) Is the program cost effective? (See Policy IL and Strategy 3 of the Strategic Plan.) This section will provide answers to each of the six research questions, along with summaries of supporting data. 1. Are the new curriculum standards/benchmarks, instructional strategies, and materials effective in teaching primary grade students how to read independently and understand words on a page? (See Section 5.2.1a of the Revised Desegregation and Education Plan and Strategy 2 of the Strategic Plan.) When the program design committee published their plan for the PreK-3 Literacy Program, they included a section on Effectiveness (see pp. 12-14). Using the available 2data at that time, the 1997-98 and 1998-99 Grade 4 Literacy Benchmark results and the fall 1998 grade 3 SAT9 results, they noted that only approximately 30 percent of the students were achieving at an acceptable level. Another concern at that time was the achievement gaps that were evident. The committee wrote, These gaps are, of course, unacceptable and are indicators that current practice is not effective" (p. 12). They also wrote, ... far too few students are becoming good readers by grade 3 (p. 13). To determine effectiveness of the new program components, the percent of students meeting the performance standard (readiness) on the Developmental Reading Assessment (DRA) was used since this measurement most closely approximates the type of measurement used on the grade 4 Literacy Benchmark. In summary, the standard for kindergarten is Level 2, for grade 1 it is Level 16, and for grade 2 it is Level 24. The percent readiness is the percent of students at each grade level who met the standard. (See Section III for a description of the DRA and the process used to establish performance levels or standards.) Tables 22 and 23 in Section V provide the data that are summarized here. The reader may also wish to review Tables 75-83 in Section X for school-level data. A summary of those results follows: Summary of K-2 Percent Readiness Data on the Developmental Reading Assessment All three grade levels improved in spring 2001. Grade 1 showed the greatest improvement. Grade Level 1999-2000 2000-2001 Change Kindergarten Grade 1 Grade 2 72.2 53.6 67.5 80.7 63.8 75.4 +8.5 +10.2 +7.9 More than 75 percent of the schools improved in spring 2001. Grade Level # Impr. % Impr. Kindergarten Grade I Grade 2 n 29 29 77% 83% 83% Both area and magnet schools did well. The five highest performing schools at each grade level for each year follow. McDermott Elementary was among the top five schools in 2000-2001 at all three grade levels. 3Grade Level 1999-2000 2000-2001 Kindergarten Grade 1 Grade 2 McDermott* Fulbright* Rightsell* Terry * Mitchell* Woodruff* Williams Wilson* Western Hills* McDermott* Rightsell* Williams Forest Park* Western Hills* Otter Creek* McDermott* Baseline* Fulbright* Gibbs Brady* Williams Carver McDermott* Booker Forest Park* Carver Williams Western Hills* Otter Creek* McDermott* Denotes area schools. Many schools improved dramatically in spring 2001. Schools improving 20 or more points were as follows. All are area schools, and most are high poverty schools. Stephens Elementary and Badgett Elementary were among the most improved schools at all three grade levels in 2000-2001. Grade Level 2000-01 Kindergarten Grade 1 Grade 2 Baseline (42.9)* Badgett (28.4)* Forest Park (27.1) Cloverdale (26.1)* Stephens/Garland (25J)* Wakefield (44.6)* Watson (41.9)* Baseline (41.2)* Stephens/Garland (27.5)* Western Hills (25.8) Chicot (24.4)* Badgett (20.6)*_________ Dodd (31.1) Badgett (31.1)* Stephens/Garland (30.1)* Pulaski Heights (29.3) McDermott (22.5) Denotes schools with 75% or higher eligible for free/reduced lunch. The Incentive Schools, in general, improved. With the exception of Mitchell and Rightsell at grade 1, a majority of the students performed at or above the readiness level, although significant gaps between the lowest and highest-performing schools are a concern. Franklin, Rockefeller, and Stephens had more than 50 percent of their students at or above readiness level at all three grade levels. 4Kindergarten School 1999-2000 2000-2001 Change Franklin Mitchell Rightsell Rockefeller Stephens 64.3 90.6 92.1 75.8 40.8 58.6 92.3 80.5 76.2 66.1 -5.7 1.7 -11.6 0.4 25.3 Grade 1 School 1999-2000 2000-2001 Change Franklin Mitchell Rightsell Rockefeller Stephens 57.6 25.0 35.7 76.3 23.5 58.9 25.0 41.7 65.2 51.0 1.3 0.0 6.0 -11.1 27.5 Grade 2 School 1999-2000 2000-2001 Change Franklin Mitchell Rightsell Rockefeller Stephens 81.2 48.6 94.7 71.4 31.3 83.6 50.0 70.5 84.2 61.4 2.4 1.4 -24.2 12.8 30.1 The Newcomer Centers improved, except for Terry at kindergarten and grade 2. All five schools had more than a majority of their students performing at or above the readiness level at the kindergarten level, all except one at grade 1, and all at grade 2. Second-language students scores are included in these results. Kindergarten School 1999-2000 2000-2001 Change Brady Chicot Romine Terry_____ Washington 76.9 56.1 66.7 91.9 81.2 93.4 70.9 86.4 86.7 84.1 16.5 14.8 19.7 -5.2 2.9 Grade 1 School 1999-2000 2000-2001 Change Brady Chicot Romine Tea_____ Washington 34.9 26.8 59.6 47.1 35.5 53.5 51.2 76.5 59.8 41.1 18.6 24.4 16.9 12.7 5.6 Gradel School 1999-2000 2000-2001 f haiigc Brady Chicot Romine Terry_____ Washington 70.8 38.6 68.8 81.2 63.3 79.6 52.1 81.6 67.1 81.4 8.8 13.5 12.8 -14.1 18.1 5 j iThere are seventeen (49 percent) elementary schools in the District where 75 percent or more of the students are eligible for ffee/reduced lunch. Many of these schools improved dramatically in spring 2001 and/or some performed in the highest range of scores (80 percent or higher). Kindergarten was the strongest performing grade with six of the 17 schools at 80 percent or higher. Sixteen of the 17 schools (94 percent) had the majority of their students at or above the readiness level at kindergarten, 11 of 17 ( 65 percent) at grade 1, and 15 of 17 (88 percent) at grade 2. First grade was the lowest of the three grades tested, but where the most growth occurred. Six schools improved 20 points or more. Kindergarten School 1999-2000 2000-2001 Change Badgett (94%) Franklin (90%) Stephens (90%) Chicot (87%)______ Baseline (86%) Woodruff (86%) Cloverdale (85%) Wilson (85%) Mabelvale (85%) Mitchell (84%) Watson (83%) Geyer Springs (83%) Rightsell (82%) Meadowcliff (81%) Wakefield (80%) Fair Park (78%) 21.6 64.3 40.8 56.1 51.1 69.2 56.4 66.7 61.0 90.6 56.4 85.1 92.1 77.4 46.8 68.3 so.o S8.6 66.1 70.9 94.0 46.2 82.5 80.0 73.3 92J 73.7 87.7 80.5 77.1 61.1 75.6 28.4 -5.7 25.3 14.8 42.9 -23.0 26.1 13.3 12.3 1.7 17.3 2.6 -11.6 -0.3 14.3 7.3 Grade 1 School 1999-2000 2000-2001 Change Badgett (94%) Franklin (90%)_____ Stephens (90%) Chicot (87%) Baseline (86%) Woodruff (86%) Cloverdale (85%) Wilson (85%) Mabelvale (85%) Mitchell (84%) Watson (83%)_____ Geyer Springs (83%) Rightsell (82%) Meadowcliff (81%) Wakefield (80%) Fair Park (78%) 5.9 57.6 23.5 26.8 29.6 84.2 28.4 82.9 50.8 25.0 24.7 46.8 35.7 70.0 22.0 62.5 26.5 58.9 51.0 51.2 70.8 61.5 33.9 53.8 60.5 25.0 66.6 38.6 41.7 66.6 66.6 72.7 20.6 1.3 27.5 24.4 41.2 -22.7 5.5 -29.1 9.7 0.0 41.9 -8.2 6.0 -3.4 44.6 10.2 6Grade 2 School 1999-2000 2000-2001 Change Badgett (94%) Franklin (90%) Stephens (90%) Chicot (87%)______ Baseline (86%) Woodruff (86%) Cloverdale (85%) Wilson (85%) Mabelvale (85%) Mitchell (84%) Watson (83%)______ Geyer Springs (83%) Rightsell (82%) Meadowcliff (81%) Wakefield (80%) Fair Park (78%) 11.8 81.2 31.3 38.6 47.1 78.3 57.9 60.4 43.4 48.6 54.4 72.5 94.7 57.9 40.0 62.9 42.9 83.6 61.4 52.1 60.5 86.5 45.1 61.4 63.0 50.0 51.2 66.0 70.5 75.0 54.4 67.7 31.1 2.4 30.1 13.5 13.4 8.2 -12.8 1.0 19.6 1.4 -3.2 -6.5 -24.2 17.1 14.4 4.8 Five of the seven Success for All (SFA) schools improved at the kindergarten and grade 1 levels in spring 2001, and six of the seven improved at grade 2. Some are now enriching the SFA program with ELLA strategies. Kindergarten had four schools in 2000-01 with scores at 80 percent or above, and grade 2 had two. All except one school had the majority of their students performing at or above readiness level at kindergarten, grade 1, and grade 2. Baseline posted high levels of growth at all three grade levels, especially kindergarten and grade 1. Kindergarten School 1999-2000 2000-2001 Change Brady Baseline Cloverdale Fair Park Meadowcliff Romine Woodruff 76.9 51.1 56.4 68.3 77.4 66.7 69.2 93.4 94.0 82.5 75.6 77.1 86.4 46.2 16.5 42.9 26.1 7.3 -0.3 19.7 -23.0 Grade 1 School 1999-2000 2000-2001 Change Brady Baseline Cloverdale Fair Park Meadowcliff Romine Woodruff 34.9 29.6 28.4 62.5 70.0 59.6 84.2 53.5 70.8 33.9 72.7 66.6 76.5 61.5 18.6 41.2 5.5 10.2 -3.4 16.9 -22.7 7Grade 2 School 1999-2000 2000-2001 Brady Baseline Cloverdale Fair Park Meadowcliff Romine Woodruff 70.8 47.1 57.9 62.9 57.9 68.8 78.3 79.6 60.5 45.1 67.7 75.0 81.6 86.5 Change 8.8 13.4 -12.8 4.8 17.1 12.8 8.2 Schools using both the ELLA strategies in grades K-2 and the Reading Recovery program in grade 1 performed well, especially in grade 2 evidence of the power of the grade 1 intervention. Eleven of the 12 schools (92 percent) had more than half of their students performing at or above the readiness level at grade 1, and all of them were over the 50 percent mark at grade 2. At grade 1, only two of the 12 schools (17 percent) had at least 80 percent of their students at or above readiness, and at grade 2 there were eight (67 percent). Grade 1 School 1999-2000 2000-2001 Change Booker Chicot* Dodd Franklin Fulbright** Geyer Springs Gibbs Jefferson** Otter Creek Pulaski Heights' Williams Wilson* 69.3 26.8 58.3 57.6 61.0 46.8 65.9 69.1 67.7 50.0 84.1 82.9 87.4 51.2 73.5 58.9 66.6 38.6 71.4 73.9 69.6 61.7 97.1 53.8 18.1 24.4 15.2 1.3 5.6 -8.2 5.5 4.8 1.9 11.7 13.0 -29.1 *2000-2001 was a training year. **Reading Recovery not continued in 2000-2001 due to loss of Title I funding. Grade 2 School 1999-2000 2000-2001 Change Booker Chicot* Dodd Franklin Fulbright' Geyer Springs Gibbs Jefferson** Otter Creek Pulaski Heights' Williams Wilson* 79.8 38.6 51.7 81.2 79.3 72.5 80.5 71.1 87.2 45.2 89.7 60.4 81.4 S2.1 82.8 83.6 88.7 66.0 82.9 85.0 90.5 74.5 92.6 61.4 1.6 13.5 31.1 2.4 9.4 -6.5 2.4 13.9 3.3 29.3 2.9 1.0 *2000-2001 was a training year. **Reading Recovery not continued in 2000-2001 due to loss of Title 1 funding. 8The number of schools with a majority of students performing below the readiness level declined, with the most improvement at grade 1. Grade Level 1999-2000 2000-2001 Kindergarten Grade 1 Grade 2 3 (9%) 13(37%) 8 (23%) 1 (3%) 6 (17%) 2 (6%) The number of schools with at least 80 percent of the students performing at the readiness level increased. Kindergarten was the strongest grade. Grade Level 1999-2000 2000-2001 Kindergarten Grade 1 Grade 2 15 (43%) 5(14%) 10(29%) 21 (60%) 5 (14%) 18(51%) The gap between the lowest and highest performing schools decreased. Grade Level 1999-2000 2000-2001 Change Kindergarten Grade 1 Grade 2 73.2 78.3 82.9 48.1 72.1 50.2 -25.1 -6.2 -32.7 Black students improved at every grade level and at a higher rate than non-black students. Grade Level 1999-2000 2000-2001 Change Kindergarten Grade 1 Grade 2 69.3 48.3 63.8 77.0 57.4 69.8 7.7 9.1 6.0 Non-black students also improved at every grade level. Grade Level 1999-2000 2000-2001 Change Kindergarten Grade 1 Grade 2 84.7 71.2 81.6 88.8 77.3 86.8 4.1 6.1 5.2 The achievement gap is much lower in grades K-2 now than in higher grade levels and is decreasing. It is lowest at kindergarten and highest at grade 1. Grade Level 1999-2000 2000-2001 Change Kindergarten Grade 1 Grade 2 15.4 22.9 17.8 11.8 19.9 17.0 -3.6 -3.0 -0.8 All of the data just cited are evidence that reading achievement improved in grades K-2 since fall 1999 when the new curriculum standards^enchmarks were implemented, along with new instructional strategies, materials, and assessments. In general, in some schools 9to a greater extent than others, but at all schools to some extent, students are performing at a higher level than they were in the pastboth black and non-black. So what can the District expect as these children move into higher grade levels? As discussed in Section III, it is too early to determine whether performance at the readiness level on the Developmental Reading Assessment (DRA) at grade 2 predicts proficient performance (or above) on the grade 4 Literacy Benchmark test administered by the state. The students who began the new curriculum in kindergarten in fall 1999 will not take the grade 4 Benchmark examination until spring 2004. If readiness on the DRA in grade 2 proves to predict only the basic level of performance (rather than proficient) in grade 4, then the new program would be expected to produce results that exceed the performance in 1997-9858 percent performing at or above the basic level. If readiness on the DRA in grade 2 predicts the proficient level of performance in grade 4, then the new program would be expected to produce results higher than 30 percent (the percent performing at or above the proficient level in 1997-98). Continued improvement on the grade 4 Literacy Benchmark test is anticipated since each year more teachers complete more training, and more students have more experience in the new program. It is important to remember, however, that test scores reflecting the performance of different groups of students each year do not necessarily go up every year- !ven with high levels of implementation. There will be some variances due to a number of different influences. In spring 2004, when the 1999-2000 kindergarten students will take the grade 4 Literacy Benchmark test, the District can more accurately calculate how performance on the grade 2 DRA predicts achievement on the grade 4 Literacy Benchmark. At that time, more accurate pronouncements of the effectiveness of the early literacy program can be determined. 2. Is the new program effective in improving and remediating the academic achievement of African American students? (See Section 2.7 of the Revised Desegregation and Education Plan.) As discussed in Section II, the District chose to emphasize the prevention of failure to every extent possible in its efforts to improve and remediate the achievement of African American students. The philosophy behind the Districts instruction program for early literacy development mirrors that of the International Reading Association in their position paper entitled Making a Difference Means Making It Different: Honoring Childrens Rights to Excellent Reading Instruction. The first principle in that document is that Children have a right to appropriate early reading instruction based on their individual needs (3). The position is that No single method or single combination of methods can successfully teach a child to read. Instead, each child must be helped to develop the skills and understandings he or she needs to become a reader (3). A major challenge to District staff was to design a total program that would take into account that 10Children of poverty are more likely than others to enter school without the knowledge and background necessary for learning to read and wnte. They are more likely to fail at school tasks, thus decreasing motivation. Reducing the impact of poverty will require instruction that is sensitive to the childrens own knowledge and background and consistent in supporting children as individual learners each with his or her own set of strengths and needs (13-14). The program design had to be one that would allow every child, regardless of prior knowledge and skill, to grow, to develop increasingly more complex knowledge and skills, and to be challenged and successfuland, at the same time, to accelerate the growth of children from poverty so that the gap narrowed over time. To determine the effectiveness of the new program in improving and remediating the academic achievement of African American students, the District used the performance results of the Observation Survey and the Developmental Reading Assessment. The basic criterion established in determining the program effectiveness for black students was that black student achievement would have to improve and then that growth over the two-year period of the programs implementation would need to be equal to, but preferably greater than, the growth of non-black students. A caution in interpreting the scores of some of the measures is that four of the five subtests on the Observation Survey have relatively low ceilingsfor at least two reasons. (See Section III for a description of the tests and the maximum scores for each.) The first is that they are designed so that all children can experience some measure of success in the assessment process, and the second is that they measure leaming-to-read skills, not higher-level comprehension skills foimd in many reading assessments for older children. For example, there are only 54 items on the Letter Identification test, which measures whether a child can recognize upper and lower-case letters. There are only 20 words to decode on the Word Test, and those words are the most frequently occurring words in the basal reader. There are only 24 questions on the Concepts about Print test, all relatively simple. And there are only 37 points that can be earned on the K-1 Hearing and Recording Sounds test and 64 points at grade 2. In most of the sub-tests the greatest growth for both black and non-black students occurs at the kindergarten level where the ceiling is less of a factor. One of the Observation Survey sub-tests does not have a ceilingWriting Vocabulary, where the child is asked to write, within ten minutes, all the words he/she can write. And, as one might expect, children who come to school with rich literacy experiences in their backgrounds do better on this test than those who come from poverty backgrounds. Also, students who come to school with substantial vocabularies have more to build upon, and they learn how to write the words they already know and how to wnte new words at a faster rate than those coming in with limited vocabulary. In fall 1999 the black kindergarten scores were only 62 percent of those of non-black students. The data will show, however, that even though this test concludes with a higher gap than the other sub-tests of the Observation Survey, the gap is considerably more narrow at the end of 11 grade 2 than it was in fail of kindergarten, and black students grew a great deal on this measurement over a two-year period. The Developmental Reading Assessment (DRA) has a maximum of 44 levels. Since this test was constructed to be used through grade 3 or even beyond for students who are still learning to read, the ceiling effect is not evident at the end of grade 2 in the LRSD. The DRA is the most difficult of the measurements used in K-1 since it does test comprehension and more complex cognitive operations. The Achievement Level Test at grade 2 has far less of a ceiling than any of the measurements used in K-2. Staff reviewed the data from various perspectives to determine program effectiveness, specifically for Afiican American students. They looked, for instance, for evidence that black students growth in the leaming-to-read skills was progressing at the same or higher amount than non-black students growth and that the percentage (ratio) of black performance to non-black performance was increasing from the fall pre-test to the spring post-test and from grade-level to grade-level. Scores were examined as well to determine the amount of summer regression and to determine the growth of two-year cohorts (kindergarten in 1999-2000 and grade 1 in 2000-01 and then grade 1 in 1999-2000 and grade 2 in 2000-01). Comparisons of black and non-black performance for each test administration and each grade level were made of the average scores on each sub-test, the percent of those averages of the maximum scores, and the percent of students achieving the readiness standard for each grade level. These detailed analyses for each sub-test are found in Section VI of this program evaluation, with references to the tables in Section V. The following findings based on Observation Survey, Developmental Reading Assessment, and Achievement Level Test results make it possible to conclude that the new early literacy program has so far been effective in improving and remediating the reading achievement of African American students, as well as all students. It is unusual in any District to find gains by both blacks and non-blacks over a two-year period on eight different measurements, as this study finds. Again, however, experts on program implementation advise that it takes approximately five years to determine program effectiveness, so this year 2 study at best establishes baseline and early trend data for comparisons in future years. Also, the Year 2 study does not include an examination of the variance in learning opportunities for students from school to school or an examination of the different forms of interventions. It is a look at the total program, including all its components. Letter Identification The gap was almost closed on Letter Identification by the end of kindergarten in 2000-01, and it was totally closed at the end of grade 1. 'The percentages in the following table represent the ratio of black to non-black scores for each testing administration. 12Letter IdentificationBlack to Non-Black Ratios Grade Level Kall 1999 Kindertarten Grade 1 Spring 2000 Fall 2000 Spring 2001 81% 96% 96% 100% 83% 99% 97% 100% i Black growth exceeded that of non-black students at both kindergarten and grade 1 in both 1999-2000 and 2000-01. The Letter Identification test is not given in grade 2. Letter IdentificationGrowth/Improvement Grade Level 1999-00 Growth 2000-01 Growth KinderBlack KinderNon-B Difference Grade 1Black Grade 1Non-B Difference 20.89 16.22 4.67 5.36 3.42 1.94 21.95 18.04 3.91 4.06 3.42 0.64 The ratio of the number of points of growth for black students as compared to non-black students over a two-year period (kindergarten in 1999-2000 and grade 1 in 2000-01) was 134 percentthe highest growth ratio on any of the sub-tests for this cohort. (See Table 7.) Ratios over 100 percent indicate that black students grew more points than non-black students. Letter IdentificationGrowth Ratios of Black to Non-Black Grade Level 1999-2000 2000-2001 Kinderearten Grade 1 Two-Year Cohort 129% 157% 122% 119% 134% Black students experienced no regression during the summer between kindergarten and grade 1 on this measurement, although non-black students did. Letter IdentificationSummer Regression Race Spring 2000K Fall 2000Gr. I Diff. Black Non-Black 48.48 50.30 48.95 49.66 +0.47 -0.64 In fall 1999 black kindergarten students achieved 51 percent of the maximum score of 54 on this measurement, as compared to 63 percent for non-black students. By the end of grade 1 in 2000-01, the black students scored 98 percent of the maximum, as did the non-black students. 13Letter IdentificationPercent of Maximum Score Grade Level Fall 1999 Spring 2000 Fall 2000 Spring 2001 KinderBlack Kinder-Non-B Grade 1Black Grade 1Non-B 51% 63% 88% 92% 90% 93% 98% 98% 51% 61% 91% 92% 91% 95% 98% 98% Word Test Although black students began kindergarten in fall 1999 scoring at only 57 percent of non-blacks, by the end of grade 2 in 2000-01 the black scores were 96 percent of non-black scoresvirtually closing the gap on this measurement in the second year of program implementation. This test includes only 20 of the most frequently occurring words, so it measures a very limited amount of decoding achievement. Word TestBlack to Non-BIack Ratios Grade Level Fall 1999 Spring 2000 Fall 2000 KinderEarten Grade 1 Grade 2 Spring 2001 57% 73% 89% 76% 92% 96% 83% 68% 91% 97% 94% 96% Even though black students did not grow as many points as non-blacks in kindergarten, they grew more than non-blacks in both grade 1 and grade 2 for both years, 1999-2000 and 2000-01. They also grew at a faster pace in kindergarten on this measurement than they did in either grade 1 or grade 2. Neither black nor non-black students grew much on this measurement in grade 2 since both groups were close to maximizing their scores by the end of grade 1. The larger gro3vth in the second year of the program at the kindergarten level may be an indication of teachers being more explicit in developing these necessary skills. The growth ratios indicate the more rapid improvement of black student achievement in grades 1 and 2. The higher growth ratios for black students may also reflect that non-black students had come close to maximizing the total possible score on this low-ceiling measurement. Word TestGrowth/Improvement Grade Level 1999-00 Growth 2000-01 Growth Kinder-Black KinderNon-B Difference Growth Ratio Grade 1Black Grade 1Non-B Difference Growth Ratio Grade 2Black Grade 2Non-B Difference Growth Ratio 9.58 16.22 -6.64 81% 11.12 10.45 0.67 106% 2.82 1.73 1.09 163% 12.03 13.73 -1.70 88% 11.52 10.04 1.48 115% 2.06 1.31 0.75 157% 14Black students grew more than non-black students during the two-year period of 1999-2000 kindergarten and 2000-2001 grade 1 (cohort 1) and during the 1999- 2000 grade 1 and 2000-2001 grade 2 (cohort 2). Cohort Growth Cohort I Cohort 2 Black Non-Black Difference Ratio 15.58 15.48 0.10 101% 12.31 11.02 1.29 112% Both black and non-black students regressed in their performance on this measurement during the summer between kindergarten and grade 1. There was slight regression for both groups between grade 1 and grade 2, but not nearly as much as the previous summer. In both cases, black students regressed less than non-blacks on this measurement. Word TestSummer Regression Race Spring 2000 Kinder Fall 2000 Grade I Diff. Spring 2000 Grade I Fall 2000 Grade 2 Diff. Black Non-Black 11.33 14.91 5.81 8.49 -5.52 -6.42 16.87 18.34 16.00 17.60 -0.87 -0.74 1 i In fall 1999 the black kindergarten students scored 9 percent of the maximum score of 20 on this measurement, as compared to 15 percent for non-black students. By the end of grade 2 in 2000-01, after two years in the program, black students scored 90 percent, as compared to 95 percent for non-black students. Word TestPercent of Maximum Score Grade Level Fall 1999 Spring 2000 Fall 2000 Spring 2001 KinderBlack KinderNon-B Grade 1Black Grade 1Non-B Grade 2Black Grade 2Non-B 9% 15% 29% 39% 81% 90% 57% 75% 84% 92% 95% 99% 7% 13% 29% 42% 80% 88% 67% 82% 87% 93% 90% 95% Concepts about Print Black students started kindergarten in fall 1999 scoring at 69 percent of non-black students scores. By the end of grade 1 in 2000-01 the ratio was 93 percent almost closing the gap on this measurement. Concepts about PrintBlack to Non-Black Ratios Grade Level Fall 1999 Spring 2000 Fall 2000 Spring 2001 Kindergarten Grade 1 69% 88% 81% 93% 72% 84% 87% 93% 15Just as in the Word Test, black kindergarten students did not grow as many total points as did non-black students in either 1999-2000 or 2000-01, although the growth was almost the same in 2000-01. But their growth did exceed non-black growth in grade 1 for both years. The highest amount of growth on this measurement was in kindergarten for both blacks and non-blacks. Teachers were clearly more effective in teaching the skills measured on this sub-test the second year of implementation. This test is not administered in grade 2. Concepts about PrintGrowth/lmprovement Grade Level 1999-00 Growth 2000-01 Growth Kinder-Black KinderNon-B Difference Growth Ratios Grade IBlack Grade 1Non-B Difference Growth Ratios 7.76 8.06 -.30 96% 5.65 5.21 .44 108% 10.07 10.11 -.04 100% 6.25 5.11 1.14 122% Black students grew more than non-black students in the two-year period of 1999- 2000 kindergarten and 2000-2001 grade 1. Cohort Growth Cohort 1 Black Non-Black Difference Ratio 13.22 11.72 1.50 113% Both black and non-black students experienced a slight regression in their performance over the summer between kindergarten and grade 1 on this measurement. The non-black students regressed a little more than did the black students. Concepts about PrintSummer Regression Race Spring 2000K Fall 2000Gr. I Dill. Black Non-Black 14.30 17.56 13.51 16.11 -0.79 -1.45 In fall 1999 the black kindergarten students scored 27 percent of the maximum score of 24 on this measurement, as compared to 40 percent for non-black students. By the end of grade 1 in spring 2001,, after two years of instruction, the black students scored 82 percent of the maximum, as compared to 88 percent for non-black students. 16Concepts about PrintPercent of Maximum Score Fall 1999 Spring 2000 Fall 2000 Spring 2001 KinderBlack KinderNon-B Grade 1Black Grade 1Non-B 27% 40% 58% 65% 60% 73% 81% 87% 25% 35% 56% 67% 67% 77% 82% 88% Writing Vocabulary The gap at the end of grade 2 on this measurement is the highest of any of the gaps on the five sub-tests of the Observation Survey, but it is still much smaller than the gaps that have been in existence in the past, especially those on the SAT9. This sub-test is the only one of the Observation Survey sub-tests that is without a ceiling. Black kindergarten students started in fall 1999 performing at only 62 percent of non-black scores. At the end of grade 2 in 2000-01, the ratio had improved to 87 percentan improvement of 25 points. Writing VocabularyBlack to Non-Black Ratios Fall 1999 Spring 2000 Fall 2000 Spring 2001 Kindergarten Grade 1 Grade 2 62% 87% 95% 66% 84% 82% 58% 80% 84% 71% 88% 87% Black Student growth lagged behind non-black growth in terms of numbers of points at all three grade levels on this measurement. This finding reinforces what teachers know as the power of prior knowledge. Students entering school already knowing much of what is measured on Letter Identification, Concepts about Print, and even the Word Test begin to amass vocabulary and spelling skills immediately, while those who come without that knowledge must take the time to learn them before they can perform well on a measurement such as the Writing Vocabulary sub-test. As on other measures, there was considerably more growth for all students the second year of program implementation. Also, the difference between the growth of black and non-black students lessened in year 2 at both grade 1 and grade 2. 17Writing VocabuiaryGrowlh/Improvement Grade Level 1999-00 Growth 2000-01 Growth Kinder-Black KinderNon-B Difference Growth Ratios Grade 1Black Grade 1Non-B Difference Growth Ratios Grade 2Black Grade 2Non-B Difference Growth Ratios 11.57 17.43 -5.86 66% 23.57 28.39 -4.82 83% 15.18 24.08 -8.90 63% 16.86 23.06 -6.20 73% 27.22 29.29 -2.07 93% 25.96 28.54 -2.58 91% Black students grew at a slower pace than non-black students in both cohort 1 (1999-2000 kindergarten and 2000-2001 grade 1) and cohort 2 (1999-2000 grade 1 and 2000-2001 grade 2) on Writing Vocabulary. Also, the results of this subtest reveal a widened gap over the two-year period, even though the number of points of improvement for both blacks and non-blacks was the highest^perhaps due to the test not having a ceiling. Cohort Growth Cohort I Cohort 2 Black Non-Black Difference Ratio 37.23 40.74 -3.51 91% 42.22 48.32 -6.10 87% Black students experienced only a slight regression on this measurement between kindergarten and first grade, and the regression for non-black students was about three times as many points. Both groups also regressed on this measurement over the summer between grade 1 and grade 2. Again, non-black students regressed more than black students. Writing VocabularySummer Regression Race Spring 2000 Kinder Fall 2000 Grade I Diff. Spring 2000 Grade I Fall 2000 Grade 2 Diff. Black Non-Black 14.50 22.13 12.94 16.15 -1.56 -5.98 37.11 44.04 29.80 35.43 -7.31 -8.61 18 I iHearing and Recording Sounds The gap almost closed (91 percent) on this measurement by the end of grade 2 in 2000-01, even though the black kindergarten scores in fall 1999 were only 54 percent of those of non-black students. The 46 percent in fall 2000 was the lowest beginning ratio of all the sub-tests on the Observation Survey. Hearing and Recording SoundsBlack to Non-Black Ratios Fall 1999 Spring 2000 Fall 2000 Spring 2001 Kindergarten Grade 1 Grade 2 54% 78% 86% 70% 91% 88% 46% 74% 87% 76% 92% 91% Black students did not grow as many points on this measurement as non-black students during kindergarten of either 1999-2000 or 2000-01. However, their growth did exceed that of non-black students in grade 1. It lagged only slightly behind non-black students in grade 2 during 1999-2000, but exceeded the non- black students in 2000-01, again indicating the growing effectiveness of the program. As seen on other measurements, the greatest growth or improvement for both black and non-black students was at the kindergarten level. This measurement, like Writing Vocabulary, reveals the importance of prior knowledge. Children cannot write down a dictated sentence without knowing Letter Identification and Concepts about Printinformation that many had not mastered at the kindergarten level. Black student growth at grade 1 and grade 2 are evidence that they have mastered the prerequisite knowledge and skills for good performance on this sub-test. Hearing and Recording SoundsGrowtb/Improvement Grade Level 1999-00 Growth 2000-01 Growth Kinder-Black KinderNon-B Difference Growth Ratios Grade 1Black Grade 1Non-B Difference Growth Ratios Grade 2Black Grade 2Non-B Difference Growth Ratios 13.44 17.71 -4.27 76% 13.62 12.13 1.49 112% 8.18 8.21 -0.03 100% 17.43 21.03 -3.60 83% 14.21 10.85 3.36 131% 6.10 4.34 1.76 141% The black growth exceeded non-black growth for cohort 1 (1999-2000 kindergarten and 2000-2001 grade 1), but it lagged slightly behind non-black growth in cohort 2 (1999-2000 grade 1 and 2000-2001 grade 2). 19Cohort Growth Cohort I Cohort 2 Black Non-Black Difference Ratio 28.12 27.74 0.38 101% 34.35 34.80 -0.45 99% Black students did not regress over the summer between kindergarten and grade 1, although non-black students did on this measurement. Both black and on-black students gained multiple points over the summer between grade 1 and grade 2 on this measurement. Hearing and Recording SoundsSummer Regression Race Spring 2000 Kinder Fall 2000 Grade I Diff. Spring 2000 Grade I Fall 2000 Grade 2 Diff. Black Non-Black 17.02 24.37 17.49 23.55 4-0.47 -0.82 30.87 34.11 45.50 52.44 4-14.63 4-18.33 In fall 1999 black kindergarten students scored 10 percent of the maximum score of 37 on this measurement, as compared to 18 percent for non-black students. By the end of grade 1 in 2000-01 black scores had improved to 86 percent, as compared to non-black scores at 93 percent. At the end of grade 2, black students in 2000-01 scored 81 percent of the maximum score of 64, as compared to 89 percent for non-black students. Hearing and Recording SoundsPercent of Maximum Score Fall 1999 Spring 2000 Fall 2000 Spring 2001 KinderBlack Kinder^Non-B Grade 1Black Grade 1Non-B Grade 2Black Grade 2Non-B 10% 18% 47% 59% 66% 77% 46% 66% 83% 92% 79% 89% 6% 13% 47% 64% 71% 82% 53% 69% 86% 93% 81% 89% Developmental Reading Assessment This test assesses significantly more complex xmderstandings and skills than does any one of the sub-tests on the Observation Survey. The Observation Survey provides information on each students leaming-to-read skills, while the Developmental Reading Assessment tests recall and comprehension. In spite of that, however, LRSD students, both black and non-black, made significant improvements in their performance over the past two years. Black kindergarten students in fall 1999 were scoring at only 35 percent of the level of non-black studentsthe widest gap seen on any of these K-2 assessments. But by the end of grade 2 in 2000-01, after only two years in the program, the black to non-black ratio was 80 percent. The staff members consulted for this program evaluation are confident that once students have been in the program for three years that the end 20of grade 2 ratio will be closer to 90 percentvirtually closing the gap in ensuring that all students learn to read independently by the end of grade 3. Developmental Reading AssessmentBlack to Non-Black Ratios Grade Level Fall 1999 Kindergarten Grade 1 Grade 2 Spring 20QQ Fall 2000 Spring 2001 35% 64% 74% 43% 68% 78% 41% 47% 70% 48% 71% 80% Although black students did not grow as many points as non-black students in kindergarten or grade 1 for both years of the test or during grade 2 in 1999-2000, they grew more than non-black students in grade 2 during 2000-01again evidence of the growing effectiveness of the program in the second year of implementation. Performance on this measurement again is evidence of the importance of prior knowledge. Children entering school with strong knowledge and skills learn to read much quicker than those who come without them, e.g., the rapid growth of non-black students in kindergarten and grade 1. Non-black children, in general (with some exceptions), are learning to read at grade 1, and black children, again in general (but with many exceptions), are learning to read at grade 2. Developmental Reading AssessmentGrowth/Improvement Grade Level 1999-00 Growth 2000-01 Growth Kinder-Black KinderNon-B Difference Growth Ratios Grade 1Black Grade 1Non-B Difference Growth Ratios Grade 2Black Grade 2Non-B Difference Growth Ratios 2.14 4.40 -2.26 49% 12.38 17.69 -5.31 70% 10.11 11.79 -1.68 86% 3.21 6.62 -3.41 48% 14.22 17.46 -3.24 81% 10.55 9.87 0.68 107% As on the Writing Vocabulary sub-test, non-black growth on the Developmental Reading Assessment surpassed black growth for both cohort 1 (1999-2000 kindergarten and 2000-2001 grade 1) and cohort 2 (1999-2000 grade 1 and 2000- 2001 grade 2). Cohort Growth Cohort I Cohort 2 Black Non-Black Difference Ratio 16.99 22.69 -5.70 75% 24.46 29.20 -4.74 84% 21Both black and non-black students slightly improved their performance over the summer between kindergarten and grade 1 on the DRA. Both also improved over the summer between grade 1 and grade 2. Developmental Reading AssessmentSummer Regression Race Spring 2000 Kinder Fall 2000 Grade I Diff. Black Non-BIack 3.09 7.12 Spring 2000 Grade I 3.72 7.95 Fall 2000 Grade 2 Diff. 16.67 24.37 18.20 26.01 + 1.53 +1.64 In fall 1999 black kindergarten students achieved 2 percent of the maximum score of 44 on the DRA, as compared to 6 percent for non-black students. By the end of grade 2 in 2000-01, after two years of instruction in the new program, the black students scored at 65 percent of the maximum, as compared to 82 percent for non- black students. Developmental Reading AssessmentPercent of Maximum Score KinderBlack KinderNon-B Grade 1Black Grade INon-B Grade 2Black Grade 2Non-B Fall 1999 2 6 10 15 40 55 Spring 2000 7 16 38 55 63 82 Fall 2000 1 2 8 18 41 59 Spring 2001 8 17 41 58 65 82 Achievement Level TestGrade 2 Reading The Achievement Level Test for grade 2 reading differs significantly from the Developmental Reading Assessment, although both are measurements of reading comprehension. The Developmental Reading Assessment is a one-on-one administered test that uses in its instructions to students the same language as the teacher used in the instruction of the students. Also, the teacher, using a rubric, scores the students responses, so the student is not distracted by having to manipulate a test booklet, plus an answer sheet and pencil and be moving back and forth between the test booklet and the answer sheet. The Achievement Level Test at grade 2 is the first experience that elementary students have with a group-administered examination that requires them to bubble in their responses to questions on a separate answer sheet. The directions to the students may or may not be in familiar language. So even though some of the same concepts are tested on these two measurements, they are tested in entirely different ways, and most early childhood experts advise against administering this kind'of test to students below grade 3. (See position statements by the International Reading Association and the National Associate for the Education of Young Children.) 22Black students scores were 93 percent of the national median in spring 2001, as compared to non-black students scores at 99 percent. The gap between the RIT scores narrowed from spring 2000 to spring 2001 by one pointfrom 13 to 12. Black students RIT scores improved two points from spring 2000 to spring 2001, as compared to only one point of improvement for non-black students. Achievement Level TestGrade 2 Language Usage This sub-test is the first test in language usage experienced by LRSD elementary students. The same issues as described for the ALT reading test exist for the language usage test relating to the unfamiliarity and complexity of the testing situation. In spite of these potential difficulties, performance improved from spring 2000 to spring 2001. Black students scores were 95 percent of the national median in spring 2001, as compared to non-black students scores at 102 percent. The gap between the RIT scores narrowed from spring 2000 to spring 2001 by two pointsfrom 14 to 12. Black students RIT scores improved two points from spring 2000 to spring 2001, as compared to no improvement for non-black grade 2 students. National Research on the Achievement Gap National research and other credible studies on the achievement gap between black and non-black students have been consistent in finding that black students test scores are lower than non-black students test scores. Educational Achievement and Black-White Inequality, just released in July 2001 by the National Center for Education Statistics (a department of the Office of Educational Research and Improvement), found the following in their analyses of educational achievement of children at various points between grades 1 and 12: Black-white gaps in mathematics and reading achievement appeared at every grade studied. Even for children with similar levels of prior achievement one or two grades earlier, mathematics and reading scores of blacks were generally lower than the corresponding scores of whites (p. xi). The black-white reading gap also differed in size across grades, but not in an entirely consistent manner it grew wider between grades within two elementary school cohorts, but was narrower in cohorts observed in grades 9 and 12 than in a cohort observed in grade 2 (p. xii). 23Within the same samples of children, the black-white reading gap increased by one-third between grades 1 and 2 and one-fifth between grades 3 and 5, while remaining the same between grades 7 and 9, and between grades 10 and 12 (p. xiv). These findings imply that black-white disparities in educational achievement can widen as students progress through elementary or secondary school. Possible explanations for these differences in achievement growth include differences in the school or home environments of children of different racial backgrounds that make it more difficult for blacks to acquire math or reading skills at the same pace as whites (p. xv). The black-white reading gap widened by one-third between grades 1 and 2, and by one-sixth between grades 3 and 5 (p. 37). Within each of the elementary school samples, black children acquired reading skills at slower rates than white children. Between grades 1 and 2, and between grades 3 and 5, blacks acquired reading skills at a rate onefifth slower than the rate of whites (p. 37). The existence of a smaller black-white reading gap in grade 12 than in grade 2 suggests that these gaps are not immutable, and that appropriately designed public policies can reduce the educational disparities between black and white children (p. 42). The results of two years of changes in the LRSD policies, programs, and procedures in grades PreK-2 indicate that both black and non-black children in the Little Rock School District are learning to read and are well on their way of reaching the district goal of all children being able to read independently by grade 3 (see Section 5.2.1 of the Revised Desegregation and Education Plan). The findings and analyses in this report indicate trends in the opposite direction of the national research findings cited above and of scores of other similar studies. Instead of black students growing at slower rates than non- blacks, in most of the measurements the LRSD results indicate higher rates of growth of black students than non-blacks. Instead of the gap widening between grades 1 and 2 as it does in national studies, it narrows significantly in the LRSD by every one of the eight measures (five sub-tests of the Observation Survey, the Developmental Reading Assessment, and two sub-tests of the Achievement Level Test). The black to non-black ratios on each of the measures are summarized below for each of the two years of testing: 24Kindergarten Black to Non/Black Ratios: Reductions in the Achievement Gap Sub-Test hall 1999 Spring 2000 Change hall 2000 Spring 2001 Change Letter Identification Word Test Concepts about Print Writing Vocabulary Hearing/Recording DRA 81% 57% 69% 62% 54% 35% 96% 76% 81% 66% 70% 43% -15 -19 -12 -4 -16 -6 83% 53% 72% 58% 46% 41% 97% 82% 87% 71% 76% 48% -14 -29 -15 -13 -30 -7 Grade 1 Black to Non-Black Ratios: Reductions in the Achievement Gap Sub-Test hall 1999 Spring 2000 Change h all 2000 Spring 2001 Change Letter Identification Word Test Concepts about Print Writing Vocabulary Hearing/Recording DRA 96% 73% 88% 87% 78% 64% 100% 92% 93% 84% 91% 68% -4 -19 -5 +3 -13 -4 99% 68% 84% 80% 74% 47% 100% 94% 93% 88% 92% 71% -1 -26 -9 -8 -18 -24 Grade 2 Black to Non-Black Ratios: Reductions in the Achievement Gap Sub-Test hall 1999 Spring 2000 Change h all 2000 Spring 2001 Change Word Test Writing Vocabulary Hearing/Recording DRA ALT Reading_____ ALT Lang. Usage 89% 95% 86% 74% 96% 82% 88% 78% 93% 93% -7 + 13 -2 -4 91% 84% 87% 70% 98% 87% 91% 80% 94% 94% -7 -3 -4 -10 Another prominent study on black vs. white achievement in reading is entitled, Does the Black-White Test Score Gap Widen after Children Enter School? by Meredith Phillips, James Crouse, and John Ralph, published in The Black-White Test Score Gap (1998) that was edited by Christopher Jencks and Meredith Phillips. That comprehensive studys major findings included the following ... About half of the total black-white math and reading gap at the end of high school can be attributed to the fact that blacks start schools with fewer skills than whites. The other half can be attributed to the fact that blacks learn less than whites who enter school with similar initial skills. This does not necessarily mean that schools are a major contributor to the black-white test score gap. Although blacks may attend worse schools than whites, may be treated differently from whites in the same schools, or may be less interested in school than initially similar whites, it is also possible that blacks parenting practices, peer influences, summer learning opportunities, or beliefs about their academic ability could explain why 25they learn less between first asid twelfth grades than initially similar whites (p. 232). ... The black-white vocabulary' gap is about the same size at the end of the ninth grade as at the beginning of first grade (p. 233). ... Race appears to have a stronger negative effect on reading growth than on math growth (p. 248). ... Race also has more impact on vocabulary growth than on math growth during elementary school (p. 249). Our calculations imply that 56 percent of the math gap and 43 percent of the reading gap can be attributed to the fact that blacks start school with fewer skills than whites. It follows that 44 percent of the math gap and 57 percent of the reading gap is unrelated to racial differences in initial skills (p. 254). If equal educational opportunity means that black and white children should start school with the same basic skills, then we have yet fulfilled that ideal. The average black child starts elementary school with substantially weaker math, reading, and vocabulary skills than the average white child. Because these skill differences persist throughout elementary and secondary school, the average black child finishes the twelfth grade with weaker math, reading, and vocabulary skills than the average white child. Our results imply that we could eliminate at least half, and probably more, of the black-white test score gap at the end of the twelfth grade by eliminating the differences that exist before children enter first grade (pp. 256-257). Again, the results in the LRSD are, for the most part, in the opposite direction of these studies. Black students are making significant progress in catching up with their non- black peers through the Districts early literacy program, especially in the leaming-to- read skills that are tested on the Observation Survey. It is true, however, that the vocabulary gaps and the gaps on the more difficult measures, such as the Developmental Reading Assessment, while greatly narrowed, do still persistundoubtedly due to the much wider gaps that existed at the beginning of kindergarten. 3. Is there a relationship between teacher participation in professional development and student achievement? (See Policy IL expectation to examine cost effectiveness and Strategy 7 of the Strategic Plan.) The most expensiveand the most important-piece of the cost of any program implementation designed to improve student achievement is always the cost of professional development. The cost of student materials is a one-time expense, generally, but the cost of professional development is ongoing. New teachers have to begin their 26 initial training every year. Ineffective teachers have to be retrained. All teachers need on-going opportunities to refine and enhance their understandings and skills. Implementation has to be coached and monitored. And all are necessary if there is to be continuous improvement. Richard Allington summarizes the research literature as follows: In study after study, it is the quality of the teacher, not variation in curriculum materials, that is identified as the critical factor in effective instruction. That is not to say that materials are wholly unimportant, but that investing in teacher development has a better result than investing in curriculum materials (9). It is very important, therefore, for the District to know if there is a positive relationship between teacher participation in the LRSD training on early literacy and the achievement of their students. Patricia Busbeas Education Specialists thesis examined this question for a sample of kindergarten students. In her findings, she concluded, The current study strongly indicated Early Literacy Learning in Arkansas (ELLA) did influence the student outcomes of the ten students included in the study (p. 77). She added, It can be concluded that the teachers participation in the ELLA training did influence changes in the teachers classroom practices (p. 78). Another study relating to the black-white test score gap is also helpful here. Ronald Fergusons study, Teachers Perceptions and Expectations and the Black-White Test Score Gap was published in The Black-White Test Score Gap (1998), edited by Christopher Jencks and Meredith Phillips. He wrote the following: Simply cajoling teachers to raise their expectations for black childrenusing phrases such as All children can learnis probably a waste of time. However, good professional development programs can make a difference. Recall that some teachers in Oklahoma responded to the Great Expectations program with the assertion, My kids couldnt do that. If they had gone on teaching as they had always done, that judgment would have been correct. But when they changed their teaching methods, they learned that they were wrong. Similarly, Guskey shows that teachers can learn responsive teaching methods that weaken the link between past and future performance. Teachers who have been helped to improve their classroom practices can have seeing is believing experiences that challenge their prior biases (312). Other studies on the effectiveness of the ELLA training, as well as studies in general on the impact of professional development on teacher practice and student achievement would suggest a positive relationship, if the training is closely aligned with the curriculum standards and assessments, if the training is delivered well, if the training is followed by coaching and feedback, and if the training is in depthsufficient to build the new skills and understandings. Twelve days of ELLA training have been offered to LRSDs K-2 teachers during the past two years, in addition to the required training on the administration of the Observation Survey and Developmental Reading Assessment, and in addition to the days in the school-year calendar for professional development. In Section VII are summaries of two 27 studies conducted to determine, by program, the degree of teacher participation in professional development. The highest average number of days of participation has been among kindergarten teachers for the schools implementing ELLA strategies and for those implementing both ELLA and Reading Recovery. The greatest participation at all three grade levels has been among the teachers in schools implementing both ELLA strategies and Reading Recovery. These schools, perhaps, have the greatest degree of commitment to improved literacy, as evidenced both by the commitment of their Title I funds to literacy by funding Reading Recovery and by the teachers willingness to attend professional development sessions. Almost half of their grade 1 teachers have completed the entire 12 days of training. The following table indicates by school the participation level of teachers in ELLA training over the two-year period: Teacher Participation in ELLA Training, by School School # Kinder Tchers Kinder Days #Gr. 1 Tchers Grade I Days #Gr. 2 Tchers Grade 2 Days Total Days Total # Tchers Avg. # Days Badgett Bale Baseline Booker Brady________ Carver Chicot Cloverdale Dodd Fair Park Forest Park Franklin Fulbright_____ Geyer Springs Gibbs Jefferson King_________ Mabelvale McDermott Meadowcliff Mitchell Otter Creek Pulaski Heights Rightsell______ Rockefeller Romine Stephens_____ Terry________ Wakefield Washington Watson Western Hills Williams Wilson Woodruff Total Average Days 1 2 3 5 4 4 4 3 2 2 2 3 4 3 3 3 5 3 3 2 2 3 3 2 4 4 3 4 3 4 3 2 3 2 2 105 0 7 0 37.5 6 32.5 17 19 21 0.5 0 36 42 24 23 24 42.5 9 0 0 8 24 33 5 10.5 21 10 32.5 20 0 0 11 34 14 9.5 573.5 5.46 2 3 3 5 3 4 3 3 2 2 3 4 3 2 3 3 3 2 3 2 2 4 2 2 3 2 2 4 3 4 3 3 3 3 2 100 9 6.5 5 43.5 0 33 22 0 24 0 4.5 48 .42 9.5 22 25 16 12.5 0 0 2 38 0 0 18 0 15 20.5 16 7 3 6 22.5 27 0 497.5 4.98 2 3 3 5 4 4 6 2 3 2 3 3 3 2 2 3 5 3 3 2 2 3 2 2 2 3 2 5 3 4 3 2 4 3 2 105 0 3.5 0 47 0.5 10 24.5 12 36 0 4 36 18 14.5 22 13.5 7.5 17 13 0 1 11 10 5 0 0 2.5 22.5 17 7 0 0 3.5 24 0 372.6 3.54 9 17 5 126 6.5 75.5 62.5 31 79 0.5 8.5 120 92 48 67 63.5 66 38.5 13 0 II 73 43 10 28.5 21 27.5 73.5 53 14 9 17 60 65 9.5 1,443.6 5 8 9 15 1 12 13 8 7 6 8 10 10 7 8 9 13 8 9 6 6 10 7 6 9 9 7 13 9 12 3 7 10 8 6 310 Teachers designated as Special Education or other special programs were added to grade 2 to simplify the table. 1.8 2.1 0.6 8.4 0.6 6.3 4.8 3.9 11.3 .08 1.1 12 9.2 6.9 8.4 7.1 5.1 4.8 1.4 0 1.8 7.3 6.1 1.7 3.2 2.3 3.9 5.6 5.9 1.2 0.3 2.4 6 8.1 1.6 4.65 28From the table above, one can infer that implementation is, in general, still at a low level since the average number of days of ELLA training experienced by teachers is 4.65 of the 12 possible days available. Kindergarten teachers have the highest level of participation, then grade 1 and then grade 2. Kindergarten, probably not coincidentally, is the highest performing grade level. Interestingly, participation does not seem to vary as much within a school as it does across schools. In other words, for the most part, either every teacher at a grade level or in a school participates fully, or they participate minimally. For example, every teacher at Franklin has participated all twelve days, and the average number of days at Dodd is 11.3 days. Other high-participation schools include Booker, Fulbright, Gibbs, and Wilson. ELLA schools with two or fewer average days of participation include Badgett, Bale, Forest Park, McDermott, Mitchell, Rightsell, Watson, and Western Hills. The major obstacle the District has had in providing the 12 days of professional development to all teachers has been the lack of availability of substitute teachers. Too many times when District staff pull teachers out of class for professional development, the school is not able to secure a substitute, so a class is divided up and sent to other classrooms. In either case, even when a substitute can be secured, valuable instructional time is lost. The staff has attempted, therefore, to provide the training in the evenings, on week-ends, and during the summer, paying stipends to teachers for attendance rather than paying for substitutes. This strategy makes participation optional, and fewer teachers participate. The following table displays by quarter the number of classrooms where teachers were absent in 2000-01 and there were no substitutes available to cover their classes. The absences were for all possible reasons, not just for ELLA training, but the numbers indicate the extent of the problem in attempting to provide teacher training during school time. Teacher Absences without Substitutes, Elementary Schools, 2000-01 Quarter First Second Third Fourth Total Number 63 120 295 479 957 One can assume approximately half of those 957 teachers would be K-2 teachersor 478. The other half can be assumed to be 3-5 teachers. If there is an average of 20 students in the K-2 classrooms and given the mandated two and one-half hours daily for literacy instruction, then 23,900 student hours of literacy instruction were lost in 2000-01 in grades K-2 alone due to teacher absences and the lack of quality substitutes. The number of lost hours due to teacher absences covered with a substitute substantially increases this number. Given the necessity of teachers being out from time to time due to illness and the resulting loss of quality instructional time, it is clearly a major problem to 29 pull teachers out of classrooms for professional developmenteven when their participation in these activities results in improved sludesit achievement. The statistician in PRE conducted statistical analyses for each of the past two years to determine which of the four instructional approaches (ELLA only, ELLA plus Reading Recovery Success for All and Direct Instruction) was producing the most powerful results. For both years the most effective program was clearly the ELLA plus Reading Recovery, and the second most powerful program was ELLA only. In 2000-01 the Success for All schools did perform at a significantly higher level than they did in 1999- 2000, especially at grade 1. It is important to note that the schools implementing ELLA plus Reading Recovery were also the schools with the highest participation in professional development. 4. Is there evidence of success in each of the four literacy models in useEarly Literacy Learning in Arkansas (ELLA) only ELLA and Reading Recovery Success for All and Direct Instruction? (See Section 2.7 of the Revised Desegregation and Education Plan.) When the early literacy program plan was developed, it allowed schools to implement either ELLA, ELLA plus Reading Recovery, or Success for All strategies. If a school wished to do something different, that school was required to submit a waiver request for approval. Only Washington Magnet did so. Their waiver application requested approval for them to continue the implementation of Direct Instruction, and it was granted. To establish evidence of success for each of these models, at least one school implementing each of the four models was examined. Except for Direct Instruction where there is only one school involved, there is, of course, more than one successful school implementing each model. One is reminded here, however, of an oft-repeated quotation from Ron Edmonds: How many effective schools would you have to see to be persuaded of the educability of poor children? If your answer is more than one, then I submit that you have reasons of your own for preferring to believe that basic pupil performance derives from family background instead of school response to family background. Ron Edmonds was the seminal researcher who established the first correlates of effective schools, i.e., those schools that were effective in teaching all students, regardless of economic or racial background. A Successful ELLA School Carver Magnet School is a good example of a school that has implemented the ELLA strategies. Although a magnet school. Carver has many students from poverty backgrounds and who enter kindergarten performing considerably below their more affluent peers. Because they have so many students from middle and upper-class 30 backgrounds, their average scores are usually among the highest in the District. Effective schools, however, are those schools with not only high average (mean) scores, but also high scores among each of their sub-groups. Carvers progress in improving the percent of their students performing at or above the readiness level on the Developmental Reading Assessment and in closing the black to non-black achievement gap is displayed in the following table. Without exception, both black and non-black students improved their performance at grade-level during the second year of implementation. And, without exception, the gap was considerably narrowed the second year of the program. Finally, the grade 2 gap in 2000-2001 is only about a third of what it was for kindergarten students. Improvements in Percent Readiness Developmental Reading Assessment Can er Magnet 1999-2000 2000-2001 KindergartenAll KindergartenBlack KindergartenNon-Black KindergartenGap Grade 1All Grade 1Black Grade 1Non-Black Grade 1Gap__________ Grade 2All Grade 2Black Grade 2Non-Black Grade 2Gap 80.5% 74.4% 91.7% 17.3 76.1% 64.7% 90.3% 25.6 81.8% 78.3% 94.7% 16.4 90.4% 83.8% 97.2% 13.4 91.8% 83.3% 100% 16.7 93.1% 91.1% 95.2% 4.1 An area school that has performed well in the last two years is McDermott Elementary. This school performed in the highest range at the kindergarten and grade 1 the first year of the new program and at all three grade levels in 2000-01. McDermott Elementary ranked in the top five schools on the DRA in 1999-2000 at kindergarten and grade 1 and at all three grade levels in 2000-01. McDermotts kindergarten class was the highest performing in the District both years, and McDermotts grade 2 was among the schools that improved the most in 2000-01. It is important to note, as well, that McDermott virtually closed the achievement gap in 2000-01 between black and non-black students at grades 1 and 2 on the Developmental Reading Assessment. 31Improvements in Percent Readiness Developmental Reading Assessment McDermott Elementan' 1999-2000 2000-2001 Kindergarten.All KindergartenBlack KindergartenNon-Black KindergartenGap Grade IAll Grade 1Black Grade 1Non-Black Grade 1Gap__________ Grade 2--A11 Grade 2Black Grade 2Non-Black Grade 2Gap 94.8 94.7 97.3 2.6 80.4 85.7 76.9 *8.8 67.1 62.8 86.4 23.6 94.3 86.4 100.0 13.6 87.9 87.5 87.9 0.4 89.6 89.5 89.7 0.2 A Successful ELLA, Plus Reading Recovery. School Dodd Elementary, a school with high percentages of minority and poverty, is an example of a successful school implementing both ELLA strategies and Reading Recovery. Because Dodd is very smalt and because it has high mobility, it is difficult to make valid comparison of scores from one year to the next. One child represents several percentage points in a small school, and the children who were in kindergarten last year are likely to be very different from the children in grade 1 this year. Nevertheless, the efforts of the Dodd faculty are making major differences in the literacy of the children who attend there. In the second year of the program implementation, both grade 1 and grade 2 improved significantly. The gap was virtually closed in grade 1, and that is the year of the Reading Recovery intervention for the lowest performing students. Improvements in Percent Readiness Developmental Reading Assessment Dodd Elementary' 1999-2000 2000-2001 KindergartenAll KindergartenBlack KindergartenNon-Black KindergartenGap Grade 1All Grade 1Black Grade 1Non-Black Grade 1Gap__________ Grade 2All Grade 2Black Grade 2Non-Black Grade 2Gap 86.5% 88.2% 85.0% *3.2 58.3% 60.0% 62.5% 2.5 51.7% 52.9% 71.4% 18.5 80.0% 76.5% 88.2% 11.7 73.5% 72.2% 73.3% 1.1 82.8% 73.7% 100% 26.3 Denotes that black students are higher performing than non-black students. A Successful Success for All School Baseline Elementary made such dramatic improvements in 2000-01 that they were selected as an example of a successful Success for All school. Baseline Elementary has 86 percent of its children eligible for the free/reduced lunch program and very few 32 non-black children, so their challenge is to overcome the effects of poverty and the disadvantages that go with it. The improvements at Baseline at the kindergarten and grade 1 levels are exemplary. Grade 2 did not perform as well, but black students did improve sufficiently to close the gap at that grade level. Improvements in Percent Readiness Developmental Reading Assessment Baseline Elementary 1999-2000 2000-2001 KindergartenAll KindergartenBlack KindergartenNon-Black KindergartenGap Grade 1All Grade 1Black Grade 1Non-Black Grade 1Gap Grade 2All Grade 2Black Grade 2Non-Black Grade 2Gap 51.0% 48.6% 100% 51.4 29.6% 30.2% 20.0% *10.2 47.1% 46.2% 57.1% 10.9 94.0% 92.5% 100% 7.5 91.8% 68.3% 85.7% 17.4 60.5% 60.5% 60.0% *0.5 Denotes a school where blacks scores are higher than non-blacks. A Successful Direct Instruction School Washington Magnet is the only school in the Little Rock School District implementing the Direct Instruction model. Their overall performances at kindergarten and grade 2 are strong, although the gap did increase at both grade levels in the second year. The gap at grade 1 nanowed, but the scores are low. Success is very mixed at this school, with few consistent patterns. Both sub-groups improved only at grade 2. Improvements in Percent Readiness Developmental Reading Assessment Washington Magnet 1999-2000 2000-2001 KindergartenAll KindergartenBlack KindergartenNon-Black KindergartenGap Grade 1All Grade 1Black Grade 1Non-Black Grade 1Gap Grade 2-All Grade 2Black Grade 2Non-Black Grade 2Gap 81.2% 81.8% 83.3% 1.5 35.5% 24.4% 55.2% 30.8 63.3% 61.7% 73.1% 11.4 84.1% 78.0% 95.7% 17.7 41.1% 36.8% 50.0% 13.2 81.4% 74.3% 88.6% 14.3 As demonstrated in the data under Research Question #1 and with the information in this section, the literacy programs are working well in a variety of school models and with both black and non-black students. 335. What are the programs strengths and weaknesses? (See Policy IL.) When a District chooses to implement a program that is already research-based, already tested with students of similar demographics, then the program itself does not require evaluation. Rather, it is the program implementation that requires scrutiny. Thus, in response to this question, the program evaluation team must consider the areas of strong implementation and the areas where the implementation has been weak. Program Strengths The elementary literacy programs (ELLA only, ELLA plus Reading Recovery, Success for All, and Direct Instruction) in the Little Rock School District are producing improvements in student achievement on every one of the eight different measurements examined in this program evaluation they are producing improvements in every individual school, even if only at one grade level and they are narrowing and sometimes totally closing the achievement gap, even as non-black students are also improving. It would be impossible to determine which individual components of the new programs have had the most powerful effects since everything was implemented at once, without setting up controlled experiments. There tends to be a consensus, however, among teachers and reading specialists who have been asked that the designation of the two and one-half hour uninterrupted block of time for literacy instruction has made a major difference in the degree of emphasis given to literacy at every school. Also cited as important and powerful are the ELLA strategies that teachers are learning through the professional development program and the availability of the new materials in every classroom. Animated Literacy has made a difference in childrens mastery of phonemic awareness at the kindergarten level. Another definite strength is that teachers are using the assessment results to drive their instructionone of the staffs objectives in selecting the Observation Survey and the Developmental Reading Assessment with their rich sources of data and their tight alignment with the Districts curriculum standards and grade-level benchmarks. Teachers also cite the preponderance of children who now have access to the Districts pre-kindergarten program as another strength. More and more children are having an opportunity for another year of experiences that promote higher levels of literacy. Program Weaknesses Program weaknesses can be found in the examination of school-by-school performances. There was in 2000-01 a narrowing of the achievement gap among the performance of the 35 elementary schools, but gaps still exist, the widest on the Developmental Reading Assessment., And at each grade level the gap widened in 2000-01 on one sub-testLetter Identification at grade 1 and Writing Vocabulary at kindergarten and grade 2. 34Another program weakness is in the variability of the amount of teacher training experienced from school to school. There is one school where not a single teacher has participated in the ELLA training and there are two where all the teachers have participated fully, with other schools somewhere in between. Teachers in the Success for All and Direct Instruction schools have had lower rates of participation in the ELLA training. Grade 1 is the lowest performing grade of the three grades tested, so that grade level requires attention in improvement efforts. There is likely a need for further intervention. Lack of a monitoring plan through classroom observations to document the level of implementation is a problem. This weakness not only resulted in a late identification of poor implementation in some cases, but it was also a weakness in evaluating the consistency of program implementation. Finally, more high quality parent involvement would likely improve results, especially to address the summer regression phenomenon, but also in general. 6. Is the program cost effective? (See Policy IL and Strategy 3 of the Strategic Plan.) The literacy plan was adopted in spring 1999, so the first purchases of classroom materials and the first professional development sessions designed to prepare teachers for implementation occurred in summer 1999. The District has invested over the two-year period $405,948 on materials for teachers and books for classroom libraries to support the implementation of the new program. The State provided a grant of $198,440 that was also spent on students instructional materials. During the two-year period the District spent $119,450 on the salary and fringe benefits for one teacher leader and approximately $12,000 on teacher stipends for inservice participation. Other professional development conducted during the school day at times required the cost of substitutes (paid by the Human Resources Department) and at times was absorbed by the school staff. One additional expense was the $29,500 cost for the Developmental Reading Assessment materials. In summary, the total investment over two years in this program has been as follows: Instructional materials Professional development Assessments Total $604,388* $131,450 $ 29,500 $765,338* *0f these amounts, $198,440 was funded by the State through a grant. The total cost to the District, therefore, was $566,898 for ELLA implementation. During the past two 35 years all ELLA training was conducted during the summer and after school hours. Teachers were not paid a stipend. One group of 35 teachers, however, was paid stipends for the training-of-trainers training for live days of pailscipalion in summer 1999. The schools implementing Success for All and Direct Instruction paid for their professional development and student materials from their Title I budgets. A reading specialist from the Elementary Literacy Department has been assigned to supervise the Success for All school programs, so her salary, travel, and benefits are costs to the District, not Title I. Reading Recovery costs were also funded through school-level Title I budgets. An average salary for a Reading Recovery teacher is $40,000, plus fringe benefits. The cost of six hours of graduate-level tuition for their training is approximately $1,000. And the cost of materials to use with students is about $3,000 per teacher. Schools have expended about $1,100,000 over the past two years in paying for training costs and salaries for their Reading Recovery teachersa total of eight. Are all these expenditures cost effective? It is really too early to make a determination. If the program results in virtually all children learning to read independently by grade 4 when they take the Benchmark examination, then this program is a bargain on that measurement alone. If only a few children become proficient enough to go to college and, as a result, earn a much higher salary than they would have had they not learned to read well, then the investment will be returned several times. If the District moves from only 30 percent of its students performing at or above the proficient level to even^O percent, then the investment will see incremental returns. The staff and the Board of Education must make a determination at this juncture as to whether the current results are encouraging enough to continue the course that has been set. The data indicate positive results at all three grade levels, even though the data also indicate that the teachers have received only about one-third of the training for full implementation. Ongoing investments, plus possible policy decisions, will be necessary to ensure that all teachers receive the minimum 12 days of training, with appropriate follow-ups and enhancements. Recommendations for Improvement are made for the consideration of The following recommendations for improvement District and building-level staff, as well as for the Board of Education. Instructional Recommendations Given the gap between black and non-black achievement in the fall kindergarten scores for both 1999-2000 and 2000-01, there is an indication that Letter Identification should be stressed more with black and other low-performing prekindergarten students. Schools should monitor their implementation of the balanced literacy program, including Animated Literacy implementation, to ensure that all kindergarten children learn their letters and sounds by the end of the kindergarten year. The National Research Council in Preventing Reading Difficulties in Young Children makes a similar recommendation: 36It is clear from the research on emergent literacy that important experiences designed to reduce the number of children with inadequate literacy-related knowledge (e.g., concepts of print, phonemic awareness, receptive vocabulary) at the onset of formal schooling would considerably reduce the number of children with reading difficulties and, thereby, the magnitude of the problem currently facing schools (317). Adams points out that it is not simply the accuracy with which children can name letters that gives them an advantage in learning to read, but it is the ease or fluency with which they can do so (p. 43). Therefore, if it takes two years of school just to learn the letters, then some time is lost in developing ease and fluency. She concludes, Thus the speed with which they can name individual letters both strongly predicts success for prereaders and is strongly related to reading achievement among beginning readers (43). Schools and teachers should assess why the 2000-01 grade 2 students, both black and non-black, performed lower on the Word Test than those in 1999-2000. One implication is that decoding development should be emphasized even more in grade 1. Adams notes that weaknesses in basic decoding skills may be the most common and can be the most serious source of reading difficulties (92). Acceleration of black students decoding skills in reading frequently occurring words is recommended in pre-kindergarten and kindergarten so that the black students performance on the Word Test catches up to non-black performance before grade 2. The gap closed in Letter Identification at grade 1 and for the Word Test in grade 2. Even though it is almost closed at the end of grade 1 for Concepts about Print, it is not quite, and instruction must continue in this area in grade 2 until the concepts are mastered for students who are low-performing. Adams points out that A lack of phonemic awareness ... appears to be characteristic of children who are failing or have failed to learn to read (57). The skills and knowledge that are prerequisite to high performance on Concepts about Print should be consciously and explicitly taught at the pre-kindergarten and kindergarten levels so that little remains to be done in this area by grade 1. This kind of acceleration is essential if the gap is to be closed on this measure. Schools must ensure the full implementation of the Pre-ELLA program at the prekindergarten level and the ELLA strategies at the kindergarten level. Even though both black and non-black students recorded their highest rates of growth on Writing Vocabulary, the achievement gap is highest on this measure at the end of two years than of any other sub-test. Even at that, the gap is only 13 points, with black scores achieving 87 percent of those of non-black students. There are implications here, again, for African-American students, who clearly 37 need even greater attention to vocabulary and spelling development and an accelerated program to bring their achievement to the level of that of non-black students. None of the measurements used in K-2 is a vocabulary test, per se. They all, however, test vocabulary, and as research indicates, higher-level, more standardized tests, such as the Achievement Level Tests, essentially measure students vocabulary, regardless of the subject matter being tested. Marzano, Kendall, and Gaddy report in their recent book. Essential Knowledge: The Debate Over What American Students Should Know, that their research indicates that vocabulary instruction should be the focal point of education, especially for students from more disadvantaged backgrounds (143). They continue as follows: Research indicates that even when there is no attempt to ensure that the words students are taught are ones they will need to know when learning new content, the effect on their achievement is substantial. Specifically, teaching vocabulary has been shown to increase students ability to understand new content by 12 percentile points (146-147). The effects of vocabulary instruction are even more powerful when the words selected are those that students most likely will need to know as they encounter new content. Specifically, research indicates that student achievement will increase by 33 percentile points when vocabulary instruction focuses on specific words that are important to what they are learning (147). The National Reading Panels findings indicate that vocabulary should be taught both directly and indirectly. Repetition and multiple exposures to vocabulary items are important. Learning in rich contexts, incidental learning, and use of computer technology all enhance the acquisition of vocabulary. Direct instruction should include task restructuring as necessary and should actively engage the student (p. 14). Adams encourages teachers to teach vocabulary explicitly: direct vocabulary instruction is generally shown to result in an increase in both word knowledge and reading comprehension (p. 29). Teachers should include in their lessons at least some degree of instruction on test wiseness. That is, since the ALT is the first test with the requirement to bubble answers that the students take, their lack of familiarity with the format and expectations may depress achievement. An excellent resource for teachers in ways to do this work effectively and ethically is A Teachers Guide to Standardized Reading Tests: Knowledge Is Power by Lucy Calkins, Kate Montgomery, and Donna Santman. 38Parent Involvement Recommendations Students who begin kindergarten knowing a lot about literacy learn faster than those who do not. They have the necessary prior knowledge to make meaning from grade-level instruction. A concern, therefore, is that while some black students take two years of school to master letter identification, non-black students had achieved 95 percent of the maximum score by the end of kindergarten. While many black students continued to master letter identification in grade 1, non-black students were, no doubt, moving forward in other areas, such as vocabulary development. Adams states, One interpretation impossible to avoid is that the likelihood that a child can succeed in first grade depends, most of all, on how much he or she has already learned about reading before getting there (p. 44). The poverty-related factors, therefore, that created the gap for children before they started to school continue after they start to school, even though the school programs may be highly effective. These findings have implications for public policy at the federal, state, and community level for pre-natal care and early childhood health care, parenting, cognitive development, and social development. It takes a village to teach children high levels of literacy. The schools and classroom teachers can do their best by seeking individual ways to accelerate student learning in pre-kindergarten and kindergarten so that grade 1 students can spend their time on more complex cognitive operations that are prerequisite to learning to read. Both black and non-black students at the end of both kindergarten and grade 1 lost significant Writing Vocabulary during the siunmer, indicating, most likely, little reinforcement of this skill during summer months. Along with summer reading and spelling lists that are sent home, teachers should also include exercises that allow students to maintain, at a minimum, and ideally to improve their Writing Vocabulary during the summer. Schools should organize, perhaps with the assistance of the HIPPY staff, a preschool PT A so that support and assistance can be provided to as many parents as possible in the development of early literacy experiences, from birth forward. For instance, parents need to know the research cited by Adamsthat the beginmng of phonemic awareness is seeded in childrens knowledge of nursery rhymes (p. 42). She continues ... success in reading can be found in traditional rhymes and word games such as Baa Baa Black Sheep and Humpty Dumpty (p. 43). Adams also found that The single most important activity for building the knowledge and skills eventually required for reading appears to be reading aloud to children. In this, both the sheer amount of and choice of reading materials seem to make a difference (46). The staff should design appropriate reports and processes to keep parents well informed about their childrens performance on the early literacy assessments, as 39well as about ways that they can support higher achievement. The Student Academic Improvement Plan (SAIP) and the Parent-School Compact are two processes that address the need for close communication with parents. Schools should evaluate the effectiveness of their implementation of these processes. Rasinski summarizes the research on the effect of parental involvement on literacy as follows: For if you are talking about research-based information, we have it in the effects of community-home-school connections and student achievement in reading. Study after study has shown positive effects of parental involvement in the literacy education of their children. .. (161). District-level should collaborate with school-level staff in designing processes for effective communication of the results of K-2 assessments. Intervention Recommendations Given the amount of summer regression that is evident on several of the measurements, attention to these issues should be a priority for those children participating in the inter-sessions of the Extended Year Education (EYE) schools and for children participating in after-school programs and summer school. A concern of the Superintendents Cabinet for the past three years has been that several schools are no longer eligible for Title I funding due to changes in their demographic make-up. A school must have among its students at least 35 percent who are eligible for ffee/reduced lunch services in order to qualify. Both the staff in those schools and staff at the District level have noted that some of the Districts most economically-deprived children attend those schools, and there is now no funding for special services or support for them. The area schools without Title I funding are as follows: Forest Park, Fulbright, and Jefferson. Fulbright and Jefferson lost their Reading Recovery programs when Title I funds were no longer available, and Pulaski Heights lost theirs due to inadequate Title I funding. Not surprisingly, those very schools have some of the widest achievement gaps of any of the Districts schools. The District should continue to explore options for providing the necessary support services to these schools for the lowest-performing students. Otherwise, wide achievement gaps will continue to exist. The District should take all necessary steps to continue its efforts to secure external grant funding for additional support for individual schools. Funds are needed to provide more Reading Recovery teachers, especially in the lowest performing schools. Ideally, there would be one Reading Recovery teacher for every two grade 1 classrooms. This step, plus the addition of Literacy Coaches in every school, would greatly enhance the likelihood'of success. Schools with Title I funding should redirect their budgets to fund these effective interventions. 40Since grade 1 is the lowest performing grade of the three grades tested, attention must be devoted to improvement at this level. The National Research Council makes the following recommendations for students in grade 1 who still are having reading related difficulties: Additional instructional services in supplemental reading programs should be provided in the first grade. Instruction should be provided by a well-qualified reading specialist who has demonstrated the ability to produce high levels of student achievement in reading. Materials and instructional techniques should be provided that are well integrated with ongoing excellent classroom instruction and that are consistent with the findings, conclusions, and recommendations identified above in Reading Instruction in Kindergarten through Third Grade. Children who are having difficulty learning to read do not, as a rule, require qualitatively different instruction from children who are getting it. Instead, they more often need applications of the same principles by someone who can apply them expertly to individual children who are having difficulty for one reason or another (327). To these ends, the following recommendations are critically important for continued improvement: Expand Animated Literacy into grade 1 (or even grade 2) for students who need it or design another effective intervention Encourage schools to implement their own research-based interventions during after-school, tutoring, inter-sessions, and summer school. Reallocate Title I dollars to these high-quality interventions and away from the employment of aides and provide as many Reading Recovery and Literacy Coaches as possible through all sources of funds. Continue or expand teacher training in all components of Balanced Literacy for all teachers. These components include instruction in concepts about print, phonemic awareness and letter identification, phonics, language acquisition, vocabulary development, and emergent writing. Professional Development Recommendations Given the high positive correlation in national studies and in LRSD of the relationship between teacher participation in high quality professional development and the achievement levels of their students, it is evident that professional development should continue to be a high priority in the ongoing implementation refinements of the PreK-2 Literacy Program, including for 41teachers in the Success for All and Direct Instruction schools. Too, participation should not be optional, given the findings of the National Reading Panel: the results indicated that inservice professional development produced significantly higher student achievement (17). District staff should ensure that every LRSD teacher knows the skills that students must have to perform at least at the national median on the Achievement Level Tests. The publications available from the Northwest Evaluation Association delineate the specific skills and understandings that are needed to perform in various levels of each test. It is important that teachers ensure that those same skills and understandings are included in their lesson plans so that students are as well prepared as possible to perform well on the grade 2 ALTs. Schools Identified for Improvement at K-2 The following schools are identified for improvement. They are among the lowest performing schools in at least two of the three grade levels tested. According to the Districts Priority Intervention Procedures, the District should provide special assistance for improvement for the staff of the following schools: Badgett, Cloverdale, Meadowcliff, Mitchell, and Watson. Badgett, Mitchell, and Watson are ELLA schools, but with very low levels of teacher participation in professional development and, thus, low levels of program implementation. Both Cloverdale and Meadowcliff are Success for All schools, again with low levels of implementation in that program. Year 3 Program Evaluation Recommendations The District should design and implement procedures to document the quality of program evaluation in all K-2 classrooms. The District should add to the year 3 study an analysis of how the new literacy program has impacted the referral and placement of K-2 children for 504 or Special Education services. The District should add to the year 3 study an analysis of the impact of participation in the Districts pre-kindergarten program on early literacy achievement. The District should measure Effect Size as an alternative to the growth index in determining improvement. The District should examine the impact of the new literacy program on the retention rate. 42Section IX: Bibliography Adams, Marilyn Jager (1990). Beginning to Read: Thinking and Learning about Print. Cambridge, Massachusetts: The MIT Press. Adams, Marilyn Jager (1990). Beginning to Read: Thinking and Learning about Print A Summary. Cambridge, Massachusetts: The MIT Press. Allington, Richard L. (2001). Teaching Children to Read: What Really Matters. Preventing Early Learning Failure. Bob Somson, Ed. Alexandria, VA: Association for Supervision and Curriculum Development. Beaver, Joetta (1997). Developmental Reading Assessment. Parsippany, NJ: Celebration Press. Busbea, Patricia A. (2000). A Study of the Effects of the Professional Development Model. Early Literacy Learning in Arkansas (ELLAl on Changes in Kindergarten Teachers Instructional Practices and Students Outcomes. Little Rock, AR: University of Arkansas at Little Rock. Calkins, Lucy, Kate Montgomery, and Donna Santman (1998). A Teachers Guide to Standardized Reading Tests: Knowledge Is Power. Portsmouth, NH Heinemann. Celebration Press (1996). Preliminary Validation of the Developmental Reading Assessment. Parsippany, NJ: Celebration Press. Clay, Marie M. (1993). An Observation Survey of Early Literacy Achievement. Portsmouth, NH Heinemann. Compliance Committee (March 2000). Interim Compliance Report. Little Rock, AR: Little Rock School District. Compliance Committee (March 2001). Compliance Report. Little Rock, AR: Little Rock School District. Dom, Linda (1999). Partnerships in Literacy: School-Wide Design for Success. Little Rock, AR University of Arkansas at Little Rock. Dom, Linda J., Cathy French, and Tammy Jones (1998). Apprenticeship in Literacy: Transitions Across Reading and Writing. York, Maine: Stenhouse Publishers. Edmonds, Ron (1979). Effective Schools for the Urban Poor. Educational Leadership. 37(1). 15-24. 43Ferguson, Ronald F. (1998). Teachers Perceptions and Expectations and the Black- White Test Score Gap. In The Black-White Test Score Gap edited by Christopher Jencks and Meredith Phillips. Washington, DC: Brookings Institution Press. 273-317. Flippo, Rona, Ed. (2001). Reading Researchers in Search of Common Ground. Newark, Delaware: International Reading Association. International Reading Association and National Association for the Education of Young Children (1999). Learning to Read and Write: Developmentally Appropriate Practices for Young Children. Newark, DE: International Reading Association. International Reading Association (July 1998). Phonemic Awareness and the Teaching of Reading. Newark, DE: International Reading Association. Jencks, Christopher and Meredith Phillips, Eds. (1998). The Black-White Test Score Gap. Washington, DC: Brookings Institution Press. Joint Committee on Standards for Educational Evaluation. James R. Sanders, Chair (1994). The Program Evaluation Standards (2"*^ Edition): How to Assess Evaluations of Educational Programs. Thousand Oaks: Sage Publications. Marzano, Robert J., John Kendall, and Barbara Gaddy (1999). Essential Knowledge: The Debate Over What American Students Should Know. Alexandria, VA: Association for Supervision and Curriculum Development. National Association for the Education of Young Children (1990). Guidelines for Appropriate Curriculum Content and Assessment in Programs Serving Children Ages 3 Through 8. Washington, DC: NAEYC. National Association for the Education of Young Children (1987). Standardized Testing of Young Children 3 Through 8 Years of Age. Washington, DC: NAEYC. National Center on Education and the Economy and the University of Pittsburgh (1999). Reading and Writing Grade by Grade: Primary Literacy Standards for Kindergarten through Third Grade. Washington, DC: National Center on Education and the Economy. National Center for Education Statistics (2001). Educational Achievement and Black- White Inequality. Washington, DC: United States Department of Education Office of Educational Research and Improvement. 44National Reading Panel (2000). Teaching Children to Read: An Evidence-Based Assessment of the Scientific Research Literature on Reading and Its Implications for Reading Instruction. Washington, DC: National Institute of Child Health and Human DevelopmentNational Institutes of Health. National Research Council (1998). Preventing Reading Difficulties in Young Children. Washington, DC: National Academy Press. Neuman, Susan B. and David K. Dickinson (2001). Handbook of Early Literacy Research. New York: The Guilford Press. Phillips, Meredith, James Crouse, and John Ralph (1998). Does the Black-White Test Score Gap Widen after Children Enter School? in The Black-White Test Score Gap, edited by Christopher Jencks and Meredith Phillips. Washington, DC: Brookings Institution Press. 229-272. Rasinski, Timothy V. (2001). A Focus on Communication with Parents and Families. Reading Researchers in Search of Common Ground. Newark, DE: International Reading Association. 159-166. Reeves, Douglas (2000). Accountability in Action: A Blueprint for Learning Organizations. Denver, CO: Center for Performance Assessment. Reeves, Douglas (1998). Making Standards Work: How to Implement Standards-Based Assessments in the Classroom. School, and District. Denver, CO: Center for Performance Assessment. Strickland, Dorothy S. (2001). Early Intervention for Afncan American Children Considered to Be at Risk. In Handbook of Early Literacy Research. Susan B. Neuman and David K. Dickinson, Eds. New York: The Guilford Press. Wiggins, Grant P. (1993). Assessing Student Performance: Exploring the Purpose and Limits of Testing. San Francisco: Jossey-Bass Publishers. Williams, E. Jane (1999). Developmental Reading Assessment: Reliability Study. http://intranet.prenhall.eom/edtech/mcp/Docs/research/DRA.doc4/13/00 45Cb'') /i/ Year 2 Evaluation: The Effectiveness of the PreK-2 Literacy Program in the Little Rock School District 1999-2000 and 2000-2001 Presented to the Board of Education Little Rock School District October 2001 Prepared by Dr. Bonnie A. Lesley Dr. Ed Williams Patricia Price Pat Busbea Ann Freeman Ken Savage Anita Gilliam Sharon KiilsgaardTable of Contents Section I: Introduction Introduction Research Questions Methodology Outline of Program Evaluation Sections Outline of Appendices 1-6 1-2 3 3-5 5-6 6 Section II: Background on Program Design Background on Program Requirements: Design of the PreK-3 Literacy Program Background on Program Requirements: LRSD Strategic Plan Background on Program Requirements: Revised Desegregation and Education Plan 7-13 7-8 8 8-13 Section III: The Assessments The Assessments: Observation Survey The Assessments: Developmental Reading Assessment Definition of Readiness vs. Proficiency Reliability and Validity: National Study Reliability and Validity: LRSD Study Developmental Appropriateness of Testing Instruments The Assessments: Achievement Level Tests in Reading and Language Usage 14-25 14-15 15-21 16-19 19-20 20-21 21-23 23-25 Section IV: Alignment with National Research on Early Literacy 26-29 Section V: Description of Tables Table 1: Kindergarten, 1999-2000, Fall to Spring Black and Non-Black Performance Table 2: Kindergarten, 2000-01, Fall to Spring Black and Non-Black Performance Table 3: Grade 1, 1999-2000, Fall to Spring Black and Non-Black Performance Table 4: Grade 1, 2000-01, Fall to Spring Black and Non-Black Performance Table 5: Grade 2, 1999-2000, Fall to Spring Black and Non-Black Performance 30-42 31 31 32 33 33Table 6: Grade 2, 2000-01, Fall to Spring Black and Non-Black Performance Table I'. Cohort 1, Kindergarten Fall 1999 and Grade 1 Spring 2001 Table 8: Cohort 2, Grade 1 Fall 1999 and Grade 2 Spring 2001 Table 9: Grades K-2, 1999-2000, Fall to Spring Performance, All Students Table 10: Grades K-2, 2000-01, Fall to Spring Performance, All Students Table 11: Percent of Maximum Scores, Kindergarten Black Students Table 12: Percent of Maximum Scores, Kindergarten Non-Black Students Table 13: Percent of Maximum Scores, Grade 1 All Students Table 14: Percent of Maximum Scores, Grade 1 Black Students Table 15: Percent of Maximum Scores, Grade 1 Non-Black Students Table 16: Percent of Maximum Scores, Grade 1 All Students Table 17: Percent of Maximum Scores, Grade 2 Black Students Table 18: Percent of Maximum Scores, Grade 2 Non-B lack Students Table 19: Percent of Maximum Scores, Grade 2 All Students Table 20: Cohort 1All Students, Kindergarten Fall 1999 and Grade 1 Spring 2001 Table 21: Cohort 2All Students, Grade 1 Fall 1999 and Grade 2 Spring 2001 Table 22: Percent Readiness, DRA, Black and Non-Black Students Table 23: Percent Readiness, DRA, All Students Table 24: Grade 2 Reading, ALT, Black and Non-B lack Comparisons Table 25: Grade 2 Reading, ALT, All Students Table 26: Grade 2 Language Usage, ALT, Black and Non-Black Comparisons Table 27: Grade 2 Language Usage, ALT, All Students 34 35 35 36 37 37 38 38 38 38 39 39 39 39 40 40 41 41 41 42 42 , 42Section VI: Analysis of Results, 1999-2000 and 2000-2001 Letter Identification Word Test Concepts about Print Writing Vocabulary Hearing and Recording Sounds Developmental Reading Assessment 43-67 43-46 47-51 52-54 55-59 60-63 64-67 Section VII: Additional Data Achievement Gap Among Schools Impact of Professional Development 68-71 68-70 70-71 Section VIII: Program Evaluation Findings and Recommendations for Improvement Research Question 1Program Effectiveness Research Question 2Achievement Disparities Research Question 3Professional Development Research Question 4Four Literacy Models Research Question 5Program Strengths and Weaknesses Research Question 6Cost Effectiveness Recommendations for Improvement Instruction Parent Involvement Interventions Professional Development Schools Identified for Improvement Year 3 Program Evaluation 72-113 72-80 81-96 96-100 100-103 103-105 105-106 106- 107-109 109-110 110-112 112 112 112-113 Section IX: Bibliography 114-116 Section X: School-Level Data Letter Identification, Kindergarten Word Test, Kindergarten Concepts about Print, Kindergarten Writing Vocabulary, Kindergarten Hearing and Recording Sounds, Kindergarten Developmental Reading Assessment, Kindergarten 117-205 119-122 123-126 127-130 131-134 135-138 139-142 Letter Identification, Grade 1 Word Test, Grade 1 Concepts about Print, Grade 1 Writing Vocabulary, Grade 1 Hearing and Recording Sounds, Grade 1 143-146 147-150 151-154 155-158 159-162Developmental Reading Assessment, Grade 1 163-166 Word Test, Grade 2 Writing Vocabulary, Grade 2 Hearing and Recording Sounds, Grade 2 Developmental Reading Assessment, Grade 2 167-170 171-174 175-178 179-182 Cohort 1Letter Identification, Black and Non-BIack Cohort 1Word Test, Black and Non-BIack Cohort 1Concepts about Print, Black and Non-BIack Cohort 1Writing Vocabulary, Black and Non-BIack Cohort 1Hearing and Recording Sounds, Black and Non-BIack Cohort 1Developmental Reading Assessment, Black and Non-BIack 183 184 185 186 187 188 Cohort 2Word Test, Black and Non-BIack Cohort 2Writing Vocabulary, Black and Non-BIack Cohort 2Hearing and Recording Sounds, Black and Non-BIack Cohort 2Developmental Reading Assessment, Black and Non-BIack 189 190 191 192 Percent Readiness, Developmental Reading Assessment, K-2 Percent Readiness, DRA, Rank Order, K-2 Percent Readiness, DRA, Black and Non-BIack 193-195 196-198 199-201 Grade 2 ALT, ReadingAll Students Grade 2 ALT, Reading, Black and Non-BIack Grade 2, ALT, Language Usage, All Students Grade 2, ALT, Language Usage, Black and Non-BIack 202 203 204 205 Appendices A. B. C. D. PreK-3 Literacy Program Plan Section 5.2.1 of the Revised Desegregation and Education Plans March 2000 Interim Compliance Report Section 5.2.1 of the Revised Desegregation and Education Plans March 2001 Compliance Report Presentation to the Board of Education, January 2000 (update on program implementation and early results)E. Update on the Implementation of the PreK-3 Literacy Program Plan, Highlights of Grades K-2 Results: Developmental Reading Assessment, 1999-2000 and 2000-2001, and a copy of the slides for the June 2001 presentation to the Board of EducationYear 2 Evaluation: The Effectiveness of the PreK-2 Literacy Program in the Little Rock School District 1999-2000 and 2000-2001 Section I: Introduction Introduction During March 2000 the Little Rock School District provided to the Board of Education, the federal court, the Office of Desegregation Monitoring, and administrators an Interim Compliance Report, which included a status report on the implementation of the PreK-3 Literacy Program (pp. 93-105) relating to the Revised Desegregation and Education Plan (RDEP). In August 2000 the Planning, Research, and Evaluation (PRE) office provided to the Board and staff a draft copy of a program evaluation for the first year of implementation of the K-2 Literacy Program. At least two subsequent drafts were developed as more data became available, but these were not presented to the Board of Educationjust discussed among staff members. An implementation update was provided to the Board in January 2001 by the curriculum staff, on the status of program implementation and including an analysis of available data, along with an outline of next steps. Then in March 2001 the staff provided a summary evaluation in the Compliance Report (pp. 72-93) relating to the Revised Desegregation and Education Plan that was filed with the federal court and provided to members of the Board of Education. The Board of Education approved on second reading in March 2001 a new policy on program evaluation. Policy IL: Evaluation of Instructional Programs requires that the staff evaluate the instructional programs designated by the Board of Education in their annual approval of the program evaluation agenda. Each evaluation is to provide valuable insights into how programs are operating, the extent to which they are serving the intended purpose of increasing student achievement, the strengths and weaknesses, the cost-effectiveness, and directions for the future. In August 1999, 2000, and 2001, the Board of Education included the PreK-2 literacy program on its approved research agenda for the following year. An interim program evaluation was provided to the Board of Education in June 2001, the first analysis of the scores on the Developmental Reading Assessment in grades K-2 for 1999-2000 and 2000-2001. At that time the scores were reported as the percent of students at each grade level, by race, who met the standard for readiness, the level that would predict success at the next grade level (level 2 at kindergarten level 16 at grade 1 and level 24 at grade 2), Copies of that report, plus the summary and the slides were immediately sent via e-mail to principals to use in their own analysis and to provide to 1teachers and parents. (See Appendix E.) Elementary principals used these materials in their August 2001 preschool inservice sessions. This Year 2 Evaluation of the Effectiveness of the PreK-2 Literacy Program in the Little Rock School District builds on the information provided in all earlier reports. It is intended to meet the requirements specified in Policy IL for the 2000-01 school year, as well as to fulfill the requirements in Section 2.7.1 of the Revised Desegregation and Education Plan for the PreK-3 Literacy Program Plan. The grade levels evaluated include only grades kindergarten through grade 2. Another report will include grades 3 through 5. The curriculum staff received from PRE on July 19, 2001, the report on the mean scores for K-2 students on both the Observation Survey and the Developmental Reading Assessment for 2000-01. Achievement Level Test data were available earlier, but they had not yet been disaggregated by race. This program evaluation, therefore, differs from, but builds upon, the evaluation report that was presented to the Board of Education in June. It includes a much more detailed analysis of data it includes the results of the five sub-tests of the Observation Survey and it includes the average performance scores for each school on each sub-testnot just the percent of students meeting the standard. It also includes the results of the grade 2 Achievement Level Tests in reading and language usage. The new data permit the staff to calculate and analyze the scores in a different way (mean performance vs. percent readiness), and they permit the calculation of a black to non- black student ratio so that the degree to which the achievement gap in narrowed can be measured, as well as how the gap has changed over the two years of program implementation. One caution in comparing the 1999-2000 and 2000-01 pre-test scores on the Observation Survey and the Developmental Reading Assessment is that some schools did not complete their fall testing by the deadline in 1999 and so their pre-test scores were higher than they would have been had the testing been done in a timely maimer. There were instances when there were several weeks difference in the test date, so this variance This project was supported in part by a Digitizing Hidden Special Collections and Archives project grant from The Andrew W. Mellon Foundation and Council on Library and Information Resoources.