Mr. John W. Walker October 4, 2002 Page 2 We respectfully request that any obj ections you may have to the compliance plan be provided in wnting to Ken James before the October 10,2002, Board meeting, so we will have an opportunity to resolve any disagreements before the Board meeting. Thank you for your time and attention to this matter. Please do not hesitate to call if you have any questions. Sincerely, ohn C. Fendley, Jr. / John JCF/bgb enclosure(s) cc: Dr. Ken James (w/enclosures) F:\H0ME\BBrowD\Feodley\LK5]>desegkouiMel b.wpd1 Friday Eldredge & Clark HERSCHEL H. FRIDAY (1922-1994) WILLIAM H. SUTTON. P.A. BYRON M. BISEMAN. JR. P.A. JOE D. BELL, P.A. JAMES A. BUTTRY, P.A. FREDERICK S. URSERY, P.A. OSCAR E. DAVIS. JR. P.A. JAMES C. CLARK, JR.. P.A. THOMAS P. LEGGETT. P.A. JOHN DEWEY WATSON, P.A. PAUL B. BENHAM UI. P.A. LARRY W. BURKS, P.A. A. WYCKLIFF NISBET. JR. P.A. JAMES EDWARD HARRIS. P.A. J. PHILLIP MALCOM. P.A. JAMBS M. SIMPSON. P.A. JAMES M. SAXTON. P.A. J. SHEPHERD RUSSELL III. P.A DONALD H. BACON. P.A. WILLIAM THOMAS BAXTER P.A. RICHARD D. TAYLOR P.A. JOSEPH B. HURST. JR, P.A. ELIZABETH ROBBEN MURRAY. P.A. CHRISTOPHER HELLER, P.A. LAURA HENSLEY SMITH. P.A. ROBERT S. SHAFER. P.A. WILLIAM M. GRIFFIN III, P.A. MICHAELS. MOORE. P.A. DIANE S. MACKEY, P.A. WALTER M. EBEL III, P.A. KEVIN A. CRASS, P.A. WILLIAM A. WADDELL. JR, P.A. SCOTT J. LANCASTER P.A. ROBERT B. BEACH, JR, P.A. J. LEE BROWN. P.A. JAMES C. BAKER JR, P.A. HARRY A. LIGHT, P.A. SCOTT H. TUCKER P.A. GUY ALTON WADE. P.A. PRICE C. GARDNER P.A. TONIA F. JONES, P.A. DAVID D. WILSON. P.A. JEFFREY H. MOORE. P.A. DAVID M. GRAF, P.A. ATTORNEYS AT LAW A LIMITED LIABILITY PARTNERSHIP www.fridayflrm.cofn 2000 REGIONS CENTER 400 WEST CAPITOL LITTLE ROCK, ARKANSAS 72201-3493 TELEPHONE 501-376-2011 FAX 501-376-2147 3425 NORTH FUTRALL DRIVE. SUITE 103 FAYETTEVILLE, ARKANSAS 72703-4811 TELEPHONE 479-685-2011 FAX 478-695-2147 CARLA GUNNELS SPAINHOUR, P.A. JOHN C. FENDLEY, JR., P.A. JONANN ELIZABETH CONIGLIO. P.A. R. CHRISTOPHER LAWSON. P.A. FRAN C. HICKMAN. P.A. BETTY J. DEMORY, P.A. LYNDA M. JOHNSON. P.A. JAMES W. SMITH. P.A. CLIFFORD W. PLUNKETT. P.A. DANIEL L. HERRINGTON, P.A. MARVIN L. CHILDERS K. COLEMAN WESTBROOK. JR. ALLISON J. CORNWELL ELLEN M. OWENS JASON 6. HENDREN BRUCE B. TIDWELL MICHAEL E. KARNEY KELLY MURPHY MCQUEEN JOSEPH P. MCKAY ALEXANDRA A. IFRAH JAY T. TAYLOR MARTIN A. KASTEN BRYAN W. DUKE JOSEPH G. NICHOLS ROBERT T. SMITH RYAN A. BOWMAN TIMOTHY C. EZELL T. MICHELLE ATOR KAREN S. HALBERT SARAH M. COTTON PHILIP B MONTGOMERY KRISTEN S. RIGGINS ALAN G. BRYAN LINDSEY MITCHAM SLOAN KHAYYAM M. EDDINGS JOHN F. PEISERICH AMANDA CAPPS ROSE BRANDON J. HARRISON RECEIVED OCT - 4 2002 office OF desegregation monitoring 208 NORTH FIFTH STREET BLYTHEVILLE. ARKANSAS 72315 TELEPHONE 870-762-2696 FAX 870-762-2918 October 4,2002 OF COUNSEL B.S. CLARK WILLIAM L. TERRY WILLIAM L. PATTON. JR. H.T. LARZELERE. P.A. JOHNC. ECHOLS. P.A. A.D. MCALUSTBR JOHN C. FENDLEY, JR. LITTLE ROCK TEL 501-370-3323 PAX 501-244-5341 fndlyOfc.nt ( By Hand Delivery ) Mr. John W. Walker Mr. Sam Jones Mr. Steve Jones John W. Walker, P.A. 1723 Broadway Little Rock, Arkansas 72201 Wright, Lindsey & Jennings 2200 Worthen Bank Bldg. 200 West Capitol Little Rock, Arkansas 72201 Jack, Lyon & Jones, P.A. 425 W. Capitol, Suite 3400 Little Rock, Arkansas 72201 Mr. Richard Roachell Roachell Law Firm ( By Hand Delivery ) Ms. Ann Marshall 11800 Pleasant Ridge Road, #146 Desegregation Monitor 1 Union National Plaza Ms. Sammye Taylor Office of the Attorney General 323 Center Street P.O. Box 17388 Little Rock, Arkansas 72222 124 W. Capitol, Suite 1895 Little Rock, Arkansas 72201 200 Tower Building Little Rock, Arkansas 72201 RE: Compliance Remedy Dear Counsel & Ms. Marshall: Enclosed please find the Little Rock School Districts proposed Compliance Plan for implementing the courts Compliance Remedy set forth in the District Courts September 13,2002, Memorandum Opinion. Included in the Compliance Plan is a timeline for implementation. It contemplates submitting the Compliance Plan to the Board for approval at the October 10, 2002, agenda meeting. All parties are invited to attend the meeting and will be provided an opportunity to address the board. F:\HOME\BBrown\FcDdley\LRSD\deseg\couQsel LwpdLittle Rock School District Compliance Committee I Proposed Compliance Plan Revised Plan 2.7.1 RECEIVED OCT - 4 2002 OFFICE OF DESEGREGATION MONITORINGThe District Courts Compliance Remedy On September 13,2002, the District Court issued its Memorandum Opinion (hereinafter Opinion) finding that the Little Rock School District (LRSD) had substantially complied with all areas of the Revised Desegregation and Education Plan (Revised Plan), with the exception Revised Plan 2.7.1, Section 2.7.1 provided: LRSD shall assess the academic programs implemented pursuant to Section 2.7' after each year in order to determine the effectiveness of the academic programs in improving African-American achievement. If this assessment reveals that program has not and likely will not improve Afiican-American achievement, LRSD shall take appropriate action in the form of either modifying how the program is implemented or replacing the program a The District Courts Opinion set forth a detailed Compliance Remedy to be implemented by the LRSD. The Opinion first stated: Because LRSD failed to substantially comply with the crucially important obligations contained in 2.7.1, it must remain under court supervision with regard to that section of the Revised Plan until it
(a) demonstrates that a program assessment procedure is in place that can accurately measure the effectiveness of each program implemented under 2.7 in improving the academic achievement of Afincan-American students
and (b) prepares the program evaluations identified on page 148 of the Final Compliance Report and uses those evaluations as part of the program assessment procedure contemplated by 2.7.1 of the Revised Plan. The Opinion then outlined the details of the Compliance Remedy as follows
A. For the entire 2002-03 school year and the first semester of the 2003-04 school year, through December 31,2003, LRSD must continue to assess each of the programs implemented under 2.7 to improve the academic achievement of African-American students. LRSD now has over three years of testing data and other information available to use in gauging the effectiveness of those programs. I expect LRSD to use all of that available data and information in assessing the effectiveness of those programs and in deciding whether any of those programs should be modified or eliminated. 'Revised Plan 2.7 provided, LRSD shall implement programs, policies and/or procedures designed to improve and remediate the academic achievement of Afiican-American students, including but not limited to Section 5 of this Revised Plan. 1B. C. F. LRSD must maintain written records regarding its assessment of each of those programs. These written records must reflect the following information: (a) the written criteria used to assess each program during the 2002-03 school year and the first semester of the 2003-04 school year
(b) the results of the annual assessments of each program, including whether the assessments resulted in program modifications or the elimination of any programs
and (c) the names of the administrators who were involved -with the assessment of each program, as well as at least a grade level description of any teachers who were involved in the assessment process {e.g., all fourth grade math teachers
all eighth grade English teachers, etc.). LRSD must use Dr. Nunnerl/ or another expert from outside LRSD with equivalent qualifications and expertise to prepare program evaluations on each of the programs identified on page 148 of the Final Compliance Report. I will accept all program evaluations that have already been completed by Dr. Nunnerly or someone with similar qualifications and approved by the Board. All program evaluations that have not yet been completed on the remaining programs identified on page 148 of the Final Compliance Report must be prepared and approved by the Board as soon as practicable, but, in no event, later than March 15, 2003. In addition, as these program evaluations are prepared, LRSD shall use them, as part of the program assessment process, to determine the effectiveness of those programs in improving African-American achievement and whether, based on the evaluations, any changes or modifications should be made in those programs. In addition, LRSD must use those program evaluations, to the extent they may be relevant, in assessing the effectiveness of other related programs. * * On or before March 15,2004, LRSD must file a Compliance Report which documents its compliance with its obligations under 2.7.1. Any party, including Joshua, who wishes to challenge LRSDs substantial compliance with 2.7.1, as specified above, may file objections with the court on or before April 15,2004. Thereafter, I will decide whether the LRSD has substantially comphed with 2.7.1, as specified in the Compliance Remedy, and should be released from all further supervision and monitoring. ^The Court is clearly referring to Dr. John Nunnery. 2Proposed Compliance Plan As the Compliance Committee understands the District Courts Opinion, the Compliance Remedy requires the LRSD to: 1. 2. 3. 4. 5. Continue to administer student assessments through the first semester of2003-04
Develop written procedures for evaluating the programs implemented pursuant to Revised Plan 2.7 to determine their effectiveness in improving the academic achievement of Afiican-American students
Maintain written records of (a) the criteria used to evaluate each program
(b) the results of the annual student assessments, including whether an informal program evaluation resulted in program modifications or the elirmnation of any programs
and (c) the names of the administrators who were involved with the evaluation of each program, as well as at least a grade level description of any teachers who were involved m the evaluation process
Prepare a comprehensive program evaluation of each academic program implemented pursuant to Revised Plan 2.7 to determine its effectiveness in improving the academic achievement of African-American students and to decide whether to modify or replace the program
and Submit for Board approval the program evaluations identified on page 148 of the LRSDs Final Compliance Report that have been completed, and complete, with the assistance of an outside expert, the remaining evaluations identified on page 148 of the LRSDs Final Compliance Report. What follows is an explanation of how the Compliance Committee derived these five requirements from the District Courts Opinion, and what the Compliance Committee proposes to do to comply with each requirement. Assessment and Evaluation When first read, the District Courts Compliance Remedy seemed simple and straightforward, but as the Compliance Committee attempted to develop this Proposed Compliance Plan, numerous questions arose. The most fundamental question related to the District Courts use of the term assessment in Paragraphs A and B of the Compliance Remedy. The ambiguity of this term was the subject of testimony at the hearing. The District Court included in its Opinion Dr. Lesleys testimony on the difference between assessment and evaluation, see Opinion, p. 152, but it is unclear whether the Court accepted this testimony. 3It is clear that the District Court understood the distinction between testing data, which are derived from student assessments, and program evaluations, which are used to determine the effectiveness of programs. See Opinion, p, 152 (LRSD acknowledged in the Interim Comphance Report that it was required: (a) to use both the testing data and the program evaluations to detemune the effectiveness of the key academic programs implemented pursuant to 2,7 ,,, (emphasis in original)). Even so, the District Court appears to have used the term assessment in some instances to refer to only student assessments and in other instances to refer to both student assessments and evaluations. This required the Compliance Committee to determine the District Courts intended meaning. In making this determination, the Compliance Committee considered the context in which the term was used, the District Courts findings of fact as set forth in the Opinion, what would be in the best interest of African-American students, and hopefully, common sense. An explanation of each requirement of the Compliance Remedy is provided below. To avoid any ambiguity, Compliance Committee hereinafter uses the term assessment to refer to student assessments and the term evaluation to refer to the program evaluations, whether formal or informal. 1. Continue to administer student assessments through the first semester of 2003-04. This requirement derives from Paragraph A of the Compliance Remedy, Given Paragraph As reference to testing data, it seems clear that Paragraph A concerns, in part, student assessments. The Compliance Committee proposes to comply with this part of Paragraph A by implementing the 2002-03 Board-approved assessment plan. The 2002-03 Board-approved assessment plan incorporates four changes that have been made since the LRSDs Final Compliance Report, First, the Board eliminated the fall administrations of the Achievement Level Tests (ALTs) in 2001-02, The administration recommended this for three reasons: (1) the loss of instructional tune resulting from testing and test preparation
(2) fall results did not provide sigmficantly different information from the previous springs results
and (3) the cost of administering and scoring the tests. Second, the fall admimstration of the Observation Surveys and Developmental Reading Assessment will only be used by the teacher for diagnostic purposes. The scores will not be reported to or maintained by the LRSD, This change saves considerable time in test administration and allows more time for instruction. It was approved by the Board on September 26,2002, Third, the LRSD will no longer administer the ALTs. The administration recommended the complete elimination of the ALTs for the following reasons: (1) the lack of alignment with the content and format of the State Benchmarks
(2) the loss of instructional time resulting from 4testing and test admimstration
(3) the new federal accountability requirements in the No Child Left Behind Act require annual testing by the State in grades 3-8, making the LRSDs administration of the ALTs redundant
and (4) the costs of administering and scoring the tests. The Board approved this change on September 26,2002. Finally, the Arkansas Department of Education (ADE) has moved the administration of the SAT9 from the fall to the spring, effective 2002-03. The 2002-03 Board-approved assessment plan calls for the administration of the following student assessments in English language arts and mathematics: Kindergarten Grade 1 Grade 2 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Grades 7-10 Grades 9-11 Grade 10 Grade 11 Observation Surveys (5) Developmental Reading Assessment Observation Surveys (5) Development Reading Assessment Observation Surveys (3) Development Reading Assessment Norm-referenced test to be identified for gifted/talented screening Benchmark Literacy examination Benchmark Mathematics examination SAT9 Total Batteiy Benchmark Literacy examination Benchmark Mathematics examination SAT9 Total Battery Benchmark Literacy examination Benchmark Mathematics examination End-of Course Algebra I examination End-of Course Geometry examination SAT9 Total Battery End-of-Level Literacy examination All of these assessments are admimstered in the spring. Consequently, the final student assessment before March 15, 2004, will be administered in the spring of 2003. 2. Develop written procedures for evaluating the programs implemented pursuant to 2.7 to determine their effectiveness in improving the academic achievement of African-American students. This requirement derives from the opening paragraph of the Compliance Remedy. To comply with this requirement, two proposed regulations have been drafted, IL-Rl for formal evaluations and IL-R2 for informal evaluations, attached as Appendixes 1 and 2, respectively. 5as Proposed regulation IL-Rl combines generally accepted principles of program evaluation with practices that have been in place in the LRSD for the past two years. See, e.g,, Robby Champion, Map Out Evaluation Goals, Journal for Staff Development, Fall 2002, attached _ Appendix 3. This regulation will be subimtted to the Board, Office of Desegregation Monitoring ( ODM) and the Joshua Intervenors (Joshua) for review and comment before being finalized Proposed regulation IL-R2 specifically addresses the next requirement and is discussed therewith. 3. Maintain written records of (a) the criteria used to evaluate each program
(b) the results of the annual student assessments, including whether an informal program evaluation resulted in program modifications or the elimination of any programs
and (c) the names of the administrators who were involved with the evaluation of each program, as well as at least a grade level description of any teachers who were involved in the evaluation process. This requirement derives from Paragraph B of the Compliance Remedy. Paragraph B apparently came about as a result of the District Courts concern about the LRSD making program modifications based on informal evaluations of student assessment data. See Opinion, p. 155 ( I have grave reservations about anyone this side of Solomon being wise enough to use two or three semesters worth of erratic composite test scores to make reliable decisions about which remediation programs for LRSDs African-American students were actually working.). Proposed regulations IL-R2 was drafted to specifically address this requirement. It prohibits substantial program modifications from being made without a written record as required by Paragraph B. This regulation will also be submitted to ODM and Joshua for review and comment before being finalized. Proposed regulation IL-Rl also complies with this requirement. It mandates that the criteria used to formally evaluate a program be identified as the research questions to be answered, the first of which will be, Has this cuiriculum/instruction program been effective in improving and remediating the academic achievement of African-American students?. S^ Appendix 1, IL-Rl, p. 5. Recommended program modifications and the members of the evaluation team are routinely included in formal evaluations. As to the results of annual student assessments, the LRSD will continue to maintain a computer database with the results of annual students assessments administered pursuant to the Board-approved assessment plan. 64. Prepare a comprehensive program evaluation of each academic program implemented pursuant to 2.7 to determine its effectiveness in improving the academic achievement of African-American students and to decide whether to modify replace the program. or This requirement derives from Paragraph A of the Compliance Remedy. To comply with this requirement, the Compliance Committee proposes to prepare the following new, comprehensive evaluations: (a) Primary ReadingLanguage Arts, (b) Middle and High School Literacy and (c) K-12 Mathematics and Science. Each evaluation will be prepared in accordance with proposed Regulation IL-Rl and will incorporate all available student assessment data relevant to the program being evaluated. Based on Paragraph F of the Compliance Remedy, the LRSD understands these evaluations must be submitted to the Court on or before March 15 2004. Some may argue that Paragraph A and Paragraph C together require the LRSD to prepare new, comprehensive evaluations of all the programs identified on page 148 of the LRSDs Final Compliance Report. The Compliance Committee considered and rejected this argument for three reasons. First, Paragraph As description of the programs to be evaluated differs from that of Paragraph C. Paragraph A states that the LRSD must continue to assess each of the programs implemented under 2.7 . .. The Compliance Committee understands this to mean that the LRSD should continue to prepare evaluations of some of the key programs, as identified in the Interim Compliance Report. See Opinion, p. 151 (In addition to the Assessment Plan, 2.7.1 of the Interim Compliance Report noted that the LRSD was preparing ^"evaluations" of some of the key programs designed to improve Afifrcan-American achievement in order to provide a more in-depth look at the effectiveness of those programs. (emphasis in original)) In contrast to Paragraph A, Paragraph C requires the LRSD to prepare evaluations of each of the programs identified on page 148 of the Final Compliance Report. The Compliance Committee understands this to mean that the LRSD should complete all of the evaluations identified on page 148 of the Final Compliance Report and submit those to the Court. See Opinion, p. 156 ( [A]s of March 15,2001, the date the Final Compliance Report was filed with the Court: (1) PRE had prepared only draft evaluations of some of the programs in question
(2) none of those evaluations had been approved by the Board .... (emphasis in original)). The District Courts statement in Paragraph C that it will accept evaluations already completed and approved by the Board further indicates that Paragraph C does not require new, comprehensive evaluations. Second, recognizing this distinction between Paragraph A and Paragraph C resolves a potential conflict between Paragraph C and Paragraph F. Paragraph C provides, All program evaluations that have not yet been completed on the remaining programs identified on page 148 7of the Final Compliance Report must be prepared and approved by the Board as soon as practicable, but, in no event, later than March 15,2003. However, Paragraph F does not require the LRSD to file a compliance report on its compliance with Revised Plan 2.7.1 until March 15,2004. The Compliance Committee concludes that March 15,2004, is the deadline for submitting the new, comprehensive evaluations of the programs implemented pursuant to 2.7. S^ Paragraph A of Comphance Remedy. This is consistent with Paragraph As requirement that the LRSD include assessment data through December 31,2003. Obviously, such data could not be included in an evaluation filed on or before March 15,2003. Finally, it makes the most sense for the LRSD to expend the greatest tune and resources preparing evaluations of the programs designed to improve Afiican-American achievement. While the requirement for new, comprehensive evaluations derives from Paragraph A, some may argue that Paragraph Cs requirement that the LRSD use an outside expert to prepare evaluations of each of the programs identified on page 148 of the Final Compliance Report applies to the new, comprehensive evaluations. The Compliance Committee hopes the District Court and the parties agree that the team approach to program evaluation set forth in proposed regulation IL-Rl renders this argument moot. Proposed Regulation IL-Rl states that the program evaluation team must include [a]n external consultant with expertise in program evaluation, the program area being evaluated, statistical analysis, and/or technical writing .. . . Appendix 1, p. 4. The exact role of the external consultant may vary, depending upon the expertise required for the production of the program evaluation. Id. The Compliance Committee believes that the LRSDs practice over the last two years of using the team approach to program evaluation has produced credible evaluations. Moreover, participation of the LRSD staff on the evaluation team provides them an excellent learning experience that they do not typically receive when an evaluation is prepared entirely by an outside expert. The evaluations prepared over the last two years using the team approach are as follows: 1. 2. Dr. Steve Ross was the external consultant in the production of the Early Literacy program evaluation for 1999-2000 and 2000-01. He was asked to read a nearfinal draft and to provide feedback, which he did. His suggestions were then incorporated into the final report before it was published and disseminated. Other team members included Bonme Lesley (associate superintendent), Patricia Price (program director), Pat Busbea (program specialist), Ed Williams (statistician), and Ken Savage (computer programmer). Dr. Julio Lopez-Ferraro is the National Science Foundation (NSF) program officer who over-sees the LRSDs implementation of the grant-funded 8Comprehensive Partnership for Mathematics and Science Achievement ( CPMSA ). NSF trained a team of LRSD staff to produce the mandated annual program evaluations for this initiative and then assembled an external team of practitioners and researchers who came to the LRSD each year to validate findings and provide written feedback. The LRSD team members who our participated in writing of the annual progress reports included Vanessa Cleaver (project director), Dennis Glasgow (director of mathematics and science), Bonnie Lesley (associate superintendent and co-project investigator), Virginia Johnson (CPMSA program evaluator), Ed Williams (statistician), and Ken Savage (computer programmer). 3. 4. Mr. Mark Vasquez, an attorney and former employee of the Office for Civil Rights in Dallas, has been retained by the LRSD for the past three years to provide guidance in the design and production of the English as a Second Language (ESL) program evaluation. Other team members have been Bonnie Lesley (associate superintendent), Karen Broadnax (program supervisor), Ed Williams (statistician), Ken Savage (computer programmer), and Eddie McCoy (program evaluator). Dr. Larry McNeal, a professor at the University of Arkansas at Little Rock in education administration and a private consultant in program evaluation, was retained by the LRSD to lead the team that produced the program evaluation for the Charter School. Other members of that team included Linda Watson (assistant superintendent), Krista Young (program director), and Ed Williams (statistician). Dr. McNeal wrote this report. The team approach, supported by an external expert, ensures that all areas of expertise (program, implementation, techmeal and evaluative) are included. No one person would have all the knowledge and skills that a team would have. As these examples show, the external expert does not always perform the same role in every project. Rather, the role changes, depending the expertise that is required for a credible report. on 5. Submit for Board approval the program evaluations identified on page 148 of the LRSDs Final Compliance Report that have been completed, and complete, with the assistance of an outside expert, the remaining program evaluations identified on page 148 of the LRSDs Final Compliance Report. The follovnng program evaluations identified Report have been completed: on page 148 of the Final Compliance 1. Early Literacy. A comprehensive report for 1999-2000 and 2000-01 was prepared, completed, and presented to the Board in fall 2001. An update to this report for 2001-02 was presented to the Board in June 2002, with an emphasis on 9the improved achievement of African-American students and closing the achievement gap. 2. 3. 4. 6. 7. 8. 9. 10. Mathematics and Science. Three years (1998-99,1999-2000, and 2000-01) of program evaluations as required by the NSF were prepared, presented to the Board, and submitted to NSF, and NSF has responded to each evaluation. Extended Year Schools. The LRSD staff prepared, completed, and presented to the Board in the spring of2002 an evaluation of the Extended Year Schools. Elementary Summer School. The LRSD staff prepared, completed, and provided to the School Services Division an evaluation of elementary summer school programs for 2000-01. 5 HIPPY. The HIPPY program was evaluated by the LRSD staff in July 1999. The , report was prepared, completed, and submitted to the program director and the Cabinet. Charter School. This program evaluation was prepared, completed, and presented to the Board in June 2001. ESL. The Office for Civil Rights has required the LRSD to prepare a program evaluation in this area for each of the past three years
1999-2000, 2000-01, and 2001-02. The first two of these reports have been prepared, completed, submitted to the Board, and submitted to OCR. (A third program evaluation will be completed in October when state scores arrive and will be ready by the March 15, 2003 deadline). Lyceum Scholars Program. Two separate evaluations of this alternative education school program were prepared by the LRSD staff. Southwest Middle Schools SEDL Program. Southwest Middle School was the recipient of a two-year technical assistance grant from the Southwest Educational Development Lab (SEDL) to build professional community. SEDL prepared a comprehensive program evaluation that included Southwest among other grant recipients outside the LRSD. The LRSD staff provided SEDL data for this evaluation. Onward to Excellence (Watson Elementary). A grant from ADE funded a partnership between Watson Elementary and the Northwest Educational Development Lab to implement a school improvement initiative. The LRSD staff provided data to Watsons principal for preparation of program evaluations. The principal submitted two annual program evaluations to ADE. 1011. 12. Collaborative Action Team ("CAT). This one-year partnership with SEDL provided in 2000-01 for establishing and training a Collaborative Action Team of parent and community volunteers supported by LRSD staff to improve parent involvement. SEDL wrote a 249-page evaluation of their three-year grant-funded program, of which LRSD was included only the last year. The LRSD staff provided SEDL data for this evaluation. Vital Link. The LRSD staff prepared a program evaluation, and it was provided to the project director. A question arises as to which of these evaluations are acceptable to the Court without additional work. The first sentence of Paragraph C of the Compliance Remedy provides, LRSD must use Dr. Nunnerly (sic) or another expert from outside LRSD with equivalent qualifications and expertise to prepare program evaluations of each of the programs identified on page 148 of &e Final Compliance Report. The second sentence of Paragraph C states that the District Court will accept all program evaluations that have already been completed by Dr. Nunnerly (sic) or someone with similar qualifications. It is unclear whether an expert from outside the LRSD' must have prepared the completed evaluations for them to be accepted by the District Court, whether it is sufficient that they were prepared by someone within LRSD with similar qualifications. I l or The District Courts findings of fact suggest that the District Court will accept only program evaluations already completed by an outside expert. The District Court noted that Dr. Lesley testified that, by the end of November 2000, it was her opinion that no one in PRE had the expertise to prepare program evaluations. Opinion, p. 153. Thus, the District Court likely concluded that the only acceptable program evaluations would be those prepared by persons outside the LRSD. Applying this standard, the Compliance Committee believes that the following evaluations are acceptable to the Court, following Board approval, without additional work: Early Literacy, Mathematics and Science, Charter School, ESL, Southwest Middle Schools SEDL Program and CAT. The remaining program evaluations identified on the bottom of page 148 of the Final Compliance Report must be completed by an outside expert. They are: Extended Year Schools, Middle School Implementation, Elementary Summer School, HIPPY, Campus Leadership Teams (CLTs), Lyceum Scholars Program, Onward to Excellence and Vital Link. The Compliance Committees proposal for completing each of these evaluations will be discussed below. In deciding how to go about completing these evaluations, the Compliance Committee focused on what makes sense to do at this time considering the goal of improving African-American achievement and the limitations inherent in asking an expert to complete an evaluation. 11Extended Year Schools. This evaluation was completed by the LRSD staff. The Comphance Committee proposes retaining an outside expert to review the report and, if possible, draw conclusions and make recommendations based on the existing data. Middle School Implementation. A draft of this evaluation was presented to the Board in July and August 2000, but it was never completed. The Comphance Committee proposes retaining an outside expert to rewrite the report and, if possible, prepare an evaluation based on the existing data. Elementary Summer School. This evaluation was completed by the LRSD staff. The Comphance Committee proposes retaining an outside expert to review the report and, if possible, draw conclusions and make recommendations based on the existing Hata HIPPY. This evaluation was completed by the LRSD staff. The Comphance Committee proposes retaining an outside expert to review the report and, if possible, draw conclusions and make recommendations based on the existing data. CLTs. The LRSD staff conducted a survey of CLTs during 2000-01. A summary of the survey findings was presented during a CLT training session, but no formal report was ever prepared. The Comphance Committee proposes retaining an outside expert to review the survey data and, if possible, prepare an evaluation based on the existing survey data. Lyceum Scholars Program. This evaluation was completed by the LRSD staff. The Compliance Committee proposes retaining an outside expert to review the report and, if possible, draw conclusions and make recommendations based on the existing data. Onward to Excellence. This evaluation was completed by the LRSD staff. The Compliance Committee proposes retaining an outside expert to review the report and, if possible, draw conclusions and make recommendations based on the existing data. Vital Link. This evaluation was completed by LRSD staff The Comphance Committee proposes retaining an outside expert to review the report and, if possible, draw conclusions and make recommendations based on the existing data. 12Action Plan Timeline The Compliance Committee proposes implementation of this Compliance Plan in accordance with the following timeline. 1. Provide copies of this proposed Compliance Plan to ODM and Joshua for their reactions. 2. Incorporate, as possible, suggested revisions from ODM and Joshua. 3. Place Compliance Plan on the agenda for Board review and approval. 4. Place 2002-03 Program Evaluation Agenda on the Boards agenda for review and approval. 5. Place on Board agenda for approval two previously presented program evaluations (early literacy, and charter school). 6. Place on Board agenda for approval the evaluations of Southwest Middle Schools SEDL program and the Collaborative Action Team (also conducted by SEDL). 7. Place on Board agenda for approval the previously presented ESL program evaluations for 1999-2000 and 2000-01, plus the new evaluation for 2001-02. Week of September 30, 2002 Week of October 7, 2002 October 10,2002 October 24,2002 October 24,2002 November 2002 November 2002 13 I Clay Fendley Ken James Attorneys Ken James Compliance Team Ken James Attorneys Ken James Bonnie Lesley Bonnie Lesley Linda Watson Bonnie Lesley Bonnie Lesley Karen Broadnax8. Place on Board agenda for approval the three previously presented program evaluations for the NSF-funded CPMSA program, plus the new Year 4 report for 2001-2002. 9. Issue Request for Proposals (RFPs) from available external experts to review and complete the eight remaining program evaluations listed on page 148. _____________ 10. Form a screening team to determine recommendations to the Superintendent for designating external experts to review and complete the eight remaining program evaluations listed on page 148._________________ 11. Select and negotiate consulting contracts with designated external experts. 12. Assign appropriate staff to each external expert to provide needed information, data, access to program staff, etc. 13. Monitor the work to ensure timely completion. 14. As each paper is completed and ready for circulation, send copies to ODM and Joshua for their review and comments. December 2002 Mid-October 2002 Late October 2002 Mid-November 2002 Mid-November 2002 Mid-November 2002^February 2003 December 2002February 2003 14 Bonnie Lesley Vanessa Cleaver Dennis Glasgow Bonnie Lesley Darral Paradis Ken James Compliance Team Bonnie Lesley Ken James Bonnie Lesley Bonnie Lesley Bonnie Lesley15. As each paper is completed, place on the Boards agenda the item to be reviewed and approved. 16. Write Interim Compliance Report relating to programs on page 148 to be completed. 17. Establish staff teams for each of the three programs on the Boards Program Evaluation Agenda to be completed fori2002-2003 (Elementary Literacy, Secondary Literacy, and K- 12 Mathematics/ Science). 18. Publish RFPs to identify external experts to serve on each of the two staff teams for the Boards Program Evaluation Agenda (K-12 mathematics/science external experts are provided by NSF). 19. Establish consulting contracts with the two external experts required for the Elementary Literacy and Secondary Literacy program evaluations. 20. Train each program evaluation team, including the external expert, on the requirements of the approved Compliance Plan and IL-R. December 2002February 2003 March 15,2003 March 1, 2003 March 1, 2003 Late March 2003 May 2003 15 Ken James Bonnie Lesley Attorneys Compliance Committee Bonnie Lesley Bonnie Lesley Darral Paradis Bonnie Lesley Bonnie Lesley21. Monitor the completion of the work on all three program evaluations required in the Boards Program Evaluation Agenda. 22. Send copies of the completed Elementary Literacy program evaluation to ODM and Joshua for information. 23. Complete the evaluation of the Elementary Literacy program and place on the Boards agenda for approval. 24. Send copies of the Secondary Literacy program evaluation to ODM and Joshua for information. 25. Complete the evaluation of the Secondary Literacy program and place on the Boards agenda for approval. 26. Send copies of the completed CPMSA program evaluation to ODM and Joshua for information. 27. Complete the five-year evaluation of the CPMSA project (science and mathematics) and place on the Boards agenda for approval. 28. Write Section 2.7.1 Final Compliance Report for federal court and file with Court. MayOctober 2003 With October 2003 Board agenda packet October board meeting, 2003 With November 2003 Board agenda packets November board meeting, 2003 With December 2003 Board agenda packet December board meeting, 2003 March 15, 2004 16 Bonnie Lesley Ken James Bonnie Lesley Bonnie Lesley Pat Price Ken James Bonnie Lesley Bonnie Lesley Pat Price Ken James Bonnie Lesley Bonnie Lesley Vanessa Cleaver Dennis Glasgow Ken James Attorneys Compliance TeamI Appendix 1 Proposed IL-RlLITTLE ROCK SCHOOL DISTRICT NEPN CODE: IL-R1 PROGRAM EVALUATION AGENDA Purpose The purpose of these regulations is to provide guidance to the staff involved in the evaluation of programs required in the Boards Program Evaluation Agenda. They do not necessarily apply to grant-funded programs if the funding source requires other procedures and provides funding for a required evaluation. Criteria for Program Evaluations Policy IL specifies that the evaluations of programs approved in its Board- approved Program Evaluation Agenda shall be conducted according to the standards developed by the Joint Committee on Standards for Educational Evaluation. (See Joint Committee on Standards for Educational Evaluation, James R. Sanders, Chair (1994). The Program Evaluation Standards, 2^^ Edition: How to Assess Evaluations of Educational Programs. Thousand Oaks, CA: Sage Publications.) They are as follows: Utility Standards The utility standards are intended to ensure that an evaluation will serve the information needs of intended users. These standards are as follows: Stakeholder identification. People involved in or affected by the evaluation should be identified so that their needs can be addressed. Evaluator credibility. The people conducting the evaluation should be both trustworthy and competent to perform the evaluation so that the evaluation findings achieve maximum credibility and acceptance. Information scope and sequence. Information collected should be broadly selected to address pertinent questions about the program and should be responsive to the needs and interests of clients and other specified stakeholders. Values identification. The perspectives, procedures, and rationale used to interpret the findings should be described carefully so that the bases for value judgements are clear. Report clarity. Evaluation reports should describe clearly the program being evaluated, including its context and the purposes, procedures, and findings of the evaluation, so that essential information is provided and understood easily. 1Report timeliness and dissemination. Significant interim findings and evaluation reports should be disseminated to intended users so that they can be used in a timely fashion. Evaluation impact. Evaluations should be planned, conducted, and reported in ways that encourage follow-through by stakeholders, so that the likelihood that the evaluation will be used is increased. Feasibility Standards Feasibility standards are intended to ensure that an evaluation will be realistic, prudent, diplomatic, and frugal. Practical procedures. Evaluation procedures should be practical so that the disruption is kept to a minimum while needed information is obtained. Political viability. The evaluation should be planned and conducted with anticipation of the different positions of various interest groups j so that their cooperation may be obtained, and so that possible attempts by any of these groups to curtail evaluation operations or to vias or misapply the results can be averted or counteracted. Cost-effectiveness. The evaluation should be efficient and produce information of sufficient value so that the resources expended can be justified. Propriety Standards The propriety standards are intended to ensure that an evaluation will be conducted legally, ethically, and with due regard for the welfare of those involved in the evaluation, as well as those affected by its results. Service orientation. Evaluations should be designed to assist organizations to address and effectively serve the needs of the full range of targeted participants. Formal agreements. Obligations of the formal parties to an evaluation (what is to be done, how, by whom, and when) should be agreed to in writing so that these parties are obligated to adhere to all conditions of the agreement or to formally renegotiate it. Rights of human subjects. Evaluations should respect human dignity and worth in their interactions with other people associated with an evaluation so that participants are not threatened or harmed. Complete and fair assessments. The evaluation should be complete and fair in its examination and recording of strengths and weaknesses of the program being evaluated so that strengths can be built upon and problem areas addressed. Disclosure of findings. The formal parties to an evaluation should ensure that the full set of evaluation findings, along with pertinent limitations, are made accessible to the people affected by the 2evaluation, as well as any others with expressed legal rights to receive the results. Conflict of interest. Conflict of interest should be dealt with openly and honestly so that it does not compromise the evaluation processes and results. Fiscal responsibility. The evaluators allocation and expenditure of resources should reflect sound accountability procedures and be prudent and ethically responsible so that expenditures are accounted for and appropriate. Accuracy Standards Accuracy standards are intended to ensure that an evaluation will reveal and convey technically adequate information about the features that determine the worth of merit of the program being evaluated. Program documentation. The program being evaluated should be described and documented clearly and accurately so that it programs is identified clearly. Context analysis. The context in which the program exists should be examined in enough detail so that its likely influences on the program can be identified. Described purposes and procedures. The purposes and procedure of the evaluation should be monitored and described in enough detail so that they can be identified and assessed. Defensible information sources. The sources of information used in a program evaluation should be described in enough detail so that the adequacy of the information can be assessed. Valid information. The information-gathering procedures should be chosen or developed and then implemented in a manner that will ensure that the interpretation arrived at is valid for the intended use. Reliable information. The information-gathering procedures should be chosen or developed and then implemented in a manner that will ensure that the information obtained is sufficiently reliable for the intended use. Systematic information. The information collected, processed, and reported in an evaluation should be review systematically so that the evaluation questions are answered effectively. Analysis of quantitative information. Quantitative information in an evaluation should be analyzed appropriately and systematically so that the evaluation questions are answered effectively. Analysis of qualitative information. Qualitative information in an evaluation should be analyzed appropriately and systematically that the evaluation questions are answered effectively. so Justified conclusions. The conclusions reached in an evaluation should be justified explicitly so that stakeholders can assess them. 3Impartial reporting. Reporting procedures should guard against distortion caused by personal feelings and biases of any party the evaluation reports reflect the evaluation findings fairly. Metaevaluation. The evaluation itself should be evaluated so formatively and summartively against these and other pertinent standards so that its conduct is appropriately guided, and on completion, stakeholders can closely examine its strengths and weaknesses. Program Evaluation Procedures The following procedures are established for the evaluation of programs approved by the Board of Education in its annual Program Evaluation Agenda: 1. 2. 3. The Division of Instruction shall recommend to the Superintendent annually, before the budget for the coming year is proposed, the curriculum/instruction programs for comprehensive program evaluation. The recommendation shall include a proposed budget, a description of other required resources, and an action plan for the completion of the reports. Criteria for the proposed agenda are as follows: A. Can the results of the evaluation influence decisions about the program? B. Can the evaluation be done in time to be useful? C. Is the program significant enough to merit evaluation? (See Joseph S. Wholey, Harry P. Hatry, and Kathryn Newcomer (1994). Handbook of Practical Program Evaluation. San Francisco, CA: Jossey- Bass Publishers. 5-7.) The Superintendent shall recommend to the Board of Education for approval the proposed Program Evaluation Agenda^with anticipated costs and an action plan for completion. For each curriculum/instruction program to be evaluated as per the Program Evaluation Agena, the Associate Superintendent for Instruction shall establish a staff team with a designated leader to assume responsibility for the production of the report according to the timelines established in the action plan approved by the Board of Education. 4. Each team shall include, at a minimum, one or more specialists in the curriculum/instruction program to be evaluated, a statistician, a programmer to assist in data retrieval and disaggregation, and a technical writer. If additional expertise is required, then other staff may be added as necessary. 5. An external consultant with expertise in program evaluation, the program area being evaluated, statistical analysis, and/or technical writing shall be retained 4as a member of the team. The role of the external consultant may vary, depending upon the expertise required for the production of the program evaluation. 6. The team leader shall establish a calendar of regularly scheduled meetings for the production of the program evaluation. The first meetings will be devoted to the following tasks: A. B. C. Provide any necessary training on program evaluation that may be required for novice members of the team, including a review of the Boards policy IL and all of the required criteria and procedures in these regulations, IL-R. Assess the expertise of each team member and make recommendations to the Associate Superintendent for Instruction related to any additional assistance that may be required. Write a clear description of the curriculum/instruction program that is to be evaluated, with information about the schedule of its implementation. D. Agree on any necessary research questions that need to be established in addition to the question, Has this curriculum/instruction program been effective in improving and remediating the academic achievement of African-American students? (See Policy IL, 2.7.1 of the Revised Desegregation and Education Plan, and Judge Wilsons Compliance Remedy.) E. Generate a list of the data required to answer each research question, and assign responsibility for its collection and production. All available and relevant student performance data must be included. (See Judge Wilsons Compliance Remedy.) F. Decide who will be the chief writer of the program evaluation. G. Plan ways to provide regular progress reports (e.g., dissemination of meeting minutes, written progress reports, oral reports to the Superintendents Cabinet and/or Compliance Team) to stakeholders, including the Associate Superintendent for Instruction, the Superintendent of Schools, the Office of Desegregation Monitoring (until Unitary Status is achieved), and the Joshua Intervenors (until Unitary Status is achieved). (See Joellen Killion (2002). Assessing Impact: Evaluating Staff Development. Oxford, OH. National Staff Development Council (NSDC)
Robby Champion (Fall 2002). Map Out Evaluation Goals. Journal of Staff Development. 78-79
5Thomas R. Guskey (2000). Evaluating Professional Development. Thousand Oaks, CA: ConA/in Press
Blaine R. Worthen, James R. Sanders, and Jody L. Fitzpatrick (1997). Participant-Oriented Evaluated Approaches. Program Evaluation: Alternative Approaches and Practical Guidelines
153-169
Beverly A. Parsons (2002). Evaluative Inquiry: Using Evaluation to Promote Student Success. Thousand Oaks, CA: Corwin Press
and Joseph S. Wholey, Harry P. Hatry, and Kathryn E. Newcomer (1994). Handbook of Practical Program Evaluation. San Francisco, CA: Jossey-Bass Publishers.) 7. Subsequent meetings of the program evaluation team are required for the following tasks: to monitor the completion of assignments
to collaborate in the interpretation and analysis of data
to pose any necessary new questions to be answered
to review drafts and provide feedback to the writer
to formulate recommendations, as required, for program improvement, especially to decide if a recommendation is required to modify or abandon the program if the findings reveal that the program is not being successful for the improvement of African- American achievement: to assist in final proofreading
and to write a brief executive summary, highlighting the program evaluation findings and recommendations. 8. A near-final copy of the program evaluation must be submitted to the Associate Superintendent for Instruction at least one month before the deadline for placing the report on the Boards agenda for review and approval. This time is required for final approval by staff, for final editing to ensure accuracy, and for submission to the Superintendent. 9. When the program evaluation is approved for submission to the Board of Education for review and approval, copies of the Executive Summary and complete report must be made for them, for members of the Cabinet, for ODM (until Unitary Status is achieved), and for the Joshua Intervenors (until Unitary Status is achieved). 10.The program evaluation team shall plan its presentation to the Board of Education on the findings and recommendations. 611 .The Associate Superintendent for Instruction shall prepare the cover memorandum to the Board of Education, including all the required background information (see Judge Wilsons Compliance Remedy)
A. B. C. D. If program modifications are suggested, the steps that the staff members have taken or will take to implement those modifications. If abandonment of the program is recommended, the steps that will be taken to replace the program with another with more potential for the improvement and remediation of African-American students. (See Section 2.7.1 of the Revised Desegregation and Education Plan and Judge Wilsons Compliance Remedy.) Names of the administrators who were involved in the program evaluation. Name and qualifications of the external expert who served on the evaluation team. Grade-level descriptions of the teachers who were involved in the assessment process (e.g., all fourth-grade math teachers, all eighth grade English teachers, etc.). 10. When the program evaluation is approved by the Board of Education, the team must arrange to have the Executive Summary and the full report copied and design a plan for communicating the program evaluation findings and recommendations to other stakeholders. This plan must then be submitted to the Associate Superintendent for approval. 11. Each program evaluation team shall meet with the Associate Superintendent for Instruction after the completion of its work to evaluate the processes and product and to make recommendations for future program evaluations. (See Joellen Kiliion (2002). Evaluate the Evaluation. Assessing Impact: Evaluating Staff Development. Oxford, OH: National Staff Development Council. 46, 123-124.) 7Appendix 2. Proposed IL-R2LITTLE ROCK SCHOOL DISTRICT NEPN CODE
IL-R2 INFORMAL PROGRAM EVALUATION Introduction The purpose of this regulation is to ensure that a written record exists explaining a decision to significantly modify an academic program. It is not the intent of this regulation to require a formal program evaluation before every significant program modification. Definitions Academic Program means one of the core curriculum programs of English/Language Arts, Mathematics, Science or Social Studies. Significantly modify means a material change in the content or delivery of an academic program implemented throughout the entire District. Written Record A written record must be prepared and maintained explaining a decision to significantly modify an academic program. The written record required by this regulation must include the following information: (a) the written criteria used to evaluate the program
(b) a summary of the student assessment data or other data on which the decision was based
and (c) the names of the administrators who were involved with the evaluation of each program, as well as at least a grade level description of any teachers who were involved in the evaluation process (e.g., all fourth grade math teachers
all eighth grade English teachers etc.). 1Appendix 3 Robby Champion, Map Out Evaluation Goals, Journal for Staff Development, Fall 2002 99I a It i n g m e a s ROBBY CHAMPION Map out evaluation goals A master plan can guide you down the rocky path of evaluation w 'hen you launch a major professional development evaluation, regardless of the projects scope, you may quickly find yourself on a slippery, often rocky road, with twists and unexpected turns. Before venturing too far and becoming disillusioned about program evaluation, create a master plan. While it requires an upfront investment of time and may delay starting, it quickly becomes an invaluable road map that helps you avoid delays and detours along the way. Developing an evaluation master plan is most useful when you are launching a major, summative program evaluation. A summative evaluation is done at major junctures in a programs life cycle and emphasizes documenting impact. Information from summative evaluations is used to make important decisions about the initiative, such as whether to continue, alter, expand, downsize, or eliminate it. A formative evaluation, on the other hand, means monitoring and collecting data, often informally and spontaneously, throughout program implementation. Formative evaluation helps show implementers where to make adjustments so a program can eventually achieve significant results. A thoughtfully prepared master plan for a major evaluation effort would: Focus the evaluation effort and help implementers avoid being sidetracked by leadership changes and new opinions
Create a realistic timeline and work plan that Robby Champion is president of Champion Training & Consulting. You can contact her at Champion Ranch at TnimbeD Canyon, Mora. NM 87732, (505) 387-2016, fax (505) 387-5581, e-mail: RobbychampioD@aol.com. provides needed momentum for the work
Be a key informational document to provide an overview and answer specific questions throughout the process
Help recruit people to assist with the project on the myriad evaluation tasks
Give the message that the evaluation wUl be open and not secretive. Whether your evaluation must be completed within a few months or will extend for several years, think through four phases of work before starting. PHASE I: ORGANIZE THE PROCESS 1. Form a steering committee, including any needed outside expertise. 2. Learn more about program evaluation together. 3. Write a clear description of each program to be evaluated. 4. Agree on the primary purpose of the evaluation. .5. Plan how you will keep everyone informed along the way. Steering committees, charged specifically with program evaluation, are important to focus attention and maintain the energy and momentum needed for the evaluation. They also help build a spirit of collaboration and open inquiry. And they keep the evaluation on track when other priorities might push the effort aside. Provide steering committee members with the tools to succeed. Members need not be evalu- 78 National Staff Development Council JSD Fall 2002I i I i I m e a s u r e ation-experts, but they do need information, support, and guidance to make informed decisions. They need background material to learn about program evaluation and examples of good evaluation studies. Finally, they need access to experts on professional development, measurement, and the content areas of the training programs. Before launching any evaluation effort, have a written description of each program to be evaluated. You would be amazed at the number of people who do not have a clear idea of what you mean by the New TeacherJnduction Program or the Early Literacy Initiative since so many different initiatives are being undertaken simultaneously around the school or district. PHASE II: DESIGN THE EVALUATION 1. Generate questions to guide the evaluation. 2. Generate potential data sources/ instruments to address the questions. 3. Using a matrix to provide a birds-eye view, agree on the most important questions and the best data sources. 4. Decide if collecting data from a sample group is warranted to make the evaluation manageable. 5. Determine the evaluation approach that makes sense: quantitative vs. quahtative/naturalistic. 6. Gather or create the instruments for data collection. 7. Determine a realistic schedule for collecting data. 8. Create a system for collecting, analyzing, and interpreting data. Decisions made in Phase n are critical. They determine the technical quality of your evaluation. In the questions you select, you determine what to examine and what to ignore. When you finish with the design phase, your program evaluation will be shaped to use a quantitative or a , qualitative model or a mixture of the two. In the design phase, you make other major decisions, such as whether to use a sample group. You also decide whether to do an in-dep case study, whether to ON THE WEB. See an example of a matrix to help guide evaluations at: www.nsdc.org/libraryrjsd/ champion234.html. survey the whole population, whether to use examples of student work instead of official documents such as student grades or standardized test scores, or whether to judge adult learners understanding of the training content with performance tasks during training or by exit tests, classroom observations, or student feedback. If the programs to be evaluated already have stated indicators of longterm impact, generating appropriate evaluation questions is much simpler than when programs have only vague, lofty goals. The steering committee may drift into the realm of program planning as you encounter hurdles tike fuzzy program outcomes. To avoid making misinformed evaluation design decisions, involve program leaders in your discussions. Developing or gathering instruments and then collecting the data are the most expensive steps in any evaluation. Think strategically about which data to collect, from whom to collect it or where to find it, and the best time to collect it. Your organization may already be collecting data for another purpose that now can be used for program evaluation. Some pubhc records, such as student attendance, may be valuable if, for example, 20% increase in student attendance at all grade levels is one of your programs indicators of impact PHASE III: PREPARE TO REPORT 1. Determine which audiences will want to know the results. 2. Consider several forums and formats to disseminate the results. 3. Plan reports, presentations, photo displays, graphs, charts, etc. Remember that your job is to make the evaluation results useful to your organization, so consider a range of ways to provide information to various groups. Consider briefs in the school or district newsletter, a handout updating staff about the schedule for data collection, five- minute progress updates in faculty meetings, bulleted statements on your web site, a digital picture- album of the programs results in classrooms with photos of students, and hallway displays of student work. If your final report is a formal document complete with examples of your data collection instruments, consider writing an executive summary of five pages or less to help,readers get the essential information. PHASE IV: CREATE THE WORK PLAN 1. List all tasks to be completed for the whole evaluation. 2. Create a realistic timeline. 3. Assign work. 4. Distribute the master plan. You will have to be creative to accomphsh all the evaluation tasks. In education, we rarely have the luxury of contracting outsiders for the entire project Enlist steering committee members, partners, graduate students from the local university, and other talented critical friends to get the work done. One caution: For formal or summa- tive evaluations to be credible, avoid using insiders such as the program designers or implementers (coaches, mentors, trainers, or facilitators) to perform critical evaluation tasks that call for objectivity and distance. And be sure to get ongoing, high-quality technical expertise for the critical technical analysis. A CATALYST FOR REFLECTION Completing a major program evaluation usually serves as the catalyst for serious reflection on the current designs, policies, and practices of your professional development programs their goals, content, processes, and contexts. In fact, revelations are often so powerful that they bring about the realization that major changes are needed if significant results' are really expected from professional development. People frequently conclude that designing the evaluation should be the first step in the program planning process, rather than an afterthought during implementation. E I I JSD Fall 2002 National Staff Development Council 79Office of Desegregation Monitoring United States District Court Eastern District of Arkansas Ann S. Marshall, Federal Monitor One Union National Plaza 124 West Capitol, Suite 1895 Little Rock, Arkansas 72201 (501)376-6200 Fax (501) 371-0100 October 7, 2002 Dr. Ken James, Superintendent Little Rock School District 810 West Markham Street Little Rock, AR 72201 Dear Ken: I enjoyed our telephone chat today and look forward to meeting for coffee and more conversation on October 31 (all treats, no tricks!) Thanks for understanding our frustration at receiving the districts Proposed Compliance Plan at noon last Friday, October 4, 2002, giving us very few days in which to respond by your deadline of this coming Thursday, October 10'\ Although youd been kind to notify me September 23' that your proposal would be forthcoming in the near future, Id not known exactly when to expect it. As a result, were trying to squeeze our review into an already crowded schedule. rd As I explained, the tight timeline will preclude as thorough a review of your plan as we might otherwise have been able to do. Nevertheless, as you and I agreed, well write up some of our preliminary observations and questions for you to factor into your deliberations. My associate. Gene Jones, has been monitoring program evaluation in the district and will continue to do so. Unless otherwise notified, we will assume that our primary contacts for information are those named in the Action Plan Timeline. We intend to do our best in continued support of the district, and will always appreciate ample lead time for our input. Thank you very much. Sincerely yours, Ann S. Marshall Office of Desegregation Monitoring United States District Court Eastern District of Arkansas Ann S. Marshall, Federal Monitor One Union National Plaza 124 West Capitol, Suite 1895 Little Rock, Arkansas 72201 (501)376-6200 Fax (501) 371-0100 Date: October 10, 2002 To: Dr. Ken James, LRSD Frorn^^ni in Marshall, ODM Re: LRSDs Evaluation Compliance Plan Thank you for forwarding the LRSDs Proposed Compliance Plan, Revised Plan 2.7.1, which was delivered to us last Friday, October 4, 2002. ODM is happy to provide comments when requested, and we will appreciate being accorded adequate time in which to do so. Our feedback, attached, is limited as a result of the very short lead time. We hope the enclosed observations and questions will be helpful as you fine-tune your compliance plan. We look forward to working with you over the coming months as we monitor implementation of the plan. Please let us know how we may be of assistance. Enc. cc: Judge William R. Wilson Judge J. Thomas Ray All Counsel of RecordOctober 10, 2002 ODMs Observations and Questions LRSD Proposed Compliance Plan, Revised Plan 2.7.1 1. Of the eight program evaluations the district proposes to have re-written by outside experts (page 12), will new data for continuing programs be added to those evaluations, or will the re-writes be limited to the data that were used at the time the eight evaluations had been written? 2. Page 7 of the Proposed Compliance Plan identifies Primary Reading/Langiiage Arts as a program to be evaluated as part of the evaluation agenda, while the term Elementary Literacy is used on page 15. The first term connotes PK-3, while the second connotes K-5. Do the different titles actually identify different programs? Which grades are to be included in the evaluation? 3. Another evaluation agenda item is K-12 Mathematics and Science. The list of assessments on page 5 includes nothing for math and science in grades K-3. What data will be used in evaluating the primary grades, considering the lack of assessments in math and science at that level? 4. 5. 6. 7. 8. In the evaluation of primary literacy, what measures will the district institute to off-set potential bias or conflict-of-interest inherent in the process of teachers administering assessments to their own students? The proposed regulation, IL-Rl, requires a clear description of each program that is to be evaluated. To what extent will those descriptions include not only the subject content that students are expected to learn, but also teaching methods, materials, time allotment, and so on? The proposed regulation requires that all relevant student performance data will be used in the evaluations. Will other data, such as interviews with teachers or students, case studies, and classroom observations, supplement the performance data for the evaluation? The degree and quality of program implementation have received little attention in previous evaluations. What procedures will the LRSD develop to measure program implementation (such as its quality, uniformity, and completeness throughout the district) in order to assure that the student performance data reported in the evaluations are actually the result of the programs described? The compliance plan section headed, Continue to administer student assessment through the first semester of 2003-04 (page 4) details how the LRSD has recently altered its assessment plan. The 2002-03 assessment plan, board-approved on September 26, 2002, provides only for spring testing, as it eliminates all fall testing that is reported to or maintained by the LRSD and, of course, incorporates ADEs move of SAT-9 testing to the spring. Page 5 of this section reads that the final student assessment before March 15, 2004 will be administered in the spring of 2003. We note that the spring 2003 tests will not only be the final assessment, but also the only student assessment reported to or maintained by the LRSD between now and March 15, 2004. Will the data from this one testing cycle be complemented by that of previous years?9. We note that the new assessment plan includes administration of student assessments only in English language arts and mathematics. Also, the 3'* grade has no assessments. 10. 11. 12. Given the assignments in the Action Plan Timeline, what is the role of the districts Planning, Research and Evaluation department under the new compliance plan? Under the envisioned Program Evaluation Agenda? The proposed regulation, IL-R2, Informal Program Evaluation, requires that a written record be prepared and maintained to support any decision to modify an academic program. The purpose of this regulation is unclear. Are the program evaluation standards relevant to informal evaluations? Does the regulation mean that LRSD will informally evaluate all programs? Or, will suspect programs be targeted for informal evaluation? Or, is the regulation intended to protect programs from unfair criticism? How is the LRSD planning to alter its budget to purchase the services of outside evaluation experts? Which budget items will be reduced to accommodate the purchases of evaluation services?Friday Eldredge & Clark HERSCHEL H. FRIDAY (1932-1994) WILLIAM H. SUTTON, P.A. BYRON M. EISEMAN. JR.. P.A. JOE D. BELL, P.A. JAMES A BUTTRY, P.A FREDERICK S. URSERY, P.A OSCAR E. DAVIS. JR., P.A. JAMBS C. CLARK, JR., P.A. THOMAS P. LEGGETT. P.A. JOHN DEWEY WATSON, P.A PAUL B. BENHAM III, P.A. LARRY W. BURKS, P.A. A. WYCKLIFP NISBET, JR., P.A. JAMES EDWARD HARRIS. P.A. J. PHILLIP MALCOM, P.A. JAMES M. SIMPSON. P.A. JAMES M. SAXTON. P.A. J. SHEPHERD RUSSELL III. P.A DONALD H. BACON. P.A WILLIAM THOMAS BAXTER, P.A RICHARD D. TAYLOR, P.A. JOSEPH B. HURST. JR.. P.A. ELIZABETH ROBBEN MURRAY. P.A. CHRISTOPHER HELLER. P.A. LAURA HENSLEY SMITH. P.A. ROBERT S. SHAPER. P.A. WILLIAM M. GRIFFIN Ill. P.A. MICHAEL S. MOORE. P.A. DIANS S. MACKEY. P.A. WALTER M. EBEL HI. P.A. KEVIN A. CRASS. P.A. WILLIAM A. WADDELL. JR.. P.A. SCOTT J. LANCASTER. P.A. ROBERT B. BEACH. JR.. P.A. J. LEE BROWN. P.A. JAMES C. BAKER. JR.. P.A. HARRY A. LIGHT. P.A. SCOTT H. TUCKER. P.A. GUY ALTON WADE. P.A. PRICE C. GARDNER. P.A. TONIA P. JONES. P.A. DAVID D. WILSON. P.A. JEFFREY H. MOORE. P.A. DAVID M. GRAP. P.A. ATTORNEYS AT LAW A LIMITED LIABILITY PARTNERSHIP CARLA GUNNELS SPAINHOUR. P.A. JOHN C FENDLEY. JR.. P.A. BRYAN W, DUKE JOSEPH G. NICHOLS JONANN ELIZABETH CONIGLIO, P,A. ROBERT T. SMITH www.frldayfirm.com 2000 REGIONS CENTER 400 WEST CAPITOL LITTLE ROCK. ARKANSAS 72201-3493 TELEPHONE 501-376-2011 FAX 501-376-2147 3425 NORTH FUTRALL DRIVE. SUITE 103 FAYETTEVILLE. ARKANSAS 72703-4811 TELEPHONE 479-805-2011 FAX 479-605-2147 R. CHRISTOPHER LAWSON, P.A. FRAN C. HICKMAN. P.A. BETTY J. DEMORY. P.A. LYNDA M. JOHNSON. P.A. JAMES W. SMITH, P.A. CLIFFORD W. PLUNKETT. P.A. DANIEL L. HERRINGTON. P.A. MARVIN L. CHILDERS K. COLEMAN WESTBROOK. JR. ALLISON J. CORNWELL ELLEN M. OWENS JASON B. HENDREN BRUCE B. TIDWELL MICHAEL E. KARNEY KELLY MURPHY MCQUEEN JOSEPH P. MCKAY ALEXANDRA A. IF RAH JAY T. TAYLOR MARTIN A. KASTEN RYAN A BOWMAN TIMOTHY C. EZELL T. MICHELLE ATOR KAREN S. HALBERT SARAH M. COTTON PHILIP B. MONTGOMERY KRISTEN S. RIGGINS ALANG. BRYAN LINDSEY MITCHAM SLOAN KHAYYAM M. EDDINGS JOHN F. PEISERICH AMANDA CAPPS ROSE BRANDON J. HARRISON OFCOUNSEL B.S. CLARK WILLIAM L. TERRY WILLIAM L. PATTON. JR. H.T. LARZELERE, P.A. JOHN C. ECHOLS. P.A. A.D. MCALLISTER RECEIVED 208 NORTH FIFTH STREET BLYTHEVILLE. ARKANSAS 72315 TELEPHONE 870-762-2808 FAX 870-762-2918 OCT 1 1 2002 OFFICE OF DESEGREGATION MONITORING October 11,2002 JOHN C. FENDLEY. JR. LITTLE ROCK TEL 501-970-3323 FAX 501-244-5341 fandltyQf^c.ntt ( By Hand Delivery ) Mr. John W. Walker Mr. Sam Jones Mr. Steve Jones JohnW. Walker, P.A. 1723 Broadway Little Rock, Arkansas 72201 Wright, Lindsey & Jennings 2200 Worthen Bank Bldg. 200 West Capitol Little Rock, Arkansas 72201 Jack, Lyon & Jones, P.A. 425 W. Capitol, Suite 3400 Little Rock, Arkansas 72201 Mr. Richard Roachell Roachell Law Firm ( By Hand Delivery ) Ms. Ann Marshall Mr. Dennis Hanson 11800 Pleasant Ridge Road, #146 Desegregation Monitor 1 Union National Plaza Office of the Attorney General 323 Center Street P.O. Box 17388 Little Rock, Arkansas 72222 124 W. Capitol, Suite 1895 Little Rock, Arkansas 72201 200 Tower Building Little Rock, Arkansas 72201 RE
Compliance Remedy Dear Counsel and Ms. Marshall: At a special meeting last night, the LRSD Board voted to approve the Proposed Comphance Plan provided to you by letter dated October 4,2002, with the exception of proposed regulation IL- R2. It was decided by the administration that IL-R2 was unnecessary and it was withdrawn. The administrations reasoning will be discussed below as a part of the Districts response to questions submitted by Ms. Marshall. Ms. Marshalls memorandum to Dr. James dated October 10, 2002, set forth 12 observations/questions related to the Proposed Compliance Plan. The Districts response to each observation/question is set forth below: F:\HOME\FENDLEY\LRSD 200I\unitary-all-counsd-10-l 1-02.wpdAll Counsel and Ms. Marshall October 11, 2002 Page 2 1. Of the eight program evaluations the district proposes to have re-written by outside experts (page 12), will new data for continuing programs be added to those evaluations, or will the re-writes be limited to the data that were used at the time the eight evaluations had been written? RESPONSE: The re-writes will be limited to the data that were used at the time the eight evaluations were written. 2. Page 1 of the Proposed Compliance Plan identifies Primary Reading/Language Arts as a program to be evaluated as part of the evaluation agenda, while the term. Elementary Literacy is used on page 15. The first term connotes PK-3, while the second connotes K-5. Do the different titles actually identify different programs? Which grades are to be included in the evaluation? RESPONSE: There is one program to be evaluated for PK-5, and the title of that evaluation will be Elementary Literacy, rather than Primary Reading/Language Arts. 3. Another evaluation agenda item is K-12 Mathematics and Science. The list of assessments on page 5 includes nothing for math and science in grades K-3. What data will be used in evaluating the primary grades, considering the lack of assessments in math and science at that level. RESPONSE: The data to be used for the K-12 Mathematics and Science evaluation will be the same as that previously used for the NSF Mathematics and Science evaluations with the exception of the ALTs which will no longer be administered. 4. In the evaluation of primary literacy, what measures will the district institute to off-set potential bias or conflict-of-interest inherent in the process of teachers administering assessments to their own students? RESPONSE: The District attempts to off-set potential bias resulting from teachers administering student assessments through its training of teachers in administering the assessments and monitoring the results of assessments. 5. In the evaluation ofprimary literacy, IL-Rl, requires a clear description of each program that is to be evaluated. To what extent will those descriptions include not only the subject content that students are expected to learn, but also teaching methods, materials, time allotment, and so on? F:\HOME\FENDLEY\LRSD 2001\unitary-aJl-counsel-10-l 1-02. wpdAll Counsel and Ms. Marshall October 11, 2002 Page 3 RESPONSE: The program description will be prepared by the evaluation committee pursuant to IL-Rl. Teaching methods, materials and time allotment may be included in the program description to the extent the evaluation committee deems it relevant. It is important to note the evaluation will be of the overall program and not individual program components. 6. 7. 8. 9. The proposed regulation requires that all relevant student performance data will be used in the evaluations. Will other data, such as interviews with teachers and students, case studies, and classroom observations, supplement the performance data for the evaluation? RESPONSE: The data necessary to answer the research questions will be determined by the evaluation committee. The degree and quality of program implementation have received little attention in previous evaluations. What procedures will the LRSD develop to measure program implementation (such as its quality, uniformity, and completeness throughout the district) in order to assure that the student performance data reported in the evaluations are actually the result of the programs described. RESPONSE: The evaluation committee will determine the necessity of such data and the manner in which it will be gathered. The compliance plan section headed, Continue to administer student assessment through the first semester of 2003-04" (page 4) details how the LRSD has recently altered its assessment plan. The 2002-03 assessment plan, board-approved on September 26, 2002, provides only for spring testing, as it eliminates all fall testing that is reported to and maintained by the LRSD and, of course, incorporates ADEs move of SAT-9 testing to the spring. Page 5 of this section reads that the final student assessment before March 15,2004 will be administered in the spring of 2003. We note that the spring 2003 tests will not only be the final assessment, but also the only assessment reported to or maintained by the LRSD between now and March 15, 2004. Will the data from this one testing cycle be complemented by that of previous years. RESPONSE: Yes, for the three new, comprehensive evaluations. As stated above, no new data will be gathered for the evaluations to be completed pursuant to Paragraph C of the Compliance Remedy. We note that the new assessment plan includes administration of student assessments only in English language arts and mathematics. Also, the 3"* grade has no assessments. F:\HOMEVFENDLEY\LRSD 2001\unitary-ail-counsd-10-ll-02.wpdAll Counsel and Ms. Marshall October 11, 2002 Page 4 RESPONSE: The assessment plan outlined was only that for English language arts and mathematics. Even so, it is noted that the SAT9 will be administered and the total battery of that test includes a science and social studies component. It is correct that there will be no assessment of 3"^ graders this year or next. The State will be developing and administering a test after that to comply with the No Child Left Behind Act. 10. 11. 12. Given the assignments in the Action Plan Timeline, what is the role of the districts Planning, Research and Evaluation department imder the new compliance plan? Under the envisioned Program Evaluation Agenda? RESPONSE: PRE staff may serve on the evaluation committees. Proposed regulation, IL-R2, Informal Program Evaluation, requires that a written record be prepared and maintained to support any decision to modify an academic program. The purpose of this regulation is xmclear. Are the program evaluation standards relevant to information evaluations? Does the regulation mean that LRSD will informally evaluate all programs? Or, will suspect programs be targeted for informal evaluation? Or, is the regulation intended to protect programs from unfair criticism? RESPONSE: The administration withdrew IL-R2 before approval of the Proposed Compliance Plan by the Board. The administration decided that the regulation would be redimdant of information to be included in the new, comprehensive evaluations required by Paragraph A of the Compliance Remedy. Rather than a separate written record, the program description in the new, comprehensive evaluations will include a description of program modifications made during each year of implementation satisfying the requirements of Paragraph B of the Compliance Remedy. How is the LRSD planning to alter its budget to purchase the services of outside evaluation experts? Which budget items will be reduced to accommodate the purchase of evaluation services? RESPONSE: This decision has not yet been made. The District will make this decision when it has a better idea of the total cost of these services. The Board has instructed the administration and counsel to work with the parties in an effort to achieve a consensus that the LRSDs Compliance Plan meets the requirements of the District Courts Compliance Remedy. The Board contemplates some give and take before a final consensus may be reached. So that this may be done in a timely manner, the LRSD asks all parties to F:\HOME\FENDLEY\LRSD 200l\uniury-al!-counsci-10-11-02. wpdAll Counsel and Ms. Marshall October 11, 2002 Page 5 specifically identify in writing any perceived deficiency in the Board-approved Compliance Plan on or before Monday, October 21,2002. Please let us know if this deadline presents areal problem for you or your client. We will be happy to answer any questions the parties may have before that date. We ask that the questions be submitted in vmting, and the LRSD will respond in writing so that all parties will have the benefit of the question and response. Thank you in advance for your cooperation. Sincerely, John C. Fendiey, Jr. cc: Dr. Ken James F:\HOME\FENDLEY\LRSD 2001\unitary-all-counsd-10-l 1-02. wpdec: JOHN W. WALKER SHAWN CHILDS Mr. Chris Heller Friday, Eldredge & Clark 2000 Regions Bank Center 400 West Capitol Little Rock, AR 72201 John W. Walker, P.A. Attorney At Law 1723 Broadway Little Rock, Arkansas 72206 Telephone (601) 374-3758 FAX (501) 374-4187 Via Facsimile -376-2147 October 9, 2002 OF COUNSEL ROBERT McHENRY. P.A. DONNAJ.McHENRY 8210 Hendesson Road Little Rook, Aekans.as 72210 Phone (501) 372-3425 F.ax (501) 372-3428 Email
mclieiiryd@swbelLnet RECEIVED OCT 11 2102 OmCEOF DESEGREGATION MONITORING Re: LRSD Compliance Plan Dear Chris: This is to inform you that I received your Compliance Plan on Monday, October 1, 2002. I was in court on Monday and Tuesday. Incidentally, I am again in court today and have not had adequate time to even read what you propose. I understand that you will be presenting this matter to the Board on tomorrow evening. I have called you several times and have been informed that you were not in. Accordingly, when you present this matter to the Board, please let them know that I have not had an opportunity to review what you have proposed nor to have any discussions regarding the matter with you or the ODM. 1 // Sincerely, / 4 jefhn W. Walker JWW:js cc: Ms. Ann Marshall All Counsel of Record e<r: John w. Walker, p.a. Attorney At Law 1723 Broadway Little Rock, Arkansas 72206 Telephone (501) 374-3758 FAX (501) 374-4187 JOHN W. WALKER SHAWN CHILDS Via Facsimile - 376-2147 October 10, 2002 OF COUNSEL ROBERT McHENEY. P.A DONNA J. McHenry 8210 Henderson Ro.ad Little Rock, Arkansas 72210 Phone: (501) 372-3425 Fax (501) 372-3428 Email: mchenryd@swbell.net Mr. Chris Heller Friday, Eldredge & Clark 2000 Regions Center 400 West Capitol Little Rock, AR 72201 RECEIVED OCT 1 1 2002 Re: Little Rock School District v. PCSSD, et al. Case No. 4:82CV00866 OFRCEOF DESEGREGATION MONITORING Dear Chris: Plan. This refers to your letter of October 4, 2002, providing LRSDs proposed Compliance The courts remedy and the general subject matter are too complex for us to provide all comments and objections we may ultimately have before todays Board meeting. We do note the following: 1. More consideration is needed of the programs to be identified as implementation pursuant to Section 2.7 . . . , which are to be subjected to a comprehensive program evaluation. . . Your document at page 7 identifies three areas. We note the absence of specific reference and detail regarding interventions / scaffolding - areas of vital importance given the achievement patterns of African American students. We note also that the LRSD compliance report cited many more programs as designed to fulfill Section 2.7. 2. In a discussion prior to his testimony in the hearing Judge Wilson, we understood Dr. Ross to indicate that the existing evaluation of the Pre-K - 2 literary program was not adequate. The notation at page 4 of your document of the changed use of the Observation Survey and the DRA relates to part of the concerns he expressed. This undermines the LRSD argument (page 11) that the existing evaluation, upon Board approval, will satisfy a part of the courts remedy. 3. The LRSD discussion about satisfying the courts order regarding the evaluationsmentioned at page 148 of the compliance report does not seem to take account of the material provided, which describes an adequate evaluation. 4. We question the period for implementation of a remedy which the court has identified and, therefore, the LRSD schedule. Once again, these comments should not be taken to be the full range of concerns which Joshua may ultimately have about the courts remedy and the Compliance Plan. Nor do we intend to waive our concerns about the court setting forth a remedy, without first hearing from the parties and the ODM with regard to the courts views on an appropriate remedy. Sitic^ly, / .>John W. Walker JWW.'js cc: Ms. Ann Marshall All Counsel of RecordFriday Eldredge & Clark HERSCHEL H. FRIDAY (1922-1994) WILUAM H. SUTTON. P.A. Byron m. eiseman, jr, p.a JOE D. BELL. P.A. JAMES A. BUTTRY. P.A. FREDERICKS. URSERY. P.A. OSCAR B DAVIS, JR., P.A. JAMES C. CLARK. JR.. P.A. THOMAS P. LEGGETT, P.A JOHN DEWEY WATSON. P.A PAUL B. BENHAM UI. P.A. LARRY W. BURKS. P.A. A. WYCKLIFP NISBET. JR.. P.A. JAMES EDWARD HARRIS. P.A J. PHILLIP MALCOM. P.A. JAMES M. SIMPSON. P.A. JAMES M. SAXTON. P.A J. SHEPHERD RUSSELL UI. P.A. DONALD H. BACON. P.A. WILLIAM THOMAS BAXTER. P.A. RICHARD D. TAYLOR, P.A. JOSEPH B. HURST. JR.. P.A. ELIZABETH ROBBEN MURRAY. P.A. CHRISTOPHER HELLER, P.A. LAURA HENSLEY SMITH. P.A. ROBERT S. SHAFER. P.A. WILLIAM M. GRIFFIN HI. P.A. MICHAEL S. MOORE. P.A. DIANE S. MACKEY. P.A. WALTER M. EBEL HI. P.A. KEVIN A. CRASS. P.A. WILLIAM A. WADDELL, JR.. P.A. SCOTT J. LANCASTER. P.A. ROBERT B. BEACH. JR., P.A. J. LEE BROWN. P.A. JAMES C. BAKER, JR., P.A. HARRY A. LIGHT, P.A. SCOTT H. TUCKER. P.A. GUY ALTON WADE. P.A. PRICE C. GARDNER, P.A. TONIA P. JONES, P.A. DAVID D. WILSON, P.A. JEFFREY H. MOORE. P.A. DAVID M. GRAF. P.A. ATTORNEYS AT LAW A LIMITED LIABILITY PARTNERSHIP www.fridayfirm.cofT> 2000 REGIONS CENTER 400 WEST CAPITOL LITTLE ROCK. ARKANSAS 72201-3493 TELEPHONE 501-376-2011 FAX 501-376-2147 3425 NORTH FUTRALL DRIVE. SUITE 103 FAYETTEVILLE. ARKANSAS 72703-4811 TELEPHONE 479-695-2011 FAX 479-665-2147 CARLA GUNNELS SPAINHOUR. P.A. JOHN C. FENDLEY, JR.. P.A. JONANN ELIZABETH CONIGLIO. P.A. R. CHRISTOPHER LAWSON. P.A. FRAN C. HICKMAN, P.A. BETTY J. DEMORY, P.A. LYNDA M. JOHNSON, P.A. JAMES W. SMITH, P.A. CLIPFORD W. PLUNKETT, P.A. DANIELL. HERRINGTON. P.A. MARVIN L. CHILDERS K. COLEMAN WESTBROOK. JR. ALLISON J. CORNWELL ELLEN M. OWENS JASON B. HENDREN BRUCE B. TIDWELL MICHAEL E. KARNEY KELLY MURPHY MCQUEEN JOSEPH P. MCKAY ALEXANDRA A. IFRAH JAY T TAYLOR MARTIN A. KASTEN BRYAN W. DUKE JOSEPH G. NICHOLS ROBERT T. SMITH RYAN A. BOWMAN TIMOTHY C. EZELL T. MICHELLE ATOR KAREN S. HALBERT SARAH M. COTTON PHILIP B. MONTGOMERY KRISTEN S. RIGGINS ALAN Q. BRYAN LINDSEY MITCHAM SLOAN KHAYYAM M. EDDINGS JOHN F. PEISERICH AMANDA CAPPS ROSE BRANDON J. HARRISON Honorable William R. Wilson, Jr. United States District Court 423 U.S. Post Office & Courthouse 600 West Capitol Avenue Little Rock, Arkansas 72201-3325 RE: 208 NORTH FIFTH STREET BLYTHEVILLE. ARKANSAS 72315 TELEPHONE 870-762-2868 FAX 870-762-2918 October 14,2002 OPCOUNSEL B.S. CLARK WILLIAM L. TERRY WILUAM L. PATTON. JR H.T. LARZELERE, P.A. JOHN C. ECHOLS. P.A AD. MCALLISTER JOHN C. FENDLEY. JR. LITTLE ROCK TEL S01-370-3323 FAX 501-244-S341 fn4lyQfc.nt RECEIVED OCT 1 6 2002 OmCEOF DESEGREGATION MONITORING Little Rock School District vs. Pulaski County Special School District et al. United States District Court, Eastern District, No. 4:82CV00866 WRW/JTR Dear Judge Wilson: We note that Ms. Marshall provided you a copy of her October 10,2002, memorandum to Dr. James setting forth observations/questions related to the Little Rock School Districts Compliance Plan for complying with the Compliance Remedy set forth in the Courts Memorandum Opinion of September 13,2002. We submitted a response to Ms. Marshall on October 11,2002, but were uncertain whether we should provide a copy of that to the Court. While we would be happy to provide copies to the Court, it would be presumptuous on our part to presume that the Court wants to review all correspondence between counsel in this case. We respectfully request the Court advise the parties what it would like them to do in this regard. Thank you for your time and attention to this matter. Sincerely, John C. Pendley, Jr. V F:\HOME\BBrown\Fa>dley\LRSD\deseg\judge wilsoo k.wpdHonorable William R. Wilson, Jr. October 14,2002 Page 2 cc: Mr. John W. Walker Mr. Sam Jones Mr. Steve Jones Mr^chard Roachell ^Ms. Ann Marshall Ms. Sammye Taylor Mr. Ken James F:\HOME\BBrown\FcndJcy\]J<SEMcscg\/udge wibon h.wpdI . JOO JOHN W. WALEtER SHAWN CHILDS John W. walker, pa. Attohney At Law 1723 Broadway Little Rock, Arkansas 72206 Telephone (501) 374-3758 FAX (501) 374-4187 October 23,2002 OF COUNSEL ROBERT McHENEY, PA. DONNA J. McHENRY 8210 Hendesson HoaL Uttle Rock, Arkansas 72210 Phone: (601) 372-3426 Pax (501) 372-3428 RUAn,: mchearyd^awLelLxket Mr. Christopher Heller FRIDAY. ELDREDGE & CLARK . 400 W. CapitoU Suite 2200 Little Rock, Arkansas 72201 RECEIVED OCT 2 4 2002 Re
LRSD V, PCSSD OFFICE OF DESEGREGATION MONITORING Dear Chris
This letter sets forth additional comments of the Joshua Intervenors concerning the I PSD Compliance Plan. We are offering these comments, although we are unable to discern that the comments we offered earlier were given consideration, 1. In using historical student assignment results, attention should be given to the quality of the data. In the past, LRSD has used results on the RA and the Observation Survey in ways not consistent with the purposes of those instruments. In addition, because teachers provided scores for their own students, the past use made of the data was in conflict with the districts recognition in the newly enacted Regulation IL-Rl that Conflict of Interest must be avoided. 2. We are concerned about the manner in which the regulation describes the team process for preparing evaluations, again in the context of conflict of interest. In order to insure that conflict of interest is avoided, the external consultant needs to write the report and control the context of the analysis. Paragraphs 3, 5 and 6 of the Program Evaluation Procedures do not guarantee that the external expert will have these roles. Of course, if reports were prepared in the manner which we describe, there would be no bar to LRSD staff preparing commenTs to the Board with a differing interpretation of the evaluation results. 3. We continue to be concerned about the global, general manner in which the content of planned evaluations is described (page 7 of the document, first paragraph). For example, the Board has adopted a policy and two regulations dealing with remediation for students whose perform anrA is below par. Studying the actual implementation of these standards (in all or a representative sample of schools) is of vital importance to the Intervenor class because class members are so much more likely than other students to exhibit unsatisfactory performance on the Benchmark and Stanford Achievement Tests. A satisfactory description by the School Board of the evaluations which itI, Page Two October 23,2002 ^^0 dertake should make clear that the actual implementation of remediation ^vittes in distnct schools is to receive careful consideration. This is surely an important contextual factor (see Accuracy Standards, para. 2). / 1 the LRSD plans evaluations of programs deemed to be paiticul^ly directed to achievement of African American students for the indefinite future, not smply for the period necessary to satisfy the court. We would like to receive the Boards assurance that this IS the case. We would appreciate yourproviding this letter to the Superintendent and the members of the school board. RffW. Walker Sincere^, JWW:lp cc: All Counsel Ms. Ann Marshall Judge Thomas Ray Dazs: .J- O JOHN W. NHLKER, Aitomsy ar Law 1723 Sroad'.way J2od^ Aakar<aas 72206 Telephons (501) 37a.375S Fea (501) 374-4137 RAH TR.liVSMI3SICiN COVTS S7TTF.F e Ltui L I fk- J 271-d/a-i} [. L Rz: SznRsr: / 70USHOULD HSCDD/E [ (iTiclzuimg caver shssi)} PAC-D(S). OfCLUDlHG THIS COVER SHEET. IF TOUDO HOT RECEIVE ALL THE PACES, r.4r.r "<(501) 574-375S>" The iMarmationconiaiaedin diis iiesiniilemessage is sitoiaeypri'/iiegedand coafideatial infSrmgrinn mr.qnripd eniy for the use of the iadivimial ar enrity named above, if ms .^ader of this message is not the intended tecipieat. ot the employes or agent responsible to deliver it to the intended recipient, you are hereby notified that any dissetnination, oistibiinon or copying of mis conmunicahon. is strictly prohibited. If you have received dhs communication in error, please immediate aoui^us by teleptons. and return the original message to us at the above address via the U.S. Postal SerAce. Thank you. received OCT 2 9 2002 OFFICE OF DESEGREGATION MONITORING John W. Walker, P.A. Attorney At Law 1723 Broadway Little Rock, Arkansas 72206 Telephone (501) 374-3758 FAX (501) 374-4187 JOHN W. WALKER SHAWN CHILDS OF COUNSEL ROBERT McHENRY, P a DONNA J. McHenry 3210 Henderson Road Little Rock, Ahkans.as 72210 Phone: (501) 372-3425 P.4X (501) 372-3428 Email
mchenryd@swbeil.net Via Facsimile - 376-2147 October 25, 2002 Mr. Chris Heller Friday, Eldredge & Clark 2000 Regions Center 400 West Capitol Little Rock, AR 72201 Re: Little Rock School District v. PCSSD, et al. Case No. 4:82CV00866 Dear Chris: We are in receipt of your letter dated October 25, 2002 regarding LRSDs compliance remedy. Instead of these exchanges of paper, it would be advantageous to all concerned for us to have some meetings in order to respond to these issues so that we can better understand each others position. Not everything is black letter and there should be room for discussion in mutual agreement. Si-Hcerely, .hn W. Walker JWW
js cc: Ms. Ann Marshall All Counsel of Record(TC: Friday Eldredge & Clark HERSCHEL H. FRIDAY (1922-1994) WILLIAM H. SUTTON. P.A. BYRON M. EISEMAN. JR.. P.A. JOE D. BELL, P.A. JAMES A. BUTTRY. P.A. FREDERICK S. URSERY. P.A. OSCAR E. DAVIS. JR.. P.A. JAMES C. CLARK, JR.. P.A. THOMAS P. LEGGETT. P.A. JOHN DEWEY WATSON. P.A. PAUL B. BENHAM IIL P.A. LARRY W. BURKS. P.A. A. WYCKLIFF NISBET. JR.. P.A. JAMES EDWARD HARRIS. P.A. J. PHILLIP MALCOM. P.A. JAMES M. SIMPSON. P.A. JAMES M. SAXTON, P.A, J. SHEPHERD RUSSELL III, P.A. DONALD H. BACON. P.A. WILLIAM THOMAS BAXTER. P.A. RICHARD D. TAYLOR. P.A. JOSEPH B. HURST. JR.. P.A. ELIZABETH ROBBEN MURRAY. P.A. CHRISTOPHER HELLER. P.A. LAURA HENSLEY SMITH. P.A. ROBERT S. SHAFER. P.A. WILLIAM M. GRIFFIN III. P.A. MICHAEL S. MOORE. P.A. DIANES. MACKEY. P.A. WALTER M. EBEL III. P.A. KEVIN A. CRASS. P.A. WILLIAM WADDELL. JR.. P.A. SCOTT J. LANCASTER. P.A. ROBERT B. BEACH. JR., P.A. J. LEE BROWN, P.A. JAMES C. BAKER. JR.. P.A. HARRY A. LIGHT. P.A. SCOTT H. TUCKER. P.A. GUY ALTON WADE. P.A. PRICE C. GARDNER. P.A. TONIA P. JONES. P.A. DAVID D. WILSON. P.A. JEFFREY H. MOORE. P.A. DAVID M. GRAF. P.A. RECEIVED OCT 2 5 2002 OFFICE OF DESEGREGATION MONITORING ATTORNEYS AT LAW A LIMITED LIABILITY PARTNERSHIP www.fridayfirm.com 2000 REGIONS CENTER 400 WEST CAPITOL LITTLE ROCK. ARKANSAS 72201-3493 TELEPHONE 501-376-2011 FAX 501-376-2147 342S NORTH FUTRALL DRIVE. SUITE 103 FAYETTEVILLE. ARKANSAS 72703-4811 TELEPHONE 479-665-2011 FAX 479-695-2147 208 NORTH FIFTH STREET BLYTHEVILLE. ARKANSAS 72315 TELEPHONE 870-762-2696 FAX 870-762-2918 October 25, 2002 CARLA GUNNELS SPAINHOUR, P.A. JOHN C. FENDLEY. JR.. P.A, JONANN ELIZABETH CONIGLIO. P.A R. CHRISTOPHER LAWSON, P.A. FRAN C. HICKMAN. P.A. BETTY J. DEMORY, P.A. LYNDA M. JOHNSON, P.A. JAMES W. SMITH. P.A. CLIFFORD W. PLUNKETT. P.A, DANIEL L. HERRINGTON. P.A MARVIN L. CHILDERS K. COLEMAN WESTBROOK. JR. ALLISON J. CORNWELL ELLEN M. OWENS JASON B. HENDREN BRUCE B. TIDWELL MICHAEL E. KARNEY KELLY MURPHY MCQUEEN JOSEPH P. MCKAY ALEXANDRA A. IFRAH JAY T. TAYLOR MARTIN A. KASTEN BRYAN W. DUKE JOSEPH G. NICHOLS ROBERT T. SMITH RYAN BOWMAN TIMOTHY C. EZELL T. MICHELLE ATOR KAREN S. HALBERT SARAH M. COTTON PHILIP 8. MONTGOMERY KRISTEN S. RIGGINS ALAN 0. BRYAN LINDSEY MITCHAM SLOAN KHAYYAM M. EDDINGS JOHNF. PEISERICH AMANDA CAPPS ROSE BRANDON J. HARRISON OF COUNSEL B.S. CLARK WILLIAM L. TERRY WILLIAM PATTON. JR. H.T. LARZELERE, P.A. JOHN C. ECHOLS, P.A. AD. MCALLISTER JOHN C. FENDLEY, JR. LITTLE ROCK TEL 501-370-3323 FAX 501-244-5341 fsndlay^fac.nct (By Hand Delivery ) Mr. John W. Walker Mr. Sam Jones Mr. Steve Jones John W. Walker, P.A. 1723 Broadway Little Rock, Arkansas 72201 Wright, Lindsey & Jennings 2200 Bank of America Bldg. 200 West Capitol Little Rock, Arkansas 72201 Jack, Lyon & Jones, P.A. 425 W. Capitol, Suite 3400 Little Rock, Arkansas 72201 Mr. Richard Roachell Roachell Law Firm ( By Hand Delivery ) Ms. Ann Marshall Mr. Dennis Hanson Plaza West Building 415 N. McKinley, Suite 465 Little Rock, Arkansas 72205 Desegregation Monitor 1 Union National Plaza Office of the Attorney General 323 Center Street 124 W. Capitol, Suite 1895 Little Rock, Arkansas 72201 200 Tower Building Little Rock, Arkansas 72201 RE
Compliance Remedy Dear Counsel and Ms. Marshall: In our letter dated October 11,2002, we asked the parties to specifically identify in writing any perceived deficiency in the Board-approved Compliance Plan on or before Monday, October 21, 2002. No responses were received on or before that date. However, Mr. Pressman called on October 21,2002, and advised that Joshua would rely on tlie comments contained in Mr. Walkers October 10, 2002, facsimile. On October 24, 2002, additional comments were received from Mr. Walker. All of Mr. Walkers comments will be addressed in turn.All Counsel and Ms. Marshall October 25, 2002 Page 2 October 10, 2002 Facsimile 1. More consideration is needed of the programs to be identified as implementat[ed] pursuant to Section 2.7 .. which are to be subjected to comprehensive program evaluation .. > Your document at page 7 identifies three areas. We note the absence of specific reference and detail regarding interventions/ scaffolding - areas of vital importance given the achievement patterns of Afiican-American students. We note also that the LRSD compliance report cited many more programs as designed to fulfill Section 2.7. Mr. Pressman clarified this concern during our October 21, 2002 telephone conversation. Mr. Pressman explained that Joshua was concerned that interventions designed to assist low achieving students, for example SAIPs, were not being fully implemented and wanted some assurance that the comprehensive program evaluations would assess implementation of these programs. LRSD RESPONSE: On October 24,2002, the Board approved the Division of Instructions Plan to Support Low-Performing Schools, a copy of which is enclosed for your review. Under that plan, the LRSD will conduct curriculum, instruction and classroom management audits at low performing schools. Data gathered through these audits and other monitoring under the plan may be used by a program evaluation team to identify possible causes of poor performance, including poor implementation of interventions such as SAIPs. The LRSD lacks the resources to implement this plan at every school. Approximately 10 schools will receive the full compliment of services outlined in the plan. Those 10 schools will be identified based on the priority system set forth in the plan. 2. In a discussion prior to his testimony in the hearing [before] Judge Wilson, we understood Dr. Ross to indicate that the existing evaluation of the PreK-2 literacy program was not adequate. The notation on page 4 of your document of the changed use of the Observation Survey and the DRA relates to part of the concerns he expressed. This imdermines the LRSD argiunent (page 11) that the existing evaluation, upon Board approval, will satisfy a part of the Courts remedy. LRSD RESPONSE: As the LRSD understands this statement, Joshua objects to the LRSD considering the PreK-2 literacy evaluation to have been completed pursuant to Paragraph C of the Compliance Remedy. Attached are the comments received by the LRSD from Dr. Ross related to that evaluation. As can be seen. Dr. Ross did not advise the LRSD that the evaluation was inadequate. Moreover, it does not make sense for the LRSD to expend resources to have this evaluation completed by an outside expert while it also prepares a new, comprehensive evaluation of the same program with the assistance of an outside expert.All Counsel and Ms. Marshall October 25, 2002 Page 3 3. The LRSD discussion about satisfying the courts order regarding the evaluations mentioned at page 148 of the compliance report does not seem to take accoimt of the material provided, which describes an adequate evaluation. LRSD RESPONSE: As the LRSD understands this statement, Joshua objects to the LRSD not completing the evaluations identified on page 148 of the Final Compliance Report in a manner consistent with IL-Rl. As the LRSD understands Paragraph C of the Compliance Remedy, the District Court simply wants the LRSD to do what it said it did and complete the evaluations identified on page 148 of the Final Compliance Report. That is what the LRSD intends to do. It is true that those evaluations, even after being completed, may not be model program evaluations as envisioned by IL-Rl. The LRSD decided, however, that the most prudent use of its limited resources would be to focus on the new, comprehensive evaluations of programs designed to improve Afiican-American achievement. 4. We question the period of implementation of a remedy which the court has identified and, therefore, the LRSD schedule. LRSD RESPONSE: The LRSD is willing to agree that any agreement between the LRSD and Joshua related to implementation of the Compliance Remedy will not prejudice Joshuas appeal of the District Courts September 13,2002, Memorandum Opinion. October 24, 2002 Facsimile 1. In using historical student assigiunent results, attention should be given to the quality of the data. In the past, LRSD has used results on the [D]RA and the Observation Survey in ways not consistent with the purposes of those instruments. In addition, because teachers provided scores for their ovra students, the past use made of the data was in conflict with the districts recognition in the newly enacted Regulation IL-Rl that Conflict of Interest must be avoided. LRSD RESPONSE
Paragraph A of the Compliance Remedy requires the LRSD to use all available data in its evaluations. It will be the responsibility of the evaluation team to weigh the reliability and validity of the available data. The Arkansas Department of Education and national organizations with expertise in early literacy recommend the use of the DRA and Observation Surveys. The primary piupose of those assessments is to determine whether students are learning the essential components of the reading curriculum. As to the integrity of the data from those assessments, the LRSD monitored student scores year-to-year to discoinage teachers from inflating scores in an effort to show improvement. Moreover, the ultimate success of the LRSDs early literacy program willAll Counsel and Ms. Marshall October 25,2002 Page 4 be judged by performance on the States Benchmark examinations, rather than the DRA and Observation Surveys. 1. We are concerned about the manner in which the regulation describes the team process for preparing evaluations, again in the context of conflict of interest. In order to insure that conflict of interest is avoided, the external consultant needs to write the report and control the context- of the analysis. Paragraphs 3, 5 and 6 of the Program Evaluation Procedures do not guarantee that the external expert will have these roles. Of course, if reports were prepared in the maimer which we describe, there would be no bar to LRSD staff preparing comments to the Board with a differing interpretation of the evaluation results. LRSD RESPONSE: The LRSD rejects the implication that LRSD personnel cannot be trusted to write an honest program evaluation. The LRSDs commitment to improving student achievement is second to none. To fulfill that commitment, it is in the LRSDs best interest to effectively evaluate its programs. The success of the programs and program evaluations will ultimately be measured by the States Benchmark evaluations. All evaluation team members will be actively involved in the evaluation process and are expected to provide a check against the self-interest of any one team member. The evaluation team will decide who writes the report based on the expertise of team members. The outside expert will be asked to take to the Superintendent any concerns about the evaluation not being addressed by the evaluation team. The outside expert will also be asked to be present when the evaluation is presented to the Board so that the Board can be advised of any concerns the outside expert may have about the final evaluation. 3. We continue to be concerned about the global, general manner in which the content of planned evaluations is described (page 7 of the document, first paragraph). For example, the Board has adopted apohcy and two regulations dealing with remediation for students whose performance is below par. Studying the actual implementation of these standards (in all or a representative sample of schools) is of vital importance to the Intervenor class because class members are so much more likely than other students to exhibit unsatisfactory performance on the Benchmark and Stanford Achievement Tests. A satisfactory description by the School Board of the evaluations which it requires the staff to undertake should make clear that the actual implementation of remediation activities in district schools is to receive careful consideration. This is surely an important contextual factor (see Accuracy Standards, para. 2). LRSD RESPONSE
As the LRSD understands this comment, it is a restatement of the first number paragraph in Mr. Walkers October 10, 2002 facsimile, and the LRSD hereby incorporates its response thereto.All Counsel and Ms. Marshall October 25, 2002 Page 5 4. We understand from the Plan that the LRSD plans evaluations of programs deemed to be particularly directed to achievement of African-American students for the indefinite term, not simply for the period necessary to satisfy the court. We would hke to receive the Boards assurance that this is the case. LRSD RESPONSE: The Boards approval of IL-Rl was not limited to the term of the Compliance Remedy, and at this time, the Board anticipates continuing to evaluate programs pursuant to Policy IL after the term of the Compliance Remedy. Conclusion The LRSD hopes that it has been able to address all of Joshuas concerns. If any party has any questions about the LRSDs responses to Joshuas comments, we ask that those be submitted in writing, and the LRSD will promptly provide a written response. If Joshua continues to have concerns about the LRSDs Compliance Plan, Joshua should consider this the LRSDs written response to alleged noncompliance in accordance with Revised Plan 8. Pursuant to Revised Plan 8.2.4, Joshua has 15 days of receipt of this letter to submit the issue to ODM for facilitation of an agreement. Thank you for your cooperation. Sincerely, ohn C. Jr. John Pendley, cc: Dr. Ken James (via hand-delivery)received cc 1 2 5 2002 OESEGREGWION MONITORING Plan to Support Low-Performing Schools Division of Instruction Little Rock School District 2002-2003 Background The Little Rock School District obtained approval from the federal court in spring 1998 for its Revised Desegregation and Education Plan. In submitting its plan to the. judge, the District willingly assumed accountability for its numerous obligations, several of which addressed the issues of student achievement. The following sections were most relevant: 2.7 LRSD shall implement programs, policies, and/or procedures designed to improve and remediate the academic achievement of African-American students. 2.7.1 LRSD shall assess the academic programs implemented pursuant to Section 2.7 after each year in order to determine the effectiveness of the academic programs in improving African- American achievement. If this assessment reveals that a program has not and likely will not improve African-American achievement, LRSD shall take appropriate action in the form of either modifying how the program is implemented or replacing the program. 5.1 Early Childhood Education. LRSD shall implement an early childhood education program which shall include a HIPPY program and a four-year-old program with no less than 720 seats. LRSD contemplates that the four-year-old classes will remain at their present site or in the same general location. 5.2.1 Primary Grades. LRSD shall implement at least the following strategies to improve the academic achievement of students in kindergarten through the third grade: a. Establish as a goal that by the completion of the third grade all students will be reading independently and show understanding of words on a page
g. Monitor student performance using appropriate assessment devices. 5.2.2 Intermediate Grades. LRSD shall implement at least the following strategies to improve the academic achievement of students in grades four through six: a. Adopt as a goal that by completion of the sixth grade all students will master and use daily higher-level reading comprehension skills for learning in all subject areas, for 1making meaning in real life experiences, and for personal growth and enjoyment
e. Monitor student performance using appropriate assessment devices. 5.2.3 Secondary Schools. LRSD intends to implement the following strategies to improve the academic achievement of students in grades six through twelve: a. Adopt as a goal that upon graduation all students will read independently with comprehension in all subject areas and be proficient in language arts, as necessary to be successful workers, citizens, and life-long learners
f. Monitor student progress and achievement using appropriate assessment devices. 5.3 Mathematics. LRSD shall implement the following strategies to improve mathematics instruction. 5.3.2 Develop appropriate assessment devices for measuring individual student achievement and the success of the revised curriculum. 5.3.5 Adopt as a goal that all students in regular classes will complete Algebra and Geometry by the end of their eleventh grade year and that students will be proficient in mathematics by graduation. On September 13,2002, the federal court ruled that the District was substantially in compliance in all areas of the obligations, except Section 2.7.1, and the judge established a Compliance Remedy for the completion of a number of program evaluations. The work to meet those requirements is in progress. In the Districts Covenant for the Future, the Little Rock School District pledged to the commumty and to all employees, students, and parents to continue its efforts for the improved academic achievement of aU the children: improve the academic achievement of all students, comply with the Constitution, and ensure that no person is discriminated against on the basis of race, color, or ethnicity in the operation of the District, and provide equitable educational resources, programs, and opportunity in a nondiscriminatory environment for all students attending District schools. In June 1999 the Arkansas Department of Education published its statewide accountability plan, the Arkansas Comprehensive Testing, Assessment, and Accountability Program (ACTAAP), which was subsequently revised in June 2000. The 2'f introduction states that ACTAAP is a comprehensive system encompassing high academic standards, professional development, student assessment, and accountability for schools and students. The four stated purposes are as follows: To improve student learning and classroom instruction
To provide public accountability by establishing expected achievement levels and reporting on student achievement
To provide program evaluation data
and To assist policymakers in decision-making. Expected outcomes include the following: It will result in improved teaching and learning. It will identify successful schools and programs and encourage replication of those successes. It will encourage individual schools and school districts to reflect on then- practices, take corrective actions, and receive support from state agencies. It will fulfill the requirements of various Arkansas statutes, including Act 999 of 1999, which mandates that all students in the public schools of this state demonstrate grade-level academic proficiency through the application of knowledge and skills in the core academic subjects consistent with state curriculum frameworks, performance standards, and assessments. At the centerpiece of ACTAAP is the states new assessment program, which in 2002- 2003 includes annual criterion-referenced Benchmark examinations in literacy and mathematics at grades 4,6, 8, and high school end-of-course/level tests in Algebra I, Geometry, and Literacy. The states testing program also includes the administration of the SAT9 at grades 5,7, and 10. Accountability indicators for the three tiers of the accountability program are established. Tier I includes the following: Performance on Benchmarks and End-of-Course/Level tests School dropout rates Average daily attendance Classes taught by an appropriately licensed teacher 3Professional development (at least 30 hours) School safety. Tier n includes the same indicators, but the assessment is based on growth over multiple years, not absolute scores. Mandated subgroups to examine include the general population, special education, limited English proficient, and high mobility students. Tier HI is a narrative of approximately 500 words, developed by staff of each school. The narrative may include the progress that the school has made in implementing its school improvement plan. Page 19 of the plan includes the following definition of adequate yearly progress to be used until the ACTAAP can be fully implemented.
In order to meet federal mandates, a temporary system will be developed to identify those schools designated for school improvement. Beginning in 2000- 2001 and continuing until the ACTAAP accountability system is fully operational, a school will be designated in school improvement under the following condition: Seventy-five percent or more of the students perform below proficient on either the literacy (reading and writing) or the mathematics section of the Benchmark Exam for the designated grade or grades represented by the school. It has been this paragraph upon which the Little Rock School District has based much of its decision-making relating to school improvement since 1999-2000. Clearly, both in the description of Tier I and Tier n indicators and in the page 19 paragraph, it was not the performance on the SAT9 that mattered. SAT9 scores were included only as an optional indicator. A schools entire accountability for Title I depended on its performance on the Benchmarks. In January 2001 the No Child Left Behind (NCLB) Act, which included the reauthorization of the Elementary and Secondary Education Act (ESEA), became law. The accountability provisions in Title I of this new law included, for the most part, the same language as several of the most recent reauthorizations^requirements that states and districts identify low-performing schools for improvement and requirements that progressively more intense consequences should occur for schools continuing not to improve over time. One difference in the 2001 law is that the first year a school is identified for improvement, the school must, in addition to the usual consequences, offer public school choice to the children and parents of that school so that they may elect to attend another higher performing school in the district. This option will remain in effect until the home school has earned its way out of the school improvement designation. Transportation costs must be paid out of the Title I allocation. The second year that a school is identified for improvement, the school must continue to offer choice, but must also offer, at no cost to the parent, a choice of state-approved supplemental services, or tutoring. Again, the cost of this service must be paid from Title I funds. 4NCLB requires states to establish curriculum standards and to test all students every year in both language arts and mathematics, grades 3-8, and once in grades 10-12. States must also establish an accountability system for all its public schools. The new law was to become effective at the beginning of the 2002-2003 school-year. However, implementation has been guided chiefly at both the state and local levels through Dear Colleague Letters fiom the Secretary of Education, since the formal regulations were still not complete as of late October 2002. Many, many questions regarding definitions of the mandated adequate yearly progress (AYP), what flexibility districts have in implementing the choice options, and other major issues remained unanswered. Arkansas educators assumed that the Arkansas Department of Education (ADE) plan described in the ACTAAP document for accountability remained in force, and ADE announced in June 2002 that no Arkansas schools would be identified for improvement under NCLB until the end of the 2002-2003 school year, given the phase-in of the states Benchmarks and high school tests. In September 2002 everyone awoke one morning to a headline in the newspaper that the state had been required by the federal government to change its accountability plan, that schools would be identified based upon SAT9 data, not Benchmark data, and that the public school choice and supplemental services requirements must be implemented no later than the beginning of the 2002-2003 schoolyears second semester. ADE aimounced its list of identified schools on October 16, 2002. The 1998-99 schoolyear was established as the baseline year for making the identification decisions, the percent of students performing at/above the 50* percentile of the SAT9 Basic Battery score was selected as the measurement, and the first screening of schools included all Title I schools with fewer than 25 percent of its students performing at/above the 50* percentile. Schools in that pool could also be identified for improvement if they had failed to decrease the percentage of students performing in the bottom quartile for two consecutive years. From that pool of low-performing schools, schools failing to make AYP for the two subsequent years (1999-2000 and 2000-2001), based on SAT9 data, were identified for improvement. These ongoing commitments in the Revised Desegregation and Education Plan and in the Covenant for the Future, plus the adoption of the Arkansas Comprehensive Testing, Assessment, and Accountability Program (ACTAAP) by the State of Arkansas, the passage of the No Child Left Behind (NCLB) act by the United States Congress, and ADEs identification of schools for improvement all compel the Little Rock School District to enhance its efforts to provide a higher level of monitoring, support, and intervention for its low-performing schools. Through the activities outlined in this plan, the District adds another strategy to improve the academic achievement not only for Afiican American children, but for all children, whether representing a race/ethnicity, gender, firee/reduced lunch status, disability, or limited English proficiency subgroup. Stq)s have already been taken by the Division of Instruction to assume new roles and responsibilities in supporting school improvement. For the past two years curriculum 5staff and PRE staff have provided principals meeting in small groups a complete set of school performance data, including assistance in analyzing those data and support in implementing their school improvement plans. Division of Instruction staff concentrated many of their efforts on low-performing schools in 2001-2002, including the monitoring of Student Academic Improvement Plans (SAIPs), targeting those schools for grant proposals (Comprehensive School Reform grants and the Reading Exellence Act grants), providing follow-up professional development, consultation, and school visits. Needs Assessment According to the Arkansas Department of Education in its October 16, 2002, communication to the Superintendent, the following Little Rock School District schools have been identified in 2002-2003 for improvement: Bale Elementary (1 year) Baseline Elementary (2** year) Chicot Elementary (2" year) Dodd Elementary (1* year) Fair Park Elementary (P* year) Mabelvale Elementary (L year) Mitchell Elementary (Pyear) Stephens Elementary (2" year) Wakefield Elementary (2" year) Watson Elementary (2" year) Henderson Middle (1 year) Mabelvale Middle (1 year) Southwest Middle (l year) ADEs letter to the Superintendent included the following information relating to Removal from School Improvement: All schools, except high schools, listed in this notification will be eligible for removal from school improvement based on AYP as measured by the Benchmark exams and described in the Arkansas Comprehensive Testing, Assessment, and Accountability Program (ACTAAP). Specifically, these schools will be eligible for removal following the 2003 administration of these exams based on two consecutive years ofAYP with 2000-2001 designated as the baseline year. Several issues remain unclear
1. Whether AYP will be defined based on ten years or twelve years to reach 100 percent proficient/advanced
2. Whether AYP will be defined for one year or over a two-to-three year period
63. 4. 5. 6. Whether schools identified for school improvement in this first group will be accountable for subgroup performance
How AYP for subgroups will be determined
What the minimum number of students will be to be considered a subgroup
Whether the high priority schools will continue to be defined as those with fewer than 25 percent performing at the proficient/advanced levels. The deadline for states to submit their accountabihty plans, definitions for AYP, and related issues to the U. S. Department of Education for approval is the end of the calendar year. It is likely, therefore, that districts will not know until late spring or early summer what the rules will be for some of these issues related to schools already identified for improvement and for the new identification that will occur when 2002-2003 Benchmark scores are received. The Little Rock School District, however, cannot wait for answers to all these questions in order to begin providing the necessary support, assistance, interventions, and monitoring required to help the identified schools exit the School Improvement designation and to prevent as many additional schools as possible from being identified at the end of 2002-2003. To assess greatest need and to enable the Division of Instruction to determine priorities, a more complete needs assessment was conducted. The following matrix lists the schools that have been formally identifi
This project was supported in part by a Digitizing Hidden Special Collections and Archives project grant from The Andrew W. Mellon Foundation and Council on Library and Information Resoources.