Compliance remedy

J. As to the educational acronyms, Counsel has requested that the authors of the comprehensive evaluations immediately prepare a glossary of acronyms used in their respective evaluations. These will be consolidated into a single glossary for all exhibits and provided to the Court as soon as possible. Respectfully Submitted, LITTLE ROCK SCHOOL DISTRICT Friday, Eldredge & Clark. Christopher Heller (#81083) 2000 Regions Center 400 West Capitol Little Rock, AR 72201-3493 (501) 376-2011 BY: Christopher Heller 1 Page 2 of 3CERTIFICATE OF SERVICE I certify that a copy of the foregoing has been served on the following people by depositing a copy of same in the United States mail on May 13, 2004: Mr. John W. Walker JOHN W. WALKER, P.A. 1723 Broadway Little Rock, AR 72201 Mr. Sam Jones Wright, Lindsey &. Jennings 2200 Nations Bank Bldg. 200 West Capitol Little Rock, AR 72201 Ms. Arm Marshall Desegregation Monitor 1 Union National Plaza 124 W. Capitol, Suite 1895 Little Rock, AR 72201 Mr. Tim Gauger Mr. Mark A. Hagemeier Office of the Attorney General Mr. Steve Jones JACK, LYON & JONES, P.A. 425 W. Capitol, Suite 3400 Little Rock, AR 72201-3472 323 Center Street 200 Tower Buildin! Little Rock, AR 72201 Judge J. Thomas Ray U. S. District Courthouse 600 West Capitol Avenue, Suite 149 Little Rock, AR 72201 Mr. Clayton Blackstock Mr. Mark Burnett 1010 W. Third Street Little Rock, AR 72201 ig Christopher Heller Pa-. Page 3 of 3 0.!- IN THE UNITED STATES DISTRICT COURT EASTERN DISTRICT OF ARKANSAS WESTERN DIVISION received may 1 4 200^ DKEGREGATIO? fSoNITORJNG LITTLE ROCK SCHOOL DISTRICT PLAINTIFF V. NO.4
82CV00866 WRW PULASKI COUNTY SPECIAL SCHOOL DISTRICT NO. 1,ETAL DEFENDANTS MRS. LORENE JOSHUA, ET AL INTERVENORS KATHERINE KNIGHT, ET AL INTERVENORS PLAINTIFFS NOTICE OF FIT.iNG DOCUMENTS IN RESPONSE TO THE COURTS ORDER FILED MAY 12, 2004 Plaintiff Little Rock School District (^ERSD) for its Notice of Filing states: 1. Attached are the following documents requested by the Court in its Order filed May 12, 2004: A. Little Rock School District Proposed Compliance Plan Revised Plan 2.7.1 (Appendix 1 of which is IL-Rl")
B. Letter from John W. Walker to Chris Heller dated October 10, 2002
and. C. Letter from John W. Walker to Chris Heller dated October 23, 2002 (received by fax on October 24, 2002). 2. As to Mr. Walker s references to your document, the Court is correct that Mr. Walker is referring to the Proposed Compliance Plan attached hereto as Exhibit A. Page 1 of 3Little Rock School District Compliance Committee Proposed Compliance Plan Revised Plan 2.7.1.The District Courts Compliance Remedy On September 13, 2002, the District Court issued its Memorandum Opinion (hereinafter Opinion ) finding that the Little Rock School District (LRSD) had substantially complied with all areas of the Revised Desegregation and Education Plan (Revised Plan), with the exception Revised Plan 2.7,1. Section 2.7.1 provided: LRSD shall assess the academic programs implemented pursuant to Section 2.7 after each year in order to determine the effectiveness of the academic programs in improving Afincan-American achievement. If this assessment reveals that program has not and likely will not improve Afiican-American achievement, LRSD shall take appropriate action in the form of either modifying how the program is implemented or replacing the program a The District Court s Opinion set forth a detailed Compliance Remedy to be implemented by the LRSD. The Opinion first stated: Because LRSD failed to substantially comply with the crucially important obligations contained in 2.7.1, it must remain under court supervision with regard to that section of the Revised Plan until it: (a) demonstrates that a program assessment procedure is in place that can accurately measure the effectiveness of each program implemented under 2.7 in improving the academic achievement of African-American students
and (b) prepares the program evaluations identified on page 148 of the Final Compliance Report and uses those evaluations as part of the program assessment procedure contemplated by 2.7.1 of the Revised Plan. The Opinion then outlined the details of the Compliance Remedy as follows: A. For the entire 2002-03 school year and the first semester of the 2003-04 school year, through December 31, 2003, LRSD must continue to assess each of the programs implemented under 2.7 to improve the academic achievement of Afidcan-American students. LRSD now has over three years of testing data and other information available to use in gauging the effectiveness of those programs. I expect LRSD to use all of that available data and information in assessing the effectiveness of those prograrhs and in deciding whether any of those programs should be modified or eliminated. Revised Plan 2.7 provided, LRSD shall implement programs, policies and/or procedures designed to improve and remediate the academic achievement of Afiican-American students, including but not limited to Section 5 of this Revised Plan. 1 1B. C. F. LRSD must maintain written records regarding its assessment of each of those programs. These written records must reflect the following information: (a) the written criteria used to assess each program during the 2002-03 school year and the first semester of the 2003-04 school (b) the results of the annual assessments of each program, including whether the assessments resulted in program modifications or the year
elimination of any programs
and (c) the names of the administrators who were mvolved with the assessment of each program, as well as at least a grade level description of any teachers who were involved in the assessment process {e.g., all fourth grade math teachers
all eighth grade Enghsh teachers, etc.). LRSD must use Dr. Nunnerlw or another expert from outside LRSD with equivalent qualifications and expertise to prepare program evaluations on each of the programs identified on page 148 of the Final Compliance Report. I will accept all program evaluations that have already been completed by Dr. Nunnerly or someone with similar qualifications and approved by the Board. All program evaluations that have not yet been completed on the remaining programs identified on page 148 of the Final Comphance Report must be prepared and approved by the Board as soon as practicable, but, in no event, later than March 15, 2003. In addition, as these program evaluations are prepared, LRSD shall use them, as part of the program assessment process, to determine the effectiveness of those programs in improving African-American achievement and whether, based on the evaluations, any changes or modifications should be made in those programs. In addition, LRSD must use those program evaluations, to the extent they may be relevant, in assessing the effectiveness of other related programs. * * * On or before March 15, 2004, LRSD must file a Comphance Report which documents its compliance with its obligations under 2.7.1. Any party, including Joshua, who wishes to challenge LRSDs substantial compliance with 2.7.1, as specified above, may file objections with the court on or before April 15, 2004. Thereafter, I will decide whether the LRSD has substantially complied with 2.7.1, as specified in the Compliance Remedy, and should be released from all further supervision and monitoring. The Court is clearly referring to Dr. John Nunnery. 2 iProposed Compliance Plan As the Compliance Committee understands the District Courts Opinion, the Compliance Remedy requires the LRSD to: 1. 2. 3. 4. 5. Continue to administer student assessments through the first semester of 2003-04
Develop written procedures for evaluating the programs implemented pursuant to Revised Plan 2.7 to determine their effectiveness in improving the academic achievement of Afiucan-American students
Maintain written records of (a) the criteria used to evaluate each program
(b) the results of the annual student assessments, including whether an informal program evaluation resulted in program modifications or the elimination of any programs
and (c) the names of the administrators who were involved with the evaluation of each program, as well as at least a grade level description of any teachers who were involved in the evaluation process
Prepare a comprehensive program evaluation of each academic program implemented pursuant to Revised Plan 2.7 to determine its effectiveness in improving the academic achievement of Afiican-American students and to decide whether to modify or replace the program
and Submit for Board approval the program evaluations identified on page 148 of the LRSD s Final Compliance Report that have been completed, and complete, with the assistance of an outside expert, the remaining evaluations identified on page 148 of the LRSDs Final Compliance Report. What follows is an explanation of how the Compliance Committee derived these five requirements from the District Courts Opinion, and what the Compliance Committee proposes to do to comply with each requirement. Assessment and Evaluation When first read, the District Courts Compliance Remedy seemed simple and straightforward, but as the Compliance Committee attempted to develop this Proposed Compliance Plan, numerous questions arose. The most fundamental question related to the District Courts use of the term assessment in Paragraphs A and B of the Compliance Remedy. The ambiguity of this term was the subject of testimony at the hearing. The District Court included in its Opinion Dr. Lesleys testimony on the difference between assessment and evaluation, see Opinion, p. 152, but it is unclear whether the Court accepted this testimony. 3It is clear that the District Court understood the distinction between testing data, which are derived from student assessments, and program evaluations, which are used to determine the effectiveness of programs. Opinion, p. 152 (LRSD ackno^vledged in the Interim Compliance Report that it was required
(a) to use both the testing data and the program evaluations to determine the effectiveness of the key academic programs implemented pursuant to 2.7 . .. (emphasis in original)). Even so, the District Court appears to have used the term assessment in some instances to refer to only student assessments and in other instances to refer to both student assessments and evaluations. This required the Comphance Committee to determine the District Courts intended meaning. In making this determination, the Compliance Committee considered the context in which the term was used, the District Courts findings of fact as set forth in the Opinion, what would be in the best interest of African-American students, and hopefully, common sense. An explanation of each requirement of the Compliance Remedy is provided below. To avoid any ambiguity, Compliance Committee hereinafter uses the term assessment to refer to student assessments and the term evaluation to refer to the program evaluations, whether formal or informal. 1. Continue to administer student assessments through the first semester of 2003-04. This requirement derives from Paragraph A of the Compliance Remedy. Given Paragraph A s reference to testing data, it seems clear that Paragraph A concerns, in part. ^'1* A X WAA w A IO XXX XXX Uj student assessments. The Compliance Committee proposes to comply with this part of Paragraph A by implementing the 2002-03 Board-approved assessment plan. The 2002-03 Board-approved assessment plan incorporates four changes that have been made since the LRSDs Final Compliance Report. First, the Board eliminated the fall administrations of the Achievement Level Tests (ALTs) in 2001-02. The administration recommended this for three reasons
(1) the loss of instructional time resulting from testing and test preparation
(2) fall results did not provide significantly different information from the previous springs results
and (3) the cost of administering and scoring the tests. Second, the fall administration of the Observation Surveys and Developmental Peading Assessment will only be used by the teacher for diagnostic purposes. The scores will not be reported to or maintained by the LRSD. This change saves considerable time in test administration and allows more time for instruction. It was approved by the Board on September 26, 2002. Third, the LRSD will no longer admimster the ALTs. The administration recommended the complete elimination of the ALTs for the following reasons: (1) the lack of alignment with the content and format of the State Benchmarks
(2) the loss of instructional time resulting from 4testing and test admimstration
(3) the new federal accountability requirements in the No Child Left Behind Act require annual testing by the State in grades 3-8, making the LRSDs administration of the ALTs redundant
and (4) the costs of administering and scoring the tests. The Board approved this change on September 26, 2002. Finally, the Arkansas Department of Education (ADE) has moved the admini stratinn of the SAT9 from the fall to the spring, effective 2002-03. The 2002-03 Board-approved assessment plan calls for the administration of the following student assessments in English language arts and mathematics: Kindergarten Grade 1 Grade 2 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Grades 7-10 Grades 9-11 Grade 10 Grade 11 Observation Surveys (5) Developmental Reading Assessment Observation Surveys (5) Development Reading Assessment Observation Surveys (3) Development Reading Assessment Norm-referenced test to be identified for gifted/talented screening Benchmark Literacy examination Benchmark Mathematics examination SAT9 Total Battery Benchmark Literacy examination Benchmark Mathematics examination SAT9 Total Battery Benchmark Literacy examination Benchmark Mathematics examination End-of Course Algebra I examination End-of Course Geometry examination SAT9 Total Battery End-of-Level Literacy examination All of these assessments are administered in the spring. Consequently, the final student assessment before March 15, 2004, will be administered in the spring of 2003. 7 Develop written procedures for evaluating the programs implemented pursuant to 2.7 to determine their effectiveness in improving the academic achievement of African-American students. This requirement derives from the opening paragraph of the Compliance Remedy. To comply with this requirement, two proposed regulations have been drafted, EL-Rl for formal evaluations and IL-R2 for informal evaluations, attached as Appendixes 1 and 2, respectively. 5 1Proposed regulation IL-Rl combines generally accepted principles of program evaluation with practices that have been in place in the LRSD for the past two years. See, e.g., Robby Champion, Map Out Evaluation Goals, Journal for Staff Development, Fall 2002, attached as Appendix o. This regulation will be submitted to the Board, Office of Desegregation Monitoring ( ODM ) and the Joshua Intervenors (Joshua) for review and comment before being finalized. Proposed regulation IL-R2 specifically addresses the next requirement and is discussed therewith. 3. Maintain written records of (a) the criteria used to evaluate each program
(b) the results of the annual student assessments, including whether an informal program evaluation resulted in program modifications or the elimination of any programs
and (c) the names of the administrators who were involved with the evaluation of each program, as well as at least a grade level description of any teachers who were involved in the evaluation process. This requirement derives from Paragraph B of the Compliance Remedy. Paragraph B apparently came about as a result of the District Courts concern about the LRSD making program modifications based on informal evaluations of student assessment data. See Opinion, p. 155 ( I have grave reservations about anyone this side of Solomon being wise enough to use two or three semesters worth of erratic composite test scores to make reliable decisions about which remediation programs for LRSD s Afihcan-American students were actually working.). Proposed regulations IL-R2 was drafted to specifically address this requirement. It prohibits substantial program modifications from being made without a written record as required by Paragraph B. This regulation will also be submitted to ODM and Joshua for review and comment before being finalized. Proposed regulation IL-Rl also complies with this requirement. It mandates that the criteria used to formally evaluate a program be identified as the research questions to be answered, the first of which will be, Has this curriculum/instruction program been effective in improving and remediating the academic achievement of Afincan-American students?. See Appendix 1, IL-Rl, p. 5. Recommended program modifications and the members of the evaluation team are routinely included in formal evaluations. As to the results of annual student assessments, the LRSD will continue to maintain a computer database with the results of annual students assessments administered pursuant to the Board-approved assessment plan. 64. Prepare a comprehensive program evaluation of each academic program implemented pursuant to 2.7 to determine its effectiveness in improving the academic achievement of African-American students and to decide whether to modify replace the program. or This requirement derives from Paragraph A of the Compliance Remedy. To comply with this requirement, the Compliance Committee proposes to prepare the following new, comprehensive evaluations: (a) Primary Readingkanguage Arts, (b) Middle and High School Literacy and (c) K-12 Mathematics and Science, Each evaluation will be prepared in accordance with proposed Regulation IL-Rl and will incorporate all available student assessment data relevant to the program being evaluated. Based on Paragraph F of the Compliance Remedy, the LRSD understands these evaluations must be submitted to the Court on or before March 15 2004. Some may argue that Paragraph A and Paragraph C together require the LRSD to prepare new, comprehensive evaluations of all the programs identified on page 148 of the LRSDs Final Compliance Report. The Compliance Committee considered and rejected this argument for three reasons. First, Paragraph As description of the programs to be evaluated differs from that of Paragraph C. Paragraph A states that the LRSD must continue to assess each of the programs implemented imder 2.7 . . . The Compliance Committee understands this to mean that the LRSD should continue to prepare evaluations of some of the key programs, as identified in the Interim Compliance Report. See Opinion, p. 151 (In addition to the Assessment Plan, 2.7.1 of the Interim Compliance Report noted that the LRSD was preparing evaluations of some of the key programs designed to improve African-American achievement in order to provide a more in-depth look at the effectiveness of those programs. (emphasis in original)), In contrast to Paragraph A, Paragraph C requires the LRSD to prepare evaluations of each of the programs identified on page 148 of the Final Compliance Report. The Compliance Committee understands this to mean that the LRSD should complete all of the evaluations identified on page 148 of the Final Compliance Report and submit those to the Court. S^ Opinion, p. 156 ( [A]s of March 15, 2001, the date the Final Compliance Report was filed with the Court. (1) PRE had prepared only draft evaluations of some of the programs in question
(2) none of those evaluations had been approved by the Board . .. . (emphasis in original)). The District Court s statement in Paragraph C that it will accept evaluations already completed and approved by the Board further indicates that Paragraph C does not require new, comprehensive evaluations. Second, recognizing this distinction between Paragraph A and Paragraph C resolves a potential conflict between Paragraph C and Paragraph F. Paragraph C provides, All program evaluations that have not yet been completed on the remaining programs identified on page 148 7 1 Iof the Final Compliance Report must be prepared and approved by the Board as soon as practicable, but, in no event, later than March 15, 2003. However, Paragraph F does not require the LRSD to file a compliance report on its compliance with Revised Plan 2.7.1 until March 15, 2004. The Compliance Committee concludes that March 15,2004, is the deadline for submitting the new, comprehensive evaluations of the programs implemented pursuant to 2.7. See Paragraph A of Compliance Remedy. This is consistent with Paragraph As requirement that the LRSD include assessment data through December 31, 2003. Obviously, such data could not be included in an evaluation filed on or before March 15, 2003. Finally, it makes the most sense for the LRSD to expend the greatest time and resources preparing evaluations of the programs designed to improve Afincan- American achievement. While the requirement for new, comprehensive evaluations derives from Paragraph A, some may argue that Paragraph Cs requirement that the LRSD use an outside expert to prepare evaluations of each of the programs identified on page 148 of the Final Compliance Report applies to the new, comprehensive evaluations. The Compliance Committee hopes the District Court and the parties agree that the team approach to program evaluation set forth in proposed regulation IL-Rl renders this argument moot. Proposed Regulation IL-Rl states that the program evaluation team must include [a]n external consultant with expertise in program evaluation, the program area being evaluated, statistical analysis, and/or technical writing . . . Appendix 1, p. 4. The exact role of the external consultant may vary, depending upon the expertise required for the production of the program evaluation. Id. The Compliance Committee believes that the LRSDs practice over the last two years of using the team approach to program evaluation has produced credible evaluations. Moreover, participation of the LRSD staff on the evaluation team provides them an excellent learning experience that they do not typically receive when an evaluation is prepared entirely by an outside expert. The evaluations prepared over the last two years using the team approach are as follows: 1. 2. Dr. Steve Ross was the external consultant in the production of the Early Literacy program evaluation for 1999-2000 and 2000-01. He was asked to read a nearfinal draft and to provide feedback, which he did. His suggestions were then incorporated into the final report before it was published and disseminated. Other team members included Bonnie Lesley (associate superintendent), Patricia Price (program director), Pat Busbea (program specialist), Ed Williams (statistician)^ and Ken Savage (computer programmer). Dr. Julio Lopez-Ferraro is the National Science Foundation (NSF) program officer who over-sees the LRSDs implementation of the grant-funded 8 j iComprehensive Partnership for Mathematics and Science Achievement ( CPMSA ). NSF trained a team of LRSD staff to produce the mandated annual program evaluations for this initiative and then assembled an external team of practitioners and researchers who came to the LRSD each year to validate findings and provide written feedback. The LRSD team members who OUT participated in writing of the annual progress reports included Vanessa Cleaver (project director), Dennis Glasgow (director of mathematics and science), Bonnie Lesley (associate superintendent and co-project investigator), Virginia Johnson (CPMSA program evaluator), Ed Williams (statistician), and Ken Savage (computer programmer). 3. 4. Mr. Mark Vasquez, an attorney and former employee of the Office for Civil Rights in Dallas, has been retained by the LRSD for the past three years to provide guidance in the design and production of the English as a Second Language (ESL) program evaluation. Other team members have been Bonnie Lesley (associate superintendent), Karen Broadnax (program supervisor), Ed Williams (statistician), Ken Savage (computer programmer), and Eddie McCoy (program evaluator). Dr. Larry McNeal, a professor at the University of Arkansas at Little Rock in education admimstration and a private consultant in program evaluation, was retained by the LRSD to lead the team that produced the program evaluation for the Charter School. Other members of that team included Linda Watson (assistant superintendent), Knsta Young (program director), and Ed Williams (statistician). Dr. McNeal wrote this report. The team approach, supported by an external expert, ensures that all areas of expertise (program, implementation, techmeal and evaluative) are included. No one person would have all the knowledge and skills that a team would have. As these examples show, the external expert does not always perform the same role in every project. Rather, the role changes, depending the expertise that is required for a credible report. on 5. Submit for Board approval the program evaluations identified on page 148 of the LRSDs Final Compliance Report that have been completed, and complete, with the assistance of an outside expert, the remaining program evaluations identified on page 148 of the LRSDs Final Compliance Report. The following program evaluations identified on page 148 of the Final Compliance Report have been completed: 1. Early Literacy. A comprehensive report for 1999-2000 and 2000-01 was prepared, completed, and presented to the Board in fall 2001. An update to this report for 2001-02 was presented to the Board in June 2002, with an emphasis on 9the improved achievement of African-American students and closing the achievement gap. 2. 3. 4. 6. 7. 8. 9. 10. Mathematics and Science. Three years (1998-99, 1999-2000, and 2000-01) of program evaluations as required by the NSF were prepared, presented to the Board, and submitted to NSF, and NSF has responded to each evaluation. Extended Year Schools. The LRSD staff prepared, completed, and presented to the Board in the spring of 2002 an evaluation of the Extended Year Schools. Elementary Summer School. The LRSD staff prepared, completed, and provided to the School Services Division an evaluation of elementary summer school programs for 2000-01. 5 HIPPY. The HIPPY program was evaluated by the LRSD staff in July 1999. The report was prepared, completed, and submitted to the program director and the Cabinet. Charter School. This program evaluation was prepared, completed, and presented to the Board in June 2001. ESL. The Office for Civil Rights has required the LRSD to prepare a program evaluation in this area for each of the past three years: 1999-2000, 2000-01, and 2001-02. The first two of these reports have been prepared, completed, submitted to the Board, and submitted to OCR. (A third program evaluation will be completed in October when state scores arrive and will be ready by the March 15, 2003 deadline). Lyceum Scholars Program. Two separate evaluations of this alternative education school program were prepared by the LRSD staff. Southwest Middle School's SEDL Program. Southwest Middle School was the recipient of a two-year technical assistance grant from the Southwest Educational Development Lab (SEDL) to build professional community. SEDL prepared a comprehensive program evaluation that included Southwest among other grant recipients outside the LRSD. The LRSD staff provided SEDL data for this evaluation. Onward to Excellence (Watson Elementary). A grant from ADE funded a partnership between Watson Elementary and the Northwest Educational Development Lab to implement a school improvement initiative. The LRSD staff provided data to Watsons principal for preparation of program evaluations. The principal submitted two armual program evaluations to ADE. 10 t11. 12. Collaborative Action Team (CAT). This one-year partnership with SEDL provided in 2000-01 for establishing and training a Collaborative Action Team of parent and community volunteers supported by LRSD staff to improve parent involvement. SEDL wrote a 249-page evaluation of their three-year grant-funded program, of which LRSD was included only the last year. The LRSD staff provided SEDL data for this evaluation. Vital Link. The LRSD staff prepared a program evaluation, and it was provided to the project director. A question arises as to which of these evaluations are acceptable to the Court without additional work. The first sentence of Paragraph C of the Compliance Remedy provides, LRSD must use Dr. Nunnerly (sic) or another expert from outside LRSD with equivalent quahfications and expertise to prepare program evaluations of each of the programs identified on page 148 of the Final Compliance Report. The second sentence of Paragraph C states that the District Court will accept all program evaluations that have already been completed by Dr. Nunnerly (sic) or someone with similar qualifications. It is unclear whether an expert from outside the LRSD must have prepared the completed evaluations for them to be accepted by the District Court, whether it is sufficient that they were prepared by someone within LRSD with similar or qualifications. >5 The District Courts findings of fact suggest that the District Court will accept only program evaluations already completed by an outside expert. The District Court noted that Dr. Lesley testified hhat, by the end of November 2000, it was her opinion that no one in PRE had the expertise to prepare program evaluations. Opinion, p. 153. Thus, the District Court likely concluded that the only acceptable program evaluations would be those prepared by persons outside the LRSD. Applying this standard, the Compliance Committee believes that the following evaluations are acceptable to the Court, following Board approval, without additional work: Early Literacy, Mathematics and Science, Charter School, ESL, Southwest Middle Schools SEDL Program and CAT. The remaining program evaluations identified on the bottom of page 148 of the Final Compliance Report must be completed by an outside expert. They are: Extended Year Schools, Middle School Implementation, Elementary Summer School, HIPPY, Campus Leadership Teams (CLTs), Lyceum Scholars Program, Onward to Excellence and Vital Link. The Compliance Committees proposal for completing each of these evaluations will be discussed below. In deciding how to go about completing these evaluations, the Comphance Committee focused on what makes sense to do at this time considering the goal of improving Afincan-American achievement and the limitations inherent in asking an expert to complete an evaluation. 11Extended Year Schools. This evaluation was completed by the LRSD staff. The Compliance Committee proposes retaining an outside expert to review the report and, if possible, draw conclusions and make recommendations based on the existing data. Middle School Implementation. A draft of this evaluation was presented to the Board in July and August 2000, but it was never completed. The Compliance Committee proposes retaining an outside expert to rewrite the report and, if possible, prepare an evaluation based on the existing data. Elementary Summer School. This evaluation was completed by the LRSD staff. The Compliance Committee proposes retaining an outside expert to review the report and, if possible, draw conclusions and make recommendations based on the existing data. HIPPY. This evaluation was completed by the LRSD staff. The Compliance ComTnirtee proposes retaining an outside expert to review the report and, if possible, draw conclusions and make recommendations based on the existing data. CLTs. The LRSD staff conducted a survey of CLTs during 2000-01. A summary of the survey findings was presented during a CLT training session, but no formal report was ever prepared. The Compliance Committee proposes retaining an outside expert to review the survey data and, if possible, prepare an evaluation based on the existing survey data. Lyceum Scholars Program. This evaluation was completed by the LRSD staff. The Compliance Committee proposes retaining an outside expert to review the report and, if possible, draw conclusions and make recommendations based on the existing data. Onward to Excellence. This evaluation was completed by the LRSD staff. The Compliance Committee proposes retaining an outside expert to review the report and, if possible, draw conclusions and make recommendations based on the existing data. Vital Link. This evaluation was completed by LRSD staff. The Compliance Committee proposes retaining an outside expert to review the report and, if possible, draw conclusions and make recommendations based on the existing data. 12Action Plan Timeline The Compliance Committee proposes implementation of this Compliance Plan in accordance with the following timeline. 1. Provide copies of this proposed Compliance Plan to ODM and Joshua for their reactions. 2. Incorporate, as possible, suggested revisions from ODM and Joshua.______ 3. Place Compliance Plan on the agenda for Board review and approval.______ 4. Place 2002-03 Program Evaluation Agenda on the Boards agenda for review and approval. 5. Place on Board agenda for approval two previously presented program evaluations (early literacy, and charter school).______ 6. Place on Board agenda for approval the evaluations of Southwest Middle Schools SEDL program and the Collaborative Action Team (also conducted by SEDL). 7. Place on Board agenda for approval the previously jresented ESL program evaluations for 1999-2000 and 2000-01, plus the new evaluation for 2001-02. Week of September 30, 2002 Week of October 7, 2002 October 10, 2002 October 24, 2002 October 24, 2002 November 2002 November 2002 13 Clay Pendley Ken James Attorneys Ken James Compliance Team Ken James Attorneys Ken James Bonnie Lesley Bonnie Lesley Linda Watson Bonnie Lesley Bonnie Lesley Karen Broadnax8. Place on Board agenda for approval the three previously presented program evaluations for the NSF-funded CPMSA program, plus the new Year 4 report for 2001-2002. 9. Issue Request for Proposals (RFPs) from available external experts to review and complete the eight remaining program evaluations listed on page 148.________________ 10. Form a screening team to determine recommendations to the Superintendent for designating external experts to review and complete the eight remaining program evaluations listed on page 148.____________________ 11. Select and negotiate consulting contracts with designated external experts. 12. Assign appropriate staff to each external expert to provide needed information, data, access to program staff, etc.________ 13. Monitor the work to ensure timely completion. 14. As each paper is completed and ready for circulation, send copies to ODM and Joshua for their review and comments. December 2002 Mid-October 2002 Late October 2002 Mid-November 2002 Mid-November 2002 Mid-November 2002^February 2003 December 2002^February 2003 14 Bonnie Lesley Vanessa Cleaver Dennis Glasgow Bonnie Lesley Darral Paradis Ken James Compliance Team Bonnie Lesley Ken James Bonnie Lesley Bonnie Lesley Bonnie Lesley15. As each paper is completed, place on the Boards agenda the item to be reviewed and approved. 16. Write Interim Compliance Report relating to programs on page 148 to be completed. 17. Establish staff teams for each of the three programs on the Boards Program Evaluation Agenda to be completed for 2002-2003 (Elementary Literacy, Secondary Literacy, and K- 12 Mathematics/ Science). 18. Publish RFPs to identify external experts to serve on each of the two staff teams for the Boards Program Evaluation Agenda (K-12 mathematics/science external experts are provided by NSF). 19. Establish consulting contracts with the two external experts required for the Elementary Literacy and Secondary Literacy program evaluations. 20. Train each program evaluation team, including the external expert, on the requirements of the approved Compliance Plan and IL-R. December 2002F ebruary 2003 March 15, 2003 March 1, 2003 March 1, 2003 Late March 2003 May 2003 15 Ken James Bonnie Lesley Attorneys Compliance Committee Bonnie Lesley Bonnie Lesley Darral Paradis Bonnie Lesley Bonnie Lesley21. Monitor the completion MayOctober 2003 of the work on all three program evaluations required in the Boards Program Evaluation Agenda.___________ 22. Send copies of the completed Elementary Literacy program evaluation to ODM and Joshua for information.___________ 23. Complete the evaluation of the Elementary Literacy program and place on the Boards agenda for approval. 24. Send copies of the Secondary Literacy program evaluation to ODM and Joshua for information. 25. Complete the evaluation of the Secondary Literacy program and place on the Boards agenda for approval._____________ 26. Send copies of the completed CPMSA program evaluation to ODM and Joshua for information. 27. Complete the five-year evaluation of the CPMSA moject (science and mathematics) and place on the Boards agenda for approval.___________ 28. Write Section 2.7.1 "inal Compliance Report or federal court and file with Court. With October 2003 Board agenda packet October board meeting, 2003 With November 2003 Board agenda packets November board meeting, 2003 With December 2003 Board agenda packet December board meeting, 2003 March 15, 2004 16 Bonnie Lesley Ken James Bonnie Lesley Bonnie Lesley Pat Price Ken James Bonnie Lesley Bonnie Lesley Pat Price Ken James Bonnie Lesley Bonnie Lesley Vanessa Cleaver Dennis Glasgow Ken James Attorneys Compliance Team JAppendix 1 Proposed IL-Rl ILITTLE ROCK SCHOOL DISTRICT NEPNCODE: IL-R1 PROGRAM EVALUATION AGENDA Purpose The purpose of these regulations is to provide guidance to the staff involved in the evaluation of programs required in the Boards Program Evaluation Agenda. They do not necessarily apply to grant-funded programs if the funding source requires other procedures and provides funding for a required evaluation. Criteria for Program Evaluations Policy IL specifies that the evaluations of programs approved in its Board- approved Program Evaluation Agenda shall be conducted according to the standards developed by the Joint Committee on Standards for Educational Evaluation. (See Joint Committee on Standards for Educational Evaluation, James R. Sanders, Chair (1994). The Program Evaluation Standards, 2^ Edition: How to Assess Evaluations of Educational Programs. Thousand Oaks, CA: Sage Publications.) They are as follows: Utility Standards The utility standards are intended to ensure that an evaluation will serve the information needs of intended users. These standards are as follows: Stakeholder identification. People involved in or affected by the evaluation should be identified so that their needs can be addressed. Evaluator credibility. The people conducting the evaluation should be both trustworthy and competent to perform the evaluation so that the evaluation findings achieve maximum credibility and acceptance. Information scope and sequence. Information collected should be broadly selected to address pertinent questions about the program and should be responsive to the needs and interests of clients and other specified stakeholders. Values identification. The perspectives, procedures, and rationale used to interpret the findings should be described carefully so that the bases for value judgements are clear. Report clarity. Evaluation reports should describe clearly the program being evaluated, including its context and the purposes, procedures, and findings of the evaluation, so that essential information is provided and understood easily. 1Report timeliness and dissemination. Significant interim findings and evaluation reports should be disseminated to intended users so that they can be used in a timely fashion. Evaluation impact. Evaluations should be planned, conducted, and reported jn ways that encourage follow-through by stakeholders, so that the likelihood that the evaluation will be used is increased. Feasibility Standards Feasibility standards are intended to ensure that an evaluation will be realistic, prudent, diplomatic, and frugal. Practical procedures. Evaluation procedures should be practical so that the disruption is kept to a minimum while needed information is obtained. Political viability. The evaluation should be planned and conducted with anticipation of the different positions of various interest groups so that their cooperation may be obtained, and so that possible attempts by any of these groups to curtail evaluation operations to vias or misapply the results can be averted or counteracted. or Cost-effectiveness. The evaluation should be efficient and produce information of sufficient value so that the resources expended can be justified. Propriety Standards The propriety standards are intended to ensure that an evaluation will be conducted legally, ethically, and with due regard for the welfare of those involved in the evaluation, as well as those affected by its results. Service orientation. Evaluations should be designed to assist organizations to address and effectively serve the needs of the full range of targeted participants. Formal agreements. Obligations of the formal parties to an evaluation (what is to be done, how, by whom, and when) should be agreed to in writing so that these parties are obligated to adhere to all conditions of the agreement or to formally renegotiate it. Rights of human subjects. Evaluations should respect human dignity and worth in their interactions with other people associated with an evaluation so that participants are not threatened or harmed. Complete and fair assessments. The evaluation should be complete and fair in its examination and recording of strengths and weaknesses of the program being evaluated so that strengths can be built upon and problem areas addressed. Disclosure of findings. The formal parties to an evaluation should ensure that the full set of evaluation findings, along with pertinent limitations, are made accessible to the people affected by the 2 (evaluation, as well as any others with expressed legal rights to receive the results. Conflict of interest. Conflict of interest should be dealt with openly and honestly so that it does not compromise the evaluation processes and results. Fiscal responsibility. The evaluators allocation and expenditure of resources should reflect sound accountability procedures and be prudent and ethically responsible so that expenditures are accounted for and appropriate. Accuracy Standards Accuracy standards are intended to ensure that an evaluation will reveal and convey technically adequate information about the features that determine the worth of merit of the program being evaluated. Program documentation. The program being evaluated should be described and documented clearly and accurately so that it programs is identified clearly. Context analysis. The context in which the program exists should be examined in enough detail so that its likely influences on the program can be identified. Described purposes and procedures. The purposes and procedure of the evaluation should be monitored and described in enough detail so that they can be identified and assessed. Defensible information sources. The sources of information used in a program evaluation should be described in enough detail so that the adequacy of the information can be assessed. Valid information. The information-gathering procedures should be chosen or developed and then implemented in a manner that will ensure that the interpretation arrived at is valid for the intended use. Reliable information. The information-gathering procedures should be chosen or developed and then implemented in a manner that will ensure that the information obtained is sufficiently reliable for the intended use. Systematic information. The information collected, processed, and reported in an evaluation should be review systematically so that the evaluation questions are answered effectively. Analysis of quantitative information. Quantitative information in an evaluation should be analyzed appropriately and systematically so that the evaluation questions are answered effectively. Analysis of qualitative information. Qualitative information in an evaluation should be analyzed appropriately and systematically so that the evaluation questions are answered effectively. Justified conclusions. The conclusions reached in an evaluation should be justified explicitly so that stakeholders can assess them. 3Impartial reporting. Reporting procedures should guard against distortion caused by personal feelings and biases of any party the evaluation reports reflect the evaluation findings fairly. Metaevaluation. The evaluation itself should be evaluated so formatively and summartively against these and other pertinent standards so that its conduct is appropriately guided, and on completion, stakeholders can closely examine its strengths and weaknesses. Program Evaluation Procedures The following procedures are established for the evaluation of programs approved by the Board of Education in its annual Program Evaluation Agenda: 1. 2. 3. The Division of Instruction shall recommend to the Superintendent annually, before the budget for the coming year is proposed, the curriculum/instruction programs for comprehensive program evaluation. The recommendation shall include a proposed budget, a description of other required resources, and an action plan for the completion of the reports. Criteria for the proposed agenda are as follows: A. Can the results of the evaluation influence decisions about the program? B. Can the evaluation be done in time to be useful? C. Is the program significant enough to merit evaluation? (See Joseph S. Wholey, Harry P, Hatry, and Kathryn Newcomer (1994). Handbook of Practical Program Evaluation. San Francisco, CA: Jossey- Bass Publishers. 5-7.) The Superintendent shall recommend to the Board of Education for approval the proposed Program Evaluation Agenda^with anticipated costs and an action plan for completion. For each curriculum/instruction program to be evaluated as per the Program Evaluation Agena, the Associate Superintendent for Instruction shall establish a staff team with a designated leader to assume responsibility for the production of the report according to the timelines established in the action plan approved by the Board of Education. 4. Each team shall include, at a minimum, one or more specialists in the curriculum/instruction program to be evaluated, a statistician, a programmer to assist in data retrieval and disaggregation, and a technical writer. If additional expertise is required, then other staff may be added as necessary. 5. An external consultant with expertise in program evaluation, the program area being evaluated, statistical analysis, and/or technical writing shall be retained 4as a member of the team. The role of the external consultant may vary, depending upon the expertise required for the production of the program evaluation. 6. The team leader shall establish a calendar of regularly scheduled meetings for the production of the program evaluation. The first meetings will be devoted to the following tasks: A. Provide any necessary training on program evaluation that may be required for novice members of the team, including a review of the Board s policy IL and all of the required criteria and procedures in these regulations, IL-R. B. Assess the expertise of each team member and make recommendations to the Associate Superintendent for Instruction related to any additional assistance that may be required. C. Write a clear description of the curriculum/instruction program that is to be evaluated, with information about the schedule of its implementation. D. Agree on any necessary research questions that need to be established in addition to the question, Has this curriculum/instruction program been effective in improving and remediating the academic achievement of African-American students? (See Policy IL, 2.7.1 of the Revised Desegregation and Education Plan, and Judge Wilsons Compliance Remedy.) E. Generate a list of the data required to answer each research question, and assign responsibility for its collection and production. All available and relevant student performance data must be included. (See Judge Wilsons Compliance Remedy.) F. Decide who will be the chief writer of the program evaluation. G. Plan ways to provide regular progress reports {e.g., dissemination of meeting minutes, written progress reports, oral reports to the Superintendent s Cabinet and/or Compliance Team) to stakeholders, including the Associate Superintendent for Instruction, the Superintendent of Schools, the Office of Desegregation Monitoring (until Unitary Status is achieved), and the Joshua Intervenors (until Unitary Status is achieved). (See Joellen Killion (2002). Assessing Impact: Evaluating Staff Development Oxford, OH. National Staff Development Council (NSDC)
Robby Champion (Fall 2002). Map Out Evaluation Goals." Journal of Staff Development. 78-79
5Thomas R. Guskey (2000). Evaluating Professional Development. Thousand Oaks, CA. Corwin Press
Blaine R. Worthen, James R. Sanders, and Jody L. Fitzpatrick (1997). Participant-Oriented Evaluated Approaches. Program Evaluation: Alternative Approaches and Practical Guidelines
153-169
Beverly A. Parsons (2002). Evaluative Inquiry: Using Evaluation to Promote Student Success. Thousand Oaks, CA: Corwin Press
and Joseph S. Whoiey, Harry P. Hatry, and Kathryn E. Newcomer (1994). Handbook of Practical Program Evaluation. San Francisco, CA: Jossey-Bass Publishers.) 7. Subsequent meetings of the program evaluation team are required for the following tasks: to monitor the completion of assignments
to collaborate in the interpretation and analysis of data
to pose any necessary new questions to be answered
to review drafts and provide feedback to the writer
to formulate recommendations, as required, for program improvement, especially to decide if a recommendation is required to modify or abandon the program if the findings reveal that the program is not being successful for the improvement of African- American achievement
I? to assist in final proofreading
and to write a brief executive summary, highlighting the program evaluation findings and recommendations. I s 8. A near-final copy of the program evaluation must be submitted to the Associate Superintendent for Instruction at least one month before the deadline for placing the report on the Boards agenda for review and approval. This time is required for final approval by staff, for final editing to ensure accuracy, and for submission to the Superintendent. 9. When the program evaluation is approved for submission to the Board of Education for review and approval, copies of the Executive Summary and complete report must be made for them, for members of the Cabinet, for ODM (until Unitary Status is achieved), and for the Joshua Intervenors (until Unitary Status is achieved). s I 3 I 10. The program evaluation team shall plan its presentation to the Board of Education on the findings and recommendations. i j 611 .The Associate Superintendent for instruction shall prepare the cover memorandum to the Board of Education, including all the required background information (see Judge Wilsons Compliance Remedy): A. If program modifications are suggested, the steps that the staff members have taken or will take to implement those modifications. If abandonment of the program is recommended, the steps that will be taken to replace the program with another with more potential for the improvement and remediation of African-American students. (See Section 2.7.1 of the Revised Desegregation and Education Plan and Judge Wilsons Compliance Remedy.) B. C, D. Names of the administrators who were involved in the program evaluation. Name and qualifications of the external expert who served on the evaluation team. Grade-level descriptions of the teachers who were involved in the assessment process (e.g., all fourth-grade math teachers, all eighth grade English teachers, etc.). 10. When the program evaluation is approved by the Board of Education, the team must arrange to have the Executive Summary and the full report copied and design a plan for communicating the program evaluation findings and recommendations to other stakeholders. This plan must then be submitted to the Associate Superintendent for approval. 11. Each program evaluation team shall meet with the Associate Superintendent for Instruction after the completion of its work to evaluate the processes and product and to make recommendations for future program evaluations. (See Joeilen Killion (2002). Evaluate the Evaluation. Assessing Impact: Evaluating Staff Development. Oxford, OH: National Staff Development u Council. 46, 123-124.) 7Appendix 2 Proposed IL-R2LITTLE ROCK SCHOOL DISTRICT NEPNCODE: IL-R2 INFORMAL PROGRAM EVALUATION Introduction The purpose of this regulation is to ensure that a written record exists explaining a decision to significantly modify an academic program. It is not the intent of this regulation to require a formal program evaluation before every significant program modification. Definitions Academic Program means one of the core curriculum programs of English/Language Arts, Mathematics, Science or Social Studies. Significantly modify means a material change in the content or delivery of an academic program implemented throughout the entire District. Written Record A written record must be prepared and maintained explaining a decision to significantly modify an academic program. The written record required by this regulation must include the following information: (a) the written criteria used to evaluate the program, (b) a summary of the student assessment data or other data on which the decision was based
and (c) the names of the administrators who were involved with the evaluation of each program, as well as at least a grade level description of any teachers who were involved in the evaluation process (e.g., all fourth grade math teachers
all eighth grade English teachers etc.). 1Appendix 3 Robby Champion, Map Out Evaluation Goals, Journal for Staff Development, Fall 2002 ?9t a k i n g ROBB-Y CHAMPION Map out evaluation goals A master plan can guide you down the rocky path of evaluation when you launch a major professional development evaluation, regardless of the projects scope, you may quickly find yourself on a slippery, often rocky road, v/ith twists and unexpected turns. Before venturing too far and becoming disillusioned about program evaluation, create a master plan. 'While it requires an upfront investment of time and may delay starting, it quickly becomes an invaluable road map that helps you avoid delays and detours along the way. Developing an evaluation master plan is most useful when you are launching a major, summative program evaluation. A summative evaluation is done at major junctures in a programs fife cycle and emphasizes documenting impact. Information from summative evaluations is used to make important decisions about the initiative, such as whether to continue, alter, expand, downsize, or eliminate it A formative evaluation, on the other hand, means monitoring and collecting data, often informally and spontaneously, throughout program implementation. Formative evaluation helps show implementers where to make adjustments so a program can eventually achieve significant results. A thoughtfully prepared master plan for a major evaluation effort would
Focus the evaluation effort and help implementers avoid being sidetracked by leadership changes and new opinions
Create a realistic timeline and work plan that B Robby Champion is president of Champion Training & Consulting, You can contact her at Champion Ranch at Trumbeli Canyon, Mora. NM 87732, (505) 387-2016, fax (505) 387-5581, e-mail
Robbychampion@aol.com. provides needed momentum for the work
Be a key informational document to provide an overview and answer specific questions throughout the process
Help recruit people to assist with the project on the myriad evaluation tasks
e Give the message that the evaluation will be open and not secretive. Whether your evaluation must be completed within a few months or wdl extend for several years, think through four phases of work before starting. PHASE 1: ORGANIZE THE PROCESS 1. Form a steering committee, including any needed outside expertise. 2. Learn more about program evaluation together. 3. Write a clear description of each program to be evaluated. 4. Agree on the primary purpose of the evaluation. .5. Plan how you will keep everyone informed along the way. Steering committees, charged specifically , with program evaluation, are important to focus attention and maintain the energy and momentum needed for the evaluation. They also help build a spirit of 'collaboration and open inquiry. And they keep the evaluation on track when other priorities might push the effort aside. Provide steering committee members with the tools to succeed. Members need not be evalu- 78 National Staff Development Council JSD Fall 2002 t. a
ltj 3l jg: JE St a JI r X I ation experts, but they do need information, support, and guidance to make informed decisions. They need background material to learn about program evaluation and examples of good evaluation studies. Finally, ey need access to experts on professional development, measurement, and the content areas of the training programs. - Before launching any evaluation effort, have a written description of each program to be evaluated. You would be amazed at the number of people who do not have a clear idea of what you mean by the New TeacherJmduction Program or the Early Literacy Initiative since so many different initiatives are being undertaken simultaneously around the school or district PHASE II: DESIGN THE EVALUATION 1. Generate questions to guide the evaluation. 2. Generate potential data sources/ instruments to address the questions. 3. Using a matrix to provide a birds-eye view, agree on e most important questions and the best data sources. 4. Decide if collecting data from a sample group is warranted to make the evaluation manageable. 5. Determine the evaluation approach that makes sense
quantitative vs
qualitative/naturalistic. 6. Gather or create the instruments for data collection. .7. Determine a realistic schedule for coUecting data. 8. Create a system for collecting, analyzing, and interpreting data. Decisions made in Phase EI are critical. They determine the technical quality of your evaluation. In the questions you select, you determine what to examine and what to ignore. When you finish with the design phase, your program evaluation wiU be shaped, to use a quantitative or a , quahtative model or a mixture of the two. In the design phase, you make oer major decisions, such as whether to use a sample group. You also decide whether to do an in-depth case study, wheer to ON THE WEB. See an example of a matrix to help guide evaluations at
www.nsdc.org/library/jsd/ champion234.html. survey the whole population, whether to use examples of smdent work instead of official documents such as student grades or standardized test scores, or whether to judge adult learners understanding of the training content with performance tasks during training or by exit tests, classroom observations, or smdent feedback. If the programs to be evaluated already have stated indicators of longterm impact, generating appropriate evaluation questions is much simpler than when programs have only vague, lofty goals. The steering committee may drift into the realm of program planning as you encounter hurdles like fuzzy program outcomes. To avoid making misinformed evaluation design decisions, involve program leaders in your discussions. Developing or gathering instruments and then collecting the data are the most expensive steps in any evaluation. Think strategically about which data to collect, from whom to collect it or where to find it, and the best time to collect it. Your organization may already be collecting data for another purpose that now can be used for program evaluation. Some public records, such as smdent attendance, may be valuable if, for example, 20% increase in smdent attendance at aU grade levels is one of your programs indicators of impact. PHASE III
PREPARE TO REPORT 1. Determine which audiences wiU want to know the results. 2. Consider several forums and formats to disseminate the results. 3. Plan reports, presentations, photo displays, graphs, charts, etc. Remember that your job is to make the evaluation results useful to your organization, so consider a range of ways to provide information to various groups. Consider briefs in the school or district newsletter, a handout updating staff about the schedule for data collection, five- minute progress updates in faculty meetings, bulleted statements on your web site, a digital picture- album of the programs results in classrooms with photos of smdents, and hallway displays of smdent work. If your final report is a formal document complete with examples of your data collection instruments, consider writing an executive summary of five pages or less to help.readers get the essential information. PHASE IV: CREATE THE WORK PLAN 1. List aU tasks to be completed for the whole evaluation.. 2, Create a realistic timeline. 3. Assign work. .4. Distribute the master plan. You -will have to be creative to accomplish ail the evaluation tasks. In education, we rarely have the luxury of contracting outsiders for the entire project Enlist steering committee members, partners, graduate smdents from the local university, and other talented critical friends to get the work done. One caution: For formal or summa- tive evaluations to be credible, avoid using insiders such as the program designers or implementers (coaches, mentors, trainers, or facilitators) to perform critical evaluation tasks that call for objectivity and distance. And be sure to get ongoing, high-quaiit)' technical expertise for the critical technical analysis. A CATALYST FOR REFLECTION Completing a major program evaluation usually serves as the catalyst for serious reflection on the current designs, policies, and practices of your professional development programs their goals, content, processes, and contexts. In fact, revelations are often so powerful that they bring about the realization that major changes are needed if significant results' are reaUy expected from professional development People frequently conclude that designing the evaluation should be the first step in the program planning process, rather than an afterthought during implementation. B JSD Fall 2002 National Staff Development Council 79John W. Walker, P.a. Attorney At Law 1723 Broadway Little Rock, Arkansas 72206 Telephone (501) 374-3758 FAX (501) 374-4187 JOHN W. WALKER SHAWN CHILDS Via Facsimile - 376-2147 October 10, 2002 OF COUNSEL ROBERT McHenry, pa donna J. McHenry 8210 Henderson Road Little Rock, Arkansas 72210 Phone: (501) 372-3425 Fax (501) 372-3428 Email: mchenryd^wbeU.net Mr. Chris Heller Friday, Eldredge & Clark 2000 Regions Center 400 West Capitol Little Rock, AR 72201 Re: Little Rock School District v. PCSSD, et al. Case No. 4:82CV00866 Dear Chris: Plan. This refers to your letter of October 4, 2002, providing LRSDs proposed Comphance The carts remedy and the general subject matter are too complex for us to provide all comments and objections we may ultimately have before todays Board meeting. We do note the following: 1. More consideration is needed of the programs to be identified as implementation , which are to be subjected to a comprehensive program evaluation cJocument at page 7 identifies three areas. We note the absence of specific reference and detail reg^dmg mteiventions / scaflFolding - areas of vital importance given the achievement patterns of .African Amencan students. We note also that the LRSD compliance report cited pursuant to Section 2.7 . . many more programs as designed to fulfill Section 2.7. 2. In a discussion prior to his testimony in the hearing Judge Wilson, we understood Dr Ross to indicate that the existing evaluation of the Pre-K - 2 literary program was not adequate' hp nntfitinn oorta A J_______ r-.i , , * The notation at page 4 of your document of the changed use u 3 DRA relates to part of the concerns he expressed. This undermines the LRSD of the Observation Survey and the in r . -------- ------------------- argument (page 1) that the existing evaluation, upon Board approval, will satisfy a part of the courts remedy. 3. The LRSD discussion about satisfying the courts order regarding the evaluations EXHIBITmentioned at page 148 of the compliance report does not seem to take account of the material provided, which describes an adequate evaluation. 4. We question the period for implementation of a remedy which the court has identified and, therefore, the LRSD schedule. Once again, these comments should not be taken to be the full range of concerns which Joshua may ultimately have about the courts remedy and the Compliance Plan. Nor do we intend to waive our concerns about the court setting forth a remedy, without first hearing from the parties and the ODM with regard to the courts views on an appropriate remedy. ihc^ly. ohn W. Walker JWW:js cc: Ms. Ann Marshall All Counsel of Record, OCT. 24.2002 8:06ftM JOHN W WALKER P A NO.963 P.E JOHN w. Walker SHAWN CHILDS I John W. Wllker, p.a. Attorne? At Law 1723 Broadway Little Rock, Arkansas 72206 Telephone (501) 374-3758 FAX (501) 374-4187 October 23, 2002 OR COUNSEL ROBERT McHENKT P^ 8210 Hendeksqn BqaD Lrrtm Sock, Arkansas 722i n Phone: (501) 372-3425 Faz (501) 372-3428 Email
mcheciydgawbolLaet Mr. Christopher Heller FRIDAY, ELDREDGE & CLARK 400 W. Capitol, Suite 2200 Little Rock, Arkansas 72201 Re
LRSD V. PCSSD Dear Chris
This letter sets forth additional Compliance Plan. We comments of the Joshua Intervenors concerning the LRSD , ----------------* LLiC jUIkOU the dataJ In tol^TSD attention should be given to the quality of e oata. m_the past, LRSD has used results on the RA and the Observation in consistent Txnth rr ^nd the Observation SuTvcy in wcvs not nsistent with the purposes of those instruments. In addition, because teanherc 4., In addition, because teachers provided scores for to own students, the past use made of the data was in conflict with the diS newly enacted Regulation IL-Rl that Conflict of Interest .55 must be avoided. :s recognition in the for 2. We are concerned about the . manner in which the regulation describes the team nropAg-
prepari^ evaluauons, again in the context of conflict of interest L - - isavoided,e external consultant needs to write the report and control the conflict of interest In order to insure that context of the analysis. Paragraphs 3, 5 and 6 of the guarantee that the external ti' which w= describe,-JI Program Evaluation Procedures do not iizitk , cNfv .' --------LRSD staff preparing comments to the Board with a differing interpretation of the evaluation results. Continue to be concerned about the global, general manner in which the content of P desenbed (page 7 of the document, first paragraph). For example the Board is bXwX'^sSStoe with remediation for students whose petoanc, is oe ow par Studying the actual implementation of these stotuds im an or a re more T' because class membersare so much AcibeiSn^esS Srf i 1 P^rfoxmauce on the Benchmark and Stanford /vcnievement lests. A. satisfactory desenpuon by e School Board :e standards (in all or a representative of the evaluations which it : exhibit I 10/24/2002 THU 09:03 [TX/Ri: NO 85S0] @002' . CX^. 24.2002 8:07flM JOHN W WALKER P R N0.9S3 P.3 Page Two October 23, 2002 ^dert^c should make clear that the actual implementation of remediation acmvines m ehstnet schools is to receive careful consideration. This is surely an important contextual factor (see Accuracy Standards, para. 2), i^onicxiuai . Plan that the LRSD plans evaluations ofprograms deemed pamculmly direcmd to achievement of African American students for the Uidefiuiu: ' necessary to satisfy the court. We would Uke to receive the Boards assurance mat inns is me case. to be indefinite future, not We would appreciate your providing this letter to the Superintendent and the members of the school board. Walker Sincerely, JWW:Ip cc: AU Counsel Ms. Ann Marshall Judge Thomas Ray 10/24/2002 THU 09:03 [TX/RI NO 8580] @003 MEMORANDUM To: From: Subject: Date: Dennis Glasgow Gene Jones Compliance Remedy August 9, 2004 Thank you for providing time for Margie and me Wednesday and for making our conversation both pleasant and informative. We are pleased to be invited to participate in the planning for and implementation of the Courts June 30 Compliance Remedy. We will be available as much as necessary to complete the work. The proposed responses to the Compliance Remedy that you shared with us should be helpful to the people to be assigned those tasks. As you refine and update the document, it may be worth considering that the language of paragraph A of the Compliance Remedy includes the term statisticians in the list of people to be hired. The language of the list of staff members of the Research, Assessment and Evaluation Department in the proposed response document does not. 1 also note that the term is plural in the Compliance Remedy. Significant tasks ordered by the Compliance Remedy in paragraphs F and G include written notification to Joshua and ODM of the eight programs to be targeted for step 2 evaluations and the quarterly updates. The proposed response document does not include these and may not need to, but they should be prominent on the list of things to do when the staff of the Research, Assessment, and Evaluation Department is assembled. The first of the quarterly updates is due December 1. Thank you for arranging our meeting with the superintendent this Thursday.PROJECTED SCHOOL IMPROVEMENT STATUS OF LRSD SCHOOLS BASED ON 2004 ACTAAP ASSESSMENT RESULTS SUMMARY: Fifteen (15) elementary schools fully met standards for all subgroups in both mathematics and literacy. These schools are not on school improvement status. Of the remaining nineteen (19) elementary schools that are on school improvement status, seven (7) met Adequate Yearly Progress (AYP) in both math and literacy. Bale, Baseline, Chicot, Cloverdale (Elementary), Dodd, Fair Park, and Rockefeller met AYP for 1 year. These seven schools can come off school improvement if they make AYP next year. One elementary school moves up to Year 3 status: Mitchell. Meadowcliff, Rightsell, and Western Hills are projected to be on alert since they failed to meet AYP for one or more subgroups in mathematics. These schools will be on Year 1 of School Improvement should they fail to meet AYP next spring. At the middle level Mann and Pulaski Heights Middle Schools met the adequate yearly progress standard. They will remain on Year 1 of School Improvement. The other six middle schools will move up a year on school improvement, five to Year 2 and one (Southwest) to Year 3. All five high schools were on Alert status last year. Four of them will move to Year 1 status due to not meeting AYP in literacy and math. Parkview met AYP in literacy and math and will move off school improvement status. SCHOOLS THAT MET OR EXCEEDED THE STANDARD IN MATH AND LITERACY FOR 2004: Bale Booker Chicot Dodd Forest Park Geyer Springs Jefferson McDermott Pulaski Heights Romine Wakefield Mann Middle Parkview High School Baseline Carver Cloverdale Elementary Fair Park Fulbright Gibbs Mabelvale Elementary Otter Creek Rockefeller Terry Williams Pulaski Heights Middle received AIIR - A 200H OFFICE OF desegregation monitoringSCHOOLS THAT WERE ON SCHOOL IMPROVEMENT STATUS FOR THE 2003-04 SCHOOL YEAR: Current Status (Based on 2002-03 Scores) Level/School ELEMENTARY Chicot Year 3 Bale Baseline Fair Park Mitchell Wakefield Brady Cloverdale Dodd Franklin ML King Mabelvale Rockefeller Stephens Washington Watson Year 2 Year 2 Year 2 Year 2 Year 2 Year 1 Year 1 Year 1 Year 1 Year 1 Year 1 Year 1 Year 1 Year 1 Year 1 Projected Status (Based on 2003-04 Scores) Year 3 Year 2 Year 2 Year 2 Year 3 Off School Improvement Year 2 Year 1 Year 1 Year 2 Year 2 Off School Improvement Year 1 Year 2 Year 2 Year 2 Subgroup/Subject not meeting AYP All students: Math and Literacy All students
Math All students: Math African-American students
Math All students
Math and Literacy African-American students: Math and Literacy All students: Math and Literacy Comments Met AYP for 1 year Met AYP for 1 year Met AYP for 1 year Met AYP for 1 year Met AYP for 2 consecutive years Met AYP for 1 year Met AYP for 1 year Met AYP for 2 consecutive years Met AYP for 1 yearWilson Woodruff Year 1 Year 1 Year 2 Year 2 All students: Math All students: Math Meadowcliff Alert Rightsell Western Hills Alert Alert African-American students: Math All students: Math All students: Math MIDDLE SCHOOL Cloverdale Dunbar Forest Heights Henderson Mabelvale Mann Pulaski Heights Southwest HIGH SCHOOL Central Hall Fair McClellan Parkview Year 1 Year 1 Year 1 Year 1 Year 1 Year 1 Year 1 Year 2 Alert Alert Alert Alert Alert Year 2 Year 2 Year 2 Year 2 Year 2 Year 1 Year 1 Year 3 Year 1 Year 1 Year 1 Year 1 Not on School Improvement All students: Math and Literacy African American students: Math and Literacy All students: Math All students: Math and Literacy All students: Math and Literacy All students: Math and Literacy African-American students: Literacy All students: Literacy All students: Literacy All students: Literacy Met Safe Harbor provision Met Safe Harbor provision 34% improvement in black students below basic in math compared to last year 88% of white students proficient in literacy Met AYP in literacy and mathEASTI IN THE UNITED STATES DISTRICT COURT EASTERN DISTRICT OF ARKANSAS LITTLE ROCK DIVISION FILED U.8. DISTRICT ^URT 'ESN district ARKANSAS JUN 30 2004 JAMES W. McCORMACK, CLERK By
. DEP CLERK LITTLE ROCK SCHOOL DISTRICT PLAINTIFF V. No. 4:82CV00866 WRW/JTR PULASKI COUNTY SPECIAL SCHOOL DISTRICT NO. 1,ETAL. DEFENDANTS MRS. LORENE JOSHUA, ET AL. INTERVENORS KATHERINE KNIGHT, ET AL. INTERVENORS MEMORANDUM OPINION' I. Warning to the General Reader The general reader, if any there be, should realize that educators, like lawyers, have developed their own language. To the extent that time, patience, and skill would permit, I have tried to Gamerize'^ this Memorandum Opinion. I have fallen short, but hope the effort will be of some help. IL Background On September 13,2002,1 entered a Memorandum Opinion (the September 13 Decision) holding that the Little Rock School District (LRSD) had substantially complied with all of its I would be seriously remiss if I did not once again note the tremendous amount of work United States Magistrate Judge Joe 'Hiomas Ray has done on this case. -Bryan Gamer, of Dallas, Texas, has published several excellent books and articles for the legal profession on the use of plain, understandable English. .0 72A Rev.a/82)desegregation obligations set forth iii the January 16,1998 Revised Desegregation and Education Plan (the Revised Plan)/ except those obligations contained in 2.7.1, LRSD V. Pulaski County Special Sch. Dist, et al., 231F. Supp. 2d 988 (E.D. Ark. 2002)
afPd, 359 F.3d 957 (8* Cir. 2004). Section 2.7 of the Revised Plan obligated LRSD to implement programs, policies and/or procedures designed to improve and remediate African-American achievement. 2.7.1 ensured that the promise made in 2.7 would have teeth by requiring that: Section LRSD shall assess the academic programs implemented pursuant to 2.7 after each year in order to determine the effectiveness of the academic programs in improving Afiican-American achievement. If this assessment reveals that a program has not and likely will not improve Afidcan-American achievement, LRSD shall take appropriate action in the form of either modifying how the program is implemented or replacing the program. ex 871. As stated above, m the September 13 Decision, I found that LRSD had substantially complied with its obhgations under 2.7 of the Revised Plan
however, I detennined there were numerous, substantial deficiencies in LRSDs efforts to comply with its obligations under 2.7.1. See LRSD, 237 F. Supp. 2d at 1076-1082. The September 13 Decision gave LRSD until March 15, 2004, to demonstrate that it had substantially complied with 2.7.1 of the Revised Plan, as specified in subparts A, B, and C of the Compliance Remedy. LRSD, 237 F. Supp. 2d at 1087-88. LRSD has been involved continuously in desegregation litigation since 1956. See LRSD, 237 F. Supp. 2d at footnote 18. As far as I can tell from the reported cases, LRSD now has the During the 2001-02 unitary status hearings, the Revised Plan was introduced into evidence as CX871. -2- AO72A (Rev.8/82)dubious distinction of having been under federal court supervision longer than any other school district in history. Thus, LRSD is well seasoned when it monitoring. comes to court supervision and On November 12, 2002, Joshua Intervenors" (Joshua) appealed (docket no. 3704) the Septembers Decision. On March 2,2004, the Eighth Circuit Court of Appeals affirmed. LRSD V. Joshua, 359 F.3d 957 (8- Cir. 2004). Thus, all aspects of the September 13 Decision are now final and the law of the case. On March 12, 2004, LRSD filed its Compliance Report (docket no. 3837) seeking complete unitary status on the ground that it had substantially complied with the obligations imposed under the Compliance Remedy and 2.7.1 of the Revised Plan. On April 15, 2004, Joshua filed an Opposition to LRSDs Request for Release from Court Supervision of Its I Desegregation Efforts (docket no. 3856), along with a supporting Memorandum (docket no. 3857). 1 must now decide whether LRSD has met its obligations under the Compliance Remedy, and whether it should be released from almost five decades of court supervision. in. The September 13, 2002 Compliance Remedy Almost 70% of LRSDs students are African-American. Historically, the academic achievement of many of these students, as gauged by standardized test scores, is low and poses a significant long-term challenge to LRSD teachers and administrators. Of course, because this so-called achievement gap IS a nationwide phenomenon, it is a problem that educators must are '^e Joshua Intervenors are a group of Afiican-American school children.. enrolled in each of the three Pulaski County school districts. Thus, Joshua some of whom serves as the class representative for all African-American students enrolled in LRSD, the Pulaski Countv Snecial School District, and the North Little Rock School District. -3- A0 72A (Rev.a/82)I ( ' AO72A (Rev.8/82) confront in schools throughout the-country. See LRSD, 237 F. Supp. 2d at 1073-74. Importantly, 2.7 of the Revised Plan promised only that LRSD would implement programs, policies and/or procedures designed to improve and remediate Afidcan-American achievement, See docket no. 3410 at 51. However, 2.7.1 went on to require LRSD to assess the 2.7 programs annually in order to determine their effectiveness, and to modify or replace any programs that were shown not to be working to improve African-American achievement. Id. at 148. Read together, the obligations set forth in 2.7 and 2.7.1 of the Revised Plan required LRSD not only to design academic programs that were int achievement of African-American students, but also to programs to ensure that they were, in fact, effective in improving^ Expressed in the vernacular of my native Scott County, Arkan^
^ 1 c ise It. id 2.7.1 contained the bacon. These two sections of the Revised Plan are crucially important to the future educational success of a large number of LRSDs current and future students. During the November 2001 hearings on unitary status, Dr. Bonnie Lesley, LRSDs Associate Superintendent of Instruction and Curriculum, defined a program assessment as something that is dynamic, it is interactive, its ongoing, it happens frequently, and it is a measurement, along with the analysis that you would make of whatever results are available. LRSD, 237 F. Supp. 2d at 1077. In contrast, she defined a program evaluation as more long term, it may consider observations or measurements in addition to test scores, and is guided by a series of research questions that are usually provided by whoever the consumer is of that report. Id. In other words, a program assessment is a relatively informal process that may not result in much documentation, while a program evaluation is a formal process that always A-AO 72A (Rev.B/82) I i I I I mvolves the preparation of an often lengthy written program evaluation which is centered around carefully prepared research questions that the evaluation is designed to answer. Section 2.7.1 of the Revised Plan provided that LRSD must make assessmentsnot evaluations-of the 2.7 programs in order to determine their effectiveness in improving the academic achievement of African-American students. However, as early as March 15, 2000, LRSD acknowledged, m its own Interim Compliance Report (docket no. 3356), that 2.7.1 of the Revised Plan obligated it to prepare evaluations on the key 2.7 programs so that LRSD admimstrators could make an informed decision on the effectiveness of those programs. See LRSDs Interim Compliance Report at 51-55. Furthermore, during her testimony in November of 2001, Dr. Lesley admitted that, even though 2.7.1 of the Revised Plan did not mention anything about LRSDs obligation to prepare program evaluations to determine the effectiveness of the 2.7 programs, she and other administrators interpreted that section of the Revised Plan as requinng LRSD to perform evaluations covering the most important 2.7 programs. LRSD, 237 F. Supp. 2d at 1077. Because it is so important to an understanding of the Compliance Remedy, I want to be very clear on this pomt: The evidence overwhelmingly establishes that LRSD has always construed the obligations contained in 2.7.1 of the Revised Plan as requiring It to prepare formal program evaluations on the key 2.7 programs. See Interim Compliance Report at 51-55
Final Compliance Report dated March 15, 2001 (docket no. 3410) at 148
Dr. Lesleys testimony cited in the September 13 Decision, LRSD, 237 F. Supp. 2d at 1077. Since my decision will turn on whether LRSD has properly implemented the September 13, 2002 Compliance Remedy, it is set forth below in full
-5-Vn. Compliance Remedy Because LRSD failed to substantially comply with the crucially important obligations contained in 2.7.1, it must remain under court supervision with regard to that section of the Revised Plan until it: (a) demonstrates that a program assessment procedure is in place that can accurately measure the effectiveness of each program implemented under 2.7 in improving the academic achievement of Afiican-Amencan students
and (b) prepares the program evaluations identified on page 148 of the Final Compliance Report and uses those evaluations as part of the program assessment procedure contemplated by 2.7.1 of the Revised Plan. The details of this compliance remedy are set forth below
A. For the entire 2002-03 school year and the first semester of the 2003-04 school year, through December 31, 2003, LRSD must continue to assess each of the programs implemented under 2.7 to unprove the academic achievement of Afiican-American students. LRSD now has over three years of testing data and other information available to use in gauging the effectiveness of those AO72A (Rev.8/82) i i I i I I I B. C. programs. I expect LRSD to use all of that available data and information in assessing the effectiveness of those programs and in deciding whether any of those programs should be modified eliminated. or LRSD must maintain written records regarding its assessment of each of those programs. These written records must reflect the following information: (a) the written criteria used to assess each program during the 2002-03 school year and the first semester of the 2003-04 school year
(b) the results of the annual assessments of each program, including whether the assessments resulted in program modifications or the elimination of any programs
and (c) the names of the administrators who were involved with the assessment of each program, as well as at least a grade level description of any teachers who were involved in the assessment process {e.g., all fourth grade math teachers
all eighth grade English teachers, etc.). LRSD must use Dr. Nunnery or another expert from outside LRSD with equivalent qualifications and expertise to prepare program evaluations on each of the programs identified on page 148 of the Final Compliance Report.^ I will accept all program evaluations On page 148 of its March 15, 2001 Final Compliance Report (docket no. 3410), LRSD flatly stated that it had prepared program evaluations on fourteen separate programs listed on that -6-that have already been completed by Dr. Nunnery or someone with similar qualifications and approved by the Board. All program evaluations that have not yet been completed on the remaining programs identified on page 148 of the Final Compliance Report must be prepared and approved by the Board as soon as. In practicable, but, in no event, later than March 15, 2003. D addition, as these program evaluations are prepared, LRSD shall use them, as part of the program assessment process, to determ in p the effectiveness of those programs in improving Afiican- American achievement and whether, based on the evaluations, any changes or modifications should be made in those programs. In addition, LRSD must use those program evaluations, to the extent they may be relevant, in assessing the effectiveness of other related programs. 72A Rev.8/82) D. Joshua must monitor LRSDs compliance with 2.7.1 and must immediately bring to the attention of LRSD all problems that detected in its compliance with its obligations under 2.7.1, as those obligations are spelled out in this Compliance Remedy. Thereafter, Joshua and LRSD must use the Process for Raising Compliance Issues set forth in 8.2, et seq., of the Revised Plan are E. to attempt to resolve those compliance issues. If those efforts are unsuccessful, Joshua must present the issues to me for resolution, as required by 8.2.5. Any such presentation must be timely. The ODM must also monitor LRSDs compliance with 2.7.1 and help to ensure that LRSD fulfills its obligations, as specified in this Compliance Remedy. F. On or before March 15, 2004, LRSD must file a Compliance Report which documents its compliance with its obligations under 2.7.1. Any party, including Joshua, who wishes to challenge LRSD s substantial compliance with 2.7.1, as specified above, may file objections with the court on or before April 15, 2004 Thereafter, I will decide whether LRSD has substantially complied with 2.7.1, as specified in this Compliance Remedy, and should be released from all further supervision and monitoring. page. During the 2001-02 unitary status hearings, the evidence overwhelmingly established that, as of March 15, 2001, LRSD had not prepared any of those fourteen program evaluations. See LRSD,231'?. Supp.2dat 1079-80. Henceforth, I will refer to these fourteen program evaluations as the Page 148 Evaluations. -7- I I I ILRSD,131 F. Supp. 2d at 1087-88 f-emphasis added). Even a casual reading of this Compliance Remedy reveals that it imposed four essential obligations on LRSD, Joshua, and the Office of Desegregation Monitoring (ODM): 1. 2. Preparation of Annual Assessments of 2.7 Program .s LRSD was required to annually assess each of the programs implemented under 2.7 during the 2002-2003 school year and the first semester of the 2003-2004 school year and then use those assessments, the more than three years of testing data, and all other relevant available information to determine the effectiveness of those programs and to decide whether any of the programs should be modified or eliminated. LRSD also was required to maintain written records reflecting: (a) the criteria used to assess each program
(b) the results of the annual assessments of each program, including whether any programs were modified or eliminated
and (c) the administrators and teachers who were involved in preparing the assessment of each program. I I Subparts A and B of the Compliance Remedy obligated LRSD to assess each of the 2.7 programs implemented during the 2002-03 school year and the first semester of the 2003-04 school year and to maintain written records of its armual assessments of each of those programs. I made no mention of LRSD preparing evaluations of 2.7 programs because, on its face, nothing in 2,7.1 of the Revised Plan obligated LRSD to perform program evaluations. However, Dr. Lesley made it clear in her testimony that LRSD administrators knew and understood that the assessment obligation in 2.7.1 included the obligation of preparing program evaluations. See discussion, supra, at 4-5. Therefore, I concluded it would be best to use the same terms in the Compliance Remedy that the parties themselves had chosen to use in 2.7.1 of the Revised Plan. Because I was tracking the parties own language, I never dreamed the use of the tenns assess and assessment would suddenly create confusion for LRSD administrators in deciding how to comply with subparts A and B of the Compliance Remedy. Preparation of Page 148 Program Evaluations. I LRSD was required to hire experts to prepare the fourteen program evaluations identified on page 148 of its March 15, 2001 Compliance Report. The Court agreed to accept all program evaluations that had already been prepared by outside consultants and ordered LRSD to hire outside consultants to complete the unfinished program evaluations, which were to be approved by the LRSD Board and filed with the Court no later than March 15, 2003. Finally, as those evaluations were prepared, LRSD was required to use them - to the extent they I -8- AO72A (Rev.e/82)might be useful - in its annual assessment of the effectiveness of the 2.7 programs. 3. Monitoring. Joshuas counsel was required to continue with its monitoring of LRSDs implementation of the Compliance Remedy and to use the procedures set forth in 8.2 of the Revised Plan to resolve any compliance problems that might arise. If Joshua and LRSD were unsuccessful in using the ODM to facilitate and resolve those problems, they were required to bring those compliance issues directly to me for resolution. Finally, the ODM was directed to monitor LRSDs implementation of the Compliance Remedy and to help ensure that LRSD fulfills its obligations specified therein. The sole purpose of subparts D and E the Compliance Remedy was to ensure that, if the ODM was unable to successfully facilitate the resolution of any compliance issues raised by Joshua, those compliance issues would be brought to my attention so that I could resolve them on a timely basis, thereby avoiding any surprises when LRSD filed its Compliance Report. 4. Preparation of Compliance Report. V LRSD was ordered to file a Compliance Report by March 15,2004, documenting its substantial compliance with 2.7.1 of the Revised Plan and the Compliance Remedy. TV. The Parties Compliance Activities after September 13, 2002 A. The Court Clarifies Joshuas Monitoring Obligation. After the entry of the September 13 Decision, LRSD did not seek clarification of any terms used in the Compliance Remedy or any of its compliance obligations. Based on its silence, I concluded that LRSD understood what it was required to do under the Compliance Remedy, and that it was proceeding apace to meet those obligations. In contrast, on October 1, 2002, Joshuas counsel wrote a letter (docket no. 3680) objecting to the monitoring obligations imposed on them under subpart D of the Compliance Remedy. Among other things, Joshuas counsel challenged the Courts decision to: (1) impose monitoring obligations on them that were contemplated to be the responsibility of the ODM
-9- AO 72A (Rev.8/82)i and (2) place a greater burden upon Joshua than it has imposed upon the ODM. Id. By way of relief, Joshuas counsel sought clarification of their monitoring obligations under subpart D ( . of the Compliance Remedy and a hearing on this matter so that an appropriate record on the issues of the role of the ODM monitoring and Joshuas monitoring may be fully developed. Id. On October 11, 2002, I entered an Order (docket no. 3685) clarifying the monitoring obligations imposed on Joshuas counsel under the Compliance Remedy. After noting that Joshua s counsel had been engaged in monitoring LRSDs compliance with its desegregation obligations since at least 1990,1 made it clear that subpart D of the Compliance Remedy only obligated Joshua s counsel to continue to perform their monitoring role according to the same procedure they and LRSD have followed for many years in this case. LRSD, 237 F. Supp. 2d at 1091. Because the October 1,2002 letter could be read to suggest that Joshuas counsel would only continue their monitoring role, if ordered to do so by the Court, I also made it clear that: I do not believe I can force Joshuas counsel to perform monitoring duties something that I may have mistakenly assumed they wanted to do. I will leave it up to Joshua s counsel to decide if they have an ethical and professional duty to continue monitoring LRSDs compliance with its sole remaining obligation under the Revised Plan. I hope Joshuas counsel resolves that question in favor of continuing their long-standing commitment to monitoring LRSDs compliance with its desegregation obligations. However, since they complain about my expressly directingthem to continue monitoring LRSDs compliance with 2.7.1 of the Revised Plan - something I never expected to hear -1 believe I must now clarify Section VII.D. of the Memorandum Opinion to read as follows
Joshua may monitor LRSDs compliance with 2.7.1 and, if they choose to do so, they should bring to the attention of LRSD, on a timely basis, all problems that are detected in its compliance with its obligations under 2.7.1, as those obligations are spelled out in this Compliance Remedy. Thereafter, Joshua and LRSD must use the process for raising compliance issues set forth in ^-2. et seq., of the Revised Plan to attempt to resolve those compliance issues. If those efforts are unsuccessful, Joshua shall -10- XO72A Rev.8/82) present the issues to me for resolution, as required by 8.2.5. Any such presentation must be timfily Id. at 1091. Finally, I emphasized that, regardless of whether Joshuas counsel decided to continue to monitor LRSDs compliance with its obligations under 2.7.1 of the Revised Plan, the ODM staff most certamly will continue their close monitoring ofLRSDs compliance with that section of the Revised Plan. Id. at 1091. My final admonition follows
on the subject of monitoring was as K Joshua s counsel decide to continue their monitoring role, which is independent from the momtonng work performed by the ODM,... I expect counsel for Joshua and LRSD to cooperate and work together to ensure that things go smoothly with regard to monitoring LRSDs implementation of its obhgations under 2.7.1. However, if actual disputes arise regarding monitoring, I wiU be available to resolve them. I at 1091. B. LRSD Adopts Compliance Plan, Approves Regulation IL-R1, and Designates Areas for 2.7.1 Program Evaluations. On October 10,2002, LRSDs Board of Directors (the Board) adopted a Compliance Plan that was specifically designed to satisfy the Courts Compliance Remedy. See Exhibit A to LRSDs May 14, 2003 Notice of Filing Program Evaluations (docket no. 3745)? The 72A Rev.8/82) During the June 14 and 15, 2004 compliance hearing (hereafter referred compliance neanng (hereafter referred to as the ), the Proposed Compliance Plan was introduced into evidence as LRSDs In most respects, the Proposed Compliance Plan is identical to the final Compliance Plan approved by the Board on October 10, 2002. However, the Proposed Compliance Plan raises a number of questions about the meaning of subparts A and B of the Compliance Remedy. All of those questions were deleted from the final Compliance Plan approved by the Board on October 10,2002. As will be discussed later, LRSD did not bnng any any of those questions to my attention, or otherwise seek clarification of the requirements of the Exhibit No. 2. Compliance Remedy. -11-Compliance Plan recognized that, in order for LRSD to meet its obligations under the Compliance Remedy, it would have to satisfy three core obligations
(1) develop a written procedure for evaluating the programs implemented pursuant to 2.7 to determine their effectiveness in improving the academic achievement of African-American students
(2) maintain written records of the criteria used to evaluate each [ 2.7] program
and (3) repare a comprehensive program evaluation of each academic program implemented pursuant to ... 2.7 to determine its effectiveness in improving the academic achievement of African-American students and to decide whether to modify or replace the program.'" Id. at 3-5 (emphasis added). Significantly, there is nothing in the Compliance Plan adopted by the Board that suggests LRSD was confused about the meaning of any of the terms in the Compliance Remedy or any of its compliance obligations. Additionally, the Compliance Plan makes it clear that LRSD construed subpart A of the Compliance Remedy as requiring it to prepare a comprehensive program evaluation of each academic program implemented pursuant to .. . 2.7 . The Compliance Plan also included a detailed Action Plan Time Line that: (1) identified the LRSD employees who were responsible for implementing each activity necessaiy to satisfy the Compliance Remedy
and (2) provided a schedule for completing each of those activities. Dr. Bonnie Lesley, the Associate Superintendent for Curriculum and Instmction, and Dr. Ken James, LRSD s Superintendent of Schools, were assigned personal responsibility for each of the twenty-eight (28) activities identified by the Action Plan Time Line. Id. at 7-10. Thus, It was up to Dr. Lesley and Dr. James to spearhead the timely implementation of all twenty- eight activities necessary to satisfy the Compliance Remedy. Finally, at the same time it approved the Compliance Plan, the Board also adopted -12- \O72A Rev.e/82)Regulation IL-Rl, which set forth the written procedures for evaluating the programs implemented pursuant to 2.7 to determine their effectiveness in improving the academic achievement of African-American students. See Exhibit A at 2 (docket no. 3745).' According to the Compliance Plan, Regulation IL-Rl established the criteria for preparing the program evaluations necessary to satisfy LRSDs obligations under subparts A and B of the Compliance Remedy. On October 24,2002, the Board approved a Program Evaluation Agenda for the 2002- 03 school year that authorized the preparation of evaluations in three broad areas: (1) Elementary literacy
(2) Secondary literacy
and (3) the National Science Foundation (NSF) K-12 Math and Science Project. See LRSDs Exhibit No. 3
ODMs March 30, 2004 Report on LRSDs Implementation of the Courts Compliance Remedy at 4 (hereinafter referred to as the ODMs Compliance Report) (docket no. 3854). LRSD subsequently construed the 2002-03 Program Evaluation Agenda as requiring it to prepare only two 2.7.1 evaluations in order to satisfy its ! I I I i I i I obligations under subpart A of the Compliance Remedy: (1) I I a comprehensive Literacy Evaluation
and (2) a comprehensive Math and Science Evaluation. Of course, Literacy and Math and Science are not programs they are broad I i ! I i I academic areas that roughly correspond to the grouping of college courses into Arts or Sciences. Because LRSD administrators, such as Dr. Lesley, had always constmed 2.7.1 of the Revised Plan as requiring LRSD to prepare evaluations of the key 2.7 programs implemented to improve African-American achievement, the Board should have been aware that I I I During the compliance hearing. Regulation IL-Rl was introduced into evidence Joshuas Exhibit No. 2. as I -13- AO72A (Rev.a/ea)they were beingtoo general in dividing the academic universe into Literacy and Math/Science and then preparing a global evaluation of each of those areas. Nevertheless, in its March 12,2004 Compliance Report (docket no. 3837), LRSD contends that these two 2.7.1 evaluations fully satisfy all of its obligations under subpart A of the Compliance Remedy/ C. Joshua Invokes Facilitation Provision of Revised Plan. On November 4, 2002, Ms. Ann Marshall, the Director of the ODM, wrote Joshuas counsel a letter confirming that she: (a) had received their November 1,2002 letter requesting that she facilitate the dispute that had arisen between Joshua and LRSD regarding the adequacy of LRSD s Compliance Plan
and (b) was willing to facilitate that dispute.' I Thereafter, I was never contacted by Joshuas counsel or anyone else about the outcome of the ODMs facilitation efforts. Because subpart D of the Compliance Remedy obligated Joshuas counsel to contact me only if the facilitation was unsuccessful, I concluded from the parties silence (and the ODMs silence) that the ODM had successfully resolved this dispute. On November 6, 2002,1 wrote Ms. Marshall to reinforce the crucially important role I expected herto playmmomtoring LRSDs compliance withits obligations under the Compliance As I explain later in some detail, I was notmade aware ofLRSDsPropoW Compliance Plan and Regulation IL-Rl until two months after LRSD filed its March 12, 2004 Comphance Report. LRSD did not file the October 10,2002 Compliance Plan with the Court until March 14 2003. I Section 8.2 of the Revised Plan required the parties to use the ODM to facilitate the resolution of any compliance issues. Since the words facilitation and facilitate come directly TT*r\-m fkio "D c&irt Din., T _____j.i____... from the Revised Plan, I will use them. A copy of Ms. Marshalls November 4,2002 letter, marked Exhibit A, is attached to this Memorandum Opinion. -14- ^O72A Rev.8/82)I Remedy?* The instructions to her were: It seems to me it would be best if you worked with the parties toward implementing the remedies
but you should feel free to contact me in writing if a serious impasse develops. In other words, as long as everything is going along smoothly, I see no reason for you to make regular reports to me in this respect. - J. -------------- A wuywi. I emphasize, however, that you should feel free to call on me if serious problems anse. Neither the parties nor the ODM ever contacted me to request that I become involved in resolving any compliance disputes or other serious impasses between LRSD and Joshua, despite the requirement for such contact by subpart D of the Compliance Remedy and my November 6,2002 letter. D. The Six Completed or Substantially Completed Page 148 Evaluations. The September 13 Decision stated that the Court will accept all program evaluations that have already been completed by Dr. Nunnery or someone with similar qualifications and approved by the Board. LRSD, 273 F. Supp. 2d at 1088. By the time I entered that decision, LRSD had completed or substantially completed six of the fourteen Page 148 Evaluations. These SIX evaluations covered the following programs
(1) Pre-K-3 Literacy
(2) NSF Math and Science Project
(3) Charter Schools
(4) English as a Second Language
(5) SEDL*^ Program at Southwest Middle School
and (6) Collaborative Action Team Project. By December 31,2002, LRSDs Board had approved all six of these Page 148 Evaluations. II A copy of my November 6, 2002 letter, marked Exhibit B, is attached to this Memorandum Opinion. ^SEDL is an acronym for Southwest Educational Development Laboratory. ODMs Compliance Report at 8 (docket no. 3854). See -15- AO72A (Rev.8/82)E. The Eight Remaining Page 148 Evaluations. LRSD contracted with Dr. Steven Ross, a program evaluation expert and a member of the faculty at the University of Memphis, to prepare guidelines for completing or revising the eight remaining evaluations.'^ These eight evaluations covered the following programs
(1) Middle Schools
(2) Extended Year Schools
(3) HIPPY
'"* (4) Campus Leadership Teams
(5) Summer School - Elementary
(6) Lyceum Scholars Program
(7) Onward to Excellence - Watson Elementary
and (8) Vital Link. In late November of 2002, Dr. Ross prepared a document captioned Guidelines for Completing Eight Program Evaluations in the LRSD.'^ On January 10,2003, LRSD contracted with Dr. Ross and two other program evaluation experts. Dr. William Moore and Dr. Larry McNeal, to prepare these eight program evaluations. Dr. Ross prepared or completed evaluations I on Vital Link, Onward to Excellence, HIPPY, and Campus Leadership Teams
Dr. Moore prepared or completed evaluations on Middle School Transition and Extended Year Education- and Dr. McNeal completed evaluations on Lyceum Scholars and Elementary Summer School. j On March 14,2003, LRSD filed all fourteen ofthe Page 148 Evaluations with the Court, as required by subpart C of the Compliance Remedy. See LRSDs Notice of Filing Program Evaluations (docket no. 3745). The six evaluations, which were substantially completed as of 1 t I 'A number of years ago, Joshua formally agreed that Dr. Ross has the qnalificatinns necessary to prepare program evaluations. 'hippy is an acronym for Home Instruction for Parents of Preschool Youngsters. See ODMs Compliance Report (docket no. 3854) at 8. '^During the compliance hearing, this document was introduced as LRSDs Exhibit No. 5. -16- ^O72A flev.e/82)September 13, 2002, are bound together in Volumes I and II. The remaining eight Page 148 Evaluations are bound together in Volumes HI and IV. On April 14,2003, Joshua filed Comments on the Submission of Page 148 Evaluations. (Docket no. 3752.) These comments identify and discuss numerous alleged deficiencies, most of which are contained in the eight Page 148 Evaluations that the Board after January 1, 2003. were completed and approved by F. Dr. Lesley and Dr. James Resign. On March 14, 2003, the same day LRSD filed the fourteen Page 148 Evaluations, Dr. Lesley, who was responsible for overseeing the preparation of those evaluations, resigned. and, two months later. Dr. Janies, LRSDs Superintendent, also resigned. As indicated ( ) previously, LRSDs Compliance Plan had assigned Dr. Lesley and Dr. James direct responsibility for each of the twenty-eight time line activities necessary for LRSD to implement the Compliance '^During the compliance hearing, these fourteen program evaluations were introduced as LRSDs Exhibit No. 13. During its implementation of the Revised Plan in 1998 through early 2001, LRSD originally intended to use its own Department of Planning, Research, and Evaluation (PRE) I to prepare the fourteen Page 148 Evaluations. Dr. Kathy Lease was the Assistant Superintendent who headed PRE and reported to Dr. Lesley. See ODMs Compliance Report at 2. According to Dr. Lesley s testimony during the November 2001 hearings on unitary status. Dr. Lease dropped the ball in preparing these evaluations, which resulted in only a few partially completed and woefully inadequate evaluations being available on March 15,2001, the deadline for LRSD to file its Compliance Report seeking unitary status. Additionally, Dr. Lesley testified that, in her opimon, no one in PRE-including Dr. Lease-had the expertise to prepare program evaluations. See LRSD, 237 F. Supp. 2d at 1077-81. I In early 2001, Dr. Lease resigned and left LRSD for other job opportunities. Since that time, PRE has functioned with only a statistician and several support employees, but no one was hired to replace Dr. Lease or take over and operate the department. In November of2003, LRSD appointed the statistician as the acting head of PRE. As a result, since Dr. Leases departure in early 2001, LRSD has essentially functioned without a meaningful Department of Planning, Research, and Evaluation. See ODMs Compliance Report at 6. -17- KO72A Rev.B/82)Remedy. i Thus, less than a year after the September 13 Decision, LRSD lost both of the crucially important leaders to whom all responsibility had been assigned for implementing LRSDs Compliance Plan. Both Dr. James and Dr. Lesley had been deeply involved for years in LRSDs implementation of its desegregation obligations under the Revised Plan, and both were thoroughly familiar with the intricacies of satisfying judicially imposed desegregation obligations. The almost simultaneous departures of Dr. James and Dr. Lesley during the early stages ofLRSDs implementation of its Compliance Plan clearly created problems for LRSD in its compliance efforts. In the ODMs Compliance Report, the authors observe that the loss of Dr. Lesley and Dr. James at a crucial time in the implementation ofLRSDs Compliance Plan, and the delays and difficulties LRSD encountered in filling those positions with acting or interim employees created a period of some uncertainty for LRSD. Id. at 5. In June of2003, LRSD appointed an Interim Superintendent to replace Dr. James. Later that month, on June 26, 2003, Mr. Dennis Glasgow, who previously had been the Director of i I LRSDs Math and Science Department, was appointed to succeed Dr. Lesley as Interim Associate Superintendent of Instruction and Curriculum. According to the Compliance Plans Action Plan Time Line, Mr. Glasgow originally was assigned responsibility for only two of the twenty-eight time line activities. Although it was not mentioned in LRSDs Compliance Report (docket no. 3837), during the recent compliance hearing, Mr. Glasgow testified that, on March 18,2003, he assumed responsibility for all of Dr. Lesleys twenty-eight activities under the Compliance Plan. In August of 2003, after the Interim Superintendent, hired only three months earlier, left that position, LRSD hired Dr. Morris Holmes as the second Interim Superintendent. Dr. Holmes -18- AO72A (Rev.a/82)has continued to serve as Interim Superintendent through the date of this Memorandum Opinion 18 LRSDs Compliance Report does not address what role, if any. Dr. Holmes had in implementing the Compliance Plan. It seems that Dr. Holmes or someone else should have been assigned I responsibility for the twenty-eight time line activities that had been assigned to Dr. James in the October 10, 2002 Compliance Plan
but the Compliance Report is silent on this point. G. LRSD Files Its Compliance Report Seeking Release from Court Supervision. When LRSD filed its Compliance Report (docket no. 3837) on March 12,2004, IQ it was supported by documents attached to it as Exhibits A through G. According to LRSD, the documents attached to the Compliance Report establish that it has substantially complied with the Compliance Remedy and is entitled to be released from all fiirther court supervision and monitoring. A careful reading of the Compliance Report reveals a number of revelations that cast LRSDs efforts in a less than favorable light. First, the Report states that: (a) in October of2002, less than sixty days after the Courts September 13 Decision, Joshuas counsel raised concerns about the Board-approved Compliance Plan
(b) subsequently, Joshuas counsel invoked the Process for Raising Compliance Issues set forth in Revised Plan 8.2" and [the parties] met with Ms. Ann Marshall to facilitate .5, an agreement
and (c) the parties last such meeting with Ms. Marshall was on February 28,2003, but the parties failed to reach an agreement on Joshuas or 'On June 11,2004, the Board announced that it hired Mr. Roy Gregory Brooks to serve as LRSDs new Superintendent of Schools. It is my understanding that sometime in July August Mr. Brooks will assume his new duties as Superintendent. During the compliance hearing, this document was introduced into evidence as LRSDs Eichibit No. 14. -19- AO72A (Rev.B/82)( AO72A (Rev.e/82) objections to the adequacy of the Compliance Plan. See Comphance Report (docket no. 3837) at 1-2. Thus, LRSDs March 12, 2004 Compliance Report constituted my first notice that Ms. Marshalls facilitation efforts in late 2002 had failed-neither the parties nor the ODM brought this to my attention, although they were required to do so by subpart D ofthe Compliance Remedy and by my November 6, 2002 letter to Ms. Marshall Second, LRSD contends that, by failing to bring the parties disagreement over the Comphance Plan to the Courts attention, Joshua waived any objections to the Board-approved Compliance Plan. Id. at 1-2. As I explained earlier, the sole purpose of subpart D ofthe Compliance Remedy was to require the parties to bring potential compliance problems to my attention. as soon as they arose, so that I could resolve them early enough to prevent them fi-om becoming stumbling blocks to LRSDs compliance with its obligations under the other subparts ofthe Compliance Remedy. After the ODMs facilitation efforts failed, it must have soon become obvious to LRSDs counsel and the ODM that Joshuas counsel was not going to notify the Court. It seems to me that, at this point, both LRSD and ODM should have realized that it was in LRSDs best interest to let me determine, in March of 2003-while there was still time to do something about it~if the compliance issues raised by Joshua had merit. In a school desegregation case that has its origins in the infamous 195 7 Little Rock school desegregation crisis, no court is likely to hold the silence of Joshuas counsel-even if they are to be cnticized-against the Aftican-American students they represent, and who now fill almost 70% of the total number of seats in LRSDs classrooms. I believe I would be ill advised to adopt waiver as a way to avoid reaching the merits of the adequacy of the board-approved -20- i I I I iCompliance Plan. In other words, I find that it would be inappropriate for me to default approximately 17,000 LRSD students. Third, LRSD acknowledges in the Compliance Report (docket no. 3837) that it was required to do two things to satisfy the core obligations imposed by subparts A and B of the Compliance Remedy: (1) develop written procedures for evaluating the programs implemented pursuant to 2.7 of the Revised Plan to determine their effectiveness in improving the academic achievement of African-American students
and (2) prepare a comprehensive program evaluation of each academic program implemented pursuant to 2.7 of the Revised Plan to determine its effectiveness in improving the academic achievement of African-American students and to decide whether to modify or replace the program. See Compliance Report at page 2, paragraph 4 (docket no. 3837
LRSDs Exhibit No. 14) (emphasis added). To satisfy this first core obligation, which is contained in subpart B of the Compliance Remedy, the Compliance Report states that the Board adopted Regulation IL-Rl. As indicated previously, the Board adopted Regulation IL-Rl on October 10,2002. But, LRSD did not make I this key Regulation an Exhibit to either its March 14,2003 Notice of Filing Evaluations Pursuant to Paragraph C of the Compliance Remedy (docket no. 3745) or its March 12,2004 Compliance Report (docket no. 3837). On May 12, 2004, after I had been unable to locate this document anywhere in the record, I entered an Order (docket no. 3864) requiring that LRSD provide a copy I of Regulation IL-Rl. The next day, LRSD filed its Response (docket no. 3865), which attached I Regulation IL-Rl as Appendix 1 to Exhibit A of that document. Regulation IL-Rl contained the following procedures that were to be followed in preparing all future 2.7.1 program evaluations: 1 -21- AO72A (Rev.8/82)I i (1) Write a clear description of the curriculum/instruction program that is to be evaluated, with information about the schedule of its implementation. (2) Agree on the necessary research questions that need to be established in addition to the question: Has this curriculum/instruction program been effective in improving and remediating the academic achievement of African-American students? (3) If program modifications are suggested, the steps that the staff members have taken or will take to implement those modifications. If abandonment of the program is recommended, the steps that will be taken to replace the program with another program with more potential for the improvement and remediation of AO 72A (Rev.8/82) I II I I I (4) (5) Afiican-American students. (See 2.7.1 of the Revised Designation and Education Plan and Judge Wilsons Compliance Remedy.) Plan ways to provide regular progress reports (e.g. dissemination of meeting minutes, written progress reports, oral reports to Superintendent
Cabinet and/or Compliance Team) to stakeholders, including the Associate Superintendent for Instruction, the Superintendent of Schools, the ODM (until Unitary Status is achieved) and the Joshua Intervenors (until Unitary Status is achieved). >5 The team preparing the program evaluations had to meet to monitor the completion of assignments
to review drafts and provide feedback to the writer
and to formulate recommendations ... for program improvement, especially to decide if a recommendation is required to modify or abandon the program if the findings reveal that the program is not being successful for the -22-I improvement of Afiican-American achievement. (6) (7) (8) A final draft of the program evaluation had to be submitted to the Associate Superintendent for Instruction at least one month before placing the report on the Boards agenda for review and approval. After the program evaluation was approved by the Board, a copy of the complete report had to be made available to the ODM and Joshua Intervenors (until Unitary Status is achieved). Each program evaluation team shall meet with the Associate Superintendent for Instruction after the completion of the work to evaluate the process and product and to make recommendations for future program evaluation. See Regulation IL-RI at 3-7 (emphasis added). In paragraph 8 of the Compliance Report, LRSD states that it satisfied the second core obligation specified in subpart A of the Compliance Remedy, by doing the following things: (1) (2) Dr. Steve Ross was hired to prepare evaluations of the Districts elementary and secondary literacy programs. Ultimately, Dr. Ross authored the global Literacy Evaluation, which the Board approved in November of2003. See Exhibit F to Compliance Report (docket no. 3837).^ Dr. Don Wold, a retired member of the faculty at UALR, authored the global Math and Science Evaluation, which covered the overall math and science curricula (grades K-12) that LRSD had implemented with the grant it received ^As indicated previously, during the compliance hearing, the Compliance Report was introduced into evidence as LRSDs Exhibit No. 14. -23- AO72A (Rev.S/82)from the NSF. The Board approved this evaluation in December 2003. See Exhibit G to Compliance Report (docket no. 3837). According to the Compliance Report, these two global evaluations satisfy all of LRSDs obligations under subpart A ofthe Compliance Remedy and constitute substantial compliance with the obligations contained in 2.7.1 of the Revised Plan. Ominously, the Compliance Report says nothing about whether these two evaluations complied with the mandatory requirements of Regulation IL-Rl. H. The ODMs Compliance Report on LRSDs Implementation ofthe Compliance Remedy. On March 30, 2004, the ODM filed its Compliance Report (docket no. 3854)^ commenting on various aspects of LRSDs implementation of the Compliance Remedy. The first four pages of the ODMs Compliance Report re-plow the now largely irrelevant events related to LRSDs earlier compliance efforts under the Revised Plan."- The remaining nineteen pages of the Compliance Report contain Findings and Conclusions that, for the most part, criticize LRSDs compliance efforts. Among the more significant Findings and Conclusions in the ODMs Compliance Report are the following: . (1) As of September 13,2002, the date the Court imposed the Compliance Remedy, LRSD had implemented at leastforty-six programs designed to improve African- I During the compliance hearing, the ODMs Compliance Report was introduced into evidence as LRSDs Exhibit No. 15. -"The September 13 Decision addressed in detail all of the events described in these four pages ofthe ODMs Compliance Report. -24- ^0 72A Rev.8/82)I American achievement under 2.7 of the Revised Plan?" Id. at 10. I AO72A (Rev.8/82) (2) (3) (4) The ODM had difficulty getting LRSD administrators to identify the specific 2.7 programs that would be evaluated under 2.7.1 of the Revised Plan. LRSD administrators finally acknowledged to the ODM that the program evaluation agenda for the 2002-03 school year would include only elementary literacy, secondary literacy, and the NSF math and science project. LRSD administrators never explained to the ODM which 2.7 programs would be covered in the literacy and math and science evaluations. Id. at 12. While the Compliance Remedy directed LRSD to use all available testing data in assessing the effectiveness of the 2.7 programs, LRSD only did so in the Evaluation of the Math and Science Programs. (Exhibit G to LRSDs Compliance Report.) In the Literacy Program Evaluation (Exhibit F to LRSDs Compliance Report), LRSD limited the testing data to the SAT9 and the benchmark literacy exams. Id. at 14. Subpart B of the Compliance Remedy required LRSD to maintain written records reflecting the results of the annual assessments of each program, including whether the assessments resulted in program modifications or the elimination of programs. LRSD, 237 F. Supp. 2d at 1087-88. The ODM concluded that, while Exhibit C to LRSDs Compliance Report describes the annual program modifications for elementary and secondary literacy and math, there is no ^During the compliance hearing, LRSD witnesses correctly pointed out that the chart on page 10 of the ODMs Compliance Report was flawed. Among other things, a number of programs are listed twice, and some of the things that are listed as programs are not. -25-( AO 72A (Rev.8/82) (5) (6) (7) discussion of annual program modifications for science.^'* Id. at 15. The ODMs Compliance Report noted numerous shortcomings in the Literacy Program Evaluation. Id. at 1677. However, the most serious criticism was that the evaluation draws no conclusions about the extent to which student performance might be affected by program components, such as Reading Recovery or Accelerated Reader, nor does it correlate any teaching practices with student achievement. Id. at 18. The ODMs most serious criticism of the Math and Science Evaluation was that it does not offer data relative to the level of uniformity of program implementation and does not identify which 2.7 programs most directly improved the academic achievement of African-American students in math and science courses. Id. at 19. By the time the Court entered its September 13 Decision, six of the fourteen Page 148 Evaluations had already been completed or substantially completed by outside consultants. Subpart C of the Compliance Remedy provided that the Court will accept all program evaluations that have already been completed by Dr. Nunnery or someone with similar qualifications and approved by the Board. Between October 24, 2002, and December 19,2002, the Board voted to approve these six evaluations which covered the following programs: (a) Pre K-3 Literacy
(b) NSF Math and Science
(c) Charter School
(d) English as a Second Language
(e) 2'Although not mentioned in the ODMs Compliance Report, Exhibit C also fails to discuss the reasons for making each of the modifications in the math and literacy programs, or and how those modifications were expected to improve the effectiveness of those programs. -26-SEDL Program.- Southwest Middle School
and (f) Collaborative Action Team. The ODMs Compliance Report does not contain any significant criticism of these six evaluations. Id. at 20. (8) (9) As of September 13,2002, eight ofthe Page 148 Evaluations still remained to be prepared. Under subpart C of the Compliance Remedy, LRSD was obligated to hire outside consultants to prepare those eight evaluations and the
This project was supported in part by a Digitizing Hidden Special Collections and Archives project grant from The Andrew W. Mellon Foundation and Council on Library and Information Resoources.