{"response":{"docs":[{"id":"bcas_bcmss0837_445","title":"English as a Second Language","collection_id":"bcas_bcmss0837","collection_title":"Office of Desegregation Management","dcterms_contributor":null,"dcterms_spatial":["United States, 39.76, -98.5","United States, Arkansas, 34.75037, -92.50044","United States, Arkansas, Pulaski County, 34.76993, -92.3118","United States, Arkansas, Pulaski County, Little Rock, 34.74648, -92.28959"],"dcterms_creator":null,"dc_date":["2002"],"dcterms_description":null,"dc_format":["application/pdf"],"dcterms_identifier":null,"dcterms_language":["eng"],"dcterms_publisher":["Little Rock, Ark. : Butler Center for Arkansas Studies. Central Arkansas Library System."],"dc_relation":null,"dc_right":["http://rightsstatements.org/vocab/InC-EDU/1.0/"],"dcterms_is_part_of":["Office of Desegregation Monitoring records (BC.MSS.08.37)","History of Segregation and Integration of Arkansas's Educational System"],"dcterms_subject":["Little Rock (Ark.)--History--20th century","Little Rock School District","Education--Arkansas","Educational planning","English language--Study and teaching--Foreign speakers","Education--Evaluation","Educational statistics","Student activities"],"dcterms_title":["English as a Second Language"],"dcterms_type":["Text"],"dcterms_provenance":["Butler Center for Arkansas Studies"],"edm_is_shown_by":null,"edm_is_shown_at":["http://arstudies.contentdm.oclc.org/cdm/ref/collection/bcmss0837/id/445"],"dcterms_temporal":null,"dcterms_rights_holder":null,"dcterms_bibliographic_citation":null,"dlg_local_right":null,"dcterms_medium":["documents (object genre)"],"dcterms_extent":null,"dlg_subject_personal":null,"dcterms_subject_fast":null,"fulltext":"\n \n\n\n\n\n\n\n\n  \n\n\n   \n\n   \n\n\n   \n\n\n   \n\n\n\n\n   \n\n\n\n\n   \n\n\n\n\n\n\n\n\n\n\n\n   \n\n   \n\n \n\n\n   \n\n   \n\n  \n\n\n   \n\n  \n\n \n\n\n   \n\n  \n\n \n\n\n   \n\n  \n\n  \n\na .- RECEIVED TO: LITTLE ROCK SCHOOL DISTRICT INSTRUCTIONAL RESOURCE CENTER 3001 PULASKI STREET LITTLE ROCK, AR 72206 February 7, 2002 Ann Marshall, ODM FEB 1 2 2002 (SnCEOF DESEGREGATION MONITORING bl-i FROM: Dr. Bonnie Lesley, Associate Superintendent for Instruction SUBJECT: 2000-2001 ESL Program Evaluation We are attaching a copy of the 2000-2001 ESL program evaluation. Please share it also with Horace Smith. We are rather pleased with the findings. Weve come a long way. Let us know if you have questions. BAL/adg Attachment frtn tJlSC) i-it-IZ LITTLE ROCK SCHOOL DISTRICT INSTRUCTIONAL RESOURCE CENTER 3001 PULASKI STREET LITTLE ROCK, AR 72206 TO: Board of Education FROM: PREPARED BY: T. Kenneth James, Superintendent of Schools ^^onnie Lesley, Associate Superintendent for Instruction DATE: February 28, 2002 SUBJECT: Program Evaluation: Alternative Language Program Background Information A part of the Districts Commitment to Resolve with the Office for Civil Rights in 1999 was to conduct an annual program evaluation of the Alternative Language Program and to modify the programs and services based on the findings. The first program evaluation was presented to the Board of Education in August 2000 and submitted to OCR in October 2000. The second annual evaluation is now ready for Board review. Eddie McCoy was assigned the lead responsibility for writing the program evaluation in fall 1999. Other staff (Karen Broadnax, Ed Williams, Ken Savage, and Bonnie Lesley) have worked with her to verify data and the overall accuracy of the report. Mark Vasquez served as consultant to the process. Fiscal Impact No fiscal impact is anticipated. Recommendation That the Board of Education review and provide feedback to the ESL program evaluation. BAL/adg Attachment cc: Karen Broadnax Dr. Eddie McCoy I The Second Annual Program Evaluation for Alternative Language Program (ALP) School Year: 2000-2001 (Introduction and Key Findings/Recommendations) January 30, 2002 Division of Instruction Little Rock School District Prepared by Eddie Williams-McCoy Karen Broadnax Dr. Ed Williams Ken Savage Dr. Bonnie Lesley Mark Vasquez, Consultant T Table of Contents I I 1 I Section I: introduction Introduction Program Goals and Objectives Research Questions Methodology Evaluation Design and Focus Assessments Outline of Program Evaluation Section II: Policies, Procedures, and Programs to Ensure Compliance i I Top Ten Languages of PHLOTE, LEP, and FEPE Students PHLOTE Students LEP Students FEPE Students Policy, Procedural, and Program Changes Program Objective 1 (Student Identification) Program Objective 2 (Assessment) Program Objective 3 (Program Placement) Organization and Role of LPACs Student Assignment Program Objective 4 (ESL Program) Curriculum Standards and Benchmarks Assessments ESL InstructionElementary Schools ESL InstructionMiddle Schools ESL InstructionHigh Schools Program Objective 5 (Staffing/Development) ESL Training Program (Elementary) ESL Training Program (Secondary) Schedule for ESL Training Summary of ESL Training Participation ESL Endorsement Program Summary of ESL Endorsement Participation Program Objective 6 (Instructional Materials) Program Objective 7 (Program Exit) Program Objective 8 (Parental Involvement) Program Objective 9 (Special Programs) Special Education Gifted and Talented Special Programs by PHLOTE and Grades Third Grade G/T PHLOTE Students Fourth Grade G/T PHLOTE Students Fifth Grade G/T PHLOTE Students 1 1-2 2-3 3-4 4-5 5 5-6 7-9 7 8 8-9 9 9-10 10 10-12 11 11-12 13-16 13-14 14 14-15 15 16 17-19 17 17 18 18 18-19 19 21 22 22 22-27 23 23-24 24 24 24-25 25r-  Middle-Level G/T and Special Education PHLOTE Students Special Education PHLOTE Students GfT PHLOTE Students Senior-Level G/T and Special Education Special Education PHLOTEs (9-10) G/T PHLOTEs (9-10) Special Educatoin PHLOTEs (11-12) G/T PHLOTEs (11-12) Program Objective 10 (Exited LEPs) Students Who Exited Prior to Fall 2000 Students Who Exited in 2000-2001 Program Objective 11 (Program Evaluation) Program Objective 12 (Student Records) Program Cost Section III: English Language Acquisition of LEP and FEPE Students Developmental Reading Assessment Kindergarten Performance Overall Performance of PHLOTEs Kindergarten PHLOTES with PreK Kindergarten PHLOTES without PreK Overall Performance of LEPs Kindergarten LEPs with PreK Kindergarten LEPs without PreK Kindergarten FEPEs Kindergarten PerformanceTwo Years Overall Performance of PHLOTEs Overall Performance of LEPs Summary and Conclusions First Grade PerformanceTwo Years Overall Performance of PHLOTEs Performance of LEPs Summary and Conclusions Second Grade PerformanceTwo Years Overall Performance of PHLOTEs Performance of LEPs Performance of FEPEs Summary and Conclusions Newcomer Centers School Enrollment by PHLOTE Elementary Newcomer Centers Elementary Non-Newcomer Centers DRA Results by Schools DRA by Newcomer Centers DRA by Non-Newcomer Centers Overall Performance of PHLOTEs 25 25 26 26 26 27 27 27 28-34 29-32 32-34 34-35 35-36 36 37-38 38- 38-39 39 39-40 40 40-41 41 42 42-44 42-43 43 43-44 44-45 44 44-45 45 45-47 46 46-47 47 47 48- 48 48-49 49-50 50-54 50 50-51 51-52 iiOverall Performance of LEPs Conclusions Achievement Level Tests 52-54 54 54-55 Data Interpretation for LEPs and FEPEs 55 ALT ReadingGrade 2 PHLOTEs ALT ReadingGrade 2 LEPs Summary and Conclusions ALT ReadingGrade 3 PHLOTEs ALT ReadingGrade 3 LEPs ALT ReadingGrade 3 FEPEs Summary and Conclusions ALT ReadingGrade 4 PHLOTEs ALT ReadingGrade 4 LEPs ALT ReadingGrade 4 FEPEs Summary and Conclusions ALT ReadingGrade 5 PHLOTEs ALT ReadingGrade 5 LEPs ALT ReadingGrade 5 FEPEs Summary and Conclusions ALT ReadingGrade 6 PHLOTEs ALT ReadingGrade 6 LEPs ALT ReadingGrade 6 FEPEs Summary and Conclusions ALT ReadingGrade 7 PHLOTEs ALT ReadingGrade 7 LEPs ALT ReadingGrade 7 FEPEs Summary and Conclusions ALT ReadingGrade 8 PHLOTEs ALT ReadingGrade 8 LEPs ALT ReadingGrade 8 FEPEs ALT ReadingGrade 9 PHLOTEs ALT ReadingGrade 9 LEPs ALT ReadingGrade 9 FEPEs Summary and Conclusions ALT ReadingGrade 10 PHLOTEs ALT ReadingGrade 10 LEPs ALT ReadingGrade 10 FEPEs Summary and Conclusions Conclusions\nALT Reading 55-56 56 56 56-57 57 58 58-59 59-60 60 60-61 61 61-62 62-63 63 63 63-64 64-65 65 65 65-66 66-67 67 67 68 68-69 69-70 70 71 71-72 72 72-73 73 73-74 74 74 ALT Language Usage ALT Language Grade 2 LEPs ALT Language Grade 2 FEPEs ALT Language Grade 3 LEPs ALT Language Grade 3 FEPEs ALT Language Grade 4 LEPs ALT Language Grade 4 FEPEs ALT Language Grade 5 LEPs 74-75 75 75 75-76 76 76-77 77 78 iiiALT Language Grade 5 FEPEs ALT Language Grade 6 LEPs ALT Language Grade 6 FEPEs ALT Language Grade 7 LEPs ALT Language Grade 7 FEPEs ALT Language Grade 8 LEPs ALT Language Grade 8 FEPEs ALT Language Grade 9 LEPs ALT Language Grade 9 FEPEs ALT Language Grade 10 LEPs ALT Language Grade 10 FEPEs ALT Language Grade 11 LEPs ALT Language Grade 11 FEPEs 78-79 79 79-80 80 80-81 81 81-82 82 82-83 83 83 83 83-84 Section IV: Content Area Knowledge and Skills of LEP and FEPE Students ALT Mathematics Grade 2 LEPs ALT Mathematics Grade 2 FEPEs ALT Mathematics Grade 3 LEPs ALT Mathematics Grade 3 FEPEs ALT Mathematics Grade 4 LEPs ALT Mathematics Grade 4 FEPEs ALT Mathematics Grade 5 LEPs ALT Mathematics Grade 5 FEPEs ALT Mathematics Grade 6 LEPs ALT Mathematics Grade 6 FEPEs ALT Mathematics Grade 7 LEPs ALT Mathematics Grade 7 FEPEs ALT Mathematics Grade 8 LEPs ALT Mathematics Grade 8 FEPEs ALT Algebra I LEPs and FEPEs ALT Algebra 11 LEPs and FEPEs ALT Geometry LEPs and FEPEs 85 85 86 86-87 87 87-88 88 88-89 89 89-90 90 90 91 91 91-92 92 92 ALT Biology LEPs and FEPEs ALT Physics LEPs and FEPEs ALT Chemistry LEPs and FEPEs 93 93 93-94 Stanford Achievement Test, 9* Edition Grade 5 PHLOTEs Grade 5 LEPs Grade 5 FEPEs Grade 7 PHLOTEs Grade 7 LEPs Grade 7 FEPEs Grade 10 PHLOTEs Grade 10 LEPs Grade 10 FEPEs 94-95 95 95-96 96 96-97 97 97 97-98 98 98 ivArkansas Benchmark Examinations Primary and Middle Level Definitions Grade 4 PHLOTEs Grade 4 LEPs Grade 4 FEPEs Grade 8 PHLOTEs Grade 8 LEPs Grade 8 FEPEs 99 99-100 100 100-101 101 101 101-102 102 Section V: Behavioral Performance Indicators of LEP and FEPE Students Background Information on PHLOTEs in District Grade Distribution of PHLOTEs Grade Distribution of LEPs and FEPEs Six Highest PHLOTE Student Enrollments School Enrollment by LEP and FEPE PreKindergarten PHLOTEs Kindergarten PHLOTEs First Grade PHLOTEs Second Grade PHLOTEs Third Grade PHLOTEs Fourth Grade PHLOTEs Fifth Grade PHLOTEs Sixth Grade PHLOTEs Seventh Grade PHLOTEs Eighth Grade PHLOTEs Ninth Grade PHLOTEs Tenth Grade PHLOTEs Eleventh Grade PHLOTEs Twelfth Grade PHLOTEs 103 103-104 104 105 105 106 106 106-107 107 108 108-109 109-110 110-111 111 112 112-113 113-114 114-115 115-116 Graduation Rates for PHLOTEs Summary Who Are the Dropouts? 116 116 116-117 Section VI\nKey Findings and Recommendations Research Question 1 (Policies, Procedures, Programs) Research Question 2 (Special Opportunity Programs) Research Question 3 (Quality of Instruction) Research Question 4 (Learning English) Effect of PreKindergarten Two-Year Comparison of Kindergarten DRAs Two-Year Comparison of Grade 1 DRAs Two-Year Comparison of Grade 2 DRAs Summary of K-2 Findings 118 118 118-119 119-125 119-120 120 120-121 121 121 VComparison of K-2 Students in Newcomer and Non-Newcomer Center Schools Summary of Findings 121-122 122 Cohort Performance on ALT Reading Summary of Findings 122-123 123 Cohort Performance on ALT Language Usage Summary of Findings 123-124 124 Stanford Achievement Test Summary of Findings 124-125 125 Arkansas Benchmark Examinations Summary of Findings Conclusions 125 125 125 Research Question 5 (Content Knowledge and Skills) Cohort Performance on ALT Mathematics Summary of Findings 125-127 125-126 126 ALT High School Mathematics/Science Summary of Findings 126-127 127 Stanford Achievement TestMathematics Summary of Findings 127 127 Arkansas Benchmark Examinations Summary of Findings Conclusions 127 127 127 Research Question 6 (Behavioral Indicators) Attendance Discipline and Suspensions Retention Dropouts Graduation Rate Summary of Findings 127-129 128 128 128 128 128 128-129 Recommendations 129-130 References 131 vi Second Annual Evaluation of the Little Rock School Districts Alternative Language Program Section I: Introduction This Second Annual Evaluation of the Effectiveness of the Little Rock School Districts (LRSD) Alternative Language Program builds on the information provided in an earlier report that was submitted to the Board of Education in August 2000 and to the Dallas Office of Civil Rights (OCR) in October 2000. The report for school year 2000-2001 is a part of the Little Rock School Districts continuing efforts to meet the requirements outlined in the LRSDs Commitment to Resolve (CTR) agreement with the Office for Civil Rights (September 29, 1999), to comply with the requirements of Title VI of the Civil Rights Act of 1964, and to meet the requirements of the Arkansas Department of Education. The requirement to conduct an annual evaluation of the Alternative Language Program was formally established by the Board of Education in its adoption of Policy IHBEA: English as a Second Language, on November 18,1999. Specific requirements and procedures were delineated in the administrative regulations (IHBEA-R) as adopted in October 1999 and amended in November 2001. This report also is in compliance with the Boards Policy IL: Evaluation of Instructional Programs, adopted on March 22, 2001, that requires program evaluations to include valuable insights into how programs are operating, the extent to which they are serving the intended purpose of increasing student achievement, the strengths and weaknesses, the cost effectiveness, and directions for the future. Program Goals and Objectives According to the Districts Administrative Regulation IHBEA-R: English as a Second Language (adopted in September 1999 and amended in November 2001), the first goal of the ESL program is to enable identified students to master English language skills (reading, writing, speaking, and comprehension/understanding) and content area concepts and skills so that the students are able to participate effectively in the regular program as quickly as possible. The second goal of the program is to provide identified students with the cultural literacy necessary for them to feel comfortable in participating in the school, community, and greater community. To reach those goals, the District also established twelve program objectives. Each one addressed a policy, procedural, or program requirement mandated by federal, state, and/or local governance bodies. They are as follows: 1 I 1. 2. 3. 4. 5. 6. 7. 8. To implement and maintain consistent procedures for student identification processes. To assess all students who have a primary home language other than English (PHLOTE), regardless of whether they are enrolled in a Newcomer Center. To establish and administer consistently appropriate criteria for entry and placement into an ESL program. To diagnose student needs and provide appropriate ESL standards/ benchmarks, instruction, and assessments to meet identified students individual needs for English-language instruction, for understandable instruction in other content areas, and for positive self-concept and identification with personal/family cultural heritages. To hire, train, and continually develop highly motivated, sensitive, and caring ESL teachers and other staff to provide effective ESL instruction, interact one-to- one with the identified students and their families, and serve as liaisons between school and relevant community. To provide appropriately aligned instructional materials. To establish and administer consistently appropriate criteria for exit from an ESL program. To provide for parental/family involvement in the school setting to support improved student learning. 9. To provide equitable access to other district programs and services, including special education and gifted/talented education and all procedural safeguards. 10. To monitor the progress of all identified students during program participation and after program exit and to reclassify students as needed. 11 .To evaluate the ESL program and make program modifications as needed. 12.To maintain accurate and useful student records, including procedural safeguards. Research Questions To address the requirements of federal, state, and local governance, the following research questions were selected to guide the evaluation of the effectiveness of the Districts Alternative Language program\nProgram Implementation 1. Has the District implemented appropriate policies, procedures, and programs to comply with federal law, the Commitment to Resolve with OCR, the ADE, and local LRSD policy? What has been the cost of implementation? 2. What is the evidence that LEP students have appropriate access to the range of special opportunity programs in the District, including special education, gifted/talented programs, 504 programs, and co/extra-curricular programs? 3. Has the District been successful in improving the quality of instruction through either ensuring an adequate number of ESL-endorsed teachers to serve the identified LEP students or in providing adequate training to ensure competent performance? I 2 I 5 t Academic Progress 4. Are identified LEP students being served in the Alternative Language Pmnrorr* _ __________  ____  .. . _ iy prggress in learning reading and English language arts soki/iiHllcsO? How does their _p__e__rfco__r_m___a nce compare wi_t_h__ _t_h e general popiyulation with PHLOTE students in general, and with fluent English-proficient students who have exited the program (FEPEs)? 5. Are identified LEP students being served in the Alternative Language Program making progress in learning content knowledge and skills? How does their performance compare with the general population, with PHLOTE students in general, and with fluent English-proficient students who have exited the program (FEPEs)? Other Behavioral Indicators 6. What are the attendance rates, retention rates, discipline/suspension rates dropout rates, and graduation rates of identified LEP students being served i the Alternative Language Program? How does their performance compare with that of the general population, with PHLOTE students in general and in with fluent English-proficient students who have exited the program (FEPEs)? Methodology A team comprised of staff from the Division of Instruction and from the Department of Testing, Research and Evaluation was assembled early in the year to determine the scope and focus of the evaluation, to formulate the research questions, and to ensure the availability of appropriate data upon which to base findings and conclusions. During all stages of the process, this team relied on findings from best practices in the development aanud ceififceucutivvec iimiiippileemmeennitaaitiioonn oorf AAlltteerrnnaattiivvee LLaanngguuaaggee PPrrooggrraammss and effective techniques for monitoring and assessing the performance of limited-EnqIish Mr. Mark Vasquez, an expert on the requirements for compliance with the Office of Civil Rights relating to LEP issues, served as a consultant to the team who prepared the program evaluation. Since this study is a year-two evaluation, the team decided to focus on the gualitv of the iimmnplloemmoennttoattiiroinn of the program, Xto__ _m___o__n:ixt_o_r 1t1h. e. _p errf ormance of _L__E__P_ and__ _F__EjPr ^E1 sUtudents academic performance in relation to the performance of the general student population in both acquiring English language skills and content knowledge and skills, and to monitor certain behavioral variables such as attendance, retention, discipline/ suspension, dropout, and graduation rates. Compiled data results for each academic or behavioral variable are displayed Tables follow the descriptions of the data. The descriptions always start with an overview of the demographics of the table, i.e., how many LEPs, FEPs, etc. The demographic overview is followed by An Overview of the Overall Performance of PHLOTE Students (Primary Home Language Other Than English) students. Caul description proceeds with the performance of LEP (Limited English Proficient) and in tables. Each FEPE (Fluent English Proficient Exited) students. A summary follows each description 3 of each PHLOTE category. Conclusions are included at the end of each grade-level section. Unless otherwise indicated, pre assessment refers to the spring of the 1999-2000 school year, and post refers to the spring of the 2000-2001 school year. The exception is with the Development Reading Assessment (DRA) for the kindergarten, first grade, and second grade students who took the DRA during the 2000-2001 school year. In these cases, pre refers to the fall of the 2000-2001 school year, and post refers to the spring of the 2000-2001 school year. Evaluation Design and Focus The general population for this report will consist of all students in the LRSD except the PHLOTE (Primary Home Language Other Than English) students. PHLOTE students include the following sub-populations: LEP, FEP, FEPE, NALMS, REFUSE, LEPREC, and LEPREF which are designated by acronyms. The acronyms and the definitions of the sub-populations of PHLOTE students are as follows:  LEP (Limited-English Proficient): Students administered the Language Assessment Scale (LAS) upon admission to the LRSD and determined not to be proficient in reading, writing, listening comprehension, and/or speaking English.  NALMS (Not Assessed Language Minority Students): PHLOTE students whose English language proficiency has not been assessed.  FEP (Fluent-English Proficient): PHLOTE students administered the LAS upon admission to the LRSD and determined to be proficient in reading, writing, listening comprehension, and speaking English.  FEPE (Fluent-English Proficient and Exited from the program): PHLOTE students initially identified as LEP\nreceived LRSD ESL program services\nand then exited the program after the LAS indicated a proficiency in reading, writing, listening comprehension, and speaking English.  REFUSE (Parent Refused Initial Assessment): PHLOTE students whose parents refused permission to assess the students English proficiency. I i I J  LEPREC (Limited English Proficient Reclassified): FEPE students whose performance after exiting the ESL program required that the student to again be identified as LEP.  LEPREF (Limited English Proficient Refused Services)\nPHLOTE students who were identified through language assessment to be LEP\nhowever, the parents refused permission to place the student in the ESL program. 4Although the evaluation analyzes the achievement of PHLOTE students as a group, the primary focus of the evaluation is the performance (behavioral and academic) of LEP (Limited- English Proficient) and FEPE (Fluent-English Proficient Exited) students. LEP students are the students receiving ALP (Alternative Language Program) services, and FEPE students are former LEP students who have been exited from the AL program and are no longer receiving direct services. The performance (academic) of FEPE students must be monitored in case reclassification is necessary. Reclassification is the process by which a FEPE student re-enters the ALP because he/she was not successful in the regular school program and is, thus, reclassified as LEP. Assessments The descriptions of academic performance in the English language arts are based upon student performance on the following national, state, and local assessments\n\" Stanford Achievement Test Series - Ninth Edition (grades 5, 7 and 10),Total Reading and Total Language\n Arkansas Benchmark Examination (grade 4 and 8), Literacy Examinations\n Developmental Reading Assessment (grades K-2)\n LRSD Achievement Level Tests\nReading and Language Usage (grades 2 through 11). The descriptions of academic performance in specific content knowledge and skills are based upon student performance on the following assessments\n LRSD Achievement Level Tests\nElementary Mathematics (Grades 2-5)\nMiddle School Mathematics (Grades 6-8), and the following content courses at the secondary level\n Physics  Chemistry  Biology,  Algebra I and II  Geometry. Outline of Program Evaluation This document is divided into six sections. Each of sections ll-V addresses specific research questions. Section I: Introduction The first section includes the introduction, an outline of the Districts program goals and objectives, a delineation of the research questions for the study, a description of methodologies, the evaluation focus, a list of assessments used to gather academic and behavioral data, and an outline of the sections in the stud/ Section II\nPolicies, Procedures, and Programs to Ensure Compliance Section II includes evidence of the Districts implementation of numerous new policies, procedures, and programs to ensure compliance and quality, including cost data, all aligned with the twelve program objectives. Also included are demographic datanumbers of PHLOTE, LEP, and PEPE students, as well as statistics on the top ten language communities represented by each subpopulation. 5 s Section III: English Language Acquisition Performance of LEP and FEPE Students This section includes an evaluation of the effectiveness of programs that develop students' English language arts skills. Section IV: Content Area Knowledge and Skills of LEP and FEPE Students This section evaluates LEP and FEPE students' progress in attaining content area knowledge and skills. Section V\nBehavioral Performance Indicators of LEP and FEPE Students This section evaluates LEP and FEPE students' performance in relation to attendance, retention, discipline and suspension, dropout, and graduation rates. Section VI: Key Findings and Recommendations\nBibliography 6Section VI: Key Findings and Recommendations Program Implementation 1. Has the District implemented appropriate policies, procedures, and programs to comply with federal law, the Commitment to Resolve with OCR, the ADE, and local LRSD policy? What has been the cost of implementation? A review of documents submitted to OCR as per the agreed-upon schedule, of the annual report to the ADE, and of LRSD data and reports, along with the information provided for this program evaluation provides adequate evidence for the Districts compliance with federal, state, and local expectations. Although the District has invested substantial funding in these efforts, most of the funds have come from the federal Class-size Reduction grant, from Title I, and from the States allocation for LEP students. District funds have been used to pay the salaries and office expenses of the ESL Supervisor, one-half secretary, and 20 percent of the ESL Parent Coordinators salary. The remaining 80 percent of her salary is paid through Title I. 2. What is the evidence that LEP students have appropriate access to the range of special opportunity programs in the District, including special education, gifted/talented programs, 504 programs, and co/extra-curricular programs? 5 J i As described under Program Objective 9 above, the LRSD has established and implemented appropriate policies and procedures, as well as training of various levels of staff, to ensure compliance with the expectation that LEP students will have appropriate access to the range of special opportunity programs. The District has provided evidence of the inclusion of LEP students in its special education and gifted/talented programs. In addition to the data provided, LEP students are also represented in the Pre-Advanced Placement and Advanced Placement courses at all secondary schools and in the University Studies (concurrent enrollment with the University of Arkansas at Little Rock) at Hall High School. Section III of this study also provides evidence of the participation of LEP students in the Districts pre-kindergarten program. 3. Has the District been successful in improving the quality of instruction through either ensuring an adequate number of ESL-endorsed teachers to serve the identified LEP students or in providing adequate training to ensure competent performance? Program Objective 5 activities described above includes evidence of the Districts effort's to support teacher endorsement in ESL, to provide training for scores of other teachers to enhance their effectiveness, to provide administrator training, and to schedule students appropriately with the trained or endorsed teachers. The ESL Supervisors database provides further evidence of the Districts commitment to provide quality instruction for LEP students. At all three levels of schools (elementary, middle, and high), the District already had at the end of the second year (2000-2001) more endorsed teachers than they projected a need for in the fall of 1999. 118Having more and more teachers endorsed allows a school to be more flexible in scheduling students, and it also enables the District to serve without so much stress its growing population of LEP students. 1 i Academic Progress 4. Are identified LEP students being served in the Alternative Language Program making progress in learning reading and English language arts skills? How does their performance compare with the general population, with PHLOTE students in general, and with fluent English-proficient students who have exited the program (FEPEs)? i i i 5 Effect of Prekindergarten on Academic Progress The performance of six groups of kindergarten students were displayed in the tables in the first part of Section 111\nall kindergarten PHLOTE students\nkindergarten PHLOTE students with pre-kindergarten\nkindergarten PHLOTE students without pre- kindergarten\nall kindergarten LEP students\nkindergarten LEP students with pre- kindergarten\nand, finally, kindergarten LEP students without pre-kindergarten. Each of the groups is rank-ordered below in terms of its performance on the spring 2001 DRA the percent of students performing At/Above the readiness level: PHLOTE kindergarten with pre-kindergarten LEP kindergarten with pre-kindergarten LRSD General Population All PHLOTE kindergarten All LEP kindergarten PHLOTE kindergarten without pre-kindergarten LEP kindergarten without pre-kindergarten 86% 81% 81% 78% 76% 68% 67% The rank order of the groups according to gains from the fall 2000 pre-test to the spring 2001 post-test is as follows: LEP kindergarten with pre-kindergarten PHLOTE kindergarten with pre-kindergarten All LEP kindergarten All PHLOTE kindergarten LEP kindergarten without pre-kindergarten PHLOTE kindergarten without pre-kindergarten 76%age points 72%age points 67%age points 67%age points 57%age points 57%age points Either way the data are examined reveals the importance of students participation in the pre-kindergarten program for more rapid acquisition of English language skills. The PHLOTE and LEP students with LRSD pre-kindergarten experience both out-performed those without pre-kindergarten, and they grew at faster rates. In comparison to all kindergarten students in the District, both the PHLOTE and LEP kindergarten students with LRSD pre-kindergarten experience out-performed the general population. The LRSD average in spring 2001 for all students was 80.7. The PHLOTE kindergarten average was 86%, and the LEP kindergarten average was 81%. It is also important to examine the differences in the performance of PHLOTE kindergarten students with LRSD pre-kindergarten (86% At/Above readiness) and those 119 without (68%)a difference of 18 percentage points. A similar disparity is found between the LEP kindergarten students with LRSD pre-kindergarten (81% At/Above readiness) and those without (67%)a 14 point difference. These differences are especially important, given that there is almost no difference between the performance of PHLOTE kindergarten students in general (78% At/Above readiness) and LEP kindergarten students in general (75% At/Above readiness), although the LEP students comprise 86 percent of the total kindergarten PHLOTE population. This study further finds that LEP kindergarten students who have attended LRSD pre-kindergarten (81 % At/Above readiness) out-perform PHLOTE kindergarten students in general (78% At/Above) and LRSDs general population (80.7% At/Above readiness). The students with the pre-kindergarten experience had an advantage over the students without the pre-kindergarten experience. This finding validates the research on the importance of early childhood education (Bredekamp, Knuth, Kunesh, and Shulman. 1992\nKagan, 1995\nSchwartz, 1996). Two-Year Comparison of Kindergarten DRA Scores Just as the students in the second year of the Districts new literacy program improved at each grade level, K-2, over the performance of students in the first year of the programs implementation, so did the PHLOTE and LEP students.  LRSDs general kindergarten population improved from 72% readiness to 81%-9 points.  Kindergarten PHLOTE students improved from 67% to 79%12 points.  Kindergarten LEP students improved from 59% to 76%-17 points. Both PHLOTE and LEP students improved more on average than the general population, with LEP students achieving the most improvement. The gap between the general population and PHLOTE students decreased from 5 points in 1999-2000 to only 2 points in 2000-2001 The gap between the general population and LEP students decreased from 13 points to 5 points in the second year. Two-Year Comparison of Grade 1 DRA Scores At grade 1, the general population, PHLOTEs and LEPs students all improved the second year of the program:  LRSDs general grade 1 population improved from 54% to 64%-10 points.  Grade 1 PHLOTE students improved from 33% to 60%--17 points.  Grade 1 LEP students improved from 23% to 59%--36 points. Again, both PHLOTE and LEP students improved more on average than the general population, and, again, with LEP students improving the most. The gap between the general population and PHLOTE students decreased from 21 points in 1999-2000 to 4 120 I points in 2000-2001, and the gap between the general population and LEPs decreased from 31 points to 5. Two-Year Comparison of Grade 2 DRA Scores The pattern of improved achievement during the second year of program implementation continued for grade 2 for the general population, PHLOTEs, LEPs, and FEPEs.  LRSDs general population at grade 2 improved from 68% to 75%--7 points.  Grade 2 PHLOTE students improved from 56% to 65%--9 points.  Grade 2 LEP students improved from 50% to 57%--7 points.  Grade 2 FEPE students improved from 60% to 100%~40 points. The grade 2 PHLOTE and FEPE students both improved more in the second year than did the general population. The LEP students improved the same. The gap between the general population and PHLOTE grade 2 students decreased from 12 points to 10 points in the second year and for the FEPE students decreased from 8 points to -25. The gap remained the same for LEP students18 points. Summary of K-2 Findings LEP students grew more than PHLOTE students or the general population in both kindergarten and grade 1. FEPE students, although small in number, all achieved the District standard at grade 2 (100% readiness), and their improvement in the second year was 40 points. FEPE students outperformed the general population in grade 3. Comparison of K-2 Students in Newcomer and Non-Newcomer Schools At the K-2 levels, there were 145 PHLOTE students attending Newcomer Schools in 2000-2001, and 140 attended non-Newcomer Center schools. Eight elementary schools had no PHLOTE students.  At the kindergarten level the Newcomer Center PHLOTE students (82%) outperformed the District (81%) and non-Newcomer Center schools (76%) with the percent of students reaching the readiness standard. LEP students (83%) at Newcomer Centers also outperformed the District and non-Newcomer Center schools (70%).  At grade 1 the PHLOTE (66%) students at the non-Newcomer Center schools outperformed the PHLOTE students at Newcomer Centers (53%), as well as the general population (64%). LEP students at the Newcomer Center schools performed as well as those in non-Newcomer Center schools (59%), but not as well as the general population (64%), 121 i Grade 2 non-Newcomer Center school PHLOTE students (70%) performed significantly better than those in the Newcomer Center schools (56%) and just five points below the general population (75%). Non-Newcomer Center LEP students (76%) also significantly outperformed LEP students in the Newcomer Centers (44%) and outperformed the general population by one point (75%). Summary of Findings Both PHLOTE and LEP students at the kindergarten level at the Newcomer Center schools outperformed the PHLOTE and LEP students at the non-Newcomer Center schools, as well as the Districts general population. At grades 1 and 2, however, both PHLOTE and LEP students at the non-Newcomer Center schools outperformed their peers in the Newcomer Center schools. In grade 1 the PHLOTE students in the nonNewcomer Center schools outperformed the general population. Cohort Performance on the Achievement Level Reading Test The Northwest Evaluation Association recommends the use of median RIT scores for the purpose of evaluating student performance.  The Grade 2 general population (183) outperformed both PHLOTE (171) and LEP (179) students.  The 1999-2000 grade 2 FEPE students grew 21 points by the end of grade 3, as compared to an 18-point growth by PHLOTEs, a 10-point growth by LEPs, and only one point by the general population. Both PHLOTE and FEPE students scored higher than the general population in grade 3 during 2000-2001. i 1  The 1999-2000 grade 3 PHLOTE students grew 16 points by the end of grade 4, as compared to a 13-point growth by LEPs, 10 points by FEPEs, and no growth by the general population. FEPE students outperformed the general population both in grade 3 in 1999-2000 and in grade 4 in 2000-2001. PHLOTE students also outperformed the general population in 2000-2001.  The 1999-2000 grade 4 LEP students grew 11 points by the end of grade 5, as compared to 10 points by the FEPEs, 3 points by the general population and a -9 points by PHLOTES, Both PHLOTEs and FEPEs outperformed the general population in grade 4 in 1999-2000, and FEPEs outperformed the general population in grade 5 in 2000-2001.  The 1999-2000 grade 5 LEP students grew 6 points by the end of grade 6 in 2000- .- 2001, as compared to 4 points each by PHLOTEs, FEPEs, and the general population. The general population outperformed PHLOTEs, LEPs, and FEPEs at the end of grade 6.  The 1999-2000 grade 6 FEPE students grew 15 points by the end of grade 7 in 2000-2001, as compared to 10 points by PHLOTEs, 3 points by LEPs, and 2 points by the general population. The general population scored higher than PHLOTEs, 122LEPs. and FEPEs in grade 6. but both FEPEs (222) and PHLOTEs (221) outperformed the general population (216) in grade 7.  The 1999-2000 grade 7 PHLOTE students grew 15 points by the end of grade 8 in 2000-2001, as compared to 10 points by FEPEs. 7 points by LEPs, and 2 points by the general population. The general population outperformed the PHLOTEs. LEPs. and FEPEs in grade 7. but PHLOTE students significantly outperformed the general population in grade 8.  The 1999-2000 grade 8 LEP students grew 7 points by the end of grade 9 in 2000- 2001. as compared to only one point of growth by PHLOTE and FEPE students and none by the general population. The general population outperformed the PHLOTE. LEP. and FEPE students in both grades 8 and 9.  The 1999-2000 grade 9 PHLOTE students grew 6 points by the end of grade 10 in 2000-2001. as compared to only one point by the general population, no growth by FEPEs. and a -2 by LEPs. Summary of Findings In general. PHLOTE and FEPE students performed better than the general population on the ALT Reading test in grades 3-4 and 7-8. This finding provides evidence of the strength and effectiveness of the elementary Alternative Language Program. The lower performance of these students in grades 6 and 9-10 requires examination to determine whether their lower performance is a result of the schools to which they are assigned, changes in the LEP and FEPE populations themselves from year to year, or the quality the Alternative Language Program at these levels. I Cohort Performance on the Achievement Level Language Usage Test Both the DRA and ALT reading scores already reported provide evidence that PHLOTE, LEP, and FEPE students are improving in their ability to read in English. The following analysis provides evidence of the acquisition of English usage skills for these students.  At grade 2 in 2000-2001, the FEPE students outperformed the general population.  The 1999-2000 grade 2 LEP students grew 14 points by the end of grade 3 in 2000- 2001. as compared to 11 points by the general population and 10 points by FEPE students. The grade 2 FEPE students outperformed the general population, as did the grade 3 FEPE students in 2000-2001.  The 1999-2000 grade 3 FEPE students grew 8 points by the end of grade 4 in 2000- 2001, as compared to 5 points by the LEPs and no growth by the general population. The grade 3 FEPE students outperformed the general population, as did the FEPE students in grade 4 in 2000-2001.  The 1999-2000 grade 4 LEP students grew 12 points by the end of grade 5 in 2000- 2001, as compared to 5 points by FEPE students and 4 by the general population. 123The grade 4 FEPE students outperformed the general population in 1999-2000, as well as in grade 5 in 2000-2001.  The 1999-2000 grade 5 LEP students grew 3 points, as compared to no growth by FEPEs and -2 by the general population by the end of grade 6 in 2000-2001. The general population outperformed the LEP and FEPE students in both grade 5 and grade 6.  The 1999-2000 grade 6 FEPE students grew 5 points by the end of grade 7 in 2000- 2001, as compared to 1 point for LEP students, and -2 for the general population. The general population outperformed LEP and FEPE students in grade 6, but the FEPE students outperformed the general population in grade 7.  The 1999-2000 grade 7 FEPE students grew 7 points by the end of grade 8 in 2000- 2001, as compared to 2 points for LEP students and none for the general population. The general population outperformed the LEP and FEPE students in both grades 7 and 8.  The 1999-2000 grade 8 LEP students grew 4 points by the end of grade 9, but both FEPEs and the general population declined. The general population outperformed both LEP and FEPE students in both grades 8 and 9.  The 1999-2000 grade 9 general population posted no growth by the end of grade 10, and both the LEPs and FEPEs posted declines. The general population outperformed both the LEPs and FEPEs in grades 9 and 10. i Summary of Findings FEPE students in grades 2-5 consistently outperformed the general population on the English Language Usage ALT. With only one exception, however, neither LEP nor FEPE students scored as well as the general population in grades 6-10. I Stanford Achievement Test The degree to which PHLOTE students, as well as LEPs and FEPEs, are acquiring English reading and language arts skills can also be measured by the SAT9 reading and language subtests, although not all LEP students take these tests, as they do the DRA and ALTS. j I  At grade 5, the FEPE students (64' %ile) significantly outperformed the general population (42 %ile) on the language subtest. PHLOTE students (45 %ile) also outperformed the general population.  At grade 7 the PHLOTE students (45' %ile) outperformed the general population (39^ %ile) on the reading subtest. The FEPE students scored at the 38' %ile.  At grade 7 both the PHLOTES (49' %ile) and FEPEs (51 %ile) outperformed the general population (SS^^ %ile) on the language subtest. i I 124 At grade 10 the PHLOTE students (44** %ile) outperformed the general population (42\" %ile). I Summary of Findings PHLOTEs outperformed the general population at all three grade levels tests on the language subtest, and FEPE students did better in grades 5 and 7. Only in grade 7 did PHLOTEs and FEPEs outperform the general population in reading. Arkansas Benchmark Examinations Not all LEP students take the Benchmark Examinations, but the scores on these tests can also be analyzed to determine the degree to which PHLOTE, LEP, and FEPE students are acquiring English literacy skills.  At grade 4, FEPE students (75%) significantly outperformed the general population (35%). PHLOTEs were close behind with 33%.  At grade 8, both PHLOTE (28%) and FEPE (25%) students outperformed the general population (17%). Summary of Findings Both PHLOTE and FEPE students are doing very well on the state Benchmark Examinations, as compared to the general population. i Conclusions A review of PHLOTE, FEPE, and LEP performance on the DRA, ALT reading, ALT language usage, SAT9 reading, SAT9 language usage, and Arkansas Benchmark Literacy Examinations, as compared to the general population, shows strong performance by PHLOTE and FEPE students at the elementary level and to some degree at the middle school level. Both PHLOTE and FEPE students compared to the general population more favorably on the SAT9 and Arkansas Benchmark examinations than they did on the ALTs. Kindergarten students who attended the LRSD prekindergarten program performed significantly better than those without this experience. Grades 1-2 students in the non-Newcomer Center schools outperformed their peers in the Newcomer Centers. ! I 5. Are identified LEP students being served in the Alternative Language Program making progress in learning content knowledge and skills? How does their performance compare with the general population and with fluent English- proficient students who have exited the program (FEPEs)? This question will be answered through analysis of student performance on the ALT mathematics test, the SAT9 mathematics subtest, and the Arkansas Benchmark Mathematics Examinations. Scores are also included for high school science ALTs. Cohort Performance on the Achievement Level Mathematics Test  The 2000-2001 grade 2 FEPE students outperformed the general population, and the LEPs did as well as the general population. 125 The 1999-2000 grade 2 LEP students grew 17 points by the end of grade 3 in 2000- 2001, as compared to 9 points by FEPEs and 2 points by the general population. In grade 2 the general population outperformed LEPs and FEPEs, but FEPEs outperformed the general population in grade 3, and the LEPs tied their score.  The 1999-2000 grade 3 FEPE students grew 8 points by the end of grade 4, compared to 5 points for LEPs and 2 points for the general population. The FEPE students outperformed the general population in grade grade 3 and grade 4. . The 1999-2000 grade 4 LEP students grew 11 points by the end of grade 5, as compared to 3 for the general population, and none for FEPEs. FEPE students tied the performance of the general population in grade 4, and the general population outscored both LEPs and FEPEs in grade 5.  The 1999-2000 grade 5 general population grew 4 points, as compared to none for LEP students and 6 for FEPEs. The FEPEs tied the performance of the general population in grade 5, but the general population outperformed the LEP and FEPE students in grade 6.  The 1999-2000 grade 6 LEP students grew 5 points by the end of grade 7 in 2000- 2001, as compared to 2 for the general population and none by FEPEs. The general population outperformed both groups in both grades 6 and 7.  The 1999-2000 grade 7 LEP students grew 8 points by the end of grade 8 in 2000- 2001, as compared to 3 by the general population and none for FEPEs. The general population performed slightly higher than FEPEs in grade 7 and slightly higher than LEPs in grade 8. Summary of Findings FEPE students generally outperformed the general population in mathematics in grades 2-4. They fall behind, however, in grades 508. Achievement Level Test: High School Mathematics and Science  FEPE students outperformed the general population in 2000-2001 in Algebra I.  FEPE students outperformed the general population in 2000-2001 in Algebra II.  FEPE students outperformed the general population in 2000-2001 in Geometry.  The general population outperformed FEPEs in 2000-2001 in Biology.  The general population outperformed FEPEs in 2000-2001 in Physics.  FEPE students outperformed the general population in 2000-2001 in Chemistry. 126Summary of Findings FEPE students outperformed the general population in all three required mathematics courses and in Chemistry. Stanford Achievement Mathematics Test  At grade 5 PHLOTE students (40* %ile) outperformed the general population (31  %ile).  At grade 7 PHLOTE students (53^' %ile) and FEPEs (41 %ile) both outperformed the general population %ile).  At grade 10 PHLOTE students (52\"'^ %ile) and LEPs (48* %ile) both outperformed the general population (46 %ile). Summary of Findings PHLOTE students outperformed the general population at all three grade levels on the SAT9 mathematics subtest. FEPE students outperformed the general population at grade 7 and LEPs outperformed them at grade 10. Arkansas Benchmark Mathematics Examination  PHLOTE (46%), LEP (33%), and FEPE (75%) students all outperformed the general population on the grade 4 Benchmark.  PHLOTE (25%) students outperformed the general population (17%) in grade 8. Summary of Findings As seen in elementary reading and language usage and in elementary mathematics on the ALT, PHLOTE students outperformed the general population in both grades 4 and 8. In addition, both LEP and FEPE students outperformed the general population in grade 4 on this important measurement. Conclusions Based on the available data on mathematics achievement, PHLOTE and FEPE students appear to be doing very well at all three levels of schools, even though the scores were weaker at the middle school level on the ALT. PHLOTE and FEPE achievement was good in chemistry, but weaker in biology and physics. Other Behavioral Indicators 6. What are the attendance rates, retention rates, discipline/suspension rates, dropout rates, and graduation rates of identified LEP students being served in the Alternative Language Program? How does their performance compare with that of the general population, with PHLOTE students in general, and with fluent English-proficient students who have exited the program (FEPEs)? The data indicate that kindergarten and grade 1 have the largest PHLOTE enrollments\nkindergarten through grade 3 the largest LEP enrollment, and grades 3 and 4 the 127 largest FEPE enrollment. The elementary school with the largest enrollment is Chicot with 109\nthe middle school with the most PHLOTEs is Cloverdale\nand the high school with the largest PHLOTE enrollment in Hall High School. Attendance  FEPE students attendance is better than the general population in grades kindergarten, 2-6, and 8-12. PHLOTE students attendance is better than the general population at grades 4-7 and grade 12. LEP students attendance is better than the general population at grade 12.  LEP students missed an average of 18 days at grade 9, apparently as a result in part of their being suspended. Discipline/Suspensions  There were no PHLOTE, LEP, or FEPE students disciplined in grades K-4. At grade 5 there were 2 students with 3 sanctions\nat grade 6 there were 6 students with 9 sanctions\nat grade 7 there were 6 students with 10 sanctions\nand at grade 8 there were 9 students with 15 sanctions. I  At the high school level at grade 9 there were 17 students with 41 sanctions and 416 missed days of school for suspensions. Grade 10 had 12 students, 26 sanctions, and 93 days of suspension. Twenty of the 29 students receiving discipline were LEP.  At grade 11 there were 5 students, 6 sanctions, and 19 days of suspension. Only 5 seniors were disciplined, with 10 sanctions, but with 371 days of missed school. Retention  A total of 50 PHLOTE students were retained in 2000-2001. Nineteen of these were in grades K-5\n6 in grades 6-8\nand 26 in grades 9-12. Dropouts  Only 4 PHLOTE students dropped out in 2000-2001. Graduation Rate  In 2000-2001,42 of the 48 PHLOTE students (88%) graduated, as compared to 27 of 27 the year before. Summary of Findings  Attendance of FEPE and PHLOTE students generally was better than the attendance of the general population. LEP attendance typically was not.  Discipline/suspensions does not appear to be a problem for any PHLOTE students, except at grades 9-10 and 12. The largest sub-group of those receiving sanctions is LEP students.  Retention occurs infrequently for PHLOTEs at the elementary and middle schools. 128 The retention rate of LEP students at the high school level is a weakness. . The dropout rate of PHLOTE students is less than that of the general population.  The graduation rate of PHLOTE students is about the same as the general population. Recommendations  The District's school-level staff must ensure that LEP students are scheduled with teachers who are either endorsed, partially endorsed, or fully trained in ESL methodologies. District-level staff should monitor carefully at the beginning of each school year to ensure compliance.  The District should continue to be advocates for more state funds for the education of LEP students.  The District should develop a plan in spring 2002 for the potential loss of the Classsize Reduction grant from the federal government and how extra teachers and professional development may be funded in 2002-2003.  District-level staff should complete and publish the procedural handbook for school staff in identifying and serving students in the Alternative Language Program. . Parent recruiters, the ESL Parent Coordinator, and other staff should actively recruit LEP students for the prekindergarten program, given the importance of that extra year in LEP students reading performance.  The District should continue with its plan to phase out elementary Newcomer Centers, given the performance of students in the non-Newcomer Center schools.  The District should examine the dip of performance of middle school LEP students to determine the cause.  The District should examine the attendance and discipline problems of LEP students in grades 9-10 and develop plans for improvement.  PHLOTE, LEP, and FEPE student performance on the elementary ALT science tests should be analyzed in 2002-2003 to help determine whether these students are acquiring content area knowledge and skills. The 2002-2003 program evaluation should include an analysis of PHLOTE, LEP,  Ine ZUUZ'ZUUO piuyidi11 cvdiuduiut oiluuiu \u0026gt;v- j i and FEPE participation in Pre-AP, AP, and University Studies courses at the middle and high school levels. 129 The District should continue to provide high levels of professional development for staff responsible for the Alternative Language Program to ensure continuous improvement. i I I s i 130Agenda RECEIVED FEB 2 8 2002 OFFICE OF DESEGREGATION MONITORING Little Rock School District Board of Directors' Meeting February 2002 R. LITTLE ROCK SCHOOL DISTRICT INSTRUCTIONAL RESOURCE CENTER 3001 PULASKI STREET LITTLE ROCK, AR 72206 TO: Board of Education FROM: PREPARED BY: T. Kenneth James, Superintendent of Schools ^/Bonnie A. Lesley, Associate Superintendent of Instruction DATE: April 25, 2002 SUBJECT: Proposed Revision of Policy IHBEA: English as a Second Language Background Information The District gathered data in 1999-2000 on its identified limited-English proficient students (LEP) relating to where they were attending school. Significant percentages were assigned to the five elementary Newcomer Centers, the two middle school centers, and the one high school center. What surprised everyone, however, is how many were attending their neighborhood schools. Virtually every school in the District had LEP students in attendance. In 1999-2000 the District also conducted its first ESL program evaluation. That study included a look at the performance of students in the Newcomer Centers as opposed to other schools. Tentative findings indicated that the elementary students in the nonNewcomer schools were performing as well as or better than those assigned to the Newcomer Centers. That finding was repeated in the 2000-2001 study, with the exception of kindergarten LEP students. This important finding told the staff several things: 1. The literacy curriculum was working well for both native English speakers and for second-language students\n2. The assignment of LEP students to trained/endorsed teachers, regardless of whether the school was a Newcomer Center, was reaping benefits\n3. Significant savings could result in transportation if LEP students, at least at the elementary level, could attend their neighborhood schools\n4. Parental involvement for LEP students would more likely improve if LEP students were not bused to Newcomer Centers but attended school closer to their homes. Also, of course, the Districts obligation to provide an alternative language program is the same in the non-Newcomer schools as it is in the Newcomer schools. Board of Education April 25, 2002 Page Two District staff began informally to phase out the automatic assignment of elementary LEP students to Newcomer Centers in 2000-2001 and have continued to allow (and not discourage) attendance at the neighborhood school in 2001-2002. Those who wished to stay in the Newcomer schools to complete elementary school were allowed to do so. The proposed deletions to Policy IHBEA remove the language relating to Newcomer Centers to reflect this proposed change. At this time LEP secondary students continue to be offered an assignment to a Newcomer Center. If, however, enrollment continues to grow in other secondary schools, this option may in the future also be discontinued as more and more students are served well in their zoned schools. Fiscal Impact This proposed change will not cost the District anything. Instead, it should continue to provide opportunities to save money formerly required for transportation. Recommendation That the Board of Education approve on first reading the proposed amendments (deletions of language) to Policy IHBEA: English as a Second Language. BAL/adgLITTLE ROCK SCHOOL DISTRICT NEPN CODE: IHBEA ENGLISH AS A SECOND LANGUAGE The Board of Education is committed to providing appropriate and equitable programs and services in compliance with federal and state mandates and which are designed to ensure that students who are limited-English-proficient (LEP) achieve the curriculum content standards and benchmarks established by the State of Arkansas and the Little Rock School District. The Districts program will be designed to address the affective, linguistic, and cognitive needs of the limited-English proficient (LEP) students. A research-based English-as-a-Second-Language (ESL) program will be provided in designated Newcomer Centers at the elementary, middle, and high school levels for all identified students, regardless of the students grade levels and home language, except where parents have denied placement in the program. The District will meet its obligations to provide language services to students whose parents have denied services and placement in a Nowcomer Center through teachers professional development in ESL methodologies, tutoring, summer school opportunities, parental involvement, and monitoring of students academic progress. The District will ensure the provision of appropriate ESL curriculum standards and benchmarks, professional development, technical assistance, parent involvement, staffing, materials, access to special education and other special opportunity programs, qualified staff, and othqr resources to ensure compliance and effectiveness. The District will annually conduct an evaluation of the ESL programs and services, report the findings to the Board of Education, and make appropriate program modifications based on the evaluation. Revised: Adopted: November 18,1999 Legal Reference: Title VI of the Civil Rights Act of 1964, 42 U. S. C. 2000d seq., and sections 100.3(a), (b) of its implementing regulation, at 34 C.F.R. Part 100.0b/iy/2002 08: 48 501-324-2023 LRSD COMMUNICATIONS RAISE 01/05 810 West Markham Is Little Rock, AR 72201 Phone: (501) 447-1027 \" (501)447-1161 Fax: DATE: June 19, 2002 TO: FROM: SUBJECT: Central Arkansas Media Cynthia Howell, Arkansas Democrat-Gazette JuHe Davis, Communications Specialist ESL Graduate Academy Continues through June 28 Forty-one teachers from the Little Rock School District have joined forces with teachers from other central Arkansas school districts at the two-week ESL (English As a Second Language) Graduate Academy, held at the Oasis Renewal Center, 14913 Cooper Orbit Road. Technology Day, held on June 27, will take place at three locations: Williams Magnet Elementary (7301 Evergreen St.), Southwest Middle School (3301 S. Bryant St.) and the Instructional Resource Center (3001 S. Pulaski St.). The academy is sponsored by the Little Rock School District, the Arkansas Department of Education and Arkansas Tech University. The Little Rock School District is home to students who speak 42 foreign languages. Spanish-speaking children comprise the largest segment of this population. A schedule of activities and a list of presenters are attached. ###06/19/2002 08:49 501-324-2023 LRSD COMMUNICATIONS PAGE 02/05 ESL GKADUATE ACADEMY VI Week One at a Glance June 17 - 22,2002 Little Rock, Arkansas Time Monday 6/17 Tuesday 6a8 Wednesday 6/19 Thursday 6/20 Friday 6/21 Saturday 6/22 9:00 Welcome Addresses 10:00 11:00 Limguage Acquisition/ ESL . Medtodology JOr-Carmen Sarudtez-Sadek Life of an Immigrant Child Dr. Urisula Chandler Teaching with Arkansas Frameworks MaTgaret Zoller Language  Acquisition (Continued) -\nL--\nTauent-\u0026amp; .  Commiini^ \"'ReJationslups Al Lopez Culture (Continued) 12:00 1:00 2:00 3:00 ' Dmguage Aeqaieitiotd- ESL Methodology . (Continued) Language Acquisition/ ESL Methodology (Continued) Lanj^a^ Acquisition Dr. tlena Iztftiierdp L^guage Acquiritioh (Cdntiiiuiid)'  Culture , . Dr. Afa^ . Bmitei Culture (Continued ~ ~ L U N C H 4:00 5:00 --DINl^ER-- 6:00 7:00 I^guage Acquisitioii/ f'jSL Methodology (Cqnrinuedl Language '/Acquisition/ ,, I^thodolo^ (Continued) Laiigua^ , Acquisition (Continued). .Language Acqulsitiqiv . Vitali ' }- 8:00 --DISMISSAL-- fKv. 't ESL Graduate Academy VI Coordinators: Little Rock School OiatricL Karen Broadnax Arkansas Tech University# Dr- Ursula Chandler 06/13/2002 08:43 501-324-2023 LRSD COMMUNICATIONS PAGE 03/05 Time 9:00 10:00 11:00 12:00 1:00 2:00 3:00 4:00 5-.00 6z00 7:00 8:00 Sunday 6/23 Group Projects Project^ Showcase Asiui Odtnie Mai te Ngttyeii ESL GRADUATE ACADEMY VI Week Two at a Glance June 23 - 28,2002 Little Rock, Arkansas Monday 6724 Tuesday 6/25 Wedaesday 6/26 Thursday 6/27 Triday 6/28 ESL Standards Workshop Dehhtr Sabo \u0026amp; Lisa Poteet ADE  Ron Tolson Methodology Dr. Prank Gonzales (Ltem.) \u0026amp; Diana Conzalez Worthen (Secondary) Language Assessment Dr. Carmen Sanchez-Sadei Technology. Dr. Yvonne Watts, .. -CItzistote z  B^k \u0026amp; Grace Kerr /tssesemeiA Karen Broadnax '--LUNCH Office of Civil Rights Recjuirements Mark Vas/fuez ESL Strategies Karen Broadnax AXKTESOt Resouice Display Methodology \u0026amp; Children's Literature (Continued) Language Assessment (Condhued) Technolo^ (Cp'ntUaed) Gtaduation , --DINNER-- Assessmmt Karen Broadnax . .. Hispanic Cultiue S tella Lt^a i ^^DISMISSAL-- ESI Graduate Academy VI Coordinators: Little Kock School DistricL Karen Broadnax Arkansas Tech University, Dr. Ursula Chandler 06/19/2002 08:49 501-324-2023 LRSD COMMUNICATIONS PAGE 04/05 Little Rock School District ESL Graduate Academy Oasis Renewal Center Little Rock, AR June 17-28, 2002 Presenters and Topics 1. Dr. Mario Benitez, Chair (Emeritus), Curriculum and Instruction, University of Texas at Austin Topic: Socio-Cultural Influences of Language on Learning 2. Dr. Carmen Sanchez-Sadek, Educational Consultant/Program Evaluator, Los Angeles, California Topic- Second Language Acquisition, ESL Methodology and Assessment 3. Dr. Ursula Chandler, Department Head, Foreign Languages and International Studies. Arkansas Tech University, Russellville, AR Topic: The Life of an Immigrant child 4. Dr. Elena Izquierdo, College of Education. University of Texas at El Paso Topic: Second Language Acquisition 5. Mark Vasquez, Esq., Consultant/Civil Rights Attorney, Dallas, Texas Topic: Civil Rights Requirements for Language Minority Students 6. Dr. Frank Gonzales Topic: Methodology/Childrens Literature (Elementary) 7. Christine Black, ESL Services/Gifted Education, Lake Hamilton School District Grace Kerr, Media Specialist, The New School. Fayetteville, AR Dr. Yvonne Waits, Consultant/ADE, Little Rock, Arkansas Topic: Utilizing technology for ESL/ Core Content Instruction with LEP students 8. Dr. Diana Worthen Topic: Methodology/Childrens Literature (Secondary) The following presenters will provide special presentations^ Arkansas Department of Education Ron Tolson, Director Professional Licensure Topic: The ESL Endorsement Application Process Al Papa Rap Lopez, Springdale Public Schools/N.W. Arkansas Multicultural Youth Clubs Topic: Parent and Community Relationships06/19/2002 08:49 501-324-2023 LRSD COMMUNICATIONS PAGE 05/05 ESL Graduate Academy Oasis Renewal Center Little Rock, AR June 17-28, 2002 The following presenters will provide special presentations: Fort Smith Public Schools Mai Le Nguyen Topic: Asian Culture Little Rock School District Debbie Sabo and Lisa Poteet Topic: ESL Standards Workshop Little Rock School District Karen Broadnax, ESL Supervisor Topic: The Language Assessment Scales/An Introduction Little Rock School District Stella Loya, ESL Parent Liaison Topic: Hispanic Culture0 f/ov. LITTLE ROCK SCHOOL DISTRICT INSTRUCTIONAL RESOURCE CENTER 3001 PULASKI STREET LITTLE ROCK, AR 72206 TO: Board of Education FROM: T. Kenneth James, Superintendent of Schools PREPARED BY: ^^Sonnie A. Lesley, Associate Superintendent for Instruction DATE: SUBJECT: November 21,2002 Approval of the ESL Program Evaluations for 1999-2000 and 2000-2001 Background The LRSDs Compliance Plan for completion of the tasks relating to page 148 of the Final Compliance Report (Section 2.7.1) requires the Boards approval of already completed program evaluations that were conducted with the assistance of external experts. Two of those completed program evaluations are those for the English-as-a-Second Language program for 1999-2000 and 2000-01, which the Board has previously reviewed. The external expert who participated on those evaluation teams was Mr. Mark Vasquez, a former employee of the Office for Civil Rights in Dallas and an expert on OCRs expectations for the evaluation of alternative language programs. Those participating in the design and writing of the 1999-2000 evaluation included Dr. Ed Williams, Karen Broadnax, Ken Savage, and Mr. Vasquez. The Board of Education initially reviewed this program evaluation in fall 2000. Those participating in the design and writing of the 2000-2001 evaluation included Dr. Eddie McCoy, Dr. Ed Williams, Karen Broadnax, Ken Savage, Dr. Bonnie Lesley, and Mr. Vasquez. The Board of Education initially reviewed this program evaluation in fall 2001. Neither of these program evaluations included data related to the improvement or remediation of student achievement for African American students. Rather, the data were disaggregated for limited-English proficient students and fluent-English proficient students who had exited the LRSDs alternative language program. These data were then compared to the data for the general population. The research questions, the data to be collected, and the disaggregations were Board of Education - Memo November 21,2002 Page Two specified, for the most part, in the Commitment to Resolve, the District's voluntary agreement with OCR to put into place the necessary programs, policies, and procedures to be in full compliance with Title VI of the Civil Rights Act. Fiscal Impact None Recommendation That the Board of Education formally approve the following: 1. 1999-2000 program evaluation for LRSDs alternative language program 2. 2000-2001 program evaluation for LRSDs alternative language program \u0026amp; i BAL/adg 3 Attachment 1?ESL Standards for Pre-K-12 Students *1 Teachers of English to Speakers of Other Languages, Inc.die ess .on .tor 197 Introduction: Promising Futures ESL Standards for Pre-K-12 Students All elementary and secondary school students currently in the United States will be Ihing in and contributing to an increasingly diverse society and interdependent community' of nanons in the 21st century. To realize their personal, social, and long-term career goals, indincuals will need to be able to communicate with others skillfully, appropriately, and effectively. The challenge of contemporary education is to prepare all students for life in this new world, including those learners who enter schools with a lang'uage other than English. The purpose of this document is to identify the ESL standards and their role in meeting this challenge. Why ESL Standards Are Needed Schools and communides throughout the United States are facing increased linguistic and cultural diversity. Ever}' year, more and more studencs who speak languages ocher than Er h r from homes and communities with diverse histones, traditions, world news, and experiences, populate classrooms in urban, suburban, and rural settings. Th num: school-age children and youth who speak languages other than English at home increas 68.6% in the past 10 years. By 1993 English lang'uage learners in U.S. public schools m more than 2.5 million. Current projections estimate that by the year 2000 the majonty school-age population in 50 or more major U.S. cities will be from language minor.?.' b: grounds. I I 1 fn this document use [WO acronyms. E5L and ESOL. ES refers io [hefieid oj ing'.ish as a se: 0 com.e tonal\nen oy imbered of the .1 fSOL students vary greatly in proficiency level and academic needs. Some ESOL students are recent immigrants, brought to the United States by families seeking refuge from political repression or persecution or by families seeking economic opporrunim Others are members of ethnolinguisuc groups that have lived on this continent for generations, some for longer than the United States has e.xisted as a nation. Some'have had prior education, including literacy, in their native languages. Others hav'e had limited formal schooling. Some have had normal dev'elopmental histories, while others have identified disabilities that challenge their learning. Our'  primary concerns in this document are with students in elementary and secondary' schools who are not native speakers of English, and whom we refer to as ESOL students and learners. ESL Standards describe the language skills necessary for social and academic purposes. The standards described in this document specify the language competencies ESOL students tn elementary and secondary schools need to become fully proficient in English, to have unre- ESL Standards for Pre-K-12 Students I I I i i language ana te cht scandards chemsecves. EiCL ( En^iish io sve-iic' oy ocher languagesj refers co che ieamer u ho are idencifiea a sccll in the process o acquiring Engiish a an additional language. 1 stricted access to grade-appropriate instruction in challenging academic subjects, and ultimately to lead nch and productive lives. The development of these standards has been informed by the work of other national standards groups, particularly by the English language arts and foreign language standards. All three language standards projects share an emphasis on the importance of:  language as communication  language learning through meaningful and significant use  the indixddual and societal value of bi- and multilingualism 8' th  the role of ESOL students native languages in their English language and general acade- -SU mic development o' o' cultural, social, and cognitive processes in language and academic development * assessment that respects language and cultural diversity let th The ESL Standards provide the bridge to general education standards expected of all Ths ESL standards articulate the developmental Endish language needs of ESOL learners and highlight special instructional and assessment considerations that must be given to ESOL learners if they are to benefit from and achieve the high standards proposed for other subjects. 2 dents in the United States. stu- Standods do not ano cannot stand alone. Other professional have developed standards that organizations and groups Of useful. Thes\nincluding E and strategic are world-class, imponant. developmentally appropriate, and standards mandate high levels of achievement in content learning for ail learners. :Oi- students. But the content standards do not orovide educators the directions\ns mey need to assist ESOL leaimers to attain these standards because t student understanding of and ability to use English to engase with content. Many tent stanna\nO' do not acknowledge the central role of language in the achievement of iSor do they highlight the learning styles and particular instructional and assessme learners who are still developing proficiency in English. In sum. the content stand: 14 Er bii ac e\\ assume f the Con: content, needs of s do not in addiess the specific needs of ESOL students who are adding znglish to their home !an\u0026lt;maoes. Therefore. ESL standards are needed. o O or mt The uSL standards recognize that upon entry to scnool ESOL learners must acquire an addi- tional language and culture and learn the English language com.petencies that are characteristic of native Englisn speakers of the same age and, most importantly, that are fundamental to the full attainment of English language arts and other content standards. The ESL standards articu- late the developmental English language needs of ESOL learners and highlight special mstruc- tional and assessment considerations that must be given-to. ESOL learners if they are to benefit from and achieve the high standards proposed for other subjects. Thus, the ESL Standards important because they:  aniculate the English language development needs of E5OL learners  pro\\-ide directions to educators on how to meet the needs of ESOL learners  emphasize the central role of language in the attainment of other standards are Introduction: Promising Futures if. T fo\ndu In ESely he 1 ce Myths About Second Language Learning Several myths regarding second language learning prevail both among many lay persons and some educational professionals and policy makers. One intent of this document is to refute these myths. Myth 1\nESOL students learn English easily and quickly simply by being exposed to and surrounded by native English speakers. Fact: Learning a second language takes time and significant intellectual effort on the part of the learner. Learning a second language is hard work\neven the youngest learners do not simply pick up\" the language. Myth 2: When ESOL learners are able to converse comfortably in English, they have developed proficiency in the language. Face It can take 6-9 years for ESOL students to achie^/e the same le'.'cls of proficiency in academic English as native speakers. Moreover, ESOL students participating in thoughtfiilly designed programs of bilingual or sheltered content instruction remain in school longer and attain signifii academic achie'/ement in comparison to students without such advantages. I Myth 3: In earlier times immigrant children learned English rapidly and into American life. '.dy higher ra[es of niiated .1- I i I Fact: Many immigrant students during the early part of this century did not learn English auicklv or well. Many dropped out of school to work in Jobs that did not require the kinds of academic achie.'e-ment and communication skills chat substantive employment oppotiunities reauire :odav. 1- I i TESOLs Vision of Effective Education for All Students The role of ESL standards can only be fully understood in the broader context of education for ESOL students. Therefore, before presenting the ESL Standards, it is important to desenbe our overarching vision of effecave education. In TESOLs vision:  Effective education for ESOL students includes nativelike levels of proficient?/ in English.  Effective education for ESOL students includes the maintenance and promotion of ESOL students' native languages in school and community contexts. \u0026gt; All educational personnel assume responsibility for the educadon of ESOL students. - Effective education also calls for comprehensive provision of first-rate services and full access to those services by all students \u0026gt; Knowledge of more than one language and culture is advantageous for all students. !S ESL Standards for Pre-K-12 Students 3 Effective education for ESOL students includes nativelike levels of proficiency in English. For ESOL students to be successful in school and ultimately in the world outside school. they must be able to use English to accomplish their academic, personal, and social goals with the same proficiency as native speakers of English. In school environments, ESOL students need to be able to use spoken and written English both to acquire academic content and to demonstrate their learning. ESOL learners also need to be able to follow routine classroom instructions given o' in English and understand and use appropriate communication patterns so that they can be successful learners in academic environments. Finally, ESOL learners need to use English to function effectively in social settings outside the school, as well as in academic settmgs. The ESL standards in this document are concerned with these types of social and acad- emic skills. Moreover, appropriate performance and assessment standards that distinguish between language and academic achievement are also required if ESOL students are* to be given full credit for learning academic content while acquiring English. o' Effective education for ESOL students includes the maintenance and promotion of ESOL students native languages in school and community contexts. The auHinmeni. oi ihallenging, wo.'-ld- ciass eaucatioria\nBy definition. ESOL learners already know and use another language. Both the academic Siam SLUG .cards by ui! ier.LS is oniv j achievement and the school completion of ESOL learners is significantly enrianced when they are able to use their native languages to leam in school. In fact, full proficiency in the native language (including literacy) facilitates second language development. Developing and using COSSfOL scnoos i design their i I ESOL students guistic and cul\nBilingualism is o native languages also sen'es U.S. national interests because it increases the lin- rral resources available as the United States competes in the global economy. UI dsset whose \\aiue for the indhidual and for societv can onlv increase as the a, Kmc Sim Sdiv |at} id me U Ian ^Th S sp\u0026lt; i she cot eaucanonai | :ons v.-.d- EiCi.\nL .S. role in the global market place expands in the next centurv. siuden Olh: \u0026lt; as well ss i All educational personnel assume responsibility for the education of ESOL students. Ai in mi: ' I I I i I The attainment of challenging, worid-class educational standards by al! students is only possible it scnools design their educational missions with ESOL students, as well as others, in mind, u.omprehen.si'. e education calls tor shared responsibility by and collaboration among all educational protessionals wort-ting wuth ESOL students. It also calls for professionals to expand their knowledge to encompass issues of relevance to the education of ESOL students. This e.xpanded knowledge base includes an understanding of similarities and differences in first and second language acquisition, the role ot the native language in second language and content learning, instructional methods and strategies that facilitate both English language and content learning, instructional practices that accommodate individual differences in learning styles, the interrelationships between culture, cogniuon and academic achievement, alternative approaches to assessment, and the imponance of community-school linkages in education. These are all  for tx'scf f?- tin \u0026amp; ^wh det ^coi aca 4 I 1 I I part of the professional development of ESL specialists that general educators must tap into if educational reform is to result in the attainment of high standards by all students. Effective education also calls for comprehensive provision of first-rate services and full access to those services hy all students. Quality educational experiences and ser\\nces must be made fully accessible to all ESOL students. These include, among others, comprehensive and challenging curricula, access to the full range of curricula (e .g., gifted classes, laboratory sciences, college preparatory courses), safe and Introduction: Promising Futures we us\u0026lt; off\nSr' wc ?- ^0 ESi a:h 1 ) id- 2n well-equipped classrooms, appropriate instructional practices and assessment measures, inclusion in extracurricular activities, fully and appropriately certified teachers, and other educational specialists and resources. However, this is often not the case in most schools. To have quality programs and to serve ESOL students appropriately on their way to mastery of English, instruction must take into account the different entry-level abilities in English that ESOL learners have. Some learners come to school with oral and written skills\nothers do not. In addition, where necessary, programs should provide some instruction in the native languages of ESOL students. TESOLs Access Brochure provides a description of the conditions needed to provide ESOL students with equitable opportunities to leam. (See Appendix A.) Knowledge of more than one language and culture is advantageous for all students. Internationalism is the hallmark of modem U.S. education and of the education reform movement, and linguistic and cultural diversity are the hallmarks of internationalism. The challenge of contemporary education is to contribute to students abilities to live in increasingly diverse local communities and an ever-shrinking world community. Effective education for the 21st century' must provide firsthand opportunities for students to leam about the cultural diversity around them and to leam world languages. Cross-cultural competence can be fostered bv meaningful and long-term interactions with others with different world views, life e.xperiences. languages, and cultures. Language learning can be fostered by interactions with native speakers. This means that, not only should ESOL students leam about the U.S. from native-Envli^h speakers, but native-English-speaking students, teachers, administrators, and school staff should leam about the world and its languages from ESOL students, their families, and their communities. IfESOLsiudii iO have full aci challenging i cumeuia and: i achieve :o che I high level content a: native Ens Audience \u0026gt;5- 1 This document is written for educators who work with ESOL learners. First, it is intended for educators who work directly with ESOL students at elementary\nmiddle, and secondarv school levels. This includes designated ESL teachers (whether in resource or self-contained set- tings), bilingual teachers who work with ESOL students in their native languages and English, 'O 4 t e .es and teachers who work with ESOL students with special needs and talents. Other educators who will use these standards are content area teachers who teach ESOL students. If ESOL students are to have full access to challenging curricula and to achieve to the same high level in the content are:^ as native English speakers, then content area specialists must become aware of the importance of language in relationship to their disciplines, so that they' can better facilitate the ^^demic achievement of their ESOL students. Curriculum developers and program coordinators are likely to refer to thus document as The standards and descriptors will be helpful for developers of ESL curricula and may be ^rj^^ out learning objectives. The sample progress indicators, vignettes, and discussions ^yoffer ideas for learning activities, assessment and program design. 111 id speakers, \"ler. content area specialists rat-t become awareate imponar.ee of language in relationship to ateir disciplines sc that they can better facilitate the academic achievement of their ESOL students. document may be used as a reference for educators such as counselors, school social ^ud psychologists who provide additional service to ESOL students and for profes- ^onals whose activities and decisions affect programs for ESOL students, that is, building es i KL Standards for Pre-K-12 Students 5i i ! i I 6 administrators, preservice and in-service teacher educators, and local, state and national policy makers. Parents and communities with ESOL learners may also wish to consult this document so that they may better understand what constitutes appropriate and effective education for their children. General Principles of Language Acquisition . A number of general principles derived from current research and theory about the nature of language, language learning, human development, and pedagogy, underlie the ESL standards described in this document. These principles are described briefly here.  Language is functional\nO o  Language varies. o  Language learning is cultural learning.  Language acquisition is a long-term process.  Language acquisition occurs through meaningful use and interaction.  Language processes develop interdependently.  Native language proficiency contributes to second language acquisition. ,x Etc ca idf t f-  Bilingualism is an indixidual and societal asset. Language is junctional. Language, oral and written, is primarily a means of communication used by people in multiple and varied social contexts to express themselves, interact with others, learn about the world, and meet their indi\\-idual and collective needs. Successful language learning and language teaching emphasize the goal of functional proficiency. This is a departure from traditional pedagogical approaches that new language learning and teaching primarily as mastety of the elements of language, such as grammar and vocabulary, without reference to their functional usefulness. Therefore, what is most important for ESOL learners is to function effectively in English and through English while learning challenging academic content. Language varies. Language, oral and written, is not monolithic\nit comes in different varieties. Language varies according to person, topic, purpose, and situation. Everyone is proficient in more than one of these social varieties of their native language. Language also varies with respect to regional, social class, and ethnic group differences. Such language varieties are characterized by distinctive structural and functional characteristics, and they constitute legitimate and functional systems of communication within their respective sociocultural niches. Additionally, language varies from one academic domain to anotherthe language of mathematics is different from the language of social studies. As competent language users, ESOL students already use their own language varieties. They must also learn the oral and written language varieties used in schools and in the community in large. What is most important for ESOL learners is to function effectively in academic environments, while retaining their own native language varieties. Introduction: Promising Futures Bl S ar be le\nac ac \u0026gt; er F.i/ Li e L er ^ac ^le Seb L-f' to ^in g'lb ^al feicy .nt -e of s Language learning is cultural learning. Patterns of language usage vary across cultures and reflect differences in values, norms, and beliefs about social roles and relationships in each culture. 'When children learn their first language, they learn the cultural values, norms, and beliefs that are characteristic of their cultures. To learn another language is to learn new norms, behaviors and beliefs that are appropriate in the new culture, and thus to extend one's sociocultural competence to new environments. To add a new language, therefore, is to add a new culture. Learning a new language and culture also provides insights into ones own language and culture. This is imponant for ESOL students because general education in U.S. schools tends to reflect a culture other than their own. If ESOL students are to attain the same high standards as native-English-spealdng students, educational programs must be based on acknowledgment of, understanding of, respect for, and valuing of diverse cultural backgrounds. What is imponant for all language learners is to develop attitudes of additive bilingualism and biculturalism. Language acquisition is a long-term process. iki- rld. dae- j fo ties .f 5- Language acquisition occurs over time with learners moving through developmental stages and gradually growing in prohciency. Individual learners however move through these stages at variable rates. Rates of acquisition are influenced by multiple factors including an indhiduals educational background, first language background, learning style, cognitive style, motivation, and personality. In addition, sociocultural factors, such as the influence of the English or native language communir. in the learners life, may play a role in acquisition. In many instances. learners pick up\" conversation skills related to social language more quickly than th acquire academic language skills. Educational programs must recognize the length of time it takes to acquire the English language skills necessary for success in school. This means that ESOL !eam- ers must be given the time it takes to attain full academic proficiency in English, often fro.m 5 to 1 years. Language acquisition occurs through meaningful use and interaction. Research in first and second language acquisition indicates that language is learned most effectively when it is used in significant and meaningful situations as learners interact with others (some of whom should be more proficient than the learners are) to accomplish their purposes. Language acquisition takes place as learners engage in activities of a social nature with opportunities to practice language forms for a variety of communicative purposes. Language acquisition also takes place during actixnties that are of a cognitive or intellectual nature where learners have oppormnities to become skilled in using language for reasoning and mastery of S- challenging new information. This means that ESOL learners must have multiple opportunities to Efigiish, to interact with others as they study meaningful and intellectually challengng content, and receive feedback on their language use. ^Language processes develop interdependently. Learner K' conversation\nrelated to sect language more 5- quickly man: acquire acace: O' / Educui ?ro.^ra: recoi^r.izs 'O of time it acquire tf language le le :e: necessar.' ior 5UCCSS5 in sc\nICCl. I lan^. Traditional distinctions among the processes of reading, listening, writing, and speaking are ^^cial. So is the conceptualization that language acquisition as linear (with listening preced- ion jres peaking, and speaking preceding reading, and so forth). Authentic language often entails simultaneous use of different language modalities, and acquisition of functional language ^^lities occurs simultaneously and interdependently, rather than sequentially. Thus, for exam- 'depending on the age of the learner, reading activities may activate the development of Standards for Pre-K-12 Students 71 -I 8 speaking abilities, or vice versa. Additionally, listening, speaking, reading, and writing develop as learners engage with and through different modes and technologies, such as computers. music, film, and video. This means that ESOL learners need learning environments that provide demonstrations of the interdependence of listening. e\u0026gt; speaking, reading, and writing. They also need to develop all of their language abilities through the use of varied modes and technologies. Native language proficiency contributes to second language acquisition. Because, by definition, ESOL students know and use at least one other language, they have acquired an mtuitive understanding of the general structural and functional characteristics of language. They bring this knowledge to the task of second language learning..students also come to the task of learning English and learning content through. English already lit- erate in their native languages. These learners know what a Some E5OL stu- it means to be literatethey know that they can use written forms of language to leam more about the world, to convey information and receive information from others, to establish and maintain relationships with others, and to e.xplore the perspectives of others. Literacy in the native language correlates positively with the acquisition of literacy' in a second language. In addition, academic A fl i - ! 1 I I i i j I i I I 1 I I I includes the use of E5OL students instraction that native languages, especially if they are literate in that Ian- guage. promotes learners' academic achievement while they are acquiring the English needed to benefit fully from instruction through English. Native language literacy abilities cin a_____ students in English-medium classrooms to construct meaning from academic materials and o ESOL expenences in English. And. in learning a new language, students also leam more about their natii e tongae-. This means that for aSOL leatmers the most effective teaching and learning are those th romote ESOL students' native langua\u0026lt; environments for secona language as a foundation jor English language and academic dc'.elopment. Bilingualism is an individual and societal asset. o' '.ge and literacy development .Acquisition of two languages simultaneously is a common and normal developmental ph? nomenon and that acquisition of a second (or third) language can confer certain cognitive and linguistic advantages on the indindual. To realize these benefits, however, advanced levels of pne- proficiency in both languages are necessary. Therefore, the most effective educational environ- ments for ESOL learners are those that promote the continued development of learners' primary' languages for both academic and social purposes. In addition, as noted earlier, bilingual proficiency enhances employment possibilities in the international marketplace and enhances the competitive strength of U.S. industry' and business worldwide. This means that bilingualism benefits the individual and serves the national development of multiple languages. Goals for ESOL Learners o' interest, and schools need to promote the retention and TESOL has established three broad goals for ESOL learners at all age levels, goals that include personal, social, and acadermc uses of English. Each goal is associated wth three dis- tinct standards. In TESOLs vision, ESOL learners will meet these standards as a result of the instruction they receive, thereby achieving the goals. Our schools need to ensure that all stu- Introduction\nPromising Futuresop dents achieve the English language competence needed for academic success and for life in a literate culture. 0 Goal 1: To use English to communicate in social settings\nve if lU- y lit- V .la- s, V SB' A primary goal of ESL instruction is to assist students in communicating effectively in English, both in and out of school. Such communication is vital if ESOL learners are to avoid ' the negative social and economic consequences of low proficiency in English and are to panici-  pate as informed participants in our democracy. ESOL learners also need to see that ere are j personal rewards to be gained from communicating effectively in English. This goal does not S'suggest, however, that students should lose their native language proficiency. Standards for Goal 1 Students will\n:d to SOL\n.x gi- 1. 2. 3. use English to participate in social interaction interact in, through, and with spoken and written English for personal expression and enjoyment use learning strategies to extend their communicative competence\nir nags r.snt  Goal 2: To use English to achieve academically in ail content areas In school settings, English competence is critical for success and expectations fcr -SOL I learners are high. They are expected to leam academic content through the English .anguase I lend m- T'and to compete academically with native-English-speaking ^/kamers use spoken and written Engiish in their schooiwor ig^ndards for Goal 2 -Students will\neers. This process requires that\nal :es im nd use English to interact in the classroom use English to obtain, process, construct, and proxide subject matter information in spoken and written form use appropriate learning strategies to construct and apply academic knowledge s- 1- in socially and culturally appropriate ways ^tt^udents in U.S. schools come into contact with peers and adults who are different ^^rlmguistically and culturally. The diversity in U.S. schools mirrors the diversity in g^^y and around the world that young people will encounter as they move into the 21st of work. In order to work and live amid diversity, students need to be able to appreciate people who are different and communicate effectively with them. ^^^gmcation includes the ability to interact in multiple social settings. itures for Pre-K-12 Students 9 aStandards for Goal 3 Students will: 1. 2. use the appropriate language variety, register, and genre according to audience, purpose, and setting use nonverbal communication appropriate to audience, purpose, and setting 1. J. use appropriate learning strategies to extend their sociolinguistic and sociocultural competence Conclusion 11 The ESL standards dsscnoe the proficiencies in Enaiish that ESOL students need to acquire sc they can Full proficiency in English is critical for the long-term personal, social, and economic development .of all students in the United States. In this document, TESOL outlines a framework for considering and planning language education for ESOL students and for interpreting and making use of the ESL standards. The ESL standards describe the proficiencies in English that E50L students need to acquire so.they can attain the same high level standards in other content domains, including English language arts, as fully proficient English-speaking students. Thus, the ESL Standards for Pre-K-12 Siudenis is the starting point for developing effective and equitable education for ESOL students. Planning effective English language instruction for ESOL students cannot be don isolation. It must be part of a comprehensive and challenging educational program that takes into a same hish itandaros in 0! lonisnt domains, including English language arts, as .fully proficient English- speaking students.  10 account OL students social, educational, and personal backgrounds as well as the: existing skills and knowledge bases. It must understand and respond appropriatelv to the interrelation- ships between language, academic, and sociocultural developm^ent. The linguistic, cosnitive. and sociocultural competencies that ESOL students bring to school are a solid base for buildin: 'g their future, in terms of educational and career success. Only if tSL instruction is part of a comprehensive, challenging, and enriching educational program, however, will the promising futures of ESOL learners be realized. Introduction: Promising Futures 31ie Hie ^Haku! ?K\nHo\u0026amp;r ^^Krash lihdfc II cLat rCac ilii I* Sta s m- .'el-for ik- OL References for Further Reading Bilingualism and Second Language Acquisiti(^n \u0026lt; Ai d: August, D., \u0026amp; Hakuta, K. (Ed.). (1997). Improving schooling^^rl(ingiiage-mindri^,cifyi\u0026gt;^, A' research agenda. Washington, DC\nNational Academy K^sS' 'V Bialystok, E., \u0026amp; Hakuta, K. (1994). In ocher words: The scienc'^^nd:psvchplogy oj secori^^ldtx^iage acijuisicion. New York: Basic Books. A'-.. . d- Collier, V (1987). Age and rate of acquisitio'^.bt^econd langua^)fe?.\nach\n^^ic purposes. ' TESOL Quarterly, 21 (3), 617-641.' 'f fi ' ' ' ' \"(S'' % Collier, V (1989). How long? A synthesis of research on academ.ic achievement in a second language. TE5OL Quarterly, 23(2), 509-532' W- p Hak-uta, K. (1986). mVirror o/language\n1 he dehafil^n'New York^^^sic Books. Hoffman, C. (1991). Introduction to bilingualism. NetV.-YSfk: Longmian. Krashen, S. (1982). Princi_p!es of first and second lang'.iage^cduisicion. O.xford: P. iamon. Lindfors, J. (1987). Children's language and leatming. (2nd ed\u0026lt;)\n^ Englewood Cliffs. NJ: Prentl Hall. .ulus McLaughlin, B. (1984). Second language acquisition in children. Volume 1: Preschool children. Hillsdale, NJ: Lawrence Erlbaum. McLaughlin, B. (1985). Second language acquisition in children. Volume 2\nSchool-age children. Hillsdale, NJ: Lawrence Erlbaum. McLaughlin, B. (1992). Myths and misconceptions about second language learning:,.What teacher needs to know. Educational Practice Report No. 5. Santa Cruz, CA and'^asl^.'gt^n^ DC: Nauqnal Center for Research on Cultural Diversity and Second Language t^rhins. a\nPease-Alvarec, L, \u0026amp; Hakuta, K. (1992). Enriching our views of bilingualism and bilin'^i^i^U' cation. Educational Researcher, 21(2), 4-6. Bilingual Education: Importance of Native Language Collier, V (1992). A synthesis of studies'e-xamining long-term language-minority student data on academic achievement. Bilingual Research Journal, 16(1/2), 187-212. ESL Standards for Pre-K-12 Students Si-f* ! 1 II I si-.- \\ hi, v\u0026lt;- %%. '.v 11 V Exhibit No. 66: LI230-90 Program Evaluation for English as a Second Language.Section L. Program Evaluation Program Goals The Little Rock School district shall provide a research-based English-as-a- Second Language (ESL) program for its students who are identified limited-English proficient (LEP). The first goal of the ESL program is to enable students to master English language skills (reading, writing, speaking, and listening), content area concepts, and skills so that the students are able to participate effectively in the regular program as quickly as possible. The second goal of the ESL program is to provide identified students with the cultural literacy necessary for them to feel comfortable in participating in the school, community, and greater community. Program Objectives ESL program objectives are as follows: 1. 2. J. 4. 5, 6. 1. 8. 9. 10. 11. 12. To implement and maintain consistent procedures for student identification processes. To assess all students who have a primary language other than English (PHLOTE), regardless of whether they are enrolled in a Newcomer Center. To establish and administer consistently appropriate criteria for entry and placement into an ESL program. To diagnose student needs and provide appropriate ESL standards/benchmarks, instruction, and assessments to meet identified students individual needs for English-language instruction, for understandable instruction in other content areas, and for positive self-concept and identification with personal/family cultural heritages. To hire, train, and continually , develop highly motivated, sensitive, and caring ESL teachers and other staff to provide effective ESL instruction, interact one-on-one with the identified stodents and their families, and serve as liaisons between school and relevant community. To provide appropriately aligned instructional materials. To establish and administer consistently appropriate criteria for exit from an ESL program. To provide for parental/family involvement in the school setting to support improved student learning. To provide equitable access to other district programs and services, including special education, gifted/talented education, and all procedural safeguards. To monitor the progress of all identified students during program participation and after program exit and to reclassify students as needed. To evaluate the ESL program and make program modifications as needed. To maintain accurate and useful student records, including procedural safeguards (See Administrative Regulations, IHBEA-R, approved November, 1999). Categories of English Second Language Students An objective of the Little Rock School District (LRSD) is to assess all PHLOTE students regardless of whether they are enrolled in a Newcomer Center. After attempting to assess these students, they are tagged within the Districts database as: (1) LEP - Limited English proficient, (2) NALEP - not assessed, or (3) FEP - Fluent English proficient. As of June 2,2000 the District has identified 805 PHLOTE students (LEP = 467, N-AfcEP = 45, and FEP = 293). -4here-i3-\u0026amp;fte-horTte-schQoled-sludent that has heerLassessed-EEPrbtrrwiltTTorbe included in this~' Supporting charts reflect three data gathering points, 11/30/99, 3/24/00, and 6/2/00, and correspond to the Districts reporting to the Arkansas State Department of Education, Districts Quarterly Report, and the Annual OCR report. The Districts primary database is the AS400 mainframe computer using CIMS, an electronic student database software. Student data are collected at school level. Cunent year data are filed with the students permanent record file (PRF). Baseline data are stored in ESL Department files. 'BSi7pTograirrand'DRtricrstaff1iave-reviewed-studenliiala_aud_mlLcnittinueJu44pdate4he mainframe database. As i^fereRecririrHlhHApdan. District staff are members nCthe Divi.sion of Curriculum andTnstraction. The Divisions-Associate Superintendent is Dr. Bonnie-Lesley^ The initial portion of this section will report demographic data (i.e,, gender, language, school location, grade level distribution, and any sub-groups) on LEP, NALEP, and FEP students. The subsequent sections will report academic and behavioral progress (e.g., attendance) of LEP students and how LEP students compare on academic and behavioral variables to NALEP, LEP, and the Districts general school population. Description of Limited English Proficient (LEP) Students. As of June 2, 2000, there are 467 LEP students in the Little Rock School District (LRSD). Demographics on this population are:  255 male (54.6%) and 212 female (45.4%),  Newcomer Centers serve most of the LEP students, 309 students (66.2%) (see also Figure#!), 3  LRSD has 48 schools, LEP students are being served in all of the High schools. Middle schools, and all but six of the Elementary schools (i.e., Badgett, Dodd, Jefferson, Mitchell, Western Hills, Rightsell, and Woodruff),  Non-Newcomer Centers serve the remaining LEP students (N = 158), with the top seven schools serving between 7 and 20 students (see also Figure 2),  The elementary level (grades Kindergarten through 5*) has the highest number of students per grade level with an average grade level enrollment of 46. Middle school (grades 6* through 8*') average grade level enrollment is 29, and high school (grades 9* through 12'*) per grade level enrollment averaged 19. The pre-kindergarten program (i.e. LRSDs 4-year- old program) had 29 LEP students enrolled, and  Among LEP students Spanish is the most common language spoken (71.1%), followed by Chinese (4.5%), Arabic (3.6%), Assyrian (2.3%), Korean (2.3%), and Vietnamese (1.5%). An additional 29 languages account for the remaining 14.3% of LEP students. One student has no language listed. Within the LEP population are several sub-groups: LEPNC - LEPSO - In an ESL program at a Newcomer Center (N = 309) Being served at a school other than a Newcomer Center (N = 158) LEPNS - Not being served (N = 9) LEPSPD - Receiving special education services (N = 9) LEPGT- Receiving gifted and talented (GT) services (N = 7) LEPREC - Students who have left the program, have been reclassified, and have re-entered the program (N = 0) Additional demographic data on these sub-populations are:  Newcomer Centers serve all of the students receiving gifted and talented services (N = 8),  Five of the Center, ^e^ students receiving special education services (LEPGT) attend a Newcomer  Five of the nine LEP students not receiving services (LEPNS) attend Newcomer Centers, and  Students not rei being monitored (N = 4y lirect services (LEPNS) have either refused services (N = 5) or areNewcomer Center LEP Enrollment Brady Chicot Romine Terry V\\feshington 11/30/99 26 _____68 33 22 38 3/24/00 25 71 30 18 33 6/2/00 ___  74 28 23 30 Cloverdale Middle Dunbar Middle 42 16 44 16 47 13 Hail High Total Languages LMS [ Figure 2 Top 7 Non-Newcomer Centers serving LEP Students Wakefield Cloverdale El McDermott Bale Garland Henderson Middle Forest Heights 11/30/99 21 17 17 10 6 7 11 3/24/00 22 14 18 9 8 8 10 6/2/00 20 14 13 12 8 8 7 Total I 97 I 73  88 Description of Not Assessed Limited English Proficient (NALEP) Students. There are 45 NALEP students identified in the Districts database. Demographics for this population are:  23 Male (51.1%) and 22 Female (48.9%),  Spanish is the most common language (31.1%), followed by Korean (15.5%), Gujarati (8.9%), German (4.4%), Farsi (4.4%), and Chinese (4.4%),  Grades kindergarten through 5* have an average grade level enrollment of 1, grades 6* through 8* have an average enrollment of 2, and grades 9* through 12* have an average enrollment of 8, 5  Most NALEP students, 80% (N = 36), have refused to be assessed, three (6.7%) are special education students and were unable to complete the Language Assessment Survey (LAS), three (6.7%) have not been at school when the LAS was administered, one (2.2%) is auditing, one (2.2%) has not returned the signed letter from a parent, and one (2.2%) has been assessed but the score report has not been received by the ESL office, and  Only 6.7% (N - 3) attend a Newcomer Center. Parkview High School has the most NALEP students, 44.4% (N = 20). Description of Fluent English Proficient (FEP) Students. There are 299 FEP in the Little Rock School District. Demographics for this population are:  147 male (50.2%) and 146 female (49.8%),  Spanish is the most common language (32.2%), followed by Chinese (11.2%), Vietnamese (6.4%), Urdu (4.1%), Korean (3.1%), Arabic (3.1%), and Russian (2.7%). While 34 remaining languages constitute an additional 31.4%, 5.8% (N = 17) have no language listed,  One hundred and eighteen (N = 118, 31.8%) of the students attend a Newcomer Center, with Chicot Elementary having 75 students, Cloverdale Middle having 46 students, and Hall High having 72 students.  For grades Kindergarten through 5 the average grade level distribution is 26, grades 6* through 8the average grade level distribution is 27, and grades 9 through 12^'the average grade level distribution is 13. There are nine FEP students in the Districts 4-year old program. Evaluation Plan Across four components listed below data will be collected, analyzed, and used to make appropriate, research-based, modifications to the program. The process of this evaluation is to report not only the basic program, demographic, and achievement data, but to used the data and data analysis to investigate and develop new and innovative instructional strategies. 1.(2,  Discussion of variables. All LEP academic and other data (e.g/lattendance) will be compared to the general population. The general population for this report consists of all students in the Little Rock School District except those students classified as LEP and students receiving special education services. The District will not compile the graduation, attendance, or retention rates for the 1999-2000 school year until August of 2000y These data will not be reported. LAS scores have only been entered into the Districts debase for those students who have become fluent after October 1999. Since most of the LEP, r and Pep students were not tagged in the Districts database 1999-2000 school ye^graduation data for 1998-1999 is not available. Retention and attendance data only will be reported for the 1998-1999 school year. Database information indicates that no students dropped out during the 1998-1999 or 1999-2000 school year. / Evaluation Components Curriculum service delivery, including curriculum for students who are identified for both Special Education and ESL. Materials and resources\nstaffing, including the number of teachers and aides trained, frequency of training, scope of training and results\nand student progress. Student progre^^^ademic progress of LEP students in ESL programs, LEP students who have been exited, anoifEP students not served in ESL programs. Examination of the number of retentions, drop-outs, and the student attendance rates of LEP students, former LEP students, and non-LEP students, and the extra become proficient in English/^ extent to which the aforementioned groups are becoming or have _______ .eview and comparison of data reflecting the gradtiation-and-\n(d^but rates of LEP a^former LEP students with that of their non-LEP peers. 7 Description of LRSDs Academic Testing to Assess LEP Student Progress State Mandated ACTAAP Benchmark Examination, Grades 4 \u0026amp; 8. The State is in the process of implementing its Arkansas Comprehensive Testing, Assessment \u0026amp; Accountability Program (ACTAAP) which includes a Benchmark Examination containing a measure of mathematics achievement and literacy. The intent and purpose of this component is to identify students in need of additional instruction in mathematics and literacy. This examination process is being developed, piloted, and implemented in a sequential and cumulative process beginning with 4* grade in SY 1997-98, and including 8* grade in SY 1998-99. SY 2000-01 will incorporate the math measure for 6* grade currently being piloted in other schools across Arkansas. Also end-of- course measures for Algebra I, Geometry, and Biology I are currently in the item development phase. All ESL students are to take the test unless parental permission to exempt is given. The comprehensive mathematics and literacy components contain multiple-choice and openresponse questions based on The Arkansas Mathematics, Reading, and English/Language Arts Curriculum Frameyvorks. Items are developed with the assistance and approval of the Arkansas Content Advisory Committees composed of active Arkansas educators with expertise in mathematics and literacy. The committees developed and reviewed both multiple-choice and open-response items to ensure they reflect the Arkansas Curriculum Frameworks and are grade-appropriate. While multiple-choice questions are scored by machine to determine if the student chose the correct answer from four options, responses to open-response mathematics questions are scored by trained readers using a pre-established set of scoring criteria. Students are given scores in math and literacy. Students can receive a test score of one through four with four representing Advanced followed by Proficient, Basic, and Below Basic. Achievement Level Test (ALT), Grades 2-11. The recently implemented Achievement Level Test (ALT) includes a series of mathematics, reading, language, usage, and science achievement measures that increase in difficulty across eight levels. This type of measurement is designed to document growth by assessing students at the cutting edge of their individual achievement level. Fall and spring administration across grades 3-11 permit measurement of growth within and across school years expressed in two kinds of scores: percentile scores and scale or RIT (Rasch Interval Scale) scores. Percentile scores can be used to compare students to the large group of test takers using the ALT developed by the Northwest Evaluation Association. It is important to note that this is a comparative group currently involving 104 schools districts and 500,000 students and growing 4 to 13 points annually. This is not a norm group configured to represent public school populations. All ESL students are to take the test unless parental permission to exempt is given. More importantly, demonstration of growth within and across an individuals matriculation in grades 2 - 11 is documented using the RIT score designed to make direct comparisons to a criterion performance level along a scale from 160 to 250. Students typically start at a RIT score of about 170-190 in the fall of the 3\"^* grade and progress to the 230-260 range by high school. Students at 235 have reached a readiness level for Algebra I. It is very important to note that along the Rasch Interval Scale, scores have the same meaning regardless of the individual students grade level. This type of measurement allows some students to start at a higher RIT level and some low-achieving students to never reach the top level. The design provides an accurate measure of each students achievement where the typical standardized test, by its nature, provides inadequate measures for many students, especially those at the high and low ends of the scale. Also important is the fact that tests are aligned with The Arkansas Mathematics, Reading, and English/Language Arts Curriculum Frameyvorks, thus enabling the District to determine impact and effectiveness of its instructional programs. The pool of test questions, developed by the Northwest Evaluation Association, has been extensively field tested to insure items of the highest quality and fairness. A balance of math teachers and curriculum specialists (i.e., race, gender, and grade level) matched the pool of questions to the standards and their attending benchmarks included in the aforementioned Frame^vorks. During test development activities, questions were calibrated for difficulty and assigned to a level (e.g.. Math levels 1-8). For example: An appropriate expectation of a Level 1 student is to multiply whole numbers, while a Level 6 student should be able to multiply fractions. This calibration makes it possible to calculate the RIT score which is tied directly to the curriculum. ALTs are administered during the f  and 4th quarters. Stanford Achievement Test- O*** edition (SAT-9). The Sat-9 is a overall measure of achievement in reading, mathematics, language arts, science, and social science. The SAT-9 is designed to measure student achievement in relation to the performance of a national sample selected to be representative of the nations students in each of the grades tested. The test provides a method for comparing the achievement of students with that of students in the same grade across the country. The SAT-9 is administered to students in grades 5, 7, and 10 during the month of September. The SAT-9 is a timed test. In addition to a raw score, mean scaled scores, normal curve equivalent (NCE), percentile, and stanine scores are provided. For this report, NCE scores are to be used.Marie M. Clav: An Observation Survey (AOS). The AOS is an observational measure that is used to guide student work on tasks related to reading and writing. The AOS consists of six measurable tasks.  Running Records records student behavior as text is read aloud.  Letter Identification records the what letters a student knows.  Concepts in Print records what a student is attending to on the printed page.  Word Test records the number of words a students knows.  Writing records a students writing behavior, and  Writing Vocabulary records all the words a student can write in ten minutes. Running records is considered the pivotal task among the remainder of the observation tasks. During this observation the student reads materials that are typically used within the classroom and the observer records the directional movements that are made, errors or miscues, and the student is assessed on deriving meaning, structure and information from the material. Given the number of observational tasks, only the running record data will be reported. The AOS is administered in the fall and spring to Kinderg^en through 2\"* grade. The AOS is written for classroom teachers who want to become careful observers of young children as they learn to read and write. The Survey is useful to teachers who work one-on-one with students who are having difficulty in learning to read and write. The AOS is administered to all students. Results State Mandated ACTAAP Benchmark Examination. Grades 4 \u0026amp; 8. The State Benchmark exam for the 1999-2000 school year was given in April 2000 and the results will not be available to the District until October 2000. The Benchmark exam was given to 4* and 8* grade students in February 1999 for the 1998-1999 school year. Data from the 1998-1999 school year will be used Jbr this report. Initial test results are provided to the District in hard copy form, only disagregating the data by race and ethnicity. At this time database data (e.g. Excel file) are only available for 4* grade students. Several LEP students (N = 18) did not take the test. The State of Arkansas allows LEP students, with permission from their parents, to be exempted from taking the test. This is probably the case for most of these students. The Arkansas Department of Education is currently field testing an alternative assessment for exempted students. Training on the administration of the alternative assessment will begin in fall 2000. There were no 4^ grader NALEP students, thus no test results. LEP students that took the test performed below District results and FEP students performed above District results (see Figures 3 \u0026amp; 4). (insert Figures 3 \u0026amp; 4) CL Stanford Achievement Test, 9^ edition (SAT-O). Students in the 5*, 7*, and lO'^ grades took the SAT-9 during September 2000. The complete battery scores reported are a compilation of reading, mathematics, language, spelling, study skills, science, social science, using information, and thinking skills subtest results. Percentile scores are reported (see Figure 5). The general population consists of all student with the exception of exempted LEP and special education students. A number of students (18 5 graders, 15 graders, and 18 10* graders) did not take the SAT-9. The State of Arkansas allows LEP students, with permission from their parents, to be exempted from taking the test. %js=?rpF^ably=^e\u0026lt;ise-fb-iaastailise-stedeHts. The Arkansas Department of Education is currently field testing an.alteBtrative assessment for exempted students. Training on the administration of the altemativ\nassessment will begin in fall 2000. Figure 5 SAT-9 Complete Battery Percentile Scores Grade General Population LEP NALEP FEP Sth 7th 10th 36 N = 1481 42 N = 1363 40 N = 1448 15 N = 11 27 N = 22 30 N=2 78 N = 2 40 N = 5 55 N = 30 71 N = 21 55 N = 16 Marie M. Clay: An Observation Survey (AOS). During the 1999-2000 school year, kindergarten through 2\"' grade students were assessed during the early fall and late spring on the six categories of the AOS. No students were to be exempted from taking the test. However, there were some students that were not assessed. During the fall testing 10 kindergarten, 19 1 grade, and 19 2\" grade students and during spring testing 11 kindergarten, 15 1 grade, and 5 2\"'' grade wee not assessed. There was an improvement from fall to spring on the number tested. On most of the State and District assessments (i.e., SAT-9 and State Benchmark), LEP students are allowed to be exempted, with parental permission, from testing. Teachers were asked to assess all students. With the exception of Kindergarten, LEP students did not grow as fast as the general population (see Figures 6, 7, \u0026amp; 8). V (insert figures 6, 7, \u0026amp; 8) rd Retention rate. Twenty-five students (16 LEP, 1 NALEP, arid 8 FEP)^ere retained after the 1998-1999 school year. Retention rates of these subpopulatiQns_were generally higher than the general school population (see Figure 8). Among the LEP students there were 10 male and 6 female students, and 14 students spoke Spanish, with 1 speaking Chinese and 1 Filipino. The one NALEP student was female and spoke Spanish. Among the FEP students there were 6 male and 2 females, with 7 speaking Spanish and 1 speaking Portuguese Figure 4 Retention Rate for the 1998-1999 school year by Level Level General Population LEP NALEP lElementar^ |Middle f HZ 1.33%| 4.29% I [High EI 0.98%| 2.94^ 0.42%| 1,31^ 3.30% I ^.00%| I 0.65%l iJ--  I 5.06%] EP I Attendance. The average daily attendance for the District in 1998-1999 was 92%. District data is not reported by level. Average Daily Membership (ADM) data is sent to the Arkansas State Department of Education and the Dept, of Ed. calculates the attendance rate. Dropout. According to the Districts database, no LEP, NALEP, FEP students dropped out during the 1998-1999 or 1999-2000 school years. However, 63 students did leave the District. Reasons for leave were:  Moved to another country, 23%, N = 15  Transferred to a private school, 10%, N = 6  Moved out of state, 30%, N = 19  Moved to another school District within the state, 35%, N = 22  Withdrew from school, 2%, N = 1 / ^eview^f academic dataj)f students reclassified and exited from ESL programs. Review of data reflecfing fh'e number of LEP students who have exited from ESL but returned to e ESL program to address academic deficits. Twenty-five students, since 11/30/99, have beeia-r/clasai-fied fro\n200 school year, no students have exited the program and ret\n, ,^me^ De data for this subpopulation are: ,EP to FEP. ^uring the 1999- le^ Demographic and academic  12 males (48%) and 13 females (52%),  10 students with aLASof3/4andl5 students with a L AS o f 3/5,  2 students exited March 2000, 5 exited April 2000, 10 exited May 2000, and 8 have no exit date,  17 of the 25 students do not have a language code in the database.  All of the students receive Gifted and Talented (GT) services.  64% of the exited students were from grades 2\" through 5*, 28% from grades 6* through 8*, / and 8% from grades 9 and 11, /  11 of the students (44%) were from Newcomer Centers. /fcr 4 P Z L L 1^- Figure 5 SAT-9 Complete Battery Percentile Scores Grade General Population LEP NALEP FEP Sth 7th 10th Figure 9 36 N = 1481 42 N = 1363 40 N = 1448 15 N = 11 21 N =22 30 N =2 78 N = 2 40 N = 5 55 N = 30 71 N = 21 55 N= 16 Retention Rate for the 1998-1999 school year by Level Level General Population LEP NALEP LEP 3 J I Elementary} 1.33% I 4.29% I I 0,65^ [Middle 0.98%| 2.9\nThis project was supported in part by a Digitizing Hidden Special Collections and Archives project grant from The Andrew W. Mellon Foundation and Council on Library and Information Resoources.\n   \n\n   \n\n  \n\n\n   \n\n  \n\n \n\n\n   \n\n   \n\n \n\n\n   \n\n  \n\n \n\n   \n\n \n\n  \n\n\n   \n\n \n\n  \n\n\n\n   \n\n  \n\n  \n\n\n   \n\n   \n\n  \n\n \n\n \n\n\n   \n\n  \n\n \n\n\n\n\n\n\n\n\n\n   \n\n \n\n\n\n  \n\n\n   \n\n\n\n  \n\n\n\n "},{"id":"bcas_bcmss0837_1043","title":"External Evaluations, correspondence and personal vitas","collection_id":"bcas_bcmss0837","collection_title":"Office of Desegregation Management","dcterms_contributor":null,"dcterms_spatial":["United States, 39.76, -98.5","United States, Arkansas, 34.75037, -92.50044","United States, Arkansas, Pulaski County, 34.76993, -92.3118","United States, Arkansas, Pulaski County, Little Rock, 34.74648, -92.28959"],"dcterms_creator":null,"dc_date":["2002/2003"],"dcterms_description":null,"dc_format":["application/pdf"],"dcterms_identifier":null,"dcterms_language":["eng"],"dcterms_publisher":["Little Rock, Ark. : Butler Center for Arkansas Studies. Central Arkansas Library System."],"dc_relation":null,"dc_right":["http://rightsstatements.org/vocab/InC-EDU/1.0/"],"dcterms_is_part_of":["Office of Desegregation Monitoring records (BC.MSS.08.37)","History of Segregation and Integration of Arkansas's Educational System"],"dcterms_subject":["Little Rock (Ark.)--History--21st Century","Education--Arkansas","Education--Evaluation","School improvement programs"],"dcterms_title":["External Evaluations, correspondence and personal vitas"],"dcterms_type":["Text"],"dcterms_provenance":["Butler Center for Arkansas Studies"],"edm_is_shown_by":null,"edm_is_shown_at":["http://arstudies.contentdm.oclc.org/cdm/ref/collection/bcmss0837/id/1043"],"dcterms_temporal":null,"dcterms_rights_holder":null,"dcterms_bibliographic_citation":null,"dlg_local_right":null,"dcterms_medium":["documents (object genre)"],"dcterms_extent":null,"dlg_subject_personal":null,"dcterms_subject_fast":null,"fulltext":"\n \n\n\n\n\n\n\n\n  \n\n\n   \n\n   \n\n\n   \n\n\n   \n\n\n\n\n   \n\n\n\n\n   \n\n\n\n\n\n\n\n   \n\n   \n\n \n\n\n   \n\n   \n\n  \n\n\n   \n\n  \n\n \n\n\n   \n\n  \n\n \n\n\n   \n\n  \n\n  \n\nMemorandums related to \"\"Extended Year Education (EYE) Program Evaluation\"\" and \"\"Middle School Transition Program Evaluation.\"\" Personal vitas for Theresa Akey and William P. Moore.\nThis transcript was created using Optical Character Recognition (OCR) and may contain some errors.\nMd:l!c: 5:/4col ll'cJnsffe,IJh /Jrd?l'i:2#7 Educo/c.tJh TO: FROM: LITTLE ROCK SCHOOL DISTRICT INSTRUCTIONAL RESOURCE CENTER 3001 PULASKI STREET RECEIVED LITTLE ROCK, AR 72206 Board of Education T. Kenneth James, Superintendent of Schools FEB 11 2003 OFFICE OF DESEGREGATION MONITORING PREPARED BY: ~~onnie A. Lesley, Associate Superintendent for Instruction DATE: February 27, 2003 SUBJECT: Middle School Transition Program Evaluation Background Information One of the group of programs requ ired by the federal court to be evaluated with the participation of an external evaluator was the Middle School Transition Program. The District made the transition to middle schools in fall 1999. One of the four \"draft\" program evaluations presented to the Board of Education in summer 2000 was the one on the Middle School Transition for school year 1999-2000. Steps Taken as a Result of the Program Evaluation The District took no steps as a result of the 1999-2000 program evaluation, since, as is noted in the document, the data gathered that year were considered to be baseline data, against which to evaluate progress in the future. That statement does not mean that no changes in the middle schools have occurred since 1999-2000. In addition to numerous individual school level changes, the following significant District-led changes have occurred:  Full phase-in of new mathematics curriculum, instructional strategies, materials, and assessments\n Full phase-in of new science curriculum, instructional strategies, materials, and assessments\n Refinement of implementation of the new middle school Reading/Writing Workshop program (curriculum, instructional strategies, materials, and assessments)\n Implementation of new curriculum for Research and Writing Pre-AP at grades 6-8\n New writing curriculum implemented in 2002-2003.  Beginning of phase-in of new social studies curriculum, instructional strategies, materials, and assessments\nBoard Memo February 27, 2003 Page Two  New magnet programs implemented in fall 2001 at Cloverdale Middle School and Mabelvale Middle School\n Full implementation of the Safe Schools/Healthy Students grant project, focused almost exclusively at the middle school level, providing a range of comprehensive programs and services to improve learning climate and student achievement\n Increased enrollment in grades 6-8 Pre-AP courses in the core areas\n Increased enrollment in Algebra I in grades 7-8\n Administration of new Benchmark examinations in grade 6\n Involvement of four middle schools (Cloverdale, Mabelvale, Henderson, and Southwest) in Successful Transitions and Making Middle Grades Work, as developed by the Southern Regional Education Board\n Strengthening of the middle school fine arts programs\n Implementation of Act 1748, requiring physical activity at each grade level, K-9\n Implementation of Fred Jones' classroom management strategies. Designation of External Consultant and His/Her Qualifications On December 2, 2002, the District awarded the contract for the Middle School Transition program evaluation to the firm, Youth Policy Research Group, Inc., Dr. William Moore, Senior Partner. The resumes of Dr. Moore and his associate, Dr. Theresa Akey, are attached, establishing their qualifications. Administrator Participation in Conducting the Program Evaluation In addition to Dr. Moore and his associates, specifically Dr. Theresa Akey, the following LRSD staff participated in the evaluation: Dr. Ed Williams, Department of Planning, Research, and Evaluation Ms. Mona Briggs, Department of Planning, Research, and Evaluation Dr. Kathy Lease, Assistant Superintendent for Planning, Research, and Evaluation Principals of the eight middle schools, who participated in the administration of assessments and who completed surveys and administered student surveys Teacher Participation in Conducting the Program Evaluation All grade 7 teachers who administered the SAT-9\nand all grade 8 teachers who administered the Benchmarks\nAll grades 6-8 teachers who completed surveys and/or who administered student surveys I Board Memo February 27, 2003 Page Three Impact on African-American Student Achievement The study conducted by the external evaluator did not attempt to draw any conclusions related to this research question since the student performance data available for the study were \"baseline, and there are serious questions about the appropriateness of the achievement measures and about the validity of some of the other performance outcome measures.\" Recommendation That the Board of Education approve the Middle School Transition program evaluation for submission to the federal court. BAL/adg ' ~ ~ ~ \"\" \"\"- \"\" - - ~    .a. T. AJceY f l uuw, v .. - . -- RFQ23-01 a-Response of Youth Policy Research Group, Jnc. THERESA AKEY, PH. D. BIOGRAPHICAL SKETCH OCTOBER, 2002 Dr. Toeresa J,/\n.ey bas extensive experience in the areas of evaluation, research and assessment. She earned her doctorate in educationil psychology, research and statistics from the UniversitY of](ansas in 1995 and her master's in comrnunitY and school counseling from Delta Staie University in !990. She bas approximately 12 years experience working as a consultant with nonprofit service agencies. Her major work bas focused on agencies that provide supports to families of children with disabilities, including evaluation of grants, staff development traming, and site-based research projects. She bas also worked with nonprofit organiza!ions in the areas of alcohol and drug abuse, comrnunitY counseling, and other services in both rural and urban areas. She served as project director and research director of the Beach Center on Families and Disabilities where sbe co-anthored grants, conducted research, and coordinated research and evaluation activities for a large research center focused on service provision to fumilies of children with disabilities. Dr. J,/\n.ey bas extensive background in educationil assessment. research, evaluation, and analytic methods. She served as an assistant professor in the area of research and analytic methods at Auburn University for two years, and roost recently in the role of director of educationil research. evaluation, and assessment in a ](ansas City area school district for the last four years. 38 ~   '  '-   I -..       J. T. A~ (Y01J1h Policy Research Group, inc.) 2 RFQ23-0JO--Respon.se ojYowh Policy Research Group, inc. CURRICULUM VITAE THERESA M. AKEY, PH.D. P.O. Box 4196 Kansas City, KS 66104-0196 (816) 935-0852 (V) (816) 62S-1927 (F) tak~rg.org PROFESSIONAL STATEMENT A collaborative, innovative professional with interests in educational evaluation and assessment issues, including school reform, teacher professional development and evaluation, instructional strategies, standards-based education, and alternative educational environments. Seeks to promote continuous improvement of educational and youth-serving organizations through collaborative partnerships, action-oriented research, and application of theory to practice. AREAS OF PROFESSIONAL EXPERTISE o Research design and analysis including complex quantitative analyses (HLM, SEM), categorical modeling, and qualitative design and analysis. o Development of educational assessment materials and psychometric validation of those materials. o Program evaluation in educational and community settings o Curriculum development in K-12 and higher education settings o Professional development in the use of data for organizational improvement, assessment development, and program evaluation o Grant and proposal writing o Information management development for educational information EDUCATIONAL HISTORY Ph.D. (1995). University of Kansas, Lawrence, KS, Educational psychology and research with emphases in developmental psychology and quantitative research methods  Ed.S. (1992). University of Kansas, Lawrence, KS, School psychology  M.Ed. (1991). Delta State University, Cleveland, MS, School psychology B.S. (1990). University of Arkansas, Little Rock, AR, Psychology and sociology 39 ,     '  ''  ' - II       T. Akev (Yowh Policy Research Group. inc.) 3 RFQ23-0JO-Response of Youth Policy Research Group, bu\n. PROFESSIONAL EXPERIENCES RESEARCH AND EVALUATION Youth Policy Research Group, Kansas City, Mo (June, 2002 to present). Senior Partner D Acting managing partner in research and evaluation corporation- administration and company management D Indiana Teaching ~ality Center (May 2002 to present). Evaluation project of best practices in teacher evaluation. ~alitative policy analysis of national standards in teacher evaluation and comparison of Indiana school districtS to those standards.  YDSI, Inc (May 2002 to present). Multivariate statistical analyses of insuuctional survey and observation data, including data display, technical assistance, and interrupted ti.me-series analyses   Kansas City, Kansas Public Schools (May 2002 to present). Development and implementation of case management system to display and organize student information  Kansas City, Kansas Public Schools Qune 2002 to present). Alignment and development of standards-based mathematics and language artS assessments  Park Hill School District, Kansas City, MO (July 1998 to May, 2002), Director of Research Evaluation and Assessment D Oversight and coordination of all district assessment and program evaluation activities. D Preparation of short-term and long-range budget and operational plans\nD Fiscal authority for budget of $200,000+ annually. D Responsibility for all personnel and resource utilization within the Department D Development and implementation of a comprehensive local curriculum-cm.bedded assessment system integrated with the Missouri Standards  D Development and implementation of district-wide continuous improvement planning and  eDvoacluuamtieonnt ation and analysis of district performance on all state accreditation performance and evaluation guidelines . D Development and implementation of district-wide: program evaluations of instructional practices, special education, professional development, assessment, supplemental instruction (Title 1, gifted, ESL), curriculum, and otha educational initiatives  D Served on regional and state comminees related to K-U assessment and evaluation issues. D Led applied and theoretical research projectS to address pressing educational issues relevant to local policy and instructional practice. D Collaborated with district stakeholders and school leadership to find innovative solutions to chronic educational problems in the Disrrict. Akey Consulting, Auburn, AL (June 1996 to July 1998) and Kansas City, MO. (Aug. 1999 to May, 2002), Founder and Owner O Online assessment and data information coordinating consultant for Kansas Cir-y, Kansas Public Schools. Design and implementation of online assessment and case management sysrem for reporting assessment information to teachers and adminiscrarnrs  D Assessment and data consultant for Kansas City, MO Regional Professional Development Center. Facilitated development of regional local assessments tied tO state standards and benchmarks  ::J Statistical analysis consultant for Kansas City Kansas Public Schools. Systems reform data analysis 40 I t   ' ' '          t  T Akey (Yowh Policy Research Group. Jnc.J ./ RFQ23-0JO-Response of Youth Policy Research Group. Inc D Educational consultant to Missouri school disrrictS- using data to make insuuctional decisions and best practices in standards-based assessment. Have consulted with approximately 10 different school districts to do uaining and planning in this area. D Consultant to Kansas City, MO School District- review of performance indicators and data management system and statistical consultant for desegregation case. D Educational consultant to Alvin Nash, Kansas City, Missouri Mayor's office. Suategic planning. D Educational consultant for the West Alabama Learning Coalition of Schools. Program evaluation and grant/proposal writing on teacher professional development and collaboration. D Educational evaluation consultant for Celebration School and Disney Corporation. Evaluation of integrated assessment practices. D Consultant for Chambers County-Auburn University Partnership. Program evaluation of insuuctional effectiveness and professional development. Beach Center on Families and Disabilities (1993 to 1996), Lawrence, Kansas, Research coordinator (1995-1996) and Research project director (1993-1996). D Coordinated center research activities and data analysis as research coordinator D Provided technical assistance in instrument development o Developed insuuments to measure consuucts of psychological empowerment D Primary research investigator on two projectS (one qualitative and one quantitative) concerning responsive services and psychological empowerment in family members of a child with a disability  D Worked in collaborative manner with community-based family support programs in several states to evaluate responsiveness of services to families of a child with a disability  D Grant and report writing Jones Research Consulting (1991-1996), Lawrence, Kansas, Founder and owner D Training consultant in empowering professionals and families. Beach Center on Families and Disabilities, University of Kansas. D Research and statistics consultant. Department of Educational Psychology and Research, University of Kansas D Family support consultant to Wisconsin Family Support program. Program evaluation D Insuuctional consultant to Douglas County Christian Schools, Lawrence, KS TEACHING AND ACADEMIC POSITIONS University of Kansas, Lawrence, KS (September 2001 to present). Psychology and Research in Education Department, Adjunct Professor, Educational 'Research, Statistics, and Classroom Assessment University of Missouri-Kansas City, Kansas City, MO Qanuary 2000 to present). Department of Educational Psychology and Research, Adjunct Professor, Classroom Assessment Park University, Parkville, MO Oanuary 2000 to present). Education Department, Adjunct Professor, Standards-Based Assessment Auburn University, Auburn, AL (1996 to 1998). Assistant Professor, Department of Educational Foundations, Leadership and Technology, Educational Statistics (including suucrural equation modeling, hierarchical linear modeling, multivariate statistics, and basic statistics), Research Methods, and pre-service classroom assessment University of Kansas, Lawrence, KS (1995-1996), Department of Educational Psychology and Research, Insuuctor, Basic educational statistics  41 I '    '           T. Akev (fowh Policy Research Group, Jnc.) ) RFQ23-01 O-Response of Youih Policy Research Group, Inc. University of Kansas, Lawrence, KS (1991-1995). Department of Educational Psychology and Research, Graduate Teaching Assistant, Developmental psychology, basic educational statistics, and multiple regression. CLINICAL AND APPLIED PROFESSIONAL EXPERIENCES Topeka Public Schools (1991-1993). Topeka, KS, School Psychologist Clarksdale Public Schools ( 1990-1991 ), Clarksdale, MS, School Psychologist Arkansas Children's Hospital (1989-1991), Little Rocle., AR, Behavioral Intervention Unit, Classroom Teacher Arkansas Children's Hospital (1989-1991), Little Rocle., AR, Counselor GRANT FUNDING St. Clair, MA, \u0026amp; Akey, TM. (1999). Using Reflective Practice Groups to Improve Mathematics Instruction. Goals 2000 Grant, $40,000, .!J\u0026lt;ey, T. M. (1993). Family support influences on the development of parental skill and efficacy. NIDRR Grant # H133330070, RTD-5. Funded June 1993 as part of $650,000 grant, $45,000 per year for 5 years. /\\key, T. M. (1993). Validating the Psychological Empowerment Scale for parents of children with disabilities. NIDRR Grant# H133330070, RTD-7, Funded June 1993 as part of $650,000 grant, $35,000 for 5 years . PUBLICATIONS Aley, T. M. \u0026amp; Ares, N. M. (in press). Using a Learning Partnership to Teach Classroom-based Assessment in Context: .Conceptual and Belief Changes in Preservice Teachers. Teaching Education  Aies, N.M., \u0026amp; /\\key, T. (2000/2001). Self-organization in educational systems. Louisiana Education Research 1oumal. 25(1), 49-71  /\\key, T. M., Marquis, J. M., \u0026amp; Ross, M. E. (2000). The development of the Psychological Empowerment Scale: Evidence of its construct validity. Educational and Psvchological Measurement. 60Q), 419-437  Jensen, C., Hansen, C., Green, S. B., \u0026amp; /\\key, T. M. (1997). An investigation of item difficulty incorporating structure of listening tests: A hierarchical linear modeling analysis. Proceedings of the Language Testing Research Colloauium  Green S.B., Salkind, .J., \u0026amp; /\\key, T.M. (1999). Using SPSS for Windows: Analvzing and Understanding Data, 2nd Edition. New York: Prentice Hall  Green, S. B., .AJzey, T. M., Fleming, K. K., Hershberger, S. L., \u0026amp; Marquis, J. G. (1997). The effecrs of the number of scale points on chi square fit indices in confirmatory factor analysis. Srrucrural Eauation Modeling, 1(2) . 42 t t            J,.. T. Akey (fouih Policy Research Group, Jru:.) 6 RFQ23-0JO-Response of YOUih Policy Research Group, In!:. Akey, T. M. (1996). Empowering families of children with a disability: Implications for professional educators and service providers from a family-centered program- Teacher Education Research and Practice, 12(2). Green, S.B., Salkind, N .J., \u0026amp; Akey, TM. (1996). Using SPSS for Windows: Analvzing and Understandin_g Dara~ 1st Edition. New York: Prentice Hall. Jones (Akey), T. M., Garlow, J. G., Turnbull, H. R., \u0026amp; Barber, P. A. (1995). Family empowerment in a family suppon setting. In G. Singer, L E. Powers, \u0026amp; Olson, A.L (Eds.) Redefining farnilv support Introduction to oublic-orivate oarmershios. Jones (Akey), T. M. (1995). To hdp o, not to hdpc A ,n,dy of group ,nd in\u0026lt;fi,idu,l wriabks in , monl system. Unpublished doctoral dissertation. Depanment of Educational Psychology and Research, University of Kansas Jones (Akey), T. M. (1994). Development of the Psychological Empowerment Scale: Preliminary investigations. Unpublished master's thesis. Department of Educational Psychology and Research, University of Kansas. Frey, B., Jones (Akey), T. M., \u0026amp; Saxon, T. F., (1993). Exoloring Research: Teacher's Manual ITeacher's Manual for N.J. Salkind's Exploring Researchl. New York: MacMillan. CONFERENCE PRESENTATIONS A.ND WORKSHOPS Akey, T. M. (2000). Using Data to Improve Instruction. Workshop presented to the Regional Professional Development Center, Kansas City, MO, April 16, 2001. Akey, T. M. (2000). Using Data to Improve Instruction. Workshop presented to East Central Show-Me Curriculum Administrators Association, St Louis, MO, April 1, 2000  Akey, T. M. (2000). Using Data to Improve Instruction. Workshop presented to West Central Show-Me Curriculum Administrators Association, Kansas City, MO, February 16, 2000  Pennell, J., Eick, C., and Akey, T. M. (2000). Teacher beliefs and grading practices. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, April 2\u0026amp;-30, 2000  Akey, T. M., Sanders, S., Boyd, P., Gorrell, J. J., Kamen, M., \u0026amp; Salisbury-Glennon, J. (1999). Assessment and evaluation .in the Celebration School: Links to learning and curriculum. Paper presented at the annual meeting of the American Educational Research Association, Montreal, Canada, April 19-23, 1999  Ares, N. \u0026amp; Akey, T. M. (April, 1999). Modeling and understanding of social interactions. Roundtable presented at the annual meeting of the American Educational Research Association, Montreal, Canada, April 19-23, 1999 . Akey, T. M. \u0026amp; Lawrence, F. (April, 1998). Undersranding the effecrs of non-normal data on latent growth curve models. Violations of normality and influence of sample size. Paper accepted as part of a ,ymposi= (also chai, wd o,g,niw of session) Madding Non-norm,\\ Dau fo, the annual m~cing of the American Educational Research Association, San Diego, CA., April 12-17, 1998  43 I 9 ~ --~ ~ ~  ~ w l.l!.t ~ - T. Akey (J'owh Policy Research Group, inc.) 7 RFQ23-0 I (}-Response of }'ouch Policy Research Group, inc. Mey, T. M., \u0026amp; Ares, N. M. (November, 1997). Conceptual and Belief Changes: Implications of a Sociocultural Approach to Educating Preservice Teachers in Classroom-Based Assessment. Paper presented at the annual meeting cif the MidSouth Educational Research Association, Memphis, TN, November 12- 14, 1997 Mey, T. M., Marquis, J. G., \u0026amp; Turnbull, H. R. (August, 1997). Effecr.s of Family Support Programs on Parental Empowerment. Paper presented at the annual meeting of the American Psychological Association, Chicago, IL, August 1997. Lmcig, D., \u0026amp; M\u0026lt;y, T. M. (April, 1997) Family Adapucion in f,mili,s wiili Adult Cbild\u0026gt;-en wiili Mcnul Retardation: Impact of Family Suengths and Appraisal. Paper presented at the annual meeting of the American Counseling Association, Orlando FL, April 1~13, 1997. Mey, T. M. \u0026amp; Green, S. B. (March, 1997). A Model of Moral Decision-Making: A Srudy of Group and Individual Influences. Paper presented at the annual meeting of the American Educational Research Association, Chicago, IL. March 24-28, 1997. Mey, T. M. \u0026amp; Green, S. B. (March, 1997). The Relationship Between Power and Intercorrelated Dependent Variables in MANOVA. Poster presented at the annual meeting of the American Educational Research Association, Chicago, IL. March 24-28, 1997. Mey, T. M. (February, 1997). Confirmatory Factor Analysis of a Moral Dilemma Helping Measure. Papa p,esmt\u0026lt;d at th, ,nnu,I mming of ili, E,,tem Educaciou,I Ros=ch Assnciacion, Hilton Hou!, SC, February 19-22, 1997. Jensen, C., Hansen, C., Green, S. B., \u0026amp; Mey, T. M. (1996). An investigation of item difficulty incorporating structure of listening tests: A hierarchical linear modeling analysis. Paper presented at the Language Testing Research Colloquium, Tampere, Finland, July 31-August 3, 1996. Mey, T. M. (December, 1995). Effecr.s of family support programs on parental empowerment. Workshop presentation at the annual Empowering Families Conference, Chicago, IL. Jones (Mey), T. M. (August, 1994). The development of the Psychological Empowerment Scale: Preliminary investigations. Poster presentation at the annual meeting of the American Psychological Association, Los Angeles, CA. Jones (Akey), T. M. Qune, 1993). Roundtable discussions on family empowerment and family services. Invited round table participant and presenter at the annual-Family Support and Family Empowerment Conference, Dartmouth, NH. RECENT TECHNICAL AND EVALUATION REPORTS Mey, T.M. (2002). Effecrs of standards-based assessment on insuuctional practices in middle school math teachers. Technical report for the Park Hill School District, February 2002. Akey, T.M (2001). Using data as a method for school improvement planning and decision-making: Administrators. Technical report for Park Hill School District, August 2001  Akey,~- M. (2000). 2000-2001 Disuict Performance Update: Assessment of the Show-Me Standards. Evaluaoon report prepared for Park Hill Scnool Disuict, October 1::, 2000.   J t '  t 'I           ill T. Akey (fouth Policy Research Group, inc.) 8 RFQ23-0JO-Response of YOUJh Policy Research Group, inc. Akey, T.M (2001). Using data as a method for insrructional planning and decision-making: Teachers. Technical repon for Park Hill School Disuict, August 2001. Akey, T.M. (2001) Validation of learning expectations for a standards-based curriculum in communication am. Technical repon for Park Hill School Disuict, January, 2001 Akey, T. M. (2001). Effectiveness of the Class-Within-a-Class Model .. Evaluation report for Park Hill School District. February, 2001 Akey, T. M. (2001). Evaluation of class rank procedures: Examination of alternate methods. Evaluation report for Park Hill School District. February, 2001 Akey, T. M. (2001). EffectS of Block Scheduling on lnsrructional Practices. Evaluation repon prepared for Park Hill School Disuict, February, 2001  Akey, T. M. (2000). Title I Evaluation Report Adequate Yearly Progress and Recommendations for Program Improvement. Evaluation report prepared for Park Hill School Disuict, December 7, 2000. Akey, T. M. (2000). 1999-2000 District Performance Update. Evaluation report prepared for Park Hill School District, October U, 2000. Akey, T. M. (2000). EffectS of Block Scheduling on Student Outcomes: Mastery achievement, credits, and test scores. Evaluation report prepared for Park Hill School District, April, 2000. Akey, T. M. (1999). Effectiveness ofFull-Day Kindergarten: Third Year Evaluation. Evaluation report prepared for Park Hill School District. Akey, T. M. (1999). 1998-1999 Disuict Performance Update. Evaluation repon prepared for Park Hill School District, October 10, 1999  BOOKS Green, S. B., Sal.kind, N. ]., \u0026amp; Akey, T. M. (1997). Using SPSS for Wmdows: Analyzing and Understanding Data. New York: Prentice Hall  Green, S. B., Sal.kind, N. ]., \u0026amp; Akey, T. M. (1997). Using SPSS for the Macintosh: Analyzing and Understanding Data. New York: Prentice Hall. PROFESSIONAL AND COMMUNITY SERVICE Kansas City, Kansas YWCA, (Dec 2001 to present), Board of Directors Heartland Student Achievement Gap Organization, (August 2001 to present). Board of Directors Park Hill Educational Foundation, (August 2000 to present), Advisory Board 45 I  ~    ~ -  t t         Auburn University T. Akey (Youth Policy Research Group, Inc.) 9 RFQ23-0JO-Response of Youth Policy Research Group, Inc. 1998: Undergraduate Core Curriculum Development Committee for Teacher Education 1997: Chair, Library Appeals Committee 1997: Reviewer. Professional Educator, College of Education Universitv of Kansas 1993: Representative to the Graduate Student Council Student representative for faculty meetings of Educational Psychology and 1993, 1995: Research .Deparonent. RECENT HONORS Nomination for Dissertation of the Year: University of Kansas, School of Education, May 1996 Who's Who in College, 1987, 1991 President's List 1987-1991 Outstanding Psychology Student, 1990 PROFESSIONAL ORGANIZA TIONS Show-Me Curriculum Administrators Association 1998 to present ASCD National Organization, 1998 to present Missouri ASCD: 1998 to present American Psychological Association: 1990-1998 Society for Research In Child D~elopment 1992 to present American Educational Research Association: 1993 to present 1997: Reviewer for American Educational Research Association Arinual Meeting MidSouth Educational Research Association: 1996-1998 1998: Graduate Student Involvement Committee 1997: Reviewer for MidSoutb Educational Research Association Annual Meeting 1997: Graduate Student Involvement Committee Eastern Educational Research Association: 1996-1998 American Evaluation Association 1998-present 46 I                  W. P. Moore (Youth !'Otley J(esearcn vruu , ,,,.__ , , RFQ23-01 O--Response of Youth Policy Research Group, Inc Wn.,LIAM P. MOORE, PH. D. BIOGRAHICAL SKETCH OCT0BE~ 2002 Dr. Moore holds a doctorate in Educational Psychology from the University of Kansas and is a Senior Parmer with Youth Policy Research Group, Inc. located in Kansas City, Kansas. Currently be is the co-investigator for the evaluation of a systemic change initiative (First Things First) in the Kansas City, Kansas Public Schools and just completed a post-doctoral fellowship with Juniper Gardens Childrens Research Project exploring the dimensions of effective learning communities in schools. Dr. Moore serves also as a lecturer at the University of Kansas where he teaches Evaluating School Programs, a doctoral-level educational evaluation course  Dr. Moore has served as Research and Evaluation Director in two urban school districts\nSenior Research Associate with the Ewing Marion Kauffman Foundation\nand held faculty positions in research and measurement at the University of Kansas- Dr. Moore co-founded and directed the Dramatic AIDS Education Project, a collaborative AIDS education program for school-age youth and has spent the last 6 years conducting research and evaluation srodies on the efficacy of this AIDS education program for youth. Two years ago, the Project received Kansas City's Ribbon of Hope Award for outstanding community service towards the elimination of HIV /AIDS  Previously, Dr. Moore was Senior Ma.na:,oing Consultant with GPR\u0026amp;E, a research and evaluation consulting firm. He has consulted with school districts, not-for-profits, health care institutions, national foundations, youth development intermediaries and universities. He is a past member of the Board of Directors of Kansas City Public Achievement, a Minnesota-based youth empowerment program\nand recently retired, after 6 years, from the Board of Directors of Sunflower House: A Child Abuse Prevention Center, where be was Vice-President of Education and a member of the agency's Executive Committee. He now serves as a member of the Advisory Board . Dr. Moore has served as an expert witness in Federal school desegregation litigation\nwas a member of the Panel of Writers for the development of the new Student Evaluation Standards coordinated by The Joint Committee on Standards for Educational Evaluation\nand served a twoyear term as a member of the National Research Development Committee for the American Occupational Therapy Foundation  Dr. Moore's research and evaluation interests have focused on effectively documenting the success of educational reform initiatives\nthe impact of mandated assessment -programs on teacher instructional practice\nand the impact of education on the HIV-related knowledge, attitudes and behavioral intentions of youth. Dr. Moore has published in Applied Measurement in Educarion, Educational Assessment, the International Journal of Educational Research, and The American Occupational Therapy Journal. Dr. Moore served 4 years on the Editorial Board of the Occupational Therapy Journal of Research and is currently a grant reviewer for the W. T. Grant Foundation . 7       t  W. P. Moore (Youth Polic-y Research Group. inc.) 2 RFQ23-0JO--Response of Youth Poiicy Research Group, inc. CURRICULUM VITAE WILLIAM P. MOORE, PH. D. OCT0BE~ 2002 Home 114 78 South Wtlder Street Olathe, KS 66061 913. 829. 3077 gprekc@aol.com Office P. 0. Box4196 Kansas City, KS 66104-0196 816. 564. 0143 01) 913. 390. 6162 (F) wmoore@YPrg.org Educational History 1991 Ph.D. 1984 M.A. 1981 B.S. University of Kansas, Lawrence, KS Educational Psychology Major: Evaluation, Research, and Measurement University of Kansas, Lawrence, KS Education Major: Secondary Curriculum and Instruction University of Kansas, Lawrence, KS Education Major: Secondary Soci.al Sciences Emphasis: Psychology and Sociology Current Appointments and Positions Senior Parmer. Youth Policy Research Group lnc  Kansas City, KS Lecturer. University of Kansas. Lawrence. KS Granc Reviewer. Willi.am. T. Grant Foundation., ew York.. JY Advisory Board, Sunflower House: A Child Abuse Prevention Center. Overland Park.. KS Advisory Board. Community Health Promotion Program. University of Kansas Medical Center, Kansas City. KS 8 t  t t '' ' -' ~,, ,  ~  ii  ..  W. P. Moore (l outh Policy Research Group. mc.1 j RFQ23-0JO-Response of YouJh Policy Research Group. inc. Professional Experience Consultation 2002-Present Senior Parmer, Youth Policy Research Group, Inc., Kansas City, KS. Youth Policy Research Group is a cmpora!e partnership formed by youth and education researchers located ID the Kansas City merropofuan area. The mission of this fum is to engage ID applied research and evaluation studies that will conrnDnte to policy and practice decisions and dialogue about the necessary supports and resources youth require to aclrieve long-term developmental success both acaderoicallY and behaviorally. Mucb of the work already complered by YPRG researchers is ID the K-12 education arena. Acting n,anagmg partner in research and evaluation corporation- administration and company management. 1993-2002 Senior Manrurin9- Consu)tan~ Great Plains Research and Evaluation, Olathe, KS. GPR\u0026amp;E was an education and youth development research and policy fum focused on improving the programming and organizations that children and youth experience- Qverall responsil,ility for managing research and evaluation conrractual work. Oversaw a staff of part-time consultants. managed resources and budget, negotiated contraets, oversaw project development, implementation, coordination of cla1a collection, analysis, rnssemmation, and action planning for future decision-making with clients. Clients included:  Ewing Marion Kauffman Foundation. Through contactual work awarded by EMKF we assisted the following clients:  State of Kansas, SRS, Alcohol \u0026amp; Drug Abuse Services + State of Missouri, Dept. of Mental Health, Div. of Alcohol \u0026amp; Drug Abuse + State of Colorado, Team Fort Collins + Kansas City-St. Joseph Diocese Schools  youth Opportunities unlimited  Coalition for Positive Family Relationships Shawnee Mission East High School, St. Thomas Aquinas High School, Shawnee Mission North High School, Grandview High School, Lee's Summit High School Blue Valley High School, Wmnetonka High School, Park Hill School District, Center High School, Hickman Mills Management School II  Telemedicine Research Center, Portland OR.  Gambone \u0026amp; Associates: Youth Policy and Research Firm, Philadelphia, PA  Department of Preventive Medicine, University of Kansas  Shawnee Mission Medical Center Foundation  Johnson County Regional Prevention Center  Kansas City Young Audiences-KCY A  Center for Educational Testing and Evaluation, University of Kansas  Child Abuse Prevention Coalition  The Coterie Theatre 9 I  .. ~ ~ ~ ~ ~ t  i   ~ ii8 W. P. Moore (Yourh Policy Research Group. Jnc.J .: RFQ23-0JO-Response of Youth Policy Research Group, Inc. Tbrough contactual work awarded by KCY A., we worked with the following clients:  Kansas City Arts Partners  Kansas City Lyric Opera  Kansas City Symphony  Fort Osage School District 1988 Evaluation Consultant. Deparrment of Art Education, Universitl' of](ansas, Lawrence, Kansas- Evaluation and data analysis of survey data examining the impact on participants of the Artist-in-Residence program, an integrated art education approach. 1987-88 Research Consultant. iawrence Unifred School District, Lawrence, ](ansas. Citywide telephone survey of attitudes towards Lawrence Public Schools. Development of sampling procedures, u,strumen1 development and revision, training of telephone interviews and protocol development. Oversee all technical aspects of data collection and analysis for survey oftaXPayers and for survey of district employees. Research \u0026amp; Evaluation Management and Staff Positions 1998-2002 Director, Department of Educational Research and Assessroent. Kansas Ciry, Kansas Public Schools, K,nsas City, ](ansas. Cabinet-level position with direct report to superintendent of Schools. Responsibilities included the overall IIJllllll,,oeroent of research, evaluation and assessment functions within the District. Created and organized the department from separate functional units. Prepared short-term and long-range budget and operational plansResponsibility for all personnel and resource utilization within the Department- Coordinated the actions of 8 professional and suppart staff members, as well as ten consultants, to respond to policy and strategic decision making ~ of senior district leadership, board memb=, building ~tors and district staff_ Met with State-level )ea\u0026lt;lership, communitl' 1eadershiP, and Unified Government staff to respond to external stakeholder needs for information. Planned for. and led applied and theoretical research projects to ,rldress pressing educational lssues relevant to local policy and i:nstnJCtional practice. Collaborated withJdistrict stakeholders and school leadersmP to find innovative solutions to cbronic educational problems in the District. 1997-1998 Senior ReseaICh Associate, Department of Research and Evaluaiion. Ewing Marion Kauffman Foundation. K,nsas City, Missouri. Responsibilities include providing research and evaluaiion support to internal clients and external grantees. PriroarY responsibility for leading all research and evaluation support to Foundation youth developm.ent-k-12 educational program initiatives. Client initiatives include local and national resrrucruring efforts in urban school districts. Provide leadership and management of teams associated with research and evaluation efforts. supervise three junior research and evaluation associates, liaison between research and evalUBiion and clientso creaxe, plan and manage research and evaluation projects: identifl' and select external vendors for specific projectS\ncreate and review RFPs: review grant proposals requiring research and evaluation oversight or support. Provide information for senior Foundarion and school district leadersmP to infonn policy and program decisions regarding urban school restructuring. 10   ' ~ ~  ~     ~ ~,,,, ,,.. \",.., ,,, W. P. Moore ffouth Policy J(esearc:11 v, uuy, ,,__  RFQ23-0J{}-Response ofYouch Policy Research Group. Jnc. 1996-1997 Director, Department 0 [Educarional Research and Improvement, Km,sas Ciry. Missoud School Dis1ricl, K,msas City, Missouri. Cabinet-level position with direct report to Superintendent of Schools. Responsibilities included the overall management of research and evaluation functions within the Disrrict. prepared short-tenn and loog-range budget plans, fiscal authority for budger of $600,0Q(},-. Resjl\u0026lt;\u0026gt;DS1DilitY for all personnel and resource utilizarion withio the Department. Coordinared the actioos of IO professional and support staff. Established partnerships with communirY groups and school facu]t)' to discuss innovative solutions to chronic educational problems in the Disrrict. Represented the District through information and expert testimony to attorneys and the Federal Court in desegregation litigation. !993-1995 Director of Research and Assist\u0026lt; Professor, Deparnnent of OT Education, University of Kansas Medical Cenrer, K,msas City, Kansas ResponsibiJities included mentoring faculty and sn,dents to develop and complete researcb. coordjnariog and facilitating federal and private grant development, and conducting educational and psychological research. Collaborating with other researchers in health and education. In addition, provided technical support, ins(rumelll and methods development- analytic assistance, and evaluation support to researchers, students  and faculty rnernb= univefsiry-wide. MernbershiP in Data Envelopment Analysis Users Group and Tele-educarion users group. Served on~ and school-wide committees representing researcb. curriculum and instrUCUon interests. As a member of the faculty of the School of Allied Health I coordinated all school-wide professional development events for faculty. 1989-92 Proer.mEvaluator. Km,sas City, Missouri School District, Department of Research. Evaluation, and Testing, K,msas City, MissourL Complete responsibilitY for the evaluation of seven to nine magnet school programs ,nnually. Developed evaluation plans, instmlllentation data collection procedures, trained ev-tion personnel, com:dinated all data  collection schedules and activities, assessed reliability, analyzed all data, and wrote formative, summ,rive, and achievement-enrollment surnn,aries annually. Reports informed local policy decisions for Board and senior district )eadership. Reports were used to infonn Federal court decisions regarding coropJiance with court orders and progress toward established desegregation outcomes as well as state-level policy reg,rding desegregation effectiveness in Missouri. I 987-88 Research A,sistallt. Center for Educational Testing and Evaluation. Kansas Internship Program, University ofK,msas, Lawrence, Kru,sas. Collected, analyzed, and presented da1a concerning the development of an observational system for intern teacher/trainer performance-based assessment. Teaching and Academic Appointments 2002 Lecrorer. Depanment ofPsycbology and Research in Educarion, University of Kansas. Teaching PRE 816 Evaluation of School Programs, a graduate-level evaluation comse for doctoral students. j  1993-1995 Assistant Professor. Department of OT Education. University of Kansas Medical Center. Tenure-line faculty appointment in the o,aduate School Concurrent appointment as J\nlirector of Research. Taught research methods course, graduate s,aristics and research seminar, graduate professional development ge,ninar\nprovided thesis advising to gtudents and contnDuted to the development of graduate progrmtl policies and curriculum. Mentored faculty and students in applied social science and behavioral research. o,airperson of Program Evaluation Corornittee. W. P. Moore (Youth Policy Research Group, inc.) 6 RfQ23-01 a-Response of Youth Policy Research Group. Inc Coordinator of school-wide faculty Development Education semmars. Elected to School of Allied Health Research Committee. 1993-1995 B,,search Assi-t ProfesSO'- University of l(ansaS eancer Center, University of Kansas Medical Center. Honornr)' appointroent ,eeogrn,:ing contnDution to research and scholarly activitl' ContnDute consulting and coJlaborarive service to research projects- 1991-1992 J.se'U\"\" I)epartro\"nt of Educational Psychology-and Rese\u0026amp;ch. University of Kansas. Courses taught included EPR 720 a graduate level classroom assessment course\nEPR 790 Research and EvaJuarion Proposal Development\nand EPR 710 a graduate level statistics course. 1988-89 g,.,,uate Teaching As~ Department ofEducational Psychology and Research, University of Kansas- Complete responsibility for teaching two sections of EPR 3 JO, a first year graduate level educational assessment and measurement c\\asS. 1984-86 Social Sciences m=ctor and De artmen' Actin\" C  erson. Turner High School, Tumer Unified School District, 1(ansaS City, l(ansaS- Taught sections of psychology, sociology, political science, and history. Assutned acting cl,airperson pos\\tion in Spring of 1986    1981-82 fu,cial Sci\"\"'es ]I,@!!Ctor. parsons Junior High SciJool, Parsons Unified School District, parsons, Kansas- Taught ,ections of American rnstotY and introductory Jaw. -- .... .... -rt   ~ t8 ~ ~.. Assessment and Testing I 998-2002 Drr\u0026lt;Ct% Educational Resemch and Assessroent Kansas City, Kansas Public Schools, K,,nsas City, l(ansaS- l see above for description of responsibilities)  t 996-1997 n_rrecrgr, Dep,rttnent of Educational Research and 1mproveroont, Kansas Ci,y, Missouri School District, Kansas City, J,!issouri (see above for description of responsibilities)- 1988-1989 filsistmt to T estjng Coordina19\u0026lt; Center for Educational Testing and Evaluation, K,,nsas J,,linimum eorrrpetency Testing Program, University of l(ansaS, 1..awrence, K,,nsasResponsible for assisting the coordinator in the dailY operation and coordination of a state-wide legislative mandated basic skills testing program. Developed system for rtern banking 3.000 test rterns\ntest paste-up and consrruction\nreviewins printing drafts, a\u0026lt;Jroinistering pilot srod\nes of items\nassisting with test equating and srandard setting activities.    ' -'.. .. .. ~ ~.. ~ ~ ~ ~.... ~ ~ '8 ~ .~. a..t W. P. Moore (Youth Policy Research Group, inc.} - RFQ23-01 O-Response of Youth Policy Research Group, inc. 1987 Test Item Writer. Center for Educational Testing and Evaluation, University of Kansas, Lawrence, Kansas. Developed original and revised existing multiple choice mathematics and reading items for inclusion on the 1988 K.MCT tests (grades 2, 4, 6, 8, and 10). 1986-1987 Testing Research Assistant. Center for Educational Testing and Evaluation, Kansas Minimum Competency Testing Program, University of Kansas, Lawrence, Kansas. Assisted in test construction., item tracking, and general test development activities and research regarding the KCMT. Publications Educational Psychology Gambone, M.A., Klem, A., Moore, W. P., and Summers, J. A. (2002). First Things First: Creating the conditions \u0026amp; capacity for community-wide reform in an urban school district. Ewing Marion Kauffman Foundation: Kansas City, MO  Gambone, M.A., Klem, A., Moore, W. P., and Summers, J. A. (2002). Executive Summary of First Things First: Creating the conditions \u0026amp; capacity for community-wide reform in an urban school district. Ewing Marion Kauffman Foundation, Kansas City, MO. Moranetz C.A., Hammig B.J., Moore W.P., and Coleman-Henderson K. Violence and Teens: Characteristics of High Risk Youth, Submitted to the The American Journal of Health Behavior. Moore W.P., Moranetz C.A., Owens M.M., Wordlaw-Watkins S.D., Parmet D.B., Enenbach M.J., Arroyo E.J., Hanna M.T., and Hammig B.J. The Effectiveness of a Dramatic HWIAIDS Education Program on the .Knowledge, Attitudes, Behavioral Intentions and Perceived Personal Risk of Urban Adolescents, Submitted to the Journal of Adolescent Health Moranetz C.A., Hammig B.J., Moore W.P., Turkman D.F., Miller J.W., Stebbins D.R., Brown C.M., and Mosier M.C. The Community Health Project: An Example of Effective Service Learning at the University of Kansas, Submitted to Academic 1\n1:edicine. Moore, W. P. (1996). Successful grantspersonship: Achieving the three clarities. Ewing Marion Kau:ffi.:nan Foundation, Kansas City, MO  Brown, C., Moore, W. P., Heroman, D., \u0026amp; Yunek, A. (1996). Influence of instrumental activities of daily living assessment method on judgments of independence. The American Occupational Therapy Journal_ 50(3), 202-206. Moore, W. P. (1995). Action research: A primer for teacher professional development. Ewing Marion Kauffman Foundation, Kansas City, MO. 13 I  I  I ~ '_,  ~ ' -' -' ---- ' W. P. Moore (Youth Policy Research Group, Inc.) 8 RFQ23-0J(}-Response of Youth Policy Research Group. Inc Moore, W. P. (1994). Toe devaluation of standardized testing: One district's response to a mandated assessment. Applied Measurement in Education, 7(4), 343-367. Moore, W. P. (1994). Appropriate test preparation: Can we reach a consensus? Educational Assessment, 2(1), 51-68. Miller, M. D. \u0026amp; Moore, W. P. (1991). Private-public school differences in the United States: Findings from the Second International Mathematics Study. International Journal of Educational Research, 15 (5), 433-444. Moore, W. P. Relationships among teacher test performance pressures, perceived testing benefits, test preparation strategies, and student test performance (Doctoral dissertation, University of Kansas, 1991 ). Dissertation Abstracts Internationale. Health Research and Psychology Beisecker, A. E., Murden, R., Moore, W. P., Graham, D., Helmig, L. (1996) Attitudes of medical students and primary care physicians regarding input of older and younger patients in medical decisions. Medical Care, 34(2), 126-13 7. Fabian, C. J., Kamel, S., McKittrick, R., Zalles, C., Simon, C., l(imler, B. F., Zeiger, S., Moore, W. P., and Chin, T. (1995). Breast tissue biomarkers in women at high and low risk for breast cancer. Proceedings of Cancer Symposium. Wayne State University: Kluwer Publishers. Beisecker, A. E., Helmig, L., Graham, D., and Moore, W. P. (1994)._Attitudes of oncologists, oncology nurses, and patients regarding medical decision-making by older and younger breast cancer patients. The Gerontologist, 34, 505-512 Fabian, C. J., Zalles, C., Kamel, S., IGmler, B. F., McKittrick, R., Trainin, A. S., Zieger, S., Moore, W. P., Hassanein, R.S., Simon, C., Johnson, N., Vergara, B., Jewell, W.R., Lin, F., Bhatia, P., and Chin, T. (1994). Prevalence of aneuploidy, overexpressed ER and EGFR and dysplastic cytology in random breast aspirates of women at high and low risk for breast cancer. Breast Cancer Research and Treatment, 30, 263-274  Beisecker, A. E. \u0026amp; Moore, W. P. (1994). Oncologists' perceptions of the effects of cancer patients' companions on physician-patient interactions. Journal of Psychosocial Oncology, 12 (1/2), 23-39. Fabian, C. J., Zalles, C., Kamel, S., McKittrick. R., Moore, W. P., Zeiger. S  Simon, C., IGmler. B., Cramer, A., Garcia, F., and Jewell, W. (1993). Biomarker and cytologic abnormalities in women at high and low risk for breast cancer. Journal of Cellular Biochemistry, J 7G, 153- 160. ~ I ' '  '  ' ' ' ' ' ' ' ' ' -' --' -\" ' ' --'  W. P. Moore() owh Policy Research Group. Inc.} 9 RFQ23-0J(}-Response of Youth Policy Research Group. inc. Beisecker, A. E. \u0026amp; Moore, W. P. (1992). Physician-cancer patient-companion interactions. Proceedings from the first annual meering of Psychosocial Oncology: Enhancing Parient and Family Care, Beverly Hills. Research, Evaluation and Planning Reports Moore, W. P. and Husman, M. (in press). Constructing youth engagement: A synthesis of literature, observation and professional opinion. Prepared under contract to Kansas City Young Audiences by Youth Policy Research Group, Inc.: Kansas City, KS. Moore, W. P. and Gramlich, A. (Sep. 2002). Evaluation of the impact of the school to enrrepreneurship program: Findings from the 2001-2002 implementation. Prepared under contract to the Ewing Marion Kauffman Foundation by Youth Policy Research Group, Inc.: Kansas City, KS. Gambone, M. A, Klem, A., Moore, W. P., and Summers, J. A. (2002). First Things First: Creating the conditions \u0026amp; capaciry for community-wide reform in an urban school district. Gambone \u0026amp; Associates: Philadelphia: PA. Gambone, M. A, Klem, A., Moore, W. P., and Summers, J. A. (2002). Executive Summary of First Things First: Creating the conditions \u0026amp; capaciry for community-wide reform in an urban school district. Gambone \u0026amp; Associates: Philadelphia: PA. Moore, W. P., Anthony, J., and Husman, M. (Nov. 2001). Evaluation of the TeacherArtist Residency Professional Development Model. Great Plains Research and Evaluation: Olathe, KS. Moore, W. P. (June, 2001) Effectiveness of the Young Audiences' Artist Training Conference. Great Plains Research and Evaluation: Olathe, KS. Moranetz, C., Moore, W. P., \u0026amp; Coleman-Henderson, K. (Dec, 1999). Evaluation of a Violence Prevention Program for High Risk Youth Phase One: Profiling Participants in Project SAFE. Great Plains Research and Evaluation: Olathe, KS. Moore, W. P. (1999). Learning about evaluation: Who are the stakeholders? Research Brief 4, Department of Educational Research and Assessment, Kansas City, Kansas Public Schools. Moore, W. P. (1999). Summary of parent and staff support for the Early Release Program, Research Brief, 2, Department of Educational Research and Assessment, Kansas City, Kansas Public Schools. Moore, W. P. (1999). School leadership mobility in the Kansas City, Kansas Public Schools. KCKPS Research Brief 1, Department of Educational Research and Assessment. Kansas City, Kansas Public Schools. 15 I I   I '    ' ' ' ' ' ' '  -    W. P. Moore (Yauch Policy Research Group, Inc.) JO RFQ23-0JO-Response of Youth Policy Research Group, Inc. Moore, W. P., Wright, D., \u0026amp; Cantwell, D. (1999). Assessment 2000: A Plan to create a world class assessment system in the Kansas City, Kansas Public Schools. Kansas City, Kansas Public Schools. Moore, W. P., Wright, D., \u0026amp; Cronister, J. (1998). 1997-1998 Student perceptions of instructional environment: District summary. Kansas City, KS: Department of Educational Research and Assessment, Kansas City, KS Public Schools. Richtermeyer, G., Moore, W. P., \u0026amp; Jones, R. L. (1998). Project Choice: A summary of parent perceptions five years later. Kansas City, MO: Ewing Marion Kauffman Foundation. Moore, W. P., Helliner, T., fomas, L., Jones, R., \u0026amp; Gonzalez, S. (1998). Public achievement in Kansas City: Evaluation of the first year of implementation. Kansas City, MO: Ewing Marion Kauffman Foundation. Moore, W. P., Jianas, L., Helliner, T., \u0026amp; Jones, R. (1998). The scholarship initiative: Synthesis of findings from phases 1 and 2 of the implementation evaluation. Kansas City, MO: Ewing Marion Kauffman Foundation. Moore, W. P., Helliner, T., Jianas, L., Jones, R., \u0026amp; Robinson, R. (1998). Evaluation of the first year of implementation of Public Achievement in Kansas City: Mid-course findings  Kansas City, MO: Ewmg Marion Kauffman Foundation. Moore, W. P. (1998). Long-term planning for the evaluation of Arts Partners programming in schools. Olathe, KS: Great Plains Research and Evaluation. Moore, W. P., Richtermeyer, G., Helliner, T., \u0026amp; Jones, R. (1998). Professional development for urban principals: Findings from a needs assessment of Kansas City principals. Kansas City, MO: Ewing Marion Kauffman Foundation. Moore, W. P. \u0026amp; Jones, R. (1998). Evaluation of the third annual Successful Schools network meeting. Kansas City, MO: Ewing Marion Kauffman Foundation. Moore, W.P. \u0026amp; Jones, R. (1998). Evaluation of the second Missouri Superintendent's Forum. Kansas City, MO: Ewing Marion Kauffman Foundation. Moore, W. P. \u0026amp; Jones, R. (1998) Evaluation of the second First Things First educational roundtable for school-site reform . Kansas City, MO: Ewing Marion Kauffman Foundation. Moore, W. P. \u0026amp; Jones, R. (1998). Evaluation of the Missouri Superinrendent's Forum. Kansas City. MO: Ewing Marion Kauffman Foundation  16  '  '' '  '' ' ' ' ' ' ' ' ' ' ' ' ' ' W. P. Moore (fouih Policy Research Group, 1 nc.J JI RFQ:!3-0JO-Response of Youih Policy Research Group. Jn:: Moore, W. P., Jones, R. (1997). Evaluarion of the First Things First Educational Roundtable for Schaal-Sire Reform. ].(ansas City, MO: Ewing Marion Kauffman F oundarion Daniels, S. D., fomas, L., \u0026amp; Moore, W. P. (1997). Sustainability study of youth development granrees. Kansas City, MO: Ewing Marion Kauffman Foundation. Moore, W. P. (1997). Teacher perceprions of school leadership: Decision making support informarion. Kansas City, MO: Kansas City. Missouri School District. Moore, W. P. ( 1997). College prepatory education' Results of observations at Lincoln College P,eparatory Academy. ].(ansas CIT)'. MO: ].(ansas CIT)', Missouri School District. Moore, W. P. \u0026amp; )(ass. H.K. (1997). Evaluation of the educational resrructuring efforr in the Kansas City-St. Joseph Catholic Diocese elementary schools: Findings from the second year of evaluation. Kansas City, MO: Ewing Marion Kauffman Foundation. Moore, W. P. (1997). Intern.al audit of Iowa Tests of Basic Skills special testing to determine swdent eligibility for Admission to Lincoln College P,epartory: A report to the Superincendent. Kansas City, MO: Kansas City, MO School District. Moore, W. P. \u0026amp; Kass, H.K. (1996). Evaluarion of the implementation of the FYI Training Insritute. Kansas City, MO: Ewing Marion Kauffman Foundation  Moore, W. P. \u0026amp; Kass, H. K. (1996). Evaluation of the initial implemenJation af the Bug H U. G. S. educarion.al program. Shawnee Mission, KS: Shawnee Mission Medical Center Foundation.. Moore, W. P ., Kass, H.K., \u0026amp; Welch., K. (1996). Evaluation of the education.al rest,ucruring effort in the Kansas City-St. Joseph catholic diocese elementary schools: Findings from the first yew of evaluation. ].(ansas CIT)', MO: Ewing Marion Kauffman Foundation Moore, W. P. \u0026amp; Kass, H. K. ( 1996). Implementation evaluation and identification of our comes for selected music programs offered through Arts Partners. ].(ansas City, MO: )(ausaS City Young Audiences. Moore, W. P. \u0026amp; Kass, H.K. (1996). Evaluarion of the implementation of Arts Farmers prog,ams in the Fort Osage School District. Kansas CIT)', MO: J(ausaSCity Young Audiences. Moore, W. P. (1996). Summary of technical assistance ourcomesfor nation-wide substonce use projectsfimded by the Ewing Mwion Kauffman Foundation. Kansas City, MO, Ewing Marion Kauffman Foundation. Moore, W. P. \u0026amp; Welch. K. (1996). Evaluarion report for the 3-on-3 youth ourreach prog,am Context, implementation and preliminary outcomes. ].(ansas City. MO: Emlg Marion Kau:ffinan Foundation. I   ' -~ '' ''' ' ' ' ' ' ' ' ' ' ' ' ' ' ' W. P. Moore (Youth Policy Researcn Group. inc.) !2 RFQ23-0JO-Response of Youth Policy Research Group. Inc Moranetz., C. A., Moore, W. P. Hutcheson, D. \u0026amp; Wendt, K.. J. (1996). Summary of impact of the Dramaric AIDS Education Project at catholic high sclwols: Technical Report. Kansas City, KS: University ofl(ansas Medical Center. Moore, W. P. \u0026amp; Kass, H. ( I 996). Evaluation report for Baseline Training in the midwest region of the United States: Implementation, rechnicol specifications, and perceived outcomes. Kansas City, MO: Ewing Marion K.auffinan Foundation. Moore, W. P., Welch, K.., \u0026amp; Kass, H. (1995). Final evaluation report for the STAR continuarion grants program. I(ansas City, MO: Ewing Marion Kauffman Foundation. Moore, W. P. \u0026amp; Welch, K. (1995). Evaluation report for the Ewing Marion Kauffino-n Foundation's Project ST AR continuation granrs program: Unintended outcomes and future role off oundation in substance abuse prevention. Kansas City, MO: Ewing Marion Kanffinan Foundation. Moore, W. P. (1994). Evaluarion of the Mind's Edge arts integration program: An examination of the process and products. Kansas City, MO: Kansas City Young Audiences. Atwater, J. \u0026amp; Moore, W. P. (1992). Achievement and enrollment evaluation of the Applied Learning ElementarY Magnet Schools, I 990-I 99 I  Kansas City, MO: Kansas City  Missouri School District, Evaluation Office. Moore, W. P. (1992). Achievement and enrollment evaluation of the Faxon Monressori Elemenrary Magnet Sclwol Program, 1991-/992. Kansas City, MO: Kansas City, Missouri School District, Evaluation Office. Moore, W. P. (1992). Achievement and enrollment evaluation of the Science/Math Middle Magnet Sclwols Program. I 991-199 2. Kansas City, MO: Kansas City, Missouri School District, Evaluation Office. Moore, W. P. (1992). AchievemenI and enrollment evaluation of the Southwest High School Math/Science Magnet Program, I 991-1992. Kansas City, MO: Kansas City. Missnuri School District. Evaluation Office. Moore, W. P. (1992). Summarive evaluation of the Applied Learning Elemenrary Magnet Sclwols Program. I 99 J-199 2. Kansas City. MO: Kansas City. Mi,souri School D\n,\nrrict. Evaluation Office. Moore, W. P. (1992). Summarive evaluation of the Science and Mathematics Magner Elemenrary Sclwols. Longiwdinal Report, I 989-1990. Kansas City. MO: Kansas City. Missouri School District.. Evaluation Office. 18 I  ~ - ' '  '  ' ' '' '''''' 1ft -~ ~ Ir P. Moore (T'oucn Poucy Research Group  .1 n:. J 3 RFQ23-0I (}-Response of J'ouih Policy Research Group. in::. Moore, W. P. (1992). Summative evaluation of the lnvesrigarive Learning Elementary Magnet Schools, 1991-1992. Kansas City, MO: KansaS City. Missouri School District, Evaluation Office. Moore, W. P. (1992). Formative evaluation of the Holliday Monressori Elementary Magnet School Program, 1991-1992. J(\n,nsas City, MO: Kansas City, Missouri School District, Evaluation Office. Moore, W. P. (1992). Formative evaluation of the Southeast Health Professions Magnet High School, 1 991-199 2. J(\n,nsas City, M 0: J(\n,nsas City, Missouri School Disrrict. Evaluation Office. Moore, W. P. (1991). Achievement and enrollment evaluation of the Investigative Learning Elemenrary Magnet Schools, 1990-1991. I(ansas City, MO: 1(ansas City, Missouri School District, Evaluation Office. Moore, W. P. (1991). Achievement and enrollment evaluation of theScience/Math Middle Magnet Schools, 1990-1991. I(ansas City, MO: l(ansaS City, Missouri School District, Evaluation Office. Moore, W. P. (1991). Formative evaluation of the Holliday Montessori Elementary Magnet Schaal, 1990-1991. Kansas City, MO: K\n,nsas City, Missouri School Disrrict, Evaluation Office. Moore, W. P. (1991). Formative evaluation of the Science and Mathematics Magne1 Elementary School Program, 1990-1991. Kansas City. MO: l(ansaS City, Missouri School District, Evaluation Office. Moore, W. P. (1991). Formative evaluation of the Southeast Health Professions Magnet Wgh Schaal, 1990-1991. Kansas City, MO: K\n,nsas City, Missouri Schoo 1 Disrrict, Evaluation Office. Moore, W. P. (1991). Summative evaluation of the Fax.on Montessori Magnet School Program, Longicudinal Report, 1989-1991. Kansas City, MO: Kansas City, Missouri School District, Evaluation Office. Moore, W. P. (1991 ). Summative evaluation of the Southwest Science and Mathemacics Magnet High School Prag.-am, Longitudinal Report, 1989-1991. K\n,nsas City, MO: Kansas City. Missouri School District, Evaluation Office. Moore, W. P. (1990). AchievemenI and enrollment evaluation of the Jnvescigarive Learning Elemenrary Magnet Schools, 1989-1990. Kansas City. MO: Kansas City. Missouri School District, Evaluation Office. 19 I   ' ' ' '  '  '  '    W. P. Moore (iourh Policy Research Group, Jnc.j /4 RFQ23-0JO-Response of Youth Policy Research Group, Jnc. Moore, W. P. (1990). Formative evaluation of the New Faxon Montessori Magnet School, 1989-1990. Kansas City, MO: Kansas City, Missouri School District, Evaluation Office. Moore, W. P. (1990). Formative evaluation of the First Year Science and Mathematics Magnet Elementary Schools, 1989-1990. Kansas City, MO: Kansas City, Missouri School District, Evaluation Office. Moore, W. P. (1990). Formative evaluation of the Southwest Science and Mathematics Magnet High School, 1989-1990. Kansas City, MO: Kansas City, Missouri School District, Evaluation Office. Moore, W. P. (1990). Mid-year formative evaluation of the First Year Science and Mathematics Elementary Magnet Schools, 1989-1990. Kansas City, MO: Kansas City, Missouri School District, Evaluation Office. Moore, W. P. (1990). Mid-year formative evaluation of the Science/Math Middle Magnet Schools, Longitudinal Report, 1987-1988, i988-1989, 1989-1990. K,,nsas City, MO Kansas City, Missouri School District, Evaluation Office. Moore, W. P. (1990). Summative evaluation of the Science/Math Middl.e Magnet Schools: Longitudinal Report, 1987-1988, 1988-1989, 1989-1990. Kansas City, MO: Kansas City, Missouri School District, Evaluation Office  Moore, W. P. (1988). Evaluation report on the Artist-in-Residence Program. Lawrence, KS: University of Kansas, Department of Art Education. Published Abstracts Wendt K.J., Moranetz C.A., Wallace D.D .. HoffG.L., Moore W.P., Hamm.R.H., Bindman A.B., Lehman J.S. Deterrents to HIV Testing and Care: An Analysis of the HIV Referral Syscem in Missouri, Presented at 125th Annual American Public Health Association Meeting. Indianapolis. Indiana. November 1997  Wendt K.L Moranetz C.A., and Moore W.P. Treatment Group Composition Effects in A HIV Education Program For Teenage Youth. Presented at 18th Annual Scientic Sessions of The Society of Behavioral Medicine. San Francisco, California., April 1997  Moore, W.P .. Moranetz C.A., and Wendt K.J. , Artitudes and Predicrors of HIV Risk Reduction Behavior in Carholic Teenagers. Presented at 26th International Congress of Psychology. Monrreal. Canada. August 1996. 20 I I ' ' ~ ' -~ \\ ' ' '' ' ' ' ' -' ' ' -'  W. ?. Moore (Youth POiicy Researcn Group, inc.J ,J RFQ23-0JO-Response o_( l'outh Policy Research Group, In=. Conference Presentations and Session Chairs Wright. D ., Moore, W. P ., and Heiro=. J. (2002, Apr.). Whom did we miss and why? Character\nstics of non-participating general education srudents in mandated achievement assessments. Paper presented at the annual meeting of the American Educational Research Association, ew Orleans, LA. Moranetz, C. A., Hamming, B. J., Moore, W. P., Mosier, M. M., Turla:nani, D. F., Young. B., \u0026amp; Anthony, A. (2000, Apr.). Service learning Merging medicine and public health. The Community Healrh Project at the University af Kansas. Paper presented at the annual meeting oftbe Association for Teachers of Preventive Medicine, Atlanta. GA. Coleman-Henderson K., Moranetz C.A., Moore W.P. (2000, Apr.). Violence and Teens Characteristics of High-Risk Youth, Srudent National Medical A.,sociarion 35 Annual Medical Education Conference, Los Angeles, CA. Wendt, JU., Moranetz, C. A., Wallace, D. D., Hoff, G. L., Moore, W. P., Har=, RH  B-A. B., and Lebman, J. S. ( l 997, Nov.). Deterrents to HIV testing and carec An analysis of the HIV referral system in Missouri. Paper presented at the annual meeting of the American Public Health Association, Indianapolis, IN. Moore, W. P. (1997, Apr.). Using multiple measures in evaluation. Chairpenon for conference session in Division R Research on Evaluation Special Interest Group. Session delivered at the annual rneerinll of the American Educational Research Association, Chicago, IL. Wendt, IC, Moranetz., C. A, \u0026amp; Moore, W. P. (1997, Apr.). Treatment Group Composition Effects in A HW Education Program For Teenage Youth. Paper pre-ed at the annual meeting oftbe Society for Behavioral Medicm.e, Washington, DC. Moore, W .P ., Moranetz C.A., and Wendt K.l. ( 1996, Aug.). Attitudes and Predicrors of HIV Risk Reduction Behavior in Catholic Teenagers, Presented at the 26th International Congress of Psychology, Montreal.\nCanada- Moore, W. P. \u0026amp; Mercer, C. (1995, Apr.). The role of teachers in the development of the national educational research agenda Is ar,yone in Washington listening? Paper presented at the I 995 annual meeting of the AJDerican Educational Research Association, San Francisco. Moranetz, C. A. \u0026amp; Moore, W. P., Owens, M., \u0026amp; Wordlaw, S. (1995, Apr.). A collaborative AIDS education model for changing knowledge and behavioral intentions of adnlescenrs. Paper presented at the I 995 annual meeting of the Society of Behavioral Medicllle. San Diego. 21 W. P. Moore () 'owh Pnlicy Research Group, inc./ /6 RFQ23-0JO-Response of YouJh Policy Research Group, Inc Moranetz., C. A., Moore, W. P., Owens, M., \u0026amp; Wordlaw, S. (1995, Mar.). The impact of a dramatic AIDS education program on knowledge and behavioral intentions of teenagers. Paper presented at the 1995 annual meeting of the Association for Teachers of Preventive Medicine, New Orleans, LA. Moranetz, C. A., Moore, W. P., Owens, M. M., \u0026amp; Wordlaw, S. D. (1994, Nov.). AIDS education and prevention: A medicine and drama model for changing knowledge and behavioral intentions of adolescents. Paper presented at the annual University of Kansas Medical Center Faculty Research Day, Kansas City, KS. Kamel. S., Fabian, C., Zalles, C., McKittrick, R., Kimler, B., Zeiger, S., Simon, C., Moore, W. (1994, Nov.). Correlation of breast tissue biomarkers with hyperplasia and dysplasia in fine needle aspirates (FNA's) of women at high and low riskfor breast cancer. Paper presented at the annual Faculty Research Day at the University of Kansas Medical Center. Kansas City, KS. Moranetz, C. A., \u0026amp; Moore, W. P. (1994, July). The dramatic AIDS education project: The impact of an innovative program on the knowledge, attitudes and behavior of teenagers. Paper presented at the 19th annual meeting of the National Wellness Conference, Stevens Point, Wl. Scott, D. A., \u0026amp; Moore, W. P. (1994, Jun.). Predicting academic success and failure. Paper presented at the annual meeting of the Commission on Education, Boston. Moore, W. P., Moranetz, C. A., Hanna, M. T., \u0026amp; Arroyo, E. Y. (1994. Jun.). Effects of a dramatic education intervention on AIDS-related knowledge, attitudes, and behavioral intentions. Paper presented at the sixth annual meeting of the American Psychological Society. Washington. DC. Moore, W. P. \u0026amp; Esselman., M. E. (1994, Apr.). Exploring the context of teacher efficacy: The role of achievement and school climate. Paper presented at-the annual meeting of the American Educational Research Association, New Orleans. Fabian. C., Zalles, C., Kamel. S., McKittrick. R., Kimler, B., Zeiger. S., Simon, C  Moore, W. (1994, Apr). Correlation of breasr tissue biomarkers with hyperplasia and dysplasia in fine needle aspirates (FNA's) of women at high and low riskfor breast cancer. Paper presented at the 85th annual meeting of the American Association for Cancer Research, San Francisco. Moranetz. C. A., Arroyo, E. Y .. Hanna. M. T .. \u0026amp; Moore, W. P. (1994. Feb). The implementation and evaluation of an innovative AIDS education program. Paper presented at the fifth annual meeting of the American Journal of Health Promotion research conference. Colorado Springs. CO. 22   ' ' ~ ' ' ' ~ ' ' ' ' ' '\\ '.. '''\" -.. W. P. Moore (Youth Policy Research Group, Inc.) Ji RFQ23-0JO-Response of Yourh Policy Research Group. Jnc. Beisecker, A. E. \u0026amp; Moore, W. P. (1993, Nov.). Attirudes regarding desire for information and patient input in medical decisions for breast cancer. Paper presented a! the annual meeting of the American Public Health Association, San Francisco. Beisecker, A. E. \u0026amp; Moore, W. P. ( l 993, Sept.). The effects of patient age on attitudes of oncologists. oncology nurses and female patients regarding patient input in medical decisions for breast cancer. Paper presented at the snnual meeting of the American Association for Cancer Research, San Diego. Moranetz, C. A., Moore, W. P., Arroyo, E. Y., \u0026amp; Hanna, M. T. (1993, Nov.). The impact on teenagers of an innovafive AIDS education program: Results from the first pilot year Paper preseoted at the University of Kansas Medical Center annual faculty research day, Ki,nsas City, KS. .  Moore, W. P. \u0026amp; Mercer, C. ( l 993, Nov.). Identifying research prioriries in education: Results from a national sample of university educarors. Paper presented at the snnual F acuity Research Day, University ofl(ansas Medical Center, Kansas City, KS. Moore, W. P. ( l 993, Apr.). Preparation of swdents for testing: Teacher differentiation of appropriare and inappropriate practices. Paper presented at the snnual meeting of the National Council on Measurement in Education, Atlanta- Beisecker, A. E. \u0026amp; Moore, W. P. ( l 992, Nov.). Attiwdes of oncologists and oncology nurses regarding medical decision-making by older and younger breast cancer patients: Implications for provider-patient communication. Paper presented at the snnual meeting of the American Public Health Association, Washington, D. C. BeIBecker, A. E. \u0026amp; Moore, W. P. (1992, Nov.). Attitudes of medical sn,dents and primary care physicians regarding input of older and younger patients in medical decisions. Paper presented at the annual meeting of the Gerontological Society of America, wasmngton, D c. Moon,, W. P. \u0026amp; Beisecker, A. E. (l 992, Nov.). Oncologist's perceptions of physician cancer patient-companion interactions. Poster presented at the University of Kansas Medical Center Faculty Research Day, Kansas City, KS. Beisecker, A. E. \u0026amp; Moore, W. P. (1992, Sept.). Physician-cancer patient-companion interactions. Presented at the first Psychosocwl Oncology: Enhancing PatiOnt and Family Care Conference, Beverly Hills, CA. Essebnan, M. E. \u0026amp; Moore, W. P. ( 1992, Apr.). In search of organiiationnl variables which can be altered to promote an increased sense of efficacy. Paper presented at the American Educational Research Association annual meeting, San Francisco. 23 W. P. Moore ffouch Policy Research Group. Inc.) 18 RFQ23-0JO-Response of Yauch Policy Research Group, Jnc. Moore, W. P. (1992, Apr.). Testing perceptions, practices, and malpractice: The impacr on teachers of court-ordered achievement testing in a desegregation setting. Paper presented at the National Council on Measurement in Education annual meeting, San Francisco. Moore, W. P. \u0026amp; Esse~ M. E. (1992, Apr.). Teacher efficacy, empowerment, and a focused instructional climate: Does student achievement benefit? Paper presented at the American Educational Research Association annual meeting, San Francisco. Moore, W. P. (1990, Apr.). Ethnic group differential mathematics achievement: Some findings from the Second International Mathematics Study. Paper presented at the American Educational Research Association annual meeting, Boston. Moore, W. P. \u0026amp; Miller, M. D. (1990, Apr.). Differential insrructional environments in public and private school classrooms: Some findings from the Second International Mathematics Study. Paper presented at the American Educational Research Association annual meeting, Boston. Moore, W. P. \u0026amp; Peckover, R. (1989, Nov.). Integrating the phases of item management utilizing Apple's HyperCard. Poster presented at the annual meeting of the Association for Psychological and Educational Research in Kansas, Pittsburg, KS. Moore, W. P. \u0026amp; Hsia, S. (1989, Nov.). Qualitatively better education in private school classrooms? Some findings from the Second International Mathematics Study. Paper presented at the annual meeting of the Association of Psychological and Educational Research in Kansas, Pittsburg, KS. Hsia. S. \u0026amp; Moore, W. P. (1989, Nov.). Factors associated with minority group achievement: Some findings from the Second International Mathematics Study. Paper presented at the annual meeting of the Association for Psychological and Educational Research in Kansas. Pittsburg, KS.  Miller, M. D. \u0026amp; Moore, W. P. (1988, Apr.). Private-public achievement differences in the Second International Mathematics Study. Paper presented at the annual meeting of the American Educational Research Association, ew Orleans. Salkind, N. \u0026amp; Moore, W. P. (1988, Apr.). Leaming about children and families. Paper presented at the meeting of the Southwestern Society for Research in Human Development. New Orleans. Moore, W. P. (1 987. ov.). Teacher observation under a microscope: A iechniquefor evaluating the effectiveness of beginning teachers. Paper presented at the annual meeting of the Association for Psychological and Educational Research in Kansas- Lawrence, KS. lnvited Addresses, Seminars, and Training Presentations I ~  ' '' '''''' ' ' ' ' -''' .. '..  '  '.. W. P Moore(} ouih Polzet Researcn Grou_o. 1nc.1 , ~ RFQ23-010---Response of fown Policy Researcn Group, Jru: Moore, W. P. and Gambone. M.A. (2002. Aug.). A theory ofchange approach to evaluating community-wide reform. Invited presentation to the Los Angeles. California County Commissioners meeting, Long Beach, CA. Gambone, M. A. and Moore, W. P. (200'.?.. Feb.). A theory of change approach to evaluating education system reform. Invited presentation at The Aspen Institute's Roundtable on Comprehensive Community Initiatives for Children and Families. Glen Cove, Long Island, New York. Moore, W. P. (2001. Aug.). Re-casting the role of the arts in k-12 education: How the arts can contribute to the national educational reform dialogue . Invited panel presentation at the Annual National Conference of Young Audiences Organizations. Kansas City, MO. Moore, W. P. (1998, Aug.). Next steps for the evaluation of Public Achievement in Kansas City. Invited presentation to the Kansas City Partners of Public Achievement. Kansas City, MO. Moore, W. P. (1998, May). Understanding the process of benchmarking and establishing benchmarks w guide internal improvement efforts. Invited presentation to the Department of Research and Evaluation, Ewing Marion Kauffman Foundation. Independence. MO. Moore, W. P. (1998, Feb.). Re-casting the role of the arts in education: How evaluation can shape the local and national agenda. Invited panel presentation to the Young Audiences Board of Directors. Kansas City, MO. Moore, W. P. (1997, Jun.). Furure direcrionsfor the West-Cenrral Associated Schools: What do evaluation results tell us? Invited presentation to community and school stakeholders of the West-Central Associated Schools sponsored by the Ewing Marion Kauffman Foundation. Kansas City, MO. Moore, W. P. (1997, Apr.). Setting realistic and meaningful system and school goals. Invited kevnote address at the Kansas City, Missouri School District's District-wide administrators workshop on school improvement planning. Kansas City, MO. Moore, W. P. (1996, May). Consequences of mandared achievement testing: Resulrs from a desegregating school district. Invited presentation to College of Education faculty and staff, Auburn University. Auburn, AL. Moore, W. P. (1996, Apr.). Introduction to positive youth development through asset building in our communities and schools. Invited keynote address to the Advisory Board ofthe HelpNet Community Project. Overland Park. KS. 25 ' ~ ' -- \" \"-   -~ \"\" \" ~ -  -  W. P. Moore rTouzh Policy Research Group, h,c.) 2(J RFQ23-0JO-Response of You1h Policy Research Group. Inc. Moran.etz, C. A. \u0026amp; Moore, W. P. (1996, Apr.). Adolescent health behaviors: The good, the bad and the ugly. Invited keynote address to parents of O'Hara High School. Kansas City, MO. Moore, W. P. (1996, Apr.). Consequences of mandated achievement testing: Results from a desegregating school district. Invited presentation to College of Education faculty and staff, University of Wyoming. Laramie, WY. Moore, W. P. (1996, Mar.). Teacher leadership through action research. Invited keynote address to Park Hill School District Administrative Leadership. Kansas City, MO. Moore, W. P. (1996, Feb.). Preparing your grant application to increase the chances of funding. Invited professional development seminar to Baseline Cad.re members from Colorado, Kansas, and Missouri. Kansas City, MO. Moore, W. P. (1996, Feb.). Professional development through action research. Invited staff development seminar to Park Hill School District faculty and staff. Kansas City, MO. Moore, W.P. (1995, Dec.). Evaluation as a stimulant for creating a learning organization. Invited presentation to the Board of Directors of l(ansaS City Arts Partners. Kansas City, MO. Moore, W. P. (1995, Nov.). The role of evaluation in a learning organization. Invited presentation to the Board ofDirectors of Kansas City Young Audiences. Kansas City, MO. Moore, W. P. (1995, Nov.). Locatingfunders and.preparing grant applications. Invited staff development seminar to ST AR schools faculty. Sponsored by the Ewing Marion Kauffman Foundation, Kansas City, MO. Moore, W. P. (1995, Oct.). Achieving the three clarities of successful grantspersonship  Invited staff development seminar to Baseline program cad.re in Kansas, Missouri, and Colorado  Sponsored by the Ewing Marion Kauffman Foundation, 1(ansaS City, MO  Moore, W. P. (1995, Sept.). Designing client-focused program evaluacions. Invited staff development seminar sponsored by the University of Kansas Medical Center, Child Development Unit, Kansas City, KS. Moore, W. P. (1995, May). Outcomes-Based Evaluation. Invited staff development seminar to the staff of Kansas City Young Audiences. Kansas City, MO. Moore, W. P. (1995. Apr.). Using SPSS (Statiscical Package for the Social Sciences) in applied research. Invited staff development seminar to the faculty of the University of Kansas Department of Occupational Therapy Education. Kansas City. KS  26 I  ' ' ~ ~  .. .. ~ ~ ~ ..   ~ ~ ~ ~ ~ ~.-. W P. Moore (Youih Policy Research Group, Inc. ) ~l RFQ23-0JO--Respome of Youth Policy Research Group, Inc. Moore, W. P. (1995, Mar.). Improving your classroom-based assessments. Invited staff development seminar to the University of Kansas Department of Physical Therapy Education, Kansas City, KS. Moore, W. P., \u0026amp; Richards, L. (1994, Dec.). Assessing independent living. Invited presentation at the University of Kansas Medical Center OT Education conference: The Competent Professional: A Balance of Work-Play-Leisure, Kansas City, KS. Moore, W. P. \u0026amp; Moranetz, C. (1994, Aug.). An AIDS education program. Invited presentation to community and business leaders sponsored by the Child Abuse Prevention Coalition, Overland Park, KS. Moranetz, C. \u0026amp; Moore, W. P. (1994, Apr.). The implementation of an innovative AIDS education project. Invited presentation to the Kansas University Affiliated Programs Child Development Unit (CDU), Kansas City, KS. Moore, W. P. (1994, Apr.). How to improve your classroom-based assessments  Svmposium Chair for the University of Kansas School of Allied Health Faculty Development Education Seminar, Kansas City, KS. Moore, W. P. (1994, Mar.). Clinician as researcher. Invited address to the Kansas University Affiliated Program Child Development Unit (CDU), Kansas City, KS. Moore, W. P. (1994, Mar.). The clinician as researcher. Invited address to the Leadership in Occupational Therapy Service Systems (LOTSS) Group at the University of Kansas, Kansas City, KS . Moore, W. P. (1994, Mar.). Problem-based learning and critical thinking. Symposium chair for the University of Kansas School of Allied Health Faculty Development Education Seminars, Kansas City, KS.  Moore, W. P. (1994, Mar.). How children are assessed in schools. Invited address to community and business leaders sponsored by the Child Abuse Prevention Coalition, Overland Park, KS. Moore, W. P. (1994, Feb.). How to be creative and effective in the classroom  Svmposium Chair for the University of Kansas School of Allied Health Faculty Development Education Seminars. Kansas City, KS  Moore, W. P. (1993, Dec.). Developing educationally useful and defensible classroom tests. Invited training seminar to the faculty of the Department of Occupational Therapy Education, University of Kansas, Kansas City, KS. Moore, W. P. Clinician as researcher. (1993, Nov.). Invited kevnote address at the annual meeting of the Kansas Occupational Therapy Association, Topeka, KS. 27 ~ ' \\ ~ \\ ~ '  ' ~ ' ' '  ~ ~ W. P. Moore (Yourh Policy Research Group. j 11!:.} 2] RFQ23-0W-Response ofYourh Policy Research Group, Inc. Moore, W. P. \u0026amp; McCl~ L. (1993, Nov.). Understanding quantitative and qualitative research approaches. Invited presentation to the Leadership in Occupational Therapy Service Systems (LOTSS) Group at the University of Kansas, Kansas City, KS. Moore, W. P. \u0026amp; Esselman, W. P. (1992, Mar.). The influence of school climate on student achievement and teacher efficacy. Invited presenta1ion to Kansas City, Missouri School Di.strict Conflict Resolution subcommittee, l(an.sas City, MO. Moore, W. P. \u0026amp; Seever, M. (1992, Mar.). Large-scale testing: The impact on instrUction and school curriculum. Invited presentation to Area 2 Di.strict principals, Hartman Elementary, Kansas City, MO. Seever, M. \u0026amp; Moore, W. P. (1992, Jan.). How students learn: Implications for insrrucrional design. Invited presentation to Area 2 Di.strict principals, Hartman Elementary, Kansas City, MO. Moore, W. P. (1992, Mar.). Test performance and test preparation: How instruction and curriculum are influenced Invited presentation during spring staff development to administration and faculty at district elementary schools, West Rock Creek Elementary, Kansas City, MO. Moore, W. P. (1992, Mar.). Testing practices and preparation: How to benefit your students' test performance. Invited presentation to Swinney and Volker elementary school administration and faculty, Swinney Elementary, Kansas City, MO. Moore, W. P. (1992, Feb.). Testing programs and teacher practices. Invited presentation to Swinney and Volker elementary school School Advisory Committee, Swinney Elementary. Kansas City. MO. Moore, W. P. (1991, Aug.). Health professions educational programs in the Kansas Cicy, Missouri School Disrrict, A sta1US report. Invited kevm,te address to fi,oulty at the 1991 District Inservice Conferences. University of Missouri-Kansas City, Kansas City, MO. Moore, W. P. (1990, Aug.). A report on the status of mathematics and science education in the Kansas Ciry, Missouri School Disrrict. Invited kevnote address to di.strict-wide faculty at the 1990 Opening of School Conference, Kansas City, MO. Externally Funded Research \u0026amp; Training The Evaluation of the Kansas Superintendent's Forum. Principal Investigator: W. Moore. Total CostS: 96.157.25. Funded by the Kansas Health Foundation.. November 2002-December. 2004. 28 W P. Moore (Youth Policy Research Group, Inc.) 23 RFQ23-0!0-Response of Yau.th Policy Research Group, Inc. Evaluation of the Impact of the School to Entrepreneurship Program. Principal InveStigator: W. Moore. Total Costs: 9750.00. Funded by the Ewing Marion Kauffman Foundation., July 2002- ovember, 2002. The Dramatic AIDS-Education Project: A program for extreme risk youth. Co-principal Investigators: J. Pelster, C. Moranetz, W. Moore. Total Costs: 18,000. Funded by Heart of America Community Foundation for AIDS and Design Industries Foundation Fighting AIDS, October, 1995-September, 1996. Development of a Consolidated Interdisciplinary Model for Providing Inclusive Education and Related Services for Children and Youth with Disabilities. Co-Principal Investigators: W. -Dunn and W. Moore, Dept. of OT Education. US Department of Education. Total Costs: 157,962. Application Approved, May, 1995. Institute on Statistical Analysis for Education Policy. American Educational Research Association Grants Program for the Enhancement of Education Statistics, Mathematics, and Science Education., and the Educational Research Infrastructure. Funded. April, 1995. The Dramatic AIDS Education Project: Development of a Middle School Educational Intervention. Principal Investigator: C. Moranetz\nCo-Investigator: W. Moore. Joint submission with The Coterie Theatre. Total Costs: 34,000. Funded by The Kansas City Junior League, June, 1995-May, 1997. The Dramatic AIDS Education Project: Continuation for Year Two. Principal Investigator: C. Moranetz\nCo-Investigator: W. Moore. Joint submission with Toe Coterie Theatre. Total Costs 36,000. Funded by Twentieth Century Companies, AU:,crust, 1994-August 1995. Establishing Educational Priorities in Kansas. Principal Investigator: W. Moore, Dept. of OT Education\nCo-investigator: C. Mercer. Funded by the Kansas State Board of Education. Total Costs: 5,000. January 1993-September 1994. Establishing Educational Priorities in Kansas. Principal Investigator: W. Moore, Dept of OT Education. Funded by the University of Kansas, Graduate Studies and Faculty Research Grant, Total Costs: 1,000. April, 1994 and March, 1995 The Validity of JADL Assessments for Predicting Function in Natural Environments. Principal Investigator: W. Moore, Dept of OT Education\nCo-Investigator: C. Brown. Funded by the University of Kansas, School of Allied Health, Dean's Research Fund. Total Costs: 750.00. July 1993-June 1994. The Dramatic Aids Education Project. A joint project with Toe Coterie Theatre and the University of Kansas Medical Center. Principal Investigator: C. Moranetz, Associate Professor Preventive Medicine. Co-investigator: W. Moore. Total Costs: 23,000. Funded by Twentieth Century Companies, Inc. August 1993-July 1994. 29 I  ~ '   ..   \".... .... .... .... .... ..  W. P. Moore (fouth Policy esearcn RFQ23-0JO-Response of Youth Policy Research Group, inc. Honors, Awards, Elected and Appointed Positions Adv\u0026lt;\nory Council, CommunitJ' Health Promotion Project, University of Kansas Medical Center, 2A0d0v1i-sPorreys Benota rd, Sunflower House: A Child Abuse Prevention Ctr, 1998-Present National Panel of Writers, Joint Cornrrrinee on Standards for Educational Evaluation, Subcommittee to develop standards for student evaluation, Western Michigan University, 1998 Vice-President of Education, Sunflower House, A Child Abuse Prevention Ctr, 1998 Kansas City Ribbon of Hope Award, The Dramlictic AIDS Education Project, 1997 Board Secretary, Sunflower House, A Child Abuse Prevention Center, 1997 Board of Directors, Sunflower House, 1997-1998 Vice-President for Education, Child Abuse Prevention Coalition, 1996. \"Faculty Marshall, University of Kansas Commencement Ceremonies, 1995 Who's Who in American Education, 5th Edition. A. N. Marquis Publishers, 1996-1997 Univ. ofKansas School of Allied Health, Research Committee, Elected by Faculty, 1994 Univ. of Kansas Graduate Studies and Research faculty Travel Grant, 1994, 1995 Board of Directors, Child Abuse Prevention Coalition, 1994 Appointment to the KU Cancer Center as Honorarv Research AssisW\u0026gt;t Professor, 1993 Univ. of Kansas School of Allied Health, Dean's Research Award, Fall, 1993 Selected by faculty to Madeline Hunter Effective Tear.rung Cadre, 1985-1986 Kansas National Education Association-Turner, Building Representative, 1985-86 Professional Service National Reviewer. W. T. Grant Foundation., 2002-Present Member, Panel of Writers, The Joint Committee on Standmds for Educational E valuarilm  Western Michigan University., 1997-1998 National Succes.eful Schools planning T earn. 1997-1998 Editorial Board Member. Occupational Therapy Journal of Research, 1994-1998 Member. ational Research Development Committee. American Occupational Therapy Foundation. 1995-97 Grant Reviewer, U.S. Deparrment of Health and Human Services. Administration on Children- Y outb and Families, 1992-1998 Program Reviewer. American Psychological Sociery. 1992-1995 Manuscript Reviewer, Macmillan Publishing. Educational Psychology. 1991-1995 Manuscript Reviewer. Wadsworth Publishing. Educational Psychology. 1995-1996 30  t       .. .... .... .... .... ....  \" W. P Moore (You1h Policy Research Group. inc.) __ RFQ23-0JO--Response of }'ou1h Policy Research Group, Jr::. Regional and Local Kansas Ciry Merropolitan Task Force to Establish Local Standards for Evaluarion, 1998 Kansas Ciry Urban Principal's Leadership Planning Team. 1997-1998 Kansas Ciry Metropolitan Area Early Childhood Care and Education Planning Team. 1997-1998 Missouri Superintendent's Forum Planning Team, 1997-1998 Kansas Ciry, Missouri School-to Career Communiry Steering Commitree, 1997-1998 BE2 School-to-Career Functional Team Member, Benchmarking and Evaluation, 1997-2000 Kansas Ciry Public Achievement: Leadership Team member. 1998-2000 Dramatic AIDS Education Projecr: Co-Director, Research and Evaluation. 1993-1997 Consuhant, Research and Evaluation, 199'.?., 1998-Present Coalition for Positive Family Relationships: Education and Training Committee member, 1995-1997 Family Research Team Committee member, 1995-1997 Sunflower House: A Child Abuse Prevention Center Advisory Board Member, 1998-Present Vice-President of Education, 1997-1998 Executive Committee, 1997-1998 Strategic planning Crurirperson, 1997-1998 Board of Directors, 1997-1998 Education Committee member, 1997-199 Child Abuse Prevention Coalition: Board of Directors, 1994-1997 Sunflower Children's Advocacy Center/CAPC Merger Joint Task Force, 1995-96 Wyandotte County, Child Abuse Task Force, CAPC 1995-1996 Vice-President, Education, 1996 Executive Committee, 1996-1997 Executive Director Search Committee, 1994 Strategic Planning Chairperson\n1994-1997 Education Committee member, 1993-1997 HelpNET:A Communiry Response to Youth Development, Overland Park, KS, Co-Organizer, 1995-1996 Institutional Summer School Planning Committee, Member, Kansas City. Kansas Public Schools. 1998-2002 KCKPS and KNEA Instructional Improvement Initiative Planning Committee, 1999-2002 Overcrowded Schools Committee. Member. Kansas City, Kansas Public Schools, 1999-200'2 Superintendent's Assessment Advisory Committee. Chairperson. Kansas City. Kansas Public 31 W. P. Moore (YouJh Policy Research Group, Inc.) 26 R.FQ23-0IO-Response of Youth Policy Research Group, inc. Schools, 1998-2002 Kansas City, Kansas Public Schools District Data Team, Chairperson., Kansas City, Kansas Public Superintendent' s Management Team (Cabinet), Kansas City, Kansas Public Schools, 1998-2002 Schools, 1998-2002 First Things First: Kansas City, Kansas School District, Research Management Team, 1997-2002 Superintendent's Cabinet, Kansas City, Missouri School District, 1996-97 Desegregation Transition Planning Team, Kansas City, Missouri School Dist., 1996-97 Education Leadership Planning Team, Kansas City, Missouri School District, 1996-97 Missouri School Improvement Plan, Advisory Council, KCMSD, 1996-97 Superintendent's Planning Team for School Reorganization and Closings, KCMSD, 1996-1997 Superintendent's Testing Advisory Committee, Kansas City, Missouri School District, 1991-92 Kansas City, Missouri School District Magnet Program Planning Committees, 1989-91 Graduate Faculty, University ofKansas, 1993-1995 Internal Advisory Panel, University of Kansas Medical Center, Primary Care Physician Education Grant, 1994 Coordinator, School of Allied Health Faculty Development Seminars, 1993-95 Research Committee, Member \u0026amp; Secretary, School of Allied Health, 1994-1995 Thesis Committee Member, Departments of OT Education., Preventive Medicine, Speech and Hearing, 1993-96 Co-Chairperson., Program Evaluation Committee, Dept of OT Education, 1993-1995 Graduate Committee, Dept of OT Education, Member, 1993-1995 Senior Directors Committee, Dept of OT Education, Member, 1993-1995 Chairperson, Student Research Handbook Committee, Dept of OT Education., 1993-1995 Acting Department Chairperson, Dept. of Social Sciences, Turner High School, 1986 Turner SRA Testing Committee, 1985-86 Faculty Sponsor, Turner Students Mental Health Patients Cbristma:S Fund, 1985 orth Central Accreditation Steering Committee, Turner Unified School District, 1985 Chairperson., North Central Accreditation Goals and Objectives Committee, Turner School District, 1985 32 Likmd ,Yeh~ Mu~(0G) /12?/(}A'I ~/uQ-/u)n TO: FROM: LITTLE ROCK SCHOOL DISTRICT INSTRUCTIONAL RESOURCE CENTER RECEIVED 3001 PULASKI STREET LITTLE ROCK, AR 72206 FEB 11 2003 Board of Education T. Kenneth James, Superintendent of Schools OFFICE OF DESEGREGATION MONITORING PREPARED BY: 1 ?--.Bonnie A. Lesley, Associate Superintendent for Instruction DATE: February 27, 2003 SUBJECT: Extended Year Education (EYE) Program Evaluation Background Information One of the group of programs required by the federal court to be evaluated with the participation of an external evaluator was the Extended Year Education (EYE) program. District staff had completed a program evaluation after the end of the 2000-2001 school year, and it was presented to the Board of Education for information. Another report was also completed for the 2001-2002 school year. Both of these preliminary studies are attached for the Board's information. Steps Taken as a Result of the 2000-2001 Program Evaluation Three schools initially participated as EYE schools: Stephens Elementary, Mabelvale Elementary, and Woodruff Elementary. In fall 2002, two additional schools were added : Cloverdale Elementary and Mitchell Elementary. Some modifications to the intersessions were made in 2002-2003, but these were not a result of the program evaluation. Designation of External Consultant and His/Her Qualifications On December 2, 2002, the District awarded the contract for the Extended Year Education (EYE) program evaluation to the firm , Youth Policy Research Group, Inc., Dr. William Moore, Senior Partner. Dr. Moore's and Dr. Theresa Akey's resumes are attached, establishing their qualifications. Administrator Participation in Conducting the Program Evaluation In addition to Dr. Moore and his associates, specifically Dr. Theresa Akey, the following LRSD administrators participated in the evaluation: Ms. Frances Jones, Assistant Superintendent for Elementary School Services Dr. Ed Williams, Department of Planning, Research, and Evaluation Ms. Patricia Price, Director of Early Childhood and Elementary Literacy Board Memo February 27, 2003 Page Two Mr. Ken Savage, Department of Computer Information Systems Ms. Sharon Brooks, Principal, Stephens Elementary School Ms. Tab Phillips, Principal, Mabelvale Elementary School Ms. Janice Wilson, Principal, Woodruff Elementary School Teacher Participation in Conducting the Program Evaluation Teachers at Stephens, Mabelvale, and Woodruff who administered the various assessments-ORA and Observation Surveys at grades K-2\nBenchmarks at grade 4\nAL Ts at grades 2-5\nand SAT-9 at grade 5. Impact on African-American Student Achievement The external evaluator's conclusion wast hat \" Unfortunately, the limited nature oft he original design and existing data do not afford us an opportunity to answer in a rigorous manner the key evaluation question oft he extent of impact of the initiative on black student performance.\" Recommendation That the Board of Education approve the Extended Year .Education (EYE) program evaluation for submission to the federal court. BAL/adg ' ~ -~ \"\" ~ ' ~ ~ \"\"- --  Jl T A/CeV fl UWr\u0026lt;, v .. - . -- RFQ13-0JO-Response ~[Yauch Policy Research Group, Jnc. THERESA AKEY, PH. D. BIOGRAPHICAL SKETCH OCTOBER, 2002 Dr. Theresa Akey bas exrensive experience m the areas of evaluation, research and assessment She earned her doctorate m educational psychology, research and sraristics from the University of Kansas m 1995 and her master's m cornrnunitY and school counseling from Delta State University m 1990. She bas approximately 12 years experience working as a consultant with nonprofit service agencies. Her major work bas focused on agencies that provide supports to families of children with disabilities, ll}cludlllg evaluation of grants, staff development training, and site-based research projects. She bas also worked with nonprofit organizations mt he areas of alcohol and drug abuse, cornrnunitY counseling, and other services m both rural and urban areas. She served as project director and research director oftbe Beach Center on Families and Disabilities where she co-authored grants, condDcted research, and coordinated research and evaluarion acrivities for a large research cenrer focused on service provision to fumilies of children with disabilities. Dr. Akey bas extensive background m educational assessment, research evaluation, and analytic methods. She served as an assisrant professor mth e area of research and analytic methods at Auburn University for twO years, and mosr recently in the role of director of educational research, evaluation, and assessment m a J(mlsas Ciry area school district for the last four years. 38 I ~   '''- ..  .... \" ..     .... T. Akey (Yowh Policy Research Group. Jnc. J 2 RFQ23-0JO--Response of YouJh Policy Research Group, Jnc. CURRICULUM VITAE THERESA M. AKEY, PH.0. P.O. Box 4196 Kansas City, KS 66104-01% (816) 935-0852 (V) (816) 628-1927 (F) takey@YErg.org PROFESSIONAL STATEMENT A collaborative, innovative professional with interests in educational evaluation and assessment issues, including school reform, teacher professional devdopment and evaluation, insrructional strategies, standards-based education, and alternative educational environments. Seeks to promote continuous improvement of educational and youth-serving organizations through collaborative partnerships, action-oriented research, and application of theory to practice. AREAS OF PROFESSIONAL EXPERTISE  Research design and analysis including complex quantitative analyses (HLM, SEM), categorical madding, and qualitative design and analysis.  Development of educational assessment materials and psychometric validation of those materials. :::i Program evaluation in educational and community settings  Curriculum devdopment in K-12 and higher education settings  Professional devdopment in the use of data for organizational improvement, assessment devdopment, and program evaluation  Grant and proposal writing Information management development for educational information EDUCATIONAL HISTORY Ph.D. (1995). University of Kansas, Lawrence, KS, Educational psychology and research ,vith emphases in devdopmental psychology and quantitative research methods  Ed.S. (1992). University of Kansas, Lawrence, KS, School psychology  M.Ed. (191). Delta State University, Cleveland, MS, School psychology B.S. (1990). University of Arkansas, Little Rock. ~ Psychology and sociology 39 I   -t   I I  ' -        T. Ake\\ (Youih Policy kesearcn Group, inc.J J RFQ23-0JO-Response offouth Policy Research Group. Inc. PROFESSIONAL EXPERIENCES RESEARCH AND EVALUATION Youth Policy Research Group, Kansas City, Mo (June, 2002 to present). Senior Partner D Acting managing partner in research and evaluation corporation- administration and company  mInadniaagneam Teenatc hing ~ality Center (May 2002 to present). Evaluation project of best practices in teacher evaluation. ~alitative policy analysis of national standards in teacher evaluation and comparison of Indiana school districtS to those standards. D YDSI, Inc (May 2002 to present). Multivariate statistical analyses of instructional survey and observation data. including data display, technical assistance, and interrupted time-series analyses  o Kansas City, Kansas Public Schools (May 2002 to present). Development and implementation of case management system to display and organize student information o Kansas City, Kansas Public Schools Qune 2002 to present). Alignment and development of standards-based mathematics and language artS assessments  Pane Hill School District, Kansas City, MO (July 1998 to May, 2002), Director of Research Evaluation and Assessment  Oversight and coordination of all district assessment and program evaluation activities.  Preparation of short-term and long-range budget and operational plans\n Fiscal authority for budget of $200,000+ annually. D Responsibility for all personnel and resource utilization within the Department  Development and implementation of a comprehensive local curriculum-embedded assessment system integrated with the Missouri Standards  D Development and implementation of district-wide continuous improvement planning and D eDvoacluuamtieonnt ation and analysis of district performance on all state accreditation performance and D eDveavluealotipomn egnuti daenldin iems .p lementation of district-wide program evaluations of instructional practices, special education, professional development, assessment, supplemental instruction (Title 1, gifted, ESL), curriculum, and otha educational initiatives  D Served on regional and state committees related -to K-12 assessment and evaluation issues.  Led applied and theoretical research projecrs to address pressing educational issues relevant to  local policy and instructional practice  Collaborated with district stakeholders and school leadership to find innovative solutions to chronic educational problems in the District. Akey Consulting, Auburn, AL (June 1996 to July 1998) and Kansas City, MO. (Aug. 1999 to May, 2002), Founder .and owner O Online assessment and data information coordinating consultant for Kansas City, Kansas Public Schools. Design and implementation of online assessment and case management system for reporting assessment information to teachers and administrators  :J Assessment and data consultant for Kansas City, MO Regional Professional Development Center. Facilitated development of regional local assessments tied to state standards and benchmarks  D Statistical analysis consultant for Kansas City Kansas Public Schools. Systems reform data analysis 40 ~ ~  ' ' ' '         t t  T Akey (Yowh Policy Research Group, Im:. ) ./ RFQ23-0JO-Response of Youzh Policy Research Group, inc. D Educational consultant to Missouri school districtS- using data to make instructional decisions and best practices in standards-based assessment. Have consulted with approximately 10 different school districtS to do training and planning in this area. D Consultant to Kansas City, MO School District- review of performance indicators and data management system and statistical consultant for desegregation case. D Educational consultant to Alvin Nash, Kansas City, Missouri Mayor's office. Strategic planning. D Educational consultant for the West Alabama Learning Coalition of Schools. Program evaluation and grant/proposal writing on teacher professional development and collaboration. D Educational evaluation consultant for Celebration School and Disney Corporation. Evaluation of integrated assessment practices. D Consultant for Chambers County-Auburn University Partnership. Program evaluation of instructional effectiveness and professional development. Beach Center on Families and Disabilities (1993 to 1996), Lawrence, Kansas, Research coordinator (1995-1996) and Research project director (1993-1996). o Coordinated center research activities and data analysis as research coordinator D Provided technical assistance in instrument development  Developed instruments to measure constructS of psychological empowerment  Primary research investigator on two projectS (one qualitative and one quantitative) concerning responsive services and psychological empowerment in family members of a child with a disability.  Worked in collaborative manner with community-based family support programs in several states to evaluate responsiveness of services to families of a child with a disability   Grant and report writing Jones Research Consulting (1991-1996), Lawrence, Kansas, Founder and owner  Training consultant in empowering professionals and families. Beach Center on Families and  DRiessaebairlcihti easn, dU nstiavteisrtsiictsy coofn Ksualntsaanst.. Department of Educational Psychology and Research, University of D KFaamnsialsy support consultant to Wisconsin Family Support program. Program evaluation o lnscructional consultant to Douglas County Christian Schools, Lawrence, KS TEACHING AND ACADEMIC POSITIONS University of Kansas, Lawrence, KS (September 2001 to present). Psychology and Research in Education Department, Adjunct Professor, Educational Research, Statistics, and Classroom Assessment University of Missouri-Kansas City, Kansas City, MO Qanuary 2000 to present). Department of Educational Psychology and Research, Adjunct Professor, Classroom Assessment Park University, Parkville, MO Qanuary 2000 to present). Educacion Department, Adjunct Professor, SAtuanbduarrnd sU-Bniavseerds iAtys,s eAsusmbuernnt , AL (1996 to 1998). Assistant Professor, Department of Educational Foundations, Leadership and Technology, Educational Statistics (including suucrural equation modeling, hierarchical linear modeling, mulcivariate statiscics. and basic statistics), Research Methods, and pre-service classroom assessment University of Kansas, Lawrence, KS (1995-1996), Department of Educational Psychology and Research  Instructor, Basic educacional statistics  41        I    T. Akey (J\"outh Policy Research Group. JnC.) J RFQ13-0JO-Response offowh Policy Research Group, Jn:: University of Kansas, Lawrence, KS (1991-1995). Department of Educational Psychology and Research, Gradu~te Teaching Assistant, Developmental psychology, basic educational sratistics, and multiple regression. CLINICAL AND APPLIED PROFESSIONAL EXPERIENCES Topeka Public Schools (1991-1993). Topeka, KS, School Psychologist Clarksdale Public Schools (1990-1991), Clarksdale, MS, School Psychologist Arkansas Children's Hospital (1989-1991), Little Rock, AR, Behavioral lntervention Unit, Classroom Teacher Arkansas Children's Hospital (1989-1991), Little Rock, AR, Counselor GRANT FUNDING St. Clair, MA, \u0026amp; M.ey, TM. (1999). Using Reflective Practice Groups to Improve Mathematics Instruction. Goals 2000 Grant, $40,000, Af.ey, T. M. (1993). Family support influences on the development of parental skill and efficacy. NIDRR Grant # H133330070, RTD-5. Funded June 1993 as part of )650,000 grant, $45,000 per year for 5 years. Af..ey, T. M. (1993). Validating the Psychological Empowerment Scale for parents of children with disabilities. NIDRR Grant t H133330070, RTD-7, Funded June 1993 as part of $650,000 grant, $35,000 for 5 years . PUBLICATIONS Af..ey, T. M. \u0026amp; Ares, N. M. (in press). Using a Learning Partnership to Teach Classroom-based Assessment in Context Conceptual and Belief Changes in Preservice Teachers. Teaching Education  Ares, N.M., \u0026amp; Af..ey, T. (2000/2001). Self-organization in educational systems. Louisiana Education Research 1ournal 25(1), 49-71  Af..ey, T. M., Marquis, J. M., \u0026amp; Ross, M. E. (2000). The development of the Psychological Empowerment Scale: Evidence of its construct validity. Educational and Psvcholorical Measurement. 60Q), 419-437  Jensen, C., Hansen, C., Green, S. B., \u0026amp; Af..ey, T. M. (1997). Ari investigation of item difficulty incorporating structure of listening tests: A hierarchical linear modeling analysis. Proceedings of the Language Testing Research Colloauium  Green, S.B., Salkind, N.J., \u0026amp; Af..ey, TM. (1999). Using SPSS for Windows: Analvzing and Understandin_g Data, 2nd Edition. New York: Prentice Hall. Green, S. B., Af..ey, T. M., Fleming, K. K., Hershberger, S. L.. \u0026amp; Marquis, J. G. (1997). The effects of the number of scale points on chi square fit indices in confirmatory factor analysis. Srrucrural Eauation Modeling, ~(2) . 42   t           I T. Akey ffouJh Policy Research Group, inc.) 6 RFQ:23-0 I a-Response of Youth Policy Research Group. Inc. Afcey, T. M. (1996). Empowering families of children with a disability: lmplicati.ons for professional educators and service providers from a family-centered program. Teacher Education Research and Practice, 12(2). Green, S.B., Salkind, N.]., \u0026amp; Mey, T.M. (1996). Using SPSS for Windows: Analvzing and Understanding Data, 1st Editi.on. New York: Prentice Hall. Jones (Akey), T. M., Garlow, J. G., Turnbull, H. R..., \u0026amp; Barber, P.A. (1995). Family empowerment in a family support setting. 1n G. Singer, LE. Powers, \u0026amp; Olson, A.L. (Eds.) Redefining farnilv support: Introduction to public-private partnerships. Jones (Akey), T. M. (1995). To hdp o\u0026lt; not to help, A ,o,dy of group and individual v,,iabks in a monl system. Unpublished doctoral dissertation. Depanment of Educational Psychology and Research, University of Kansas Jones (Afcey), T. M. (1994). Development of the Psychological Empowerment Scale: Preliminary investigations. Unpublished master's thesis. Department of Educational Psychology and Research, University of Kansas. Frey, B., Jones (Afcey), T. M., \u0026amp; Saxon, T. F., (1993). Exoloring Research: Teacher's Manual (Teacher's Manual for N.J. Salkind's Exploring ResearchJ. New York MacMillan  CONFERENCE PRESENTATIONS AND WORKSHOPS Akey, T. M. (2000). Using Dau to !mprnve lmtruction. Wo,kshop p,esenwl to tl Regional P,.ofcssioual Development Center, Kansas City, MO, April 16, 2001. Afcey, T. M. (2000). Using Data to lmprove lnstr11ction. Workshop presented to East Central Show-Me Curriculum Administrators Association, St Louis, MO, April 1, 2000  Afcey, T. M. (2000). Using Data to lmprove lnstr11ction. Workshop presented to West Central Show-Me Curriculum Administrators Association, Kansas City, MO, February 16, 2000  Pennell, J .. Eick, C., and Afcey, T. M. (2000). Teacher beliefs and grading practices. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, April 26-30, 2000  Afcey, T. M., Sanders, S., Boyd, P., Gorrell, J. J., Kamen, M., \u0026amp; Salisbury-Glennon, ). (1999). Assessment and evaluation .in the Celebration School: Links to learning and curriculum. Paper presented at the annual meeri.ng of the American Educational Research Association Montreal, Canada, April 19-23 , 1999  Ares, K \u0026amp; Akey, T. M. (April, 1999). Modeling and understanding of social interactions. Roundtable presented at the annual meeting of the American Educational Research Association, Montreal, Canada, April 19-23, 1999 . Afcey, T. M. \u0026amp; Lawrence, F. (April, 1998). Understanding the effeccs of non-normal data on latent growth curve models. Violations of normaliry and influence of sample size. Paper accepted as pan of a \"\"\"posiuro (al.so chai, rnd o,gmiu\u0026lt; of smion) Modeling on-norm,! D,u fo, ,he rnnu,l ro~ring of the American Educaoonal Research Association, San Diego, CA. April 12-17, 1998  43 I  , t i ~  I 9 8  ~ ~ W!f ~ l.-il.t -- ~- T. Akey (foUih Policy Research Group, inc. , R.FQ23-0 Io-Response of Youih Policy Research Group. Jn::. /Cm, T. M, \u0026amp; A=, N. M. (Novanbe\u0026lt;, 1997). Conttpa\u0026gt;al and Bclicf ChanS\"' lmplicarions of, Socioculnml Apptn\u0026gt;ch to EducaUUg ps=,vicc Tcachm\n. Cl\u0026gt;,s,ooroB.scd Assessment P\u0026gt;pcr p,cscnd \u0026gt;t ili, mnual rowing of ,he MidSouili Eduational R,scatch Associ\u0026gt;rion. Memphis, TN, ovcnbet 12 14, 1997 /Cm, T. M., M\u0026gt;tqui,, J. G., \u0026amp; Tum bull, H R. (August. 1997). Effrro of f,mily Suppon Ptogr,uns on P\u0026gt;tcntal Eropow=ncnL P\u0026gt;pct P\"\"ntcd \" ,he ,nnual meeting of ,he Aroctiosn Psychological Associ,tion, Chicago, IL, August 1997. Lusti~ D., \u0026amp; /Cm, T. M. (Ap,il, 1997) f,mi\\y Adapurion\n. f,milics wiili Adult Cbildten wi,h Mental Rewdationc lmp\u0026gt;ct ofF.unily So-cngilis and Apprais\u0026gt;l PP'\" p,cscnusl \u0026gt;t m, ,nnual meeting of m, American Counseling Association, Orlando FL, April 10-13, 1997. Akey, T. M. \u0026amp; Gtccn, S. B. (M,tch, 1997). A Modd of Mo,al Deci,ionM,ltingc A Srody of Gwu p wd Individual Influcnw. P,pcr p,=ntcd \" ,he ,nnual mccUUg of ,he An,ctican Educational Rcs=ch Association, Chicago, IL March 24-28, 1997  Akey, T. M. \u0026amp; G,w,, S. B. (M=h, 1997). Tb\u0026lt; Rchuionsbip Jk,wccn Pomt md Intcnnrrdansl Dependent V,riablcs m MANOVA. Posre, p,,,,,ntcd \"m, ,nnual mcering of ,he An,ctiosn Educ\u0026gt;rional R,s=ch Association, Chicago, IL March 24-28, 1997  Akey, T. M. (FcbnW'/, 1997). Coofum\u0026gt;to,Y F,ctot Analysi, of, Moral Dilcnun Hdping Mcasut\u0026lt; P,p,s p,cscntI \u0026gt;t m, ,nnual mceUUg of m, E,w= Educ,rion\u0026gt;l R=ch A,soci,tion. Hilton Head, SC, February 19-22, 1997. Jensen, C., Hmscn, C., G=n, S. B., \u0026amp; Ny, T. M. (1996). An invarig,rion of i= difficuley inro,poraUUg srructut\u0026lt; of li,twing tcsuc A l,i,wchical lin\u0026lt;\u0026gt;t mod ding ,n,lysi.. P,pcr p,csW,I \u0026gt;t ili\u0026lt; Language Testing Research Colloquium, Tampere, Finland, July 31-August 3, 1996. Akey, T. M. (D=mbu, 1995). Effects of family support p,ogrnms on P\"'W ,mpow,nncnt. Wmkshnp presentation at the annual Empowering Families Conference, Chicago, IL. Jones (A'y), T. M. (August, 1994). Tb\u0026lt; dcodopmcnt of ,he Psychological EmpQWWO'' Sc,k Plimin,ry invcsrig,rions. Pom\u0026lt; p,cscnrarioo \u0026gt;t ,he ,nnual roceUUg of m, An,aican Psychological Association, Los Angeles, CA. Jones (Akey), T. M. Uun\u0026lt;, 1993). Roundublc discussioos on family cmpnwcnn\u0026lt;nt wd f=ily sctvises. Invited ,oundrabl\u0026lt; p,rticip\u0026gt;n\u0026lt; wd P\"''\"' ,t m, ,nnual f,mily Support ,nd f,mily Empowenn\u0026lt;n\u0026lt; Conference, Dartmouth, NH. RECENT TECHNICAL AND EVALUATION REPORTS /Cm, T.M. (2002). Effects of sw,dwl.s-b.a,d ,sscssmon\u0026lt; on insnuctional pnctiscs in middle school m\u0026gt;ili teachers. Technical report for the Park Hill School District, February 2002. Akey, T.M (2001). Using d,u ,s , method fo ,chool imp\"\"'\"'\"'' plannffig ,nd dctisionmkingc Administrators. Technical report for Park Hill School District, August 2001. Akey, T. M. (2000). 2000-2001 Diso-ict P,.fonnan Updac Asscssmcns of ili\u0026lt; Show-Mc Srand\u0026gt;tds Evaluation report prepared for Park Hill School District, October 12, 2000. 44 t ~ ~ t  t ~ I          .. T. Akey (YOUih Policy Research Group. inc) 8 RFQ23-0JO--Response of Youth Policy Research Group. Jnc. AJ\u0026lt;ey, T.M (2001). Using data as a method for insuuctional planning and decision-making: Teachers. Technical report for Park Hill School Disuict, August 2001. Akey, T.M. (200 l) V ,lidacion nf l,aming c\u0026lt;p\u0026lt;ctacinn, fo, , ,rand,uds-b,sol cuniculum in rommunicacinn ans. Technical report for Park Hill School Disuict, January, 2001 Akey, T. M. (2001). Effecrivon\u0026lt;Ss of th\u0026lt; Cl,ss-W\u0026gt;thin-,-Cl,,, Modd  .,..Ju,rion ,\u0026lt;port fo, Puk Hill School Disuict. February, 2001 Akey, T. M. (2001). Ev,lu,rion of cl,ss rank p,ocoluresc Ex,min,cion of ,!,en, m,thod,. Ev,lu,cion report for Park Hill School District. February, 2001 Akey, T. M (2001). Eff\u0026lt;ru nf Block Scboluling on !nsrrucrinnal P,wi= Ev,lu,cion ,\u0026lt;port p,opu,d fo, Park Hill School Disuict, February, 2001. ll\u0026lt;ey, T. M. (2000). Titl, l v,lu,rinn F,pnnc Ad\u0026lt;qu\u0026gt;. Y=ly P,ng,css ,nd F,cnmmondarions fa, Program Improvement. Evaluation report prepared for Park Hill School District, December 7, 2000. ll\u0026lt;,y, T. M. (2000). 1999-2000 District P\"fnnnmcr Upd,tt. Evalu,rinn report pp=d fo, Puk Hill School Disuict, October 12, 2000  Al=f, T. M. (2000). Eff,ru of Blnck Scboluling on Srod,nt Outt\u0026lt;\u0026gt;m\u0026lt;'' Man\u0026lt;,Y ,chi=\"'' uolin, ,nd test scores. Evaluation report prepared for Park Hill School Disuict, April, 2000. AJ\u0026lt;ey, T. M. (1999). Effectiveness of Full-Day Kindergarten: Third Year Evaluation. Evaluation report prepared for Park Hill School Disuict. N\u0026lt;ey, T. M. (1999). 199\u0026amp;-1999 District PWOnnancr Upd\"' v,lu,rion ,\u0026lt;port pel\"'ol fo, Puk Hill School Disuict, October 10, 1999  BOOKS Gmn, S. B., Salkind, N. J., \u0026amp; !1,_ey, T. M. (1997). Using SPSS fa, Windows, An,!yzing ,nd Undc\u0026gt;unding Data. New York: Prentice Hall  Green, S. B., Salkind, N. J., \u0026amp; AJ\u0026lt;ey, T. M. (1997). Using SPSS for the Macintosh: Analyzing and Understanding Data. New York: Prentice Hall  PROFESSIONAL AND COMMUNITY SERVICE Kansas City, Kansas yWCA, (Dec 2001 to present), Board of Directors Heartland Student Achievement Gap Organization, (August 2001 to present). Board of Directors Park Hill Educational Foundation, (August 2000 to present), Advisory Board 45 I ~ ~ t   ~        ..    T. Akey (Youth Policy Research Group, Inc.) 9 RFQ23-0JO--Response of Youth Policy Research Group, Jru:. Auburn Universitv 1998: Undergraduate Core Curriculum Development Committee for Teacher Education 1997: Chair, Library Appeals Committee 1997: Reviewer. Professional Educator, College of Education Universitv of Kansas 1993: Representative to the Graduate Student Council Student representative for faculty meetings of Educational Psychology and 1993, 1995: Research Deparonent. RECENT HONORS Nomination for Dissertation of the Year: University of Kansas, School of Education, May 1996 Who's Who in College, 1987, 1991 President's List 1987-1991 Outstanding Psychology Student, 1990 PROFESSIONAL ORGANIZA TIONS Show-Me Curriculum Administrators Association 1998 to present ASCD National Organization, 1998 to present Missouri ASCD: 1998 to present American Psychological Association: 199~ 1998 Society for Research In Child Development 1992 to present American Educational Research Association: 1993 to present 1997: Reviewer for American Educational Research Association Annual Meeting MidSouth Educational Research Association: 1996-1998 1998: Graduate Student Involvement Committee 1997: Reviewer for MidSouth Educational Research Association Annual Meeting 1997: Graduate Student Involvement Committee Eastern Educational Research Association: 1996-1998 American Evaluation Association 1998-present 46 l I  -t              ,,  W. P. Moore (Youth J'ollcy J{esearcn uruu, \"\"-- '  RFQ23-0JO-Response ofl'OU1h Policy Research Group, Inc. WILLJAMP. MOORE, PH. D. BIOGRAHICAL SKETCH OCTOBER, 2002 Dr. Moore holds a doctorate in Educational Psychology from the University of Kansas and is a Senior Partner with Youth Policy Research Group, Inc. located in Kansas City, Kansas. Currently he is the co-investigator for the evaluation of a systemic change initiative (First Things First) in the Kansas City, Kansas Public Schools and just completed a .post-doctoral fellowship with Juniper Gardens Cbildrens Research Project exploring the dimensions of effective learning communities in schools. Dr. Moore serves also as a lecturer at the University of Kansas where he teaches Evaluating School Programs, a doctoral-level educational evaluation course. Dr. Moore has served as Research and Evaluation Director in two urban school districts\nSenior Research Associate with the Ewing Marion Kauffman Foundatio~ and held faculty positions in research and measurement at the University of Kansas- Dr. Moore co-founded and directed the Dramatic AIDS Education Project, a collaborative AIDS education program for school-age youth and has spent the last 6 years conducting research and evaluation studies on the efficacy of this AIDS education program for youth. Two years ago, the Project received Kansas Ci-rys Ribbon of Hope Award for outstanding community service towards the elimination of HIV /AIDS  Previously, Dr. Moore was Senior Managing Consultant with GPR\u0026amp;E, a research and evaluation consulting furn. He has consulted with school districts, not-for-profits, health care institutions, national foundations, youth development intermediaries and universities. He is a past member of the Board of Directors of Kansas City Public Achievement, a Minnesota-based youth empowennent program\nand recently retired, after 6 years, from the Board of Directors of Sunflower House: A Child Abuse Prevention Center, where he was Vice-President of Education and a member of the agency's Executive Committee. He now serves as a member of the Advisory Board . Dr. Moore has served as an expert witness in Federal school desegregation litigation: was a member of the Panel of Writers for the development of the new Student Evaluation Standards coordinated by The Joint Committee on Standards for Educational Evaluation\nand served a twoyear term as a member of the National Research Development Committee for the American Occupational Therapy Foundation  Dr. Moore s research and evaluation interests have focused on effectively documenting the success of educational reform initiatives\nthe impact of mandated assessment programs on teacher instructional practice\nand the impact of education on the HIV-related knowledge, anirudes and behavioral intentions of youth. Dr. Moore has published in Applied Measurement in Education, Educational Assessment, the Jnrernational Journal of Educarional Research, and The American Occupazional Therapy Journal. Dr. Moore served 4 years on the Editorial Board of the Occupational Therap} Journal of Research and is currently a grant reviewer for the W. T. Grant Foundation. 7 I  ~ ' ' '         W. P. Moore (}'ouzh Policy Research Group. inc.) 2 RFQ13-0JO-Response of You.th Policy Research Group, Inc. CURRICULUM VITAE WILLIAMP- MOORE, PH. D. OCTOBER, 2002 Home 11478 South Wilder Street Olathe, KS 66061 913. 829. 3077 gprekc@aol.com Office P. 0. Box 4196 l(ansas City, KS 66104-0196 816. s64. 0143 M 913. 390. 6162 (F) wrooore@YPrg.org Educational History 1991 Ph.D. 1984 M.A. 1981 B.S. University of Kansas, Lawrence, KS Educational Psychology Major: Evaluation, Research, and Measurement University ofl(ansas, Lawrence, KS Education Major: Secondary Curriculum and Instruction University of Kansas, Lawrence, KS Education Major: Secondary Social Sciences Emphasis: Psychology and Sociology Current Appointments and Positions Senior Partner, Youth Policy Research Group lnc., Kansas City, KS Lecturer, University of Kansas, Lawrence, KS Grant Reviewer, William T. Grant Foundation, New York., NY Advisory Board, Sunflower Housec A Child Abuse Prevention Center, Overland j'ark, KS Advisory Boa,d, Communitl' Health Prornotion Program, University of Kansas Medical Center, Kansas City, KS 8 I  ~ , -' ---I I I ~ ~ ~     -- W. P. Moore (l'outh Poiicy Research Group, me.\n3 RFQ23-0JO-Response o_(Youth Policy Research Group, Inc. Professional Experience Consultation 2002-Present Senior Partner, Youth Policy Research Group, Inc., Kansas City, KS. Youth Policy Research Group is a corporate partnership formed by youth and education researchers located in the Kansas City metropolitan area. Toe mission ofthis firm is to engage in applied research and evaluation studies that will contribute to policy and practice decisions and dialogue about the necessary supports and resources youth require to achieve long-term developmental success both academically and behaviorally. Much of the work already completed by YPRG researchers is in the K-12 education arena. Acting mana::.aing partner in research and evaluation corporation- administration and company management. 1993-2002 Senior Mana...oin2: Consultant, Great Plains Research and Evaluarion, Olathe, KS. GPR\u0026amp;E was an education and youth development research and policy furn focused on improving the programming and organizations that children and youth experience. Overall responsibility for managing research and evaluation contractual work. Oversaw a staff of part-time consultants, managed resources and budget, negotiated contracts, oversaw project development, implementation, coordination of data collection, analysis, dissemination, and action planning for future decision-making with clients. Clients included:  Ewing Marion Kauffman. Foundation. Through contactual work awarded by EMKF we assisted the following clients:  State of Kansas, SRS, Alcohol \u0026amp; Drug Abuse Services  State of Missouri, Dept. of Mental Health. Div. of Alcohol \u0026amp; Drug Abuse  State of Colorado, Team Fort Collins  Kansas City-St. Joseph Diocese Sc\nThis project was supported in part by a Digitizing Hidden Special Collections and Archives project grant from The Andrew W. Mellon Foundation and Council on Library and Information Resoources.\n   \n\n   \n\n  \n\n\n   \n\n  \n\n \n\n\n   \n\n   \n\n \n\n\n   \n\n  \n\n \n\n   \n\n \n\n  \n\n\n   \n\n \n\n  \n\n\n\n   \n\n  \n\n  \n\n\n   \n\n   \n\n  \n\n \n\n \n\n\n   \n\n  \n\n \n\n\n\n\n\n\n\n\n\n   \n\n \n\n\n\n  \n\n\n   \n\n\n\n  \n\n\n\n "},{"id":"tmll_hpcrc_50074555","title":"Funding federal civil rights enforcement, 2000-2003 [electronic resource]","collection_id":"tmll_hpcrc","collection_title":"Historical Publications of the United States Commission on Civil Rights","dcterms_contributor":null,"dcterms_spatial":["United States, 39.76, -98.5"],"dcterms_creator":["United States Commission on Civil Rights"],"dc_date":["2002"],"dcterms_description":["A digital version of the report published by the United States Commission on Civil Rights.","The Civil Rights Digital Library received support from a National Leadership Grant for Libraries awarded to the University of Georgia by the Institute of Museum and Library Services for the aggregation and enhancement of partner metadata."],"dc_format":["application/pdf"],"dcterms_identifier":null,"dcterms_language":null,"dcterms_publisher":null,"dc_relation":["Forms part of online collection: Historical Publications of the United States Commission on Civil Rights.","Requires Acrobat plug-in to view files."],"dc_right":["http://rightsstatements.org/vocab/NoC-US/1.0/"],"dcterms_is_part_of":null,"dcterms_subject":["Civil rights--United States","Affirmative action programs--United States--Finance"],"dcterms_title":["Funding federal civil rights enforcement, 2000-2003 [electronic resource]"],"dcterms_type":["Text"],"dcterms_provenance":["Thurgood Marshall Law Library"],"edm_is_shown_by":null,"edm_is_shown_at":["http://purl.access.gpo.gov/GPO/LPS20090"],"dcterms_temporal":null,"dcterms_rights_holder":null,"dcterms_bibliographic_citation":null,"dlg_local_right":null,"dcterms_medium":["reports","records"],"dcterms_extent":null,"dlg_subject_personal":null,"dcterms_subject_fast":null,"fulltext":null},{"id":"bcas_bcmss0837_1046","title":"\"Guidelines for Completing Eight Program Evaluations in the Little Rock School District\"","collection_id":"bcas_bcmss0837","collection_title":"Office of Desegregation Management","dcterms_contributor":null,"dcterms_spatial":["United States, 39.76, -98.5","United States, Arkansas, 34.75037, -92.50044","United States, Arkansas, Pulaski County, Little Rock, 34.74648, -92.28959"],"dcterms_creator":["Ross, Steven M."],"dc_date":["2002"],"dcterms_description":null,"dc_format":["application/pdf"],"dcterms_identifier":null,"dcterms_language":["eng"],"dcterms_publisher":["Little Rock, Ark. : Butler Center for Arkansas Studies. Central Arkansas Library System."],"dc_relation":null,"dc_right":["http://rightsstatements.org/vocab/InC-EDU/1.0/"],"dcterms_is_part_of":["Office of Desegregation Monitoring records (BC.MSS.08.37)","History of Segregation and Integration of Arkansas's Educational System"],"dcterms_subject":["Little Rock (Ark.)--History--21st Century","Little Rock School District","Education--Arkansas","Education--Evaluation","Educational planning","School improvement programs","Student assistance programs","Educational statistics"],"dcterms_title":["\"Guidelines for Completing Eight Program Evaluations in the Little Rock School District\""],"dcterms_type":["Text"],"dcterms_provenance":["Butler Center for Arkansas Studies"],"edm_is_shown_by":null,"edm_is_shown_at":["http://arstudies.contentdm.oclc.org/cdm/ref/collection/bcmss0837/id/1046"],"dcterms_temporal":null,"dcterms_rights_holder":null,"dcterms_bibliographic_citation":null,"dlg_local_right":null,"dcterms_medium":["documents (object genre)"],"dcterms_extent":null,"dlg_subject_personal":null,"dcterms_subject_fast":null,"fulltext":"\n \n\n\n\n\n\n\n\n  \n\n\n   \n\n   \n\n\n   \n\n\n   \n\n\n\n\n   \n\n\n\n\n   \n\n\n\n\n\n\n\n\n\n\n\n   \n\n   \n\n \n\n\n   \n\n   \n\n  \n\n\n   \n\n  \n\n \n\n\n   \n\n  \n\n \n\n\n   \n\n  \n\n  \n\nThis transcript was created using Optical Character Recognition (OCR) and may contain some errors.\nUSCHEL H FR10AY(lt22-1'94) JLLIAM H SUTTON, P.A iON M EISEMAN. JR, P.A  O BELL, PA JAMES A BUTTRY, PA FREDERICK. S URSERY,,. A. ,..SCARE DAVIS, JR, P.A MES C. CLARJC, JR , PA. OMAS p LEGGETT, P.A t{N DEWEY WATSON, PA PAUL B BENHAM Ill.PA LARRY W BUJlK.S, P A  WYCK.LIFP NISBET, JR..., P.A MES EDWARD HARRJS, P.A PHILLIP MALCOM, PA MES M SIMPSON, P A JAJllfES M SAXTON, PA J SHEPHERD RUSSELL Ill, PA DONALD H BACON. P.A ILLIAM THOMAS BAXTER. P A (HARD D TAYLOR.PA SEPH B HURST, JR.., PA ELIZABETH ROBBEN MUR.llAY, PA CHRISTOPHER HELLER. P.A LAUR.A HENSLEY SMITH, PA ROBERT S. SHAPER. P A WILLIAM M GRIFFIN Ill, P.A MICHAELS MOORE, P.A DIANE S MACKEY, PA WALTER M EBEL 111, P A KEVIN A CRASS, PA WILLIAM A WADDELL, JR., PA SCOTT J LANCASTER. P.A ROBERT 8 BEACH, JR . P A J LEE BROWN, P.A JAMES C. BAKER, JR.PA HAR.AV A LIGHT, PA SCOTT H TUCKER. PA GUY ALTON WADE, PA PRICE C. GARDNER.PA TONIA P JONES, P A DAVID D WILSON, PA JEFFREY H MOORE, P A DAVID M GRAF. P .A RECEIVED 1/'11/M DEC - 4 2002 j-/?Vid - Oc /,vu e cL OFFICE OF DESEGREGATION MONITORING ( By Hand Delivery) Mr. John W. Walker John W. Walker, P.A. 1723 Broadway Little Rock, Arkansas 72201 Mr. Richard Roachell Roachell Law Firm Plaza West Building 415 N. McKinley, Suite 465 Little Rock, Arkansas 72205 FRlDA Y ELDREDGE \u0026amp; CLARK ATTORNEYS AT LAW A LIMITED LIABILITY PARTNERSHIP www.fridayfirm.com 2000 REGIONS CENTER 400 WEST CAPITOL LITTLE ROCK, ARKANSAS 72201-3493 TELEPHONE 501-376-2011 FAX 501-376-2147 3'25 NORTH FUTRALL DRIVE, SUITE 103 FAYETTEVILLE, ARKANSAS 72703-41!111 TELEPHONE 479-l!llil5-2011 FAX 479-l!llil5-2147 208 NORTH FIFTH STREET BLYTHEVILLE. ARKANSAS 72315 TELEPHONE 1!170-7\u0026amp;2-2898 FAX 870-7152-2911!1 December 4, 2002 Mr. Sam Jones Wright, Lindsey \u0026amp; Jennings 2200 Worthen Bank Bldg. 200 West Capitol Little Rock, Arkansas 72201 ( By Hand Delivery) Ms. Ann Marshall Desegregation Monitor 1 Union National Plaza 124 W. Capitol, Suite 1895 Little Rock, Arkansas 72201 RE: Compliance Remedy Dear Counsel \u0026amp; Ms. Marshall: CARLA GUNNELS SPAINHOUR. P A JOHN C. FENDLEY, JR, PA JONANN ELIZABETH CONIGLIO, P A R. CHRISTOPHER LAWSON, P A FRAN C. HICK.MAN, P A BETTY J DEMORY, P.A LYNDA M JOHNSON, P A JAMES W SMITH, PA CLIFFORD W PLUNKETT, PA DANIEL L. HERRJNGTON, P A MARVIN L. CHILDERS K. COLEMAN WESTBROOK, JR ALLISON J_ CORNWELL ELLEN M OWENS JASON B HENDREN BRUCE B TIDWELL MICHAEL E KARNEY KELLY MURPHY MCQUEEN JOSEPH P MCKAY ALEXANDRA A IFRAH JAY T TAYLOR MARTIN A KASTEN Mr. Steve Jones BRYAN W DUKE JOSEPH G NICHOLS ROBERT T SMITH RYAN A BOWMAN TIMOTKY C. EZELl T MICHELLE ATOR KAREN S. HALBERT SARAH M COTTON PHILIP B MONTGOMERY KJUSTEN S RIGGINS ALAN G. BRYAN LINDSEY MITCHAM SLOAN KHAYYAM M EDDINGS JOHN F PEISERICH AMANDA CAP'PS ROSE BRANDON 1. HARRISON OFCOUNS[L BS CLARK. WILLIAM L TERJt.Y WILLIAM L PATTON, JR H T LARZELERE, P A JOHN C. ECHOLS, P A A O MCALLISTER JOHN C. FENDLEY, JR. LITTLE ROCK TEL 501-l70ll23 FAX 5D1244-5341 f  ndl  yOftc.n  t Jack, Lyon \u0026amp; Jones, P.A. 425 W. Capitol, Suite 3400 Little Rock, Arkansas 72201 Mr. Dennis Hansen Ofc of the Attorney qeneral 323 Center Street 200 Tower Building Little Rock, Arkansas 72201 Enclosed please find, \"Guidelines for Completing Eight Program Evaluations in the Little Rock School District\" prepared by Dr. Steven M. Ross. The Little Rock School District intends to retain Dr. Ross, Dr. Larry McNeal and Dr. William Moore to complete the eight evaluations identified on page 148 of the Little Rock School District's March 15, 2001 compliance report that were not completed with the assistance of an outside expert. These were identified on page 12 of the Compliance Committee's proposed compliance plan previously provided to you. F \\HOME\\BBrown\\Fmdley\\LRSD\\dcseg\\counsel2 It. wpd n :o ro 0::, ~rt n\"\"\" ' ro l\"1 \u0026lt;  H'I tn 0 l\"1 l\"1 ro ~ en ro g~ ::, Cl C/l l\"1 ro n ::,' ,... = rt,, (I \u0026gt; Cl ~::r 00 n::, I\"\" Cl Qt All Counsel December 4, 2002 Page2 Little Rock School District hopes that its decision to follow the guidelines prepared by Dr. Ross will eleviate the Joshua Intervenors concerns about the preparation of these evaluations. If not, we respectfully request that ODM schedule facilitation related to the preparation of these evaluations as soon as possible so that the District may meet the court's March 15, 2003, deadline for Board approval and submission of the evaluations to the Court. Also enclosed for your reference are the responses to the RFQ submitted by the experts identified above. JCF/bgb enclosure(s) cc: Dr. Ken J arnes F:IHOME\\BBrownlfendleyll.RSDldcseg\\counscl2 h wpd Sincerely, John C. Fendley, Jr. () \"\"'1) 0 :, 1--' rt ~-Cl) n l'\"1 \u0026lt;  HI en o l'\"1 l'\"1 '1) ,:j en ro \"'Cl en 0 '1) :, Ill en l'\"1 ro n ::r Guidelines for Completing Eight Program Evaluations in Little Rock School District Prepared by Steven M. Ross, Ph.D. The present guidelines are based on my review of the Revised Compliance Plan, the LRSD standards for program evaluation, and evaluation report drafts and associated materials related to the eight programs identified as requiring \"final\" evaluation reports. My analysis of this material, combined with my experiences as an educational researcher and familiarity with the Joshua case as it affected LRSD, was influenced by the following assumptions:  Invalid or questionable evaluation results can be much more detrimental than helpful to efforts to improve educational practices, and should not be disseminated without strong cautions and qualifications. Accordingly, studies that lack proper controls against bias or contamination from extraneous factors (e.g., differential sampling, history, diffusion of treatments) have limited value for guiding policies.  Program evaluations that focus predominately on student achievement outcomes while lacking sufficient implementation data have reduced value due to inability to determine the nature of the \"treatment.\" The study will also fail to inform policymakers about the practicality of the program, how it was used and reacted to by stakeholders, or whether and/or how it needs to be improved to impact atrisk learners.  Evaluations of programs that have been discontinued in the district are of much less interest relative to ones that are presently being implemented or informing ongoing practices.  To raise the achievement of African American students in LRSD, attempting to resuscitate existing studies that have insufficient data available, limited relevance to current practices, or require substantial time and resources with little promise of yielding useful information for policy decisions would be less productive than employing the \"lessons learned\n' from the prior evaluation work to support high quality and informative future studies. One such lesson is that the LRSD research department (formerly PRE) was understaffed to perform evaluations of the quality and quantity needed. Based on the above assumptions, I will recommend below a basic strategy for the third-party evaluators to use in preparing the eight identified evaluations for approval by the school board. Four of the evaluations concern programs that are no longer in use by LRSD and have limited or no relevance to programmatic decisions (Lyceum Scholars, Elementary Level Summer Schools, Vital Link, and Onward to Excellence). Of the remaining four evaluations, two have limited available data (Middle School Transition and Campus Leadership Teams) that, even with supplementary analyses, would not permit confident (valid) decisions to be made about program effectiveness (') \"ti (I) 0:::, ~rt ~-(I) 0 l\"1 \u0026lt;  HI en o l\"1 l\"1 (I) !:C en ro -c, en 0 (I) :::, Ill en l\"1 (I) 0 ::r 2 in general or about African American student achievement resulting from program participation. A seventh evaluation (Extended Year Education) could possibly yield informative evidence about an ongoing program, but to be sufficiently refined would require time and resources extending significantly beyond the current conditions for project completion. An eighth evaluation (HIPPY) also deals with an ongoing program, but unlike the others could possibly provide useful evidence through revisions completed within the available time frame. Accordingly, the HIPPY report is currently being rewritten by Dr. Ed Williams from LRSD. The suggested plan for the third-party evaluators is presented below followed by a brief review of each evaluation. A. Submit the current evaluation report as an attachment to a supplemental document as described in B-D. B. The supplement should begin with an expanded description of the program, its goals, and its history in LRSD. It should then describe the evaluation methodology and summarize and interpret the key findings. C. Most importantly, the supplement should discuss the limitations (and any strengths where indicated) of the evaluation with regard to: (a) informing current practices in LRSD\n(b) using appropriate methodology\nand (c) addressing student achievement effects, especially in reference to African American students. D. Finally, the supplement should present suggestions for conducting stronger studies of similar programs in future evaluation studies. 1. Middle School Transition (Moore) Thi/evaluation is in near-completed form and needs mostly editing and expansion. Because the middle school program is current and continuing, this evaluation study can be useful (mostly for guiding professional development and implementation improvement) for informing district strategies. The achievement results are fairly minimal and uninformative, but at the time of the evaluation (1999-2000), only baseline data existed. Thus, aside from providing additional description of the results (the tables and the narrative are sparse) and a more meaningful interpretation of trends (especially with regard to African American vs. Caucasian students), there is probably little more that needs to be done for this essentially baseline time period. The survey data appear to be reasonably analyzed and reported, but the interpretation and discussion should be extended to provide more meaningful conclusions and recommendations. Suggestions: The third-party evaluator should follow the basic strategy outlined in the introductory section. (') \"O Cl) 0::, .... rt\" I-'('!) n ~ '\u0026lt; t-'11 en o ~ ~ Cl)?:-' en m \"Cl en 0 Cl) ::, Ill en ~ m n ::r )\" en en 0 n I- Ill Q rt Cl) 3 2. Lyceum Scholars (McNeal) The Lyceum Scholars' High School Program, which was evaluated in 1998-99 and 1999- 2000, is no longer being implemented in LRSD. The latter consideration, coupled with the obvious limitations of the evaluation design with regard to rigor, depth, and meaningfulness of the data, substantially reduce the value of the study and the need for devoting more than minimal resources to it, beyond perhaps a supplemental summary and explanation. Suggestions: The third-party evaluator should follow the basic strategy outlined in the introductory section. 3. Elementary Level Summer School (McNeal) Similar to the Lyceum Scholars' High School Program (#2 above), the Elementary Level Summer School program is no longer being implemented in LRSD. In addition, the evaluation study conducted in the summer of 2001 is limited in its design and methodology. Among the major concerns are the lack of: (a) implementation data to describe the program strategies and the degree to which they were actually used by teachers, (b) an adequate control group or norms to which the achievement scores of summer school students could be compared, and (c) qualitative data to describe the experiences of students and teachers in the program. Due to \"differential sampling\" the multiple tables provided are neither overly meaningful nor informative regarding the progress of summer school students in general and African American summer school students in particular. Seemingly, there is little useful information to be gained for informing future policies by. investing substantive resources in revamping the study. While more suitable control samples might be established using archival data, the absence of implementation assessments would still make the \"treatment\" essentially unknown. Therefore, suggestiJns similar to those made for the Lyceum Scholars program are also offered here. Suggestions: The third-party evaluator should follow the basic strategy outlined in the introductory section. 4. Vital Link (Ross) The Vital Link program, designed to provide students with on-the-job experiences, was offered to 394 middle school students in the summer of 1999. Because the program was of very limited duration (only one week) and is not focused on either academic curriculum or learning strategies, it is highly unlikely to have affected students' academic achievement. Although such a program would still potentially serve a useful purpose for fostering student motivation to achieve and complete school, it is no longer being implemented in LRSD. Further, the evaluation study conducted was so limited (a brief post-test only, closed-ended survey) that the policy implications of the results are minimal and even potentially misleading if derived. Therefore, suggestions similar to (\") \"Cl ro 0 ::, ~M' n'\"'' lr'oi -\u0026lt;  1'11 en o l'i l'i ro ~ en ro ~ en o ro ::, QI en l'i ro n ::,' 4 those made for the Lyceum Scholars Program and the Elementary Level Summer School Program (#'s 2 and 3 above) are again offered here. Suggestions: The third-party evaluator should follow the basic strategy outlined in the introductory section. 5. Onward to ExceIJence CSRD Program (Ross) The OTE model was implemented at Watson Elementary School for several years, starting in 1999. It has since been discontinued and was never formally evaluated, except for achievement data reports sent by the principal to ADE. Thus, in essence, there is no longer any program in LRSD to evaluate and no evaluation report to revise, expand, or redraft. It would seem wasteful of resources to reexamine historical data from this program, especially since implementation data are lacking. That is, if positive or negative results were found, it would be impossible to determine whether OTE or numerous others factors were the main cause. Suggestions, therefore, are similar to those for #'s 2-4 above. Suggestions: The third-party evaluator should follow the basic strategy outlined in the introductory section. 6. HIPPY (Ross) Because HIPPY is a continuing program, this evaluation can be potentially useful to LRSD by providing initial program results on student achievement and benefits to African American children. A limitation of the study, which unfortunately cannot be remedied retroactively, is the lack of implementation data to describe the fidelity with which HIPPY program components were actually used. The quantitative achievement results must therefore be viewed cautiously, but should sti11 be at least suggestive regarding program influences. Substantive expansion and revision, however, are needed to increase the readability and meaningfulness of the report. For example, there is inadequate description of the program, context, methodology, and analysis design. Tables and findings need to be presented in a more readable (\"user-friendly\") manner. Suggestions: A. Reorganize and expand the introduction and methodology to be in line with district evaluation standards (i.e., more context, more detailed methodology, clearer questions and organization). B. Ed Williams needs to run the revised analysis and write up results by January 31, 2003. A program description needs to be provided. Results need to be disaggregated, if possible, for African American and Caucasian students. Expand the Results sections to provide more informative reporting of outcomes, clearer tabular presentations, etc. C\"l \"O Cl) 0::, ~ rt' ~-Cl) n l'1 ~ - H'I en o l'1 l'1 Cl) ,:I en m \"O en 0 Cl) ::, llJ en l'1 m n ::,' ~- ::, t:l .0, C. Expand the Conclusions section to: (a) directly address whether there are implications for the achievement of African American and other disadvantaged groups (there probably are not at this stage), (b) more fully discuss implications and recommendations associated with the findings, and ( c) propose further evaluation research that will validly determine both implementation quality and influences of HIPPY on student achievement. D. The third-party evaluator should follow the basic strategy in expanding this report. 7. Extended Year Education (EYE) Report (Moore) 5 The EYE program is relevant to LRSD's current interests in improving academic achievement of its students. Unfortunately, the present evaluation design does not seem sufficiently sensitive to detect effects that might be attributable to EYE. Specifically, usage of whole-school data compared descriptively to district norms gives only a very surface examination of the schools' progress, with susceptibility to contamination by student mobility, differences in SES, etc. A more precise analysis would match students at the three schools to similar students at comparable schools not using EYE, and then examine progress using a multivariate-type (regression or MANOV A) analysis. It is questionable, however, that such analyses could be completed in the time remaining for the required submission of the final report. Also, the findings would be limited by having only two years of post-program data. Aside from the design limitations, the organization of the report is difficult to follow due to the many tables and brief but not very informative narrative descriptions. The survey data might be interpretable, but also need a much clearer and better organized presentation. Suggestions: The third-party evaluator should follow the basic strategy outlined in the introductory section. , 8. Campus Leadership Teams (Ross) This initiative seems highly relevant to current and future goals of LRSD. However, the \"evaluation data\" collected to date consist of only results from two district-wide surveys that assessed team members' reactions to various activities. No information exists to verify the representativeness of the samples, the validity of the data collection in general, or the implementation of the CL Ts at the various schools. The aggregate survey results on the 24 combined items (14 in the team member survey\n10 in the certified/noncertified staff member survey) do not appear overly interesting or meaningful with regard to informing practice. Suggestions: The third-party evaluator should follow the basic strategy outlined in the introductory section. C\"'l \"d Cl) 0::, r-' rt\" .... Cl) n l\"1 \u0026lt; - Hi tll 0 l\"1 l\"1 Cl) ~ Ill Cl) 'Cl Ill 0 Cl) ::, 0J Ill l\"1 ro n ::,\" .... ::, t'l .Q , ~The University of Memphis Memphis, Tennessee 38152-3340 A State of Tennessee Center of Excellence Center for Research in Educational Policy 325 Browning Hall October 28, 2002 Director of Procurement Little Rock School District 1800 East Sixth Street Little Rock, AR. 72202 Dear Mr. Paradis, RECEIVED 'j : I 2p rvJ DEC - 4 2002 I f.t,i,,J . /)C'l1 ~ ~~ c J., OFFICE OF DESEGREGATION MONITORING Local 901/678-2310 Toll 866/670-6147 FAX 901/678-4257 Enclosed are five copies of the Center for Research in Educational Policy's response to RFQ #23-010: Revised Desegregation and Education Plan Program Evaluation Consultant. If additional information is needed or ifl can be of further assistance, please contact the Center toll free at 1-866-670-6147. il~ Steven M. Ross Director A Tennessee Board of Regents Institution An Equal Opponunity/Afflrrnsliw Action University Response from the Center for Research in Educational Policy at the University of Memphis to: RFQ Number: 23-010 Title: Revised Desegregation and Education Plan Program Evaluation Consultant Opening date/time: ovember 6, 2002 (2:00 p.m.) CREP Response to RFQ23-010 1. Curriculum vitae with specific documentation of successful experience in education program evaluation. PERSONAL DATA Steven M. Ross 224 Eagle Spring Cove Cordova, TN 38018 Institution Pennsylvania State University Undergraduate Major: EDUCATION Psychology Home (901) 755-6654 Office (901) 678-3413 Degree-Year B.A. 1969 M.S. 1972 Ph.D. 1974 Graduate Major: Educational Psychology PROFESSIONAL ASSOCIATIONS American Psychological Association, Fellow American Educational Research Association, Member Mid-South Educational Research Association, Member Association for Educational Communications \u0026amp; Technology, Member International Congress for School Effectiveness and School Improvement, Member EXPERIENCE Instructor, Continuing Education, 1973-74, Pennsylvania State University Instructor, Psychology, Spring Semester, 1974, Lock Haven State College, Lock Haven, Pennsylvania Evaluator, Summer, 1974, Mitre Corporation, McLean, Virginia Assistant Professor, Educational Psychology, 197 4-79, University of Memphis Associate Professor, Educational Psychology, 1980-1985 Professor, Educational Psychology, 1985 - Present Senior Researcher, Center for Research in Education Policy, University of Memphis, 1995-2001 Director, Center for Research in Educational Policy, University of Memphis, 2001 - present COURSES RECENTLY TAUGHT Theories of Learning (undergraduate) Individual Differences and Learning (Graduate) Educational Statistics (Undergraduate and Graduate) Educational Research (Graduate) Computers in Education (Graduate and Undergraduate) Thesis Writing (Graduate) Educational Assessment (Graduate) CREP Response to RFQ23-010 2 HONORS AND DISTINCTIONS 1. NDEA Fellowship for graduate study at the Pennsylvania State University, 1971-1973. 2. Graduate Student Associate, Southwest Regional Laboratory, Summer, 1971. 3. Distinguished Teaching Service Award, University of Memphis, 1980. 4. Phi Delta Kappa Professional Research Award, Memphis Chapter, 1983. 5. Elected Fellow, Division 15, American Psychological Association, 1986. 6. Visiting Scholar, National Center for Research on Improving Postsecondary Teaching and Learning. University of Michigan, Summer 1987. 7. Distinguished Research Award, University of Memphis, 1987. 8. Distinguished Teacher Service Award, University of Memphis, 1988. (First eligibility since 1980\nno longer eligible) 9. Memphis State University nominee, CASE Professor of the Year Award, 1989 10. Superior Performance in University Research (SPUR) Award, University of Memphis, 1990, 1991, 1992 11. Distinguished Research Award, University of Memphis, 1993. 12. Board of Visitors Eminent Faculty Award, University of Memphis (first recipient), 1993 13. Editor, Educational Technology Research and Development, 1993-present 14. Editorial Board, Journal of Education for Students Placed At Risk. 1995-present 15. Editorial Board, Computers and Human Behavior, 1994-present 16. Invited testimony, U.S. House of Representatives Subcommittee on Early Childhood, Youth, and Families, Committee on Education and the Workforce, June 26, 1998. 17. Invited panelist on comprehensive school reform, discussion with Secretary of Education Richard Riley, March 16. 1999. 18. Lillian and Morrie Moss Chair of Excellence in Urban Education, 2001 Publications in Refereed Journals Books Book Chapters SCHOLARSHIP Papers Presented at Professional Meetings 122 7 28 224 SELECTED RECENT PUBLICATIONS Ross, S. M., Henry, D., Phillipsen, L., Evans, K., Smith L., \u0026amp; Buggey, T. (1997). Matching restructuring programs to schools: Selection, negotiation, and preparation. School Effectiveness and School Improvement,~' 45-71. Ross, S. M., Troutman, A., Horgan, D., Maxwell, S., Laitinen, R., \u0026amp; Lowther, D. (1997). The success of schools in implementing eight restructuring designs: A synthesis of first-year evaluation outcomes. School Effectiveness and School Improvement, 8_, 95-124. Ross, S.M., Smith, L. J., \u0026amp; Casey, J. (1997). Preventing early school failure: Impacts of Success For All on standardized test outcomes, minority group performance, and school effectiveness. Journal for Research on Students Placed At Risk, i, 29-54. CREP Response to RFQ23-010 3 Stringfield, S., \u0026amp; Ross, S. M. (1997). A \"reflection\" at mile three of marathon: The Memphis restructuring initiative in mid-stride. School Effectiveness and School Improvement, 8, 151-161. Ross, S., \u0026amp; Smith, L. J. (1997). Improving the academic success of disadvantaged children: An examination of Success For All. Psychology in the Schools, ~..1, 171-180. Jayasinghe, M. G., Morrison, G. R., \u0026amp; Ross, S. M. (1997). The effect of distance learning classroom design on student perceptions. Educational Technology Research and Development,~ 5-20. Ross, S. M., \u0026amp; Smith, L. J. (1998). Improving school achievement and inter-group relations for children placed at risk. European Journal oflntercultural education, 9_(2), 141-154. Smith, L. J., Ross, S. M., McNelis, M, Squires, M., and others (1998), The Memphis restructuring initiative: Analysis of activities and outcomes that impact implementation success. Education and Urban Society, 30(3), 296-325. Stringfield, S., Datnow, A., Ross, S., \u0026amp; Snively, F. (1998). Scaling up school restructuring in multicultural multilingual contexts: Early observations from Sunland County. Education and Urban Society, 30(3), 326-357. Ross, S. M., Smith, L. J., \u0026amp; Casey, J.P. (1999). \"Bridging the gap\": The effects of the Success For All Program on elementary school reading achievement as a function of student ethnicity and ability level. School Effectiveness and School Improvement, lQ(2), 129-150. Morrison, G. R., Ross, S. M., \u0026amp; Kemp, J.E. (2000). Designing effective instruction (3'd ed.). New York, tN: Macmillan College Publishing. Ross, S. M., Alberg, M., Smith. L., Anderson, R., Bol, L., Dietrich, A., Lowther, D., \u0026amp; Phillipsen, L. (2000). Using whole-school restructuring to improve educational outcomes: The Memphis story at year 3. Teaching and Change, 1(2), 111-126. Ross, S. M., \u0026amp; Seidel, S. (2000). The introduction to the NEA Teacher Education Initiative. Teaching and Change,~' 5-9. Nath, L.R., \u0026amp; Ross, S.M. (2001). The influence of a peer tutoring training model for implementing cooperative groupings with elementarJ students. Educational Technology, Research and Development, 49(2), 41-56. Ross, S. M., Sanders, W. L., Wright, S. P., Stringfield, S., Wang, L. W., \u0026amp; Alberg, M. (September 2001). Two- and three-year achievement results from the Memphis Restructuring Initiative. School Effectiveness and School Improvement, 1]_, 323-346. CREP Response to RFQ23-010 4 Summary of Interests During the past ten years, I have worked extensively with school districts, both regionally and locally, to develop and evaluate programs for improving student achievement. The primary focus of these studies bas been schools predominantly serving disadvantaged inner-city minority children. Currently, I am working on the formative and sumrnative evaluation of Comprehensive School Reform (CSR) projects at schools in various states. Additional ongoing research projects are studies of school restructuring designs as they are implemented in Memphis City Schools and other school districts and of professional development schools in seven national sites as part of the NEA Teacher Education Initiative (NEA-TEI). 2. Capacity and capability to perform education program evaluations including a writing sample (see appendix for writing sample). The Center for Research in Educational Policy (referred to in this document as the Center or CREP) is funded by the State of Tennessee as one of five Centers of Excellence located at The University of Memphis. The mission of the Center is to implement a research agenda associated with educational policies and practices in the preK-12 public schools of Tennessee and the nation, and to disseminate research findings so that they inform decisions made by educational practitioners and policymakers. Since 19.89, the Center bas served as a mechanism for mobilizing community and university resources to address educational problems and to meet the I University's commitment to primary and secondary schools. Functioning as a part of the College of Education, the Center seeks to accomplish its mission through a series of investigations conducted by Center personnel, College and University faculty, and graduate students. The Center's research agenda is developed through analysis of persistent or emerging issues in schools and their communities, changes occurring in teacher education programs, and recommendations from educational authorities. In order to plan and conduct inquiries relevant to issues associated with public schools, the Center supports research reflecting the following characteristics: potential for contributing to the solution of educational policy and practice issues\nan applied research and development focus\ncollaborations and partnerships with schools and other external organizations\ninterdisciplinary research teams\nmultiple modes of inquiry\nCREP Response to RFQ23-010 5 immediate and long-range planning\nand creation of databases to foster secondary analyses. Research outcomes are intended to provide a knowledge base for use by educational practitioners and policymakers by providing insight into the complexities of educational phenomena and offering recommendations for action. Through work in schools for over a decade, the Center has contributed to Tennessee policy decisions regarding teacher preparation and licensure, school governance and site-based decision making, and public school reforms. Additionally, the Center has gained national recognition for its contribution to discussions of issues such as reform of teacher education, educational equity, educational technology, school reform and restructuring, urban and multicultural education, interventions for at-risk students, and using formative evaluation methods for school improvement decision-making. In summary, the Center for Research in Educational Policy has extensive experience in evaluating diverse educational programs. The Center has developed numerous valid and reliable evaluation tools that have been effectively used by hundreds of schools and districts to examine the extent to which programmatic goals are being realized, and also fotmaking data-based improvement decisions. The Center's professional expertise, available evaluation resources, and proximity to the Little Rock School District all make it likely that CREP can assist the District in complying with the court order related to its Revised Desegregation and Education Plan. 3. Indication of the amount of available time between November 11, 2002 and March 1, 2003 for consulting and active work on this project. Dr. Steven Ross and staff from the Center for Research in Educational Policy can initially commit to ten hours per week for this project. This commitment is flexible, however, and can be negotiated based on the needs of the Little Rock School District. CREP Response to RFQ23-010 6 4. Two professional references who can attest to quality of work and ability to meet schedules and deadlines. RickBasoin Deputy Executive Director of Planning and Development SERVE 915 Northridge St., 2nd floor Greensboro, NC. 27403-2112 (800) 755-3277 RBASOM@serve.org Doris Redfield Director of Research AEL P.O. Box 1348 Charleston, WV. 25325-1348 (800) 624-9120 redfield@ael.org 5. Pending Lawsuits. None. IC !3: C Ill ll ::, ~ Ill., ..a r ro ~ 5ro1 c, ::, C: rt i: r. I \u0026gt;en lrl en., 0 C n....: : Ill Q 1\nCREP Response to RFQ23-010 7 Appendix Writing Sample 1 1 The writing sample is an executive summary. The full report will be provided upon request. PROGRESS AND OPTIONS REGARDING THE IMPLEMENTATION OF DIRECT INSTRUCTION AND SUCCESS FOR ALL IN TOLEDO PUBLIC SCHOOLS EXECUTIVE SUMMARY Prepared by: Center for Research in Educational Policy (CREP) at the University of Memphis www .memphis.edu/crep July 30, 2002 :J:\u0026gt;'lll t/l rl t/l 0 C (l ::i I ~- Ill Q rt 1m Progress and Options Regarding the Implementation of Direct Instruction and Success for All in Toledo Public Schools EXECUTIVE SUMMARY Purpose and Background The major goals of this research study were to evaluate the outcomes and provide recommendations on the implementation of two improvement models being used by multiple schools in the Toledo Public School (TPS) district. The district has taken bold steps to turn around some of its lowest performing schools, including the investment of about $2 million over the past four years to adopt two research-based comprehensive school reform models with solid national track records of effectiveness. These models, Direct Instruction (DI) and Success for All (SF A), have been implemented in a total of nine Toledo elementary schools. The design and methodology of the study, to be described below, was oriented towards answering the following research questions.  How do \"program\" (i.e., DI and SFA) schools compare to \"control\" (i.e., similar) schools in student achievement outcomes over time?  How well are program schools implementing their chosen models?  How do program schools compare to control schools in measures of school climate that make a difference in program implementation or student achievement?  What are key stakeholder reactions (e.g., district leaders, principals and teachers) to the effectiveness of the programs and their schools' ability to fully implement the programs?  Which factors appear to differentiate between program schools that are most and least successful in raising student achievement? Study Design The design of this project relies on a number of different approaches to determine program implementation. The findings and options for action reported here draw on multiple data sources consisting of:  Comparisons of student achievement results at program schools against similar TPS schools\n Targeted observations of the extent and quality of DI and SPA implementation\no Interviews of district and union leadership\n School climate inventories administered to teachers at program and control schools\n Teacher questionnaires administered to teachers teaching reading at program and control schools\n Focus groups comprised of program teachers that explored issues related to program implementation\nand  Interviews of principal and building representative. Nine schools participated in the evaluation, including three DI schools that began implementation in 1997-1998\nthree DI schools that began implementation in 1999-2000, one that began in 2001-2002\nand two SFA schools that began implementation in 1999-2000. (Note: As data was not yet available, the DI school that began implementation this year was not considered in the analysis of student achievement.)  For the student achievement study, the comparison sample consisted of all other (not SF A and DI) schools in the District, the scores for which (as explained below) were adjusted for school and student characteristics.  For the implementation analyses, experts in the school district selected matched control schools for DI and SF A sites based on prior achievement, SES factors, and ethnicity. There were six DI control schools and two SF A control schools. Again, the DI school that began implementation this year did not have a control school. Results Achievement Analysis Student achievement data was gathered from all available sources, including the Ohio proficiency test and the Stanford Achievement Test (SAT-9). Student results on these standardized tests in schools implementing DI and SF A were examined as well as results for all other (not DI and SFA) district schools. Results were compared for program and control schools and the analysis revealed the following. Overall Results. DI and SF A program schools included in the evaluation posted student achievement gains nearly equal to what would be expected of other Toledo schools serving similar student populations. DI Results.  Achievement gains tended to improve in DI schools from 1999 to 2001 in 2nd and 6th grade, but were still slightly below what would be expected given the pf)verty rate and pretest levels of the schools.  In 2001, DI schools as a whole produced a statistically significant negative effect for fourth grade.  Among the schools that implemented DI in 1997-1998, Mt. Vernon second grade students tended to post higher achievement gains than would be expected. Fourth and sixth grade achievement gains improved at Mt. Vernon from moderately below to roughly equal to the district average. Among 1999-2000 DI schools, student achievement improved substantially at King, particularly in 2nd and 6th grade. Second grade student performance declined precipitously at Warren from 1999 to 2001. 2 3: Ill ::, Ill IQ m 5l m ::, rt- ,~\u0026gt; 0 (') ~- Ill rtm SF A Results.  In both SF A schools, students at each grade level made achievement gains at a rate nearly equal to the district average for each year considered. School Climate Inventory (SCI) The main purpose of the School Climate Inventory (SCI) is to assess impacts of reform initiatives in relation to seven dimensions logically and empirically linked with factors associated with effective school organizational climates. 1 The inventory contains 49 items and responses are scored using a five-point scale ranging from strong disagreement (1) to strong agreement (5). Overall Results. SCI results showed no unusual positive or negative trends for any of the program or control groups, with the exception of one of the two SF A sites that had negative school climate results. DI Results. DI teachers reacted comparably to their control counterparts. SF A Results. SF A teachers reacted significantly more negatively than did the control teachers on three out of the seven dimensions (Environment, Leadership, and Order). One of the SF A schools bad very positive school climate results, while the other SF A site bad negative school climate results compared to both the controls and to national norms. Reading Teacher Survey (RTS) All teachers of reading at each DI, SF A, and control school were asked to complete the RTS, which contains 20 items teachers respond to using a five-point scale ranging from strongly disagree (1) to strongly agree ~). Items identified the specific program in the case of DI and SF A, but referred generically to the \"reading program\" for the control schools. Among the areas assessed were professional development, impacts on students, changes in teaching, support for the program, effects on technology use, and involvement of parents and the community. Overall Results. Findings for both DI and SF A sites indicated significant favorable teacher attitudes toward the DI and SF A reading programs relative to control group impressions of the district's general reading program. DI Results. DI teachers expressed more positive attitudes toward their reading program than did control teachers of their schools' approaches. Significant differences were obtained on 12 out of 20 items, with the largest effects indicated for: (1) external guidance and support\n(2) 1 The dimension on the Environment addresses pride in the school and caring about others\nthe Order dimension focuses on student behavior, discipline, and attendance\nthe Leadership dimension deals with the degree to which the administration is supportive, communicative, and effective\nInvolvement concerns the extent to which parents and the community are involved in the school\nthe dimension on Instruction considers the extent to which the instructional program is well developed and implemented\nthe Expectations dimension is the extent to which students are expected to learn and be responsible\nand the Collaboration dimension is the extent to which the administration, faculty, and students cooperate and participate in problem solving. 3 the reading program changing classroom learning activities a great deal\n(3) external professional development being valuable\nand, ( 4) student achievement being positively impacted. SF A Results. SF A teachers were significantly more positive than their control counterparts on 13 of the 20 items. The largest effects were obtained for items indicating a more positive teacher evaluation of: (1) the guidance provided by the school facilitator, support team, or others\n(2) changes in learning activities due to the reading program\n(3) the school's plan for evaluating the reading program\n( 4) the value of the professional development\nand, the involvement of parents. Interviews and Focus Groups Interviews were conducted with key stakeholders ( district leadership, union leadership, and a school board member), principals of schools implementing DI and SF A as well as Control schools, building level union representatives, and DI and SFA school facilitators. Teacher focus groups of about one-hour in length were held to provide background information about schools' usage of their selected programs to support more informed interpretations of outcome measures such as student achievement. Questions addressed teachers' experiences with and reactions to program implementation with regard to such areas as program appropriateness, resources, professional development, and outcomes. The principal at each of the nine schools participated in a one-hour, on-site interview. Interview questions addressed the principal's experiences and reactions to the program implementation and the associated outcomes for the school, students, faculty, and parents/community. Interviews with control school principals were conducted via telephone, using the same instruments. Results from DI Interviews and Focus Groups According to respondents (principals, teachers, and teacher union building representatives), the strengths of DI appear to be in the primary grades, especially with regard to phonics. The model also appears to them to be highly positive for special needs children, attributed to DI's structure, repetition, and appropriate ability grouping. Weaknesses in DI were perceived in the intermediate grades, including a lack of instruction in comprehension and higher-order thinking skills. Implementation, according to one teacher focus group, has gone well in the lower grades, but as reported by one building representative, the program lacks support among intermediate grade teachers. Almost all principals, building representatives, and teacher focus groups perceived a positive impact of DI on reading and interest in reading. High student mobility was identified as one of the biggest obstacles to DI implementation, since new students enter the school with no DI experience. Large class size was mentioned as another perceived impediment to implementation. A third impediment reported at some schools was high teacher turnover. In some schools, boredom and lack of interest during DI was identified as an issue, although others report increases in student motivation and enthusiasm about reading. 4 3: Ill ::, ill \"\" IQ (D s ~ g rt ~ :t\u0026gt;'P. tll \" ~ a n  I-' Ill ,~ Most respondents viewed initial DI training as positive. Several noted that training was inadequate for new/transfer teachers. Model developers appeared to have inconsistent contact with the schools. Across all schools, there appeared to be a need for improved district training and support, which was reported to be minimal and unspecific. In summary, teachers and principals in DI schools perceive the program to be very effective, especially in grades K-3, at teaching larger numbers of students to read. Teachers and principals indicate that students are more motivated and interested in reading and that they see progress in their classrooms on a regular basis. There appears to be a weakness in the training provided by model consultants and the district, particularly for new teachers entering the program buildings. Results from SFA Interviews and Focus Groups According to respondents, SF A implementation has been improving, but has been highly dependent on the rate of teacher turnover. The most effective elements of the SF A model were perceived to be the 90-minute block of reading time, ability grouping for reading, the use of phonics and tutoring at the primary levels, and program consistency from grade to grade. The least effective SFA elements were reported as the lack of nonfiction materials (an issue identified at both \"program\" and \"control\" schools), limited writing emphasis, time constraints, large class sizes, the scripted lessons, and the lack of alignment to Ohio standards. Focus groups in both schools perceived that SFA supports cooperative and team-based approaches. Other SF A classroom changes identified include smaller reading groups, flexible levels, individualized learning, interdisciplinary and project-based instruction, improved partnering, and more student enthusiasm for learning. Staff reported that special\"needs SF A students are ability-grouped according to reading level, and are given extra tutoring if necessary. I In summary, staff observed that students in both SF A schools appear to be more motivated to read. SF A schools reported that, according to SF A assessments, more students are reading on level, and students are reading more often. SF A principals reported increasing proficiency scores, although teachers in one school reported that test scores are down: One SF A school also noted improved communication, stronger student relationships, and fewer discipline problems during SF A implementation. In both SF A schools, teacher collaboration and teacher collegiality has reportedly improved, despite high teacher turnover in one of the schools. According to the focus groups, professional development and SF A training have been helpful overall at both schools. However, follow-up training and training for new teachers were viewed to be inadequate. Targeted Observations Independent, out-of-state consultants with expertise in the implementation of DI and SF A conducted site visits to the Toledo schools implementing these models. Targeted observations for each model follow. 5 IQ xi::: 0J 0J ::, 0J 1- -0 ro~I s ('I) ::, Cl rt C: 0 :x:,. 0J en rt en .... 00 0 ::i I-' 0J QI rt ('I) DI Targeted Observations In-depth observations of Toledo's DI schools by outside experts in the program indicate a rather inconsistent and incomplete level of implementation of the components of the DI model that are critical for accelerating student progress to reach grade-level performance. These expert observers noted that implementation is not yet geared to produce significantly more than a year's progress each year so that children can close the \"academic gap.\" On an encouraging note, the observers reported that school staff nearly unanimously identified a significant reduction in the number of children who are nonreaders, and a significant increase in the number of children who are encountering success during reading instruction. Specific recommendations to improve DI implementations are as follows: 1. Place more emphasis on accelerating student performance in kindergarten and first grade. 2. Increase reading instructional time to accelerate and expand student reading proficiency. 3. Provide children who are functioning below grade level with daily extra DI reading instruction. 4. Provide for structured reading in a wide variety of materials. 5. Place more emphasis on implementation of the DI language curricula. 6. Provide professional development to enable all teachers to reach high levels of proficiency in teaching DI. 7. Provide training and support for building principals to take a more active role in supporting implementation of DI model. 8. Provide more of a sense of urgency from the district level to ensure implementations are producing desired levels of student learning. This recommendation further stres,ses the need to establish a district-wide DI coordinator to monitor implementation and 'ensure consistency and quality in uses of DI across schools. 1 SF A Targeted Observations According to the SF A expert who visited the program schools, implementation in Toledo is uneven. While observations revealed that implementation of the SF A program is generally above average in curricular areas and both schools provided the requisite 90-minutes of reading instruction each day, several concerns exist. Implementation of the \"Reading Wings\" reading component in one school was below average. Both schools are below average in family support implementation and one school needs to improve in the area of teacher training. 6 ' tO 3:C:: Ill Ill ::, Ill 1- IO (I) I\u0026lt; El (I)~ ::, c:1 rt Cl \u0026gt;en r en 0 (\"l I-' Ill rt I (I) In terms of student performance, based on their analysis of SF A-provided student assessments, both schools report approximately 90% of first graders reading at or above grade level and close to 70% at or above grade in most second through sixth grades.2 According to the expert observer, teachers at each school appear to be working together to successfully implement the program in the face of implementation challenges. For instance, one site has been without a permanent facilitator for most of the 2001-2002 school year. The other SF A site has two facilitators, however, program implementation is challenging there as well due to the large student population and other conditions. The observer noted that the district appears to lack attention and focus to ensure that high quality program implementation is o.ccurring at each SF A site. Specific recommendations to improve SF A implementation are as follows: 1. Improve the alignment of SFA with the Ohio Proficiency Standards. 2. Ensure that adequate program facilitation is provided. 3. Improve the quality of professional training that SFA teachers receive. 4. Strengthen the implementation of SF A's writing component. 5. Implement fully the family support program component of the SFA model. Options for Action Our research team's goal in conducting this analysis was to support the ongoing efforts of all stakeholders in Toledo to engage in effective, evidence-based action to improve student achievement. We hope this report will provide the District the information it needs to weigh its options regarding the future implementation of two research-based, research-proven reading/school improvement programs. This report does not seek to promote or discredit either program or any alternative approach, or to call into question the well-intended and hard work of many district administrators and school staff. Rather, it lays out the available faqts, analyzes them and, on that basis, provides the following, forward-looking options. The District has several options regarding the future of Direct Instruction and Success for All in these schools. They range from the abandonment of the models entirely to their full, sustained, and potentially expanded implementation. At one end of this spectrum, before abandoning the models completely, the District would have to answer two key questions: 2 This statement on grade level performance refers to an SF A student assessment. This information contradicts other student assessment data provided by the Toledo Public Schools Office of Research showing that on the district assessment one SFA site has about 50% of its first graders and 60% of second graders performing on grade level while the other site has about 80% of first and second graders performing at grade level. 7 1. Have we done all that we could at the district and school level to fully implement these two programs in order to achieve successful student outcomes? 2. If we do decide to drop these programs, what do we have in hand, ready to go, that will produce better results? Given the observations detailed in our report, it would appear that the District could talce several more steps to assure high-quality implementations of these models. Furthermore, while the district may develop equally, or more effective, alternative approaches for raising reading achievement in these program schools, and perhaps across the entire district, no such option currently exists. Therefore, the district may wish to explore ways in which these programs could continue to operate-for a limited period of time, and with increased support and monitoring. The programs that the District has adopted have demonstrated success in many similar settings across the country. Therefore, given the right circumstances they may yet yield the desired-but as yet unattained---outcomes in Toledo. Based on the results of this study and our experiences as consultants and researchers in the area of comprehensive school reform, the following recommendations are offered for consideration by the TPS School Board and Superintendent. Improving Model Implementation this School Year. In the event the District decides to continue to operate these two models, for the 2002-2003 school year, there are a number of actions that should be considered in order to improve implementation. The District should:  Carefully review the detailed findings of the targeted observations provided in this report and make every possible suggested change in implementation. In the future, the district should monitor and act on the information provided by DI and SF A ongoing implementation checks and consider supplementing this data with other formative evaluation tools.  Establish a support, accountability and monitoring structure at the district level with one administrator charged with oversight of these models. This would ensure an ongoing focus on providing the support required for success. This individual-knowledgeable in the research-based programs and, preferably, about literacy instruction as well.-would oversee the quality of implementation across sites. The administrator would work closely with principals and school staff to meet their needs, and should have the authority and budgetary power to: a. Identify and oversee high quality providers of technical assistance, professional development, and supplies and materials\nb. Help identify and support school-based facilitators of the model\nc. Malce certain that materials and supplies are provided in a timely fashion to appropriate school staff\nd. Set and keep a master schedule for training new teachers, current teachers, and school-based program facilitators that complements District-provided training\nand 8 I-' 0J rt Cl) e. Increase principal involvement in and preparation for implementing models.  Negotiate and hold models and consultants to clear, transparent, performance-based agreements that specify what it is the District expects of the model provider(s), and consequences for success and failure. Improving Medium and Long Term Model Implementation. At the other end of the spectrum, indefinitely continuing and/or expanding implementation, would not appear to be a wise option given the limited academic success so far experienced, and the program implementation improvement needs noted in this report. Despite a national track record of success, unless a program can demonstrate substantial, long-term success in raising student achievement for Toledo's children, the program may not be the best option for meeting student needs. The District, and program advocates, must answer some tough questions before the programs are deemed worthy of long term support, including: 1. Why are we seeing less than adequate student progress in some of the program schools implementing these models? How can we track student achievement more accurately? 2. What is our capacity, interest, and commitment to fully implementing these models? Does the continuation of these models best serve the interest of Toledo's children? 3. How can we better monitor and assess the implementation of the models at the District level? 4. In the long term, what are the better solutions to meet the educational needs of Toledo's children? Other, intermediate and perhaps more viable choices exist for the District. For example, it might decide to choose a path of cautious optimism and allow the implementation of these models to continue for one or two more years-pending more results, and with additional supports. In our view, all schools wanting to continue their selected models should be given at least one year to show tangible progress. Presently, teacher support for the models being used is fairly strong at most schools. There are also community stakeholders who are both active and vocal in their advocacy of their chosen model. Using the present, third-party evaluation as a starting point for requiring tangible progress to be demonstrated within the next school year should present ostensibly a fair and reasonable plan to all stakeholder groups. The District should consider making subsequent policy decisions about model continuance on a case-by-case (school specific) basis. Even if, for example, the District chose to install a research-based, districtwide reading curriculum and approach, a decision could be made at that time whether the results at any given school were sufficient to justify allowing the particular school to continue with its current program. Some districts have chosen to allow schools to pursue individual program options, as long as they can demonstrate performance success, and prepare their students for further success in case they transfer, or when they graduate to their next grade level. 9 3: Ill ::, Ill IQ Cl) s Cl) ::, rt :i,,i tll tll .. 0 C C'l::, I-' Ill Ir 1~ Specific Options for DI Implementation. Although some DI schools have had sufficient time to achieve full implementation and raise student achievement, applications of DI still appear to be weak in many critical areas. Schools and the District need to show in the coming year that they are able and willing to achieve the effective program implementation required to significantly improve student achievement. Otherwise, there is little reason to believe that results will be more successful than shown in the present analysis. Over time the District might consider reducing the number of DI sites based on the different schools' performance and the quantity of teachers willing to make a long-term commitment to implementing the program. Creating more structures like the magnet schools reviewed in this study may be desirable both to promote greater teacher \"buy-in\" and reduce student mobility. Also, the District might consider whether to limit DI to the lower (preK-3) grades and phase out implementation in the intermediate grades ( 4-6). Alternatively, the District should provide substantially better and more consistent training to teachers working with the older primary students. The DI model has a track record of effectiveness in the intermediate grades, but only when proper, regular teacher training is conducted and a literature-rich student environment is cultivated. Specific Options for SF A Implementation. In the case of SF A, more time is needed for schools to gain experience in implementing the model. On the whole, SF A implementation is uneven and the district must pay more attention to implementing the model with fidelity. Before scaling up with any further SF A sites, the district should ensure full program implementation at both current sites. Particular attention must be paid to placing a trained SFA facilitator at each site, providi~g adequate professional development for all staff, and fully implementing the family support component of the model. Creating more structures like the magnet schools reviewed in this study may be desirable both to promote greater teacher \"buy-in\" and reduce student mobili~ The district should expect mixed student achievement results across SF A sites if implementation is not improved. Conclusion. Several years ago, the Toledo Public Schools initiated an exemplary pilot effort to implement two reading improvement programs that have among country's best track records of effectiveness in raising student achievement. The leadership of the District was to be commended then, for making the commitment to take on \"evidence-based\" reform. It has continued that commitment by commissioning this impartial, rigorous, and thorough examination of the results to date of those decisions. While there is room for disappointment that student outcomes are less than anticipated, and concern that program implementation needs to be improved, the District's reliance on following the trail of evidence to appropriate conclusions should provide hope that-whichever options are taken in the coming months and years-the result will be a better education for Toledo's children. 10 Quality Education and Management Associates A CONSULTANT COMPANY P.O. Box 26166 Little Rock, Arkansas 72221-6166 (501) 221-1178 or lxmcneal@netscape.net Dr. Kathy K. Franklin, Principal Consultant Dr. Larry McNeal, Principal Consultant Dr. Tom E. C. Smith, Principal Consultant Dr. Gordon E. Watts, Principal Consultant RECE:VED ./'/2pn7 DEC - 4 2002 / /,\n11/- /).:..I, 'v ( v, .f. OfACEOf DESEGREGATION IIONiTORJNG I \"For every complex problem there is a simple solution that is one hundred percent wrong. QEMA knows the right solution.\" cfQ...uahut cgdcawuv QM\n~ ~~ A CONSULTANT FIRM P. 0. Box 26166 Little Rock, Arkansas 72221-6166 (501) 221-1178 or lxmcneal@netscape.net November 4, 2002 Darral Paradis, CPPB, C.P.M. Director, Procurement and Materials Management Department Little Rock School District 1800 East 6th Street Little Rock, Arkansas 72202 Dear Mr. Paradis: 1bis letter is to introduce the consulting finn of Quality Education \u0026amp; Management Associates (QEMA) of Little Rock, Arkansas. Members of the firm are university faculty with specialized expertise in educational issues, P-12, as well as postsecondary. Firm members have actively consulted with public school districts, private businesses, and institutions of higher education in a variety of areas. Members of the firm are offering their services to the Little Rock School District in the area of program evaluation. In particular, members of the firm are able and prepared to assist Little Rock School District personnel in evaluating the various programs outlined in the recent school desegregation court ruling. We are looking forward to assisting the Little Rock School District as it moves forward to address the program evaluation issue. Sincerely, r'..__ 'L~\\ ~ Dr. Larry McNeal Principal Consultant 2 1. Curriculum Vitae Quality Education and Management Associates (QEMA) Vitae A summary of Quality Education and Management Associates (QEMA) members' evaluation activities are listed below. Dr. Kathy K Franklin 2001 to 2004, Evaluator, Alternative Classroom Experience Project (Joseph Pfeifer Kiwanis Camp) 1998 to 2000, Evaluator, Freshman Year Experience program (University of Arkansas at Little Rock) 1998 to 1999, Evaluator, Adult First Year Experience course (University of Arkansas at Little Rock) 1998 to 1999, Evaluator, Learning Communities Program (University of Arkansas at Little Rock) 1997, Evaluator, Learning Community Experiment (University of Arkansas at Little Rock) Dr. Larry McN eal 2001 to 2004, Evaluator, Alternative Classroom Experience Project (Joseph Pfeifer Kiwanis Camp) 2000 to 2001, Evaluator, Little Rock Scliool District Charter Elementary School (Little Rock School District) 2001 to 2002, Evaluator, Little Rock School District Charter Elementary School 1998 to 2000, Chair, College of Education Assessment Committee (University of Arkansas at Little Rock) 1997 to 1998 Dr. Martin Luther King, Jr. Elementary School, Gales Public School District (Galesburg, Illinois) 1997, Evaluator, Title 1: Summer School Evaluation - Peoria Public School District (Peoria, Illinois) 3 1995 to 1996, Evaluator, Illinois School for the Visually (Illinois Department of Rehabilitation Services) 1995 to 1996, Evaluator, Illinois Center for Rehabilitation and Education (Illinois Department of Rehabilitation Services) 1995 to 1996, Evaluator, Illinois School for the Deaf (Illinois Department of Rehabilitation Services) Dr. Tom E. C. Smith 2001 to 2004, Evaluator, Alternative Classroom Experience Project -Joseph Pfeifer Kiwanis Camp 1999 to Present, National Evaluation System 2001, Charlotte-Mecklenburg School District Dr. Gordon E. Watts 2002 to 2004, Evaluator, Alternative Classroom Experience Project (Joseph Pfeifer Kiwanis Camp) 2002, Program Evaluator, Teacher Quality Enhancement Grant (Delta Teacher Education Consortium) 2000, Program Evaluator, Higher Education Consortium for Careers in Early Childhood (University of Arkansas at Little Rock) 1992 to present, Consultant/Evaluator, Higher Learning Commission of the North Central Association of Colleges and Schools 1998 Program Reviewer (Oklahoma State Regents for Higher Education) The curriculum vitae for individual QEMA members are in Appendix A 4 2. Capacity And Capability To Perform Program Evaluations. QEMA has both the capacity and capability to perform program evaluation within the timelines identified in the recent school desegregation court ruling. The capacity and capability of QEMA is imbedded in its structure as a research based consulting firm with specialized expertise in educational issues, P-12, as well as postsecondary. Principal Consultants of QEMA have conducted a variety of types of program evaluations at the public school, community college, and university levels. The unique backgrounds and experiences of the principals enable them to bring together a wide spectrum of program evaluation models and statistical methodologies in order to create a program evaluation that is specifically tailored to their client's needs. - Currently, firm members are engaged in conducting a multi-year evaluation of the Alternative Classroom Experience Project for the Joseph Pfeifer Kiwanis Camp in Little Rock, Arkansas. The project provides educational services to at-risk elementary children from the Little Rock School and the Pulaski County Special School Districts. The experience acquired from this venture enhances QEMA's understanding and ability to provide program evaluation services to public school districts in Central Arkansas. A copy of a program evaluation completed by a QEMA principal consultant in Appendix B 5 3. Time Available For Consultation In response to the timeline requirements, QEMA consultant principals can be available between November 11, 2002 and March 1, 2003 on the following dates. The actual dates are subjected to mutual agreement between Little Rock School District personnel and QEMA personnel. Dates after March 1, 2003 are also available upon request. November 11-15: November 18-22 November 25-29 Pecember 2-6 December 9-13 December 16-20 January 6-10 January 13-17 January 20-24 January 27-31 February 3-7 February 10-14 February 17-21 February 24-28 4. Professional References A. QEMAGroup: Sanford Tollette Director Joseph Pfeifer Camp 5512 Ferndale Co I Little Rock, AR 72208 501.821.3714 Sanford@pfeifercamp.com B. Dr. Kathy Franklin Dr. Thea Zidonowitz Hoeft Division Chief for University College Director of Academic Advising Coordinator of First Year Experience University of Arkansas at Little Rock 501.569.3386 tmhoeft@ualr.edu 6 C. Dr. Larry McNeal Dr. Jean Brown Principal Dr. Martin Luther King, Jr. Elementary School Galesburg, Illinois 309. 691.8023 343.0409 D. Dr. Tom Smith 5. Pending Lawsuits There are no lawsuits or any other legal actions pending on Quality Education and Management Associates as a.consulting company or any of its principal consultants as individuals. I 7 Appendix A Curriculum Vitaes For Dr. Kathy K. Franklin Dr. Larry McNeal Dr. Tom E.C. Smith Dr. Gordon E. Watts 8 I EDUCATION KATHY K. FRANKLIN, Ed.D. Department of Educational Leadership, Policy, and Technology Studies The University of Alabama Box 870302 Tuscaloosa, Alabama 35487-0302 kfranldi@bamaed.ua.edu 205-348-2470 Kathy K. Franklin 1 Ed.D. Educational Leadership with a concentration in higher education. Department of Educational Leadership and Policy Analysis, East Tennessee State University, Johnson City, Tennessee, 1996. Dissertation: Exploring the Congruency Between Student Satisfaction and Institutional Effectiveness in Higher Education. Utilized a qualitative research design with a focus group method of inquiry to explore the congruency between criteria used by students to determine satisfaction with the higher education experience and criteria used by administrators to evaluate institutional effectiveness. The purpose of this research was to investigate the validity of using student satisfaction assessment as a definition of institutional effectiveness. M.B.A. Concentration in marketing and economics. Middle Tennessee State University, Murfreesboro, Tennessee, 1991.  B.S. Commerce and Business Administration with a concentration in marketing. University of Alabama, Tuscaloosa, Alabama, 1977. PROFESSIONAL EXPERIENCE Associate Professor of Higher Education Administration, Department of Educational Leadership, Policy\nand Technology Studies, The University of Alabama, 2002 to present. Associate Professor of Higher Education Administration, Department of Educational Leadership, University of Arkansas at Little Rock, 1999 to 2002. Taught courses on the following topics: history and philosophy of higher education, governance and public policy in higher education, finance of higher education, leadership theories in higher education, and qualitative research methods. Chair, Department of Educational Leadership, University of Arkansas at Little Rock, 1999 to 2001. I Assistant Professor of Higher Education, Department of Educational Leadership, University of Arkansas at Little Rock, 1996 to 1999. Taught courses on the following topics: history and philosophy of higher education, overview of higher education in the United States, college teaching problems and issues, designing a college curricula, qualitative research methods, dissertation seminar\nand advanced research methods. Team Instructor, Department of Educational Leadership and Policy Analysis, East Tennessee State University, Spring 1996 and Summer 1996. Taught the following courses in collaboration with senior faculty from the ELP A department: Professional Needs of Individuals and Groups\nGraduate Internship Program\nand Theories of Educational Administration. Instructor, Kathy K. Franklin 2  East Tennessee State University, College of Business, Department of Management and Marketing, Johnson City, Tennessee\n1992 to 1996. Full-time Temporary faculty status -Taught Principles of Marketing, Sales Force Management, Retail Management, Introduction to Business, and Organizational Behavior to undergraduate students.  Milligan College, Adult Education Program, Johnson City, Tennessee\n1994 to 1996. Adjunct faculty status -- Taught Marketing Communications in the adult accelerated M.B.A. program.  Northeast State Technical Community College, Blountville, Tennessee\nJanuary 1992 to August 1992. Adjunct faculty status -- Taught courses in business economics, finance, and marketing.  Tusculum College, Greeneville, Tennessee\nJune 1992 to October 1992. Adjunct faculty status -Taught graduate courses in strategic marketing management to adult students in the accelerated M.B.A. program. Graduate Assistant, Office of the Dean, College of Education, East Tennessee State University, Johnson City, Tennessee\nMay 1995 to August 1995. Worked with the Associate Dean to prepare for the 1995 NCATE visit. Assistant Director, Bell South Exemplary Superintendent Training (BEST) Program, East Tennessee State University, Department of Educational Leadership and Policy Analysis, Johnson City, Tennessee\nAugust 1994 to May 1995. Coordinated all activities related to the professional development of forty Tennessee educational leaders in preparation for a future superintendency position. The BEST program was a collaborative effort with the Bell South Foundation and East Tennessee State University. PUBLICATIONS Journal and Newsletter Publications Franklin, K.K. (2002). Qualitative coding: The allegory of the quilt. Research in the Schools 9(1 ), 65-71. Franklin, K.K. (2001, winter). The \"Mikey\" phenomenon: Reflections of a first year chair. The Department Chair 11(3), 26-28. Franklin, K.K., Chesser, J.S., Edleston, R.J., Edwards-Schafer, P., Marvin, S.R., \u0026amp; Satkowski, T. (2001). Faculty attitudes about instructional technology in a metropolitan university classroom. _Metropolitan Universities: An International Forum 12(1), 50-61. Franklin, K.K. \u0026amp; Lowry, C. (2001). Computer-mediated focus group sessions: Naturalistic inquiry in a networked environment. Qualitative Research 1 (2), i 169-184. Franklin, K.K. (2000). Shared and connected learning in a freshman learning community. Journal of the First-Year Experience and Students in Transition 12(2), 33-60. Conners, N.A. \u0026amp; Franklin, K.K. (Spring, 2000). Using focus groups to evaluate client satisfaction in an alcohol and drug treatment program. Journal of Substance Abuse Treatment, 18, 313-3\nW. Franklin, K.K. (1999). A theoretical framework for metropolitan student satisfaction. Metropolitan Universities: An International Forum 10(3), 81-88. Franklin, K.K. (1999). Forging the bonds: learning communities on an urban campus. The First-Year Experience Newsletter 11(3), 8.  Franklin, K.K. (1998). Looking in the looking glass: How administrators describe the effectiveness of the metropolitan university. Metropolitan Universities: An International Forum 9(3), 9-18. Manuscripts in Press Kathy K. Franklin 3 Franklin, K.K., Cranston, V., Perry, S.N., Purtle, D.K., \u0026amp; Robertson, B.E. (in press). Conversations with mTreatnrosiptioolnit.a n university freshmen. Journal of the Freshman Year Experience and Students in Manuscripts in Blind Review Perry, S.N. \u0026amp; Franklin, K.K. (2002). I am not the Gingerbread Man! Exploring the Experiences 'of College Students Diagnosed with AD/HD. The Journal of Research on Learning Diabilities. Manuscripts in Progress Franklin, K.K. A Tale of Four Freshman Friends: The Urban University Experience. Trawick, K. \u0026amp; Franklin, K.K. Responsibilities for the Effective Metropolitan University Department Chair. Franklin, K.K. Assessing a Freshman Year Experience Program. Franklin, K.K. The Freshman Year Experience: Increasing Confidence in the Ability to Succeed Crum, C. \u0026amp; Franklin, K.K. Mentoring Female Graduate Students. Kondrick, L. \u0026amp; Franklin, K.K. Predicting Success for Female Graduate Students Enrolled in Math, Science, and Engineering Programs. Marvin, USn. i\u0026amp;ve rFsritayn. klin, K.K. Gardner's Theory of Multiple Intelligences: Students in Transition at an Urban Franklin, K.K., Conners, N.A., \u0026amp; Perry, S.N. Teaching Naturalistic Inquiry: A Constructivist Approach. ERIC Publications Perry, S.N. \u0026amp; Franklin, K.K. An Analysis of College Students with AD/HD at a Private and Public Institution in Arkansas using a Grounded Theory Approach. (submitted, November 2002) Conners, N.A. \u0026amp; Franklin, K.K. (November, 1999). Fostering research by female graduate students through mentoring. Washington, D.C.: Educational Resources Information Center (ERJC) Clearinghouse on Higher Education. (ERIC Document Reproduction Service No. ED 435 757). Lowry, C. \u0026amp; Franklin, K.K. (November, 1999). Utilizing networked computer workstations to conduct electronic focus group sessions. Washington, D.C.: Educational Resources Information Center E(EDR I4C3)5 C7l6e2a)ri.n ghouse on Assessment and Evaluation. (ERIC Document Reproduction Service No. Marvin, S., Franklin, K.K., Edleston, R., Oberste, C., Routen, I.J., \u0026amp; Satkowsk.i-Harper, T. (November, 1999). Faculty attitudes about the use of technology in the college classroom. Washington, D.C.: Educational Resources Information Center (ERIC) Clearinghouse on Higher Education. Reference number IRO 19901. Franklin, K.K., Boggs, K.J., Conners, N.A., Crum, C.B., Nawarat, P., Ramirez, C.T., \u0026amp; Trawick, K.C. (1997). Defining institutional effectiveness for a metropolitan university. (Report No. HE 030 940). Washington, D.C.: Educational Resources Information Center (ERIC) Clearinghouse on Higher Education. (ERJC Document Reproduction Service No. ED 416 747). Kathy K. Franklin 4 Chamberlin, G.D. \u0026amp; Franklin, K.K. (November, 1997). Implications of the baby boom echo in arkansas. (Report No. HE 030 872). Washington, D.C.: Educational Resources Information Center (ERIC) Clearinghouse on Higher Education, (ERIC Document Reproduction Service No. ED 418 610). Franklin, K.K. \u0026amp; Shemwell, D.W. {1995). Disconfirmation theory: An approach to student satisfaction assessment in higher education. Washington, D.C.: Educational Resources Information Center (ERIC) Clearinghouse on Higher Education. (ERIC Document Reproduction Service No. ED 388 199). Franklin, K.K. \u0026amp; Knight, W.H. (1995). Using focus groups to explore student opinion. Washington, D.C.: Educational Resources Information Center (ERIC) Clearinghouse on Higher Education. (ERIC Document Reproduction Service No. ED 388 200). Franklin, K.K., Shemwell, D.W., \u0026amp; Witta, E.L. (1994). Hierarchical confirmatory factor analysis of a student satisfaction survey. Washington, D.C.: Educational Resources Information Center (ERIC) Clearinghouse on Assessment and Evaluation. (ERIC Document Reproduction Service No. ED 388 694). Franklin, K.K. (1994). Multivariate correlation analysis of a student satisfaction survey. Washington, D.C.: Educational Resources Information Center (ERIC) Clearinghouse on Assessment and Evaluation. (ERIC Document Reproduction Service No. ED 388 695). Workbooks Published for Conference Workshops Franklin, K.K., (1999). The search for a needle in a haystack, Part II: The art of coding. Workbook published for the annual meeting of Mid-South Educational Research Association, Point Clear, AL. Franklin, K.K., Donaldson, C.W., Hoeft, T.M., \u0026amp; Chapman, C.A. (1999). Assessing the service-learning component in a freshman year experience course. Workbook published for the American Association for Higher Education Assessment Conference. Denver, CO. Franklin, K.K., Donaldson, C.W., \u0026amp; Hoeft, T.M. (1999). Inspiration and perspiration: Combining service-\nlearning with the freshman year experience. Workbook published for the annual meeting of the National Association of Student Personnel Administrators. New Orleans, LA. Franklin, K.K., \u0026amp; Nawarat, P. (1998). The search for a needle in a haystack: Qualitative data analysis. Workbook published for the annual meetipg of the Mid-South Educational Research Association. New Orleans, LA. i Franklin, K.K. \u0026amp; Hoeft, T.M. (1998) Assessing the learning community: A survival kit. Workbook pSupbrilnisghse, dA Rfo. r the annual meeting of the South Central Freshman Success Initiatives Association. Hot Franklin, K. K. \u0026amp; Donaldson, C.W. (June, 1998). Assessing the learning community: A survival kit. Workbook published for the American Association for Higher Education Assessment Conference. Cincinnati, OH. PRESENTATIONS International Kathy K. Franklin 5 Franklin, K.K. \u0026amp; Donaldson, C.W. (1999, April). An analysis of two postsecondary learning environments: Learning communities and freshmen year experience courses. Poster session at the annual meeting of the American Educational Research Association, Montreal, Canada. Franklin, K.K. \u0026amp; Conners, N.A. (1999, April). Mentoring female graduate students in a qualitative research methods course. Roundtable discussion at the annual meeting of the American Educational Research Association, Montreal, Canada. National Marvin, S., Franklin, K.K., Chesser, J.S., Edleston, R., Edwards-Sharer, P., Oberste, C., Routen, I.J., \u0026amp;  Satkowski-Harper, T. (2001, October), Traveling through the Land of Oz: Teaching Qualitative Research. Poster Session at the Nursing Research Conference sponsored by Gamma Xi Chapter, Beta Chi Chapter, Sigma Theta Tau International Honor Society of Nursing, the University of Arkansas for Medical Services College ofNursing, and the Central Arkansas Veterans Healthcare System. Nursing Service. Little Rock, AR. Franklin, K.K, Donaldson, C.W., Hoeft, T.M., \u0026amp; Chapman, C.A. (1999, October). Assessing the servicelearning component in a freshman year experience course. Presentation at the annual meeting of the Coalition of Urban and Metropolitan Universities. Boise, ID. Franklin, K.K., Donaldson, C.W., Hoeft, T.M., \u0026amp; Chapman, C.A. (1999, June). Assessing the servicelearning component in a freshman year experience course. Workshop presented at the annual meeting of the American Association for Higher Education Conference on Assessment. Denver, CO. Franklin, K.K., Hoeft, T.M., \u0026amp; Donaldson, C.W. (1999, March). Inspiration and perspiration: _ Combining service-learning and the freshmen year experience. Paper presented at the annual meeting of the National Association of Student Personnel Administrators. New Orleans, LA. Franklin, K.K. \u0026amp; Donaldson, C.W. (1998, November). Developing the assessment strategy for a linked course learning corf-zmunity. Poster session at the Assessment Institute Best Practices Fair, Indiana University-Purdue University Indianapolis. Indianapolis, IN. Conners, N.A. \u0026amp; Franklin, K.K. (1998, November). Using focus groups to evaluate client satisfaction in an alcohol and drug treatment program. Poster session at the annual meeting of the American Evaluation Association. Chicago, IL. Franklin, K.K. (1998, June). Assessing the learning community: A survival kit. Workshop conducted at the 1998 American Association for Higher Education Assessment Conference. Cincinnati, OH. Franklin, K.K. (1998, February). Explaining student satisfaction with a metropolitan university.  Grant funded research presented at the annual meeting of the Coalition of Urban and Metropolitan Universities. San Antonio, TX. Franklin, K.K. (1996, April). Exploring the congruency between student satisfaction and institutional effectiveness in higher education. Dissertation research presented at the 17th annual National Graduate Student Research Seminar in Educational Administration sponsored by the University Council for Educational Administration. New York City, NY. Re~ional Perry, S.N. \u0026amp; Franklin, K.K. (2002, November) .. College Students with AD/HD. Presentation made at the 2002 annual meeting of the Mid-South Educational Research Association. Chattanooga, TN. Kathy K. Franklin 6 Trawick, K. \u0026amp; Franklin, K.K. (2002, November). Exploring Department Chair Roles in Metropolitan Universities. Presentation made at the 2002 annual meeting of the Mid-South Educational Research Association. Chattanooga, TN. Hoeft, T.M., Franklin, K.K., \u0026amp; Donaldson, C.W. (2002, April). Conversations with First Year Students: Implications for Teaching and Learning. Presentation made at the 2002 annual meeting of the Arkansas Academic Advising Network. Jonesboro, AR. Franklin, K.K., Cranston, V., Peny, S.N., Purtle, D.K., Roberston, B.E. (2001, November). Conversations with metropolitan freshmen. Paper presented at the annual meeting of the Mid-South Educational Research Association. Little Rock, AR. Franklin, K.K. (1999, November). The search for a needle-in-a-haystack, Part II: The art of coding. PWooinrkt sCholepa rp, rAesLe.n ted at the annual meeting of the Mid-South Educational Research Association. Conners, N. A. \u0026amp; Franklin, K.K. (1999, November). Fostering research by female graduate students through mentoring. Paper presented at the annual meeting of the Mid-South Educational Research Association. Point Clear, AL. Lowry, C. \u0026amp; Franklin, K.K. (1999, November). Utilizing networked computer workstations to conduct electronic focus group sessions. Paper presented at the annual meeting of the Mid-South Educational Research Association. Point Clear, AL. Marvin, S., Franklin, K.K., Edleston, R., Oberste, C., Routen, I.J., \u0026amp; .Satkowski-Harper, T. (1999, November). Faculty attitudes about the use of technology in the college classroom. Paper presented at the annual meeting of the Mid-South Educational Research Association. Point Clear, AL. Franklin, K.K., Conners, N. A., Edleston, R., Marvin, S., Oberste, C., Routen, I.I., \u0026amp; Satkowski-Harper (1999, November). Traveling through the Land of Oz: Teaching qualitative research. Poster pArLes. ented at the annual meeting of the Mid-Sotith Educational Research Association. Point Clear, Franklin, K.K. \u0026amp; Hoeft, T.M. (1998, October). Assessing a learning community: A survival kit. Workshop conducted at the South Central Freshman Success Initiatives Association. Hot Springs, AR. Franklin, K.K., Nawarat, P., \u0026amp; Ramirez, C. (1998,\nNovember). The search for a needle-in-a-haystack: Qualitative data analysis. Workshop conducted at the annual meeting of the Mid-South Educational Research Association. New Orleans, LA. Franklin, K.K., Chamberlin, G.D., Kennedy, R.L., \u0026amp; Sewall, AM. (1997, November). A dissertation survival Mkite. mPpahpise,r TprNe.s ented at the annual meeting of the Mid-South Educational Research Association. Franklin, K.K., Boggs, K.J., Conners, N.A., Crum, C.B., Nawarat, P., Ramirez, C.T., \u0026amp; Trawick, K.C. (1997, November). Defining institutional effectiveness for a metropolitan university. Paper presented at the annual meeting of the Mid-South Educational Research Association. Memphis, TN. Franklin, K.K., \u0026amp; Chamberlin, G.D. (1997, November). Implications of the baby boom echo in the midMsouemth. p hiPsa, pTeNr p. resented at the annual meeting of the Mid-South Educational Research Association. Kathy K. Franklin 7 Chamberlin, G.D., \u0026amp; Franklin, K.K. (1997, November). The baby boom echo: Implications for two-year colleges in Arkansas. Paper presented at the annual meeting of the Arkansas Association of Two- Y ear Colleges. Little Rock, AR. Franklin, K.K., \u0026amp; Knight, W.H. (1997, February). Exploring the congruency between student satisfaction and institutional effectiveness in higher education. Paper presented at the annual meeting of the Eastern Educational Research Association. Hilton Head, SC. Franklin, K.K., \u0026amp; Knight, W.H. (1996, November). Exploring the definition of institutional effectiveness in higher education. Paper presented at the annual meeting of the Mid-South Educational Research Association. Tuscaloosa, AL. Franklin, K.K., \u0026amp; Bartell, N.O. (1996, November). How to survive a qualitative dissertation. Training session- for graduate students at the annual meeting of the Mid-South Educational Research Association. Tuscaloosa, AL. Franklin, K.K., \u0026amp; Shemwell, D.W. (1995, November). Disconfirmation theory: An approach to student satisfaction assessment in higher education. Paper presented at the annual meeting of the Mid-South Educational Research Association. Biloxi, MS. Franklin, K.K. \u0026amp; Knight, W.H. (1995, November). Using focus groups to explore student opinion. Paper presented at the annual meeting of the Mid-South Educational Research Association. Biloxi, MS. Franklin, K.K., Shemwell, D.W., \u0026amp; Witta, E.L. (1995, March). Hierarchical confirmatory factor analysis of a student satisfaction survey. Paper presented at the annual meeting of the Eastern Education Research Association. Hilton Head, SC. Franklin, K.K. (1995, March). Relationship between student satisfaction and assessing program effectiveness. Paper presented at the annual meeting of the Eastern Education Research Association. Hilton Head, SC. I Franklin, K.K. (1994, November). Multivariate correlation analysis of a student satisfaction survey administered at East Tennessee State University. Paper presented at the annual meeting of the MidSouth Education Research Association. Nashville, TN. Local Franklin, K.K. (2000, January). Faculty attitudes about the use of instructional technology in the metropolitan. university classroom. Presentation of research findings to the University of Arkansas AatR L. ittle Rock (UALR) Strategic Planning Committee on Instructional Technology. Little Rock, Franklin, K.K. (1999, January). Explaining student satisfaction with a metropolitan university.\nPresentation of research findings to the UALR Retention Committee. Little Rock, AR. Conners, N.A. \u0026amp; Franklin, K.K. (1999, January). Using focus groups to evaluate client satisfaction in an alcohol and drug treatment program. Poster session at the University of Arkansas Medical School, Department of Psychiatry, Grand Rounds. Little Rock, AR. Franklin, K.K. (1998, August). Assessing learning communities. Faculty development workshop at UALR. Little Rock, AR. Franklin, K.K. (1998, April). Explaining student satisfaction with a metropolitan university. Presentation of research findings to the UALR Faculty Senate. Little Rock, AR. Kathy K. Franklin 8 Franklin, K.K. (1998, April). Explaining student satisfaction with a metropolitan university. Presentation of research findings to the staff of the UALR Office of Educational and Student Services. Little Rock, AR. SPONSORED RESEARCH Evaluator for the Freshman Year Experience program sponsored by a grant from the Office of Educational and Student Services at UALR. Completed a two-year student learning assessment for the PEA W 1300 courses based on the six learning objectives identified for the course by the PEAW 1300 Curriculum Committee. The assessment included a freshmen profile survey administered to approximately 1100 students, a freshman attitude survey administered to approximately 900 students, a service-learning evaluation, and course portfolio evaluation (n = 161). Data provided evidence to support that the six learning objectives were accomplished. 1998 to 2000. Evaluator for the Adult First Year Experience course sponsored by a UALR Curriculum Development Grant. 1998 to 1999.   Evaluator for the fall and spring, 1998-1999 learning communities sponsored by a UALR Curriculum Development Grant.. 1998 to 1999. Conducted four focus group sessions with administrators and four focus group sessions with students on the University of North Texas campus to explore student perception of educational satisfaction and administrator perception of institutional effectiveness. Purpose of research was to explore the congruency between the criteria used by students to determine student satisfaction and the criteria used by administrators to evaluate institutional effectiveness. Research sponsored by a Coalition of Metropolitan and Urban Universities. 1997. Conducted four focus group sessions with administrators and seven focus group sessions with students on the. University of Arkansas at Little Rock campus to explore student perception of educational satisfaction and administrator perception of institutional effectiveness. Purpose of research was to explore the congruency between the criteria used by students to determine student satisfaction ai\nid , the criteria used by administrators to evaluate institutional effectiveness. Research sponsored by a UALR Faculty Development Grant. 1997. Evaluator for a learning community experiment, conducted during the 1997 fall semester, linking three freshmen core courses: speech communication, rhetoric and writing, and anthropology sponsored by an UALR Assessment Grant. 1997. ' PROFESSIONAL SERVICE Service to the Profession Continuous Service Member, Editorial Board, Research in the Schools, Mid-Sou.th Educational Research Association. 2002 to present. Member, Editorial Board, Journal of the First-Year Experience and Students in Transition, University of South Carolina Press. 1999 to present Member, Editorial Board, The Qualitative Report. http://www.nova.edu/ssss/QRJindex.html. 2001 to present. Referee, Qualitative Research, Sage Publications. 2001 to present. Research Coordinator, Coalition for Urban and Metropolitan Universities. Responsible for managing the Ernest Lynton Research Grant Program and Paige E. Mulhollan Dissertation Award Program. 1999 to 2002 2001 Kathy K. Franklin 9 Panel Member, Plenary Session, Developmental Education - Issues and Expectations, Arkansas Association for Developmental Education 2001 Fall Conference. Hot Springs, AR: October, 2001. 2000 Accepted an invitation to serve on the Strategic Study Committee for Roanoke College, Roanoke Virginia as a member of the Southern Association of Colleges and Schools (SACS) Visiting Team. November, 2000 . Mid-South Educational Research Association - 2003 President Mid-South Educational Research Association - 2002 Vice-President and President-Elect, 2002 Chair, Outstanding Paper and Dissertation Awards Committee Mid-South Educational Research Association - 2001 Site Coordinator, 2001 Annual Meeting, Little Rock Arkansas Member, Program Review Committee for the 2001 annual meeting Representative-at-Large, Board of Directors Mid-South Educational Research Association - 2000 Nominated by Board of Directors to serve as Secretary/Treasurer for MSERA. 2000 Site coordinator for the 2001 annual meeting to be held in Little Rock, Arkansas. Member, Program Review Committee for the 2000 annual meeting. Mid-South Educational Research Association - 1999 Elected by members of the Board of Directors to serve as a Representative-at-Large for MSERA. Member, Program Review Committee for the 1999 annual meeting. Mid-South Educational Research Association - 1998 Chair, Graduate Student Advisory Committee. Member, Program Review Committee for the 1998 annual meeting. Discussant, Mid-South Educational Research Association Conference, Discussion on Teaching and Learning. November, 1998. Session chair, Mid-South Educational Research Association Conference. Discussion on Early Childhood Education. November 1998. Session chair, Mid-South Educational Research Association Conference. Discussion on Teaching and Leaming. November, 1998. Mid-South Educational Research Association - 1997 Chair, Graduate Student Advisory Committee. .. Member, Program Review Committee for the 1997 annual meeting. Panel member, Mid-South Educational Research Association. Training session for graduate students titled: Life After Graduation: More Things You Need to Know About the Job Market. November 1997. Panel member, Mid-South Educational Research Association Conference. Training session for doctoral candidates titled: Dissertation Survival Kit. November 1997. g r. i ~ rt 01 .r.t. g Kathy K. Franklin I 0 Session chair, Mid-South Educational Research Association Conference, presentation of the 1997 Outstanding Paper awards. November 1997. Mid-South Educational Research Association - 1995 to 1996 Member, Graduate Student Advisory Committee. Session chair, Mid-South Educational Research Association Conference. Discussion on Higher Education. November, 1996. Session chair, Mid-South Educational Research Association Conference. Discussion on Test-Ta.long. November, 1996. Session chair, Mid-South Educational Research Association Conference. November, 1995. Service to The University of Alabama Member, Department of Educational Leadership, Technology, and Policy Studies Assessment Committee. 2002 to present. Service to the University of Arkansas at Little Rock Service to 2002 Member, UALR Department of Excellence Selection Committee. Jan. 2002 to April 2002 Service to 2001 Completed a two-year assessment of the PEA W 1300 courses at UALR for the Office of Educational and Student Services. The final assessment report included a detailed profile of approximately 1100 UALR freshmen emolled from the fall 1998 to spring 2000. The report also included the findings from a pre- and post-administered attitude survey, evaluation of course portfolios, and evaluation of service-learning activities. Data were analyzed using SPSS-PC v.9. Vice-Chair, UALR Graduate Council. 2000 to 2001 Chair, UALR Graduate Council Personnel Advisory Committee. 2000 to 2001 Evaluator, Freshman Year Experience Program at UALR. Responsible for developing and implementing an assessment strategy for all PEA W 1300 courses taught as a component of the Freshman Year Experience Program at UALR. 1998 to 2001 Member, Advisory Committee to the UALR Chancellor.  Serving on a committee of approximately 25 UALR faculty and administrators providing the Chancellor with feedback on proposed policies designed to enhance the metropolitan mission of the university. 1997 to 2001 Member, UALR Graduate Council. 1996 to 2001 Service to 2000 Member, elected by the Faculty Senate to serve 3$ a member of the UALR Research Committee. Responsibilities of the committee include  reviewing grant proposals written by UALR faculty for funding by the Office of Research and Sponsored Programs. 1998 to 2000. Chair, UALR Graduate Council Curriculum Subcommittee. 1998 to 2000. Evaluator, Service-Learning component of the Freshman Year Experience Program at UALR. Responsible for developing and implementing an assessment strategy for the service-learning component of PEAW 1300 courses. Member, UALR Faculty Appeals Committee. 1998 to 2000. Member, UALR Retention Committee. 1998 to 2000. Member, UALR Graduate Council Subcommittee on Graduate Curriculum. 1996 to 2000. Service to 1999 Member, Freshman Year Experience Brochure Development Committee. Spring, 1999. Member, First-Year Experience Curriculum Committee. 1998 to 1999. Member, UALR Academic Appeals Committee. September 1997 to 1999. Service to 1998 Member, Grant writing team to develop a grant proposal for the Kellogg Foundation visit. April 1998. Kathy K. Franklin 11 Member, UALR World Humanities Core Assessment Group. Served as the Core Assessment Committee liaison on this committee. Responsibility of the committee was to develop a core assessment strategy. March 1997 to 1998. Member, Advisory Subcommittee to the UALR Chancellor. Purpose of subcommittee was to provide s1u9p9p8o. rt for the development of a grant proposal for a 1997 Kellogg Foundation grant. 1997 to Member, UALR Core Assessment Committee. 1996 to 1998. Member, UALR Core Assessment Subcommittee on Implementing the University Core Assessment Plan. 1996 to 1998.  Co-authored the nomination packet for the John Templeton Award for the Freshman Year Experience Program at UALR. December, 1998. Attended the First-Year Experience Seminar hosted by the National Resource Center for The First-Year Experience and Students in Transition and the University of South Carolina held in Little Rock, AR. April 6 - 7, 1998 Attended the Developing Larning Communities conference sponsored by the University of Miami in Miami, Florida. January 1998. Service to 1997 Developed and implemented an assessment strategy for a learning community experience at UALR at the request of the Office of the Provost and sponsored by a UALR assessment grant. The learning community linked three freshmen courses: Anthropology, Rhetoric and Writing, and Speech Communications. This course was designed based on the research of Vincent Tinto. Summer 1997 Conducted a focus group session of student leaders at UALR, analyzed the data, and drafted an executive summary of the findings for UALR administrators at the request' of the Office of the Provost. The purpose of the focus group session was to explore student leader attitudes concerning the strengths and weaknesses of UALR. The information was used to develop a SWOT analysis for the UALR five-year strategic plan. July 1997 Attended the 1997 Service Learning retreat sponsored by the Office of the Chancellor and the Office of the Provost. Invited by the Chancellor to join 65 UALR faculty and administrators in a daylong retreat discussing service-learning strategies for UALR. Attended the Critical Thinking Weekend Workshop, featuring Dr. Stanley Paul, at the request of the Office of the Provost in Seattle, Washington. May 1997. Visited with key change agents at Portland State University at the request of the Chancellor for the purpose of \"bridge-building\" between the two metropolitan campuses. May 1997. Service to 1996 Attended the Indiana University - Purdue University Indianapolis Assessment Conference~ held in Indianapolis, Indiana, at the request of the Office of the Provost. November 1996. Service to the College of Education, University of Arkansas at Little Rock Service to 2002 Chair, Department of Educational Leadership Annual Review Committee. Spring, 2002. Service to 2001 Department Chair. 1999 - 2001 Attended the annual conference for the American Association of Colleges of Teacher Education. Dallas, Texas. 2001. Member, Annual Peer Review Committee. 2001 Member2, 0F0a0c.u lty Search Committee to select an Associate Professor of Higher Education Administration. Member, Faculty Search Committee to select an Assistant Professor of Higher Education Administration. 1999 to 2001. a ( E g r: Q Service to 2000 Member, Faculty Search Committee for the Vision Education Program. Spring, 2000. Member, Admissions, Retention, and Exit Committee. 1998 to 2000. Service to 1999 Kathy K. Franklin 12 Discussant, Millennium Forum, Metropolitan Focus Lecture on Instructional Technology, Dr. James In.man, featured speaker. September 30, 1999 Chair, UALR, College of Education Assessment Committee. Responsible for the on-going assessment of ANuCgAuTstE 1 s9t9a7n dtaor d1s9 9a9n.d the implementation of continuous-improvement strategies for the college. Chair, College of Education Assessment Readers Team to critique the college 1999 assessment reports. 1999 Elected by faculty in the Department of Educational Leadership to serve as the faculty reporter for all faculty meetings and to serve as acting chair for the department during the absence of the department chair. 1998 to 1999. Service to 1998 Chair, College of Education NCATE Readers Team. Responsible for reviewing and critiquing the NCATE report for the 1998 spring visit. August 1997 to April 1998. Member, Readers Team for the 1997-98 UALR Annual Assessment Progress Report. May, 1998 Member, College of Education NCATE Steering Committee. Coordination of the preparation for the NCATE 1998 spring visit. August 1997 to April 1998. Guided College of Education faculty through a S.W.O.T. analysis to develop the five-year strategic plan. September 1998. Service to 1997 Chair, Department of Educational Leadership Assessment Coordination Committee. 1996 to 1997. Submitted an application to serve as a Research Mentor for the McNair Scholars Program. FebI1\n1ary 1997. PROFESSIONAL MEMBERSHIPS Member, Arkansas Academic Advising Network. 2002 to present. Member, American Association of University Women. 2000 to present. Member, Mid-South Educational Research Association. 1994 to present. Member, The American Educational Research Association. 1999 to present. Member, The American Association of Higher Education. 1997 to 2001. Member, Eastern Educational Research Association. 1995 to 1997. PROFESSIONAL HONORS Kathy K. Franklin 13 Nominated for the 2001 Faculty Excellence Award in Research for the College of Education at the University of Arkansas at Little Rock. December, 2001 A warded the 1999 Faculty Excellence Award in Service for the College of Education at the University of Arkansas at Little Rock. April, 1999. Nominated by students in the Higher Education Administration program for the 1997-1998 Faculty Excellence Award in Teaching. December 1997. Awarded the East Tennessee State University, College of Education, 1997 Outstanding Dissertation Award. Nominated by Dr. Hal W. Knight, Associate Dean, College of Education. Selected by the University Council for Educational. Administration to attend the 17th annual National Graduate Student Research Seminar in Educational Administration held in New York City, 1996. I CURRICULUM LIST East Tennessee State University Higher Education Administration ELPA 6581 Internship with the Office of Enrollment Management ELPA 7820 Administration in Higher Education ELP A 68 80 Higher Education Finance and Law ELP A 6810 Theory of Educational Admjnistration ELP A 6010 Seminar in Education Administration and Organizations HDAL 5720 Student Personnel Services ELPA 6957 ELPA 6957 ELPA 6957 ELPA 6957 ELPA 6957 Special Topics - \"Leadership in Professional Development Centers.\" Special Topics - \"Program Planning in Continuing Education.\" Special Topics - \"Proposal and Grant Development.\" Special Topics - \"Politics of Higher Education.\" Special Topics - ''Leadership Studies.\" Education Foundations ELPA 6730 History and Philosophical Foundations of Education ELP A 6906 History and Philosophical Issues in Higher Education HDAL 5200 Human Relations and Group Processes Education Statistics Advanced Research and Analysis Kathy K. Franklin 14 Research ELPA 7810 ELPA 6950 ELPA 6901 MKTG5900 MKTG 5900 Independent Study - \"Multivariate Correlation Analysis\" Independent Study - \"Hierarchical Confirmatory Factor Analysis using LISREL 7.0\" Independent Study - \"Investigating the Use of Disconfirmation Theory to Measure College Student Satisfaction\" Middle-Tennessee State University Management and Marketing MGMT0660 Study of Organizations MGMT0665 Seminar in Operations Management BAD 0698 Business Policy MKT MKT MKT MKT 0680 0683 0685 0682 . Marketing Strategy Marketing Systems International Marketing Seminar Marketing Behavior General Business Theory ECON 5110 Economics and Business Decisions (East Tennessee State University) ECON 5240 Applied Macro Economic Theory (East Tennessee State University) FIN 0671 Advanced Financial Analysis lNFS 0671 Systems Analysis ACTG 0691 Accounting and Business Decisions REFERENCES Dr. Candace W. Burns, Chair Department of Educational Leadership University of Arkansas at Little Rock 2801 South University Ave. Little Rock, Arkansas 72204-1099 Phone: 501-569-3367 E-mail: cwbums@ualr edu Dr. Larry McNeal, Professor Department of Educational Leadership University of Arkansas at Little Rock 2801 South University Ave. Little Rock, Arkansas 72204-1099 Phone: 501-569-3552 E-mail: lxmcneal@ualr.edu Dr. Gordon E. Watts, Professor Department of Educational Leadership University of Arkansas at Little Rock 2801 South University Ave. Little Rock, Arkansas 72204-1099 Phone: 501-569-3267 E-mail: gewatts@ualr.edu Kathy K. Franklin 15 DR. LARRY MCNEAL BUSINESS ADDRESS University of Arkansas at Little Rock 2801 S. University Avenue Department of Educational Leadership Little Rock, Arkansas 72204-1099 Office 501-569-3552 Fax 501-569-3547 lxmcneal@ualr.edu HOME ADDRESS 15806 Patriot Drive Little Rock, Arkansas 72212-2606 501-221-1178 lxmcneal@netscape.net or 1.mcneal59@hotmail.com PROFESSIONAL PREPARATION Ph.D. Educational Administration, University of Wisconsin, Madison, WI (1990) Emphasis: Fiscal and Community Support for Public Education Concentrations: Educational Finance, School-Co=nity Relations, and Public School\\ Administration M. S. Educational Administration, University of Wisconsin, Madison, WI (1989) Emphasis: Administration and Organization of Public Education Concentration: Educational Administration (Public School) M. A. Public Affairs. University oflowa, Iowa City (1976) Emphasis: Public Administration Concentration: Urban Administration B. A. Business Administration and Political Science, Dakota Wesleyan University, Mitchell, SD {1975) Emphasis: Political Economics Concentrations: Business Administration, Political Science and Economics Licensure: School Business Management (No longer active) Insurance Agent (No longer active) Entrepreneurship: Quality Education and Management Associates, President PROFESSIONAL WORK EXPERIENCE Professor, Department of Educational Leadership, Graduate School of Education, University of Arkansas at Little Rock, Little Rock, Arkansas, 1998 to the present Visiting Professor, Department of Educational Management \u0026amp; Development, Graduate School of  Education, New Mexico State University, Las Cruces, New Mexico, Summer 2000 Educational Administration and Supervision Program Coordinator, Department of Educational Leadership, Graduate School of Education, University of Arkansas at Little Rock, Little Rock, Arkansas, 1998 to 2001 Visiting Professor, Department of Educational Management \u0026amp; Development, Graduate School of Education, New Mexico State University, Las Cruces, New Mexico, Summer 1998 Associate Professor, Department of Educational Administration and Foundations, Graduate School of Education, Illinois State University, Normal, Illinois, 1995 to 1998 Visiting Professor, Bellver International College, Trenton State College (now College of New Jersey), Graduate School of Education, Palma de Mallorca, Baleares, Spain, Summer 1996 Associate Director, Office of Educational Finance, Center For Higher Education and Educational Finance, qraduate School of.Education, Illinois State University, Normal, Illinois, 1995-1997 State Coordinator, Illinois Education Policy Fellow Program, Institute for Educatio~ Leadership, Graduate School of Education, lliinois State University, Normal, lliinois 1994-1996 Research Associate, Center for the Study of Educational Finance, Graduate School of Education, lliinois State University, Normal, lliinois, 1993-1995 Assistant Professor, Department of Educational Administration and Foundations, Graduate School of Education, lliinois State University, Normal, Illinois, 1993-1995- Visiting Professor, Department of Educational Administration and Foundations, Graduate School of Education, University ofNorthern Iowa, Cedar Falls, Iowa, Summer 1993 Research Associate, Center for the Study of Small/Rural Schools, Graduate School of Education, University of Oklahoma, Norman, Oklahoma, 1992-1994 Adjunct Fellow, Center for Research on Multi-Ethnic Education, University of Oklahoma, Norman, Oklahoma, 1991-1992 Danforth Principal Preparation Program Co-Facilitator, Graduate School of Education, University of Oklahoma, Norman, Oklahoma, 1991-1993 Assistant Professor, Department of Educational Leadership and Policy Studies, Graduate School of Education, University of Oklahoma, Normal, Oklahoma, 1991-1993 Coordinator of Multicultural Affairs, Wisconsin Alumni Association, University of Wisconsin, Madison, Wisconsin, 1990-1991 Budget Analyst Intern, University of Wisconsin System Administration, Madison, Wisconsin, 1989-90 Equal Rights Officer, Division of Care and Treatment Facilities, Wisconsin Department of Health and Social Services, Madison, Wisconsin, Fall/Spring, 1988/1989 Budget and Management Analyst Intern, Wisconsin Department of Public Instruction, Madison, Wisconsin, Summer, 1988 Marketing Manager, WarBuc Educultural Publications, Madison, Wisconsin, 1985-1987 Business Mathematics Instructor, Business Department, Madison Area Technical College, Madison, Wisconsin, Fall 1985 and Spring 1987 Finance Marketing Representative, John Deere \u0026amp; Company, Moline, lliinois, based in Madison, Wisconsin, 1978-1984 Commercial Service Representative, Honeywell, Incorporated, Minneapolis, Minnesota, based in Des Moines, Iowa, 1977-1978 Public Administration Intern, Mayor's Office, City of Davenport, Davenport, Iowa, 1976 2 DISSERTATION McNeal, L. (1990). The role of education for employment councils in education for employment programs. University of Wisconsin-Madison. RESEARCH INTERESTS School Cornrnunitarianism (the fundamental relationship between schools and their co=unities) Organizational Change (change processes and organizational effectiveness) Educational Finance ( adequacy and equity of funding for public education) Program Assessment and Evaluation GRADUATE COURSES TAUGHT Administration and Organization of Schools Advanced Administrative Theory and Behavior Administrative Problem Solving Organizational Change Educational Politics and Policy Dissertation Proposal Development Educational Finance School Business Administration Human and Fiscal Resources Management Educational Public Relations Introduction to Doctoral Studies Organizational Development PUBLICATIONS: REFERRED McNeal, L. \u0026amp; Christy, W.K. (In Press, 2003). The locus of control issue in standard-based accountability. Educational Considerations. Christy, W .K. \u0026amp; McNeal, L. (2002). Influence of school board members on state legislation in Arkansas. ERIC Document Reproduction Service No. EA 031517. McNeal, L. (2002). The school-community relations profile model: Combining school district and community-based data. In J. Thomas Owens and Jan C. Simmons (Eds.), In creating quality reform: Programs, communities, and governance (67-81). Upper Saddle River, NJ: Pearson Education. Chesser, J.S., \u0026amp; McNeal, L. (2001). Educational community study circles: How superintendents can enhance school improvement through community dialogue. ERIC Document Reproduction Service No. ED 446 370. Christy, W.K., \u0026amp; McNeal, L. (2000). Implications oflegislative policy development for public school districts. F. Kochan (Ed~). Southern Regional Council on Educational Administration Yearbook Chesser, J.S., \u0026amp; McNeal, L. (1999). School improvement through community dialogue: The first community study circles on education in Arkansas and Oklahoma. ERIC Document Re:production Service No. ED 436 694..  Mogilka, J, Ashby, D.E, \u0026amp; McNeal, L., (Eds.). (1996). Planning \u0026amp; Changing, 27(1\u0026amp;2). McNeal, L. (1995). Fulfilling promises in the land of Will Rogers: A look at performance indicators in selected school districts since the enactment of Oklahoma house bill 1017. School Finance Policy Issues in the States and Provinces: Annual Update 1995 (135-138), C. Edlefson (Ed). The Ohio State University: Policy Research for Ohio-Based Education. McNeal, L. \u0026amp; Reed, R. (1995). Building a school-community relations profile through sociological inventorying. People \u0026amp; Education: The Human Side of Schools, 3(3), 371-386. 3 McNeal L., et al. (1994). National Sallie Mae winners and their principals. National Forum of Educational Administration and Supervision Journal, 11(3), 3-10. McNeal, L. (1994). Focusing on at-risk students: Case study of John Wilkinson Elementary School. Illinois School Research and Development Journal, 31(1), 7-10. McNeal L., \u0026amp; Ashby, D. (1993). Site-based management and changing relationships. Illinois School Research and Development Journal, 31(1), 7-10. McNeal L., \u0026amp; Lehman, B. J. (1993). A vision of the future: The full-service school. Planning and Changing, 24(3/4), 140-154. BOOK PROPOSAL IN PROGRESS Christy, W.K., \u0026amp; McNeal, L. (2001). Working Title: The Superintendency: Theory to reflective practice. To be submitted to Wadsworth. Belmont, CA. MANUSCRIPTS IN PROGRESS McNeal, L. (2001). The contextual world of education for children and the school-co=unity: James coleman and the effective schools movement. To be submitted to Humanistic Counseling, Education and Development. McNeal, L., \u0026amp; Christy, W.K. (2001). Rethinking the school district model of funding for individual schools: Co=ents about site-based management of resources. To be submitted to Planning and Change. Christy, W.K., \u0026amp; McNeal, L. (2001). Implications of charter schools and home schooling for the funding of public schools. To be submitted to Journal of School Leadership. PUBLICATIONS: MONOGRAPHS , McNeal, L., et al. (1993). Common sense: Plain talk to legislators about school finance. Center For the Study of Educational Finance. Illinois State University. PUBLICATIONS: NON-REFERRED McNeal, L. (December, 1990). The role of education for employment councils in education for employment programs. Dissertation. McNeal L. (1993). The education of African-American children in Oklahoma. State of Oklahoma: Annual Report. Urban League of Greater Oklahoma City, Inc., 36-45. McNeal, L., First, P. F., \u0026amp; Knudson, D. P. (1993). Evaluating the University of Oklahoma Danforth Principal Preparation Program. Connections. 1 (2), 3. McNeal, L. (1992). University of Oklahoma report. Danforth Programs for the Preparation of School Principals Newsletter, 1 (2). 3.  McNeal, L. (1987). From the desk of. National Multicultural Banner, 5 (6), 2. McNeal, L. (1986). From the desk of: Literacy, who's problem is it anyway? National Multicultural Banner, 5 (3), 2. 4 McNeal, L. (1986). The Black collegians guide to graduate fellowships for minority students. National Multicultural Banner, 5 (2), 9. CITED IN EDUCATION WEEK In the area of educational finance, Education Week has quoted me on several occasions. I have been quoted in the following articles: 11/26/97 in News ILL. Lawmakers Get One More Try To Pass School Funding Reforms 6/11/97 in News ILL. Lawmakers Duck Vow To Revamp Funding 3/26/97 in News ILL. Audit Questions Oversight of ILL. Education Agency 2/5/97 in News ILL. Odds Seen Better for F1.lllding Reform in ILL. REPORTS McNeal, L. Little Rock school district charter elementary school evaluation report for the 2001- 2002 school year. Prepared for the Little Rock School District, November, 2002. McNeal, L., et. al. The college of education assessment report: 2000-2001: University. of Arkansas at Little Rock. Prepared for the Provost's Advisory Group on Assessment, June, 2001.  McNeal, L. Little Rock school district charter elementary school evaluation report for the 2000- 2001 school year. Prepared for the Little Rock School District, June, 2001. McNeal, L. Projected student enrollment for the 2000-2001 school year: 6th and 7th grade student racial make up report. Prepared for Pulaski County Charter School Inc., April, 2001. McNeal, L. Enrollment trends in the Little Rock, North Little, and Pulaski County Special school districts: 1995-1996 to 1999-2000. Prepared for Pulaski County Charter School Inc., November, 2000. Coleen, B.C., Driskill, G., Leslie, S., McNeal, L., Mitchell, W., Taylor, C., \u0026amp; Webb, R. Provost's advisory group on assessment: University of Arkansas at Little Rock. Summer 2000 report, July, 2000. I McNeal, L., et. al. The college of education assessment report: 1999-2000: University of Arkansas at Little Rock. Prepared for the Provost's Advisory Group on Assessment, June, 2000. McNeal, L. Student enrollment needs assessment study of the Illinois school for the visually impaired, Illinois Center for Rehabilitation and Education and Illinois School for the Deaf. Prepared for the Illinois Department of Rehabilitation Services, Odober, 1995. McNeal, L, First, P., Walker, V., \u0026amp; Hobson, B. An inquiry into alleged cultural insensitivity at Capitol Hill High School. Prepared for Oklahoma City Public School District, March, 1993. McNeal, L, et al. School choice: Open enrollment and post secondary options. Prepared for the Association of Wisconsin School Administrators, March, 1990. McNeal, L. County veterans service officer training manual. Wisconsin Department of Veterans Affairs, June, 1990. McNeal, L. A Review of health care and medical services provided by the United States department of veterans affairs. Prepared for the Division of Veterans Programs, Wisconsin Department of Veterans Affairs, August, 1989. McNeal, L. A review of health care grants and the Wisconsin Veterans Home in King, Wisconsin. Prepared for the Division of Veterans Programs, Wisconsin Department of Veterans Affairs, August, 1989. 5 McNeal, L. Mendota mental health institute: An analysis of an organization in crisis. Prepared for the Division of Care and Treatment Facilities of the Wisconsin Department of Health and Social Services, December 1988. PRESENTATIONS: REFERRED McNeal, L., Christy, W.K., \u0026amp; Lewis, R. (2002). New leaders and new implications for educational administration. Southern Regional Council on Educational Administration Annual Conference, Kansas City, MO. Christy, W.K., \u0026amp; McNeal, L. (November, 2001). Mid-South Educational Research Association Annual Conference, Little Rock, AR. McNeal, L. \u0026amp; Christy, W .K. (November, 2001 ). A discussion of change theory, systems theory, and state designed standards and accountability initiatives. Southern Regional Council on Educational Administration Annual Conference, Jackspnville, FL. McNeal, L. \u0026amp; Christy, W.K. (November, 2001). State designed standards and accountability initiatives in the southwestern regional educational development laboratory service area. Southern Regional Council on Educational Administration Annual Conference, Jacksonville, FL. McNeal, L. (2001, July). The institutionalization of the assessment process: One story in one college of education. The Consortium for Assessment \u0026amp; Planning Support. San Juan, PR. MeNeal, L. (2001, July). Faculty perceptions of their involvement in the assessment (evaluation) process. The Consortium for Assessment \u0026amp; Planning Support. San Juan, PR. Chesser, J., \u0026amp; McNeal, L. (November, 2000). The use of the study circle in school reform: Bringing all the voices to the table. Mid-South Educational Research Association Annual Conference, Bowling Green, KY. McNeal, L. \u0026amp; Christy, W.K. (November, 2000). Charter schools under construction: An analysis of a charter school evaluation plan. Southern- Regionaf Council on Educational Administration Annual Conference, Nashville, TN. Christy, W.K., \u0026amp; McNeal, L. (November, 2000). Implications of charter schools and home schooling. Southern Regional Council on Educational Administration Annual Conference, Nashville, TN. I Christy, W.K. \u0026amp; McNeal. L. (November, 2000). The process of making sausage in the factory of program reform. Southern Regional Council on Educational Administration Annual Conference, Nashville, TN. Michaelis, K, \u0026amp; McNeal, L. (April, 2000). From indifference to injustice: The politics of teen violence. American Educational Research Association for the Spring 2000 Conference in New Orleans, LA.  Caram, C. A., Christy, W. K., Altom, B, \u0026amp; McNeal, L. (April, 2000). The sausage factory: The process of planning for accountability. Arkansas Association of Colleges of Teacher Education, Little Rock, AR. Caram, C. A., Christy, W. K., Altom, B., \u0026amp; McNeal, L. (April, 2000). Responding to the call for accountability of a school leader preparation program Arkansas Association of Colleges of Teacher Education, Little Rock, AR. Chesser, J. S., \u0026amp; McNeal, L. (March, 2000). Educational community study circles in Arkansas: How superintendents can enhance school improvement through community dialogue. American Association of School Administrators 11 th Annual Conference Within A Conference, San Francisco, CA. 6 Chesser, J. S., \u0026amp; McNeal, L. {November, 1999). School improvement through connnunity dialogue: The first community study circles on education in Arkansas and Oklahoma. Mid-South Educational Research Association Annual Conference, Clear Point, AL. McNeal, L., \u0026amp; Christy, W. K. (November, 1999). From preparation to practice in Arkansas: The relationship between program preparation standards and entry-level administrators' success. Southern Regional Council on Educational Administration Annual Conference, Charlotte, NC.  Christy, W. K., McNeal, L. (November, 1999). Implications oflegislative policy development for public school districts. Southern Regional Council on Educational Administration Annual Conference, Charlotte, NC. McNeal, L., Gonzalez, M. L., \u0026amp; Noley, Grayson. (October, 1999). The ethics of silencing in school accountability: Listening to the voices of Hispanic, Native American, and African-American Researchers. University Council for Educational Administration Convention, Minneapolis, MN. Christy, W. K. \u0026amp; McNeal, L. (March, 1999). Future policy implications of an Arkansas referendum initiative. American Education Finance Association, Seattle, WA. McNeal, L. (March, 1998). The link between quality and school cormnunitarianism. Creating the Quality School: 7th Annual National Conference, Arlington, VA. McNeal, L., Place, A. W., Tilbnan, L.C., Beaumont, J. J. \u0026amp; Sanders, E. T. W. (October, 1997). A cross-cultural discussion of the 1997 UCEA conference theme. University Council for Educational Administration Convention, Orlando, FL. McNeal, L. (October, 1997). The contextual world of education for children and the schoolconnnunity: James Coleman and the effective schools movement. MidW estem Educational Research Association Annual Meeting, Chicago, IL. McNeal, L. (March, 1997). Influencing instructional strategies. to enhance leartti.ng by using the school-connnunity relations profile model. Creating the Quality School: 6th Annual National Conference, Oklahoma City, OK. McNeai L. (1996, March). The implications of community based information for caring schools: SCRPING along. Creating the Quality School: 5$ Annual National Conference, Oklahoma City, OK. McNe~ L. \u0026amp; Chi, J. (1996, March). Performance indicators and curriculum offerings: Is there a connection in Oklahoma? American Education Finance Association Conference, Salt Lake City, UT. McNeai L., Parks, J., Watson, L., Jackson, D., Midgette, T., \u0026amp; Glenn, E. (1996, March). Our pedagogy: Culture as a major variable. Pedagogy of the Oppressed Conference, Omaha, NE. McNeal, L. \u0026amp; Ashby, D. (1995, October). School-community relations profiling: Re examining leadership for community. University Council for Educational Administration, Salt Lake City, UT. McNeal, L. (1995, April). Fulfilling promises in the land of Will togers: A look at performance indicators in selected school districts since the enactment of Oklahoma house bill IO 17. Sponsored by the Special Interest Group on Fiscal Issues, Policy, and Educational Finance (FIPEF). American Educational Research Association Conference, San Francisco, CA. McNeal, L. (1995, March). Promoting quality in education through the SCRPING of schools. Creating the Quality School: 4th Annual National Conference, Oklahoma City, OK. 7 McNeal, L., Higham, R, \u0026amp; Boyd, M.A. (1994, October). Establishing community between higher education, public education, and self:. An effort of compromise in infusing multiculturalism. Midwestern Educational Research Association, Chicago, IL. McNeal, L. (1994, April). The state of educational finance in Oklahoma. American Educational Research Association Conference, New Orleans, LA. McNeal, L. (1994, March). Governance structures in decentralized schools and school improvement: Lessons from Chicago school reform. Creating the Quality School: 3rd Annual National Conference, Oklahoma City, OK. McNeal, L. (1994, March).\nThis project was supported in part by a Digitizing Hidden Special Collections and Archives project grant from The Andrew W. Mellon Foundation and Council on Library and Information Resoources.\n   \n\n   \n\n  \n\n\n   \n\n  \n\n \n\n\n   \n\n   \n\n \n\n\n   \n\n  \n\n \n\n\u003cdcterms_creator\u003eRoss, Steven M.\u003c/dcterms_creator\u003e\n   \n\n \n\n  \n\n\n   \n\n \n\n  \n\n\n\n   \n\n  \n\n  \n\n\n   \n\n   \n\n  \n\n \n\n \n\n\n   \n\n  \n\n \n\n\n\n\n\n\n\n\n   \n\n \n\n\n\n  \n\n\n   \n\n\n\n  \n\n\n\n "},{"id":"pbs_hueypnewton_hueypnewtoncr","title":"Huey P. Newton Story","collection_id":"pbs_hueypnewton","collection_title":"A Huey P. Newton Story","dcterms_contributor":["Luna Ray Films","Black Starz! (Firm)","African Heritage Network","KQED-TV (Television station : San Francisco, Calif.)","National Black Programming Consortium","Ford Foundation"],"dcterms_spatial":["United States, California, Los Angeles County, Watts, 33.94057, -118.24285","United States, California, Placer County, 39.06343, -120.71766"],"dcterms_creator":["Public Broadcasting Service (U.S.)"],"dc_date":["2002"],"dcterms_description":["Web site companion to the film adaptation of Roger Guenveur Smith's off-Broadway solo performance of the same name. The site explores many of the subjects only briefly touched on in the film, including background of several individuals and events influential to the Black Panther Party.","The Civil Rights Digital Library received support from a National Leadership Grant for Libraries awarded to the University of Georgia by the Institute of Museum and Library Services for the aggregation and enhancement of partner metadata."],"dc_format":null,"dcterms_identifier":null,"dcterms_language":["eng"],"dcterms_publisher":["http://www.pbs.org/hueypnewton/"],"dc_relation":null,"dc_right":["http://rightsstatements.org/vocab/InC/1.0/"],"dcterms_is_part_of":null,"dcterms_subject":["African Americans--Politics and government","African American social reformers--California","Social reformers--California","Black Panther Party","United States--Race relations--History--20th century","African Americans--Civil rights","Civil rights movements--United States","Race relations","Watts Riot, Los Angeles, Calif., 1965","Vietnam War, 1961-1975--Public opinion","Public opinion--United States","Black militant organizations--United States","African American political activists--California","Black power--United States"],"dcterms_title":["Huey P. Newton Story"],"dcterms_type":["Text"],"dcterms_provenance":["Public Broadcasting Service (U.S.)"],"edm_is_shown_by":null,"edm_is_shown_at":["http://www.pbs.org/hueypnewton/"],"dcterms_temporal":["1942/1989"],"dcterms_rights_holder":null,"dcterms_bibliographic_citation":null,"dlg_local_right":null,"dcterms_medium":["web sites","photographs","texts (document genres)","biographies (literary works)","essays"],"dcterms_extent":null,"dlg_subject_personal":["Newton, Huey P.","Seale, Bobby, 1936-","Cleaver, Eldridge, 1935-1998","Hilliard, David","Carmichael, Stokely, 1941-1998"],"dcterms_subject_fast":null,"fulltext":null},{"id":"fqr_asm0656_559","title":"Letter to members of the American Jewish Committee announcing the honoring of Seymour Samet and others at the 50th anniversary of the founding of the Miami Chapter","collection_id":"fqr_asm0656","collection_title":"Seymour Samet Papers","dcterms_contributor":null,"dcterms_spatial":["United States, Florida, Miami-Dade County, Miami, 25.77427, -80.19366"],"dcterms_creator":null,"dc_date":["2002"],"dcterms_description":["Letter from Felice Traktman, President to Florida members dated February 26, 2002 and letter from Gail Fels, Executive VP and Ad Journal Chair to northern members dated March 15, 2002"],"dc_format":["image/tiff"],"dcterms_identifier":null,"dcterms_language":["eng"],"dcterms_publisher":null,"dc_relation":null,"dc_right":["http://rightsstatements.org/vocab/InC/1.0/"],"dcterms_is_part_of":["University of Miami. Library. Special Collections","Seymour Samet Papers","ASM0656"],"dcterms_subject":["American Jewish Committee","American Jewish Committee. Miami Chapter"],"dcterms_title":["Letter to members of the American Jewish Committee announcing the honoring of Seymour Samet and others at the 50th anniversary of the founding of the Miami Chapter"],"dcterms_type":["Text"],"dcterms_provenance":["University of Miami. Library. Special Collections"],"edm_is_shown_by":null,"edm_is_shown_at":["http://digitalcollections.library.miami.edu/cdm/ref/collection/asm0656/id/559"],"dcterms_temporal":["2000/2009"],"dcterms_rights_holder":null,"dcterms_bibliographic_citation":null,"dlg_local_right":["This material is protected by copyright. Copyright is held by the creator. For additional information, please visit: https://digitalcollections.library.miami.edu/digital/custom/copyright-guidelines"],"dcterms_medium":["correspondence artifacts"],"dcterms_extent":["2 letters"],"dlg_subject_personal":["Samet, Seymour, 1919-2014"],"dcterms_subject_fast":null,"fulltext":null},{"id":"bcas_bcmss0837_527","title":"Little Rock School District Professional Teacher Appraisal System (PTAS), revised draft","collection_id":"bcas_bcmss0837","collection_title":"Office of Desegregation Management","dcterms_contributor":null,"dcterms_spatial":["United States, 39.76, -98.5","United States, Arkansas, 34.75037, -92.50044","United States, Arkansas, Pulaski County, 34.76993, -92.3118","United States, Arkansas, Pulaski County, Little Rock, 34.74648, -92.28959"],"dcterms_creator":["Little Rock School District"],"dc_date":["2002"],"dcterms_description":null,"dc_format":["application/pdf"],"dcterms_identifier":null,"dcterms_language":["eng"],"dcterms_publisher":["Little Rock, Ark. : Butler Center for Arkansas Studies. Central Arkansas Library System."],"dc_relation":null,"dc_right":["http://rightsstatements.org/vocab/InC-EDU/1.0/"],"dcterms_is_part_of":["Office of Desegregation Monitoring records (BC.MSS.08.37)","History of Segregation and Integration of Arkansas's Educational System"],"dcterms_subject":["Little Rock (Ark.)--History--20th century","Little Rock School District","Education--Arkansas","Educational planning","Education--Evaluation","Teachers"],"dcterms_title":["Little Rock School District Professional Teacher Appraisal System (PTAS), revised draft"],"dcterms_type":["Text"],"dcterms_provenance":["Butler Center for Arkansas Studies"],"edm_is_shown_by":null,"edm_is_shown_at":["http://arstudies.contentdm.oclc.org/cdm/ref/collection/bcmss0837/id/527"],"dcterms_temporal":null,"dcterms_rights_holder":null,"dcterms_bibliographic_citation":null,"dlg_local_right":null,"dcterms_medium":["documents (object genre)"],"dcterms_extent":null,"dlg_subject_personal":null,"dcterms_subject_fast":null,"fulltext":"\n \n\n\n\n\n\n\n\n  \n\n\n   \n\n   \n\n\n   \n\n\n   \n\n\n\n\n   \n\n\n\n\n   \n\n\n\n\n\n\n\n\n\n\n   \n\n   \n\n \n\n\n   \n\n   \n\n  \n\n\n   \n\n  \n\n \n\n\n   \n\n  \n\n \n\n\n   \n\n  \n\n  \n\nDRAFT Little Rock School District Professional Teacher Appraisal System(PTAS) Revised Edition 2002 In Pursuit of Excellence for Teaching and Learning Board of Directors H. Baker Kurrus, President Judy Magness, Vice-President Tony Rose, Secretary Larry Berkley R. Micheal Daugherty Dr. Katherine Mitchell Sue H. Strickland Dr. T. Kenneth Janies, Superintendent 810 West Markham Street Little Rock, Arkansas 72201 501-447-1000 1 FOREWORD The Professional Teacher Appraisal System (PTAS) is the result of the collaboration, consultation, and cooperation of the Little Rock School District classroom teachers, principals, central office administrators, and nationally recognized authorities in the field of effective teaching research. In a single document, we have expressed what we believe effective teachers do in their classrooms and what the most recent research reflects on effective teaching and learning. The Professional Teacher Appraisal System consists of four domains via Core Teaching Standards, four levels of performance, and three professional tracks of development. The ultimate aim is to increase student achievement as exhibited through quality and accountable teaching and learning. As research in the area of effective teaching continues to be examined, professional will continue to learn more about teaching and its relationship to student learning. We have done our best to capture the dynamic and interactive processes of teaching as found within this document. We expect our teaching professionals to incorporate the four domains with their cunent and successful practices. As with any effort of this scope, this document will be reviewed and modified to ensure that it continues to reflect our best and most current thinking. It is with pride and pleasure that the Professional Teacher Appraisal System (PTAS) is presented to the teachers of the Little Rock School District. This document is an example of the commitment that the Districts educators, administrators. Board, and association have to the improvement of public education and teacher performance in our community. All of those professionals involved in the development of the Professional Teacher Appraisal System (PTAS) hope it will serve as a valuable tool in our continuous pursuit of excellence for teaching and learning. H. Baker Kurrus LRSD Board of Director President Date Clementine Kelley Classroom Teacher Association President Date T. Kenneth James Superintendent DateThe Little Rock School District Professional Teacher Appraisal System (PTAS) is the result of collaboration and cooperation of our teachers, central office administrators, principals, and recognized authorities in the field of teacher evaluation systems and teaching. The committee respectfully submits this appraisal system with the purpose of creating excellence in teaching and learning in our school district for all teachers while building a community of learners for these professionals. Committee Co-Chairperson Sadie Mitchell, Associate Superintendent Dr. Lloyd Sain, Jr., Principal, ALC Kay Gunter, Administrative Assistant Committee Members Dr. T. Kenneth James, Superintendent Tami Bennett, Teacher Karen Broadnax, ESL Supervisor Dr. Linda Brown, Principal Susan Colford, Teacher Suzi Davis, Director, Secondary English/Foreign Languages Mable Donaldson, Director, Gifted \u0026amp; Talented Frederick Fields, Principal Dennis Glasgow, Director, Math \u0026amp; Science Susan Bestir, Teacher Dr. Richard Hurley, Director, Human Resources Mayrean Johnson, Teacher Frances Cawthon-Jones, Assistant Superintendent, Elementary Schools Clementine Kelley, CTA President Dr. Marian Lacey, Assistant Superintendent, Secondaiy Schools Dr. Bonnie Lesley, Associate Superintendent Victor McMurray, Teacher Marie McNeal, Director, Social Studies Ann Mangan, Principal Patricia Price, Director, Early Childhood Marion Woods, Coordinator, Staff Development Educational Consultant Dr.Thomas L. McGreal 3Little Rock School District Professional Teacher Appraisal System(PTAS) Mission Statement The Little Rock School District is committed to the pursuit of excellence in teaching and learning. We further believe that teaching and learning must be supported by a comprehensive appraisal system that identifies clear, rigorous, and measurable standards and that allows for individualized professional growth opportunities for its teachers. We believe that evaluation must be embedded in a collegial, collaborative, reflective, and supportive atmosphere where individual goal setting, continuous improvement, quality professional development, positive reinforcement, and ongoing dialogue are sustained practices and behaviors of the teachers and administrators. Designed to enhance the highest standards of excellence and professional expertise, the LRSD Professional Teacher Appraisal System provides a comprehensive process for providing success for all teachers while ensuring quality performance and accountability in the classroom. The Professional Teacher Appraisal System requires commitment, trust, and support from all members of the school community with the ultimate goal of improving and enhancing classroom instruction and student achievement. 4Little Rock School District Professional Teacher Appraisal System(PTAS) I. Evaluation Overview A. Purpose of Evaluation The Little Rock School District believes that the principle responsibility of the certified teacher is to direct and assess teaching and learning experiences of students. The District places high regard on the quality of teaching and learning in each classroom with high expectations for improved student outcomes. In order to ensure such expectations are met, this appraisal system has been designed. The purposes of this evaluation system are: 1) to enhance the quality of instruction, 2) to provide a basis for professional development, 3) to encourage collegiality and professionalism, and 4) to serve as the basis for sound and defensible employment decisions. B. Goals of Evaluation The Little Rock School District has established the following goals for the teacher evaluation process: 1. 2. 3. To aim for excellence in the educational curriculum through improving teaching techniques and providing opportunities for professional staff development and growth for every teacher. To improve instruction by collecting specific data and by analyzing the individual teaching performance for each certified person. To improve teaching and learning by cooperatively developing a professional growth plan for each certified person in the career track. 4. To identify for the teacher the critical skills and expectations that are essential and clear indicators for effective teaching performance. 55. To offer a supportive process/track to any teacher who warrants additional assistance when further preparation and refining of skills are needed in meeting the expectations. 6. To offer feedback to the individual being evaluated in expressing the extent in which his/her performance or undertaking has met the District's expectations or school improvement initiatives. 7. To provide accountability for the District's decision to continue employment. C. Review of the evaluation The appraisal system shall be reviewed every three years by the Classroom Teacher Association and the Little Rock School District. Any necessary proposed changes or clarifications will be recommended to the Little Rock School District Board of Directors for review and approval. D. Training of the evaluators To ensure that evaluators are prepared to implement the Professional Teacher Appraisal System, the Little Rock School District shall: 1. Train evaluators and administrators in observation techniques that will enable them to identify and to describe teaching behavior in alignment with the established teaching competencies as found in the Professional Teacher Appraisal System. 2. Offer training to evaluators to assist them in developing and assessing professional growth plans. 3. Ensure that a district appraisal training program is established and completed by all pertinent evaluators and administrators. 4. Train yearly new principals and vice-principals on the expectations and procedures for the district's revised teacher appraisal system. 6E. Orientation of Program Appraisal System The following steps will be implemented to familiarize teachers of the District's appraisal system\n1. 2. 3. To acquaint Track 1 and Track II certified teachers to the LRSD evaluation process, policy, procedures, and forms\nTo explain and discuss the core teaching standards and coordinating rubrics for each domain\nTo assign Track I teachers to an assigned evaluator at the building level\n4. To link the state's mandated Pathwise processes to the Little Rock School District Appraisal System\n5. To outline and explain the scope and importance of Professional Growth Plan\nand 6. To acquire the teacher's signed acknowledgment of the training and the receipt of the new appraisal booklet. F. Evaluation of Certified Teachers All certified teachers will be evaluated on the Core Teaching Standards as outlined on pages 12-39. These standards have been established as the core and essential skills that all certified teachers must master in the Little Rock School District. Teachers will be evaluated on five major categorical areas with each area containing specific expectations\nDomain I\nPlanning and Preparation\nDomain II\nThe Classroom Environment\nDomain III\nInstruction\nand Domain IV\nProfessional Responsibilities. The level of performance descriptors are Below Basic, Basic, Proficient and Distinguished. Track 1: Beginning/Non-Traditional Teachers Teachers in Track 1 will be provided with several opportunities to exhibit competency in the four domains and their related and identifiable skills. Any teacher who has less than 3 years of experience in the District will be placed in Track 1. Any teacher who has more than three years of experience but new to the 7District will be placed in year two of Track 1. Any teacher who receives an unsatisfactory or below basic mark in the weighted component must show improvement for continued employment in the Little Rock School District. Track II: Non-Probationary Teachers A certified teacher who has successfully completed Track I shall be placed in Track II: Professional Growth. While still expected to meet the Core Teaching Expectations on a daily basis, a teacher in Track II will concentrate on the development of professional growth plan and will identify the coordinated goals and activities needed to achieve the plan. A teacher in Track II will be evaluated yearly on Domain IV: Professional Responsibilities. Effective implementation of the plan is based on the dialogue between the teacher and evaluator and self-reflection of the teacher. In fulfilling the Professional Growth Plan, a teacher may choose to work independently or with a team or another colleague in establishing and achieving the plan. Plans will be developed collaboratively with the supervising principal/evaluator and must reflect District and/or building goals or objectives. Components of the Professional Growth Plan will include the Following (see page 45):  Goals and/or objectives  Activities  Outcomes  Resources  Time Frame  List of participants There are no regular or assigned number of observations for a teacher in Track II. However, an evaluator can observe a teachers classroom at any given time. If a teacher's performance becomes less than acceptable during the Track II phase, the teacher will be placed in the Awareness Phase or the Assistance Phase of Track III pending the degree of the deficiency or problem. 8Track III: Professional Support Track III is designed to offer technical and supportive assistance to any experienced or non-probationary teacher who may exhibit deficiency in a particular domain or competency. This track is intended to offer specific strategies or intervention that will alleviate or improve the teachers area of weakness and allow the teacher to return to Track II with mastery of the area(s) of improvement. This track is also designed to provide the teacher with due process while the teacher undertakes and implements plan of action in addressing the identifiable areas of deficiency. a The Awareness Phase of Track III will alert the teacher to a particular problem and will offer a solution for correction. A specific time line and plan of action will be established between the teacher and the evaluator in rectifying the problem. If the problem is not corrected at this phase, the evaluator will then place the teacher in the Professional Support Phase. The Professional Support Phase will be more intense and will require other personnel in supporting and assisting the teacher in meeting the district's expectations. A specific plan and time line will be established in assisting the teacher in meeting the expectations and returning to Track II. If the evaluator determines that the teacher has failed to meet the expectations, then the evaluator may recommend non-renewal or termination of contract. 9Track One: Probationary Teacher 10 G, Observation of the Teacher In Track I, a teacher will be observed and evaluated for three consecutive years (see page 10). During year one, the teacher will receive two formal observations and four informal observations. A portfolio will be required yearly and a mid year and summative appraisal will be done yearly by the evaluator. While the teacher in Track I will receive two observations during year one by an assigned Pathwise mentor, these two observations will not be included or used in the evaluators collection of data or evaluation of the teacher's summative appraisal. During the second year, the teacher will receive the same number of formal and informal observation as year one and will be required to maintain a portfolio. The requirements for the portfolio is further discussed and outlined on pages 49-51 of this handbook. The teacher will receive a mid year and summative appraisal. During the third year, the teacher will receive one formal, four documented walk-through observations, and a mid year and summative appraisal. The teacher will also be required to maintain a portfolio. Both formal and informal observations will be a minimum of thirty minutes in length. The formal observations will require a pre-conference, a scheduled or announced observation, and a reflection/post conference between the teacher and the evaluator. The informal or unannounced observation will require the evaluator to offer a written summation or notation to the teacher describing the observation experience. A post conference following the informal observation will be held between the teacher and the administrator to reflect on the observation. A walk-through observation may be done at any time as deemed necessary by the evaluator. A walk-through observation is an unannounced, non-timed visit to the classroom that will result in a written summation of the evaluators observation being placed in the teachers mailbox or given to the teacher. 11CORE TEACHING STANDARDS The Core Teaching Standards consist of four primary focuses: Domain I: Planning and Preparation\nDomain II: The Classroom Environment\nDomain III: Instruction\nand Domain IV: Professional Responsibilities. Each of the four domains of this document refers to a distinct aspect of teaching. To some extent, the defined components within each domain form a coherent body of knowledge and skill, which can be the subject of focus independent of the other domains. The common thread that links all four of these domains is that they all engage students in learning important knowledge( Danielson, 1996). There are four levels of performance: unsatisfactory, basic, proficient, and distinguished. The levels range from describing teachers who are still attempting to master the fundamentals of teaching (unsatisfactory) to highly successful professionals who are able to communicate their expertise to other professionals (distinguished). Each component of a particular domain defines what is an unacceptable to highly acceptable performance level for each component. 12' Ay' V J lA. IB IC. ID. IE. IF. PLANNING AND PREPARATION Demonstrating Knowledge of content and Pedagogy Knowledge of content Knowledge of prerequisite relationships Knowledge of content-related pedagogy Demonstrating Knowledge of Students Knowledge of characteristics of age group Knowledge of students' varied approaches to learning Knowledge of students' skills and knowledge Knowledge of students' interest and cultural heritage Selecting Instructional Goals Value Clarity Suitability for diverse students Demonstrating Knowledge of Resources Resources for teaching Resources of students Designing Coherent Instruction Learning activities Instructional materials and resources Instructional groups Lesson and unit structure Assessing Student Learning Alignment with instructional goals, standards and benchmarks Criteria and standards Use for planning Variety of strategies 130) 0) C 0) Z o \u0026lt; Q. lU 0. o z \u0026lt; 13 Z 1-4 z z Q. \u0026gt; Ol o cn re D U 0. o c re c 0) c o u XJ CD V) CT E 4-* \\fi b C y y p S  o-g  0) O -k: ST'S - J9 5 Q. O O Kf} c \\z V) \u0026gt; /,.  QJ CT U o 0) x: c 0) *_ c O (U O .-t! -03^0 2^004-'=. -  \u0026gt; t) i2  p 3 o XJ . B CT u 3 (/) c o 0) *J m ^ \u0026gt; $ 8 :y -=5 3 O. E u) .c \u0026lt;u *- (U o c -^ _ o 2 0^ u E \"CT 5 o c (D g. o X3 O 2  \"^ i: ro to \u0026lt;u JZ in 1- to **- _ -.9- Ui U) \u0026gt; *\n.C C 0) jj O U)  U! 2 \u0026lt;u 5 cn 01.2 .2 3 i2 0) S CT .2 . 5 2 z \u0026lt; z o o O V 01 ? 5 o c cn c s re u W c o E V Q 0) u c TO E u t V a. o U 0) TH to CD re c V c a E o u  o y \u0026gt;2  g c: H Z UJ Z UJ lU ui  E E -2 jn u ID T3 ID 2 S' i_ \u0026lt;U .2 u \u0026lt;U -H C 5 gE 8 x: c \u0026lt;0 .- Q. w .0 3 = 3 \u0026gt;- \"O  j_i  (u UJ J2 ~ o) .u\u0026gt; F \u0026gt; 2 -S E 2 S ' UI 2^ 5 2 \u0026lt;u m  c\nH .1^: -S $ CT o V) Q. i/\u0026gt; I- C c 8 c o E 8 cn c 3 in  O K X) \u0026gt; CT ^^2 21.^ -C T-l C XJ lO (O tj O c c oi 01.2 F. *- ! o 3 5 S CL~  s 2 to  0.10 TO c.Q g g-^o s 'oi S\nS' (u 4-. 01 S h: S o c o in c CT C to  8 =5 .8  \u0026gt; g 2i  S fl) TO H * * (U c to liini u- o CT y CT O \u0026gt; U 4- .S5 c to (P x: \u0026lt;1^ -i tj ^ ._ u  g TO c c i2 o h- u \u0026lt;u c x: o u -  g . o 4_\u0026gt; ~ ~ UI a o 5 Q. 2 c C o E o 8 8 S .2   tJ 0) .2 t 8  \"' .g 5 S.5 O 3 i_ TO C U fl) O .id  - ^   n p o x: (V O 1- 0)  _ t  o to o o 2 tn O sB  . u 2 c 2 5 o c u c C o u in \u0026lt;u (U t (/ JS o \"2 g y _ o O C x: (O .\u0026lt;1^ H in s X3 O ) c u 0) t\n= c 3 4- in 5 o o c 8 -S o 'B  c 8 u 2 CT 3 O 2 o. U c U (D U \u0026lt;u c O  =3 a? TO  ex o .-t\nx: E y D \u0026lt;U 2 o S. o CT \u0026gt;, y S 2 'p ? TO o O ^ - W X5 3 CTl (U Q. o .y ^-c o - s J E P C E .s 5 S 8 (D (U 5  TO c y 5 3  t s  =? w c *- Q i/\u0026gt; a\u0026gt;  C Q. c ID o g g -C 2 5 O u o O) (u . U TO S S2 \"ct H O O ? 5 -  g E n to o  Q. H-l 2r m 3 C o a\u0026gt; 22 c Q. 0) 0^ -\"Isf 2 o. E O tJ E .y w \u0026lt;u C XJ JO CT (TJ c \"O u. V) O \" if} c 5. g o ^ 01 H- \"O \"5 to CT C SE  g TO (U XJ 4-t o tn u Q. tn o o'-E i_ .2 \"o c q\u0026gt; c o \u0026lt;u O) TJ 0) y B o o H \u0026lt;u  \"CT to c o 3 trt in O -o n 5* = o Ql 4J  0) w re \u0026gt; C fli at re 5 o \"o o 5 at  C Q.ro ifl .y -CT ID QJ CT ID 0) P w o   to $ c \u0026lt;/) Q. \u0026lt;U Z O s \u0026lt; 0. lU 0^ 0. o z \u0026lt; t9 z z z ^5 0. fl C QI D 3 (/) O QI CT o QI s o CT C fU I Z h4 \u0026lt; z o o vt c o E V o .fi V o Q. E o u fl u_ 2 o to QJ \u0026lt;v S .2 ro c j_ CT-q fl fl - fl iC ro ro p S  S *J  Ui p o o c ~ \"2 ro5 ro\n8 5    CT CT ro P P c p -52 'I E ^ro 5 si 6 c = s  ro o y \"ro \"5 tj 5 E s 2 S 5 (- C CL     i's I s-Bb-S CT g'O S B \u0026gt;^ g \" ro 5 -g 2 5s \u0026lt;-. ^ ro c O to QI c 5^ ? EB 0) (U Q. CT C in b c S \"S S c S 3 P 'r Q. ' in Q. c ro 3 ti Q -C S'  u e g .y CT ID -5 Q. 5 c fO 0) CT P Jr. P S \u0026lt;1^ ID tn OJ i*_ ID 3 O = OJ -- Q\u0026gt; CT   ro K! i? i2 \u0026gt; CT P  ^ P F ID \u0026lt;- 5 4- . - 0^ 2  . o IX 4= c QI o 3 (D Oro*\" 5 QI F u! 5 0 ID Wl * P to X? QI ' O  to o -E C -3 Jr O  S  a F a' 3 = cn 5 u ru u a = ro tz 8 - .\u0026lt;1^ Q) O O 3 U u c n E u t V Q. O g u r\u0026gt; tn ,. ^xi o 'c ro g Ji  c ro ro Q. o S js \u0026gt; 6 = -p  ir. iO QI ID pool \"O *- hx QI Q) ID CT u tJ O s5 S 2 - -5 55 \u0026gt; o ro -  l-s XI c fo  3  ro fl U -O ID 3 ,0) 3 2 *0 0) \u0026lt;U S m c CT S (D 3 (D CT O *5 X \u0026lt;u to o t: (U 9- c a 05 -2 _ S 9^ F  -S   c fc   X *- 3 CT QI ? c \"D D QI J\nro - ro D X)  CD CT ID O 'I \u0026gt; \u0026lt;11 J'l \u0026gt;* S p fl Q- ro 2 F -b! = QI to iw = *- -# -J ro Q. 5 S- -2 P CO o-C 2 \"o S \u0026gt; -S \u0026lt;Tt /. = Jr 2 fl  3 o u ID x: D ID J= fD JZ (J ID Q. ID 3 5 E ro g  E CT J-L o p flj t5 ,p s 3 : i/i tu O1 0) ID f? 0^ O o O ex .i2 O J \u0026lt;U 5 Os CD J 2 O ID 3 x: u F O Ifi ro ro ^   Xz to c ro Z B -g Oo u B O 3 \u0026lt;D 5- O' o t?! *j Js -- 15 P ID -*- X \u0026lt;U C ? ro ro g  O^S U .\u0026lt;1^ O  CT QI 3 to  3 o 3 ID  5S .ti QI 3 QI flJ \u0026gt; \" lO -m I) O to u 3 .2 ID (U to -D O1 :c g = E u X E ro fl ID QI *. S^ 9- C S,\" * 3 c \"5, -\u0026gt; in Q. 0 c  m Tn m fl)  ID \"D to p ro c -2 ro Z Q ro 0) ro t l5^it S \"O  F *o z UI z UJ -J UJ o-^ = st o CT Z 2'^ g * St ? P 2* ra Q. O G 4- o c U o mroroinO^ro\"^ \u0026gt;- 0)43 n, -F o\u0026gt; ro ro^BOroro-oro ^^QI(U-^\u0026lt;UQ|C 5 \"O \u0026gt; H 5 \"a P. ft fl 0) ^ogBroojO-- ro ro^ rof'g O S   ro-^rowNSro^ HginO-cPro-St O = 3 to5.i2~  P R xj .X CT u ro cji = a ro w ro ro  o \"S  \"5 ' \u0026gt; c (U \u0026lt;u x: Q  -S c ~ 3 QI n CT 5^ QI 3 CT f/k 8 -TO \" \u0026lt;V  to fl) -is j\nO .i2 !C fl fl o - c ro S 5 -2 ro   ID CT \u0026gt; S Jr. ? iSsS^I ( . u ID .\u0026lt;1^ ctB  oJ 5 \u0026lt;U I -S 3 tt-F o o 2 8 5 o 2 CT . r-  A -c C o 3 (U \u0026lt;U CT n V 0) -S ~ OT 5 ^t! '  So^'Sco^S\n'S^SS'^inSjS 3 3 P .0^ ID u HI 01 CT CT :s^ u 3 to \u0026lt;u 5 5 m ro  c^ jii  !S  QI *0 in 25 ? ~ ^?S  QI ro xj CT  'S2 3 (D =6^ 3 Vi 5 Si-S u 3 ro m ro u_ H o O O O 3  XJ 3 ID u- fl O S.a -n S ? gi  |35 eSsS5 o (/) \u0026amp; \"J o C (/) Q.-I o. \u0026lt; O = 4-1 C 10 ro .p ID CT m O W ro tn 0) ro  tFi !  o ro J-- -c 8 ? . Qj O) E .c (V N \"D W) 3 s\n' 5 \u0026gt; . B S o -E  .- ^ F O Q .E 0) to to \u0026gt;- c ro 5 J2 5 ro -S 3 S, \u0026lt;U ID S P 3 ID 4= ID to Q. fj} m \u0026lt;p ID roi ro  3-  3.5 ro 3 _ -O ~ -X O 3 5 3 \u0026lt;u ai S B tn O - = p c 0) sc. S tfi CT g'S __\"TO 5 UJ  -a o o c c c ro ro 2^ ro \"o ro j: B Di u ex- ro2 = |5^ o ID 3 .fcj t/\u0026gt; o-  g^a s ro g, 2a O Oga 5 -S S S t\n5 3 1-30) qK 2UX 5 2 c V) W) *\nc Mc 5 I z o 2 \u0026lt; 0. IIJ Of 0. o z \u0026lt; o z z z 3 Q. (/) 75 o u m c o  3 u W c 3) C U Z \u0026lt; Z O Q c V c E o u S S? *3 CT in Q c 'y p u) 2 o ro -c o CD GJ GJ F, \u0026lt;2 p \u0026lt;u V u c TO E L. t V Q. o  V u TO CO s   s c z Uj z UJ lU J J. o e-  io O \"O UJ GJ E S E O s 3 GJ 3 ^.c  g\n c aJ c = = si ft 1 O \u0026lt;U c U a (o 02 oj y _c   S 3  \"o 51 .y = c UI nj t (O \"P p m ro . _ ~  P JS TO GJ TO ro ja w (D  XI \u0026gt;~ UI c XI \u0026lt;-\u0026gt; \u0026gt;~ \u0026lt;u o ir .2 c c C GJ TO in g TO * O TO IS in w TO \u0026lt;-\u0026gt; O y O = 2 - o fl-i O O\u0026gt; (O J2 \u0026gt; TO d X GJ gj CT (/) - . - -B  3 C 2 i5 S \"O ai E a ^Bs-g C \u0026gt; GJ  GJ GJ a o 3 :z S o ~ O u c TO C TO 13 3 c in S  u) (D U  X3 12 GJ GJ o *^ - - -A y i fc. TO TO B5 t! - E '1' o 0,0 01 a UJ    -3^ .2 TO p E X L\u0026gt; c c X  \u0026lt;u Q c -S  i jri \u0026lt;5 0) \"o 2 S t\n! c - 3  \u0026lt;S 5 B-S X c GJ 3 a c TO TO GJ \u0026lt;0 \u0026lt;u c  s U ui .2 \"R O .E  -g GJ ~ g .2 vy E \u0026gt; TO E o-^ 3 in o. w 5 c t _ p CT ro GJ O c p B P P !5 u) ? TO 3 TO c 2 2 .2 = \" S = \" p y\u0026gt; o \u0026lt;U \"O c  X \"2 jn -D 2 3 5 c 3 Q nj 3 P sr O TO O in p re L S ui c Q. ui 4 7= 01 re a 2 UI u c o 0 re c re re GJ 43 3 re \u0026gt; ft in \u0026lt;U \u0026lt;U 01 ' 5 t! -O .ti - I S E \u0026lt; 5 o5 TO u  E P o. !\n: re re IT'S c Ji! -D u! o  re 3 o 3 c EXi\u0026gt; JO TO \u0026gt; TO o GJ Q. W 3' o (O z GJ S GJ in J2 5 OJ . -g 0, 10 o c ti XI o o\u0026gt; t E c a O' 0) \u0026lt;u = y \u0026gt;5 ES O TO in C 1_ c (/) \u0026gt; GJ O O P Jg 2^ TO TO  E  -g '5 B o tj u TO c TO O 05 S 2 TO .2 O \u0026lt;n O *E flj 2 c J :\u0026gt; = -g 5 c \"J -9 E I. 4J .2 w S C \u0026gt; UI X \u0026lt;u 2 C -S -5 n !\n] i F 3  2 TO in in TO TO o c sB 5 s  in u \u0026lt;0 u OJ GJ 3 .X  c CT IS \u0026gt; a  5 o in O E w TO .P-S U 01 01 S  . S 3 GJ C E TO \"O jn 5 TO V) O o TO in - TO  \"o GJ 5^ XJ -C A in a S nj \"O u, 3 O TO O P tn in TO u GJ o GJ XJ S 3 UI o c GJ TO tn TO O CJ ,o u 5** c n= u \u0026gt; .So 3 W  XJ 3z o 1-4 s \u0026lt; Q. lU Of 0. Q z \u0026lt; O z 1-4 z z Q. tf) 0^ H 3 SI V Q O V Ot T3 tt) s o c tn c s 2  c o E V o D (D 52 3 O' c 52 b K y p z hH \u0026lt; z o o c V c a E o u 8 i'C n :E V a. Q $  U in Q CD p Id 5 tS \u0026gt; *2 o 2 J oiw e s t \" -5 c 5  Q   . 2 0 o 9  0, o p cn 3 \u0026lt;v cn w 3 E o p i/TS \u0026lt;u'*'S S -g g  \u0026lt;i5-.S V c X o p fc c n 3 o\u0026gt;.i2 O (U c c \"O u .g s? 5 I c o ro Il u Q i/i \"O M- ru o c fD i'^ 'te \u0026lt;U O 2 3 O tn \u0026lt;P  k 0) 3 o 5  o 5 ro o ' \u0026gt; \"5 \u0026gt;- \u0026lt;0 w .i2 3 O o s -P  52 cn 3 Qj tn -c P o u E = .JZ P fD \"O W P \u0026lt;1^ \u0026lt;U E 3 y -c 5-? J- V) O) p J2 u_ 2 Q. O -c 1. .52 V) O C in p TT . \u0026lt;U r? Q 22 J3 P = ru O U in (D 5 \u0026gt; .01 S ro H ro Ife 2:^  (Q s y I tn c Z) w H Z\nlU Z LU LU tn 2 3 O o 8 tn p u o in OJ \u0026lt;u \u0026lt;D 5 (O c s 3 in 2 3 .C .52 0) aj -S j:  y 'm 3 o \u0026lt;D \u0026lt;D sl I\" 8^2  oi 2^ =-\u0026lt;= 5 8 5 Si 2 $ \u0026lt;U tD \u0026lt;1^ i::: o JO 2^ -g 2  S \u0026lt;5 O .C \"D  5 w S ^^5 5 - \u0026gt;. 5 S JJ .S2 O ) P 2 3 in ai 8 \u0026gt; \u0026gt; 8 g Si 8 tt  oj = .c *3 . p Q .55 tn (U (U y -c s = i'= (rt in O) p p 3 w JS M_ 2 Q. O ^ \u0026gt;- .52 O KTi '*'  2 = -5 y ro (? in m (D 5 \u0026gt; QJ \u0026gt; (U H in 3 o .c $ 8 a 2!  I*- *o O 5 0) V)  \u0026lt;p tn $ tn 52 c \u0026lt;0 3 .52 D \u0026lt;V \u0026lt;U Q c 0) \u0026lt;y .O jc 25 fD P (U \u0026gt; H p U 0^ 3 o 3 O w M Vi   O\n5' kn *3 CT .C in Q \u0026lt;u  c O  = o o E 5 S y s fl 2 85= ^2 \n-u- 0 c c 2: tj C p 10 TO 2 CT \u0026lt;u c c (D g.3  Ci C O) 3 CT S^SiCTinc.Stn E-g 2fl^^ fl) p P 3 0? C \u0026lt;1\nVi ffi 3 H E 5 oJ c 2 rti O- 9 3 IQ xn  STS  O) ID 2 x: F o- o P \"J Q. o O ID 8  - \"O .-hi c 3 V) 0) (O S  2 (0 ~ P fl S^ -n -E  - -1 ID  c 13 . c -2  fet^cE^-Sa s ? E ^o3 - -^ - = TO \u0026lt; Q. o . E CT a\u0026gt; c O' to r\u0026gt; to Io tj P 9- Q c g is. P CT   3 \u0026gt; .9 o  p \u0026lt;D W O\u0026gt; S2 S O S fl 2  3 c (0 c \u0026lt;u S  o t/, o jc tn 0 tj fl   tj 3 \"O E - E P H 0 XJ jn o  fl) r2 C (0 Ji o M- O c Q..b to \u0026lt;v \u0026gt;  CTO - s E o) 3 J 2 .S i/i ID \"D \"O o fl O g jn 'J -o o - ~ O _ 0) M- in o  m '^'D 55- O \u0026gt; c .! o , in ! U c C 0 3^^ i5  (o *' H kn to Q. 3 V) Z o IH 2 \u0026lt; Q. UJ Q Q. O Z \u0026lt; 0 Z z z 0. c o 0 3 u tt \u0026gt;* ^- c u P n ID i_ in ID S o i^ -m fl c g* 0) -2 O fl  O = S ti c in .\u0026gt; in C 9 rt  +\u0026gt;* t to 3 .2 4- u 0) to := B lA = to 'fe fl =  S 3  2  tn c 2: tn Q.  2^ S\nto o o .c U _ 2 S.3 3 -2 a (U O o c \u0026lt;0 o o g g Z \u0026lt; z o o  V  o u Ot g g 3) 55 u o s c (0 E . k. t V Q. o u in nj ca qj g  g a E o u f I i J ' ! ti'  E -o in 3 to -  \"(1)0 tn -ti '+- 9*   tj Ji H c - 2  S   s i_ 3 ' CT c 8 tJ \u0026gt;_ .^ 0 3 0 3 c c o E fl -D  fe ID  c in a, '8 t\n c Id afl'-g E g-SS = 8 cn c JZ 3 P S--H CT O \"tn Q..E  . . (P 3 \u0026gt; 2 O 5 \u0026amp;! \" Si  ID 3  O m   . 2 fl fl  L- 2 g .-t: -g fl ^  - O \u0026gt; rt) 10 'J Q fi S o 5 S p 8 3 in c Q V) t) S O to \u0026lt; to o. C fU c 0 in 2 ti fl 3 T3 E  .9 \"CT io o C O CT C  cc-e zit Z s-g SS _ o S ~ .2 c (V fO o = to 9  IQ u- tn \u0026lt;1^ . flo'^2 ffi . 5) ti - P - \"D to O o  c to O O CT C 'S 2 ~ S e'g 2 ? In'^nl'''n-2 E fl . -2 c'fl iJ! O 4- c tn 'x \u0026lt;u fl) in '5 lU -S \u0026gt; \u0026gt; \u0026gt; fl p   -y 2 t) 2 o c .0 fl o c to c jn 3 ID fl .2 t in fl c fl S.  3 \u0026lt;5 g o. Q, in fl E m = \u0026lt;U \"5  8 -2 o 3 3 O \u0026lt;u  fl   E P S2 2 E 00 in o .ti \u0026lt;u -P 5 CT c  y --  \u0026lt;0 \u0026lt;v ro to pi *- '* V) O O to  O 3 CT io  in S '^ 3 JS g, ID 2 '5 \u0026gt; O) m ID \"(5 \"id -2 o E o ti .2 fl 2 E fl g eg e o i o c u '9 C .9 5 tn \"CT  9 E o :5   o - 2 i S B c P to (/i to c Si 3 Es: O fl 0) 3 1E^ S .p 2 jn c o)Jis CTO Ji CT 3 C 3 S \u0026lt;u 8 i = P p i5 2 H ID ro c 8 =s 00 r  45 in 4- to (n c O  O O o O 2   o Td ID 2 ~ - 5 ? :i ? I 2 7 fl  S Sos' C Cn Q. C tn J. C7J QJ .2 p c I3  tx  2 8 tj to p |5 o. 5^ z \u0026gt; Ui LU O) fli c. C  \u0026gt; IQ 'iS Of U -J \u0026lt; O CT C tn  1^ OJ to O ^0^  O -3 ? 2 2 fl c Ct lA T ?.-S $ to O 3 _tn -C tn .9 t: Ji, t_ t. CT \u0026lt;U Q  Sc Z 3 O to CT V) = \"O 5 S (A \u0026lt;0 u A *** MZ s22 E U (0 3  co \"So  z V) \u0026gt; g'Sjn 2 -  ID O 01 Q-fl ? o 5 .2 o ? O in O c o (ft .c to *-' u \u0026gt;c 4 O to 2 3 sT M 5 C o tj c to .= 2 g V) \"O c c - to (0 P \u0026lt;5 fS Q. tJ 3 2 P w c L  i - O tJt-  3 O -b y tn tn -y -i-t O fl   li 'J lE .2 O c 3 3 C 3 k. = s o t3 \u0026lt;*\u0026gt; 3  u o 4^ -J V)X\u0026gt; TO J3 .iG 5 7^ -L o C TO SB in *3 a c !5 to lA c Q o c TO 2 5 0 . E W ID iS B B C E \u0026lt;u B  B JC Q.  F E oj C XJ c TO C U TO to c -e o 85 O u XJ -o c c TO TO a c OJ TO ~ ro i6 S oj  IH LA fl \u0026lt;U TO TO --* TO -r^ u O) \u0026lt;U \"O TO Ji  O C (J P C _, u - s \u0026lt;u  -Q ro .. .c 5 *-' \u0026lt;^ c 7 P B TO c U -S -Q TO O E B 0) .2 -^2 u in ID \u0026gt; c 01 u)  c E Qj o O aS El^-S 5' \u0026lt; to E ' 3 El^ O U $ O -C \u0026lt;u TO o p o (U c -A (/) Xp E is c tn TO in D c TO to .Si in M_ XJ TO O C TO z o h4 c TO E o. B vy o c TO 5 O' E -S V ro o B E  . \" fl) JI J i- TO B is \u0026lt;1' B TO Q. = TO \u0026lt;D W *0 a\nrti TO Si -Q .y XJ 5    3 K c \u0026lt;n  TO s -t in ID (U Q. c ID TO \u0026gt; \u0026gt; Al P I 2 lU 0^ Q. o z \u0026lt; 13 z I-I z z Ol g c k. m 0) 4 S Si u e \"S  m O o C ID . p JZ 0) O E in  in  O ro E t  2 4^ 8 a c w !\u0026lt;! c ID  2 (0 .R* c p E E ID -2 ID -g -d -o 8 c - 8 B ~ \u0026lt;V to in to to \"J in o C Di a -o c c TO TO .JS c \u0026lt;1^ ^\"3 3 \u0026lt;0 (D -S -) l/l u 2 = U p C TO S TO m EB . to TO \u0026gt; c 5n (S 3 c TO E V)  V) o in o. 3 O \u0026lt;U C - in o a ID Q.T3 c tA o E Ji TO *5 3 T \u0026lt;'1 JS -is 3 Q. c  D 3 M O1 C '35 V) V M w \u0026lt; ra E u 2 TO a o V u 'to TO CD c o (J c 8 SJ *- iw C JZ C to m C ^3 B 2 \u0026lt; to o u \u0026lt;U Vi 3 3 O x: TO .= TO o c ID E o E u? O t :S ID p . g 5 S oi ID o E ? o $ $ cn XJ c C TO TO S-S tH Z i-i \u0026lt; z o o C  c a s o u Si 4'^ 2: .p to 4-* TO to c O .7^ lU Z UJ lU E - . B c a - o -B S ?  s 2 8 TO 4- TO \u0026gt; C to iv t- o 5 c a- ID C ? O ~ Q. \u0026gt; TO JZ u TO VI  TO  TO I/) o to C \u0026lt;D w- \"O o c \u0026gt;, \"J gj -u m _ CZ \u0026lt;1^ TO 3 !\u0026gt; 'C  c m 3J TO P  TO TO 0^ c TO 2 E .Si 'K S B in *J in ID O E O i_ 2 -O c \u0026lt;u E \u0026gt; \u0026lt;u in \"O ii? U 3 ^0 P TO TO m P. V) TO in in D TO C c TO E ? 8 TO -IH z I I TO 0) S ill TO TO 3 u w Ji tn .c s. TO E \u0026lt;u rti in I- Ji c 2 f- tn .2 -8 - B s s^:g g-o E O C  o CD ^  E jT ? IV F c Q to TO E c c E = h u) u) (/) c TO X\u0026gt; u c 2 3 j: TO (O TO TO 8^ XZ C TO O tn O to TO c O to x: 4-* tn J'S -n .. TO TO TO in 2*  s E o a\nO' C o i5_^ C c C D s 3i.R S W w 3 in in in c TO  o C U) (0 'M. - g-sbI = o (0 -5 i= w-g c E V B co E c m \u0026lt;A S c \u0026lt; V) * X5 g o .^  \u0026gt;8 in c 5 TOTO Ch E 45 5. -y io p 8 ? o o 4-\u0026lt; TO ra c- CJ to jS TO in to TO \u0026lt;  a TO C TO .ti TO TO X3 I - R ra TO 5 a ,S 3 E 5 w -1-1 !  m to .-t\np \u0026lt;TJ TO TO U O C in S ^s = C S jg c in O U O C .2 TO t_ E B tn 'z tn u TO in o ra o B TO CZ SB (J U)  a o -g a 3 3 to 4_ TO O O 3 c 2 \"o x: 3 Si c O\u0026gt; c TO 4 c c E S 25 9! in TO t5 TO t 15 \"\u0026gt; O) .E c c TO 0. p a\nV)  O1 Sfs = E 3 to in J3 o C TO in ID -= - 4) H) C c I\" P O S ID E 8 S -- ^ 8 1 B iS o 0) j: U !? TO ro l/l .I u. Vi O V \u0026gt;\u0026amp; Al V B 'Z m m h \u0026gt; wre? THECLASSROOM ENVIRONMENT 2A\nEstablishing a Culture for Learning Importance of content Expectations for learning and achievement Teacher interaction with students Physical Environment 2B: Managing Classroom Procedures Management of instructional groups Management of transitions Management of materials and supplies Performance of noninstructional duties 2C: Managing Student Behavior Expectations Monitoring of student behavior Response to student misbehavior 2D. Organizing Physical Space Safety and arrangement of furniture Accessibility to learning and use of physical resources 20Z lU Z z o fi O) c 'E u ra u O a \u0026lt;u 5G 3 cr c 4~\u0026gt; in b c p 2 o. \u0026gt; Z lU Z o o Qi (/) in u lU z ro u 3 3 u n ot c Z .!2 Z 5 (A UJ s c R3 E u  V Q. O  ro u in ro co ri z \u0026lt; Z o o n N 4-\u0026gt; c u c a E o u z UJ Z UJ UJ p C c g CUJO)  2 m  m \u0026gt; G m 8 13  3 Ci  tn P - c i_ ro P s ^13 S O \"O 5 \"o o \" 5s I  I 8:5 o G p 5 C ro P s -'P 5 y .E m  8   t -jj O1 ro p C CL 0) w At c ro in fD * Mi 01 c i3 ttJ  rtJ TX C ro X) P E 5 5  o I ro ro 8 E I S P c m 3 -g E II o 5 ro O i! m p c ro ro X3 c c \u0026lt;u  i_ ro in ro X3 5 5  S \u0026lt;u -  ^-g o 8 H 3 in -Q c o c O c ro o A .E o G c Prob 13 jz g- S O .!\u0026gt; OJ ~ '3 \"9 3 S CTl W K) XJ c in c c o b Qj .b ro IS ro E ro E s O 8 G ^ 6  8.-2 ro ro \u0026gt; X  C C fli u O P tj ~ E .S' c 3 - o ro B Jft P in 2' O TO r: tn \u0026lt;u H .2 ro c , tj u o o ro '- b -3 ro m TJ g \u0026gt;- -. o ro G 5 2 E ro E OJ ro 2: 2 S S  -8 S y ro 3\ng 13 u ro \"C iu _-t= ro ro g E o ro .E ~ -8  S c  -  ro u cn in D c \u0026gt;2 ro 4_, C 1- .tz = O \u0026lt;U 3 X3 ro-appi~ .j -'-EE rou-i-'zi-Eaj E^rT.roEoroJjz ?roEiD2'' mu! G-g Q.S^ ro c c _-- :3 O c 5 .2 2  E \u0026lt; x: ro c a\u0026gt; 5 ro in tJ .is e Qj c  .5 \u0026lt;u a? c E \u0026gt; ro  u (Z rt ro jL t-i 2 2? c  \u0026lt;u oT 5 \"s\n I' .3 ro in \u0026gt;  \u0026gt;. o ro  c q= o- c ro \u0026lt;u u .1^ iiiih Il eI c 8^ d\u0026gt; - 'o t! 5) ro c ro Vi rotj^E .SE .9^.y ro\u0026lt;oroi-+JroQ.\u0026lt;nin HlL'oroy?.ocw\u0026gt; H S-'oro^ oc^ ?58.a^-^ai \"  5^5 S 8 c c ro P \"O ID .2 S 2- W  m E o  i ro ro = \u0026gt;-is  8 -y g \u0026gt;* P u O c R X ro XJ C 4- ro c ro = P ro ifi o c 8  m S ro  S' P x: u ro - E  ro $ o .iG \"5 s = 8 ? y 8  ___ .E o 13 x: c 0) _ ,  c p 2 8 o c 5 .2 Q. c V) in 5 c E E  ro 3 ? 2G g- Ef-g E A P c c ro o \"D o n cn C c Q. ro ti r= Et3 ro c 3 3 P ro c X in t o ro ^.3 0. g 3 ro Qj TJ o .y c c u ' P 2 .. _ ro 2 \u0026gt;   E o S S g g G. E-\n..o.2 .SG in K_ c ro 3 si \u0026lt;u c  8 '- o ,3 - 8 -E g 3 fO 1= P o 3 X3 G  E ro in X3 E u ro .P ro 2 I O c c o 0) \u0026gt; c .S2 9 P \u0026amp; o tJ 15 vt s-\u0026gt; 3 c 3 ro S ro in ro c o u c lU 19 c a\u0026gt; E \"O 4J O O . U -- ro !2 lU cz c o u ro ,p 5 2 c O  (0 \u0026lt;u S' o o .E S. g -3 m E S.' O) 3 in I ID o w c S  c c o a E 2 uf 8 G s  x in c ro c P I U ro i CM ij \u0026gt; iS u s- *-\u0026lt; \"n iS ro ro tn c S  5 5 p 6 g ~ - \u0026gt;-2 ^,11 ro 8  E a ro 8 .2 .2 in in 3 rti I c S E ti Ic o .2 c 5 G e 3 S \u0026gt; u ro H ro c X K -g P P (A C ot c o .t  = E -r. 5 ro? u v re ro -I Q. I. X O UJ M- c V E u \u0026gt; V U \u0026lt; SsSslsS i Ta ' fe o - 5 g. 5  - g- o  ro '* *5:^ E = J p \"S o ro ro P ro SJ 3 lo \u0026gt; o 3  .y o 13 ! \u0026lt;u E y ro 5 S\u0026gt; s ? 2 3.^ wp c .iG c O'S 13 tj - o 8. o 8- ro ro \"01 E m S .2 2 g ro E w .8 E p p \" -o g c 0^ J d) O) .b c \u0026gt; ro C -=5 .b  c E 3 .E in in iX 4- c ro g c 3-2 c \"O Q- 2 S  5 P c 5^ n ro  9--C CL -M ro M- c o (U 3 \"S '5) O ^.S2 x: -a Q. J3 .2 ro = y  i u u I- ti = C 4J V) '! 5 c 8 E W 5 V* A \u0026gt; 9  .h Q. \u0026gt; c UJfD ,o S in '3 CT C SQ b CT c 9 c 5 CL 3 O re^  S? S CT t! J2 \u0026amp; nj S  s -j S 9i re -H i Q-tj  V) 5\n2 -o r _ p c o TO .2 S re 83 2 a3 S re c Q o  re a\ncn re .2 g] lu =5:2-0 rest C Q. S C TO TO Q. in O i_ j: .0 0 13 c c 2 u c .ti Cn P TO c in = CT 5 o E .P 5 c w -.3 t/) ,p =\u0026gt; ? Le = E -5 = E 4s: O ( P z= 3 o TO Vi TO in c  = E.2 ifi 0) 25 u E 01  -o \u0026gt; c re S3 tn- 11 B g ? E 9 TO re 2 E CL (A TO o t -o  TO _  CL TO . c 5 P  5 o '  in 3 Q. tn in c 2^ TO s in \u0026lt;u o t  o .9 -Q in   tJ S s .2 E 2 S3 -g ? r n +z _ (U c in EJ w{ u./ *w 1= \"a oi E in = c -Q TO in .9 TO tn 'Co -. c \u0026gt; O cE c 8 z lU z z o of M \u0026gt; z 111 z o o Of in in u lU z re 2^ TO TO ifi ID o Cl in 0) O TO O 3 O cn (N Z h4 \u0026lt; z o Q W re u 3 o re u s a. E o o L. w M (0 u 01 c '5t ID C ID z\n c P y e CL 5 2 E a . cn r\u0026gt; d CT) o c 0) E u 3 43 re c _ r-\\  ^ TO * g:2^E 9-4: p _ Q) 3 -O TO P P C \"D TO = 01 re lU g\n--' c\" cn re = o = TO m JZ c c .2 2 s B :t^ .. TO U C re E 0 t 0) Q. o  \u0026gt;  in TO cn   2 13 re 5\nS  -s S. TO CT O in TO re  ^ H 8E E o in a? TO in tn  in  TO Q. 3 .c 2 O o 2 o CL TO P' P O 2 2 TO E c - sis C O p .2 cn k $ $ .tx 01 s \"O S  w = 2 .re in TO 5 o TO c 2^ d .t \u0026lt;u.2Sg C TO in U E 2 = -  2 .2  o  S -E  c CT P T7 P s\ni=\u0026gt; H Se 8. in t/1 i/i C P CM C re c E o u Z UI Z UJ UJ o o\u0026gt; c c 2 lU c re re i_ re e:  .TO '4-1 5! c Z) ti _ c o u ~ c TO P P S Cl re TO re O ? 10 5 re re re '6 cn r S re -o c 2 LO o c TO 3 TO Q- re c c g .2 g. E C 3 Si 3 O \" \"S c w re Z c l-l o 3 c 4^ in 55 c O p .55 TO E in c TO z o 4iJ W c c u o E 5 V M a\u0026gt; c (0 10 re Z E re aj \u0026lt;u .2 E re -E E-S in re 's c in *  2 \" 0 Sis? 5ii 3   O O 3 in .E = 5 T3 s = s J5 i^ . \"O p c TO E 01 a o in in c c i4 re o o to tj E c p TO E C TO O 3 \u0026lt;1^ H in c Se . =5 o re c in fc re w *3 -cP P c TO CT Ji! c .2 E TO in ID z P O -o C 0^ E 22 = re Q. V 01 C m s w u Q. U 3 o in f Cj 'J  E C .TO O) c c o . -D g tJ re i! P 7 c 3 .2 -2 . re 0) Tj   CL o -S re a v 1 ,r o LU a\u0026gt; c TO in O Cl  O' in 1 re . E re o E tn -r-.  = = = 'J  B E .E t TO 4- 'JI = TO C 55 C .2 5. O in s -H .fe R  2 a\nc P TO CT S 3 u in i= . E c 2 T - in 5 re \u0026gt;- O P C TO o in c  Vi .z tn E 8 TO  fN CM 10 Z Z .22 TO E t o . Jn \u0026lt;n C C -2 o  tJ CT c a a c S TO c in  c TO \"D in w O c \u0026lt;3 re o c V .2 t3 - re 3.2 c S * a p C Q t: i c 0-  z\u0026lt;1^ \"D X3 di 52 3 CT C tn b ra o c o ra o\u0026gt; o di y Xi t5 3 3 tn ? di o X3 \u0026gt; X3 S .2 tn O . O 2S o 5 .9 2-D.y I- 2 t fa Q- ro nj ii 0) Q Q. 0) 3 \u0026gt;- tr MJ C ID O. \u0026gt; ra (U TO (U (/5 (D E 0, 4 QJ .hl .\u0026gt; di Vi CJ di O tn Z Ul Z Z o D IH \u0026gt; z Ul z o o Q (Z) Vi 3 u Ul Z o \u0026gt; re  V m c o 3 (A O1 c O1 re c re Z .\u0026lt;\u0026lt;' c d) y 2 Q. ID  X3 tJ c c 3 ID .2 S - X3 C 8 3 tn .52 = o Xi UI IS ID p 8 c ra E u Oi o. o  \u0026gt; V u *U) ID CO ID C Hi u ID tn Ui S ID o c JS Ui c di X3 3 \u0026lt;u c p uy o  O : (N Z \u0026lt; z o Q u CM c V c E o u I  .p .52 c Z3 z UJ Z UJ UJ M_ X3 tn O c O) 5 8 4-?^ C 9 tJ c  -2 -S Ui c := O li -Q tJ XJ =- - c ID ^o c ID s. Q. 5 u\u0026gt; UJ O 3 s .a !:\nQj (C Je ID s. o 3 0) tn nJ k. 2  '5 a\n \"n P ra O'S C f- tn 8\n iS'2  -n u\u0026gt;  x: u- Xi o IS Ui tn D Oi c X3 C I 3 IS c 8 01 U) B Ui di ID O S Z .c Vt c .2 *3 dJ Q. X UJ XJ 5  di r -c s Ji UI $ .i2 di Oi 3 .2 ID C Ui 75 \u0026gt; hr C 2 ID Q. 9 .a S -2 -c o S di  di o -o .\u0026gt; - *- 5Z r= *a U' S c - 2 t o S $ i \" E S o E  I I c S a\nX) CL O U) CL  0)  c c Z ID C (O O  O S ID U \u0026lt;u 0) \u0026lt;u F s \"S 5 fo 9 X di . O tn 3 di U) p E \u0026gt; c-o ~ *- ID ID C di jc L. .2 a a ID -^.\u0026gt;252 ^eO\n di tn \u0026lt;D Qj .2 o (D O 'S *\u0026gt; t (U ID (T5 ro .52 .2 ro H E ID 3 o 5^ }\u0026lt;J ro t. -2 E ro tJ S 2 -S o -a cu -S C 3 di 5 tn --gS E 2 5 E 8 (J U) '3 ')=: 2   E ID ID P -E a g g O di C Oi c 5 s  .52 ID p c 3 ID 2 W i9 ro x: di x c _ s C -3 0) o 3 (U \"o to x\u0026gt; c ID o ?c.2 z 0) \u0026gt; o \"o ra  3   co z -c 5 .2 g ra'ft tn c g .12 S -S ra 5 -t^ g ra ID p tn 0} a\na Xi 3 XJ c a \"s\nP O (U o -E 51 52 E E (D 5 5 3 tn t{ P ID u. XJ tn *_ P .2 ID x: c \u0026lt;1^ E _ \"5 ._g S p -52 E .52  2 \u0026lt;U (U s 2^ .52 t:  S! \u0026lt;u c 3 xJ c a .g $ tn Ui C di Q. P *- q\ndi 0) o  m CM C U) (U o .2 o (n X3 U-E c XJ fc ro o g O o 8 = r*  O-S5 S w .2 Z c 0^* z UI z z o Q 7?i\u0026amp; 8 (Q Ct (0 a 4) .55 *3 O' E 4- 55 b c 'u a. o c CD \u0026lt;U (S^  CD \u0026lt;u cr c 'c 8 fe in tn C .32 .12 -5, \u0026gt;  E S'^i p s S V) a fl) 3 W -I (U I^ 3 Q 3 Q. \u0026lt;/\u0026gt; c c a  C c OT CD \u0026lt;U S c  .S2 p k. (Z t .o 4) ID '*- S J2 c UI i3 5 . 4) 4) c J=  y S 5 3 \"S ? = o 3 .E . -D 35 *0 V) .V} tQ (/\u0026gt; c nj -= c A XJ .55 i= r-  V) . O' V) S i5 . g ai  \u0026gt;:S c  4 - ID SL ID S  E  .32 \u0026lt;u Z3 x:  CD  ns fft u u ID id c g X CD S=^' Q.= O- I/) \u0026gt; z UJ Z O o Q (/) (fl u UJ z 8 M \u0026gt; Q. O) c 'c re S' o 8 c n E P t QI O. o s u V) CD CO C\n= 8 Si  P 2 P = in .-ti o 4J ifl w J2 t 4J U  - p \u0026lt;1^ = S = v\u0026gt; 2  O V) S' ^ cy  u in .S 42 5E5 ci ID Ul 0) \u0026lt;u I^ ns  - (0 3 4, UI O ~ ?2 I w H VI ' /Il (D U ID D p c ID .2 di\" gj \"S S .3 ? .12  ^   CD O \u0026gt; in E ,3 O C 33 41 O 5  81 C  nj o\u0026gt; ns .3 ^iz = V) CD O   ro  i3 \"c 5 s 33 4\nin XJ \"J !8 S 3 (U V) 3 rsl (N Z hM z o o D fS c  c a E o (J SS\u0026lt; a I x i I 9  .p .SQ a 3 1-i z.\u0026lt; UJ Z -4 UJ * ? u ro c 92 \u0026lt;-\u0026gt; 8 (5 c  \u0026lt;1' K  U E VI u lU CD ID V) '\" O  - H V) fU o 4, S  5 , in 5  O r\n\u0026lt;u c O .S? = C\n= 4) \u0026gt; i2 E  EOT 8 c c ID O t Vi V\u0026gt; m V) V) ns oi JS cu 4J .-L  '^.1^ o = c Oj c ra S 3 E . 8.E ro (n C 3 fD IL \u0026lt; a\u0026gt; E ! 3 \u0026lt;1\u0026gt; o H V) y s 2-s 'ui -- O) w Q..S 41 V) C  ^S O \u0026lt;U O y \u0026gt; CD \"C /ii t .\u0026lt;U 2 -o g g li** s ffx i: XJ C  3 8b 8 S v i U Jl w \u0026lt; D 3 V) 0)3A: 3B: 3C: 3D: 3E: 3F: aasESsgR INSTRUCTION Communicating Clearly and Accurately Directions and procedures Oral and written language Using Questioning and Discussion Techniques Quality of questions Discussion techniques Quality of discussion Engaging Students in Learning Presentation of content Activities and assignments Grouping of students Instructional materials and resources Structure and pacing Providing Feedback to Students Quality Timeliness Demonstrating Flexibility and Responsiveness Lesson Adjustment Response to Students Persistence Utilizing Technology Curriculum Resource Communication Instruction Student Use 255 C CT C D CT 4^ E 3 s -C .52 '3 CT C in b C CT y p TO -F, XI E w _2 J----- o '- 5j g - ti a\ng- P CT Js *o CT CT c in \u0026lt;U o C in TO CT :_, CT .52 g s s S (^ Q.  C (D re B re (0 - \"O in Qj TO 2 'J  -5 tj a\nre re GJ TO P m u D \u0026lt;U C tj = \u0026lt;u a) fc S (D x: 3 8 -C o c .-y CT  $ JU \u0026lt;p o CT ^3  S. S' S - P  r 5  =  s  ^V) CTl I/)  y) \u0026gt;- c p \u0026gt; (U -2 a c -5 tj (- X 0) S S  S c 0) V) D C x: 5 TO O u ro O .S2 c Z o 1^ b 3 Ct 8 \u0026lt; o c re \u0026gt; (/) z l-l re re u O) _c s V U c re E u t re a. Q OJ S y 'in TO CD ~ in c  T3 J) (D re re a ? y re g- re X xj P -tn CL TO a c /\u0026gt; \u0026lt;U \u0026lt;P i_ g. in in re gtt a? 3 :\u0026gt; -a . KTt o 0^ CT C J -a ro CT E CT  CT C CT 5 c t g- re y re o P 01 S e re g. ro l~ 9- 5 c ro H TO Q .ifi a\u0026gt; m z \u0026lt; z o Q c 3 E E o u rihi F  S p a\n2 o 5 ~ ro ~ g p a? \u0026gt; I .a g '^-2 O . tn Q. m k 51 }= J2 S 5fi . (V E -g -T S  - -g S s (5? g E u TO X r\u0026gt; CT \u0026lt;u (0 m c re c a E o u 'A \u0026amp;  .p s ^\" c z LU Z  LU LU in 3 \"S a p s Q. o CT 3 4- in p in CT o . '6 p 3 c x: u TO o u P H TO c U) re o Vi c 3 T3 .2 re tJ ' U re S .b Q. O C CT CT -5 2 CT y? 15 a S'? '52 c re S -g g S I E s ro W S -jj tn TO in  in P fN -C+jPJ-.- y -z CT t  - 8 TO .\u0026lt;y H P TO X E E (P in -u in S? 2r 3 CT c 0) - i,' \"o \u0026lt;n E CT __ TO 3 CT  ~ E re S - ffi C CT \u0026gt; ftf S re I * S a P .(s 8 o ro . -^ c E i2  E 0) _ o \u0026amp; o -X . i- ? re ri /T. : ro *- P O B 2 c  u o c 0) c on  S 3 Sg ? o \u0026gt; *5XJ \u0026lt;U xz b ex c b CD JZ g - te J3 g.^ \"O i- co  3 C ' to -C CD 4-\u0026gt; (XJ Z3 o O (U c TJ 0) CD \u0026amp; yj E c fU XJ x: 3 CO = .-^QJ fo  !5 E =5 43 CT o o to fO tU 0) nj S OJ x: oi c S TJ 0) QJ O (XJ Xj C Ol -hi O (D JP Q- i:\n3 CO 5 \u0026lt;u o *- u c CD E i_ p in \u0026lt;D CT Z O IM b 3 Of (A 0) 3 C  U c S y o L. Q. V) z m z IH \u0026lt; z o Q c o '55 in 3 U in Q D C re O) c 'c V u c m E u t V Q. o s V r- cn lo ? F a? .g XJ r p Q cn co x cn -r CD .VJ  0. - 2  3 to (O \u0026lt;U o In  3 O 3) C 'in O A M 4\u0026gt;* e  c a E o u u 'co (XJ CQ ?  .p 4-t (XJ CO c I-  Z UJ 2 \" UI UJ c -g  10 UJ ._ f= re 2^ 3 X3 (0 yj C c E S .2 O = tn _'-S 5 \u0026lt;u =3-^0 D CO XI O g CJ (XJ ' c o g 43 c  :  ^co5SSGu3 O QJ U 2 3 3  -= Q. in o Q.  tn O 1= fD O .\u0026lt;1^ =3 H to m OJ XJ m c o cj \u0026lt;u  ij o a c T1 co CD U_ ? I 3 c g're s c CO O\" o  P S\u0026lt;U-53~^a3ErnC*^tnC 2^Srea-in\u0026amp;ocrreCreaj\"g u- , 0) u_ re tt) ^-ojrtntn'rooS' Sc*-  ^.^1'2  re 5- n--\u0026gt;.-^^reg'5 5E^ - J. .-^.g -gEti^-^-gSSre CT si 4J *- \u0026gt; CT fA m cu o %i y XJ 5 XJ CD  vm -S CT CD C 2 XJ O co ro OJ (q m c._____ o o (XJ c o .   s  CJit! Q. M -f^ p3 C := O t  co CD 3 IW V-* *- .\u0026amp;_ = O V, - E O -ti  c -S 5 S -5 15 E =\u0026gt; a- n r- in c 3 4- CD -C CJ (XJ ru O c xz  .  cu \"O E-2 o (J (D (XJ u (XJ .03 H .E o o ki c CD 3 \u0026lt;D 2 XJ  = \u0026lt;U in  C -ri 3 fe g -F E o :p c c Q. tn O C T.E O c i- (q co OJ Q- O\u0026gt; gj 43 C 5 = S tj oj x: T .2 CT ? a S CD H . _H H\" CD I t XO c -X 03 \u0026gt; 2. ti = tz in c o \u0026lt;U o C ** (A c C 9- CL OQ (Q 01 3 3 O'O' in (U (XJ 0) e g s 2 cy (q CD ' -B  s c g 42 S c E u) c S 2? o  Q. 'in S co co fq U 3 u i/i fO tJ CO (XJ QJ C E S^.2 a o -  in S O) :: '015 cn 3 if! if! c E \u0026lt;u if (D O E t: i_ Q. C OJ c \u0026gt;  CD E  -H ro E t3 -' (^\"3 5 OJ -= (fi (N 0) c \u0026lt;u o 5-5g Sc'-' 2 = re ?i-s Q. \u0026gt; c \u0026lt;u (U c -S \u0026lt;U S '1 - -^ 2- (Zi -2^ iJ 01 = c (D OJ 5 co c (D B -g -S ? iJ rtJ Xj (XJ CD 4^ c C co rn XJ X3 P -\"S c c c (XJ (A (A m .zz w  3 C U  W U oc o u w V 3 O c  U .V H O \u0026lt;u -C i/i Z3 O' c 4- b C y e Q. r- c cn -f-5 Q r* V)  r^  S 5 In '3 = cn \u0026lt;=: ^  I- m fO -J S in i2 =  S' -b! ? \n .2 \u0026gt; ^ .2 CD \u0026lt;D C \u0026lt;U ^ \u0026lt; 3 C CT\u0026gt;^ B 2 =\u0026gt; 2 \u0026gt; dj 5 Xi 3\n\u0026lt;u U -o \u0026gt; C ro 3 .-2 \u0026lt;s t -C t?! ro w a 5 cr g Q TJ c c (D  in 13 \u0026lt;u c TJ c c o . c o d3 5 \"O \"O CTl 0) 03 W L. \u0026lt;u Z o 6 3 Q V) z M Z \u0026lt; z o Q c o '35 (A 3 u M Q o c re ) c 'E 0 ! V 3 (y Ot g '35 3  M C  c o Q. E o u 03 U C: (0 E U'   a. - o s V U {ft 03 (3 i^ ?:i 2?  i2 4-J (O v\u0026gt; c Z) fe z lu UJ UJ _ _. 2 \u0026gt; u in id .2 g  -M ? c 8 g js !S (1) me  ro 3 m V) 2^ 5 -o (U \u0026lt;U -S^ h- o c -C te } \"O 03 5 in - S (-) ro Q.  3 - !={ S oin b5 _ LV 2 =6 -s -a .2 g- a -c  S -F 3 E  b! 2 Bq TO (U C -Si C := O 3 o O Sts (D \u0026lt;U fU O ZJ fU O\" c (/\u0026gt; \u0026lt;u .2 O -Ci C in (5 42 B   in 03 v\u0026gt; C X .y 00 fN Ifc. c o o \u0026gt; 'u! is ra u 3 M 05\"O re JZ .!2  .SC re fl D 9J Di c .2 \u0026gt; \"o re -t\nin re -D re a fl c o ~ o Si c p u D OJ O fl c (D O' c *4-' l/t a O C 03 C ? k- 03 CL (U ro lo u  fl is S 8 Sil 8.^0 re 2 \u0026lt;\" o  re 9-5 -D c IX  5 g 8 Q. 03 !2 re re C31 5 -o 3 c z o b 3 O) c 'E u re re 2 *u 2 Q. co . 5 03 0.^0^ U  C S u \"D ^ .=  (- .. fl c U OS'! (O z M c M Z HH \u0026lt; z o Q 33 c  o 3 4.\u0026gt; U) O) c 5\u0026gt; ra oi c 111 8 c (0 E u t 03 \u0026amp;: o  10 03 CO u fO c re c E o u  .p (/) 4-1 u? c Z) Z lU Z Ul UJ 03 c -2 .2 fl 5 g fl -Z \"S - S re re 9-^ IX (D 5 03 3 o c .52 03 c \u0026gt;-\u0026gt;= *=^0 4.. P 3 4- rD ' C 3 = to p O-jji -2 (3  ifi CLfe S - re E -re 5 e: c o X 2 :2 O 3 \"o (p 03 u lo .52 W -o J2 Jj V3 O 5 c c 03 o .2 5i 8 E  t 2 . \u0026lt;S  . o. 10 03 lO 52 =* - (0 C O U 10 .03 03 c 03 O c -u M- 3 C o o 03 C C in o re \" E (D o X  9- 03 03 O ID C .2 ID lO s 03 C o c 2^ 4.* U re \u0026lt; _l_l c u w Q. c o u re o s \"c o _____2 c c re D _ \"E__-S? re O - tn g c\nra 2 .2\u0026gt; re tn D 9 c oSsSc.-^cia '-LfSore.\u0026gt;rec 2 ofl X fl ID in  X3 re )=! re tn ^, E c CD cn 2 \u0026lt;U S .re p 03 O 03 c-o 03 - C ^\u0026lt;utn.= mo\ni= e = K 5 tJ  -  D 3 03 O O O - 44 re -S c nj \u0026lt;u 2-.hz ci 2 c  g,fl \"I ^ ejinj \u0026lt;u ejiQ.s oi*^ c \"iu P.re'i6-Sfl'i5 ufl c.\u0026amp;recy-'-'cecin o-HQ-oflreoro^ 6 1 2 tJ '.g tj-g -re  S  P . 10 .= v\u0026gt; Q. c 3 T) O i  ,03 .2=S 03 \u0026gt; \u0026gt; fl g c re 2 cn -1 ?  2 ? rr m -\n\u0026gt; s re re \u0026amp;fl D 03 03 E S -iS S 03 o \u0026gt; S S S 5 re 03 a\u0026gt; 03 \u0026lt;U *0 .4. U/ ^*5   j\n3 o \"o P o 2 h: c o 03 03 03 C _ .M O  O s L= fl 2 p ? 2 S S - llal? O. c 03 V) \"O C \u0026lt;13 -S fO *3 03 in l\nI A c O D .re j3 in c .-tiCOtu- \u0026gt; ID *- ^ p d 2 fl ij *- xJ  03 re  re re c SiJ o ? ti .2 re .g Ul Q. 03 T3 C 03 fJl E \u0026lt;u *- fe s ^.S 1 re fl S DI c re 5* re .2' g VI j-i J3 ~ in tn 03 D c o c: iS a\nre \"3 fl g*^ in (D  c \u0026lt;13 X3 Q. a\n\"o \u0026gt; o  5 \u0026lt; a? D C 03 E 03 c 33 I re E 'iS c \u0026gt; .S tj v\u0026gt; \u0026lt; \u0026lt; 03 C o 2^ S \u0026gt;, .2 ! re .2  -= .2 Q. re I*- \"tj c 9- P 9  re o o O oj H to 2 2- tn O In y? !=h re- fl y c re Oi Q c c 3  ^^-g !? 03 c  3 \u0026gt;..c -i- O .2 fl -5  :p -H to 03 jn y 03 03 S .- 2  ^-g S g, c I ? -' O c -D \u0026lt;D Q. 3 p o p  E 9v 3 03 P 03 .2 03 3 oS - ti fl tn 2 .re .2 -, a 2  2  ex ex re c o o\u0026gt; c 33 c .h  3 3 o A! O W) 03 rslk C o u \"O S to 3 O1 c 4-t w b .3? e Q. Z o b O) C c u re re -J c (A Z M M Z i-i \u0026lt; z o o 1 c re D 3 4U V) 31 C 51 re 31 c UJ u M c 0) c a E o u 8 c ro E u c V Q. o \"i 5 O in re CQ t s ro .52 3 z\u0026gt; Z UJ Z UJ UJ .in 8 *0  a? s s ro-S O \"O 3 i_ O (U ro is $.c- 5 S ro J3 5 8 -S fe.-^oiS'SoSEro x^3 rOJ-^t.ro^ tx rz -= y rx I 1 n ii n fO \u0026lt;D u to E ai o tn o ro n K. c Cl 3 2 i o\u0026lt;uE5S.-^t4py,i- n u CD.-ti .2 s $ ti S2 ro c ..^ ro .2 o C if} p Q.  -n 'ro  a  i- -o Q- Q. i CL c S CU ro rj u i\n2.-L..-: O to 3 3 3 o .E C \u0026lt;U 8 to 01 X '-' u .c O. Q\u0026gt; ro -ti D ro ro E c flj sy JO 2: 3 -jf* to c 1) o .2   O 'S o o to U \u0026lt;U P to ro '..'TO in ~ -S' ro Q-ro re -c I- 3 \u0026lt;V XJ 3 to tJ O ro fD QJ -C U- fO 4-' o c Q C Q ro g ro  . 5 rz y A] 5 \u0026gt;\" ro ro ro E g) c to 'r re ro -S re c c m Tj P\u0026gt; O w, W SL' o ro 3 ro p  u \u0026gt; cn ro tJ 3 3 c o 6 . to 0) (U a 0) . c \u0026gt; tu P ti = O fO -i2 3 .2 t ro J2 S -o ro S.5  g re re  2*2  ro \u0026lt;u 55 Q\u0026gt; J. -= 3 ** *: O ? ro  ro \u0026gt;w u/ rutOeOofli  O .-2 '-g r  ti g  2 ro to p 3 3 a 3 3 to o 3 (U x: *- ? CD to (U ro c -D O) 3 c reS *= 9\u0026gt; O \u0026lt;o QJ tn t= -= '^ C .IS -Q I- ro \u0026lt;L\u0026gt; .-fci ro = O S in u E s ro = 3 ro TO .o io 5 3 P .2 - re tj ro t I PWS c 5 tu U cu E y 3 cn tn 55. uy a i: 3 O to - to 3 cu  cn c \u0026lt;D to (U re c \"O c M n V .2 (fl y tj re 3 3 'Z O   !G 4^ re re (t 4^ (/) c A z S P  ro 3 -Si C in o ro ro re - ro re -5 c -Si! re ro  ti28 S 3 \u0026lt;U  .52 w *  re ro \u0026gt; \" So = to to ro w *0 rti y - ^ro g a\nro \u0026lt;v  t E^- H X3 o D E p ro 5 to Q) p 3 0) 4J 3 Q\u0026gt;  E .2 o - ro 2 3 3 i/i C 3 O O .C O O \u0026lt;U C u u  s .in  ro ro ro ro .w -o .Qi  \u0026lt;u 5^ o Q.  .52 O m \u0026lt;U 3 C C C   g S 8 .g .E ro *- 5 ro---------- ro E XJ c Q\u0026gt; O J2 Q, M_ XJ O *0 0^ \u0026gt;- cn in  ro 3 n ro u CL O o Qj 5  C O Q in .Q to ro    81\nA  ?  \u0026lt;U P to to 0) H  o D C re  O\u0026gt; c  U tt 3 4Ju c srs  5 s s r.  (/) c c \u0026lt;U \"S ? c 11 'lit 5^ z o b I (A Z m z a c 0) o 3 4.) M O 4.1 .i U re J3 o 8 u. O) c \u0026gt; o u CU s c m E . u   Q. o 5 .ff\".'- \u0026lt; z o o o n c V c a E o u 1^. 2 3 O' c b 0^ p u s t! .n\u0026gt; tn o SI c 3 H Z  UJ Z \u0026lt; UJ UI S- \"a 5, .  \"7 c gj Qj  in re - c tn \"S ? z n o C F 03 fn 2 - -- gT 'c O .h: g .= 2 \" a) io re ij u) i3 D u ij o 03 .2 a\no 2 fT \u0026lt;u nj g Q- 3 8 U c \u0026lt;1^ s o 3  vj Q \u0026lt;5  u 3 fu S2 (V 0? .\u0026gt; fO (0 -O 3 O u  gj o w o 5 ti- P 3 u  L a gj- S 5 tJ E S = 'in re *3 c D 3 S . E re  tt .!2 w o fj IZ) C it E cn c I' E  (6 c o C O ,_ \"g gJ $ c  w __ 4-  T3 to (D 35 c ^i= y p s  g *  O. a fc! .8 o. c (TJ  \"O u CD F \u0026gt; \"O O c o O (D XT u. nj 5 = o -h* S TO 3 (/) 0) O' o \u0026gt; Q O 3 o O ex o c I v (/) it  c 3 (TJ X5 o 1'^ 15 3 O' 3 ro c 1) .2 E s C O o oi - C C c  \"O IXJ (U E y \"O S \u0026gt; e Q. XJ I .52 u fC X3 o ai F. D 5J .52 O *52 c Vi O (/) u . E gj E \"8 p o P O ex o c C (XJ  .52 \u0026lt;U  U XJ (D XJ I tf) VI 0) c 0^ E ro6 z o b I M Z M Vi TO C TO M C o Q. (A TO D C re r A X TO Ik DI C w- S ,A c o in in  3 CT C S .3^ P -c re s w 0. O  \u0026gt; u 'w s tn O TO m'  3 fO in \u0026lt;n o I. 03 TO 2 -C TO E u c   TO t .\u0026lt;v o C \u0026lt;u 3 TO C O   0) \u0026gt; *- ni \u0026lt;  tu w  E Z s TO to in o 3 O E ID V) S a) S2 TO J 5 1 ID .\n,TO XJ H TO tn *0 3 c Q 3 U 8 2 5^ J3 5 \"5 Q. S s S C TO CO z l-H Z o Q (0 M c o E  Q 1^  -2 c TO TO c TO cn C V c E o u \u0026amp;  J tzi c 03 03 fi -C \u0026gt; \u0026gt;S :o s s e CL E 03 \u0026gt;  \u0026lt;y ^   9 c ro D g  Q.U x: -ri V3 O. U (X3 \"S 2 *5 TO  TO CT 03 y 3 c ID  \" C TO C c  o E (A 35 !fl J \u0026lt; in c o k TO O U K   s :^p3 \u0026lt;u 3 5 'S S \"'c ?o nj ^  a) c t E S Illg H o c O J a  C i3 -= \u0026lt;U to i5 \"a P in 3 03 w p c H \u0026lt;/ - 3 JM Vi fO O t. \"O \u0026lt;n C3 O C x: E P u E  jn ,TO o 9} H \u0026lt;-3 2. u CT TO Q. in o 5 o S c in in 2-3 2 \u0026lt;u E 05 -E ro - = ID \u0026lt;5 O iiy sIS\" lZ U in TO y \u0026lt;13 :p TO 3 S 1/5 CT X) 1- in TO TO CT 1- i_ O O m {fl C. TO O 3 CT TO 03 XJ TO .C in TO 3 D* 13 tj \u0026lt;13 TO \"O \u0026lt;13 5 H O in CT  M M  - Vi c o Q. V) V Ct C 0) o 3 E c TO cn ,- - O E 2 - TO TO .-ti '*- {fl \u0026gt; P .E .hz \u0026lt;-3 ifi .^utns-esy-^c -TO^38 ^TO** 0) 03 C i/i^ c 0) O) __ TO TO  in u i_Oiin^*-vTOin \u0026lt;13cJ3 ^jn_(y g \"u TO TO ETOtoTOJGCTC-^ Si 2 = S S .2 I- Q. Q -H TO S 0) CX-C nJ S c cu \u0026gt; cn 03 TO .. S' 2 .E CT  P *5  C O TO TO E O \u0026lt;1^  m CL Q. \u0026gt;  *- CL in 03 \u0026lt;1^ C 3 E Si 5 O  u  O ro . y it 01 -t .01 2 ~ in '\"\" al in O (D \u0026lt;u  -a p u 3 .E u  = ID \u0026amp;  in c p ro - V) \u0026lt;V TO O : 3^ cn 5 *5 o 2 S' \u0026lt;u S. S o p I-  5 \"t !\nSi 8^ \\n 03 U TO \u0026lt;*^\nsi Zr \u0026gt;   5 Ol U I. in O * c in -ti TO \u0026lt;13 \"D 5 J2 in 03 in c y a* 3 I S i. u 3 3 in _ OJ 0) 4J o ^   O  ID js 2 o y-o 5 \u0026lt;U O , c  E o .2 \u0026gt; ID D. \u0026gt; TO 3 C  TO  u c  V) '55 u U Q. (N CQ\u0026gt;-? \u0026gt; iO g x= .15 '3 CJ c i2 Q c Ji u o 2 = g C in (U \u0026gt;S J- o  c \u0026lt;U IO x: C CT y  C ^\"S  ID _ !y R  s jv ifi u 3 0)  tu E o 3 S2 O 3 u (U D x: oj U c- !G 2 8 S ID :-y I- W .E nj 0^ p 3 C (D .15 Ji v\u0026gt; \u0026lt;7 E 3 ? =\n3 IO S  8  y (U *- V) 3 3 fc nj u lO O) U ID ID .^J3 (U ^ . . s*^ g 5 2 UJ F U .UJ d) in 3 10 O c in i=- o. \u0026lt;n V) - (7 t \"a - S'^ O g S o 2 JO : = 3 E W 8  S \u0026gt;-E3a'-=222 J? o CT c 2 = o e c u nj 0^ c _ -. s 2=  n \u0026lt;* ro 03 I/) ^.6^ 2 0 is p u c D E XJ \u0026amp; p C no O = u 3 S o 3 ID g 5 -C (U E .3 dJ D 3 10 c \"go u) 5 S s \u0026lt;U \u0026gt; s- g \"5 .n I' CT 5i 5 \u0026lt;u S9 to c  V) - Sl^ S'  2 d) dJ  ss^S ro  c \u0026gt;.  o t m 3 h c z o l-l b o c s ^5 C Q. in z l-l o\u0026gt; c ' c (0 E u V Q. 3 in p o   a? I g S. in g 3 OJ o CT -F U ID g R m -' O H U \u0026lt;u . y CT  P m z hH \u0026lt; z o o 3 o s 3 u 'in (7 CD  = .e Q c c E to c  E IS  CL HE in \u0026lt;7 8 s S  (0 3 3 \u0026lt;D u  p c a\n\u0026lt;u M 4.1 C u c a E o u t5 3^\" g e \u0026gt; 3 g-y C H :u w 0^ .p  E 2 3 c to {7 to 5^ c c 3 o 3 \u0026lt;J C go\" S 8 E H 2 (7 OJ o v\u0026gt; (U 3  .p in 4-\u0026gt; 5! c 3 1 3 dJ u 7 3 o V) d C .p .5 c E S s s P (J t7  3 Q. \u0026lt;U (7 u in (G \u0026lt;7 1- Z '. UJ Z lu  3 y R 3 W O I. w lu : b 3 u  J 2 S 5:8 QJ O U (U (/\u0026gt; 3 : 5 c 2 o 2 \u0026lt;D E E tj A (7 ,\u0026lt;1^ p c o Io u = 3 E E o u 2 CT - . 216 g \u0026lt;D O 3 t)  E o \u0026lt;u QJ ID ID U .-' to -S \"S So a\n3 Ji \u0026gt;. c ro 2 8 If O) 0^ fc * X3 c XJ \u0026lt;U OJ \u0026lt;7 MJ \u0026lt;/) = -F F o dJ w S ID 2 S^.2 H CT g \u0026lt;u JZ u c o a 3 tn c .c in \u0026lt;U u. XJ O X3 c o o =1\"'^ \u0026gt;2 \u0026gt; n ** S \u0026lt;u \u0026lt; p w Q. 3 S C C O S Sis i3 Oi in  C -J U) w VI q\n\u0026gt; S 2 g p c E Q- XZ .'=^  tj c gj c o tJ t o  a.E 3 H d) CL to - c 3 O c .= -^ d) dJ dJ 5 \"S  --g Lj  \u0026lt;Z) O c I o c x: in U dJ dJ o *' 0 ct8 c = p 2 to C C 3 t c o 7 3 i_ 4J W c l-l o a. Q. o \u0026lt;7 dJ X nJ OJcoo-E,- P  c o E 3 10 -fi E C Q o c (y V) -A 2 u dJ .2\u0026gt; Q. 2 tn c fl) P S \u0026lt;U 3 Ji  77 ^7 a) r\njp - -Q -C m I* JP \u0026lt;0 S S u 5 0) 3 'a ID \u0026gt; I- CT 0) 17 H 10 in 'x (U o o c o 'U:g o in O D p dJ -  t! O C  0) _ = t\nt\ny  H C 3 c 55 \u0026lt;u _o k Q -g E S. 3 S S Q. 4-\u0026gt; CT 3 CT O I'l CT ID c  C 5 g R S' (7 o c OJ u O) o o = 8 P =3 r7 ,o J? o C 3 2 t - V w 3 4.) c OJ o 3 4.1 n u (U u c dJ co co . Q. Os iSSRHiatlSiR A PROFESSIONAL RESPONSIBILITIES 4A: Reflecting on Teaching Accuracy Use in future teaching 4B: Maintaining Accurate Records Student completion of assignments Student progress in learning Noninstructional records 4C: Communicating with Families Information about the instructional program Information about individual students Engagement of families in the instructional program 4D: Contributing to the School and District Relationships with colleagues Service to the school Participation in school and district projects Collaborative pedagogy and assessment 4E: Growing and Developing Professionally Enhancement of content knowledge and pedagogical skill Service to the profession 4F\nShowing Professionalism Service to students Advocacy Decision Making 34fD \u0026lt;u  in D resH S re re \u0026gt; c O .X 0, in m cn -I- Q_ (P -2 -C Q. (V rf cj TTr  \u0026lt;U 4^ \u0026lt;u (/) lU I-I (U x: VA O\u0026gt; c b  nj re .t\nW S \u0026lt;D c u- m 4Z re i = E o if! u E E , I- re re o re c S \u0026gt; Pl  .! p 2 re Sf 5 c re St c .S rit CD tz 4- O TT uz oj m m u CJ iz S in 0^ XJ y* Itz c jn E O in A   in *-* (D XJ o c iff 0) re nj ID \"o o X  0) Di 01 re in c s TO .ti u \u0026lt;D 2 c y p cn -C *2 IM \u0026lt;p cn u c y c y CP fo E tz CP C c -g P Ol in p \u0026gt; -i2 Jr, \"O re if! \u0026gt; o re \u0026gt;Si = c S! P 2 o o) '^ = if! tJ t\ncnj- g, fl) co St x: 01 01 (P c .^ E : CP .-2 lii \u0026gt; re u 01  \u0026lt;*- 2 o? re 5 45 5 . X = 9- = P St ui re 4,c tJ re _ oi in (D rei-iD QJ c c (D c n -Q in fU c S\u0026gt; 5 re o  o co p O 2 Q. 5 CP E S |^S\"8g 5 o  O re o E (f! SJ . (U res s? E -J CD . 3 - in E u CO 1-4 (Z) z o CL V) UJ O) c Z u re  c o O1 c u c (0 E u t 0) o. g n\u0026gt; 'c a\nre s. o re 5 cn a s's  re y -n co \u0026lt;U u its CD y u_. H \u0026amp; in (D 5 \u0026lt; Z o I-I (A W lU 'B c  0 o  \u0026gt; V y 'in (O CD O 0^ CL z IH z o o re \"t c  c a E o u B  u\n' .p .12 c: Z) o 4Z g re c S .y 5 \u0026gt;- \u0026lt;D p o P C O OJ -g 01 o  ro E -2 is cn re c: H---- tn Q. u ii 5= CP 0^ X .2 re -M 01 re ti y r^ g u in * CJ in (U \u0026lt;v  (D ru O) c c o V) l/i in j3 \u0026lt;U = \"Z \"O  CO CP M (V o\u0026gt; X5 - o u- \u0026gt; w in \" \u0026lt;P F cn 52^2 re re\u0026lt;re o c tfi CP  sS i  re 2 v\u0026gt; e s o - \u0026lt;1^ (P (/) in (U 5 ?5 (D .\u0026lt;1^ ir lu $ O O  -a 2 5 re re re gr 2 3 U U \u0026lt; Q. O) 01 t 1- O g -1 p 3 u 0) S  S ? iS H = (U in -iii o \"P P vi c \u0026lt;u p O Q. SE . g^.E =3 X cn ro 1-. 2 !^ C o c !5 6 '5 in i\n-c in re i_ a\u0026gt; CP  -c nj c-\u0026gt; \u0026gt; CD \u0026gt; OJ o h- -c  P O) c  .5 U. J= .E re n. . V) 3 tn mi:: O  o cn UJ h4 h4 i-i EQ Z o Q. (O LU Q \u0026lt; z o l-l tZ) V) UJ u. O Q Q. w o u O u s 3 U u \u0026lt; Ot c c 5 c re Z .o t 8 c ro E u c V Q. o s u D ro x: '3 CT C ifi b c S u e Q. u in ro co \u0026gt;2^0 = ro .y = c P  - 3  (V cn w S c E E ro o -c u 4-/ (U (/) (p . (D 0) Q. o ro u - .y ro tj ro -a .S\u0026gt; t ro  3 K it g.^ \" ro ro ro Eg E ro .\u0026lt;1^ = H E o c \u0026gt;. ro .9 -5 E E tJ  fe-5..2 aj c E \u0026lt;3 \u0026gt; in  O c +{ 9 S cn J S ro c E  q5 c ro \" (U Q CT .t in iZ Q ID I- 2 E o c o  n (D .9 E E  P  in E in  O y\u0026gt; cn ro c ro \u0026gt; ro xt P c \u0026lt;u - ro \u0026gt; c C .S C 7= ro in ro .2 lU UI UI  ro 2  !fl E t\nro \"c 2 ro ,0)  to Ct H ?9 s  (p in 3 p S3 c Dl c .9  (P in C \"2 ro - 3 \u0026lt;u ro CT^ in -n \u0026lt;V CT 2 E 0 O Q. u ro ro ro 1^ lU H -o |- E E \u0026lt; c R g\u0026gt; E 8 E o ro .o in \u0026gt;* c in \" (J Z Hl \u0026lt; z o o c w c a E o u -\"A 2? I in cn o c = t: c E ro ro E O .S2 T3 e-g \" a? 55 E Vi i3 c 1 .-r. C Z) \u0026gt;* o E '\u0026gt; c c yi O Oi i_ T. rz Ju W 4- VJ ro ro in ro .\u0026lt;1^ E J! P O . \u0026gt; O J3 C C COO) ro  E 9 ro c  -s Q. W w E 5 5\u0026lt; I- o  -ro - \u0026gt;2 B S E E B   oi ^? 8 SB 8 \u0026gt; c _H ro b cz (2 tn  c ro \" o,.2 3^5 5  c ro ro' u  ^rog^f^^E S 5 E 1= \u0026gt; Q. c - ~ - 0) H ro o ro 8 p -B ro f, E E   'p^ D 3 to 3 3 u u s in 2 ro-o \u0026gt;\u0026gt; c in  Q. O o Cl .2 c c ?  ro cn ro l-S 5  E S ro   ro t 1 ID C 01 3  o ~ u in cn^-i2 ro c r in i_ I m ro c c ro P? -c = o ro y s c \u0026gt; Q. S . c -K E 8 c o uO 4 .9 -E = ti i/\u0026gt; ro in ro ro c o c E ro in o E E 3 55^ S E S' c c lu y\u0026gt; CT   -  c C c: . O ro  -S .i2 xJ ro JS 2 w S ro c to c SJ \u0026lt;p = fc I (D c t  ' 3 E O ro tJ c o  o cn \u0026gt;. c  ro o E != \"' ro g ro ~ \u0026gt; E - y c o  - - p . = . SS .i^ ro (S CT - f-E B ro -i F, ro g -S  Vi ro ro ro O S n '*- -ti *-* QJ  V\u0026gt; \u0026gt; -t! 3 o P i-! 3 P \u0026gt; tJ (u ro o ro ro ro ro p c ro .b o jn p 3 3 c S tj o -c ? *o KJ x3 - o c c p ro c o E p M M C xi ro ti ro ro o KO m E ro D c ro e ro o .Si \u0026lt;1^ S ? -S 5 9't! g ro   ro 5 (- in rs zu 12  9 B6 \u0026lt; U D ro H lE V) = H\" c  .S2 re c o 3 O Vi h P M C U ro .E 0^ C O Z c o uc Sfl - ID (A UJ IH hH s l-l (A z o a. w UI Bi Vi .2 1 (0 LL X I V u o ID tn 'zi CT C n b c jj 2 ID 3 CT ID 2 C 4^) -- ^ Al C = 2 S Id ID p CT re c g re 2--S _ o S .51 E S re ID ID * - - - C ex CT a 0) in ID ex 2 g Q. O u, C CL .2 o P  3 E Be -S \u0026amp; u ID E 9- P C P - c \u0026lt;u 3 in ID .. \u0026lt;1^ c E p it P 3 5^ tn ID 2 p P c C .ID 5  B q. '6 oi 2 2 ? -c E CL *5 \u0026lt; z o l-l (A lA UJ O Bi 0. c o s s E 3 E E o u R) E u t V 0. o Z \u0026gt; y in ID co se E 1^  ID 3 i_ -Cl  .  c 2 tn tn '3  ro .y\n o V Q-.\u0026gt; c ^ '! = S tj ID ID 2 tn E .5 o   i2 ~ cxL' .\u0026lt;1^ H ID ex z IH \u0026lt; z o o 4.1 c V c o Q. E o u ?  .p .52 Z) c o o '-  E 2 I .E ex ai (D 2 S tn tj ID 3 E D \u0026gt; in o c Q. (U \u0026lt;D *-* U 3 ID O P XJ H ID c   p J3  O c E u O 3 3 O C p C \u0026lt; 1-4 .a 4J \u0026gt; - w C 0\u0026gt; . in ID ID qj re \u0026gt; ID P c = S   C ID = F - \u0026gt; p 5 E . otncnc^*-''Eo  .2 e c  \u0026gt;.B \u0026lt;u^ q.~E-Q S tnE_'gCin-Q.\u0026amp; 'i_iDn3oMy Qj't-c'auUgp \u0026lt;u .g -M .P, c \" 'G S 23 S s ID .! in g  -3 g m S aj ai ? S\u0026gt;3 o t cn Oo^inin(j--= M_ re c *5 ^c^cocjeo\u0026gt;.-fc! re.o-giDj .^Pc yregi!tiffi?:ire e Q. C c in c \u0026lt;U 3 in o E Q. ID 52 ID E O) ID o -E ID u tn E D--.3 K \" i  in S' .P C a ID S \u0026lt;u -o c u re 0) c g 0^ CT p 3 \u0026lt;v ex c H C *3 CTCT i_ 0) O l/\u0026gt; P 5 o ID (D c: co tn Q) ID 4- c in p P oi is *1^ O .\u0026gt; E -6 E ID E a OT-S2 E g S e sn 2 p 2  !g.g Qi   E B-a  2 S 0) S P U _ re \"Q S ^/^ *- ID c o B g aj OT:= aJ 5 ai ID 2 ai ^ = '- \u0026gt; u C u .y^ o OJ .O .y  t c. 9tj 2. y \u0026lt;y S cn 2 S- IZL \u0026lt; H CT H in ID H c h er ' /II \u0026gt;n ni (V m p c ID  O P is o t_ \" 5i ro  .E  cn p tn g. tn ~ O E fc p 2 s x: - 3 .y o t -J ex o ID ) P 0) c .2 Q in -CT c Qi o c m 3 /I JJ 3 P 5 , O U c . XJ S P 2 c c 4= -g Q. E c ai tn O ID is /n \u0026lt; \u0026gt; t. ID ID ex ID in u ex tn _ g tn 2 2\n a VI c \u0026lt;u \u0026lt;u 3 \u0026lt;6  aS .2 5 ? in in ID ID \"CT CTl ro 2 re ID E gS S' !^! \u0026lt;^ c Q cn ID ID u ID tJ ID in c o m c 8 J3 01 c c *- 0) ID o S.B D C O in 3 .Sep I- P C in c ID in n 3 llg re X fc .= T3  c M 3 3 (A O \u0026lt; p- ai E  E p p -^ B* Al T ID O .P Q. c j= in P ID O c CT i_ ID d (U cn -c K tn c u \u0026lt;u . ID  _ 5 .E .2 E  D 2 O E  .2 3 2* h 2 \u0026amp;E c re LU LUtJ (A .y LU IH M o s \"O c re 8 iG o ? O O -c  J o \u0026lt; *- ? o ~ .l_l c re E u c 0. Q s s l-H LZ) 3 .O (/) :3 LU t C O o Of U Q\u0026gt; .. z F, hH Ci c z a 2 s O O u ii CT q\u0026gt; SQ 3 O' c b c p u p u 'in \u0026lt;D CD tJ 45 5 :i ti 5? c O Jr 6 .9-^ s -c u . . re jG TO c jz flj E .re - c Ct 8 re 0 O (TJ . ~ \"O JO re E re -2 S QJ c P Olli cn t re :i re 'I g 8. re 8  5 E  16  - in -5 '5 c o .9- CT CT qj tn c in 7ti 8.S o re Di P. '53 re u P D C ru qj \u0026lt;D O KI X ti o !l  t s 5  2 3 to {J 2 CT o  \u0026lt;u o (U = 8 ui .= in 3 S .\u0026amp;p - in c C -c \n(fl 2 c E O q\nCT x: p (U in q\u0026gt; 3 CT fO s in lu  rv (ft .x: CT gJ CT q3 O [QJ u u D o _ Ct c IF o _ '3 S2 c ,re (fl o re 0, t fc re  cnxj \"o p__  c o -ore ^.e-i_ o c oi c o re c .2 !2 n C ft 12 3 re o  \u0026lt;u  re \"o -(j S -5 E  S bi .re Q. CT CT C T3 4- ru c o c a I E trr 3  \"o  ^ B B S S U in c 3 w tn CT Ct ro S C c o a, 4_ (U (D O) tn *- \u0026lt;u 3 CT S re 8 [(Ij re * Q. 13's E re o Si 're S c D c tj 5  5! S [2  re y t \u0026gt;  u! (u 2 9 Bl -o  3 -a I- ro C 8 8 -2, \u0026gt;  t re I i S I  -c  \" 2 s 'fl (U t i: _ u/ I re .ifl 3 re -= V) o . Ct^Q S Q (fi in c -Q in qj I/) \u0026lt;u C P \u0026lt; uofa.9-\u0026lt;uc^c5(D flj= u c 5 ..x: qj = J F  o .w  0- ro u -o g-o J- . q^ qj \u0026gt; \u0026lt;u CT U in c ru O . H E \u0026gt;3  S a) Q. p 2 \u0026lt;-\u0026gt; Q. in V s y =\nru O' p c 8 p *= .K b! in .  - a H in c P 0) c (U }n -P- (U . \u0026lt;y c \u0026lt;u c re w re  tj - 8 Si, .y o' tut (TJ CtCT tj I- q^ -c q) .b CT 3 .iG Q. nj ._ A -y q3  .re g. H c CT c nj Ct . :y 40 t^ c ro p Ct 0) tn  s o 9 3 CD u tfi Id CT :g-S c ~ sIS 0) (D ..=is S'? c U Q. - s s q\u0026gt; O t? CT Q. in ro :y i5? ti H q3 Ct c rti (V S. ' O Q. q) Ct b tn  u \u0026lt;0 .\u0026lt;v \"2 O CT \u0026gt; c qj qj q^ \"o CT O (0 u 0) tn \"3 M M V a. 3 Z O' Is 2 o u  2 Q re u flj =  0 : s .y u 01 V) C c  O C B  2 :11s o t ro Q.  C L-. ei5?sI^ C 3 C 0) E \u0026gt; 13 5 CT \u0026lt;5 CT  CT CT c in 5 (x\u0026gt; ID (D C r- O O E I'' -M  10 c (u y CT c 0) = D c (U js .y \u0026lt;/\u0026gt; in S 2 -o re 5 3 CT Q. *- q\u0026gt; tn 03 3 c qj ti = C u := in c tJ  rr. Ct 55 55 ? u \"CJ .- re . re 3 2  9-13 ~  o \n- 8 -s 2 c re re re g 8-o  C u c c c o c 0) F top T5 CT E C.  = '- y 2 .8^=2 8 =\nS qj c c \" c 9 c CT .E CT (S :i : 2  E re I y i E tn Q. O re o S= -2 8| S y P S  (D in in c o X 23 *-* O u :9 C c q^ c  '2 o -   re re 2  ifl Xi c  tn .9- qj c - E tn in q? 00 m E CT 2 ~ a  re -Jg =2 o E  U c \u0026gt;1 73 re XJ c f \u0026gt; 0) E Vi 5 ft uS\u0026lt;XJ c ID c E Vi lU 1-4 CO (Z) z o Q. lU \u0026lt; Z o tn (n lU u. O CL re c o w tf) o u Q. O) c 'E o V \u0026gt; V o o c re oi c s o u U Z \u0026lt; z o o c V c E o u XJ 0 in ' CT .C E? b 3 -is \u0026gt;_ E -55  P E^ - P 2 S .2 p -S \u0026lt;u S 'c o s QJ S. C XJ S OJ 3  := tn y 4_ nj Q. . S r\u0026gt; TO - t: o 5 = OJ (5 S. .2 ro - U ID .\u0026lt;1^ o \u0026gt; - Q. a \u0026lt;u ID ID c o tn CT C  tn GJ .= tn tn GJ ID -C Q GJ GJ -C _e- y u:- ID S? O y S tn c : \"  \u0026gt; -O -2 GJ *~ E ID -t* C GJ tZ ID w ? O CZ ID ID C ID tn P \u0026lt;u  -Q -2  C E n) oi cn o CJ ID is cn c c S s t\nC -c . re - E -s  Q. Q. C (U E c y p 8 c lU E u 2 V Q. O s u 'cn ID co   45 .i2 s c Z\u0026gt; 4-. P .5 Q. S XJ S ID _ P \"c Q. GJ GJ O O^sSft Q. gj re fc 5 o -c CT cn (J GJ cn tn ID \u0026lt;U tn : ID c (U ,p o 2 Q. ID E o S 2 ID iJ S ii=: = Cl \u0026gt; Qj ^2 -F t B 5  \"o Q- ID lU .2 p F. - u tn  ID 4) 2 Q. .C GJ 4-* OJ CT S S. g 5 s t o S \u0026lt;1^ n O' \u0026gt; u : 4J \u0026lt;U C ~  IP c p E  o P o -C cn cn in (U -C o I/) E 2 S o Ip c I u ID \u0026gt; tJ ID OJ M- O)~ o *o s .y ?? s S o - U E o\u0026gt; = V u * J? -5 o .X re  43 1) C 0. V \"O  g g u (Z) a o Q. S . 2 s E! t: .sc o 3 GJ .E XJ r: \" GJ S S' ID -C ro .-y 5 5 2 a o) 2 .a\u0026gt; CT S \"O \u0026lt;/\u0026gt; 5 -Si c E  ni 5 = = S Q ui S VI in cn o w, c k tE c c O - (U ID  u tn 0) 12 (V e S. O Vi \u0026lt;1^ O ID 1^ O O E ID  in o 2 o -s\ni2 = .   \u0026lt; j: in .2 kC 4-^ GJ O GJ = o x: *0 -9 ^  UI 5 S t .p in -S' g . t: in 3 \u0026lt;U tn *- i-J E ID OJ  :  GJ OJ u 5 O  OJ (/) Q.b* (A UI E M CO -l (/) z o Q. V) UI Q \u0026lt; Z o V) V) UJ u. O of Q. E .2 re c o 'm M o k. Q. O) c 5 o  W 8 c ro E u t V Q. O 5 V   o c O c \u0026lt;c I \\Z) OJ 5  c  \u0026gt; OJ GJ 2 -g -i \u0026gt;. x: S' s . -C \u0026lt; 5) 5 c S2 . 8 w 5? 8 Si \" 8 -5  ,$ 1 E Q. 3 to c 2 3 p o S \u0026gt; c \u0026lt;u c - = S 8 GJ O y GJ E i*' a c ft C O) P C (J .55 \u0026amp; \u0026lt;u l_ \u0026lt;/) (U c ro (U .\u0026gt; H tj ro o GJ s ro D .i2 ti  GJ (V w o u ro GJ -S \u0026lt;V t/i Z \u0026lt; Z o Q c V c o Q. E o u W -V\n\u0026amp; te 33 : ,  iS fl I. 'U a c GJ o in p ui vj GJ -Q ro $ 4_, GJ o c c V) GJ U ro .\u0026lt;y Q J3 V C U V  n '5 \"O  S a S  ro  c ? 2 p O) Q. GJ  GJ rot GJ 05 tL \"D i_mGJ*o^ro^ OJ J2 OJ c = =\n\u0026gt;**0 3C(Xjrou^P u GJ ?.  I/) GJ c ti . 53 2\n-y ro 2 t: ro 2 2 ro 2 3 (Z) .! nj \u0026gt; V) C \u0026lt;U \u0026lt;V o .-^ o c: H in C -o Xj 3 OJ l: S ti ft 1 \u0026gt;30. h  \u0026lt;u  \"5 j? TO -M * (V (D H  O GJ C 5 c 3 ti 2 Q. Vi GJ 8 3 IZ) Q.^ O ft  -c ,ro 3 o U) (fi GJ ro o GJ 3 ft -C \u0026lt;/\u0026gt; \u0026gt; U C OJ ro GJ u .\u0026lt;1^ I- p P fu 3  V) .ro c ** GJ ro p GJ s \u0026gt; 8 S 8 8 3 2  p J3 5 \"al o O c 8 a CL \u0026lt;u ro Q. P -hi o u ro ro x: (U 4-\u0026lt; B-  \u0026gt; o \u0026lt;  c i 2 4- tn .2  \"J \"2 S 52 S tn -c itJ tn S O c cn-O tu a 2 'F o Q- 5i N  GJ 'u 2 VJ GJ (v V' v.* -S-  ro ~  2 -a -a c S u .2- S -5 o 0 ,GJ (/) t: o\u0026gt; S. - -t^ .c F \"a K \" 2  5.2 F p v5 cP ro D c t\n5 c g : \u0026gt; 23 E S U o 3 ro ~ \u0026lt;u S ro c c 2 ra .2 .2 GJ TO \u0026lt;0 - in -=1^ 8.E 8 2 Q. o   1- OJ ro ro 3 c w ?. 4= ( o ^-gtS-S 8 E g f5 o 5 jn  m X. C OJ Xj GJ O C t'J ^1 F 5 -5 3 O 0) (/) L. i/\u0026gt; 2-^ in OJ I ro GJ E c tu x: u O ro V} ,GJ ro H -Q o\u0026gt; c IS re Z c o \"C! u 01 O D CProfessional Teacher Appraisal System Forms The Little Rock School District believes that teaching and learning are the most critical aspects of the teacher's responsibilities but while still holding that the professional responsibilities of the teacher are essential to the development of the school's culture, climate, programs, and in meeting necessary building or/and district goals. To ensure this goal, several forms have been developed for this process: PTAS-1 PTAS-2 PTAS-3 PTAS-4 PTAS-5 PTAS-6 Classroom Observation Professional Growth Plan Instruction and Reflection Profile Teacher Summative Appraisal Instructional Unit Developed for Portfolio Professional Development Log The asterisk (*) components on the LRSD Summative Appraisal form (see page 51-52) have been identified as critical and are reviewed to be important skills of a successful teachers repertoire. Thus, if a teacher receives an unsatisfactory or below basic marking in one of these critical components, the teacher will be placed in Track III and placed in the appropriate phase as deemed necessary by the evaluator in accordance to the degree of assistance needed to improve the teachers deficiency. The summative appraisal form also offers an overall review of the teacher's ability to have met the district established expectations that all teachers will have to meet in offering a quality education to all children of the district. When assessing the skills of the non-probationaiy teacher in Domain IV, the evaluator will only assess components 4a thru 4f. The Professional Tools in this section are not apart of the non-probationary teachers expectations. Lastly, because Professional Development will be a critical part of the teachers experience, a list of necessary and meaningful workshops have been identified in enhancing the teaching components identified within this document (see pages 58-70 ). 42PROFESSIONAL TEACHER APPRAISAL FORMSLITTLE ROCK SCHOOL DISTRICT CLASSROOM OBSERVATION PTAS-1 FORMAL INFORMAL WALK-THROUGH Name: School: Grade Level: Subject: School Year: Observer Name: Position: Domain 2: Classroom Environment Component 2a: Establishing a Culture for Learning Domain 3: Instruction_________ Component 3a: Communicating Clearly and Accurately Component 2b: Managing Classroom Procedures Component 3b: Using Questioning and Discussion Techniques Component 2c: Managing Student Behavior Component 3c: Engaging Students in Learning Component 2d: Organizing Physical Space Component 3d: Providing Feedback to Students General Comments: Component 3e: Demonstrating Flexibility and Responsiveness Component 3f: Utilizing Technology 44Name LITTLE ROCK SCHOOL DISTRICT PROFESSIONAL GROWTH PLAN: TRACK II PTAS-2 School Year Participants Growth Goal(s): The goal addresses components in (check all that apply) ___Domain 1 ___Domain 2 ______DDoomaaiinn 33 ___Domain 4 Activities and Steps To be Taken Persons/Resources Needed Documentation Timeline and Deadlines Expected Outcome(s): Evaluator Date 45 1/if\n= 5?^ 5s: 'Ki S CSJ no  s: .\u0026lt;0 -tS CJ 3 IO s: '4^41 \" '-k A (Z) \u0026lt; H u X co H co co a V Q O 43 CJ CZD s c o co 3 (9 \u0026gt; tu o u co Q o eg' e . .2 ZS K i-1 u W-5 U  55 a o ft.  0\u0026gt;  fe 'J  '5 .o I o  \u0026lt; - \"ss ca X O 5 3 Q Q -O OX \"O V TJ C VC 3 O^ n E co o o S co CO o CQ 3 \u0026gt;W V T3 CO  o 3 c/:) CZ3 I Ip 3 o 73 5 co o c a I \u0026lt;u o CJ co 8 X 1\u0026gt; co 43 O H ^o e ^15 sJI Z\" S' E-i y| P?l ZiS \u0026gt;4 o c 1^ c 3 O o no co 43 cz\u0026gt; CO eS e o co co u CO5 ,o CxO c E co u c o o 2V] co r- 00  1\u0026gt;  5 O 4u\u0026gt;. *c*o 2 c c o CO  o Ofl 1\u0026gt; co 1\u0026gt; J= C U to O 43 3 o o \u0026gt;  43 s\nB g :P' S o s ,3 u 5 3 3 O Q s C/3 S 1 (/3 ,O C/1 3 0) *3 a CZ) ex. 3 O \u0026amp; 3 2 *3 O o X 3 Q O 3 H g (CO c 2 o (/) S ,o -J2 c u o 2 W5 O. 3 s 00 3 O o X 00 S 3 g \u0026amp; 2 C o Vi O X 3 O^ V c f o o TJ O u fa 4\u0026gt; WJ TJ O a u E bO c C4 V 3 C^ 2 V C/3 (9 J3 C c o (/) o V)  S u 3 Q Vi O o e OD C LS u \u0026lt;9 V eS TJ O O s o)  c o V) o -C o 3 O^ 4\u0026gt; \u0026gt; e J=o' t/5 O) p H CQ s o 3 O^ o 3 o as \u0026gt; o \u0026gt; 1? Q \u0026lt;U (O Si \u0026gt; u eq o o t/5 eq 1q c TJ 1\u0026gt; eS u o \u0026lt; o E P c/i s i 1/5 ,o O W) 73 O O s 3 LS 4\u0026gt; 3 O (Z3 c o I \u0026gt; u c ,eq *5. G .2 W 3 \"eq \u0026gt; o o *3 V G 15 3 G C G Q co to qj\u0026gt; to 00 G  3 c^. 15 \"5 G Cl. S eq w .^2 TJ w 15 2  c eq 00 .c S C (Q cj M 5 o S 43 00 \u0026gt; \u0026lt; TJ 1\u0026gt;  'S. 3 o^ V \u0026gt; e f/i V \u0026gt; R} o O O \u0026lt; o E H C CQ V c/) S 3 O {/5 a\u0026gt; w s c o 1 co .s  Wi o eS E u VJ V s o OT o 3 o o \u0026gt; {Q X t/5 Q C C O 1\u0026gt; -S c o 00 .s E (Q O G O T) 5 vi \u0026lt;lj W 3 \u0026lt;3 u o c eq o. 3 O^ O 3 C o u= 3 c e\u0026lt;3 O CZ500 B 2 3 2 00 cn 2 B \u0026lt;u tn O 3 u \u0026gt; eg 43 f o 1) ?2 o \u0026amp;  o^ 4\u0026gt;4 u eg \u0026lt; 1\u0026gt; ,o u o 3 *0 . o c eg cn cn eg S C^ . eg cn C V o 2 i/i O U S eg 2 o 43 u o G u 2 o Vi O cn fS c V I op cn cn eg cn 2 c o o c V XJ 2 in O f/i o a  o- c 5 (/) CL  2 Q -5 \u0026gt; o cn o c o t/1 :2 5= \u0026gt; p o s Cl c/\u0026gt; z o p ca n o I 0^ o R 'I. 43 o' cn -3 o 2 73 o g eg 5- O \u0026gt;. 00 c 2 c* eg s p ts eg CL \u0026lt;U o\n3 XJ q o E eg cn t\u0026gt; 2 o o 3 o 2 o c u .V TJ a o 2 3 Q US czT C V T3 I CJ w o 2 O C 00 CQ M tn eg \"o tn eg o 2 00 .S o 00 2 u o 00 cn cn eg \"q cn 2 i X o 3 O X3 2 B eg *2. 3 O^ O O (4 2 XJ Q c: a\u0026gt; CL CL CQ t c o *0 m c\u0026gt;V Q 3 CO g ,o o ex CA \"ex 3 O \u0026amp; o co 3 3 CA .o c  o CO 3 Q O ffi c o (A (A o CO *3 O .g 2 o X CA 3 O *3 2 CA o o. 3 g W) o co 3 3 .3 3 co c o o o 3 CO g ,p o ex ex 3 2 0/) o CO 3 3 CA Lc ,o C 3 O o o co CO 42 c o (A CA o CA Ck. *3 CA o c 3 o  co o bo bO .3 2 co o OJ 3 -5 co ** US 4\u0026gt; o \u0026gt; -3 -g wj co 3 \u0026lt;U 3 2 c o o 3 CA o, S O p (A V o (A  3 g o. C 3 S o 3 \u0026gt;'  = 53 '5 3 S: Q O T3 X o o o bO CQ 3 o ' co ex ,u s o co J3 bO -S 3 co u V 4= V) QJ K E CO X o o Um c o (A CA 5 3 o x* co CA 3 .2 CA O g- o CA 3 .2 o co 0) CA 3 1) I 3 Q 43 O nJ -O U  5 3 O 3 co 8 o  'O *0 co Q CO g !5 ,2 (N o tnlittle rock school district TEACHER SUMMATIVE APPRAISAL PTAS-4 Teacher School Year Mark One\nMark One: Mid-year Track I Summative Track II Track III U=Unsatisfactory B=Basic P=Proflcient denotes heavily weighted components D=Distinguished DOMAIN 1: PLANNING AND PREPARATION Component Level of Performance U B P D la. Demonstrating Knowledge of Content and Pedagogy* 1b. Demonstrating Knowledge of Students*_____________ 1c. Selecting Instructional Goals*_______________________ Id. Demonstrating Knowledge of Resources_____________ Ie. Designing Coherent Instruction*_____________________ If. Assessing Student Learning* Strengths DOMAIN 2: THE CLASSROOM ENVIRONMENT Component 2a. Establishing a Culture for Learning* 2b. Managing Classroom Procedure* 2c. Managing Student Behavior*_______ 2d. Organizing Physical Space Strengths 51 Areas to Address Level of Performance U B P D Areas to AddressDOMAIN 3: INSTRUCTION Component Level of Performance U B P D 3a. Communicating Clearly and Accurately*________ 3b. Using Questioning and Discussion Techniques* 3c. Engaging Students in Learning*_______________ 3d. Providing Feedback to Students*_______________ 3e. Demonstrating Flexibility and Responsiveness* 3f. Utilizing Technology Strengths Areas to Address DOMAIN 4: PROFESSIONAL RESPONSIBILITIES Component Level of Performance U B P 0 4a. Reflecting on Teaching_________________ 4b. Maintaining Accurate Records*_________ 4c. Communication with Families___________ 4d. Contributing to the School and District 4e. Growing and Developing Professionally* 4f. Showing Professionalism PROFESSIONAL TOOLS Professional Development Log Portfolio Strengths Areas to Address (If the teacher disagrees with a part of the appraisal or with a recommendation made by the prindpal, the teacher may elect to attach a written response to the appraisal instrument. Both documents shall then be placed in the teachers personnel file.) Principals Signature Teacher's Signature (Indicates that the teacher has read the report but does not necessarily indicate agreement with it.) Recommended for re-election Yes No (To be completed at the end of the school year.) 52PROFESSIONAL TEACHER APPRAISAL TOOLSPortfolio Requirement  J. Track 1 W' ' ackground Information: Resume (Profile Sheet) Individual Improvement Plan/Professional Growth Plan ^assroomManagen^^ Class Schedule Sample of units developed in support of the curriculum Sample of modifications made for identified students Daily/Weekly Schedule Sample of weekly lesson plans Collaborative Skills: Log of Parent Contact Samples of colleagues collaboration ssessment Information' Sample of Assessment Student Achievement Data Student Academic Improvement Plan Professional Development Documents^ Professional Growth Reading List of Professional Journals/Materials 54REQUIREMENTS FOR TRACK 1 TEACHERS/SPECIALISTS A Portfolio will be required with the following included:  Resume Provide a summary of previous employment experiences, education, and extra-curricular activities.  Sample of weekly lesson plans A copy of lesson plans for each subject taught. This should include standards and benchmarks, materials, activities, and assessments.  Sample of units developed in support of the curriculum This unit should include objectives, materials, activities, and assessments. The unit objectives must be aligned to the core curriculum of your subject area.  Sample of modifications made for identified students Identify students with Individual Education Plan (lEP) and provide written evidence in the plan book of modifications by subject area.  Log of Parent Contact This should include phone, conferences, notes, open house, and any other form of parent communication.  Professional Growth Provide documented evidence of professional training (A list of required staff development will be provided.  Sample of Assessment Provide samples of various ways in which students have been assessed to determine academic achievement. These should include teacher developed or modified forms of assessment. (Teacher-made tests, activities for students portfolio, projects, etc.)  Student Achievement Data Provide data indicating academic achievement forthose students for whom the teacher is responsible for teaching. The teacher should indicate that data has been evaluated to help improve instruction. Example: Documentation showing that the teacher is aware of strengths and concerns of students in class or classes.) The grading outline should be included to show point system for various types of assessment done throughout the year. Grade Distribution sheets can be included under this section. The information should demonstrate that the teacher has evaluated the data as indicated by notes, graphs, or narratives.  Individual Improvement Plan/Professional Growth Plan This plan is developed around the components of professional practice. The teacher and the evaluator/administrator collaborate on the plan. The teacher takes the lead in determining the particular component that will be developed. Class Schedule Documentation shows the daily and weekly scheduled activities for which the teacher is responsible. This should include the times each subject area is being taught. An elementary class schedule will look different than a secondary class schedule. Student Academic Improvement Plan A sample of the plan or plans that have been developed for the student(s) that require additional academic support in order to meet the state guidelines for achievement. (The name of the student(s) should not be on the sample(s). Reading List of Professional Joumals/Materials A listing of professional journals and/or reading materials that the teacher has experienced during the year. The format may follow the APA or MLA style. A new portfolio will be developed each year. The portfolio will be reviewed midyear and at the end of each school year by the administrator. 55PTAS -5 INSTRUCTIONAL UNIT DEVELOPED BY TRACK 1 TEACHER FOR PORTFOLIO The teacher is to provide an instructional unit of at least one week in length, artifacts from that unit, and the following information prior to established date set by the administrator. This evidence will be submitted in the portfolio in preparation for the final summative conference. Name Subject of Unit Grade/Subject Taught Dates of the Unit Unit Concept/Topic: Objectives/Goals for the Assignment/Student Artifacts Attached: Attach a copy of a unit that you have used in your classroom this year. The unit can be in any format you wish and taken from any point during the year. Attach an activity or assignment directions that engaged students in authentic work related to the concept or topic cited above, (e.g. project guidelines, problems, homework assignment, center activity) Provide some evidence of student learning. This should reflect the full range of student achievement levels in your class and should include feedback you provided to your students on their work. (e.g. samples of student work, photographs, audio or video tapes) Be prepared to reflect on the artifacts and the unit in your final summative conference. 56\u0026lt; (00) \u0026gt;o oo w D a\u0026gt; \u0026gt; 0) Q cV 0) mo oc 3 XI O q: O) O co o 5o2 oo oI oo 0^ J c o E Q. O 0) \u0026gt; o Q ro c o w w .Q 1O-Q. o o u (n i0n) L. 3 O U d o  m - k. c p P 5 \u0026gt; III \u0026lt;u  o c  p c o o 0) LO o RJ O V Enj Z -o 4^ 8 a-  E fa CM \u0026gt;\u0026gt; a a CM cm a   43 Vi E ti 3  OX) 8 o = J3 a T O + u  -o 3 R a 3 CM \u0026gt; a a  E 4\u0026gt; JS 3 O JS s 3  .8 \u0026lt;*\u0026gt; a sT a \"O a \u0026gt; a -a XjM 3 O  4) IM) o CM a m C s ex o \u0026gt; 1  -I Q -I o ,o a XI a *-w a 0 ^a LE  a   c Q o  O ( 4m (J  co Cm   O S   W u   CM   5   E a *   Vi * 4S 3 fa a t. tS U a\u0026gt; a .S2  H 43 w a Cm  43 \u0026lt; a 4^ \u0026gt; a SB a  a  a ox .E a  .a 5 o o CA a 4\u0026gt; OX a a r\nw ti a O o I V) a\u0026gt; 4\u0026gt; JB 2 a u o I o O U \u0026gt;.5 R ( a o 5 O k. s o .S a\u0026gt; fa o 3 O o x: a  O a a a  u .1^ a a u a o fa v V u 43 . **  5 'S  e*  V) O 0^ V) u .9  (/3 43 H e#  a  H '5i^^ (2 gj OJD a 8 o a IZ5 E   u 1^ 4\u0026gt; 3 V t/i \u0026gt;  fa a  E  ox a a a E a  E  OX) a   o S \u0026lt; E Q  (/3 a *  43 ,o a 4) w a o u V a Z a X S    \u0026gt; .M U ?* U) a\u0026gt; a\u0026gt; a Si E IXI 3  w 8 MM i  ti   a s  E a    ox a a a CM   2 P a fas \"W S II   OS \u0026lt; \u0026lt; es E I  ox . *3 3 oi . 's a 8  a ti *8 i OX) 8 2  a  .a  c a ja p a  E CM  CM  M ** a ,o Q\u0026gt; V3 a 5 .2 -2 a CQ  OX) a 3 OX) a a 4\u0026gt; a o d a a\u0026gt; \"o ^a *a a 4^   z Cl 03 Cli 1-3 w a 2 a 43   H \u0026lt;   M -o a 3 .0 R .0 '5 3 E E rP M V \u0026lt;! 3 CA u a O) I. s 3 O  fa \u0026gt; a a  s  \u0026gt; 0^ o a  \u0026gt; a \u0026lt; o in a V) O CX) B - \"S  S 2 o 85 w A y- -5 4\u0026gt; CO a Q 2 .S  ' -w U 3 fl s Vi O a s o r: CM \"O a a S3 CM -  a \"O ti a    o (A 'L 'E^   \u0026gt; a o V CA a o a ox a .2f M 7\nfa O \u0026lt; a 2 5 \u0026lt;M .S  i S S s - g 3 2 \"O ja E  .ts  -S 5 .2 XI  a a S'\" XJ 4\u0026gt; u -o -a o a: 5  M E V3 a u 0^ a\u0026gt; o x3 o'\" E  CM  8  \u0026amp; S 5 \"E  5 5  *- . CM o - 2f  a .E i\nB a .2 s a S: a i  -2 fa  \"0 a  E U CM   O V5 3 a R V vi 4\u0026gt; f/3 s a  1  CM OX .SS o S SoS  \u0026lt;-  E  0 B o -a o a U o B  43 o 2  * S E S ti ti 3  fa fa R 00 D a  a a *3 s? U S .a) u U A I ^  a js -C a  as V3 5 5 \u0026lt; .B*  U R .! 3 Q  Q ti co ti CM 11 CO ti V3 \u0026lt; o a  -a  U 4 5 ti \u0026gt;fl  a  V33 fl GJ (Z3 tS = 3 CT fl Pi fl E a o o u 43 fl S ox CT .5 a ,2 \u0026gt;\u0026gt; B 3 - a 3 fl O' E f/i fl 3 3 fl fl 9 OX *8 CA CA fl fl o 4^: fi. G 4) -fl O cu a OX) :9 B fl fl \u0026gt; 69 fl \u0026amp; O o fl fl V) fl Q B Q, 'F \"O o 2 \u0026gt;-r s I 2  \nE CA fl fl fl P c fl q\u0026lt;5 O OX fl 0. E a 8 'b -b H a fl S o H CA \" fl O a o E fl 2 \u0026gt; 2 Q a C 43 .S fl a Ct o C\u0026lt;3 -S fl 6M B JS fl \"g fl a a a B fl B CA CA fl fl fl a \u0026lt;A \"3 B fl ea C B fl B O 43 CA fl 43 fl B fl .2 a u fl fl t \u0026gt; S' 3 a B fl 'F ,o \"3 fl B CA B fl CA 3 O fl E M M * \"p (J ** B 3 H V3 fl \u0026gt; 3 3* fl Vi OX ,a  u B 2 CA 18 J3 o H a ox .5 B . a u 2 e9 0 fl U J o  fl 3 fl V} r4 fl  T3 ti B B CA \u0026gt;\u0026gt; B 3 fl 43 fl oe at a C9 fl ,2 fl 3 h \u0026lt; .S a \" K a Cm fl - fl O a s \u0026gt; .2 5 Vi fl CA A fl a fl E fl CA B 3 \\O fl 43 ,o CA fl 'a fl B B a ,2 3 h \u0026lt; 2 J .S J  ? CA CA a  Cm - fl fl a a at .2 5 2 a \u0026lt; E CA fl CA CA fl a fl E fl CA \u0026gt;\u0026gt; B -3 00 fl X fl 'a fl B Vi B a a fl 3 CA a J a \u0026gt; w (tJ (m f 1 fl fl fl a a  .2 5 S a \u0026lt; E CA \u0026gt;\u0026gt; B *3 fl J3 ,o fl fl 3 a CA fl E 3 3 fl 3 fl fl fl Ct ti 43 fl (/i fl CA A fl 8 fl E fl 2 a \u0026lt; S CA \u0026gt;\u0026gt; B 3 fl A p \u0026lt;M OX a S B fl M O OJ * I a B N fl fl 3 O A fl E a a u 3 fl E fl Ct ti 43 fl CA \u0026gt;\u0026gt; B 3 B o 00 CA \u0026gt;\u0026gt; B -3 CA B 3 ifi \u0026gt; B -3 *B 43 iJ fl H a a B ti ,fl 3 fl \"P 2 fl a (J B fl ti ti I fl a fl 43 fl B fl fl a kJ U I fl a \u0026lt; ti ti ti fl 43 fl B fl fl a B fl Zl B OX fl 3 fl 43 fl B a 2 fl a s \u0026lt; u 2 at Ml B M E .2 fl o fl CA Cm 1 O J 4\u0026gt; V) fl O 'F s g a U fl 43 .P CA fl a fl B a at \u0026gt; 2 \u0026gt; at .a  2 0 2 B fl *a fl V3 B a o fl 3 fl J3 .o V) fl fl 3 C CA fl E 3 3 fl fl 43 \u0026lt;0 V) fl 3 O CA fl E 3 3 fl .M OJ \"E ox V) fl U3 a . ^ti fl ti % ti at fl - fl fl :S \u0026lt; fl 2 fl Q-E o ti a  E .2 B O at (B fl \" J A fl fl fl 'E fl 2 fl B E fl CL \u0026gt; 2 a E w B fl M O fl a  M I r\" E-i w CA a a E S fl E ja fl B wS a fl CA fl CA OX g B \u0026gt; o fl B OX fl -3 fl 43 fl B a 2 2 a b M r\" H w fl 1 Ij o I B a fl fl fl P  fl fl fl .3 ti a fl CA B OX fl \"3 fl 43 fl B a 2 2 O 3 3 fl o B a fl  .2 5 ^5 V) E fl VJ fl CA CA fl T \"O a fl E fl a fl E fl a ti V) 43 O OX 43 fl *- B fl M 0 fl 3 fl fl E fl Ct \"W CA .8 at  \u0026gt; at J3 o 020 a  ua fl fl  \u0026lt;2 .S S o. \u0026lt; S OX a \u0026gt; fl  I fl U \"2 .fl fl \"2 ox E ** B fl z 0 fl  I i H (/) fl ja S fl\u0026lt; lA OX B -- u 5 B ! P J O 3 3 fl t o , *fl Ct  a E fl T \"O H 5k H M  \u0026lt;2 .E Ch in CA .2 -3 3 (Z! S *fl o Zl \u0026lt; u ti fl M fl fl ** Ifl u -O 5k s ox fl V3 fl 43 fl B fl 43 fl B fl a a a fl E E fl 43 B a at O) (J 8 ti fl a fl -w 2 O A fl -3 B O at 3 2 S E  E  a\u0026gt; \u0026gt; M U EiJ 02 f/i 3 IZ5 .S fl o fl 43 fl B fl fl -3 B O fl 43 fl B fl fl -3 B O fl fK -3 fl fl fl -3 B f/i O -5 X fl B fl fl ** O CAa 4J 3 C a a 2 H CA \u0026gt;\u0026gt; B a B a V) \u0026gt;\u0026gt; B a n CA \u0026gt;\u0026gt; B a B a 00 CA \u0026gt;\u0026gt; B a CA \u0026gt;\u0026gt; B \"a CA \u0026gt;\u0026gt; B a cs \u0026gt;\u0026gt; B a 00 CA B a CA \u0026gt;\u0026gt; B a CA B a n 4\u0026gt; X CA a 'ox a B V3 4\u0026gt; a 4^ .O CA 4^ 3 O 4^ B .2 a* a B a o Q s 2 3 3 a u 4^ Q CA B \" \u0026lt; L\" J f/i 4\u0026gt; U 4^ ti ti Cm 3 a a g a a Q- a a 3  .2 5 CA 4\u0026gt; CA f/i 4^ B a 2 4\u0026gt; ox -B 5 W  s a a OX) B \u0026gt; 4\u0026gt; o  I B  S \u0026lt; . a (3 S \"O n H 5k O a a H \u0026lt; ti ti ti Vi a a 4^ a B 4) fM 4\u0026gt; a B B 4.\u0026gt; \u0026gt; CA ?-s B B 2 fl - a 2 J3 4^ B 5 S CA 4\u0026gt; ja u B 4\u0026gt; fS o a B i.5 \u0026lt;2 a a 3 O 4) 2 3 3 a 3 U U a  a, CA a Q CA a s s \u0026lt; a B \u0026lt;/i b. a .l B M 2 .2 B o a CA fl   -Si B g \u0026gt; E  u a 4\u0026gt; A CA a 'OJD a B (/3 R R fl ba o a a *-w .ta 4\u0026gt; \u0026gt;  \u0026gt; \u0026lt;U J3  S a o o B W J3 o B U S  -a 2 M a 2 8 2^ O Si  I   CS 4^ ja ,o f/i 4^ 3 O CA 4\u0026gt;  ti 3 2 3 a 0^ *- \u0026gt; ts 4\u0026gt; 5 ti 4\u0026gt; a CA 4\u0026gt; U B O CA O 2 5 3 U 4^ s *- CA f/i CA ox -o - u a R T\" 0.) 4^ o H M L* B B 4\u0026gt; V u E  a a a fl - a ti c/:) CA a a B a 4^ -a B O a 3 o Q. E \u0026gt; .! o O B O I a 2 'W fl a 4^ l/i CA a 2 T \"O Si \" I-  o  o a Ba \u0026lt;3 .3 CA .2 a 3 CO .2 *a a co f/i a a 3 CA a a __ a B a u s -a B O CA V) 4^ fl C \u0026lt; a B u a fits o S 2 .s CA B  2 .2 a o a CA ( fl  - B a a B \u0026gt; E  4) X3 4\u0026gt; J3 4\u0026gt; ja .o .o ,o a 'ox a B \u0026lt;/) V3 4\u0026gt; U 3 O 4) CA 4\u0026gt; o 3 O f/i 4\u0026gt; a s s \u0026lt; a B CA u a  o '* Vi 3 _ 2 .2 a o V W V) 3 a *a  a vs a ox a a B a S B \u0026lt; . O ... o a  M I fl  5 * 2 B H w 3 a a E 'a a \u0026amp; ti CA a  ox -fi S s 2^ o a a a \u0026gt;  \u0026gt; a a   ja OJ'-S 2 fl u o a I-] 3 2 3 3 a 2 3 3 \u0026lt;M I  'J ca  .s u 3 u a\u0026gt; V R \u0026gt; E  1) a \u0026gt; .u \u0026gt; a V U aij 4^ a u B 4\u0026gt; \u0026lt;*) 4\u0026gt; -a B B B a a 2 2 a a 4\u0026gt; B 3 S 4^ a o B 4\u0026gt; \u0026lt;*) 4^ B  0 2  R ** \u0026gt; V3 tS ua a\u0026gt; .s M .. 0) \"2 4\u0026gt; 4\u0026gt; ,4\u0026gt; CA a\u0026gt; 4S S CA OX) -o -- a B B T- O \u0026gt;) a 3 B ti V) CA - 0^ o  fl a o B .S ox .5 b i a B E-i w B fl a a a 2 g a U co CA 4\u0026gt; a u B 4\u0026gt; 4\u0026gt; -a B \u0026lt;u O O o  h -^ s 5? a V3 CA  7 TJ u a 2 U 4 o a \u0026lt;S .S .2 -a 3 'W CC .2 a o f/i a cn a 3 -a B U-0 w 4\u0026gt; a u B 4^ o CA 3 a a E a  B. ti V) a OX 3 a a E a  . O. ti M a \u0026amp; g Si \u0026gt;  \u0026gt; U 3  s c:  2 o R R X3 o a J h \"S .a J= 2 w B o 2^ o a 8 \" I a  B \u0026lt; . B Si 'r a 0) OX V) 4) \u0026gt; \u0026gt;. -w B V M -- CA 4\u0026gt; a B 4) a a B B 2 ** B o 2^  8  * I a V 3 \"O a a 0) OX g a a ! 1 4\u0026gt; T3 E-I M CA ^.2 B B 2 B - o a 2 ja 4.\u0026gt; B 3 S (/i 4\u0026gt; a w B 4\u0026gt; a a B  O \u0026gt;) a .S S o S S ^a \u0026gt; .! o fi. 4^ T T? r *\" f-i w L* B -w fl a a a fl a a a a fl - a ti Vi 4\u0026gt; ja Q B 4) a a B   \u0026lt;2 .2 .2 3 55 S *S a co 2 -S V S o -o B f/i B u -g JB U B 4\u0026gt; Q ** O CA o MO3 4) 3 O' o Qi 4\u0026gt; E H CM \u0026gt;\u0026gt; a 3 X CM \u0026gt;% a 3 c/i \u0026gt;\u0026gt; a 3 V3 \u0026gt; a 3 CM \u0026gt;\u0026gt; a 3 a 3 a   -o a a C -3 N V3 \u0026gt; a 3 CM w s/1 a a P  = S CM O) 3 CM T CM 3 'x a 3 CM \u0026gt;\u0026gt; a 3 CM 3 4^ a O E 4^ a. \u0026lt; B R X3 4) a o X3 CM O Sd 4^ w a 4) 43 fl a a u .2 S \" o- r \"1 (J CM O s 4^ \u0026gt;   ^a u 4\u0026gt; a W V) 4\u0026gt; u O S.2 (/i (/i 4\u0026gt; 4\u0026gt; E o 2  S . CM 4\u0026gt; U 3 O CM 4) E 3 3 O 4\u0026gt; 4a c2 CM 4) U 3 O CM 4^ E 3 3 U o a CM u 4\u0026gt; a-3 2 a M E .2 a o   4- a V 'E 4\u0026gt; a 4\u0026gt; E 4\u0026gt; 3 4\u0026gt; CM 4\u0026gt; U U a o o o O \u0026gt; u o ja a 4\u0026gt; \u0026gt; Q 1/3 R BC .S B. V) 4) 3 V) *O .2  -22 u .2^4) 5 -fl a a a .S x) ..s \" SV) rt r fl 4\u0026gt; o 3 O CM 4^ CM Q u \u0026lt;w 4\u0026gt; 3 U o a CM X3 \u0026lt;u OX = s ** a c 2^ Q 4^ a   t S V 3 \"fl a a 3 E *s Q. 'U 43 \u0026gt; - a S a o V XI  2 fl a fl ua o Q Q .- (V \"S 0 J3 3 \u0026lt; -fl 2 ** a fl 4\u0026gt; H M t-C a a 0^ CM OX) g a fl OX) a \u0026gt; o (J a E 4^ E ja u \u0026lt;-i 4\u0026gt; a V3 4 X u a 4^ m 4) \"O a u 4^ A U a 4^ 3 a  c \u0026gt;4 B _ S? .r. fl     c  U 4) O -C E 3 3 Q  3 CM U Q 0 4\u0026gt; a \"O a \u0026gt;\u0026gt; 3 3 O S a a  .5 fl 5 c w a 4\u0026gt;  S fl F- ' B E *\nCM a a Q 3 U E 4\u0026gt; V) 3 OX) .a a 'fl 2 \"b H M L- a w B 4^ 4^ Zi E a  4\u0026gt; 4) \u0026gt;3 1 a U o X (J a 4\u0026gt; 4\u0026gt; 3 a o 4\u0026gt; 2 -fl O. ' ' E \u0026gt; J o fl .2 3 a 'W fj fl. X 4\u0026gt; E c o a ^2 *-w a a o E 4\u0026gt; \"S- E ,o OX) .2 *S *a CM CM a U 4a CM 1  S M - a 5 fl Q. XT .3 *^*3 3 OX) S u fl 5r fl f\" C \u0026gt; \"a do X w** E o o 't** S u fl \u0026gt; OX) Vi \u0026amp;( 4\u0026gt; O .S o  o y o OJD o eu CM O ja u J3 O \" U o 4\u0026gt; W)  a a. .3 o 7 \"fl a-  .S .2 3 3 .2 S o Oi CM 2 IX) (A V 3 Q\u0026gt; 3 a CM a  -3 ua a 4^ o ** c CM T3 I 5 - .\"E 5 ? E E s - R H E 5 Q a J B .s R 2  a u a * R \"     h 5 - \"o T a F -a ati S 4 \"O R a't\" o a 2 a u a B H mu ft. u a E a a o .2 *CM a 23 00 a W 4\u0026gt; ja 'O ^a -C a 4\u0026gt; 4\u0026gt; 0^ a o JS u ZT) 4) 3 3  QQ I 'O va V3 4\u0026gt; 0^ 3 a o a OX) 4^ \u0026lt; i/i B B a a 4^ 5 -a O w o a 4^ s ri u fl E CA V) 4^ \"O a -w Q a m U ja R u s 'Sb a a o OX) 4a fl 2 4a U -2  B C O a 4) CM o (A /iK -fl MO y CM a 4) -5 O a X3 CM U \"oi) fl 4) ua .2 \"El S 'fl fl \u0026gt;fl CM td y 4^ 3 au 2 \u0026lt; R lU-3  3 O'  4\u0026gt; E H \u0026gt; fl 3 3 fl BL (A 4^ Q o *a H a 4 s 4\u0026gt; E 3 3 no V u 3 \u0026gt; S  U .S U fl  E (A 4\u0026gt; (A M \u0026lt; r\u0026lt; O I O 'C fl 3 JH a .2  OX) fl Ca^ \u0026lt;Z) fl fA .o \u0026gt; fl T5  I \u0026gt; fl S OX S .S 3 CO \u0026gt; fl 3  -C .o u S O (A  E 3 3 tJ 3 Q Q E u - a  a a \"a 3 -3 fl fl CZ)  O a. (Z) fl 3 (M CA \u0026gt;\u0026gt; fl 3 n V) 4\u0026gt; fl 3 \u0026lt; fl Vi U 4^ ^3 i: 3 O *\nV3 U .o 3 OX) .E  L CZ} s  a -w n h-l u ox .5 3 62 ^B fl ! fl M S .2 u a  4^ -  J a B g \u0026gt; s u  lU \u0026gt; V)  Q fl \u0026gt;\u0026gt; fl 3 a tA w '5d a -2 gj 3 g M o a  (A 3 3 OX) cr a  B ja 3 3   -M \u0026gt;  B  *3 *- 4J 3 fl 4Z s v u a .5  t=\n-a ft- Q\u0026lt; v O \u0026gt; ua (Z3 4^ \u0026gt; 3 - fj a fl - .E ^25   \u0026gt; o fl 4\u0026gt; U fl o o ja u fl 3 43 a .2 u 43 I  fl (A  4\u0026gt; 3 .B a .2 OX fl Cs^  fl  \u0026gt; o o o -- - OX) Jl\u0026gt;  ki* \" ka  B OX) fl fl fl 3 fl 3 ( a 0 o a 3 V 43 3 J3 'a  CT u a -w   - \u0026gt;. . a M M t a u (/i u fl fl 'W Q fl cx^ o 4\u0026gt;  .2 u ex 5 .fl a iZ u **  fl 3 U 33 .\u0026lt;s e - a. -w .2  s \"IS w  flw43 M ja 4^ OX) \u0026lt;/3 j a .S 5s S s 0^ *2 -C 0) fl V) S S jMfc a -w J\" fl (N \\O 3 fl B  (/i O o 3 3 a  3 3 o o OX)  V)  3 u 4Z OX) u fl s T _  **  * u fl .2 3 M^XpauMM a *T fl f\" .S .S 'fl 4Z^ Q a B fl S 5 g a' fl V \u0026gt; -5! SJ -w fl  fl o Q.-a  'o ka^33fl^aZ H mU a\u0026lt; U K bcH o a o fl OJ o lA i: -fi 3. *\nE \u0026gt; .i o a r 3  \u0026lt;2 .5  V5  fl u  fl fl  M fl U O u \"O fl a a y \u0026lt; (/} fl  2 \u0026lt;Z3 OX) fl ft- C \u0026lt; u  -. A \"B ft\u0026lt; ^-  3 u -O eui \u0026lt; n, (Z)  o u fl 3 B 3  fl  u .fl fM 4Z i-f 43 \\o cz\u0026gt; 4.\u0026gt; fl o 0 fl 3 4\u0026gt; fl  k. fl U .2 Q W3 (Z9 .M 3 3 -w CZ3 .2 '3 o CZ5 (Z) .M -o (/i O 3 a  fl fl .2 U o V5 .fl (J fl  fl \"    -3 , o -3 s  -w u a g 3 W =  J  - -S 5 O \"   fl 3 3  w r O H W r4 u fl i/i u 8 s u (A  s 3 - H S ft-  CU \u0026lt; 3  - -2 B = -2 5 -So a wi g g \"  I   -   * fl cc  V3 3 fl X3 a 3 Aj c W s . .2 43 ft, a 2 a g  ^ S  S fl 4^ K  CO 3 O u E e9 \"O 4^ 3 O' V v E \u0026gt;\u0026gt; B \"B \u0026gt;\u0026gt; 5 *-   a E  Qi va ^S  3 T3 a \u0026gt;\u0026gt; B -3 \u0026gt;\u0026gt; a -o 9D fl O a  vi Q  Q  H  43 \u0026gt;\u0026gt; B 3 (za 3 CU  E  Ct in B 3 V \u0026lt;U B fl n V3 \u0026gt;\u0026gt; B \"O \u0026gt;* B - 3  43 s E 3 3   (J B 43 OX) s u .a 'o B a\u0026gt; 2 va  A  B  va U O -w B 3 fl   ya  \u0026gt; a a Q (Z5 V J3 43 'W B E .S OX) B E    CQ E 3 3  3 u va fl .2 (/i \u0026lt;X w '^   \u0026lt;za *3 w 3  2  (3 E o B U va  CQ  OX) B 3 OX) fl B fl .2f *  Cbi  va w fl  fl  Qi E o U a\u0026gt; u  B CU  CQ a M 52 OX) W Q I 2 1-5  g S r  J *- OJD 3  M {S \u0026gt;g  B 3 12  B \u0026gt; 2 o E o Ct B S \u0026amp;.S E o ,o 3  \u0026lt;U \"O  2S E O (za . 3  za * '^  B B 43  B cza cza  3 -g \u0026lt; (41 (za O  E  \u0026gt; o .2  B O Ui g J3 .2f B s 43 cK (A cza  \"fl B OX)  43 u B  \u0026lt; B E  -fl w B E  43 3 -fl     pfl  c/i 43 .Sf Ifl  B CU O fl  2 =3 B OX) B B E 4\u0026gt; .o   .C OX) ,a b  CQ \u0026gt;\u0026gt;  a  3 -O a ua 3 0^  c/:)  B cu l-J *0 in -w a V a o a. E o U (e w B B (za i^ fl  -o 3 \u0026amp;  *3 *B fl (za I. 3 B -S a-   B in Ji 2 4 \"O O \"O B \u0026gt; 3 u O .a i .B ja .f\u0026gt;  ffi S * B  (N o Cs B E  cza  43   ^ _ _ S -O  B -w  B ** o E va  3 (za u  43  B  -B 3 .2 va  B \" va -  a E -  o B a BLi o   'Sa cza 3 V9 Qi   ^ c 2 B a ya g \u0026lt; b (Z) 9 a\u0026gt; 43 .a  fl  'W B \u0026gt; ja ?  B  \u0026gt; OX) .2 va 3 E   (A va B \"  43 ** o \u0026gt; 5 \"O (2 A \u0026lt;u r JS a\u0026gt; M Cd e B Q  v i\u0026gt;   S H B S  o -  1-1 g in 3 B -S b  B B \"O 3 B S B .12 =  O \" P o Bi - o S- I- M za w A J3  U S 1_ rt 1 **  43 ox 4^ O B u  s Su u iO Vi  JS  u a\u0026gt; o o J3 o OX) J3 a\u0026gt; O u \u0026amp; o 3 Si .2 E S I-  H  u H  \u0026gt; tb va  fj fl o E  \u0026gt;  * S A\u0026gt; .B U a u  -a S B H OJ .2 \"B B O \u0026gt;  s CU M r\u0026lt;a O  o Ct .S \u0026amp;JD b- va va va B u va   fl  E  va B B  E   ~   s-s   ^ J Q  u = 1 cu fl o B cu \u0026gt; fl OX)  \"S B 43   H \u0026lt; \u0026lt; \u0026lt; \u0026lt; r U a u n i B  \u0026gt;-a  ) s 3 3 O'  Qj  E H a \u0026gt; a a xT* a V) 3  \u0026gt;\u0026gt;  4= J3 M S OX) O E S fl u u -*  a Q ft 3  \u0026gt;\u0026gt; a a a a CM a -a cn ja u a  \u0026gt; a a .a a  O' E CM \u0026gt;\u0026gt; a -a CM a ja a  a a -w M   M . fl   E   o a , *a a    Q a CM a Q a  a  E CM CM  t/3 (A a a a a CM V3 a OJD ^B a  ja .2 'O o - O) 69 \"o 2 a a \u0026lt; ^  ^ U -s ts Q w .a fl aa   2 S .-S 4\u0026gt; C *\u0026gt; u ua  S - OS 0  ft O H a ,2 *- a a w   riJ O X3 \u0026lt; a*   5 Q a 'ox B  o ts 69  a CM \u0026gt;M \"a M a a Qj a ' 'L 2 fl S s .-s -fl  o* a a CM a 12  t_ u ja 2 -fl  - *- \u0026lt;  5 S 2 (A a u v \u0026gt; \"S  .2 3 CS -a \" -3  a a a fl ri is V3 (5 S \u0026lt; V) a  \u0026gt; E a's v \u0026amp; ? UD V es a \"O a a a 4\u0026gt; a a \"Q  ri a E Vi 'W  M  ft a  a a G. a  \"O  00 \"O a a   \u0026gt; e   w a  a  a U w .a a -a cn \u0026amp; o A o CM \u0026gt; a a c\u0026lt;^   a  \u0026gt;\u0026gt; a U o .a Vi \u0026gt; a -a CM a   Sd m P 4 a *- S *5 M ft .S M 3 O o CM V3 a  S 2 .2 M.2 .S \u0026gt; fe a .2 ja  t a  a *- \"7 fl 3 CM ft 8 3 2 O n\" 9i := .-o 2 -a  -K a \"fl s ' R \u0026lt;  .s  V) Q \u0026lt; 2 H O .2 o (/3 r*  ft w O s U o ft  \"a ua B ** a a CM \u0026amp; s a S TS a o a 5  H Q,  o Ck a  2 ft 2 fl Q Ck a 3 \u0026lt; ft o C co a  a   ii 2  \" 3 a V eu  ft o o   t-C \u0026lt; w  G. I  ST dj w h \u0026gt; E* a a a a a Vi a Q  CM  a  (/i  a a ,2 *-w 3 a .S ox .S 's a VS a J a riiJ )S U r \u0026gt;  o  a OX ,a *V3 3 CM  a OX) ,a (/3 3 2 w ja   a a a. s I o a a  .2 3 t: *- a a v E u M a *a 4^ a VI u 2  a  fi H 4^ \u0026gt; (/3  M a 2 . |s 2 fl c  u w 4\u0026gt; 3 -3 aSri U S 3 4)  cm CM ** 2 \u0026lt; 2 -o Ori 3 W  CM  a  E  8 u 3 \u0026gt; a E a V 2 ft 5 E  a   fl ft u \u0026lt; u I  a \u0026lt; a a E S V E J3  a 5 S CM  X  a   04 \u0026lt;s a  a ac u  a Vi u  JS  a a 2 fM a  a 0  a u  A  a a  2 6S .a riri a. riri 4\u0026gt; a -a a a 3 a \u0026gt; a M u 5 -ft o 1 \u0026amp; E .S .Q C3 w .\nCA 0^  u *3 (M 3 u  2 U 3  us  *0X1  w a M a a ,2 *a  3 CZ) B V3  a ox .S *VJ 3 - .S-5 V) \u0026lt;Z5 J  a 43 U ox 2 fl a  8  3 \u0026gt; a *- a  ja 3 ri M 2 2 3 I ft o   o   - .* ft U H a a   E a    .3  U tn a  a ox  a u  X  a B a 2 a  t fl a-s a  o ua U  *-  CM M 3 E    L* J f 1 u  u  a a  9^ .2 a  B OX M B V5 *2  B V3 U 2 fl a  a   E -fl a *- a S 8  .2 S \u0026gt;\u0026gt; r-g^ ft a a CM  CM  o a  E  a o a \u0026lt; .1 V 3 \u0026gt; a \"W E a Lri 41 .o a. t: o S o 0- U .2 \"a 3 w .s ' a co \u0026lt; a (/)  .2 I- T) a a f ii __ \"O .2 CM  ja  a Vi  ja  a  .J  Bq*- 2 CM  a a O \u0026lt;S CM V) L-g a a E Q  E A  a 5 S V)    a   -a a o CM3 3 3 3 3 06 E H  3 a 3 \u0026gt; M Ba O o 3 \u0026gt; a 3 ro CA \u0026gt; a 3 \u0026lt;*) ua 3 a 3 \u0026gt;  a O 3 \u0026gt; a 3 n CA 3 a ,3 M Q A 3 Q 4\u0026gt; Q OX) ^a K 3 a 4a 3 a 3 \u0026gt;\u0026gt; a a .2 CA \u0026gt; a 3 f*) a. 3 a 3 \u0026gt;\u0026gt; a 3 4a 3 a 3 \u0026gt; a o .a a. o 4a 3 C9 3 \u0026gt; :? a a\u0026lt; O o \u0026gt; a 3 vs \u0026gt; a 3 vs \u0026gt; a 3 vs \u0026gt; a 3 \u0026lt;*) 4a M fia 'W 3 a 3 S.S 5 3 OXI at .2 .\"o o o S' Q. E .S \"o s .O V3 CA vs 3 .2 3 ua 3 'ei 3 M a a a o Q 3 (A 3 -a 3 CA 3 .a 3 A B. w at a -e a \u0026gt; a S Ig OX) at .s  m '\u0026gt; 2 a S' O. a\u0026gt; ja ,o CA 3 *0X1 3 -M a CA u vs u 43 a* O. at B \"O a .2 5 B 03 OX) at .E \"O \u0026gt; \u0026gt; a 5 o o ja .o \u0026amp; 3 a 2  a u o 3 S' o-  3 a * a La a E-a . B \"O , a o v OXI a 5 a ex a a ,O OX ,a *5 a 3 s y CA B vs 3 3 a vs La 2 a a 3 OX) ^a vs 3 a 9 A CA V3 3 3 Cm Cm vs 3 3 3 f/i 3 .2 0^ 03 B 4^ CA ., S '-I *5 \"S S  s o M5 U E o CA H '* ! M _ U B 4a ex 2  3 A at S at a \u0026gt; o ua a 'W E 2 .2 S S a a ,P Gg 2  3 3 a\u0026gt; -  C4 U H ^3 3 H a M a 3 3 3 E g at -B u u a CA t a O.-5 \u0026amp; 4^ U J3 u  *- 3 CA cM a E '2 3 3 3 B i: a S 43 E 3 ua CA 3 .a 3 a 3 3 3 a a 4\u0026gt; \u0026gt; rq B ti S ti 3 3 3 9 a 3  .2 a ts w A a 3 E M a CA 3 3 a CA La CA a a 3 a at .r. E s V 'a 43 va  = -s 2  S Q \u0026lt; Ib W a vs La bX 2 a a 3 U (A a M b B a-S a. 4\u0026gt; u ja *3 \u0026gt; a u a U g  .2 S \u0026gt;3 \u0026gt;22 K A a VS vs 4^ CA CA 4\u0026gt; a 3 E 3 o 3 B \u0026gt; a -w S a 2 a 2 A 3 3 3 U y a a 'S E a ox a U 3 *- 3 vs cM a^ B E 2 o 4\u0026gt; 4\u0026gt; B h g s 4a s Sc: M (m VS 4^ Q 4^ o a O B -a  A 5 B u 3 B \u0026gt; \"o a \u0026lt; 2 o at O Bt U S: 2 r al U X B a r2 t Bu U H J! o .2  \u0026gt;\u0026gt; K A a CA CA vs 4^ 2- 3 E E a 3 a CA a M eM .X 3 5 I 2 CA 3  3 a 4a CA La ej[ 2 B s a 3 - g at a \u0026gt; a w U M a M U CQ ej o xa ,a a 4) u u 43 U o ** 3 CA Cm O E.m 4) a td CA Cm CA - 4\u0026gt; u o o a a .2 cn o 2  \u0026lt; E u at 2 o- u a 5 o 04 U ii M w E 2 s i \" 2 2 5-u at o at CLt U H a a 4\n3 4a E si  o 3 2 \u0026gt;S-g V \u0026gt;1 K A vs vs 4.) 3 E 3 2 a \u0026lt; E f/i .2 3 3 M CO .2 *3 3 co \u0026lt; U S A E a   3 E 43 \u0026lt;u a 5 S a w a 3 3 9 Sa 3 3 M 3 ti CO vs ^3 3 3 w CO .2 *3 3 CO 3 \u0026gt; \u0026gt;1 a a .M W w a 3 U 2  U -] vs 2 3 05 v a V 3 a 4a _ 3 ea a 0 -S S O vs a 3 E E 3 .a a a 3 a (J 4\u0026gt; a E g 3 .M \"\" 3 ti CO .M 3 3 M .2 3 3 3 \u0026gt; u \u0026gt;s 3 a at ti ti 4\u0026gt; U a 4^ (M 3 3 a e\u0026gt; yi 4\u0026gt; A U a 4^ \u0026lt;s 3 3 a VS 3 ja u a 4^ es o *3 a o \u0026lt;s f/i 3 M 0) 3 4^ -3 a 4a __ fj a B o s n O vs u 3 ja 3 a 3 \u0026lt;*) 4\u0026gt; *3 a o \u0026lt;s 3 ja 3 a 3 4\u0026gt; 3 a CA 3 4a 3 a 3 M 9 3 a  r4 vs .2 fO 3 3 -3 a CA O -3 y\u0026gt; 3 ja 3 a 3 CA 3 ja a 3 o vs 3 3 a u n73 o 3 3 o o E H 45 O a o a .a 45 O a o \u0026gt;\u0026gt; a a. o T3 .Si Ct o \u0026gt;\u0026gt; a 3 M \u0026gt;\u0026gt; a *3 45 O a o \u0026gt; a 45 O a o JS V3 o .a a o a o M a 3 (A \u0026gt; a *3 \u0026lt;*) M \u0026gt;\u0026gt; a \"3 \u0026lt;S M o M O 45 3 a o M o Q ex ^a M 3 a o 45 3 ex) .a m 3 a E at .S *! V .fl M M a a 0^ u V a \u0026gt; a E a ta O 2 a o o o - U M L-s a a O E S o E 4a o a s s M o 45 O 45 O a o o 3 a O a o \u0026gt; \u0026lt;s .ex fl ex - fl M M ' -S.a ft O o (/3 \u0026gt; a 3 \u0026lt;*) a 3 \u0026gt;\u0026gt; a 3 fl E 0 .2 Em M M 3 o Vi a. a 3 ^5^\no M a 3 XN ca' ,fi \u0026gt; a 3 s 3 a I \u0026gt; a J3 'W ex w O fl 3 O I \"o .S g ex \u0026lt;u .E -a 2 o S- s. E .S  a n M M M O o *3 .M fc S -g t = ^3 5 fl 3 O M u a a M i: fl o 4a .o V3 *ex o -w a M M o M O 45 a o fl 3 a .E  a ex u .S -a 2 o Oi .S J - a n 3 L. .S 2 W .* 3 u \u0026gt; M  ,a  3 3 s- O. V 3 J3 S i: -5 a a o a o T s \u0026gt; =? a *'C E a ex a  \" 2 \u0026lt;2 g\u0026lt; o S 0) 3 u SU U H a w fl o o E g O .fl o U 05 Vi o o a o o -O a u a-n a o b O O J3 a U u o *- O M Oa 5? fl E Z a o fl o o o o ua E 5: O \u0026gt; 2 \"S' K a M 3 w 05 .s *0 3 (7) M o o u a v \u0026gt;\u0026gt; M o o E o 3 M M o rr -3 V 3 73 a 51 M o 4a o a o o -w O M rM ex) .S *M 3 ex .S M 3 \"O . 3 .0 a M M M o .9 a o a E at  .S *: F M ta 2 fl a o u V 3 \u0026gt; a 'W E a 2 a \u0026lt; S I\"  2 ft U O S o SU U O \u0026gt; M a a u a 3 9 9 o a o E E o .a (13 i_l o a ss 3 .r. 3 ex K 3 S 3 43 *3 .M Za 3 5 I 2 a V) ex 2 fl 3 a o - s  a \u0026gt; a 'W E a O M a M fl fl-S o 4a ex a a \" 2 2 Su ^23 V O e\u0026gt; SU U H a 3 O o o s g -S U co a o a *0 a MO.: M o 3 (/3 M V} w .M H .2 U .2 \"O  \"fl n o \"W -o a o 3 .-o Em M JT H 11  5 O 3 o o 'S o a o o ja U 0 *- O M Oa a^ fl E Z o o o fl h g S 2 \" E:: S! o fli ?\nB S fl 5 a w 3 a E\u0026gt;  M o E o a -  3 E s\nM a J _ fl ex S o o ** .2 u 3 M fi. 0 M 4^ o 45 \"ex fl tE] o 45 ex) a \" ft a \u0026amp; o o y 0 .E o S -ft s O ~ U ^ = 5 Li rtJ M o O *0 o ex M a a a c o VO VO - .2 3 S \u0026gt; 'S \"O K a 3 M Is - S 5 ? E s a -3 3 H S 2 3 a a ex .S .E o M .Sa \"3 3 O) .S o o CZ5 o o 45 o O) o 3 3  ox .E 3 o -C o\u0026lt; fl C 0! fl o E M M O M M o ex st^ s I E -S a 2 ! -S M O ja o a o M o 45 o a o M O 45 O a o Vi O m \"3 O 3 o 73 a 73 a o 3 a -o a M 51 M o .a o a o 00 I VO M (/3 O *- M u O o 3 45 a o ta a ex o ja 'o y M a o -3 M 1-H afl I o VO a 73 a 4a M o o  g-H 45 I O VO a o o o M \u0026lt; O w a \u0026lt;s fS rs \"O 4a fl .2 rq O o ex a 3 45 fl .2\"O 4\u0026gt; 3  06 4\u0026gt; E H 69 -o o ja ,o V3 O U 3 O (A o \u0026gt;\u0026gt; a 3 -5 -o \u0026gt;\u0026gt; 69 o O ua (A o 3 O (A 4) fl .2 *'W a E 3 3 U w i/) 4^ Q 3 U u  *E a a 2 a  -s ja e9 S ir a Ci.  V3 ja S? a c9 -fl N \u0026gt;\u0026gt; 69 -o 4a w fl. 4) fl \"fl 3 .S 3 3 a 0 u .s  2 o \u0026gt;\u0026gt; fl fl \u0026gt; fl o CO Qi CL\u0026lt; H fl o \u0026gt;\u0026gt; 69 \"fl OXI s 4\u0026gt; 4a *s a 1 g c fl 'fl E 3 3 U s u U  tS e\u0026gt;j 2 W o fi o 2 o  Sb5 o 2 = ^ 0 o o Maa S  'a _ o u u *3 fit S T a -7  IM  =\u0026gt; a n a a u 5 o a 4\u0026gt; -w o 4a F* mU S .w *5. o E-I V3 2 c? a a g \"  a g  co U 2* S. 2t. , a .o a fl .2 u 3 u s fl a \u0026gt; 2 a E S Ci. 4? U \u0026amp; .S u Im VI a\u0026gt; \"\u0026gt; - .2 W eM _ S -g t 3^ I 2 U M u B .,. a  T- 69 . i: a a-s Ja 2  a 2 *3 * 0 2 .5 5  *fl 5 O 4) U ** \u0026gt; Si 9^ *- a .E T a = a i  XI a  5 a j_ ? a a\u0026gt; -w O 0^ H mU H fl T3 fl O 4) B u .2 o u fl OJD B Q. - u J3 u o *- O V3 .M 3^ a E.M B o o a i: 0^ ja s S!  .2 I u  CB a fl \u0026lt;fl 4\u0026gt; Qtf \"fl fl fl OXI fl  \u0026gt;\u0026gt; .2 (va s \u0026gt; 4\u0026gt; C^ tf3 fl o U) fl 3 u fl fl fl .2? 'S o Exm OXI ^fl *fl 4^ Qii o 4a I -w a h SO) ja V3 a _ _ \"2  69 -w 4) U fl *- u U E (S 4\u0026gt; i\"( 4a SO (A O Q fl Q \"fl 4\u0026gt; 69 U fl U .2 U 69 4\u0026gt; \u0026gt; (A \u0026gt;\u0026gt; 69 T3 (S o 0) o S 73 .f 2 a 'C CLi W5 a -\u0026lt;j fl V) 4a tS .2 a oe Q .fl hi*l WAI ii o a 2  M 5 r S o O 4\u0026gt; O M  ** .S E3 S? \"3 \u0026gt; fl \"O Q u Q fl SLi 3 O ja 'C M .2 -S 3 w CO .S 'o o co fl .2 *-w Q 3 a .2 *-w U 3 \u0026lt;s .2  \"fl so c/s 3  ?? ts - \"a a U o wi \u0026lt;S B fl fl .2 ja u 69 4^ u **  J 69 U V) a\u0026gt; lu a 2 2 S U S o fl *a - o U o iM V ' a M 2 'a M a 5 *a u s\nM\"g 2  S a a r^ B O) \u0026gt; * J * ?? S a_  \u0026lt;S UI 4\u0026gt; CQ \u0026gt; u fl o E 4) E o o o -a * u V3 0) ja E o\u0026lt;.E s o fl a\u0026gt; o -O o  S u V) O 3 o  fl fl M iQ 4\u0026gt; 4\u0026gt; w r o Vi E Y \u0026lt;.1 o \"O o  -o \u0026lt; iZi a M a i\nu o A M 5 cK M 4^ V o ifl a y U) \u0026lt; r\u0026lt; a o fj B o B o  \"O B O B B 69  a. 4\u0026gt; 69 CL. Q a\u0026gt; W E w 3 S a E -a o H n R cu u a\u0026gt; \"O 5 o u fl fl (A o Q. 69 a S  S 4\u0026gt; fl O 69 B o 4) o tf) fl 4) fl O o. E o 4\u0026gt; 4a\n V) 69 73 4\u0026gt; 4a B .fl  0^ 'sO a )2 a S a cr 3 B s E  fl -a M Si Clm \u0026lt; a V \u0026lt; 69 X o OXI fl  M V3 \u0026gt; 4\u0026gt; Di o a cu ox ^H *3 9.\u0026gt; U fl s fl E fl 4\u0026gt; \u0026gt; Si a ,2 \u0026lt; eu -o 4) V3 Q S O u Su a 2 1^ S \u0026lt; u 69 S 4\u0026gt; ja u 69 4\u0026gt; OS C4 u fl V CQ \u0026lt;s 4\u0026gt; 7 J3 6 y V5 o T3 69 O 4a a :2 ox o \"o fl .fl o 4) \u0026lt; r h U a a r\u0026lt; \u0026lt;*)o  3 3  Qj  E H fl 3 CA fl 3 n (/S fl 3 (/i fl 3 S.N CA \u0026gt;\u0026gt; fl 3 \u0026gt;\u0026gt; fl -o CA \u0026gt; fl 3 fS CA \u0026gt;\u0026gt; fl 3 xT* fl 3 fl *3 fl 3 rs \u0026gt; fl 3 xT* fl 3 SN \u0026gt;% fl 3  x: .O CA  '3d  fl fl . tf3  3 3 fl fl  .2 a   Q .fl   0\u0026lt; .a IE fl  \u0026lt; h-] I   3 CA fl Z.\u0026lt;  43 a. M  B 3  I * .s a 2 ts = 00 o .S -o o P  a- Q, S X) a  . .2 3d  M fl CA 43 M \u0026amp;  a 3  I -o fl I A 00 o .S -s  43  3d  M fl 43 M \u0026amp; M  B 3  I .E Ss   *  I V \u0026gt; o .fl 01 .5 .'s fl J a U % EaJ Cm * k o 4^ o S fl   .2 a. 3 I   tbi B t\n** fl fl E  43 fl E .fl .fl ^fl fl ,o fl fl o M  3 V) Ii S 5, E CA CA  CA CA  u fl  E 2 \u0026lt; e fl  fl  u . *\u0026amp; Q H 0\u0026lt; .B la fl H I  U \u0026lt; I  \u0026lt; V3  43 \u0026lt;  43  fl  5^ \nu fl  W fl 00  o  X  fl fl S 2 fl  \u0026gt; f*) t/i fl fl E a Z7 Q  E ja  fl 5S fl  CA fl fli W\u0026gt; d  \"3  fl a S 2  B .2  (/i .S 00 .B ^E fl fl  fl  U t* fl M a     .M \"*  w fl  V) fl 00  \"O  Ji  fl a S 2 00 CJ a \"S ri fl fl  2- 5- E fl fl fl  \"o   3 a C9 . fl E  JH fl E .E 00 .E 'e fl fl E  xa M fl E fl 00 .a '5 w CA o CA .2 2 9 73 * u s  Vi fl  t a a.-s  fl fii  u J3 u  *- O CA CM fl a E 3 fl  a    xa a S: gj O fli T. fl J a % U   o fl fl a CA  CA CA M a\u0026gt;    fl .  .a 00 .2 ^E fl ) .2 2 S \" M CA  fl  fl t  cu-r fl.  U JS u o *- O V3 Cm W a  S\u0026gt;M 4)  .2 5  T3 CC fl 3 M  fl  E 2 a \u0026lt; s fl  w fl O U fl  w fl  u a h a  43 E V3 a J a (/i CA ( y  o fl fl  .5  -a tl M CQ   fl .2 * .s 00 .fl\ne fl M   .2 . \u0026gt;o 3 V3 u 9 fl i:  a  u ja u o ** O CA Cm a a E r a    00 o .2 5 M 03 .2  o 03 w CA B '    CA ox 5   izi =  43  fl  fl .2  fl  w  o fl b-c a a E E A  fl w S fl 'M fl    E g  .2  M cn  .fl  fl   .fl  fl   -o fl  -o fl S\no .2  i* K fl s  CA  fl  E 2 a \u0026lt; S fl  M fl  u fl  M fl  u a  .a  -H a:: S!  .2 S \u0026gt; \"S = A fl Vi M .2 -3 3 M 03 .2  03 \u0026lt; fl fl E a ZT  E ja  fl s s fl M fl    E fl \"   .M  1^ 03 .2 3 3 M 05 2 * o 03 V) .2 3 M  s  \"2 c -fi * fl  O.\n\u0026lt;*) fl  03  43  fl   43  fl  u  43  fl  V3 \u0026lt;M 3   s   \" fl  -o fl  o fl \"S -E fl  fl o  fl   cn  -w-o 4.\u0026gt; 3 C 4\u0026gt; 4\u0026gt; E H vs \u0026gt;\u0026gt; fl 3 CM CA fl 3 (A fl *0 xT* \u0026gt;\u0026gt; fl 3 \u0026gt; fl \"3 fS VS fl 3 vs \u0026gt; fl xT* \u0026gt; fl 3 vs \u0026gt; fl TS CM CA \u0026gt;\u0026gt; fl -3 XM fl T3 \u0026gt; fl -o 4^ X] P  53d  w fl CA fl ^2 5 a fl U .2 JS \u0026amp; -S *? 4\u0026gt; (A 4\u0026gt; Q 3 u ** \u0026gt; VS ,C u a U CA Cm V) - Q U 4) o fl a .2 K a 5 a fl E 4^ ja fl E .S on .S ^8 fl 4^ U a .2 3 vs .s 00 ,3 'a fl f/) vs 4\u0026gt; 4\u0026gt; E 4\u0026gt; 2 a \u0026lt; s 3 4\u0026gt; 8 O U fl 4^ fl a U ^ fl ^3 4\u0026gt; \u0026gt; u fl  u 3 U f/i 3 X3 4\u0026gt; 43 Q fl 4\u0026gt; \u0026lt;*) *3 fl o vs a a 8 4\u0026gt; E E 4\u0026gt; JS 0^ fl vs V J3 U fl 4) rn 4\u0026gt; 3 fl o t* fl fl 4i\u0026gt; a\u0026gt; u E ' \"   .. fl 4^ \" u U CA 4^ 43 u fl 4^ 4\u0026gt; \"3 fl  fl \u0026gt; fn *) 43 a -w  3 *3  I \"o S I an 4) .E XJ 2 o E .5 \"O a .0 03 A VS  .2 a\u0026gt; \u0026lt;B -w .y w s  vs fl ..IM Q.-S fl 4^ 43 53d 4\u0026gt; fl fl \u0026gt;5 a u .2 a a 3 *3 \u0026gt; CA  fl  M a o u ,4J u A a W VS 4\u0026gt; U  *- 0 VS  9 o E.m q Q fl 4)  43 E Si  fl_i ?. .Si 5 \u0026gt;5 \" \"a OS a vs 3 w (/:i .2 *s o co (A V3 .2 m 3 2 s O CO fl ^  U-3 3 CO vs Cm (/i 4\u0026gt; U O S.2 vs vs 0^ 4\u0026gt; E u V fl 43 w a  8 -3 o .S 2 a CA flXl oj .E -a 2 o 03 a Stj fl 4\u0026gt; .0 Vi .2 53d 4^ fl 2 a a u 43 -w a 3  I o a 3 = CQ at 4, .S -s 2 o \u0026amp; fl E 4\u0026gt; 43 w fl E 8 .0 03 VS V \u0026amp; ,fl  *fl 2 a \u0026lt; s fl 4) w fl O U \u0026gt; \u0026gt;) U vs fl fl t 2   4) u _) (A 4^ 43 u fl 4^ Tf 4^ \"3 fl u *) g S 2 ja O fl w S VS O 43 U fl 4\u0026gt; w 3 fl e\u0026gt; 4^ U a 3 M 3 \u0026amp; ^3 *3 'fl fl V fl o U fl fl 4^ V (J E a    .3 M Q U co VS 4^ 43 U fl 4^ 4\u0026gt; 3 fl  .2 .Si T3 y s W :S Z .2 5 W 3 2 W \u0026gt; *2 u E fl  gr  ^fl a L* u .4^ U X! a U u  *-  vs Cm o   - E.m \u0026lt;M  H i: B S ua 4) '= E vs Cm rx CA O fl o I J  a 5 fl E 4^ 43 w fl E ,3 OD ^8 's fl 8 .2 3 ,8 W\u0026gt; ,3 fl CS -3 3 fl  - VS  .2  V \"B a .y o U fl 3 fl O--3 a U X3 u o *- O vs tM o a  S*\"*   s h Cs o   \u0026gt;1 \u0026gt; 2 \"2 K a .2 \"3 8 S *3 o vs .2 4\u0026gt; 3 VS CO -S fl _  fl w-S fl 4.) Q ** O 05 vs vi 4^ E 4^ 2 a S fl o fl Q u s 4\u0026gt; s o U 4\u0026gt; \u0026gt; U *-* fl  U ,4\u0026gt;  W vs 4^ 43 Q 03 4\u0026gt; 4^ 3 fl g fl s c: o flj tt .2 5 I'-sl 05 fl CA CA fl fl E g o E ja a\u0026gt; fl u S 4^ 43 u fl 4\u0026gt; 4^ *3 fl \u0026lt;*\u0026gt; t* fl w 8    9  B   .3 U vs .2 \"S s co .2 *3 3 co 4\u0026gt; 43 U fl 4^ IO o 3 fl o CA .2 IO *3 0) 3 a\u0026gt; fl 12 fl  -2 o a CZ3 \u0026lt;*) fl 4\u0026gt;3 fl 'a c w u E P \u0026gt;\u0026gt; 3 3 Q t* Cf) 3 Oi .E fl fl \"fl a. CA fl 3 CA V3 o .2 a 2 sS  M .2 o 03 V T3 ca o 3 fl a fl fl Q a .3 -a  .M a E -S   E  .M H o 3 a '5 o \"a 4\u0026gt; - r fl fl fl fl E fl  5 B -w Qm o  E  cza 3 fl E fl 3 3 43 w 3 3 CA fl fl fi. w) \"o S Pm O  - 3 OD U fl ,3 S \u0026amp; fl 3  ^  fl va fl fl fl 43 (J va u V v W \u0026lt;*5 3 S 2:S -3 I 2 a T3 3 = H E 4\u0026gt; O a. (O fl O *\u0026gt; fl fl *a 3 o o J3 u c aj O i fl fl fl 3 f/i fl 3 3 QC  (za fl cza fl \"3 3 3 fl u es bX) fl \u0026lt; 3 fl\nza fl ,O '5 va fl va V3 fl -a i u 'fl cza 3  u V -w 3 3 43 (/3 cza 3 \u0026gt;\u0026gt; 3 a  a \u0026gt; 3 3 3 o I (S \u0026gt; 3 fl fl .= \u0026gt;\u0026gt; 3 3 r4 u ex 2-5 S I cc i: va L* a a 2 g \"  5 a S V3 k. 2 T-M \u0026gt;3 I u 'O a U \"u a U e*) (/3 tJ (/a fl fl 3 K fl fl *3 fl ' \" . s g tm fl .E  s: \u0026gt; ? P \u0026amp; a. .2^ r a .2 s o a a  PM fl  J3 ,o fl V) o M .2 o rtJ  v3 .H  ! ex a a B S a 5 ** W fl .E fl 3 1 3 fl U H S' 3 3 3 O fl fl fl fl fl .fl fl (J fl cza 3 M ft-r fl 3 .3 OX) 5 fl 3 tM o fl fl S B O W 2 2 o 43 \u0026lt; 3 S a 11  - fl CA t\n4* B \u0026gt; E  S 3\u0026lt; 43 fl Q P, o \u0026amp; MW t. 3 fl fl fl ** o fl \u0026gt;J 08 fl fl ti 3 o fl fl  a.E s ja u o . (m X3 S. fl u ja U o *- O va M es a E 3 fl fl fl g 5 -B E c:  \"O flj 0.) a iS  V CA fl 3 va 3 CA -W *2 -* 3 fl s?  .2  \u0026gt; \u0026gt;58 - a ja OS 3 w A V v t: a V3 B S 4\u0026gt; 4\u0026gt; O \u0026gt; o fl fl o .2 fl -fl 4) 4* \u0026lt; (Z3 a M s -M Vi .s fl o c Vi O 4^ -a b u fl 43 .f  s 43 S \"S B O .S \u0026amp; fl M  I \u0026gt; a 11\" Q E v\u0026gt; = r: 3 \u0026lt; 5  . 9 fl a. -w   a i   -w 2  B a \u0026gt; 1 g .a o) s s   ! E A 3 \" \u0026lt;R1 va CS a = -a V O o S 3 3 Q. a 4) 3 a a E a V \u0026gt; 2 \"8 3 \u0026lt; 0 fl 3 fl fl \u0026gt; fl -3 fl fl (/) .E T \"3 CM fl 3 S3 fl 3 /S 3 fl 3 fl U 3 U .SJ fl va fl V3 fl 3 M fl u Vi  -B .2 4^ 4 3 -B a 3 -w fl - fl *- \\fl 69 a .a B .SS E5 u 6*^ E O E \u0026lt;*) \u0026lt;*) \\fl 3 (za fl fl a  a .2 -g 3 fl fl 3 3 CJ 3 CZ) fl ** fl 43 fl 3 fl B e \u0026lt; (M fl fl 3 fl 43 fj a fl f*) \u0026lt; \u0026lt;*\u0026gt; fl\\Friday Eldredge \u0026amp; Clark HERSCHEL H. FRIDAY \u0026lt;1922-1994) WILLIAM H. SUTTON, P.A. BYRON M. EISEMAN, JR., P.A. JOE D. BELL, P.A. JAMES A. BUTTRY, P.A. FREDERICK S. URSERY, P.A. OSCAR E. DAVIS, JR.. P.A. JAMES C. CLARK. JR.. P.A. THOMAS P. LEGGETT. P.A. JOHN DEWEY WATSON. P.A. PAUL B. BENHAM 111, P.A. LARRY W. BURKS, P.A. A. WYCKLIFF NISBET, JR., P A. JAMES EDWARD HARRIS, P.A. J. PHILLIP MALCOM. P.A. JAMES M. SIMPSON, P.A. JAMES M. SAXTON. P.A. J. SHEPHERD RUSSELL III, P.A. DONALD H. BACON. P.A. WILLIAM THOMAS BAXTER, P.A. JOSEPH B. HURST. JR., P.A. ELIZABETH ROBBEN MURRAY, P.A. CHRISTOPHER HELLER. P.A. LAURA HENSLEY SMITH. P.A. ROBERT S. SHAFER. P.A. WILLIAM M. GRIFFIN HI. P.A. MICHAEL S. MOORE. P.A. DIANES. MACKEY. P.A. WALTER M. EBEL Hi. P.A. KEVIN A. CRASS, P.A. WILLIAM A. WADDELL, JR., P.A. SCOTT J. LANCASTER. P.A. ROBERT B. BEACH, JR., P.A. J. LEE BROWN, P.A. JAMES C. BAKER, JR., P.A. HARRY A. LIGHT, P.A. SCOTT H. TUCKER, P.A. GUY ALTON WADE, P.A. PRICE C. GARDNER, P.A. TONIA P. JONES, P.A. DAVID D. WILSON, P.A. JEFFREY H. MOORE, P.A DAVID M. GRAF, P.A. CARLA GUNNELS SPAINHOUR, P.A. ATTORNEYS AT LAW A LIMITED LIABILITY PARTNERSHIP www.fridayfirm.com 2000 REGIONS CENTER 400 WEST CAPITOL LITTLE ROCK. ARKANSAS 72201-3493 TELEPHONE 501-376-2011 FAX 501-376-2147 3425 NORTH FUTRALL DRIVE. SUITE 103 FAYETTEVILLE. ARKANSAS 72703-4811 TELEPHONE 479-695-2011 FAX 479-695-2147 JOHN C. FENDLEY, JR., P.A. JONANN ELIZABETH CONIGLIO, P.A. R. CHRISTOPHER LAWSON, P.A. FRAN C. HICKMAN. P.A. BETTY J. DEMORY, P.A. LYNDA M. JOHNSON, P.A. JAMES W. SMITH, P.A. CLIFFORD W. PLUNKETT. P.A. DANIEL L. HERRINGTON, P.A. MARVIN L. CHILDERS K. COLEMAN WESTBROOK. JR., P.A. ALLISON J. CORNWELL ELLEN M. OWENS. P.A. JASON B. HENDREN BRUCE B. TIDWELL MICHAEL E. KARNEY KELLY MURPHY MCQUEEN JOSEPH P. MCKAY ALEXANDRA A. IFRAH JAY T. TAYLOR MARTIN A. KASTEN BRYAN W. DUKE JOSEPH G. NICHOLS ROBERT T. SMITH RYAN A. BOWMAN TIMOTHY C. EZELL T. MICHELLE ATOR KAREN S. HALBERT SARAH M. COTTON PHILIP B. MONTGOMERY KRISTEN S. ROWLANDS ALAN G. BRYAN LINDSEY MITCHAM KHAYYAM M. EDDINGS JOHN F. PEISERICH AMANDA CAPPS ROSE BRANDON J. HARRISON OF COUNSEL B.S. CLARK WILLIAM L. TERRY WILLIAM L. PATTON. JR. H.T. LARZELERE. P.A. JOHN C. ECHOLS. P.A. 208 NORTH FIFTH STREET BLYTHEVILLE. ARKANSAS 72315 TELEPHONE 870-762-2898 FAX 870-762-2918 March 4, 2003 JOHN C. FENDLEY, JR. LITTLE ROCK TEL 501**70-3323 FAX 501-244-5341 fendleyQfeC.nt Ms. Joy Springer John W. Walker, P.A. 1723 Broadway Little Rock, Arkansas 72201 RECEIVED MAR - 5 2003 OFFICE OF DESEGREGATION MONITORING RE: Professional Teacher Appraisal System Dear Ms. Springer: Enclosed please find a May 2002 draft of the Little Rock School Districts Professional Teacher Appraisal System. This is the background information on the new teacher appraisal system which I agreed to provide to you during our facilitation with ODM on February 28,2003. Please do not hesitate to call if you have any questions. Sincerely, John C. rendley, Jr. f JCF/bgb cc: IxNJs. Ann Marshall (w/enclosure(s)) Dr. Ken James (w/o enclosure(s)) F:\\HOME\\BBrown\\Pendlcy\\l-RSD\\deseg\\springer2 h wpd\nThis project was supported in part by a Digitizing Hidden Special Collections and Archives project grant from The Andrew W. Mellon Foundation and Council on Library and Information Resoources.\n   \n\n   \n\n  \n\n\n   \n\n  \n\n \n\n\n   \n\n   \n\n \n\n\n   \n\n  \n\n \n\n\u003cdcterms_creator\u003eLittle Rock School District\u003c/dcterms_creator\u003e\n   \n\n \n\n  \n\n\n   \n\n \n\n  \n\n\n\n   \n\n  \n\n  \n\n\n   \n\n   \n\n  \n\n \n\n \n\n\n   \n\n  \n\n \n\n\n\n\n\n\n\n\n\n   \n\n \n\n\n\n  \n\n\n   \n\n\n\n  \n\n\n\n "},{"id":"tmll_hpcrc_50990069","title":"A new paradigm for welfare reform [electronic resource] : the need for civil rights enforcement","collection_id":"tmll_hpcrc","collection_title":"Historical Publications of the United States Commission on Civil Rights","dcterms_contributor":null,"dcterms_spatial":["United States, 39.76, -98.5"],"dcterms_creator":null,"dc_date":["2002"],"dcterms_description":["A digital version of the report published by the United States Commission on Civil Rights.","The Civil Rights Digital Library received support from a National Leadership Grant for Libraries awarded to the University of Georgia by the Institute of Museum and Library Services for the aggregation and enhancement of partner metadata."],"dc_format":["application/pdf"],"dcterms_identifier":null,"dcterms_language":null,"dcterms_publisher":null,"dc_relation":["Forms part of online collection: Historical Publications of the United States Commission on Civil Rights.","Requires Acrobat plug-in to view files."],"dc_right":["http://rightsstatements.org/vocab/NoC-US/1.0/"],"dcterms_is_part_of":null,"dcterms_subject":["Public welfare--United States","Welfare recipients--Civil rights--United States","United States. Personal Responsibility and Work Opportunity Reconciliation Act of 1996"],"dcterms_title":["A new paradigm for welfare reform [electronic resource] : the need for civil rights enforcement"],"dcterms_type":["Text"],"dcterms_provenance":["Thurgood Marshall Law Library"],"edm_is_shown_by":null,"edm_is_shown_at":["http://purl.access.gpo.gov/GPO/LPS23972"],"dcterms_temporal":null,"dcterms_rights_holder":null,"dcterms_bibliographic_citation":null,"dlg_local_right":null,"dcterms_medium":["reports","records"],"dcterms_extent":null,"dlg_subject_personal":null,"dcterms_subject_fast":null,"fulltext":null},{"id":"pbs_pov","title":"Point of view","collection_id":null,"collection_title":null,"dcterms_contributor":["American Documentary, Inc.","Point of view (Television program)","PBS Online"],"dcterms_spatial":["United States, 39.76, -98.5"],"dcterms_creator":null,"dc_date":["2002"],"dcterms_description":["Web site companion to the Public Broadcasting Service program Point of View. With background information about programs, a schedule of showings, and other supporting resources for the show. Civil rights content includes a program on civil rights leader Bayard Rustin.","The Civil Rights Digital Library received support from a National Leadership Grant for Libraries awarded to the University of Georgia by the Institute of Museum and Library Services for the aggregation and enhancement of partner metadata."],"dc_format":null,"dcterms_identifier":null,"dcterms_language":null,"dcterms_publisher":null,"dc_relation":null,"dc_right":["http://rightsstatements.org/vocab/InC/1.0/"],"dcterms_is_part_of":null,"dcterms_subject":["Documentary films--United States","African American civil rights workers","Civil rights workers--United States","African American gay men","Gay men--United States","Social history"],"dcterms_title":["Point of view"],"dcterms_type":["Text"],"dcterms_provenance":["Public Broadcasting Service (U.S.)"],"edm_is_shown_by":null,"edm_is_shown_at":["http://www.pbs.org/pov/"],"dcterms_temporal":null,"dcterms_rights_holder":null,"dcterms_bibliographic_citation":null,"dlg_local_right":null,"dcterms_medium":["instructional materials","texts (document genres)","documentaries and factual works","study guides"],"dcterms_extent":null,"dlg_subject_personal":["Rustin, Bayard, 1912-1987"],"dcterms_subject_fast":null,"fulltext":null},{"id":"tmll_hpcrc_50543430","title":"Racism's frontier : the untold story of discrimination and division in Alaska","collection_id":"tmll_hpcrc","collection_title":"Historical Publications of the United States Commission on Civil Rights","dcterms_contributor":["United States Commission on Civil Rights"],"dcterms_spatial":["United States, Alaska, 64.00028, -150.00028"],"dcterms_creator":["United States Commission on Civil Rights. Alaska Advisory Committee"],"dc_date":["2002"],"dcterms_description":["A digital version of the report published by the United States Commission on Civil Rights.","The Civil Rights Digital Library received support from a National Leadership Grant for Libraries awarded to the University of Georgia by the Institute of Museum and Library Services for the aggregation and enhancement of partner metadata."],"dc_format":["application/pdf"],"dcterms_identifier":null,"dcterms_language":null,"dcterms_publisher":["Los Angeles (300 North Los Angeles Street, Los Angeles, 90012) : U.S. Commission on Civil Rights, Western Regional Office"],"dc_relation":["Forms part of online collection: Historical Publications of the United States Commission on Civil Rights.","Requires Acrobat plug-in to view files."],"dc_right":["http://rightsstatements.org/vocab/NoC-US/1.0/"],"dcterms_is_part_of":["Historical Publications of the United States Commission on Civil Rights Collection (Thurgood Marshall Law Library)"],"dcterms_subject":["Indians of North America--Alaska--Civil rights","Race discrimination--Alaska","Indians of North America--Alaska--Government relations","Alaska--Race relations"],"dcterms_title":["Racism's frontier : the untold story of discrimination and division in Alaska"],"dcterms_type":["Text"],"dcterms_provenance":["Thurgood Marshall Law Library"],"edm_is_shown_by":["http://www2.law.umaryland.edu/marshall/usccr/documents/racisimsfrontier.pdf"],"edm_is_shown_at":["http://crdl.usg.edu/id:tmll_hpcrc_50543430"],"dcterms_temporal":null,"dcterms_rights_holder":null,"dcterms_bibliographic_citation":null,"dlg_local_right":null,"dcterms_medium":["reports","records"],"dcterms_extent":["x, 58 p. ; 28 cm."],"dlg_subject_personal":null,"dcterms_subject_fast":null,"fulltext":null},{"id":"tehc_tncr","title":"Tennessee encyclopedia of history and culture","collection_id":null,"collection_title":null,"dcterms_contributor":["Tennessee Historical Society","University of Tennessee Press"],"dcterms_spatial":["United States, Tennessee, 35.75035, -86.25027"],"dcterms_creator":null,"dc_date":["2002"],"dcterms_description":["Online encyclopedia with information about the Civil Rights movement in Tennessee, including articles about the history of the movement, important individuals in the civil rights struggle, and important events. Some articles have links to associated images.","The Civil Rights Digital Library received support from a National Leadership Grant for Libraries awarded to the University of Georgia by the Institute of Museum and Library Services for the aggregation and enhancement of partner metadata."],"dc_format":null,"dcterms_identifier":null,"dcterms_language":["eng"],"dcterms_publisher":null,"dc_relation":null,"dc_right":["http://rightsstatements.org/vocab/InC/1.0/"],"dcterms_is_part_of":null,"dcterms_subject":["Civil rights movements--Tennessee","African Americans--Civil rights--Tennessee","Tennessee--Race relations--History--20th century","African American civil rights workers--Tennessee","Civil rights workers--Tennessee","Discrimination--Tennessee","Segregation in education--Tennessee","Segregation--Tennessee","Highlander Folk School (Monteagle, Tenn.)","Highlander Research and Education Center (Knoxville, Tenn.)","National Association for the Advancement of Colored People","Fisk University","Civil rights demonstrations--Tennessee","National Civil Rights Museum"],"dcterms_title":["Tennessee encyclopedia of history and culture"],"dcterms_type":["Text"],"dcterms_provenance":["Tennessee Encyclopedia of History and Culture"],"edm_is_shown_by":null,"edm_is_shown_at":["http://tennesseeencyclopedia.net/"],"dcterms_temporal":null,"dcterms_rights_holder":["Online Edition copyright ?2002 The University of Tennessee Press, Knoxville, Tennessee. All Rights Reserved."],"dcterms_bibliographic_citation":null,"dlg_local_right":["For permission to use passages or images in the on-line edition, please contact the Tennessee Historical Society."],"dcterms_medium":["encyclopedias","articles","photographs","portraits"],"dcterms_extent":null,"dlg_subject_personal":["King, Martin Luther, Jr., 1929-1968","Du Bois, W. E. B. (William Edward Burghardt), 1868-1963","Horton, Myles, 1905-1990","Horton, Zilphia, 1910-1956","Lawson, James M., 1928-","Nash, Diane, 1938-","Wells-Barnett, Ida B., 1862-1931"],"dcterms_subject_fast":null,"fulltext":null},{"id":"bcas_bcmss0837_730","title":"Unitary status","collection_id":"bcas_bcmss0837","collection_title":"Office of Desegregation Management","dcterms_contributor":null,"dcterms_spatial":["United States, 39.76, -98.5","United States, Arkansas, 34.75037, -92.50044","United States, Arkansas, Pulaski County, 34.76993, -92.3118","United States, Arkansas, Pulaski County, Little Rock, 34.74648, -92.28959"],"dcterms_creator":null,"dc_date":["2002/2007"],"dcterms_description":null,"dc_format":["application/pdf"],"dcterms_identifier":null,"dcterms_language":["eng"],"dcterms_publisher":["Little Rock, Ark. : Butler Center for Arkansas Studies. Central Arkansas Library System."],"dc_relation":null,"dc_right":["http://rightsstatements.org/vocab/InC-EDU/1.0/"],"dcterms_is_part_of":["Office of Desegregation Monitoring records (BC.MSS.08.37)","History of Segregation and Integration of Arkansas's Educational System"],"dcterms_subject":["Little Rock (Ark.)--History--20th century","Little Rock School District","Education--Arkansas","Educational law and legislation","School management and organization","School integration"],"dcterms_title":["Unitary status"],"dcterms_type":["Text"],"dcterms_provenance":["Butler Center for Arkansas Studies"],"edm_is_shown_by":null,"edm_is_shown_at":["http://arstudies.contentdm.oclc.org/cdm/ref/collection/bcmss0837/id/730"],"dcterms_temporal":null,"dcterms_rights_holder":null,"dcterms_bibliographic_citation":null,"dlg_local_right":null,"dcterms_medium":["documents (object genre)"],"dcterms_extent":null,"dlg_subject_personal":null,"dcterms_subject_fast":null,"fulltext":"\n \n\n\n\n\n\n\n\n  \n\n\n   \n\n   \n\n\n   \n\n\n   \n\n\n\n\n   \n\n\n\n\n   \n\n\n\n\n\n\n\n\n\n   \n\n   \n\n \n\n\n   \n\n   \n\n  \n\n\n   \n\n  \n\n \n\n\n   \n\n  \n\n \n\n\n   \n\n  \n\n  \n\nr IN THE UNITED STATES DISTRICT COURT EASTERN DISTRICT OF ARKANSAS WESTERN DIVISION LITTLE ROCK SCHOOL DISTRICT V. LR-C-82-866 PULASKI COUNTY SPECIAL SCHOOL DISTRICT NO. 1, ET AL. received DEFENDANTS MRS. LORENE JOSHUA, ET AL. MAY 3 0 2002 INTERVENORS KATHERINE KNIGHT, ET AL. OFRCEOF DESEGREGATION MONITORING INTERVENORS The Joshua Intervenors' Opposition to the LRSD's Motion for an Immediate Declaration of Unitary Status This memorandum responds to the LRSD's \"Motion for an Immediate Declaration of Unitary Status,\" filed on March 15, 2001. Introduction During the 1997-98 school year, representatives of the Joshua Intervenors and the LRSD completed the proposed \"[LRSD] Revised Desegregation and Education Plan.\" They then filed a joint motion seeking its approval by the court on January 21, 1998. On April 10, 1998, the court (Judge Susan Webber Wright) approved the revised plan. The plan provided for three-year term assuming a substantial_and good faith compliance with its terms. [Sections 2.1, 9, and 11] The plan further provided for \"[t]he 1997-98 school year and the first semester of the 199899 school year [to] E^, 977 Ark E.^, Sturgis v. Skosos, ,977 S.W.2d 217, 223 (Ark. 1998) (interpret contract not by emphasizing one clause to the exclusion of others, \"but from the entire context of the agreement\"). 1 ibe a transition period in preparation for implementation of [the] Revised plan.\" [Section 10] Section 11 of the revised plan provides, in part: \"In anticipation of release, LRSD shall issue a report on March 15, 2001 indicating_the state of LRSD's compliance with the Revised Plan\" (emphasis added). The LRSD submitted an Interim Compliance Report on March 15, 2000 (cited as March 2000 report at ) and a Compliance Report on March 15, 2001 (cited as March 2001 report at ), On June 25, 2001, the Joshua Intervenors filed n an [OJpposition to [the] Little Rock School District's Compliance Report. II The court (Judge Wright) conducted 5 1-2 days of hearings concerning the LRSD's effort to secure release from court supervision [i.e. . on July 5-6, 2001, August 1-2, 2001, and November 19-20, 2001). Thereafter, on March 15, 2002, the LRSD filed its motion for an immediate declaration of unitary status. This memorandum responds to the LRSD motion, with regard to the subjects addressed in the hearings conducted by Judge Wright: Revised Plan Sections 2.1 (general requirement of good faith compliance)\n2.5-2.5.4 (student discipline)\n2.7 (academic achievement)\n2.7.1 (program evaluation)\n2.12.2 (general 3 2 See Tr., 8-2-01, at 890, 3-9 (comment by Judge Wright on the limited information on student discipline set forth in the March 2001 report). 3 While Section 2.7.1 refers to program assessment. assessment and evaluation the terms at times, used interchangeably. [Tr. , 11-19-01, 13-17 (Associate Superintendent Bonnie Lesley)] This memorandum shows, in detail, that LRSD acted on the premise that Section 2.7.1 addressed program evaluation. until. at 242, are, in 2requirement of activities \"for investigating the cause of racial disparities in programs and activities and developing remedies where appropriate\")\nto 6.7 (generally applicable LRSD 6.0 Compliance program). This memorandum also addresses the obligation of the LRSD to narrow the racial achievement gap, as required by the \"Pulaski County School Desegregation Case Settlement Agreement as revised on September 28, 1989.\" See revised plan, Section l(a.)\nTr., 7-6-01, at 378, 21-24 (recognition of obligation by former Superintendent Les Gamine)\nTr. , 11-20-01, at 564, 1-4 (recognition of obligation by Associate Superintendent Leslie). It is necessary to consider in connection with the LRSD motion and this response that the Joshua Intervenors' have the opportunity to submit some additional evidence. See Order by Judge William R. Wilson, May 9, 2002 at 13 (30 minutes of rebuttal), at 14 (during additional hearings, Joshua Intervenors may offer certain evidence bearing upon \"good faith obligations\" and \"the academic achievement of African-American students\"). Subsequent to the filing of the LRSD motion. this court scheduled a hearing in July 2002 on several issues. Order, May 9, 2002, at 14. Intervenors, therefore. do not respond to LRSD's argument that as to issues other than those addressed here, \"the LRSD should be granted unitary status and released from court supervision without further evidentiary hearings.\" [LRSD Mem.- the hearings, it faced the task of defending its performance in this sphere. See, e^^, Sturgis v. Skokos, supra. 977 S.W.2d at 223 (\"If there is an ambiguity, a court will accord considerable weight to the construction the parties themselves give to it, evidenced by subsequent statements, acts and conduct.\" [citation omitted]) 3Brief, at 34] This memorandum begins with a summary of the evidence. The summary, in the form of proposed findings of fact, encompasses the issues addressed here by the Joshua Intervenors: Student Discipline (at 4-19), Improving and Remediating Academic Achievement of African-American Students (at 19-36), Racial Disparities in Achievement (at 37-40), and Program Evaluation (at 40-46). An argument relying upon the factual summaries follows (at 47). The argument is not lengthy, the court's principal task seemingly being the examination of the facts in the light of concept of \"substantial compliance. Intervenors' factual presentation shows in each instance why substantial compliance is lacking, in the light of the concept of substantial compliance advanced. Results on the Arkansas Benchmark Examinations are set forth as an appendix. Summary of the Evidence I Student Discipline A- The Relevant Provisions of the Revised Plan (1.) The provisions of the revised plan relevant to the matter of student discipline are the following. 2.5. LRSD shall implement programs, procedures designed to ensure that there is discrimination with regard to school discipline. policies and \\or no racial a 2.5.1. The LRSD shall strictly adhere to the policies set 4 Intervenors dispute, in the argument, suggestion [e.,^, Mem.-Brief at 2, 34] that termination of jurisdiction would be appropriate if this court found substantial noncompliance, but somehow also was without doubt as to the system's intent to comply with the Constitution absent court supervision. Substantial compliance and future fealty to the Constitution are, in fact, separate components of the exit formula. 2, LRSD's repetitive termination as 4forth in the Student Rights and responsibilities Handbook to ensure that all equitable manner. students are disciplined in fair and 2.5.2. LRSD shall purge students' discipline records after the fifth and eighth grades of all offenses, except weapons offenses, arson and robbery, unless LRSD finds that to do so would not be in the best interest of the student. 2.5.3. LRSD shall establish the position of \"ombudsman\" the job description for which shall include the following responsibilities: ensuring that students are aware of their rights pursuant to the Student rights and Responsibilities Handbook, acting as an advocate on behalf of students involved in discipline process, investigating parent and student complaints of race based mistreatment achieve equitable solutions. and attempting to 2.5.4 LRSD shall work with students and their parents to develop behavior modification plans for students who exhibit frequent misbehavior. 2.12.2. LRSD shall implement policies and procedures for investigating the cause of racial disparities in programs and activities and developing remedies where appropriate. SECTION 6: LRSD Compliance Program. LRSD shall implement a desegregation compliance program which shall include the following components: 6.1. Compliance standards and procedures reasonably capable of reducing the prospect of noncompliance\n6.2. Oversight of compliance with such standards and procedures by the superintendent\na * * *  * * 6.3. Communication of compliance standards and procedures to employees\n6.4. Utilization of monitoring and auditing systems reasonably designed to detect noncompliance\n.... 6.6. Enforcement of compliance standards and procedures through appropriate- disciplinary mechanisms, including the discipline of individuals responsible for compliance and individuals responsible for any failure to report noncompliance\nand for compliance failure to 5 I\u0026amp;.1. After has been detected, implementation of all reasonable steps to correct past noncompliance and to prevent further noncompliance, including modification of the compliance program as necessary to prevent and detect further similar noncompliance. noncompliance B. The LRSD Interim Compliance Report (March 15, 20001 (2.) The LRSD \"Interim Compliance Report\" (March 15, 2000) discusses the five sections of the revised plan, which focus on student discipline, at pages 13-17. (a) The text concerning Section 2.5 addresses: adoption of policies (a general policy on non-discrimination and policies on discipline records)\nrevision of student handbooks\ncreation of \"an online student discipline reporting system for each school building . . .\"\nstaff development\na decrease in suspensions and expulsions in the LRSD\nthe sampling of parent, student, community and teacher attitudes on safety and order in the schools\nand expansion of the number of alternative learning sites, [at 13-15] (b) The text concerning Section 2.5.1 (on the Student Rights and Responsibilities Handbook) addresses: the adoption of general district standards on racial disparities in programs and activities and student rights and responsibilities\ndirecting principals to comply with the handbook\ninforming students and parents of standards\nand employing the ombudsman, [at 15] (c) The text concerning Section 2.5.2 (purging students' discipline records) addresses: adoption of standards\nin-service training\nand implementation by the Assistant Superintendent for School Discipline (Dr. Linda Watson), [at 15] (d) the text concerning Section 2.5.3 (the ombudsman) 6addresses: the filling of the position in February 1999 (half-way through the first year of the plan)\nestablishment of goals for the ombudsman's work, including \"[i]nvestigat[ing] parent and student complaints of alleged race-based mistreatment and . . . work[ing] to achieve equitable solutions\"\nincreasing community awareness of the ombudsman and monthly reports on his work, [at 15-16] (e) the text concerning Section 2.5.4 (behavior modification plans for students) addresses: the general process for developing such plans and an exit process for students eligible to return to a home school from the ft Alternative Learning Center.\" [at 16-17] The totality of the text on behavior modification plans is follows: as Students who exhibit frequent misbehavior have their cases refereed to the schools' Pupil Services Team. The team is comprised of the building administration, the students' teacher, the counselor, the parents and any specialists deemed necessary. The team develops a behavior modification plan warranted, [at 16] as (3.) The March 2000 Interim Compliance Report omits coverage of Section 2.12.2 (investigating causes of racial disparities in programs and activities and developing remedies). [See report at 82-86] (4.) The portion of the March 2000 Interim Compliance Report concerning Plan Sections 6.0 through 6.7 (general desegregation compliance program) does not discuss school discipline, [at 127-29] C. The LRSD \"Compliance Report\" (March 15, 2001] (5.) The LRSD \"Compliance Report\" (March 15, 2001) discusses the five sections of the revised plan, discipline, at pages 24-26. which focus on school 7(a) The text concerning Section 2.5 addresses: decreases in the numbers of suspensions and expulsion system-wide and for black and white students\nthe decrease in the number of students committing offenses\nthe sampling of community and teacher attitudes on school issues (positive views on safety and pupils' feelings on \"belonging at schools\"), [at 24-25] The report also includes this text [at 24]: The number of African-American students suspended decreased 20 percent consistent with the overall reduction in disciplinary sanctions. The proportion of suspensions issued to African- American students remained in the neighborhood of 85 percent. The Report describes no particular action directed at the continuing racial disparity. (b) The text concerning Section 2.5.1 (on the Student Rights and Responsibilities Handbook) addresses: school board approval of general district standards on racial disparities in programs and activities and student rights and responsibilities. [at 25] (c) The text concerning Section 2.5.2 (purging students' discipline records) addresses: asserted compliance with this provision by school principals and the \"Student Hearing Office.\" [at 25] (d) The text concerning Section 2.5.3 (the ombudsman) addresses [at 25-26]: training received by the ombudsman\nsteps taken to increase public awareness of the ombudsman's services\na and description of the ombudsman's activities, which reads as follows: Efforts to raise public awareness of the ombudsman appear to have been successful. In the last year, the ombudsman has been contacted by over 250 parents or students and provided 8services related to over 450 incidents. the In addition, ombudsman has implemented intervention activities at Badgett Elementary and McClellan High School designed to assist African-American males who demonstrate unacceptable behavior. Efforts are underway to expand these activities to include other schools. (e) The text concerning Section 2.5.4 (behavior modification plans for students) contains only a general description of the asserted process for developing such plans. Contrary to other instances, there is no reference to a school board policy or the numbers of students and schools involved, [at 26] (6.) The part of the March 2001 Compliance Report addressing Section 2.12.2 (investigating causes of racial disparities in programs and activities and developing needed remedies) contains only six lines of text. This text cites the school board's adoption of the general policy on racial disparities in programs and activities and then provides in part: \"In implementing its obligations under the revised plan, the District has addressed racial disparities in . . . discipline (Section 2.5) . . 165] (7.) The March 2001 Compliance Report omits mention of Plan . \" [at Sections 6.0 through 6.7 (general desegregation compliance program), which had been discussed in only a cursory fashion in the March 2000 report (see paragraph 4 above), [at i-iii] 5 Section 11 of the revised plan for the LRSD provided for release of court jurisdiction \"provided that LRSD has substantially complied with its obligations set forth in [the] Revised plan, added: \"In anticipation of release, LRSD shall issue a report on March 15, 2001 indicating the state of LRSD's compliance with Fthel It It [thel Revised Plan\" (emphasis added). 9D. The Evidence Presented to the Court (8.) The LRSD March 2000 and March 2001 reports and Dr. Linda Watson's testimony stressed reduction in the overall number of suspensions and expulsions. [March 2000, at 13-14\nMarch 2001, at 24\nTr., 11-19-01, at 48, 13-21\nat 55, 22 to 56, 15\nat 83, 14-21 (Dr. Watson)] 6 However, the data set forth in CX 743, introduced by LRSD, revealed that in 2000-2001, the third year of the plan, while white student suspensions\\expulsions continued to drop (69 fewer. 11.2 percent lower than 1999-2000), black student suspensions\\expulsions increased in that school year (496 more. 12.3 percent higher than 1999-2000). [CX 743 (\"Discipline Suspension Index by Year\")] (9.) While asserting that suspensions and expulsions decreased in number, the LRSD acknowledged that racial disparity continued. [March 2001 report, at 24)\nWatson testimony, 11-19-01, at 83, 21\nat 113, 14 to 114, 1\nCX 743] 14- (10.) The LRSD reports in March of 2000 [at 13-15] and 2001 [at 24-25] presented no data showing discipline by school. The 6 In LRSD's affirmative presentation in presentation, LRSD identified Dr. Linda Watson as responsible for implementation of Sections 2.5, 2.5.1, 2.5.2, 2.5.3, and 2.5.4 of the revised plan. [Tr. at 25, 16-19] 11-19-01, 2000, the Office of Desegregation Monitoring (ODM) issued a report titled Disciplinary Sanctions in the Little Rock School District, it sets forth data by school, by sex and race, on the students subjected to one or suspensions. This allowed the LRSD to identify the extent of representation of black students in discipline in a meaningful manner, and to single out schools with atypical disparities. See Appendix at 5. The LRSD chose to ignore the ODM report. See para. 19, infra. On June 14, number of more over- 10reports set forth no data by sex and race [id.], with the omission of data on disciplining of black males being particularly significant [Tr., 8-2-01, at 892, 5-9 (Associate Superintendent Sadie Mitchell)\nTr., 11-19-01, at 124, 4-14\nat 132, 12-24 (Dr. Watson)\nCX 583, at 125 (ODM report noting black males' being suspended \"at significantly higher rates than any other subgroup\")\nsee also Tr., 8-2-01, at 890, 23 to 891, 13 (Judge Wright)] (11.) The discipline process at the school level involves referrals of students by teachers and imposition of sanctions by administrators. [Tr., 11-19-01, at 151, 155] (12.) The March 2000 and 2001 LRSD reports show no evidence of the development of criteria to identify schools, teachers or administrators involved in atypical racial disparities in discipline [e^^, departing from system averages, or in the case of a teacher or administrator in a particular school, departing from the pattern for colleagues in that school). [March 2000 report, at 13-15\nMarch 2001 report, at 24-25] Assistant Superintendent Watson identified no such criteria in her testimony on November 19, 2001. (13.) The LRSD has the ability, by computer, to identify particular teachers, vice principals, and principals. whose referrals or sanctions evidence atypical racial disparities. This has not been done systematically, if at all. [Tr. , 11-19-01, at 123, 7-16\n128, 6-18\n149, 10 to 150, 25\n155, 7-12\n161, 4-13]] (14.) After acknowledging the absence of such disaggregation of data. Dr. Linda Watson testified as follows: Q Okay. So, it wouldn't be possible to correct it, if it was not disaggregated and in writing, would it? 11A. I guess not sir. [Tr., 11-19-01, at 149, 23-25] (15.) Dr. Watson testified as follows: that are helping Q. All right. Is there a group within the District or made up teachers, administrators, support staff, liiaL aj-t\niifcij.pj.ng to identify and to be responsible for correcting the disparate impact, discipline in the District? A. Not to my knowledge. [Tr. 11-19-01, at 162, 18-23] (16. ) Dr. Watson agreed that she \"[has] not prepared a monitoring report with respect to disparities in discipline.\" [Tr., 11-19-01, at 114, 4-7] Asked \"[d]id you make a written analysis of discipline data to reveal any potentially systemic problems,\" Dr. Watson testified, \"No sir.\" [Tr. , 11-19-01, at 142, 8-10] Asked whether former Superintendent Les Carnine or Associate Superintendent Junious Babbs had II prepare[d] a causation analysis of discipline disparities,\" Dr. Watson testified. II Not to my knowledge no, sir.\" [Tr., 11-19-01, at 130, 1-4] Faced with the question, \"[s]o, there are no plans by which to reduce disparate impact of black students?,\" Dr. Watson, the person responsible for implementation of the discipline sections of the revised plan, testified: \"Not, to my knowledge.\" [Tr., 11-19-01, 135, 6-8\nsee also id. at 112, 9-17] (17.) Dr. Watson testified as follows: Q. Have you made any recommendations regarding how to address the gross over representation of black boys, in the disciplinary process? A. No, I have not. Q. Have you not publicly stated that there needs to be some because more attention devoted to dealing with this problem, apparently there is a fear factor associated with black boys? A. Yes. [Tr., 11-19-01, at 132, 12-20] 12(18.) The following testimony of Dr. Watson is particularly significant in view of LRSD's acknowledgement of continuing racial disparities in school discipline and her own recognition of the particular issue regarding black male students. Q. Other than what you have told me, what is the Little Rock School district doing to  and what you told Mr. Walker, what is the Little Rock School District doing, in addition, to correct the disparity based upon race? A. I can't say that we are looking at it based on race. We are looking at the number of suspensions. We are trying to offer programs that African-American students, as well as other students, to participate in. [Tr., 11-19-01, at 163, emphasis added] 16-25\n(19.) The Office of Desegregation Monitoring distributed on June 14, 2000 a report titled Disciplinary Sanctions in the Little Rock School District. [CX 583] This report set forth discipline statistics by race, by school, for the school years 1993-94 through 1998-99, including the numbers of student in each school receiving one or more sanctions. The report also contained seven recommendations. [CX 583, at 127] Dr. Watson testified as follows regarding the ODM report. Q. Now, did you ever meet with the ODM after the ODM issued its report for the purpose of either better understanding their recommendations or for seeking ways to implement their recommendations? A. No, I did not, but I sure wanted to. Q. Why didn't you? 8 The data by student, by race, allows a comparison of the proportions of black and white students in a school receiving suspension or expulsion as a form of discipline. The comparisons in the individual schools can then be compared to those of other schools, allowing identification of schools with atypical disparities. of 13we A- Because I took  once the report came out, we discussed it in the cabinet, and it was the decision at that time that would not respond or do anything. Q. That's right. Dr. Carnine told you not to meet with them didn't he. A. At that time, yes he did. Q. I see. A. That was the decision that came from cabinet, going to address the issues. we were not emphasis added] [Tr., 11-19-01, at 177, 11-25\n(20. ) Dr. Watson testified as follows regarding behavior modification plans. Q. [Y]ou have indicated that you have responsibility under 2.5.4 for creating Behavior Modification Plans, is that correct? 2.5.4 is A. Yes, I did say that. Q. How many such plans did you develop each year? A. I couldn't say how many I developed. Q. You never had  you do not have a report, which documents the number you have developed? A. No sir, I do not. Q. What is the evidence to show that it was actually done? A. In cases that I heard in student hearings, when we needed to do Behavior Modification Plans, there were times that we stopped and did the plans there in the office. Q. I see. A. There were times that I referred them back to the schools. Pupil Services Team, to do Behavior Modification Plans. Q. I see. Do you agree with this statement? The district does not have any document compiling the total number of Behavior Modification Plans or the race or gender of students for whom Behavior Modification Plans have been prepared? A. I agree. I do not have the numbers. 14Q. All right. Do you agree with this statement? The District does not have nay document entitled, \"Monitoring Report of Behavior Modification Plans.\" A. I would agree.[Tr., 11-19-01, at 135, 9 to 136, 12] (21.) The testimony revealed that Dr. Watson had a vast array of responsibilities, more than one person could reasonably be expected to accomplish. The evidence also reveals that Dr. Watson, an \"assistant superintendent,\" sought additional personnel, that her plea did not bear fruit, and that additional personnel were needed to address racial disparity in discipline in individual schools. [Tr., 11-19-01, at 114, 4 to 119, 8\nsee also id. at 142, 25 to 146, 23 (example of type of effort needed to work with one school)]. E. Findings Concerning Overall Compliance with the Plan (22. ) Section 2.5 of the revised plan is devoid of any statement that the reguisite \"programs, policies and\\or procedures\" to be \"implement[ed]\" pursuant to this section are limited to those set forth in Sections 2.5.1, 2.5.2, 2.5.3, and 2.5.4. of the plan. Ms. Linda Watson's affirmative testimony presented by LRSD was not limited to the subject matter of these four sections. [jELSi, Tr. , 11-19-01, at 27-30] The text of Sections 2.12.2, 6.1, 6.2, 6.3, 6.4, 6.6, and 6.7 of the revised plan show that these sections are relevant to the subject of racial disparities in school discipline. Moreover, the text of these sections contains no indication that their content as to the discipline sphere can be satisfied merely by the fulfillment of the reguirements of Sections 2.5.1, 2.5.2, 2.5.3 , and 2.5.4. of the plan (assuming that LRSD substantially 15complied with each of these sections). (23.) There is no predicate for the court to find a lack of substantial compliance with Sections 2.5.1, 2.5.2, and 2.5.3 of the revised plan. However, the record does establish lack of a substantial compliance with Sections 2.5 and 2.5.4. (24.) The record establishes a lack of substantial compliance with Section 2.5 for the following reasons. (a) The LRSD report of March 2000 [at 13, 15] and the testimony of Dr. Linda Watson [Tr., 11-91-01, at 27-28] identified LRSD Policies AC, ACB, JB and JBA as steps implementing Section 2.5 of the revised plan. [CX 719 (cited standards)] However, these standards merely restate the LRSD's existing obligation to comply with the Egual Protection Clause of the Fourteenth Amendment and Title VI of the Civil Rights Act of 1964, 42 U.S.C. Sec. 2000d- d(4)(a) (barring racial discrimination in programs receiving federal financial assistance). Moreover, these standards do not even mention disciplining of students. [CX 719] (b) Dr. Watson's outlines for training of principals omitted the matter of disparate discipline generally and discipline of black males [CX 672-76\nTr. , 11-19-01 at 122, 14 to 123, 6], despite awareness of these issues. See paras. 9, 10. (c) The LRSD was aware of continuing racial disparities in the imposition of school discipline generally and in particular with  LRSD regulation JBA-R implements policy JBA. It is noteworthy that this regulation addresses explicitly each school's obligations to insure nondiscrimination in \"programs and activities,\" ..... JBA. __ with three required strategies, but does not mention discipline. [CX 719] 16regard to black male students. See paras. 9, 10. The system had the capability by computer of identifying schools with atypical racial disparities in discipline\nthe system also had the capability of identifying teachers whose referrals and administrators whose discipline actions were marked by atypical racial disparity. See paras. 9 \u0026amp; n. 6, 13. The LRSD did not implement any programs, policies and\\or procedures geared specifically to such schools personnel. See paras. 12-18. or (25.) The LRSD's discussions of \"behavior modification plans for students who exhibit frequent misbehavior\" [Section 2.5.4], and other evidence on this topic, show mere lip service to the concept, rather than \"work[ing] with students and their parents to develop\" such plans. See paras. 2(e), 5(e), 20\ncompare paras. 5(d) and 5(e) (in the March 2001 report, discussion of the ombudsman contains statistics on parent contacts and matters worked on, while coverage of behavior modification plans is limited to general description of process for developing plans). (26.) LRSD's failure to comply with Section 2.12.2 as applied to discipline is obvious. The system was aware of racial disparity and had the capability of isolating schools and staff with atypical problems. The system did not investigate the matter\nand, therefore, could not develop remedies. The system did not commit sufficient personnel to the issue. See paras. 9, 10, 12-18. Indeed, Dr. Linda Watson, the official responsible for compliance with the discipline sections of the plan [Tr., 11-19-01, at 25, the system's major witness on the topic, testified: ' 16-19], and \"I can't say 17 that we are looking at it [discipline issue] based on race.\" [Tr., 11-19-01 at 163, 21-25] (27.) Similarly, LRSD's failure to comply with Part 6 of the revised plan (\"LRSD Compliance Program\"), as applied to discipline. is obvious. Again, LRSD was aware of the general pattern of discipline disparity, and the particular issue about black male students. The system did not adopt standards to identify schools and staff with atypical discipline patterns. It did not analyze available data based upon such standards. It did not inform staff of such standards and procedures. It did not enforce such standards, or require remedial actions to address problems identified. Neither the superintendent, nor his designees oversaw compliance with any such standards and procedures. See Sections 6.1, 6.2, 6.3, 6.4, 6.6, and 6.7 and paras. 9, 10, 12-18, 21. (28.) Finally, LRSD's performance with respect to student discipline does not evidence substantial compliance with its agreement to \"in good faith exercise its best efforts . . to ensure that no person is discriminated against on the basis of race, color or ethnicity in the operation of the LRSD. . It [Section 2.1] The evidence supporting this conclusion includes the following, [i] The district did not commit adequate personnel to the issue of discipline. See para. 21. [ii] Despite knowledge of the continuing racial disparity in discipline, the system, did not study the causes, or identify and follow-up on schools and personnel with atypical disparate patterns. See paras. 9, 10, 12- 18. [iii] Dr. Watson testified, as noted, that \"I can't say that we 18 Iare looking at it [discipline issue] based on race.\" See para. 18. [iv] Upon receipt of the Office of Desegregation Monitoring report on school discipline in June 2000, the decision of the superintendent and his cabinet was \"at that time . . . we would not respond or do anything\"\n\" . . we were not going to address the issues. II See para. 19. II. Improving and Remediating Academic Achievement of African- American Students_________ A. The Relevant Provisions of the Revised Plan (29.) The provisions of the revised plan relevant to the subject of improving and remediating the academic achievement of African-American students are the following. 2.7. LRSD shall implement programs, policies and\\or procedures designed to improve and remediate the academic achievement of African-American students, including but not limited to Section 5 of this revised plan. 2.7.1. LRSD shall assess the academic programs implemented pursuant to section 2.7 after each year in order to determine the effectiveness of the academic programs in improving African-American achievement. If this assessment reveals that a program has not and likely will not improve AfricanAmerican achievement, LRSD shall take appropriate action in the form of either modifying how the implemented or replacing the program. program is * * * 2.12.2. LRSD shall implement policies and procedures for investigating the cause of racial disparities in programs and activities and developing remedies where appropriate. ft ft ft SECTION 6: LRSD Compliance Program. LRSD shall implement a desegregation compliance which shall include the following components: program 196.1. Compliance and procedures capable of reducing the prospect of noncompliance\nstandards reasonably 6.2. Oversight of compliance with such standards procedures by the superintendent\nand 6.3. Communication of compliance standards and procedures to employees\n6.4. Utilization of monitoring and auditing systems reasonably designed to detect noncompliance\n.... 6.6. Enforcement of compliance standards and procedures through appropriate disciplinary mechanisms, including the individuals responsible for compliance individuals responsible for any failure to report noncompliance\nand discipline of any to and 6.7. After has been detected, implementation of all reasonable steps to correct past noncompliance and to prevent further noncompliance, including modification of the compliance program as necessary to prevent and detect further similar noncompliance. noncompliance been to B. The Shortcomings in the Educations Afforded Black Students and the Standards Adopted to Address the Problem______ (30.) Two aspects of Section 2.7 of the revised plan are particularly noteworthy. First. The LRSD obligation is not limited to \"design[ing]\" programs and other initiatives\nrather, the initiatives must also be implementPed.\" [See Tr., 8-1-01, at 686- 87 (Leslie)] Second. The programs and other initiatives \"[include] but [are] not limited to [those in] Section 5 of [the] revised plan.\" (31.) Dr. Leslie Carnine became Superintendent of the LRSD effective with the 1997-98 school year. [Tr., 11-19-01, at 341-42] During testimony on November 19, 2001, when he had served for four years. Dr. Carnine provided the following overview. if you and Mr. Walker, when we put the new plan together, if will_remember, and in fact  if i can find the document, L.d I think we might be able to present it, but I said at the time 20that it was my statement to you that I firmly believed that if we remediated the education of black students and made a real effort,where I_felt that it had been missing, that by that very remediation effort of increasing their achievement, we would help to, in fact minimize the disparity between black and white achievement. Now, that statement is the one that I have made continuously over the past four years, I have said nothing f erent, not that I am not  I am certainly concerned about that disparity issue. It has been my life's work. But my point is the only way you can do it is not worry about the disparity, but let's just teach kids. And I didn't feel that we were doing that good a job. I think we are doing a better job now. Are we where we need to be? Not absolutely, but we are getting there. . 51\nemphasis added] . . [At 450- (32.) During the 1998-99 school year, year one of the new plan, LRSD staff under the direction of Associate Superintendent Bonnie Lesley, who joined the staff at the end of June 1998 [Tr. 8- 1-01, at 670, 18-19], undertook a comprehensive review of the educational program, including students' test scores. This review yielded, ultimately, the view that the curriculum for grades K-12 in language arts (including literacy), mathematics, science and social studies needed to be replaced. [March 2000 report, Tr., 11-20-01, at 550, 10-14] at 45\n(33. ) The review of programs during 1998-99 examined the development of early literacy skills in the light of results for LRSD students on the Arkansas Grade 4 Benchmark Examination (Spring 1998 and Spring 1999)^ and the Stanford Achievement Test (SAT 9) 10 Arkansas has adopted curriculum frameworks for language arts, mathematics and other subjects. For each framework, there are benchmarks, identifying, in grade level bands, knowledge and skills which it is hoped students will master. districts mathematics to give benchmark in grades four and examinations The State requires local eight. in These literacy and instruments are 21(Grade 3, Fall 1998). 11 On the state-mandated examination, 42 percent of LRSD students performed at the lowest level (\"Below Basic\") and only 30 percent of students at the levels deemed acceptable. Significantly, \"[f]ifty-three percent of African- American grade 4 students performed at the 'Below Basic' level, compared to 20 percent of white students.\" The results for grade 3 on the nationally normed SAT 9 reading test were consistent. \"In both cases only approximately 30 percent of LRSD students performed at the 'Proficient' or 'Advanced' levels . . , again indicating that far too few students are becoming good readers by grade 3.\" [CX 703, Doc. 1 at 12-13] (34.) The federal educational program known as tl Title I\" originated in the Elementary and Secondary Education Act of 1965. It provides financial assistance to local school districts to support help for low achieving students. The March 2000 report noted LRSD's receipt of $4.2 million in Title I funds, annually. designed to show a student's level of mastery of benchmarks appropriate to the students' grade level. The results are reported in terms of four levels (below basic, basic, proficient, and advanced). The State's goal is that all students reach at least the level of proficient. [March 2001 report, at 56\nTr. , 8-1-01, at 692, 18 to 694, 9\n\"[LRSD] Memorandum Brief in Support of Motion for an Immediate Declaration of Unitary Status, Tab 5, at 9] basic, proficient II March 15, 2002, at 56. For a description of the SAT 9, see the March 2001 report. The exhibit did not discuss SAT 9 scores by race. The record contains SAT 9 reading scores by race for grade 5 for 1998-99 (but not grade 3 scores). On \"total reading\" the average percentile score for black LRSD fifth grade students was 27 and that for LRSD white students 69. [CX 741] 22Almost all LRSD schools received some Title I funds in recent years. \"The goal of Title I is to provide supports so that all children can achieve the rigorous curriculum content standards established by the State and the [local] District.\" [March 2000 report at 47, 68\nsee 20 U.S.C. Sec. 2701] (35.) The review of instructional programs during 1998-99 encompassed Title programs. The March 2000 interim report I described some of the findings as follows: An analysis of performance data found that most Title I schools had not been meeting their improvement goals. The stafffound_that part of the problem was the absence of or lack of consistent delivery of District-established literacy and math curriculum. These problems were corrected through the new Pre-K Literacy Plan and the new NSF mathematics curriculum. Another part of the problem was the lack of alignmentbetween the Title I programs and the regular curricula in literacy and mathematics. [At 68\nemphasis added] Dr. Leslie provided consistent testimony about the findings of the review. [Tr. , 81-01-01, at 700-02 (noting use of \"pull out programs\" which \"even result in the child missing the instruction on the things that are going to be tested\")] (36.) Testimony by Dr. Lesley revealed the consequences for Title I participants, disproportionately black youth [Tr., 8-1-01, at 702], of the lack of alignment of the content of the Title I program \"and the regular curricula in literacy and mathematics.\" are  . . . Alignment is absolutely critical, particularly in an urban school district, because alignment means that you are going to test kids over, that you have given them a chance to learn that, that you have got that included in the curriculum. And so without alignment, poor kids in particular suffer the most, because there may not be an opportunity for them to get that knowledge and skill anywhere else. . . . [Tr. 11-19-01 at 199-200] ' (37.) The perverse and ironic impact of the content of the 23 ITitle I program on LRSD's black students has been substantial. The program is longstanding f i.e . . originated in 1965) and in the relevant time frame has supported activities in almost all Little Rock schools. The low scores of LRSD's black middle school and high school students on the state benchmark and the SAT 9 tests are no doubt due, in part, to their isolation from important parts of the curriculum by Title I programs (which were supposed to help them attain the knowledge and skills which LRSD identified as important for all students). (38.) Dr. Lesley also identified general problems in the math curriculum, prior to its revision. \"The old curriculum really focused on two strands of the [State] standards, and now we have a curriculum that encompasses all six strands, which include such things like statistics and problem solving and geometry and algebra, even for young children. So, that has been a big change for teachers.\" [Tr., 11-19-01, at 272, 6-11] (39.) The information gleaned about the content of the educational program and student outcomes on standardized tests led the LRSD leadership to conclude that a complete overhaul of the educational program was necessary. [CX 703, Doc.l at 12-13] This overhaul involved many areas. For example, the March 2000 report described the changes needed to implement the PreK-3 literacy plan. alone, as follows: \"The plan required restructured schools and school days, alignment of special programs with general education, new standards-based curriculum, appropriate pedagogy (instruction) , materials, and assessments, high-quality and intensive professional 24development, effective interventions, and parent involvement.\" [At 97] The restructuring also involved mathematics, science and social studies curricula. See para. 32\nsee also Dr. Leslie's testimony regarding the magnitude of the attempt to completely overhaul the educational program. [Tr., 11-19-01, at 198, 23 to 207,12] (40.) In the March 2000 and March 2001 reports, the LRSD identified many policies, practices and procedures. some general and some specific, as designed to fulfill the obligation which it assumed in Section 2.7 of the revised plan. Sub-paragraphs (a)-(e) describe central elements of the LRSD commitments. (a) \"The District developed in 1997-98 and 1998-99 comprehensive curriculum content standards, plus grade-level and course benchmarks in K-12 English language arts. mathematics, science, and social studies. In addition. curriculum maps were constructed for each area to ensure that the LRSD standards were aligned with the state's curriculum frameworks and assessments.\" [March 2000 report, at 45\nTr., 11-20-01, at 513, 17-21] (b) Staff members developed during 1998-99 and the directors approved in June 1999 a PreK-3 literacy plan. Board of The March 2000 report stated that II PreK-3 literacy is a major, if not the major priority of LRSD.\" \"The plan required restructured schools and school days, alignment of special programs with general education, new standard based curriculum, appropriate pedagogy (instruction), materials, and assessments, high-quality and intensive professional development. effective interventions, and parent involvement. [March 2000 report, at 96-99\nsee also id. at 2590 [assessment to \"[identify] [students] for early interventions\"] (c) The March 2001 report states: ..... Implementation of standards-based, instruction in mathematics and science, intensive and sustained professional development for teachers, and multiple assessment measures have been put in place to ensure improvement. New standards-based curricula in mathematics in grades K-8 and in science for grades 1-9 have been adopted. The curricula for other grade levels are being adapted locally to reflect a standards-based, inquiry-centered approach. The number of K-12 mathematics teachers who received training and materials to fully implement the new mathematics program increased from 215 teachers in the 1999-2000 school year to 515 teachers during the 2000-2001 school year. The number of K-12 science teachers who received training and materials to fully implement the new science program increased from 50 teachers in the 1999-2000 school year to 243 teachers during the 2000-2001 school year. Another 108 mathematics teachers inquiry-based intensive and 453 science teachers began implementing part of the standards-based program during 2000-2001. All teachers in mathematics and science are scheduled to fully implement the standards-based program during the 2001-02 school year. [At 115] 453 (d) Dr. Leslie testified that \"interventions\" for students whose achievement is not at the standards deemed desirable is vitally important part of the new literacy proqram. [Tr., 8-1-01, at 679,14 to 681, 15] Interventions (and remediation) are a point of emphasis in the LRSD reports of March 2000 and 2001 and in policies adopted by the LRSD Board of Directors to which they refer (summarized in next paragraph). This emphasis is in keeping with Section 2.7 of the revised plan which requires designing and implementing actions \"to improve and remediate the academic a achievement of African-American students. . . \" (emphasis added). (e) The LRSD adopted the following relevant standards. X3 See March 2000 report at 43, report at 51-52, 62, 64, 125-26. 44, 47, 48, 49\nMarch 2001 26(i) The Board of Education adopted Policy IHBDA (\"Remedial Instruction\") on July 22, 1999, after year one of the plan. It requires \"the district and each school\" to make \"comprehensive and aggressive early intervention efforts, especially in PreK-3 reading and mathematics, with continuing support through complementary remediation efforts on an as-needed basis to promote and sustain the standard levels of achievement.\" \"Intervention\\remediation efforts of the Little Rock School District will be comprised of a broad range of alternatives . . I* [CX 719, Policy IHBA] ( ii) The Board approved Regulation IHBDA-R tl Intervention\\remediation\") on October 21, 1999, after the start of year two of the plan. It provides, in part: Assistance will be provided for any student who is performing below the standard levels of achievement in the areas of mathematics and reading\\language arts. Intervention\\remedial programs include reteaching, tutoring, extended-day programs, Saturday programs, summer school, and special courses offered within the school day in addition to the core instruction. Program designs may differ from school to school, depending upon funding sources, needs of students, and decisions made by the Campus Leadership Team. [CX 719, Reg. IHBDA-R] (iii) The School Board approved Regulation IHBDA-R2 (\"Student Academic Improvement Plan\" (SAIP)) on August 24, 2000, after year two of the plan, in compliance with Act 999 of 1999. It requires teachers tl of English language arts and mathematics\" at each level to prepare individual SAIPs for \"each student who [i] is not performing on grade level (K-4)\n[ii] is not 'proficient' on any part of the state's benchmark examinations - primary (grade 4), intermediate (grade 6), middle school (grade 8)\nand [iii] is not scoring 'proficient' on End-of-Course examinations in literacy. 27geometry, and\\or algebra.\" IV School and individual teachers are encouraged to develop plans for additional students who, in their judgment, reguire remediation or intervention.\" The regulation further provides: student's The Student Academic Improvement Plan (SAIP) will document a student's achievement through District-adopted assessment tools, consideration of personalized education services (special education, English-as a- Second language. Title I, gifted_ programs, etc.) identification of areas of need^ specific skills to improve, strategies that will be implemented (see IHBDA-R), and progress. [CX 719, Reg. IHBDA-R] etc. ) C. Deficiencies in Implementation Establishing a Lack of Substantial Compliance with Secs. 2.7.1, 2.12.2 \u0026amp; Part 6.0 (41.) The content of paragraphs (42) through (57) supra shows that the deficiencies in implementation of the Section 2.7 activities identified by the LRSD are such that a finding of substantial compliance with Section 2.7 is not warranted. The LRSD's failure to substantially comply with Section 2.12.2 and Part 6 of the revised plan, as to the area of academic achievement, also apparent. is (42.) The LRSD Board of Directors approved the PreK-3 Literacy plan in June 1999, after year one of the revised plan. [March 2000 report, at 99] (43.) Teachers did not receive \"their copies of the new curriculum documents\" until \"August 1999\" the start of the school year (and the start of the second year of the revised plan). [March 2000 report at 45] \"All teachers did not begin the [1999- 2000] year with the training to implement the new curriculum, teaching strategies, and materials. Training occurred throughout 28 the year, and some teachers were not trained at all in 1999-2000.\" [March 2001 report, at 91] (44.) LRSD has emphasized that the training and retaining of teachers is a vitally important component of the new educational programs. (a) Dr. Leslie testified as follows: It [professional development] is probably the most important thing that we have done, and we've spent all of our treasury on that. A great deal of time, a great deal of energy, a great deal of money, trying to be sure that every teacher has at least a minimum level of training in several areas, because one of the things that was overwhelming about the plan and its implementation is that  particularly for elementary teachers, is that they had to learn new curriculum, they had to learn new materials, they had to learn new instructional strategies, and many of them had to change some belief systems, in order to make it work. over, much. And so, it takes more than a one workshop approach to get 311 that done. It has to be followed up over and over and And so, that is one reason we have emphasized it so The Board has allocated every dime they could to that effort over the last three years. [Tr., 11-19-01, at 207 13 to 208, 8] (b) The importance of teacher training was described as follows in the \"Year 2 Evaluation: The Effectiveness of the PreK-2 Literacy Program in the Little Rock School District 1999-2000 and 2000-2001\" (October 2001) by Dr. Lesley and other LRSD staff. The most expensive - and the most important - piece of the cost of any program implementation designed to improve student achievement is always the cost of professional development. . . . 'In study after study, it is the quality o: . , it is the quality of the teacher not variation in curriculum materials that is identified as the critical factor in effective instruction. That is not to say that materials are wholly unimportant, but that investing in teacher development has a better result than 14 This document appears at Tab 4 of the \"[LRSD] Memorandum Brief in Support of Motion for an Immediate Declaration of Unitarv Status.\" 29investing in curriculum materials.'. . . [At 96] (45.) Nevertheless, the II Year 2 Evaluation\" above identified serious shortcomings in the teacher training needed to implement the Pre-K-3 literacy program. The report states that 12 days of \"Ella training\"have been offered to K-2 teachers during the last two years. [At 97] It then sets forth a table, by school, showing the amounts of training for K-2 teachers. The average number of days per school is 4.65 across all levels. Moreover, in 15 of the 35 schools listed, the average number of days is 2.4 or fewer days. [At 98] The report states: From the table above, one can infer that implementation is, in general still at a low level since the number of days of ELLA training experienced by teachers is 4.65 of the 12 possible days available. Kindergarten teachers have the highest level participation, then grade 1 and then grade 2. Kindergarten, probably not coincidentally, is the highest performing grade level. [At 98] (46.) The LRSD employed lead teachers in the areas of math and science to promote the change from the traditional math and science curriculum to the new curriculum. Among other things, the lead teachers used an observation form to assess \"the implementation level and quality of implementation of the teachers\\schools in their cluster.\" LRSD reported the survey results for 1999-2000 in the March 2001 report. The report explains implementation codes as follows: \"3 - fully implementing standards-based\n2 - partially implementing standards based\n1 - minimally implementing standards based, 0 - not implementing standards based.\" The average score for 33 sites was 2.2. However, there were 10 scores of 1.8 or lower. The report explains scores for quality of implementation as 30follows: II 4 - excellent\n3 - good\n2 - fair\n1 - poor.\" The average score for quality of implementation was 2.6 for 3 2 sites. The report characterizes the results as follows: The District's average implementation score was 2.2, which represents a beginning shift from partial implementation to full implementation of a standards-based curriculum. The District's average quality score was 2.6, which represents trend toward quality instruction in math and science. Based on a the data provided, the District is in an active transition and the quality from the traditional curriculum to standards-based curriculum in both quality of implementation and the quality of implementation. See March 15, 2001 report at 122-24 (reports for elementary schools and middle schools only). (47.) The LRSD has also reported on the implementation of the new math and science curricula for the 2000-01 school year. Based upon reports by lead teachers, the average implementation score was 2.4 (on a scale of 0 to 3) and the average quality of implementation score 2.7 (on a scale of 1 to 4). Unlike 1999-2000, in 2000 01 LRSD reported only district averages and not scores by school. [ See \"Little Rock Comprehensive Partnerships for Mathematics and Science Achievement - Annual Progress Report for 2000-2001,\" Tab 5 to LRSD memorandum brief previously cited] (48.) The LRSD did not implement the new social studies curriculum until 2001-02, after year three of the plan. [Tr., 11-20-01, at 427, 2-3\ncompare para. 40(a) above] Indeed, Dr. Leslie seemingly testified at one point that the entire new curriculum was implemented for the first time in the \"Fall of 2000\" [Tr., 11-20-01, at 518, 22-25], rather than in the Fall of 1999. Compare para. 43. 31 I(49.) The October 2001 report on the Pre-K-3 literacy program after year two, previously cited, states that the study \"does not include  32examination of the different forms of interventions . II [Tab 4, at 83] Paralleling this admission, Associate Superintendent for Instruction Lesley, and Ms. Sadie Mitchell, Associate Superintendent for School Services, could not provide concrete information on the implementation of SAIPs, or other interventions for students requiring additional assistance to satisfy learning standards (see para. 40(e)). [Tr., 8-1-01, at 609, 18 to 611, 23 (Mitchell)\nat 679, 18 to 684, 4 and 736, 17 to 739, 18 (Dr. Lesley)] It is obvious from test results that black students are more likely to need interventions. See para. 33. (50.) As part of the new Pre-K-3 literacy curriculum. LRSD teachers have administered in the Fall and the Spring in grades K-2 the tl Developmental Reading Assessment.\" The results have varied sharply from school to school and even within the same school from year to year. Dr. Leslie attributed these variations to \"the degree to which teachers had implemented the new curriculum.\" [Tr., 8-1-01, at 731, 21 to 732, 2] (51. ) LRSD staff have recognized that there has been insufficient monitoring of classrooms to evaluate whether PreK-3 literacy curriculum is actually being implemented. the new Lack of a monitoring plan through classroom observations document the level of implementation is a problem, weakness not only resulted in a late identification of implementation in some cases, but it was also a weakness in evaluating the consistency of program implementation. to This poor See Mem. Brief in Support of Motion for an Immediate Declaration of 32Unitary Status (March 15, 2002), Tab 4, at 105. (52.) As noted, the LRSD is required to administer, each year in April, state Benchmark Examinations in literacy and mathematics to fourth and eighth graders. The State's goal is that all students reach the levels of proficient or advanced on each examination. which measure mastery of knowledge and skills. identified as important for each student to master. Results by race for the school years 1998-1999, 1999-2000, and 2000-2001 appear in the appendix, infra\nsee also note 10, para. 33, supra (description of Benchmark Examinations). (53.) On August 1, 2001, Dr. Lesley testified, in part, about the 1998-99 benchmark results in math and literacy for black fourth graders (administered in April 1999). Only 8 percent of these black youth attained the levels of proficient\\advanced in math and only 20 percent in literacy. Dr. Lesley's testimony included the following content. Q. Now, in terms of the 1998-'99 results for Little Rock black students in the fourth grade on math, proficient or advanced, is that right? eight percent were A. Let me look, check for sure. In '98-99, American students were eight proficient. yes. African- Q. Eight percent, okay, proficient. A. At or above proficient. Q. Correct. So that's basically one out of 12 of the students who have been tested, is that right, roughly? A. Eight percent, uh-huh. Q. Now, you regarded that as a serious problem, correct? A. Certainly. 33Q.-.An\u0026lt;^ a maior part of your explanation for that result is fcb.at thosestpdentg,__in terms of the curriculum they had received,had not_ been exposed to what you call many of the strands of the benchmarks for math, right? A. Yes. Q. For students to show mastery on a test like that, they need to be exposed to the material, right? A. Absolutely. Q. Now, in terms of black fourth graders in the literacy, percent were proficient or advanced, is that right? 20 A. I want to check and make sure I don't misrepresent. In '98- '99, yes, 20 percent. Q. Did you see that as a serious problem? A. Of course. Q, And do you think that, again, that part of the reason for that_was that the curriculum those students had coveredmany_of the strands in the state benchmarks literacy? had not for A. Yes. added] J_ [Tr., 9-1-01, at 694, 8 to 695, 21\nemphasis (54.) As noted, state benchmark exam results are also available for 1999-00 and 2000-01 (year three of the new plan). The results for 1999-2000 show some improvement. However, in 2000-01 (April 2001 test), the proportions of LRSD black youth attaining the levels of proficient\\advanced were  4th grade literacy 19%\n4th grade math 14%\n8th grade literacy 18%\nand 8th grade math only 4%! See tables infra in appendix. These results are on a par with the results for 1998-99, which evidenced to Dr. Lesley that black youth had not been exposed to curricula covering all of the grade- appropriate strands in the state curriculum frameworks. To be fair to the LRSD, no child will have had five years of exposure to the 34new curricula (if it is implemented) until those children tested in 2003-04 (April 2004).^ (55.) The results on the April 2001 State Benchmark Examinations and the other evidence reveal that LRSD had not implemented for the black students tested: (a) a curriculum marked by alignment with the state benchmarks\n(b) teaching by teachers with the training which LRSD identified as an essential part of its program pursuant to Section 2.7\nor (c) the interventions for students experiencing difficulties, also identified by LRSD as an essential facet of its program for compliance with Section 2.7. (56.) Scores on State Benchmark Examinations as of April 1999 for African-American students evidenced a situation where they had not been exposed to the content of the curriculum. See para. 53. The longstanding, massive Title I program was organized in a manner detracting from, rather than. as required by federal law, contributing to low-achieving students (disproportionately black) mastery of system instructional goals for all pupils. See paras. 34-37. The LRSD identified the need for a complete overhaul of the K-12 educational program in core courses, with implementation not commencing until year two of the plan. See paras. 32, 39, 40(a). The overhaul required change in many aspects of system operation. There were shortcomings in teacher in-service training, a pivotal area, as well as in implementation of the new math and science curricula. See paras. 43-47. There was admittedly no systematic 15 A student in kindergarten in 1999-2000, who makes normal progress, will reach the fourth grade and take the grade four benchmark examinations in April of the 2003-04 school year. 35 review of actual implementation of interventions for those students not doing well, another area of high importance, particularly for African-American youth given their achievement levels. See para. 49. Results of State Benchmark Examinations administered in April 2001 again established the lack of delivery of curriculum to African-American students. See para. 54. Finally, the SAT 9 tests for 2001 evidenced some backsliding in terms of addressing racial disparities in achievement. See para. 61(b), infra (SAT 9 results). (57. ) In light of the condition of education for black students in the LRSD at the outset of the revised plan, the program changes which the LRSD identified as necessary, and the lack of implementation of key facets of those changes (as shown by evidence about those initiatives and test results), the court finds that the LRSD did not substantially comply with the obligation which it assumed in Section 2.7 of the revised plan to implement certain programs, policies and\\or procedures. (58.) There was also a lack of substantial compliance in this area with Sections 2.12.2 and Sections 6.1 6.7, generally applicable elements of the revised plan. The LRSD did not adopt, and therefore could not follow-up on. \"compliance standards\" [Section 6.1]. [Tr., 8-1-01, at 671, 21 to 675, 6 (Dr. Lesley)] The LRSD was of the view that it need not address the racial gap in achievement, as such\nthe staff therefore did not seek to devise a remedy directed to decreasing this \"racial [disparity]\" as such. violating Section 2.12.2 of the revised plan. See Part III of this memorandum, infra. 36Ill Racial Disparities in Achievement (59.) The Revised Desegregation and Education Plan provides for the continuation in force of \"The Pulaski County School Desegregation Case Settlement Agreement as revised on September 28, 1989.\" [Section l(a.)] Testifying after being superintendent of schools for four years. Dr. Les Carnine agreed that he understood that the agreement with the State required the LRSD to narrow the achievement gap between black and white students. [Tr., 7-6-01, at 378, 21-24] See also at 378, 2-7 [\"Mr. Walker: Well, Your Honor, let me say this. We have the State agreement - The Court (Judge Wright) That's a settlement agreement, that's correct. about the achievement disparities, about reducing that, that's true.\"]\nTr., 11-20-01, at 564, 1-4 (Dr. Leslie).^ (60.) The LRSD did not II [develop] any particular program by which to remediate achievement disparity between African-American students and other students. . . \" during Dr. Carnine's tenure as superintendent. [Tr., 7-6-01, at 374, 25 to 375, 1-4 (Dr. Carnine)\nTr. , 8-1-01, at 622, 18 to 623, 9 (Associate Superintendent Mitchell)\nsee also Tr. 7-6-01, at 375, 14 to 379, 18 (Dr. Carnine)] (61.) The results of the State Benchmark Examinations and the Stanford Achievement Test show. at best. continuing massive disparities in achievement between black and white students and, at See also CX 594, at 11 [\"The achievement gap between African American and other students is always an issue of concern in the Little Rock School District. A major emphasis in the PreK-3 Literacy Plan is the significant narrowing and eventual elimination of that gap.\"] 37 worst, increasing disparities during the term of the revised plan. (a) On the Benchmark Examinations: [i] the proportion of fourth grade white students attaining the proficient or advanced levels in literacy has exceeded the like proportions for black students by 3.1 times (1998-99), 2.2 times (1999-2000), and 3.3 times (2000-01)\n[ii] the proportion of fourth grade white students attaining the proficient or advanced levels in mathematics has exceeded the like proportions for black students by 6.6 times (1998-99), 4.1 times (1999-2000), and 3.7 times (2000-01)\n[iii] the proportion of eighth grade white students attaining the proficient or advanced levels in literacy exceeded the like proportion for black students by 4.1 times (2000-01)\nand [iv] the proportion of eighth grade white students attaining the proficient advanced levels in mathematics exceeded the like proportion for black students by 10.3 times (2000-01). See appendix infra. (b) Results on the nationally normed SAT 9 test for LRSD students for the period 199697 through 200102 seemingly reflect an increase in the achievement gap. Twenty-one comparisons are possible in the data which covers grades 5, 7, and 10. The following comparisons are for the first (1996-97) and last (2001- 02) years of the six year period, [i] The gap between the average percentile scores of black and white youth increased in 20 of 21 instances\n[ii] over the six years, the average percentile scores for black students increased in 6 instances, remained the same in 6 instances, and declined in 9 instances\n[iii] over the six years, the average percentile scores for white students increased in 17 38 I iinstances, remained the same in 2 instances, and declined in 2 instances. [C 741, at 1] (62.) The LRSD has administered the Developmental Reading Assessment (DRA) in the Fall and Spring in grades K-2 beginning in 1999-2000. The LRSD maintains that the results show a narrowing of the achievement gap in reading.[E.g. . Tr., 11-20-01, at 409, 21 to 410, 4] However, test results on the DRA depend on a classroom teacher's judgments on his\\her students' abilities to read and to comprehend a series of progressively more difficult reading selections. In the spring, the teacher is in part judging her\\his own performance. The LRSD has recognized this issue: \"One caution, therefore, interpreting the data is that the teacher has scored his\\her own students' performance, and bias may be possible.\" [Year Two Report on the PreK-3 reading program. Tab 4 at 21] There is yet to be like progress, if any progress, on either the State Benchmark Examination or the SAT 9. See also Tr., 8-1-01, at 721, 12 to 726, 12 (lack of a predicate for LRSD to use DRA to evaluate achievement gap by race)] (63.) The LRSD has acknowledged problems prior to the effort to completely overhaul the K-12 program, which would harm black students disproportionately and exacerbate the achievement gap. The LRSD curriculum did not cover various strands of the State benchmarks. The Title I program emphasized \"pull out programs\" which isolated participants, disproportionately black. from the mainstream curriculum. See paragraphs 31, 34-38, 53-54. (64.) The LRSD has not substantially complied with its 39 iobligation under Section 2.7 of the revised plan to implement the activities which it identified \"to improve and remediate the academic achievement of African-American students . , II See paragraphs 41-55. (65.) The LRSD has not provided a predicate to end court jurisdiction with regard to its voluntary undertaking \"to narrow\" the achievement gap between black and white students. IV. Program Evaluation (66.) Three aspects of Section 2.7.1 (quoted above at page 19) are noteworthy. First. In the first sentence and the second sentence, the words \"assess\" and \"assessment\" refer to programs (rather than to assessment of students). Second. The assessment obligation is not limited to the programs described in Part 5 of the revised plan, but instead pertains to those \"implemented n pursuant to Section 2.7 . .  which as noted is not confined to the programs in Part of the plan. Third. The assessment 5 obligation is annual in nature. (67.) Asked during the hearing on November 19, 2001, \"to discuss the difference between an assessment and an evaluation,\" Associate Superintendent Bonnie Lesley began by testifying \"[w]ell, I think part of the confusion has been that we have sometimes used those terms interchangeably . . .\" [Tr., at 242, 13-17] The evidence shows that prior to the hearings on whether or not LRSD had attained unitary status, the LRSD had indicated repeatedly, by its actions. that compliance with Section 2.7.1 required the carrying out of program evaluations. Indeed, Dr. Lesley agreed with 40this proposition, when called as witness by the Joshua a Intervenors. The relevant evidence is summarized in sub-paragraphs (a) to (e). (a) LRSD issued a \"Compliance Plan for the LRSD Revised Desegregation and Education Plan\" on June 10, 1999. The text concerning Section 2.7.1. identified relevant \"Board Policies\" to include those on \"Testing Programs\" and \"Program Evaluation.\" The text on \"Procedures (Regulations, Administrative Directives, Handbooks, etc.\") relevant to Section 2.7.1 provided as follows: 1. Program Evaluation Agenda - in progress 2. Title I Restructuring Plan provides for Title I evaluation 3. National Science Foundation Project provides for program evaluation 4. Application for waiver includes an evaluation design from State or District rules 5. In progress: second-year evaluation of Success for All Thus, as seen, every sub-paragraph referred to \"evaluation.\" [See CX 544, at 11-12] (b) In a June 1999 position paper on the PreK-3 literacy program LRSD staff wrote: PreK-3 Literacy Program evaluation. ----------------------- In keeping with the obligations in the Revised Desegregation and Education Plan, the District shall employ with Title I funding a program evaluator, who shall annually report on the level of effectiveness of the innovations in this PreK-3 Literacy Plan. [CX 703, Doc. 1, at 44\nemphasis added] See also Tr., 11-19-01, at 278, 19-21 (Dr. Leslie). (c) The material in the March 2000 interim compliance report addressing Section 2.7.1 refers to \"[i]mprovements in the assessment of academic programs.\" [At 51] It also cites. inter 41alia, the \"Program Evaluation Plan\" [at 51], a draft policy on \"Curriculum Evaluation\" [at 52], and \"[t]he 1999-2000 program evaluation agenda . ,  approved by the Board of Education in August 1999.\" [At 53] (d) The material in the March 2001 compliance report addressing Section 2.7.1 is headed \"Program Evaluation\" which is repeated at a later point in the discussion.  a title The text (page 148 of the report) includes at least nine other references to fl evaluation.\" (e) During her testimony on August 1, 2001, Dr. Leslie agreed that the District had interpreted 2.7.1, which does not use the word evaluation. as nevertheless raising the topic of program evaluation. [Tr., 8-1-01, at 705, 24 to 707, 12\nsee also Tr. , 8-2-01, at 843 , 7-15 (Judge Wright noting that LRSD voluntarily undertook . . . obligation to have program evaluations of the programs that are designed to enhance African-American achievement\"] (68.) The LRSD took a different tact in seeking to defend its implementation of Section 2.7.1, at the hearing on November 19, 2001. Dr. Lesley cited testing of students and other \" assessment\" activities as satisfying Section 2.7.1. [Tr., 11-19-01, at 242, 18- 22\n243 , 6 to 249, 14\nsee also at 253, 22 to 254, 6 (colloquy between Judge Wright and LRSD counsel)]^ The content of paragraphs Dr. Lesley distinguished such assessment a \"proqram evaluation.\" [Tr., 11-19-01, at 242, 23 to 243, 5] She described a program evaluation as \"more long term\" [at 242, 23]  a feature congruent with the reference in Section 2.7.1 to an activity \"after each year . . . .\"in contrast, her discussion of \"assessment\" as from 23] fl In contrast, her discussion of \"assessment\" 42(a) through (h) supra provide the likely explanation for the LRSD's seeking to defend its performance by discussion of assessment rather than evaluation. The deficiencies in evaluation activities have been such that a finding of substantial compliance with Section 2.7.1 is not warranted. (a) The LRSD Board of Directors did not \"[adopt\"] its Policy IL on \"Evaluation of Instructional Programs\" until March 22, 2001 near the end of year three of plan implementation. [CX 575] As noted, Section 2.7.1 refers to assessments (evaluations) \"after each year.\" The LRSD \"voluntarily undertook [this] obligation.\" (b) The LRSD Planning, Research and Evaluation unit (PRE) presented evaluation documents covering four areas to the Board of Education in August 2000. The documents concerned the PreK-3 literacy program, the implementation of middle schools (including the effectiveness of new curriculum in English language arts and science), the effectiveness of the ESL program and the national Science Foundation project components. The Board of Education tabled the consideration of these documents because they were incomplete and there were no recommendations. [Tr., 7-6-01, at 362, 24 to 365, at 389, 18 to 392, 18\nat 400, 16 to 401, 22 23\n(Superintendent Carnine)] During the three year period of the plan, the LRSD recognized that it did not have the capability, interencompassing teachers \"us[ing] the data that they have available on a daily basis to decide what to do next for one chid, for a group of children or for the whole class\" [at 245, 12-20] was incongruent with the provision in 2.7.1. 43nally, to prepare the required evaluations. [Tr., 7-6-01, at 400, 2-19 (Dr. Carnine)\nTr. , 8-2-01, at 710, 3 to 713, 21 (Dr. Lesley)\nat 829, 20 to 831, 6\nTr., 11-20-01, at 334, 496, 3 (Dr. Lesley)] 5-14\nat 495, 16 to (c) The versions of the evaluation of the implementation of the PreK-3 Literacy Program prepared during the three-year period were drafts. [CX 577, Tr. , 7-6-01, at 418, 17-23 (Dr. at 1\nCarnine)\nTr., 8-1-01, at 709, 3 to 710, 8 (Dr. Lesley)\nTr., li20 01, at 321, 21 to 322, 22\nat 472, 25 to 473, 8 (Dr. Lesley)] (d) The versions of the evaluation of the implementation of the new mathematics and science curricula prepared during the three-year period were drafts. [CX 577, at 1\nTr. , 7-6-01, at 398, 1 to 399, 9\nat 418, 17-23 (Dr. Carnine)\nTr., 8-2-01, at 829, 20 to 831, 6\nTr., 11-20-01, at 473, 25 to 476, 14 (Dr. Lesley)] (e) The version of the evaluation of the implementation of the new middle school program prepared during the three-year period was a draft. [CX 577, at 1] (f) The LRSD did not conduct during the three-year period an evaluation of the implementation of the several policies requiring interventions\\remediation for students performing below para. 49. par. See (g) The LRSD identified the summer school program as an important component of its effort \"to improve and remediate the academic achievement of African-American students\" [Section 2.7]. [March 2000 report, at 47\nMarch 2001 report, at 62, 125-26] In its March 2001 report, LRSD asserted that \"PRE has evaluated\" the 44\"Summer School [Program].\" [At 148] However, the evaluation of the summer school program for \"Summer School 2000\" was only in draft form as of April 5, 2001 and July 17, 2001. [CX 721\nTr., 8-1-01, at 645, 12 to 652, 12 (Assoc. Superintendent Mitchell)\nTr., 11-20-01, at 357, 1 to 358, 8] (h) In the March 15, 2001 report the LRSD asserted that \"PRE has evaluated . . . [11] programs . . II [At 148] This assertion is misleading. [aa] The \"Extended Years Report\" existed in draft form as of July 13, 2001\nthere is a version of the report dated September 28, 2001. [CX 720] [bb] The LRSD presented only drafts of the \"Summer School\" report. [CX 721] [cc] There is an evaluation of the \"Hippy Program\" dated July 1999. [CX 722] [dd] The report on the \"Charter School\" is dated June 25, 2001\nit was written by an external consultant. [CX 723] [ee] The report on \"Campus Leadership Teams\" contains survey data dated May 11, 2001 and lists of participants, without further discussion. [CX 724] [ff ] There is an evaluation of the ESL program dated * October 30, 2000. [CX 725] [gg] There is a draft evaluation on the \"Lyceum Scholars Program at Philander Smith College\" dated September 22, 2000. [CX 726] 45[hh] With regard to the \"Southwest Middle School's SEDL Program,\" there is a request for data from an external source and some data, not an evaluation. [CX 727\nTr., 11-20-01, at 361, 17 to 362, 5] [ii] With regard to \"Onward to Excellence (Watson Elementary),\" there is collection of information provided by the \"Site Facilitator\" on November 1, 2001, not an evaluation. [CX 728\nTr., 11-20-01, at 362, 7-17] [jj] With regard to \"Collaborative Action Team (CAT),\" there is a collection of survey data and some comments, of anonymous authorship. dated November 6, 2001. [CX 729\nTr., 11-20-01, at 363, 10-24] [kk] Regarding \"Vital Link,\" there is a brief. a undated evaluation of anonymous authorship. [CX 730] (67.) The LRSD did not substantially comply with the program evaluation obligation which it voluntarily assumed by virtue of Section 2.7.1. The LRSD did not evaluate the academic programs which it implemented pursuant to Section 2.7 after each year to determine their effectiveness in improving African-American achievement and to use the results to make program changes. Indeed, it has not fulfilled this agreement after three years. In fact, the LRSD tacitly acknowledged its failure by seeking to recast the nature of its obligation during the hearings. 46Argument A. Introduction and the Standard for Substantial Compliance In this matter, the court is called upon to \"[apply] the terms of a contract between [two of] the parties ...\" [LRSD v, PCSSD, 11, 83 F.3d 1013, 1017 (Sth Cir. 1996)]  the LRSD and the Joshua Intervenors. \"Because this case has been settled, the settlement agreement becomes, in a sense, a particularization of federal law applicable to these parties.\" Knight v. Pulaski County Special School District. 112 F.3d 953, 955 (Sth Cir. 1997). Put another way, \"the terms of the settlement agreement became the law of the case.\" Little Rock School District V. Pulaskui County Special School District. No. 96-2047, Slip Opinion, Dec. 15, 1997, at 6. In sum. this court is called upon to apply the parties' agreement in the form of the revised plan, which left in place among other things \"The Pulaski County School Desegregation Case Settlement Agreement as revised on September 28, 1989.\" [Section l(a.)] The revised plan identifies the standards which this court is to apply to determine, for example, whether the LRSD fulfilled its obligations regarding student discipline and program evaluation. and whether it is entitled to a \"release from court supervision.\" That other systems face less onerous criteria^ is irrelevant. LRSD is held to the obligations which it \"voluntarily undertook\" [see Tr., 8-2-01, at 843, 7-15 (Judge Wright referring to \"obligation to have program evaluations\")\nthey form \"the law of [this] case.\" 18 See LRSD Me.-Brief, at 18-19, 28. 47Construed as an entirety [see n. 1 at 1, supra 1 . the terms of the agreement support the construction that the court's jurisdiction continues as to an area in which a party meets its burden of proof of showing \"that LRSD has [not] substantially complied with its obligations set forth in [the] Revised Plan.\" [Section 11] In this light, a principal task for this court is to define the term \"[substantial compliance].\" The opinion in Cody v. Hillard. 139 F.3d 1197 (Sth Cir. 1998) provides guidance on this topic. 19 There, the district court had dissolved a consent decree, merely asserting in a conclusory manner that \"the defendants have conscientiously and in good faith complied substantially with its terms.\" [At 1199] In explaining the inadequacy of the district court's terse ruling, the appellate court wrote, in part: . . . . The record indicates that there have been failures in the past to comply with the decree and supplemental orders, and that there are still at least some violations of the decree. The district judge's order does not give us enough information to determine whether he ignored the evidence of past and present violations or whether he violations inconsequential in considered any the context of substantial compliance. If the conditions Powitz complained of constitute violations of the consent decree. exercise violations noncompliance its I were discretion I serious and to in the district court must determining whether those enough to constitute substantial cast doubt compliance with the Constitution. . added] on defendants' future . . [At 1199\nemphasis The Cody court focuses on two related matter. These are. first, whether any violations are \"inconsequential\" in the light of 19 Cody is, however, largely distinguishable\n\"[t]he decree did not state the time of its duration.\" consent 1198. See 139 F.3d at 48the parties' overall performance and, second, whether the particular violations, given their subject matter, involve \"serious\" natters. The Court of Appeals for the First Circuit twice addressed the meaning of \"substantial compliance\" in the context of appeals from judgments of civil contempt. See Fortin v. Com'r of Mass. Dept. of Public Welfare, 692 F.2d 790 (1982) and Morales-Feliciano V. Parole Bd. of Commonwealth of Puerto Rico. 887 F. 2d 1 (1989) (Judge Breyer). These decisions are also helpful. In Fortin, the court wrote [692 F.2d at 795]: . 'substantiality' must depend Finally, no particular percentage of compliance can be a safe-harbor figure, transferable from one context to another. Like ^reasonableness,' . . . 'substantiality' must depend on the circumstances of each case, including the nature of the interest at stake and the degree to which non-compliance affects that interest. In the present case, the interest at stake - entitlement to subsistence-level benefits - is great . . ., making the consequences of failure to comply quite serious. The district court properly weighed the seriousness of the harm . . - in considering the substantiality of the Department's compliance. . . . [citations omitted] The court also considered the duration of noncompliance. Id. at 796. The Morales-Feliciano court followed the Fortin standard. See 887 F.2d at 4-5. Intervenors next apply these standards to the facts. The argument shows that all of the shortcomings cited in the Intervenors' factual presentation involve substantial noncompliance. Because all areas of noncompliance impinge on and harm the education of youth, an interest of great importance [see Fortin and Morales-Feliciano, supra], Intervenors address that matter once at the conclusion of the argument. 49B. Student Discipline In this case involving racial discrimination in public education, the person responsible for compliance with the discipline sections of the plan testified: \"I can't say that we are looking at it based on See para. 18. Dr. Watson's description of inaction concerning discipline was entirely race.\" consistent with her admission. See paras. 12-17. The violation of Section 2.5 was \"serious. II Cody, supra. The system argues that the revised plan \"did not require the LRSD to reduce the discipline disparity.\" [At 28] However, it did require actions \"designed to ensure that there is no racial discrimination with regard to student discipline . . It [ Sec. 2.5\nemphasis added] Compliance would necessarily require scrutiny of disparity to determine whether it originated in discrimination in any schools. In any event, the LRSD plainly assumed this obligation in Section 2.12.2. See para. 26. Lastly, there are also obvious and serious violations of Section 2.1 (good faith efforts) and Part 6 (compliance program). See paras. 27-28. The LRSD merely accepted disparate discipline as a fact of lif e. That tact may be open to other school systems. See LRSD Mem. -Brief, at 28. However, the LRSD pledged to implement the revised plan in good faith. The plan became, therefore. \"the law of this case.\" The failure to implement the provisions of Section 2.5.4 (behavior modification plans) can not be dismissed as \"inconsequential.\" The March 2001 report listed 4,274 suspensions 50of black pupils in 1999-2000. [At 24] There was a need for such plans\nthe LRSD merely gave \"lip service\" to the concept. Para. 25. C- Improving and Remediating the Achievement of Black Students The LRSD pledged not only to design, but also to implement actions \"to improve and remediate the academic achievement of African-American students.\" [Section 2.7] 11 [T]he circumstances of [this] case ...\" [Fortin, supra] highlight the centrality of this pledge. The evidence shows that at the time that the parties drafted the revised plan and its implementation began, LRSD polices and practices isolated black students, disproportionately, from the curriculum content LRSD identified as important for all students. This practice included the operation of the Title I program a mode of operation diametrically opposed to the requirements federal law. See paras. 31-38, 53. of The LRSD determined, essentially, that it needed to replace its curriculum and that this step would fulfil its Section 2.7 obligation to the plaintiff class. Intervenors factual presentation shows that implementation fell short in areas deemed significant by LRSD (training of teachers for the new literacy curriculum. implementation of the new math-science and social studies curricula. provision of interventions to students [mostly black youth] not performing well. and monitoring of classrooms. See paras. 32, 43 51. Indeed, State Benchmark Examination results in April 2001 revealed that the vast majority of black pupils in grades 4 and 8 continued to show signs of isolation from the curriculum content deemed essential by the State and the LRSD in 51 math and language arts. See paras. 52-55. These shortcomings in compliance obviously involved \"serious\" and not \"inconsequential\" matters. Cody, supra. D. Racial Disparities in Achievement Former Superintendent Carnine, Associate Superintendent Lesley, and Judge Wright recognized the continuing requirement of an effort to narrow the achievement gap between black and white students. See paragraph 59. LRSD did not argue during the hearing that it could not narrow the achievement gap. It argued that its Section 2.7 activities would do so  and that it was doing so in the area of early grade literacy. See Tr., 7-6-01, at 375, 379, 18 (Dr. Carnine)\nsee para. 62. 14 to The LRSD did not develop any particular program designed to remedy achievement disparity between black and white pupils [see para. 60]\nthere have been, as noted, serious shortcomings in its implementation of the strategies to overhaul the educational program, K-12, which were to improve black achievement. The results of State benchmark and SAT 9 testing provide evidence that the educations of countless African-American students in the system have been tainted by isolation from the mainstream curriculum. See paras. 53-54, 61. The LRSD had promised in the prior plan to deal with achievement disparity. [At para. B] Manifestly, 1. a curriculum isolating black students from core content was not the way to make progress in this sphere. The failures to address the achievement gap, as such, and to implement major parts of the reforms encompassed in Section 2.7 are 52\"serious\" shortcomings. E. Program Evaluation The LRSD elected voluntarily to make a major commitment which it understood to involve program evaluation until such time as it determined that it could not show substantial compliance with Section 2.7.1, as so construed. Para. 67. The commitment encompasses not only evaluating the programs designed to benefit black students' achievement \"after each year,\" but also making changes if programs prove to be ineffective. Section 2.7.1. The LRSD seeks to defend its performance in this area by arguing that Section 2.7.1 required something less than a program evaluation. [LRSD Mem.-Brief, at 22-23] Intervenors have addressed this point. LRSD also contends that Intervenors' focus is on a set of evaluations, which the system had identified on page 148 of its March 2001 report. [LRSD Mem.-Brief, at 20] LRSD asserts: \"The evaluations were introduced as Court's Exhibits 720-730.\" [Id., at 21] Our response is two-fold. First. Intervenors's focus is on the LRSD's failure to complete, even in the threeyear period. program evaluations of central features of the LRSD program to implement Section 2.7. See paras. 68. Second. The content of paragraph 68(h) shows that the assertion that the page 148 \"evaluations were introduced\" is vast overstatement. The evaluations were to be an integral part of the LRSD effort to provide programs to improve and remediate the achievement of African-American students. The default here is \"serious,\" not 53inconsequential\" in the overall scheme of things. Cody, supra. F. The Setting in which Noncompliance Occurred \"In the present case, the interest at stake - [education] - is great . . . , making the consequences of failure to comply quite serious.\" Fortin,_supra. Indeed, the Supreme Court has recognized the importance of education in the contexts of racially discriminatory school systems [Brown V. Board of Education. 347 U.S. 483, 492-93 (1954)] and school suspensions [Goss v. Lopez. 419 U.S. 565, 576 (1975)]. See also Appeal of Little Rock School District, 949 F.2d 253, 256 (Sth Cir. 1991) (in identifying \"those elements of the 1989 plan that we consider crucial, and to which no retreat should be approved,\" court notes \"the agreed effort to eliminate achievement disparity between the races\"). States and local districts identify benchmarks in areas such as literacy and mathematics so that they can graduate youth prepared to enter postsecondary programs and later participate in all of life's callings and positive experiences. The noncompliance cited by intervenors continues to threaten these ends. The setting in which noncompliance has occurred adds to its substantiality. Conclusion The Joshua Intervenors bargained for. and secured by the district court's approval of the revised plan, the right to three years of substantial compliance with its terms. The appropriate remedy for substantial noncompliance in the areas addressed in this memorandum is compensatory relief in the form of a three year 54period of actual substantial compliance in those areas (and any other areas identified as ancillary to the areas marked by violations). See Miener v. Missouri. 800 F.2d 749 (Sth Cir. 1986)\nPihl V-_Massachusetts Dept, of Ed.. 9 F.3d 184 (1st Cir. 1993)\nLester H. v. Gilhool, 916 F.2d 865 (3rd. Cir. 1990), cert, denied, 111 S.Ct. 1317 (1991)\nJefferson County Board of Education V. Breen, 853 F.2d 853 (11th Cir. 1988). The same criteria concerning ultimate termination of the court's role should apply to the fresh three-year period. Finally, the LRSD suggestion [Mem.-Brief at 2] that the court could terminate its role despite finding of substantial a noncompliance should be rejected. The Cody opinion (at 1199) contains this language. citing the Supreme Court decision in Dowell: \"if district court finds that defendant operating in compliance with Constitution and unlikely to return to 'its former ways,' purposes of injunction have been achieved\" (emphasis added). This identifies a two-part standard. In any event, the LRSD agreed to make substantial compliance the test for the end of a court role and is bound by that choice. .'VX/' xR^pectfully .submitted, oecttully Robert Pressman 22 Locust Avenue Lexington, MA 02421 781-862-1955 Mass. 405900 ,Aohn W. Walker ^ohn W. Walker, P.A. 1723 Broadway Little Rock, AR 72206 501-374-3758 Ark. 64046 55Arkansas Benchmark Examination 4th Grade Literacy Note: See footnote 10 (description of Benchmark Examinations) Black Students 1998-99 [4-99] 1999-00 [4-00] 2000-01 [4-01] Below Basic Basic Proficient Advanced 52% 36% 49% 28% 80% 70% 81% 34% 32% 20% 29% 19% 0% 20% 30% 19% 1% 0% White Students 1998-99 [4-99] 1999-00 [4-00] 2000-01 [4-01] Below Basic Basic Proficient Advanced 17% 10% 14% 21% 38% 58% 62% 4% 23% 33% 60% 66% 6% 23% 37% 56% 7% 63% Appendix 1Arkansas Benchmark Examination 4th Grade Mathematics Note: See footnote 10 (description of Benchmark Examinations) Black Students 1998-99 [4-99] 1999-00 [4-00] 2000-01 [4-01] Below Basic Basic Proficient Advanced 74% 91% 17% 6% 2% 69% 85% 16% 10% 69% 17% 8% 8% 5% 15% 6% 86% 14% White Students 1998-99 [4-99] 1999-00 [4-00] 2000-01 [4-01] Below Basic Basic Proficient Advanced 24% 24% 25% 23% 28% 25% 47% 40% 48% 16% 21% 23% 21% 53% 40% 61% 31% 52% Appendix 2Arkansas Benchmark Examination 8th Grade Literacy Note: See footnote 10 (description of Benchmark Examinations) Black Students Below Basic Basic Proficient Advanced 1998-99 [4-99] 1999-00 [4-00] 2000-01 [4-01] 43% 39% 82% 17% 1% 18% White Students Below Basic Basic Proficient Advanced 1998-99 [4-99] 1999-00 [4-00] 2000-01 [4-01] 8% 19% 27% 51% 22% 73% Appendix 3Arkansas Benchmark Examination 8th Grade Mathematics Note: See footnote 10 [description of Benchmark Examinations) Black Students Below Basic Basic Proficient Advanced 1998-99 [4-99] 1999-00 [4-00] 2000-01 [4-01] 75% 21% 4% 0% 96% 4% White Students Below Basic Basic Proficient Advanced 1998-99 [4-99] 1999-00 [4-00] 2000-01 [4-01] 25% 34% 59% 31% 10% 41% Appendix 4The Extent of the Racial Disparity in Discipline by School, Race, and Sex in 1998-99 The Extent Black Male Suspensions Per 100 Students Exceed White Male Suspensions Per 100 Students The Extent Black Female Suspensions Per 100 Students Exceed White Female Suspensions Per 100 Students Junior High Schools Cloverdale Dunbar Forrest Heights Henderson Mabelvale Mann Pulaski Heights Southwest 1.23 2.05 1.56 1.64 1.39 2.07 3.00 0.79 1.53 2.71 2.18 1 to 1 1.11 2.80 7.00 0.71 Total 1.83 2.40 Senior High Schools Central Fair Hall McClellan Parkview 2.00 1.50 1.65 2.25 1.13 3.50 1.27 2.44 1.17 0.67 Total 1.83 1.71 Source\nODM Report, June 2000, at 90-120 Appendix 5 ICERTIFICATE OF SERVICF. This is to certify that a copy of the foregoing has been mailed, postage prepaid to the following counsel or record, postage prepaid on this day of May, 2002. Mr. Clay Fendley Friday, Eldredge \u0026amp; Clark, P.A. 400 W. Capitol, Suite 2200 Little Rock, Arkansas 72201 Ms. Ann Brown Marshall ODM One Union National Plaza 124 West Capitol, Suite 1895 Little Rock, Arkansas 72201 Mr. Sam Jones Wright, Lindsey \u0026amp; Jennings 2200 Worthen Bank Building 200 West Capitol Little Rock, AR 72201 Mr. Mark Hagemeier Office of Attorney General 323 Center Street 200 Tower Building Little Rock, AR 72201 1. fehifW.ap RECEIVED FILED us DISTRICT COURT EASTERN DISTRICT ARKANSAS JUL J 8 2002 OFRCEOF DESEGREGATION MONITORING IN THE UNITED STATES DISTRICT COURT EASTERN DISTRICT OF ARKANSAS WESTERN DIVISION JUL J 6 2002 LITTLE ROCK SCHOOL DISTRICT vs. 4:82CV00866-WRW PULASKI COUNTY SPECIAL SCHOOL DISTRICT NO. Let al MRS. LORENE JOSHUA, et al KATHERINE KNIGHT, et al ORDER JAME! 0 c DEP CLERK PLAINTIFF DEFENDANTS INTERVENORS INTERVENORS Attached is a transcript of the hearing held Friday, last (July 12). Since time is short (the evidentiary hearing will commence next Monday, July 22), this transcript, rather than a detailed. separate order, is adopted as the order of the court (court solecisms and all). IT IS SO ORDERED this I ri* day of July, 2002. UNITED STATES DISTRICT JUDGE THIS DOCUMENT ENTERED ON DOCKET SHEET IN COMPLIANCE WITFH RULE 58 AND/OR 7^ RCP ON BY.  1 2 3 4 5 6 1 8 9 10 11 1 IN THE UNITED STATES DISTRICT COURT EASTERN DISTRICT OF ARKANSAS WESTERN DIVISION LITTLE ROCK SCHOOL DISTRICT, Plaintiff, V . PULASKI COUNTY SPECIAL SCHOOL DISTRICT NO. 1, et al., Defendants. MRS. LORENE JOSHUA, et al.. Intervenors. KATHERINE KNIGHT, et al.. Intervenors. No. 4:82CV00866WRW Friday, July 12, 2002 Little Rock, Arkansas 8:30 a.m. 12 13 TRANSCRIPT OF HEARING BEFORE THE HONORABLE WILLITkM R. WILSON, JR., UNITED STATES DISTRICT JUDGE 14 APPEARANCES: 15 On Behalf of Little Rock School District: 16 17 18 19 MR. CHRISTOPHER HELLER, Attorney at Law MR. JOHN C. FENDLEY, JR., Attorney at Law Friday, Eldredge \u0026amp; Clark Regions Center, Suite 2000 400 West Capitol Avenue Little Rock, Arkansas 72201-3493 20 On Behalf of Pulaski County Special School District: 21 22 MR. M. S7H4UEL JONES, III, Attorney at Law Wright, Lindsey \u0026amp; Jennings 2200 NationsBank Building 23 200 West Capitol Avenue Little Rock, Arkansas 72201 24 25 [Continued] Christa R. Newburg, RMR, CRR, CCR United States Court Reporter1 2 APPEARANCES CONTINUED: 2 On Behalf of North Little Rock School District: 3 4 5 MR. STEPHEN W. JONES, Attorney at Law MR. GUY W. MURPHY, JR., Attorney at Law Jack, Lyon \u0026amp; Jones, P.A. 425 West Capitol Avenue, Suite 3400 6 7 Little Rock, Arkansas 72201-3472 On Behalf of the Joshua Inteirvenors: 10 11 MR. JOHN W, WALKER, Attorney at Law MR. John W. Walker, P.A. 1723 Broadway Little Rock, Arkansas 72206\nand ROBERT PRESSMAN, Attorney at Law 22 Locust Avenue Lexington, Massachusetts 02421 8 9 12 On Behalf of the Knight Intervenors: 13 14 MR. RICHARD W. ROACHELL, Attorney at Law Roachell Law Firm 11800 Pleasant Ridge Road, Suite 146 15 16 17 Post Office Box 17388 Little Rock, Arkansas 72222-7388 RECEIVED 18 JUL J 8 2002 19 OFFICE OF 0KE6REGATI0N MONITORING 20 21 22 Proceedings reported by machine stenography and displayed in realtime\ntranscript prepared utilizing computer-aided transcription. 23 24 25 Christa R. Newburg, RMR, CRR, CCR United States Court Reporter 11 3 (Proceeding at 8:30 a.m., as follows:) 2 THE COURT: We're here this morning for a short 3 hearing in the Little Rock School District against the Pulaski 4 5 County Special School, et al. It's Case No. LR-C-82-866. I 10 might first introduce counsel and the people present to Ms. Christy Conrad. Would you stand up, please, ma'am? She is my new lawyer on this case, commenced to work this morning. case. She will be the law clerk especially assigned to this That's Ms. Christy Conrad. We might start with Mr. Walker. I got my letter off late 6 7 8 9 11 yesterday, and if you don't mind outlining for me briefly what 12 13 your two rebuttal witnesses will say. Ms. Marshall go ahead. If you don't mind, come to the lectern. When we start 14 the trial next week, week after next, we'll have mikes on the 15 table, but I don't have them now. 16 MR. WALKER: Your Honor, my I inquire whether you 17 received my letter? Apparently our letters -- 18 THE COURT: I did get a letter from you. I've got it 19 right here, as a matter of fact. I don't believe it addressed 20 that issue. If it did, I overlooked it. Like I say, my letter 21 got out later than I thought. 22 MR. WALKER: Your Honor, it does attempt to address 23 it on page 2, paragraph four, sub six. 24 25 THE COURT: Paragraph four? MR. WALKER: Sub six, the bottom of the second page. Christa R. Newburg, RMR, CRR, CCR United States Court Reporter1 It begins, 4 \"Plaintiff suggests that if not\" 2 3 4 5 6 7 8 THE COURT: All right. Let me read that. I do have this letter, and I've read it. That's another one of my lawyers. We're all working on this case, and she needs to be able to hear us back in chambers, and she has just announced, Mr. Walker, she couldn't hear you. So both of us need to speak right into the mike. 10 11 12 13 MR. WALKER: THE COURT: sir. I have read that. Can you be a little more specific with us? MR. WALKER: Dr. Lesley in her testimony indicated that the evaluation process was not flawed. in part because it was it involved the ODM, Mr. Gene Jones specifically. and Yes, 9 14 Ms. Ann Marshall to some extent. And she submitted an exhibit 15 that relates to or was attempting to relate to the 16 participation of the ODM, in order to demonstrate that 17 involvement. We wanted to establish what ODM's role was and 18 also the fact that ODM at all times through Mr. Jones had 19 expressed difficulty and problems with the evaluation approach 20 that was being used by the district and the lack of 21 evaluations. 22 THE COURT: And Ms. Marshall and Gene Jones are both 23 going to address that issue? 24 25 MR. WALKER: There were two separate points where -- which differ. Ms. Marshall can only relate to an exhibit that Christa R. Newburg, RMR, CRR, CCR United States Court Reporter1 5 Dr. Lesley introduced where she made reference to, if I 2 understand to be correct, where she made reference to the 3 comments and the like that had been made about in criticism or 4 critique of a document which she had prepared, and it would be 5 our intention to show that that was misrepresentative of the 6 involvement of ODM. 7 THE COURT: All right. Thank you. 8 MR. WALKER: And the other will be Mr. Gene Jones, 9 and Mr. Jones was sometimes invited to some of the sessions 10 that dealt with the subject. And his -- the way his 11 participation was presented, we would address, and also the 12 comments and the like that he made or his obse3rvations from the 13 perspective from which he sits we thought would be useful to 14 the Court in explaining the overall evaluation. The ODM was 15 supposed to have a special role in relationship to the whole 16 process, and we would like to at least take that time to put 17 that in. 18 THE COURT: All right. I'll hear from Mr. Heller, 19 see if he continues his objection in view of that. 20 21 MR. HELLER: Good morning, your Honor. We do continue our objection. Mr. Walker didn't identify any of the 22 exhibits he's talking about. Dr. Lesley's testimony, which. as 23 we've said, could have been anticipated in its entirety because 24 it didn't concern anything other than the compliance reports 25 which were filed by the district, her testimony regarding ODM Christa R. Newburg, RMR, CRR, CCR United States Court Reporter1 6 10 was minimal. As I recall, all she did was point out that an ODM monitoring report which is in the record did not require anything other than what Dr. Lesley was doing. in the record. That report is Mr. Walker had a chance to question Dr. Lesley about it, and there shouldn't be any issue about that. With regard to Mr. Jones, all that was said about him was that he was a participant in several meetings. I think that's undisputed. Dr. Lesley didn't say that she thought Mr. Jones' position on a particular issue would be X or Y. The only thing in the record that I recall is that Mr. Jones participated in a 2 3 4 5 6 7 8 9 11 meeting, and I think that's undisputed. 12 With respect to Mr. Walker's comment about Mr. Jones' 13 perspective would be helpful on the evaluation process, which 14 really, as the Court is aware, was a requirement for 15 16 assessments rather than evaluations, that's something that clearly could have been presented in Mr. Walker's case in 17 18 chief, if he believed that someone from ODM had a perspective about the assessment process that was important, because 19 Mr. Walker knew that's exactly what Dr. Lesley was going to 20 testify about. 21 THE COURT: Let me say this before you leave the 22 lectern, if you will, because I may ask you another question: 23 I generally take a pretty dim view of rebuttal evidence because 24 I've found that most of it -- I've found in practice over the 25 years that most of it is not true rebuttal. And I point that Christa R. Newburg, RMR, CRR, CCR United States Court Reporter1 7 10 11 out in some of my letters or orders, and, as a matter of fact. in my standard scheduling order or letter with that scheduling order, one or the other, I point out that rebuttal witnesses must be identified if known. Well, that's almost by definition that if they're known, they're not rebuttal witnesses. always had a hard time with that. So I've But this case was tried by Judge Wright, the first roughly half of evidentiary hearing on the issues before the Court now. She did reserve 30 minutes' rebuttal time. Truthfully, I'm inclined to agree that this doesn't sound like rebuttal, but out of an abundance of caution. since it's only 30 minutes. I'm 2 3 4 5 6 7 8 9 12 going to allow these witnesses to be called, with these 13 requirements: Number one, I'm going to require Mr. Walker to 14 identify the documents -- are you prepared to do that now. 15 Mr. Walker, exhibits, or would you rather do this by a pleading 16 in the next -- by, say, Monday afternoon? 17 18 19 20 you 21 MR. WALKER: THE COURT: MR. WALKER: THE COURT: A letter, your Honor. All right. Then by 4 p. m. Monday. Your Honor, before you finish, could Let me finish, and then I'm going to let 22 you have the floor again, Mr. Walker. 23 24 MR. WALKER: All right. Thank you. THE COURT: By 4 p.m. Monday, identify the exhibits 25 that you plan to address with Jones or Marshall. Christa R. Newburg, RMR, CRR, CCR United States Court Reporter1 2 3 4 5 6 7 8 Number two, Mr. Heller, if you want to -- you can interview these people, I assume. If you can't interview them. I'll allow you to take a telephone deposition of them next week to prepare you to meet this rebuttal testimony. If you want to do that, notify Mr. Walker and me by 11 a.m. Monday, if you want to take their depositions as opposed to interviewing them. All right. If you don't have any other comments. 8 9 Mr. Heller, Mr. Walker looks like he's going to swell up and burst if he doesn't get to say something else on this. 10 MR. HELLER: There is just one thing, your Honor. I 11 think it's at least implicit in all of the orders, but we'll 12 13 certainly have an opportunity for cross-examination. I'm not sure how that counts against our time in the overall process. 14 but 15 THE COURT: I'm going to be somewhere between Judge 16 Woods and Judge Eisele on timing. 17 18 19 20 21 22 23 24 25 MR. HELLER: I think we'll have plenty left from our 20 hours, even if the cross-examination counts against THE COURT: All right. us. MR. WALKER: Well, your Honor, I have no objection to them interviewing these people, but Ms. Brown has always -- and the ODM, for the Court's benefit, has always taken the position that it's available to speak with either or both of the parties about any matter that they are related to. THE COURT: It looks to me like, Mr. Walker, you've Christa R. Newburg, RMR, CRR, CCR United States Court Reporter9 1 2 3 4 just won. Are you piling on now? MR. WALKER: No, sir. No, sir. All I'm saying -- no. sir. All I'm saying is that it's not an order that's necessary. I mean, they have that as a standing -- that's been 5 longstanding in the district, as long as the ODM has been in 6 the process. 7 THE COURT: I'm going to enter the order even if it's 8 pure surplusage. 9 MR. WALKER: All right. Now, with respect to the 10 testimony, we will provide that. 11 12 13 14 15 THE COURT: MR. WALKER: THE COURT: MR. WALKER: THE COURT: You mean the exhibits? The exhibits. Right. Your Honor, by way of background -- Let me -- I want to change that. If you 16 want to depose them, I don't think you would, Mr. Heller, but 17 if you do, let me know by -- let Mr. Walker and me know by 9 or 18 9:30, by 9:30 Tuesday morning, because you may not know until 19 you see the exhibits. 20 Go ahead, Mr. Walker. 21 MR. WALKER: Your Honor, I hate to say this. The 22 Court -- you indicated you were going to follow much of Judge 23 24 25 Wright's process that she followed. She steadfastly refused to allow us to depose the ODM, which was part of her staff. as she said it, because that would, in effect. in a way be like Christa R. Newburg, RMR, CRR, CCR United States Court Reporter1 2 10 deposing an arm of the Court. And I would ask that the Court not enter an order requiring depositions but allowing instead 3 for them to just have the interviews. 4 Once you start doing that, then 5 THE COURT: You know, that's a pretty good point. 6 Let me hear from Mr. Heller on that. 7 10 11 aware, MR. WALKER: MR. HELLER: Okay. Your Honor, as I'm sure the Court is we raised an issue with Judge Wright concerning the role of the monitors in this case. THE COURT: Let me say something on that, and I'll 8 9 12 try not to interrupt you too much, but I probably won't do a 13 14 very good job since I have -- I'm a type A. When I assumed the case, I met with Ms. -- or was assigned the case. I met with 15 Ms. Marshall, and we exchanged pleasantries right after I was 16 17 appointed. We did not talk about the substance of the case in any way. After that, after thinking about it and after 18 reviewing the file some and seeing what had been discussed. I 19 asked a member of my staff to contact Ms. Marshall and advise 20 her that all of our communications would be in writing. And I 21 have had no further conversations with her, do not intend to. 22 Everything will be in writing. And I can't imagine that I 23 wouldn't share the -- whatever writings I send to her or she 24 sends to me with counsel. So there will be no - just out of 25 an abundance of caution -- and I see the ODM as a fact- Christa R. Newburg, RMR, CRR, CCR United States Court Reporter11 1 gathering institution, and I'm going to use ODM in a different 2 way than Judge Wright used the ODM, although I'm not 3 criticizing or passing judgment on the way she used it. 4 But if 10 11 12 13 14 that helps you, that's I hope it does. But at any rate. that will be the relationship. MR. HELLER: And I think, your Honor, that addresses Mr. Walker's argument, because our position with Judge Wright IS , the monitor's office is either more like a law clerk and cannot be deposed but can't testify either, or more of a fact gatherer and not so closely related to the Court that testimony would be prohibited. And Judge Wright allowed us to take Ann Marshall's deposition. her testimony. We've done it once in anticipation of So your ruling is entirely consistent with what Judge Wright had previously ordered. And in any event, I can 5 6 7 8 9 15 say right now that if interviews can be arranged, I won't be 16 17 asking for a deposition. I will be perfectly satisfied with an interview. But only in the event that we couldn't reach an 18 agreement about arranging an interview -- 19 20 THE COURT: Let me ask you this, Mr. Heller: Assuming you wanted to discuss conversations. you or 21 Mr. Walker, either one, that Ms. Marshall had had with Judge 22 Wright, I don't see how that would be relevant now that I'm the 23 trier of fact. So that's something that I want to avoid being 24 delved into if a deposition is allowed. 25 MR. HELLER: I agree that would not be relevant. Christa R. Newburg, RMR, CRR, CCR United States Court Reporter1 2 3 4 5 6 12 again. shot. THE COURT: All right. Well, I'm going to flip-flop Y'all are going to give me a nervous breakdown. MR. WALKER: THE COURT: Don't do it yet. All right. I'll give Mr. Walker one last MR. WALKER: This is not a subject that I asked that 7 should be dealt with perhaps today. 8 9 I think that the Court of Appeals was very clear about what it wanted monitoring to do. There was a special concurrence from Judge Wollman in the -- 10 11 THE COURT: I'm familiar with that concurrence. MR. WALKER: -- that anticipated that monitoring 12 would be conducted in a certain manner, and the manner that it 13 was being conducted has, in effect, been approved by the Court 14 of Appeals. Now, if it's to be changed, then I would certainly 15 think that the Court ought to at least invite the positions of 16 the parties in writing and a brief on the subject so that we -- 17 18 19 20 21 22 23 24 25 THE COURT: MR. WALKER: On what subject? On the subject of the way the monitors should react or act with the Court. Recall in this situation. your Honor, there is a situation where once when Little Rock came to court and demonstrated that it was not aware of all the employees they had, even the total number and what they were doing and things like that. Judge Wright then gave the  office a function that was to do an investigation and to do things, and then she had hearings on those things. The Christa R. Newburg, RMR, CRR, CCR United States Court Reporter1 13 role of the monitor is distinctly different from that of 2 a party where orders are given and things --so what we'd like to 3 4 do is at least preserve that. In the field of desegregation 10 11 12 law, monitors me to do. THE COURT: MR. WALKER: I'm not understanding what you're asking I'm asking that you do nothing to change the way that that office operates. Because if you say that you're going to - THE COURT: I'm not changing the way, as far as I know. MR. WALKER: Even if you communicate with them and 5 6 7 8 9 13 communicate with them each time in writing, I think that that's 14 not something that should be necessarily -- 15 16 is saved. THE COURT: Your objection is noted. Your exception You can file a motion for reconsideration if you 17 want to, but I've made my mind up on that at this point, and 18 you can file a motion and -- but don't do it with a great deal 19 20 21 of optimism. But feel free to do it. MR. WALKER: Here's the reason I raise the issue THE COURT: I've decided on that issue, Mr. Walker. 22 Put whatever objection you have in your motion for 23 24 25 reconsideration. We need to move on to some other issues. MR. WALKER: Half a minute, please? THE COURT: Yes. Christa R. Newburg, RMR, CRR, CCR United States Court Reporter I I I14 1 MR. WALKER: We have the matter of -- the ruling here 2 necessarily has to apply to the other districts as well. 3 THE COURT\nAbsolutely. 4 MR. WALKER: We have the matters of Pulaski County 5 that are still pending. There is no motion before the Court. 6 7 The role of the monitor there would seem to be being limited by the Court's ruling now because -- 8 10 11 12 13 14 THE COURT: MR. WALKER: THE COURT: Put that in your motion. All right, your Honor. Thank you. All right. All right. I want to remind the parties, I've said it several times in writing, and Judge Wright said it, but I want to remind you at the outset of that, we have three discrete issues left, and one of them is advanced placement courses, another is extracurricular activities. the 9 15 third is guidance and counseling. And then, of course, we have 16 good faith, but only as good faith applies to those one, two. 17 three things that I just mentioned. And we have academic 18 achievement on the table, but only as it relates to those 19 three one, two, three issues. As Mr. Walker just noted. 20 this applies to both sides with equal force. 21 Now, as I have read the transcript, the issues tried to 22 23 conclusion, and I emphasize the phrase \"to conclusion. II by Judge Wright were lack of good faith under Section 2.1\ntwo. 24 25 improving African-American achievement, lack of good faith by the Little Rock School District\nand, three, student Christa R. Newburg, RMR, CRR, CCR United States Court Reporter1 2 15 discipline. Now, those issues are closed. They've been tried to conclusion, save and except the 30 minutes for rebuttal 3 which we will start the hearing a week from Monday with. 4 All right. Now, I want to turn to Mr. Pendley's letter 5 which was in response to the letter I got out about one o'clock 6 yesterday. Let me find Mr. Pendley's letter. It's dated July 7 11. 8 Now, on the first page, there's a reference to Joshua's 9 witness list and a reference to Ms. Sharon Brooks. 10 It appears to me that Mr. Walker's testimony would go to student 11 discipline there. If that's true, I think it would be 12 appropriate to object at the trial, but I don't know that I 13 14 need to deal with it now. If anybody thinks I do. I'll hear from you. But if it does go to student discipline. I'm likely 15 to exclude it at the trial. 16 17 18 MR. WALKER: THE COURT: MR. WALKER: Your Honor, may I be heard? You may. Some of these matters overlap. A matter 19 that may be related to student discipline may also relate to 20 counseling. And I would say in this situation, the discipline 21 part of this 22 23 THE COURT: Related to guidance and counseling? MR. WALKER: Yes, sir. And the first part of it. 24 where students collectively are punished for ringing an alarm 25 by putting them in a room with an aide for two months where Christa R. Newburg, RMR, CRR, CCR United States Court Reporter10 16 there's no record of the discipline appears to be disciplinary. But at the same time, it certainly goes to the educational experience and the need for there to have been at least some counseling with respect to what those -- the educational needs of those children were and how they were being addressed. we say that it has two purposes. The first, on discipline, So though, has to relate -- you remember, you've seen the records. They have disciplinary records showing students who are suspended for this, this, and this. Normal discipline relations, those kinds of things. But putting kids in a room 1 2 3 4 5 6 7 8 9 11 where they -- where there's no record of it clearly is 12 13 14 15 16 17 18 19 20 discipline. but it is also something else. THE COURT: I'm going to think about that issue. inclined to think that I'm relates directly to discipline, but I'll think about it and we'll take it up at the trial. Mr. Heller address it right now, briefly. MR. HELLER: I'll let if he wishes to. Thank you, your Honor. I would just like to point out that at the last hearing, Mr. Walker argued that this was a discipline issue, and he presented evidence about it and argued about it and argued precisely what he just 21 told the Court, that this is - this situation, he alleges. was 22 a way to avoid the recorded discipline statistics but was 23 nonetheless discipline. I think it's going to be easy for him 24 to say anytime something happened to any student at any time in 25 the district, whether it relates to academic achievement or Christa R. Newburg, RMR, CRR, CCR United States Court Reporter10 11 17 some issue that's already been litigated, oh, by the way, they missed some counseling or could have been counseled otherwise. But I don't think that draws it into the scope of any legitimate objection he might have about guidance and counseling. THE COURT: this issue. Mr. Walker, you're on the downside of If you want to submit a trial brief to me to try to get me in a right frame of mind, in your view, by trial day. do a -- and this applies to either side. If you want to do another trial brief, get it to me by noon Wednesday of next week. By noon Wednesday. And I will guarantee you I will -- 1 2 3 4 5 6 7 8 9 12 if it's not too long, I will have read your brief and your 13 citations of authorities. if you will avoid string cites. On 14 string cites, I read only the first one and sometimes the 15 second one. 16 17 18 19 Now we come to exhibits not directly related. some of them have been withdrawn and so forth, but right. Let's go to No. 746. MR. HELLER: And I think all Your Honor, there was one other witness 20 mentioned in Mr. Pendley's letter, and that's Ethel Dunbar. 21 THE COURT: Yes. I don't believe that it' s Yes. 22 my opinion, and, again, you can put this in the trial brief. 23 24 Mr. Walker, if you think I am wrong-headed on this issue, that what it looks like what Ethel Dunbar would testify to goes 25 to the gifted and talented issue, as far as I'm concerned. 1 I Christa R. Newburg, RMR, CRR, CCR United States Court Reporter t1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 18 That's not on the table. If you want to persuade me otherwise, you can put it in the trial brief. Let's go to the exhibits now. 746. I'm having a hard time reading my -- what is 746, Mr. Walker? Why don't you just hand me a copy of it so I can look at it? MR. WALKER: Your Honor THE COURT: can hand me? MR. HELLER: THE COURT: annotations on there. MR. HELLER: THE COURT: MR. WALKER: a copy of it. THE COURT: Does the school district have Yes, your Honor. a copy you If you've got any of your inked I don't want to see them. I've got a circle and an underline. I promise not to accept your emphasis. Your Honor, we have given you our copy, I just don't have it out here with me. and I just -- I need to look at something. MR. WALKER: MR. PENDLEY: MR. HELLER: is merely highlighted. MR. WALKER: THE COURT: sure. MR. HELLER: This one has been Here you go. We'll give you Mr. Pendley's copy, which Is this the new number given by your -- I think this is the old number. I'm not That's correct, your Honor. Our Christa R. Newburg, RMR, CRR, CCR United States Court Reporter19 1 objections use the old numbers. 2 3 over. THE COURT: 746 . All right. I'm working my way Mr. Walker, if you'll approach the lectern. Are you 4 telling me you don't have a copy either, Mr. Walker? 5 MR. WALKER: No, no. We have a copy, but we have 6 taken the old exhibits -- after your courtroom deputy told us 7 the new numbers, we changed them. 8 THE COURT: Okay. I'll give you time to get your sea 9 legs. 10 MR. WALKER: What is now 747 was 746. I don't 11 understand the objection. 12 13 THE COURT: All right. I'll have him state his objection then. Mr. Heller -- why don't you stand aside. 14 Mr. Walker, and let him state his objection. 15 16 MR. WALKER: All right. MR. HELLER: Your Honor, our objection is that 17 Exhibit 746, using the old number, relates to ALT testing and 18 not to any of the issues before the Court for next week. 19 MR. WALKER: Your Honor, if you look at that exhibit. 20 we're looking at the way the district has referred to the 21 numbers that are related -- I don't see anything in 746 in the 22 middle of the page which happens to be -- and I stand to be 23 corrected, the e-mail from Babbs to Kathy Lease. 24 25 THE COURT: You're going to have to -- I'm computer illiterate, Mr. Walker. You're going to have to quote the Christa R. Newburg, RMR, CRR, CCR United States Court Reporter10 11 12 13 14 15 16 17 18 19 20 21 22 20 language. I can't tell on e-mails from who to what unless it's written on there. MR. WALKER: Mr. Babbs, your Honor, is the person responsible for monitoring desegregation compliance. THE COURT: MR. WALKER: THE COURT: MR. WALKER: And he sent something to Kathy Lease. It apparently comes from Babbs to Lease. Right. And the third paragraph says, \"It would be appropriate to list current data that is available. Be reminded that when writing materials for our report submission. we will include district-wide numbers. We may not be there yet, but this will help serve as an indicator of established baseline information from which we will jump off.\" Now, this relates to, your Honor, the data that relates to pre-AP and AP courses, along with some other data, but it will be related to testimony regarding advanced placement. will be related to extracurricular activities. It also So you'11 understand the concept, when you've got two black schools. McClellan and Fair, for all practical purposes, when you lump the extracurricular participation from those schools with the other schools, it gives a picture of real inclusivity. If you take it out. it may not. When you lump the -- when you do a 1 2 3 4 5 6 7 8 9 23 lumping process, we're saying that it gives a false picture. 24 There was an intent here. and this goes -- this is an 25 intent, it goes to good faith, an intent to make a presentation Christa R. Newburg, RMR, CRR, CCR United States Court Reporter1 21 of a reality that did not exist. 2 3 THE COURT: All right. Mr. Heller? 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 MR. HELLER: Your Honor, Mr. Walker's explanation shows why this exhibit should be excluded. When we objected to their response was that it was related to extracurricular activities. Now the first thing that he said was that it's related to advanced placement and pre-advanced placement. not related to any of those things. and the compilation of documents. It' s It's related to testing It doesn't have anything specifically to do with any of the issues before the Court. Mr. Walker has now given the Court two different explanations of how it relates, none of which can be shown from the face of the exhibit. THE COURT: All right. I don't believe I need briefs on this one. forthwith. I will do a letter order ruling on that Mark No. 746 down and remind me so I don't -- with the other issues involved so I get right on it. I think it's already been noted, but I want to re-note it. It's now 747 under the new numbering system. All right. 754 . MR. WALKER: THE COURT: MR. WALKER: Mr. Walker, will you comment on 754? Yes, sir. Which now is what? It should be 755, if I'm not mistaken. MR. PRESSMAN: Same number. 4 5 6 7 8 9 it, Christa R. Newburg, RMR, CRR, CCR United States Court Reporter1 2 3 4 5 6 7 22 THE\nThis project was supported in part by a Digitizing Hidden Special Collections and Archives project grant from The Andrew W. Mellon Foundation and Council on Library and Information Resoources.\n   \n\n   \n\n  \n\n\n   \n\n  \n\n \n\n\n   \n\n   \n\n \n\n\n   \n\n  \n\n \n\n   \n\n \n\n  \n\n\n   \n\n \n\n  \n\n\n\n   \n\n  \n\n  \n\n\n   \n\n   \n\n  \n\n \n\n \n\n\n   \n\n  \n\n \n\n\n\n\n\n\n\n\n\n   \n\n \n\n\n\n  \n\n\n   \n\n\n\n  \n\n\n\n "}],"pages":{"current_page":125,"next_page":126,"prev_page":124,"total_pages":3369,"limit_value":12,"offset_value":1488,"total_count":40428,"first_page?":false,"last_page?":false},"facets":[{"name":"educator_resource_mediums_sms","items":[{"value":"lesson plans","hits":307},{"value":"online exhibitions","hits":37},{"value":"teaching guides","hits":34},{"value":"timelines (chronologies)","hits":23},{"value":"bibliographies","hits":15},{"value":"worksheets","hits":5},{"value":"annotated bibliographies","hits":4},{"value":"study guides","hits":4},{"value":"learning modules","hits":3},{"value":"slide shows","hits":2}],"options":{"sort":"count","limit":16,"offset":0,"prefix":null}},{"name":"type_facet","items":[{"value":"Text","hits":40428},{"value":"Sound","hits":1050},{"value":"StillImage","hits":803},{"value":"MovingImage","hits":213},{"value":"Collection","hits":10},{"value":"InteractiveResource","hits":4}],"options":{"sort":"count","limit":16,"offset":0,"prefix":null}},{"name":"creator_facet","items":[{"value":"Thurmond, Strom, 1902-2003","hits":2076},{"value":"Mississippi State Sovereignty Commission","hits":1425},{"value":"Newman, I. DeQuincey (Isaiah DeQuincey), 1911-1985","hits":1003},{"value":"Herrera, John J.","hits":777},{"value":"Connor, Eugene, 1897-1973","hits":567},{"value":"Jordan, Barbara, 1936-1996","hits":464},{"value":"South Carolina Council on Human Relations","hits":397},{"value":"AFL-CIO. Civil Rights Department","hits":326},{"value":"Hunter, Charles N., approximately 1851-1931","hits":323},{"value":"United States. District Court (Arkansas: Eastern District)","hits":289},{"value":"Smith, Lillian (Lillian Eugenia), 1897-1966","hits":244}],"options":{"sort":"count","limit":11,"offset":0,"prefix":null}},{"name":"subject_facet","items":[{"value":"Race relations","hits":4661},{"value":"Civil rights","hits":3223},{"value":"People--Ethnic Groups--Hispanics","hits":2928},{"value":"African Americans--Civil rights","hits":2321},{"value":"African Americans","hits":1792},{"value":"Education--Arkansas","hits":1747},{"value":"Civil rights--South Carolina","hits":1739},{"value":"Civil rights movements--Mississippi","hits":1550},{"value":"League of United Latin American Citizens","hits":1531},{"value":"LULAC","hits":1517},{"value":"Race","hits":1483}],"options":{"sort":"count","limit":11,"offset":0,"prefix":null}},{"name":"subject_personal_facet","items":[{"value":"Smith, Lillian (Lillian Eugenia), 1897-1966--Correspondence","hits":1888},{"value":"Meredith, James, 1933-","hits":1501},{"value":"Baker, Augusta, 1911-1998","hits":1413},{"value":"Herrera, John J.","hits":1312},{"value":"Smith, Lillian (Lillian Eugenia), 1897-1966","hits":713},{"value":"Mizell, M. Hayes","hits":639},{"value":"Silver, James W. (James Wesley), 1907-1988","hits":623},{"value":"Jordan, Barbara, 1936-1996","hits":608},{"value":"Connor, Eugene, 1897-1973","hits":582},{"value":"Snelling, Paula","hits":582},{"value":"McCain, James T.","hits":418}],"options":{"sort":"count","limit":11,"offset":0,"prefix":null}},{"name":"name_authoritative_sms","items":[{"value":"Smith, Lillian (Lillian Eugenia), 1897-1966","hits":2592},{"value":"Meredith, James, 1933-","hits":1497},{"value":"Herrera, John J.","hits":1331},{"value":"Silver, James W. (James Wesley), 1907-1988","hits":622},{"value":"Jordan, Barbara, 1936-1996","hits":608},{"value":"Snelling, Paula","hits":580},{"value":"Connor, Eugene, 1897-1973","hits":579},{"value":"Thurmond, Strom, 1902-2003","hits":360},{"value":"De Laine, Joseph A. (Joseph Armstrong), 1898-1974","hits":345},{"value":"King, Martin Luther, Jr., 1929-1968","hits":315},{"value":"Samet, Seymour, 1919-2014","hits":279}],"options":{"sort":"count","limit":11,"offset":0,"prefix":null}},{"name":"event_title_sms","items":[{"value":"Ole Miss Integration","hits":1567},{"value":"Housing Act of 1961","hits":663},{"value":"Little Rock Central High School Integration","hits":386},{"value":"Martin Luther King, Jr.'s Nobel Prize","hits":299},{"value":"Freedom Summer","hits":188},{"value":"Birmingham Bombing (Sixteenth Street Baptist Church)","hits":125},{"value":"Memphis Sanitation Workers Strike","hits":104},{"value":"University of Georgia Integration","hits":83},{"value":"Freedom Rides","hits":68},{"value":"Brown versus Board of Education","hits":55},{"value":"Civil Rights Act of 1964","hits":43}],"options":{"sort":"count","limit":11,"offset":0,"prefix":null}},{"name":"location_facet","items":[{"value":"United States, 39.76, -98.5","hits":10939},{"value":"United States, South Carolina, 34.00043, -81.00009","hits":4304},{"value":"United States, Georgia, 32.75042, -83.50018","hits":3804},{"value":"United States, Arkansas, 34.75037, -92.50044","hits":3648},{"value":"United States, Mississippi, 32.75041, -89.75036","hits":2519},{"value":"United States, Arkansas, Pulaski County, 34.76993, -92.3118","hits":2032},{"value":"United States, Arkansas, Pulaski County, Little Rock, 34.74648, -92.28959","hits":1863},{"value":"United States, Tennessee, Shelby County, Memphis, 35.14953, -90.04898","hits":1801},{"value":"United States, Texas, Harris County, Houston, 29.76328, -95.36327","hits":1564},{"value":"United States, Georgia, Fulton County, Atlanta, 33.749, -84.38798","hits":1236},{"value":"United States, New York, New York County, New York, 40.7142691, -74.0059729","hits":1198}],"options":{"sort":"count","limit":11,"offset":0,"prefix":null}},{"name":"us_states_facet","items":[{"value":"South Carolina","hits":6466},{"value":"Georgia","hits":5551},{"value":"Arkansas","hits":3826},{"value":"Mississippi","hits":3452},{"value":"Texas","hits":3432},{"value":"Tennessee","hits":2359},{"value":"Alabama","hits":2352},{"value":"North Carolina","hits":1657},{"value":"New York","hits":1561},{"value":"Florida","hits":1185},{"value":"District of Columbia","hits":1167}],"options":{"sort":"count","limit":11,"offset":0,"prefix":null}},{"name":"year_facet","items":[{"value":"1962","hits":5885},{"value":"1964","hits":5684},{"value":"1963","hits":5401},{"value":"1965","hits":5249},{"value":"1966","hits":5016},{"value":"1961","hits":4941},{"value":"1968","hits":4914},{"value":"1967","hits":4901},{"value":"1969","hits":4708},{"value":"1960","hits":4681},{"value":"1957","hits":4182},{"value":"1958","hits":4010},{"value":"1977","hits":3976},{"value":"1959","hits":3971},{"value":"1971","hits":3904},{"value":"1970","hits":3870},{"value":"1976","hits":3790},{"value":"1955","hits":3718},{"value":"1974","hits":3664},{"value":"1972","hits":3650},{"value":"1975","hits":3649},{"value":"1956","hits":3611},{"value":"1973","hits":3461},{"value":"1994","hits":3357},{"value":"1995","hits":3332},{"value":"1950","hits":3315},{"value":"1978","hits":3266},{"value":"1954","hits":3261},{"value":"1979","hits":3249},{"value":"1996","hits":3225},{"value":"1980","hits":3158},{"value":"1948","hits":3126},{"value":"1953","hits":3050},{"value":"1997","hits":3036},{"value":"1949","hits":3022},{"value":"1998","hits":2990},{"value":"1999","hits":2983},{"value":"1952","hits":2967},{"value":"1947","hits":2958},{"value":"1951","hits":2907},{"value":"1981","hits":2856},{"value":"2000","hits":2833},{"value":"2001","hits":2792},{"value":"2002","hits":2693},{"value":"1982","hits":2691},{"value":"1946","hits":2690},{"value":"2003","hits":2675},{"value":"1983","hits":2652},{"value":"1945","hits":2610},{"value":"1985","hits":2584},{"value":"1943","hits":2578},{"value":"1944","hits":2577},{"value":"1984","hits":2573},{"value":"1942","hits":2499},{"value":"1941","hits":2473},{"value":"1940","hits":2447},{"value":"1986","hits":2438},{"value":"1939","hits":2375},{"value":"1930","hits":2352},{"value":"1938","hits":2341},{"value":"1937","hits":2323},{"value":"1936","hits":2303},{"value":"2004","hits":2296},{"value":"2005","hits":2270},{"value":"1931","hits":2252},{"value":"1990","hits":2186},{"value":"1987","hits":2183},{"value":"1991","hits":2181},{"value":"1935","hits":2174},{"value":"2006","hits":2165},{"value":"1934","hits":2161},{"value":"1933","hits":2160},{"value":"1932","hits":2150},{"value":"1992","hits":2139},{"value":"1993","hits":2123},{"value":"1989","hits":1879},{"value":"1929","hits":1874},{"value":"1988","hits":1753},{"value":"1928","hits":1509},{"value":"1921","hits":1393},{"value":"1925","hits":1275},{"value":"1927","hits":1272},{"value":"1926","hits":1266},{"value":"1924","hits":1263},{"value":"1923","hits":1208},{"value":"1922","hits":1188},{"value":"1920","hits":1187},{"value":"2007","hits":1029},{"value":"2011","hits":1029},{"value":"2008","hits":1017},{"value":"2010","hits":992},{"value":"2009","hits":980},{"value":"2012","hits":967},{"value":"2013","hits":965},{"value":"2014","hits":924},{"value":"2016","hits":876},{"value":"1919","hits":850},{"value":"1918","hits":849},{"value":"1900","hits":831},{"value":"2015","hits":826}],"options":{"sort":"count","limit":100,"offset":0,"prefix":null},"min":"0193","max":"2035","count":301323,"missing":11},{"name":"medium_facet","items":[{"value":"correspondence","hits":9610},{"value":"documents (object genre)","hits":4200},{"value":"letters (correspondence)","hits":3564},{"value":"newspapers","hits":1925},{"value":"manuscripts (documents)","hits":1690},{"value":"records (documents)","hits":1429},{"value":"oral histories (literary works)","hits":1385},{"value":"reports","hits":1362},{"value":"clippings (information artifacts)","hits":1156},{"value":"articles","hits":903},{"value":"transcripts","hits":816}],"options":{"sort":"count","limit":11,"offset":0,"prefix":null}},{"name":"rights_facet","items":[{"value":"http://rightsstatements.org/vocab/InC/1.0/","hits":13560},{"value":"http://rightsstatements.org/vocab/InC-EDU/1.0/","hits":9168},{"value":"http://rightsstatements.org/vocab/UND/1.0/","hits":8110},{"value":"http://rightsstatements.org/vocab/CNE/1.0/","hits":6150},{"value":"http://rightsstatements.org/vocab/InC-NC/1.0/","hits":1673},{"value":"http://rightsstatements.org/vocab/NoC-US/1.0/","hits":1494},{"value":"http://rightsstatements.org/vocab/NKC/1.0/","hits":40},{"value":"http://rightsstatements.org/vocab/NoC-NC/1.0/","hits":20},{"value":"http://rightsstatements.org/vocab/InC-RUU/1.0/","hits":12},{"value":"http://rightsstatements.org/vocab/InC-OW-EU/1.0/","hits":2},{"value":"https://creativecommons.org/publicdomain/zero/1.0/","hits":1}],"options":{"sort":"count","limit":11,"offset":0,"prefix":null}},{"name":"collection_titles_sms","items":[{"value":"John J. Herrera Papers","hits":3271},{"value":"Strom Thurmond Collection, Mss 100","hits":2068},{"value":"Lillian Eugenia Smith Papers (circa 1920-1980)","hits":1887},{"value":"Office of Desegregation Management","hits":1843},{"value":"Isaiah DeQuincey Newman, (1911-1985), Papers, 1929-2003","hits":1717},{"value":"Augusta Baker papers, 1911-1998","hits":1696},{"value":"Memphis World","hits":1484},{"value":"James W. Silver Collection","hits":1430},{"value":"Sovereignty Commission Online","hits":1423},{"value":"Integration Correspondence","hits":1420},{"value":"Land of (Unequal) Opportunity: Documenting the Civil Rights Struggle in Arkansas","hits":1403}],"options":{"sort":"count","limit":11,"offset":0,"prefix":null}},{"name":"provenance_facet","items":[{"value":"John Davis Williams Library. Department of Archives and Special Collections","hits":3821},{"value":"South Caroliniana Library","hits":3419},{"value":"University of North Texas. Libraries","hits":3316},{"value":"Hargrett Library","hits":3287},{"value":"University of South Carolina. Libraries","hits":3250},{"value":"Butler Center for Arkansas Studies","hits":2174},{"value":"South Carolina Digital Library","hits":2167},{"value":"University of South Carolina. South Carolina Political Collections","hits":1793},{"value":"Atlanta University Center Robert W. Woodruff Library","hits":1663},{"value":"Rhodes College","hits":1641},{"value":"Mississippi. Department of Archives and History","hits":1435}],"options":{"sort":"count","limit":11,"offset":0,"prefix":null}},{"name":"class_name","items":[{"value":"Item","hits":40191},{"value":"Collection","hits":237}],"options":{"sort":"count","limit":100,"offset":0,"prefix":null}},{"name":"educator_resource_b","items":[{"value":"false","hits":40278},{"value":"true","hits":150}],"options":{"sort":"count","limit":100,"offset":0,"prefix":null}}]}}