Program evaluation

(ra' LITTLE ROCK SCHOOL DISTRICT INSTRUCTIONAL RESOURCE CENTER 3001 PULASKI STREET LITTLE ROCK, AR 72206 TO: Board of Education FROM: PREPARED BY: T. Kenneth James, Superintendent of Schools ij^onnie A. Lesley, Associate Superintendent for Instruction DATE: October 24, 2002 SUBJECT: Program Evaluation Agenda, 2002-03 Background Section 2.7.1 of the Revised Desegregation and Education Plan included the following obligation: LRSD shall assess the academic programs implemented pursuant to Section 2.7 after each year in order to determine the effectiveness of the academic programs in improving African-American achievement. If this assessment reveals that a program has not and likely will not improve African-American achievement, LRSD shall take appropriate action in the form of either modifying how the program is implemented or replacing the program. In response to Section 2.7.1 of the Revised Desegregation and Education Plan, the District implemented in 1999-2000 a new student assessment plan and proposed to the Board of Education a program evaluation agenda to include the most critical programs implemented pursuant to Section 2.7 and detailed in Section 5: elementary, middle, and/or high school English language arts and mathematics programs. Each year the agenda has also included the evaluation of one or more additional programssuch as the ESL program mandated by the Office for Civil Rights. Judge Wilsons Compliance Remedy A summary of the Compliance Teams interpretation of Judge Wilsons order of September 13, 2002, pertaining to the 2002-2003 programs to be evaluated is a follows: Continue to assess the programs implemented under 2.7 to improve the academic achievement of African-American students for 2002-03 and through the first semester of 2003-04. Since the District has now eliminated all fall testing, except for certain diagnostic tests that teachers administer without 1 Board of Education - Memo October 24, 2002 Page Two reporting to the district the results, this obligation to assess students ends at the end of the 2002-03 school year. The judge stated that he expected the District to use all of that available data and information in assessing the effectiveness of those programs and in deciding whether any of those programs should be modified or eliminated. Therefore, the program evaluations that will be completed as per the 2002-2003 program evaluation agenda will include all available data, including scores from previously administered fall tests, and they will all be completed by the end of the first semester of 2003-04. Each one will answer several research questions, including the one most critical to compliance, Was this program effective in improving and remediating the achievement of African American students? The Revised Desegregation and Education Plan obligated the District in Section 5 to assess students in the following programs, as follows: 5.2.1 5.2.2 5.2.3 5.3.2 Primary Reading/Language Arts. g. Monitor student performance using appropriate assessment devices. Intermediate Reading/Language Arts e. Monitor student performance using appropriate assessment devices. Secondary Schools Reading/Language Arts f. Monitor student progress and achievement using appropriate assessment devices. Mathematics Develop appropriate assessment devices for measuring individual student achievement and the success of the revised curriculum. The 2002-03 Program Evaluation Agenda outlined in this proposal includes these required components. Status of the Requirements of the 2001-02 Program Evaluation Agenda The Boards program evaluation agenda for 2001-02 was as follows: Primary Reading/Language Arts Middle and High School Literacy K-12 Mathematics and Science (CPMSA) English as a Second Language 2Board of Education - Memo October 24, 2002 Page Three None of the four program evaluations for 2001-02 has been completed as yet due to the tardiness of our receipt of the States Benchmark scores for literacy and mathematics in grades 4, 6, and 8. The scores arrived on Thursday, October 3, 2002. The Board received an update on the Early Literacy program evaluation in June 2002 based on the Observation Surveys and the Developmental Reading Assessment data. That update confirmed the findings of the 2000-01 report and also documented even higher achievement. The previous evaluation, along with the update, are on the Boards agenda for approval on October 24, 2002. I Staff have planned to produce a brief report with data and analysis as a beginning program evaluation of the grades 3-5, 6-8, and 9-12 literacy programs with a more thorough evaluation report to come at the end of 2002-03. However, without the Benchmark scores, that report has not yet been drafted. The Board has received annual reports on assessment results for each of these programs. Comprehensive program evaluations of the grades K-5 and 6-12 literacy programs will be produced at the end of the 2002-03 school year for Board approval. The CPMSA (NSF-funded project) program evaluation (the fourth annual) will be presented at the December meeting and then will be submitted to the National Science Foundation for their review and feedback. At that time, the staff will bundle each of the previous three annual reports produced thus far, along with the NSF feedback, for the Boards formal approval of these program evaluations. The ESL report was originally scheduled for an October presentation, but we have had to reschedule since we do not yet have the Benchmark data. We anticipate being able to present that study in November. The Office for Civil Rights in Dallas has been very complimentary of our 2000-01 ESL program evaluation and has asked us to assist them in providing technical assistance to other districts on how to conduct this study. When the 2001-02 report is presented, the staff will bundle the reports for 1999-2000 and 2000-01, along with the 2001-02 report for the Boards formal approval. 2002-03 Proposed Program Evaluation Agenda The District will provide for the evaluation of the following programs for 2002-03. 1. Elementary Literacy Staff will produce, with the assistance of an external expert, a comprehensive evaluation of the elementary literacy program (grades K-5) at the end of 2002-03. This study will include findings for the following four literacy programs being implemented in LRSD: Balanced Literacy (Early Literacy Learning in Arkansas or ELLA at grades K-2 and Effective Literacy at grades 3Board of Education - Memo October 24, 2002 Page Four 3-5), Balanced Literacy with Reading Recovery, Success for All, and Direct Instruction. This report will be completed and presented to the Board for approval prior to the winter break in 2003. 2. Secondary Literacy Staff will produce, with the assistance of an external expert, a comprehensive evaluation of the secondary literacy program (grades 6-12) at the end of 2002-03. This study will include findings for the Reading/Writing Workshop implemented at grades 6-8 and the English I Workshop implemented in three schools at grade 9. All available data will be used in determining the effectiveness of the overall program. 2. CPMSA (K-12 Mathematics and Science) Staff will issue a final report on and evaluation of the five-year NSF-funded project for grades K-12 mathematics and science, and it will be presented to the Board for approval prior to the winter break in 2003. When NSF (external experts) provides its feedback, that report will be added to the documents submitted to the court. Fiscal Impact The District will be able to complete all the program evaluation requirements outlined in the 2002-03 program evaluation agenda through funds already budgeted, except for the cost of the external experts who will serve on each team. The costs for external consultants to complete the 2002-03 program evaluations are not yet known since the District has not yet had the opportunity to identify who they might be and to negotiate contracts. Recommendations That the Board of Education approve the 2002-03 program evaluation agenda as outlined. BAL/adg 4LITTLE ROCK SCHOOL DISTRICT INSTRUCTIONAL RESOURCE CENTER 3001 PULASKI STREET LITTLE ROCK, AR 72206 TO
Board of Education FROM
PREPARED BY
T. Kenneth James, Superintendent of Schools ^^onnie A. Lesley, Associate Superintendent for Instruction Pat Price, Director of Early Childhood and Elementary Literacy DATE
October 24, 2002 SUBJECT
Early Literacy Program Evaluations Background The Board of Education approved in its 1999-2000 program evaluation agenda the Early Literacy program, which began implementation in fall 1999. During July and August 2000 the Assistant Superintendent of PRE presented to the Board drafts of this evaluation, which the Board tabled in August 2000 pending completion. That early draft was never completed and was not again submitted to the Board of Education for review and approval. During summer 2001 Dr. Bonnie Lesley, on behalf of the Early Literacy Program Evaluation team (Pat Price, Pat Busbea, Ann Freeman, Ed Williams, Ken Savage, Anita Gilliam, and Sharon Kiilsgaard) presented a completed 204-page program evaluation
Year 2 Evaluation
The Effectiveness of the PreK-2 Literacy Program in the Little Rock School District (1999-2000 and 2000-2001). This report was presented for information, but our Section 2.7.1 Compliance Plan now requires that all program evaluations be presented for Board acceptance and approval. Dr. Steve Ross of the University of Memphis had served as an external consultant to the team. He read both a near-complete draft and made several suggestions for its improvement, which were incorporated into the final draft. He also read the final draft and responded. All grades K-2 teachers administered the assessments, both fall and spring, in all three years, 1999-2000, 2000-2001, 2001-2002. All elementary principals supervised both the fall and spring administrations of the Developmental Reading Assessment and the Observation Surveys and the Achievement Level Tests at grade 2. Central office Elementary Literacy staff conducted the training for the assessments, collected the answer documents, and participated in the analysis of data
Patricia Price, Pat Busbea, Judy Milam, Judy Teeter, Kris Huffman, and Ann Freeman. Both Dr. Ed Williams and Board of Education - Memo October 24, 2002 Page Two Ken Savage assisted in the production and analysis of score reports. Anita Gilliam and Sharon Kiilsgaard assisted in checking the data tables for accuracy and in preparing the final reports. Copies of this program evaluation were provided to Mr. John Walker, to Ms. Ann Marshall at ODM, and to all elementary principals and elementary literacy staff. Executive summaries, including the program evaluation recommendations, were sent to all K-2 teachers with a cover memorandum congratulating them on their successes. The program evaluation was comprehensive, including the following: an introduction: a chapter on the literacy program design and its relationship to the Districts Strategic Plan and the Revised Desegregation and Education Plan
a description of all of the K-2 assessments used to measure student progress
a chapter aligning the program with national research studies on effective early literacy programs
numerous tables displaying the data in several different ways, disaggregated by grade level and race
an analysis of the results (based on student performance data)
an analysis of additional data relating to achievement gap among schools and the impact of professional development on student achievement
a chapter on findingsanswers to the six research questions originally posed
a bibliography
and tables of school-level data on each assessment for the two-year period. 1 An important chapter of the program evaluation relating to Section 2.7.1 of the Revised Desegregation and Education Plan was the one on findings. Research Question 2 was as follows
/s the new program effective in improving and remediating the academic achievement of African American students? The discussion filled pages 81-96. The following paragraph includes the criteria that were used to determine effectiveness. i To determine the effectiveness of the new program in improving and remediating the academic achievement of African American students, the District used the performance results of the Observation Survey and the Developmental Reading Assessment. The basic criterion established in determining program effectiveness for black students was that black student achievement would have to improve and then that growth over the two-year period of the programs implementation would need to be equal to, but preferably greater than, the growth of non-black students, (p. 81)Board of Education - Memo October 24, 2002 Page Three The report included a detailed analysis of all available data
The following findings based on Observation Survey, Developmental Reading Assessment, and Achievement Level Test results make it possible to conclude that the new early literacy program has so far been effective in improving and remediating the reading achievement of African American students, as well as all students. It is unusual in any District to find gains by both blacks and non-blacks over a two-year period on eight different measurements, as this study finds. Again, however, experts on program implementation advise that it takes approximately five years to determine program effectiveness, so this year 2 study at best establishes baseline and early trend data for comparisons in future years, (pp. 82-83) In this sections conclusions, the program evaluation included not only a summary of findings as they relate to the achievement gap, but also how they compare to the findings in recent national research on reading achievement among African American and white students: The results of two years of changes in the LRSD policies, programs, and procedures in grades PreK-2 indicate that both black and non-black children in the Little Rock School District are learning to read independently by grade 3 (see Section 5.2.1 of the Revised Desegregation and Education Plan). The findings and analyses in this report indicate trends in the opposite direction of the national research findings cited above and of scores of other similar studies. Instead of black students growing at slower rates than non-blacks, in most of the measurements the LRSD results indicate higher rates of growth of black students than non-blacks. Instead of the gap widening between grades 1 and 2 as it does in national studies, it narrows significantly in the LRSD by every one of the eight measures (five sub-tests of the Observation Survey, the Developmental Reading Assessment, and two sub-tests of the Achievement Level Test), (p. 94) Pages 107-113 included recommendations for improvement in instruction, parent involvement, interventions, and professional development. Five schools were identified for improvement since they were the lowest performing schools in at least two of the three grades tested. Recommendations for the next program evaluation were also included. These recommendations were all considered by the program staff and by school-level staff and many were immediately implemented, as well as others identified in formative evaluations during year 3. The major recommendations made to principals for program improvement included (1) ensuring that all teachers are fully trained and are implementing the District program
and (2) adding Reading Recovery and literacy coaches wherever possible, since both of these actions in some schools had resulted in higher achievement.Board of Education - Memo October 24, 2002 Page Four At the end of 2001-02 the staff decided that another comprehensive study was not necessary so early in the program's implementation (year 3). They, therefore, presented to the Board of Education in June 2002 an update that included all the 2001- 02 scores on the Observation Surveys and Developmental Reading Assessment, along with a summary of analysis of performance, especially comparisons of African American student achievement with other students. Those findings not only confirmed the findings of the 1999-2001 study, but the results were even stronger in year 3. At the end of year 3, African American students scores were at least 90 percent of other student scores on all five measures of the Observation Survey by the end of grade 2. In other words, the achievement gap was either closed on these measurements or almost closed, given the standard of 90 percent as an acceptable ratio. On the Developmental Reading Assessment, the most difficult of the measurements, the black to non-black ratio grew from 35 percent at the beginning of kindergarten in fall 1999 to 82 percent at the end of grade 2 in 2002. Deeper analysis also revealed that although many African American children from poverty were not learning to read in grade 1, they did successfully learn to read in grade 2, so they will most likely reach the goal of independent reading by grade 3, even though they began far behind their peers. Interestingly, the grovtrth of other students generally exceeded African American student growth on the DRA in grade 1, but African American growth exceeded other student growth in grade 2. Copies of the program evaluation and the update are attached for Board members' review. Recommendation That the Board of Education accept and approve, as submitted, the following: Year 2 Evaluation
The Effectiveness of the PreK-2 Literacy Program in the Little Rock School District, 1999-2000 and 2000-2001 Update on the Implementation of the PreK-2 Literacy Program, Little Rock School District, 1999-2000, 2000-01, and 2001-02 BAL/adg AttachmentsLITTLE ROCK SCHOOL DISTRICT INSTRUCTIONAL RESOURCE CENTER 3001 PULASKI STREET LITTLE ROCK, AR 72206 TO
Board of Education FROM
T. Kenneth James, Superintendent of Schools PREPARED BY
I Linda Watson, Assistant Superintendent ^bBonnie A. Lesley, Associate Superintendent for Instruction DATE
October 24, 2002 SUBJECT
Approval of the Charter School Program Evaluation Background Information Dr. Linda Watson and Ms. Krishna Young, former director of the LRSD Charter School, presented to the Board of Education in June 2001 the program evaluation for the Charter School. That report was presented as information, but the Section 2.7.1 Compliance Plan requires that the Board formally approve each of the program evaluations listed on page 148 of the Final Compliance Report. The Charter School Program Evaluation was prepared by Dr. Larry McNeal, Professor at the University of Arkansas at Little Rock. Serving on the team with him were Dr. Linda Watson, Ms. Krishna Young, and Dr. Ed Williams, members of the LRSD staff. All of the Charter School teachers, grades 3-5, participated in administering the assessments
the Achievement Level Tests at grades 3-5, the SAT9 at grade 5, and the State Literacy and Mathematics Benchmark examinations at grade 4, and the Success for All quarterly assessments in reading. The program evaluation included not only student achievement data, but also demographic data, student attendance rates, records of suspensions, student grades, and financial costs for the program. Performance data for the program evaluation were not disaggregated by race. The student body, however, was 87 percent African American. Due primarily to budget constraints, the District eliminated funding for the Charter School in summer 2002 after two years of operation, so this program has now been abandoned. Recommendation That the Board of Education accept and approve the LRSD Charter School Program Evaluation for 2000-2001. BAL/adg Attachment 1 -ZDOi. LITTLE ROCK SCHOOL DISTRICT INSTRUCTIONAL RESOURCE CENTER 3001 PULASKI STREET LITTLE ROCK, AR 72206 TO: Board of Education FROM: T. Kenneth James,. Superintendent of Schools PREPARED BY: Ionnie A. Lesley, Associate Superintendent for Instruction DATE: November 21,2002 SUBJECT: Approval of Program Evaluation for Southwest Middle Schools Partnership with Southwest Education Development Lab (SEDL) Background Information During 1998-99 and 1999-2000 SEDL formed a two-year research partnership with five schools in their five-state region. The project was entitled Facilitating Implementation of Reform Strategies and Tactics (FIRST). Southwest Middle School was the only middle school to be selected for participation. The focus areas for the project at Southwest were literacy achievement: mathematics achievement
student attendance
and student health, wellness, and safety. The SEDL staff published two evaluations of their work with the five schools and their findings relating to the effectiveness of certain school improvement efforts. Those two reports are attached. LRSD s PRE department through Dr. Ed Williams provided data over the two-year period as requested by SEDL for their analysis. In addition, SEDL conducted informal surveys, observations, and interviews and documented each schools conditions with journal entries (see p. 2, Volume 9, Number f. Issues about Change). SEDL staff, primarily Dr. DeEtte Cowan, conducted the study and wrote the program evaluation. Southwest Middle Schools principal and grades 6-8 teachers participated in data collection and in administering student assessments. Five core issues were identified as having significant impact on the five schools past and present efforts at improvement: organizational structures focus of improvement work personal and social dynamics contextual influences , leadership. Board of Education - Memo November 21, 2002 Page Two Although the purpose of the SEDL study was not specifically to determine the rrQ<'Tn/Anr,r.n _Z___ . rr . . . reform efforts on the academic achievement of African American children, whatever findings were made would be relevant t:..,, Z__L___' Schools students are almost all African Americans. The findings for Southwest found as follows in the Volume 9, Number 2 report: since Southwest Middle are organizational structuresp. 4, first column (year 1) and second column (Year 2) focus of improvement workp. 2, second column (year 1) personal and social dynamicsp. 5, second column (year 1) and p 6 first column contextual influencesp. 6, second column leadership p. 8, first column (year 1) and p. 8, second column (year 2) In conclusion, when improved academic achievement did not including Southwest, the researchers stated the following: occur in the five schools, While increased student achievement is the goal of any responsible school improvement effort, administrative and organizational difficulties must often be addressed before a coherent view of the student body and its needs can be formulated and connected to staff capabilities and goals. Left unaddressed issues of leadership, organization, and context, as well as personal and social dynamics can detail school improvement efforts and sap the energy of the most gifted and talented teachers. Most critically, if these /ssues are not addressed and a schoolwide improvement effort is not advanced, the quality of education individual students receive can become simply a matter of chance and class assignment (pp. 9-10, Volume 9, Number?). Recommendation That the Board of Education approve the attached research reports on the Facilitatinq Implementation of Reform Strategies and Tactics (FIRST) project / M C* . .Xi_________ X 1 I I I Al I. ... ' ' ' ..UHCU.CIUOUUII UI r\erorm iiraiegies and lactics (FIRST) project as the proqram evaluations for Southwest Middle School's participation in the SEDL partnership. BAL/adg AttachmentVolume 9, Number 2 2000 Issues . about Change Year One and Year Two: What Do You Do In Comprehensive School Improvement Introduction During the summer of 1998, the Southwest Educational Development Laboratory (SEDL) began a partnership project entitled Facihtating Implementation of Reform Strategies and Tactics (FIRST). A two-year initiative, FIRST partnered staff from SEDLs Strategies for Increasing School Success (SISS) program with staff at five schoolsone in each of the five states defining SEDLs service region^Arkansas, Louisiana, New Mexico, Oklahoma, and Texas. These initiatives would differ from former models of school improvement. FIRST partnerships would be long-term, broad scale, in-depth. u. joined at the hip partnerships focusing on the identified needs of the schools and including all relevant staff. Identified needs were not hmited to a particular program or organizational aspect of the school, but could encompass curriculmn, instruction, assessment, classroom management, professional development, parental and community involvement, school management, and a consideration of how these parts fxmctioned as a whole to create a particular school culture. For two years, SEDL staff provided information, guidance, and technical support for improvement efforts determined by the school personnel. FIRST schools collaborated with SEDL in conducting interviews, surveys, and observations about the course these improvement efforts took, including major accomplishments and stumbling blocks. The entire school program was examined and prioritized
specific academic areas and/or organizational structures were chosen as the focus of improvement work at each of the FIRST schools. The intent of the FIRST project was to develop the capacity of school personnel to plan, monitor, and continue improvement efforts. To that end, technical assistance providers were enlisted and coordinated with SEDL staffs work to assist the schools during the FIRST initiative. FIRST schools represented the regions diversity on many levels
three high schools (Banner, Community, and Pelican), one middle school (Tall Pines), and one K-8 school (San Fernando) were chosen. These schools, whose names are pseudonyms, served students across a range of ethnic and socioeconomic backgroimds, and struggled with issues both specific to their schools and common to many schools (e.g. low student achievement, lack of parent involvement). In the course of the research into school improvement strategies and particularly through the first year of experience in partner schools, SEDL staff identified and confirmed the primacy of five core issues for school improvement efforts. The staffs analysis of school issues and strategies for advancing school improvement was framed within these areas: focus of the improvement work, orgamzational structures that support school change, personal and social dynamics of the indi'viduals and organizations involved, the Southwest Educational Development Laboratory 211 East Seventh-Street, Austin, Texas 78701 (512) 476-6861widening circles of contexts that influence school work, and leadership that influence all of the preceding can Much of the first year of SEDL staffs work was devoted to becoming familiar -wi-th the school culture and context, and building relationships -wi-th school personnel. First year efforts tended to identify the issues that impeded school improvement. As these issues had often been hidden by routine and low expectations prior to the partnership, many participants became discouraged as improvement efforts caused these issues to emerge or re-emerge. I Major tasks facing the SEDL staff in the second year thus included maintaining momentum in ongoing changa efforts, continuing relationship-building, and celebrating accomphshments as they occurred. SEDL partners had bmlt trust -with school staff in their roles as external changa facilitatorsneutral parties in district, school, and interpersonal pohtics. In order to advance change efforts and maintain this trust, SEDL personnel focused their efforts on each schools identified needs, with particular attention to the five core school change issues. Focus of the Improvement Work Year 1. In order for partner schools to fully engage in the work of school improvement, the focus of the improvement work that was undertaken was identified and chosen with the full participation of staff at each school. The level and root of difficulty in achieving this objective varied from school to school. For example, Barmer High School was reconstituted during the first year of its partnership with SEDL. In its effort to provide students with a sense of community and to encourage personal relationships between students sind teachers, this single large high school was divided into four academies. The focus of improvement work quickly became apparent
implementation of the academies concepts and structures. At San Fernando School, SEDL assisted staff in re-viewing achievement data and performing action research. As a result of their learning, staff at San Fernando decided to focus improvement work on student retention of skills, and on curriculum, particularly in mathematics. The SEDL facilitator at Cnm-munity High School conducted student focus groups for faculty observation, supported professional development for faculty and administrators, and assisted in the collection and analysis of data from multiple so-urces in the community and from students. Freshman student success was chosen as the focus of improvement work, and the rest of the first year at Commimity High was spent identifying leverage points and making plans for implementation in the second year. At Pelican High School, issues of communication between the central office and the school inserted themselves into efforts to define the focus of improvement work. These issues were worked on as the facilitator introduced the study of student achievement data and its analysis, followed by tie-fining a -vision focused on student outcomes. While staff and administrators quickly agreed to focus efforts on improving instructional effectiveness, the consensus broke down when the SEDL facilitator pushed for more specifics. The staff ultimately decided to focus on planning^ as a way of addressing instructional effectiveness. Student achievement and other forms of data were collected by SEDL staff and utilized to inform staff at Tall Pines Middle School about the academic needs of students and the factors operating -within the school that impacted student learning. SEDL facilitated development of a shared -vision of exemplary SEDL 2schools through the distribution of researchbased studies on successful middle school reform. With this support and guidance, Tall Pines staff identified four areas for improvement
hteracy achievement, mathematics achievement
student attendance
and student health, wellness, and safety. Year 2. In Year 2, SEDL staff sought to ensure that students remained a visible and vital target of school improvement efforts. At Tall Pines, an administrator from a neighboring school district, trained by staff from SEDLs Program for Teaching and Learning (PITL), developed Uid delivered professional development for teachers that focused on what students learnednot what teachers taught. As part of their year-long staff development in mathematics instruction, teachers at San Fernando were trained in conducting student interviews, in order to gain insight into the process and success of student learning To discover and disseminate student priorities and perceptions, SEDL staff conducted student focus groups at San Fernando School, Community High, and Pehcan High. At Pehcan High School, development of a school improvement plan began as SEDL engaged Pehcan staff in a return to the schools migainn and vision, and led the staff in reflecting on the skills and attributes of an ideal Pehcan High graduate. - These efforts to keep school improvement work focused on students helped to diminish political issues and increase staff enthusiasm and commitment. At Pelican High School, for example, staff who had been adept at blaming external circumstancedistrict pohcy, student socio-economic background^for students lack of success began to see and take responsibility for their impact on students. Staff became both more accountable and more enthusiastic as they planned, implemented, assessed, and revised specific strategies for improved student achievement. SEDL made available the resources of both its SISS and PITL programs, creating and dehvering professional development activities specifi.c to each schools needs
conducting, analyzing, reporting and guiding revision in response to the assessment of school improvement strategies that were initiated
leading, planning, and attending meetings of school and/or district staff
and providing each school an observer and aUy both removed from divisive school issues and fuUy committed to school improvement and increased student achievement. In addition, SEDL staff brought in outside experts and assisted schools in accessing local resources, and devp1 oping relationships between school and district staff, and between the FIRST school and education professionals at neighboring schools and universities. Reflection. Lack of access to and understanding of student achievement data played a large role in the difficulties encountered as each partner school sought to define and maintain the focus of their improvement efforts. SEDL spent time at each school gathering existing data, collecting new data, and training school staff in interpreting data and identifying logical, research-tested strategies for school improvement In the absence of empirical information about their students' achievement, school staff had developed their own rationale for student achievement and lack of achievement. SEDL staff had to address these straw men,' ft including: student socioeconomic background
government-mandated program .q, standards, and measurements
interpersonal disputes
and other factors outside staff control. SEDL partners pushed school staff to assume responsibility for student learning, and to believe in their ability to positively affect that learning 3 SEDLOrganizational Structures Year 1. During Year 1, SEDL staff famiharized themselves with the organizational structures that existed at partner schools, and assisted in the development of orgamzational structures where there were none. 'The reconstitution of Banner High School into academies required that new communication structures be developed and implemented. Pehcan High School, which had recently been created by the division of a K-12 school into an elementary, middle, and high school, had a similar need to develop new organizational structures, particularly addressing commumcation between central office staff and the school. At Community High and San Fernando School, organizational structures that were already in place were strengthened and supported through the FIRST partnership. At Commumty High, the principal provided meals and, in some cases, stipends, in support of afterschool planning meetings among teachers
this was to change in year two. Facffitation of meetings at San Fernando School helped to assure that all staff were included in plannings and all voices were heard at staff meetings. At San Fernando, SEDL also assisted in the creation and operation of action research teams, which collected and presented the data utilized in developing that schools focus of improvement work. Organizational structures at Tail Pines provided little support for change. Systems for routine procedures, such as tracking attendance and communicating hallway duties, were lacking. Communication among teachers and between the school administration and teachers was sporadic and ineffective. Structures such as grade level teams and the schools steering committee, while in place, were not utihzed effectively. Grade level teams collaborated on student discipline, parent/teacher conferences, and planning for special events. The schools steering committee existed primarily to commumcate the principals unilaterally developed agenda to the rest of the staff. Expectations and opportunities for teacher leadership or problem-solving were nearly nonexistent. Year 2. During the FIRST initiatives second year, orgamzational structures remained a focus of attention at partner schools. SEDL facihtators sought to enhance structures and practices that worked and mitigate the effects of structures that did not exist or work well. When funds to provide meals and stipends for professional development dried up at Community High School, SEDL supported the principal in utilizing other school resources to support unprovement efforts. Communication via e-mail helped fill the gap created by fewer meetings
the schools video equipment (and students) provided a means to create videotapes of training activities. At Pelican High School, in light of a new school structure, and with the cooperation of a new administrator, SEDL staff supported school personnel in adapting necessary procedures to the new organization and structures. At Banner High School, FIRST efforts targeted one academy of the four (that understood the value of FIRST and articulated interest) created by the division of a large high school. The lack of organizational structure at Tall Pines Middle School remained a .signifirant obstacle to school improvement efforts. The SEDL facilitator at Tail Pines advocated for fuller use of existing organizational structures, and helped the principal realize the strength and skills of the campus leadership team. In addition, the facilitator created opportunities from external mandated policies
the district required the completion of a campus plana difficult task given the lack of organization and staff expertise that characterized Tall Pines. The SEDL coordinator volunteered to assist in this effort, and in the process modeled I i SEDL 4inclusion, data driven decisions, planning and communicationand embedded greater capacity for ongoing school improvement into Tall Pines future. District requirements provided accoimtabhity and legitimacy to school efforts at San Fernando School. Here, the SEPT, facilitator utihzed district pressure to align curriculum within the school, and with state assessments helped to unify teachers^by quelling resistanci in taking the next steps in the schools focus on improved mathematics instruction and achievement. Reflection. As a result of two years work in partner schools, SEDL staff learned that they had been too optimistic in terms of organizational structures at partner schools. Partner schools functioned with minimal organization, allowing informal networks and unspoken expectations to guide and define their work, and reinforcing the isolation of teachers in their individual classrooms. Without clear access to information, the means to express opinions, or the assurance that their perspective would be honored, teachers and staff retreated from one another, convinced themselves that school-wide improvement was impossible, and focused their efforts narrowly. SEDL facilitators found themselves working within organizational structures that were cumbersome and ineffective, or creating new organizational structures with personnel who were often skeptical and occasionally resistant. Facihtators found this work to be critical to maintaining improvement efforts, but also to be quite difficult, thankless, and slow to show benefits. improvement efforts at each school, advanced dynamics that supported those efforts, and most criticallyavoided being drawn into or ahgned with any one side of personal and social conflicts involving the partner schools. At Banner and Community High Schools, personal and socieil dynaTnirs were generally positive. Banner High School was marked by strong, clear, and widespread teacher conTmitrnp-nt. to and knowledge of students. For the most part, this commitment fueled supportive professional relationships between teachers, although there was little opportunity for teachers suggestions to be incorporated into administrative decisions about the new academy structure. At Commumty High School, trust in the principal translated into the ability for teachers to work well with one another, and to ignore small factions that arose in opposition to improvement efforts. In schools where the personal and social dynamics were less positive, SEDL facihtators sought to engage all staff in the improvement effortsand in the meetings eind decisionmaking that informed and shaped those efforts. Nudging teachers to learn one anothers names by asking a group who is missing provided a small but vital fotmdation for teachers to begin talking to one another about instruction at Pelican High School. At San Fernando, providing the opportunity for staff to discuss the issues that simmered between them allowed some of these issues to be resolved. At Tall Pines, attending to organizational structures helped to dispel some of the tensions that precluded the development of strong personal relationships among teachers and staff. Personal and Social Dynamics of Individuals and Organizations Year 1. SEDL facilitators observed the personal and social dynamics that impacted Year 2. For the most part, issues of personal and social dynamics were addressed only tangentially during the second year. SEDL staff maintained their focus on building relationships among teachers and between 5 SEDLteachers and adm i n i strators in the context of professional development and school improvement efforts. When facilitating meetings, SEDL staff worked to ensure that every voice was heard, and maintained professional neutrahty when tensions emerged. SEDL staff utilized emerging teacher leaders and other individuals invested in school change efforts to encourage schoolwide commitment to the project. In this project and in others, SEDL has learned that professional growth, focused efforts toward school improvement, and the academic success of students form the strongest bonds between school professionals. Banner High School and Tall Pines Middle School did provide notable exceptions, where personal and social d5mamics jeopardized improvement efforts. At Banner, three of the four academy principals began to isolate and undermine the fourth principal, and to reduce their commitment to the FIRST project. These three principals ultimately left the FIRST project, and the SEDL facflitator worked exclusively with that fourth principal, in the academy she administered. Issues related to leadership negatively impacted personal and social dynamics at Tall Pines Middle School. A school climate survey was conducted during each year of the FIRST projectall indicators fell in the second year, and the subscale of Collegial support fell most dramatically. Unfortunately, SEDL could do little to directly address these issues until issues of leadership improved. Reflection. Personal and social dynamics is perhaps the least distinctly bounded of the five critical areas the FIRST project identified and addressed. Problems that are rooted in leadership, context, and organizational structures almost always affect personal and social dynamics negatively. Similarly^ advancement in any of the four other areas tends to support positive dynamics. But, while SEDL facihtators acknowledged that personal and social dynamics tended to bleed into and out of the other critical areas, they found it remained an important area to consider independently. one When school personnel know and trust another, the work of school change gets easier. Communicating expectations of respect and participation, bringing parties in conflict together to dialogue, and teaching techniques for assuring full participation and equal representation helped to advance positive personal and social dynamics, which in turn advanced school improvement efforts. Contextual Influences Year 1. Each of the five partner schools operated within a different community and district context. At Community and Banner High Schools, contextual issues were minimal, and easily addressed through assuring clear communication between all stakeholders. This was a central tenet of the focus of improvement efforts, although some attention was required at Banner because of the schools reaction to the community and city politics that were at play. Contextual issues were most significant at Tall Pines, where new state and di strict, pohcies and mandated curricular changes created new roles, relationships, and responsibflities among administrntors, teachers, parents, and students. In addition. Tail Pines operated under a long-standing desegregation order that required careful scrutiny of any changes that might affect the racial composition of the school. A strong teachers union actively monitored the effect of these changes and FIRST improvement efforts on teachers work and responsibflities. At Tall Pines, SEDL sought to connect all these factors to develop a coherent improvement effort. SEDL 6 1Similarly, tensions between the digfrict- and school at Pelican High School were addressed in the first year by providing opportunities for representatives of each orgamzation to meet, dialogue, and problem- solve. The establishment of regular meetings had a powerful and immediate impact on communication and trust between school and district personnel. The students at San Fernando School provided this school its most challenging contextual issue. While most of the core staff at San Fernando were Euro-American, the majority of instructional assistants and students were Native American and T-Tispanic SEDL staff worked to assure that the voices of instructional assistants were included and honored at staff meetings, and brought in research-based materials on connecting school curricula to students ethnic and cultural backgrounds. Year 2. SEDL facilitators continued to assist school personnel in identifying and responding to a variety of contextual influences. Peril a pi more critically, SEDL facilitators sought to maintain school focus on improvement goals and specific activities toward those goals, in order to prevent contextual influences from becoming excuses for stasis. SEDL helped school personnel learn to define their real IS spheres of influence and to handle issues of context. They also provided technical support in the completion of campus improvement plans, offered advice on utilization of Title I funds, and guided professional development in areas of multiculturalisni and age-appropriate teaching strategies. Reflection. While schools are the location of improvement work, they are critically affected for better or worse by the context in which they exist. External change facilitators must have a wide and deep range of strategies and uiformation in oi
der to be ready to anticipate and address issues of context during improvement efforts. Bringing all parties to the table, where possible, is the best first strategy in addressing contextual issues. In aU cases, more and better communication, and more and better understanding mitigated the negative aspects of context, and allowed stakeholders to begin to imagine context as a strength of, and not a hindrance to, their school. Leadership Year 1. Gathering an accurate impression of each partner school was the first step for SEDL facihtators across each of the five critical arpas for school improvement. Doing so quickly was particularly important in the area of leadership. While shared leadership is acknowledged as a pwerful form of school administration, at the FIRST partner schools, the principals retained most of the power, made most of the admimstrative decisions unilaterally, and thus wielded tremendous influence on the staffs perception of the FIRST initiative and their willingness to fiflly participate. At Community High School, relationships between staff and a d m i m'.gtr a tors were generally positive, and the Assistant Principal was particularly enthusiastic about the FIRST project. His enthusiasm would eventually lead him to overload the project with mitiatives, but at the beginning it served to create momentum, interest, and commitment among school staff. During the first year, tensions between the four academy principals at Banner High School came to a head. When the fm-mer superintendent had made them equals as administrators, she had not provided them with any model for operating as such. The resulting power struggle defused improvement efforts and negatively impacted interpersonal dynamics. The SEDL facihtator sought to assuage hurt feelings and discover and i 7 SEDL Ldisseminate leadership models that might work between these four administrators. She was unsuccessful at finding such a model, and ultimately decided to focus the FIRST project on only one of the academies. Leadership at Tall Pines was a very apparent area for potential improvement. The principal, though well meaning, seemed not to understand the function or practical value of shared leadership. While both a steering committee and campus leadership team were in place, there were no clearly defined areas of responsibihty for each or between the two. The principal rarely shared substantive decisionmaking with either body, and in fact appointed some members of the campus leadership team despite district guidelines calling for their election. When the principal did delegate responsibility, he neither monitored nor followed up to gauge progress or to identify how he could support staff efforts. As a result, many tasks were never completed, or if completed, were not recorded. Within this environment, the SEDL facihtator sought to develop a shared focus and a sense of self-efficacy among the staff. While this approach yielded enthusiasm in small group settings, plans were often jettisoned in responses to some crisis or other, and no coherent improvement plan could be developed and maintained. The principalship at San Fernando School and Pehcan High School changed hands after the first year of the FIRST initiative. At each of these schools, SEDL facihtators began again with new administrators, and built upon the relationships they had established with school staff. At Pehcan the facihtator was able to establish an immediate positive relationship with the new principal, while at San Fernando, teacher leaders maintained continuation of the project and specifically asked the new principal in the hiring interview if she was willing to support the SEDL project. In both instances, this turnover negatively impacted the momentum of the project, but did not completely erase the achievements nor void the plans made for implementation of school improvement efforts. Year 2. SEDL staff worked with FIRST school principals where they were, and in some cases, where they were not. When three of four principals in Banner High Schools new academy structure evidenced a lack of interest in or focus on how SEDL might support them, SEDL shifted its focus to full, supportive cooperation with the one principal who remained active and interested. At Tall Pines Middle School, the need for better management of routine procedures impeded efforts at communication, change, and improvement. The SEDL facihtator worked to develop leadership skills of the principal and of school staff. She advocated for utilizing existing structmes to share decision-making with the staff, and by the end of the project year, the campus leadership team was more involved in important decisions about school personnel and pohcies. In addition, the SEDL FIRST facihtator ultimately met with the school principal behind closed doors and confronted him about the need for stronger management and greater administrator visibihty in the school. The principal was able to accept this counsel, and made changes. At Community High School, one principals enthusiasm for the focus on freshman students led him to overbuild that program, nearly to the breaking point. The SEDL facilitator at Community High advocated for the staff and brought this principal to an awareness ofand sense of humor about^his tendency to take on too much. f i i At two FIRST schools, the projects second year began with new principals. SEDL facihtators took responsibihty for educating these fldTnini.qtrat.ors on the history, purposes. I SEDL 8and progress of the FIRST initiative. At Pelican High School, SEDL consistently supported and advanced the strengths of the new principal, even in trying times of adjustment. The SEDL facilitator pushed the new Pelican principal to define his vision of leadership, and supported the principal in implementing that vision throughout the predictable highs and lows of adjusting to a new school and a new assistant principal. At San Fernando school, the SEDL facilitator explained the staffs choice of mathematics as a school focus, and supported the principal in advancing this focus even as the district pushed for a shift to reading. The SEDL facihtator helped to assure the principals interest in professional development and assessment were incorporated into the school improvement plans, and reminded the principal that change takes time. When this principal also left the school, SEDL began again with San Fernandos third leader, explaining the FIRST initiatives focus, detailing the history and achievements of the staff, and offering continued assistance. Reflection. Through FIRST and other school improvement initiatives, SEDL has developed an abiding respect for the role of leadership in any school change effort. These efforts advance most effectively and smoothly in schools where principals are committed to high quality mstruction leading to success for every student
are adept at handling both day-to-day operations as well as the crises that routinely break these routines
enjoy strong working relationships with district and school staff
and have both the professional security and commitment to advance and utihze teacher leadership. Unfortimately, principals with such broad and deep strengths are few and far between. In addition, all school leaders subject to relocation, retirement, and are reassignment. Shifts in leadership, even when anticipated, can have profound, lingering, deleterious effects on teacher morale and school improvement efforts. SEDL supported principals who had skills and strengthened the skill base of principals who struggled with leadership. Developing personal relationships with these administrators was an important first step. Once this foundation was estabhshed, SEDL facilitators shared professional literature on leadership with principals and coached the principals on leadership strategies ranging from use of active verbs and first person plural (in their communication with teachers) to implementing significant shared leadership. SEDL also sought to connect these school administrators with a wide web of ongoing support, and so assisted in building relationships between FIRST school principals, and supported their professional development and attendance at national and local conferences. Conclusions Through the FIRST initiative, SEDL has developed a body of research on school improvement efforts that includes close study of five schools in the process of change, a widening library of strategies for supporting school change, and confirmation of the critical role change agents can play in schools undertaking improvement and change. As outside agents, change facihtators can develop a clearer view of dynamics that support and impede change efforts, and provide and build resources and abihties. In addition, they can offer assistance that is free of existing power relationships and requirements, can advocate for aU school personnel, md, most importantly, can maintain the focus of improvement efforts on unproved instruction and increased student achievement. While increased student achievement is the goal of any responsible school improvement effort, administrative and organizational difficulties must often be addressed before a coherent view of the student body and its needs can be formulated and connected to staff capabihties and goals. Left unaddressed, issues 9 SEDLof leadership, organization, and context, as well as personal and social dynetmics can derail school improvement efforts and sap the energy of the most gifted and dedicated teachers. Most critically, if these issues are not addressed and a schoolwide improvement effort is not advanced, the quality of education individual students receive can become simply a matter of chance and class assignment. School change is a daunting proposal, and school change professionals must develop and utilize a deep and broad variety of strategies for assisting schools in change and improvement. Change facilitators must be able to respond to the particular issues of a school, and must be willing to devote time and attention to developing a clear understanding of the schools readiness and its cultural ethos, in order to adapt and implement change strategies that are specific to each schools circnmstance In addition, change agents must maintain an awareness of more universal issues in education^including administrative turnover, fluctuating funds, and student populations that are increasingly diverse and face increasing demands, both academic and personal. i i 1 j i Issues.-.about Change is published twice a year by Southwest Educational Development Laboratory. This issue was written by Melissa Capers, consultant
D'Ette Cowan, SEDL Program Associate
and Grace Fleming, Tara Leo, and Melanie Morrissey, Program Spedahsts, SEDL. SEDL I OERI This publication is based on work sponsored wholly, or in part, by the OfBce of Educational Research & Improvement, U.S. Department of Education, under Contract Number RP91002003. The contents of this publication do not necessarily reflect the views of OERI, the Department, or any other agency of the U. S. Government. This publication may be reproduced and copies distributed to others. Please acknowledge SEDL as the source on all copies. I i I I I J i SEDL 10Voltime 9, Number 1 2000 Issues . . about Change Comprehensive School Improvement: Addressing the Challenges The report of the 1996 National Commission on Teaching and Americas Future bore a strong message regarding the need for educational change
School transformation cannot succeed unless it focuses on creating conditions in which teachers can teach and teach well. This report, coupled with current nationwide interest in encouraging schools to adopt comprehensive reform strategies or programs, urgently communicates the need for school improvement. Such reform may not be much to ask of schools that are already supported by adequate funding, continuous professional development programs, and active parent involvement. Many schools dont fit that picture however, and are crying out for help. It is with these schools, the ones that are struggling to meet the needs of their students, that SEDL has engaged in the Facilitating Implementation of Reform Strategies and Tactics (FIRST) project. The goals of this paper are to provide a deeper understanding of how schools experience comprehensive reform and to identify the issues that affect schools efforts at improvement. Taking a Different Approach This project took a systemwide look at comprehensive school improvement while simultaneously working with schools that were undertaking reform efforts. For the purposes of this work, comprehensive school improvement is an inclusive term for engaging an entire school staff in an in-depth study of the teaching and learning process. Emphasis is placed on the examination of all aspects of the school curriculum, instruction, assessment, classroom management, professional development, parental and community involvement, school management, and so on and identification of how all of these parts can work together to improve student results. SEDL formed in-depth partnerships with five schools, one in each state of the educational laboratorys diverse service region. These partnerships were intended to differ from former models of school improvement in three specific ways. First, the focus of attention and assistance was on the entire school program and all factors that have the potential to affect student learning. This work was in no way to be a quick fix to show a fast turnaround in standardized test scores. Rather, the intent was to develop the capacity of school personnel and to promote their engagement in continuous improvement. Second, the existing and proposed structures and practices at each school site were examined in terms of their expected benefits for student learning. Each schools needs were viewed as individual and unique, which required the creation of a program tailored to that specific site and a process to address specific areas of need. Third, multiple technical assistance providers were coordinated to assist in the improvement efforts at each site during the FIRST project, and also beyond that time, to enable staff to stay informed and current in their practice. Again, because of the individual needs of each school site, the providers and the assistance that they offered varied. Developing Alliances with Schools Similarities existed among the schools in that each entered the partnership acknowledging that it was at risk of fading to meet the learning needs of its students, and each lacked experiences in school change. More important, staff at the five sites expressed their interest and commitment to the partnership as a means of producing meaningful changes in their schools and positively affecting student results. Southwest Educational Development Laboratory 211 East Seventh Street, Austin, Texas 78701 (512) 476-6861Together, the sites- displayed characteristics that are symptomatic of the challenges in pubhc education across the nationachievement scores were consistently low or falling, students were unhappy and/or unmotivated, parents were ignored, community members were disengaged, and school staff did not beheve they could affect student learning. The sues varied in terms of geography and demographics (race, ethnicity, socioeconomic status, students cultural background), as well as in their capacity for reform planning and implementation. What We Are Learning SEDL staff conducted ongoing informal surveys, observations, and interviews, and documented each schools existing conditions with journal entries. In the process of reviewing the first years work across all five sites, the researchers found five core issues, each of which had signifirant impact on the schools past and present efforts at improvement: organizational structures focus of improvement work personal and social dynamics contextual influences leadership. While the school sites themselves were more dissimilar than similar, the five core issues were factors at each site, to varying degrees. Such commonalities motivated SEDL staff to examine more thoroughly the core issues and their impact on each of the schools. Each of the core issues is thematic, encompassing a number of related areas in which the schools have needed assistance. The issues are highly interactive and interrelated, some to a stronger degree than others. Growth and progress at each site required that the external agent be aware of the five core issues. Actions were taken to nurture, support, and encourage positive developments within each of the core issues, in attempts to build up the capacity of staff and administration and to develop a system that is supportive of change. Findings regarding these issues have evolved from the experiences, observations, and documentation of work done at the five sites by SEDL staff and by the shared observations and commentary of the school staffs. This paper will clarify and discuss each core issue, using observations from the FIRST sites. While the situation in the schools with respect to these core issues does raise concerns, SEDL staff feel it is increasingly important to assess the overall picture, to acknowledge school staffs weaknesses along with their strengths, in order to knowledgeably facihtate school change. While SEDL staff continue to work on these issues with schools, it is hoped that other external school improvement facihtators will benefit from the insights provided regarding the challenges that all schools may face as they undertake comprehensive reform. Organizational Structures With appropriate structures and processes in place, effective schools run efficiently. At the FIRST schools, many of the necessary organizational supports were lacking, and the result was disorganization, unclear directions and processes, few to no avenues for problem solving or collaboration among staff, and frustrated teachers. Three specific areas within the organizational structures of schools were identified as areas that needed attention: time, communication, and organization. Finding time. One of the schools had regularly scheduled in-services for the whole staff to meet for professional development or collaboration. At the other four sites, faculty meetings were often held after school and were kept short to stay within state or union guidelines. Such meetings functioned primarily as a time to address administrative items, rather an to provide an opportunity for staff to come together as a whole for learning, problem solving, or decisionmaking. The result was fragmented understanding of the schools vision or collective purpose and continued isolation of teachers within their own classroom, grade level, and/or subject area. f 1 Though in-service days and/or daily planning periods of 45 minutes or longer were scheduled at each school site, the staffs tended to use the time SEDL 2 Iindependently for grading, planning, etc. No times or structures were designated specifically to facilitate collaboration among teachers. The inabihty to find time and/or the inefficient use of time greatly affected staffs opportunities to discuss issues regarding their vision, their goals, the school, the students, and the curriculum. 1110 result at each of the sites was a disconnection of purpose, intent, and action. Staff and administrators at each site identified a need for making time to work together as a staff and for using allotted time efficiently. This particular issue required ongoing negotiation and creative problem solving between SEDL staff and the campus administrator(s). In some instances, the large size of the staff and limited time constraints have meant that only, part of a staff was together at any one point. Within this configuration, the staffs were just learning how to use their time most productively so that their collective work has a positive impact on student learning. Communication. Although aU the schools could identify some form(s) of communication between administrators, staff, students, and parents, several of them did not have a regular means of daily or weekly communication of events. Rather, they rehed on word-of-mouth messages and/or PA announcements, which were disruptive and frequently consumed valuable classroom instruction time. Each of the schools lacked efficient methods for regular, ongoing communication regarding events whether scheduled or unscheduled. Such inadequate communication often resulted in last minute scrambhng on due dates, changes in staff or student meetings, and in hasty decisionmaking. At three of the schools in particular, information from the central office was not communicated to staff and/or administrators effectively or in a timely manner. Since much of the communication between schools and the central office is in the form of requests, concerns, or questions, the lack of efficient systems for such communication led to misinformation and confusion. Several of the school sites did not have any consistent means of communicating calendar events to parents, and so parental involvement in school activities was limited. School stakeholdersthe students, parents, and community members were rarely informed of or invited to participate in school activities, especially those that required decisionmaking. Teachers contact with parents was generally in the context of reporting concerns or grievances regarding their students. Parents and surrounding community members were viewed as unsupportive of schools efforts, and yet the staffs did not know how to nurture more positive relationships. Organization. In the time that SEDL staff spent in the schools during the first year, it appeared that few organizational systems were in place for making requests, identifying concerns, allocating materials or resources, or handling necessary paperwork. At one school site, it was not unusual for administrators to be unaware of a students location during the day, since no system existed for accessing students class schedules. If a parent came to pick up his or her child, or the administrator wanted to talk to a particular student, school office personnel would have to interrupt instructional time by making a call over the PA system asking the child to report to the office. Most office personnel and administrators had not designed an efficient system for filing or accessing pertinent information, whether it was a state mandate regarding curriculum, personnel information, or student records. Focus of Improvement Work Maintaining an undeviating focus on students is central to identifying and articulating purposeful intent for any schools reform work. Such a focus was lacking at the sites. Often, small groups or individuals appeared to have a grasp of the overall intent of improvement work at the sites, but staffwide common focus and effort were not apparent. Also significant were the low levels of teacher empowerment found within these schools teachers ability and willingness to access information, identify needs and potential solutions, and engage in self-study were hmited. The result was inconsistency of purpose, mixed messages, and inefficient implementation of instructional strategies across the sites. Four specific areas of need 3 SEDLwere noted this first year: data analysis, problem solving, access to inrormation/resources, and conflict resolution/celebration. Data analysis. SEDL staff involved each school staff in examining their schools strengths and areas of concern, engaging the entire staff in data collection, analyzing trends, and developing hypotheses. Four of the five sites found it diffimit to provide recent student achievement data for this process, often not having the scores from the past years assessments on hand at the school. Accessing longitudinal dau for interpretation and analysis was even more difficult. When SEDL staff did access student achievement data, the staff were not clear about the usefulness of information gained from examining these data. Teachers and administrators exhibited limited understanding of alternative student assessment techniques, and this restricted their ability to accurately identify the needs of their students. Therefore, SEDL staff began to teach the school staffs how to interpret and analyze testing data. Once data were accessed, and teachers were taught how to read the scores and use them to identify strengths and needs, dialogue about the impact of instruction in the classroom began. Collaborative problem solving. Beyond the examination of data is the opportunity for school staffs to engage in dialogue regarding the needs of their students and their school. Before estabhsh- ing a partnership with SEDL, the five school sites had devoted little to no time to such discourse. As noted in the Organizational Structures section above, there was limited time provided for collaborative work. When administrators or staff attempted to discuss needs, it was often in a context of hurried decisionmaking, without referring to data or acknowledging everyones concerns. Dialogue techniquesallowing everyones voice to be heard without judgementwere not employed at any of the school sites. No clear norms were established for the school personnel to use in group discussions. The result was that a few vocal staff members at each school site were heard, while the test of the group was quiet and less involved. Often the more vocal staff members were aware that not everyone was involved. and yet did not seem to know how to address the problem. On the other hand, the quieter, less involved staff members spoke of feehng alienated from the discussions and decisionmaking, and therefore were admittedly less committed to making things work. The schools, in general, did not spend time identifying the potential for their students, their school, or themselves. Staffs were continually inundated by demands to improve student test scores, improve discipline referrals, or improve something else that was seen by someone else as unsatisfactory. School staffs did not spend any time identifying their strengths, or their vision for their students, schools, or themselves, and therefore they could not collectively relate to their successes or to their potential for improvement. Access to information/resources/training. Only one of the schools had access to and made use of available technical assistance in the form of training or resources. The other sites have either had difficulty in the past accessing outside resources, were unaware of what assistance was available to them, or simply could not identify what kind of assistance would benefit them. Directly related to the schools need for a vision and goals for their improvement work, the schools staff did not regularly seek information by tapping into research or literature regarding best practices. Staffs were more likely to attempt implementation of programs that another school in the district was doing, or what they heard from other teachers about what was working in their schools, rather than investigating the claims first. School staffs often did not receive adequate training in the programs they were trying to implement. An example of this was the implementation of block scheduling at one high school. Though the block scheduling strategy was adopted three years ago, the staff never received training regarding their instructional practices while teaching in longer periods. Conflict resolution and celebration. In any organization involving creative and energetic individuals, a certain degree of conflict will be present. Such friction does not result in negative attitudes or perceptions when there are clear norms and strategies for resolution in effect. SEDL 4These school sites, however, all struggled with conflict and they had limited resolution strategies in place. The results ranged firom the development of factions within a school staff, to complete ignorance of conflict, to individuals resigning their positions and leaving the school or district. Celebration strategies were very limited at the school sites. Since the schools seldom acknowledged their progress, there was no apparent need to celebrate accomplishments, learning, or growth. When one high schools state assessment scores significantly increased this year, the SEDL staff member who had written congratulatory notes to the instructional teams was informed by teachers that the note was the only acknowledgment they had received upon learning of the improved scores. Without celebration of even the little things, staff motivation was low, which in turn affected the students perceptions of school and learning, resulting in disenfranchised staff and students. Personal and Social Dynamics A trustful culture, mutual respect and regard within relationships, and collective engagement of staff and administrators are key components of effective cultures within schools. The personal and social dynamics at these sites varied substantially. Trustful culture. Change of any kind is a very difficult process. SEDL staff recognize that when a group has personal or information concerns, it is unlikely that sustainable progress will be made until those concerns are resolved. At some schools, the staff were open with each other, and a certain level of trust had been established over time. At other sites, however, the culture was distrustfulor at best, unsupportive of staff-wide openness and respect. At each of the sites, there was the need for SEDL staff to establish norms with the group about working together and set some precedence regarding group involvement. Several of the schools displayed a pervasive feehng of distrust toward district office staff. The causes of the distrust are unknown, but the lack of trust and respept significantly affected the ability of the staff to learn to work together with district staff. Relationships. The development of a trustful culture requires strong professional relationships, and the key to developing those ties is to strengthen the personal relationships as well. Too often, the workplace is seen as the place for work, and there is no acknowledgment that everyone has a life outside. Each of the five school sites was limited in the development of relationships among staff members. There were very few opportunities, either within school or outside of it, for staff to do fun things together, learn together, laugh together, or just get to know each other. Relationships that were nurtured occurred primarily, in small groups in grade level or subject area, because of proximity in location or similar scheduling. The groups that did engage in these types of interactions and relationship building worked more effectively together within the school as a result. Collective engagement. Since the school staffs had had few experiences of working together in these schools, it is understandable iat they had not had many opportunities to experience differences, develop mutual regard, or engage in collective learning. Little to no work had been done with school staffs to acknowledge and value the differences in culture, experience, and expertise that they brought to the school environment. Due to the limited interactions between staff, opportunities for building trust and collegial growth were hindered. In several instances, pockets of staff members had worked together over a long period of time and had established some trusting relationships. In only a few instances, however, did staff use these relationships to engage in learning with and from each other regarding classroom practice. Contextual Influences A school does not operate separate or apart from surrounding entities. Four specific areas were found to have the most direct impact on the school staff and their improvement efforts: the school itself, the community, the district, and the state. School context. The most apparent issue at two of the five sites was the quality and maintenance of the facilities and grounds. Each day students 5 SEDLcame to a school that was not well cared for, and the result was a continuing lack of respect for the facilities, displayed by ripped wallcoverings, beat- up lockers, trash on the floors, writing on the desks, and general classroom and hallway disrepair. Such an environment had become so common to school staff that little was done to address the issue of facilities maintenance, either among themselves, with the custodial staff, or with the students. Upon entering the buildings, one encountered an environment that was dismal, unkempt, and drabnot an atmosphere that would encourage positive self-esteem, communicate value and respect, or nurture pride. and parents, which further alienated the two groups and kept them from developing positive relationships and understanding. Finally, the schools teachers and administrators had low expectations with regard to themselves as professionals and as self-learners. This was com- mtmicated in many ways, but the most obvious to the teachers at several of the sites was the reality that resources, materials, and training were not available. Although each of these schools struggled with low funding for such items, teachers perceived the lack of supplies and opportunity as indications of disrespect for their work. Deeply intertwined with the context at the school sites was. the quality of relationships between the students and the staff. At several of the schools these relationships were noticeably strained, and poor commimication, behavior, and morale were the result. In classes and while engaging in one-on-one conversation with students, teachers were frequently disciplining students rather than refocusing them on their work or encouraging then- creativity in class. Students were overheard complaining about how the teachers treated them, and they rebelled by acting out in class or skipping classes altogether. There was significant emphasis on maintaining control through discipline. It was questionable, however, whether such tactics had the intended positive effects on student/staff relationships and student learning. Directly related was a comment made by staff and students alike
We dont talk to each other. Since most students see their teachers more than they see their own parents, such unsupportive relationships can be detrimental to student learning, self-esteem, and personal growth. In three of the five schools, there were significant attitudes and/or beliefs among the school staff affecting perceptions of students, parents, and community. Issues of culture, race, and education surfaced in many overt ways. Staff did not appear to acknowledge or understand the cultural beliefs or environmental situations of their students, and community members. This affected the students and their families perceptions of the worthiness and value of the school and staff. Sometimes school staff communicated their own economic or educational superiority to students There were also low expectations held for the students as learners and for the district/community/state as viable support systems. Community context. Every school ftmctions within a community, which can maintain a limited undersunding of the school, how it works, and what its impact is on students. The community comprises the parents of schoolchildren and also the area businesses that support and prosper from the education of the communitys students. To varying degrees, each of the sites struggled with relationships within the community. It was not uncommon to hear that the community held the school in less-than-supportive regard. Often articles appeared in the local media that reflected negatively on the school, the staff, or the students. Active community members and board members at several sites voiced their concerns about their schools effectiveness publicly, which served to further deepen the divide between the school and the community. However, the school staffs engaged in little outreach to encourage more parental or community understanding. The general attitude of the school staffs seemed to be, Theres nothing we can do about it anyway. f f Staff at each site discussed the importance of involving parents and community members, informing them of the work that is done at the school, and enlisting their assistance, but at most of the sites, staff experienced difficulty with this component and were unable to overcome their discomfort in working with parents and community members'. i I SEDL 6District context. Insofar as schools work within a larger system of education, they must be responsive to the requests, mandates, and desires of that surrounding system. District offices, charged with communicating state requirements, often make demands on schools regarding policies, curriculum, discipline, and professional development. With regard to policies, administrative demands, record keeping, facility maintenance, access to data, and availability of resources and materials, the numerous interactions between the schools and their district offices were less than smooth. Until the partnership with SEDL, most of these school sites took ho action to improve the lines of cornmunica- tion between themselves and the district offices. State context. The states demands on these school sites either have changed significantly within the last few years (with the adoption of a new accountability system, for example), or are frequently changing. Therefore, clear communication of pohcies, adoptions, and mandates is very important. While district offices are often the voice of such communication, the inconsistency of messages and constant changes from the state departments continued to cause schools to struggle. Leadership The most critical of the themes emerging from the first year of work was the leadership capacity of the principals. Such administrative development includes the principals ability to communicate a clear vision, inspire others to maintain high expectations, create strong organizational systems for themselves and the school, understand what is possible regard- mg improvement, and develop a culture of murntil respect and regard. In other words, leadership capacity has significant impact and influence on the other four core issues. Clear vision. Strong leadership is a necessary component for successful school reform. In order to provide such leadership, administrators need to be clear in identifying the vision they have for their school, their staff, their students, and themselves. The administrators at these sites simply held a common vision to improve achievement scores. Although this goal is certainly desirable for each of these schools, it was unclear how the administrators envisioned achieving it, and why that goal would be importantboth'necessary components of a strong vision. Without identifying a shared focus for improvement, administrators could not guide their staff in developing and articulating a collective vision for their students or their school. This lack of clarity made it difficult for the administrators to model the image through his or her actions with staff, students, parents, and community. Without strong vision as a path toward improvement, the a schools often lost their way. 11 Expectations. Closely related to the vision are the expectations that a school leader communicates to his/her staff and students. At these school sites, high expectations were rare. There is some relationship between the Personal and Social Dynamics and Contextual Influences in this subcategory, as expectations are often based on historical norms and professional relationships. In the case of one high school, the historical norms took precedence over the principals desire to set higher expectations. Being a new principal, he deferred to the existing norms rather than estabhsh- ing his own strong expectations directly related to a clear vision for improvement. Several of the administrators at the sites spoke of high expectations for staff and students, but they rarely modeled or followed through on such expectations. Decisionmaking. As discussed in the Organizational Structures section, there were few clear procedures for decisionmaking at the school sites. The absence of decisionmaking structures prevented teachers from being involved in long-range planning and resulted in unilateral decisions made by the administrator(s). If issues were brought to the staff, they were often voted on without accurate or thorough information. Organization. School administrators at several of the sites had difficulty organizing the daily tasks and paperwork with a user-friendly system. It did not appear that office personnel were utilized effectively for organizational assistance, and it was difficult to locate something when it was needed for a teacher, a parent, or district office staff. This lack of organization was apparent in plaiming efforts, in meetings, and in daily work. 7 SEDLSystems for communication among staff and between school and home were also inadequate. Both of these forms of communication were mentioned in the Organizational Structures section. Knowledge. One of the most important roles of a school leader is to function as a model for learning. It is imperative that administrators have a deep understanding of student learning and of teaching for learning, best practices, and current educational research. Such a foundation allows them to recognize and model strong teaching strategies for teaching staff. It is also important that administrators be familiar with state and district curriculum expectations and be able to communicate them to teachers and parents. We found administrators to be willing to pursue their own learning in these areas, but too overwhelmed by the daily routine to devote any time to increasing their own understanding. Administrators who were involved in their own advanced studies were more likely to be current with best practice and aware of developing research. This strongly supports the idea that administrators need to pursue opportunities for their own professional growth, in order to increase the effectiveness of their leadership. Administrators who valued continuous learning were more likely to make the cormection between teacher efficacy and continuous improvement in their schools. They were also more likely to admit they didnt always have the right answers and to encourage teacher leadership among their staff. It is crucial that administrators be able to model the will to develop the skill. Culture. The role of the administrator includes nurturing a positive, learning culture of mutual respect and regard among staff setting high expectations, but it also requires commitment to the vision of such a culture. At the schools studied, administrators did not address the culture among the staff and students. In some cases, they appeared oblivious to the needs of the staff or students, or unable to figure out how to respond to those needs. Some of the teachers felt that the principal didnt really know what'was going on in classrooms, but should, and in some cases, students echoed that sentiment. These teachers wanted to see the principal around the school on a daily basis. Administrators also needed to develop skills that would allow them to use resolution or mediation strategies appropriately in times of conflict, whether it was between students, parents, or staff. Too often, conflict was left to resolve itself and became detrimental to the school and/or the staff. Some of the unproductive norms at these sites had arisen from unresolved conflicts that were buried and in turn, had festered into a bigger issue. Finally, the culture of a school includes the parents, and few administrators were comfortable or skilled in communicating clearly with parents and/ or community members. In some instances, parents had been caUing directly to the district office regarding issues or concerns, since they did not feel they received adequate responses from the school administrator. In Summary Although much of the work being done in schools today is called comprehensive, in fact many efforts continue to focus on a quick fix to improve student achievement results rather than addressing the system as a whole and building up the parts in need. Such reforms may suffice in the short term, but they can rarely be sustained over time, or through administrative turnover, staff changes, or legal mandates. i SEDLs FIRST project attempted to address the needs of low-performing schools at a system level and to increase the capacity of staff to address continuous improvement for the purposes of increasing student learning. In order to do that, however, it was necessary to evaluate how these schools were functioning regarding their student results, the staffs professional development, and their capacity for growth. i i This project has found that schools that fail to meet the achievement needs of their students often also lack the necessary structures and skills for initiating and sustaining continual, growth and improvementsupportive organizational structures, focus for the work, attention to human dynamics, ability to wor^ within multiple contexts, and highly skilled leadership. In order to support meaningful t I 1 SEDL 8growth and change over time, significant attention must be devoted to strengthening the schools capacity in each of these areas. Successful comprehensive reform work will necessarily include such a focus. Although results of the identification and discussion of core issues in these schools seem discouraging, we have found them to be an accurate representation of the challenges currently confronting schools that undertake comprehensive school reform efforts. The next Issues.. .about Change paper will report the actions taken by SEDL and school staff at these sites to address the five core issues that emerged in this work. References National Commission on Teaching and Americas Future. (1996). What matters most: Teaching for Americas future. New York: Author. SEDL is addressing the challenges that accompany efforts at school reform. The difference between the FIRST project and previous reform efforts lies in this acknowledgment and identification of the issues that affect the schools past and current efforts to make changes. Without addressing the underlying issues, reform efforts will merely scratch the surface and are unlikely to be sustainable over time to benefit student learning. Identification of the core issues provides insights about schools current conditions while engaged in comprehensive reform efforts and proves invaluable in determining capacity strengths and needs at each of the school sites. Comprehensive school reform as it is defined here- ingaging an entire school staff in an indepth, broad-scope examination of the teaching and learning process and working with them to improve student outcomesplants seeds of change that will continue to grow beyond the limits of this project. We hope that other external school improvement facilitators will be informed by the findings of this work and will use this information to promote school staffs grovrth and learning more effectively, and thus have stronger impact on building schools capacity for continuous improvement. We have much yet to learn. True comprehensive reform requires a thoughtful, reflectively adapting pace. As observers and participants in this process, we are learning that continuous reform can be encouraged by practicing tolerance for the investment of time that is necessary, and by nurturing continuous development within the core issues that surface in the process. Issues... about Change is published twice a year by Southwest Educational Development Laboratory. This issue was written by Melanie S. Morrissey, Program Specialist, Strategies for Increasing School Success at SEDL. SEDL I OERI This publication is based on work sponsored wholly, or in part, by the Office ofEducational Research & Improvement, U.S. Department of Education, under Contract Number RP91002003. The contents of this publication do not necessarily reflect the views of OERI, the Department, or any other agency of the U. S. Government. This publicationmay be reproduced and copies distributed to others. Please acknowledge SEDL as the source on all copies. 9 SEDLRECEIVED DEC 1 1 2002 A.n Individual Approach to a World of Knowledge OFRCEOF DESEGREGATION MONITORING December 3, 2002 DNiaOilNOW N011VD3a03S3a 30331330 Ms. Ann Marshall Office of Desegregation Monitoring One National Plaza 124 W. Capital, Ste. 1895 Little Rock, AR 72201 2002 I I 330 a3AI333a Dear Ms. Marshall: So that you are aware, we are presenting to the Board of Education for approval at the December 19, 2002, meeting the 2001-2002 evaluation of the Alternative Language Program for limited-English proficient students. Since it is not relevant to African American student achievement, we have not included a copy for you. If you would like to have one. however, just give me a call. We are also requesting the Boards formal approval of the first three years (1998- 99. 1999-2000, 2000-01) of the program evaluations of the Comprehensive Partnerships for Mathematics and Science Achievement, along with the feedback we received on each from the National Science Foundation, and then, for each, the next years Strategic Plan for the project. We are including in that package the program evaluation for 2001-2002 and will be asking the Board to review and formally approve that document as well. Copies of all the CPMSA documents are included in this mailing for your information, although all the documents for the first three years have already been provided to you as part of the exhibits for the court. If you have questions, please let us know. As a review of the progress we have made thus far, the Board will have approved, by the end of December 2002, six of the 14 program evaluations listed on page 148 of the Final Compliance Report: 1. Pre-Kindergarten through Grade 2 Literacy. 1999-2000 and 2000- 2001. plus the Update provided to the Board in June 2002formally approved at October 2002 meeting
810 W Markham Little Rock, Arkansas 72201 www.lrsd.kl2.ar.us 501-324-2000 fax: 501-324-2032 Mr. Ann Marshall December 3, 2002 Page Two 2. Charter School (first year)formally approved at October 2002 meeting
3. Southwest Middle Schools Partnership with Southwestern Education Development Lab (Austin)formally approved at November 2002 meeting
4. Collaborative Action Team (also a partnership with SEDL)formally approved at November 2002 meeting: 5. 1999-2000 and 2000-2001 Evaluations of the Alternative Language Programformally approved at November 2002 meeting: the 2001- 2002 evaluation will be approved in December
6. 1998-99, 1999-2000, 2000-01, and 2001-02 evaluations of the Comprehensive Partnerships for Mathematics and Science Achievementwill be formally approved at December 2002 meeting. Yours truly. Bonnie A. Lesley, Ed. D. Associate Superintendent for Instruction Enclosures BAL/adg cc: Kenneth James Chris Heller Clay Pendley Junious Babbs Sadie Mitchell Don Stewart John Walker The University of Memphis Memphis, Tennessee 38152-3340 A State of Tennessee Center of Excellence Center for Research in Educational Policy 325 Browning Hall Local 901/678-2310 Toll 866/670-6147 FAX 901/678-4257 October 28,2002 Director of Procurement Little Rock School District 1800 East Sixth Street Little Rock, AR. 72202 RECEIVED y. / 2p eV? DEC - 4 2002 desegregation MONITORING Dear Mr. Paradis, Enclosed are five copies of the Center for Research in Educational Policys response to RFQ #23-010: Revised Desegregation and Education Plan Program Evaluation Consultant. If additional information is needed or if I can be of further assistance, please contact the Center toll free at 1-866-670-6147. Sincerely, Steven M. Ross Director / A Tennessee Board of Regents Institution An Equal Opportunity/Affirmative Action University Friday Eldredge & Clark HERSCHEL H. FRIDAY (1922-1994) WILLIAM H. SUTTON. P.A. BYRON M. EISEMAN. JR.. P.A. JOE D. BELL. P.A. JAMES A. BUTTRY. P.A. FREDERICK S. URSERY. P.A. OSCAR E. DAVIS. JR., P.A. JAMES C. CLARK. JR.. P.A. THOMAS P. LEGGETT. P.A. JOHN DEWEY WATSON, P.A. PAUL B. BENHAM HI. P.A. LARRY W. BURKS. P.A. A. WYCKLIFP NISBET. JR.. P.A. JAMES EDWARD HARRIS. P.A. J. PHILLIP MALCOM. P.A. JAMES M. SIMPSON. P.A. JAMES M. SAXTON. P.A. J. SHEPHERD RUSSELL III, P.A. DONALD H. BACON. P.A. WILLIAM THOMAS BAXTER. P.A, RICHARD D. TAYLOR. P.A. JOSEPH B. HURST. JR.. P.A. ELIZABETH ROBBEN MURRAY. P.A. CHRISTOPHER HELLER. P.A. LAURA HENSLEY SMITH, P.A. ROBERT S. SHAFER, P.A. WILLIAM M. GRIFFIN HI. P.A. MICHAEL S. MOORE. P.A. DIANE S. MACKEY. P.A. WALTER M. EBEL HI. P.A. KEVIN A. CRASS. P.A. WILLIAM A. WADDELL. JR.. P.A. SCOTT J. LANCASTER. P.A. ROBERT B. BEACH, JR.. P.A. J. LEE BROWN, P.A. JAMES C. BAKER. JR.. P.A. HARRY A. LIGHT. P.A. SCOTT H. TUCKER. P.A. GUY ALTON WADE, P.A. PRICE C. GARDNER. P.A. TONIA P. JONES. P.A. DAVID D. WILSON. P.A. JEFFREY H. MOORE. P.A. DAVID M. GRAF. P.A. ATTORNEYS AT LAW A LIMITED LIABILITY PARTNERSHIP www.fridayfirm.com 2000 REGIONS CENTER 400 WEST CAPITOL LITTLE ROCK, ARKANSAS 72201-3493 TELEPHONE 501-376-2011 FAX 501-376-2147 3425 NORTH FUTRALL DRIVE, SUITE 103 FAYETTEVILLE. ARKANSAS 72703-4811 TELEPHONE 479-695-2011 FAX 479-695-2147 CARLA GUNNELS SPAINHOUR. P.A. JOHN C. FENDLEY. JR.. P.A. JONANN ELIZABETH CONIGLIO. P.A. R. CHRISTOPHER LAWSON. P A. FRAN C. HICKMAN. P.A. BETTY J. DEMORY. P.A. LYNDA M. JOHNSON. P.A. JAMES W. SMITH. P.A. CLIFFORD W. PLUNKETT. P.A. DANIEL L. HERRINGTON. P.A. MARVIN L. CHILDERS K. COLEMAN WESTBROOK. JR. ALLISON J. CORNWELL ELLEN M. OWENS JASON B. HENDREN BRUCE B. TIDWELL MICHAEL E. KARNEY KELLY MURPHY MCQUEEN JOSEPH P. MCKAY ALEXANDRA A. IFRAH JAY T. TAYLOR MARTIN A. KASTEN BRYAN W. DUKE JOSEPH G. NICHOLS ROBERT T. SMITH RYAN A. BOWMAN TIMOTHY C. EZELL T. MICHELLE ATOR KAREN S. HALBERT SARAH M. COTTON PHILIP B. MONTGOMERY KRISTEN S. RIGGINS ALAN G. BRYAN LINDSEY MITCHAM SLOAN KHAYYAM M. EDDINGS JOHN F. PEISERICH AMANDA CAPPS ROSE BRANDON J. HARRISON RECEIVED DEC - 4 2002 fje/, 208 NORTH FIFTH STREET BLYTHEVILLE. ARKANSAS 72315 TELEPHONE 870-762-2898 OF COUNSEL B.S. CLARK WILLIAM L. TERRY WILLIAM L. PATTON. JR. H.T. LARZELERE. P.A. JOHN C. ECHOLS, P.A A.D. MCALLISTER FAX 870-762-2918 OFFICE OF DESEGREGATION MONITORING December 4, 2002 JOHN C. FENDLEY, JR. LITTLE ROCK TEL 501-370-3323 FAX 501-244-5341 fendleyQfec.n*! ( By Hand Delivery ) Mr. John W. Walker Mr. Sam Jones Mr. Steve Jones John W. Walker, P.A. 1723 Broadway Little Rock, Arkansas 72201 Wright, Lindsey & Jennings 2200 Worthen Bank Bldg. 200 West Capitol Little Rock, Arkansas 72201 Jack, Lyon & Jones, P.A. 425 W. Capitol, Suite 3400 Little Rock, Arkansas 72201 Mr. Richard Roachell Roachell Law Firm ( By Hand Delivery) Ms. Ann Marshall Mr. Dennis Hansen Plaza West Building 415 N. McKinley, Suite 465 Little Rock, Arkansas 72205 Desegregation Monitor 1 Union National Plaza Ofc of the Attorney General 323 Center Street RE: 124 W. Capitol, Suite 1895 Little Rock, Arkansas 72201 200 Tower Building Little Rock, Arkansas 72201 Compliance Remedy Dear Counsel & Ms. Marshall: Enclosed please find, Guidelines for Completing Eight Program Evaluations in the Little Rock School District prepared by Dr. Steven M. Ross. The Little Rock School District intends to retain Dr. Ross, Dr. Larry McNeal and Dr. William Moore to complete the eight evaluations identified on page 148 of the Little Rock School Districts March 15, 2001 compliance report that were not completed with the assistance of an outside expert. These were identified on page 12 of the Compliance Committees proposed compliance plan previously provided to you. F \HOME\BBrown\Fendky\LRSD\deseg\counseI2 It.wpd/ All Counsel December 4, 2002 Page 2 Little Rock School District hopes that its decision to follow the guidelines prepared by Dr. Ross will deviate the Joshua Intervenors concerns about the preparation of these evaluations. If not, we respectfully request that ODM schedule facilitation related to the preparation of these evaluations as soon as possible so that the District may meet the courts March 15, 2003, deadline for Board approval and submission of the evaluations to the Court. Also enclosed for your reference are the responses to the RFQ submitted by the experts identified above. Sincerely, John C. Pendley, Jr. JCF/bgb enclosure(s) cc: Dr. Ken James F:\HOME\BBrown\Fcndley\LRSD\dcscg\counsel2 k.wpd1 Guidelines for Completing Eight Program Evaluations in Little Rock School District Prepared by Steven M. Ross, Ph.D. The present guidelines are based on my review of the Revised Compliance Plan, the LRSD standards for program evaluation, and evaluation report drafts and associated materials related to the eight programs identified as requiring final evaluation reports. My analysis of this material, combined with my experiences as an educational researcher and familiarity with the Joshua case as it affected LRSD, was influenced by the following assumptions: Invalid or questionable evaluation results can be much more detrimental than helpful to efforts to improve educational practices, and should not be disseminated without strong cautions and qualifications. Accordingly, studies that lack proper controls against bias or contamination from extraneous factors (e.g., differential sampling, history, diffusion of treatments) have limited value for guiding policies. Program evaluations that focus predominately on student achievement outcomes while lacking sufficient implementation data have reduced value due to inability to determine the nature of the treatment. The study will also fail to inform policymakers about the practicality of the program, how it was used and reacted to by stakeholders, or whether and/or how it needs to be improved to impact at- risk learners. Evaluations of programs that have been discontinued in the district are of much less interest relative to ones that are presently being implemented or informing ongoing practices. To raise the achievement of African American students in LRSD, attempting to resuscitate existing studies that have insufficient data available, limited relevance to cunent practices, or require substantial time and resources with little promise of yielding useful information for policy decisions would be less productive than employing the lessons learned from the prior evaluation work to support high quality and informative future studies. One such lesson is that the LRSD research department (formerly PRE) was understaffed to perform evaluations of the quality and quantity needed. Based on the above assumptions, I will recommend below a basic strategy for the third-party evaluators to use in preparing the eight identified evaluations for approval by the school board. Four of the evaluations concern programs that are no longer in use by LRSD and have limited or no relevance to programmatic decisions (Lyceum Scholars, Elementary Level Summer Schools, Vital Link, and Onward to Excellence). Of the remaining four evaluations, two have limited available data (Middle School Transition and Campus Leadership Teams) that, even with supplementary analyses, would not permit confident (valid) decisions to be made about program effectiveness 2 in general or about African American student achievement resulting from program participation. A seventh evaluation (Extended Year Education) could possibly yield informative evidence about an ongoing program, but to be sufficiently refined would require time and resources extending significantly beyond the current conditions for project completion. An eighth evaluation (HIPPY) also deals with an ongoing program, but unlike the others could possibly provide useful evidence through revisions completed within the available time frame. Accordingly, the HIPPY report is currently being rewritten by Dr. Ed Williams from LRSD. The suggested plan for the third-party evaluators is presented below followed by a brief review of each evaluation. A. Submit the current evaluation report as an attachment to a supplemental document as described in B-D. B. The supplement should begin with an expanded description of the program, its goals, and its history in LRSD. It should then describe the evaluation methodology and summarize and interpret the key findings. C. Most importantly, the supplement should discuss the limitations (and any strengths where indicated) of the evaluation with regard to: (a) informing current practices in LRSD
(b) using appropriate methodology
and (c) addressing student achievement effects, especially in reference to African American students. D. Finally, the supplement should present suggestions for conducting stronger studies of similar programs in future evaluation studies. 1. Middle School Transition (Moore) This^evaluation is in near-completed form and needs mostly editing and expansion. Because the middle school program is current and continuing, this evaluation study can be useful (mostly for guiding professional development and implementation improvement) for informing district strategies. The achievement results are fairly minimal and uninformative, but at the time of the evaluation (1999-2000), only baseline data existed. Thus, aside from providing additional description of the results (the tables and the narrative are sparse) and a more meaningful interpretation of trends (especially with regard to African American vs. Caucasian students), there is probably little more that needs to be done for this essentially baseline time period. The survey data appear to be reasonably analyzed and reported, but the interpretation and discussion should be extended to provide more meaningful conclusions and recommendations. Suggestions'. The third-party evaluator should follow the basic strategy outlined in the introductory section.3 2. Lyceum Scholars (McNeal) The Lyceum Scholars High School Program, which was evaluated in 1998-99 and 1999- 2000, is no longer being implemented in LRSD. The latter consideration, coupled with the obvious limitations of the evaluation design with regard to rigor, depth, and meaningfulness of the data, substantially reduce the value of the study and the need for devoting more than minimal resources to it, beyond perhaps a supplemental summary and explanation. Suggestions: The third-party evaluator should follow the basic strategy outlined in the introductory section. 3. Elementary Level Summer School (McNeal) Similar to the Lyceum Scholars High School Program (#2 above), the Elementary Level Summer School program is no longer being implemented in LRSD. In addition, the evaluation study conducted in the summer of 2001 is limited in its design and methodology. Among the major concerns are the lack of: (a) implementation data to describe the program strategies and the degree to which they were actually used by teachers, (b) an adequate control group or norms to which the achievement scores of summer school students could be compared, and (c) qualitative data to describe the experiences of students and teachers in the program. Due to differential sampling the multiple tables provided are neither overly meaningful nor informative regarding the progress of summer school students in general and African American summer school students in particular. Seemingly, there is little useful information to be gained for informing future policies by. investing substantive resources in revamping the study. While more suitable control samples might be established using archival data, the absence of implementation assessments would still make the treatment essentially unknown. Therefore, suggestions similar to those made for the Lyceum Scholars program are also offered here. Suggestions: The third-party evaluator should follow the basic strategy outlined in the introductory section. 4. Vital Link (Ross) The Vital Link program, designed to provide students with on-the-job experiences, was offered to 394 middle school students in the summer of 1999. Because the program was of very limited duration (only one week) and is not focused on either academic curriculum or learning strategies, it is highly unlikely to have affected students academic achievement. Although such a program would still potentially serve a useful purpose for fostering student motivation to achieve and complete school, it is no longer being implemented in LRSD. Further, the evaluation study conducted was so limited (a brief post-test only, closed-ended survey) that the policy implications of the results are minimal and even potentially misleading if derived. Therefore, suggestions similar to 4 those made for the Lyceum Scholars Program and the Elementary Level Summer School Program (#s 2 and 3 above) are again offered here. Suggestions: The third-party evaluator should follow the basic strategy outlined in the introductory section. 5. Onward to Excellence CSRD Program (Ross) The OTE model was implemented at Watson Elementary School for several years, starting in 1999. It has since been discontinued and was never formally evaluated, except for achievement data reports sent by the principal to ADE. Thus, in essence, there is no longer any program in LRSD to evaluate and no evaluation report to revise, expand, or redraft. It would seem wasteful of resources to reexamine historical data from this program, especially since implementation data are lacking. That is, if positive or negative results were found, it would be impossible to determine whether OTE or numerous others factors were the main cause. Suggestions, therefore, are similar to those for #s 2-4 above. Suggestions: The third-party evaluator should follow the basic strategy outlined in the introductory section. 6. HIPPY (Ross) Because HIPPY is a continuing program, this evaluation can be potentially useful to LRSD by providing initial program results on student achievement and benefits to African American children. A limitation of the study, which unfortunately cannot be remedied retroactively, is the lack of implementation data to describe the fidelity with which HIPPY program components were actually used. The quantitative achievement results must therefore be viewed cautiously, but should still be at least suggestive regarding program influences. Substantive expansion and revision, however, are needed to increase the readability and meaningfulness of the report. For example, there is inadequate description of the program, context, methodology, and analysis design. Tables and findings need to be presented in a more readable (user-friendly) manner. Suggestions: A. Reorganize and expand the introduction and methodology to be in line with district evaluation standards (i.e., more context, more detailed methodology, clearer questions and organization). B. Ed Williams needs to run the revised analysis and write up results by January 31,2003. A program description needs to be provided. Results need to be disaggregated, if possible, for African American and Caucasian students. Expand the Results sections to provide more informative reporting of outcomes, clearer tabular presentations, etc.5 C. Expand the Conclusions section to: (a) directly address whether there are implications for the achievement of African American and other disadvantaged groups (there probably are not at this stage), (b) more fully discuss implications and recommendations associated with the findings, and (c) propose further evaluation research that will validly determine both implementation quality and influences of HIPPY on student achievement. D. The third-party evaluator should follow the basic strategy in expanding this report. 7. Extended Year Education (EYE) Report (Moore) The EYE program is relevant to LRSDs current interests in improving academic achievement of its students. Unfortunately, the present evaluation design does not seem sufficiently sensitive to detect effects that might be attributable to EYE. Specifically, usage of whole-school data compared descriptively to district norms gives only a very surface examination of the schools progress, with susceptibility to contamination by student mobility, differences in SES, etc. A more precise analysis would match students at the three schools to similar students at comparable schools not using EYE, and then examine progress using a multivariate-type (regression or MANOVA) analysis. It is questionable, however, that such analyses could be completed in the time remaining for the required submission of the final report. Also, the findings would be limited by having only two years of post-program data. Aside from the design limitations, the organization of the report is difficult to follow due to the many tables and brief but not very informative narrative descriptions. The survey data might be interpretable, but also need a much clearer and better organized presentation. Suggestions: The third-party evaluator should follow the basic strategy outlined in the introductory section. / 8. Campus Leadership Teams (Ross) This initiative seems highly relevant to current and future goals of LRSD. However, the evaluation data collected to date consist of only results from two district-wide surveys that assessed team members reactions to various activities. No information exists to verify the representativeness of the samples, the validity of the data collection in general, or the implementation of the CLTs at the various schools. The aggregate survey results on the 24 combined items (14 in the team member survey
10 in the certified/non- certified staff member survey) do not appear overly interesting or meaningful with regard to informing practice. Suggestions: The third-party evaluator should follow the basic strategy outlined in the introductory section.JOHN W. WALKER SHAWN CHILDS John w. Walker, P.A. Attorney At Law 1723 Broadway Little Rock, Arkansas 72206 Telephone (501) 374-3758 FAX (501) 374-4187 OF COUNSEL ROBERT McHenry. P A DONNA J. McHENRY' S210 Heoterson Ro.ad Little Rock. .Aekans.as 72210 Phone: (501) 372-3425 Pjs (501) 372-3428 Email: mche1u7d@swbell.net Via Facsimile January 6, 2003 Dr. Bonnie Lesley Associate Superintendent for Instruction Little Rock School District 3001 Pulaski Little Rock, AR 72201 RECEIVED JAN -8 2003 OFFICE OF DESEGREGATION MONITORING Dear Dr. Lesley: On December 10, 2002 you mailed me a package which I have just received. I enclosing a copy of the address page for your information. am Please note that you have had my correct address for years. I am sending a copy of this letter to Judge Tom Ray because I am not certain whether there are any time response requirements involved, but I do not wish to be disadvantaged by receiving your document almost a month after its mailing. Very truly yours. -''^John W. Walker JWW:js Enclosure cc
Mr. Chris Heller Ms. Ann MarshallBonnie Lesley gl 0 W. Markham IRC Little Rock, Arkansas 72201 >t4 www.lrsd.kl2.ar.us 501-324-2000 501-324-2032 (M C. 19 sw SBfflS K.h't- Hl i<!
'Ox .J/ X An Individual Approach (o a ]]7orUofKnonledff I : Mr. John Walker 3601 S. Broadway UUle Ror
k, AR 72206received FEB 1 1 2003 An Individual Approach to a World of'Knowledge OFFICE OF DESEGREGATION MONITORIHG February 10, 2003 Mr. John Walker Attorney at Law 1723 S. Broadway Little Rock, AR 72206 received FEB 1 2003 DESEGREGOAFTFIIOCN** OMFO NITOWNG Dear Mr. Walker: I am attaching four items that will be on the Board of Educations agenda for approval at their February 27, 2003, meeting. The four items are as follows: 1. 2. 3. 4. Approval of the Elementary Summer School program evaluation Approval of the Lyceum Scholars program evaluation Approval of the Extended Year Education program evaluation Approval of the Middle School Transition program evaluation The cover memoranda and attached program evaluations are included for your review and information. Please give us a call if you have questions. Yours truly, A Bonnie A. Lesley, Ed. D. Associate Superintendent for Instruction cc
Ken James Chris Heller Junious Babbs, Jr. Don Stewart Sadie Mitchell /Ann Marshall BAL/adg Attachments 810 W Markham Little Rock, Arkansas 72201 www.lrsd.kl2.ar.us 501-324-2000 fax
501-324-2032 u I f 7 -fi 'Xw Individual Approach to a World ofKnowledge January 9, 2003 RECEIVED Mr. John Walker 1723 Broadway Little Rock, AR 72206 JAN 1 4 2003 OFFICE OF DESEGREGATION MONITORING Dear Mr. Walker: I received your letter admonishing me for sending my December 10, 2002, mailing to you to an incorrect address. I apologize profoundly, and I want to explain. The address of the packet to 3601 S. Broadway, rather than to 1723 Broadway, was a clerical error. In every other time that I have drafted a letter to you, I have included the address in my file in my draft and then sent it to my administrative assistant to complete and mail. In that particular letter, I just asked her to address it to you and to Ms. Marshall at ODM, and I failed to list the addresses. The address she had in her file was the old one. Neither of us knew that the mistake had occurred until we returned to work on January 6 after the winter break. When the person who delivers mail told me that it had been returned that Monday, 1 was very concerned and immediately requested that he hand-deliver the packet to you that day, which he did. That the package arrived in your office almost a month after its initial mailing, please note, was the result of our being out of the office for the winter break. The letter and accompanying document were related to the Boards approval at their December meeting of the program evaluations for the 2000-2001 ESL program and for the 1998-99, 1999-2000, 2000-01, and 2001-02 mathematics/science programs (CPMSA). You were advised earlier that these two programs would be considered at that meeting, and, of course, you received the Board agenda materials. Again, 1 apologize for the error, and 1 hope that you see that I sent the information and materials in good faith. Yours truly, L>- Bonnie A. Lesley'Ed. D. Associate Superintendent for Instruction BAL/adg cc: Kenneth James Chris Heller I'Ann Marshall Don Stewart Junious Babbs Sadie Mitchell 810 W Markham Little Rock, Arkansas 72201 6 www.lrsd.kl2.ar.us 501-324-2000 c fax: 501-324-20326^ 810 West Markham Little Rock, AR 72201 Direct Phone: Communications Office: (501) 447-1030 (501) 447-1025 DATE: February 13, 2003 TO: Central Arkansas Media Cynthia Howell, Arkansas Democrat-Gazette FROM: Suellen Vann, Director of Communications SUBJECT: Special School Board Meeting MESSAGE: The Little Rock School District (LRSD) Board of Directors will hold a special meeting Thursday, February 13, in conjunction with its agenda meeting. Tire special meeting agenda will include tire review of program evaluations and employee hearings. Tlie meetings will begin at 5:00 p.m. in the Board Room of the LRSD Administration Building, 810 West Markliam. Pages (including cover) 1 To Fax * An Individual Approach to a World of Knowledge *A Ze % tl 1 An Individual Approach to a World ofKnowledge February 10, 2003 RECclVED Mr. John Walker Attorney at Law 1723 S. Broadway Little Rock, AR 72206 FEB 1 1 2003 OFFICE OF DESEGREGATION MONITORING Dear Mr. Walker
I am attaching five items that will be on the Board of Educations agenda for review/ approval at a special session on February 13, 2003, the evening of their agenda meeting. (We are asking them to approve four of the eight remaining program evaluations at their agenda meeting and the last four at the regular February meeting.) The five items are as follows: 1. Guidelines for Completing Eight Program Evaluations in the Little Rock School District: Fiscal Impact 2. Approval of the HIPPY program evaluation 3. Approval of the Campus Leadership program evaluation 4. Approval of the Vital Link program evaluation 5. Approval of the Onward to Excellence program evaluation Please note that all four of the programs evaluated were previously abandoned by the District either due to d ata analysis that suggested program ineffectiveness or due to fiscal constraints. The cover memoranda and attached program evaluations are included for your review and information. Please give us a call if you have questions. truly, Yoprs truly. Bonnie A. Lesle/ Ed. D. Associate Superintendent for Instruction cc: Ken James Chris Heller Junious Babbs, Jr. Don Stewart Sadie Mitchell i/Ann Marshall BAL/adg Attachments 810 W Markham Little Rock, Arkansas 72201 www.lrsd.kl2.ar.us 501-324-2000 fax: 501-324-2032V IN THE UNITED STATES DISTRICT COURT EASTERN DISTRICT OF ARKANSAS WESTERN DIVISION LITTLE ROCK SCHOOL DISTRICT PLAINTIFF V. No. 4:82CV00866 WRW/JTR PULASKI COUNTY SPECIAL SCHOOL DISTRICT NO. 1,ET AL received MRS. LORENE JOSHUA, ET AL MAR 1 4 2003 DEFENDANTS INTERVENORS KATHERINE KNIGHT, ET AL OFFICE OF DESEGREGATION MONITORING INTERVENORS PLAINTIFFS NOTICE OF FILING PROGRAM EVALUATIONS REQUIRED BY PARAGRAPH C OF THE COURTS COMPLIANCE REMEDY Plaintiff Little Rock School District (LRSD) for its Notice of Filing Program Evaluations Required by the Courts Order of September 13, 2002 states: 1. On September 13, 2002, the District Court issued its Order finding that the LRSD had substantially complied with all areas of the Revised Desegregation and Education Plan (Revised Plan), with the exception Revised Plan 2.7.1. The Courts Order set forth a detailed Compliance Remedy as to Revised Plan 2.7.1. Paragraph C. of the Compliance Remedy stated: LRSD must use Dr. Nunnerly or another expert from outside LRSD with equivalent qualifications and expertise to prepare program evaluations on each of the programs identified on page 148 of the Final Compliance Report. I will accept all program evaluations that have already been completed by Dr. Nunnerly or someone with similar qualifications and approved by the Board. All program evaluations that have not yet been completed on the remaining programs identified on page 148 of the Final Compliance Report must be prepared and approved by the Board as soon as practicable, but, in no event, later than March 15, 2003. In addition, as these program evaluations are prepared, LRSD shall use them, as part of the program assessment process, to determine the effectiveness of those programs in improving African-American achievement and whether, based on the evaluations, any changes or modifications should be made in those programs. In addition, LRSD must use those program evaluations, to the extent they may be relevant, in assessing the effectiveness of other related programs.2. On October 10, 2002, the LRSD Board of Directors adopted a Compliance Plan designed to meet the requirements of the Courts Compliance Remedy. A copy of the Compliance Plan is attached hereto as Exhibit A. 3. As to Paragraph C of the Compliance Remedy, the LRSD concluded that the following evaluations had already been completed as required by Paragraph C and only needed to be submitted to the Board for approval: Early Literacy, Mathematics and Science, Charter School, English-as-a-Second Language, Southwest Middle Schools SEDL Program and Collaborative Action Team. The Charter School and Early Literacy evaluations were approved by the Board on October 24, 2002. The Southwest Middle Schools SEDL Program, 2000
This project was supported in part by a Digitizing Hidden Special Collections and Archives project grant from The Andrew W. Mellon Foundation and Council on Library and Information Resoources.