CREP V ftaearck M Educaaonai Policy Little Rock School District Center for Research in Educational Policy The University of Memphis 325 Browning Hall Memphis, Tennessee 38152 Toll Free: 1-866-670-6147 21 ** Century Community Learning Centers Evaluation FINAL TECHNICAL REPORT CREP * Center for Research in Educational Policy Center for Research in Educational Policy The University of Memphis 325 Browning Hall Memphis, Tennessee 38152 Toll Free: 1-866-670-6147 Little Rock School District 21^ Century Community Learning Centers Evaluation 2005-2006 Final Technical Report November 2006 Heidi Kenaga, PhD Jerry A. Bates, EdS Fethi Inan, MS Center for Research in Educational Policy John Nunnery, PhD Old Dominion University Table of Contents: 21 CCLC Technical Report iiTable of Contents Executive Summary Technical Report Introduction 1 10 10 Evaluation Questions Evaluation Design and Measures Evaluation Instrumentation 10 12 14 Procedure 18 Results 20 Teacher Questionnaire (CCLCTQ) Parent Questionnaire (CCLCPQ) Student Questionnaire (CCLCPQ) K-2 Students 21 28 33 34 3-5 Students 36 6-12 Students 39 Interviews 42 Site Coordinators 42 School Principals Mabelvale Case Study Classroom Observations 52 56 56 Student Focus Group Results Student Achievement Results 62 64 Conclusions 79 Evaluation Questions Compliance Remedy Questions References 79 84 90 Appendices Appendix A: Survey Instruments Appendix B: Teacher Survey Comments by School Appendix C: Parent Survey Comments by School 91 92 97 104 Table of Contents: 2 CCLC Technical Report hiTables Table 1. Table 2. Table 3. Table 4. Table 5: Table 6: Table 7. Table 8. Description of 2L CCLC Program Components at the Six Study Schools Demographics of the 2L' CCLC Schools in the Study Summary of Instruments, Participants, and Data Sources by Evaluation Question Data Collection Summary 21st CCLC Teacher Questionnaire (CCLCTQ) Results: Implementation 21st CCLC Teacher Questionnaire (CCLCTQ) Results: Impact 21st CCLC Parent Questionnaire (CCLCPQ) Results 21st CCLC Student Questioimaire (CCLCSQ) Results, Grades K-2 11 12 13 19 22 23 30 35 Table 9. Table 10. Table 11. Table 12. Table 13. Table 14. 21st CCLC Student Questionnaire (CCLCSQ) Results, Grades 3-5: Implementation . 21 st CCLC Student Questionnaire (CCLCSQ) Results, Grades 3-5: Impact . 21 st CCLC Student Questionnaire (CCLCSQ) Results, Grades 6-12: Implementation . 21st CCLC Student Questionnaire (CCLCSQ) Results, Grades 6-12: Impact . Race of Students Enrolled in 21st CCLC Programs in Study Schools, 2006-07 . Gender of Students Enrolled in 21st CCLC Programs in Study Schools By Grade Level, 2006-07 37 38 40 42 66 66 Table 15: Adjusted Means and Effect Size Estimates by School, Grade, and Treatment Level, 2006 ITBS Scores Table 16. Percentage of Students (Treatment and Control) Proficient in Literacy and Mathematics on the Arkansas Benchmark Exams by School, Grade, and Treatment Level 73 76 Figures Figure 1. Proportion of Male 21st CCLC Participants, Broken Down by Grade Level 67 Figure 2. Number of 21st CCLC Sessions Attended by Students at Henderson Middle School 69 Table of Contents: 2P CCLC Technical Report ivFigure 3. Number of 21 st CCLC Sessions Attended by Students at Mabelvale Middle School 70 Figure 4. Number of 21 st CCLC Sessions Attended by Students at McClellan High School Figure 5. Number of 21st CCLC Sessions Attended by Students at Southwest Middle School Figure 6. Number of 21st CCLC Sessions Attended by Students at Woodruff Elementary School Figure 7. Proportion of 21st CCLC Participants Substantially Served, by Grade Level Figure 8. Percentage of Students Obtaining Proficiency on the Arkansas Benchmark Exams in Literacy and Mathematics by Treatment Level Figure 9. Standardized Residuals of ITBS Total Reading NCE Scores vs. Total Days Attended Figure 10. Standardized Residuals of ITBS Total Math NCE Scores vs. Total Days Attended 70 71 71 72 75 77 78 Table of Contents: 2P CCLC Technical Report VEXECUTIVE SUMMARY INTRODUCTION This report summarizes the results of an evaluation of the 2005-06 Little Rock School Districts (LRSD) 2P Century Community Learning Centers (2L CCLC). The overall purpose of the evaluation was threefold: (1) to assess the effects of the 2L CCLC on raising the academic achievement of African Americans, (2) to examine implementation of the 2L CCLC program in the study schools, and (3) to document the perceptions of students, teachers, principals, and school staff regarding the 21' CCLC program. BACKGROUND Seven schools in the Little Rock School District host 21' Century Community Learning Centers, which offer academic support
math/science activities
music/arts/drama
entrepreneurial programs
drug/violence prevention, counseling, and character education
tutoring/mentoring
parent involvement
technology and communication
family literacy/education
recreational programs
extended library hours
and services for truant, suspended or expelled students. Individual centers provide a subset of the possible activities. Each school develops a slate of activities specific to the needs of the student population. RESEARCH QUESTIONS Primary Evaluation Question: 1. Have the 21 ** Century Community Learning Center (21 CCLC) programs been effective in improving and remediating the academic achievement of African American students? Supplemental (Qualitative/Step 2) Evaluation Questions: 1. What is the nature and level of implementation of the 2 CCLC programs? 2. What is the level of participation in 2U CCLCs by African American students? 3. What are the perceptions of teachers and school administrators regarding program implementation, impacts, strengths, and weaknesses? 4. What are the perceptions of parents/guardians and students of program impacts, strengths, and weaknesses? 2U CCLC Technical Report 1DESIGN Participants. LRSD identified 6 schools implementing a 21' CCLC program to participate in the evaluation: 1 elementary school, 3 middle schools, and 2 high schools. Collectively, the schools enrolled approximately 4,200 students and employed approximately 410 classroom teachers. Over 84% of the student enrollment was African American. Design. The evaluation design was based on both quantitative and qualitative data collected from the participating schools. Quantitative analyses examined district test scores to determine the effect of the 21' CCLC on Afncan American students learning as compared to non-21' CCLC students. In order to identify the level and quality of program implementation, the quantitative results of surveys distributed to teachers, parents, and student participants were descriptively analyzed, and qualitative analysis examined interviews conducted with 21' CCLC site coordinators and principals. One school was selected for a more focused case study, Mabelvale Magnet Middle School. INSTRUMENTATION Teacher, Student, and Parent Survey Teacher survey. This instrument was used to collect teacher knowledge of and perceptions about the 21' CCLC program as it operated at their school. Parent survey. This instrument was designed to capture parent perceptions regarding their childs participation in the 2L' CCLC program. Student surveys. Because the evaluation assessed students in elementary, middle, and high schools, different surveys were developed for each of three age groups: K-2, 3- st 5, and 6-12. These surveys were used to gauge student perceptions about the 21 CCLC program activities at their school. Interviews The site coordinator and principal interviews focused on basic features, implementation, support, and strengths and weaknesses of the 2P CCLC program at the participating school. Classroom Observation Measure (case study school only) An observation instrument (Classroom Observation Tool) was used to record student and teacher activities during a typical 2 CCLC classroom meeting at the case study school, Mabelvale Magnet Middle School. 2 CCLC Technical Report 2Student Focus Group (case study school only) The student focus group protocol was used to document student impressions of the nature of 2P CCLC program activities
the usefulness of the program
and student satisfaction with the range of activities offered. The focus group was conducted at the case study school, Mabelvale Magnet Middle School. Student Achievement Researchers used scores from the following standardized achievement tests to examine the effect of the 2P CCLC program on African American students learning: Iowa Test of Basic Skills (ITBS) (ITBS, 2002) and Arkansas Benchmark Exams (Arkansas Benchmark Exams, 2005). PROCEDURE The evaluation involved the collection of a variety of information at the 6 schools by researchers who had no association with LRSD or the 21* CCLC program. Surveys were distributed to students who were participating in the program and their parent or guardian, and to all teachers at each school in order to obtain their perceptions of the 21* CCLC program. In addition, researchers conducted interviews with the site coordinator and principal at each school. At Mabelvale Magnet Middle School, the case study school, researchers observed 21* CCLC classrooms during prearranged sessions and also conducted a student focus group. Analyses of student achievement on the ITBS and the Arkansas Benchmark Exams were conducted to determine the impact of the 21* CCLC on the academic achievement of African American as compared to non-African American students. RESULTS Key results of the study are presented by measurement strategy. In the conclusion section, findings are synthesized across instruments in order to address each research question. Survey Results Teacher Survey. A total of 187 teachers at the participating LRSD schools completed surveys, the 21* CCLC Teacher Questionnaire (CCLCTQ). (Results are summarized below.) Parent Survey. A total of 82 parents/guardians of primarily African American children completed surveys, the 2L CCLC Parent Questionnaire (CCLCPQ). (Results are summarized below.) Student Survey. A total of 270 students (50 elementary, 140 middle school, and 80 high school) completed surveys, the 21* CCLC Student Questionnaire (CCLCSQ). (Results are summarized below.) 21 CCLC Technical Report 3Interview Results Site Coordinators. Nearly all the site coordinators had high praise for the 21 CCLC program and the afterschool staff and affiliates working at their schools. They consistently identified three strengths of the program: (1) that it provided a safe, secure environment for children at a time (after school) when they were most at risk, even those working at or above proficiency who found a place to further enhance their skills
(2) that it afforded students many opportunities (the community activities, access to technology) which they otherwise may not have had
and (3) that it constituted a psychological reward, as the students were made more acutely aware that there was a group of adults devoting their extra time and energies toward helping them achieve academic success as well as improve their classroom behaviors. st While the site coordinators never experienced problems recruiting students into the 21 CCLC program at their school, all except one (at Woodruff) identified student retention/ attendance rates as an ongoing challenge. In general, those days following the administration of standardized tests saw less frequent attendance, and as the warmer weather returned in the spring semester there were general declines in participation rates as well. Another area for improvement identified by the coordinators (except at Woodruff Elementary and Southwest Middle School) was parental involvement, particularly in the upper grades. The high school coordinators in particular tried several strategies for eliciting parents participation, without too much success. However, the site coordinators at Woodruff and Southwest reported that any public performance or display (e.g., music, drama, art) by their child/ren in the 21 CCLC program was guaranteed to bring out the parents. Principals. Most of the principals responses to questions during interviews were similar to those provided by the site coordinators, reiterating the major themes described above. Classroom Observation Results On the basis of the three 21 CCLC sessions observed at Mabelvale Magnet Middle School, both Afidcan American and non-African American students participated equally in activities and with a fairly high level of engagement. The academic relevance of the range of activities was generally clear
the quality of instruction and classroom environment was usually good
and the commitment and enthusiasm of the teaching staff was much in evidence. Student Focus Group The student focus group conducted at the case study school, Mabelvale Magnet Middle School, comprised 6 students. They liked having useful activities in which to engage after the close of the regular school day, and most felt that participation was having a positive impact on their schoolwork. In terms of areas for improvement, student indicated that greater variety and perhaps more creative development of 21 CCLC program activities (e.g., recreation, enrichment or arts instruction) would enhance student enjoyment of afterschool, and also contribute to higher attendance rates. In addition, although most students felt that afterschool activities were improving their academic performance, it did not cause them to like school more. 2P CCLC Technical Report 4Student Achievement Results According to Federal guidelines, students must attend a program at least 30 days to be considered substantially served. The three groups assessed in CREPs statistical analysis control, not substantially served, and substantially served students performed equally on ITBS Reading and Mathematics after controlling for prior achievement, free or reduced-price lunch status, special education status, and gender. There was no relationship between student attendance of the 21^ CCLC program nor numbers of days attended and student achievement scores. CONCLUSIONS The conclusions of this study are presented in association with each of the major research questions in the respective sections below. Primary Evaluation Question Have the 21^ Century Community Learning Center (21^ CCLC) programs been effective in improving and remediating the academic achievement of African American students? Although CREP employed rigorous methods to estimate 2L CCLC effects, the small number of students who were substantially served mitigate against ascribing the observed effects to the 2L CCLC programs. The available evidence suggests that there was no relationship between student achievement on standardized tests and attendance of 2L CCLC programs. No relationship was observed between the number of sessions attended and student achievement in reading or mathematics. A significant negative relationship was observed between treatment and literacy proficiency on the Arkansas benchmark examination, but given the preponderance of students who were not substantially served, it is quite plausible that this effect is attributable to students self-selecting out of the program, rather than to the interventions themselves. Supplemental (Qualitative/Step 2) Evaluation Questions What is the nature and level of implementation of the 21^ CCLC programs? On the basis of survey results as well as observations conducted at the case study school Mabelvale Middle, it would appear that many participants are highly engaged with 2L' CCLC activities. The academic relevance of the classes was generally clear
the level of instruction and quality of classroom environment was usually good
and instructor preparation, commitment, and enthusiasm were clearly evident. Most of the teachers were very positive in their comments about the programs efficacy and impact on their students, as were the parents of 2L CCLC program participants. However, in regard to the level of implementation, the attendance rates for the majority of 2 L' CCLC participants were quite low. Only two of the six schools that received 2L CCLC funding substantially served a majority of the students who enrolled in the program. The two exceptions were Woodruff Elementary, where essentially all participants were 2P CCLC Technical Report 5substantially served, and Southwest Middle, where slightly more than half of participants were substantially served. Thus, while the quality of the observed implementation of the 2L CCLC program was good, the level of implementation (on the basis of attendance rates) fell far short of expectations, especially at the middle and high school levels. Overall, only about 20% of students in grades 6- 12 who enrolled in 2P CCLC programs attended 30 or more sessions, which is comparable to the 16.4% who attended no sessions at all. What is the level of participation in 21^ CCLCs by African American students? Since the majority of students in the study schools were African American, the level of participation by this ethnic group in 2L CCLC activities was quite high: 87% of the K-2 respondents, 85% of the middle school, and 82% of the high school identified themselves as Afncan American. Interviews with site coordinators and principals confirmed this level of participation. What are the perceptions of teachers and school administrators regarding program implementation, impacts, strengths, and weaknesses? Staff at the 6 study schools were generally very positive about and supportive of the program. In terms of implementation, nearly all agreed that the CCLC offered students a safe place to be after school and that it offered academic help to students who needed it. Large proportions also felt that the CCLC program appealed to a wide variety of students and that the program offered students enough choice of activities. Also, many teachers praised the afterschool staff for their commitment and dedication to helping students improve their skills. In their survey responses, teachers were somewhat more ambivalent about the programs positive impact on participants in terms of changes in academic and social domains. A little less than half agreed that almost all or many 2L CCLC participants were performing better academically, and on average approximately 40% saw improved behaviors in terms of how they related to classmates and teachers as well as in-class conduct. In terms of students facility in using technology, almost half reported that almost all or many participants showed improvements in computer literacy skills. In general, school staff identified three strengths of the 2L CCLC program. First and foremost, the provision of safe and secure environment in which to build skills was seen by most respondents as crucial. Second, participants now had multiple opportunities (such as computer access and arts enrichment), to which they otherwise may not have had access. Third, students were reaping the psychological rewards of knowing that teachers cared enough about their development and progress to work in the afterschool program. These responses by administrative persoimel, who are in the best position to observe the operation and impacts of the 2P' CCLC on students, acknowledge a salient aspect of the program: oversight of participants activities beyond the regular school day may have had positive impacts not easily captured nor sufficiently measurable by standard survey methods. The two most commonly noted areas for improvement included increasing students rates of attendance and expanding efforts to reach and engage the parents of program participants. In addition, some of the site coordinators noted that they had occasional problems finding qualified and available staff, especially in math and science. 21 CCLC Technical Report 6What are the perceptions of parents/guardians and students of program impacts, strengths, and weaknesses? st A majority of parents who returned the surveys responded very positively to the 21 CCLC program at their child/rens school. They felt that the center was a safe place for their children to be and that the 21 CCLC had good programs with different activities from which to choose. A large proportion would sign their child up again and would encourage other children to attend as well. More than three quarters of responding parents thought that the CCLC program was having a positive impact on their child/rens academic performance. There were some differences between the parents responses and the school staff responses. Higher percentages of parents than teachers thought that their child/ren were exhibiting improvements in other areas: feeling better about her/himself, more clearly understanding the importance of graduating from high school, and working better with other students. No parent identified the same areas for improvement - student attendance and parental engagement - as did the teachers and administrators. In fact, when asked whether they had attended any of the 2L CCLC programs for parents, almost two thirds said that they had, which did not match data gathered from school staff. These results should be viewed with caution, since (1) there were fewer parent responses (n = 82) than teacher responses (n = 187) and (2) this data set comprises less than 10% of all parents of participants. Thus, the parent responses constitute a self-selected sample and may be biased (that is, it is more likely that parents who returned surveys were more engaged with the program and had children who attended more often). In regard to student perceptions of the 2P' CCLC, the elementary school children (all at Woodruff) appear to be enjoying and benefiting from the program. For both the younger cohort (K-2) and the older cohort (3-5), the afterschool teachers and staff are strong features of the program
they also like the range and variety of activities offered, although the older children were somewhat less satisfied with some of the programs. The safety issue was broached in the surveys: approximately three quarters of both cohorts said that they felt safe at the center. These children identified participation with having a positive impact on their self-image, as a large majority said that the program made them feel better about themselves. Almost two thirds of the older cohort felt that participation had helped them behave better at school and understand the importance of graduating from high school. Like the two younger cohorts, the participant respondents in grades 6-12 identified the teachers and staff in their 21^ CCLC as a real strength. Similarly, a majority of the older students indicated that they felt safe in the afterschool program. Approximately two thirds liked the programs offered, would sign up again, and would recommend the program to other kids. Still, as perhaps might be expected of this age group, only about half said that being in the program was better than other things they could be doing after school. Although in general the responses of the older students indicate they believe the 21 St CCLC to be of much benefit, they were less sure than the younger students of its impact on their academic performance: raising their grades, doing better at school, talking to teachers more, and simply wanting to attend school. At the same time, a little less than three quarters of students said that participation in the center had helped them understand the importance of graduating from high school, a very positive impact of the program. 2P CCLC Technical Report 7COMPLIANCE REMEDY QUESTIONS Teacher and Administrator Involvement The evaluation involved 6 LRSD schools that were implementing the 2L CCLC Program. At each of these schools, the site coordinator of the program as well as the principal was interviewed. Survey instruments were also distributed to all faculty at each of the schools, those who were involved with the 2L CCLC and those who were not. Recommended Program Modifications The 2L CCLC program was generally praised by teachers, faculty, parents, and students in the study as a valuable supplement to the schools. There may be important benefits of the program beyond the overtly measurable outcomes. As discussed above, 2U CCLC program participants reap the rewards of a safe learning environment during the afterschool hours, the knowledge that teachers and administrators care enough about their progress to expend time and effort on their behalf, and the many extracurricular and enrichment activities probably not otherwise available to them. At the same time, statistical analysis of standardized test scores conducted by CREP did not show a correlation between improved achievements in math and literacy and student participation in the program. Possible program modifications that might produce better gains in achievement scores are described below: 1. 2. 3. 4. Staff and administrators consistently identified low student attendance as a key problem, and attendance records confirm that, with the exception of Woodruff Elementary and Southwest Middle School, the average 2P CCLC participant is attending the program only sporadically. Site coordinators and school administrators should focus on effective ways of encouraging students to attend more frequently and regularly. Possible options include (1) providing incentives or rewards for good attendance and (2) the addition of more enrichment activities, such as arts or drama-oriented events, in order to gamer more student interest, as well as parental involvement. Closer attention should be paid by 2L CCLC site coordinators to the mandated attendance record-keeping, so that during the evaluation process more accurate and specific assessments can be developed regarding the impacts of the program on student achievement scores and other domains. Greater efforts should be made to identify those students who are consistently working below proficiency levels and encourage them to attend the 2P CCLC program as much as possible. Interviews with staff indicated that primarily mid-range students and those who didnt need the extra help constituted the majority of participants. Especially at the high schools, school staff felt that the 2L CCLC provided such crucial academic assistance to the many students who needed it that it should not be voluntary but in fact compulsory. At the same time, both the qualitative and quantitative data collected for this report indicate that the 2P CCLC program is most effectively implemented at the elementary and middle school levels
85% of 21 CCLC participants 21^ CCLC Technical Report 8in grades 6-12 survey were in the 6* through 9* grades. Correspondingly, mandatory attendance might be most practical and feasible for high school freshmen only, assuming compliance measures were acceptable to relevant stakeholders. 5. Staff at the study schools had a fairly high degree of success in eliciting community involvement in their 21 CCLC program. However, they had little luck in involving the parents of participants in center activities. Site coordinators might explore innovative tactics, such as offering continuing education courses in skills or career development, to involve more parents in their 21 CCLC program. 6. Implementation of a self-evaluation by each of the schools receiving a 21 CCLC grant may be a good way for individual schools to pinpoint specific areas for improvement and develop workable solutions for addressing problem areas. CREP recommends that the LRSD consider partnerships with local universities to institute a structured evaluation component for each school in the district receiving a 21 CCLC grant. Expectations of Program Modifications The 21 CCLC holds promise for addressing the gaps in the academic achievement of African American students, even if the statistical analysis revealed no relationship between participation and attendance of the program and standardized test scores. As noted by teachers and staff at the study schools, there are social and psychological benefits of program participation which may be as equally valuable as test outcomes. CREP believes that if the recommendations outlined above are followed, 21 CCLC program staff can expect 1. Progressive gains in all participants scores on standardized literacy and mathematics tests, but particularly those students working at or below proficiency levels and who have been attending the afterschool program regularly. 2. Higher rates of attendance by participants in the middle and high schools, or at least higher proportions of those who, following Federal guidelines, would be considered substantially served. If participation by high school freshmen were rendered mandatory, school staff might see gains in math and literacy in this cohort which are extended through subsequent years of these students secondary school education, as well as (given the socialization function of the 21 CCLC program in the first year of high school) improved high school graduation rates. 3. Greater parental involvement in the 21 CCLC programs, with potentially the positive outcomes associated with such engagement. Further, the initiation of continuing education courses specifically for parents would have multiple educational and prosocial benefits, not the least of which would include additional reinforcement of the value of education for their children. 4. Improved program implementation in the study schools via the incorporation of a selfevaluation component. 21 CCLC Technical Report 9LITTLE ROCK SCHOOL DISTRICT 2C CCLC Community Learning Centers Evaluation TECHNICAL REPORT Introduction This report summarizes the results of an evaluation of the 2005-06 Little Rock School Districts (LRSD) 2P' Century Community Learning Centers (2P' CCLC). The overall purpose of the evaluation was threefold: (1) to assess the effects of the 2P CCLC on raising the academic achievement of African Americans, (2) to examine implementation of the 2P CCLC program in the study schools, and (3) to document the perceptions of students, teachers. principals, and school staff regarding the 2P CCLC program. st This evaluation study examined the following 6 schools who were implementing a 21 CCLC program in LRSD
Woodruff Elementary
-Henderson, Southwest, and Mabelvale Magnet Middle Schools
and Hall and McClellan High Schools. Mabelvale was selected as the case study school. More detailed descriptions of the 2P CCLC program at each study school can be found in Table 1. EVALUATION QUESTIONS The 21* CCLC evaluation was structured around one overarching, primary question concerning the impact of the program on student achievement. Four supplemental questions addressed contextual factors related to the implementation of the 21* CCLC program. 21* CCLC Technical Report 10Table 1. Description of 21 CCLC Program Components at the Six Study Schools School Woodruff Elementary Henderson Middle Southwest Middle Mabelvale Middle Hall High McClellan High ______________Description of 2L CCLC Program______________ Before and afterschool, on Saturdays, and during intersessions (Woodruff is a year-round school) Visual and performing arts enrichment Cultural activities and educational and recreational field trips Educational technology Parent and family services____________________________________________ Before and afterschool
summer camp Before school provides homework help, extended access to the library, and technology resources. Tutoring Club afterschool designed to assist students academically Enrichment opportunities are coordinated with community partners Summer camp focuses on mathematics, literacy and science, with technology a strong component____________________________________________ Afterschool, Saturday Sojourns Focus on improving standardized test scores in literacy and math Homework Club Community partnerships for cultural enrichment activities Parent and family services____________________________________________ Before and afterschool program
summer camp Homework Club before school provides homework assistance and extended library hours Tutoring Club afterschool offers computer-based programs and personalized instruction for algebra preparation and literacy development Enrichment Club provides extended learning opportunities relating to magnet programs of Environmental Science, Medical Studies, and Technology . Summer camp focuses on mathematics and literacy, primarily through the use of technology Parent programs and family services____________________________________ Before and afterschool
Saturdays
summer camp Drug and violence prevention program and a youth development component emphasizing counseling, service learning, mentoring, and employment opportunities . Summer camp offers a six-week Algebra 1 program and a two-week transition program for students entering 9* grade Arts enrichment Parent/family sessions_______________________________________________ Before and afterschool
Saturdays
summer camp Drug and violence prevention program and a youth development component emphasizing counseling, service learning, mentoring, and employment opportunities Summer camp offers a six-week Algebra 1 program and a two-week transition program for students entering 9' grade Arts enrichment Parent/family sessions 2P CCLC Technical Report 11Primary Evaluation Question'. 1. Have the 2P Century Community Learning Center (2L' CCLC) programs been effective in improving and remediating the academic achievement of African American students? Supplemental (Qualitative/Step 2) Evaluation Questions: 2. What is the nature and level of implementation of the 2P CCLC programs? 3. What is the level of participation in 2L CCLCs by African American students? 4. What are the perceptions of teachers and school administrators regarding program implementation, impacts, strengths, and weaknesses? 5. What are the perceptions of parents/guardians and students of program impacts, strengths, and weaknesses? EVALUATION DESIGN AND MEASURES Participants LRSD identified 6 schools with a 2L CCLC program to participate in the evaluation: 1 elementary school, 3 middle schools, and 2 high schools. Collectively, the schools enrolled approximately 4,400 students and employed approximately 340 classroom teachers. Over 84% of the overall student enrollment was African American. LRSD data indicated a total of 859 students registered for the program (see Table 2). Table 2. Demographics of the 21 st CCLC Schools in the Study School Name Elementary School Woodruff Elementary Middle Schools Mabelvale Middle School Southwest Middle School Henderson Middle School High Schools Hall High School McClellan High School Grade Level Percentage of African American Students* Number of 21 * CCLC registrants K-5 6-8 6-8 6-8 9-12 9-12 89% 74% 96% 80% 73% 93% Total 55 260 167 107 6 264 859 * Common Core of Data, 2006 2L' CCLC Technical Report 12Design The evaluation was conducted during the spring of 2006. Both quantitative and qualitative data were collected from the 6 participating schools using survey instruments and established research procedures (Ross, Smith, Alberg, & Lowther, 2004) (see Table 3). Table 3. Summary of Instruments, Participants, and Data Sources by Evaluation Question Evaluation Question Primary Question Have the 21 Century Community Learning Center (21 CCLC) programs been effective in improving and remediating the academic achievement of African American students? Supplemental Questions What is the nature and level of implementation of the 21 CCLC programs? What is the level of participation in 21 CCLCs by African American students? What are the perceptions of teachers and school administrators regarding program implementation, impacts, strengths, and weaknesses? What are the perceptions of parents/guardians and students of program impacts, strengths, and weaknesses? Participants Data Sources Students participating in 21 CCLC programs in 6 LRSD schools Control group matched to participating students 21 CCLC Program/School administrators 21 CCLC Teachers 21 CCLC Students 21 CCLC Students 21 CCLC Teachers 21 CCLC Program/School Administrators Parents of 21 CCLC students 21 CCLC Students ITBS and Arkansas Benchmark Reading and Math Subtests Principal Interviews (phone and on site) Site Coordinator Interviews (phone and on site) Teacher Survey (administered at faculty meetings) Student Survey Student Focus Group (case study) 21 CCLC Observations (case study) School records/participation rosters 21 CCLC Observations (case study) Teacher Survey (administered at faculty meetings) Principal Interviews (phone and on site) Site Coordinator Interviews (phone and on site)_______________ Parent Survey Student Survey Student Focus Group (case study) Quantitative analyses examined district test scores to determine the effect of the 21 CCLC on African American students academic achievement. In order to identify the level and quality of program implementation, the quantitative results of surveys distributed to teachers, parents, and 21 CCLC Technical Report 13student participants were descriptively analyzed. Qualitative analyses were conducted on information gained during staff interviews conducted at the 6 study schools. The evaluation design also included a case study of an individual school, Mabelvale Magnet Middle School. The case study approach was selected so that a comprehensive understanding of that instance obtained by extensive description and analysis taken as a whole and in its context (GAO, 1990) would provide richer contextual detail regarding the implementation of the community learning center concept. Researchers employed a triangulated method, which entailed multiple classroom observations and a focus group in addition to the surveys and staff interviews. The one school chosen for the case study was regarded by district personnel as having an exemplary 2P CCLC program. EVALUATION INSTRUMENTATION Five strategies were used to collect evaluation data from all 6 schools
teacher. parent/guardian, and student surveys
staff interviews
and assessment of student achievement scores. In addition, direct classroom observations and a focus group were conducted at the case study school, Mabelvale. Copies of the instruments can be found in Appendix A. Teacher, Parent/guardian, and Student Surveys Teacher survey. The 21' Century Community Learning Center Teacher Questionnaire (CCLCTQ) was used to collect teacher knowledge of and perceptions about the 21' CCLC program as it operated at their school. The first section of the survey asked teachers to identify whether they were part of the 21' CCLC program at their school. The second section asked them for their opinion of the program by rating their level of agreement with 5 general 21' CCLC Technical Report 14statements, using a 5-item Likert-type scale that ranged from Strongly Agree/Agree/ Neutral/Disagree/Strongly Disagree. The third section of the CCLCTQ section asked teachers to assess how many students had been impacted by the program in a variety of areas, including academic performance, classroom and school behavior, interpersonal and social development. and computer literacy, using a 5-item Likert-type scale that ranged from Almost all of them/Many of them/Some of them/A few of them to Almost none of them. The final section asked teachers to list the strengths of the program and ways it might be improved. (See the example of the teacher survey in Appendix A.) Parent/guardian survey. The 2L Century Community Learning Center Parent Questionnaire (CCLCPQ) instrument was designed to capture perceptions of parents or guardians regarding their child or childrens participation in the 2L CCLC program. The first section of the instrument asked parents or guardians to identify their child/rens ethnic group. The second section asked respondents to indicate which (if any) clubs or activities affiliated with the 2P CCLC their child/ren participated in (Yes/No/Dont Know). The list of possible programs (e.g.. Enrichment, Tutoring, Saturday program. Summer Camp) was compiled from the schools grant applications, which described the various activities implemented at the schools. The third section of the CCLCPQ asked parents opinion of the program by rating their level of agreement (5-item Likert-type scale that ranged from Strongly Agree to Strongly Disagree) with a list of 6 statements. The fourth section asked respondents to indicate their agreement with any of 8 general statements describing the programs potential impact on their child/rens views about school, schoolwork, and interpersonal and social behavior. The final section asked parents to list the strengths of the program and ways it might be improved. (See the example of the parent/guardian survey in Appendix A.) 21* CCLC Technical Report 15Student surveys. The 21' Century Community Learning Center Student Questionnaire (CCLCSQ) was used to gauge student perceptions about the 21 CCLC program activities at their school. Because the evaluation assessed students in elementary, middle, and high schools. different surveys were developed for each of three grade cohorts: K-2, 3-5, and 6-12. (See the examples of these surveys in Appendix A.) All students were asked to identify their grade. gender, and ethnic group membership
the wording of these questions was altered to fit the age level of the respondent. The K-2 instrument. This questioimaire contained fewer items (12) and used more simple language than the other student surveys. Students were asked to fill in the appropriate 3- st point Likert bubble (Agree/Dont Know/Disagree) regarding statements about aspects of the 21 CCLC program at their school. To aid the children in understanding the 3 options, the instrument contained small face icons, e.g., a smiley face next to Agree, a nonsmiling face next to Dont Know, and a frowning face next to Disagree. Teachers were also instructed to read the statements out loud as necessary, and they were advised that they might also explain the question about ethnicity. (See the sample survey found in Appendix A.) The 3-5 and 6-12 instruments. These two versions were the same except for the initial questions about gender and grade (see the sample surveys found in Appendix A). The first section asked students to indicate (Yes/No) whether they participated in clubs or activities affiliated with the 21* CCLC, such as Enrichment, Saturday program, or Summer Camp. The next section asked students to rate their level of agreement with 9 statements about the program. using a 5-point Likert-type scale (Strongly Agree/Agree/Neutral/Disagree/Strongly Disagree). The final section asked students to complete a set of statements assessing the programs impact 21' CCLC Technical Report 16on their views about school, schoolwork, and interpersonal and social behavior, using a 4-point Likert-type scale (A great deal/Somewhat/A little/Not at all). Interviews An interview protocol was developed for both the site coordinators and principals of all 6 schools in the study. Six of the interviews were conducted in person onsite, and six via telephone. Some of the protocol was designed to gather basic information from staff, such as questions about their role in the operation of the 2L CCLC program
whom the program serves
and the types of approaches and scheduling used (e.g., meeting times. Homework or Enrichment Club, Saturday program. Summer Camp). Most of the protocol required staff to elaborate on the specific implementation of the 21' CCLC at their school: the programs primary purpose. recruiting and retention rates, typical participant, school and parental support, community partnerships, and strengths and weaknesses. At the conclusion of the interview, staff were asked to contribute any additional comments. Classroom Observation Tool The Classroom Observation Tool was custom-designed to document the processes and practices evident in the 21' CCLC classrooms in the case study school. The instrument was used to record contextual data (e.g., number of students observed participating in the afterschool program, number by ethnicity, number of teachers, number of parents) and the type of activities and materials in use. Data were also recorded regarding student involvement or engagement in the activities, how well the period observed represented a learning center. While this tool guided the observations in ensuring key data were observed, qualitative field notes were also to be used as part of the data collection process. 21' CCLC Technical Report 17Student Focus Group The student focus group protocol was used to document student impressions regarding 21 CCLC program activities
the usefulness and effectiveness of the program
satisfaction with the range of activities offered
and strengths and areas for improvement. At the conclusion of the focus group, students were asked to contribute any additional comments. The focus group protocol was designed for use in the case study school. Student Achievement CREP used scores from the following standardized achievement tests to examine the effect of the 2L CCLC program on African American students learning
(1) Iowa Test of Basic Skills (ITBS, 2002), and (2) Arkansas Benchmark Exams (Arkansas Benchmark Exams, 2005). PROCEDURE The ten data collection measures are summarized in Table 4 by type of measure. instrument, number completed, and the data collection procedure. Surveys. There were three types of survey instruments: the 2L* CCLC Teacher Questionnaire (CCLCTQ), the 2L' CCLC Parent Questionnaire (CCLCPQ), and 21 CCLC Student Questionnaire (subdivided into K-2, 3-5, and 6-12). In March 2006, packets containing instruments, instructions, and return envelopes were sent to the site coordinators at each study school, who administered and then collected all instruments for return via mail to the evaluator. The site coordinator distributed the teacher questionnaires at the faculty meeting
provided each student participating in the 2L CCLC program with a parent/guardian questionnaire (enclosed in envelopes with an instruction label)
and administered the student questioimaires to program 2L CCLC Technical Report 18participants. In some instances, researchers picked up completed instruments at the school itself. All three surveys were received from each site, with one exception: parent surveys were not received from Southwest Middle School. Interviews. The 2P CCLC site coordinator and the principal at each study school were interviewed, either via telephone or in person on site, using a standard interview protocol. The site coordinator interviews usually took about 45 minutes and the principal interviews about 30 minutes. Each interview was audiotaped and then a detailed transcription made that incorporated both interview notes and the audiotape version. Twelve interviews were completed and transcribed by May 2006. Table 4. Data Collection Summary Type of Measure Instruments Number Completed Data Collection Procedure Surveys 1) 21st Century Community Learning Center Teacher Questionnaire (CCLCTQ) 187 Administered during faculty meetings at each of the 6 study schools Interviews Classroom Observations Student Focus Group Student Achievement Analysis 2) 21st Century Community Learning Center Parent Questionnaire (CCLCPQ) 3) 21st Century Community Learning Center Student Questionnaire (CCLCSQ) 1) Site Coordinator Inten/iew Protocol 2) Principal Interview Protocol Site Classroom Observation Tool Student Focus Group Protocol 1) ITBS 2) Arkansas Benchmark Exams 82 270 (30 K-2,20 3- 5,220 6-12) Distributed by site coordinators at all 6 study schools. Surveys were returned to the schools by the parent/guardian and then mailed to the evaluator. Distributed to all students participating in the 2V CCLC at all 6 of the study schools. There were 3 different surveys developed for grade cohorts: K-2,3-5, and 6-12. Researchers individually interviewed site coordinators and principals at all 6 of the study schools
the inten/iews lasted between 30 and 45 minutes One morning and two afternoon sessions were observed at the case study school, Mabelvale Middle School. Additional visits were scheduled but canceled by the site coordinator 1 focus group conducted at the case study school, Mabelvale Middle School: 6 students (3 in the 6*', 2 in the 7** grade, and 1 in the 8 grade) Analyses of student achievement on the ITBS and the Arkansas Benchmark Exams were conducted to determine the impact of the 21 CCLC on the academic achievement of African Americans 2P' CCLC Technical Report 19 6 6 3 1Classroom Observations. There were three observations (two in the afternoon and one in the morning) of 2P' CCLC classrooms in operation at the case study school, Mabelvale Magnet Middle School. The researcher prearranged these visits with the site coordinator. In the afternoon session, the researcher observed the activities of separate groups of 6*, 7*, and 8* graders, and in the morning session all grades were observed in one classroom. Student Focus Group. One focus group was conducted by the researcher at Mabelvale Magnet Middle School using a focus group protocol
there were 6 students, 3 in the 6*, 2 in the 7**, and 1 in the 8* grade. The students selected for the focus group were those for whom parental consent had been received. The discussion lasted approximately 30 minutes and took place in the site coordinators office. Achievement Tests. The results of two tests taken in April by LRSD students, the Iowa Test of Basic Skills (ITBS) and the Arkansas Benchmark Exams, were forwarded to the evaluator for analysis. The ITBS scores arrived in July and the Benchmark scores in September. Researchers examined scores of those African Americans who participated in the 2P' CCLC program and matched them to control students within the same grade and school in order to discern the possible impact of the program on participants academic achievement. Researchers also compared the results of students who were enrolled in the program and substantially served to those who were enrolled but not substantially served. Substantially served was defined according to Federal guidance for such programs, based on program attendance of at least 30 days over the course of the year. RESULTS The results of the study are presented in the section below. In the Conclusion section, the findings are synthesized across instruments to address each research question. 21 CCLC Technical Report 20Survey Results This section contains the results of the 2P Century Community Center Teacher Questionnaire (CCLCTQ), the 2L' Century Community Parent Questionnaire (CCLCPQ), and the 2L Century Community Student Questioimaire (CCLCSQ). 2r CCLC Teacher Questionnaire (CCLCTQ) The CCLCTQ was completed by a total of 187 teachers working at the 6 schools implementing a 2L CCLC program, out of a possible 410 (the total number of teachers at all six schools), which represents a 46% response rate. Approximately 69% of the respondents were teachers at the school but not affiliated with aftercare program, and approximately 25% were st both teachers and involved with the program
only 1 respondent was employed only by the 21 CCLC program and not a faculty member at the school. Eleven did not respond to the question about their role in the program. The first section was used to gauge teacher perception of the 2P CCLC as it was implemented in their school. As shown in Table 5, two of the statements were rated very highly on the survey (as indicated by the combination of the categories Strongly Agree and Agree). A large majority strongly agreed or agreed that the CCLC offered students a safe place to be after school (91.4%) and that it offered academic help to students who needed it (88.2%). Two other statements were rated highly: almost three quarters (74.9%) strongly agreed or agreed with the statement that The CCLC program appeals to a wide variety of students and more than two thirds (66.8%) with the statement that The CCLC program offers students enough choice of activities. This last statement about the range of program activities elicited the highest percentage (9.6%) of respondents who strongly disagreed or disagreed, although the level of disagreement was still quite low. Respondents were more clearly ambivalent about the last 2L CCLC Technical Report 21statement, The CCLC program reaches student families
a little more than half (56.1 %) agreed or strongly agreed, with about a third (34.2%) responding neutral about their agreement with this statement. Table 5. 2V CCLC Teacher Questionnaire (CCLCTQ) Results (all schools combined)
Implementation Responses(%) Items The CCLC program offers students enough choice of activities. The CCLC program offers academic help to students who need it. The CCLC program offers students a safe place to be after school. The CCLC program appeals to a wide variety of students The CCLC program reaches Strongly Agree and Agree 66.8 88.2 91.4 74.9 56.1 Neutral 21.9 8.6 5.3 18.7 34.2 Disagree and Strongly Disagree 9.6 2.7 1.6 4.8 8.0 Mean (SD) 3.86(.99) 4.38(79) 4.52(.67) 4.08(88) students' families_______________________________________________________________________________ Notes. Percentages for responses may not total 100% because of missing input from some respondents, n = 187 3 All items measured using a 5-point Likert scale: 1 = strongly disagree
5 = strongly agree 3.73(.95) The concluding section of the CCLCTQ asked teachers to assess the extent of the CCLC programs effect on student participants by indicating whether Almost all of them, Many of them, Some of them, or Almost none of them exhibited positive changes in a variety of academic, social, and technological domains. As shown in Table 6, teachers responses indicate that less than half of them think that the CCLC program is having a positive impact, as indicated by the combination of the Almost all of them and Many of them categories. For example, 47.6% of respondents agreed that almost all or many students in the program exhibited improved academic performance, and 39.6% thought that at least some were exhibiting improvement. Slightly more than two fifths (41.2%) of teachers said that almost all or many participants attendance had improved, with a similar proportion (43.9%) noting better attendance during the regular school day by some of the students. In addition, several teachers 2P CCLC Technical Report 22saw improved behaviors for 2P' CCLC participants in terms of how they related to classmates (38.5%) and teachers (40.6%) and how they behaved in class (36.3%) (see Table 6). Table 6. 21st CCLC Teacher Questionnaire (CCLCTQ) Results (all schools combined): Impact How many students that you know who participate in the 2V CCLC program exhibit... Improved academic performance? Increased school attendance? Improvement in how they relate to classmates? Improvement in how they relate to teachers? Improvement in how they behave in class? Almost All of Them and Many of Them 47.6 41.2 38.5 40.6 36.3 Responses (%)> Some of Them 39.6 43.9 43.3 40.6 43.3 A Few of Them and Almost None of Them 9.6 11.8 13.3 15.0 16.5 Mean (SD)^ 3.50 (.90) 3.39 (.98) 3.33 (.99) 3.37(1.01) 3.25 (.98) Greater self-worth and self-esteem? Involvement in community services? Development as leaders? Greater awareness of health issues? Greater awareness of drug and violence issues? Improvement in computer skills/ 44.4 26.7 27.3 27.8 32.1 38.5 39.6 41.2 39.0 37.4 12.8 27.2 26.7 27.2 23.6 3.41 (.94) 2.93 (1.09) 2.98 (1.04) 2.98 (1.07) 3.10(1.04) 45.5 38.5 10.2 computer literacy?_______________________________________________________________ Note: Percentages may not total 100% because of missing input from some respondents, n = 187 All items measured using a 5-point Likert scale: 1 = almost all of them, 5 = almost none of them. 3.49 (.94) Percentage response calculated after collapsing points. b In terms of students psychosocial lives, 44.4% of teachers responded that almost all or many 2L CCLC participants were exhibiting greater self-worth and self-esteem, and another 38.5% responded that some students were exhibiting improvements in this area. For other items. somewhat lower percentages and a wider distribution range were found. For example, while 41.2% of respondents observed that some students exhibited greater development as leaders. 17.6% said that only a few did so, and another 9.1 % said that none did so. Taken together. this means that more than a fourth (26.7%) of teachers thought that few or none of the program participants exhibited development as leaders. Further, while 39.0% of respondents said that 2L CCLC Technical Report 23some of the students were more aware of health issues as a result of participation, again well over a fourth (27.2%) of teachers said that few or almost none exhibited heightened awareness. In terms of students use of technology, however, almost half (45.4%) of teachers reported that almost all or many and nearly two fifths (38.5%) that some students showed improvements in computer literacy skills (see Table 6). The teachers also responded to 2 open-ended questions on the CCLCTQ which asked them to identity the strongest aspects of the 2P CCLC program as it operated at their school, as well as suggest areas for improvement. In the main, the tenor of the comments reflects the survey results shown in Tables 5 and 6, although there are some divergences. A full transcription of teacher comments to the open-ended questions can be found in Appendix B. A summary of these comments follows, presented in grade-level order (elementary school, middle school, high school). Elementary School Teacher Comments. At Woodruff Elementary School, the teachers who responded to the survey ( = 13) were quite positive about the program. One was effusive
[The program] is a safe, vibrant place for students to grow. Two others commended the enrichment activities which form the core of Woodruffs 2P CCLC program, specifically mentioning the piano classes, chess club, and Tae Kwon Do as strengths. They also thought students extra time in the computer lab was helpful and mentioned the health and nutrition class as benefits. Another respondent liked how the program attended to the individual needs of the students, and one commended the leadership and organizational skills of the site director. Only one respondent out of the 7 from Woodruff who included comments noted an area for improvement: adding new activities each semester that will encourage and enhance thinking skills.' 55 2P CCLC Technical Report 24Middle School Teacher Comments. Faculty respondents who taught in the middle schools (Southwest [zi = 23], Henderson [n = 21], and Mabelvale [n =36]) were generally quite positive about and supportive of the program, in terms of its academic impact, student engagement, and teacher commitment. Many teachers commended the academic assistance provided to students who were benefiting from the reinforcement of skills and concepts taught during the regular school day. As one put it, the program provides valuable help for students who have potential to succeed but need additional support, and another noted that the program gives students some extra incentives to work harder in class. Several mentioned homework help and the assistance with core subjects, literacy and math, as well as test-taking skills as strengths. Two respondents observed that the program improved students grades. These responses to the open-ended questions are in accord with the surveys quantitative results about the programs provision of academic assistance and its effect on school performance (see Tables 5 and 6). Another common theme in middle school teacher responses was students enjoyment of the afterschool activities, which respondents thought enhanced participants desire to attend regularly and improve their skills. As one respondent offered anecdotally, when the announcement comes over the intercom No After School [today] there is a groan, because Kids St Love It! Some individuals commented on the level of commitment and involvement in the 21 CCLC program by the teachers at their school (e.g., very dedicated). Comments from these respondents indicate that not only were students benefiting academically from more individualized attention, they reaped the psychological rewards of knowing that teachers cared enough about their development and progress to work in the afterschool program. This is in accord with the response to the survey item concerning the programs impact on student 21 ** CCLC Technical Report 25 participants (see Table 6), which indicated that nearly 45% of faculty thought that almost all/many exhibited greater self-worth and self-esteem. As one respondent put it, the potential to positively influence students was seen as a real strength of the program. One aspect seen often in the middle-school teacher comments (also a common feature of the staff interviews) was that the 2P CCLC gives our students a safe place to stay after school, providing a secure environment in which to engage in CCLC activities and concentrate on their academic work, which may not have been available in students neighborhoods, and perhaps in their homes. One respondent phrased this more negatively: the CCLC gives the students a place to go after school instead of getting into trouble. Still, this acknowledges a very salient aspect of the CCLC program as it operated in the LRSD study schools, that greater oversight of students activities beyond the regular school day was seen as a potential benefit both academically and behaviorally. In terms of areas for improvement, two areas were by far most commonly cited by middle school respondents: expansion of the program entailing more student participation and attendance, and greater parental engagement. There were few specific suggestions for how to achieve higher attendance rates, but one suggested additional incentives or rewards for coming to afterschool' regularly. Several others opined that their schools program should include more of an enrichment component to encourage greater attendance, such as crafts, art. or PE component,' 95 44, more camp-like/social activities, or just a more educationally fun dimension. Conversely, some teachers felt there should be more homework time, a greater focus on basic skills, or use of test scores to identify and focus on problem areas. Other respondents simply observed that a greater variety of activities with more flexible scheduling would improve the 2V CCLC at their school, which reflects the quantitative survey result * Afterschool was the term commonly used by respondents to refer to the 2L' CCLC program 21^ CCLC Technical Report 26indicating less satisfaction with the choice of activities (see Table 5). No respondent had a concrete suggestion for how to improve parental involvement
again, the quantitative survey results confirm that the program has had only measured success in reaching students families (see Table 5). High School Teacher Comments. The comments of those high school faculty who returned the questiormaires (at Hall [n = 36] and McClellan (n = 58]) to the open-ended questions on the CCLCTQ were generally close to those expressed by the middle school teachers, although there was more frequent address of the CCLCs impact on students academic achievement. The following are sample responses: Students are provided with a place in which they can improve their skills and grades to succeed academically. Students who take advantage of this service almost always do better in class and on tests. The strengths of the tutoring program are offering help to students who are behind academically, access to computers, and individual help with assignments. I think that this program has helped many borderline students to become academically successful. Reinforces classroom teaching in an alternative setting. Addresses a need for No Student Left Behind. Helps guide our students toward proficiency. Provides a vital service to students needing help with academics. CCLC Technical Report 27The central importance of the CCLC to the performance of students at Hall and McClellan is indicated by the last comment, and these others: should be mandatory for students who are failing, and if students with low academic performance could be required to attend the tutoring program, it would greatly benefit our students and our school. Other respondents cited as strengths of the program the 2L CCLC faculty and staff
its convenience and accessibility (on-site location)
availability before and after school
and, like the middle school teachers, the provision of a safe environment for students, although all of these were noted less often than the academic benefit of the program. In terms of improvement, again greater student and parental involvement was cited, and at one school (McClellan) more instructors (especially in science and foreign languages) to teach in the program. 21 CCLC Parent Questionnaire (CCLCPQ) St The CCLCPQ was completed by 82 parents (or guardians) of participants in the 21 CCLC program. The number is especially low because (1) parent surveys were not returned by Southwest Middle School and (2) only 1 parent returned a survey to Hall High School. Thus, the parental response rate can be estimated as just under 10% (9.55%), assuming a total possible 859 responses from one parent of the 859 students who, according to district enrollment data, were registered in the program. Thirty-three were parents of elementary school students, 22 were parents of middle school students, and 27 were parents of high school students. With the low response rate (9.55%) to the parent survey, its findings are to be regarded with caution, as they may not be entirely representative of the perceptions of parents of children participating in the program. The first section of the questionnaire was primarily devoted to gathering basic information about their child/rens activities in the program, such as whether they belonged to a 21 CCLC Technical Report 28 particular afterschool club, attended a Saturday program, or would participate in the Summer Camp. Parents responses indicated that more students were involved in academic groups such as the Homework Club (54.9%) and the Tutoring Club (62.2%) than in the Enrichment Club (39.0%), and that the typical 2P CCLC student in the study schools attended the program after school, during the week. Participation in before-school, Saturday program, and Summer Camp was low (approximately 10%, 2%, and 7%, respectively), a result that accords with information provided by 21** CCLC staff during interviews. In the first section, respondents were also asked if they had attended any of the 21** CCLCs programs for parents. While almost two thirds (64.6%) said that they had, this does not accord with staff interviews. Site coordinators and principals frequently commented on the lack of parental engagement and involvement, which sometimes resulted in discontinuing such activities. Given the low response rate to the parent survey, the picture of parental involvement drawn from other data sources may be the more accurate. The first section concluded with a question asking parents to identify which ethnic group they belonged to. The largest proportion of respondents (86.6%) identified their ethnic group as African American
7.3% as Caucasian American
1.2% as Hispanic American
and 1.2% as other. No respondent was identified as either Native American or Asian American. The second and third sections of the CCLCPQ were designed to elicit parents perceptions of the 21** CCLC as it was being implemented at their child/rens school and gamer a sense of how participation may have helped students academically and psychosocially. As shown in Table 1, the majority of parents responded very positively to the Centers programs and its impact on their children. They indicated a high level of agreement with all statements on the survey (as indicated by the combination of the categories Strongly Agree and Agree): that the 21** CCLC Technical Report 29Center was a safe place for their children to be (90.2%), that they would sign their child up again (90.2%), and that overall they were pleased with the CCLC at their school (89.0%). Two other statements were rated almost as highly: 85.4% strongly agreed or agreed that My childs Community Learning Center has good programs and that I would encourage other children at my childs school to join the Center. More than three quarters (80.5%) agreed with the statement that My child has enough different activities to choose from the Centers programs. The third section of the CCLCPQ asked parents to indicate the impact of the CCLC program on student participants in academic and social areas. Respondents clearly thought that the CCLC program is having a positive impact on academic performance, as more than three quarters (79.3%) responded that participation had helped their child get better grades. Table 7. 21 st CCLC Parent Questionnaire (CCLCPQ) Results Responses(%) Items My child's school Community Learning Center has good programs My child has enough different activities to choose from the Centers programs I feel my child is safe in the afterschool program I would sign my child up again for the program I would encourage other children at my childs school to join the Center Overall, I am pleased with the Strongly Agree And Agree 85.4 80.5 90.2 90.2 85.4 89.0 Neutral 9.8 13.4 4.9 7.3 12.2 8.5 Disagree and Strongly Disagree 0.0% Mean (SD) 4.58(.67) 0.0% 0.0% 0.0% 0.0% 0.0% Community Learning Center_______________________________________________________ Note. Percentages may not total 100% because of missing input from some respondents, n = 82 All items measured using a 5-point Likert scale: 1 = strongly disagree
5 = strongly agree 4.39(.72) 4.56(.59) 4.58(.63) 4.44(.7O) 4.55(.65) More than half of parent respondents indicated a positive impact of the program in other areas. For example, when asked to select all those ways in which they felt participation in the 2L CCLC had positively impacted their child/ren, 57.3% responded that that he/she felt better 21 CCLC Technical Report 30about her/himself
56.1% that their child better understood the importance of graduating from high school
and 54.9% that their child worked better with other students as a result of the program. A little less than half (48.8%) of respondents thought that participation had helped their child to talk more to their teachers, and 41.5% said it helped him or her to like school more. However, considerably fewer parents thought that participation in the program had helped their child behave better at school (32.9%) and come to school more often (23.2%). The CCLCPQ also asked parent'guardians to respond to 2 open-ended questions which asked them to identify the strongest aspects of the 2P CCLC program as it operated at their school, as well as suggest areas for improvement. A full transcription of these comments can be found in Appendix C. A summary of these comments follows, presented in grade-level order (elementary school, middle school, high school). Comments by Parents of Elementary School Children. The survey responses indicate that responding parents {n = 33) of children attending the 2L CCLC at Woodruff are generally satisfied with the program. Many identified additional help with homework as the best thing about the program. Like the teacher respondents, several parents liked its safe and secure environment, and one said that the program makes [students] do positive things that keep them out of trouble. Others mentioned the enrichment activities and field trips as the best aspect of the 2L' CCLC at Woodruff, with some commending the computer, chess, Tae Kwon Do, keyboard/piano, and exercise and nutrition classes in particular. In the program, noted one parent, students get to experience things and learn about things they wouldnt [otherwise] get a chance to do. Another parent praised the CCLC staff as very helpful and passionate about their work. They work closely with the student as well as with the parents in order to improve St the students work habits and learning abilities. Not all parents were pleased with the 21 2L CCLC Technical Report 31CCLC staff, however
in response to the question concerning areas for improvement, one respondent cited the after school workers and another wished that the teachers/caregivers [were] more patient. One respondent asked for more positive reinforcing. In terms of the activities offered, one parent asked for more variety, another felt more math was warranted. St and two others would have preferred more field trips. Finally, one respondent hoped that the 21 CCLC would receive money to keep the program going and alive. So many programs start and stop even though it is working for kids and really makes a difference. Comments by Parents of Middle School Children. The following summary is based on parent responses from Henderson (n = 9) and Mabelvale ( = 13)
Southwest did not return parent surveys to the researchers. In general, the parents of middle school children attending the 2P CCLC were positive about the program, although they did not seem as familiar with its activities as did the parents of elementary school children. Several respondents focused on the academic help offered by the 2P CCLC at their childs school, citing assistance with homework and getting better grades, more exposure to math and science, and improvement of students 'overall learning skills as strengths. Some parents praised the interaction between the teachers and students and the individualized attention students received, which was facilitated by the smaller groups attending the various afterschool activities. There were few recommendations for improvement from the parents of middle school students in the 2L CCLC, other than more structure and one on one tutoring. However, one parent felt that the various components of the 2L CCLC program at their childs school were not sufficiently publicized: I didnt know anything about any clubs, summer camp, or programs for parents. Comments by Parents of High School Children. Only one parent survey was returned by Hall High School, so the following summary largely reflects the responses of parental 2CCLC Technical Report 32 responses returned by McClellan High School (n = 26). Like the elementary and middle-school survey responses, parents of high schoolers attending the 2P CCLC were quite supportive of the program. Most respondents praised the academic assistance it provided their children: responses commonly referred to the help with homework and specific subjects like algebra, the improvement of grades, and the individualized work with teachers. Several commended the teaching staff, the way the teachers stay late just to work with my children. Once more, a number of parents mentioned the safety and security issue
for example, one parent liked the fact that the 2L CCLC provided children [with] an environment they can work efficiently in. The social and psychological benefits of the program (a theme seen also in the teacher survey comments) were praised by these parents, who commended the staffs concern for and reaching out to help their children not just to do better in school but to affect how they view the long-term benefits of education. For example, one parent described the best thing about the 2L CCLC this way: My child realizes how important his grades are to be able to graduate and have a good life, better job and go to college. The 2L CCLC program, commented another parent, gives the children a future when they think no one cares. In terms of areas of improvement, the parents of high schoolers had few recommendations. One parent mentioned greater variety of classes (especially more science and algebra), more staff for different subjects. and longer hours, but in general these respondents were quite happy with the 2L CCLC at their childs school. 2r CCLC Student Questionnaire (CCLCPQ) A total of 270 surveys were completed by 2L CCLC participants in the 6 study schools: Woodruff Elementary (n = 50, with 30 K2 surveys and 20 3-5 surveys), Mabelvale 2L' CCLC Technical Report 33Magnet Middle School (n = 56), Southwest Middle School ( = 58), Henderson Middle School (n = 26), Hall High School {n = 26), and McClellan High School {n = 54). This represents an overall 31% response rate, based on district information reporting that 859 students were registered for the program. A summary of the quantitative survey results, subdivided by grade level (K-2, 3-5, and 6-12), is presented below. Kindergarten Through 2^ Grade. All 30 respondents to the K-2 survey (see Appendix A) attended the only elementary school in the study. Woodruff. Two questions were designed to collect demographic data: grade (4 were kindergarteners, 6 in the 1* grade, and 20 in the 2"** grade)
gender (12 were boys, 15 were girls, with 3 not responding)
and ethnic group membership (26 were African American, 1 Asian American, 1 Caucasian American, and 1 Native American, with one not responding). One question asked if they would attend Summer Camp (19 Yes, 11 No). The majority of the survey asked respondents to mark bubbles labeled Agree/Dont Know/Disagree in regard to a set of 12 statements, simplified to match the reading comprehension level of the younger student. As indicated by a high level of agreement with all statements on the survey (see Table 8), these children appear to be enjoying and benefiting from the 21* CCLC activities. A discussion of each of the survey items follows. The afterschool teachers and staff are strong features of the program at the elementary school (Woodruff): a large majority of students (93.3%) Agreed with the statement I like the teachers who work with me and the statement Teachers give me help when 1 ask for it. In addition, more than three quarters (76.7%) agreed with the statement Teachers and other grownups in the program make me feel good. A majority of respondents liked the CCLC program (80.0%) and the range of activities offered (83.3%). More than three quarters (76.7%) 21* CCLC Technical Report 34agreed that I would sign up for the program next year and I think other kids should join the community learning program. Table 8. 2V CCLC Student Questionnaire (CCLCSQ) Results, Grades K-2 Items 1 like the CCLC programs I can choose from different activities I like the teachers who work with me Teachers give me help when I ask for it I feel safe in this program Teachers and other grownups in the program make me feel good I would sign up for the program next year I think other kids should join the community learning program The program has helped me to like school more The program has helped me to do better in school The program has helped me to behave better at school The program has helped me to feel better about Agree 80.0 83.3 93.3 93.3 76.7 76.7 76.7 76.7 63.3 63.3 76.7 86.7 Responses (%) Dont Know 6.7 16.7 6.7 6.7 23.3 16.7 13.3 16.7 26.7 26.7 16.7 6.7 Disagree 13.3 0.0 0.0 0.0 0.0 6.7 Mean (SD) 2.67(71) 2.83(.37) 2.93(.25) 2.93{.25) 2.77(.43) 2.70(.59) 6.7 3.3 10.0 10.0 6.7 6.7 myself__________________________________________________________________________ Note. Percentages may not total 100% because of missing input from some respondents, n = 30 a All items measured using a 3-point Likert scale: 1 = disagree, 2 = don't know, 3 = agree 2.72{.59) 2.76{.51) 2.53(.68) 2.70(.59) 2.80(.55) 2.70(.59) The afterschool teachers and staff are strong features of the program at the elementary school (Woodruff): a large majority of students (93.3%) Agreed with the statement 1 like the teachers who work with me and the statement Teachers give me help when I ask for it. In addition, more than three quarters (76.7%) agreed with the statement Teachers and other grownups in the program make me feel good. A majority of respondents liked the CCLC program (80.0%) and the range of activities offered (83.3%). More than three quarters (76.7%) agreed that I would sign up for the program next year and I think other kids should join the community learning program. The safety and security issue was also broached in these quantitative results: more than three quarters (76.7%) agreed that 1 feel safe in this program and The program has helped me behave better at school. And again, participation in the 2L CCLC appears to have a positive impact on students self-image, as 86.7% agreed that The program has made me feel better 21 CCLC Technical Report 35about myself. On two items, The program has helped me to like school more and The program has helped me to do better in school, the K-2 students were somewhat more ambivalent
63.3% of respondents agreed with these statements, but 26.7% responded Dont Know, and 10% Disagreed. However, these response rates still indicate that the 2P' CCLC is having a positive impact on how many elementary student participants regard school. Third Grade Through 5^ Grade. All 20 respondents to the 3-5 survey (see Appendix A) attended the one elementary school in the study. Woodruff. The first part of the survey was designed to gather demographic data: grade (6 were in the 3^* grade, 8 in the 4* grade, and 4 in the 5* grade, with 2 not responding)
gender (13 were boys, 7 were girls)
and ethnic group membership (17 were Afiican American, 2 Caucasian American, and 1 other). Next, students were asked 7 questions about their activities in the program, such as whether they belonged to a particular afterschool club, attended a Saturday program, or would participate in the Summer Camp after school was over. Reflecting the structure of the 21* CCLC at Woodruff, all students in this age cohort reported that they belonged to the Homework Club and nearly all (89.5%) reported that they belonged to the Enrichment Club, with one half reporting that they were also in the Tutoring Club. All students attended the program after school, and as there was no before school or Saturday program, none of the Woodruff respondents reported attending either of these. The main components of the 3-5 survey were designed to elicit students perceptions of the 21* CCLC as it was being implemented at their school and to provide a sense of how participation might be benefiting them academically and psychosocially (see Table 9). Like the K-2 cohort, these older elementary students felt that the afterschool teachers and staff were a strong feature of the program. A large majority (85.0%) of the respondents Strongly Agreed or 21* CCLC Technical Report 36Agreed with the statement that Teachers in the program give me help when I need it and 70.0% Strongly Agreed or Agreed that I like the teachers who work in the community learning center program. Almost two thirds (65.0%) Strongly Agreed or Agreed with the statement that Teachers and other adults in the program make me feel comfortable. Once again, the safety and security of the program is a prominent aspect of its appeal
three quarters of students reported that they felt safe in the program, and 70.0% liked the variety of activities from which to choose. Other responses from the 3-5 cohort suggest that these students are less satisfied with some aspects of the 2P CCLC at their school. Only a little more than half (55.0%) of respondents Strongly Agreed or Agreed with the statements 1 like the programs the community Table 9. 21 CCLC Student Questionnaire (CCLCSQ) Results, Grades 3-5: Implementation Responses(%) Items I like the programs the community learning center offers There are enough different activities from which to choose I like the teachers who work in the community learning center program Being in this program is better than other things I could be doing after school Teachers in the program give me help when I ask for it 1 feel safe in the after-school program Teachers and other adults in the program make me feel comfortable 1 would sign up again for the program I would tell other kids to sign up for the community learning center activities Strongly Agree And Agree 55.0 Neutral 15.0 Disagree and Strongly Disagree 30.0 Mean (SD) 3.60(1.5) 70.0 70.0 55.0 85.0 75.0 65.0 50.0 65.0 15.0 15.0 15.0 10.0 10.0 20.0 10.0 5.0 15.0 15.0 30.0 5.0 15.0 15.0 40.0 30.0 3.75(1.2) 3.70(1.3) 3.20(1.5) 4.35(.87) 3.75(1.3) 3.80(1.4) 3.10(1.4) 3.55(1.7) Note. Percentages may not total 100% because of missing input from some respondents, n = 20 a All items measured using a 5-point Likert scale: 1 = strongly disagree, 5 = strongly agree 2L CCLC Technical Report 37learning center offers and Being in this program is better than other things I could be doing after school. In addition, only half (50.0%) of students would sign up for the program again, and 30.0% of students Strongly Disagreed or Disagreed with the statement I would tell other kids to sign up for the community learning center activities. The final section of the 3-5 survey asked students to indicate the extent to which participating in the 2P CCLC program had helped their academic performance, relationship with teachers, and other issues related to school. As shown in Table 10, three quarters of these respondents felt that it had helped improve their grades, and 70.0% said that it had contributed to greater self-esteem. Almost two thirds (65.0%) responded that participation had helped them behave better at school and understand the importance of graduating from high school. Half said that they worked better with other students. Table 10. CCLC Student Questionnaire (CCLCSQ) Results, Grades 3-5: Impact Responses(%) The Center has helped me to... Like school more Come to school more often Gel better grades Behave belter at school Work belter with other students Feel better about myself Talk to my teachers more Understand the importance of A Great Deal 45.0 60.0 75.0 65.0 50.0 70.0 45.0 65.0 Somewhat 35.0 5.0 10.0 15.0 15.0 15.0 35.0 15.0 A Little 5.0 30.0 10.0 10.0 20.0 0.0 10.0 15.0 Not At All 15.0 5.0 5.0 10.0 15.0 15.0 10.0 5.0 Mean (SD) 3.10(1.0) 3.20 (1.0) 3.55 (.88) 3.35(1.0) 3.00(1.1) 3.40 (1.0) 3.15 (.98) 3.40 (.94) graduating from high school______________________________________________________ Note, Percentages may not total 100% because of missing input from some respondents, n = 20 All items measured using a 4-point Likert scale: 1 = not at all, 4 = a great deal The potentially positive impact of the 2P CCLC was less clear in two areas. When asked if they talked to teachers more, 45% responded A Great Deal and 35% Somewhat. Although there was some divergence on this issue, responses still indicate that a large proportion of the 3-5 respondents (80.0%) were interacting with teachers more than before. Survey results 21^ CCLC Technical Report 38also indicated that participation in the 2P CCLC appeared to have less impact on whether students liked school more, with 45.0% responding A Great Deal and 35% Somewhat. Despite this divergence, responses once again do indicate that a large segment of the 3-5 students (80.0%) felt that participation had helped them enjoy the learning process more than previously. Sixth Grade Through Grade. All 220 respondents to the 6-12 survey (see Appendix A) attended one of the middle or high schools in the study (Mabelvale [n = 56], Henderson [n = 26], Southwest [ = 58] Middle Schools, and Hall [n = 26] and McClellan [n = 54] High Schools). More students in the lower grades completed surveys than those in the higher grades: 85% were in the 6* through 9* grades, and the remainder in the 10* or 11* grades. No respondent was in the 12* grade. A little more than half (114) were male, 100 were female, and 6 did not specify gender. In terms of ethnic group membership, 82.3% of students identified themselves as African American, 4.1% as Native American, 1.8% as Caucasian American, 1.8% as Hispanic American, 3.2% as other, and 15 did not respond to the ethnicity question. Like the 3-5 students, the 6-12 students were asked about their activities in the program. such as whether they belonged to a particular afterschool club, attended a Saturday program, or would participate in the Summer Camp. Three quarters of students in this age cohort reported that they belonged to the Tutoring Club, and a little over a half reported that they belonged to the Homework Club (54.5%) and the Enrichment Club (52.7%). Nearly all (91.4%) students attended the program after school, with approximately a fifth (19.5%) attending the before school program (if available) at their school. Large proportions reported not attending either the Saturday program (87.7%) or the Summer Camp (78.6%). 2CCLC Technical Report 39The major part of the 6-12 survey sought to elicit student perceptions of the 21* CCLC as it was being implemented at their school and to provide a sense of how participation might be benefiting them academically and psychosocially. The highest percentage of students who Strongly Agreed or Agreed was found for the statement Teachers in the program give me help when 1 ask for it (77.7%), a result similar to that seen in the K-2 and 3-5 surveys. High percentages for the combined categories Strongly Agree and Agree were also found for the other statements about teachers and staff: 1 like the teachers who work in the community learning center program (63.6%) and Teachers and other adults in the program make me feel comfortable (61.8%). Once again, the majority of students indicated that they felt safe in the after school program (75.0%), another result quite similar to that seen in the K-2 and 3-5 surveys. Overall, a large proportion of the 6-12 respondents (72.7%) liked the programs offered by the 21* CCLC at their school (see Table 11). Approximately two thirds (67.7%) would sign Table 11. 2V< CCLC Student Questionnaire (CCLCSQ) Results, Grades 6-12 (middle and high school responses combined): Implementation Items I like the programs the community learning center offers There are enough different activities from which to choose I like the teachers who work in the community learning center program Being in this program is better than other things I could be doing after school Teachers in the program give me help when I ask tor it I feel safe in the afterschool program Teachers and other adults in the program make me feel comfortable I would sign up again for the program Strongly Agree And Agree 72.7 61.8 63.6 47.3 77.7 75.0 61.8 67.7 Responses(%) Neutral 21.8 24.1 24.5 29.1 14.1 18.6 28.2 23.6 Disagree and Strongly Disagree Mean (SD)= 5.5 13.2 10.5 23.2 7.3 5.9 9.1 8.2 Note: Percentages may not add up to 100% because of missing input from some respondents, n = 220 3 All items measured using a 5-point Likert scale: 1 = strongly disagree, 5 = strongly agree 3.94(.94) 3.72(1.05) 3.80(1.09) 3.39(1.29) 4.14(1.00) 4.08(.94) 3.77(1.00) 3.89(1.07) 21* CCLC Technical Report 40up again for the program while 62.3% would recommend the program to other kids. Nearly the same percentage (61.8%) Strongly Agreed or Agreed that there was enough variety in the programs offered by the 2P CCLC. Like the 3-5 cohort, students in the 6-12 group expressed less agreement with the statement that Being in this program is better than other things I could be doing after school, with less than a half (47.3%) agreeing. The last section of the 6-12 survey asked students to indicate the extent to which participating in the 21 CCLC program had improved their academic performance or relationship with teachers and students and other similar school-related areas (see Table 12). The most positive impact of the program was found in students response to whether the program had helped them understand the importance of graduating from high school: 70.0% said A great deal. In regard to other items, however, the 6-12 students were more ambivalent in their responses than the 3-5 students had been, although the results still indicate that this cohort St perceives the program to be of much benefit. When asked whether participation in the 21 CCLC had raised their grades, slightly less than half of these respondents (48.2%) responded A Great Deal and approximately a third (30.5%) responded Somewhat. Likewise, 46.4% responded A great deal and 25.5% Somewhat when asked whether the program made them want to attend school often, and 45.5% said it improved their self-esteem A great deal and 27.3% Somewhat. There was broader disagreement reported for three other items: only 39.1% felt that the program helped them A great deal to do better at school, 39.5% to work better with other students, 35.5% to talk to teachers more. And only a little more than a fourth (25.9%) of students thought that participation in the 2P CCLC made them like school more
slightly more (37.3%) responded Somewhat to this item. 2P CCLC Technical Report 41Table 12. 2V< CCLC Student Questionnaire (CCLCSQ) Results, Grades 6-12 (middle and high school responses combined): Impact The Center has helped to... Like school more Come to school more often Get better grades Behave better al school Work better with other students Feel better about myself Talk to my teachers more Understand the importance of graduating from high school A Great Deal 25.9 46.4 48.2 39.1 39.5 45.5 35.5 70.0 Responses(%) Somewhat 37.3 25.5 30.5 26.4 32.3 27.3 27.7 16.4 A Little 25.0 13.6 15.5 19.5 20.0 15.0 25.5 7.3 Not At All 10.9 12.7 4.5 13.2 7.3 10.5 10.0 5.0 Mean (SD)^ 2.79(.95) 3.07(1.06) 3.24(.88) 2.93(1.06) 3.05(.94) 3.10(1.01) 2.90(1.00) 3.53(.83) Note. Percentages may not total 100% because of missing input from some respondents, n = 220 All items measured using a 4-point Likert scale: 1 = not at all, 4 = a great deal Interviews The interview results are presented in two sections: (1) interviews with the 6 site coordinators, those directly responsible for administering the implementation of the 2P CCLC program activities at their school, and (2) interview results from the 6 principals whose schools were selected for the study. The responses of both site coordinators and principals are summarized below. Site Coordinators The 6 site coordinators interviewed for this evaluation had a range of experience in directing the 2L CCLC program at their school
in one case, the coordinator had been responsible for program oversight only since January 2006, and in another case the coordinator had been the first and only director since the inception of the program at that school 4 years ago. In general, however, the researcher found the site coordinators to be seasoned education professionals (a number of whom had had much administrative experience at their current school and elsewhere) who were also quite knowledgeable about the CCLC as it operated at their school. Some of the site coordinators were also teachers at the school or held other positions. 2L CCLC Technical Report 42such as Title I facilitator, technology or reading specialist, or dual enrollment program administrator. Responsibilities. All of the coordinators reported that they performed basic administrative tasks to ensure the smooth operation of the program: hiring and assigning teachers for appropriate subject areas
deciding upon a schedule of activities
keeping a variety of records (student attendance, instructor, payroll)
ordering and providing snacks
and securing transportation at the end of the day. Some respondents also regularly monitor the classrooms in which activities are taking place and address complaints or discipline problems as they arise. Other respondents noted that they were responsible for promoting and publicizing the CCLC program around the school as well as in the community. Goals of the CCLC at Their School. Most of the coordinators responded that the primary goal of the program was to provide students who were struggling academically (sometimes in a number of areas) with a safe environment in which to improve their skills. Some specifically mentioned that the program targeted students who were not performing at the proficient or advanced level on standardized achievement tests. However, any student attending the school was eligible to participate, and on occasion students from other schools who lived near a study school would attend. One coordinator observed that since the CCLC was designed as a community based program, she welcomed participants from the neighborhood. Typical 2T CCLC Student. Most commonly, coordinators described the typical participant as a student working at below grade level who needed extra help, and often a child without a structured learning environment in the home. One coordinator noted that the program provided a vital diversion for those students who were getting into trouble a lot, although no respondent characterized the CCLC as a haven for those with discipline problems. At the same 2 P' CCLC Technical Report 43 time, some respondents said that proficient or advanced students, particularly those who were self-motivated, would attend the CCLC as a secure and safe place to expand their skills and have ready access to technology (e.g., computer and printer) that may not have been available at home. At one school, such students would act as peer tutors, assisting their classmates in various academic areas. None of the site coordinators reported any significant differences in participation by students of various ethnicities in any of the 2P CCLC activities. Most said that they had observed approximately the same demographic makeup in the afterschool program as was evident in the school as a whole (predominantly Afiican American
see Table 1). Two coordinators noted that there had been some problems recruiting and retaining Hispanic students. which they attributed to language difficulties (nonnative English speakers) and also to heightened parental concern about their child being out after dark. In terms of gender, about half of the site coordinators felt that more females than males were attending afterschool, but the other half had observed no majority of girls. Student Recruitment and Retention. The site coordinators described a variety of ways of encouraging participation in the 21 CCLC at their school. Nearly all developed some sort of flyer or application for dissemination at student orientation meetings prior to the start of school. usually August. Some coordinators continued to promote the program throughout the year at student or parent events, such as an Open House or Parent-Teacher Association meetings
others made the program a constantly visible part of the school by putting up notices or posters about afterschool activities at various locations on campus
many would routinely make announcements at faculty meetings and/or provide teachers with 2L CCLC information in preparation for conferences with parents. The high school site coordinators seemed especially 21* CCLC Technical Report 44 involved in 2P CCLC promotion and recruitment. In addition to the strategies mentioned above, Hall maintained a website detailing activities in the program, and the coordinator sent out weekly emails to the entire student/parent email list. The coordinator at McClellan used a PhoneMate system that sends automated messages about the Center to parents
arranged for the local carwash to run an LCD display message about the Center
and will be adding an announcement about after school activities to the new school marquee on the street. Several of the site coordinators stated that they targeted students who were performing below proficiency, on the basis of standardized test scores and/or teacher recommendations. In some cases, attendance was mandatory
at Woodruff Elementary, for example, the site coordinator noted that any student who scores below grade level in reading and math is automatically put into the afterschool program, after the parent approves. At McClellan, any student who had a GPA below 2.0 and was involved in any competitive extracurricular activity (e.g., band, choir, athletics) was required to attend 100 minutes a week in a tutoring program. Attending the schools 21' CCLC program was the most common way of satisfying this requirement. Implementation. Most of the CCLC programs offered activities afterschool, although a few included a before-school component. For example, at the case study school Mabelvale, there was an hour of activity (7:30 a.m. - 8:30 a.m.) prior to the start of the regular school day. Students went to the environmental lab to do homework or other independent seatwork until about 8:10 a.m., when they left to eat breakfast in the cafetorium. Others could stay until 8:30 a.m. Some of the site coordinators observed that when they offered a beforeschool program, attendance was consistently too low to justify its continuation. Others noted 21' CCLC Technical Report 45st that their school already had a before-school program funded by sources other than the 21 CCLC grant. The majority of the CCLC after school activities began about 15 minutes after the conclusion of the regular school day, usually 3:30 or 4 and running until 5:30 or 6. As expected, there was variability in terms of which activities were done at what time, since it was a basic supposition of the 21 CCLC grant that the schools would design the program in ways that best met the needs of their students. For example, at Woodruff Elementary the afterschool program began with a half-hour of outside play, followed by a snack
the Homework Club met for an hour
and the day concluded with various enrichment activities, such as violin, cello, piano, or Tae Kwon Do lessons
exercise and nutrition classes
chess and computer work
or Boy Scout meetings. At Mabelvale Magnet Middle School, afterschool students were subdivided into grades (6-8) and then directed to individual classrooms for academic work during the first hour and enrichment (art or music, crafts) during the remaining 45 minutes to an hour, although there was variation depending on the schedule. The basic schedule (academic work followed by enrichment activities) was similar in the high schools, although Hall was more oriented toward the three primary areas (algebra, geometry, and literacy) of the Arkansas Comprehensive, Assessment, and Accountability Program (ACTAAP), whereas at McClellan students were subdivided by the specific area in which they needed improvement (e.g., computers, math. English, civics/history). In regard to the relationship between the school and its 2P CCLC, it was apparent that students and staff alike made little distinction between the regular school day and afterschool program activities. Site coordinators gave several reasons for this: frequently students were engaged in homework or other seatwork that built upon those literacy and math skills that were 2P CCLC Technical Report 46the focus of exercises during the regular school day. In addition, at all of the schools except the elementary school (Woodruff), members of the 2P CCLC staff were also teachers at the school. Also, although neighborhood children who did not attend the local school offering a 2P CCLC program were permitted to attend the afterschool activities, the vast majority of program participants were students at the school. Finally, there was little or no geographic separation. since no school reported having a distinct location for program activities
for example, in the case of Mabelvale (the case study school), students would meet in common central areas such as the library, cafetorium, or the computer lab. All of these aspects helped cement the relationship between the 2L CCLC and the school as continuous rather than simply contiguous. In some of the schools, arts enrichment was a regular feature of the 2L CCLC program. This was especially true of Woodruff Elementary, because in 2005-06 it became a member of the Arkansas A+ Schools Network, a consortium dedicated to comprehensive school reform. A+ schools combine interdisciplinary teaching with daily arts instruction, offering children opportunities to learn through multiple pathways (Arkansas A-Plus Network, 2006). Thus, the arts focus during the regular school day is just continued into afterschool activities, incorporating ballet, musical instruction (piano and strings), art, acting and drama, and also sports, such as Tae Kwon Do, tennis, and golf. Woodruff has initiated a number of partnerships with local cultural and recreational groups
for example, members of the Arkansas Symphony Orchestra were recruited to conduct violin and cello classes for the 2P CCLC students
during the first year, the musicians were paid, but they now do it for free. The principal noted that weve found that these arts classes help the kids in all areas of academics. Woodruff also regularly hosted Parents Nights, at which the children would perform for their families and friends (and on one 21 CCLC Technical Report 47 occasion, for Arkansas Governor Mike Huckabee). These events helped build students self- esteem and also resulted in improved school behavior. St At Hall High School, in 2004-05 an arts enrichment component was added to the 21 CCLC program, in which students did crafts projects such as tie-dye and jewelry, and even a larger stained-glass mural depicting the school logo (a Native American peace pipe) for placement outside Halls main office. They also did more traditional arts like painting, although here too the students were encouraged to express their creativity. For example, the site coordinator at Hall showed the researcher a still life made entirely using various concentrations of coffee as paint. 2r CCLC Involvement with the Community. Every school in the study sought community involvement in their 2P CCLC programs by initiating partnerships with local organizations, businesses, civic and cultural centers, or universities and colleges. Nearly all the site coordinators reported an ongoing affiliation between their CCLC and Let Our Violence End (LOVE), a Little Rock-based nonprofit organization that uses peer volunteers to speak with students about the causes and implications of school and neighborhood violence. Aside from LOVE, however, the schools arranged CCLC extra-program activities derived from a variety of community partnerships. Woodruff Elementary has an active association with Big Brothers and Big Sisters, who help students in the before- and afterschool program with reading, math, and social science. Other schools partnered with groups such as the Audubon Society or with local colleges and universities to conduct classes or workshops in natural or biomedical science. For example, students at Remington College (Little Rock Campus) who were enrolled in a workstudy program helped Woodruffs 2L CCLC participants in areas related to their majors (education, pre-med, computers). At Henderson Middle School, afterschool students learned 21 CCLC Technical Report 48about water ecosystems and methods of conservation while working on a water sampling project at the University of Central Arkansas. Hall High School developed a partnership with Junior Achievement, a national organization that sponsors local business leaders to speak to students about business fundamentals. Development of students physical skills was also a feature of some of the afterschool programs. A parent of a student at Mabelvale offered karate classes, not (as the principal was quick to point out) to teach students how to fight but rather to learn basic self-defense techniques as a means to enhance their confidence and self-esteem. Every Thursday afternoon was drum day at Southwest, where a local percussionist taught the students using a variety of traditional African drums. Most Successful Aspects of the 2f CCLC. Nearly all of the site coordinators reported that one of the key strengths of the program was that it provided safe, secure environment for children at a time (after school was out and before parents arrived home) when they were most at risk. This was assumed to be especially true of this study population, since many students were from lower SES backgrounds, lived in transient housing, and were being raised by a single parent or parents with multiple jobs. Often the afterschool setting was the only structured environment to which the student would have access during that time. Having a quiet place to complete homework, with individualized assistance provided as needed, was very important, if not critical to addressing childrens academic deficits. Even those 2U CCLC students who were working at or above proficiency reaped the benefits of the secure setting as a place to develop their math and literacy skills. The second positive aspect of the afterschool program often mentioned by the site coordinators was the many opportunities (the community activities, access to technology, arts enrichment) afforded the students which they otherwise may not have had. Several stressed the 2 U'CCLC Technical Report 49 academic benefit of these activities, in that the experiential, hands-on nature of such projectbased learning developed skills and abilities in ways that traditional approaches such as direct classroom instruction or seatwork perhaps could not. At the same time, some of the site coordinators noted that it was these activitiesthe arts enrichment in particularthat tended to pull the students in and help keep attendance rates high. Those children who were struggling academically, particularly in the middle and high schools, were enticed by the prospect of learning in a fun way, rather than avoiding those subjects (math and reading) which had seemed daunting in the past. A third positive aspect cited by a number of the site coordinators concerned the psychological reward of simply having a program for the students, who were made more acutely aware that there was a group of adults, other than family, who were devoting their extra time and energies toward helping them achieve academic as well as social and behavioral success. As one site coordinator put it, you cant measure the importance to kids of knowing that someone cares. In this regard, most of the site coordinators had high praise for and satisfaction with the afterschool staff and all those affiliates who coordinated activities and projects for the 2P' CCLC at their school. Areas for Improvement in the 2f CCLC Program. Nearly all the site coordinators mentioned two areasstudent retention and attendance rates and parental involvementas ongoing challenges. Of all 6 schools in the study, only Woodruff Elementary reported no problems either attracting participants to the program or maintaining steady attendance (97% or above, each day). This might be expected, given the younger age of the Woodruff students (K-5). However, all other site coordinators said that while recruitment efforts at the start of the year generally were successful, and initial enrollment figures were quite high, maintaining 2L CCLC Technical Report 50regular attendance by participants was a challenge. In general, attendance rates tended to drop most in the spring semester as warmer weather returned and the days grew longer, but even in the late fall and winter parents were anxious about their children being out after dark. Problems with irregular or infrequent attendance were more associated with the older students (9*-12* grades), because as these children matured there were more demands on their time (sports, other clubs, friends and social life, or even a job) which likely interfered with their participation in activities associated with schoolwork and academic life. This is likely why the Hall and McClellan site coordinators were so active in year-round promotion of 2P CCLC activities at their high schools, as a hedge against declining attendance rates. Except for Woodruff Elementary and Southwest Middle School, parental involvement in 21" CCLC activities was another area for improvement reported by the site coordinators. One estimated that approximately 65% of parents in his school could be described as having low engagement with the program and their child, another 25% medium engagement, and the remainder (10%) high engagement. Several mentioned various strategies they attempted to elicit greater parental involvement, such as Parents Nights, meetings on Saturdays, partnerships with the PTA, and so on, without much success. Still, most of the coordinators noted that gamering parents interest and participation in any of their preteens and teens school activities was difficult. Woodruff staff attributed the younger student ages and the schools arts focus as likely reasons for their more frequent parent involvement, and Southwest cited the field trips in which parents accompanied their children as an important factor in this regard. In addition, both site coordinators at these schools reported that any public performance by their child/ren in the 21" CCLC program was guaranteed to bring out the parents. 21" CCLC Technical Report 51Some of the site coordinators mentioned occasional problems not having enough teachers (especially in math and science). In addition, one felt that not the number but the commitment of instructors in the 2P' CCLC could be better, that occasionally staff was lax in their duties. At the same time, most site coordinators believed that students in the afterschool program became closer and more responsive to their teachers, since the student-teacher ratio was lower and students and staff alike tended to be more casual than during the regular school day. Two site coordinators wished that they could offer a greater variety of activities and experiences to the students, but felt constrained by both budget and available staff. School Principals In general, the principals interviewed for the evaluation tended be somewhat less familiar with the day-to-day operation and activities of the 2P CCLC at their school than the site coordinators, and this was particularly so if the site coordinator had been administering the afterschool program for more than a year. Also, some of the principals themselves were new to the school (in one case, hired in June 2006). Still, all of the principals had valuable insights to offer. having been employed in administrative or supervisory positions in LRSD and elsewhere. Responsibilities. Most of the principals had a supervisory role in the daily operation of the 21* CCLC centers, working with the site coordinator on the basic schedule
hiring certified teachers in the core areas (which in the elementary school meant math and literacy, and in the middle and high schools English, science, and math)
developing programming appropriate to the centers goals
and ensuring that the necessary supplies were available. Some of the principals took on other responsibilities, such as helping to write the initial grant and making sure that students were utilizing technology in the 21* CCLC classrooms. 21* CCLC Technical Report 52Goals of the CCLC at Their School. Principals identified a number of goals specific to the needs of their student population, but the purpose most commonly noted was enrichment and extension of those subjects taught during the regular school day. Elementary school children often lacked supervision and assistance with homework in the evenings, and as there was seldom a skilled adult reader at home, many of the Woodruff children had no opportunity to practice reading. We just needed the funds and materials to address the problem, said the Woodruff principal, and thats what the 21 CCLC [grant] does. At the middle and high school levels, focus on math and literacy skills was paramount, with an eye toward improving student achievement scores. However, a number of the principals noted that their program included enrichment activities in music, art, drama, science as well. One high school principal commented that their program also had a social/behavioral component, as it primarily served incoming 9* graders who often needed assistance making the transition from a middle to a high school environment. Typical 2f CCLC Student. The principals said that their program was primarily designed for students performing at or below proficiency, although it was open to all those desiring additional assistance. However, like the site coordinators, they described the typical attendee as a child academically on the bubble, who just needed an extra push, extra time, the one-on-one work, as one put it. They also echoed the coordinators view that the 2 CCLC offered students a supervised place to study, without distractions, as well as access to technology (computer and printer) and print resources not often available to them. One middle school principal commented that probably 20-25% of their students had a computer at home. Again several of the principals mentioned that the high-performing participants (who really didnt 2L' CCLC Technical Report 53need to be there) often acted as tutors, benefiting from the added responsibility and extra challenge of learning how to explain concepts to their peers. Student Recruitment and Retention. Like the site coordinators, principals at the middle and high schools noted that while initial recruitment was seldom a problem, maintaining steady. frequent attendance across the course of the school year was an ongoing challenge. When queried about the possible reasons for this, one middle school principal responded that parents are not knowledgeable about the importance of keeping their kid in the program, were not fully committed
others said that social activities and athletic events took precedence. Another middle school principal said that the shift to Daylight Savings Time in the fall caused attendance to decline across the winter months, because the parents of the younger students (6* and 7'* graders) did not want their children out after dark. One high school principal speculated that the increasing number of Hispanic students, many of whom were English language learners, were reluctant to participate in the 2P CCLC program because they feared problems in communication. At the same time, he felt that this could be addressed with special outreach to this community. Overall Strengths and Weaknesses of the CCLC Program. Most of the principals echoed the observations made by the site coordinators about the best aspects of the 2L CCLC program at their school: (1) the provision of safe, supervised setting for students at a time when they were perhaps most vulnerable
(2) access to many opportunities (especially assistance with building literacy and math skills and the arts enrichment activities)
and (3) the psychological benefits of knowing that there are caring adults who want to help them succeed, as one principal put it. One middle school principal cited the programs success in improving math skills via their Algebra Camp, which helped prepare 7* and 8* graders for the basic, required 21 CCLC Technical Report 54 algebra course in 9* grade (the first year of high school). Another thought that one of their programs best features was its facility in helping students make a transition from the behavioral and social norms of an elementary school to those of the middle school. One of the high school principals also mentioned that since their 2P CCLC served predominantly 9* graders, the program served the same useful function: students see middle school as a social place, versus the culture of a high school, which is academic, and potentially a place from which they will continue their studies, he noted. One respondent summarized the general tenor of the high school principals responses regarding the significant contribution made by the 2L CCLC to their educational objectives: [It is an] extremely valuable part of our school
we need opportunities to work with kids, specifically in smaller settings, but also where we can really target assistance. whether its in literacy, or whether its in algebra or geometry, or whether its some kind of social skill, whatever it is
we need those opportunities. There is oftentimes so much crammed into the normal school day, that if we didnt have the extended program, before and after school, we would not be able to meet all of our needs as effectively as we can now. With regard to areas for improvement, once again achieving steady and frequent attendance and fomenting parental involvement were most commonly mentioned. Only Woodruff reported few problems with these issues
in fact, the principal identified as one challenge trying to get enough people [staff] to accommodate all the kids, [since] the payroll was full. But this response was atypical, and probably in some measure attributable to the fact that Woodruff, the only elementary school in the study, has a younger student population. The other principals noted that, in order to increase rates of participation, they had to confront the 21* CCLC Technical Report 55multiple concerns and distractions faced by the average middle and high school student. Even those who were academically well focused had some difficulty keeping on task after being in school all day. Gamering parents commitment, to make sure that their children were coming to the 2P' CCLC every day as well as engagement with advertised center activities, was an ongoing challenge. At one school, developing connections between the Parent-Teacher Association and the 2P CCLC to promote the centers activities and benefits helped improve parental involvement, and as noted above any public performance (e.g., involving music, drama) by their child/ren was certain to bring out the parents. Several of the principals lamented the loss of their summer bridge program, which in the main had been quite successful in helping students transition into a middle or high school and addressing the potential loss of skill development that often occurs over the summer months (described by one respondent as grade recovery camp). They hoped that it would be re-funded in the future, either by the 21* CCLC grant or via other sources. Mabelvale Case Study Classroom Observations The researcher observed three 2 CCLC sessions (two afternoon sessions and one morning session) on two separate days at the case study school, Mabelvale Magnet Middle School. In each of the two afternoon sessions, the researcher watched activities in three different classrooms, because students were divided by grade (6*, 7*, and 8*). Instructors were always present (all of whom were teachers at Mabelvale), but no parent was observed at any session. Monday, April 17. The researcher observed 6*-grade students working in a computer lab doing two activities: half the children used the interactive PLATO system on Playstations 2P CCLC Technical Report 56and half were working on PCs. Three adults were present, all teachers at Mabelvale, who were directing and supervising the students. The staff described the students activity as an extension of school day skills. There were 20 students in attendance, of whom 18 were African American, one Caucasian, and one biracial, and most were clustered into small groups of 2-4 students, who appeared to be friends. A few students (2-3) were isolated from their peers and worked alone. Most of the students were involved in the PLATO activity and computer work the majority of the time, and these activities appeared to be engaging for them most of the time. The students would choose which area (e.g., math, literacy, geography) to work on. Teachers provided individual attention when asked, and interacted in a friendly way with students, often joking with them. The researcher did not observe any significant conflicts among students
any minor difficulties were quickly and amicably resolved. There was much intergroup discussion and a generally high level of positive energy in the lab, which reflected the students level of engagement with their work, and perhaps also the lab environmental conditions - it was very cold. On April 17, the researcher also observed the 7* grade CCLC session, which took place in the schools cafeto
This project was supported in part by a Digitizing Hidden Special Collections and Archives project grant from The Andrew W. Mellon Foundation and Council on Library and Information Resoources.

<dcterms_creator>Center for Research in Educational Policy, University of Memphis, Memphis, Tennessee</dcterms_creator>