The literacy-related practices of principals in high schools with improved student reading achievement
TABLE OF CONTENTS 4 LIST OF TABLES 7 Chapter I - RESEARCH PROBLEM AND STUDY DESIGN 9 Introduction 9 Statement of the Problem 11 Purpose of Study 18 Research Questions 20 Conceptual Framework 21 Methodological Design 24 Site Selection 24 Instruments 26 Data Collection and Analysis 28 Summary of Related Literature 29 Significance of the Study 32 Limitations of the Study 33 Summary 34 Chapter II - REVIEW OF THE LITERATURE 36 Introduction 36 Concerns About Adolescent Reading Comprehension 37 Solutions from the Classroom and from the Principal' s Office 46 Classroom Strategies 47 Literacy Leadership 52 Collaborative Leadership 67 Summary 70 Chapter III-RESEARCH DESIGN AND METHODS 72 Rationale for Mixed-Methods Case Study Design 73 Research Methods 79 Selection of Research Sites 79 Selection of Participants 85 Data Collection Procedures 87 CAPT Results.. 87 Steps of Study 89 Survey Methodology 94 Sampling Procedures 97 Interview Methodology 97 Document Analysis 101 Data Analysis Methods 102 Validity and Reliability 108 Research Bias and Reactivity 108 Significance of the Study 116 Limitations of the Study 117 Summary 118 Chapter IV - FINDINGS 119 Data Collection Process 120 Major Findings 124
5 Case 1: Cannon High School 125 Case 1, Research Question 1 125 Collaborative Leadership and School Capacity Development 126 Strategic Use of Student Assessment 131 Professional Development for Administrators and Teachers 132 Instructional Practices 132 Interventions to Improve Achievement among Struggling Students.. 133 Whose Job is it to Teach Reading? 133 Case 1, Research Question 2 135 Collaborative Leadership and School Capacity Development 135 Strategic Use of Student Assessment 142 Professional Development for Administrators and Teachers 144 Instructional Practices 145 Interventions to Improve Achievement among Struggling Student.... 146 Case 1, Research Question 3 147 Case 1, Research Question 4 152 Summary of Case 1 155 Case 2: Kelly High School 157 Case 2, Research Question 1 159 Collaborative Leadership and School Capacity Development 159 Strategic Use of Student Assessment 160 Professional Development for Administrators and Teachers 164 Instructional Practices 165 Interventions to Improve Achievement among Struggling Students. 165 Whose Job is it to Teach Reading? 165 Case 2, Research Question 2 167 Collaborative Leadership and School Capacity Development 167 Strategic Use of Student Assessment 174 Professional Development for Administrators and Teachers 176 Instructional Practices 178 Interventions to Improve Achievement among Struggling Students....78 Case 2, Research Question 3 179 Collaborative Leadership and School Capacity Development 179 Strategic Use of Student Assessment 183 Professional Development for Administrators and Teachers 183 Instructional Practices 184 Interventions to Improve Achievement among Struggling Students... 184 Summary of Case 2 184 Case 3: Learner Technical High School 187 Case 3, Research Question 1 187 Collaborative Leadership and School Capacity Development 188 Strategic Use of Student Assessment 192 Professional Development for Administrators and Teachers 193 Instructional Practices 193 Interventions to Improve Achievement among Struggling Students.... 193
6 Case 3, Research Question 2 194 Collaborative Leadership and School Capacity Development 194 Strategic Use of Student Assessment 196 Professional Development for Administrators and Teachers 197 Instructional Practices 198 Interventions to Improve Achievement among Struggling Students...200 A Sixth Construct: Student Motivation 201 Case 3, Research Question 3 201 Collaborative Leadership and School Capacity Development 205 Strategic Use of Student Assessment 206 Professional Development for Administrators and Teachers 206 Instructional Practices 206 Interventions to Improve Achievement among Struggling Students .. 207 Case 3, Research Question 4 207 Summary of Case 3 208 Cross-Case Analysis 212 Collaborative Leadership and School Capacity Development 212 Strategic Use of Student Assessment 216 Professional Development for Administrators and Teachers 219 Instructional Practices 224 Interventions to Improve Achievement among Struggling Students....227 Summary of Findings 228 Chapter V -- DISCUSSION OF FINDINGS AND RECOMMENDATIONS 231 Introduction 231 Discussion of Findings 231 Research Question 1 233 Research Question 2 236 Research Question 3 241 Research Question 4 243 Recommendations for Practitioners 244 Recommendations for Leadership Development Programs 248 Recommendations for Future Research 249 Summary 253 REFERENCES 258 APPENDIX A: Definition of terms 268 APPENDIX B: Survey Instrument for School Administrators 271 APPENDIX C: Survey Instrument for Teachers 273 APPENDIX D: Interview Protocol for School Principals 275 APPENDIX E: Interview Protocol for Other Leaders 278 APPENDIX F: Interview Protocol for Teachers 281
7 LIST OF TABLES Table 1: Research Methods 74 Table 2: Methodology of Multiple-Case Study 78 Table 3: CAPT Reading Scores of Three Case Studies 82 Table 4: Data Collection Methods 90 Table 5: Sample Survey Questions 97 Table 6: Sample Interview Questions 101 Table 7: Coding System 104 Table 8: Surveys Distributed and Collected 123 Table 9: Self-reported Literacy-related Practices at Cannon High School 127 Table 10: Cannon High School's Mean Scores in the Practice of Five Literacy-related Constructs 130 Table 11: Self-reported Practice by Department of Accepting Responsibility to Teach Reading at Cannon High School 134 Table 12: Literacy Capacity Survey Statements about Cannon Principal's Practice 136 Table 13: Self-reported Literacy-related Beliefs at Cannon High School 149 Table 14: Self-reported Literacy-related Practices at Kelly High School 161 Table 15: Kelly High School's Mean Scores in the Practice of Five Literacy-related Constructs 164 Table 16: Self-reported Practice by Department of Accepting Responsibility to Teach Reading at Kelly High School 166 Table 17: Literacy Capacity Survey Statements about Kelly Principal's Practices ...168 Table 18: Self-reported Literacy-related Beliefs from at Kelly High School 180 Table 19: Kelly High School's Mean Scores in the Beliefs of Five Literacy-related Constructs 183 Table 20: Self-reported Literacy-related Practices at Learner High School 189
8 Table 21: Learner High School's Mean Scores in the Practice of Five Literacy-related Constructs 192 Table 22: Literacy Capacity Survey Statements about Learner Principal's Practices. 195 Table 23: Self-reported Literacy-related Beliefs from at Learner Technical High School 202 Table 24: Learner High School's Mean Scores in the Beliefs of Five Literacy-related Constructs 205 Table 25: Cross-case Comparison of Literacy-related Practices and Beliefs 211 Table 26: Appendix B: Survey Instrument for School Administrators 271 Table 27: Appendix C: Survey Instrument for Teachers 273 Table 28: Appendix D: Interview Protocol on Literacy-related Practices for School Principals 275 Table 29: Appendix E: Interview Protocol on Literacy-related Practices for Other Leaders 278 Table 30: Appendix F: Interview Protocol on Literacy-related Practices for Teachers 281
9 Chapter I RESEARCH PROBLEM AND STUDY DESIGN Introduction In the late 1990s, about 15 years after launching a massive school reform effort, the educational community began to realize that it had been ignoring a fundamental need among high school students: reading comprehension skills (Vacca, 1998). The reform movement had addressed complaints raised in 1983 in A Nation at Risk: The Imperative for Educational Reform, among them that children were not reading with the skills expected for their age level. Federal, state and local agencies responded with early childhood reading initiatives designed to enable all children to read by Grade 3. Meanwhile, critics identified yet another problem: adolescent readers who struggled with reading comprehension (Buehl, 1998; Vacca, 1998). Reform efforts were improving skills of elementary school children, but national test scores showed stagnating or declining reading skills among high schools students through the 1990s (Buehl, 1998) while at the same time the occupations awaiting them upon graduation demanded increasing literacy skills (Barton, 2000). The International Reading Association soon afterward issued a position paper calling on the nation to turn its attention to that group (Moore, Bean, Birdyshaw, & Rycik, 1999). By high school, the IRA argued, most students had learned fundamental reading skills, but many had not mastered the more sophisticated reading skills needed to comprehend materials in secondary schools; in short, students had learned to read but could not read to learn.
10 The specific skill that appeared to be lacking was reading comprehension, part of a five-stage process of development (National Institute of Child Health and Human Development, 2000; Snow, 2002; Snow & Biancarosa, 2003). The first wave of responses to the concerns about inadequate adolescent reading skills targeted the classroom, with a plethora of books flooding the market on reading strategies for secondary schools. A second wave of responses to the problem of struggling adolescent readers has been aimed at leadership practices. National organizations and policy makers have called upon high school principals to use their authority and skills to address what the organizations deemed a crisis and a systemic denial of a civil right (Berman & Biancarosa, 2005; International Reading Association, 2006; National Association of State Boards of Education, 2005; Phillips, 2005). For purposes of this study, "literacy" was used in its narrowest sense to refer to reading comprehension because it was that skill at the secondary level that served as an important predictor of academic success (Barton, 2000). In the professional literature, the term "literacy" was used both broadly and narrowly. In its broadest contexts, it referred to the ability to process material through reading, writing, listening, speaking and seeing (Taylor & Collins, 2003). Of those, only reading and writing was tested in Connecticut's battery of exams, known as Connecticut Academic Performance Tests (CAPT). "Literacy" was defined much more narrowly in reports dealing with secondary schools to mean reading comprehension because of its fundamental role in learning. Such was the case in the seminal work Reading Next — A Vision for Action and Research in Middle and High School Literacy: A Report to Carnegie Corporation of
11 New York (Biancarosa & Snow, 2004). While most high school students could read words, up to 60% of high school seniors could read with the understanding expected of students at their grade level (NCES, 2005). It was that gap between abilities and expectations that caused alarm and required new combinations of classroom and leadership practices to address (Biancarosa & Snow, 2004). Researchers have looked in several places in search of effective practices. Over the years, researchers concerned with literacy have examined practices at the primary school level (Dole, Duffy, Roehler, & Pearson, 1991). Since 2000, the focus on the issue has widened to include secondary schools. A shift also occurred to include not only classroom practices but also leadership practices. This study focused on the latter. This chapter provides an overview of my issue of concern - the literacy-related practices and beliefs of high school principals. It offers a statement of the problem and the purpose of this study. It also lists the study's research questions. The chapter includes a discussion of the conceptual framework in which this research is rooted along with an overview of related literature, discussed in greater detail in Chapter II. Finally, the chapter introduces the study's methodological design, with an explanation of the instruments to be used, the significance and the limitations of the study, explained further in Chapter III. Statement of the Problem In schools that showed the most progress in improving reading comprehension, leaders ensured that reading was taught across disciplines (CARR, 2001). The notion of teaching reading proved to be a hard sell to secondary teachers (Beers, 2003; Billmeyer
12 & Barton, 2002; O'Brien, Stewart, & Moje, 1995). Content-area teachers argued that they were hired to teach their specialty, not reading (Snow, 2002). Developing reading skills in students, teachers said, would take time away from students learning the content material. Besides, teachers said, they were not skilled to teach reading in their content area. It was not until 2003, for instance, that the state of Connecticut, where this study was grounded, required pre-service teachers to pass a course in content-area reading. With all of this resistance, the issue then became one of leadership (Biancarosa & Snow, 2004; Sanacore, 1994; Snow, 2002; Snow & Biancarosa, 2003; Taylor & Collins, 2003). In the past decade, some secondary schools in Connecticut whose students showed improvement in reading comprehension were led by principals who recognized the problem of inadequate student reading skills and had committed to help solve it (Baron, 1999; CARR, 2001). While knowing reading strategies was useful for individual teachers, improving reading achievement in content areas throughout a school required "getting to the guy who controls the bells" (personal conversation with researcher Kylene Beers, on March 3, 2005). Studies showed that school principals exert influence, albeit indirectly, on student achievement (Hallinger, Bickman, & Davis, 1996; Hallinger & Heck, 1998; Waters, Marzano, & McNulty, 2003). Some of the most recent literature on literacy has begun to incorporate a look at leadership (Biancarosa & Snow, 2004; Meltzer, 2002). The Carnegie Corporation, the Alliance for Excellence in Education and others concerned about this issue have called for a greater understanding of which practices - in both the classroom and the principal's office - worked to improve student achievement. The National Association of Secondary
13 School Principals (Phillips, 2005) joined a national organization of governors (Berman & Biancarosa, 2005) and state school boards (2005) to urge members to help solve this problem of inadequate reading comprehension among adolescents. During the period when the problem of inadequate adolescent literacy was gaining urgency, the federal government began to hold schools to higher standards with the No Child Behind Act of 2001 (hereinafter referred to as NCLB). The law required all states to use standardized tests to measure and track student performance. In Connecticut, the performance of high school students has been judged through exams called CAPT. Included in the battery was the Reading Across Disciplines CAPT, which measured students' ability to read and respond to not only fiction, the traditional fare of language arts courses, but also to nonfiction, a skill that was needed in content areas and that has been shown to be weak among adolescents (Beers, 2003; Billmeyer & Barton, 2003; Calkins, 2001; Peterson, Caverly, Nicholson, O'Neal, & Cusenbary, 2001; Tovani, 2000). A 2001 study (CARR) sponsored by the Connecticut State Department of Education surveyed school principals on leadership and classroom practices. A similar study had been commissioned by the federal government earlier (Baron, 1999) based on Connecticut's high reading scores on National Assessment of Educational Progress exams during the 1990's. Those two studies occurred prior to NCLB. This study built upon those studies and shed light on effective practices and beliefs during the NCLB era. This was significant because, in a state where students traditionally performed relatively well on standardized tests, high school principals and other instructional leaders were still looking for ways to improve student achievement by improving reading comprehension, as evidenced by goals identified in their strategic school
14 profiles, published by the Connecticut State Department of Education. Literacy leadership at the secondary level is an emerging field, borrowing features long associated with collaborative leadership to address the recently recognized need of struggling adolescent readers, that is, those reading below proficiency, defined in Connecticut as below grade-level. In Connecticut, collaboration was identified as a leadership model to which administrators were expected to aspire, according to a monograph published by the state department of education and called "Defining Effective Leadership for Connecticut's Schools" (Leithwood & Duke, 1998). Among the qualities characterizing effective leadership were those that shaped a "robust culture": • a widely shared set of defensible, student-centered norms, beliefs, values and assumptions about professional practices; • collaboration among faculty to maintain and improve school programs • commitment to continuing inquiry, dialogue and improvement of instruction • the valuing of diverse opinions and perspectives among staff members; and • a willingness to view problems as opportunities for continued learning. (Leithwood & Duke, 1998, p. 71) Traditionally, research on reading focused on the classroom, for decades in elementary schools and recently in middle and high schools. However, in the past few years, researchers (Biancarosa & Snow, 2004; Meltzer, 2002; Taylor & Collins, 2003) and policymakers — the National Association of Secondary School Principals, the National Governors Association and the National Association of State Boards of Education ~ have called on secondary school leaders to use their authority to support efforts to
15 improve student reading skills. Creating and implementing literacy programs "must be a collaborative effort among teachers and administrators," said NASSP, but "it is the building principal's role as a literacy leader that has the greatest impact on the plan's outcome" (Phillips, 2005, p. 13). It is beyond the scope of this study to measure whether the principal has the "greatest impact" on improved student reading achievement. The study, however, attempted to quantify whether the principals' practices and beliefs about literacy were consistent with, or shared by, their respective staff as a way of measuring collaboration. The NASSP further framed the issue of literacy leadership by specifying literacy- related responsibilities for high school principals that moved beyond the traditional role of financial supporter to include new roles as professional development participant and learner: Strong leadership from both administrators and teachers is an essential building block in constructing a successful literacy program, but the role played by the principal is key to determining success or failure...to have an effective program, the school leader must be involved in all aspects of planning and sustaining the program. Above all, this must include participation in professional development sessions so that he or she is fully aware of successful strategies to improve literacy across the curriculum. To be an effective literacy leader in the building, the principal must be viewed by the teachers as a role model of a reflective, life-long learner and have their respect as knowledgeable in the area of adolescent literacy. (2005, p.7) To guide principals, NASSP (Phillips, 2005) published a list of 41 literacy-related practices and beliefs. Drawn from a variety of field studies and advocacy literature, they were organized into five categories. This study's author used these categories and the explanation of terminology provided by the NASSP report to craft research
16 questions for this study, discussed in more detail later in this chapter. The five categories and their definitions were the following: (a) Collaborative leadership and school capacity development. This was characterized by a school culture in which the principal and other school leaders encouraged collegial decision-making among staff members across course disciplines. The principal's role in supporting literacy improvement was evident to staff members financially and through participation in professional development. And, the principal used scheduling structures to support literacy. (b) Strategic use of student assessment. This referred to a systemic use by leaders and teachers at regularly scheduled times throughout the school year of what NASSP called "formative" and "summative" data on student performance to plan classroom instruction and staff development. In an interview with me, a designer of the NASSP survey (personal telephone conversation with Peter Reed, June 1, 2006) offered the following definitions of those terms: formative referred to low-stakes assessments that could be coached, a classroom version of a musician's rehearsal or athlete's scrimmage; and summative referred to high-stakes assessments, a final academic production like a musician's concert or athlete's ballgame. (c) Professional development of administrators and teachers to support literacy. Training provided to staff members was designed by an in-house literacy leadership team and addressed student literacy needs. It involved teachers of content-area courses, such as mathematics and science, as well as English.
17 (d) Instructional practices. Teachers in all content-area courses used a variety of strategies before, during, and after reading that were well documented by researchers in their effectiveness to help students comprehend course-related reading materials. (e) Interventions to improve student achievement. Administrators and teachers created highly structured plans to meet the individual needs of struggling readers, and the school offered extra tutoring or classes for struggling readers. As discussed in Chapter II of this paper, some interventions were packaged in programs that cost tens of thousands of dollars and some were free of charge (Peterson et al., 2001). Using a mixed-methods approach, this study was designed to survey and interview principals and their staffs about literacy-related practices and beliefs in those five categories. The overarching research question was the following: What are the literacy- related practices and beliefs of high school principals in three Connecticut schools where students have shown significant and steady improvement in reading comprehension? The intention was to respond to a request for research on literacy leadership made in 2006, when Michael Pressley presented a paper entitled, "What the Future of Reading Research Could Be." His observations included the following: I am mightily impressed that often the principal will make a huge difference, a finding in my own work but also a classic finding in the effective schools literature (e.g., Reynolds, Creemers, Stingfield, Teddlie, & Schaffer, 2002; Teddlie & Reynolds, 2000). The effective principals ... have done much to ensure they have excellent teachers who work together to create an environment that promotes the literacy achievement of all students in their school. ... there are plenty of principals out there who are not providing anything like the leadership required to produce an effective literacy development environment, (p.7) He argued: ".. .we very much need research on how to develop excellent literacy teachers as well as principals who can encourage effective literacy..." (p.7)
18 Purpose of the Study This study served three purposes. The first was to determine what, if any, common characteristics existed in the literacy-related practices and beliefs of principals and other high school leaders in three schools that showed statistically significant improvement in the reading achievement of students. For purposes of this study, "other" high school leaders included assistant or vice principals, department coordinators or chairpersons, team leaders, and literacy coaches. The decision to include those job titles was informed by a dissertation on secondary school literacy leadership (Cupid-McCoy, 2003) and discussed in greater detail in Chapter III, under Study Methodology. Questions have in recent years surfaced about the practices and beliefs of high school principals in the area of literacy (Phillips, 2005). Creating a community in which all secondary teachers saw themselves as teachers of reading took the vision and execution of a literacy leader (Taylor & Collins, 2003). A state- commissioned study (CARR, 2001) showed that 10 years ago, when Connecticut - the state where this dissertation was grounded ~ introduced a reading exam for high school students, some principals added the role of literacy leader to their repertoire. Others did not. NCLB, meanwhile, has raised the stakes to find solutions to lagging reading skills, forcing high school administrators to take part in fixing a problem that in the past had not been considered within their domain (Phillips, 2005). A second purpose for this study was to assess the efficacy of the five conditions cited earlier as well the efficacy of a nationally published survey instrument built upon those conditions (Phillips, 2005). Professional literature advocated that high school principals become literacy leaders and suggested five categories of strategies.
19 Interviews and a review of archival materials were conducted to capture data about whether principals pursued literacy leadership through the five domains. A final purpose of the study was to test the validity of a survey instrument that NASSP (Phillips, 2005) used to urge principals to take stock of their literacy leadership. Called the Literacy Capacity Survey, it asked 41 questions in the five categories. Although published, the survey had previously not been field-tested and had been used only informally by NASSP consultants to initiate staff development sessions on literacy (personal telephone conversation with Melvina Phillips, May 21, 2006). This study marked the first time the survey has been used in research, she said. This study was limited in nature, a mixed-methods examination of three case studies, and therefore could add relatively little validity to the survey. Still, the study may serve the purpose of prompting other researchers to test the survey's validity in larger studies. Because researchers and policymakers have only recently recognized struggling adolescent literacy as a problem, research was not yet robust. In 2004, a national panel of reading experts convened to write a paper urging funding agencies, policymakers, researchers and educators to contribute to solutions. The paper, called Reading Next: A Vision for Action and Research in Middle and High School Literacy (Biancarosa & Snow, 2004), included a list of 15 characteristics found from a review of the professional literature to be common among effective reading programs in secondary schools. The 15 characteristics were included in this study's survey instrument and interview protocol. Of the 15, nine were instructional practices, such as direct and explicit teaching of comprehension skills and using diverse texts. Another six were "infrastructural improvements," including literacy-related professional development
20 and leadership (Biancarosa & Snow, 2004). Typically, schools did not practice all 15. However, effective programs were built upon three components - professional development, ongoing formative assessments and ongoing summative assessments (Biancarosa & Snow, 2004). The report urged researchers to conduct experiments with students receiving various combinations of the 15 variables. While the idea of such projects were far more ambitious than the scope of this study, this researcher hoped also to contribute to what was known about effective reading programs for high school students. The anticipated contribution was the answer to this overarching question: What are the literacy-related practices and beliefs of principals in high school with improved student reading achievement? By examining three high schools in Connecticut with statistically significant reading improvement, I hoped to help illuminate the roles that principals played in supporting reading improvement. Research Questions The overarching research question was the following: What are the literacy-related practices and beliefs of high school principals in three Connecticut schools where students have shown significant and steady improvement in reading comprehension? Informed by research and practice, the study explored the following questions: Research Question 1: To what extent do principals, other instructional leaders, and teachers report that the following literacy-related variables occur in their schools: (a) collaborative leadership and school capacity development; (b) strategic use of student assessment; (c) professional development for administrators and teachers; (d) instructional practices; and (e) interventions to improve achievement among struggling
21 students? As discussed earlier in this chapter, these categories and terms were identified and defined in a report from NASSP that informed this study. Research Question 2: What specific actions have principals taken to improve literacy learning in the seven years corresponding to the second and third generations of the Reading Across the Disciplines CAPT? Research Question 3: What levels of importance do principals, other instructional leaders, and teachers perceive that the following variables have in supporting literacy: (a) collaborative leadership and school capacity development; (b) strategic use of student assessment; (c) professional development for administrators and teachers; (d) instructional practices; and (e) interventions to improve achievement among struggling students? Research Question 4: To what extent does a shared vision about literacy-related practices and beliefs exist among principals, other instructional leaders, and teachers in three schools where student reading achievement has improved significantly? Conceptual Framework This study was rooted in two major emerging trends: (1) that reading comprehension, a linchpin skill to student achievement, has declined alarmingly among high school students; and (2) that high school principals have a growing responsibility as instructional leaders to shore up what some policymakers called a basic civil right (National Association of State Boards of Education, 2005). Inadequate reading skills among adolescents have surfaced in the standards era. Prior to the implementation of the No Child Left Behind Act of 2001, only a handful of