Using repeated practice and error correction procedures combined with Precision Teaching measurement approaches to improve the math calculation fluency of middle school students with emotional and behavioral disorders
TABLE OF CONTENTS Page Copyright ii Abstract iii Doctoral Committee iv Acknowledgements v List of Tables ix List of Charts x Chapter 1. Introduction 1 Statement of the Problem 5 Significance of the Study 5 Research Questions 5 Definition of Terms 7 Limitations of the Study 10 Organization of Study 11 2. Review of Related Literature and Research 13 Mathematical Implications with Fluency and Automaticity 13 Emotional Behavioral Disorders 24 Single Subject Design and Precision Teaching 31 Summary 39 3. Research Methodology Introduction 41 Research Questions 41 vi
Participants 42 Setting 46 Experimenter 49 Dependent Measures 49 Daily Formative Performance 50 Summative Performance 51 Follow-up Retention Probe 52 Interobserver Agreement 52 Procedural Integrity 54 Social Validity 56 Experimental Design 57 Human Subjects Approval 58 Procedures 58 Review of Literature 59 Analysis of Research Questions 60 Summary 60 4. Results of the Investigation 62 Question One 62 Question Two 74 Question Three 96 Question Four 96 Question Five 97 Question Six 100 vii
Question Seven 100 5. Discussion and Recommendations 107 Conclusions 109 Discussion 117 Recommendations 121 Suggestions for Additional Research 123 Limitations 124 References 127 Appendix 147 A. Informed Consent Form 147 B. Student Social Validity Survey 156 C. Teacher Social Validity Survey 158 D. Procedural Integrity Checklist 160 E. Interobserver Agreement Data Collection Form 163 F. Human Subjects Approval Letter 166 G. Letter of Support 168 vin
LIST OF TABLES Table Title Page 1 Woodcock-Johnson Scores for Students 43 2 Grade Level, Disability Category, Years of Special Education Service and Free Lunch Eligibility 44 3 Achievement and Ability 45 4 Gender, Age, Hours of service in Special Education 46 5 Medians, Ranges, Numerical Variability & Performance Change 66 6 Individual Participant's Celeration Change Values Across Computational Probes for Individual Students, Demonstrates Improvements in Performance 69 7 Individual Participant's Celeration Performance Change Values (i.e. Overall Numerical Variability or Bounce in Data) Across Computational Probes for Individual Students, Demonstrates Improvements in Performance 78 8 Medians and Ranges Related to Computational Fluency Within and Across Computational Probes 89 9 Percentage of Interobserver Agreement Per Session 101 10 Student Satisfaction Survey Responses 105 11 Teacher Satisfaction Survey Responses 107 IX
LIST OF CHARTS Chart Title 1 Robert's Celeration Change for Computational Fluency Performance 2 Jim's Celeration Change for Computational 3 Fluency Performance 4 Matt's Celeration Change for Computational Fluency Performance 5 Robert's Celeration Performance Change: Summative Improvement in Computational Fluency 6 Jim's Celeration Performance Change: Summative Improvement in Computational Fluency 7 Matt's Celeration Performance Change: Summative Improvement in Computational Fluency x
1 CHAPTER 1 Introduction The ability to educate students who exhibit disabilities is one that causes an enormous amount of discussion today across America. Schools are placed under an increasing amount of pressure to identify methods that will individualize instruction and increase a student's skills in a variety of areas. Students labeled as displaying emotional and behavioral disorders (i.e., EBD) and considered "more difficult to teach" (Gunther, Coultinho, & Cade, 2002) than the student who is in general education classrooms or the resource room. Compared with students with other disabilities, students with EBD "fail more courses, earn lower grade point averages, and miss more days of school, and are more often retained in a grade" (Wagner, Blackorby, & Hebbeler, 1993, p.28 ). Three to five years after leaving secondary school, 52% of students with EBD, compared with 30% with learning disabilities, are not competitively employed or living independently (Gunter et al, 2002). These facts demonstrate an area of need in the field of education to incorporate strategies that can assist students with a variety learning supports for the general education classroom. By definition students with emotional and behavioral disorders (EBD) display emotional and behavioral problems that adversely affect their educational performance. Academic outcomes for students with EBD are worse than those of any other group of students with disabilities. Findings from two studies (Chesapeake Institute, 1994; Valdes, Williamson, & Wagner, 1990) indicated that students with EBD (a) could not pass competency exams for their grade levels, (b) had the lowest grade-point average of any group of students with disabilities, and (c) had the highest rate of absenteeism among any
2 group of students. According to the 25th Annual Report to Congress on the Implementation of the Individuals with Disabilities Education Act (IDEA; U.S. Department of Education, 2005), during the 2000-2001 academic year only 28.9% of students with EBD graduated with a standard diploma compared to 47.6% of all students with disabilities. Furthermore, approximately 65% of students with EBD dropped out of school, compared with 41% of all students with disabilities (Hodge, Riccomini, Buford & Herbst, 2006). According to Wagner (1989) students with emotional or behavioral disabilities (i.e., EBD) are estimated to drop out of school at a rate of 55% which is the highest rate of all students in special education. The drop-out rate is approximately double the current drop-out rate in the general education arena. Students with emotional issues often display characteristics that inhibit their success in the general education classrooms. These characteristics lead to fewer successes and a diminished sense of self accomplishment (Wagner, 1989). The best predictor that students with behavioral disabilities (i.e., EBD) will drop out is the low level of competency with basic skills such as reading and math (Jolivette, 2002; Walker, Colvin, & Ramsey, 1995). National Council of Teachers in Mathematics (NCTM) emphasizes the importance of mathematics. In addition, a need for higher levels of math competence is called for in this techno logy-based world, and a lack of knowledge, understanding, and skill development can close important doors for students (NCTM, 2004). Based on principles set forth by the NCTM (2004), instructional programs in math for all students should enable them to understand numbers, number systems, ways of representing numbers, relationships among numbers, computational fluency, and the ability to make reasonable estimates.
3 While these skills are an important aspect of overall learning and long-term achievement, one of the first steps in learning math as conceptualized by Haring and Eaton (1978) is accurate responding to basic mathematical facts, often referred to as basic math computation. Initial steps in learning math includes simple (basic) computations of single digit addition, subtraction, multiplication, and division (e. g., 5 + 5 = , 5 - 5 = , 5 x 5 = , and 25/5 = )(Axtell, 2006). Many teachers traditionally focused on providing students with instruction and practice, allowing them to successfully produce correct, accurate responses when tested. After students are able to produce correct answers, teachers will typically move on to the next skill. Moving on to new skills before assuring that the student has the previous skill embedded in their skill base ignores the other essential element in the fluency equation, i.e., time. Fluency is the # of correct responses across time leading to automaticity of responding (Sweeney, 1992; Sweeney, Ring, Malanga, & Lambert, 2003). While striving for accuracy is desired by many teachers (Johnson & Laying, 1992) it may impair the development of fluency in basic skills. An example, two students who have achieved 100% accuracy may not have the same fluency level or expertise. If Student A takes five minutes to complete a worksheet accurately and it takes Student B ten minutes to complete the same worksheet with the same accuracy it is assumed that Student A is more skilled in the particular mathematical area (Miller & Heward, 1992). Thus, accurate responding should not be the sole criterion to measure the mastery of an acquired skill. A measure of fluency should also be used. Fluency is conceptualized as responding both accurately and quickly to a selected stimulus. As the student learns a new skill, he/she will become increasingly fluent in that
4 skill until it becomes automatic. Automaticity refers to the phenomenon that a skill is performed with minimal awareness of its use (Howell & Larson-Howell, 1990; Hartnedy, Mozzoni, & Fahoum, 2005). Taking a cognitive processing perspective, the ability of a student to automatically respond to a stimulus may free limited cognitive resources that are then applied to the more complex computations and concepts (LaBerge & Samuels, 1974; Skinner & Schock, 1995). For example, if a student needs to actively think about the answer to 5 X 5, he/she possesses less cognitive processing resources to think about the next step in an algebraic algorithm. As such, this reduction in cognitive processing resources increases the time and effort it takes to complete complex math skills. Not only does this increased time and effort limit skill acquisition, it may produce an underestimate of a student's true ability, especially on time-limited tests or assessments such as those on the many standardized achievement tests. According to Gagne (1983), problem solving specifically occurs in working memory which is limited to a few computations at any single moment in time. If each component of a complex, multi-step problem requires sustained attention, the completion of the problem will likely be impossible due to the limited capacity of working memory. As the student gains automaticity, working memory is freed to address more complex operations of the selected problem (Gagne, 1983; LaBerge & Samuals, 1974). Statement of Problem The purpose of this research is to demonstrate the relative improvements in celerations, performance changes, and fluency in written math calculations of middle school students identified with emotional and behavioral disorders who do not speak English as their primary language.
5 Significance of the Problem This research contributes to the body of knowledge for educators who work with students who are Emotional and /or Behavioral Disorder/English Language Learners (EBD/ELL). This research provides educators working with students who are identified as EBD/ELL with information and special distinctive characteristics that may assist in the selection of effective strategies to assist these students to improve their math fluency skills. This research study expands upon results and replicates previous studies related to the effectiveness of the intervention with students who are EBD/ELL and to other populations exhibiting calculation fluency difficulties. In addition, this research demonstrates the effectiveness of using single-subject, behavioral dynamics research methods (Bullis & Anderson, 1986; Devine, 2003; Halle, Stoker, & Schloss, 1984; Johnston & Pennypacker, 1994) to collect data on student performance in order to assess student progress and demonstrate the effectiveness of instruction. Research Questions 1. How will the one minute timings strategy affect the relative ongoing math fluency (i.e., digits per minute written correctly or incorrectly) and the weekly learning or celerations (i.e., trends of students performance as represented by the number of digits per week or series of weeks) of students with emotional and behavioral disorders whose primary spoken language is other than English. 2. What are the relative summative results of a student's overall performance change from the initial performance on a given math calculation probe sheet when compared to the completion of an individual math probe sheet?
6 3. Can the student demonstrate relatively similar math calculation celerations across multiple math probe sheets (i.e., generalization) during the use of the one-minute timings? 4. Will the student maintain their relative math fluency levels 2, 4, and 6 weeks after completion of instruction on each respective math probe sheet. 5. Are the curriculum-based assessment procedures (i.e., Precision Teaching) used to evaluate the fluency strategy and the students' computational fluency a reliable measurement procedure 6. Is the procedural integrity measure successful at ensuring fidelity of the implementation of the one-minute timings instruction and curriculum-based assessment procedures? 7. How do the students, special education teachers, and other service providers perceive the one-minute math fluency procedures combined with the curriculum- based assessment participants (i.e., measure of social validity)? Definitions of Key Terms Behavior-thai portion of an organism's interaction with its environment characterized by a detectable displacement in space through time of some part of the organism and that results in some change in the environment. A single behavior is called a response class (Johnston & Pennypacker, 1993) Aatomaticity - the phenomenon that a skill can be performed with minimal awareness of its use (Howell & Larson-Howell, 1990; Hartnedy, Mozzoni, & Fahoum, 2005). Computational fluency- is a combination of speed and accuracy when presented with a mathematical expression.
7 Emotional and Behavioral disorders- According to IDEA 1997 emotional and behavioral disorders (EBD) as those individuals who exhibit one or more of the following characteristics that adversely affect educational performance over a long period of time and to a marked degree (IDEIA, 2004) 1. An inability to learn that cannot be explained by intellectual, sensory, or health factors. 2. An inability to build or maintain satisfactory interpersonal relationships with peers and teachers. 3. Inappropriate types of behavior or feelings under normal circumstances. 4. A general pervasive mood of unhappiness or depression. 5. A tendency to develop physical symptoms or fears associated with personal or school problems. Celeration- Graphical patterns of behavior that analyze weekly learning and trends within the data (Lindsley, 2001) English Language Learner (ELL)- The term English language learner (ELL), as used here, indicates a person who is in the process of acquiring English and has a first language other than English. Other terms commonly found in the literature include language minority students, limited English proficient (LEP), English as a second language (ESL), and culturally and linguistically diverse (CLD) Computational Fluency- having and using efficient, flexible, and accurate methods for computing (NCTM, 2000)
8 Weekly learning- An analysis of celerations for one week to determine the learning that occurred during the week. Celerations would be examined to determine behavioral trends that occur during the course of a week (Lindsley, 1991; Lindsley, 2001; Pennypacker, Koenign,& Lindsley, 1972; Sweeney, Ring, Malanga, & Lambert, 2003) Precision Teaching- A measurement procedure requiring direct and daily assessment of students' skills within a given duration of time (Lindsley, 1991; Sweeney, 1992). Trend- The overall direction taken by a data path. Trends are described in terms of their direction (increasing, decreasing, or zero trend), degree of trend, and extent of viability of data points around the trend (Cooper, Heron, & Heward, 2007) Trend Line or line of progress- The inspection of graphed data by drawing a line that visually provides the best fit through the data in order to examine a data path or trend (Cooper et al., 2007) Procedural Integrity (Internal Validity)- A checklist of specific steps in implementing the intervention measures on the occurrence or non-occurrence of the independent variables (Cooper et al., 2007) This is used as a measure of avoiding procedural drift. Interobsei~ver Reliability (Agreement)- A way of comparing the judgments of two independent observers simultaneously measuring the target behavior, to see how closely they agree on the occurrence or non-occurrence of the behavior (Cooper, et al., 2007) Social Validity- Is assessed in three areas in the field of applied behavior analysis; The social significance of the target behavior, the appropriateness of the procedures and the social importance of the results (Cooper, et al., 2007)
Systematic Replication- A method of reproducing previous research and when successful in reproducing the results of previous research, systematic replication not only demonstrates the reliability of the findings but also adds to the external validity of the experiment by showing that the same findings can be obtained during varying conditions (Johnston & Pennypacker, 1994) Within subject replication- A method of arranging comparisons between experimental and control conditions in which each subject is exposed to both control and experimental conditions and data representing the performance of individual subjects are used for analysis (Cooper, et al.,2007; Johnston & Pennypacker, 1994; Kazdin,1982) Replication Logic- Repeating the previously observed change with further manipulations of the independent variable. Replication within an experiment accomplishes two important goals. First, reproducing the previously observed behavior change reduces the probability that a variable other than the independent variable was responsible for the twice-observed behavior change. Second, replication demonstrates the reliability of the behavior change; it can be made to happen again (Cooper, et al., 2007; Johnston & Pennypacker, 1994; Kazdin,1982) Limitations 1. The characteristics of the participants are a limitation in this study due to the different prerequisite skills already displayed by each student. The participants possess differing emotional or behavioral difficulties, which may also exhibit limitations on the study. 2. The curriculum and content difficulty was based upon the unique individual
10 differences and ability levels of the students. This individualization prevents you from making a direct comparison of student performance between one student and the next; therefore, the ability to generalize to other students or populations can only be achieved through ongoing replications across subjects. 3. Students with emotional and behavior disorders are absent more often from school potentially leaving gaps in the data weekly. 4. Students with emotional and behavioral problems exhibit a wide range of behavioral problems that could lead to participant turnover (i.e., research subject morbidity) during the study. 5. Each student who participates in this study exhibits a wide range of emotional and behavioral problems, which limited the generalizability of this research study. This research study further was limited, because the fluency strategy was only implemented on one population of students. 6. Students in this study may lack the necessary reading skills (i.e., phonological awareness, alphabetical principles, and word analysis) that are related to math calculations to fully participate in a meaningful way in the study. 7. Students have experience difficulty based on the differing individual primary spoken languages of the student. This language difference could cause a possible difficulty in understanding the verbal directions. Organization of the Study This chapter introduced and discussed emotional/behavior disorders (EBD), the need for specialized instruction, and the need for computational and problem solving
11 instruction. In conclusion, this section examined the experimental questions specific to this study and potential limitations of the study and defined key terms. Chapter 2 examines the literature available with regards to emotional behavioral disorder as well as students who are considered ELL learners and their individual needs and the need for instruction in computation fluency. Chapter 3 re-examines the research questions and defines the population and sample size of the study. The instrumentation, data collection, and data summary procedures are also discussed. Chapter 4 discusses the data findings for each of the three participants. Data will be presented in the form of charts, tables and graphs. The research questions will be individually discussed. Chapter 5 discusses the recommendations for further research along with limitations of the study.
12 CHAPTER 2 Literature Review This section focuses on the literature available to support, define, and explain the procedures used in the research. This section is divided into three different sections: (a) mathematical implications, (b) emotional behavioral disorders, (c) Precision Teaching and single-subject research. Mathematical Implications with Fluency and Automaticity General educators experienced an increased anxiety in the achievement of students to perform basic computation. Interesting to note, for example, is that 98% of all adult Americans "do not compute" (Horton, 1985 p. 12). Most people cannot perform computations not due to their ability, but more because their fluency is so poor that they do not compute (White, 1986). Mathematical achievement comes under attack as of late due to the increased awareness that No Child Left Behind (NCLB) creates. Mathematics performance is the 600-pound gorilla in the U.S. schools (Steen, 2007), with general educators feeling the pressure to cover more content in limited amounts of time. This added pressure increases when the students do not possess the basic fundamental skills needed to acquire the content material. Instructors are moving on in the curriculum in the content area without waiting for proficiency or mastery. The deficit in fluency is creating a large population of students who do not possess the fundamentals needed to continue to the higher mathematical processes. Teachers are beginning to experience more of a need to differentiate in the classroom than ever before due to the fact that computational fluency is not achieved at an early age.
13 The middle school mathematics classrooms continue to strive for Annual Yearly Progress (AYP) according to the standards set up by NCLB (NCTM, 2008). This problem related to students not achieving AYP under NCLB related to math performance is especially important given statistics show that students who take at least a minimum of algebra II in high school do better financially in life (Carnevale & Desrochers, 2003). To achieve AYP, middle schools are required to prepare students to take more intensive mathematics coursework. The middle schools are expected make sure that students are engaged and actively involved in the learning process (Burns, 2008). In developing intervention instructions, a commitment needs to involve all three of the areas involved in numerical proficiency; computation, number sense, and problem solving. Burns (2007) also stated that only when the basics include understanding as well as skill proficiency related to math skills will all students learn what is required for their continued success. Academic performance of youth as a whole in the United States is important (Goals,2000). We still continue to look at the statistics for subgroups that are performing below the standard (NCLB). Many of the statistics, according to scores submitted on the 2005 National Assessment of Educational Progress (NAEP) showed that 59 % of students of African-American decent, 50 % of students of Hispanic decent, and 45 % of students of Native American decent did not meet proficiency at the 8th grade level ( Mullis, Dorsey, Owen and Phillips, 1993). Similarly, 69 % of students with disabilities and 71 % of English language learners did not reach the AYP benchmark (National Center for Education Statistics, 2005). Statistics show that there is a tremendous gap that is not addressed in the area of mathematical calculation and fluency. Proficiency in
14 mathematics continues as an area that each student will need throughout their school life into adulthood (Crawford, 1998). Students with learning difficulties in mathematics are found in almost every classroom across America (Shapiro, 1996a). About 5% to 10% of the students in schools for elementary general education possess deficits in mathematics (Rivera, 1997). Many students have difficulties learning basic math facts. These difficulties are attributed to a variety of differing causes. These deficits can also emerge at different times in a student's life. Potential causes are numerous and are partly explained by such child characteristics as intellectual functioning, motivation, problem-solving, memory skills, strategy acquisition and application, and vocabulary (Kroesbergen & Van Luit, 2003). The student who displays trouble with acquiring math skills due to one of the above mentioned causes can exhibit even more difficulty when there are multiple causes. Some students experience several causes and difficulties and are in need of special educational assistance. These students will require some type of specific mathematic intervention. Research consistently shows direct instruction as very effective for students with math learning difficulties to attain both the automaticity and problem-solving skills (Kroesbergen & Van Luit, 2003). Studies show that most math difficulties exhibit an early onset (Schopman & Van Luit, 1996). Early in the schooling process children develop their number sense which focuses on classification and seriation along with counting skills. The students then learn the four basic mathematical operations for computation skills. This knowledge of operations and a student's ability to perform mental math plays a critical role in the child's future skill development ( Mercer & Miller, 1992; Crawford, 1998). Students
15 achieve automaticity with math facts when they can directly retrieve the correct answer, without any intervening thought process. The development of automaticity is critical so students can concentrate on higher order thinking in math (Crawford, 1998). When this automaticity or fluency of calculation skills is not developed, the child will struggle in all mathematics that require proficiency in these basic skills that they encounter later in their schooling. The results of these deficits indicate a growing number of children are lagging behind in the area of mathematics. This increase in the number of students with deficits in mathematics is one of the rationales behind NCLB (2008). The interventions used to assist students in their math performance also help in identifying individual student needs (Mercer & Miller, 1992). Mathematical deficits for students consist of several areas including: (a) division of whole numbers (90%), (b) basic operations with fractions (89%), decimals (85%), (d) percent (82%), (e) multiplication of whole numbers (77%) (McLeod & Armstrong, 1982). Data shows a consistent need for proficiency in calculation skills due to each of these areas requiring basic computational skills. Students display this need for calculation skills by using a variety of techniques to compensate for the lack of computational skills. For example, students rely on using their fingers to count, or use concrete manipulatives and calculators to compensate for the inability to perform basic mathematical computation. The quick acquisition of facts has been found to happen quicker when there is a small set of related facts which follow a rule such as the 9's, 2's, and 5's. These sets response time on timed trials indicate that when facts are learned response time increases (Camine & Stein, 1981; Crawford, 1998; VanHouten, 1993). When students compute facts that do not exhibit a relationship, it requires the students to
16 use more time and energy to perform the same tasks as their peers who did not experience delays in calculation (Graham, 1987). McLeod & Armstrong (1982) stated that remediation in the basic mathematical skills will continue to be a major concern. This student population suffers from significant skill deficits and it is highly unlikely ever to function at grade level in mathematics without assistance in skill acquisition. Therefore interventions need to be incorporated into the classrooms that will assist the student in gaining computational fluency of math facts. The study done by Whalen, Willis, and Sweeney in 1993 examined the effects of time trials as a strategy to evaluate and improve mathematical performance in a high school student with behavior disorders. The method used for the study used 1-minute, 2-minute, 3-minute and 4-minute time trials to assist in fluency acquisition. The study was conducted as a single subject, behavioral dynamics design that employed Precision Teaching measurement procedures. The results showed an increase in the participant's math fluency especially under the 1-minute timing condition. Fatigue as related to the length of the timing period as well as intensity of effort and rate production of students performance was identified as a possible factor in the discrepancy between the 1 -minute timings and the longer 4-minute timings. Several curriculum programs based on the idea and philosophy that students need to develop their fluency through repeated practice in computational skills (i.e., Saxton Math, Rocket Math; Mad Minute). The idea that students need automaticity for mathematical calculations is vital based on the statement by Hasselbring, Goin, & Bransford (1988), that correct answers are obtained through fluency of procedural knowledge because it is often slow and effortful and appears to interfere with learning and understanding higher-