Purchase of this document is not possible at this time because the store is temporarily disabled. Continue searching

The relationship between the Wechsler Intelligence Scale for Children-Fourth Edition and the Woodcock-Johnson III Tests of Cognitive Abilities in a clinically referred pediatric population

Dissertation
Author: Lindsay Anne Shaw
Abstract:
This research involves an investigation of the construct validity of the Wechsler Intelligence Scale for Children- Fourth Edition (WISC-IV) when compared to the Woodcock-Johnson III Tests of Cognitive Abilities (WJ III COG) to provide evidence for the utility of using the WISC-IV in assessing cognitive abilities according to the Cattell-Horn-Carroll (CHC) theory. The study was conducted using archival data consisting of 92 children and adolescents between the ages of 6 years and 16 years, 11 months referred for a comprehensive neuropsychological evaluation at a university-affiliated assessment center. Data for all participants were collected following administration of a battery of measures as part of a neuropsychological evaluation, with tests administered in no particular order. The mean age of children was 9.82 years ( SD = 2.81) with a mean grade level of 3.95 (SD = 2.63). Ten hypotheses were investigated specifically to examine the comparability of the general intellectual functioning scores for each battery among a sample of children with neuropsychiatric disorders, as well as to examine the convergent and discriminate validity of the WISC-IV index scores. The first hypothesis utilized a paired samples t-test and found that the WISC-IV Full Scale IQ score was significantly below that of the WJ III COG General Intellectual Ability-Extended score. For the remaining hypotheses, Pearson product-moment correlations revealed large correlations between the WISC-IV and WJ III COG convergent constructs of general intellectual functioning, comprehension-knowledge, fluid reasoning, working memory, and processing speed. For correlations between divergent constructs, the WISC-IV Verbal Comprehension Index and the WJ III COG Visual-Spatial Thinking ( Gv ) factor demonstrated a large correlation. Both the WISC-IV Processing Speed Index and Working Memory Index correlated moderately with the WJ III COG Gv factor, while the WISC-IV Perceptual Reasoning Index correlated moderately with the WJ III COG Auditory Processing factor. Fisher's r to Z transformation was used to assess for significant differences between the observed correlations and stipulated values determined. Results indicated that correlations between the global IQ, fluid reasoning, and short-term memory composite scores of the two measures were significantly greater than that found for the WISC-III and WJ III COG, while the relationship between the verbal ability and processing speed composite scores were consistent with past findings. Correlations between divergent constructs revealed a reliable pattern of significantly greater relationships than was found for research concerning the WISC-III and WJ III COG. Primarily, results of this study provided evidence that the substantive changes made to the WISC-IV have improved the ability to interpret the Full Scale IQ score as a measure of general intelligence similar to that obtained by the WJ III COG. However, the global IQ scores between the two measures cannot be assumed to be equivalent among children with neuropsychiatric disorders. Results also suggested that the WISC-IV appears to provide improved measurement of the CHC broad abilities of fluid reasoning ( Gf ) and short-term memory (Gsm ). Correlations between divergent constructs provided evidence for relationships between cognitive abilities suggested to be significantly related to academic achievement. This study concluded that research findings for the WISC-III cannot be applied conclusively to the WISC-IV and that the substantive changes made to the WISC-IV have improved the ability to interpret the battery under the CHC framework. However, findings underscore the importance of examining performance across second-order factors that may contribute to differences in general intelligence, as well as remaining aware of differences in narrow ability constructs measured, task demands, or shared variance between subtests when making interpretations of test performance.

Table of Contents

List of Tables............................................................................................................. vii Abstract..................................................................................................................... viii Chapter I: Statement of the Problem......................................................................... 1 Chapter II: Review of the Literature......................................................................... 12 Conceptualization and Measurement of IQ................................................... 12 CattellHornCarroll (CHC) Theory.............................................................. 15 Psychometric Validity................................................................................... 20 Substantive Validity...................................................................................... 21 Structural Validity......................................................................................... 26 WithinBattery Structural Research...................................................... 26 CrossBattery Structural Research........................................................ 28 External Validity........................................................................................... 30 Correlations with Other Measures......................................................... 30 Correlations between the Wechsler Intelligence Scale for Children and the WoodcockJohnson.................................................................. 33 Differences Between the WISCIII and WISCIV Structure........................ 42 Purpose of Investigation................................................................................ 45 Hypothesis One............................................................................................. 46 Hypothesis Two............................................................................................. 49 Hypothesis Three........................................................................................... 52 Hypothesis Four............................................................................................ 54 Hypothesis Five............................................................................................. 56 Hypothesis Six............................................................................................... 58 Hypothesis Seven.......................................................................................... 59 Hypothesis Eight........................................................................................... 61 Hypothesis Nine............................................................................................ 62 Hypothesis Ten.............................................................................................. 64

v

Chapter III: Method................................................................................................... 66 Participants.................................................................................................... 66 Descriptive Statistics..................................................................................... 67 Measures........................................................................................................ 71 Intellectual Functioning......................................................................... 71 Procedure....................................................................................................... 75 Data Collection...................................................................................... 75 Institutional Review Board Requirements............................................ 75 Chapter IV: Results................................................................................................... 76 Preliminary Analyses.................................................................................... 76 Normality.............................................................................................. 77 Outliers.................................................................................................. 78 Linearity and Homoscedasticity............................................................ 78 Study Analyses.............................................................................................. 79 Hypothesis One..................................................................................... 79 Hypothesis Two..................................................................................... 79 Hypothesis Three................................................................................... 80 Hypothesis Four.................................................................................... 80 Hypothesis Five..................................................................................... 80 Hypothesis Six....................................................................................... 81 Hypothesis Seven.................................................................................. 81 Hypothesis Eight................................................................................... 82 Hypothesis Nine.................................................................................... 82 Hypothesis Ten...................................................................................... 82 Chapter V: Discussion............................................................................................... 84 Hypothesis One............................................................................................. 84 Hypothesis Two............................................................................................. 90 Hypothesis Three........................................................................................... 92 Hypothesis Four............................................................................................ 93 Hypothesis Five............................................................................................. 95

vi

Hypothesis Six............................................................................................... 97 Hypothesis Seven.......................................................................................... 100 Hypothesis Eight........................................................................................... 102 Hypothesis Nine............................................................................................ 103 Hypothesis Ten.............................................................................................. 105 Implications of Findings................................................................................ 107 Limitations.................................................................................................... 116 Internal Validity.................................................................................... 116 Sample Size........................................................................................... 117 External Validity................................................................................... 118 Statistical Limitations............................................................................ 119 Future Research............................................................................................. 119 References................................................................................................................. 123

vii

List of Tables

Table 1: CHC Broad Ability Factors....................................................................... 19 Table 2: Descriptive Statistics for Demographic Variables..................................... 68 Table 3: Diagnoses Represented in the Sample....................................................... 70 Table 4: Descriptive Statistics.................................................................................. 77 Table 5: Tests of Normality..................................................................................... 78

THE RELATIONSHIP BETWEEN THE WECHSLER INTELLIGENCE SCALE FOR CHILDREN- FOURTH EDITION AND THE WOODCOCK-JOHNSON III TESTS OF COGNTIVE ABILITIES IN A CLINICALLY REFERRED PEDIATRIC POPULATION By

Lindsay Anne Shaw, M.S.

Nova Southeastern University Abstract

This research involves an investigation of the construct validity of the Wechsler Intelligence Scale for Children Fourth Edition (WISCIV) when compared to the WoodcockJohnson III Tests of Cognitive Abilities (WJ III COG) to provide evidence for the utility of using the WISC IV in assessing cognitive abilities according to the CattellHornCarroll (CHC) theory. The study was conducted using archival data consisting of 92 children and adolescents between the ages of 6 years and 16 years, 11 months referred for a comprehensive neuropsychological evaluation at a universityaffiliated assessment center. Data for all participants were collected following administration of a battery of measures as part of a neuropsychological evaluation, with tests administered in no particular order. The mean age of children was 9.82 years (SD = 2.81) with a mean grade level of 3.95 (SD = 2.63). Ten hypotheses were investigated specifically to examine the comparability of the general intellectual functioning scores for each battery among a sample of children with neuropsychiatric disorders, as well as to examine the convergent and discriminate validity of the WISCIV index scores. The first hypothesis utilized a paired samples ttest and found that the WISCIV Full Scale IQ score was significantly below that of the WJ III COG General Intellectual AbilityExtended score. For the remaining hypotheses, Pearson productmoment correlations revealed large correlations between the WISCIV and WJ III COG convergent constructs of general intellectual functioning, comprehensionknowledge, fluid

reasoning, working memory, and processing speed. For correlations between divergent constructs, the WISCIV Verbal Comprehension Index and the WJ III COG VisualSpatial Thinking (Gv) factor demonstrated a large correlation. Both the WISCIV Processing Speed Index and Working Memory Index correlated moderately with the WJ III COG Gv factor, while the WISCIV Perceptual Reasoning Index correlated moderately with the WJ III COG Auditory Processing factor. Fisher’s r to Z transformation was used to assess for significant differences between the observed correlations and stipulated values determined. Results indicated that correlations between the global IQ, fluid reasoning, and shortterm memory composite scores of the two measures were significantly greater than that found for the WISCIII and WJ III COG, while the relationship between the verbal ability and processing speed composite scores were consistent with past findings. Correlations between divergent constructs revealed a reliable pattern of significantly greater relationships than was found for research concerning the WISC III and WJ III COG. Primarily, results of this study provided evidence that the substantive changes made to the WISCIV have improved the ability to interpret the Full Scale IQ score as a measure of general intelligence similar to that obtained by the WJ III COG. However, the global IQ scores between the two measures cannot be assumed to be equivalent among children with neuropsychiatric disorders. Results also suggested that the WISCIV appears to provide improved measurement of the CHC broad abilities of fluid reasoning (Gf) and shortterm memory (Gsm). Correlations between divergent constructs provided evidence for relationships between cognitive abilities suggested to be significantly related to academic achievement. This study concluded that research findings for the WISCIII cannot be applied conclusively to the ix

WISCIV and that the substantive changes made to the WISCIV have improved the ability to interpret the battery under the CHC framework. However, findings underscore the importance of examining performance across secondorder factors that may contribute to differences in general intelligence, as well as remaining aware of differences in narrow ability constructs measured, task demands, or shared variance between subtests when making interpretations of test performance.

x

1

Chapter I: Statement of the Problem

In 1949, David Wechsler published the first version of the Wechsler Intelligence Scale for Children (WISC; Wechsler, 1949), developed as a downward extension of his original test, the WechslerBellevue Intelligence Scale (Wechsler, 1939) (Kaufman, Flanagan, Alfonso, & Mascolo, 2006). While the original Wechsler measure primarily sought to classify individuals based upon global aspects of intelligence, subtests included did not align with an explicit theory of intelligence (Coalson & Weiss, 2002). Nonetheless, consistencies have been demonstrated to exist between the Wechsler scales and other measures of intelligence because of the inclusion of significant areas of cognitive ability: verbal comprehension, perceptual organization, quantitative reasoning, memory, and processing speed (Carroll, 1993, 1997; Horn, 1991). Over the last 60 years, the Wechsler scales have undergone numerous revisions, reflecting the evolution of intellectual assessment, and allowing for the clinical (e.g., improved norms) and practical (e.g., simplified administration procedures) utility of the tests across a wide range of settings and purposes (Wechsler, 2003; Wechsler, 2004). The Wechsler scales have demonstrated continued diagnostic applicability, utilized for the purposes of identification of mental retardation and learning disabilities, placement in specialized programs, determining clinical intervention, and for neuropsychological evaluation (Beres, Kaufman, & Perlman, 2000). It was previously thought that intelligence tests were not useful in neuropsychological evaluations because they were reported to lack specificity regarding underlying brain impairments, limiting conclusions regarding brain function. Likewise, their ability to accurately predict functional outcomes was questionable (Yeates & Donders, 2005). However, research has provided evidence for the relationship between neurological substrates and performance on the

2

Wechsler scales, possibly allowing for the prediction of performance based on brain damage (Gläscher et al., 2009; Riccio & Hynd, 2000). Moreover, such tests are practically relevant for validating recommendations in special education and clinical practice, and provide useful hypotheses regarding a child’s cognitive strengths and weaknesses (Yeates & Donders). Among the various cognitive ability measures available, the WISC has been purported to be the most frequently used measure of intelligence among child neuropsychologists (Camara, Nathan, & Peunte, 2000). The usefulness of intelligence testing is also an area of debate among school psychologists. According to Pfeiffer, Reddy, Kletzel, Schmeizer, and Boyer (2000), concerns have continued regarding the use of the WISC with minority students or among children for whom English is not their primary language. In their survey of nationally certified school psychologists regarding the perceived usefulness of the Wechsler Intelligence Test for Children Third Edition (WISCIII, Wechsler, 1991), concerns were reported regarding the applicability of the measure in directing psychoeducational interventions and strategies for instruction. Regardless of such concerns and other perceived weaknesses of the Wechsler scale, such as outdated visual stimuli, lack of utility in reevaluations, and high verbal content, Pfeiffer et al. found that 70% of the school psychologists surveyed rated the WISCIII factor scores and profile analysis as the most practically useful feature of the measure. Moreover, it characterized the WISCIII as playing a useful role in diagnosis and educational placement. The viability of utilizing measures of intelligence for the evaluation of child psychopathology and developmental disabilities has been substantiated (see Matson, Andrasik, & Matson, 2008, for the uses of intelligence testing when evaluating childhood pathologies), with

3

diverse and intricate methods available for assessing the complicated construct of cognitive functioning (Sattler, 2008). The Wechsler measure, in its many revised forms, has historically demonstrated considerable popularity as part of a comprehensive psychological assessment (Oakland & Hu, 1992; Prifitera, Weiss, Saklofaske, & Rolfhus, 2005; Zhu, & Weiss, 2005). Nonetheless, criticism has been made regarding the continued failure of the Wechsler scales to incorporate contemporary cognitive theory and research (e.g., Braden, 1995; Little, 1992; Kamphaus, 1993; Shaw, Swerdlik, & Laurent, 1993; Thorndike, 1997; Witt & Gresham, 1985), calling into question the substantive foundation of the scales. The WISC, now in its fourth revision, has undergone numerous changes since the first publication with regards to the test’s content and structure. Each successive revision of the measure has allowed for updated norms, provided more contemporary and less biased testing materials, improved the psychometric properties of the test, and has clarified the factor structure in support of a fourfactor solution (Prifitera, Weiss, et al., 2005). However, criticism has been made regarding the failure to incorporate new developments in cognitive theory across revisions (e.g., Little, 1992). Of importance, the Wechsler Intelligence Test for Children Fourth Edition (WISCIV, Wechsler, 2003) has been reported to integrate current research regarding cognitive functions and learning (Shaughnessy, 2006). Following suit with regards to revisions made to other measures of intelligence, the WISCIV more closely aligns with the CattellHornCarroll (CHC) theory of cognitive abilities (Flanagan & Kaufman, 2004), which is considered to be one of the most widely accepted theoretical models of cognitive ability (Keith, Kranzler, & Flanagan, 2001; McGrew, 1997). The CattellHornCarroll theory is a contemporary framework that integrates two similar cognitive

4

models: Carroll’s threetier model of human cognitive abilities and the HornCattell GfGc model (McGrew, 1997), where Gf and Gc refer, respectively, to “fluid" and “crystallized" intelligence. This hierarchical theory, which provides a theoretical taxonomy for understanding the cognitive constructs measured by major intelligence test batteries, classifies intellectual abilities within a threetiered structure, integrating general abilities (g; stratum III), ten broad abilities (stratum II) [crystallized intelligence (Gc), fluid reasoning (Gf), quantitative knowledge (Gq), shortterm memory (Gsm), visual processing (Gv), auditory processing (Ga), longterm retrieval (Glr), processing speed (Gs), reaction time (Gt), and reading/writing (Grw)], and 73 narrow abilities (stratum I). The WISCIV manual provides evidence of test validity through factor analytic research and comparative studies with other Wechslerbased measures of cognitive ability, while other research has examined the within and crossbattery factorstructure of the Wechsler scales. However, the WISCIV’s correlation with measures more aligned with current CHC theory is unclear. Correlational studies provided in the manual were conducted mainly with Wechslerbased measures and limited independent research has been conducted, failing to allow for understanding of the correlational patterns between the WISCIV and more diverse cognitive tests. A shortcoming in the current research is the lack of studies evaluating the relationship between the WISCIV and other current measures of cognitive functioning based upon CHC theory. The WoodcockJohnson Tests of Cognitive Abilities Third Edition (WJ III COG; Woodcock, McGrew, & Mather, 2001) is one such primary measure. The instrument has strong construct validation that allows for CHC based evaluations, with recent revisions based on the CattellHornCarroll theory (Taub & McGrew, 2004). Research concerning the correlations

5

between the WISCIII and WJ III COG completed at the time that the WJ III COG was standardized has provided considerable empirical evidence regarding the convergent and divergent relationships of both measures. Although the WISCIII and the WISCIV demonstrate significant correlations, comparisons made between the WISCIII and WJ III COG are not applicable to the understanding of the validity of the WISCIV when considering the substantial changes made to the content and structure of the Wechsler scale. As such, research is needed to examine the correlations between the WISCIV and the WJ III COG global intellectual scores and composite factor scores. The CHCbased Cross Battery Assessment (XBA) approach (Flanagan & McGrew, 1997; Flanagan, McGrew, & Ortiz, 2000; Flanagan, Ortiz, & Alfonso, 2007; McGrew & Flanagan, 1998) was developed to provide researchers and clinicians with a comprehensive theory to interpret performance on intellectual test batteries. The CattellHornCarroll framework, supplemented by tests from other batteries, allows for a more thorough analysis of CHC abilities. This approach is based on a series of joint confirmatory factor analyses examining the classification of individual intelligence tests at both the broad and narrow ability stratums (Flanagan et al., 2000; McGrew, 1997; McGrew & Flanagan, 1998; McGrew & Woodcock, 2001). This approach not only assists in interpretations made of the WISCIV in light of its lack of a specified formal theory (Keith, Fine, Taub, Reynolds, & Kranzler 2006), but it can also be a powerful tool for understanding a student’s intellectual abilities by providing a common language by which to describe cognitive abilities (e.g., Alfonso, Flanagan, & Radwan, 2005; Flanagan & McGrew, 1997; Flanagan et al., 2007). However, it remains to be seen whether the composite index and factor scores for the WISCIV and WJ III COG characterized by the same

6

construct actually measure the same abilities. Examining correlations between these two measures will provide evidence for the comparability of the broad abilities measured, allowing for the tests to be used interchangeably. Another shortcoming of the current research concerns the failure to determine the statistical difference between the mean scores for the WISCIV and the WJ III COG. While the relationship between the mean full scale scores for earlier versions of the WISC and Woodcock Johnson has been explored, there have been no current studies to determine if any statistical difference exists between the most recently published versions of these tests. Research is needed to determine if there are any statistical differences between the WISCIV Full Scale Intelligence Quotient (FSIQ) and the WJ III COG General Intellectual Ability (GIA) scores. Given the regular application of the WISCIV in determining eligibility for exceptional student education, professional standards mandate evidence regarding the test’s psychometric robustness (American Educational Research Association, American Psychological Association, & National Council on Measurement in Education, 1999). Traditionally, the identification of children with a specific learning disability (SLD) has been based upon an abilityachievement discrepancy model. With the revision of the Individuals with Disabilities Education Improvement Act in 2004 (IDEA; 2004), identification of children with learning disabilities has moved away from requiring evidence of a significant discrepancy between intellectual functioning and academic performance. Instead, there is a focus upon the evaluation of intraindividual differences in cognitive functioning and/or achievement relative to intellectual development. This alternative discrepancy method involves examining an individual’s pattern of cognitive or academic strengths and weaknesses to determine the impact on academic

7

performance (Mather & Gregg, 2006). While some states continue to enforce the use of a single criterion for the identification of a learning disorder (Holdnack & Weiss, 2006), arguments have been made for the limited utility of global intelligence scores or using a discrepancy model when determining eligibility for SLD (Fletcher et al., 2001; Kavale & Forness, 1995; Mather & Gregg, 2006; Vellutino, 2001; Warner, Dede, Garvan, & Conway, 2002). Despite these concerns, general intelligence test scores continue to be used in the identification of individuals with learning disabilities, mental retardation, giftedness, or low achievement (Saklofske, Prifitera, Weiss, Rolfhus, & Zhu, 2005). As such, it is important to understand if different measurements of cognitive ability assess similar constructs or result in different mean IQ scores, as this could have implications for those being evaluated for special education placement (Brown & Morgan, 1991; Naglieri, Salter, & Rojahn, 2005) and could adversely impact an individual’s functioning across multiple life domains (Silver et al., 2008). It is especially important to examine the comparability of the Full Scale scores across various measures of cognitive functioning when considering the changes made to the overall factor structure of the WISCIV. The WISCIII Freedom from Distractibility Index (FDI) was renamed the Working Memory Index (WMI) with the revision of the WISCIV. The subtests that comprised the index were modified, with the Arithmetic subtest being moved to supplemental status, reducing the emphasis on school achievement. Moreover, an additional task of working memory (LetterNumber Sequencing) was added to the index. The WISCIII Perceptual Organization Index (POI) was renamed the Perceptual Reasoning Index (PRI), with the core subtests measuring distinct processes involved in fluid reasoning (Gf) and visual processing

8

(Gv). Within the Verbal Comprehension Index, the Information subtest was moved to supplemental status to reduce the influence of abilities regarding general factual knowledge. While the constructs of working memory and processing speed did not contribute as heavily to the calculation of the WISCIII FSIQ, the WISCIV four factor structure allows for a more equal weighting of performance on verbal comprehension, perceptual reasoning, working memory, and processing speed in the construct of overall intelligence. The WISCIV evidences lower composite scores than the WISCIII, indicating that prior research concerning the WISC III cannot be generalized to the WISCIV. Moreover, the changes made have resulted in a Full Scale score that is more representative of the CHC broad abilities measured by the battery (Kaufman et al., 2006). Research by Flanagan and Kaufman (2004) has examined the content validity of new and revised intelligence test batteries, including the WISCIV, based on CHC theory, whereas Keith et al. (2006) used confirmatory factor analysis to determine if the WISC IV structure is better described by CHC theory. Consistent findings were shown suggesting that the WISCIV provides measurement of the CHC broad abilities of crystallized ability (Gc), fluid reasoning (Gf), visual processing (Gv), shortterm memory (Gsm), and processing speed (Gs). Research suggests that overall intelligence scores should be equivalent to the extent that they measure g or that they may be more comparable based on the extent to which they measure similar content (i.e., broad and narrow abilities). Differences between intelligences scores may be found based on the extent to which the specific abilities measured are more closely related to academic performance. Students with learning disabilities may obtain lower scores on intelligence tests that place greater emphasis on measuring cognitive processes found to be weaker in LD samples (e.g., phonological awareness, rate, memory, and perceptual speed)

9

(Mather & Wendling, 2005). Research has shown that the WISC Full Scale IQ score is typically significantly higher than the WoodcockJohnson global score in samples of children with learning difficulties. These discrepancies exist because the WoodcockJohnson battery includes tasks that more discretely measure weaker cognitive processes associated with learning difficulties, specifically, auditory processing (Ga) and longterm retrieval (Glr). Though the WISCIV FSIQ now more accurately measures CHC broad abilities, the FSIQ is likely less influenced by abilities related to school achievement than its predecessor. It remains to be seen what effect the changes made to the structure of the Wechsler scales have had on the comparability of the WISCIV FSIQ and WJ III COG GIA scores. Research is needed to address this concern by determining if the WISCIV results in lower, higher, or equivalent mean Full Scale IQ scores when compared to the WJ III COG. A third shortcoming of the current research involves the populations studied. Research concerning the correlations between the WISCIII and the WJ III COG was conducted by comparing how children without learning difficulties performed across both measures. While these comparisons helped to provide evidence for the convergent and discriminant validity of both measures, it failed to provide evidence for the comparability of the measures for children referred for academic difficulties. Prior research examining the relationship between the Wechsler Scales and the WoodcockJohnson batteries has been conducted in samples of children with learning and behavior difficulties (Bracken, Prasse, & Breen, 1984; Phelps, Rosso, & Falasco, 1984; Thompson & Brassard, 1984; Ysseldyke, Shinn, & Epps, 1981). However, these studies utilized much earlier versions of the tests, limiting the generalizability of the results.

10

The technical manuals for both the WISCIV and the WJ III assessment instruments provide results for special populations, providing important information regarding each test’s specificity and the clinical utility for diagnostic assessment (Hebben, 2004). However, the generalizability of these results is limited for a number of reasons. Studies for each measure were completed at different times and with different samples. Moreover, the studies completed during the standardization of the WJ III COG were limited, and included only one sample of students with either AttentionDeficit/Hyperactivity Disorder (ADHD) or a learning disability who were administered both the WJ III COG and the WISCIII. Limitations also exist because of the nature of the samples used for the WISCIV special group studies. Sample sizes were generally small and participants were not randomly selected. Also, data were derived from independent clinical settings, suggesting that different criteria and procedures were used for diagnosis (Hebben). Specifically, though, the WISCIV studies do not provide comparisons of group performance across a number of different intelligence measures (outside the realm of the Psychological Corporation). Also, the correlations between the WISCIV and the WJ III COG are unclear regarding the extent to which they measure similar cognitive processes. The revisions to the WISC suggest that the WISCIV would prove to be more correlated with the WJ III COG, even among samples of children identified with neuropsychiatric impairments. This is due to the fact that the previous versions of the WISC and WJ have demonstrated adequate convergent validity and the improved structure of the WISCIV now aligns the battery with more current cognitive theory. However, such comparison studies have yet to be conducted. In sum, there are several shortcomings in the research surrounding the validity of the WISCIV. Numerous revisions were made during the development of the WISCIV, resulting in

11

a measure that is significantly different in content and structure from the WISCIII. Therefore, validity research regarding the WISCIII does not provide an adequate understanding of the current test’s convergent and discriminate validity. Additionally, because the changes made more closely align the WISCIV with CHC theory, the generalizability of findings concerning the WISCIII is questionable. Finally, research is lacking regarding the comparison of the WISCIV and other cognitive measures utilized among referred groups of children. Given these limitations, research is needed to explore the relationship between the WISCIV and the WJ III COG in a clinically referred population. More specifically, this study will examine the relationships between the WISCIV and WJ III COG Full Scale scores and Index/Composite scores, as well as help to determine if any significant differences exist between the global scores within a clinical sample of children.

12

Chapter II: Review of the Literature

In order to understand the relevance of the proposed investigation, it is necessary to present an examination of: (a) theories underlying the conceptualization and measurement of IQ; (b) the development of the CattellHornCarroll (CHC) theory of intelligence; and (c) the stages of construct validity used to establish the test validity of the Wechsler scales, including examination of its construct validity from the perceptive of the CattellHornCarroll theory. Conceptualization and Measurement of IQ

Intelligence, as defined in the psychometric sense, is the general reasoning capacity used in various problemsolving tasks (Kline, 1991) and results from variations in brain structure following the interaction of genetic and nongenetic factors (Draganski, Gaser, Busch, Schuierer, Bogdahn, & May, 2004; Thompson et al., 2001). Theories regarding the nature of intelligence have evolved across time (Carroll, 1993, 1997; Gardner, 1983; Horn & Noll, 1997; Neisser et al., 1996; Spearman, 1932; Thurstone, 1938) and have offered tremendous variation in the conceptualization and assessment of cognitive functioning (Das, Naglieri, & Kirby, 1994; Flannagan & McGrew, 1997; Sattler, 2001; Sternberg & Berg, 1986). Considerable differences have arisen over time concerning those aspects included in the measurement and definition of intelligence. Some theorists, such as Jean Piaget, conceptualized intelligence using developmentally based experiences. Piaget’s theory (1972) holds that new information is assimilated into and accommodated by existing cognitive structures, with this framework being applied to the intellectual development of all children. More recent investigations have explored the anatomical and physiological brain substrates related to intelligence. Those who have attempted to clarify intelligence according to more explicit theories

13

have sought to provide a framework by which one can take into account individual differences in comprehension, adaptation, learning, and problemsolving, which can vary across domains and according to settings and standards of performance even within a single individual (Neisser et. al, 1996). According to Sattler (2001), early theories of intelligence focused more on examining sensorimotor functions thought to contribute to mental functioning. The possibility of studying mental ability through experimental and practical means did not formally arise until the work of J. M. Cattell in 1890, whereas the focus on examining cognitive functions as typically seen in modern intelligence tests did not take place until the turn of the century following the work of Carl Wernicke and Theodor Ziehen. Intelligence tests were originally developed independent from theory. The first intelligence test, developed by Binet and Simon in 1905, attempted to assess degrees of individual mental ability (Sattler, 2001). Although the BinetSimon scale was not theory driven and lacked indicators of performance (Thorndike, 1997), it was the first of its kind to incorporate administration standards and items ranked according to level of difficulty while also acknowledging agebased cognitive development in order to measure higher mental processes associated with intelligence (Sattler, 2001). In 1916, the test was revised to include a ratiobased intelligent quotient that, despite criticism, allowed for the comparison of intellectual functioning among individuals (Thorndike, 1997). Since its conception, the psychometric approach to intelligence testing has evolved considerably, with certain tests even created to measure specific constructs, such as verbal and nonverbal intelligence (Neisser et al., 1996). More recently developed tests emphasize empirically based theories of cognitive functioning. With this in

14

mind, intelligence is thought to best be measured by instruments that take into account the multiple and fairly independent factors that contribute to the phenomenon (Strauss, Sherman, & Spreen, 2006). Spearman (1904) brought the idea of psychometric testing to life by developing an approach to understanding intelligence using factor analytic methods. Spearman formulated a two factor theory of intelligence. He proposed a general factor, g, to represent what all tests have in common and, therefore, to reflect how a person would perform across batteries of intelligence tests, regardless of the domains included (Thorndike, 1997). Spearman hypothesized that positive correlations found between diverse measures were accounted for by g, with varying amounts of g represented within each measure, such that those with higher g loadings were more highly correlated (Brody, 1999). Spearman later included smaller specific factors (s) required for particular cognitive tasks (Thorndike, 1997), which accounted for the overlapping variance between tests beyond the sole influence of g (Wasserman & Tulsky, 2005). Many factor analytic studies have found support for g, with all modern intelligence tests purported to measure a general intelligence factor that accounts for the largest proportion of variance in an intelligence test battery (Kamphaus, 2005). Another factoranalytic based theory included that of Primary Mental Abilities (PMA) (Thurstone, 1938), which posited that g resulted from the relationship between a number of primary factors (Thorndike, 1997). In contrast to Spearman’s unitary theory of intelligence, this theory offered insight into understanding how intelligence could be measured through distinct cognitive factors (Flanagan et al., 2000). Using new methods of factor analysis, Thurstone’s research suggested that intelligence was comprised of seven independent factors, or primary

Full document contains 157 pages
Abstract: This research involves an investigation of the construct validity of the Wechsler Intelligence Scale for Children- Fourth Edition (WISC-IV) when compared to the Woodcock-Johnson III Tests of Cognitive Abilities (WJ III COG) to provide evidence for the utility of using the WISC-IV in assessing cognitive abilities according to the Cattell-Horn-Carroll (CHC) theory. The study was conducted using archival data consisting of 92 children and adolescents between the ages of 6 years and 16 years, 11 months referred for a comprehensive neuropsychological evaluation at a university-affiliated assessment center. Data for all participants were collected following administration of a battery of measures as part of a neuropsychological evaluation, with tests administered in no particular order. The mean age of children was 9.82 years ( SD = 2.81) with a mean grade level of 3.95 (SD = 2.63). Ten hypotheses were investigated specifically to examine the comparability of the general intellectual functioning scores for each battery among a sample of children with neuropsychiatric disorders, as well as to examine the convergent and discriminate validity of the WISC-IV index scores. The first hypothesis utilized a paired samples t-test and found that the WISC-IV Full Scale IQ score was significantly below that of the WJ III COG General Intellectual Ability-Extended score. For the remaining hypotheses, Pearson product-moment correlations revealed large correlations between the WISC-IV and WJ III COG convergent constructs of general intellectual functioning, comprehension-knowledge, fluid reasoning, working memory, and processing speed. For correlations between divergent constructs, the WISC-IV Verbal Comprehension Index and the WJ III COG Visual-Spatial Thinking ( Gv ) factor demonstrated a large correlation. Both the WISC-IV Processing Speed Index and Working Memory Index correlated moderately with the WJ III COG Gv factor, while the WISC-IV Perceptual Reasoning Index correlated moderately with the WJ III COG Auditory Processing factor. Fisher's r to Z transformation was used to assess for significant differences between the observed correlations and stipulated values determined. Results indicated that correlations between the global IQ, fluid reasoning, and short-term memory composite scores of the two measures were significantly greater than that found for the WISC-III and WJ III COG, while the relationship between the verbal ability and processing speed composite scores were consistent with past findings. Correlations between divergent constructs revealed a reliable pattern of significantly greater relationships than was found for research concerning the WISC-III and WJ III COG. Primarily, results of this study provided evidence that the substantive changes made to the WISC-IV have improved the ability to interpret the Full Scale IQ score as a measure of general intelligence similar to that obtained by the WJ III COG. However, the global IQ scores between the two measures cannot be assumed to be equivalent among children with neuropsychiatric disorders. Results also suggested that the WISC-IV appears to provide improved measurement of the CHC broad abilities of fluid reasoning ( Gf ) and short-term memory (Gsm ). Correlations between divergent constructs provided evidence for relationships between cognitive abilities suggested to be significantly related to academic achievement. This study concluded that research findings for the WISC-III cannot be applied conclusively to the WISC-IV and that the substantive changes made to the WISC-IV have improved the ability to interpret the battery under the CHC framework. However, findings underscore the importance of examining performance across second-order factors that may contribute to differences in general intelligence, as well as remaining aware of differences in narrow ability constructs measured, task demands, or shared variance between subtests when making interpretations of test performance.