• unlimited access with print and download
    $ 37 00
  • read full document, no print or download, expires after 72 hours
    $ 4 99
More info
Unlimited access including download and printing, plus availability for reading and annotating in your in your Udini library.
  • Access to this article in your Udini library for 72 hours from purchase.
  • The article will not be available for download or print.
  • Upgrade to the full version of this document at a reduced price.
  • Your trial access payment is credited when purchasing the full version.
Buy
Continue searching

Using concept mapping as as tool for program theory development

ProQuest Dissertations and Theses, 2011
Dissertation
Author: Rebecca Orsi
Abstract:
The purpose of this methodological study is to explore how well a process called concept mapping (Trochim, 1989) can articulate the theory which underlies a social program. Articulation of a program's theory is a key step in completing a sound theory based evaluation (Weiss, 1997a). In this study, concept mapping is used to articulate the outcomes domain of a program theory, using Chen's (1990) six domains for program theory as an organizing framework. A grassroots community organization in Denver, Colorado, provides context for the study. With reference to Dubin's (1978) distinctions for theoretical units as a guide, the results of concept mapping are analyzed to determine whether they are useful in building a program theory. Results are also are evaluated to determine whether they present a comprehensive, parsimonious (Whetten, 1989) and valid representation of outcomes from the community organizing intervention. Methodological and statistical considerations for using concept mapping are mentioned. The study concludes that concept mapping is a promising tool for theory articulation. Study limitations and opportunities for future research are also discussed.

help with the table of contents! I thank Jennifer Hoeting, who already sat on one graduate committee for me and was willing to sit on another . Sue Lynham

prov ided many close readings of and detail ed comments for

drafts of this study . Thanks to Paul Speer

for traveling so far

to be a part of this work . Finally, I acknowledge Brian Cobb, who

is the wisest and savvie st PhD advisor any student ever

had. I owe Br ian additional thanks for allowing me to find my own way through this study

and for not really

retiring until I was done. My children, Renata and Carlos,

popped sorting cards from perforated pages to help me

assemble packets for the concept mapping proces s. Renata and Carlos were also the people who , more than anyone

else , assumed without question that M om‟s

dissertation would get done! An d, finally, to my husband Jared: thanks to you for listening without

fail and for never doubting that this PhD i s go od for me, for our family and for the larger community.

v

For the leaders at M etro O rganizations for P eople

and at all grassroots community

organization s:

you

do th e hard work to change the world .

vi

TABLE OF CONTENTS

ABSTRA CT

................................ ................................ ................................ ........................

ii

ACKNOWL E DGE MENTS

................................ ................................ ...............................

iii

DEDICATION

................................ ................................ ................................ ..................... v

LIST OF TAB LES

................................ ................................ ................................ .............

ix

LIST OF FIGU RES

................................ ................................ ................................ ............. x

C HAPTER 1 –

INTRODUCTION

................................ ................................ ...................... 1

Research P roblem

................................ ................................ ................................ .... 2

Research Ques tions

................................ ................................ ................................ .. 3

Conceptual F ramework

................................ ................................ ............................ 3

Definitions ................................ ................................ ................................ ................ 6

P hilosophical Paradigm

................................ ................................ ........................... 8

Delimitations

and Limitations

................................ ................................ ................ 12

Significance ................................ ................................ ................................ ............ 13

CHAPTER 2 –

LITERAT URE REVIEW

................................ ................................ ......... 14

Theory - Buil ding

................................ ................................ ................................ ..... 14

Lynham‟s general m ethod for theory - building

................................ .......... 14

Dubin‟s method for theory - building

................................ .......................... 16

Whetten: building blocks for theory development ................................ .... 20

Trochim and Leeuw: program theory

................................ ....................... 21

Theory - based Eval uation

................................ ................................ ....................... 22

Development of theory - based evaluation

................................ .................. 22

Strengths of theory - based evaluation

................................ ......................... 27

Challenges of theory - based evaluation

................................ ...................... 28

Empirical examples of theory - based evaluation

................................ ........ 29

Concept Mapping

................................ ................................ ................................ ... 32

Brief overview of concept mapping

................................ ........................... 3 2

Examples of concept mapping in social research

................................ ...... 3 3

Examples of concept mapping for theory articulation

............................... 36

Summa ry

................................ ................................ ................................ ................ 38

CHAPTER 3 –

MET HODS

................................ ................................ ............................... 40

Metro Organizations for People –

Program Description

................................ ....... 40

Sampl e ................................ ................................ ................................ .................... 42

Details of Concept Mapping Tool ................................ ................................ .......... 42

Reliability of Concept Mapping ................................ ................................ ............. 50

Validity of Concept Mapping

................................ ................................ ................ 53

Data Collection

................................ ................................ ................................ ...... 57

Data Analysis

................................ ................................ ................................ ......... 60

Outcomes Domain Conceptualization

................................ ................................ ... 66

Conclusion

................................ ................................ ................................ ............. 68

CH APTER 4 –

RESULTS

................................ ................................ ................................ . 69

vii

Generation of Statements –

Brainstorming

................................ ............................ 69

Structuring of Statements –

Sorting and Rating

................................ .................... 71

Representation of Statements –

Concept Mapping Analy sis

................................ . 75

Interpretation of Maps ................................ ................................ ............................ 79

CHAPTER 5

DISCUSSION

................................ ................................ ........................... 82

Research Question 1: Dubin‟s Units of Theory

................................ .................... 82

Properties of objects instead of objects themselves

................................ ... 82

Units of theory cannot be one - time events

................................ ................ 83

Attribute or variable

................................ ................................ ................... 84

Real or nominal

................................ ................................ .......................... 86

Prim itive or sophisticated

................................ ................................ .......... 86

Collective or member

................................ ................................ ................. 87

Concept mapping and program theory

................................ ....................... 88

Research Question 2: Parsimony and Comprehensiveness

................................ .. 89

Research Question 3: Validity

................................ ................................ .............. 8 9

Evidence from multiple clustering methods

................................ .............. 90

Interpretation session results

................................ ................................ ...... 9 2

Delimitations and L imitations

................................ ................................ ................ 97

Further Research

................................ ................................ ................................ .... 99

Theory - building research

................................ ................................ ........... 99

Applied statisti cal research

................................ ................................ ...... 100

Addi tional Questions Raised by the Study

................................ .......................... 103

An alternative interpretation of concept mapping results

........................ 103

Other frameworks for articulating program theory

................................ .. 104

Whetten and an alternative interpretation

................................ ................ 105

More questions about

concept mapping

................................ .................. 105

Conclus ion

................................ ................................ ................................ ........... 1 0 9

REFEREN CE S

................................ ................................ ................................ ................ 110

APPENDIX A : List of Brainstormed Statements –

Lunchtime Session

........................ 1 22

APPENDIX B: List of Brainstormed Statements –

Even ing Session

............................ 125

APPENDIX

C: Combined List of Brainstorm ed Statements Used for the Study

.......... 127

APPENDIX D: Packet Ins truct ions

................................ ................................ ................ 131

APPENDIX E: Rating Ins t ructions

................................ ................................ ................. 132

APPENDIX F: Demograph ic Survey

................................ ................................ ............. 133

APPENDIX G: Mop Po int Map

................................ ................................ ..................... 134

APPENDIX H: Shepard D iagram

................................ ................................ ................... 135

APPENDIX I

................................ ................................ ................................ ................... 136

AGNES

4 - Cluster

S olution

................................ ................................ .................. 1 36

AGNES 5 - Cluster

S olution

................................ ................................ .................. 137

AGNES 6 - Cluster

S olution

................................ ................................ .................. 138

AGNES 7 - Cluster

S olution

................................ ................................ .................. 139

APPENDIX J

................................ ................................ ................................ ................... 140

DIANA 4 - Cluster

S olution

................................ ................................ .................. 140

DIANA 5 - Cluster

S olution

................................ ................................ .................. 141

DIANA 6 - Cluster

S olution

................................ ................................ .................. 142

DIANA 7 - Clus ter

S olution

................................ ................................ .................. 143

viii

APPENDIX K

................................ ................................ ................................ .................. 144

FANNY 4 - Cluster

S olution

................................ ................................ ................. 144

FANNY 5 - Cluster

S olution

................................ ................................ ................. 145

FANNY 6 - Cluster

S olution

................................ ................................ ................. 146

FANNY 7 - Cluster

S olution

................................ ................................ ................. 147

APPENDIX L

................................ ................................ ................................ .................. 148

PAM 4 - Cluster

S olution

................................ ................................ ...................... 148

PAM 5 - Cluster

S olution

................................ ................................ ...................... 149

PAM 6 - Cluster

S olution

................................ ................................ ...................... 150

PAM 7 - Cluste r

S olution

................................ ................................ ...................... 151

APPENDIX M: 4 - Cluster Map for Inter pretation Session

................................ ............. 152

APPENDIX N: Final 5 - Clu ster Map

................................ ................................ .............. 153

APPENDI X O: Grouped List of Statements –

Final 5 - Cluster Map

.............................. 154

ix

LIST OF TABLES

1. Participant Characteristics

for Brain storming and Sorting/Rating Groups ............. ...... .74

2. Similarity I ndi ces Comparing Different Cluster Solutions ........................ ................ ....91

x

LIST OF FIGURES

1. Relationship between Chen‟s ( 1990 )

program theory domains and the use

of

concept mapping for developing theoretical units ( Dubin, 1978 )

in the outcomes

domain . ....................................................................... ......................................................... . 5

2.

Six steps of the concept mapping process (Kane & Troc him, 2007; Trochim,

1989) . ........................... .............. ............................................................................. ........... 43

3. Sample named cluster map from Trochim ( 1989 ) .

Used with permission...................47

4. Sample binary similarity matrix (BSM) for one participa nt with 80 sorting

statements. Statements 1 and 3 and s tatements 2 and 79 are sorted together.. .......... ...... .61

1

Chapter 1 –

Introduction

No religious festival or major holiday occurred that particular day, but on Thursday evening, November 10 th , 2005, over 500 people filled Nuestra Señora de la Paz (Our Lady of Peace) Catholic Church in northeast Gree ley, Colorado. They were attending a public meeting with Weld County District Attorney Ken Buck and Greeley Mayor Tom Selders. The organizing committee that orchestrated

this event was composed of parishioners at the church ; they were not professional ev ent planners. They had

only ten

days to plan the

meeting . How did they motivate so many people to attend? How did they get important politicians like the district attorney and the mayor to take an evening out of their busy schedules? And d id this large

meeting make a differenc e to individuals

or to the civic

community in any way?

…..

The brief anecdote about the public meeting in Greeley

raises a

salient question: h ow do es one

evaluate

a complex social intervention ?

T heory - based evaluation ( Chen, 1990 ; Weiss, 1972 , 1998 )

may offer an answer. To be executed w ell , however, theory - based evaluation ( TBE )

requires the articulation

of a robust theory that describes and explains the

intervention

in question . Using grassroots community organizing as an example of such a complex social intervention, t his

study explor es

the utility of concept mapping ( Trochim, 1989 )

as a

statistical and

analytic

tool for

program theory

development . Well - developed

theory can in turn support

strong

theory - based evalu ation s .

2

Research

P roblem

On one hand, s ocial interventions and programs are clearly present in the United States today. An example of one particularly complex intervention is the phenomenon of grassroots community organizing whic h involves itself

in urban social and political processes in dynamic ways

( Warren, 2001 ; Wood, 2002 ) . Furthermore, there exists a growing consensus in the evaluation literature that theory should play a role in the evaluation

of social programs ( Chen, 1990 ; Weiss, 1998 ) . On the other hand, there is also consensus

in the literature that current theory - based evaluations cou ld be better - executed

( Rogers & Weiss, 2007 ; Weiss, 1997a ) .

Despite a history spanning almost 40 years ( Weiss, 1997b ) , theory - based evaluation continues to suffer from numerous challenges to doing it well ( Weiss, 1997a ) . T hree

significant challenges include :

(a) lack of clarity on the differe nce between black - box

process - outcome evaluation and theory - based evaluation , (b ) difficulty in constructing program theory, and (c ) large time and resource req uirements for TBE .

Finally , Chen ( 1990 )

has suggested that statistical tools

might be used in the development of program theory , particularly in the area of outcom es evaluation . Therefore, the purpose of this study is to explore the utility

of concept mapping ( Trochim, 1989 )

as a statistical and analytic tool for progr am

theory development . More specifically, the study will use concept mapping to

specify an

outcomes domain

in the context of Chen‟s ( 1990 )

six - domain framework for p rogram theory . I t will also examine whether or not the resulting outcomes domain meets several of Dubin‟s ( 1978 )

criteria for specifying the building blocks

(i.e. units ) of a theory.

T he study‟s

exploration s

will contribute to current

discussion s

in the literature about how theory - based evaluation might be improved

via better theory articulation .

3

Research Q uestions

To achieve

the purpose s

of the study menti oned

above, the following research question s

are posed .

1.

H ow well d oes concept mapping assist in developing

the outcomes domain of a program theory for a complex social intervention ?

This first question will be answered by considering the following sub - ques tions

which use

Dubin‟s ( 1978, p. 37 )

“distinctions” for theoretical “ units ”

as a guide.

a.

D oe s concept mapping produce potential

units

for incorporation into theory

which

(a) describe properties of objects in stead of objects themselves

and which (b) do not describe one - time events?

b.

Does

concept mapping produce potential

units wh ich meet Dubin‟s fo ur sets of mutually exclusive distinctions ? That is, each unit must be classifiable

as:

(a) attribute or variable, (b) real or nominal, (c) primitive or sophisticated and (d) collective or member .

2.

Are the pot ential units articulated by concept mapping

co llectively both parsimonious and comprehensive ( Whetten, 1989 )

in describing the outcomes domain ?

3.

What evidence exists fo r the validity

of programmatic

o utcomes as articulated by the concept mapping

process ?

Conc e p tual Framework

As suggested above, this

study will investigate how researchers can use concept mapping within Chen‟s ( 1990 )

framework for articulating a program theory. Chen discusses six domains for

a complete program theory. These domains are: treatment,

4

outcome, impact, intervening mechanism(s) , implementation environme nt, and generalization. These domains are consistent with broader notions of what elements are required for a complete theory. For example, Patterson ( 1986 )

notes

that a theory includes th e following characteristics: (a ) stated postulates and assumptions, (b ) definition s of

terms and

concepts included in the theory, (c ) statement s of relationship

among

the terms and concepts in the theory, and, finally, (d ) hypotheses/predictions that follow from the

theory.

Chen and Patterson align in this manner: Chen‟s domains of treatm ent, outcome and implementation environment are, in Patterson‟s language, “terms” ( 1986, p. xix )

which require definition. Relationships between treatment and outcome are expressed in Chen‟s

intervening mechanism

domain. Expected impacts of treatments on outcomes (pe r Chen) constitute Patterson‟s

predictions of the theory. And, finally, Patterson‟s assumptions must be

made so the theory can describe

to what contexts it might generalize.

The proposed study will focus on using concept mapping to build only the portion

of a program theory that describes outcomes . Chen writes that “…normative outcome evaluation involves systematically identifying or clarifying a set of program goals or outcomes…” ( 1990, p. 54 ) . Figure 1 below illustrates

how Chen‟s program theory domains fit together and how concept mapping may

work to articulate the outcomes domain as one element

of a more comprehensive program theory.

The treatment

domain

repr esents the intervention undertaken by a social program. Outcomes

result

from this

intervention .

An

intervening mechanism

is the means by which an intervention is transmitted

so that it can a ffect

outcomes. The impact

domain describes how (e.g. in what d irection

and with what strength of association ) the program‟s

5

intervention is

expected to e ffect change in an outcome. Finally, the domains of

implementation environment

and generalization

describe the context in which an intervention occurs and whether a nd how the impact of treatment on an outcome via the intervening mechanisms can be generalized to other contexts. Chen notes that “a systematic combination of all six domain theories constitutes a superordinate theory of a program…” ( 1990, pp. 51 ) .

Chen’s Program Theory Domains

C r iteria for units of theory

Figure 1 .

Relationship between Chen‟s ( 1990 )

program theory domains and the use

of concept mapping for developing theoretical units ( Dubin, 1978 )

in the outcomes domain .

Treatment

Intervening Mechanism

Impact

Ou tcomes

I mplementation environment

Generalization

r eal/nominal

valid parsimonious/comprehensive

attribute/variable OUTCOMES UNITS properties of things

primitive/sophisticated

collecti ve/member

not historical events

Concept Mapping

6

In this study, concept mapping will be used to develop

the outcomes domain for a community organizing intervention. As the figure and th e research questions indicate, the study

wil l use Dubin‟s ( 1978 )

criteria to explore how well concept mapping can be used to develop

units which make up the

outcomes domain. It will also explore wheth er the units of the outcomes domain are both parsimonious and comprehensive ( Whet ten, 1989 )

for that domain , and what evidence exists for the validity of the articulated outcomes .

Definitions

D efinitions for the terms

theory , program , program theory

and theory - based evaluation

are now proposed, as these ideas will appear

throughout t he study. First,

consider a definition of theory .

Chen writes: “T heory

is a frame of reference that helps humans to understand their world and to function in it….theory provides not only guidelines for analyzing a phenomenon but also a scheme for unders tanding the significance of research findings”

( 1990, p. 17 ) . Lynham ( 2 002a, p. 221 )

states :

“What is the purpose of good theory other than to describe and explain how things actually work and, in so doing, to help us impro ve our actions in this world?” And Patterson notes

that : “…a theory is an attempt to organize and i nte g rate knowledge and to answer the question „ Why? ‟ ” ( 1986, p. xix ) .

T aken together, t hese ideas suggest the following definition for a theory . Theory is : a n organized

statement of assumptions and knowledge about a specified phenomenon which both (a) describe s

how it works and ( b) explain s

why it works.

Next, consider definition s

for the terms program

and program theory . This

study

will follow Chen ‟s understanding that a program “…is the

purposive and organized effort

7

to intervene in an ongoing social process for the purpose of solving a problem or providing a service” ( 1990, p. 39 ) . In other words, programs are one attempt at

improving human beings‟ circumstances in the world. Extend ing

the definition from above, one can

understand p rogram theory

to be defined as follows: an organized statement of assumptions and knowledge about a specified program which both (a) describe s

how it should and does work ,

and (b) explain s

why it should

and does work.

This definition is consistent with Chen ‟s notion

that such

a theory is two - fold. It has

one

portion which specifies “what the structure of a program should

be” and a nother portion

which states “…what are

the underlying causal mechanisms that l ink the relationships among program treatments, implementation processes and outcomes…” ( Chen, 1990, p. 43 ) . Thus , a complete program theory lays out

how the progr am both should

and in fact does

work.

Similarly, Weiss ( 1997b )

writes that theories concerning programs are two - fold in nature, consisting of an implementation theory (h ow a program should

work) and a programmatic theory (how it actually does

work).

T he distinction between prescriptive/ should

and descriptive/ does

theory can also be connected to Argyris and Schoe n‟s notions of “espoused” theory versus

“theory - in - use” ( 1974 ; 1996, p. 13 ) . An espoused theory is “…advanced to explain or justify a given pattern of activity”

(p. 13).

In contrast, theory - in - use is a theory that actually underlies an action. T heory - in - use “…must

be constructed from observation of the pattern of action in question ” ( Argyris & Schoen, 1996, p. 13 ) . All of t hese authors suggest that theory can

describe and explain an ideal situation (i.e. how the activity or action should

work) and can describe

and explain an actual situati on (i.e. how an activity or action in fact does

work). Indeed, a

complete

8

program theory must do

both.

The portions of program theory articulated in this study will be descriptive outcomes. Such descriptive theory is necessary before attempting to artic ulate intervening mechanisms which explain why

outcomes occur.

Finally, to define theory - based evaluation

one can

turn to Weiss‟s early work

( 1972 )

which appears to be the first to suggest that a program‟s theory should be a key component of that program‟s evaluation ( Weiss, 1997b ; Worthen, 1996 ) . In her

book, Weiss discuss es the fact that a model of the program‟s processes ( i.e. the prog ram‟s theory; see editor‟ s note in Weiss, 1996/1972 )

can begin to shed some light on why

certain outcomes occur or do not occur ( Weiss, 1972, p. 51 ) . This suggests that

a theory - based evaluation stands in contrast to a more black - box

type of evaluation ( Chen & Ross i, 1987 ) . In a black - box

evaluation , it may be established that certain inputs co - vary with outputs

or that there exists a statistical

association of inputs with

outputs. But a black box

evaluation provides no further information

about the nature of th e association. A

theory - based evaluation seeks to go further and explore why

such an association

occurs .

Philosophical Paradigm

Before continuing with a literature review and a description of the proposed study, the paradigmatic perspective for this resea rch

critical realism –

is described . Critical realism traces its intellectual roots to the work of Roy Bhaskar ( Bhaskar, 1975 , 1998 )

and it can be viewed as a “…middle ground between positivism and relativism” ( Bechara & Van de Ven, 2007, p. 61 ) . Bechara and Van de Ven provide

an accessible introduction to critical realism. Ontologically, critical realism acknowledges that “…there is a real world out there (consisting of material, mental, and emergent

9

products)” and that such reality is layered, stratified, multi - dimensional a nd mind - independent ( Bechara & Van de Ven, 2007, pp. 37, 64 ) . Epistemologically, critical realist thinking accepts that humans‟ understanding of reality is “limi ted” ( Bechara & Van de Ven, 2007, p. 37 ) . Furthermore, Bechara and Van de Ven state that inquiry can not

be “ impartial ”

and that , “…all facts, observations and da ta are theory - laden, implicitly or explicitly” ( 2007, p. 38 ) . In other words, no theories or accumulated knowledge are objective in the sense of being independen t of the observer who articulates them; all knowledge is interrelated with the perspective of the knower. Now consider the axiology of critical realism. An axiology of research refers to values that influence a researcher‟s choices in conducting inquiry ( Guba & Lincoln, 2005 ) . Van de Ven states the following assertion for critical realism:

Most phenomena in the social world are too rich to be underst ood adequately by any single person or perspective….[and] any given theoretical model is a partial representation of a complex phenomenon that reflects the perspective of the model builder....this requires scholars to be far more reflexive and transparent about their roles, interests, and perspectives...than they have [been] in the past. ( 2007, p. 14 )

Thus, an understanding of social phenomena as complex lead s to a set of values for research. These values include using multiple perspectives and encouraging reflexivity regarding the researcher‟s point of view. Methodologically, critical realism is also pluralistic and inclusive. Similar to the paradigm‟s epi stemological assumption, Bechara and Van de Ven note that “…no form of inquiry

[emphasis added] can be value - free and impartial; each is value - full” ( 2007, p. 38 ) . Thus, the paradigm recognizes that methods of inquiry reflect an underlying set of values and perspectives. The selection of method should depend on the research context because “…some methods are better warranted than others depending on the phenomeno n” ( Bechara & Van de Ven, 2007, p. 38 ) .

10

Finally, consider the teleology of critical realism. Teleology answers the question “T o what end ? ” is the

research condu cted ( Lincoln & Lynham, 2011 ) . In other words, what is the purpose of research? Bechcara and Van de Ven opine that “ …science is an error - correction process that is based on evidence from the world, rather than merely reflecting the scientist‟s opinions of the world” ( 2007, p. 65 ) . Further, the authors quote McKelvey (who cites Holton): “…the singular advantage of the realist method is its empirically - based, self - correcting approach to the discovery of truth” ( Holton, 1993 ; McKelvey, 2002, p. 754 ) . Thus, critical realist science pursues truthful knowledge that derives “…at least in part [from]

…the way the world is” ( Bechara & Van de Ven, 2007, p. 58 ) .

Pawson and Tilley ( 1997 )

describe a realist

perspective for evaluation which is also consistent with the critical realist

view described above. First, they note that:

Realism has sought to position itself as a model of scientific explanation which avoids the tra ditional epistemological poles of positivism and relativism. Realism's key feature is its stress on the mechanics of explanation, and its attempt to show that the usage of such explanatory strategies can lead to a progressive body of scientific knowledge.

( 1997, pp. 55 - 6 )

As with Bechara and Van de Ven ( 2007 ) , Pawson and

Tilley stress that realism is

an ontologically moderate paradigmatic framework, avoiding both completely positivist and completely relativist points of view. Pawson and Tilley also concur with Bechara and Van de Ven that realism incorporates a teleologic al end of knowledge accumulation. It is the stress on explanation and knowledge ac cumulation which is central to Pawson and Tilley‟s thinking about program evaluation from a realist perspective.

Pawson and Tilley provide a formula to express their idea ab out realist explanation: “…the basic realist explanatory formula [is]: regularity = mechanism + context ” ( 1997, p. 56 ) . In a related paper, Tilley explains thi s formula. He writes that

11

“…the realist understands causality in terms of underlying causal mechanisms generating regularities. The underlying causal mechanism will often be hidden....in the natural world, potential causal mechanism [ sic ] will only be ac tivated if the conditions are right for them” ( Tilley, 2000, pp. 4 - 5 ) . Thus, causal explanations are not of a constant conjunctive nature where it is

assumed that “like will always produce like” ( Tilley, 2000, p. 4 ) . Rather, realism allows for consideration of the effect of context when studying c ause. Pawson and Tilley modify the explanatory formula for the purpose of enunciating program theory (which they deem critical to a good evaluation). Their modified formula for program theory

Full document contains 169 pages
Abstract: The purpose of this methodological study is to explore how well a process called concept mapping (Trochim, 1989) can articulate the theory which underlies a social program. Articulation of a program's theory is a key step in completing a sound theory based evaluation (Weiss, 1997a). In this study, concept mapping is used to articulate the outcomes domain of a program theory, using Chen's (1990) six domains for program theory as an organizing framework. A grassroots community organization in Denver, Colorado, provides context for the study. With reference to Dubin's (1978) distinctions for theoretical units as a guide, the results of concept mapping are analyzed to determine whether they are useful in building a program theory. Results are also are evaluated to determine whether they present a comprehensive, parsimonious (Whetten, 1989) and valid representation of outcomes from the community organizing intervention. Methodological and statistical considerations for using concept mapping are mentioned. The study concludes that concept mapping is a promising tool for theory articulation. Study limitations and opportunities for future research are also discussed.