Effective communication techniques for eliciting information technology requirements
v Table of Contents Acknowledgments iv List of Tables viii List of Figures x CHAPTER 1. INTRODUCTION 1 Introduction to the Problem 1 Background of the Study 2 Statement of the Problem 5 Purpose of the Study 6 Rationale 7 Research Questions 8 Significance of the Study 9 Definition of Terms 10 Assumptions and Limitations 11 Conceptual Framework 12 Organization of the Remainder of the Study 15 CHAPTER 2. LITERATURE REVIEW 16 Requirements Elicitation 17 The Communication Crisis and IT Project Failure 18 Defining Success 37 CHAPTER 3. METHODOLOGY 55 Hypotheses 56 Research Design 56
vi Sample 59 Instrumentation 61 Variables 66 Design Framework 66 Design Process Flow. 67 Data Collection 67 Population and Sample 69 Data Analysis 72 Validity and Reliability 73 Statistical Analysis 74 Ethical Considerations 74 CHAPTER 4. RESULTS 76 Internet Survey 76 Data Coding 78 Participant Demographics 80 Statistical Analysis 83 Summary 104 CHAPTER 5. DISCUSSION, IMPLICATIONS, RECOMMENDATIONS 106 Review of Research Problem 106 Review of Research Purpose 110 Review of Supporting Literature 112 Review of Hypotheses 109 Review of Research Questions 109
vii Results 110 Implications 112 Limitations 113 Recommendations for Further Research 114 Conclusions 116 REFERENCES 117 APPENDIX. SURVEY QUESTIONNAIRE 131
viii List of Tables Table 1. Reasons for Project Failure 7 Table 2. Mapping Variables to Survey Questions 66 Table 3. Respondent’s Opt-In/Opt-Out 78 Table 4. Participant Functional Title 81 Table 5. Respondents’ Years in IT 82 Table 6. Project Size Participation 82 Table 7. JAD on Time Perception 84 Table 8. JAD on Scope 85 Table 9. JAD on Budget 86 Table 10. RAD on Time 88 Table 11. RAD on Scope 89 Table 12. RAD on Budget 91 Table 13. GSS on Time 92 Table 14. GSS on Scope 94 Table 15. GSS on Budget 95 Table 16. ANOVA on Time 97 Table 17. JAD, RAD, and GSS Means on Time 97 Table 18. LSD Comparisons on Time 98 Table 19. ANOVA on Scope 99 Table 20. JAD, RAD, and GSS Means on Scope 100 Table 21. LSD Comparisons on Scope 101 Table 22. ANOVA on Budget 102
ix Table 23. JAD, RAD, and GSS Means on Budget 102 Table 24. LSD Comparisons on Budget 103
x List of Figures Figure 1. Conceptual framework for studying the effects of communication techniques on IT project success 14 Figure 2. WBS model 48 Figure 3. Six steps to affordability management 52 Figure 4. Business-technology observation of a relationship between communication techniques and IT project success 68 Figure 5. JAD on time 84 Figure 6. JAD on scope 86 Figure 7. JAD on budget 87 Figure 8. RAD on time 89 Figure 9. RAD on scope 90 Figure 10. RAD on budget 92 Figure 11. GSS on time 93 Figure 12. GSS on scope 95 Figure 13. GSS on budget 96 Figure 14. Means plot of techniques used on time 99 Figure 15. Means plot of techniques used on scope 101 Figure 16. Means plot of techniques used on budget 104
CHAPTER 1. INTRODUCTION
Introduction to the Problem The elicitation of requirements has been one of the major contributing factors of Information Technology (IT) project failure. The problem of translating the needs or wants of the user to a technology solution is considered one of the most challenging tasks in a system’s development lifecycle. Furthermore, because requirements are the basic foundation that leads to the construction of an IT project solution, any failure in determining what is required of a system undoubtedly leads to a failure in the project outcome. Surveys performed by research groups such as the Standish Group (2004) identify IT project failure as a major concern for organizations whose dependence on technology continues to grow with the rapid change in the business sectors. The costs associated with project failure are at its heights as researchers continue to dissect failed projects to understand the underlying root causes for project failure. Project failure has been attributed to many different aspects of a software lifecycle ranging from technical to human challenges (Al-Rawas & Easterbrook, 1996; Bascom & Tichy, 2008; Fincham, 2002). One key area that has been commonly identified among scholars, business, and IT professionals is the problem of communicating requirements between the user and the technology communities (Havelka, 2003; Havelka & Lee, 2002; Jirotka & Goguen, 1994; Maiden & Hare, 1998). Communication is complex by nature. It
2 is, therefore, almost impossible to identify all possible factors that can lead to poor requirements communication and elicitation. Nevertheless, there are some communication techniques that have been discussed by scholars and utilized by practitioners that have demonstrated marked improvement in successfully gathering requirements. These techniques are rapid application development (RAD), joint application design (JAD), and group support system (GSS). Although effective in the requirements elicitation process, demonstrating how these techniques directly affect IT project success is lacking in the field of business and technology. This study, therefore, will present the effectiveness of the RAD, JAD and GSS techniques and their effectiveness on IT project success.
Background of the Study Identifying effective communication techniques in requirements elicitation requires a basic understanding of what requirements elicitation mean. The elicitation of requirements emerged from the field of requirements engineering as a means of ensuring all users’ needs are captured, managed, and presented in a precise, accurate, and complete manner that can be understood by user and technology stakeholders. The engineering of requirements is defined as the process in which systems analysts define and document the needed functional details of the software solution (Perse, 2001). The manner in which those needs are investigated and translated therefore becomes a very important first step in the software development process. IT researchers and IT and business professionals propose that a failure to correctly elicit requirements bears a direct correlation to IT project failure and can result in project delays, cost overruns, and out-of-scope
3 implementations (Havelka, 2003; Havelka & Lee, 2002; Jirotka & Goguen, 1994; Maiden & Hare, 1998). A summarized definition of requirements elicitation can simply be explained as the effective communication of user needs to software implementers in an accurately, defined, well understood manner, that can be translated into an information technology solution. The Communication Crisis and IT Project Failure When viewed from the perspective of requirements elicitation, communication can be simply defined as the sharing of verbal and documented information between two parties: the technical analyst and the business user. This communication exchange is critical to the elicitation process as it is the first step in the software development cycle. Getting the exchange right before moving to other phases of the development process is a key activity both parties need to master. Unfortunately most technical analysts and business users continuously fail to master such an activity resulting in today’s increased project failures. One of the major problems as identified by many researchers lies on the limitations of human factors otherwise known as the softer skills of IT development (Al- Rawas & Easterbrook, 1996; Liu & Khooshabeh, 2003; Williams & Cockburn, 2003; Yeh & Tsai, 2001). Human factor complexities arise because parties involved in the communication exchange view the exchange process through different conceptual frameworks. Conceptual frameworks define how individuals use internal representations to interpret information exchanged; in this case the user and technology communities’ present an adversely different understanding of the technology solution. This difference in perspectives wedges a gap between what is being communicated by the user and captured
4 by the analyst and what is required for the project. Others carry different perspectives and expectation on what criteria to use when defining project success (Browne & Rogich, 2001; Cushing, 1990; Darlington & Culley, 2002; G. B. Davis & Monroe, 1987; Saiedian & Dale, 2000; Yeh & Tsai, 2001). Defining Project Success Putting a definition on project success is a complex aspect of the software development lifecycle. The generally accepted industry standard for the definition of success is based on timely completion, within allocated budget, and achieved target or scope. This definition of success has been popular among researchers and practitioners because these aspects are quantifiable, measurable, and achievable (Fincham, 2002; Rodriguez-Repiso, Setchi, & Salmeron, 2007; Turner, 2004). Turner explained that the definition of project success and what is considered standard of success in the IT Industry must be closely aligned with users’ expectations for these measures to be effective. When a project’s objective is misaligned with that of users’ expectations, project failure is inevitable. Communication of success criteria is usually not quite explicit during the requirements elicitation phase of the software development lifecycle; therefore, user expectation and success measurements are never clarified and corrected prior to the start of development. The study, therefore, is to explore three group communication techniques—RAD, JAD, and GSS—used in requirements elicitation that are observed to achieve successful project delivery, with success defined by both user and technology communities as projects delivered on time, on scope, and on budget.
5 Statement of the Problem IT project failure has and continues to cost businesses millions of dollars as projects fail to be completed on time, on budget, and on scope. According to studies performed by the Standish Group, 30% of projects fail to complete on time; 70% fail to deliver on scope, and average development projects run 189% over budget (Standish Group, 1994, 1997, 2001). Zells (Development Trends, 1997) shed a positive light on the problem by reporting that slightly over 15% of projects are completed on time, on scope and on budget. A 2004 study performed by the Standish group collated 16 factors that cause IT project failure (see Table 1). The absence of clear requirements ranked highest as to the underlying cause for project failure in a studies performed by the Standish Group (1997; see Table 1). The relationship between unclear requirements and project failure is proven to be directly correlated with the lack of effective communication techniques between users and analysts (Naumann, Jenkins, & Wetherbe, 1983; Ramesh & Jarke, 2001). An InterSolv (1997) management report showed 41% of all project errors were directly attributed to incorrect requirements. The overarching question remains: How does communication contribute to incorrect requirements elicitation? Because communication is the medium used for transferring information from the user community to the technology community, it becomes a vital part of the requirements elicitation process. As was discussed before, because communication is complex requirements gathered within the exchange of information from the user to the developer are incorrectly translated, captured, and transposed into a technology solution that is different from original requests (Cannon, 1994; McManus, 2004; McManus & Wood-Harper, 2007).
6 Human limitations such as information processors and transformers are other root causes that lead to incorrect requirements being gathered and developed into failed IT projects. Several research studies demonstrate the effectiveness of communication techniques in requirements elicitation in a more general form while others discuss project failures and contributing factors of project failures (Aladwani, 2002; Bascom & Tichy, 2008; Cannon, 1994; McManus & Wood-Harper, 2007). There is a shortage of literature and data demonstrating what communication techniques yield IT project success outcomes based on the following standard success criteria: on time, within budget and on scope. Hence, the purpose of the study is to identify and present evidence that RAD, JAD, and GSS are communication techniques that can influence IT project success defined by on time, within budget, and on scope delivery (see Table 1).
Purpose of the Study The purpose of this study is to explore three communication techniques—RAD, JAD, and GSS—used in the elicitation of IT requirements that yield IT project success measured by scope, time, and budget. The aim is that this contribution will increase the awareness of such techniques in the business technology industry, thus contributing to the accurate elicitation of requirements and successful IT project implementation. Projects that utilize such techniques can be successfully managed from inception to implementation driving down IT failure costs in financial institutions.
7 Table 1. Reasons for Project Failure Project impaired factors % of responses Unclear requirements 13.1 Lack of user involvement 12.4 Lack of resources 10.6 Unrealistic expectations 9.9 Lack of executive support 9.3 Changing requirements & specifications 8.7 Lack of planning 8.1 Didn’t need it any longer 7.5 Lack of IT management 6.2 Technology illiteracy 4.3 Other 9.9
Note. From “The Chaos Report” by The Standish Group, 1994. Copyright 1994 by The Standish Research Group. Adapted with permission.
Rationale The underlying principle behind this study is to present three communication techniques used in requirements elicitation and their contribution to IT project success. The belief is that exploring these techniques as a part of the requirements gathering process of the software development lifecycle, showing their percentage utilization on successful IT projects, and how their uses are perceived by the user technology
8 community can prove or disprove their effectiveness. Although several techniques in communication are acknowledged in literature written by Bostrom (1989); Coughlan, Lycett, and Macredie (2003); and Gallivan and Keil (2003), a theoretical framework of linking such effective patterns to IT project success does not exist. As a result, this study will focus on how a sample of effective communication techniques contribute to IT project success, providing a foundation for the user–developer community to effectively elicit useful requirements, work collaboratively with the aim of understanding such requirements, and in so doing increase the probability of IT project success at the start of the project lifecycle. To summarize, the objectives of this study are threefold: 1. To identify and examine the underlying dynamics of communication techniques used in the software development domain during the elicitation of requirements. 2. Extrapolate the effectiveness of three of such techniques. 3. Demonstrate whether there is a difference in perception between the three elicitation techniques and time, scope, and budget as it relates to IT success.
Research Questions D. R. Cooper and Schindler (2003) proposed the use of the question hierarchy as a means to construct questions answered in a research study. As such, the primary research questions for this study are as follows: 1. Is there a relationship between the three group communication techniques, RAD, JAD, and GSS, and IT Project success measured by scope, time, and budget?
9 2. Can JAD, RAD, and GSS influence IT project success measured by time, budget, and scope? Answers to these questions have not been extensively researched in previously published literature. Most research studies encountered address the wider scope of IT project failures, while others focus on effective communication techniques for requirements elicitation. The argument of what is defined as IT project success is another area also widely discussed but no linkage of success criteria to elicitation techniques is explored. These gaps in research studies between effective communication techniques used in requirements elicitation and their correlation to a set of agreed upon predefined standards for project success is of a vital need for achieving IT project success in the field of business and technology.
Significance of the Study IT has become one of the necessary evils in the business domains. As businesses become more dependent on technology, IT projects become core tools needed by businesses in gaining competitive advantage over other contenders in their industry. However, IT project failure continues to augment business IT expenditure where the return on investment is considerably low (A. Davis, 1995; Hickey & Davis, 2004; Leffingwell & Widrig, 2000; Moody & Sindre, 2003; Wiegers, 2003). In the past, the responsibility for writing requirements was considered that of the user community. Several attempts at correctly eliciting and communicating such requirements from the user or business community to the technical community have been futile. While several reasons can be linked to the miscommunication and/or
10 misunderstanding of requirements, there is general consensus that eliciting and communicating requirements in a setting in which both business and technology communities can understand is desperately needed. This study, therefore, will help promote increased knowledge of communication techniques that can be utilized in the elicitation of IT requirements. The hope is by researching the topmost utilized effective techniques of communication when gathering requirements, and examining how their relationship to IT project outcomes, business analysts, project managers, and IT professionals in business organizations will now be exposed to these findings, allowing them to define effective requirements that will lead to successful IT solution implementation. The link between the human factors in information technology practices and that of the core logistics of technology activities such as design, code, and implementation, will drastically reduce the probability of project failure. Equal focus can be placed on human interaction and knowledge sharing, leading to a more collaborative contribution from both communities with the same objective—implementation of successful IT solutions.
Definition of Terms Budget. The cost of implementing the IT project solution. Elicitation. The gathering, collecting, and writing of requirements. IT. The information technology solution derived from the communicated requirements.
11 Project success. Any information technology project that was developed on time, on scope, and under or on budget. Scope. Limited to the requirements defined. Techniques. Reference to technique in this study is limited to practices or methods used to communicate the needs of the technology solution. Time. The length of time of the IT project from inception to implementation. User. A person in the business domain requesting the need for a technology solution.
Assumptions and Limitations The objective behind the study is to demonstrate the effectiveness of communication techniques used in the requirements gathering process that yield project success. Limitations of the study are largely due to its exploratory nature and include the following: 1. Techniques studied are not representative of all communication techniques used in information systems requirements gathering cross one or more. 2. Variations in detailed communication processes may influence IT project outcomes. 3. Criteria for success may differ based on different projects, companies, and experiences and other variables that cannot be covered based on the small number of participants. A more extensive study is needed to investigate other criteria that define a successful project.
12 4. The study focus is limited to three communication techniques—RAD, JAD, and GSS—in requirements elicitation as the key contributing factor for project success. There may be other contributing factors to IT project success that will not be explored. 5. The communication techniques explored are limited to what is used in requirements elicitation during a software development lifecycle and not for other purposes. However, the study can be reproduced to test the validity of the results in other domains. 6. The focus groups examined as participants in this study are the user and technology communities. Other stakeholders such as project sponsors, project managers, and many others are not considered a part of this study. 7. Results found in the study may not be representative of all users and development committee.
Conceptual Framework The conceptual framework of this study addresses the “key factors, constructions, or variables and the presumed relationship among them” (Miles & Huberman, 1994, p. 18). The relationship between communication techniques used in requirements elicitation specifically RAD, JAD, and GSS and IT project success specifically on time, on or under budget, or on scope delivery is presented in this study. Figure 1 demonstrates the conceptual framework of this study. The key independent variables to be explored are communication techniques for requirement elicitation and their relationship on IT Projects (see Figure 1). The approach
13 is to examine three group communication techniques—RAD, JAD, and GSS—as independent variables that are influencing factors on the outcome of IT projects based dependent variables time, scope, and budget. The problem domain is the communication of requirements by the user community to the technology community. Users’ understanding of what is needed from the IT solution seems to always differ from developers’ understanding of what IT solution to implement. Both communities approach the understanding of the IT requirements from different perspectives leading to the failure of the IT project. The conceptual framework was derived from the combination of works by Salaway (2002) and Tuunanen (2003). Salaway’s emphasis on communication patterns as the root cause to requirements shortfalls laid the foundation for exploring the division that exists between the user and development community. Tuunanen’s approach to communication techniques with a focus on their effective reach to the user community as a means of benchmarking IT project success introduced the idea that collaborative understanding of what IT solution is to be implemented can be the solution to the problem of IT project failure. The argument, therefore, is that if both communities are in consensus on the IT solution and work together to derive efficient requirements, then the IT solution will always meet the needs of the users ergo the organization. The resulting outcome is that IT project success can be easily achieved providing a cost advantage for the organization.
User Understanding of IT Solution Developer Understanding of IT Solution
Poor Communication User & Developer Understanding Comm Techniques Comm Techniques D E
I V E Effective Requirements B U I L
Successful IT Solution
R E S U L T
IT Project Success
Figure 1. Conceptual framework for studying the effects of communication techniques on IT project success.
15 Organization of the Remainder of the Study The remainder of the study is organized as follows. Chapter 2 presents a detailed review of literature on requirements elicitation techniques, group communication techniques such as JAD, RAD and GSS, how IT project success is measured, and causes of IT project failure. Chapter 3 presents the research design, sample, setting, instruments used in collecting and measuring data collected, the analysis of data, discussion on validity and reliability, and all ethical considerations taken when performing the research study. Chapter 4 presents the results of the study. Chapter 5 presents recommendations and conclusions.
CHAPTER 2. LITERATURE REVIEW
Requirements engineering (RE) refers to a process in systems engineering in which systems analysts describe and document specific functions required from a proposed computer information system (CIS). Because all new CIS development is initiated by determining the specific functions required of the end product, high expectations are placed on successfully exercising the RE process. The RE process itself is summarized in three stages: elicitation, presentation, and management. Mastering these three stages of the RE process is the daunting task each member of an IT project must pursue in order to achieve project success. Failure to do so raises the risk of faulty or erroneous system implementation (Browne & Rogich, 2001; Kenney & Leggiere, 2003; Standish Group, 2005; Walsh, 2003; Xia & Lee, 2004). Unsuccessful systems implementation is one of the most costly problems faced by IT organizations today and this is represented in reports of budget overruns, missed schedules, and unneeded systems development (Kenney & Leggiere, 2003; Schmidt, Lyytinen, Keil, & Cule, 2001). Although studies have been performed to understand the RE process (Herlea, 1999; Hickey & Davis, 2004; Moody & Sindre, 2003) in detail and the reasons behind failures in IT project implementation, specific studies around the requirements elicitation phase of the RE process and how it impacts project success have not been fully explored. The aim of this study is to fully explore the requirements
17 elicitation stage of the RE process with two main objectives: first, determine if communication techniques in requirements elicitation are key contributing factors for IT project success; second, show which communication technique impacts project success and at what rate such an impact is achieved. The intent of this research is to close the gap of identifying communication techniques that can be used in the eliciting of requirements to achieve project success.
Requirements Elicitation Before delving into the identification of communication patterns or defining what is meant by communication techniques, it is important to understand the meaning behind the elicitation of CIS requirements. The elicitation of requirements emerged from the field of requirements engineering in an attempt to ensure all users’ needs are captured, managed, and presented in a precise, accurate, and complete manner that can be understood by the user and technology stakeholders. The engineering of requirements is defined as the process in which systems analysts define and document the needed functions of the software solution (Perse, 2001). Derived from investigations and translations, requirements details are the means by which the wants and needs of the user community are translated into language the technology community can understand to develop a system that serves those needs. The manner in which user needs are investigated and translated becomes a very important first step in the software development process (R. B. Cooper & Swanson, 1979; G. B. Davis, 1982; Telem, 1988). Researchers and practitioners alike propose that a failure to correctly elicit requirements bears a direct correlation to IT project failure and can result
18 in project delays, cost overruns, and out-of-scope implementations (Bostrom & Kaiser, 1982). A summary definition of requirements elicitation can simply be explained as the effective communication of user needs to software implementers in an accurately, defined, well understood manner. Bostrom (1989), Nuseibeh and Easterbrook (2000), Davis, and others observed that the main difficulty in requirements elicitation lie in the limitations of the human factors in the effective communication of needs.
The Communication Crisis and IT Project Failure Communication is the exchange or sharing of information between two parties: a sender and a receiver. In the case of requirements elicitation, the analyst and the user can both play the role of a sender and a receiver at different points in time within the elicitation process. Communication is a critical aspect of the elicitation process and the fact that a large dependency is placed on the human factor adds an unwelcomed complexity to the process. Bostrom and Thomas (1983) proposed the reason for such complexity is because parties involved in the exchange “bring different conceptual frameworks” (p. 2) to the process. Conceptual frameworks are how individuals use internal representations to interpret information exchanged. The user and technology community generally bring different perspectives to the elicitation process. The views of both communities can be adversely different, wedging a big gap between what is being communicated by the user and captured by the analyst and what is required for the project. G. B. Davis (1982) proposed that these views are diverse because of two reasons:
19 1. The limitations of humans as information processors and problem solvers 2. The complexity of communication techniques between users and analysts in eliciting requirements. Human Limitations Humans as information processors. The nature of human beings as processors of information leans on the biological characteristics of the brain’s short- and long-term storage behaviors. Long-term memory bears a larger capacity for storage and recall when compared to short-term memory. Short-term memory is the information processing unit for humans and by nature proves to be the problem in communication during the requirements elicitation process. Its characteristics of processing seven plus or minus two chunks of data presented in character or visual form per second limits information absorbed during communication by the user and the analyst. Hence, the probability of gathering correct requirements diminishes with the volume and complexity of information being communicated (Bostrom & Thomas, 1983; G. B. Davis, 1982; London, 1976). Davis proposed that the method or techniques utilized in communication during this exchange of information becomes crucial in overcoming this limitation of short term storage. The use of methodologies that systematically elicit and store data in small chunks can significantly reduces the limitations of short-term memory. Humans as problem solvers. Problem solving is another area where humans exhibit limited capacity. This is due to the fact that humans simplify a problem domain into sizeable chunks that can be easily addressed in order to establish rationality. Such an approach binds rationality into a simplified model that does not portray a true representation of reality. However, the reconnection of such sizeable chunks back to the