A study of supervisor factors and their relationship to transfer of learning in a sales training course
vi Table of Contents CHAPTER 1. INTRODUCTION 1 Introduction to the Problem 1 Background of the Study 3 Statement of the Problem 9 Purpose of the Study 11 Rationale 11 Research Questions 12 Significance of the Study 13 Definition of Terms 16 Nature of the Study 17 Assumptions and Limitations 18 Organization of the Remainder of the Study 18 CHAPTER 2. LITERATURE REVIEW 19 Introduction 19 Adult learning Theory 19 Learning Transfer 22 Experiential Learning Theory and Constructivism 33 Authentic Learning 36 Designing for Transfer 37 Summary 50 CHAPTER 3. METHODOLOGY 52 Introduction 52
vii Purpose of the Study 52 Research Questions 53 Research Design 54 Population and Sample 55 Instruments 57 Data Collection Procedures 61 Inventory Data Analysis 62 Interview Data Analysis 63 Ethical Issues 65 Limitations 66 Summary 66 CHAPTER 4. DATA COLLECTION AND ANALYSIS 68 Introduction 68 Demographics of Inventory Participants 69 Descriptive Statistics for the Learning Transfer System Inventory 71 Quantitative Analysis of Research Questions 1 and 2 73 Qualitative Analysis of Research Question 3 76 Summary of Findings 105 CHAPTER 5. RESULTS, CONCLUSIONS, AND RECOMMENDATIONS 113 Introduction 113 Results Summary 115 Discussion of Results 121 Conclusions 127
Recommendations 129 Summary 133 REFERENCES 135 APPENDIX A. DISTRICT MANAGERS INTERVIEW GUIDE 144
APPENDIX B. DEMOGRAPHIC QUESTIONS 147
ix List of Tables Table 1. Summary of Research Questions, Methods, and Analysis 65 Table 2. Demographics of Inventory Participants 70 Table 3. Descriptive Statistics for the Learning Transfer Systems Inventory 72 Table 4. Supervisor Factors with Positive Transfer-Coefficients 74 Table 5. Supervisor Factors Relationship with Key Indicators-Coefficients 75
Table 6. Supervisor Factors with Negative Transfer and Key Indicators Coefficients 76
Table 7. Two-Way ANOVA Analysis of Supervisor Factors with Negative Transfer 76
Table 8. Reasons Why Transfer Occurred and Supporting Manager Comments 80
Table 9. Managers’ Interview Comments Related to Learning Transfer 83
Table 10. Manager Comments Related to Training Customization 85
Table 11. Instructional Design Phases and Number of Manager Responses 86
Table 12. Needs Analysis Summary of Recommendations, Information, and Practices 88
Table 13. Learner Analysis Summary of Recommendations, Information, and Practices 90
Table 14. Context Analysis Summary of Recommendations, Information, and Practices 92
Table 15. Training Design Summary of Recommendations, Information, and Practices 96
Table 16. Training Development Summary of Recommendations, Information, and Practices 98
Table 17. Considerations for Implementation and Recommendations, Information and Practices 100
x Table 18. Evaluation Summary of Recommendations, Information, and Practices 104
Table 19. Summary of Key Findings from the Qualitative Data 108
Table 20. Summary of Recommendations, Information and Practices in Each Instructional Design Phase 119
1 CHAPTER 1. INTRODUCTION Introduction to the Problem The issue of learning transfer and a learner’s ability to transfer learning from training environments to the job has always been an important part of learning. “Since the beginning of research on transfer, virtually every educational and cognitive researcher has repeatedly pointed out the importance of transfer” (Haskell, 2001, p. 10). The goal of education and training is the assumption that learning will be transferred, and without this assumption learning investments would be wasted (Haskell, 2001). Baldwin and Ford (1988) recognized that there was a transfer problem within workplace training, and through an extensive literature review on the topic, suggested areas for future research. Today it is estimated that American industries spend $129 billion on training annually, and that the transfer of training content to the job is as low as 10% (Baldwin & Ford, 1988; King, King, & Rothwell, 2000; Paradise, 2007). Burke and Hutchins (2007) built upon the work of Baldwin and Ford and concluded that investment in learning is still producing poor results. These poor results are specific to the gap between learning and employee’s sustained work performance or successful transfer of skills to their job. Three factors that influence transfer of learning to the job have continued to surface throughout the literature and in transfer models. The factors of learning characteristics, intervention design and delivery, and work environment influences impact workplace learning transfer significantly (Burke and Hutchins, 2007). The value of training in a corporate environment is measured by whether skills learned in training are transferred to the job and workplace performance improves. If
2 training and development is to be viewed as a critical supporting partner within an organization, then learning professionals must understand the factors that influence learning transfer and design learning to maximize transfer (Yamnill & McLean, 2005). According to Rossett (2007), a professor and author of educational technology, most professionals involved in workplace learning can describe Kirkpatrick’s (1998) levels of evaluation, but fewer than 15% are actually measuring transfer of behavior or skills (p 69). Although there are other ways of evaluating and measuring learning, Holton (1996) and Rossett (2007) both pointed out that Kirkpatrick’s work has been a standard in the industry for many years. This exposes two opportunities for instructional design professionals. First, there is an opportunity to increase efforts to measure transfer of skills in the workplace. Second, there are opportunities for instructional designers to explore the barriers and factors that impact learning transfer so they can improve their efforts at designing learning that will be transferred to the job. These opportunities require additional investigation. This research study helps instructional designers focus on the area of supervisor factors that include the measurement of the extent to which managers support and reinforce the use of learning on the job and the extent to which learners perceive negative responses from managers when applying skills learned in training. It also helps identify ways that instructional designers can involve the sales representatives’ supervisors to enhance the overall instructional design process. The identification of best practices can provide a platform that helps build relationships between instructional designers, supervisors, and the
3 learners they work with. These relationships may enhance the authentic nature of the learning event as well as enhance pre and post training support that could lead to greater learning transfer.
Background of the Study As mentioned above, American industries are spending billions of dollars on training annually (Baldwin & Ford, 1988; Yamnill & McLean, 2001). Organizations invest in their people, in hopes that the organization will remain competitive and continue to move forward in a positive direction (Pfeffr, 1994). The dollars invested are also expected to be used for training that addresses skill and performance deficiencies. The value of training is measured through the extent to which the training is transferred to the job. In other words, is the learner applying new skills on the job? Because of the low number associated with transfer to the job, instructional designers need be able to identify factors that facilitate or inhibit transfer of learning and design learning for maximum transfer. Gaining perspective about the factors that influence if a person will apply training on the job can be valuable to training departments and the companies that they support. (Seyler, Holton, Bates, Burnett & Carvalho, 1998). Understanding how to design, develop and evaluate learning solutions for maximum transfer of learning provides corporations an opportunity to maximize their learning dollars and allows instructional designers to address learning transfer head on. It has become critical that organizations know how learning impacts outcomes (Leach & Liu, 2003). This problem is creating a situation where corporations are seeing a low return on investment, and in an environment of time and cost constraints, it is important
4 to maximize the dollars invested in learning to improve performance. Researchers in the field have acknowledged that because companies are spending large amounts on sales training the pressure to justify the expense is rising (Leach & Liu, 2004; Phillips, 1998). Evaluation and understanding how to incorporate it in an instructional design process is important to the field of instructional design, and Kirkpatrick’s (1994; 1998) four levels of evaluation have been used in the corporate training environment to allow for a continuous improvement of training design. The levels allow instructional designers to evaluate reactions, learning, transfer, and results as part of a continuous improvement process. As designers begin to analyze the results from evaluations they can make important decisions about whether a training program should be continued, modified to correct disconnects, and justify the value of the training that has been designed (Kirkpatrick, 2007). The four levels of evaluation are not distinct from instructional design, but are an integral part of the process (Kirkpatrick, 2007). There is recent work with evaluation that aligns evaluation with strategy and instructional design, and this work with the four levels of evaluation is critical to driving synergies in today’s training environment (Kirkpatrick, 2007). Other recent articles have attempted to update the traditional ADDIE processes to incorporate evaluation as a central function that occurs throughout the process and as a way to facilitate continuous improvement of training design (Allen, 2006; Borin, Metcalf, & Tietje, 2007; Crawford, 2004). Transfer of learning to the job has an impact on an organization through employee performance. Research by Thompson, Brooks and Lizarraga (2003) examined 18 employees from various fields. The study demonstrated a benefit for the individuals and the organization when learners were able to transfer skills to the workplace, and
5 concluded that there was a causal linkage between learning, transfer and individual and organizational benefit. The primary reason companies invest in training is the belief that the training outcomes will help the organization achieve their objectives (Leach & Liu, 2004). A study of 2000 sales people conducted by Leach and Liu (2004) examined salesperson perceptions of training effectiveness. The findings provided evidence that measuring learning transfer or the application of the training into the salesperson’s work were the only type of evaluation that could predict organizational outcomes, and it is imperative that skills are applied on the job so that organizational goals and sales performance can be increased. The discussion emphasized that the results of this study support the “critical importance of learning transfer in learning design” (Leach & Liu, 2004, p. 335). As corporations invest in learning, instructional designers need to be aware of how to best design a learning experience for maximum impact and performance improvement. One way designers can help facilitate transfer is by understanding how supervisors contribute to the learning climate. Holton (1996) acknowledged that the Kirkpatrick model contributed to the field, but the “lack of research to develop further a theory of evaluation is a glaring shortcoming to the field” (p. 6). Although a theory of evaluation is not needed to study transfer of learning in the workplace, an approach to measuring application of learning can provide a more sound approach to demonstrating the contribution training programs make to an organization. Holton (1996) goes on to explain that for continued growth in the field, an evaluation model grounded in research is critical. Kirkpatrick (1998) pointed out his opinion “that there is little or no chance that training will transfer to job behavior if the climate is preventing or discouraging” (p. 22).
6 Additionally, the concept of management support is a relevant topic to the field of instructional design. Dick, Carey, and Carey (2001) pointed out the need to “learn about the organizational support that learners can expect to receive” after they learn new skills and attempt to apply them (p. 99). They explained that instructional designers are faced with added problems when management support is missing. By recognizing that most instructional design models encourage contextual analysis in the analysis phase, opportunity to understand how and what to include in a contextual analysis as it relates to managerial support becomes critical. If supervisor support is present, then the problems identified during a needs analysis will more likely be addressed (Dick et al., 2001). Previous transfer research has attempted to identify factors that influence transfer of learning, but have fallen short of identifying causal relationships between specific factors and learning transfer. The research has examined several different models, such as the model proposed by Phillips (1995) that added a focus on return on investment. Holton (1996) recognized this research was focused on taxonomies that contributed to conceptual thinking, but lacked the ability to identify the constructs that contribute to causal relationships between successful transfer and training. He believed that an instrument, which was theoretically based, psychometrically sound, and generalizable was necessary for the advancement of learning transfer research. In 1996, Holton and Bates developed the Learning Transfer System Inventory (LTSI) instrument “that could be used across a wide variety of organizations, training programs, and employees" (Holton, Bates, Bookter & Yamkovenko, 2007, p. 385). Analysis of transfer research that is focused on intervention design and delivery and work environment influences over the past 20 years has pointed to significant
7 opportunities for research that links transfer of learning to outcomes. Burke and Hutchins (2007) identified a significant need for qualitative research focused on design and delivery of learning, and the need for future research that can clarify the impact of transfer climate. Transfer climate refers to organizational situations and consequences that can inhibit or enhance on the job application of training (Rouiller & Goldstein, 1993). Holton (2007) also pointed out that past research has used a wide variety of instruments and measures that lack psychometric qualities and therefore little evidence exists that they measure what they are supposed to. These instruments included survey design that lacked attention to construct validity (Holton, 2007). This study addressed these gaps and contributed to the knowledge base through a mixed methods research design that utilizes a validated instrument. The Learning Transfer Systems Inventory is a standardized validated instrument that allows learning professionals to accurately diagnose transfer problems instead of guessing how to apply transfer strategies (Holton et al., 2007). It is important to use a standardized validated instrument when studying multiple organizations, and the continued research in different organizational settings will help further the ability to compare and generalize results of the Learning Transfer Systems Inventory and the factors affecting transfer (Holton, Chen, & Naquin, 2003). Holton (2007) also asserts that practitioners can increase transfer of learning by utilizing a valid and reliable instrument such as the Learning Transfer Systems Inventory to facilitate several key activities. Practitioners can use the Learning Transfer Systems Inventory to “assess potential transfer factor problems prior to conducting major learning interventions, follow up on evaluations of existing training programs, target interventions designed to enhance
8 transfer, incorporate evaluation of transfer as part of regular employee assessments, and conduct needs assessment for training programs to provide transfer skills to supervisors and trainers. Supervisor factors include positive supervisor support and reinforcement and negative supervisor responses. This research study identified if a correlation between supervisor factors and learning transfer exists. Through the exploration of this relationship, the understanding of the impact supervisors may have on learning transfer in a sales training environment was enhanced. It also helps instructional design professionals design and plan for transfer through the evaluation of what support is missing, what support employees desire from their supervisor, and the assessment of why supervisors may not be providing support. A literature review about training transfer by Burke and Hutchins (2007) pointed to several factors that warrant additional research in developing knowledge on the subject of transfer. They concluded that future research is needed to confirm linkages between transfer climate factors and utility reactions. Utility reactions are the learner’s perceived value or feeling that the training would help them perform on the job. Transfer climate factors refer to conditions or consequences in organizations, such as supervisor feedback and incentives, which enhance or discourage the use of learned skills in training back on the job. Continuing to explore the value of building stakeholder partnerships in the instructional design process can help produce training that is perceived as valuable and therefore more likely to be applied after leaving the training event.
9 Additionally, Burke and Hutchins (2007) discussed the use of needs analysis that addresses transfer barriers. They mention the Learning Transfer Systems Inventory as one such tool and suggest additional work linking the Learning Transfer Systems Inventory to transfer outcomes. This area of focus would also provide a link between practice and research, and help ground best practices with verifiable results (Burke & Hutchins, 2007). This research study built upon the foundation of research and provided results that future studies can use to continue to fill this gap. Transfer climate refers to the conditions in organizations that either prevent or encourage the application of what was learned in training to the job (Rouiller & Goldstein, 1993). It is important for instructional designers to understand transfer climate before implementing or designing transfer interventions because they need to tailor the interventions to target increased skill application on the job. In a study by Richman-Hirsh (2001), trainees who perceived a supportive transfer climate were more likely to transfer their skills to the workplace. Additionally, transfer climate aided in the understanding of the relationship between organizational learning culture and perceived innovation. This shows that climate has influence over other learning dimensions outside of training programs and supports the link between learning transfer and organizational impact (Burke& Hutchins, 2007).
Statement of the Problem There is a lack of research that addresses the relationship between supervisor support, whether positive or negative, with learning transfer. Additionally, there is a lack of research that can identify specific ways that supervisors can be involved throughout
10 the instructional design process to enhance transfer of learning and the effectiveness of course design. In a corporate training environment, training is focused on changing behavior or teaching new skills to individuals (Seyler et al., 1998). There has also been a lack of understanding surrounding what factors contribute to whether a learner will use or transfer training to the job (Seyler et al., 1998). This lack of understanding is still being observed in today’s corporate learning environment. Burke and Hutchins (2007) completed an extensive review of current literature and stated that a deficiency in designing training that transfers to the job is a core issue for learning professionals who are focused on designing interventions. Holton (1996) pointed out that one cause of failure to transfer is that the training design fails to present opportunities for transfer. As technology has emerged to offer options for training delivery, the line between training and constant on the job training has been distorted, and the need for empirical research in this area as it relates to learning transfer is needed (Burke & Hutchins, 2007). One conclusion that can drawn is that training design is an important consideration in learning transfer and should be considered no matter what the delivery method. Lack of transfer is increasing costs and reducing the return on investment of the training that is being taught (Coates, 2007). The American Society for Training and Development (ASTD) stated that corporations spend $129 billion on training every year (Paradise, 2007). This is up from $56.8 billion cited in 2001 (Phillips, 2002). It is critical that the training programs implemented today result in behavior change or skill transfer
11 on the job. Additional research is needed to expand the understanding of how to design for transfer of skills and the factors that contribute to transfer of learning.
Purpose of the Study This study helped identify the relationship between supervisor factors and the impact they may have on learners’ ability to transfer learning from a blended learning course to their job when they return to the field. Supervisor factors include a supervisor’s positive reinforcement of the skills learned in training and a supervisor’s negative responses to the learner when trying to apply new skills. This study explored supervisors’ perceptions about how supervisor support can help or hinder the transfer of learning, and identified possible strategies to overcome identified barriers. Additionally, this study examined and identified best practices and the most advantageous ways to involve supervisors in the entire instructional design process, and the relationship with supervisor support and negative supervisor reactions when the learner leaves training and attempts to apply the new skills on the job. With this foundation established, instructional designers can begin to utilize an evaluation method grounded in research and design learning solutions for increased transfer to the job.
Rationale The low numbers of skill transfer to the job associated with learning transfer show the importance of finding ways to increase learning transfer (Baldwin & Ford, 1988; King et al., 2000). As corporations invest in learning, instructional designers need to be aware of how to best design a learning experience for maximum impact and performance
12 improvement. When considering learning transfer in a sales training environment, increased learning transfer may translate into increased sales performance, more effective communication, enhanced relationships with customers, or increased customer satisfaction. One way designers can help facilitate transfer is by understanding how supervisors contribute to the learning climate. Kirkpatrick (1998) pointed out “that there is little or no chance that training will transfer to job behavior if the climate is preventing or discouraging” (p. 22). He went on to state the significance for trainers to understand the type of climate that learners will interact in after a training program, and the importance for trainers to do anything they can to create a climate of neutral or better. This study examined learning transfer in a corporate sales training environment. The sales representatives completed a blended learning training program that consisted of web based conferences, eLearning, print based home study materials, and face to face training. The content prepared the representatives to launch a new product and the study focused on the relationship a supervisor has on the transfer of these skills from a blended learning course to on the job application.
Research Questions 1. What is the relationship between supervisor support and the sales representative’s transfer of learning on the job? 2. What is the relationship between the supervisor providing negative support on the sales representative’s transfer of learning? 3. What information, practices, and recommendations should supervisors provide to instructional designers about how they may be involved in the instructional
13 design process to enhance transfer of learning and the effectiveness of course design?
Significance of the Study Research that focuses on relationships and key components of training effectiveness, such as supervisor support and supervisor sanctions, can give evidence for further exploration in a corporate training environment, and give guidance for future research related to designing for transfer. Seyler et al. (1998) pointed out key practical implications of research that focuses on critical components outside of the actual training design. They concluded that understanding the motivations to transfer training can help focus and improve training interventions by “guiding needs assessments, aiding in design of new as well as improving the design of existing training programs, and providing for more thorough training evaluations” (Seyler et al, 1998, p. 3). Tessmer and Richey (1997) stressed that many things can influence the instructional design process. They indicated that organizational climate factors, such as supervisor attitudes, play a significant role in whether the instructional content will be applied. This study stated an important implication that can be related to the current research study; before training design begins, a needs assessment that measures variables that may influence motivation to learn or transfer training, such as organizational commitment and transfer climate, should be conducted to identify weaknesses. This allows action to be taken to improve the effectiveness of the training design. By focusing on the factors of supervisor sanctions and supervisor support, it was possible to narrow down these specific influences on the training climate. The results can
14 benefit instructional designers in a corporate training environment in several ways. The identification of a relationship between supervisor sanctions and supervisor support on learning transfer will allow the field of instructional design to focus additional research on specific design strategies. For example, by grounding this theory in research an instructional designer would have evidence to support pre-training and post-training strategies and interventions, or evidence to support the inclusion of managers in the training design process. In a recent article by Gibb (2003), analysis of line manager involvement in learning and development highlighted a limited evidence base to effectively evaluate the significance of involving managers in learning and development activities. Additionally, instructional design professionals can benefit from the research findings by identifying the relationship that supervisors have on learning transfer. This may increase awareness of strategies to improve learning transfer and instructional designers can begin to accurately diagnose factors that inhibit transfer (Holton et al., 2007). As reported in a recent publication by the American Society for Training and Development, the importance of understanding a manager's role and preparing managers to support learning is often overlooked and contributes to why training fails to transfer (Coates, 2007). The key to overcoming this obstacle is through a partnership with learning professionals, learners, and managers (Coates, 2007). It is important for trainers to understand the type of climate that learners will interact in after a training program, and to do anything they can to create a climate of neutral or better (Kirkpatrick, 1998). This study also is important to corporate learning professionals and corporate executives hoping to increase return on investment and make decisions about where to
15 invest corporate learning dollars. With companies spending large amounts of money on training, there is an increase in the frequency to justify the expense (Phillips, 1998). Training professionals will need to measure their learning program outcomes consistently to better understand how transfer factors are interrelated and how they may predict outcomes (Leach & Liu, 2003). One key characteristic of the research study was the use of a validated instrument to explore this relationship. The data were collected with the Learning Transfer System Inventory. The Learning Transfer Systems Inventory was developed after critical analysis of Kirkpatrick’s evaluation model, and Holton, Bates, and Ruona (2000) proposed a new model to evaluate training transfer. This model provides the theoretical foundation for the instrument (Holton et al., 2007). The Learning Transfer Systems Inventory allows learning professionals to accurately diagnose transfer problems instead of guessing how to apply transfer strategies (Holton et al., 2007). A correlation between supervisor factors and learning transfer existed, and this helped further the understanding of factors that affect learning transfer in a sales training environment, and helps instructional design professionals design and plan for transfer through the evaluation of what support is missing, what support employees desire from their supervisor, and the assessment of why supervisors may not be providing support. By utilizing an instrument like the Learning Transfer Systems Inventory, designers can diagnose transfer problems early and include them in a needs assessment so they can address them throughout the instructional design process. They can also use this instrument as a reliable evaluation tool to make more informed decisions about what should be improved or modified in their training design or shed light on additional post