literature review on training design

Open access
Published: 01 October 2013

Effective in-service training design and delivery: evidence from an integrative literature review

Julia Bluestone 1 ,
Peter Johnson 1 ,
Judith Fullerton 2 ,
Catherine Carr 1 ,
Jessica Alderman 3 &
James BonTempo 1

Human Resources for Health volume 11 , Article number: 51 ( 2013 ) Cite this article

69k Accesses

172 Citations

54 Altmetric

Metrics details

In-service training represents a significant financial investment for supporting continued competence of the health care workforce. An integrative review of the education and training literature was conducted to identify effective training approaches for health worker continuing professional education (CPE) and what evidence exists of outcomes derived from CPE.

A literature review was conducted from multiple databases including PubMed, the Cochrane Library and Cumulative Index to Nursing and Allied Health Literature (CINAHL) between May and June 2011. The initial review of titles and abstracts produced 244 results. Articles selected for analysis after two quality reviews consisted of systematic reviews, randomized controlled trials (RCTs) and programme evaluations published in peer-reviewed journals from 2000 to 2011 in the English language. The articles analysed included 37 systematic reviews and 32 RCTs. The research questions focused on the evidence supporting educational techniques, frequency, setting and media used to deliver instruction for continuing health professional education.

The evidence suggests the use of multiple techniques that allow for interaction and enable learners to process and apply information. Case-based learning, clinical simulations, practice and feedback are identified as effective educational techniques. Didactic techniques that involve passive instruction, such as reading or lecture, have been found to have little or no impact on learning outcomes. Repetitive interventions, rather than single interventions, were shown to be superior for learning outcomes. Settings similar to the workplace improved skill acquisition and performance. Computer-based learning can be equally or more effective than live instruction and more cost efficient if effective techniques are used. Effective techniques can lead to improvements in knowledge and skill outcomes and clinical practice behaviours, but there is less evidence directly linking CPE to improved clinical outcomes. Very limited quality data are available from low- to middle-income countries.

Conclusions

Educational techniques are critical to learning outcomes. Targeted, repetitive interventions can result in better learning outcomes. Setting should be selected to support relevant and realistic practice and increase efficiency. Media should be selected based on the potential to support effective educational techniques and efficiency of instruction. CPE can lead to improved learning outcomes if effective techniques are used. Limited data indicate that there may also be an effect on improving clinical practice behaviours. The research agenda calls for well-constructed evaluations of culturally appropriate combinations of technique, setting, frequency and media, developed for and tested among all levels of health workers in low- and middle-income countries.

Peer Review reports

The need to increase the effectiveness and efficiency of both pre-service education and continuing professional education (CPE) (in-service training) for the health workforce has never been greater. Decreasing global resources and a pervasive critical shortage of skilled health workers are paralleled by an explosion in the increase of and access to information. Universities and educational institutions are rapidly integrating different approaches for learning that move beyond the classroom [ 1 ]. The opportunities exist both in initial health professional education and CPE to expand education and training approaches beyond classroom-based settings.

An integrative review was designed to identify and review the evidence addressing best practices in the design and delivery of in-service training interventions. The use of an integrative review expands the variety of research designs that can be incorporated within a review’s inclusion criteria and allows the incorporation of both qualitative and quantitative information [ 2 ]. Five questions were formulated based on a conceptual model of CPE developed by the Johns Hopkins University Evidence-Based Practice Center (JHU EPC) for an earlier systematic review of continuing medical education (CME) [ 3 ]. We asked whether: 1. particular educational techniques, 2. frequency of instruction (single or repetitive), 3. setting where instruction occurs, or 4. media used to deliver the instruction make a difference in learning outcomes; and, 5. if there was any evidence regarding the desired outcomes, such as improvements in knowledge, skills or changes in clinical practice behaviours, which could be derived from CPE, using any mixture of technique, media or frequency.

Inclusion/exclusion criteria

Articles were included in this review if they addressed any type of health worker pre-service or CPE event, and included an analysis of the short-term evaluation and/or assessment of the longer-term outcomes of the training. We included only those articles published in English language literature. These criteria gave priority to articles that used higher-order research methods, specifically meta-analyses or systematic reviews and evaluations that employed experimental designs. Articles excluded from analysis were observational studies, qualitative studies, editorial commentary, letters and book chapters.

Search strategy

A research assistant searched the electronic, peer-reviewed literature between May and June 2011. The search was conducted on studies published in the English language from 2000 to 2011. Multiple databases including PubMed, the Cochrane Library and Cumulative Index to Nursing and Allied Health Literature (CINAHL) were utilized in the search. Medical subject headings (MeSH) and key search terms are presented below in Table 1 .

Study type, quality assessment and grade

An initial review of titles and abstracts produced 244 results. We identified the strongest studies available, using a range of criteria tailored to the review methodology. Initial selection criteria were developed by a panel of experts. Grading and inclusion criteria are presented in Table 2 . The grading criteria were adapted from the Oxford Centre for Evidence-Based Medicine (OCEMB) levels of evidence model [ 4 ]. Grading of studies included within systematic reviews was reported by authors of those reviews and was not further assessed in this integrative review. Therefore, reference to quality of studies in our report refers to those a priori judgments. Only tier 1 articles (grades 1 and 2) were included in our analysis.

After prioritization of the articles, 163 tier 1 articles were assessed by a senior public health professional to determine topical relevance, study type and grade. A total of 61 tier 1 studies were selected to be included in the analysis following this second review. An additional hand search of the reference lists cited in published studies was conducted for topics that were underrepresented, specifically on the frequency and setting of educational activities. This search added eight articles for a total of 69 studies, including 37 systematic reviews and 32 randomized controlled trials (RCTs), see inclusion process for articles included in analysis, Figure 1 .

Inclusion process for articles included in the analysis.

A data extraction spreadsheet was developed, following the model offered in the Best Evidence in Medical Education (BEME) group series [ 5 ] and the conceptual model and definition of terms offered by Marinopoulos et al. in the JHU EPC earlier review of CME [ 3 ]. Categorization decisions were necessary in cases when the use of terminology was inconsistent with the Marinopoulos et al. definitions of terms for CPE [ 3 ]. For example, an article that analysed 'distance learning’ as a technique and used the computer as the medium to deliver an interactive e-learning course was coded and categorized as an 'interactive’ technique delivered via 'computer’ as the medium of instruction. See illustration of categorization terminology in panels A, B, and C, Figure 2 , for an illustration of how terminology was used to categorize and organize articles for analysis.

Illustration of categorization terminology in panels a-c.

Selected articles that best represent common findings and outcomes (effects) of CPE are discussed in the results and discussion sections; the related tables present all the articles analysed and categorized for that topic, and each article is included only once. Relevant information obtained from educational psychology literature is referenced in the discussion.

The articles or studies that specifically addressed educational techniques are summarized in Table 3 . Technique refers to the educational methods used in the instruction. Technique descriptions are based on the Marinopoulos et al. definitions of terms [ 6 ] and reflect the approaches defined in the articles analysed.

Case-based: use of created or actual clinical cases that present materials and questions

Though case-based learning was not specifically compared with other techniques in the literature reviewed, it was often noted as a method in articles that discussed interactive techniques. Case-based learning was also noted as a technique used for computer-delivered CPE courses. Triola et al. compared types of media utilized for case-based learning and found positive learning outcomes both with the use of a live standardized patient and a computer-based virtual patient [ 7 ].

Didactic/lecture: presenting knowledge content; facilitator determines content, organization and pace

Lecture was often referred to in the literature as traditional instruction, lecture-based or didactic teaching. Didactic instruction was not found to be an effective educational technique compared with other methods. Two studies [ 8 , 9 ] found no statistical difference in learning outcomes, and three studies found didactic to be less effective than other techniques [ 10 – 12 ]. Reynolds et al. compared didactic instruction with simulation. The study was limited by small sample size (n = 50), but still demonstrated that the simulation group had a significantly higher mean post-test score ( P <0.01) and overall higher learner satisfaction [ 12 ].

Several systemic reviews that compared didactic instruction to a wide variety of teaching approaches also identified didactic instruction as a less effective educational technique [ 13 – 15 ].

Feedback: providing information to the learner about performance

Multiple articles identified feedback as important for outcomes [ 16 – 18 ]. Herbert et al. compared individualized feedback in the form of a graphic (a prescribing portrait based on personal history of drug-prescribing practices) to small group discussion of the same material and found that both the feedback and the live, interactive session were somewhat effective at changing physician’s prescribing behaviours [ 16 ]. The Issenberg et al. systematic review of simulation identified practice and feedback as key for effective skill development [ 17 ]. A Cochrane review of the evidence to support CPE suggested the importance of feedback and instructor interaction in improving learning outcomes [ 18 ].

Games: competitive game with preset rules

The use of games as an instructional technology was addressed in one rigorous systematic review. The authors found only a limited number of studies, which were of low to moderate methodological quality and offered inconsistent results. Three of the five RCTs included in the review suggested that educational games could have a positive effect on increasing medical student knowledge and that they include interaction and allow for feedback [ 19 ].

Interactive: provide for interaction between the learner and facilitator

Five articles specifically compared interactive CPE to other educational techniques. De Lorenzo and Abbot found interactive techniques to be moderately superior for knowledge outcomes than didactic lecture [ 10 ]. Two other studies found interactive techniques were more effective when feedback from chart audits was added to the intervention [ 16 , 20 ].

Three systematic reviews and one meta-analysis specifically noted the importance of learner interactivity or engagement in learning in achieving positive learning outcomes [ 21 – 24 ] (refer to summary of articles focused on outcomes).

Point-of-care (POC): information provided as needed, at the point of clinical care

Two articles and one systematic review specifically addressed point-of-care (POC) as a technique. The systematic review included three studies and concluded that while the findings were weak, they did indicate that POC led to improved knowledge and confidence [ 25 ]. In an examination of media, Leung et al. determined that handheld devices were more effective than print-based, POC support, although outcome measures were self-reported behaviours [ 26 ]. You et al. found improved performance on a procedure among surgical residents who received POC mentoring via a video using a mobile device, compared with those who received only didactic instruction [ 27 ].

Problem-based learning (PBL): present a case, assign information-seeking tasks and answer questions about the case; can be facilitated or non-facilitated

Four articles specifically compared problem-based learning (PBL) to other methods. One study identified PBL as slightly better [ 11 ], and two studies indicated it to be relatively equal to didactic instruction [ 8 , 9 ]. A systematic review of 10 studies on PBL reported inconclusive evidence to support the approach, although several studies reported increased critical thinking skills and confidence in making decisions [ 28 ].

Reminders: provision of reminders

The Zurovac et al. study conducted in Kenya found that using mobile devices for repetitive reminders resulted in significant improvement in health care provider’s case management of paediatric malaria, and these gains were retained over a 6-month period [ 29 ]. Intention-to-treat analysis showed that correct management improved by 23.7% (95% confidence interval (CI) 7.6 to 40.0, P <0.01) immediately after intervention and by 24.5% (95% CI 8.1 to 41.0, P <0.01) 6 months later, compared with the control group [ 29 ]. Reminders were also noted as an effective technique by two of the systematic reviews [ 13 , 14 ].

Self-directed: completed independently by the learner based on learning needs

This term was difficult to extract for analysis due to widely varying terminology. Some authors used the term 'distance learning’, and some used it to define the medium of delivery, rather than technique. This analysis specifically discusses articles that were consistent with the description for self-directed learning, even if the authors used different terminology.

A recent systematic review identified that moderate-quality evidence suggests a slight increase in knowledge domain compared with traditional teaching, but notes that this may be due to the increased exposure to content [ 30 ]. One RCT found modest improvements in knowledge using a self-directed approach, but noted it was less effective at impacting attitudes or readiness to change [ 31 ].

Multiple studies focused on use of the computer as the medium to deliver instruction and noted that self-directed instruction was equally (or more) effective as instructor-led didactic or interactive instruction and potentially more efficient.

Simulation may include models, devices, standardized patients, virtual environments, social or clinical situations that simulate problems, events or conditions experienced in professional encounters [ 17 ]. Simulation was noted as an effective technique for promotion of learning outcomes across the systematic reviews, particularly for the development of psychomotor and clinical decision-making skills. The systematic reviews all highlighted inconclusive and weak methodology in the studies reviewed, but noted sufficient evidence existed to support simulation as useful for psychomotor and communication skill development [ 32 – 34 ] and to facilitate learning [ 35 ]. The systematic review by Lamb suggests that patient simulators, whether computer or anatomic models, are one of the more effective forms of simulations [ 36 ].

Outcomes of the four separate RCTs indicated simulation was better than the techniques to which they were compared, including interactive [ 37 , 38 ], didactic [ 12 ] and problem-based approaches [ 35 ]. A study by Daniels et al. found that although knowledge outcomes were similar between the interactive and simulation groups, the simulation team performance in a labour and delivery clinical drill was significantly higher for both shoulder dystocia (11.75 versus 6.88, P <0.01) and eclampsia (13.25 versus 11.38, P = 0.032) at 1 month post-intervention [ 38 ].

Simulation was also found to be useful for identifying additional learning gaps, such as a drill on the task of mixing magnesium sulfate for administration [ 39 ]. A systematic review focused on resuscitation training identified simulation as an effective technique, regardless of media or setting used to deliver it [ 40 ].

Team-based: providing interventions for teams that provide care together

Articles discussed here focused on the technique of providing training to co-workers engaged as learning teams. One systematic review of eight studies found that there is limited and inconclusive evidence to support team-based training [ 41 ]. Two of the articles reporting on the same CPE study did not identify any improvements in performance or knowledge acquisition with the addition of using a team-based approach [ 39 , 42 ].

This review included consideration of frequency, comparing single versus repetitive exposure. The findings regarding frequency are summarized in Table 4 .

The three articles focused on frequency all support the use of repetitive interventions. These studies evaluated repetition using the Spaced Education platform (now called Qstream), an Internet-based medium that uses repeated questions and targeted feedback. The evidence from these three articles demonstrated that repetitive, time-spaced education exposures resulted in better knowledge outcomes, better retention and better clinical decisions compared with single interventions and live instruction [ 43 – 45 ].

The use of repetitive or multiple exposures is supported in other systematic reviews of the literature, as well as one RCT conducted in Kenya that used repeated text reminders and resulted in a significant improvement in adherence to malaria treatment protocols [ 29 ].

Setting is the physical location within which the instruction occurs. We identified three articles that looked specifically at the training setting. The findings regarding setting are summarized in Table 5 . Two of them stemmed from the same intervention. Crofts et al. specifically addressed the impact of setting and technique (team-based training) on knowledge acquisition and found no significant difference in the post-score based on the setting [ 42 ]. A systematic review of eight articles evaluating the effectiveness of team-based training for obstetric care did not find significant differences in learning outcomes between a simulation centre and a clinical setting [ 41 ].

Coomarasamy and Khan conducted a systematic review and compared classroom or stand-alone versus clinically integrated teaching for evidence-based medicine (EBM). Their review identified that classroom teaching improved knowledge, but not skills, attitudes or behaviour outcomes; whereas clinically integrated teaching improved all outcomes [ 46 ]. This finding was supported by the Hamilton systematic review of CPE, which suggests that teaching in a clinical setting or simulation setting is more effective (Table 1 ), as well as the Raza et al. systematic review of 23 studies to evaluate stand-alone versus clinically integrated teaching. This review suggested that clinically integrated teaching improved skills, attitudes and behaviour, not just knowledge [ 18 ].

Media refers to the means used to deliver the curriculum. The majority of RCTs compared self-paced or individual instruction delivered via computer versus live, group-based instruction. The findings regarding media are summarized in Table 6 .

Live versus computer-based

Live instruction was found to be somewhat effective at improving knowledge, but less so for changing clinical practice behaviours. When comparing live to computer-based instruction, a frequent finding was that computer-based instruction led to either equal or slightly better knowledge performance on post-tests than live instruction. One of the few to identify a significant difference in outcomes, Harrington and Walker found the computer-based group outperformed the instructor-led group on the knowledge post-test and that participants in the computer-based group, on average, spent less time completing the training than participants in the instructor-led group [ 47 ].

Systematic reviews indicate that the evidence supports the use of computer-delivered instruction for knowledge and attitudes; however, insufficient evidence exists to support its use in the attempt to change practice behaviours. The Raza Cochrane systematic review identified 16 randomized trials that evaluated the effectiveness of Internet-based education used to deliver CPE to practicing health care professionals. Six studies showed a positive change in participants’ knowledge, and three studies showed a change in practice in comparison with traditional formats [ 18 ]. One systematic review noted the importance of interactivity, independent of media, in achieving an impact on clinical practice behaviours [ 48 ].

One article assessed the use of animations against audio instructions in cardiopulmonary resuscitation (CPR) using a mobile phone and found the group that had audiovisual animations performed better than the group that received live instruction over the phone in performing CPR; however, neither group was able to perform the psychomotor skill correctly [ 49 ]. Leung et al. found providing POC decision support via a mobile device resulted in slightly better self-reporting on outcome measures compared with print-based job aids, but that both the print and mobile groups showed improvements in use of evidence-based decision-making [ 26 ].

The systematic review of print-based materials conducted by Farmer et al. did not find sufficient evidence to support the use of print media to change clinical practice behaviours [ 50 ]. A comparison of the use of print-based guidelines to a live, interactive workshop indicated that those who completed live instruction were slightly better able to identify patients at high risk of an asthma attack. However, neither intervention resulted in changed practice behaviours related to treatment plans [ 51 ].

Multiple systematic reviews caution against the use of print only media, concluding that live instruction is preferable to print only. Another consistent theme was support for the use of multimedia in CPE interventions.

Outcomes are the consequences of a training intervention. This literature review focuses on changes in knowledge, attitudes, psychomotor, clinical decision-making or communication skills, and effects on practice behaviours and clinical outcomes. All of the articles that focused on outcomes were systematic reviews of the literature and are summarized in Table 7 .

The weight of the evidence across several studies indicated that CPE could effectively address knowledge outcomes, although several studies used weaker methodological approaches. Specifically, computer-based instruction was found to be equally or more effective than live instruction for addressing knowledge, while multiple repetitive exposures leads to better knowledge gains than a single exposure. Games can also contribute to knowledge if designed as interactive learning experiences that stimulate higher thinking through analysis, synthesis or evaluation.

No studies or systematic reviews looked only at attitudes, but CPE that includes clinical integration, simulations and feedback may help address attitudes. The JHU EPC group systematic review evaluation of the short- and long-term effects of CPE on physician attitudes reviewed 26 studies and, despite the heterogeneity of the studies, identified trends supporting the use of multimedia and multiple exposures for addressing attitudes [ 6 ].

Several systematic reviews looked specifically at skills, concluding that there is weak but sufficient evidence to suggest that psychomotor skills can be addressed with CPE interventions that include simulations, practice with feedback and/or clinical integration. 'Dose-response’ or providing sufficient practice and feedback was identified as important for skill-related outcomes. Other RCTs suggest clinically integrated education for supporting skill development. Choa et al. found that neither the audio mentoring via mobile nor animated graphics via mobile resulted in the desired psychomotor skills, reinforcing the need for practice and feedback for psychomotor skill development identified in other studies [ 49 ].

Two systematic reviews focused on communication skills and found techniques that include behaviour modeling, practice and feedback, longer duration or more practice opportunities were more effective [ 52 , 53 ]. Evidence suggests that development of communication skills requires interactive techniques that include practice-oriented strategies and feedback, and limit lecture and print-based materials to supportive strategies only.

Findings also suggest that simulation, PBL, multiple exposures and clinically integrated CPE can improve critical thinking skills. Mobile-based POC support was found to be more useful in the development of critical thinking than print-based job aids.

Several systematic reviews specifically looked at CPE, practice behaviours and the behaviours of the provider. These studies found, despite reportedly weak evidence, that interactive techniques that involved feedback, interaction with the educator, longer durations, multiple exposures, multimedia, multiple techniques and reminders may influence practice behaviours.

A targeted review of 37 articles from the JHU EPC review on the impact of CPE on clinical practice outcomes drew no firm conclusions, but multiple exposures, multimedia and multiple techniques were recommended to improve potential outcomes [ 6 ]. Interaction and feedback were found to be more useful than print or educational meetings (systematic review of nine articles) [ 24 ], but print-based unsolicited materials were not found to be effective [ 50 ]. The systematic review of live, classroom-based, multi-professional training conducted by Rabal et al. found 'the impact on clinical outcomes is limited’ [ 54 ].

The heterogeneity of study designs included in this review limits the interpretations that can be drawn. However, there is remarkable similarity between the information from studies included in this review and similar discussions published in the educational psychology literature. We believe that there is sufficient evidence to support efforts to implement and evaluate the combinations of training techniques, frequency, settings and media included in this discussion.

Avoid educational techniques that provide a passive transfer of information, such as lecture and reading, and select techniques that engage the learner in mental processing, for example, case studies, simulation and other interactive strategies. This recommendation is reinforced in educational psychology literature [ 55 ]. There is sufficient evidence to endorse the use of simulation as a preferred educational technique, notably for psychomotor, communication or critical thinking skills. Given the lack of evidence for didactic methods, selecting interactive, effective educational techniques remains the critical point to consider when designing CPE interventions.

Self-directed learning was also found to be an effective strategy, but requires the use of interactive techniques that engage the learner. Self-directed learning has the additional advantage of allowing learners to study at their own pace, select times convenient for them and tailor learning to their specific needs.

Limited evidence was found to support team-based learning or the provision of training in work teams. There is a need for further study in this area, given the value of engaging teams that are in the same place at the same time in an in-service training intervention. This finding is especially relevant for emergency skills that require the collaboration and cooperation of a team.

Repetitive exposure is supported in the literature. When possible, replace single-event frequency with targeted, repetitive training that provides reinforcement of important messages, opportunities to practice skills and mechanisms for fostering interaction. Recommendations drawn from the educational psychology literature that address the issue of cognitive overload [ 56 ] suggest targeting information to essentials and repetition.

Select the setting based on its ability to deliver effective educational techniques, be similar to the work environment and allow for practice and feedback. In this time of crisis, workplace learning that reduces absenteeism and supports individualized learning is critical. Conclusions from literature in educational psychology reinforce the importance of 'situating’ learning to make the experience as similar to the workplace as possible [ 57 ].

Certain common themes emerged from the many articles that commented on the role of media in CPE effectiveness. A number of systematic reviews suggest the use of multimedia in CPE. It is important to note that the studies that found similar knowledge outcomes between computer-based and live instruction stated that both utilized interactive techniques, possibly indicating the effectiveness was due to the technique rather than the media through which it was delivered. While the data on use of mobile technology to deliver CPE were limited, the study by Zurovac et al. indicated the potential power of mobile technology to improve provider adherence to clinical protocols [ 29 ]. Currently, there is unprecedented access to basic mobile technology and increasing access to lower-cost tablets and computers. The use of these devices to deliver effective techniques warrants exploration and evaluation, particularly in low- and middle-income countries.

CPE can positively impact desired learning outcomes if effective techniques are used. There are, however, very limited and weak data that directly link CPE to improved clinical practice outcomes. There are also limited data that link CPE to improved clinical practice behaviours, which may influence the strength of the linkage to outcomes.

Limitations

The following limitations apply to the methodology that we selected for this study. An integrative review of the literature was selected because the majority of published studies of education and training in low- and middle-resource countries did not meet the parameters required of a more rigorous systematic review or meta-analysis. The major limitation of integrative reviews is the potential for bias from their inclusion of non-peer-reviewed information or lower-quality studies. The inclusion of articles representing a range of rigor in their research design restricts the degree of confidence that can be placed on interpretations drawn by the authors of those articles, with the exception of original articles that explicitly discussed quality (such as systematic reviews). This review did not make an additional attempt to reanalyse or combine primary data.

Therefore, for purpose of this article, we also graded all articles and included only tier 1 articles in the analysis. This resulted in restriction of information on certain topics for this report, although a wider range of information is available.

We faced an additional limitation in that many articles included in the review were neither fully transparent nor consistent with terminology definitions used in other reports. This is due in part to the fact that we went beyond the bio-medical literature, to include studies conducted in the education and educational psychology literature, as was appropriate to the integrative review methodology. Certain topics were underdeveloped in the literature, which limits the interpretation that can be drawn on these topics. Other topics are addressed in studies conducted using lower-tier research methodologies (for example observational and/or qualitative studies) that were not included in this article. In addition, the overwhelming majority of studies focused on health professionals in developed or middle-income countries. There were very few articles of sufficient rigor conducted in low- and middle-income countries. This limits what we can say regarding the application of these findings among health workers of a lower educational level and in lower-resourced communities.

In-service training has been and will remain a significant investment in developing and maintaining essential competencies required for optimal public health in all global service settings. Regrettably, in spite of major investments, we have limited evidence about the effectiveness of the techniques commonly applied across countries, regardless of level of resource.

Nevertheless, all in-service training, wherever delivered, must be evidence-based. As stated in Bloom’s systematic review, 'Didactic techniques and providing printed materials alone clustered in the range of no to low effects, whereas all interactive programmes exhibited mostly moderate to high beneficial effect. … The most commonly used techniques, thus, generally were found to have the least benefit’ [ 14 ]. The profusion of mobile technology and increased access to technology present an opportunity to deliver in-service training in many new ways. Given current gaps in high-quality evidence from low- and middle-income countries, the future educational research agenda must include well-constructed evaluations of effective, cost-effective and culturally appropriate combinations of technique, setting, frequency and media, developed for and tested among all levels of health workers in low- and middle-income countries.

Abbreviations

Best Evidence in Medical Education

Confidence interval

Cumulative Index to Nursing and Allied Health Literature

Continuing medical education
Continuing professional education

Cardiopulmonary resuscitation

Evidence-based medicine

Johns Hopkins University Evidence-Based Practice Center

Medical subject headings

Oxford Centre for Evidence-Based Medicine

Problem-based learning

Point-of-care

Randomized controlled trial.

Frenk J, Chen L, Bhutta ZA, Cohen J, Crisp N, Evans T, Fineberg H, Garcia P, Ke Y, Kelley P, Kistnasamy B, Meleis A, Naylor D, Pablos-Mendez A, Reddy S, Scrimshaw S, Sepulveda J, Serwadda D, Zurayk H: Health professionals for a new century: transforming education to strengthen health systems in an interdependent world. Lancet. 2010, 376 (9756): 1923-1958. 10.1016/S0140-6736(10)61854-5.

PubMed Google Scholar

Wittemore R, Kanfl K: The integrative review: updated methodology. In J Adv Nursing. 2005, 52 (5): 546-553. 10.1111/j.1365-2648.2005.03621.x.

Google Scholar

Marinopoulos SS, Baumann MH: American College of Chest Physicians Health and Science Policy Committee: Methods and definition of terms: effectiveness of continuing medical education: American College of Chest Physicians Evidence-Based Educational Guidelines. Chest. 2009, 135 (3 Suppl): 17S-28S.

Oxford Centre for Evidence-Based Medicine (OCEBM) Levels of Evidence Working Group: The Oxford 2011 Levels of Evidence. 2011, Oxford: Centre for Evidence Based Medicine, http://www.cebm.net/?O=1025 ,

Best Evidence in Medical Education (BEME): Appendix 1-BEME Coding Sheet. 2011, http://www.medicalteacher.org/MEDTEACH_wip/supp%20files/BEME%204%20Figs%20%20Appendices/BEME4_Appx1.pdf ,

Marinopoulos SS, Dorman T, Ratanawongsa N, Wilson LM, Ashar BH, Magaziner JL, Miller RG, Thomas PA, Prokopowicz GP, Qayyum R, Bass EB: Effectiveness of continuing medical education. Evidence Report/Technology Assessment, No. 149, (Prepared by the Johns Hopkins Evidence-based Practice Center, under Contract No.290-02-0018) AHRQ Publication No. -7-E006. January 2007, Agency for Healthcare Research and Quality: Rockville, MD

Triola M, Feldman H, Kalet AL, Zabar S, Kachur EK, Gillespie C, Anderson M, Griesser C, Lipkin M: A randomized trial of teaching clinical skills using virtual and live standardized patients. J Gen Intern Med. 2006, 21 (5): 424-429. 10.1111/j.1525-1497.2006.00421.x.

CAS PubMed PubMed Central Google Scholar

Smits PB, De Buisonje CD, Verbeek JH, Van Dijk FJ, Metz JC, Ten Cate OJ: Problem-based learning versus lecture-based learning in postgraduate medical education. Scand J Work Environ Health. 2003, 29 (4): 280-287. 10.5271/sjweh.732.

White M, Michaud G, Pachev G, Lirenman D, Kolenc A, FitzGerald JM: Randomized trial of problem-based versus didactic seminars for disseminating evidence-based guidelines on asthma management to primary care physicians. J ContinEduc Health Prof. 2004, 24 (4): 237-243. 10.1002/chp.1340240407.

De Lorenzo RA, Abbott CA: Effectiveness of an adult-learning, self-directed model compared with traditional lecture-based teaching methods in out-of-hospital training. Acad Emerg Med. 2004, 11 (1): 33-37.

Lin CF, Lu MS, Chung CC, Yang CM: A comparison of problem-based learning and conventional teaching in nursing ethics education. Nurs Ethics. 2010, 17 (3): 373-382. 10.1177/0969733009355380.

Reynolds A, Ayres-de-Campos D, Pereira-Cavaleiro A, Ferreira-Bastos L: Simulation for teaching normal delivery and shoulder dystocia to midwives in training. Educ Health (Abingdon). 2010, 23 (3): 405-

CAS Google Scholar

Alvarez MP, Agra Y: Systematic review of educational interventions in palliative care for primary care physicians. Palliat Med. 2006, 20 (7): 673-683. 10.1177/0269216306071794.

Bloom BS: Effects of continuing medical education on improving physician clinical care and patient health: a review of systematic reviews. Int J Technol Assess Health Care. 2005, 21 (3): 380-385.

Satterlee WG, Eggers RG, Grimes DA: Effective medical education: insights from the Cochrane Library. Obstet Gynecol Surv. 2008, 63 (5): 329-333. 10.1097/OGX.0b013e31816ff661.

Herbert CP, Wright JM, Maclure M, Wakefield J, Dormuth C, Brett-MacLean P, Legare J, Premi J: Better Prescribing Project: a randomized controlled trial of the impact of case-based educational modules and personal prescribing feedback on prescribing for hypertension in primary care. Fam Pract. 2004, 21 (5): 575-581. 10.1093/fampra/cmh515.

Issenberg SB, McGaghie WC, Petrusa ER, Lee Gordon D, Scalese RJ: Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Med Teach. 2005, 27 (1): 10-28. 10.1080/01421590500046924.

Raza A, Coomarasamy A, Khan KS: Best evidence continuous medical education. Arch Gynecol Obstet. 2009, 280 (4): 683-687. 10.1007/s00404-009-1128-7.

Akl EA, Pretorius RW, Sackett K, Erdley WS, Bhoopathi PS, Alfarah Z, Schunemann HJ: The effect of educational games on medical students’ learning outcomes: a systematic review: BEME Guide No 14. Med Teach. 2010, 32 (1): 16-27. 10.3109/01421590903473969.

Laprise R, Thivierge R, Gosselin G, Bujas-Bobanovic M, Vandal S, Paquette D, Luneau M, Julien P, Goulet S, Desaulniers J, Maltais P: Improved cardiovascular prevention using best CME practices: a randomized trial. J Contin Educ Health Prof. 2009, 29 (1): 16-31. 10.1002/chp.20002.

Forsetlund L, Bjorndal A, Rashidian A, Jamtvedt G, O’Brien MA, Wolf F, Davis D, Odgaard-Jensen J, Oxman AD: Continuing education meetings and workshops: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2009, 2: CD003030-

Mansouri M, Lockyer J: A meta-analysis of continuing medical education effectiveness. J Contin Educ Health Prof. 2007, 27 (1): 6-15. 10.1002/chp.88.

Perry M, Draskovic I, Lucassen P, Vernooij-Dassen M, Van Achterberg T, Rikkert MO: Effects of educational interventions on primary dementia care: A systematic review. Int J Geriatr Psychiatr. 2011, 26 (1): 1-11. 10.1002/gps.2479.

Rampatige R, Dunt D, Doyle C, Day S, Van Dort P: The effect of continuing professional education on health care outcomes: lessons for dementia care. Int Psychogeriatr. 2009, 21 (Suppl 1): S34-S43.

Blaya JA, Fraser HS, Holt B: E-health technologies show promise in developing countries. Health Aff (Millwood). 2010, 29 (2): 244-251. 10.1377/hlthaff.2009.0894.

Leung GM, Johnston JM, Tin KY, Wong IO, Ho LM, Lam WW, Lam TH: Randomised controlled trial of clinical decision support tools to improve learning of evidence-based medicine in medical students. BMJ. 2003, 327 (7423): 1,090-

You JS, Park S, Chung SP, Park JW: Usefulness of a mobile phone with video telephony in identifying the correct landmark for performing needle thoracocentesis. Emerg Med J. 2009, 26 (3): 177-179. 10.1136/emj.2008.060541.

CAS PubMed Google Scholar

Yuan H, Williams BA, Fan L: A systematic review of selected evidence on developing nursing students’ critical thinking through problem-based learning. Nurse Educ Today. 2008, 28 (6): 657-663. 10.1016/j.nedt.2007.12.006.

Zurovac D, Sudoi RK, Akhwale WS, Ndiritu M, Hamer DH, Rowe AK, Snow RW: The effect of mobile phone text-message reminders on Kenyan health workers’ adherence to malaria treatment guidelines: a cluster randomised trial. Lancet. 2011, 378 (9793): 795-803. 10.1016/S0140-6736(11)60783-6.

PubMed PubMed Central Google Scholar

Murad MH, Coto-Yglesias F, Varkey P, Prokop LJ, Murad AL: The effectiveness of self-directed learning in health professions education: a systematic review. Med Educ. 2010, 44 (11): 1,057-1,068.

Young JM, Ward J: Can distance learning improve smoking cessation advice in family practice? A randomized trial. J Contin Educ Health Prof. 2002, 22 (2): 84-93. 10.1002/chp.1340220204.

Harder BN: Use of simulation in teaching and learning in health sciences: a systematic review. J Nurs Educ. 2010, 49 (1): 23-28. 10.3928/01484834-20090828-08.

McGaghie WC, Siddall VJ, Mazmanian PE, Myers J: American College of Chest Physicians Health and Science Policy Committee: Lessons for continuing medical education from simulation research in undergraduate and graduate medical education: effectiveness of continuing medical education: American College of Chest Physicians Evidence-Based Educational Guidelines. Chest. 2009, 135 (3 Suppl): 62S-68S.

Sturm LP, Windsor JA, Cosman PH, Cregan P, Hewett PJ, Maddern GJ: A systematic review of skills transfer after surgical simulation training. Ann Surg. 2008, 248 (2): 166-179. 10.1097/SLA.0b013e318176bf24.

Steadman RH, Coates WC, Huang YM, Matevosian R, Larmon BR, McCullough L, Ariel D: Simulation-based training is superior to problem-based learning for the acquisition of critical assessment and management skills. Crit Care Med. 2006, 34 (1): 151-157. 10.1097/01.CCM.0000190619.42013.94.

Lamb D: Could simulated emergency procedures practised in a static environment improve the clinical performance of a Critical Care Air Support Team (CCAST)? A literature review. Intensive Crit Care Nurs. 2007, 23 (1): 33-42. 10.1016/j.iccn.2006.07.002.

Bruppacher HR, Alam SK, LeBlanc VR, Latter D, Naik VN, Savoldelli GL, Mazer CD, Kurrek MM, Joo HS: Simulation-based training improves physicians’ performance in patient care in high-stakes clinical setting of cardiac surgery. Anesthesiology. 2010, 112 (4): 985-992. 10.1097/ALN.0b013e3181d3e31c.

Daniels K, Arafeh J, Clark A, Waller S, Druzin M, Chueh J: Prospective randomized trial of simulation versus didactic teaching for obstetrical emergencies. Simul Healthc. 2010, 5 (1): 40-45. 10.1097/SIH.0b013e3181b65f22.

Ellis D, Crofts JF, Hunt LP, Read M, Fox R, James M: Hospital, simulation center, and teamwork training for eclampsia management: a randomized controlled trial. Obstet Gynecol. 2008, 111 (3): 723-731. 10.1097/AOG.0b013e3181637a82.

Hamilton R: Nurse’s knowledge and skill retention following cardiopulmonary resuscitation training: a review of the literature. J Adv Nurs. 2005, 51 (3): 288-297. 10.1111/j.1365-2648.2005.03491.x.

Merien AE, Van de Ven J, Mol BW, Houterman S, Oei SG: Multidisciplinary team training in a simulation setting for acute obstetric emergencies: a systematic review. Obstet Gynecol. 2010, 115 (5): 1,021-1,031.

Crofts JF, Ellis D, Draycott TJ, Winter C, Hunt LP, Akande VA: Change in knowledge of midwives and obstetricians following obstetric emergency training: a randomised controlled trial of local hospital, simulation centre and teamwork training. BJOG. 2007, 114 (12): 1,534-1,541.

Kerfoot BP, Fu Y, Baker H, Connelly D, Ritchey ML, Genega EM: Online spaced education generates transfer and improves long-term retention of diagnostic skills: a randomized controlled trial. J Am Coll Surg. 2010, 211 (3): 331-337. 10.1016/j.jamcollsurg.2010.04.023.

Kerfoot BP, Kearney MC, Connelly D, Ritchey ML: Interactive spaced education to assess and improve knowledge of clinical practice guidelines: a randomized controlled trial. Ann Surg. 2009, 249 (5): 744-749. 10.1097/SLA.0b013e31819f6db8.

Kerfoot BP, Baker HE, Koch MO, Connelly D, Joseph DB, Ritchey ML: Randomized, controlled trial of spaced education to urology residents in the United States and Canada. J Urol. 2007, 177 (4): 1,481-1,487.

Coomarasamy A, Khan KS: What is the evidence that postgraduate teaching in evidence based medicine changes anything? A systematic review. BMJ. 2004, 329 (7473): 1,017-

Harrington SS, Walker BL: The effects of computer-based training on immediate and residual learning of nursing facility staff. J Contin Educ Nurs. 2004, 35 (4): 154-163. quiz 186–7

Curran VR, Fleet L: A review of evaluation outcomes of web-based continuing medical education. Med Educ. 2005, 39 (6): 561-567. 10.1111/j.1365-2929.2005.02173.x.

Choa M, Park I, Chung HS, Yoo SK, Shim H, Kim S: The effectiveness of cardiopulmonary resuscitation instruction: animation versus dispatcher through a cellular phone. Resuscitation. 2008, 77 (1): 87-94. 10.1016/j.resuscitation.2007.10.023.

Farmer AP, Legare F, Turcot L, Grimshaw J, Harvey E, McGowan JL, Wolf F: Printed educational materials: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2008, 3: CD004398-

Liaw ST, Sulaiman ND, Barton CA, Chondros P, Harris CA, Sawyer S, Dharmage SC: An interactive workshop plus locally adapted guidelines can improve general practitioners asthma management and knowledge: a cluster randomised trial in the Australian setting. BMC Fam Pract. 2008, 9: 22-10.1186/1471-2296-9-22.

Gysels M, Richardson A, Higginson IJ: Communication training for health professionals who care for patients with cancer: a systematic review of training methods. Support Care Canc. 2005, 13 (6): 356-366. 10.1007/s00520-004-0732-0.

Berkhof M, Van Rijssen HJ, Schellart AJ, Anema JR, Van der Beek AJ: Effective training strategies for teaching communication skills to physicians: an overview of systematic reviews. Patient Educ Couns. 2011, 84 (2): 152-162. 10.1016/j.pec.2010.06.010.

Rabol LI, Ostergaard D, Mogensen T: Outcomes of classroom-based team training interventions for multiprofessional hospital staff: A systematic review. Qual Saf Health Care. 2010, 19 (6): e27-10.1136/qshc.2009.037184.

Woolfolk AE: Educational Psychology. 2009, Upper Saddle River, NJ: Pearson/Merrill/Prentice Hall, 11

Mayer RE: Applying the science of learning to medical education. Med Educ. 2010, 44 (6): 543-549. 10.1111/j.1365-2923.2010.03624.x.

Gott S, Lesgold A: Competence in the workplace: how cognitive performance models and situated instruction can accelerate skill acquisition. Advances in Instructional Psychology, Volume 4. Edited by: Glaser R. 2000, Hillsdale, NJ: Erlbaum

Download references

Acknowledgments

We thank the Jhpiego Corporation for support for this research. We thank Dana Lewison, Alisha Horowitz, Rachel Rivas D’Agostino and Trudy Conley for their support in editing and formatting the manuscript. We also thank Spyridon S Marinopoulos, MD, MBA, from the Johns Hopkins University School of Medicine, for his initial input into the study and links to relevant resources. The findings, interpretations and conclusions expressed in this paper are those of the authors and not necessarily those of the Jhpiego Corporation.

Author information

Authors and affiliations.

Jhpiego Corporation, 1615 Thames Street, Baltimore, MD, 21231, USA

Julia Bluestone, Peter Johnson, Catherine Carr & James BonTempo

Independent Consultant, San Diego, CA, USA

Judith Fullerton

Research Assistant, Baltimore, MD, USA

Jessica Alderman

You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Julia Bluestone .

Additional information

Competing interests.

The authors declare they have no competing interests.

Authors’ contributions

JB performed article reviews for inclusion, synthesized data and served as primary author of the analysis and manuscript. PJ conceived the study, participated in its design and coordination, and provided significant input into the manuscript. JF provided guidance on the literature review process, grading and categorizing criteria, and quality review of selected articles, and participated actively as an author of the manuscript. CC and JBT contributed to writing of the manuscript. JA searched the literature, performed initial review and coding, and contributed to selected sections of the manuscript. All authors read and approved the final manuscript.

Julia Bluestone, Peter Johnson, Catherine Carr contributed equally to this work.

Authors’ original submitted files for images

Below are the links to the authors’ original submitted files for images.

Authors’ original file for figure 1

Authors’ original file for figure 2, authors’ original file for figure 3, rights and permissions.

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License ( http://creativecommons.org/licenses/by/2.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article.

Bluestone, J., Johnson, P., Fullerton, J. et al. Effective in-service training design and delivery: evidence from an integrative literature review. Hum Resour Health 11 , 51 (2013). https://doi.org/10.1186/1478-4491-11-51

Download citation

Received : 08 August 2012

Accepted : 02 May 2013

Published : 01 October 2013

DOI : https://doi.org/10.1186/1478-4491-11-51

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

In-service training
Continuing professional development

Human Resources for Health

ISSN: 1478-4491

Submission enquiries: Access here and click Contact Us
General enquiries: [email protected]

A-Z Publications

Annual Review of Psychology

Volume 60, 2009, review article, benefits of training and development for individuals and teams, organizations, and society.

Herman Aguinis 1 , and Kurt Kraiger 2
View Affiliations Hide Affiliations Affiliations: 1 The Business School, University of Colorado Denver, Denver, Colorado 80217-3364; email: [email protected] 2 Department of Psychology, Colorado State University, Fort Collins, Colorado 80523-1876; email: [email protected]
Vol. 60:451-474 (Volume publication date January 2009) https://doi.org/10.1146/annurev.psych.60.110707.163505
© Annual Reviews

This article provides a review of the training and development literature since the year 2000. We review the literature focusing on the benefits of training and development for individuals and teams, organizations, and society. We adopt a multidisciplinary, multilevel, and global perspective to demonstrate that training and development activities in work organizations can produce important benefits for each of these stakeholders. We also review the literature on needs assessment and pretraining states, training design and delivery, training evaluation, and transfer of training to identify the conditions under which the benefits of training and development are maximized. Finally, we identify research gaps and offer directions for future research.

Article metrics loading...

Full text loading...

Article Type: Review Article

Most Read This Month

Most cited most cited rss feed, job burnout, executive functions, social cognitive theory: an agentic perspective, on happiness and human potentials: a review of research on hedonic and eudaimonic well-being, sources of method bias in social science research and recommendations on how to control it, mediation analysis, missing data analysis: making it work in the real world, grounded cognition, personality structure: emergence of the five-factor model, motivational beliefs, values, and goals.

Publication Date: 10 Jan 2009

Online Option

Sign in to access your institutional or personal subscription or get immediate access to your online copy - available in PDF and ePub formats

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Publications
Account settings
My Bibliography
Collections
Citation manager

Save citation to file

Email citation, add to collections.

Create a new collection
Add to an existing collection

Add to My Bibliography

Your saved search, create a file for external citation management software, your rss feed.

Search in PubMed
Search in NLM Catalog
Add to Search

Effective in-service training design and delivery: evidence from an integrative literature review

Affiliation.

1 Jhpiego Corporation, 1615 Thames Street, Baltimore, MD 21231, USA. [email protected].
PMID: 24083659
PMCID: PMC3850724
DOI: 10.1186/1478-4491-11-51

Background: In-service training represents a significant financial investment for supporting continued competence of the health care workforce. An integrative review of the education and training literature was conducted to identify effective training approaches for health worker continuing professional education (CPE) and what evidence exists of outcomes derived from CPE.

Methods: A literature review was conducted from multiple databases including PubMed, the Cochrane Library and Cumulative Index to Nursing and Allied Health Literature (CINAHL) between May and June 2011. The initial review of titles and abstracts produced 244 results. Articles selected for analysis after two quality reviews consisted of systematic reviews, randomized controlled trials (RCTs) and programme evaluations published in peer-reviewed journals from 2000 to 2011 in the English language. The articles analysed included 37 systematic reviews and 32 RCTs. The research questions focused on the evidence supporting educational techniques, frequency, setting and media used to deliver instruction for continuing health professional education.

Results: The evidence suggests the use of multiple techniques that allow for interaction and enable learners to process and apply information. Case-based learning, clinical simulations, practice and feedback are identified as effective educational techniques. Didactic techniques that involve passive instruction, such as reading or lecture, have been found to have little or no impact on learning outcomes. Repetitive interventions, rather than single interventions, were shown to be superior for learning outcomes. Settings similar to the workplace improved skill acquisition and performance. Computer-based learning can be equally or more effective than live instruction and more cost efficient if effective techniques are used. Effective techniques can lead to improvements in knowledge and skill outcomes and clinical practice behaviours, but there is less evidence directly linking CPE to improved clinical outcomes. Very limited quality data are available from low- to middle-income countries.

Conclusions: Educational techniques are critical to learning outcomes. Targeted, repetitive interventions can result in better learning outcomes. Setting should be selected to support relevant and realistic practice and increase efficiency. Media should be selected based on the potential to support effective educational techniques and efficiency of instruction. CPE can lead to improved learning outcomes if effective techniques are used. Limited data indicate that there may also be an effect on improving clinical practice behaviours. The research agenda calls for well-constructed evaluations of culturally appropriate combinations of technique, setting, frequency and media, developed for and tested among all levels of health workers in low- and middle-income countries.

PubMed Disclaimer

Inclusion process for articles included…

Inclusion process for articles included in the analysis.

Illustration of categorization terminology in…

Illustration of categorization terminology in panels a-c.

Publication types

Search in MeSH

Related information

Linkout - more resources, full text sources.

BioMed Central
Europe PubMed Central
PubMed Central

Other Literature Sources

scite Smart Citations

Research Materials

NCI CPTC Antibody Characterization Program

Miscellaneous

NCI CPTAC Assay Portal

Citation Manager

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

Knowledge Base

Methodology

How to Write a Literature Review | Guide, Examples, & Templates

How to Write a Literature Review | Guide, Examples, & Templates

Published on January 2, 2023 by Shona McCombes . Revised on September 11, 2023.

What is a literature review? A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research that you can later apply to your paper, thesis, or dissertation topic .

There are five key steps to writing a literature review:

Search for relevant literature
Evaluate sources
Identify themes, debates, and gaps
Outline the structure
Write your literature review

A good literature review doesn’t just summarize sources—it analyzes, synthesizes , and critically evaluates to give a clear picture of the state of knowledge on the subject.

Instantly correct all language mistakes in your text

Upload your document to correct all your mistakes in minutes

What is the purpose of a literature review, examples of literature reviews, step 1 – search for relevant literature, step 2 – evaluate and select sources, step 3 – identify themes, debates, and gaps, step 4 – outline your literature review’s structure, step 5 – write your literature review, free lecture slides, other interesting articles, frequently asked questions, introduction.

Quick Run-through
Step 1 & 2

When you write a thesis , dissertation , or research paper , you will likely have to conduct a literature review to situate your research within existing knowledge. The literature review gives you a chance to:

Demonstrate your familiarity with the topic and its scholarly context
Develop a theoretical framework and methodology for your research
Position your work in relation to other researchers and theorists
Show how your research addresses a gap or contributes to a debate
Evaluate the current state of research and demonstrate your knowledge of the scholarly debates around your topic.

Writing literature reviews is a particularly important skill if you want to apply for graduate school or pursue a career in research. We’ve written a step-by-step guide that you can follow below.

Don't submit your assignments before you do this

The academic proofreading tool has been trained on 1000s of academic texts. Making it the most accurate and reliable proofreading tool for students. Free citation check included.

Try for free

Writing literature reviews can be quite challenging! A good starting point could be to look at some examples, depending on what kind of literature review you’d like to write.

Example literature review #1: “Why Do People Migrate? A Review of the Theoretical Literature” ( Theoretical literature review about the development of economic migration theory from the 1950s to today.)
Example literature review #2: “Literature review as a research methodology: An overview and guidelines” ( Methodological literature review about interdisciplinary knowledge acquisition and production.)
Example literature review #3: “The Use of Technology in English Language Learning: A Literature Review” ( Thematic literature review about the effects of technology on language acquisition.)
Example literature review #4: “Learners’ Listening Comprehension Difficulties in English Language Learning: A Literature Review” ( Chronological literature review about how the concept of listening skills has changed over time.)

You can also check out our templates with literature review examples and sample outlines at the links below.

Download Word doc Download Google doc

Before you begin searching for literature, you need a clearly defined topic .

If you are writing the literature review section of a dissertation or research paper, you will search for literature related to your research problem and questions .

Make a list of keywords

Start by creating a list of keywords related to your research question. Include each of the key concepts or variables you’re interested in, and list any synonyms and related terms. You can add to this list as you discover new keywords in the process of your literature search.

Social media, Facebook, Instagram, Twitter, Snapchat, TikTok
Body image, self-perception, self-esteem, mental health
Generation Z, teenagers, adolescents, youth

Search for relevant sources

Use your keywords to begin searching for sources. Some useful databases to search for journals and articles include:

Your university’s library catalogue
Google Scholar
Project Muse (humanities and social sciences)
Medline (life sciences and biomedicine)
EconLit (economics)
Inspec (physics, engineering and computer science)

You can also use boolean operators to help narrow down your search.

Make sure to read the abstract to find out whether an article is relevant to your question. When you find a useful book or article, you can check the bibliography to find other relevant sources.

You likely won’t be able to read absolutely everything that has been written on your topic, so it will be necessary to evaluate which sources are most relevant to your research question.

For each publication, ask yourself:

What question or problem is the author addressing?
What are the key concepts and how are they defined?
What are the key theories, models, and methods?
Does the research use established frameworks or take an innovative approach?
What are the results and conclusions of the study?
How does the publication relate to other literature in the field? Does it confirm, add to, or challenge established knowledge?
What are the strengths and weaknesses of the research?

Make sure the sources you use are credible , and make sure you read any landmark studies and major theories in your field of research.

You can use our template to summarize and evaluate sources you’re thinking about using. Click on either button below to download.

Take notes and cite your sources

As you read, you should also begin the writing process. Take notes that you can later incorporate into the text of your literature review.

It is important to keep track of your sources with citations to avoid plagiarism . It can be helpful to make an annotated bibliography , where you compile full citation information and write a paragraph of summary and analysis for each source. This helps you remember what you read and saves time later in the process.

To begin organizing your literature review’s argument and structure, be sure you understand the connections and relationships between the sources you’ve read. Based on your reading and notes, you can look for:

Trends and patterns (in theory, method or results): do certain approaches become more or less popular over time?
Themes: what questions or concepts recur across the literature?
Debates, conflicts and contradictions: where do sources disagree?
Pivotal publications: are there any influential theories or studies that changed the direction of the field?
Gaps: what is missing from the literature? Are there weaknesses that need to be addressed?

This step will help you work out the structure of your literature review and (if applicable) show how your own research will contribute to existing knowledge.

Most research has focused on young women.
There is an increasing interest in the visual aspects of social media.
But there is still a lack of robust research on highly visual platforms like Instagram and Snapchat—this is a gap that you could address in your own research.

There are various approaches to organizing the body of a literature review. Depending on the length of your literature review, you can combine several of these strategies (for example, your overall structure might be thematic, but each theme is discussed chronologically).

Chronological

The simplest approach is to trace the development of the topic over time. However, if you choose this strategy, be careful to avoid simply listing and summarizing sources in order.

Try to analyze patterns, turning points and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred.

If you have found some recurring central themes, you can organize your literature review into subsections that address different aspects of the topic.

For example, if you are reviewing literature about inequalities in migrant health outcomes, key themes might include healthcare policy, language barriers, cultural attitudes, legal status, and economic access.

Methodological

If you draw your sources from different disciplines or fields that use a variety of research methods , you might want to compare the results and conclusions that emerge from different approaches. For example:

Look at what results have emerged in qualitative versus quantitative research
Discuss how the topic has been approached by empirical versus theoretical scholarship
Divide the literature into sociological, historical, and cultural sources

Theoretical

A literature review is often the foundation for a theoretical framework . You can use it to discuss various theories, models, and definitions of key concepts.

You might argue for the relevance of a specific theoretical approach, or combine various theoretical concepts to create a framework for your research.

Like any other academic text , your literature review should have an introduction , a main body, and a conclusion . What you include in each depends on the objective of your literature review.

The introduction should clearly establish the focus and purpose of the literature review.

Depending on the length of your literature review, you might want to divide the body into subsections. You can use a subheading for each theme, time period, or methodological approach.

As you write, you can follow these tips:

Summarize and synthesize: give an overview of the main points of each source and combine them into a coherent whole
Analyze and interpret: don’t just paraphrase other researchers — add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
Critically evaluate: mention the strengths and weaknesses of your sources
Write in well-structured paragraphs: use transition words and topic sentences to draw connections, comparisons and contrasts

In the conclusion, you should summarize the key findings you have taken from the literature and emphasize their significance.

When you’ve finished writing and revising your literature review, don’t forget to proofread thoroughly before submitting. Not a language expert? Check out Scribbr’s professional proofreading services !

This article has been adapted into lecture slides that you can use to teach your students about writing a literature review.

Scribbr slides are free to use, customize, and distribute for educational purposes.

Open Google Slides Download PowerPoint

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

Sampling methods
Simple random sampling
Stratified sampling
Cluster sampling
Likert scales
Reproducibility

Statistics

Null hypothesis
Statistical power
Probability distribution
Effect size
Poisson distribution

Research bias

Optimism bias
Cognitive bias
Implicit bias
Hawthorne effect
Anchoring bias
Explicit bias

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

There are several reasons to conduct a literature review at the beginning of a research project:

To familiarize yourself with the current state of knowledge on your topic
To ensure that you’re not just repeating what others have already done
To identify gaps in knowledge and unresolved problems that your research can address
To develop your theoretical framework and methodology
To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

The literature review usually comes near the beginning of your thesis or dissertation . After the introduction , it grounds your research in a scholarly field and leads directly to your theoretical framework or methodology .

A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other academic texts , with an introduction , a main body, and a conclusion .

An annotated bibliography is a list of source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a paper .

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, September 11). How to Write a Literature Review | Guide, Examples, & Templates. Scribbr. Retrieved September 9, 2024, from https://www.scribbr.com/dissertation/literature-review/

Is this article helpful?

Shona McCombes

Other students also liked, what is a theoretical framework | guide to organizing, what is a research methodology | steps & tips, how to write a research proposal | examples & templates, "i thought ai proofreading was useless but..".

I've been using Scribbr for years now and I know it's a service that won't disappoint. It does a good job spotting mistakes”

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Publications
Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

Advanced Search
Journal List

Systematic review of the effectiveness of training programs in writing for scholarly publication, journal editing, and manuscript peer review (protocol)

James galipeau.

1 Ottawa Hospital Research Institute, 501 Smyth Rd, Ottawa, K1H 8L6, Canada

David Moher

2 Faculty of Medicine, University of Ottawa, 451 Smyth Road, Ottawa, ON, K1H 8M5, Canada

Becky Skidmore

3 Independent Consultant, Ottawa, Canada

Craig Campbell

4 Royal College of Physicians and Surgeons of Canada, 774 Echo Drive, Ottawa, ON, K1S 5N8, Canada

Paul Hendry

D william cameron, paul c hébert, anita palepu.

5 Department of Medicine, Centre for Health Evaluation and Outcome Sciences, University of British Columbia, St Paul’s Hospital, Vancouver, BC, V6Z 1Y6, Canada

An estimated $100 billion is lost to ‘waste’ in biomedical research globally, annually, much of which comes from the poor quality of published research. One area of waste involves bias in reporting research, which compromises the usability of published reports. In response, there has been an upsurge in interest and research in the scientific process of writing, editing, peer reviewing, and publishing (that is, journalology) of biomedical research. One reason for bias in reporting and the problem of unusable reports could be due to authors lacking knowledge or engaging in questionable practices while designing, conducting, or reporting their research. Another might be that the peer review process for journal publication has serious flaws, including possibly being ineffective, and having poorly trained and poorly motivated reviewers. Similarly, many journal editors have limited knowledge related to publication ethics. This can ultimately have a negative impact on the healthcare system. There have been repeated calls for better, more numerous training opportunities in writing for publication, peer review, and publishing. However, little research has taken stock of journalology training opportunities or evaluations of their effectiveness.

We will conduct a systematic review to synthesize studies that evaluate the effectiveness of training programs in journalology. A comprehensive three-phase search approach will be employed to identify evaluations of training opportunities, involving: 1) forward-searching using the Scopus citation database, 2) a search of the MEDLINE In-Process and Non-Indexed Citations, MEDLINE, Embase, ERIC, and PsycINFO databases, as well as the databases of the Cochrane Library, and 3) a grey literature search.

This project aims to provide evidence to help guide the journalological training of authors, peer reviewers, and editors. While there is ample evidence that many members of these groups are not getting the necessary training needed to excel at their respective journalology-related tasks, little is known about the characteristics of existing training opportunities, including their effectiveness. The proposed systematic review will provide evidence regarding the effectiveness of training, therefore giving potential trainees, course designers, and decision-makers evidence to help inform their choices and policies regarding the merits of specific training opportunities or types of training.

An estimated $100 billion is lost to ‘waste’ in biomedical research globally each year, a sizeable portion of which comes from the poor quality of published research. Chalmers and Glasziou identified four areas of waste related to: 1) the relevancy of research questions to clinicians and patients, 2) the appropriateness of research design and methods, 3) making publications fully accessible, and 4) issues of bias and the usability of reports [ 1 ]. In the latter of these categories, the authors explain that over 30% of trial interventions are not sufficiently described, over 50% of planned study outcomes are not reported and most new research is not interpreted in the context of systematic assessment of other relevant evidence [ 2 ]. In response to this, there has been an upsurge in interest and research on topics such as publication ethics, research integrity, and rigor in the scientific process of writing, editing, peer reviewing, and publishing (that is, journalology) of biomedical research. This type of waste has also become a primary focus of organizations such as the Committee on Publication Ethics (COPE), World Association of Medical Editors (WAME), and the Enhancing the Quality and Transparency of Health Research (EQUATOR) Network.

Bias in reporting and the problem of unusable reports can be attributed to shortcomings at both the production and publication phases of the research process. On one hand, some authors lack knowledge or engage in questionable practices while designing, conducting, or reporting their research. On the other hand, the peer review process for both grant giving and journal publication has serious flaws, including claims of being ineffective [ 3 ], as well as having poorly trained and poorly motivated reviewers. Similarly, many journal editors lack formal training [ 4 , 5 ], as well as having poor knowledge related to publication ethics [ 5 ]. While the causes for this type of research waste may be varied, the consequences for decision-makers, knowledge users, and tax-paying healthcare patients are ultimately negative, as indicated by Dickerson and Chalmers in their 2010 report on this topic: ‘Incomplete and biased reporting has resulted in patients suffering and dying unnecessarily [ 6 ]. Reliance on an incomplete evidence base for decision-making can lead to imprecise or incorrect conclusions about an intervention’s effects. Biased reporting of clinical research can result in overestimates of beneficial effects [ 7 ] and suppression of harmful effects of treatments [ 8 ]. Furthermore, planners of new research are unable to benefit from all relevant past research [ 9 ]’.

The lack of formal training appears to be widespread not only among authors of health research, but also among the gatekeepers of health literature - journal peer reviewers and editors, and at earlier stages, grant peer reviewers. This may be one potential reason for the large amount of waste in biomedical research. Murray [ 10 ] suggests that most academics have no formal training in writing for publication and that they developed their skills mainly through a process of trial and error. In addition, the rates of author misconduct [ 11 - 14 ], of which most incidences stem from negligence, poorly performed science, investigator bias, or lack of knowledge, rather than acts of fraud [ 15 ], suggest a need for better training among authors on journalological issues. Meanwhile, Keen [ 16 ] argued that, while there is a wealth of literature describing how to go about writing for publication, the provision of information alone may be insufficient to support potential authors. In addition, Eastwood [ 17 ] suggested that professional training opportunities may be lacking due to a faulty assumption that trainees could not have achieved their postdoctoral status without having acquired an education in critical reading and writing.

In the case of peer reviewers, very little is known about their training and experiences [ 18 ], however, research shows that many reviewers’ needs for training and support are not being met, despite the desire for it among most of them [ 19 ]. Peer reviewers have difficulty identifying major errors in articles submitted for publication [ 20 - 22 ] and in some cases agreement between reviewers of the same manuscript is not much different than would be expected by chance [ 3 ]. There is also evidence to suggest that the quality of one’s peer reviewing deteriorates over time [ 3 , 18 ] and that peer reviewers are susceptible to positive-outcome bias [ 23 ]. Similarly, the peer review process used by granting agencies also appears to be problematic. A survey of 29 international granting agencies indicated that several aspects of their peer review process were poor and had not improved in the preceding 5 years, including difficulty retaining good reviewers, reviewers carrying out poor quality reviews, and reviewers not following guidelines appropriately [ 24 ]. The same survey polled external reviewers of granting agencies (n = 258) and found that only 9% had received some form of training in how to conduct biomedical grant reviews, despite 64% of reviewers expressing an interest in peer review training [ 24 ]. The authors concluded by saying that funding organizations should help reviewers do their job effectively by providing clear guidance and training.

Many journal editors report having informal [ 5 ], or little to no [ 4 ] training in editing skills, as well as being unfamiliar with available guidelines [ 25 ]. However, they also say they would welcome more guidance or training [ 4 , 5 , 25 ]. When tested, editors performed very poorly on knowledge of editorial issues related to authorship, conflict of interest, peer review, and plagiarism [ 5 ]. Many editors also believed that ethical issues occur rarely or never at their journal [ 25 ]. These findings echo the assertion of Paul Hébert, former Editor-in-Chief of the Canadian Medical Association Journal (CMAJ), that ‘we need to train the editors of tomorrow. In Canada, we have a very small scholarly publishing industry. As a consequence, there are few medical editing positions, no obvious career paths and even fewer training opportunities [ 26 ]’.

While the reasons for the poor training of authors, peer reviewers, and editors have not been studied directly, one cause may be a lack of legitimate opportunities to obtain formal training [ 26 ], or a lack of access to these training opportunities. For example, there are currently no certification programs or degrees that allow a physician to train specifically to become a medical journal editor. While courses are offered by a few groups (for example, Council of Science Editors) and Fellowship training programs exist in the USA, Canada, and the UK, the majority of these are 1-year programs, largely require a full-time commitment, and are accessible to only a select few, annually. Some journals, such as the Journal of the American Medical Association (JAMA), the British Medical Journal (BMJ) and American Family Physician (AFP) have created 1- to 2-month electives for medical students to undergo similar training; however, these are only open to medical students. The situation for authors and peer reviewers is not much better. Eastwood [ 17 ] explains that, for medical residents, journal clubs are the primary forum in which to learn about critical appraisal of biomedical literature, however, most clinicians will receive little [ 27 ] to no [ 10 ] formal training in writing for publication. Eastwood also points out that ‘few of the programs developed to meet the National Institutes of Health requirement for training in responsible research practices devote time to the practice and ethics of biomedical reporting [ 17 ]’. Similarly, for peer reviewers, there is little to no formal training available, with most reviewers being guided by journals’ instructions to reviewers sections and being forced to learn by trial-and-error [ 28 ].

There have been repeated calls for better, more numerous training opportunities for research reporting, peer review, and publishing [ 1 , 26 , 29 ]. Although training opportunities appear to be somewhat limited, a small number of them do exist, some being offered by reputable organizations. However, little research has taken stock of the journalology training opportunities or related evaluations of their effectiveness. A systematic review of training opportunities in a related area - overcoming barriers to publication - has been identified [ 30 ]. The review, which included 17 studies published between 1984 and 2004, evaluated the effect of writing courses, writing support groups and writing coaches on author output. While publication rates were found to increase overall, whether or not opportunities exist to enhance the quality of such research output for all relevant players (that is, authors, peer reviewers and editors), is a vastly more relevant question in the age of increasing evidence towards author misconduct and misreporting. We are not aware of an existing synthesis of knowledge on this topic.

The objective of this project is to systematically review, evaluate and synthesize information on whether training in journalology effectively improves educational outcomes, such as measures of knowledge, intention to change behaviour, and measures of excellence in training domains. Collecting this information will allow knowledge users to know what training options are most effective. This will be useful for making training recommendations to authors, peer reviewers, editors, and others. In addition, it will provide a solid foundation from which to develop and build new training courses and programs for these groups, ultimately improving knowledge and the quality of research practices both within Canada and abroad.

Criteria for considering studies for this review

Those centrally involved in writing for scholarly publication, journal editing, and manuscript peer review (that is, authors, peer reviewers, journal editors), or any other group that may be peripherally involved in the scientific writing and publishing process, such as medical journalists.

Intervention

Evaluations of training in any specialty or subspecialty of writing for scholarly publication, journal editing, or manuscript peer review targeted at the designated population(s) will be included.

The following comparisons will be included: 1) before and after administration of a training class/course/program of interest, 2) between two or more training classes/courses/programs of interest, or 3) between a training class/course/program and any other intervention(s) (including no intervention).

The primary outcomes will be any measure of effectiveness of training as reported, including, but not limited to: measures of knowledge, intention to change behaviour, measures of excellence in training domains (writing, peer review, editing), however reported. Since this review is largely exploratory, where other meaningful outcomes are reported, this information will be collected as well.

Study design(s)

Comparative studies evaluating at least one training program/course/class of interest will be included in this review and henceforth be termed ‘evaluations’.

Search methods for identification of studies

A comprehensive three-phase search approach will be employed to identify evaluations of training opportunities, as follows: 1) For training which has been described in published reports, citations of these reports will be forward-searched using the Scopus citation database. 2) We will also perform a search of MEDLINE In-Process and Non-Indexed Citations, MEDLINE, Embase, ERIC, PsycINFO, and the databases of the Cochrane Library. A specific search strategy will be developed by an information specialist and will be peer reviewed prior to execution [ 31 ]. There will be no language restrictions on the search strategy, however, due to the large expected yield of the planned review and limited resources available, evaluations encountered in languages other than English and French will be set aside and listed in an appendix in the report. Letters, commentaries and editorials will not be excluded due to the possibility that they may contain reference to evaluations of particular training programs. Studies will not be excluded based on publication status. 3) A grey literature search will also be conducted, consisting of screening the results of a concurrent project to map all existing and previous training in journalology through an ‘environmental scanning’ technique [ 32 , 33 ] using the Google search engine. The administrators of any relevant training opportunities identified in the environmental scan will be contacted to inquire whether they are aware of any published or unpublished evaluations of the training opportunities.

Data collection

Following the execution of the search strategy, the identified records (titles and/or available abstracts) will be collated in a Reference Manager [ 34 ] database for de-duplication. The final unique record set and full-text of potentially eligible studies will be exported to an Internet-based software, DistillerSR (Evidence Partners, Ottawa, Canada), through which screening of records and extraction of data from included evaluations will be carried out.

Study selection

Given the broad/general nature of many of the search terms (for example, author, editor, education) we expect a large volume of initial search results. Therefore, we will conduct an initial screening of the titles only by two reviewers, using the liberal acceleration method (that is, one reviewer screens all identified studies and a second reviewer screens only excluded studies, independently). Following the title screen, titles and abstracts of identified records will be screened by two reviewers, again using the liberal accelerated method. Subsequently, the full-text of all potentially eligible evaluations will be retrieved and reviewed for eligibility, independently, by two members of the team using a priori eligibility criteria. To be included, evaluations must include one of the a priori comparison groups and examine the influence of training, as reported, using any measure. Disagreements between reviewers at this stage will be resolved by consensus or by a third member of the research team. If necessary, authors of potentially included evaluations will be contacted to clarify data needed for eligibility.

Data extraction

Separate data extraction forms will be developed to capture information needed for synthesis for each of the three comparisons and will be piloted using a subset of included evaluations and modified, as needed. One reviewer will extract general study characteristics of included evaluations, with verification carried out by a second reviewer. Data on measures of effectiveness of training for each program/course/class will be extracted by one reviewer; a second reviewer will verify the accuracy of the data from a random 20% sample of included evaluations. Any discrepancies between reviewers will be resolved by consensus or by a third member of the research team. If there is greater than a 50% discrepancy between reviewers’ answers in the random 20% sample, or only a small number of included studies, 100% data verification will be considered. Authors of included evaluations will be contacted to invite contribution of any unpublished data needed for this review that is not available in published reports.

General publication characteristics to be extracted include: first author name and contact information (of first or corresponding author), year of publication, institutional affiliation of first author, country, language of publication, type of document (full text versus abstract), and funding source. Other details to be collected include: name of training class(es)/course(s)/program(s) being evaluated (if applicable), population evaluated, sample size, whether prospective or retrospective, and mechanism of sampling (or participant assignment to groups). Extracted outcome data will include: tool(s) used to evaluate effectiveness of training, timing of measurement, effectiveness measurement (however reported), intention to change behaviour scores, and measures of excellence data (however reported).

Assessment of validity of evaluations

No tool currently exists to assess the validity (internal and external) of evaluations in methodological reviews such as this. Study designs are expected to be largely heterogeneous, however, if evaluations using randomized control trial (RCT) designs are encountered, the Cochrane risk of bias tool will be used to judge validity [ 35 ]. To assess all other evaluations, we propose assessing the following criterion with a rating of ‘yes’, ‘no’ or ‘unclear’, to help readers make their own judgments about the overall validity of the included evidence. We have used this approach elsewhere [ 36 , 37 ].

1. Whether an objective measure of training effectiveness was employed (that is, a priori questionnaires).

2. Whether the measurement tool to evaluate training effectiveness was reported to be validated.

3. Whether intended methods align with reported findings.

4. Whether data from all included participants was reported.

5. Whether sampling for comparison 2. and 3. occurred within the same time frame.

6. Whether comparison groups represent similar populations (that is, same area of health-related discipline, similar levels of training).

Data analysis

Measures of effect.

Due to the paucity of literature describing formal training opportunities in journalology, we are unable to anticipate the types of measurement tools that might be used for their evaluation. Where data is provided narratively, it will be collected as such. Where summary scores of outcomes (that is, participant knowledge using a particular tool) are presented within each evaluation, we will collect means and standard deviations (SDs). When medians and ranges are reported instead of means and SDs, suitable approximations will be used, as discussed in the Cochrane Handbook [ 38 ]. A standardized mean difference (SMD) and 99% confidence interval will be calculated for each study; a SMD >0 will indicate better overall training effectiveness. Where proportions of participants in each comparison group are reported for a particular outcome in an evaluation, this information will be collected. A relative risk (RR) and 99% confidence intervals will be calculated for each study. A RR >1.0 will indicate a higher proportion of participants with positive outcomes. Confidence levels of 99% will ensure conservative estimates of precision are obtained. If reporting and sample size allow, standard methods - depending on the approximate distribution of the data - will be used to transform medians and interquartile ranges (IQRs) to mean difference (MD) and SD, to allow visual inspection of estimates of effect. Where possible, these estimates will be included in SMD calculations for overall training effect.

Dealing with missing data

Corresponding authors of potentially included evaluations will be contacted, up to two times, where data are needed. If the data are not obtained and compromise the ability to include the evaluation in quantitative synthesis, it will be excluded from quantitative synthesis.

Data synthesis

General study characteristics will be presented in tabular format. Due to the anticipated methodological heterogeneity of potentially included evaluations (based on previous experience carrying out methodological systematic reviews), it is unlikely that data will be combined across evaluations. If this is the case, data will be described qualitatively in the text of the review. However, we will first assess for the suitability of meta-analysis, which will depend on the quantity of data and the homogeneity of studies according to methodology and content. If meta-analysis is possible, analyses will be conducted with the random effects model using the Review Manager software [ 39 ]. Any follow-up time for outcomes will be considered relevant, but only similar time points will be meta-analyzed; ‘similarity’ will need to be determined post hoc once study data are collected during the data extraction phase. Initially, all training programs will be considered together.

Subgroup analysis

If relevant data are reported and permit quantitative synthesis, the following subgroup analyses are planned: 1) modes of training delivery (that is, online, face-to-face), 2) primary role (that is, author, peer review, editor), 3) primary occupation of participant (that is, student (including medical residents), health practitioner, other health-related professional), 3) duration of training (that is, class, course, program), 4) credibility via institutional affiliation (that is, sponsored by academic institution, publishing house, industry, other), 5) setting (that is, individual versus group), or 6) associated cost (that is, cost versus no cost).

Assessment of heterogeneity

If there is any quantitatively aggregated data across included studies, we plan to measure the inconsistency of study results using the I 2 heterogeneity statistic to determine the extent of variation in effect estimates that is due to heterogeneity rather than chance [ 38 ]. Heterogeneity will be determined by visual inspection of the forest plot and I 2 statistics. For the interpretation of I 2 , a rough guide of low (0% to 25%), moderate (25% to 50%), substantial (50% to 75%), and considerable (75% to 100%) heterogeneity will be used [ 38 ]. Where considerable statistical heterogeneity exists (≥75%) data will not be pooled. Possible reasons for heterogeneity will be explored in sensitivity analyses; the pre-specified subgroup analyses, if feasible, will be examined to determine whether they provide possible reasons for any observed statistical heterogeneity. The variables outlined above for subgroup analyses will be considered statistically significant at P <0.01.

Reporting biases

Asymmetry of funnel plots is an established method for assessing the potential presence of publication bias, and others biases, in traditional systematic reviews of intervention effectiveness, subject to a sufficient number of included studies [ 38 , 40 ]. We will generate funnel plots and graphically explore the presence of asymmetry. If warranted we will evaluate asymmetry using statistical tools [ 40 ].

This project aims to provide evidence to help guide the journalological training of authors, peer reviewers, and editors, and the development of future training opportunities in this domain. While there is ample evidence that many members of these groups are not getting the necessary training needed to excel at their respective journalology-related tasks, little is known about the characteristics of existing training opportunities, including their effectiveness. The proposed systematic review will provide the evidence regarding the effectiveness of training, therefore giving potential trainees, course designers, and decision-makers evidence to help inform their choices and policies regarding the merits of a specific training opportunity or type of training.

We believe that the results of the proposed review will be of relevance to a wide variety of knowledge users, namely: authors, peer reviewers, and editors, as well as designers and teachers of training courses related to journalology, and decision-makers for continuing medical education (CME) and continuing professional development (CPD). Consumers of training (that is, potential trainees) will benefit by learning which types of training provide the most effective learning outcomes. This will empower them to make more informed choices regarding specific training, rather than making decisions based on word-of-mouth recommendations, academic affiliation, and other such unreliable methods. The designers of training will benefit by gaining access to a knowledge synthesis that outlines the characteristics of effective learning structures and environments. This will enable them to design better learning strategies and a better curriculum that takes into consideration the particularities of education in journalology. Finally, decision-makers will benefit by gaining an understanding of what works when choosing the type of training that will best benefit their organizations.

This review will provide the knowledge that is necessary for better educating authors, peer reviewers, and editors on how to reduce biomedical research waste by improving the quality and rigor in research reporting. Ultimately, the goal is to move closer to optimal reporting of health research, so that we can have full access to, and use of, the new knowledge coming from our investments in research.

Abbreviations

AFP: American Family Physician; BMJ: British Medical Journal; CMAJ: Canadian Medical Association Journal; CME: Continuing medical education; COPE: Committee on Publication Ethics; CPD: Continuing professional development; EQUATOR: Enhancing the Quality and Transparency of Health Research; IQR: Interquartile range; JAMA: Journal of the American Medical Association; MD: Mean difference; RCT: Randomized controlled trial; RR: Relative risk; SD: Standard deviation; SMD: Standardized mean difference; WAME: World Association of Medical Editors.

Competing interests

The authors declare that they have no competing interests.

Authors’ contributions

JG drafted the manuscript; DM conceived of the study, participated in its design and coordination, and helped draft the manuscript; BS designed the search strategy; and CC, PHendry, DWC, PHébert, and AP all provided content expertise and participated in the design of the study. All authors read and approved the final manuscript.

Acknowledgements

This research project is funded by the Canadian Institutes of Health Research (number: 278874). The funder has no role in the design, collection, analysis, and interpretation of the data; in the writing of the manuscript; or in the decision to submit the manuscript for publication.

Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence. Obstet Gynecol. 2009; 114 (6):1341. doi: 10.1097/AOG.0b013e3181c3020d. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Glasziou P, Meats E, Heneghan C, Shepperd S. What is missing from descriptions of treatment in trials and reviews? Br Med J. 2008; 336 (7659):1472. doi: 10.1136/bmj.39590.732037.47. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
Callaham ML. Proceedings of the Sixth International Congress on Peer Review and Biomedical Publication. Vancouver: International Congress on Peer Review and Biomedical Publication; 2009. The natural history of peer reviewers: The decay of quality. [ Google Scholar ]
Garrow J, Butterfield M, Marshall J, Williamson A. The reported training and experience of editors in chief of specialist clinical medical journals. JAMA. 1998; 280 (3):286–287. doi: 10.1001/jama.280.3.286. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Wong VS, Callaham ML. Medical journal editors lacked familiarity with scientific publication issues despite training and regular exposure. J Clin Epidemiol. 2012; 65 (3):247–252. doi: 10.1016/j.jclinepi.2011.08.003. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Cowley AJ, Skene A, Stainer K, Hampton JR. The effect of lorcainide on arrhythmias and survival in patients with acute myocardial infarction: an example of publication bias. Int J Cardiol. 1993; 40 (2):161–166. doi: 10.1016/0167-5273(93)90279-P. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Sterne JAC, Egger M, Moher D. In: Cochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series. Higgins JP, Green S, editor. Chichester: John Wiley & Sons; 2008. Addressing reporting biases; pp. 297–333. [ Google Scholar ]
Dealing with biased reporting of the available evidence. The James Lind Library. [ http://www.jameslindlibrary.org/essays/interpretation/relevant_evidence/dealing-with-biased-reporting-of-the-available-evidence.html ]
Dickersin K, Chalmers I. Recognizing, investigating and dealing with incomplete and biased reporting of clinical research: from Francis Bacon to the WHO. JRSM. 2011; 12 :532–538. [ PMC free article ] [ PubMed ] [ Google Scholar ]
Murray R, Newton M. Facilitating writing for publication. Physiotherapy. 2008; 94 :29–34. doi: 10.1016/j.physio.2007.06.004. [ CrossRef ] [ Google Scholar ]
Rennie D. In: Peer Review in Health Sciences. Godlee F, Jefferson T, editor. London: BMJ; 2003. Editorial peer review: its development and rationale; pp. 1–13. [ Google Scholar ]
Tavare A. Managing research misconduct: is anyone getting it right? BMJ. 2011; 34 :d8212. [ PubMed ] [ Google Scholar ]
Wager E. Coping with scientific misconduct. BMJ. 2011; 343 :d6586. doi: 10.1136/bmj.d6586. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Brice J, Bligh J. Author misconduct: not just the editors’ responsibility. Med Educ. 2005; 39 (1):83–89. doi: 10.1111/j.1365-2929.2004.02027.x. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Marusic A. Author misconduct: editors as educators of research integrity. Med Educ. 2005; 39 (1):7–8. doi: 10.1111/j.1365-2929.2004.02050.x. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Keen A. Writing for publication: pressures, barriers and support strategies. Nurse Educ Today. 2007; 27 (5):382–388. doi: 10.1016/j.nedt.2006.05.019. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Eastwood S, Derish PA, Berger MS. Biomedical publication for neurosurgery residents: a program and guide. Neurosurgery. 2000; 47 (3):739–748. discussion 748–9. [ PubMed ] [ Google Scholar ]
Callaham ML, Tercier J. The relationship of previous training and experience of journal peer reviewers to subsequent review quality. PLoS Med. 2007; 4 (1):e40. doi: 10.1371/journal.pmed.0040040. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
Freda MC, Kearney MH, Baggs JG, Broome ME, Dougherty M. Peer reviewer training and editor support: results from an international survey of nursing peer reviewers. J Prof Nurs. 2009; 25 (2):101–108. doi: 10.1016/j.profnurs.2008.08.007. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Baxt WG, Waeckerle JF, Berlin JA, Callaham ML. Who reviews the reviewers? Feasibility of using a fictitious manuscript to evaluate peer reviewer performance. Ann Emerg Med. 1998; 32 (3):310–317. doi: 10.1016/S0196-0644(98)70006-X. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Schroter S, Black N, Evans S, Carpenter J, Godlee F, Smith R. Effects of training on quality of peer review: randomised controlled trial. BMJ. 2004; 328 (7441):673. doi: 10.1136/bmj.38023.700775.AE. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
van Rooyen S, Godlee F, Evans S, Smith R, Black N. Effect of blinding and unmasking on the quality of peer review: a randomized trial. JAMA. 1998; 280 (3):234–237. doi: 10.1001/jama.280.3.234. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Emerson GB, Warme WJ, Wolf FM, Heckman JD, Brand RA, Leopold SS. Testing for the presence of positive-outcome bias in peer review: a randomized controlled trial. Arch Intern Med. 2010; 170 (21):1934–1939. doi: 10.1001/archinternmed.2010.406. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Schroter S, Groves T, Hojgaard L. Surveys of current status in biomedical science grant review: funding organisations’ and grant reviewers’ perspectives. BMC Med. 2010; 8 :62. doi: 10.1186/1741-7015-8-62. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
Wager E, Fiack S, Graf C, Robinson A, Rowlands I. Science journal editors’ views on publication ethics: results of an international survey. J Med Ethics. 2009; 35 (6):348–353. doi: 10.1136/jme.2008.028324. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Hebert PC. Even an editor needs an editor: reflections after five years at CMAJ. CMAJ. 2011; 183 (17):1951. [ PMC free article ] [ PubMed ] [ Google Scholar ]
Pololi L, Knight S, Dunn K. Facilitating scholarly writing in academic medicine. J Gen Intern Med. 2004; 19 (1):64–68. doi: 10.1111/j.1525-1497.2004.21143.x. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
Lu Y. Learning to be confident and capable journal reviewers: an Australian perspective. Learned Publishing. 2012; 25 (1):56–61. doi: 10.1087/20120108. [ CrossRef ] [ Google Scholar ]
Benos DJ, Bashari E, Chaves JM, Gaggar A, Kapoor N, LaFrance M, Mans R, Mayhew D, McGowan S, Polter A, Qadri Y, Sarfare S, Schultz K, Splittgerber R, Stephenson J, Tower C, Walton RG, Zotov A. The ups and downs of peer review. Adv Physiol Educ. 2007; 31 (2):145–152. doi: 10.1152/advan.00104.2006. [ PubMed ] [ CrossRef ] [ Google Scholar ]
McGrail MR, Rickard CM, Jones R. Publish or perish: a systematic review of interventions to increase academic publication rates. High Educ Res Dev. 2006; 25 (1):19–35. doi: 10.1080/07294360500453053. [ CrossRef ] [ Google Scholar ]
Sampson M, McGowan J, Cogo E, Grimshaw J, Moher D, Lefebvre C. An evidence-based practice guideline for the peer review of electronic search strategies. J Clin Epidemiol. 2009; 62 (9):944–952. doi: 10.1016/j.jclinepi.2008.10.012. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Brown A, Weiner E. Supermanaging: How to Harness Change for Personal and Organizational Success. New York: McGraw-Hill; 1985. [ Google Scholar ]
Porterfield D, Hinnant L, Kane H, Horne J, McAleer K, Roussel A. Linkages between clinical practices and community organizations for prevention: a literature review and environmental scan. Am J Preventive Medicine. 2012; 42 (6):S163–S171. doi: 10.1016/j.amepre.2012.03.018. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Reuters T. Reference Manager. New York: Thomson Reuters; 2008. [ Google Scholar ]
Higgins JPT, Altman DG. In: Cochrane Handbook for Systematic Reviews of Interventions. Version 5.0.2. Higgins JPT, Green S, editor. Chichester: John Wiley & Sons; 2008. Assessing risk of bias in included studies; pp. 187–242. [ Google Scholar ]
Turner L, Moher D, Shamseer L, Weeks L, Peters J, Plint A, Altman DG, Schulz KF. The influence of CONSORT on the quality of reporting of randomised controlled trials: an updated review. Trials. 2011; 12 (Suppl 1):A47. doi: 10.1186/1745-6215-12-S1-A47. [ CrossRef ] [ Google Scholar ]
Shamseer L, Stevens A, Skidmore B, Turner L, Altman DG, Hirst A, Hoey J, Palepu A, Simera I, Schulz K, Moher D. Does journal endorsement of reporting guidelines influence the completeness of reporting of health research? A systematic review protocol. Systematic Reviews. 2012; 1 :24. doi: 10.1186/2046-4053-1-24. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
Deeks JJ, Higgins JPT, Altman DG. In: Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1.0. Higgins JPT, Green S, editor. Chichester: John Wiley & Sons; 2011. Analysing data and undertaking meta-analyses. [ Google Scholar ]
The Cochrane Collaboration. Review Manager (RevMan) [Computer program]. Version 5.1. Copenhagen: The Nordic Cochrane Centre; 2011. [ Google Scholar ]
Sterne JAC, Sutton AJ, Ioannidis JPA, Terrin N, Jones DR, Lau J, Carpenter J, Rücker G, Harbord RM, Schmid CH, Tetzlaff J, Deeks JJ, Peters J, Macaskill P, Schwarzer G, Duval S, Altman DG, Moher D, Higgins JP. Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials. BMJ. 2011; 343 (7818):302. [ PubMed ] [ Google Scholar ]

Training design in mediating the relationship of participants’ motivation, work environment, and transfer of learning

European Journal of Training and Development

ISSN : 2046-9012

Article publication date: 29 December 2022

Issue publication date: 18 December 2023

The purpose of this paper is to examine the relationship among training design, trainee motivation and work environment on the transfer of learning for teachers enrolled in a continuing professional education (CPE) training program and the confirmation of potential positive, predictive relationships of trainee motivation, work environment and training design to transfer of learning. This study investigated the contribution of training efficiency and relevance as measured by the training design; work environment as measured by work autonomy, work complexity and work variability; and trainee’s motivation of training (learning- and job-oriented) to the transfer of knowledge and skills from the training program to their workplace. Both direct and indirect effects of mentioned components on the learning transfer were explored.

Design/methodology/approach

This study included 160 teachers working in high-needs schools with large numbers of English learners (ELs) Southwest USA. Teachers in this study primarily needed professional development to empower them and enhance their instructional capacity for ELs and economically challenged students. During the recruitment, participants completed a demographic information (e.g. gender, ethnicity, number of years teaching, age, educational background) survey.

A mediation model with training design as the mediating factor was developed and analyzed. The results revealed that training design fully mediated the relationship between trainees’ work environments and the transfer of knowledge, skills and attitude acquired from the training to their workplace. Furthermore, it partially mediated the relationship between learning-oriented motivation and the transfer of learning. These findings further amplify the significance of CPE program training design and foster important considerations for future research regarding the isolation of specific training design aspects that significantly contribute to the mediation of these relationships.

Research limitations/implications

Considering the significance of learning transfer in developing professional knowledge and skills for target employees and trainees, confirming the mediating effects of training design on training transfer holds critical implications for future research. Specific and purposeful attention needs to be given to the design of CPE training. Investigations into the effects of training design and successful elements such as the training platform (online, hybrid or in-person), sample size, group structure, facilitation and participant demographics are warranted.

Practical implications

The finding of this research provides a preliminary guide for scholar-practitioners. Results of the study confirmed the role that learning-oriented motivation, job-oriented motivation, work variability or flexibility, work complexity and training design play in transfer of learning. In practice, training professionals will be more comfortable pinpointing the factors that lead to the transfer of learning or the lack of it.

Originality/value

Learning transfer has been found to be imperative for target employees and trainees to develop professional knowledge, skills and attitudes. Results of this study reveal variables that promote the positive transfer of learning to the workplace.

Training design

Training effectiveness

Work environment

Work flexibility

Transfer of learning

Learning motivation

Nafukho, F.M. , Irby, B.J. , Pashmforoosh, R. , Lara-Alecio, R. , Tong, F. , Lockhart, M.E. , El Mansour, W. , Tang, S. , Etchells, M. and Wang, Z. (2023), "Training design in mediating the relationship of participants’ motivation, work environment, and transfer of learning", European Journal of Training and Development , Vol. 47 No. 10, pp. 112-132. https://doi.org/10.1108/EJTD-06-2022-0070

Emerald Publishing Limited

Published by Emerald Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial & non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence may be seen at http://creativecommons.org/licences/by/4.0/legalcode

Introduction

For all educators to be successful in their work, learning how to learn and the need to commit to learning for lifetime is not only a sufficient condition but, a necessary condition hence the importance of continuing professional education (CPE). There is an urgent need to understand how effective CPE transfers to improved student learning and teaching practices as influenced by national reform and current research on learning ( Gilbert, 2020 ). In recent years, there is a special attention paid to teachers’ CPE. Thus, learning and development in the workplace is now an important strategy used by organizations to reskill and retool their employees to achieve the set mission and priority goals ( Garavan et al. , 2021 ). In the case of the field of education, the importance of the quality of teaching practice for the overall learning of students has also raised the value of CPE especially in the USA. To ensure the success of schools’ mission, which is aimed at providing meaningful and powerful learning to students, teachers must be offered high-quality learning opportunities through a well-designed, a well-structured and well facilitated CPE programs ( Tannehill et al. , 2021 ). Every Student Succeeds Act (2015) emphasized an ongoing commitment to improving teacher effectiveness by using methods that strengthen school programs and meet all children’s needs. Avalos (2011) in her review of the literature published between the years 2000 and 2010 explored the topic of professional development (PD) of teachers. It became evident from the review that there are different types and forms of CPE. In this study, it was recognized that teachers’ CPE is a complex process that is affected by various factors. Avalos (2011) proposed that CPE is about “teachers learning, learning how to learn, and transforming their knowledge into practice for the benefit of their students’ growth” (p. 10). Thus, perceptions on teachers’ CPE have shifted from merely attending courses and training to a lifelong learning journey ( Fraser et al. , 2007 ). Fraser et al. (2007) presented CPE as “an ongoing process of reflection and review that articulates with development planning that meets corporate, departmental and individual needs” (p. 156). Moreover, the learning that takes place within CPE programs is considered “a process of self-development leading to personal growth as well as development of skills and knowledge that facilitates the education of young people” (p. 156).

Numerous researchers agree on the importance of transfer of learning for the success of CPE programs ( Nafukho et al. , 2017 ; Avalos, 2011 ; Daley and Cervero, 2016 ; Fraser et al. , 2007 ; Webster-Wright, 2009 ). Noe (2020) defines transfer of learning as “trainees effectively and continually applying what they have learned in training to their jobs” (p. 160). The topic of transfer of learning has occupied the minds of researchers in education and psychology for decades. The literature leaned toward the view that learning transfer is a dynamic process that is governed by complexity. Transfer of learning definition has taken different shapes throughout the years ( Blume et al. , 2010 ).

The previous researchers showed multiple factors affecting transfer of learning. Some of the variables that were identified in the literature include degree of mastery of the original content, time spent on learning, amount of practice and the design of the learning activity ( Galoyan and Betts, 2021 ). Donovan and Darcy (2011) surveyed human resource development (HRD) practitioners in Ireland to determine the factors that affect transfer of learning. The study yielded factors such as training design, trainees’ motivation, organizational and peers’ support, etc. These factors are some of the factors that the researchers deemed relevant and important in the eyes of HRD practitioners to achieve transfer of learning. In addition, individual factors are critical for learning transfer, such as training design, which moderated the relationship of learner readiness and motivation to transfer ( Dreer et al. , 2017 ).

Problem statement

There is extensive research on transfer of training in the past three decades that enriched our understanding of the concept. However, the gap between practice and research when it comes to transfer of learning is still significant. Practitioners are still not able to apply the findings of research to their practice. Training providers are calling for research that will inform the design and execution phase of training initiative that would eventually lead to transfer of learning ( Baldwin et al. , 2017 ). Banks et al. (2016) conducted a study on the science–practice gap. They found that there is a need for dialogue and collaboration between researchers and practitioners. Research has uncovered that the majority of organizational leaders are not satisfied with the results of training and development efforts pursued by their organizations ( Beer et al. , 2016 ). Baldwin et al. (2017) call for the collaboration of researchers and practitioners in defining the pressing issues in learning transfer. This paper is an answer to that calling. The authors of this paper recognize the lack of evidence-based research on the factors that affect transfer of learning in CPE programs. The research at hand investigates the most prevalent factors in the literature that affect transfer of learning. The results of the study should help researchers, as well as practitioners, understand factors that affect transfer of learning in CPE programs and get them a step closer to designing and executing effective CPE programs.

Purpose of the study and research questions

Is there a relationship between trainee motivation, work environment, training design and transfer of learning?

Is the positive, predictive relationships of trainee motivation, work environment and training design to transfer of learning confirmed?

Does training design mediate the effects of trainee motivations (learning- and job-oriented) and work environment (work variability, work autonomy and work complexity) on the transfer of learning occurring from the training?

Review of literature

effective generalization of learned materials to the job environment; and

maintenance of applying learned content over a period of time ( Baldwin and Ford, 1988 ; Baldwin et al. , 2009 ; Burke and Hutchins, 2007 ).

Considering the significance of training transfer for target employees and trainees to develop professional knowledge and skills, extensive research has been conducted to identify key components of promoting the positive transfer of training.

to investigate their contributions to the transfer of training; and

to suggest future research directions.

For instance, most research identified training objectives and content were two significant training design factors. Additionally, a cross-sectional qualitative study ( Iqbal and Alsheikh, 2018 ) examined the factors preventing or assisting the transfer of training to the workplace. The results from the interviews with program developers and faculty trainers showed that transfer of training to instructional practices is influenced by mainly three factors, including “trainee characteristics, training design features, and environmental factors” ( Iqbal and Alsheikh, 2018 , p. 3292). Both cognitive ability and self-efficacy were trainee characteristics strongly associated with training transfer.

In a study conducted by Dixit and Sinha (2022) , the researchers were able to identify tools and techniques that promoted transfer of learning. Training design through its efficiency and effectiveness had a strong correlation with transfer of learning. In addition, the work environment in the form of organizational support had an influence on transfer of learning. Motivation was at the forefront of the factors studied by Dixit and Sinha (2022) . Learning motivation had a high impact on transfer of learning.

content design ( Burke and Hutchins, 2007 ; Seeg et al. , 2021 ; Lim and Morris, 2006 ; Russ-Eft, 2002 ; Velada et al. , 2007 ; Yunus and Yasin, 2014 ); and

instruction design ( Fauth and González-Martínez, 2021 ; Burke and Hutchins, 2007 ; Taylor et al. , 2005 ; Tonhauser and Buker, 2016 ; Yunus and Yasin, 2014 ).

Content design refers to the relevance of training content, including training perspectives, materials and practices ( Burke and Hutchins, 2007 ). According to Lim and Morris (2006) , the training content should be generally relevant to the transfer task as well as the job context. Over the past decade, an increasing amount of research has been conducted to investigate the contribution of content relevance to the training transfer process. For instance, Velada et al. (2007) examined the roles of training design, trainee characteristics and work environment in promoting the transfer of learning among employees in a large grocery organization. Specifically, Nafukho et al. (2017) defined training design as both training effectiveness and relevance. The results from hierarchical regressions demonstrated that training design was a significant predictor for the transfer of learning. Yunus and Yasin (2014) investigated critical constructs, including trainee characteristics, training design and work environment, that influence the training transfer process through face-to-face interviews. The training design construct consisted of six components, including personal capacity to transfer training, training content, opportunities of applying knowledge, transfer design, training curriculum and transfer effort performance. Qualitative findings revealed that providing training content similar to trainees’ working environment was highlighted by participants. The findings indicated that relevant content was significant to promote the ability of training transfer.

Instruction design refers to the application of instructional strategies and practices during the training. Designing and providing effective training experiences were significantly associated with the training quality ( Burke and Hutchins, 2007 ; Tonhauser and Buker, 2016 ). For instance, the review by Burke and Hutchins (2007) examined the impacts of training design, trainee characteristics and work environment on the transfer of learning. Nafukho et al. (2017) identified that, in addition to the content relevance, specific training strategies such as practice and feedback, behavioral modeling and error-based examples were strongly associated with the training transfer. An updated literature review by Tonhauser and Buker (2016) further identified effective training strategies and practices for the positive transfer of learning such as instruction of error management ( Heimbeck et al. , 2003 ), real-word examples and practice-oriented tasks ( Seidel, 2012 ). On the other hand, literature indicates that there are a number of different factors that can affect trainees’ application of their learning to the workplace ( Burke and Hutchins, 2007 ; Sitzmann and Weinhardt, 2019 ). Therefore, it is essential to investigate all such factors that can leverage the transfer of training to the job ( Kodwani and Prashar, 2021 ).

Training design that links learning with individual performance provides high transfer of learning among individuals. The design and the delivery of the training proved to increase the likelihood of trainees to apply what they learned on the job ( Muduli and Raval, 2018 ). Fauth and González-Martínez (2021) studied the effect of instruction design on transfer of learning among teachers participating in continuous online training. The use of a transfer-oriented design led to higher transfer of learning. Teachers responded positively to this type of design and were able to practice what they have learned in their classrooms.

Seeg et al. (2021) conducted a longitudinal study on leadership training and transfer of learning. The purpose of their study was to examine the factors that lead to transfer of learning. The authors found a positive and significant impact of training design on learning, transfer motivation and transfer opportunity. Managers in a large public company in India were surveyed to determine the factors that affect transfer of learning in management training. The results of the study showed a significant positive impact of training design on managers applying the skills they acquired from training programs ( Yaqub et al. , 2021 ).

Trainee’s motivation

Trainees’ motivation has been considered as one of the factors that influence transfer of learning. The skills, knowledge and abilities acquired through training will not be applied to work if the motivation is not there ( Gegenfurtner, 2009 ). Kiwanuka et al. (2020) in their study of transfer of learning among farmers in Uganda were able to pinpoint trainees’ characteristics that contribute to the transfer of agronomical training. The study showed a significant influence of both trainees’ motivation and trainees’ self-efficacy on training transfer. The motivation to transfer improved the likelihood of transfer of training among the studied group. In addition, the trainees who displayed a high confidence in their ability to transfer training were able to apply the knowledge to their practice. Kodwani and Prashar (2021) highlighted the importance of voluntary enrollment in training for transfer of learning. The researchers found a significantly positive influence of voluntary enrollment in the training on the transfer of training. Moreover, the researchers confirmed that trainees with high motivation tend to transfer the learning to their practice ( Kodwani and Prashar, 2021 ).

significant impacts of trainee’s motivation on the transfer of learning; and

the mediating effect of motivation on the relationship between other factors and the training transfer.

Quratulain et al. (2021) examined the effects of organizational, individual and training-related factors on the training transfer process in public organizations. They identified that training motivation was a strong predictor of training transfer, and it partially mediated the relationships of training transfer with supervisor support and self-efficacy. Furthermore, Ismail et al. (2015) investigated the relationship between training administration, training motivation and training transfer in a training program for employees at a military-oriented health organization. The results from the path analyses indicated that training motivation was a significant component in predicting the training transfer. Additionally, Ismail et al. (2015) reported the mediation effect of the training motivation on the relationship between training administration and the transfer of training.

The work environment influences the transfer of training both directly and indirectly ( Govaerts et al. , 2018 ; Hawley and Barnard, 2005 ; Tracey and Tews, 2005 ). The relationship between the work environment and training transfer has been investigated with specific factors, including organizational support, supervisor support and peer support ( Chiaburu, 2010 ; Govaerts et al. , 2018 ; Hua, 2013 ; Na-nan et al. , 2017 ). The findings from earlier studies revealed a supportive work environment would promote trainee’s self-efficacy, motivation and the transfer of learned knowledge and skills ( Ismail et al. , 2015 ).

According to Na-nan et al. (2017) , organizational support is offered based on organization culture, management, system and policy. Thus, a supportive organization provides employees with opportunities for PD and knowledge/skills application. Additionally, Cromwell and Kolb (2004) conducted a quantitative research study to explore the effects of work environment support factors such as organization support, supervisor support and peer support on training transfer. The findings indicated that work environment factors significantly influence the transfer of training. Organizational management was found to be a significant component in promoting training transfer. In another study, Daffron and North (2006) examined how training and work environment contributed to knowledge transfer in the corporate setting. Specifically, they explored the effects of the components of training preparation, training delivery, work environment, organizational support and peer support on the likelihood of training transfer. The results from qualitative data revealed that organizational support, combined with training preparation and transfer, was a significant component for the training transfer.

Moreover, support from supervisors was also considered a significant work environment factor for training transfer. According to Chen et al. (2006) , supervisors and managers may provide various means of support such as accessibility, facilitating the training transfer, addressing the needs of employees, setting goals for training transfer and modeling of solving problems. Dermol and Čater (2013) proposed a model regarding the relationship between training transfer factors, supervisor support, peer support and training quality. The results from the structure equation modeling revealed that supervisor support was significantly associated with both training quality and training transfer. The findings indicated that the supervisor support was a significant component in predicting knowledge learning and application among trainees. Schindler and Burkholder (2016) conducted a mixed-design research study to explore the impact of supervisor support on the transfer of training. Four dimensions of supervisor support were investigated: mentoring, coaching, social support and task support. The findings from both quantitative and qualitative data demonstrated that all four dimensions of supervisor support facilitated the training transfer process.

Peer support was defined as building and developing the network of employees ( Na-nan et al. , 2017 ). Specifically, establishing a peer network provides opportunities of discussing and sharing training knowledge and experiences. Chiaburu (2010) examined the contributions of organization support, supervisor support and peer support to the training outcomes. Nafukho et al. (2017) identified that, compared with organization and supervisor support, peer support might be more significant in promoting training maintenance and training transfer. The possible significance of peer support was also examined by Hua (2013) , who investigated the impact of supervisory support and peer support on the transfer of learning in a Malaysian state health department. The results revealed that peer support was significantly associated with the transfer of knowledge and skills from training, while supervisor support was not strongly associated with the training transfer.

Transfer of training and professional development in education

The significance of training design, trainee characteristics and work environment in promoting training transfer has been also examined and highlighted in educational PD. Rijdt et al. (2013) conducted a literature review on significant variables and moderators of the training transfer. The researchers followed the conceptual models of the transfer of training by Baldwin and Ford (1988) and synthesized findings of 134 studies on staff development in higher education. Baldwin and Ford highlighted that the training motivation and motivation to transfer were significant trainee characteristics components for the ability to transfer training. In terms of the training design construct, training relevance and training strategies were significantly associated with the transfer of training in staff development programs. As for the working environment, a supportive or positive transfer environment (e.g. peer support) contributed significantly to the training transfer. Nafukho et al. (2017) provided a critique of existing research on the transfer of learning in PD in higher education, but also addressed research gaps for future study.

Nafukho et al. (2017) examined the predictive capacity of training design, trainee’s motivation and work environment for training transfer in a CPE training program for adult learners. The results from the multiple regression analyses demonstrated that training efficiency and relevance were strong components in predicting the ability to transfer knowledge and skills from the PD program. Additionally, two work environment components of working complexity and working variability were significantly associated with the training transfer profession. Moreover, trainee’s motivation for training, measured as learning-oriented motivation and job-oriented motivation, was positively related to training transfer. Jackson et al. (2019) conducted a mixed-methods research study to explore the impact of variables of the training transfer for adult learning in a work integrated learning program. Jackson et al. identified significant roles of training design and work environment in promoting trainees’ training transfer abilities. Nafukho et al. (2017) suggested that training design was significant for bridging the connection between learning and working practices with specific strategies to facilitate the transfer of knowledge and skills learned from training. With respect to the work environment, providing a supportive environment and transfer climate promoted the transfer of training.

Research context

This study was part of a federal project titled Accelerated Preparation of Leaders for Underserved Schools: Building Instructional Capacity to Impact Diverse Learners (A-PLUS; Nafukho et al. , 2017 ) under the US Department of Education SEED Program. In the A-PLUS project, we aimed to promote diversity in the Texas educator workforce and support personalized learning environments in working and understanding the diverse needs of English learners (ELs) and economically challenged students (EC) students.

Participants

This study included 200 teachers working in high-needs schools with large numbers of EL and EC students across the state of Texas. Of the 200 teachers who participated in the PD program and who were invited to complete the Transfer of Learning Questionnaire, 80% ( n = 160) responded to the questionnaire. Teachers in this study primarily needed PD to empower them and enhance their instructional capacity for EL and EC students. During the recruitment, participants completed a demographic information (e.g. gender, ethnicity, number of years teaching, age, educational background) survey. Participant background information is provided in Table 1 .

The Transfer of Learning Survey used in this study is designed to predict transfer of learning to the workplace among adult learners who are enrolled in a CPE training program. Exploratory factor analysis revealed a seven-factor solution of 42 items, evaluated with a sample-size of n = 160 adult learners.

The Transfer of Learning factor significantly loaded five items adopted from the Nafukho et al. (2017) and Renta-Davids et al. (2014) studies that describe behavior change at work after the training. The items measured the trainees’ direct application of newly acquired skills and knowledge or the new work responsibilities and activities that resulted from training.

Trainees’ motivation to participate in training was measured through job- and learning-oriented motivation. Items (five per each factor) used to measure this factor was adopted from Nafukho et al. (2017) and Daahlen and Ure’s (2009) studies that focused on work- and non-work-related motives to participate in continuing professional training and development.

Training design included 11 items adopted from Nafukho et al. (2017) and Renta-Davids et al. (2014) studies to investigate how the design of training helps determine its successful delivery as measured by training efficacy and relevance. The present research focuses on training efficiency as an indicator for training design.

The work environment has been recognized as a factor that influences the transfer of learning. A complex work environment coupled with autonomy and flexibility can inspire employees to pursue continuous professional training and development ( Nafukho et al. , 2017 ). Work autonomy (five items), work variability (four items) and work complexity (seven items) loaded independently onto three factors and were believed to be reflective of aspects of one’s work environment ( Frieling, 2006 ). Frieling (2006) Learning Dimension Inventory (LDI) was used in this study to determine the role of the work environment in the transfer of learning.

Each factor demonstrated adequate to good reliability (Cronbach’s α = 0.33 to 0.939) and all items loaded significantly onto their respective factors. Seven of these factors and their related items were used as the primary measurement instruments for this study.

Data and analysis

To investigate the research questions of this study, structural equation modeling was used as the primary statistical tool for analysis. Each variable under consideration was measured as a composite score of their related items. The corresponding measurement error of the composite variable was taken into account by the reliability-adjusted method ( Hsiao et al. , 2018 ), in which the composite score was regressed on the underlying latent factor, while the error variance was fixed to the product of the observed score variance and one minus the sample reliability. This allows for a more accurate measurement of the path coefficients.

A mediation model with training design as the mediating factor was developed and used first in the investigation of the research questions. This model is presented in Figure 1 . If the model did not demonstrate significant predictive paths from job-oriented motivation (Job Mot), learning-oriented motivation (Lear Mot), work variability (Wk Var), work autonomy (Wk Auto), work complexity (Wk Comp) or training design (Tr Des) to transfer of learning (T.Transfer), then individual models were evaluated with a single predictor of interest to substantiate the significance or non-significance of that predictive relationship.

Descriptive statistics, correlational studies and reliability statistics measured as Cronbach’s α were calculated using STATA 16.1 ( STATACorp, 2019 ). Mediation and path analyses were conducted using Mplus 8.4 ( Muthen and Muthen, 1998-2020 ). The maximum likelihood robust estimator was use due to the small sample size and slight deviations in normality. The mediation model and individual models were just-identified with zero degrees of freedom. Thus, the commonly used fit statistics including, RMSEA, SRMR and CFI were expected to produce saturated global fit results with respective values of 0.00 or 1.00 ( Hu and Bentler, 1999 ). More attention was given to the path coefficients and outcome variable R 2 values, which signifies the amount of variance in the outcome variable explained by the predicting factors in the model. Paths were evaluated based upon significance at the standard a = 0.05 significance level.

A simple correlation matrix and descriptive statistics of the variables used in this study are provided in Table 2 . Table 2 shows the levels of α coefficient obtained from the test. The levels of α fall in the acceptable range, which confirms the reliability of the instrument used in this study. Moreover, the results did not show the presence of multicollinearity. Learning motivation (M = 4.57, SD = 0.44) and work variability (M = 4.34, SD = 0.73) had the highest mean among the variables. Whereas, work autonomy (M = 2.98, SD = 0.56) and job motivation (M = 3.17, SD = 0.80) had the lowest mean scores. The correlation matrix shows a positive strong correlation between job motivation and transfer of learning ( r = 0.67, p < 0.05). Work variability was found to have the correlation of ( r = 0.60, p < 0.05) with transfer of learning followed by work complexity ( r = 0.65, p < 0.05). Table 2 shows the levels of Pearson correlation coefficient obtained from the test. The levels of α fall in the acceptable range, which confirms the reliability of the instrument used in this study. Moreover, the results did not show the presence of multicollinearity.

Multiple mediation model

Table 3 shows the direct effects from the following indicators to “Transfer Total.” Training design had a significant direct effect on transfer total ( β = 0.58, z = 10.52, p < 0.05, 95% CI [0.477, 0.696]). Job orientation had a significant direct effect on transfer total ( β = 0.120, z = 2.16, p < 0.05, 95% CI [0.011, 0.230]). Learning orientation had a significant direct effect on transfer total ( β = 0.200, z = 3.23, p < 0.05, 95% CI [0.078, 0.321]).

The model enjoys a good fit. The χ 2 badness of fit was not significant ( χ 2 (3) = 4.283, p > 0.05). The ratio of χ 2 over the degree of freedom, i.e. 4.283/3 = 1.42 was lower than 3. These results supported the fit of the model. The RMSEA index of 0.052 was between 0.05 and 0.08 ( Byrne, 2010 , Bowen and Guo, 2011 , Kline, 2016 ; Schumacker and Lomax, 2016) , which supported the fit of the model. Although the lower limit of 90% confidence interval of RMSEA, i.e. 0.000 was between 0.05 to 0.08, its upper limit of 0.152 was higher than 0.08. The probability of close fit (PCLOSE) of 0.390 was 0.05, which supported the fit of the model.

The CFI and TLI indices of 0.993 and 0.974 were higher than 0.95. They supported the fit of the model; finally, the SRMR index of 0.028 was lower than 0.05. To summarize the results, it can be claimed that except for the upper limit of the 90% confidence interval of RMSEA, all other indices supported the fit of the model. The squared multiple correlations for each endogenous variable showed that the model could explain 53% of the variance in transfer of learning. The full model results investigating the mediation effect of training design are provided in Figure 2 .

To investigate the study’s primary research questions, a mediation model was first analyzed. We were able to determine the significant paths in the model.

As expected, the model was just-identified yielding saturated global fit statistics. The transfer of learning latent factor yielded a significant R 2 value of 0.754 ( p < 0.001). Thus, 75.4% of the variance of the transfer of learning factor was explained by the predictive factors in the model. As noted in the model, both learning motivation and training design were positively and significantly related to transfer of learning with standardized path coefficients of 0.22 ( p = 0.017) and 0.77 ( p < 0.001), respectively. Job-oriented motivation, work variability and work complexity were positively, but not significantly, associated with transfer of learning. Work complexity did not have any significant direct effect to transfer total ( β = −0.048, z = −0.87, p > 0.05, 95% CI [−0.158, 0.060]). Work autonomy did not have any significant direct effect to transfer total ( β = −0.048, z = −0.87, p > 0.05, 95% CI [−0.158, 0.060]).

These are the direct effects from observed variables to training efficiency. Learning orientation had a significant direct effect on training design ( β = 0.35, z = 5.53, p < 0.05, 95% CI [0.229, 0.481]). Work variability had a significant direct effect on training design ( β = 0.359, z = 5.31, p < 0.05, 95% CI [0.226, 0.492]). Work autonomy did not have any significant direct effect on training efficiency ( β = 0.056, z = 0.84, p > 0.05, 95% CI [−0.075, 0.189]).

Training efficiency had a significant direct effect on transfer total ( β = 0.704, z = 8.41, p < 0.05, 95% CI [0.540, 0.868]). As displayed in Table 4 , the total standardized indirect effect from learning motivation to transfer of learning through training design was 0.27 ( p < 0.001). Thus, training design partially mediated the effect of learning motivation on transfer of learning. Furthermore, the total standardized indirect effect from work variability to transfer of learning through training design was 0.31 ( p < 0.001). As work variability did not directly and significantly predict transfer of learning in this model ( p = 0.141), training design fully mediated the relationship between work variability and transfer of learning. Finally, the total standardized indirect effect from work complexity to transfer of learning through training design was 0.12 ( p = 0.095). This was a marginally significant result ( p < 0.1). As work complexity did not significantly predict transfer of learning in the mediation model, training design fully mediated the relationship between these factors. As job-oriented motivation, work variability and work complexity did not significantly predict transfer of learning, each of these was investigated individually to confirm the significance or non-significance of these relationships, noting that their path coefficients could have been suppressed by other relationships in the mediation model.

The results of the study were able to answer our RQ1 . We were able to determine that there is a relationship between trainee motivation, work environment, training design and transfer of learning. The findings are in line with the previous studies that explored these factors in relation to transfer of learning. In the case of CPE, in accordance with results of this study as well as the literature, increasing employees’ motivation to participate in such program has high potential to improve transfer of learning. Moreover, creating and fostering a work environment that favors autonomy and flexibility with the adequate amount of complexity can facilitate the transfer of learning process. Training design proved to have significant role in the transfer of learning process. Careful and thorough considerations must be given to this factor in CPE programs.

From the investigation into RQ2 , the results of the prior study by Nafukho et al. (2017) were confirmed regarding the positive predictive relationship between learning-oriented motivation, job-oriented motivation, work variability, work complexity and training design with transfer or learning. However, when these variables were analyzed under the scope of a mediation model, these relationships were interestingly altered.

The findings from the investigation of RQ3 revealed only learning-oriented motivation and training design significantly predicted transfer of learning, as seen from the mediation model in Figure 2 . Though each predictive variable used in the analysis showed a positive, significant relationship to transfer of training when examined under individual models, these relationships were seemingly suppressed in the larger mediation model. This is potentially caused by the strength of the training design to transfer of learning relationship that yielded a high standardized path coefficient of 0.77. This is a noteworthy finding for investigators and further enhances the importance of training design in CPE programs.

Furthermore, the results gave reason to conclude that training design has the potential to fully mediate the relationship between work environment (as measured by work variability and work complexity) and transfer of learning. Though independently, both work variability and work complexity showed positive, significant relationships with transfer of learning, neither of these factors accounted for a significant amount of the outcome factor’s variance. Indeed, when combined with other factors in the larger mediation model, both of these predictive relationships were non-significant. However, a significant indirect effect of training design from work variability to transfer of learning and partially significant indirect effect from work complexity to transfer of learning was revealed. Thus, the relationship between the trainee’s work variability and work complexity, with their transfer of learning of training-acquired knowledge and skills to their workplace, can be fully mediated by the training design. Thus, training design is an essential component of these relationships.

Training design also partially mediated the relationship between the trainee’s learning-oriented motivation for attending training and their transfer of learning from the training to their workplace. Thus, the training design has the potential to mitigate some of the effects of trainee’s motivation for attending the training on their actual transfer of knowledge and skills acquired in the training to their workplace.

Implications for future research

Considering the significance of learning transfer in developing professional knowledge and skills for teachers teaching in a complex and an ever-changing learning environment, confirming the mediating effects of training design on training transfer holds critical implications for future research. Specific and purposeful attention needs to be given to the design of CPE programs. Investigations into the effects of training design elements such as the training platform (online, hybrid or in-person), sample size, group structure and participant demographics are warranted. This will allow scholars to continue enhancing the research base regarding specific aspects of training design that mediate relationships between trainee motivation and workplace environment to their transfer of learning from the training to their workplace. The current study provides a deeper understanding of the factors that influence transfer of learning among educators who participated in this study. The study identified the factors that significantly influence transfer of learning and the factors that were unsignificant. We recommend that future studies focus on each factor separately in and in-depth investigation on how and why they affect transfer of learning. The results of this study and similar studies conducted on transfer of learning should provide researchers and practitioners with the appropriate input for theory development to assist in easing the complexity of the process of transfer of learning among CPE programs.

As research focusing on transfer of learning among teachers participating in CPE intervention programs advances, there is an urgent need to demonstrate that CPE programs that are successful in professions such as accounting, medicine and law can also work in the teaching profession. In addition to conducting correlation studies of this nature is the beginning, and more additional questions on the importance of CPE need to be raised and answered. There is a compelling need to conduct randomized controlled studies with the treatment and control groups to determine the effect of CPE intervention programs on transfer of learning to the workplace. There is also need to use mixed-methods research studies aimed at determining the impact of CPE programs, especially for teachers. For instance, qualitative research approaches need to be used together with quantitative research approaches. Gilbert (2020) highlighted the increasing interest in CPE intervention programs for teacher in- and post-service training programs. This is of great importance to HRD and OD researchers and practitioners interested in examining the internal and external efficiency of the complex education industry.

Implications for practice

In addressing our research questions, the practical implications of the results had been given a thorough consideration. We share Blume et al. ’s (2010) point of view on the need for providing training professionals with the necessary evidence to apply the results of the study to their practice. The finding of this research provides a preliminary guide for practitioners.

This study was able to confirm the role that learning-oriented motivation, job-oriented motivation, work variability, work complexity and training design play in transfer of learning. In practice, training professionals will be more comfortable pinpointing the factors that lead to the transfer of learning or the lack of it. Our review of the literature showed a large number of factors that affect transfer of learning. This finding will limit the confusion of practitioners. The study at hand was also able to test the significance of the effect of each of the presented factors on transfer of learning. We were able to determine that learning-oriented motivation and training design are the variables that had a significant effect. Furthermore, training design proved to be essential. These findings will inform the training professionals on the area of focus. Training design is the most important factor that practitioners can focus on to improve transfer of learning.

All in all, this study provides a better understanding for CPE training professionals of the factors that lead to a successful transfer of learning. The scarcity of resources limits the practitioners’ choices in CPE programs. Therefore, we suggest the importance of allocating their resources to training design. This study can be used in conjunction with other research to improve transfer of learning in CPE programs in particular and other types of training in general.

Study limitations

This study is limited in its generalizability. With 93% of participants being female, this study lacks generalizability to the male population. Additionally, these findings are only applicable to educators and not to the general business/industry environment. Future research endeavors need to validate these findings for male educators and investigate any potential group differences in the mediating effect of training design to transfer of learning. Another limitation of this study is the use of quantitative methods to analyze the data. Using a qualitative approach would provide us with a more in-depth look into the factors studied. Collecting data through interviews will improve the accuracy of our research results.

The purpose of this study was to examine the impacts of training design, trainee motivation and work environment on the transfer of learning for teachers working in high-needs schools who enrolled in a CPE training program. The findings from this study further amplify the gravity of training design within CPE training programs. Effective training design holds extreme potential for mitigating the effects of trainees’ motivation and work environment to their transfer of learning from the training to their workplace. Training design as measured by training relevance and training efficiency confirms the critical role of the trainer as a designer and facilitator of the training. Thus, it is advisable that researchers, scholars and educational practitioners thoughtfully design their training with specific purposes and learning outcome targets. This also calls for trainers themselves to continuously invest in their own learning and engage in learning for a lifetime.

Mediation model investigating the potential mediating effect of training design using reliability-adjusted variables

Standardized full-model results investigating the mediation effect of training design

Background information of participants

Variable		(%)

Female	149	93.1
Male	9	5.6
Others	2	1.3
Total	160	100

More than 10	54	33.8
5 to less than 10	34	21.3
3 to less than 5	29	18.1
1 to less than 3	33	20.6
0 to less than 1	8	5
N/A	2	1.3
Total	160	100

Associate’s or GED	9	5.6
Bachelor’s	77	48.1
Master’s	68	42.5
Doctorate	3	1.9
N/A	3	1.9
Total	160	100

the Results of descriptive statistics and Pearson correlation coefficient

Variable	M	SD		Pearson correlation	1	2	3	4	5	6	7
1. Learning motivation	4.57	0.44	0.807	0.59	1.00
2. Job motivation	3.17	0.80	0.71	0.67	0.19	1.00
3. Work variability	4.34	0.73	0.939	0.60	0.29	−0.06	1.00
4. Work complexity	3.66	0.64	0.788	0.65	0.11	0.08	0.12	1.00
5. Work autonomy	2.98	0.56	0.338	0.67	−0.02	0.12	0.28	0.04	1.00
6. Training design	4.06	0.48	0.87	0.53	0.42	0.04	0.48	0.22	0.15	1.00
7. Transfer of learning	3.91	0.97	0.79	0.54	0.47	0.15	0.30	0.21	0.05	0.69	1.00

Standardized regression model estimates

Dependent variables	Independent variables					95% CI
Transfer of learning	Training design	0.59	0.06	10.52	<0.001	[0.478, 0.697]	0.53
	Job motivation	0.12	0.06	2.16	0.031	[0.011, 0.230]
	Learning motivation	0.20	0.06	3.23	<0.01	[0.079, 0.322]
	Work complexity	0.05	0.06	0.93	0.352	[−0.058,0.160]
	Work autonomy	−0.05	0.07	−0.87	0.383	[−0.158, 0.061]
Training design	Learning motivation	0.36	0.06	5.53	<0.001	[0.230, 0.482]	0.35
	Work variability	0.36	0.07	5.31	<0.001	[0.227, 0.492]
	Work autonomy	0.06	0.07	0.84	0.400	[−0.076, 0.190]
Overall							0.42

Bias-corrected bootstrapping indirect effects

Indirect effects	Independent variables	Observed coefficient	Bootstrap SE	z	p	95% CI
Training transfer	Training design	0	(no path)
	Job motivation	0	(no path)
	Learning motivation	0.27	0.06	3.97	0.13	0.40
	Work complexity	0.16	0.04	3.64	0.07	0.25
	Work autonomy	0	(no path)
		0.034	0.04	0.73	−0.05	0.12
Training design	Learning motivation	0	(no path)
	Work variability	0	(no path)
	Work autonomy	0	(no path)

Avalos , B. ( 2011 ), “ Teacher professional development in teaching and teacher education over ten years ”, Teaching and Teacher Education , Vol. 27 No. 1 , pp. 10 - 20 .

Baldwin , T.T. and Ford , J.K. ( 1988 ), “ Transfer of training: a review and directions for future research ”, Personnel Psychology , Vol. 41 No. 1 , pp. 63 - 105 .

Baldwin , T.T. , Ford , J.K. and Blume , B.D. ( 2009 ), “ Transfer of training 1988-2008: an updated review and new agenda for future research ”, in Hodgkinson , G.P. and Ford , J.K. (Eds), International Review of Industrial and Organizational Psychology , Wiley , Vol. 24 , pp. 41 - 70 .

Baldwin , T.T. , Kevin Ford , J. and Blume , B.D. ( 2017 ), “ The state of transfer of training research: moving toward more consumer‐centric inquiry ”, Human Resource Development Quarterly , Vol. 28 No. 1 , pp. 17 - 28 .

Banks , G.C. , Pollack , J.M. , Bochantin , J.E. , Kirkman , B.L. , Whelpley , C.E. and O’Boyle , E.H. ( 2016 ), “ Management’s science–practice gap: a grand challenge for all stakeholders ”, Academy of Management Journal , Vol. 59 No. 6 , pp. 2205 - 2231 .

Beer , M. , Finnström , M. and Schrader , D. ( 2016 ), “ Why leadership training fails – and what to do about it ”, Harvard Business Review , Vol. 94 No. 10 , pp. 50 - 57 .

Blume , B.D. , Ford , J.K. , Baldwin , T.T. and Huang , J.L. ( 2010 ), “ Transfer of training: a meta-analytic review ”, Journal of Management , Vol. 36 No. 4 , pp. 1065 - 1105 .

Bowen , N.K. and Guo , S. ( 2011 ), Structural Equation Modeling , Oxford University Press , Oxford .

Burke , L. and Hutchins , H. ( 2007 ), “ Training transfer: an integrative literature review ”, Human Resource Development Review , Vol. 6 No. 3 , pp. 263 - 297 .

Byrne , B.M. ( 2010 ), Structural Equation Modeling with AMOS: Basic Concepts, Applications, and Programming , Routledge , New York .

Chen , H.C. , Holton , E.F. III. and Bates , R.A. ( 2006 ), “ Situational and demographic influences on transfer system characteristics in organizations ”, Performance Improvement Quarterly , Vol. 19 No. 3 , pp. 7 - 25 .

Chiaburu , D.S. ( 2010 ), “ The social context of training: coworker, supervisor, or organizational support? ”, Industrial and Commercial Training , Vol. 42 No. 1 , pp. 53 - 56 , doi: 10.1108/00197851011013724 .

Chiaburu , D.S. , Dam , K.V. and Hutchins , H.M. ( 2010 ), “ Social support in the workplace and training transfer: a longitudinal analysis ”, International Journal of Selection and Assessment , Vol. 18 No. 2 , pp. 187 - 200 , doi: 10.1111/j.1468-2389.2010.00500.x .

Chow , A. , Finney , T.G. and Woodford , K. ( 2010 ), “ Training design and transfer: contributions of six sigma ”, International Journal of Productivity and Performance Management , Vol. 59 No. 7 , pp. 624 - 640 .

Cromwell , S.E. and Kolb , J.A. ( 2004 ), “ An examination of work‐environment support factors affecting transfer of supervisory skills training to the workplace ”, Human Resource Development Quarterly , Vol. 15 No. 4 , pp. 449 - 471 .

Daahlen , M. and Ure , O. ( 2009 ), “ Low-skilled adults in formal continuing education: does their motivation differ from other learners? ”, International Journal of Lifelong Education , Vol. 28 No. 5 , pp. 661 - 674 .

Daffron , S. and North , M. ( 2006 ), “ Learning transfer: tips from software company professionals ”, PAACE Journal of Lifelong Learning , Vol. 15 , pp. 51 - 67 .

Daley , B.J. and Cervero , R.M. ( 2016 ), “ Learning as the basis for continuing professional education ”, New Directions for Adult and Continuing Education , Vol. 2016 No. 151 , pp. 19 - 29 .

Dermol , V. and Čater , T. ( 2013 ), “ The influence of training and training transfer factors on organizational learning and performance ”, Personnel Review , Vol. 42 No. 3 , pp. 324 - 348 , doi: 10.1108/00483481311320435 .

Dixit , R. and Sinha , V. ( 2022 ), “ Investigating tools and techniques to promote workplace training transfer ”, Journal of Workplace Learning , Vol. 34 No. 6 , pp. 513 - 531 .

Donovan , P. and Darcy , D.P. ( 2011 ), “ Learning transfer: the views of practitioners in Ireland ”, International Journal of Training and Development , Vol. 15 No. 2 , pp. 121 - 139 .

Dreer , B. , Dietrich , J. and Kracke , B. ( 2017 ), “ From in-service teacher development to school improvement: factors of learning transfer in teacher education ”, Teacher Development , Vol. 21 No. 2 , pp. 208 - 224 .

Every Student Succeeds Act ( 2015 ), “ Pub. L. No. 114-95 ”, p. 1177 .

Fauth , F. and González-Martínez , J. ( 2021 ), “ Trainee perceptions of instructional design in continuous online training and learning transfer ”, Education Research International , Vol. 2021 doi: 10.1155/2021/3121559 .

Fraser , C. , Kennedy , A. , Reid , L. and Mckinney , S. ( 2007 ), “ Teachers’ continuing professional development: contested concepts, understandings and models ”, Journal of in-Service Education , Vol. 33 No. 2 , pp. 153 - 169 .

Frieling , E.B. ( 2006 ), Lernen Durch Arbeit: entwicklung Eines Verfahrens Zur Bestimmung Der Lernmöglichkeiten Am Arbeitsplatz , Waxmann Verlag , Münster .

Galoyan , T. and Betts , K. ( 2021 ), “ Integrative transfer of learning model and implications for higher education ”, The Journal of Continuing Higher Education , Vol. 69 No. 3 , pp. 1 - 27 .

Garavan , T. , Cahir-O’Donnell , A. and Hogan , C. ( 2021 ), Transfer of Training in Organizations: Learning and Development in Organizations Series , Oak Tree Press , Fyfield, Oxford .

Gegenfurtner , A. , Veermans , K. , Festner , D. and Gruber , H. ( 2009 ), “ Integrative literature review: motivation to transfer training: an integrative literature review ”, Human Resource Development Review , Vol. 8 No. 3 , pp. 403 - 423 .

Gilbert , J.M. ( 2020 ), “ Cognitive conditions and transfer of professional development learning in elementary school teachers ”, A doctoral dissertation Grand Canyon University .

Govaerts , N. , Kyndt , E. and Dochy , F. ( 2018 ), “ The influence of specific supervisor support types on transfer of training: examining the mediating effect of training retention ”, Vocations and Learning , Vol. 11 No. 2 , pp. 265 - 288 , doi: 10.1007/s12186-017-9190-y .

Hawley , J. and Barnard , J.K. ( 2005 ), “ Work environment characteristics and implications for training transfer: a case study of the nuclear power industry ”, Human Resource Development International , Vol. 8 No. 1 , pp. 65 - 80 .

Heimbeck , D. , Frese , M. , Sonnentag , S. and Keith , N. ( 2003 ), “ Integrating errors into the training process: the function of error management instructions and the role of goal orientation ”, Personnel Psychology , Vol. 56 No. 2 , pp. 333 - 361 , doi: 10.1111/j.1744-6570.2003.tb00153.x .

Hsiao , Y. , Kwok , O. and Lai , M.H.C. ( 2018 ), “ Evaluation of two methods for modeling measurement errors when testing interaction effects with observed composite scores ”, Educational and Psychological Measurement , Vol. 78 No. 2 , pp. 181 - 202 .

Hu , L-T. and Bentler , P.M. ( 1999 ), “ Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives ”, Structural Equation Modeling: A Multidisciplinary Journal , Vol. 6 No. 1 , pp. 1 - 55 .

Hua , N.K. ( 2013 ), “ The influence of supervisory and peer support on the transfer of training ”, Studies in Business and Economics, Lucian Blaga University of Sibiu, Faculty of Economic Sciences , Vol. 8 No. 3 , pp. 82 - 97 .

Iqbal , M.Z. and Alsheikh , M.H. ( 2008 ), “ Factors affecting transfer of training to the workplace after faculty development program: what do trainers think? ”, Biomedical Journal of Scientific and Technical Research , Vol. 3 No. 3 , pp. 3292 - 3296 .

Iqbal , M.Z. and Alsheikh , M.H. ( 2018 ), “ Factors affecting the transfer of training to the workplace after a faculty development programme: what do trainers think? ”, Biomedical Journal of Taibah University Medical Sciences , Vol. 13 No. 6 , pp. 552 - 556 .

Ismail , A. , Foboy , N.A. , Bakar , R. , Nor , N.M. and Rosnan , H. ( 2015 ), “ Training motivation as mediator of the relationship between training administration and training transfer ”, Jurnal Pengurusan , Vol. 43 , pp. 97 - 106 .

Jackson , D. , Fleming , J. and Rowe , A.D. ( 2019 ), “ Enabling the transfer of skills and knowledge across classroom and work contexts ”, Vocations and Learning , Vol. 12 No. 3 , pp. 1 - 20 .

Kiwanuka , J. , Miiro , R.F. , Matsiko , F.B. and Nkalubo , S. ( 2020 ), “ Using the learning transfer system inventory to test the effects of trainee and training design characteristics on the transfer of agricultural training in Uganda ”, International Journal of Training and Development , Vol. 24 No. 4 , pp. 374 - 383 .

Kline , R.B. ( 2016 ), Principles and Practice of Structural Equation Modeling , Guilford publications , New York .

Kodwani , A.D. and Prashar , S. ( 2021 ), “ Influence of individual characteristics, training design and environmental factors on training transfer: a study using hierarchical regression ”, Evidence-Based HRM: a Global Forum for Empirical Scholarship , Vol. 9 No. 4 , doi: 10.1108/EBHRM-09-2019-0085 .

Lim , D.H. and Morris , M.L. ( 2006 ), “ Influence of trainee characteristics, instructional satisfaction, and organizational climate on perceived learning and training transfer ”, Human Resource Development Quarterly , Vol. 17 No. 1 , pp. 85 - 115 .

Macaulay , C. and Cree , V.E. ( 1999 ), “ Transfer of learning. Concept and process ”, Social Work Education , Vol. 18 No. 2 , pp. 183 - 194 .

Muduli , A. and Raval , D. ( 2018 ), “ Examining the role of work context, transfer design and transfer motivation on training transfer: perspective from an Indian insurance industry ”, European Journal of Training and Development , Vol. 42 Nos 3/4 , doi: 10.1108/EJTD-09-2017-0078 .

Muthen , L.K. and Muthen , B.O. ( 1998 -2020), Mplus User’s Guide , 8th ed. , Muthen and Muthen , Los Angeles, CA .

Nafukho , F.M. , Alfred , M. , Chakraborty , M. , Johnson , M. and Cherrstrom , C.A. ( 2017 ), “ Predicting workplace transfer of learning: a study of adult learners enrolled in a continuing professional education training program ”, European Journal of Training and Development , Vol. 41 No. 4 , pp. 327 - 353 .

Na-Nan , K. , Chaiprasit , K. and Pukkeeree , P. ( 2017 ), “ Influences of workplace environment factors on employees’ training transfer ”, Industrial and Commercial Training , Vol. 49 No. 6 , pp. 303 - 314 .

Noe , R.A. ( 1986 ), “ Trainees attributes and attitudes: neglected influences on training effectiveness ”, The Academy of Management Review , Vol. 11 No. 4 , pp. 736 - 749 .

Noe , R.A. ( 2020 ), Employee Training and Development , 8th ed. , McGraw-Hill Education , New York, NY .

Quratulain , S. , Khan , A.K. , Sabharwal , M. and Javed , B. ( 2021 ), “ Effect of self-efficacy and instrumentality beliefs on training implementation behaviors: testing the moderating effect of organizational climate ”, Review of Public Personnel Administration , Vol. 41 No. 2 , pp. 250 - 273 .

Renta-Davids , A.I. , Jiménez-González , J.M. , Fandos-Garrido , M. and González-Soto , Á.P. ( 2014 ), “ Transfer of learning: motivation, training design and learning-conducive work ”, European Journal of Training and Development , Vol. 38 No. 8 , pp. 728 - 744 .

Rijdt , C.D. , Stes , A. , Vleuten , C. and Dochy , F. ( 2013 ), “ Influencing variables and moderators of transfer of learning to the workplace within the area of staff development in higher education: research review ”, Educational Research Review , Vol. 8 No. 1 , pp. 48 - 74 .

Russ-Eft , D. ( 2002 ), “ A typology of training design and work environment factors affecting workplace learning and transfer ”, Human Resource Development Review , Vol. 1 No. 1 , pp. 45 - 65 , doi: 10.1177/1534484302011003 .

Schindler , L.A. and Burkholder , G. ( 2016 ), “ A mixed methods examination of the influence of dimensions of support on training transfer ”, Journal of Mixed Methods Research , Vol. 10 No. 3 , pp. 292 - 310 .

Schumacker , E. and Lomax , G. ( 2016 ), A Beginner’s Guide to Structural Equation Modeling , Routledge , New York .

Seeg , B. , Gauglitz , I.K. and Schütz , A. ( 2021 ), “ Explaining and enhancing training transfer: a consumer-centric evaluation of leadership training ”, Human Resource Development International , pp. 1 - 21 .

Seidel , J. ( 2012 ), Transferkompetenz Und Transfer. Theoretische Und Empirische Untersuchung zu Den Wirksamkeitsbedingungen Betrieblicher Weiterbildung [Transfer Competence and Transfer. A Theoretical and Empirical Analysis of the Conditions Affecting Effectiveness in Further Education] , Verlag Empirische Pädagogik .

Sitzmann , T. and Weinhardt , J.M. ( 2019 ), “ Approaching evaluation from a multilevel perspective: a comprehensive analysis of the indicators of training effectiveness ”, Human Resource Management Review , Vol. 29 No. 2 , pp. 253 - 269 .

STATACorp ( 2019 ), STATA Statistical Software Release 16 , STATACorp LLC , College Station, TX .

Tannehill , D. , Demirhan , G. , Čaplová , P. and Avsar , Z. ( 2021 ), “ Continuing professional development for physical education teachers in Europe ”, European Physical Education Review , Vol. 27 No. 1 , pp. 150 - 167 .

Taylor , P.J. , Russ-Eft , D.F. and Chan , D.W.L. ( 2005 ), “ A meta-analytic review of behavior modeling training ”, Journal of Applied Psychology , Vol. 90 No. 4 , pp. 692 - 709 , doi: 10.1037/0021-9010.90.4.692 .

Tonhauser , C. and Buker , L. ( 2016 ), “ Determinants of transfer of training: a comprehensive literature review ”, International Journal for Research in Vocational Education and Training , Vol. 3 No. 2 , pp. 127 - 165 .

Tracey , J. and Tews , M.J. ( 2005 ), “ Construct validity of a general training climate scale ”, Organizational Research Methods , Vol. 8 No. 4 , pp. 353 - 374 .

Velada , R. , Caetano , A. , Michael , J.W. , Lyons , B.D. and Kavanagh , M.J. ( 2007 ), “ The effects of training design, individual characteristics and work environment on transfer of training ”, International Journal of Training and Development , Vol. 11 No. 4 , pp. 282 - 294 .

Webster-Wright , A. ( 2009 ), “ Reframing professional development through understanding authentic professional learning ”, Review of Educational Research , Vol. 79 No. 2 , pp. 702 - 739 .

Yaqub , Y. , Singh , A.K. and Dutta , T. ( 2021 ), “ An empirical study of factors influencing training transfer in the management training intervention ”, Journal of Workplace Learning , Vol. 33 No. 5 , doi: 10.1108/JWL-02-2020-0034 .

Yunus , F. and Yasin , R. ( 2014 ), “ Learning transfers in training institutions and the workplace in Malaysia ”, Journal for Technical and Vocational Education and Training in Asia , Vol. 3 , pp. 1 - 16 .

Acknowledgements

This Study was supported under the grant from the U.S. Department of Education, Project Accelerated Preparation of Leaders for Underserved Schools (A-PLUS): Building Instructional Capacity a SEED grant (Award#1894-0008).

Corresponding author

Related articles, all feedback is valuable.

Please share your general feedback

Report an issue or find answers to frequently asked questions

Contact Customer Support

Learning Outcomes and Training Satisfaction: A Case Study of Blended Customization in Professional Training

Original research
Open access
Published: 09 September 2024

Cite this article

You have full access to this open access article

Sara Torre ORCID: orcid.org/0009-0001-7399-6692 1 , 2 ,
Antonio Ulloa Severino 2 &
Maria Beatrice Ligorio 1

109 Accesses

Explore all metrics

In the case of training programs for workplace settings, design customization can help trainers to better address trainees’ needs and, at the same time, it can help them build a sense of competence and autonomy. This is particularly difficult when trainees are skeptical because of former failing training experiences. The case study presented here, is about a training program featuring customization design from the pre-training phase throughout the training process, aimed precisely at trainees with previous negative experiences. Eighteen participants (M: 10; F: 8; age average: 55,7) were involved in training senior professionals in the information communication technology (ICT) field, all of them with a history of failed training attempts and a long period of workplace inactivity. In preparation for the training, the trainers gathered information about trainees’ attitude towards training, training preferences, and baseline skills, which determined the training design. During training, feedback and intermediate learning results were considered for fine tuning. Results attested the change of attitude towards training, perceived enhancement of self-awareness, feelings of being part of a community, and successful learning outcomes.

Flexibility in Formal Workplace Learning: Technology Applications for Engagement through the Lens of Universal Design for Learning

Learning experience design (LXD) professional competencies: an exploratory job announcement analysis

Process is king: evaluating the performance of technology-mediated learning in vocational software training, explore related subjects.

Artificial Intelligence
Medical Ethics
Digital Education and Educational Technology

Avoid common mistakes on your manuscript.

1 Introduction

The need in information communication technology (ICT) organizations to efficiently update internal technical and organizational skills leads to engagement with training providers specialized in professional training. As it often is the case, the main contractor of the training program—a human resources manager, a training manager, or an executive manager—reaches out to the training provider with a plethora of needs and goals. These needs and goals could deal with a large array of issues such as organizational strategies of human resources development (Hughey & Mussnug, 1997 ), employee’s satisfaction improvement and professional fulfillment (Shen & Tang, 2018 ), and workers’ productivity expansion (Niazi, 2011 ). In other words, a company may ask for a training intervention to invest in staff development, to answer environmental challenges, or to rectify perceived shortcomings in the skill asset of the company (Isiaka, 2011 ).

Oftentimes, the company requesting a training program seeks for a training design compliant mainly with the needs expressed by the management without really considering the trainees’ needs (Morrison, 2021 ). For instance, management may assume that a training program should be implemented in a face-to-face setting to guarantee participants’ engagement, and explicitly request an in-presence training program. At the same time, employees may perceive training as a source of fatigue due to the stress of commuting and interrupting their routine and prefer online training. In other words, management’s proposals could heavily influence the training design and program. Training needs analysis in organizations can also be prominently affected by organizational politics because of conflict and power relations (Clarke, 2003 ). Therefore, compliance to management is not sufficient to conceive an effective design, as the training context and the recent research should also be considered. Successful training can be achieved through awareness of organizational culture, individual characteristics of the trainees and baseline skills (Tracey & Tews, 1995 ).

In addition, to the best of our knowledge, oftentimes training programs in corporate settings are not designed to build on the most recent innovation reported in scientific literature on education and learning (Pavliuk et al., 2017 ). While training programs based on documented theoretical background and outcomes are implemented in healthcare fields (Richmond et al., 2017 ) and school settings (Luneta, 2012 ), small and mid-sized training practitioners may lack the resources to base their practice on learning models or to share their findings and innovations with the community.

In synthesis, two types of interlinked concerns emerge about training programs for corporations. Firstly, they rarely consider the specific requirements of each organizational context and learning group, favoring management’s guidelines; secondly, they tend not to draw inspiration from research nor to produce scientific publications.

Training companies can overcome these limitations. They surely can consider the management’s requirements, but at the same time their professionality is based on the evidence coming from the specialized literature in any phase of the training program. The case we present here, originated when an ICT company was looking for a customized blended training program. The reason declared by the management was to re-skill the inactive workers and to be able to employ them in the company’s future projects. After considering management’s goals and benchmarks, the training planners developed a learning model inspired by recent approaches highlighting the ability to adapt to the training participants (Alamri et al., 2021 ). From a review—presented in the subsequent section—emerges the relevance of on-going adaptation to the individual trainees. In our case, a distinctive feature was added. The training program was designed to personalize methods and contents coherently with preliminary information, shaping the training intervention already before it started. As a result, a blended learning environment was set up, able to intercept and implement trainees’ needs throughout the course. The continuous fine-tuning of the training program was based on learning analytics and participant’s feedback.

In the next sections, we will first report a short review on the most used theoretical background and approaches in designing personalized professional training; right after, we will present our case stressing its peculiarities such as the pre-customization and the results obtained.

2 Theoretical Background

Personalized learning management systems (LMS) can support accessing learning resources (Dolog et al., 2004 ) and promoting a sustainable learning environment (Klašnja-Milićević & Ivanovi, 2021 ). Even with these advantages, training personalization is still under-utilized within workforce training (Fake & Dabbagh, 2020 ). Oftentimes, personalization and customization are used interchangeably to describe a form of platform or content adaptation to learners in an e-learning setting (Marappan & Bhaskaran, 2022 ). Only a few studies focus on the effectiveness of personalization versus customization, describing personalization as a process that requires feature adaptation to the users’ needs, while customization directly involves users’ choice (Zo, 2003 ). Personalization is a top-down process where the digital platform or the training program is adapted to participants’ characteristics by a system or an operator (Sundar & Marathe, 2010 ). On the other hand, customization is a bottom-up process that allows users to directly impact the digital platform and training program by expressing their preferences (Sundar & Marathe, 2010 ). A major advantage of customization, when compared to personalization, is the perceived control (Zo, 2003 ) and engagement participants can experience after the customization experience (Zine et al., 2014 ). Training customization was mostly explored in academic contexts (Zine et al., 2014 ; Zo, 2003 ), providing relevant information about the effects of training customization. Nevertheless, the perception of training in an academic setting can differ quite a bit from its perception in a business setting (Fayyoumi, 2009 ).

Learning analytics can be implemented into educational environments for the purpose of being utilized by instructors, learning designers and students (Reigeluth & Beatty, 2016 ; Wise & Vytasek, 2017 ). For example, learning analytics can be used for personalizing feedback and course design (Gong et al., 2018 ; Yilmaz & Yilmaz, 2020 ). When it comes to customization, learning analytics can drive learning program customization in two ways (Wise & Vytasek, 2017 ): through adaptive learning analytics (Brooks et al., 2014 ; Brusilovsky & Peylo, 2003 ) or as adaptable learning analytics (Brooks et al., 2014 ). The first approach addresses the design of the learning analytics implementation, while the second lets users determine which analytics will be attended to and how they will influence the learning process. However flexible this last approach may be, Wise and Vytasek ( 2017 ) state that users may be overwhelmed and confused about the uncertainty of the choice. Therefore, user agency has to be supported by the implementation design and by the instructors that are supposed to make decisions aimed at meeting the users’ needs and resources.

Another factor that can influence the learning experience is the formative feedback. Through this factor, personalized assignments can be created and the learning resources are tailored based on trainees’ needs (Knight et al., 2020 ). However, when considering the formative feedback, a much more interactive system is possible (Neville et al., 2005 ). When trainees are able to impact training through feedback, trainers can gather fundamental information about the use of LMS, effectiveness of the training program, students’ relationships (Neville et al., 2005 ), and about the training program acceptance (Feng, 2020 ).

Learning outcomes and training satisfaction are often measured to assess efficacy of training (Armatas et al., 2022 ; Arthur et al., 2003 ; Puška et al., 2021 ). Many dimensions are proven to enhance learning outcomes and training satisfaction. For instance, it has been found that interaction between learners improves learning performance (Wang, 2023 ). In addition, training personalization can increase retention, customer satisfaction, and learner engagement (Kwon & Kim, 2012 ; Kwon et al., 2010 ; Springer, 2014 ).

Finally, it seems useful to define the concept of blended learning (BL) and how the training model adopted in our case can be tailored to fit a trainee's needs. BL is defined by Garrison and Kanuka ( 2004 ) as “the thoughtful integration of classroom face-to-face learning experiences with online learning experiences” (p. 96). As stated by Kaur ( 2013 ), BL can offer trainees “the best of both worlds” because it guarantees training flexibility without missing out face-to-face interactions (p. 5). Indeed, in BL models the advantages of both face-to-face encounters and online platforms are combined and valued (Ligorio & Sansone, 2009 ). However, BL can clash with the trainees’ technological resistance (Prasad et al., 2018 ), especially for adult students (Lightner & Lightner-Laws, 2016 ). This is the reason why this type of user feels hindered their ability to gain access to instructional materials (Rasheed et al., 2020 ) or their ability to learn a new technology. In some cases, the online sections of BL can be burdened by feelings of learning alienation (Chyr et al., 2017 ) and isolation (Lightner & Lightner-Laws, 2016 ). Students may be hesitant in engaging in online learning communities due to lack of non-verbal communication cues in asynchronous activities (Rashed et al., 2020 ), and discomfort in using cameras and microphones for synchronous activities (Szeto & Cheng, 2016 ).

In some training cases, BL can be adapted to the trainee’s characteristics to build a learned-centered training environment (Alamri et al., 2021 ). For example, learning statistics (Çetinkaya, 2016 ) or prior knowledge (Glover & Latif, 2013 ) are used to offer trainees a personalized learning experience (Watson and Watson, 2016 ), purposely selected content (Xie et al., 2019 ), and trainee-specific modeling activities (Amenduni & Ligorio, 2022 ; Amenduni et al., 2021 ). These examples fit the definition of personalized training previously provided, as the training process is adapted by an instructor or a software.

In synthesis, the literature review has revealed that better results in professional training could be obtained by increasing personalization and flexibility, in terms of trainees satisfaction and learning outcomes. This can be surely done by exploiting the opportunities offered by technology—both in terms of LMS features and psycho-pedagogical models. However, these elements are yet to be examined jointly in a work-place training setting. It is also important to consider the timing of personalization implementation within the course. Therefore, the main research question who leads our intervention concerns the effects of introducing an early personalized training in a complex context: What are the advantages and disadvantages of early customization and constant flexibility? This overall research question will be better exploited in the next section.

3 Methodology

3.1 the research question.

As already stated earlier, our overall research question is: What are the effects of early customization and constant flexibility in a training program meant to relocate professionals coming from a complex context? This question arises from the gap in literature concerning the real application of flexible training in a professional setting. Furthermore, our research question aims to explore if the concerns about BL’s risk of learning alienation and isolation in participants can be addressed through customization and flexibility.

To answer this question, the research group focused on two aspects in particular: (a) The early customization and flexibility; as revealed by the literature reviews, most of the training programs consider customization and flexibility from the start to the end of the course. In our case, personalization and flexibility are introduced even before the start of the course, in its planning phase. This was possible because information about the trainees was available, and the trainers used this information to personalize the program; (b) The effects of such customization. In particular, we are interested in the impact of such a course on attitude towards training, perceived enhancement of self-awareness, feelings of belonging to a community, and successful learning outcomes.

3.2 The Trainees' Company

The company involved in the case study presented here, is a leading company in ICT services for digital transformation. Its main activities include the design and implementation of custom end-to-end solutions, cloud environments, digital and operations services. Two years prior to the training program described here, this ICT company acquired a branch of another company, incorporating the majority of the branch’s workers. A consistent number of these workers did not have the opportunity to acquire the technical skills demanded by the new company; therefore, a mismatch was detected between the actually possessed and the required ones. As the ICT company could not employ these workers in its projects, they became inactive and did not take part in company practices. Inactive workers were offered to terminate their contract and receive a congruent severance pay. This proposal led to the involvement of the labor union, who sought after the assurance of rightful treatment for the inactive workers. Nevertheless, most of the inactive workers accepted the proposal of termination and exited the company, while the remaining inactive workers stood their ground and kept their position in the company. This decision of the remaining workers was reportedly caused by two matters: Uncertainty of employment outside the company and impracticability of upgrading their skill set outside the company (for example, by getting back to academic studying). At approximately the same time as this decision, the COVID-19 lockdown was imposed in all Italian workplaces. This unprecedented event, in addition to the dire foregoing employment situation, led the remaining workers to spend most part of two years at home, with no client project to work on, with a few contacts with the management and almost no communication with their co-workers. During these two challenging years, the ICT company attempted to train the remaining workers in a new customer relationship management (CRM) platform. The course was delivered by a third-party training company. Unfortunately, no trainee passed the final test and none of them attained the course certification. In conclusion, the course was regarded by the trainees as another misstep from management.

On the impending opening of a new company office in the South of Italy, the ICT company decided to employ the remaining 18 workers in the new branch-office and enable them to fulfill the demands of the new position. To do so, the desired training program would need to motivate the trainees enough to help them learn new technologies and organizational methods, while easing their distrust of training due to the previous attempt. At this point, a new training company (Grifo Multimedia) entered the scene.

3.3 The Training Company

The ICT company contacted Grifo Multimedia Footnote 1 to commission a training program for a group of employees. This is a company with more than 20-year experience in the field of training solutions and with a strong reputation about e-learning.

Grifo Multimedia is specialized in supplying digital learning and gamification solutions, with a special focus on tailoring services to the client’s needs. As a company specialized in professional training programs, Grifo Multimedia takes care of all the facets of the training implementation: needs’ analysis, training design, digital learning content development and maintenance, ICT platform supply and customization. Grifo Multimedia was asked to deploy its competencies for the development of a training program that could be adapted to the delicate ICT company background. Grifo Multimedia appeared to be the best possible company to fulfill the training needs, mainly because of its good reputation in the area. In addition, this company has its headquarters in the same city where the employees in training will be relocated at the end of the training program and this made the trainees perceive them as reliable in the future given the physical proximity. Grifo Multimedia was responsible for the designing, planning, and delivering of the training course. According to the ICT company’s managers, the goal of the training program was to update the employees’ technical skills and soft skills. The purpose of re-skilling employees was to set them up to successfully fill the roles of developers and testers in the company.

No ethical guidelines were established during the contractualization of the training intervention between the training company and the commissioning ICT company, although a non-competitiveness clause was included in the contractual agreement. However, personal data were processed throughout the research, in compliance with current privacy protection regulations.

3.4 Participants

The beneficiaries of the training program were 18 workers aged between 52 and 62 (mean age of 55,7 years; Dev St = 2,7), 10 of them were men and eight were women. Workers lived in three cities in the North, Center and South of Italy: 2 in Milan, 10 in Rome, 6 in Naples. About their background, 6 of the trainees graduated in a technical institute, 9 have a high school diploma, two graduated with a bachelor’s degree, and one received a master degree. According to the managers and the labor union’s reports, none of the employees was involved in a company project at the time of the training, and all of them reported dissatisfaction towards the company and previous training attempts. Further details about participants were gathered and analyzed in a following phase of the research and are therefore reported within the context of the case.

The inclusion criteria for participants was based on two aspects: (i) Their willingness to participate in the training; (ii) The company offered them the opportunity to attend professional training to be professionally re-located.

3.5 Training Design

When planning the training, the main objective taken into account was to motivate and engage all the trainees all along the training program. To accomplish such a goal, the training team had to keep trainees involved in the decision-making process and customization for the entire duration of the training program. The ultimate goal of the training company was to have all trainees successfully pass the course and to see them efficiently relocated by their company. Learning objectives were defined in details. Learning objectives can be understood as precise descriptions of actions that learners should perform as a training result and that can be assessed during or after training (Chatterjee & Corral, 2017 , p.1). Coherently, learning objectives were defined:

During the course, trainees should demonstrably gain knowledge on the Agile-Scrum and DevOps framework, soft skills in the workplace, Micro-services architecture, Gitlab’s and Mia’s platforms, software testing, software development and service desk;

During the course, trainees should be fully engaged and satisfied with the training contents and methods;

Trainees should be able to build a support network to empower them during training

Teachers were required to adhere to the learning objectives and training model to the best of their ability. In order to do so, teachers conducted their lessons using a variety of teaching methods. Teachers conducted frontal lessons proposing anchoring ideas, following the pedagogical approach proposed by Ausubel ( 1970 ). In more details, teachers investigated the baseline knowledge and ideas of trainees through the data obtained during the pre-training stage; then, teachers during the lessons anchored the new concepts to previous knowledge, while highlighting the relevance of the new concepts. In addition, teachers often proposed case studies to further associate new knowledge to relatable events that might happen in a work setting (Schank, 1990 ). The case-studies material was posted online and discussed face-to-face. Finally, group activities, both online and in presence, were organized following the pedagogical principles of collaborative learning applied also online (McAlpine, 2000 ). The group activities were also designed to foster a sense of community among the participants (Harrison & West, 2014 ).

During preliminary meetings, the training team defined a basic structure of the training program. The training would consist of four steps:

Information gathering about participants and training needs, with the purpose of organizing training customization;

Entrance test, which would test baseline knowledge of the training’s topics;

Lessons with teachers and workshops with tutors, and a second mid-training administration of the general knowledge test;

Final general knowledge test, which would test learning outcomes of training.

Such a design was intended as just a general outline of training, that would be re-worked and enriched by the data gathered during the pre-training and the actual training phases.

It was required to end the whole training within four months; therefore, the course took place from September to December 2022. It included 10 training modules, totalizing a duration of 428 h of training. Approximately 41% of instructional sessions were conducted in-presence, while the remaining portion occurred online. The scheduling of the sessions attempted to balance in-presence and online training by proposing an agreed rotation of the two modalities ensuring, in this way, equal opportunities to experience both training methods while accommodating trainees’ availability. Commuting limitations and individual circumstances of trainees were considered during training to further accommodate trainees’ needs. The training hours were distributed across the training modules as follows:

M-1, “Aims and introduction”, 20 h, all in-presence;

M-2, “AGILE-SCRUM framework”, 32 h, of which 16 in-presence;

M-3, “Fundamentals of DevOps”, 60 h, of which 20 in-presence;

M-4, “Soft skills in the organization”, 40 h, of which 20 in-presence;

M-5, “Architecture of Micro-services and containers”, 60 h, of which 16 in-presence;

M-6, “Git-labs’ platform”, 20 h, all in-presence;

M-7, “Mia Platform’s fundamentals”, 16 h, all in-presence;

M-8, “Continuous Testing & Deployment in DevOps”, 40 h, of which 16 in-presence;

M-9, “Java Back-end Development”, 100 h, of which 16 in-presence;

M-10, “Service desk in DevOps”, 40 h, of which 16 in-presence.

3.6 The Measures

At each phase of the training, multiple instruments of measurement were used by the training team to assess attitudes, baseline skills, learning outcomes and training satisfaction. Table 1 presents a summary of used measurement tools, which will be further described within the relevant context of the case description. The curriculum vitae analysis was the only data scrutinized anonymously, as all documents were provided by the participants’ company, which requested anonymity for its employees’ documents. All other instruments clearly associated participants’ identity with the results. In this article we will continue to refer to participants anonymously even if the identity of the participants is known to the authors. All the tests and questionnaires were administered and scored automatically through the quiz and survey tools in thee-learning platform of the course.

3.7 Data Analysis

Given the limited number of our participants, we opted for a qualitative-quantitative case study. We have segmented the timing of the course into three different moments—pre-training, during training and post-training—and we used all the data available to track down the participation processes and effects on the trainees by considering these moments as benchmarkers. In this way, we can easily answer our research question—What are the effects of early customization and constant flexibility in a training program meant to relocate professionals?- because we can compare different moments and understand what actually happened during the training.

Although there is no single definition of what exactly a case study is, in general, this method is suitable whenever the aim is to study in depth the performance of a person, a group of people or an institution/agency over a certain period of time (Heale & Twycross, 2018 ). The main aim is not to simply describe a phenomenon but to understand why it went in that way, considering the specific circumstances of the case. In this way, indications to improve that situation can be gathered and generalized limited to similar cases. When this method is used, usually the data is collected in natural settings. By combining qualitative or quantitative datasets.

about the phenomenon, it is possible to gain a more in-depth insight into the phenomenon than would be obtained using only one type of data.

The intention of this case study is to test an early customized training model and consider the effects training customization can have on trainees, particularly those with a negative predisposition toward training programs due to failing previous experience.

Three researchers were involved in the data analysis: one of them—the first author of this paper—has prepared the selection of the data and their processing. The quantitative data was processed through simple analysis—frequency and average—and, when possible, with more sophisticated tests to verify the significance of the distribution. When analyzing the qualitative data, coding schemes and results were always discussed with the other two researchers. In building the qualitative categories, the Grounded Theory was employed as a method to develop ‘theory from data systematically obtained from social research’ (Glaser and Strauss, 2017 ). Grounded Theory consists of “systematic, yet flexible guidelines for collecting and analyzing qualitative data to construct theories ‘grounded’ in the data themselves’’ (Charmaz, 2006 ; p. 2). In essence, Grounded Theory allows researchers to continuously refine their theory as new data is collected and analyzed, securing a context-aware, interactive, and iterative approach to research.

In presenting the case study, we focused on and compared three specific moments of the training course: the pre-training phase when the program was conceived and prepared; the actual training phase during which the course was delivered to the trainees; finally, the post-training phase concerning the assessment of the learning and participation outcomes. Each of these phases will be illustrated in detail in the following section. More detailed information about the methodology used will be given at each phase, since each of them required some specificity.

3.8 Pre-training Phase

As a first step, Grifo Multimedia created the training team, consisting of a training manager and two training tutors, among which the first author of the present study was included. The training team analyzed the training demand coming from the ICT company management. As requested by the ICT company management, the course would have to mainly be held remotely and include several subjects’ areas: Agile-Scrum framework for organization, DevOps organizational model, soft skills for teamwork, microservices architecture and containers, Gitlab platform, Mia platform, Java back-end development, Continuous Testing and Deployment in DevOps. It was established that all these topics would be divided in separate training modules over the span of four months. Each training module needed to be self-contained and independent from the other modules.

Before the training design phase started, two main priorities were set, both of equal importance, to address requirements from the contextual setting and to conduct an effective training:

Fostering the participant’s collaboration with the training company and nurturing engagement for training;

Succeeding in gathering information useful for design training customization.

These priorities were intertwined and circular in their cause-effect relationship. Without effective collaboration and communication between the training company and the trainees, there would be no chance of early customization; at the same time, the interest displayed in the trainees’ needs had the potential to boost engagement and better collaboration and communication between the parties involved.

Soon the team realized that to fulfill these priorities a customization of the course was needed at a very early stage and a few steps were designed to this aim. Firstly, the training team analyzed the curriculum vitae of the training participants, which were provided by the ICT company. The provided curriculum vitae were entirely anonymous, by the will of the labor union who sought after the privacy for participants. Data about age, ICT skills and last position held were extracted from the CVs and analyzed. Information about technical tools and platforms were analyzed and categorized with the help of three technical professionals in the software testing and development field. Figure 1 represents professional background and Fig. 2 summarizes the ITC skills reported at the time of the CV analysis.

Employees’ professional background

Employees’ ICT skills

Employees’ professional background and ICT skills were considered during the training design. In their CVs, participants were allowed to provide information on more than one ICT skill, for the purpose of providing a comprehensive outlook on their background skills. Since many of the participants reported skills in basic computer programs (such as Microsoft Office and Database software) and obsolete programming languages (Cobol, JCL, Pascal, Clipper etc.), the training team planned to recruit experienced trainers who could lay the foundational concepts of the desired subjects for the course. However, this information was not sufficient to fully customize the training design.

To gather information about the participants’ needs and characteristics, the training team conducted face-to-face interviews with each participant. As the available curriculum vitae did not provide names for the participants, interviews were crucial to match participants with their skills and experiences. Interviews were conducted by two members of the training team in a quiet room at the ICT company office. One member of the training team had the interviewer role, while the other one presented Grifo Multimedia to the participants. The interview dealt with several topics:

Main activities conducted during the past 2 years of workplace inactivity;

The soft skills included into eLene4work project—a soft skills inventory whose aim is to create an open access tool, to help self-assess digital, interpersonal, cognitive and emotive soft skills—including the learning-to-learn soft skill, the digital problem solving soft skill and the organizational change processing soft skill (Cinque, 2017 ). The short self-assessment test consists of self-report sentences, with a four-steps Likert scale. The goal of the instrument is not to provide an objective measure of soft skills, but rather to encourage self-reflection and skills improvement. This tool was described by authors as a self-assessment questionnaire encouraging respondents’ proactivity, reflection on motivation, re-framing one’s professional experience and programming of one’s learning process. The learning-to-learn skill is defined as a learning specific meta-cognitive skill. Digital problem-solving sub-scale measures the ability to rely on online resources and new technologies to find solutions and reach one’s goals. The organizational change processing scale evaluates the ability to cope with and possibly cherish change in one’s organization;

Willingness to take part in a remote training course, expressed in three answering options: not at all; yes, under certain conditions; yes, under any conditions.

Figure 3 summarizes the main activities performed by the trainees from 2020 to 2022.

Main activities from 2020 to 2022

Participants were allowed to specify more than one activity, for the purpose of providing a comprehensive outlook on their background. As reported by management, the majority of participants confirmed that their work performance during the last two years was impaired by inactivity and/or work activities that were not coherent with their employee’s job profile. This result certainly helped researchers define the complexity of the context referred to in the research question. Participants had undergone a demotion in their work activities and, consequently, they had to cope with the lack of motivation in their job.

The content of interviews was analyzed by two separate coders. The coders looked at the data to single out the relevant excerpts, according to the research question. The coders went through several cycles of reading. During the first cycle, both coders looked at the interviews integrally. The coders confronted and discussed the divergent cases (about 10%) till a total agreement was reached. The selection of excerpts was better attuned during the second and third cycle, when the totality of the data was analyzed. The final selection was considered by both coders as well finalized in offering insights for our research question. The following excerpts are the result of this selection.

A significant example of these altered activities is the case of Jacob Footnote 2 who during the interview stated: “ For the last 10 years I worked autonomously as a second level analyst, on the mainframe for the Ministry of Education. I enjoyed being independent because it was rewarding. After my branch was acquired by the company, I mainly helped my colleagues with their access to the Cobble software ”. When asked for clarification, Jacob explained how access management for Cobble was a much more sporadic and support-adjacent role, while his precedent job entailed responsibility for an entire project and relationship with the client. A role change of this kind can be described as demotivating and affected Jacob’s sense of competence and autonomy. Jacob reported that his work was dull and unfulfilling. Another noteworthy interview result is about the failure of the previous training course—called Salesforce, mentioned in the Context section—they attended. When asked their opinion on why this training attempt failed, participants reported several factors:

The course was entirely in English. This represented a major obstacle because most of the participants only have a basic understanding of English;

At the beginning of the course, participants had the support of several training tutors. These tutors would help trainees to understand difficult topics, gather additional study material and translate some of the course content. However, tutors gradually withdrew their support and soon enough the participants felt abandoned and alone in facing the most difficult parts of the course;

Baseline skills were reportedly not sufficient to help understand the complex course content. Participants felt like they did not have enough information to grasp CRM applications.

As reported during the interviews, participants felt that the course had no clear purpose from an organizational perspective. Managers did not communicate aims and future projects in which they could be involved after the training. Consequently, participants felt as if they were required to commit to a pointless, difficult course, with no managerial support. This perception further fueled the sense of neglect for the company’s part.

Table 2 reports the results of the soft skill self-assessment tool.

Scores for the learning-to-learn skill show that participants believe to have a fair understanding of their own learning processes and preferences. We can better understand their level of understanding by considering also the statements registered during the interview. Rosy, for instance, stated: “ I think I know now well how to organize my learning strategies, but I recognize that my learning process might take a long time ”. Lucas recounted that he prefers to learn in a group, since he enjoys exchanging ideas and thinks that communication is an effective tool for learning. Furthermore, Andrew stated “ I’m not drawn to learning new technologies through theoretical study. I prefer learning through work, collaborating with my coworkers ”.

Scores for digital problem solving and organizational change processing can help highlight participants’ attitude towards digital innovations and towards the organization. These scales elicited reflection about participants’ own skills and organizational context. Regarding digital problem solving, Andrew reported: “I can’t always solve work problems on my own, because I’m not up to date with the latest technologies. Oftentimes, I don’t know how to find answers. My skills are only updated through training when I have to be included into a work project ”. During the interviews, six of the participants mentioned they did not feel knowledgeable about the latest technologies. In the matter of organizational change processing, several participants found it hard to answer the test because, as they stated, “ There is no organization to speak about ”. When asked to clarify this statement, Frank explained he did not have a point of reference for the organization, since he feels like he and his group are isolated from the company. All the answers provided by participants were crucial to define the context of the training, which represents a fundamental portion of the research question.

Finally, participants were asked if they were comfortable with a remote training program. Whilst none of the participants was opposed to remote training, most of them (83,3%) were in favor of it. A few participants (16.7%), however, consented to remote training only under certain conditions. When prompted, participants specified what circumstances needed to be provided:

Useful and functional tools, like an adequate presentation platform and a e-learning tool;

Reciprocal understanding and trust between participants and training team;

Collaboration among the participants, meaning fostering group learning and cooperation during training;

Rotation of remote and in-presence lessons and meetings.

These indications were especially useful during the designing and planning of the training model. Overall, the face-to-face interviews had a major impact on the customization of the course, which will be described towards the end of the current section.

After all the information gathered during interviews were analyzed, it became evident how participants valued group learning and collaboration with their colleagues. The training team concluded that the best way to deliver the program was the blended mode. In this way, group cohesion and collaboration could be better enhanced by exploiting the advantages of combining face-to-face and online encounters. Finding ways to interconnect moments of work in presence with those at a distance would certainly help all participants (trainers and trainees) to create a learning community capable of overcoming critical issues and achieving the aims set. Equally important was the matter of guaranteeing minimal commute stress during training, which was not a trivial task considering location of participants, as well as their personal and familiar lives. Finally, it was decided that training had to be preceded by kick-off encounters with the complete training team and all the participants.

The training team programmed a three-day meeting, where several topics would be discussed: interviews outcomes, a broader presentation of Grifo Multimedia and the training team, expectations about training, course contents and topics, attitudes towards training and organizational context, training on the digital tools that would be used during training, future working projects planned for the participants by the organization. Activities included during this training introduction aimed to give feedback about the interviews and how they would help design the training. As feedback is fundamental to learning engagement (Winstone et al., 2017 ), it was decided to constantly keep participants informed about the results of every test and questionnaire trainees were required to fill in. At the same time, there was a clear commitment in collecting continuous feedback, comments and suggestions about the training. To this aim, moments of exchange and communication among the training group members and between the trainees and the team responsible for delivering training were considered particularly relevant. The general aim was to create a collaborative learning environment for all parties involved.

Firstly, the training team introduced themselves and the company they work for (Grifo Multimedia). The training manager described the company’s most prominent activities and specialization. Then, the training team explained how tutors would be involved during training. Tutors would support training activities and access to digital tools and platforms, in addition there will always be an open channel for communication with teachers and training managers. Next, tutors presented results of the face-to-face interviews, showing them the same graphs we included in this paper (Figs. 1 , 2 and 3 ). Participants had the chance to pose any question or observation about results, which were promptly addressed by the training team. The training team announced that the training course will be delivered through a blended learning approach.

Afterwards, the training team presented an online board on Padlet, Footnote 3 named “My training expectations”. Participants could connect with the online board using their personal devices and add an anonymous note that answered the question “What are my expectations for the course about to begin?”. The whole activity was conducted by two tutors. While Fig. 4 shows the original board, Fig. 5 presents the translated content.

“My expectation about the course” results

“My expectations about the course”. Padlet translation

Participants’ contributions to the board were examined by the same two coders that examined the interviews. Again, the coders went through several cycles of reading. During the first cycle, coders read about 20% of data. The rest of the procedure followed the same steps as the qualitative analysis of the interviews.

Hearts below statements were added by participants, signaling their agreement with the statement. By reading this Padlet, tutors understood the importance participants attributed to learning and self-improvement. In addition, participants emphasized the importance of feeling supported during the training process, further confirming what emerged during face-to-face interviews. Curiosity, enthusiasm, and optimism were accompanied by caution caused by the novelty of the course. In addition, it seems they all agree about the importance of useful and well-thought-out study materials and training tools. This consideration is connected to their previous negative experience with training.

The next day, participants were divided into three groups of six to join a focus group with the training team. Two tutors and the training manager were assigned to each group. Every group met in a separate room for 30 min. The topic discussed during the focus group was “What is the group point of view about the training course about to start?”. Tutors and training managers had a facilitator role, which included: presenting the topic, explaining the topic if necessary, balancing the duration of each speech turn so that everyone could contribute, ensuring that everyone had at least one speech turn, mediating disagreements, and taking notes of the discussion. The analysis of the focus group discussions were conducted through the same analysis process of the interviews, previously described. Below are the summaries of the three group discussions:

Group 1 is composed of Jacob, Andrew, Alexander, Fern, Elias and Lucas. They expressed worries about measuring up to the difficulty of the training. They wanted to rise to the occasion, and wondered if maybe they were too late to develop their skills. However, participants expressed trust in their chance to advance. They recognized their current condition is better than two months ago and it could even improve with the right support and a good amount of practice. A certain amount of grit would be needed from their side. The issue about being not so young any more is discussed and they reckoned that youth boldness is counterbalanced by their being experienced. The prior two years of standstill fueled focus and the value of hard work motivated the participants to a new re-start. Not having a clear perception of their future was perceived as walking in the dark or like being asked to get back to first grade, and that studying new topics feels like invalidating their already long career.

Group 2 is composed by Max, Sandra, Dominique, Charlie, Pierce and Matthew. This group agreed that the course seemed well structured so far. For some of them, a blended learning course is a novelty and they hoped that such a method could be maintained during the whole training. Willingness to experiment new things is expressed and the novelty of the course is recognized as a stimulating challenge. The group focuses on the topic of their perceived professional expertise claiming that it could be a useful resource that defines their professional life and that the company did not recognize it as a resource. Charlie stated that their knowledge and experiential baggage could be restrictive if it prevented them from learning new things. Pierce added that the Project Manager role serves to polish the team's rough edges and to help leverage team members’ professional experiences.

Group 3 is composed by Rosy, Frank, Ronald, Dan, Steve, and Robert. The group worried about the difficulty of the topics the training proposed. Nonetheless, they all wanted to prove to be able to grasp new concepts. The need to not feel isolated and confused is remarked together with the intention to prove they are not obsolete workers. This group feels this is an opportunity not to be missed but it ought to be adequately accompanied all along the training. The radical difference from previous attempts of training is recognized and this could help the group in regaining trust in training.

After the focus groups’ discussions ended, the three groups were dissolved and all the following activities involved the cohort of trainees in its entirety.

Based on the focus group results, a new activity was proposed to gather positive attitudes towards participants’ professional background and their future perspective. This activity consisted in brainstorming answers to fill out a 3 × 3 table (see Table 3 ).

Entries were categorized in attitudes regarding past, present and future, and attitudes towards themselves, the training group, and the company. This brainstorming had multiple aims. Firstly, to identify past experiences and perceptions, help participants elaborate the training and professional distress they experienced, and collectively overcome their training and professional past to allow themselves to start anew. Secondly, to gather data about how their present condition and the communication with the training team are perceived. Thirdly, to detect hopes and expectations for the future. All this information would be used for training customization. The activity was conducted by the first author of this article. The following rules were set: participants would first verbally report their ideas to the conductor indicating where to place them in the table; the conductor would listen and write down the participants’ ideas; participants can comment on the written text and add more content even if they did not unanimously agree on it. Figure 6 shows a photo of the original brainstorming, while Fig. 7 translates the content.

Photo of the original brainstorming table

Translation of brainstorming table

Participants were allowed to situate some of the contributions between categories, if needed. For instance, the contributions “Effort” and “Work as a team” were placed over the group category because, in the participants’ opinion, these contributions described the group in every moment. The analysis of this content was conducted through the same analysis process for qualitative data as previously described.

As the brainstorming strived for open communication, it is interesting to notice how this activity assisted in revealing all the negative attitudes and emotions linked to previous work conditions and training. Nonetheless, participants chose to describe the group as a team that always strived for hard work, despite all the distress they experienced. This proactive attitude reflects on the “now” category, where the prevalent attitude is characterized by motivation, eagerness and willingness to get involved in training. Contributions like “being supported”, “cohesion” and “being recognized” helped the training team understand how much open communication and feedback gathering would be important during the training process. Training customization would need to include a system to support participants and a feedback-based recursive training model. Finally, it seems important to point out how participants valued hard work, dignity and satisfaction with one’s work. The importance placed on hard work and challenge allowed to program an intensive training schedule which fitted with management’s requests for a short course.

During the last day of pre-training activities, training participants were instructed about the e-learning platform that would support training. Each of the training modules included specific contents and goals. Accessing the calendar area of the platform would show lessons agenda, alerts regarding the course, updated learning contents, and personal messages. Calendar included lessons held by teachers and workshops supported by tutors, indicating the modes—in-presence or remote—and the dates. In each learning module, participants would find links for the zoom class, the power point presentations used by teachers, exercises, tests and more in-depth materials shared by tutors, teachers and by the training participants themselves.

As the last pre-training activity, participants were tested on all the topics of the training program to assess their previous knowledge and obtain a baseline score to be compared to the course outcomes. The test was created with the help of all the teachers involved in training. They generated a multiple-choice test that represented all the topics of the course: AGILE-SCRUM framework; fundamental issues in DevOps; soft skills in the organization; architecture of micro-services and containers; git-labs’ platform; Mia Platform;, continuous testing and deployment in DevOps; Java back-end development; service desk in DevOps. The set of questions composing the tests covered basic information about the topics that would be extensively explored during the course. The test was a multiple-choice test with three-answers options, among which only one answer was the correct one. The test results were transformed into quantitative data by assigning to each correct answer a value of 1, and a value of 0 to all incorrect answers. Figure 8 reports the results of this general knowledge test.

Pre-course entrance general knowledge test results for every participant. Score ranged from 0 to 26

Only two participants, Pierce and Fern, out of 18 passed the first iteration of the pre-course entrance general knowledge test, reaching or surpassing the minimum score of 16 out of 26. Based on this result, the training team concluded that participants had little prior knowledge about the topics of the course that was about to start. Participants were informed about their score, but they did not know which one of their answers were correct or wrong. All the information gathered during the pre-training phase contributed to adapt the subsequent training phase and as it will be further explored and illustrated in Fig. 9 , the pre-training phase helped in understanding the complex context and in addressing the early customization requirement, postulated in our research question.

Representation of customization model

3.9 Training Phase

Before initiating the training, the information collected was analyzed to properly customize the training program. Decisions were made on the following matters:

Teachers. Data was used to define the fitting teachers’ profile. That is an experienced professional, the same age of participants, to overcome participants’ anxiety concerning young professionals. In addition, the teachers should be able to explain complex concepts by practical evidence based on both their own work life and that of the participants. Finally, teachers should be available for continuous dialogue with participants and should be flexible enough to adapt to rising needs.

Training team periodical meetings. The training team, including tutors and training manager would need to hold fine-tuning meetings with teachers to firstly present the training context and, later, to monitor the course evolution and negotiate changes or add-ons.

Continuous feedback. The training team would need to constantly be aware of the mood of the groups and their attitudes towards the training to promptly address issues that may impair the training success. This purpose would be achieved by frequent check-ins, where tutors would ask how the course is going on and what could be done to further support learning.

Training schedule. In defining the schedule, the participants’ work commitment is considered so as to accomplish the learning goals within the three months allocated to the training. Participants showcased eagerness to dive into a new training challenge and spend a great deal of effort in learning, if it means marking a turning point in their career.

Data collection. Via participants’ feedback, tests and questionnaires, the training team could monitor participants’ progress throughout the course and adjust if necessary.

Training content. All the provided materials are in Italian, the participants’ native language. In addition, teachers provided exercises, tests, simulations, case studies and explanatory real-life examples, as requested by the participants.

Training community. The pre-training process made evident how participants valued teamwork and collaboration. To meet this propensity, group activities and workshops were scheduled. In addition, the training team built a learning community that would support every participant. Groups were encouraged to exchange insights about the course, share additional study material, solve doubts by exchanging lesson notes and discussing them. Participants felt they could rely on a support network that also included the colleagues on which they can count on even when the training course will be over. This addressed the feeling of being isolated and neglected, participants reported during the pre-training phase.

These features allowed participants to play an active role in determining training features and changes in the training program. Figure 9 represents the training customization model, while Fig. 10 shows the structure of each training module.

Representation of module structure

The same module structure was repeated for each module. The entire course featured 10 modules, including the pre-training module and the final post-training module. Before each module starts, the training team meets the module's teacher and the team presents the participants’ background. Additionally, the training team would describe contents of previous modules to the new module’s teacher, to avoid content repetition.

A usual day of training would include a four-hours lesson with the module’s teacher and a four-hours workshop with tutors. During workshops, tutors presented exercises and tests provided by the module’s teacher, group work designed by the module’s teacher and the tutors, autonomous study and research from participants. At the beginning of each workshop, tutors would ask participants how the training was going. When participants notified tutors about specific needs or made suggestions, tutors would revise workshop activities for the day or forward to the training managers and teacher. For instance, during the second module, participants told tutors they were feeling overwhelmed by the quantity of new words in the Agile-Scrum technical jargon. To solve this issue, tutors suspended programmed activities for the day and proposed a group construction of an Agile-Scrum glossary. Participants were divided in four groups, and each of the groups studied a section of the training program. Then, participants wrote down in a shared document every new term they encountered and its definition. Finally, tutors compared every group doc file, deleted repetitions and built a complete Agile-Scrum glossary that participants could consult when studying. The final list was the result of collaboration between participants and tutor, and it proved to be a valuable resource during the whole course. In another instance, during the fifth module, participants reported they were finding some difficulties studying the course's content. To be precise, the group struggled with figuring out applications of the Microservices technology and felt that the course was too theoretical. The tutor reported this struggle to the rest of the training team, which negotiated with the teacher a more practical approach to the course's content. This negotiation resulted in a change of the subsequent lesson into a Q&A session that would help solve misunderstandings and doubts, plus the teacher presented a case study that would depict practical applications of Microservices technology. From that day forward, the teacher re-tuned his lessons to always include real-life examples of the technology application.

As stated, every module was followed by a satisfaction questionnaire and a final module-specific test. Satisfaction questionnaire would ask to evaluate the module on a scale from 1 to 5. Questionnaire required to assign a score to the module's content, method of delivery, perception of teacher’s knowledge, perception of teacher’s empathy and helpfulness and module’s overall organization. In addition, participants were free to add suggestions and comments. Figure 11 represents mean scores of every module’s satisfaction questionnaire.

Results of training satisfaction questionnaire for every module. Satisfaction ranged from 0 to 5

Comments and suggestions were particularly useful for training customization. For instance, after the Agile-Scrum Module, some participants suggested adding some time of individual study during workshops with tutors. The purpose of individual study would be to allow participants to review the morning’s lesson content and be prepared for group assignments during workshops. During the following workshop, tutors and participants negotiated time for individual study, and finally agreed on one hour per session of individual study.

Although the trend does not appear to be regularly increasing, we can observe how in the first four modules the score remains at a good level. The score dramatically drops at the Microservices module. When asked for feedback about that module, participants stated that it was too technical, therefore hard to grasp for beginners. In addition, participants felt that the study material provided was insufficient and unclear. The subsequent module had the same teacher, so he was asked to provide more explanatory content and exercises. Thanks to the teacher’s collaboration, the next module had a rising score. However, the last four scores do not reach the same level of satisfaction of the first four. When asked for feedback, participants stated that the reason for this decline is due to the more technical content of the last four modules.

Figure 12 presents module-specific test scores for each module’s test. They were all generated by the teacher responsible for the module and these questions did not overlap with those included in the general knowledge test. The tests were all multiple-choice tests with three-answers options, with only one correct answer.

Module-specific tests scores for every module. Score ranged from 0 to 15

The first and last module did not have a module-specific test because they did not involve theoretical or technical content. Considering module-specific test scores, on average participants reached the passing grade of 9 out of 15 for every module. This goal was reached even in the case of the least enjoyed modules.

Halfway through training, the general knowledge test was repeated to monitor changes in knowledge. The general knowledge test was repeated after a month and a half of training. As stated in the pre-training phase section, the test was considered passed with a score of at least 16 out of 26. Figure 13 compares learning results in the first and second administration.

Test score comparison between first and second general knowledge test administration. Score ranged from 0 to 26

Overall, participants obtained a higher score the second time they answered the test. The highest percent of score improvement was attained by Charlie, with a score improvement of 65.38%. All participants but one passed the test. The lowest score improvement was found to be Fern’s, who obtained the same score as the pre-course test. Fern obtained one of the highest scores in the pre-training administration of the test and did not improve her score in the following administration. This could happen for many reasons. Fern answered the first administration casually and incidentally reached a high score or she actually did not improve her knowledge; it is also possible that for her the course was too intensive and she did not have enough time to transform the content of the course into a consolidated knowledge. At this point of the training, it was decided to address this point by persevering with a flexible model of training. Since at this moment trainees only experienced half of the training course, the percentage of passing scores was deemed coherent with the stage of the training. After the second administration of the test, the course continued for another month and a half, with the same structure as described in Fig. 10 , the same customization model represented in Fig. 9 , and a continuous feedback loop between the training group and the training team.

The continuous adaptation process described in the present section addressed the constant flexibility of the training program implicit in our research question. Some results regarding the outcomes of the training program are already anticipated in this section, in the form of the mid-training test.

After exploring the “customization during training”, the following section will deal with the final phase of the training program and try to provide a definitive answer about the effects of the training, as required by the research question.

3.9.1 Post-training Phase

The post-training phase focused on gathering information about the outcomes of the training program. In particular, the considered outcomes included: (i) the trainees’ change of perspective about themselves, (ii) their experience with the training and their work life, (iii) their perception about their skills and (iv) their knowledge improvement.

After the course was over, participants met with the training manager for the purpose of having feedback over each training module. For every module, participants described positive and negative aspects about the training experience. This meeting provided initial elements about the effects of the training program, as by our research question. The meeting results can be summarized as such:

Positive features. According to participants, there was a change in the group’s self-awareness. The group felt more competent and resourceful, more aware of their past and future. Studying DevOps methodology taught them collaboration and sharing responsibility within the work team. Moreover, participants felt that they had a clearer understanding of the difference between soft and hard skills, and they felt more aware of communication strategies and emotional intelligence. In addition, participants enjoyed how the course was tailored to their needs: participants felt that the amount of communication with the training team helped fine tuning during the training. Discovering new tools and technologies was a valued feature of the training, and practice activities were especially useful. Participants felt supported by many of the teachers, and appreciated the differences between each teacher, who kept the course from becoming monotonous.

Negative features. According to participants, they preferred to have more lessons face-to-face rather than remote lessons. In addition, participants felt that some of the modules’ contents were too advanced for their knowledge and skills. In the instance of the Microservices training module they did not feel supported by the teacher, which caused the lowest satisfaction rate (Fig. 11 ). Participants felt that in some modules practice activities were not frequent enough, and the module resulted to be too theoretical. Finally, participants felt that blended training needed to skew more in favor of face-to-face training, since they thought that cohabiting the same physical space enhanced group collaboration.

Next, once again a brainstorming was conducted, as described in the Pre-training phase section. The purpose of the second brainstorming was firstly to detect changes in attitudes and emotions regarding training and career, and secondly, to track the effects of training customization on participants. Figure 14 shows the brainstorming in Italian, while Fig. 15 reports the content translated.

Brainstorming table in original language

Brainstorming table translation

The analysis of the table content was conducted through the same analysis process for qualitative data as previously described.

Contributions placed into the “in the past” row, mostly repeat what was stated in the first brainstorming table administration (see Fig. 7 ). However, the reference to workers’ rights and the search for new jobs is a substantial change in the contributions. Comparing results to the first administration, this proactive attitude only emerged in the group setting, while after training this attitude seems to also surface in the individual setting. This change can be explained as a shift in self-perception after training, perhaps caused by experiences of competence and self-efficacy. Participants may have retroactively reinterpreted their past experiences in accordance to more recent experiences, a phenomenon known for a long time as retroactive interference (Osgood, 1948 ). In addition, participants felt more empowered, self-aware and competent than in the past. The group reported feelings of being a part of a community. However, the current perspective towards the company did not seem to be better than the previous one even if participants felt enriched by the training. As for the “in the future” row, participants’ contributions focused on self-awareness, trust in their own skills, need for more practical skills and the appreciation for travels. In Fig. 10 , pictures represent landmarks and typical foods of Apulia, the region where the new company’s branch-office would be opened. This feature could reveal some excitement or curiosity about the opportunity connected to the opening of a new office located in that region. This enthusiastic perspective is detected also in the company cell of the future row: participants highlighted an ambitious future. The effects of the training, in summary, seemed to be a newly found sense of hope, self-efficacy and trust in the community, which is a further hint to compose an answer to our research question.

Finally, participants were asked to answer, for the last time, the general knowledge test. Figure 16 shows the comparison among the three test’s scores administrations.

Score comparison of the three general knowledge test administrations. Score ranged from 0 to 26

While 33.33% experienced a decline in their scores in the third test administration (Fig. 16 ), 66.66% of participants demonstrated a general improvement. Only one participant, Rosy, received a lower score at the third general knowledge test administration when compared to the first one. This result could be due to many different factors: overload with training; lack of understanding of the content; carelessness in answering the test. To transform this result into a constructive experience for Rosy, she received personalized feedback and she could review her answers’ pattern. Then, the training team organized an individual session with her to answer her questions and doubts. Overall, the mean score at the final administration was 22.22 points, compared to 20.27 points at the second administration and 12.27 points at the first one.

Scores for the three general knowledge test administrations were further analyzed to determine statistical significance. Analysis was conducted using the software rStudio. Normal distribution of the sample was proven by the Shapiro–Wilk normality test. The general knowledge test scores were normally distributed for W = 0.917, p < 0.001. The absence of outliers and sphericity were checked during analysis. The Repeated Measures ANOVA method was chosen for its ability to detect differences in means of multiple related samples, for example, in the case of a test that is repeated by the same group more than two times. The Repeated Measures ANOVA is often used to test scores differences in within-subjects designs (Lavori, 1990 ), which is exactly the case of the present study. To each phase of the program—pre, during, post—the general knowledge tests scores were significantly different (F(2, 34) = 39.29, p < 0.01, eta2[g] = 0.58). This means, at least one test score was significantly different from at least one of the other test scores. To identify which test administration was responsible for the statistical significance, multiple pairwise pair t-tests were performed as a post-hoc analysis of the Repeated Measures ANOVA. P-values were adjusted using the Bonferroni correction method. The pairwise pair t-tests detected that the difference between the first and second test administration was significant ( p < 0.05). The difference between the first and third test administration was also significant ( p < 0.05), while the difference between the second and third test administration was not significant. This result could be because the third iteration of the test happened between the seventh and eighth training module, when the majority of the training already happened.

A month after the last training meeting, participants were asked to express their point of view on the delivered training. Specifically, participants evaluated perceived efficacy and enjoyment of all training activities. Table 4 presents perceived efficacy mean scores for each training activity, while Table 5 presents enjoyment mean scores for every training activity.

Overall, face-to-face lessons with a teacher were the most enjoyable and useful activity according to participants. Participants agreed on the value of group communication activities and group activities in general. The latest appreciated activity was remote individual workshop activities. Overall, the course was mostly evaluated in a positive way, with the group interactions and face-to-face settings as the most appreciated activities involved. The fact that these features got the highest efficacy and enjoyment scores fits with the reported needs in the pre-training phase, as group activities and face-to-face encounters were detected as prominent needs. The reason teachers’ face-to-face lessons were the most appreciated could be related to the relationship established between trainer and trainees, which was perceived as less strong during remote lessons.

In addition, since tests provided constant feedback on learning progress, tests were mostly appreciated.

While this section had the goal of describing the effects of the customized training, the answer to the research question will be explored in the next section.

4 Discussion

In this paper, we presented a quali-quantitative case study of professional training concerning trainees with a history of negative previous training experiences. This case study can be also regarded as an example of early customization and full flexibility throughout the course. Therefore, our leading research question is: What are the effects of early customization and constant flexibility in a training program meant to relocate professionals?

To answer this question, we segmented the course in three phases—pre-training, training and post-training—and for each phase we looked at several aspects. For instance, the first two phases were particularly adequate to give information about the efficacy of the training program based on background knowledge, subsequent learning outcomes, trainee’s satisfaction, and trust towards training programs in general. Table 6 summarizes the results obtained at the first two phases. The third phase is not considered in this table but it will be used later to have a wider understanding of the customization effects.

The results of the general knowledge tests, together with the module specific tests’ scores, can provide an insight into the effectiveness of the learning process. Furthermore, the training program was proven to be perceived as relevant, useful and enjoyable, as evidenced by informal feedback, enjoyment questionnaire, satisfaction questionnaire and perceived usefulness questionnaire. This result is quite coherent with other research, where customization has already proved to increase learners’ motivation, perception of a meaningful experience and it facilitates the transfer of learned knowledge and skills to a real-world working environment (Lainema & Nurmi, 2005 ). This allows us to assume that the earlier the customization is provided, the more these results are amplified.

As the most enjoyed and effective training activities were face-to-face lessons and group activities, we can infer two conclusions:

The relationship between trainer/teacher and trainees was particularly effective. This can be seen as a positive outcome of customization particularly in terms of choosing the teachers to fit the trainees’ characteristics and needs;

The preference towards group learning, expressed during the pre-training phase, was confirmed to foretell enjoyment and effectiveness. In other words, favoring group learning and expressing this preference effectively resulted in positive outcomes.

These considerations confirm literature contributions about the effects of training personalization (Dolog et al., 2004 ; Klašnja-Milićević & Ivanovi, 2021 ; Zine et al., 2014 ) and about the positive effects of collaborative learning (Postholm, 2016 ; Ritella et al., 2020 ), especially when groups work under adaptive guidance (Annese et al., 2022 ).

At the same time, the least favored learning activities, such as individual activities and remote teaching, inform us about a crucial flaw of blended learning. As stated by Batalla-Busquets and Pacheco-Bernal ( 2013 ), workers recognize the convenience of e-learning, but find face-to-face training more motivating. In addition, participants may have preferred face-to-face activities to retrieve a sense of normalcy after a long period of isolation. This preference may suggest caution when planning a full-remote training program for a group of trainees that felt isolated for a long while and are approaching their retirement age.

While authors had no reason to believe that trainees were technologically resistant, they might have felt the same alienation from training described by Chyr’s study ( 2017 ). This training alienation might have been caused by the previous training attempts, which took place in an exclusively online format. At the same time, the training team considered the isolation that might arise from online training (Lightner & Lightner-Laws, 2016 ) and addressed it in multiple ways; for instance, by setting up numerous occasions for feedback, by programming in-presence lessons, and by building a learning support system that addressed the barrier to help-seeking (Er et al., 2015 ).

It is worth pointing out that analysis procedures primarily relied on measurements based on ad hoc instruments. While this solution may fit well to the specificity of the case, the use of standardized tests may help reach more generalizable results.

Described outcomes can showcase the notable qualities of customization for a small group of participants in a relatively short amount of time.

We do not consider our results as generalizable, but surely they can be useful in those cases where professional training programs previously failed. The presented model and its results can serve as a framework for trainers and educators dealing with professionals with low motivation and negative feelings. Future educational programs might adopt a similar module-structure, alternating lessons and workshops either online or in-presence; also programmed feedback loop could be implemented involving all the participants involved in the training. The present study suggests that training practitioners might adapt training content to the trainees’ characteristics and gather data about them with the purpose of further fine-tuning the training. Finally, instruments such as the brainstorming table (Fig. 6 , 14 and 15 ), pre-training interviews and focus groups discussions might be helpful tools to explore the training context, its resources and obstacles. The brainstorming table would benefit from being paired with self-efficacy measures. Since the brainstorming table detected some changes in self-efficacy, an in-depth analysis of the changes in self-efficacy before and after training would contribute to a deeper understanding of the brainstorming table content.

Furthermore, presented outcomes could be intended as prompts for broader and more far-reaching customization. For example, enjoyment and perceived usefulness of group workshop activities reinforce the need to add more group activities and strengthen the learning community cohesion to achieve higher learning goals (Deci & Ryan, 2000 ). In addition, enjoyment of face-to-face teachers’ lessons could be an insightful factor to increase face-to-face activities during blended learning programs.

Customization could also benefit from a boost in the feedback system among participants, tutors and teachers, since it was appreciated and deemed useful. Customization can be enhanced via an anonymous feedback system regarding workshop activities and teachers’ lessons. Further research on the topic could improve the described training experience by enabling users to adapt the e-learning platform to better fit their needs. For example, the user may be able to autonomously choose the e-learning platform’s dashboard and interface, evaluation method, or their preferred learning method. Participants could have the chance to choose their preference for individual or group learning activities via a standardized tool in the e-learning platform.

During the pre-training stage, participants self-reported a large amount of information through their CVs, experiences and skills shared during interviews, and the soft skills inventory. This data could potentially be influenced by a desirability bias which, if present, might be viewed in this context as a sign of participants’ willingness to positively engage in the proposed experience.

While repeating the same knowledge test to assess learning improvements in trainees is a standard practice in professional training, future studies might benefit from administering different tests about the same topics. We are aware that repeating the same test three times has effects on trainees’ familiarity with the content of the test. However, the training team organized the general knowledge test in the e-learning platform to be randomized in the questions and answers order, for each participant, for all the instances of the test administration. We rely on the fact that respondents may not remember the answers given in previous administrations given that months have passed in which those concepts were explored in depth. As already proved by Miller and colleagues ( 1993 ), the effect of test repetition over a period of six or more weeks is negligible and does not affect significance of results. Randomization of questions and answers, combined with time passing, leads to the consideration that the effect of the repetition might have a low impact on results. Of course, different versions of a knowledge test should be written so as to be equivalent in difficulty and coverage of the training topics. Data gathered from the knowledge test could be further refined. For example, there might be answers from trainees that repeatedly selected the same answer, or there might be answers that fell outside the range of the training group. We acknowledge that addressing these cases might appreciably impact the results although, given the small size of our group, in our specific case we believe such impact could be low.

Considering the strategies to assess the metacognitive skills—such as the understanding of their own learning strategies—these could be better addressed in future research. Various other approaches can be considered for data gathering, for instance:

Participants' comparative self-perception, including how they assess themselves compared to peers and their perceived improvement over time;

Naturalistic observation of participants during the learning process;

Proposing participants study diaries to record their reflections.

Two limitations should be kept in mind when choosing this kind of training design. First of all, this model is best fit for smaller training groups, since it requires collecting individual feedback and a one-to-one relationship between trainees and training manager, teachers and tutors. Secondly, this customized blended training model could be most suitable for specialized training, namely training programs about technical skills.

Data availability

Data is available upon request to the corresponding author.

https://www.grifomultimedia.it/ .

All names are fictitious to protect privacy.

https://padlet.com

Alamri, H. A., Watson, S., & Watson, W. (2021). Learning technology models that support personalization within blended learning environments in higher education. TechTrends, 65 , 62–78.

Article Google Scholar

Amenduni, F., Annese, S., Candido, V., McLay, K., & Ligorio, M. B. (2021). Blending academic and professional learning in a university course for future E-learning specialists: The perspective of company tutors. Education Sciences, 11 (8), 415.

Amenduni, F., & Ligorio, M. B. (2022). Blended learning and teaching in higher education: An international perspective. Education Sciences, 12 (2), 129.

Annese, S., Amenduni, F., Candido, V., McLay, K. F., & Ligorio, M. B. (2022). Tutor’s role in whatsapp learning groups: A quali-quantitative methodological approach. Frontiers in Psychology, 12 , 799456. https://doi.org/10.3389/fpsyg.2021.799456

Armatas, C., Kwong, T., Chun, C., Spratt, C., Chan, D., & Kwan, J. (2022). Learning analytics for programme review: Evidence, analysis, and action to improve student learning outcomes. Technology, Knowledge and Learning, 27 (2), 461–478. https://doi.org/10.1007/s10758-021-09559-6

Arthur, W., Jr., Bennett, W., Jr., Edens, P. S., & Bell, S. T. (2003). Effectiveness of training in organizations: A meta-analysis of design and evaluation features. Journal of Applied Psychology, 88 (2), 234–245. https://doi.org/10.1037/0021-9010.88.2.234

Ausubel, D. P. (1970). The Use of Ideational Organizers in Science Teaching. Occasional Paper 3

Batalla-Busquets, J. M., & Pacheco-Bernal, C. (2013). On-the-job e-learning: Workers’ attitudes and perceptions. International Review of Research in Open and Distributed Learning, 14 (1), 40–64.

Brooks, C., Greer, J., & Gutwin, C. (2014). The data-assisted approach to building intelligent technology-enhanced learning environments. In: Learning analytics: From research to practice : Springer, New York pp. 123–156

Brusilovsky, P., & Peylo, C. (2003). Adaptive and intelligent web-based educational systems. International Journal of Artificial Intelligence in Education, 13 (2), 159–172.

Google Scholar

Çetinkaya, M. (2016). Design of personalized blended learning environments based on web-assisted modelling in science education. International Journal of Evaluation and Research in Education, 5 (4), 323–330.

Charmaz, K. (2006). Constructing grounded theory: A practical guide through qualitative analysis . Sage

Chatterjee, D., & Corral, J. (2017). How to write well-defined learning objectives. The journal of education in perioperative medicine: JEPM, 19(4).

Chyr, W. L., Shen, P. D., Chiang, Y. C., Lin, J. B., & Tsai, C. W. (2017). Exploring the effects of online academic help-seeking and flipped learning on improving students’ learning. Journal of Educational Technology & Society, 20 (3), 11–23.

Cinque, M. (2017). MOOCs and Soft Skills: a comparison of different courses on Creativity. Journal of E-learning and Knowledge Society , 13(3).

Clarke, N. (2003). The politics of training needs analysis. Journal of Workplace Learning, 15 (4), 141–153.

Deci, E. L., & Ryan, R. M. (2000). The" what" and" why" of goal pursuits: Human needs and the self-determination of behavior. Psychological Inquiry, 11 (4), 227–268.

Dolog, P., Henze, N., Nejdl, W., & Sintek, M. (2004, May). Personalization in distributed e-learning environments. In: Proceedings of the 13th international World Wide Web conference on Alternate track papers & posters (pp. 170–179)

Er, E., Kopcha, T. J., Orey, M., & Dustman, W. (2015). Exploring college students’ online help-seeking behavior in a flipped classroom with a web-based help-seeking tool. Australasian Journal of Educational Technology, 31 (5). https://doi.org/10.14742/ajet.2527

Fake, H., & Dabbagh, N. (2020). Personalized learning within online workforce learning environments: Exploring implementations, obstacles, opportunities, and perspectives of workforce leaders. Tech Know Learn, 25 , 789–809. https://doi.org/10.1007/s10758-020-09441-x

Fayyoumı, A. (2009). The effectiveness of e-learning: Academic and business comparison. Turkish Online Journal of Distance Education, 10 (1), 130–140.

Feng, Z., González, V. A., Mutch, C., Amor, R., Rahouti, A., Baghouz, A., Li, N., & Cabrera-Guerrero, G. (2020). Towards a customizable immersive virtual reality serious game for earthquake emergency training. Advanced Engineering Informatics, 46 , 101134.

Garrison, D. R., & Kanuka, H. (2004). Blended learning: Uncovering its transformative potential in higher education. The Internet and Higher Education, 7 (2), 95–105. https://doi.org/10.1016/j.iheduc.2004.02.001

Glaser, B. G., & Strauss, A. L. (2017). Discovery of grounded theory: Strategies for qualitative research . Routledge.

Book Google Scholar

Glover, I., & Latif, F. (2013, June). Investigating perceptions and potential of open badges in formal higher education. In: EdMedia+ Innovate Learning (pp. 1398–1402). Association for the Advancement of Computing in Education (AACE)

Gong, L., Liu, Y., & Zhao, W. (2018, July). Using learning analytics to promote student engagement and achievement in blended learning: An empirical study. In: Proceedings of the 2018 2nd International Conference on E-Education, E-Business and E-Technology (pp. 19–24)

Harrison, J. B., & West, R. E. (2014). Sense of community in a blended technology integration course: A design-based research study. International Review of Research in Open and Distributed Learning, 15 (6), 289–312.

Heale, R., & Twycross, A. (2018). What is a case study? Evidence-Based Nursing, 21 (1), 7–8.

Hughey, A. W., & Mussnug, K. J. (1997). Designing effective employee training programmes. Training for Quality, 5 (2), 52–57.

Isiaka, S. B. (2011). Motives for training and management development in the Nigerian banking industry. Asian Social Science, 7 (3), 210.

Karaoglan Yilmaz, F. G., & Yilmaz, R. (2020). Student opinions about personalized recommendation and feedback based on learning analytics. Tech Know Learn, 25 , 753–768. https://doi.org/10.1007/s10758-020-09460-8

Kaur, M. (2013). Blended learning-its challenges and future. Procedia-Social and Behavioral Sciences, 93 , 612–617.

Klašnja-Milićević, A., & Ivanović, M. (2021). E-learning personalization systems and sustainable education. Sustainability, 13 (12), 6713.

Knight, S., Shibani, A., Abel, S., Gibson, A., Ryan, P., Sutton, N., Wight, R., Lucas, C., Sándor, Á., Kitto, K., Liu, M., Mogarkar, R., & Shum, S. (2020). Acawriter: A learning analytics tool for formative feedback on academic writing. Journal of Writing Research, 12 (1), 299–344.

Kwon, K., Cho, J., & Park, Y. (2010). How to best characterize the personalization construct for e-services. Expert Systems with Applications, 37 , 2232–2240.

Kwon, K., & Kim, C. (2012). How to design personalization in a context of customer retention: Who personalizes what and to what extent? Economic Commerce Research and Applications, 11 , 101–116.

Lainema, T., & Nurmi, S. (2005). Customization of Industrial Training: Benefits and Problems. Information and Communication Technologies and Real-Life Learning: New Education for the Knowledge Society. US: Springer.

Lavori, P. (1990). ANOVA, MANOVA, my black hen: Comments on repeated measures. Archives of General Psychiatry, 47 (8), 775–778.

Lightner, C. A., & Lightner-Laws, C. A. (2016). A blended model: Simultaneously teaching a quantitative course traditionally, online, and remotely. Interactive Learning Environments, 24 (1), 224–238.

Ligorio, M. B., & Sansone, N. (2009). Structure of a blended university course: applying constructivist principles to blended teaching. Information technology and constructivism in higher education: Progressive learning frameworks. Igi Global.

Luneta, K. (2012). Designing continuous professional development programmes for teachers: A literature review. Africa Education Review, 9 (2), 360–379.

Marappan, R., & Bhaskaran, S. (2022). Analysis of recent trends in E-learning personalization techniques. The Educational Review, 6 (5), 167–170.

McAlpine, I. (2000). Collaborative Learning Online. Distance Education, 21 (1), 66–80.

Miller, B. J., Effney, D. J., & Gough, I. R. (1993). Do medical students remember multiple choice questions? Australian and New Zealand Journal of Surgery, 63 (11), 897–900.

Morrison, K. (2021). 5 Common Mistakes You Should Avoid When Conducting A Training Needs Analysis. eLearning Industry . https://elearningindustry.com/training-needs-analysis-5-common-mistakes-avoid-conducting

Neville, K., Heavin, C., & Walsh, E. (2005). A case in customizing E-learning. Journal of Information Technology, 20 (2), 117–129. https://doi.org/10.1057/palgrave.jit.2000041

Niazi, A. S. (2011). Training and development strategy and its role in organizational performance. Journal of Public Administration and Governance, 1 (2), 42–57. https://doi.org/10.5296/jpag.v1i2.862

Osgood, C. E. (1948). An investigation into the causes of retroactive interference. Journal of Experimental Psychology, 38 (2), 132.

Pavliuk, R. O., Liakh, T. L., Bezpalko, O. V., & Klishevych, N. A. (2017). Based training: Methodological characteristics and results of the analysis of educational programs. Social Sciences, 6 (4), 152.

Postholm, M. B. (2016). Collaboration between teacher educators and schools to enhance development. European Journal of Teacher Education, 39 (4), 452–470.

Prasad, P. W. C., Maag, A., Redestowicz, M., & Hoe, L. S. (2018). Unfamiliar technology: Reaction of international students to blended learning. Computers & Education, 122 , 92–103. https://doi.org/10.1016/j.compedu.2018.03.016

Puška, A., Puška, E., Dragić, L., et al. (2021). Students’ satisfaction with E-learning platforms in Bosnia and Herzegovina. Tech Know Learn, 26 , 173–191. https://doi.org/10.1007/s10758-020-09446-6

Rasheed, R. A., Kamsin, A., & Abdullah, N. A. (2020). Challenges in the online component of blended learning: A systematic review. Computers & Education, 144 , 103701.

Reigeluth, B. J. Beatty, & R. D. Myers (Eds.), Instructional-design theories and models, The learner centered paradigm of Education (Vol. 4, pp. 93–120). NewYork, Routledge

Richmond, H., Copsey, B., Hall, A. M., Davies, D., & Lamb, S. E. (2017). A systematic review and meta-analysis of online versus alternative methods for training licensed health care professionals to deliver clinical interventions. BMC Medical Education, 17 (1), 1–14.

Ritella, G., Di Maso, R., McLay, K. F., Annese, S., & Ligorio, M. B. (2020). Remembering, reflecting, reframing: Examining students’ long-term perceptions of an innovative model for university teaching. Frontiers in Psychology, 11 , 565. https://doi.org/10.3389/fpsyg.2020.00565

Schank, R. C. (1990). Case-based teaching: Four experiences in educational software design. Interactive Learning Environments, 1 (4), 231–253.

Shen, J., & Tang, C. (2018). How does training improve customer service quality? The roles of transfer of training and job satisfaction. European Management Journal, 36 (6), 708–716.

Springer, R. (2014). Personalizing personalization: It’s about more than just knowing your user’s name. Dow Jones . Retrieved November 12, 2017 from http://www.econtentmag.com/Articles/News/News-Feature/Personalizing-personaliztion-Its-About-More-Than-Just-Knowing-Your-Users-Name-96731.htm

Sundar, S. S., & Marathe, S. S. (2010). Personalization versus customization: The importance of agency, privacy, and power usage. Human Communication Research, 36 (3), 298–322. https://doi.org/10.1111/hcre.2010.36.issue-3

Szeto, E., & Cheng, A. Y. (2016). Towards a framework of interactions in a blended synchronous learning environment: What effects are there on students’ social presence experience? Interactive Learning Environments, 24 (3), 487–503.

Tracey, J. B., & Tews, M. J. (1995). Training effectiveness: Accounting for individual characteristics and the work environment. The Cornell Hotel and Restaurant Administration Quarterly, 36 (6), 36–42.

Wang, B., Tawfik, A. A., Keene, C. W., & Giabbanelli, P. J. (2023). Transitioning from individuals to groups in knowledge map construction. Technology, Knowledge and Learning . https://doi.org/10.1007/s10758-023-09651-z

Watson, W. R., & Watson, S. L. (2016). Principles for personalized instruction. Instructional-design theories and models (Vol. 4, pp. 93–120). Routledge.

Winstone, N. E., Nash, R. A., Parker, M., & Rowntree, J. (2017). Supporting learners’ agentic engagement with feedback: A systematic review and a taxonomy of recipience processes. Educational Psychologist, 52 (1), 17–37.

Wise, A., & Vytasek, J. (2017). Learning analytics implementation design. In: C. Lang, G. Siemens, A. F. Wise, & D. Gaševic (Eds.), The handbook of learning analytics (1st ed., pp. 151–160). Beaumont, AB: Society for Learning Analytics Research (SoLAR). https://doi.org/10.18608/hla17.013

Xie, H., Chu, H. C., Hwang, G. J., & Wang, C. C. (2019). Trends and development in technology-enhanced adaptive/personalized learning: A systematic review of journal publications from 2007 to 2017. Computers & Education, 140 , 103599.

Zine, P. U., Kulkarni, M. S., Chawla, R., & Ray, A. K. (2014). A framework for value co-creation through customization and personalization in the context of machine tool PSS. Procedia CIRP, 16 , 32–37.

Zo, H. (2003). Personalization vs. customization: Which is more effective in E-services? In Proceedings of the 9th Americas Conference on Information Systems , 2003 Proceedings. 467. http://aisel.aisnet.org/amcis2003/467

Download references

Open access funding provided by Università degli Studi di Bari Aldo Moro within the CRUI-CARE Agreement. The authors have not disclosed any funding.

Author information

Authors and affiliations.

University of Bari “Aldo Moro”, Bari, Italy

Sara Torre & Maria Beatrice Ligorio

Grifo Multimedia SRL, Bari, Italy

Sara Torre & Antonio Ulloa Severino

You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sara Torre .

Ethics declarations

Conflict of interest.

The authors have not disclosed any competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Torre, S., Ulloa Severino, A. & Ligorio, M. Learning Outcomes and Training Satisfaction: A Case Study of Blended Customization in Professional Training. Tech Know Learn (2024). https://doi.org/10.1007/s10758-024-09778-7

Download citation

Accepted : 19 August 2024

Published : 09 September 2024

DOI : https://doi.org/10.1007/s10758-024-09778-7

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Blended learning
Customization
Learning analytics
Professional training
Find a journal
Publish with us
Track your research

Academia.edu no longer supports Internet Explorer.

To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to upgrade your browser .

Enter the email address you signed up with and we'll email you a reset link.

We're Hiring!
Help Center

A Literature Review and Reports on Training and Development

Related Papers

Nasir Malik

International journal of applied research

Rajni Saini

Training is an essential activity in all professions. It plays a very important role in the effectiveness and efficiency of the human resource. The main aim of this study is to examine the role of training and the factors affecting training effectiveness. Training is an organised procedure by which people learn to perform their duties with more competency. The objective of the training is to achieve perfection in their work. This paper deals with different aspects of training and factors affecting the training effectiveness. Training is required for managers to enable them work towards taking the organisation to its expected skilfulness. This research paper also determines importance of training effectiveness in the ever changing business scenario.

Soni Agrawal

IAEME PUBLICATION

IAEME Publication

One of the main differentiators for companies, particularly the manufacturing industries, is training and growth. The manufacturing industry training and development system in Pune needs a comprehensive overhaul. There is a lack of a systemic approach to the current training and development programmes, particularly with regard to the design and planning approach. For example, there was no fixed training and development schedule and, moreover, no such schedule was disseminated or displayed. The average number of training programs representing the organization's current training strategy is only 2, reflecting the institution's apathy for training and growth. If the employees' evidence is to be believed, then there is a lack of a consistent training and development program for manufacturing sectors and seldom are employees identified to external organizations. The training approach practiced is on job training and the resource person is most commonly internal resource individuals or company workers on most occasions. The recognition of training and development needs is one of the serious grievances put forward by workers and it has been ranked as one of the most significant factors affecting their decision to engage in training and development programs. With all of the above, a systematic and empirical approach to training and development must be developed by all organizations.

International Journal of Training and Development

Er Harpreet Kaur Channi

Indian Journal of Public Administration

Azhar Kazmi

Remegius I Shiwayu , Remegius Shiwayu

The quality of well-trained workforce is an asset to any organisation because of training and development in the organisation. The study, therefore sought to scrutinize the impact of training and development using Mpact Corrugated Pty (Ltd) as a case study. The study aims to find out the influence training has on the employees’ performance. The key finding of the study was poor communication and motivation of the senior management. The company employs younger professional that has skills but need more training. Furthermore, training and development are well implemented, but the processes are not up to date. The study suggests the company must use systematic training; identification of training needs should be done more professionally in combination with the line managers as well as those that are involved with the human resource managers. The researcher also advises that it is very important for Mpact corrugated companies (PTY) LTD to develop new policies that will provide favorable working conditions for employees and boost morale the of workers. The study recommends that further studies could investigate possible interventions that can be used to address the issues of training and development.

Industrial and Commercial Training

Laurence Fink

Transstellar Journals

TJPRC Publication

This part studies about the noteworthy contribution of past research on the viability of training in private organizations. The study will assess the factors affecting the training programmes, for example, Objective of the Program, Evaluation of Performance subsequent to Training, Trainee Evaluations of Trainer and Testing of Trainees on Content displayed and so on and propose enhancements to further build Return on Investment and Return on Engagement

Loading Preview

Sorry, preview is currently unavailable. You can download the paper by clicking the button above.

RELATED PAPERS

Venkata Sandeep

Vision: The Journal of Business Perspective

Dr. Arjita Jain

Journal of Applied Psychology

Russell Day

TEM Journal

Dr. Jolly Sahni

REVISTA ESPACIOS

Subhankar Das

Ajith Sundaram

Sikandar Ali

Open Access Journal of Science

Hasan Tarik

IJAR Indexing

Tulika Thakuria

International Journal of Management Studies

Atanu Manna

European Journal of Training and Development

Gonçalo Venceslau Bernardino

CERN European Organization for Nuclear Research - Zenodo

Mohammad Tamzid

Evaluation and Program Planning

IOSR Journal of Business and Management

Dr. Harshit Topno

Sumit Prasad, DipTD

Marketing management

Archives of Business Research

Dr Ajaz Ahmad Bhat

Lamidi Yusuf

Best Solution Polytechnic, Printing Press

Olarinmoye AyodejiTemple

RESEARCH REVIEW International Journal of Multidisciplinary RESEARCH REVIEW International Journal of Multidisciplinary

Sowkarthika V

iaeme iaeme

Namrata Duara

literature review on training design

IMAGES

VIDEO

COMMENTS

Effective in-service training design and delivery: evidence from an integrative literature review

Conclusions

Inclusion/exclusion criteria

Search strategy

Study type, quality assessment and grade

Case-based: use of created or actual clinical cases that present materials and questions

Didactic/lecture: presenting knowledge content; facilitator determines content, organization and pace

Feedback: providing information to the learner about performance

Games: competitive game with preset rules

Interactive: provide for interaction between the learner and facilitator

Point-of-care (POC): information provided as needed, at the point of clinical care

Problem-based learning (PBL): present a case, assign information-seeking tasks and answer questions about the case; can be facilitated or non-facilitated

Reminders: provision of reminders

Self-directed: completed independently by the learner based on learning needs

Team-based: providing interventions for teams that provide care together

Live versus computer-based

Limitations

Abbreviations

Acknowledgments

Author information

Corresponding author

Additional information

Authors’ contributions

Authors’ original submitted files for images

Authors’ original file for figure 1

About this article

Share this article

Human Resources for Health

Annual Review of Psychology

Most Read This Month

Online Option

Save citation to file

Add to My Bibliography

Effective in-service training design and delivery: evidence from an integrative literature review

Similar articles

Publication types

Related information

Other Literature Sources

Research Materials

Miscellaneous

Have a language expert improve your writing

How to Write a Literature Review | Guide, Examples, & Templates

Instantly correct all language mistakes in your text

Table of contents

Don't submit your assignments before you do this

Make a list of keywords

Search for relevant sources

Take notes and cite your sources

Chronological

Methodological

Theoretical

Cite this Scribbr article

Is this article helpful?

Shona McCombes

Systematic review of the effectiveness of training programs in writing for scholarly publication, journal editing, and manuscript peer review (protocol)

David Moher

Becky Skidmore

Craig Campbell

Paul Hendry

Criteria for considering studies for this review

Intervention

Study design(s)

Search methods for identification of studies

Data collection

Study selection

Data extraction

Assessment of validity of evaluations

Data analysis

Dealing with missing data

Data synthesis

Subgroup analysis

Assessment of heterogeneity

Reporting biases

Abbreviations

Competing interests

Authors’ contributions

Acknowledgements

Training design in mediating the relationship of participants’ motivation, work environment, and transfer of learning

Design/methodology/approach

Research limitations/implications