|Year : 2016 | Volume
| Issue : 5 | Page : 493-497
Kirkpatrick evaluation model for in-service training on cardiopulmonary resuscitation
Safoura Dorri1, Malekeh Akbari2, Mahmoud Dorri Sedeh3
1 PhD Student of Nursing, Department of Medical Surgical Nursing, Iran University of Medical Sciences, Tehran, Iran
2 BSN of Nursing, Educational Supervisor, Shohadaye Lenjan Hospital, Isfahan University of Medical Sciences, Isfahan, Iran
3 Esfahan Steel Company, Isfahan, Iran
|Date of Submission||10-Oct-2015|
|Date of Acceptance||26-Apr-2016|
|Date of Web Publication||4-Nov-2016|
Shohadaye Lenjan Hospital, Isfahan University of Medical Sciences, Isfahan
Source of Support: None, Conflict of Interest: None
Background: There are several evaluation models that can be used to evaluate the effect of in-service training; one of them is the Kirkpatrick model. The aim of the present study is to assess the in-service training of cardiopulmonary resuscitation (CPR) for nurses based on the Kirkpatrick's model.
Materials and Methods: This study is a cross-sectional study based on the Kirkpatrick's model in which the efficacy of in-service training of CPR to nurses was assessed in the Shahadaye Lenjan Hospital in Isfahan province in 2014. 80 nurses and Nurse's aides participated in the study after providing informed consent. The in-service training course was evaluated in reaction, learning, behavior, and results level of the Kirkpatrick model. Data were collected through a researcher-made questionnaire.
Results: The mean age of the participants was 35 ± 8.5 years. The effectiveness score obtained in the reaction level (first level in the Kirkpatrick model) was 4.2 ± 0.32. The effectiveness score in the second level of model or the learning level was 4.70 ± 0.09, which is statistically significant (P < 0.001). The effectiveness score at the third and fourth level were 4.1 ± 0.34 and 4.3 ± 0.12, respectively. Total effectiveness score was 4.35.
Conclusions: The results of this study showed that CPR in-service training has a favorable effect on all four levels of the Kirkpatrick model for nurses and nurse's aides.
Keywords: Evaluation, in-service training, Kirkpatrick model, nursing
|How to cite this article:|
Dorri S, Akbari M, Dorri Sedeh M. Kirkpatrick evaluation model for in-service training on cardiopulmonary resuscitation. Iranian J Nursing Midwifery Res 2016;21:493-7
|How to cite this URL:|
Dorri S, Akbari M, Dorri Sedeh M. Kirkpatrick evaluation model for in-service training on cardiopulmonary resuscitation. Iranian J Nursing Midwifery Res [serial online] 2016 [cited 2021 May 18];21:493-7. Available from: https://www.ijnmrjournal.net/text.asp?2016/21/5/493/193396
| Introduction|| |
Nowadays, economic growth and development of societies depends on their human resources; and the efforts for increasing efficiency and effectiveness, are efforts to improve life situation for all people in the society. Thus, effective insight, knowledge, and behavior should be developed in staff. , Today, accelerated growth of information and complexity of careers has increased the importance of staff training.  Training has a positive role on the employees of an organization. Organizations that pay attention to training and human resources are more successful.  Hence, it can be claimed that in-service staff training is an important step for improving the capabilities of every organization.
In health care professions, in-service training has a special role. In nursing-as one of the most important health care profession-cardiopulmonary resuscitation training is a vital component, and it is necessary that we become confident that nurses act efficiently during care of patients who need CPR. To act efficient, it is necessary that nurses have acquired essential knowledge and skill.  One of the way for improving knowledge and skill is in-service training.
In today's organizations, in-service training for improving scientific level of staff is essential, however, the efficacy evaluation of these trainings is more important.  Evaluation is a means for determining the extent to which we have been successful in achieving educational goals. Evaluation should determine whether we have achieved goals or not. ,
Nowadays because of socioeconomic changes and training technologies related to them, evaluating training programs has also changed.  Because staff knowledge and skill in health care professions are important, choosing a proper method for evaluation is also important. There are different evaluation models that are used for evaluating effectiveness around the world such as the Context-Input-Process-Product (CIPP) evaluation model, Kirkpatrick evaluation model, and Outcome-based Evaluation model (OBE). Considering the present study and evaluation of effectiveness of in-service training of nurses, the Kirkpatrick evaluation model was used, which is one of the best evaluation methods and includes 4 levels, namely, reaction, learning, behavior, and results. ,,, Kirkpatrick evaluation model was proposed for the first time by Kirkpatrick in 1960s, and has been accepted and used extensively since then.  Kirkpatrick model has been used for more than thirty years as the primary design for organizing and evaluating training in non-profit organizations.  The popularity of this model for evaluation of training programs can be because of the fact that this model provides a system for demonstration of training results and different data that can be used for evaluation. Hence, this model simplifies the complex process of evaluation of training programs. 
Reaction is the act that learners demonstrate to all effective factors during a training period. In fact, reaction evaluates participants' feelings about a program. By learning, we refer to the way and extent of changes in participants by attending training courses. Behavior implies that whether training course has caused a desired change in learners' behaviors. And finally, results imply that whether training course has resolved the existed problem and helped in achieving organizational goals.
Unfortunately in Iran, training programs are rarely evaluated. One of its causes is that staff and managers are not serious about proper evaluation of courses, which is also associated with improper feedback. , In Iran, few studies have investigated in-service training programs based on the Kirkpatrick model, especially in health care domain, and even in these few studies based on Kirkpatrick model, evaluation is about first and second level and the next two levels are not considered significantly. Therefore, this study was done to evaluate if this model can be a proper model for evaluation of in-service training courses in health care and especially nursing field.
| Materials and Methods|| |
This study is an evaluation, cross-sectional study. Statistical sample comprised all nurses and nurses' aides in the Shohadaye Lenjan Hospital. These nurses and nurses' aides worked in the women's and men's internal and surgery wards as well as the emergency room and they participated voluntarily during 20-30 December CPR in-service training course (N = 80) conducted in 2014. Participants were divided in three groups based on working shift and their requests. For every group, 2 day and 5 h CPR in-service training sessions using slides, models, and resuscitation instruments such as laryngoscope and DC shock were held. Training content and teachers in all three groups were the same.
For evaluation of reaction level (determining relish extent and participants' feelings regarding the training course is done after the course and is based on the participants' immediate reaction), feedback questionnaires concerning training course and teachers were evaluated. The questionnaire about training course had 13 questions that evaluated content, equipment and managing. Evaluation of teachers contained rhetorical and presentation capabilities, knowledge and skill related to training course topics, making learners participate in discussions, proper use of training aids and teacher's general evaluation (discipline, behavior, Etc.). 5-point Likert's scale (absolutely agree, agree, no comments, disagree, absolutely disagree) was used for the evaluation of participants' comments. Points were calculated based on 5. Because obtaining correct and meaningful answers from participants was very important at this stage, a questionnaire with closed questions was designed so that respondents give accurate answers. Respondents were assured that their information will remain anonymous and secret. The effect of training course and the learning was evaluated by a researcher-made test in the form of pretest and posttest 1 month after training. The effect on learning was calculated by the difference between scores of pre and posttests. Face and content validity of questionnaires were investigated by professionals in two stages of pre-test. Reliability of questionnaire was evaluated and its Cronbach's alpha was 0.84, which was acceptable. For analyzing data, Descriptive and inferential statistical methods were used with SPSS (statistical package for social science) version 18 (SPSS Inc, Chicago, IL, USA). For evaluation of learning, attempts were made to all three domains, knowledge, understanding, and application, were evaluated [Table 1].
For evaluating behavior level, a 360° evaluation checklist for CPR was used. Behavior was evaluated by supervisors, head nurses, CPR physician, and one colleague after 3 months.
For evaluation of the fourth level or results, it should be noted that the results of the education can be studied in one of the four domains, i.e., avoiding costs, saving, profits, and strategic results. In this study, achieving strategic results was considered. Acceptable strategic results were considered to be the extent of achieving special and general goals of training course that had been agreed earlier by head nurses and supervisors and CPR physicians. The overall utility was calculated based on the average weight of scores in four levels, and because Kirkpatrick has evaluated the importance and sensitivity of the third level to be high, third level or behavior was given an average weight of 2 and other levels were given a weight of 1.
The study was explained verbally to participants and they also receive a written explanation. They were informed that participation was voluntary. It was emphasized that none of the information would be identifiable. Informed consent was also obtained from the participants.
| Results|| |
Using description statistics and central and dispersion indices, obtained average utility in reaction level in the Kirkpatrick model was 4.2 ± 0.32, that is average of participants' answers to every item in the feedback questionnaires of course and teacher. Score of content was 4.5 ± 0.26, equipment 3.92 ± 0.4, and teacher 4.2 ± 0.18. In learning level of the Kirkpatrick model, average score of utility, obtained from the test made by the researcher, was 4.7 ± 0.09 [Table 2]. It is worth noting that comparing results from pretest to posttest of participants by t-test and P < 0.05 were statistically significant, which implies that level of knowledge and skill of nurses in CPR was improved after one month from training course. This table showed also that this improvment was significant for both nurses and Nurses' aides.
|Table 2: Results of second level of the Kirkpatrick model (pretest and posttest)|
Click here to view
In the third level, the average utility score was 4.1 ± 0.34. This score was obtained by 360° evaluation by the head nurse, supervisor and CPR physician, and one colleague after 3 months. Criteria for being ineffective was scores less than 50 from 100 or 2.5 from 5.
In the fourth level, which was achieving strategic goals, utility score was 4.3 ± 0.12. [Table 3] lists the strategic goals. Finally, the total utility score, based on a weight of 2 for behavior level and 1 for other levels, was 4.35 [Table 4].
| Discussion|| |
Human resource is considered to be one of the most important resources of an organization. An organization, even with the best technologies and equipment, without specialist and trained staff cannot achieve success.  Hence, training should be one of the organization's duties and should be considered an investment. Today, organizations for in-service training consider budget. Organizations in the USA have reported an increase in the budget of training.  Moreover, managers want to see feedback and scientific reports regarding the effectiveness of these trainings. Receiving feedback needs evaluation. To obtain confident results, this evaluation should be based on scientific principles and objectives as well as the conditions of the organization.
Results of this study show that staff that participated in CPR training course evaluated teacher and the course as useful. In the second level, changes made in knowledge and learning of staff were desired. In the third level, it was determined that the course could make desired changes in learners up to 3 months after the training. In the fourth level, it was determined that CPR training course could achieve strategic goals. In general, despite the fact that training course was not 100% effective, but based on statistical calculations, it can be claimed that the effectiveness of this course was desired.
In Iran, no study has been conducted on the effectiveness of the Kirkpatrick model in in-service training of CPR for nurses and nurses' aides. A similar study to ours was reported by Pour-Jahromi et al., which evaluated the effectiveness of in-service training concerning working with electroshock in nurses based on the same model in every four levels. Results of their study showed the utility of the course in four levels.
Iranian articles based on the Kirkpatrick model are few and are usually limited to level 1 and 2 of this model. Studies by Bakhshandeh et al.,  Amiri-Mehr  and Tavakoli  showed evaluation utility in the reaction level. The work by Omar et al.  showed desired results based on the Kirkpatrick model. Work of Omar et al. was conducted on 35 health care managers in different provinces, and it was determined that the most effective training was in the 1 st and 2 nd levels and it was reduced in the 3 rd and 4 th levels. In a work by Abbasian et al.,  in IranKhodro company, it was determined that the general utility was satisfactory, but it was decreased in level 3 and especially 2, such that effectiveness in 4 levels were 76, 64, 72, and 13%, respectively.
Considering the importance of CPR in health care profession, proper training and its evaluation is emphasized. In explaining the importance of training, especially for nurses and the necessity of proper evaluation of CPR, it should be noted that heart diseases caused the early worldwide deaths. Madden in her article mentioned the rate of early deaths to be more than 40% annually in Ireland, and reported that 6000 of these deaths were because of sudden cardiac arrests. She pointed out that more than half of these deaths occurred in hospitals, and hence considered the role of nurses in this regard vital and emphasized on the competence of nurses in administering CPR.  Training nurses for CPR is one of the principles that has been mentioned in guidelines such American Heart Association guidelines.  It becomes clear that training nurses about CPR and receiving feedback in order to increase patients' survival and preventing early deaths is very important. It appears essential to improve the quality of these programs and receive feedback from them.
To increase quality and utility of training courses in nursing regarding CPR, studies show that it is better that training programs for CPR are held every 3 to 6 months for improving knowledge and skill. In addition, it has been recommended that computer simulation applications are used in order to provide the possibility of self-study and practice. These applications should be associated with providing feedback to user.  Hence, it is possible to improve the quality of in-service training using simple and inexpensive strategies and receive feedback and correct weaknesses.
In foreign studies, we could not find an article assessing the effectiveness of in-service training of CPR for nurses based on all 4 levels of the Kirkpatrick model. The most similar article that we found was the work by Aoki et al.  They observed the desired results of their training program concerning CPR in the 1 st , 2 nd , and 3 rd levels of the Kirkpatrick model. They studied nursing, physiotherapy, speech therapy, and work therapy students and their objective was to investigate the effectiveness of American CPR in Japanese students. Regardless of its objective and the fact that this was not in-service training, it can be said that the results of Aoki et al. demonstrate the effectiveness of CPR training programs based on the Kirkpatrick model, which is also in agreement with the present study.
| Conclusion|| |
The results of this study showed that in-service training of CPR is effective in all the levels of the Kirkpatrick model for nurses and nurses' aides. CPR training for nurses is a complex activity. Hence, it is recommended that we improve nurses' learning and functional skills. Nurses who work in intensive care units encounter more cardiac arrests and dangerous situations and require training for advanced CPR. Thus, training courses for these nurses should be based on valid guidelines and should provide the ability to diagnose at-risk patients in hospitals.
Therefore, the importance of CPR in nursing and the results of the present study show that this training can be effective in the reaction, learning, behavior and results levels; it is recommended that organizations and managers try to conduct more such in-service trainings and evaluate their effectiveness based on useful models because their results have a direct impact on improving services and patients' survival.
Considering that the behavior level compared to the reaction and learning levels is challenging and sensitive and Kirkpatrick mentioned three causes for this (first, participants should find an opportunity to change their behaviors; second, the time for change in behavior cannot be really predicted; third, organizational atmosphere can have an impact on changing behavior during work), it is recommended that evaluation be repeated enough times to be confident about the permanent changes in the behavior. In the present study, attempts were made to evaluate behavioral indices at different times from different people (head nurse, physician and colleague), however, survey of behavioral index in the Kirkpatrick model is difficult and time consuming. The limitation of the present study is a lack of control group. It is recommended that future studies employ a control group in order to omit factors effective on behavior. Furthermore, it is better that changes occur in a more natural environment and be surveyed for a longer duration, a task that was not feasible in the present study.
At the end of the research we thank all the staff of the Shohadaye Lenganan Hospital.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
Chiang FF, Birtch TA, Kwan HK. The moderating roles of job control and work-life balance practices on employee stress in the hotel and catering industry. Int J Hospitality Manage 2010;29:25-32.
Aghaee M, Aghaee R, Aghaee A. The role of behavioral dimensions job on efficiency and effectiveness of human resources. J Manage Org Culture 2015;12:809-30. [In Persian].
Alhoseini H. The Theory of Continuous Engineering C.E.O. Identification of Permanent Authenticity in the Organizations as the Field of Development of the Individual and Society. 1 st
ed.Bandar Abbas: University of Hormozgan Publication; 2000. p. 103.
Ologunowa CS, Akintunde BA, Adu BO. Manpower training and development: Pathway to efficient organizational performance. Am Int J Res Hum Arts Sci 2015;9:44-52.
Hamilton R. Nurses' knowledge and skill retention following cardiopulmonary resuscitation training: A review of the literature. J Adv Nursing 2005;51:288-97.
Shojae K, Karami M, Ahanchian MR. Evaluation the effectiveness of training programs for staff of Mashhad Ferdowsi University. Master Thesis in field of Education Management. 2011, thesis nomber: ED EDC 114. Available from: http://thesis.um.ac.ir/moreinfo-45241-pg-1.html.
Esteves J, Pastor J, Casanovas J. A framework proposal for monitoring and evaluating training in ERP implementation projects. Technical Research. 2002.
Rajeev P, Madan MS, Jayarajan K. Revisiting Kirkpatrick's model-An evaluation of an academic training course. Curr Sci 2009;96:272-6.
Kirkpatrick DL. Evaluating Training Program: The Four Level. 2 nd
ed. San Francisco, CA: Berrett-Koehler Publisher; 1998.
Bates R. A critical analysis of evaluation practice: The Kirkpatrick model and the principle of beneficence. Eval Program Plan 2004;27:341-7.
Kirkpatrick DL. Evaluation of Training. Training and Development Handbook: A Guide to Human Resource Development. New York: McGraw-Hill; 1976.
Curran VR, Fleet L. A review of evaluation outcomes of web-based continuing medical education. Med Educ 2005;39:561-7.
Shelton S, Alliger G. Who's afraid of level 4 evaluation? A practical approach. Training Dev 1993;47:43-7.
Pourjahromi Z, Ghafarian HR, Ghaedi H, Momeninejad M, Mohamadi M, Abasi A, et al
. The effectiveness of training Course on "how to work with DC Shock Devices" for nurses, based on Kirkpatrick model. Iranian J Med Educ 2011;11:896-901.
Omar M, Gerein N, Tarin E, Butcher C, Pearson S, Heidari G. Training evaluation: A case study of training Iranian health managers. Hum Resour Health 2009;7:20.
Abbasian A. Effectiveness of training course. Tadbir J 2006;170:52-5.
Bakhshandeh H, Ahmadi H, Behnam M, Hamidi M. Evaluating the effectiveness of arbitration and coaching of the students based on Kirkpatrick's model. J Sport Manage 2014;5:161-78. [In Persian].
Amiri-Mehr M. Evaluate the effectiveness of training courses Saipa Diesel Company according to Kirkpatrick model in summer 2006. Master's thesis, Department of Education and Psychology, University of Shahid Beheshti; 2007. pp. 11-25.
Tavakoli S. Assessment of learners reaction from short-term training courses for librarians of public library of Astan Quds Razavi on the first level of Kirkpatrick's model. Electronic Journal of Libraries, Museums and Documents Center of Astan Quds Razavi. 2010;2:1-16. [In Persian].
Abbasian A, Salimi G, Azin R. Evaluation of engineering training: Survey the effectiveness of resistant welding training course based on Kirkpatrick model, Irankhodro Co. as a case study. J Iran Eng Educ 2008;39:37-62. [In Persian].
Madden C. Undergraduate nursing students' acquisition and retention of CPR knowledge and skills. Nurse Educ Today 2006;26:218-27. Epub 2005/11/30.
Association AH. BLS for Healthcare Providers. Texas: American Heart Association; 2001.
Aoki T, Asada Y, Hiroe T, Tomizawa Y. Board 105-Program innovations abstract effectiveness of American style CPR training for Japanese medical students (submission #1494). Simul Healthcare 2013;8:403.
[Table 1], [Table 2], [Table 3], [Table 4]
|This article has been cited by|
||Kirkpatrick Model Evaluation of Accelerated Second-Degree Nursing Programs: A Scoping Review
| ||Hyejung Lee,Youngkwan Song |
| ||Journal of Nursing Education. 2021; 60(5): 265 |
|[Pubmed] | [DOI]|
||A COMPARATIVE ANALYSIS OF THE EFFICACY OF THREE PROGRAM-EVALUATION MODELS –A REVIEW ON THEIR IMPLICATION IN EDUCATIONAL PROGRAMS
| ||Zafar Iqbal,Muhammad Anees,Rahim Khan,Abdul Wadood,Shakila Malik |
| ||Humanities & Social Sciences Reviews. 2021; 9(3): 326 |
|[Pubmed] | [DOI]|
||Surgical data strengthening in Ethiopia: results of a Kirkpatrick framework evaluation of a data quality intervention
| ||Sehrish Bari,Joseph Incorvia,Katherine R. Iverson,Abebe Bekele,Kaya Garringer,Olivia Ahearn,Laura Drown,Amanu Aragaw Emiru,Daniel Burssa,Samson Workineh,Ephrem Daniel Sheferaw,John G. Meara,Andualem Beyene |
| ||Global Health Action. 2021; 14(1): 1855808 |
|[Pubmed] | [DOI]|
||Planning, construction and use of handmade simulators to enhance the teaching and learning in Obstetrics*
| ||Roxana Knobel,Mariane de Oliveira Menezes,Débora de Souza Santos,Maíra Libertad Soligo Takemoto |
| ||Revista Latino-Americana de Enfermagem. 2020; 28 |
|[Pubmed] | [DOI]|
||Mental health crisis training for non-mental health professionals
| ||Rebecca Maindonald,Chris Attoe,Melanie Gasston-Hales,Perah Memon,Elizabeth Barley |
| ||The Journal of Mental Health Training, Education and Practice. 2020; 15(4): 223 |
|[Pubmed] | [DOI]|
||Learning benefits of live surgery and semi-live surgery in urology—informing the debate with results from the International Meeting of Reconstructive Urology (IMORU) VIII
| ||Victor M. Schuettfort,Tim A. Ludwig,Phillip Marks,Malte W. Vetterlein,Valentin Maurer,Constantin Fuehner,Florian Janisch,Armin Soave,Michael Rink,Silke Riechardt,Oliver Engel,Margit Fisch,Roland Dahlem,Christian P. Meyer |
| ||World Journal of Urology. 2020; |
|[Pubmed] | [DOI]|
||Educational Program for Physiologic Monitor Use and Alarm Systems Safety
| ||JoAnne Phillips,Azizeh Sowan,Halley Ruppel,Rochelle Magness |
| ||Clinical Nurse Specialist. 2020; 34(2): 50 |
|[Pubmed] | [DOI]|
||Efeito de um programa educativo no conhecimento e na qualidade dos registros da pressão arterial
| ||Ana Carolina Queiroz Godoy Daniel,Eugenia Velludo Veiga,Juliana Pereira Machado,Ana Carolina Cintra Nunes Mafra,Lyne Cloutier |
| ||Revista Latino-Americana de Enfermagem. 2019; 27 |
|[Pubmed] | [DOI]|
||Impact of a compassionate care leadership programme
| ||Mohamad M Saab,Jonathan Drennan,Nicola Cornally,Margaret Landers,Josephine Hegarty,Eileen Savage,Cora Lunn,Alice Coffey |
| ||British Journal of Nursing. 2019; 28(11): 708 |
|[Pubmed] | [DOI]|
||Using Kirkpatrick’s model to measure the effect of a new teaching and learning methods workshop for health care staff
| ||Mohammad Reza Heydari,Fatemeh Taghva,Mitra Amini,Somayeh Delavari |
| ||BMC Research Notes. 2019; 12(1) |
|[Pubmed] | [DOI]|
||Evaluation of a consulting training course for international development assistance for health
| ||Pan Gao,Hao Xiang,Suyang Liu,Yisi Liu,Shengjie Dong,Feifei Liu,Wenyuan Yu,Xiangyu Li,Li Guan,Yuanyuan Chu,Zongfu Mao,Shu Chen,Shenglan Tang |
| ||BMC Medical Education. 2018; 18(1) |
|[Pubmed] | [DOI]|
||Five years’ experience of an annual course on implementation science: an evaluation among course participants
| ||Siw Carlfjord,Kerstin Roback,Per Nilsen |
| ||Implementation Science. 2017; 12(1) |
|[Pubmed] | [DOI]|
||Involving teachers in reducing childrenæs media risks
| ||Anja Stiller,Hanna Schwendemann,Paula Bleckmann,Eva-Maria Bitzer,Thomas Mößle,Katherine Weare |
| ||Health Education. 2017; : 00 |
|[Pubmed] | [DOI]|
||Using the ARCC Model to Implement the 2015 ANCC COA Criteria: A North Carolina Public Health Nursing Project
| ||Susan H. Little,Pamela S. Dickerson,Greg Randolph,Phyllis Rocco,Nancy Short |
| ||The Journal of Continuing Education in Nursing. 2017; 48(11): 501 |
|[Pubmed] | [DOI]|