|
Identification of common indicators of hospital performance evaluation models: A scoping review
Vahid Rasi, Bahram Delgoshaee, Mohammahreza Maleki
Department of Health Services Management, School of Health Management and Information Sciences, Iran University of Medical Sciences, Tehran, Iran
| Date of Submission | 29-Sep-2019 |
| Date of Acceptance | 12-Nov-2019 |
| Date of Web Publication | 31-Mar-2020 |
Correspondence Address:
Bahram Delgoshaee
Department of Health Services Management, School of Health Management and Information Science, Iran University of Medical Sciences, Tehran
Iran
 Source of Support: None, Conflict of Interest: None  | Check |
DOI: 10.4103/jehp.jehp_563_19
BACKGROUND: Hospitals in developed countries allocate itself about 40% of health-care costs and in developing countries up to 80%. In this study, researchers reviewed the related article in the field of the hospitals' performance evaluation due to the identification of common indicators used in different models with the aim of providing simple list for evaluating hospitals' performance to the researchers who are interested to these issues.
METHODS: The databases for the search included Medline, Google Scholar, and Scopus. A search strategy leads to the extraction of 403 related articles that after the removal of inappropriate and duplicate articles, 42 studies were selected for the scoping review.
RESULTS: This scoping review showed that 9 of 42 selected articles use data envelopment analysis model, 6 of them Pabon Lasso model, 3 of them balanced scorecard, 3 of them organizational excellence model, and the remaining articles used key performance indicators to evaluate hospital performance. A process approach was used to categorize the common indicators because the literature review indicated that common indicators mainly evaluate data, process, output, or impact of performance. The findings included 36 input, 39 process, 27 output, and 8 impact common indicators for evaluating performance.
DISCUSSION: This scoping review of related literature indicated that despite differentiation in the time and place of past studies in the field of performance evaluation models, there are some indicators that are common in most popular performance evaluation models. These simple lists can apply for evaluation of hospitals' performance instead of complicated models.
Keywords: Evaluation, hospital, performance
How to cite this article:
Rasi V, Delgoshaee B, Maleki M. Identification of common indicators of hospital performance evaluation models: A scoping review. J Edu Health Promot 2020;9:63 |
How to cite this URL:
Rasi V, Delgoshaee B, Maleki M. Identification of common indicators of hospital performance evaluation models: A scoping review. J Edu Health Promot [serial online] 2020 [cited 2021 Jan 16];9:63. Available from: https://www.jehp.net/text.asp?2020/9/1/63/281661 |
| Introduction | |  |
In developed countries, hospitals allocate about 40% and in developing countries up to 80% of health-care costs. Therefore, the results of hospitals' performance in the health-care system are a great priority.[1]
On the other hand, based on the literature review, more than half of national health resources are wasted in different countries, especially in underdeveloped countries, which means limited resources are inefficiently consumed or national income is spent on services that are not proportionate and effective. Therefore, ensuring the quality of health-care services requires evaluating the performance of health-care organizations to alleviate some of the government's financial problems by minimizing waste.[2] Therefore, many health-care providers have stepped to path of performance evaluation in order to achieve their goals and missions.[3]
In fact, these organizations consider performance evaluation models as a way to move toward continuous improvement and valid tool for measuring of the organization's current performance.[4]
On the other hand, an appropriate and applicable evaluation model for hospital performance potentially would lead to more accountability, service quality, and satisfied customer. When a hospital's performance is not evaluated, any claims about its service quality will be unproven, and there is no evidence to improve performance. Therefore, the evaluation and improvement of performance are two sides of the same coin that will lead to continued improvement, and the lack of an efficient performance evaluation will lead to serious problems for hospitals from the aspect of responsibility and performance improvement. Until two decades ago, designing a performance evaluation system was just a mere innovation, but today, it has become an indispensable necessity for any organization. With all these interpretations, the evaluation of performance is very complex and difficult to done, which requires pay attention to all various dimensions of performance that would affect the job results.[5]
The WHO's definition of performance is achieving the desired goals. The excellent hospital performance should be based on professional competence in benefit from the latest knowledge, advanced technology and of course regarding available resources, efficiency of resource, minimum risk for patient, and patient satisfaction. In other words, according to literature review, the performance evaluation is not worthwhile alone and will be valuable if the results of that used to continually improvement of performance.[6]
Today, several models to evaluate the performance of an organization have been introduced such as models of organizational excellence (European Foundation for Quality Management) that evaluates the performance of organizations for establishing total quality management[7] or the balanced scorecard (BSC) approach that transforms the organization's strategic goals into measurable indicators and establishes a balanced distribution across the organization's critical areas. This approach, as a measurement system while operationalizing the organization's vision and strategies, can provide a comprehensive picture of the organization's performance. In other words, a balanced scorecard can play an effective role in realizing the vision and strategy of an organization using a set of financial and nonfinancial indicators.[8]
Other ways to evaluate hospitals' performance are the use of key performance indicators (KPIs). In some studies, the performance of hospitals was evaluated by identifying and measuring KPIs.[9]
Organizations that apply any evaluation's models to measure its performance at first should identify some of the effective indicators. Improving hospital indicators means efficiency and effectiveness in hospital activities, in other words, best use of resources.[10] So far, many studies have been conducted to evaluate the performance of hospitals in different countries. In this study, we have attempted a scoping review of studies carried out over the past 10 years (2018–2008) on the performance evaluation of hospitals using different methods in Iran and other countries to reach common evaluation indicators.
| Methods | | |
In recent years, many studies have been conducted to evaluate the performance of health-care organizations in different countries. Therefore, because of the large number of studies in this field, the search strategy limited to the 10-year period (2008–2018). Furthermore, with regard to the purpose of the study, which was a scoping review of studies conducted in the field of hospital performance, the researchers excluded studies that done in other health-care organizations. Therefore, the keywords used to search the related articles were as follows: Performance Evaluation, Hospital, Performance Evaluation Models, and Performance Evaluation Methods [Table 1].
The databases used for the search included Medline, Google Scholar, Scopus, EBSCO Discovery Service, and Web of Science [Table 2]. A search of these databases leads to the extraction of 403 articles that, after the removal of duplicate and non-English articles, 123 articles with screening criteria (keywords in the title or abstract of the articles) were selected. Furthermore, in order to focus on the main purpose of the study, literature that evaluated the results of performance evaluation without using a specific model were excluded. After rescreening, studies that were repeated in the EndNote X7 for windows because of the use of the snowball method in references also were excluded. Finally, 42 studies were extracted for the scoping review [Figure 1].
After the final review of extracted articles, the selected articles were reviewed using the matrix method – a system for organizing, critically evaluating, and synthesizing research articles for scientific review of articles. Inclusion criteria were also used for reviewing and evaluating relevance articles. A scoping review is done to attain to deep perspective of a selected topic, not to evaluate or criticize the best study that have been counducted. Therefore, the studies were reviewed by a researcher, away from any critical review and weighting of the studies.
| Results | | |
After screening the studies, 42 studies that met the inclusion criteria were carefully reviewed. The characteristics of these studies are summarized in [Table 3] by target population, study location, research method, and time of study.
Out of 42 selected articles, 18 were published in other countries [Table 4] and 24 in Iran [Table 5], 9 of them use data envelopment analysis,[2],[3],[4],[5],[6],[7],[8],[9],[10],[11],[12],[13],[14],[15],[16],[17] 6 of them Pabon Lasso model,[8],[18],[19],[20],[21],[22] 3 of them BSC,[23],[24],[25] 3 of them organizational excellence model,[4],[7],[26] 3 of them used accreditation standards,[27],[28],[29] and other articles used KPIs to evaluate hospital performance.[1],[30],[31],[32],[33],[34],[35],[36],[37],[38],[39],[40],[41],[42] The Pabon Lasso model was used for the evaluation of hospital performance only in Iran in determined time period. In two studies, combined models were used for performance evaluation, and in some studies, performance evaluation indicators have been used; these indicators were hospital mortality rates and readmission rates as performance evaluation indicators.
After reviewing the indicators used in hospitals' performance evaluation-related literature review, a process approach was used to categorize the common indicators in most applied performance evaluation models because the literature review indicated that common indicators are main kind of indicators to evaluate data, process, output, and impact of performance. This category is summarized in [Table 5].
| Discussion | | |
Much has been echoed in the literature about the importance of understanding performance. According to industrial models of production, where the efficiency of production is paramount, some researchers have proposed that health care could be viewed under the same magnifying glass as the production of other goods.
Nevertheless, in many countries, the hospital environment has completely changed in the past 20 years compared to industrial zone, so health-care organizations have been forced to apply continued performance improvement approaches to survive in competitive environment. Therefore, seeking to comprehensive, reliable, strategic, and flexible model to evaluate performance has become an important priority and an undeniable responsibility for them. As mentioned above, health-care management and health-care industry have been one of the popular and complex topics that many researchers and professionals have focused on. In this study, researchers reviewed studies conducted over 10 years (2007–2017) with the aim of extracted common indicators of evaluating hospital performance.
For as much as in hospitals such as other organizations, services are delivered through determined processes and the researchers decided to summarize and categorize the common indicators in the format of process components (data, process, output, and impact) [Table 6]. In the articles reviewed, different indicators were used in varied model applied for hospitals' evaluation, but a scoping review confirms this point that despite difference among performance evaluation models, common indicators are the main body of all reviewed models. | Table 6: Effective indicators on hospital performance evaluation in a process format
Click here to view |
Therefore, the researchers in this study tried to gather indicators that were mainly used in previous related literature to evaluate the performance of hospitals to guide the practice of those interested in this kind of topics.
Study novelty
In this study researchers tried to obtaining a simple but comprehensive list of indicators that commonly used for hospital performance evaluation instead of complex performance evaluation models.
Study limitation
Researchers in this study tried to extracted related articles but some of them were not available because of publisher or journal rules so, researchers for overcoming this limitation replaced the most similar ones.
Acknowledgment
this research was part of Ph.D. thesis of the first author that was financially supported by grant number IUMS/SHMIS-1395.9221532202.
Financial support and sponsorship
This article is the part of the first author thesis that financial by grant number IUMS/SHMIS-1395/9221532202
Conflicts of interest
There are no conflicts of interest.
| References | | |
| 1. | Adel A, Mohammadi Y. Designing a math model for hospital performance evaluation with the FHNBSC, SAW, and LINMAP integrated approach. Health Syst Res 2012;10:509-24. [Last accessed on 2019 Nov 20]Available from: http://magiran.com/2322-5564  |
| 2. | Adel A, Valipour M, Moghbel A, Hassas Y. Efficiency evaluation of Tehran university of medical sciences hospitals using data envelopment analysis (2011-2010). Health Manage J 2013;16:36-46. [Last accessed on 2019 Nov 20] Available from: http://jha.iums.ac.ir/article-1-1305-fa.htm. |
| 3. | Asadi M, Mirghafouri H, Sadeghi Z, Kosravsniyan H. Evaluation of performance of Yazd public hospitals using a combination of balanced scorecard models, data envelopment analysis and servqual. J Shahid Sadoughi Univ Med Sci Health Serv Yazd 2010;18:559-69. [Last accessed on 2019 Nov 20] Available from: http://jssu.ssu.ac.ir/article-1-1335-fa.html. |
| 4. | Parham M, Fotouhi M, Jandaghi M, AlipourKh. Evaluation of performance of Shahid Beheshti hospital in Qom based on organizational excellence model. J Qom Univ Med Sci 2014;7:79-85. [Last accessed on 2019 Nov 20]Available from: http://journal.muq.ac.ir/article-1-598-fa.html. |
| 5. | Taslimi M, Zayandeh M. Challenges for creating a system for evaluating the performance of the hospital: A review of the literature and scientific work. Hakim 2014;16:35-41. [Last accessed on 2019 Nov 20]Available from: http://hakim.hbi.ir/article-1-1132-fa.html. |
| 6. | |
| 7. | Khanzadeh A, Motlagh M, Mirshakak A, Akbari N, Niyakan M, Pour shirvaniD. Evaluation of the performance of hospitals affiliated to Abadan faculty of medical sciences based on the Iranian national excellence award model. Health Inf Manage 2016;13:138-44. [Last accessed on 2019 Nov 20] Available from: http://magiran.com/p1563440. |
| 8. | Dargahi H, Tolouee SH, SadeghifarJ. Evaluation of performance of hospitals affiliated with Tehran university of medical sciences using the Pabon Lasso model. Payavard Health 2017;10:290-8. [Last accessed on 2019 Nov 20] Available from: http://payavard.tums.ac.ir/article-1-6068-fa.html. |
| 9. | Ghandehari N, Esmaeeliyan M, Teimouri H, Ghalamkari Sh. Evaluation of performance of Nour and Hazrat Ali Asghar hospitals in Isfahan using a balanced scorecard combination method and a technique for measuring attractiveness through a series of evaluations. Health Inf Manage 2017;14:58-64. [Last accessed on 2019 Nov 20] Available from: http://magiran.com/p1718495. |
| 10. | Tripathi CB, Kumar R, Sharma RC, Agarwal R. Assessment of performance of services in a tertiary care neuropsychiatric institute using Pabon Lasso model. Asian J Med Sci 2016;7:69-74. |
| 11. | Dong S, Zuo Y, Guo S, Li M, Liu X, Li H. Data envelopment analysis for relative efficiency measurements of Chinese hospitals: A systematic review. Res Health Sci 2017;2:79-103. |
| 12. | Karsak EE, Karadayi MA. Imprecise DEA framework for evaluating health-care performance of districts. Kybernetes 2017;46:706-27. |
| 13. | Hajduová RL. Using the data envelopment analysis in evaluating the technical efficiency in hospitals: General and new approaches. Acta Oecon Cassoviensia 2014;19:144-58. |
| 14. | Chuang CL, Chang PC, Lin RH. An efficiency data envelopment analysis model reinforced by classification and regression tree for hospital performance evaluation. J Med Syst 2011;35:1075-83. |
| 15. | |
| 16. | Weng SJ, Wu T, Blackhurst J, Mackulak G. An extended DEA model for hospital performance evaluation and improvement. Health Serv Outcomes Res Methodol 2009;9:39. |
| 17. | Khani F, Naderi H, Zangeneh M, Fazeli E. Measuring the relative efficiency of Ilam hospitals using data envelopment analysis. Manage Sci Lett 2012;2:1189-94. |
| 18. | Sajadi H, Sajadi Z, HadiM. Is there a method for comparing key indicators of hospital performance? Health Inf Manage 2010;8:71-81. [Last accessed on 2019 Nov 20] Available from: http://ensani.ir/fa/article/262230. |
| 19. | Rahbar A, Hadi P, KhosraviM. Evaluation of the performance of teaching hospitals in Qom and Kashan university of medical sciences and health services and their comparative comparison using the Pabonacci model. Health Inf Manage 2015;11:362-70. [Last accessed on 2019 Nov 20] Available from: http://ensani.ir/fa/article/353851. |
| 20. | Bastani P, Lotfi F, Moradi M, AhmadzadehM. Performance analysis of educational hospitals of Shiraz university of medical sciences before and after delivery of the health system using the Pabon Lasso model. J Rafsanjan Univ Med Sci 2016;15:781-92. [Last accessed on 2019 Nov 20]Available from: http://journal.rums.ac.ir/article-1-3165-fa.html. |
| 21. | Barati B, Farziyanpour F, Arab M, Rahimi A. Evaluation of the performance of Ninth and Deh Razi hospitals in Torbat Heydarieh city based on the Pabon Lasso model and its comparative comparison with the national standards in 1393-2014. J Torbat E Heydarieh Univ Med Sci 2017;5:48-55. [Last accessed on 2019 Nov 20]Available from: http://jms.thums.ac.ir/article-1-430-fa.html. |
| 22. | Mohammadkarim B, Jamil S, Pejman H, Seyyed MH, Mostafa N. Combining multiple indicators to assess hospital performance in Iran using the Pabon Lasso model. Australas Med J 2011;4:175-9. |
| 23. | Grigoroudis E, Orfanoudaki E, Zopounidis C. Strategic performance measurement in a healthcare organisation: A multiple criteria approach based on balanced scorecard. Omega 2012;40:104-19. |
| 24. | Samadi S, Gholmohammadi A, Mohammadi A, Rezapour T. Designing an organizational performance evaluation model for NAJA hospitals. Entezami Med 2011;1:38-45. |
| 25. | Rajoui M, Ramezan M, Hesari M, Bour bour M. Designing performance evaluation indicators by using AHP and BSC approaches (case of study: Social security organization of Mashhad). Soc Welf Q 2017;17:131-58. [Last accessed on 2019 Nov 20] Available from: http://refahj.uswr.ac.ir/article-1-2867-fa.html. |
| 26. | Tabatabaee SS, Kalhor R, Yousefi S, Taghadum-Kangi S, Ghamari MR. Performance evaluation of mehr hospital in Mashhad, Iran, based on European foundation for quality management (EFQM) organizational excellence model. Journal of Kerman School of Management and Medical Information 2013;1:89-1. [Last accessed on 2019 Nov 20]Available from: http://jms.kmu.ac.ir/article-1-35-fa.html. |
| 27. | Stefan MS, Pekow PS, Nsa W, Priya A, Miller LE, Bratzler DW, et al. Hospital performance measures and 30-day readmission rates. J Gen Intern Med 2013;28:377-85. |
| 28. | Arzemani M, Akaberi A, Pournaghi S. Performance evaluation of medical records department of hospitals related to North Khorasan university of medical sciences in 2013. J North Khorasan Univ Med Sci 2014;6:235-44. [Last accessed on 2019 Nov 20]Available from: http://journal.nkums.ac.ir/article-1-338-fa.html. |
| 29. | Schmaltz SP, Williams SC, Chassin MR, Loeb JM, Wachter RM. Hospital performance trends on national quality measures and the association with joint commission accreditation. J Hosp Med 2011;6:454-61. |
| 30. | Glover M, Khalilzadeh O, Choy G, Prabhakar AM, Pandharipande PV, Gazelle GS. Hospital evaluations by social media: A comparative analysis of facebook ratings among performance outliers. J Gen Intern Med 2015;30:1440-6. |
| 31. | Ash AS, Fienberg SF, Louis TA, Normand SL, Stukel TA, Utts J. Statistical issues in assessing hospital performance. Committee of Presidents of Statistical Societies 2012;. [Last accessed on 2019 Nov 20] Available from: https://escholarship.umassmed.edu/qhs_pp/1114 2012. |
| 32. | Bardach NS, Asteria-Peñaloza R, Boscardin WJ, Dudley RA. The relationship between commercial website ratings and traditional hospital performance measures in the USA. BMJ Qual Saf 2013;22:194-202. |
| 33. | Keenan PS, Normand SL, Lin Z, Drye EE, Bhat KR, Ross JS, et al. An administrative claims measure suitable for profiling hospital performance on the basis of 30-day all-cause readmission rates among patients with heart failure. Circ Cardiovasc Qual Outcomes 2008;1:29-37. |
| 34. | D'Errigo P, Seccareccia F, Rosato S, Manno V, Badoni G, Fusco D, et al. Comparison between an empirically derived model and the euroSCORE system in the evaluation of hospital performance: The example of the italian CABG outcome project. Eur J Cardiothorac Surg 2008;33:325-33. |
| 35. | Werner RM, Bradlow ET, Asch DA. Does hospital performance on process measures directly measure high quality care or is it a marker of unmeasured care? Health Serv Res 2008;43:1464-84. |
| 36. | Jha AK, Orav EJ, Li Z, Epstein AM. The inverse relationship between mortality rates and performance in the hospital quality alliance measures. Health Aff (Millwood) 2007;26:1104-10. |
| 37. | Shiekhzadeh R. Designing a model for evaluation and management of the health system of the country. Management Research. 2010;3 (10):83-108. [Last accessed on 2019 Nov 20] Available from: http://ensani.ir/fa/article/301140. |
| 38. | Janati A, Dadgar E, Sadegh Tabrizi J, Asghari Jafarabadi M, GHolamzade Nikjoo R. Health system professionals, attitude towards necessary criteria for hospitals managers, performance assessment. Sci Mag Yafte 2012;14:91-101. [Last accessed on 2019 Nov 20]Available from: http://yafte.lums.ac.ir/article-1-809-fa.html. |
| 39. | Bahadori M, Izadi AR, Ghardashi F, Ravangard R, Hosseini SM. The evaluation of hospital performance in Iran: A systematic review article. Iran J Public Health 2016;45:855-66. |
| 40. | Reisi-Nafchy M, Drees F, Mirzaeian RR. Assessment of performance indicators in hospitals university of medical sciences based on the standards of the ministry of health. J Shahrekord Univ Med Sci 2014;15:60-7. [Last accessed on 2019 Nov 20]Available from: http://journal.skums.ac.ir/article-1-1931-fa.html. |
| 41. | Jahangiri A, Jahangiri M. Evaluation of hospital performance using multi-indicative decision making: A case study at Imam Khomeini social security hospital, Arak. Q J Health Manage 2017;8:91-102. [Last accessed on 2019 Nov 20]Available from: http://jhm.srbiau.ac.ir/article_10679.html. |
| 42. | Saeedi Mehr M, Ghosi R, Emami G. Evaluating the urgency of emergency hospital based on lean principles. Health Inf Manage 2018;15:162-7. [Last accessed on 2019 Nov 20]Available from: http://dx.doi.org/10.22122/him.v15i4.3554http://dx.doi.org/10.22122/him.v15i4.3554. |
[Figure 1]
[Table 1], [Table 2], [Table 3], [Table 4], [Table 5], [Table 6]
|
Search Pubmed for