Assessment of performance in teaching hospitals: Using multicriteria decision-making techniques
Mehdi Jafari1, Maryam Seyedjavadi2, Rouhollah Zaboli3
1 Department of Health Services Management, School of Health Management and Information Sciences, Iran University of Medical Sciences; Health Managers Development Institute, Ministry of Health and Medical Education, Tehran, Iran
2 Department of Health Services Management, School of Health Management and Information Sciences, Iran University of Medical Sciences, Tehran, Iran
3 Department of Health Administration, School of Health, Baqiyatallah University of Medical Sciences, Tehran, Iran
|Date of Submission||26-Jan-2020|
|Date of Acceptance||05-Apr-2020|
|Date of Web Publication||31-Aug-2020|
Dr. Maryam Seyedjavadi
Department of Health Services Management, School of Health Management and Information Sciences, Iran University of Medical Sciences, Tehran
Source of Support: None, Conflict of Interest: None
BACKGROUND: It is essential to evaluate the performance of hospitals in the health system. Hospitals need a performance evaluation system to develop and compete in order to measure the efficiency and effectiveness of their programs, processes, and human resources. This study aimed to evaluate the performance of teaching hospitals using the Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) method and hierarchical analysis.
MATERIALS AND METHODS: This was a cross-sectional and descriptive study conducted in 2019 in all teaching hospitals affiliated to Shahid Beheshti University of Medical Sciences. The required data were collected using a standard checklist. The collected data were analyzed using the analytic hierarchy process (AHP) and TOPSIS. In the first phase, annual indicators of hospital evaluation were collected. Following the AHP, key performance indicators (KPIs) were selected and prioritized in hospitals.
RESULTS: The questionnaires were provided to 15 experts to weigh KPIs, and the most important indicators were selected. The results of hierarchical analysis showed that three main indicators in evaluating the performance of hospitals were bed turnover rate, emergency clients, and length of stay.
CONCLUSIONS: One of the problems in evaluating hospitals is the use of key indicators that alone measure the quantity or quality of their performance. Multicriteria decision-making can be used to determine key indicators first, and then by combining these indicators into a multicriteria decision-making model, a better assessment of the role and performance of hospitals can be provided.
Keywords: Hospitals, Iran, process assessment, teaching
|How to cite this article:|
Jafari M, Seyedjavadi M, Zaboli R. Assessment of performance in teaching hospitals: Using multicriteria decision-making techniques. J Edu Health Promot 2020;9:214
|How to cite this URL:|
Jafari M, Seyedjavadi M, Zaboli R. Assessment of performance in teaching hospitals: Using multicriteria decision-making techniques. J Edu Health Promot [serial online] 2020 [cited 2020 Dec 4];9:214. Available from: https://www.jehp.net/text.asp?2020/9/1/214/293955
| Introduction|| |
It is essential to evaluate the performance of hospitals in the health system because hospitals are the most expensive part of the health system and they have the most staff. Performance assessment is critical for every organization such that lack of assessment system in different dimensions of organization is considered as a symptom of disease in organization. Performance appraisal is a numerical scale that provides managers with information needed to evaluate and monitor the status and current activities of the hospital. One of the performance evaluation tools is the use of dynamic multicriteria decision-making.,
Hospitals face numerous risks to the environment that affect the performance of the system and make it less efficient. Hospitals need a performance evaluation system to develop and compete in order to measure the efficiency and effectiveness of their programs, processes, and human resources. Hospitals are the key units in the health-care system and have a key role in providing health-care services. Hospitals play the most important role in advancing health goals in countries and are recognized as the leader of health care.
Assessment as a process to judge the efficiency of predefined programs needs the special tools and patterns. Till now, different models with various attributes have been provided to assess the performance of health-care providers. Due to the necessity of observing the principle of productivity in the use of resources, a model based on hospital performance indices is provided. On the other hand, one of the criteria for measuring the success of organization and determining how to achieve its goal is organization's statistics and indices.
Hospital performance in resource utilization is also assessed by certain indices. It can be expected to promote system efficiency by analyzing and planning in order to improve performance indices. However, performance can only be used to achieve important policy, planning, and resource management while the most significant indicators are selected, and by understanding their interactions, the performance of hospitals can be monitored continuously.
Hospital indices are the most significant factors in hospital performance that should be reviewed regularly. Studies indicated that there are various indices to assess the hospital performance which four of them are the most significant and applicable ones: bed occupancy, bed turnover rate (BTR), average length of stay (LOS), and bed turnover interval (BTI).
Given that assessing hospital performance by significant performance indices is one of the methods for identifying the problem, many researchers have been interested to compute and compare these indices in the country. Some were comparing these indicators with existing standards. On the other hand, some determined the factors affecting the increasing or decreasing the indices. However, none of them pay attention to simultaneous comparison of these indices and combine them.,,
The Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) is one of the techniques that can simultaneously consider different indices for assessment and prioritization. It is a multiple attribute decision-making model. This simple and useful decision-making technique was proposed by Yoon and Hwang (1981) to solve the multiple attribute decision-making problems. This technique is based on choosing an option with shortest distance to the positive ideal solution and the longest distance to the negative ideal solution. In short, the ideal positive solution and the ideal negative solution include the best available benchmarks and the worst available benchmarks, respectively.
Many studies have evaluated the performance of hospitals using TOPSIS and other multicriteria decision-making methods. However, the main disadvantage of TOPSIS is the lack of weights as well as the lack of judgmental compatibility. Therefore, this technique needs an efficient procedure to determine the significance of different indices based on the target. Analytic hierarchy process (AHP) that is another multiple attribute decision-making model can provide such a procedure. In addition, to provide a logical conclusion, this model can cover the limitation of the TOPSIS method by comparing the pairwise indices.,
Analytic hierarchy process was invented by Thomas L. Saaty at 1980. This is one of the most reliable and strong techniques in multiple attribute decision-making that makes us able to compare indices in pairs and to measure their scores against just an index in order to achieve a sum score for each index. It is obvious that weighting the indices in the pairwise comparisons is easier and safer than other methods. Using this model in weighting also prevents bias in assessments. This study aimed to evaluate the performance of general hospitals of Beheshti University in Tehran using TOPSIS method and hierarchical analysis.
| Materials and Methods|| |
This was a cross-sectional and descriptive study conducted in 2019 in all teaching hospitals affiliated to Shahid Beheshti University of Medical Sciences (n = 11). The required data were collected using a standard checklist. The collected data were analyzed using the AHP and TOPSIS technique. In the first phase, annual indicators of hospital evaluation were collected. Following the hierarchical analysis, key performance indicators (KPIs) were selected and prioritized in hospitals. These KPIs were bed occupancy, BTR, average LOS, BTI, inpatient discharge, outpatient discharge, emergency clients, and surgery capacity.
In the second phase, we used the TOPSIS method to assess the hospital performance. TOPSIS was selected as one of the compensatory classic methods in multiple attribute decision-making to solve the problems of prioritization based on similarity to ideal solution developed., TOPSIS method, a MADM technique, was used to perform prioritization in this study. The TOPSIS technique consists of the following steps:
- Compute the normalized decision matrix. The normalized value rij is calculated as:
- Calculate the weighted normalized decision matrix. The weighted normalized value vij is calculated as:
Vij = wirij; j = 1,…, J i = 1,…., n
Where wi is the weight of the ith attribute or criterion and
- Determine the ideal and negative ideal solution.
Where I′ is associated with advantage criteria and I″ is associated with cost criteria.
- Calculate the separation measures, using the n-dimensional Euclidean distance. The separation of each alternative from the ideal solution is given as:
Similarly, the separation from the negative-ideal solution is given as:
- Calculate the relative closeness to the ideal solution. The relative closeness of the alternative aj with respect to A* is defined as:
- Rank the preference order.
The whole process was performed by MATLAB software.
The study was approved by the Student Research Committee of School of Health Management and Information Sciences of Iran University of Medical Sciences and got ethical permission for publication (IR.IUMS.REC1395.30020).
| Results|| |
The questionnaires were provided to 15 experts to weigh KPIs, and the most important indicators were selected. The results of hierarchical analysis showed that main indicators to assessment of hospital performance were BTR, LOS and emergency clients [Table 1].
|Table 1: Weight and rank of key performance indicators using analytical hierarchy process|
Click here to view
According to TOPSIS analysis of hospital performance over a 1-year period based on key indicators, Hospital A had the best performance in total university hospitals and F and K hospitals had the worst performance [Table 2].
|Table 2: Assessment of hospital performance by the Technique for Order of Preference by Similarity to Ideal Solution technique|
Click here to view
| Discussion|| |
Efficiency means maximizing the use of resources to generate efficiency. In order to determine the efficiency or inefficiency, each firm should use appropriate indexes as a benchmark. The efficiency of the hospital in utilizing the resources can also be measured with the help of specific indicators, and by analyzing it and planning to improve the efficiency indicators, one can expect to improve the productivity in the system. In selecting performance indicators, they can only be used to achieve important policy, planning, and resource management goals by first selecting the most important indicators and then by understanding the relationship between the selected indicators at a glance. Overall, but at the same time, accurate monitoring of hospital performance is possible using the simultaneous indicators.,
Studies show that there are different indicators to measure the efficiency of hospitals, the most important of which are the three indicators of bed occupancy rate, bed rotation rate, and average length of hospital stay. The common feature of most of these studies is that they are less concerned with the simultaneous comparison of indices. However, the studies have shown that use of hybrid indicators in performance appraisal can caused better results.
Using the Pabon Lasso diagram is one of the tools that can be used to compare the important hospital indices and evaluate in terms of efficiency in managing the affairs. This method, which is one of the active methods of problem identification in the hospital, enables the management to have an analysis of of hospital performance., In a similar study, Tourani et al. evaluated the performance of hospital wards using TOPSIS and concluded that using the TOPSIS method would make a better judgment on hospital performance and treatment for policy-makers and managers.
On the other hand, establishing organizational and organizational coordination in hospitals, especially in critical situations, requires a system that can measure the key indicators of the hospital in order to provide a link between hospitals in critical situations. The system is based on key indicators, so evaluating and setting up a joint evaluation system in hospitals can create an inter-hospital coordination network and improve systemic therapeutic efficiency., Hospital performance is partly a function of hospital management style. Therefore, it is possible to identify and correct the performance indicators with the exact and scientific method of troubleshooting and problems in the method of administering hospitals. It is the style of hospital management that can change key performance and indicators.
Evaluation, which is used as a process for judging the efficiency of predefined programs, requires the use of specific tools and models. Using scientific and research methods of operations in evaluating and judging the performance of hospitals can provide managers and policy-makers with a better picture. Multicriteria decision-making methods are used in hospital evaluation today and confirm the findings of this study. The use of hybrid and mathematical methods is less commonly used by organizations due to the complexity of computation, but scientific centers require the use of scientifically accurate evidence for decision-making. This study provides an opportunity for university-level managers and has been able to provide a more accurate picture of hospital performance compared to similar hospitals. The strength of this study was that it was measured using hybrid performance indices, and the disadvantage of these methods is the dominance of the efficacy perspective and the effectiveness of treatment centers is ignored.
| Conclusions|| |
Measuring the performance of a hospital is a complex task. It should be measured using combined indicators of hospital performance. The study showed that the three indicators of bed occupancy, LOS, and outpatient referral were the most important performance indicators of educational hospitals. The use of routine indicators alone cannot provide a correct view of the performance of a teaching hospital. It is recommended to determine and measure cost savings using key decision-making methods in managing key indicators.
This article is based on a research project titled “Performance measurement and ranking of teaching hospitals of Shahid Beheshti University of Medical Sciences based on efficiency indicators, using multiple criteria decision making” with code of ethics: IR.IUMS.REC1395.30020 and grant number: 95-04-193-30020, sponsored by the Student Research Committee of School of Health Management and Information Sciences of Iran University of Medical Sciences. We would like to thank all individuals who have contributed to this study.
Financial support and sponsorship
This study was financially supported by the Student Research Committee of School of Health Management and Information Sciences of Iran University of Medical Sciences (grant number: 95-04-193-30020).
Conflicts of interest
There are no conflicts of interest.
| References|| |
Malekzadeh R, Mahmoodi G, Abedi G. A comparison of three models of hospital performance assessment using IPOCC approach. Ethiop J Health Sci 2019;29:543-50.
Zhou JA, Lu ZX, Cheng JQ. Introduction on performance assessment of community health in Shenzhen. Chin General Practice 2005;5:4.
Li C, Yu C. Performance evaluation of public non-profit hospitals using a BP artificial neural network: The case of Hubei Province in China. Int J Environ Res Public Health 2013;10:3619-33.
Bahadori M, Izadi AR, Ghardashi F, Ravangard R, Hosseini SM. The evaluation of hospital performance in Iran: A systematic review article. Iran J Public Health 2016;45:855-66.
Shafii M, Hosseini SM, Arab M, Asgharizadeh E, Farzianpour F. Performance analysis of hospital managers using fuzzy AHP and Fuzzy TOPSIS: Iranian experience. Glob J Health Sci 2015;8:137-55.
Zhang L, Liu R, Jiang S, Luo G, Liu HC. Identification of key performance indicators for hospital management using an extended hesitant linguistic DEMATEL approach. Healthcare (Basel) 2019;8:2-21
Wu X, Huang Z, Shen S, editors. Comprehensive Evaluation of Medical Service Ability of TCM Hospitals in 30 Provinces, Autonomous Regions and Municipalities of China in 2017 Based on Entropy Weight TOPSIS Method and RSR Method. Proceedings of the 2019 The World Symposium on Software Engineering; 2019.
Ebrahimi P, Taghi Nattaj Darzi Naghibi M, Vatankhah S, Faghanzadeh Ganji G. The relationship between performance indicators and readmission of patients with open heart surgery: A case study in Iran. Hosp Pract Res 2019;4:62-7.
Adali EA, Tuş A. Hospital site selection with distance-based multi-criteria decision making methods. Int J Healthcare Manag 2019;1-11. [Doi: 10.1080/20479700.2019.1674005].
Hosseini SM, Bahadori M, Raadabadi M, Ravangard R. Ranking Hospitals Based on the Disasters Preparedness Using the TOPSIS Technique in Western Iran. Hosp Top 2019;97:23-31.
Liao H, Mi X, Yu Q, Luo L. Hospital performance evaluation by a hesitant fuzzy linguistic best worst method with inconsistency repairing. J Cleaner Produc 2019;232:657-71. [Doi: 10.1016/j.jclepro.2019.05.308].
Gang W, Yan S, Yiwei H, Lei L, Jingyi Y, Jing M, et al
. TOPSIS evaluation of hospital performance based on disease diagnosis related group (DRGs). Chin Med Record 2017;02:17.
Palczewski K, Sałabun W. The fuzzy TOPSIS applications in the last decade. Procedia Comp Sci 2019;159:2294-303.
Büyüközkan G, Çifçi G. A combined fuzzy AHP and fuzzy TOPSIS based strategic analysis of electronic service quality in healthcare industry. Expert Syst Appl 2012;39:2341-54.
Akkoç S, Vatansever K. Fuzzy performance evaluation with AHP and topsis methods: Evidence from turkish banking sector after the global financial crisis. Eurasian J Bus Econom 2013;6:53-74.
Bandalos DL, Finney SJ. Factor analysis: Exploratory and confirmatory. In: The Reviewer's Guide to Quantitative Methods in the Social Sciences. New York: Routledge; 2018. p. 98-122.
Zaboli R, Tourani S, Seyedin SH, Oliaie Manesh A. Prioritizing the determinants of social-health inequality in Iran: A multiple attribute decision making application. Iran Red Crescent Med J 2014;16:e12607.
Opricovic S, Tzeng GH. Compromise solution by MCDM methods: A comparative analysis of VIKOR and TOPSIS. Europ J Operat Res 2004;156:445-55. [Doi: 10.1016/S0377-2217(03) 00020-1].
Shafii M, Rafiei S, Abooee F, Bahrami MA, Nouhi M, Lotfi F, et al
. Assessment of service quality in teaching hospitals of Yazd University of medical sciences: Using multi-criteria decision making techniques. Osong Public Health Res Perspect 2016;7:239-47.
Chen CT, Hung WZ, editors. Evaluating the Service Quality of Hospital by using TOPSIS with Interval Type-2 Fuzzy Sets. 2017 International Conference on Fuzzy Theory and Its Applications (iFUZZY), IEEE; 2017.
Najafi S, Mahmoudi G, Dabbaghi F, Moradian M. The effect of health system reform plan on the performance indicators of hospitals affiliated with Mazandaran University of medical sciences. J Adv Pharm Educ Res 2019;9:177-82.
Badiee Aval S, Adel A, Hosseini SJ, Ebrahimipour H, Askarzadeh E. The effect of health evolution plan on the performance indicators in hospitals of Mashhad University of medical sciences using the Pabon lasso model. Manag Strat Health Syst 2019;3:290-8.
Rezaei S, Hajizadeh M, Nouri B, Ahmadi S, Rezaeian S, Salimi Y, et al
. Iranian hospital efficiency: A systematic review and meta-analysis. Int J Health care Quality Assurance 2019 32:385-97. [Doi: 10.1108/Ijhcqa-03-2018-0067].
Tourani S, Hassani M, Ayoubian A, Habibi M, Zaboli R. Analyzing and prioritizing the dimensions of patient safety culture in emergency wards using the TOPSIS technique. Glob J Health Sci 2015;7:143-50.
Hirano H, Higashi K, Sakamoto Y. DNA polymerase in nuceoli isolated from Ehrlich ascites tumor cells. Biochem Biophys Res Commun 1975;67:518-24.
Amerioun A, Alidadi A, Zaboli R, Sepandi M. The data on exploratory factor analysis of factors influencing employees effectiveness for responding to crisis in Iran military hospitals. Data Brief 2018;19:1522-9.
Amerioun A, Hosseini Shokouh SJ, Nejati Zarnaqi B, Zaboli R, Karimi Zarchi AA. Management style of military hospitals and its relationship with hospital performance indicators. J Military Med 2013;15:61-70.
[Table 1], [Table 2]