Problems and challenges in providing feedback to clinical teachers on their educational performance: A mixed-methods study
Sepideh Jamshidian1, Nikoo Yamani1, Mohammad Reza Sabri2, Fariba Haghani1
1 Department of Medical Education, Medical Education Research Center, Isfahan University of Medical Sciences, Isfahan, Iran
2 Pediatric Cardiovascular Research Center, ICRI, Isfahan University of Medical Sciences, Isfahan, Iran
|Date of Submission||21-Jun-2018|
|Date of Acceptance||22-Sep-2018|
|Date of Web Publication||29-Jan-2019|
Dr. Fariba Haghani
Department of Medical Education, Medical Education Research Center, Isfahan University of Medical Sciences, Isfahan
Source of Support: None, Conflict of Interest: None
INTRODUCTION: Given the importance of feedback to improve teachers' educational performance, the current study aimed to identify the challenges and problems involved in providing feedback to clinical teachers on their educational performance.
MATERIALS AND METHODS: In the first phase of this mixed-methods study, the clinical teachers, students, and university officials were interviewed on feedback problems and challenges. After an inductive content analysis, and based on the problems enlisted, a ten-item questionnaire was developed. Subsequently, 25 clinical teachers completed the questionnaire by scoring the importance and urgency of each problem.
RESULTS: Overall, 18 individual and group interviews were conducted with 24 people. A total of ten themes and five categories emerged, including “lack of transparency of feedback system,” “absence of criticizability culture,” “lack of motivation to improve performance,” “failure to consider factors affecting teacher's performance,” “lack of formative evaluation,” “inappropriate data obtained on teacher's performance,” “inappropriate feedback providers,” “inappropriate feedback recipients,” “inappropriate feedback provision,” and “no feedback follow-up.” Three items of priority involved “inappropriate feedback providers,” “inappropriate data,” and “failure to consider factors affecting the teacher's performance.”
CONCLUSIONS: All the raised problems obtained high scores; nevertheless, the interviewees had more problems with the feedback provider, the data collected on the performance, and inattention of university officials to the factors influencing performance than feedback presentation methods. Hence, gaining the trust of teachers on the collected data, providing feedback by their trustworthy individuals, and paying attention to the factors influencing teachers' performance are associated with an increased possibility of feedback acceptance.
Keywords: Challenges, clinical, evaluation, feedback, problems, teacher, teaching
|How to cite this article:|
Jamshidian S, Yamani N, Sabri MR, Haghani F. Problems and challenges in providing feedback to clinical teachers on their educational performance: A mixed-methods study. J Edu Health Promot 2019;8:8
|How to cite this URL:|
Jamshidian S, Yamani N, Sabri MR, Haghani F. Problems and challenges in providing feedback to clinical teachers on their educational performance: A mixed-methods study. J Edu Health Promot [serial online] 2019 [cited 2020 May 26];8:8. Available from: http://www.jehp.net/text.asp?2019/8/1/8/250935
| Introduction|| |
Performance evaluation and feedback are two important components of human resource management systems in organizations. Universities use the teacher evaluation system to manage their most important human resources, that are, faculty members.,,,, Different aspects of a teacher's performance are evaluated, the most important of which is teaching. Teacher performance evaluation requires collecting data on educational activities, comparing these data with specific standards, and judging about the extent to which predetermined goals have been achieved. Researches have shown that the mere delivery of the evaluation results cannot promote a teacher's performance, but the teacher needs to receive appropriate feedback as well. This feedback is an interactive process between the feedback provider and recipient with the aim of informing the feedback recipient about his/her performance to improve it.,,
Teacher evaluation has two major purposes: performance improvement and accountability. Performance improvement is the main function of formative evaluation, while summative evaluation focuses on keeping the teacher responsible for his/her performance and is associated with a range of career implications such as annual promotion and reward for the teacher. In other words, the main objective of summative assessment is management decisions.,,
Feedback can be provided for both formative and summative evaluation. Feedback based on formative evaluation, that is, formative feedback, helps the teacher to improve his/her performance before the summative evaluation is performed. Summative feedback can lead to improved performance in subsequent courses. Formative feedback is provided in a safe and nonjudgmental environment whereas summative feedback, even in the best circumstances, is associated with judgment on the teacher. Many educational organizations use the results of teacher evaluation for both formative and summative purposes, while many sources describe the integration of improvement and accountability functions in teacher evaluation as challenging.
Evidence suggests that, regardless of whether the feedback is summative or formative, appropriate feedback can play a significant role in enhancing performance. In Marten's study, executive officials played an important role in building capacity and competency of teachers by providing appropriate feedback. According to Lyon et al.'s study at two German medical schools, structured-feedback provision based on an expert rater's observation of teaching and students' completion of the evaluation form are a powerful and cost-effective process to promote teachers' performance. Many higher education institutions in North America provide feedback to their teachers based on end-of-course questionnaires completed by learners, known as end-of-course feedback which is a powerful tool to help teachers to promote their performance in the coming semesters. Although Kluger and DeNisi in their meta-analysis showed that feedback in one-third of the cases leads to poor performance, they did not question the feedback per se, but considered several contributors to feedback efficacy and presented them in their subsequent paper. In other words, if feedback is provided in an appropriate manner, it can be very effective in performance improvement.
In Iranian universities of medical sciences, the Education Development Center (EDC) is responsible for evaluating and providing feedback to teachers. The teacher evaluation unit at the EDC of Isfahan University of Medical Sciences (IUMS) has been involved in summative evaluation of teachers since 1989. Nevertheless, it has sufficed to provide no more than the evaluation scores to the teachers. Students and university officials are surveyed using questionnaires, and the results are presented to teachers through a written report. The report indicates the percentage frequency of responses to each item on a 5-point Likert scale (never = 1 and always = 5) and the average total score calculated (out of 100). Moreover, the report covers the mean score (out of 100) given by the department head and other officials to the teacher on the five items of “educational discipline,” “doing educational tasks,” “professional behavior,” “teaching quality,” and “respect for ethical principles.” In other words, no written or in-person feedback is provided to the teacher except for the evaluation report.
Since a significant change on a teacher's performance is not expectable when only evaluation scores are presented, one priority of the EDC in IUMS in recent years has been to devise a program, whereby feedback can be provided to clinical teachers. The success of feedback relies heavily on the context and culture in which it is presented. Therefore, to design an appropriate feedback provision program, it is necessary, first and foremost, to understand the existing conditions, challenges, and problems concerning both teacher evaluation and feedback provision. Alongside this, the present study aimed to identify problems and challenges of providing feedback to clinical teachers on their educational performance.
| Materials and Methods|| |
This mixed-methods study was conducted in two phases in 2017. In the qualitative phase, the views of internal medicine faculty members, students, university officials, and evaluation experts in the EDC and the Education Development Office about the problems and challenges of the feedback system were obtained through semi-structured interviews and analyzed using content analysis method. Given the importance of internal medicine as one of the major courses for medical students, the teachers of this department were selected for interview. Moreover, due to the numerous clinical challenges in the Al Zahra Hospital, most of the interviews were held with the teachers working at this large hospital.
The interviews were conducted by the researcher (S.J.) supported by the interview guide after coordination with the interviewees. All interviews were recorded with interviewees' permission. When 18 individual and group interviews were done with 24 individuals, the data were saturated. After each interview, its audio file was listened and transcribed verbatim and examined using inductive content analysis method in the Excel software (version 2016). After reading the transcripts several times, the meaningful units, which included descriptive concepts implying the problems and challenges of the feedback system, were identified and coded. Similar codes were integrated into one category and similar categories were merged into one theme. Sometimes, it was found necessary to introduce new categories for a theme. This process was reiterated several times with the help of the other researchers, and each time, changes were made to themes and categories.
In the second phase, based on the results of the first phase, a preliminary questionnaire with 16 items was developed to prioritize the problems. An internal medicine and two medical education teachers reviewed and revised the questionnaire to finalize it with ten items, each representing one problem. The faculties had to score the importance and urgency of each item with scores from 1 to 10, with larger scores indicating greater importance/urgency. Overall, 25 internal medicine faculty members returned the questionnaire of which three questionnaires were removed as they were incomplete, and 22 were analyzed using descriptive methods in the Excel software. Prioritization of problems was accomplished based on the sum of scores given to importance and urgency.
Credibility, dependability, and transferability were considered to guarantee trustworthiness. The credibility of the study describes the extent to which the data and analytical processes have been able to focus on the topic in mind. Measures taken to provide credibility in this study involved data collection from individuals with different genders, ages, and work experiences; long-term interaction between the interviewer and the teachers and visiting the teachers in different situations; content analysis being reviewed by two other researchers and interview data being confirmed by the questionnaire; and presentation of representative quotations to show how data were coded and arranged.
For dependability, it is helpful to consider changes in data and the researcher's decisions over time. In this study, while the questions and the interview process evolved over time, they did not change in their original structure.
To facilitate the readers' judgment about the transferability of the results to their own situations, cultural and background conditions of the study and all activities are described, and excerpts from the interviews are quoted.
This study was approved by the ethics committee of IUMS. Oral informed consent was obtained from each participant before the study. To observe confidentiality, transcripts were delivered to the three other researchers anonymously and quotations were cited without implying anybody.
| Results|| |
Overall, 14 individual and 4 group interviews were conducted with 24 individuals (12 women). Of the interviewees, 14 were from the internal medicine department (8 women), 5 were medical students, and the rest were either an evaluation expert or university official.
In the qualitative phase, ten themes and five categories emerged with respect to the challenges and problems of feedback provision to clinical teachers.
Lack of transparency of the evaluation and feedback system
Non-transparency of evaluation processes
The only transparent part of the evaluation is the formal summative evaluation conducted by the EDC; no formal formative evaluation is performed. However, several teacher evaluations are conducted which are not defined as part of the summative or formative evaluations. For instance, in basic medical science and preclinical courses, questionnaires and interviews supervised by the faculty deputy of education are administered to collect the views of selected students about teachers' instruction; subsequently, feedback is given to the dean of the faculty and the department head.
The unofficial evaluations in the clinical courses are conducted differently in various educational departments; in the internal medicine department, externs evaluate teachers in their logbook. This evaluation is reported to the department head and if necessary, to the teacher him/herself.
Although these evaluations can be a good basis for feedback provision to teachers, the main problem with them is the vagueness of purpose, methods, and the performances being evaluated. In fact, teachers get informed of such evaluations indirectly through colleagues. Moreover, the results of these evaluations are usually not given directly to the teacher him/herself.
Teachers Group Interview (TGI) 3: “The teachers have been deceived many times. For example, the logbook was supposedly designed for externs to see if they are learning well in the ward. But there are items on the arrival and departure time of the teachers, and practically, the logbooks are used to evaluate the teachers.”
Faculty officials mentioned several reasons for these evaluations, including the fact that in some courses, some teachers' teaching hours is not enough to be evaluated in summative evaluation. Another reason claims on the untrustworthiness of the summative evaluation results; in addition to the students, the school officials also believed that these informal evaluations are much more effective on the teachers' performance than the summative evaluation.
Students Group Interview (SGI): “These informal evaluations had a significant impact on the teachers. They know that they are being evaluated and that the evaluation is going to be effective.”
Vague criteria for desirable performance
The teachers stated that neither they themselves nor feedback providers nor learners are well aware of the desirable performance criteria.
Individual Interview (II) 3: “The teacher does not know how to work. You have put him/her in the system without training. Now you say why s/he does not know how to do things? You should ask yourselves. Have you ever taught the teacher that now you are expecting from him/her?”
Sometimes, the predetermined standards cannot be useful to teachers because they are not compatible with the circumstances wherein the teachers work.
When learners are not familiar with the standards of good performance, they may evaluate a teachers' performance with shades of inaccuracy and error.
II 5: “A few teachers teach some skills that are beyond the curriculum demands. Therefore, the students may think that other teachers have failed to teach perfectly.”
Unlike the teachers, officials stressed that the standards of clinical education was given to teachers and that even the items on evaluation questionnaires indicate the expectations from teachers.
Absence of the criticizability culture
According to the interviewees, to give and receive feedback are challenging to most people. Even when they are asked to provide feedback, they still hardly express their true opinions and prefer not to comment on somebody's performance in fear of harming their relationship. On the other hand, most people do not feel good about receiving constructive feedback and focus on the feedback-givers intention rather than on the feedback content.
SGI: “Criticizability does not make sense in our country. We all want to put the blame on others, for example, we would say that the data is not reliable.”
SGI: “Once, the feedback was given to the teacher during the semester. The teacher contacted the students complaining “Why have you rated me as such?”
Lack of motivation to improve performance
One of the contributors to teachers' indifference and unmotivatedness toward performance improvement is that the teachers do not receive positive feedback from university officials on their strengths. In their view, everyone needs to assure that their work is seen by officials and feel that their efforts have had an impact on others.
II9: “I do not know whether my work at the end of the last week was monitored at all. Was it good or not? It makes me gradually feel that it does not matter to anyone how I teach... Therefore, why should I spend so much time? you may love teaching so much, but because there is nothing to give you energy, you would lose all your motivation.”
Another reason for the lack of motivation is no differentiation between the teachers who try to teach effectively and those who seem not to value teaching.
II12: “In the summative evaluation, the teacher who scores 90 and the one who scores 70, have no difference in annual promotion. All receive the educational grade on the promotion checklist... meaning that something is being done that does not result in a particular outcome.”
Another contributor to unmotivatedness is the reverse impact of teachers' efforts on evaluation scores. Almost all of them stated that more stringent teaching would be associated with lower evaluation grades given by learners.
II 4: “At first, when someone enters the educational setting, s/he is stringent and wishes to teach energetically. After a while, others will tell him/her “Why are you so hard on teaching resulting in necessity?.”
Failure to consider factors affecting the teacher's performance
A concern of many teachers is the feedback providers' inconsideration of factors beyond the teacher's control, such as inappropriate educational setting and inappropriate planning, which result in underperformance. Clearly, feedback provided in such circumstances cannot be accepted by the teachers.
II 10: “You cannot just blame the faculty members. Al Zahra Hospital setting is set up for patient care, but well-performed teaching is expected from the teachers. If the interns fail to follow the pathology reports of patients, it may happen that the pathologic results won't be obtained in ten days.”
Lack of formative evaluation
The impact of evaluation on career promotion was challenging to faculty members. Although they did not refer to “formative evaluation,” what they mentioned was the lack of a safe environment for feedback acceptance and performance improvement, the safety that cannot be expected from the summative evaluation.
II 4: “The problem is that evaluation plays a role in the teacher's career and promotion, so that when you take a harder approach to working, it becomes to your disadvantage.”
II 2: “If it is just a matter of performance, it will be easier to accept, and then there is someone (feedback giver) who helps me to be a better person.”
Inappropriate data obtained on the teacher's performance
Most of the teachers, faculty officials, and students did not consider the data obtained from the summative evaluation questionnaire as reliable.
II9: “Teacher X, with whom everybody is dissatisfied, about whom interns say his teaching is not useful, no morning report presence, nor any satisfaction with his class,… how did he earn the score necessary for promotion. How has he promoted every year?”
The codes for this theme were sorted into three categories.
Feedback sources are people who can see a teacher's performance and can evaluate it. Although the teachers considered learners as the best source of performance evaluation, they were skeptical about the reliability of the data from learners. Their most important concerns were nonresponsible learners, poor accuracy in providing honest answers, and retaliation for teacher's rigor at the time of teacher evaluation. According to most teachers, learners are more inclined to faculty members who are lenient; in other words, the outcome of academic rigor is reflected in evaluation score directly.
II3: “Sometimes, students' dissatisfaction is not justifiable; sometimes, they do not feel like working. It has been my wish for a while to have a student in my class, at any level whatsoever, who would say “I have studied this and searched about it, and have visited the patient, but I did not understand what to do.” Rarely do we see that.”
II8: “We were residents, we realize how residents complete the questionnaires. I remember the chief distributed the evaluation questionnaires to be completed. We didn't feel like doing. Sometimes, we highlighted that the teacher in charge of residency gave difficult exams; we all filled out badly for him/her.”
Students also pointed to the inaccuracy in completing questionnaires, and one of the reasons was lack of sufficient motivation because they perceived that their opinions were not valued.
SGI: “Always, they appeared at the end of the term with a series of questionnaires to evaluate the teachers. We asked the senior students and they confirmed the same routine. We filled out, but no change occurred the next terms. How do I become motivated to complete the questionnaire?”
TGI: “The problem is that learners do not trust in the evaluations. What happens after they complete the questionnaires? Does it make a change? Learners perceive that no matter how well they complete the questionnaire. No better situation would occur, nor the teacher would make a change for the better.”
In addition to the learners, officials are not in the position to assess some aspects of teaching, because they do not observe the teaching directly.
Inappropriate tools and procedures
Problems with data collection tools and methods also made the teachers doubt the reliability of data.
II10: “I was on vacation for a long while, but they evaluated my teaching!”
SGI: “The problem is no supervisor present while completing the questionnaire. So, the students fill them out from top to bottom impatiently.”
Another problem was inappropriate time and situation for completing the questionnaires and the number of items on the questionnaires.
SGI: “After the exam, you will be asked to complete several questionnaires all at once..about. 120 items altogether. I have not filled out myself.. We went on a rotation two months ago; now they are giving us the questionnaire!”
Faculty members found it inappropriate to rely merely on questionnaires as the only tool for data collection. Sometimes, it is necessary for learners to express their constructive comments on the teachers' performance rather than rating them.
Unlike the other interviewees, the evaluation officials stressed on the correct structure of the questionnaires based on the general model of teaching and considered the results as reliable.
Both the teachers and evaluation officials were concerned about the inadequacy of the completed questionnaires. They mentioned sometimes only four or five residents return the questionnaire.
Inappropriate feedback provider
Given the lack of in-person sessions of feedback provision to faculty members, the interviewees had different perceptions of who the feedback provider was. Most of the faculty members considered university officials as the feedback provider. The greatest challenge of faculty members was inadequate information about the teachers' performance. Few faculty members considered EDC as the feedback provider; rather, they generally viewed EDC officials as inappropriate because they were hardly aware of the clinical environment.
II 3: “You are sitting in the EDC and assuming the faculty members required to do all this. This is possible only theoretically. The like of me cannot do all this together, unless 24 h becomes 98 h, and I work like a bulldozer, and unless I am stronger than my computer, which often hangs down.”
Inappropriate feedback recipient
According to most of the interviewees, it is very difficult to provide feedback to clinical teachers. Some teachers strongly resist against the feedback, even when the feedback is not directly related to their performance and is related, for example, to the executive activities under their supervision.
Another challenge was underestimation of teaching by some teachers and not caring to improve their educational performance. In spite of inadequate performance, such individuals find their performance appropriate and do not accept feedback. In fact, the number of teachers who think they need feedback is very small.
SGI: “You think how many of the teachers are looking for their problems or improving themselves?!”
SGI: “When the teacher has not changed his slides or added a new reference from 10 years, or 15 years ago; when s/he does not want to improve her/his exams, it shows there are more important concerns than teaching for her/him.”
Because teachers easily receive the required education scores for annual promotion, they are more inclined to complete their research scores.
II1: “Once, Doctor X told me: They do not ask how long your round takes. they will count your research papers and say this one is an ISI paper and what is the impact factor of that one. That's how you promote. Nobody notices to your round!”
Another problem is that most teachers overestimate their abilities. Most of those with greater years of experience think that they have become proficient in teaching and show greater resistance to feedback.
II 9: “Many of them feel they are perfect, that nobody has the right to find their mistakes.”
TGI 3: “One of the faculties once related that he was given a low teaching score, but that he was pretty sure of himself. Once I was present at one of his conferences. I understood that he was far from how he described himself. I mean he overestimates his performance.”
Inappropriate feedback provision
Given the lack of in-person feedback sessions, feedback provision referred to summative evaluation reports. Some of the faculty members who participated in teaching -review sessions perceived feedback provision as the feedback on these sessions. Overall, they do not feel good about receiving only a written feedback or the evaluation report, especially when the evaluation score is lower than expected. They need to receive more explanation about their performance and speak about their concerns; which is not fulfilled by receiving the evaluation report only. Consequently, this unpleasant feeling remains unmet without being raised anywhere or solved anyhow, and leads to feelings of disappointment and sometimes incorrect decisions.
II10: “I do not agree that the letter [evaluation report] be handed to you without any explanation. It is not useful. I did not understand at all what it wanted to say. Now what should I do in what regard?”
II 10: “I had a bad feeling. I said it is worthless to work, to try, and to strive in this university.”
Sometimes, in the case of informal evaluations conducted in the departments, feedback is provided to faculty member in the presence of others.
SGI: “Some feedback principles are common everywhere. For example, no one likes his/her problems to be reported before colleagues. Interestingly, the reverse is happening here, I mean, the faculty members' mistakes are uncovered in the department's meetings, the most sensitive place.”
In addition to feedback presentation methods, the interviewees mentioned problems with the message and content of the feedback. For example, the reasons for the scores given are not described.
II 12: “One of the faculties whose professional ethics score was low, asked about the exemplifications of the items. “I want to correct myself”. This is the faculty member's right to know. The evaluation reports are ambiguous and too general.”
One of the problems with feedback provision lies with inattention to the strengths in one's performance.
II 1: “Everyone has some strengths besides his/her mistakes. The evaluator is aware of the strengths, but does not mention. I mean, they say that “we only note the weaknesses in order to be improved.” However, this leads to “unmotivatedness and frustration.”
No feedback follow-up
In many occasions, faculty members have not accepted the evaluation report or the feedback so as to act upon, and sometimes, the faculty members do not have a proper understanding of it. Currently, there is no feedback follow-up system and only if the teacher has sufficient motivation, s/he may be able to find the reason(s) for the reported evaluation scores. Without follow-up, it cannot be understood whether the accepted feedback has led to improved performance, and if no change has occurred, what the underlying reasons are.
In the second phase of the study, the problems raised concerning feedback were prioritized based on the total mean scores of importance and urgency [Table 1].
|Table 1: Problems with feedback provision in priority in view of internal medicine faculty members|
Click here to view
| Discussion|| |
The first phase of this study examined the viewpoints of clinical teachers, students, and university officials on the challenges and problems of the evaluation and feedback system, and the second phase prioritized the problems from the perspective of clinical teachers. Lack of transparency or unintelligibility of different aspects of evaluation for all stakeholders was one of the most important challenges to the evaluation and feedback system. Except for the summative evaluation, other evaluations were quite vague to the teachers in terms of purpose, procedures, assessors, and performance criteria, hence the teachers' trust in the system is reduced. The teachers and officials had rather opposing views on the transparency of performance criteria. University officials insisted on notifying about the standards of clinical education, while almost all the faculties emphasized their lack of awareness. Therefore, it should be ensured that all teachers have received and understood these criteria. To hold a panel discussion may possibly contribute to better understanding of performance standards and adherence to them. According to Sadler's study, awareness of good performance standards is an essential requirement to benefit from feedback.
Another challenge, that is, “absence of the criticizability culture,” indicates that teachers are reluctant to receive and give feedback. In such a culture, the great opportunity that feedback could provide for improvement, would not be used. Kluger's meta-analysis and Ramani's study demonstrate the influence of cultural contexts on the acceptance or nonacceptance of feedback., In fact, the learning culture could create conditions and opportunities that facilitate feedback acceptance.,,
The consequences of feedback over time, are significant in shaping organizational culture. Sometimes, these outcomes create a climate of unmotivatedness among feedback recipients; for example, after receiving feedback, some faculty members conclude that stringency in education results in student dissatisfaction, which leads to a decrease in teacher's evaluation score.
One concern of the teachers was “failure to consider factors affecting the teacher's performance” which was identified as the second priority problem. Feedback providers' consideration of the role of undesirable environmental factors indicates that they understand the recipients' conditions, which can increase the likelihood of feedback acceptance.
A problem raised here was lack of formative evaluation. Although, there is some degree of judgment in formative evaluation, because of the comparison made between an individual's performance and the desired performance, it is not comparable to the summative evaluation. As the judgment latent in evaluation encourages the feedback recipient toward resistance to protect him/herself from emotional harm, the feedback from the summative evaluation is expectedly associated with greater resistance, especially when indicating one's undesirable performance. Some organizations consider a range of low- to high-stake evaluations instead of combining formative and summative evaluation functions or separating them completely., The greater the sensitivity and importance of evaluation, the more intense the role of judgment; similarly, the less the sensitivity and importance of evaluation, the more intense the supportive role. Importantly, regardless of the views on the formative and summative evaluations, the twofold purpose of evaluation, that is, performance improvement and accountability should be taken into focus. Performance improvement requires a safe and nonjudgmental environment in which a good situation for feedback acceptance is provided by performance description and presenting effective strategies. If evaluations always affect the faculty's career implications (summative evaluation), there are practically no opportunity for supporting them and improving their performance. Care must be taken to avoid mistrust in the evaluation system upon the incorrect integration of the formative and summative evaluations. Some organizations use formative evaluation and feedback data for the purpose of improvement, but they later use these results in decision-making situations. This reduces trust in the organization and makes people unwilling to use feedback to make changes. DeNisi emphasizes that the staff should be informed as soon as possible about the purpose of the evaluation (improvement or decision-making). Furthermore, if changes are made to the way the data are to be used, they should be informed of the reason for the change.
The third priority problem of the faculty was “inappropriate data obtained on the performance of the teacher.” The teachers considered learners as the direct observers of performance and the best feedback source. Nevertheless, application of the questionnaire as the only data collection method was inadequate and inappropriate in their view. Most interviewees emphasized on an increased number and diversity of learners, multiple methods for data collection, and informing the learners about their role in enhancing the teachers' performance. Since the positive perspective of feedback recipient on the data credibility plays an important role in feedback acceptance, it would be really worthwhile to addressing it. On the contrary, when people feel that there are no credible processes for data collection and correct judgment on their performance, they are more likely to reject feedback.
The highest priority of the “inappropriate feedback provider” indicates the importance of the feedback-provider's role for most teachers. This significant role has been investigated and highlighted in numerous studies. Feedback reception will be accompanied with more satisfaction and the effect will be greater in the long-term as much as the feedback provider is credible to the recipient. Researches have shown that feedback recipients who attribute critical comments to the negative intentions of the feedback provider, give less value to the feedback content. Sometimes, the personal relationship between the feedback provider and the feedback recipient prevents the feedback acceptance. However, similarity in their cultural background is one of the contributors to performance improvement after receiving feedback.
Some characteristics of feedback recipients can be problematic in the feedback process. For example, teachers who do not highly value their educational performance and lecturers with years of tenure are more reluctant to accept feedback as they perceive themselves of sufficient expertise and have more self-confidence. The impact of the feedback-recipient's perception of the feedback value and characteristics such as self-esteem on feedback acceptance has been investigated in numerous studies.
“Inappropriate feedback provision” involves the presentation method and the transferred content. Most of the teachers did not feel good about receiving only an evaluation report, because they have no opportunity to comment on them nor does it provide strategies for improvement and it does not include any appreciation for good performance. The teachers also complained about receiving feedback in the presence of their colleagues. Several studies have been conducted on the impact of various feedback contents. For example, not all the problems should be raised in a feedback session; rather, two or three priorities that are relevant to the needs of the recipient should be presented. Aguinis has emphasized the use of strength-based feedback instead of the traditional weakness-based approach and listed nine useful suggestions in this regard.
According to the interviewees, especially the students, there is no guarantee that the person will use the feedback in practice if there is no follow-up. First, it should be checked whether the teachers have properly understood and accepted the feedback message and whether they have made any changes required in their performance. In Veloski's systematic review, the positive effects of feedback were correlated with “credible source of feedback” and “longer-duration follow-up.”
The prioritization in the second phase of the study was somewhat consistent with the results obtained in the first phase. All the problems received high scores. The second and third priorities, that is, “failure to consider factors affecting performance” and “inappropriate data obtained on the teachers' performance,” were predictable from previous interviews. The faculties were by far more greatly in trouble with these two priorities than with the method of feedback provision. We faced a bit of a challenge in interpreting the first priority, that is, inappropriate feedback providers. Given the lack of in-person feedback sessions, the teachers' perception of the feedback provider was apparently the overall teacher evaluation system, which means the EDC as the entity in charge of evaluation as well as the faculty officials as evaluation raters.
One of the strengths of this study is the accurate interviews held with a large number of key figures associated with evaluation and feedback provision; on the other hand, the limitation involves the failure to interview with residents, as one of the most important sources of feedback data.
| Conclusions|| |
The present study addressed and prioritized the concerns and challenges of clinical teachers of the internal medicine department concerning receiving feedback on their educational performance. Overall, the most important challenges of faculty members in the feedback process relate to the “feedback provider,” “data collection,” and “factors affecting the faculty members' performance.” In other words, choosing an appropriate feedback provider and providing feedback based on the data deemed reliable by faculty members can greatly contribute to the acceptance of feedback and its efficacy, in particular, when the feedback provider and evaluation officials can demonstrate to feedback recipients that they are informed of the many factors influential on the teacher's performance (clinical problems and the like). These results could be valuable to evaluation and feedback system officials, because the system can be improved by considering them.
We, hereby, express our sincere gratitude to the faculty members of the internal medicine department who, despite their hectic time, proved that education, evaluation, and feedback are worthwhile and collaborated in this research with their valuable comments.
The authors would like to appreciate Isfahan, Medical Education Research Center, and the National Agency for Strategic Research in Medical Education for financial support of this study
Financial support and sponsorship
This study was financially supported by Isfahan Medical Education Research Center, Isfahan University of Medical Sciences, Isfahan, Iran. project No. 395719 and the National Agency for Strategic Research in Medical Education, Tehran, Iran grant No. 960217.
Conflicts of interest
There are no conflicts of interest.
| References|| |
DeNisi AS, Murphy KR. Performance appraisal and performance management: 100 years of progress? J Appl Psychol 2017;102:421-33.
Yamani N, Yousefy A, Changiz T. Proposing a participatory model of teacher evaluation. Iran J Med Educ 2006;6:115-21.
Goldfarb S, Morrison G. Continuous curricular feedback: A formative evaluation approach to curricular improvement. Acad Med 2014;89:264-9.
Shakurnia A. Faculty attitudes towards student ratings: Do the student rating scores really matter? Iran J Med Educ 2011;11:84-93.
Yarmohammadian M, Bahrami S, Foroughi Abri A. Educational Administration and Planning: Isfahan University of Medical Sciences; 2010.
Saif AA. Educational Measurement, Assessment and Evaluation. Tehran: Doran Publications; 2004. p. 128.
Boerboom TB, Jaarsma D, Dolmans DH, Scherpbier AJ, Mastenbroek NJ, Van Beukelen P, et al.
Peer group reflection helps clinical teachers to critically reflect on their teaching. Med Teach 2011;33:e615-23.
Haghani F, Fakhari M. Feedback in clinical education: Concept, barriers, and strategies. Iran J Med Educ 2014;13:869-85.
Hattie J, Timperley H. The power of feedback. Rev Educ Res 2007;77:81-112.
Dinmohammadi MR, Jalali A, Bastani F, Parvizi S, Borimnejhad L. Bazkhord: Onsore asasie amoozeshe balini (vazhehnameh tosifi). Iran J Med Educ2001;9:5.
Santiago P, Benavides F. Teacher evaluation: A conceptual framework and examples of country practices. OECD Review on Evaluation and Assessment Frameworks for Improving School Outcomes; 2009. p. 1-2.
OCDE, OCDE. Teachers for the 21st
Century. Using Evaluation to Improve Teaching. París: OECD Publishing; 2013.
Saif AA. Modern Educational Psychology. 6th
ed. Tehran: Dowran publication; 2009.
Stronge JH. Evaluating Teaching. A Guide to Current Thinking and Best Practice. California: Corwin Press; 2006.
Nolan J Jr., Hoover LA. Teacher Supervision and Evaluation: theory into practice. Hoboken, New Jersey: John Wiley and Sons; 2011.
Marten JL. To whom do they turn? Expert Teachers' Experiences with Feedback [dissertation]. Wisonsin: Marian University; 2015.
Lyon HC Jr., Holzer M, Reincke M, Brendel T, Ring J, Weindl A, et al.
Improvements in teaching behavior at two German medical schools resulting from a modified flanders interaction analysis feedback intervention process. Med Teach 2014;36:903-11.
Springgay S, Clarke A. Mid-course feedback on faculty teaching: a pilot project. In: Farr Darling L, Erickson G, Clarke A. Collective Improvisation in a Teacher Education Community. Dordrecht: Springer; 2007. p. 171-84.
Kluger AN, DeNisi A. The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychol Bull 1996;119:254.
DeNisi AS, Kluger AN. Feedback effectiveness: Can 360-degree appraisals be improved? Acad Manage Exec 2000;14:129-39.
Iman M, Noushadi M. Qualitaive content analysis. Pazhouhesh 2012;3:15-44.
Graneheim UH, Lundman B. Qualitative content analysis in nursing research: Concepts, procedures and measures to achieve trustworthiness. Nurse Educ Today 2004;24:105-12.
Jalili M, Khabaz Mafinejad M, Gandomkar R, Mortaz Hejri S. Principles and Methods of Student Assessment in Health Professions. 1st
ed. Tehran: The Academy of Medical Sciences; 2017.
Nicol DJ, Macfarlane-Dick D. Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Stud Higher Educ 2006;31:199-218.
Ramani S, Post SE, Könings K, Mann K, Katz JT, van der Vleuten C, et al.
“It's just not the culture”: A qualitative study exploring residents' perceptions of the impact of institutional culture on feedback. Teach Learn Med 2017;29:153-61.
Watling C, Driessen E, van der Vleuten CP, Lingard L. Learning culture and feedback: An international study of medical athletes and musicians. Med Educ 2014;48:713-23.
Curtis DA, O'Sullivan P. Does trainee confidence influence acceptance of feedback? Med Educ 2014;48:943-5.
Perrella A. Room for improvement: Palliating the ego in feedback-resistant medical students. Med Teach 2017;39:555-7.
Ende J. Feedback in clinical medical education. JAMA 1983;250:777-81.
Leung K, Su S, Morris MW. When is criticism not constructive? The roles of fairness perceptions and dispositional attributions in employee acceptance of critical supervisory feedback. Hum Relat 2001;54:1155-87.
van de Ridder JMM, Berk FCJ, Stokking KM, Ten Cate OTJ. Feedback providers' credibility impacts students' satisfaction with feedback and delayed performance. Med Teach 2015;37:767-74.
van de Ridder JM, McGaghie WC, Stokking KM, ten Cate OT. Variables that affect the process and outcome of feedback, relevant for medical training: A meta-review. Med Educ 2015;49:658-73.
Mann K, van der Vleuten C, Eva K, Armson H, Chesluk B, Dornan T, et al.
Tensions in informed self-assessment: How the desire for feedback and reticence to collect and use it can conflict. Acad Med 2011;86:1120-7.
Aguinis H, Gottfredson RK, Joo H. Delivering effective performance feedback: The strengths-based approach. Bus Horizons 2012;55:105-11.
Veloski J, Boex JR, Grasberger MJ, Evans A, Wolfson DB. Systematic review of the literature on assessment, feedback and physicians' clinical performance: BEME guide no 7. Med Teach 2006;28:117-28.