Exploring experience of Iranian medical sciences educators about Best Evidence Medical Education: A content analysis
Seyed Amir Hossein Emami1, Hamidreza Khankeh2, Maryam Karbasi Motlagh3, Nazila Zarghi3, Mandana Shirazi3
1 Department of Medical Oncology, Imam Khomeini Hospital, Cancer Institute; Department of Medical Education, Tehran University of Medical Sciences, Tehran, Iran
2 Department of Nursing, University of Social Welfare and Rehabilitation, Tehran, Iran; Department of Clinical Science and Education, Soder Hospital, Karolinska Institute, Stockholm, Sweden
3 Department of Medical Education, Tehran University of Medical Sciences, Tehran, Iran
|Date of Submission||22-Jul-2019|
|Date of Acceptance||15-Sep-2019|
|Date of Web Publication||31-Dec-2019|
Department of Medical Education, No. 57, Hojatdust St., Keshavarz Blvd., Tehran
Source of Support: None, Conflict of Interest: None
BACKGROUND AND OBJECTIVE: Applying the Best Evidence Medical Education (BEME) in real educational arena is a necessity in medical education. As to the literature, there are enough evidence; however, their application by educators and policymakers has been still failed. Therefore, this study conducted to explore the experience of educators about applying BEME in Iranian context.
MATERIALS AND METHODS: Qualitative approach using content analysis method was utilized for exploring 25 participants involved with medical education in different levels, introduced the study using purposeful sampling. Data were collected through a semi-structured interview by which they answered to researcher's questions in around 45 min about how they apply evidence in their educational setting. To make more clarification, probing questions were used. Interviews were recorded and transcribed and then analyzed by coding paradigm immediately.
RESULTS: Three categories were emerged as: applying different levels of evidence, substitution of evidence-based medicine for BEME, and variation of understanding BEME. The first category includes subcategories of using personal experience, textbooks, and filtered papers. The second contains lack of knowledge about BEME elements, time and motivation as well as no priority for applying available medical education evidence; and third, using different terminology and having some problems in applying process, based on individual understanding and using papers with or without modification.
DISCUSSION: For effective evidence application, it is necessary to operationalize BEME terminology and overcome any ambiguity surrounded it. It is also important to suggest educators to apply the appraised evidence as well as teach them how they search and appraise evidence independently. Certainly, in the first steps, supervision and providing a proper context for BEME applications are crucial.
Keywords: Best evidence medical education, content analysis, experience, Iranian, medical sciences educators
|How to cite this article:|
Emami SA, Khankeh H, Motlagh MK, Zarghi N, Shirazi M. Exploring experience of Iranian medical sciences educators about Best Evidence Medical Education: A content analysis. J Edu Health Promot 2019;8:247
|How to cite this URL:|
Emami SA, Khankeh H, Motlagh MK, Zarghi N, Shirazi M. Exploring experience of Iranian medical sciences educators about Best Evidence Medical Education: A content analysis. J Edu Health Promot [serial online] 2019 [cited 2020 Apr 9];8:247. Available from: http://www.jehp.net/text.asp?2019/8/1/247/274486
| Introduction|| |
Of the crucial challenges in medical education is applying the best evidence in real settings. In evidence-based sciences, critical appraisal, and the process through which research findings are applied in real setting play an important role. In medicine, evidence-based practice has been extensively accepted, it is known as making decisions through conscientious, explicit, and judicious use of best available evidence for the treatment and care of the patient. Evidence-based practice in many health professions rooted in paradigm of evidence-based medicine (EBM); however, in medical education, a new emerging movement has formed recently by which education moves more and more toward the Best Evidence Medical Education (BEME). BEME means that medical educators apply best available educational evidence, either educational approaches or educational methods in their practice. Applying the Best Evidence Medical Education has been developed in recent decades, some steps have been considered for applying evidence in medical education. In this process, educators should make a professional judgment on their practice based on the QUESTS criteria resulted from BEME collaboration guidelines as well.
Applying evidence plays an important role in medical education, so that it has been considered as one of the excellence strategies. Van der Vleuten highlighted lack of applying evidence in medical education: “I noticed that my new colleagues' clinical and biomedical researchers had the same academic values as I did, which reassured me and made me feel comfortable. However, I quickly noticed something peculiar; the academic attitudes of the researcher appeared to change when educational issues were discussed.” It seems critical appraisal and scientific scrutiny was suddenly replaced by personal experiences and beliefs, and sometimes by traditional values and dogmas. In 2000, Medical teacher, affiliated by the Association of Medical Education in Europe (AMEE), published an article and emphasized that it is necessary to move from opinion-based medical education to evidence-based medical education in which the highest level of evidence is allocated to systematic reviews and meta analyses, In AMEE 2011, it was focused that producing systematic review and meta-analyses is not the endpoint of BEME, but we should try to find strategies for their application., Geoff Norman, in annual BEME symposium, held in AMEE 2014, said that we have produced enough evidence, so, now the priority is the application of available evidence in real settings. Therefore, the mission of universities is not just running courses for conducting systematic reviews and meta-analyses, but promoting the educational system, where the evidence has been applied, is more important.
Several universities such as Ohio established evidence-based education systems, where, not only the evidence has been produced, but also applied, and the results of evidence application have been reported regularly. As to the literature, the final step of systematic review is its application, thus Best Evidence Medical Education should be utilized in real settings through which both educators and learners, involved with teaching and learning process. It plays the crucial role because of their access to evidence and their active engagement in using them, however, they should acquire related skill in this field. The evidence cannot be helpful unless educators have accessibility and know how they can apply them in their environment. On the minus side, BEME is still in its first steps; educators attended in faculty development courses, are learning gradually the basic concepts and moving to produce systematic reviews, and QUESTS criteria have been thought in graduate courses of medical education. Although education development centers are trying to move forward, it is affected by several factors and this process takes a long time. In Iranian medical sciences universities, several theses and dissertations have been conducted either for producing evidence based on the BEME protocol since 2010 or other educational guidelines before that; on the other hand, BEME International Collaborating Centre has been started its activity in Tehran University of Medical Sciences since 2014, still the application of evidence is a challenge in Iranian context. As applying evidence as a phenomenon follows a process trend, because of its context-bound, multi-dimensional and dynamic nature, it is necessary to aware of individuals' live experience who is involved with. As to the literature, no study was conducted to explain BEME process through exploring their experience. Therefore, this study conducted to explore faculty members' experience of BEME in the Iranian context.
| Materials and Methods|| |
The present study is a qualitative content analysis which explores the experience of the Iranian board of medical education members and medical sciences educators about BEME from September 2016 to June 2018. Participants were selected through purposeful sampling including educators, policymakers, national board members of medical education, people who were teaching BEME and EDC managers of top Iranian universities of medical sciences based on inclusion criteria: Experience on BEME and to be involved with medical sciences education at least more than 5 years. Besides, they accepted to interview and talk about their experience. Due to access to maximum variation of data, participants were selected from different universities and different work services. They were working in different fields of basic and clinical sciences such as cardiology, community medicine, health education, and promotion, curriculum planning, distance-learning planning, emergency medicine, epidemiology, internal medicine, medical education, orthopedics, pediatrics, pediatric dentistry, pharmacology, ophthalmology, and psychology.
Participants who met the study criteria for entering the study were interviewed. Each semi-structured interview was recorded, transcribed, and analyzed as soon as possible just before going to the next interview. The next interview was set based on the previous one; thus, the sampling was continued until data saturation,, in which no additional dimension added to the concepts.
Each interview took around 45 min. Following set an appointment with participants in an appropriate location (most of them preferred to interview in their office), aims and objectives of the study were described, and the informed consent was taken. They were assured that all data would be kept anonymously and confidential, and just the results would be reported. The permission for recording interview was also taken after approving by TUMS Committee of Ethics (IRB Code: 9121486003). Following ethical consideration, the study was switched.
The main question was: “Would you please talk about your experience about applying an educational evidence in real setting?” Then based on the answer, exploratory and probing questions were asked to get more and deeper answer or clarify the ambiguity. They were also asked to give some examples as well. Member check was done during interview by reflecting participants' responses to them to ensure the right understanding of his/her saying. Constant comparison analysis was followed through the study. The inductive approach and coding paradigm were used for the content analysis of all interviews. Meaning units were conceptualized and reached the abstract level, which were extracted from participants' quotations in the form of initial codes. Researcher's memos also helped to analysis. The codes were refreshed several times to be placed by meaning similarity to form the subcategories and categories. All data were reviewed several times for ensuring the validity of subcategories and categories resulted from codes. The researcher tried to be bracketed through the analysis process.
Four criteria for credibility, dependability, confirmability, and transferability were used to increase the trustworthiness of findings in qualitative research. In this study, validity and credibility were achieved through prolonged engagement with data, enough time for data collection and analysis, triangulation of data collection, member check, and expert check. These experts selected from people who were familiar with both qualitative methods and medical education.
Transferability was confirmed by providing a clear and deep explanation about context, participants, and setting. Dependability was attained using external evaluator to review the data, memos, and results. Some university professors confirmed the analysis by review the documents, some interviews, codes and emerged subcategories and categories. Consequently, confirmability was obtained.
| Results|| |
This content analysis was performed on 25 participants who met the study criteria. Of 25 participants, 10 people were female and the rest were male; the youngest participant had at least 5 years educational experience, and the oldest one had more than 30 years educational experience. All participants were involved with educational activities in different fields of medical sciences [Table 1].
Interviews were analyzed using constant comparison strategy in which analysis simultaneously continued during the study. Data emersion was achieved through the continuous back and forth returning to data and prolonged engagement with them. Researchers tried to be bracketed during the data analysis process.
Following analyzing 25 interviews, three categories were emerged as applying different levels of evidence, substitution of EBM for BEME and different understanding of BEME. The results have been demonstrated in [Table 2].
|Table 2: Experience of participants about applying the Best Evidence Medical Education in the Iranian context|
Click here to view
Applying different level of evidence
Although participants focused on applying evidence, their inference about the evidence was different. As to the participants, the following subcategories were emerged:
Trusting on personal experience
When applying evidence, most participants stopped at the lowest level, i.e., expert opinion. For example, one of them stated that “I taught this lesson several times and know well how I should teach each section. I strongly rely on myself. I regarded myself as a solid evidence” (P10).
Some participants just trust on textbooks when they face to some dilemma. They believed that textbooks contain updated evidence, gathered in a valuable source. One said that “I think the best evidence are textbooks. I tried to refer textbooks when facing to problem” (P17).
Using filtered papers
Another group of participants tried to take a look at different journals. They found them more updated than any other source of evidence. One of them stated that “Sometimes papers published in reputed journals could be helpful. I trust on them and try to apply them if necessary” (P13).
Substitution of Evidence Based Medicine for Best Evidence Medical Education
As most participants were clinician, so it was rational to apply EBM principles instead of BEME approach for solving their educational problems. For this reason, their statements revealed following subcategories.
Knowledge deficit about Best Evidence Medical Education elements
Most participants, did a general search in Google when face to an educational problem, it may be due to lack of knowledge about BEME and its constituents such as retrieving system and journals. One of them honestly confessed that: “Whenever I face to problem, I try to Google and solve it. That's it.”
On the other hand, normally, clinicians expected to access to similar sources which are available in EBM. They stated that when medical educators have either limited or no access to appraised sources of evidence, consequently BEME has failed. For instance, one of them stated that “As educators familiar with EBM, have access to sources such as up to date in medicine and there is no appraised source for educational evidence, the application face to some problems and challenges” (P15).
Lack of time and motivation for applying educational evidence
Some of them referred to the necessity of incentives. As to one of them: “With no rewarding system, nobody does it. Reward or grant makes people motivated.” As most of them were clinician, it is rational that their concentration was more on clinic than education. So, education has been overshadowed by clinic. A participant stated that “Well, let's say my career is focused on my clinical responsibility rather than any other thing. I try to search the-state-of-the-art evidence for treatment rather than education” (P5).
No priority for applying available medical education evidence
Despite some participants believed in applying evidence, they complained about lack of time from one side and working overload from the other side. Therefore, regarding the working overload and some encumbering regulations which created a complex situation for participants, they preferred to make a revision on their priorities. One participant clearly said that “Honestly, it is not my priority for it. I have no time to achieve my educational goals in allocated time, let alone to apply evidence. Surely it is ideal, but not for us with working overload.”
Variation of understanding Best Evidence Medical Education
Participants reported a variety of understanding of BEME, which lead to the following subcategories:
Applying different terminology
Participants moved through a spectrum from modified evidence-based to evidence-informed in their practice, for example, a participant said that “I prefer and try to be evidence-informed rather than evidence based. Because it is impossible” (P20).
Having some problems in applying process, based on individual understanding
Some participants tried to apply evidence in different levels; however, most often, they experienced some problems in a real setting. Sometimes, they tried to look at the literature for making slight changes in their activities or to be evidence-informed, for example, a participant stated that “We try take a look to current literature before making decision” (P20).
Using papers with or without modification
Some people believed that if you would like to apply evidence, you have to change it. They justified that as all educational elements are different, it is impossible to use without modification. For instance, participants believed that: “Evidence could not be useful unless we modified them, because the contexts are different” (P4).
| Discussion|| |
This qualitative study was conducted to explore the experience of educators about applying BEME in the Iranian context.
The first category was related to applying different levels of evidence by educators. As to the literature, different levels of evidence could be applied in real settings following a critical appraisal.” In medical education, the first level allocated to evidence obtained from meta-analysis of randomized controlled trials, the second level related to evidence obtained from at least one randomized controlled trial, the third level is associated with evidence obtained from at least one well-designed controlled study without randomization, the forth level referred to evidence obtained from at least one other type of well-designed quasi-experimental study, the fifth level pointed to evidence obtained from well-designed nonexperimental descriptive studies, such as comparative studies, correlation studies, and case studies, and finally, the sixth level is evidence obtained from expert committee reports or opinions and/or clinical experiences of respected authorities.” According to participants' statements, although they applied different levels of evidence such as their own personal experience as expert opinion, textbooks, and papers, they ignored critical appraisal. On the basis of the literature, Van der Vleuten believed that critical appraisal and scientific scrutiny were suddenly replaced by personal experiences and beliefs, and sometimes by traditional values and dogmas in education which is in agreement with our result, however, based on the BEME guide 1, it is necessary to use different levels of evidence for decision-making after critical appraisal. Of the most well-known appraisal criteria is QUESTS in which “Quality as how good is the evidence, Utility as to what extent can the method be transferred and adopted without modification? Extent as what is the extent of the evidence? Strength as how strong is the evidence? Target as what is the target? What is being measured? How valid is the evidence? And finally, setting as how close does the context or setting approximate? How relevant is the evidence?” Therefore, rely on just expert opinion is not enough for making educational decisions. On the other hand, at the best situation, they apply other appraisal systems which have been applied in EBM such as CONSORT for trial studies, PRISMA, MOOSE, for secondary studies, STROBE for observational studies and so on. Although these appraisal tools applied in medical sciences are valid in its own field, literature yields moving on QUESTS criteria in appraising medical education evidence means that we are in the right track.
Textbooks and papers are also considered as solid evidence; however, some papers are not filtered and publish in nonreputed journals. As medical education is a new emerging area, developed as interdisciplinary field, it is different from education and medical sciences; so, it is necessary to choose valid journals and publication for resolving educational problems in different fields. On the other hand, the systematic search and sources of evidence for medical education are somehow different. The most important challenge in searching medical education evidence is lack of inclusive and complete sources dedicated to this field. Neither bibliographic databases contain the majority of evidence, nor indexed database is allocated to medical education.
The second category is substitution of EBM for BEME. All participants were aware of applying evidence in their educational settings. Most of them were working in different clinical disciplines and experience EBM as applying clinical evidence in their own disciplines as clinicians, so it is crystal clear that they follow their own approach, EBM approach, in their education. They did not know the difference between EBM and BEME in terms of retrieval system, searching, and even the BEME sources. They reported that whenever they face any educational problem, they honestly Google. On the minus side, the retrieval systems for BEME are different from EBM such as Campbell Collaboration, ERIC, and BEME collaboration, which are specific to education in different fields and levels. However, some resources such as Cochrane and MEDLINE cover both clinical and educational evidence, applied more by clinicians.
The last category is allocated to variation of understanding BEME. Most of the participants applied evidence in either decision-making or educational settings. When asking them for their experience, they reported that they search to see what is going on the world about their problem they face to. Some of them tried to modify the evidence based on their own situation. Hence, interestingly, they search evidence; however, applying it in real setting is not based on the BEME approach. Even if it is supposed to consider utility in QUESTS criteria, this kind of modification has been failed. On the other hand, just referring to one part of an appraisal system seems to be inefficient. Based on the participant, they seems to be evidence-informed instead of evidence-based.
Although pioneers of medical education believed to use evidence in teaching and learning by educators, some scholars reported using evidence by policymakers more than teachers in daily educational activities. It is in line with the present study in which most participants stated that due to some challenges such as time limitation, lack of reward system, and fat curriculum, teachers are not able to be evidence-based. On the other hand, some issues such as context and culture seem to interfere with the establishment of evidence-based education in medical sciences fields.
In order to have an effective evidence application, it is necessary to operationalize BEME terminology and overcome any ambiguity surrounded it. The most important strength and novelty of this paper are referring to the importance of evidence application in medical education, as the educational setting area based on participants' experience. Clinical educators usually used to utilize evidence in their daily practice which has been reported in several studies, however, as far as we know, no study especially qualitative one has been conducted to explain the status of applying medical education evidence and doing practice-based education. This is in line with the literature that yields clinical educators report unsatisfactory educational performance because they are not familiar with evidence-based practice in education.
The main limitation of this study was occurred during sampling. As participants had several administrative positions beside their educational roles, on the other hand, they were working in different universities, making an appointment for doing interview was so difficult, especially for the second interviews. It caused a 2-year duration for data gathering and analysis.
| Conclusion|| |
As to the results, applying BEME is not only related to teachers' lack of knowledge and abilities about BEME approach, but also related to the context and educational policies such as time limitation due to fat curriculum, working overload, and lack of rewarding system for people who use educational evidence, priority of treatment to education in our context, etc. Hence, it is strongly recommended that both policymakers and teachers involved with the BEME establishment. Consequently, it is important to teach educators how they can search and appraise evidence independently in their real settings. On the other hand, policymakers seem to provide proper context for using BEME.
Authors offer their special thanks to participants for their valuable participation in this study.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
Hart IR, Harden R. Best evidence medical education: A plan for action. Med Teach 2000;22:131-5.
Davies P. What is evidence-based education? Br J Educ Stud 1999;47:108-21.
Harden PL, Best R. evidence medical education: The simple truth. Med Teach. 2000;22:117-9.
Harden RM, Grant J, Buckley G, Hart IR. BEME guide no. 1: Best evidence medical education. Med Teach 1999;21:553-62.
Haig A, Dozier M. BEME guide no. 3: Systematic searching for evidence in medical education – Part 1: Sources of information. Med Teach 2003;25:352-63.
van der Vleuten C. Evidence-based education? Am J Physiol 1995;269:S3.
Thistlethwaite J, Davies H, Dornan T, Greenhalgh T, Hammick M, Scalese R. What is evidence? Reflections on the AMEE symposium, Vienna, August 2011. Med Teach 2012;34:454-7.
Ramani S. Twelve tips to promote excellence in medical teaching. Med Teach 2006;28:19-23.
And finally. Med Teach 2014;36:186-90.
Picus LO, Odden A. Review and Analysis of Ohio's Evidence-Based Model. Vol. 2. Picus Odden and Associates; 2009.
Patrício M, vaz Carneiro A. Systematic reviews of evidence in medical education and clinical medicine: Is the nature of evidence similar? Med Teach 2012;34:474-82.
Best evidence medical education: Report of meeting-3-5 December 1999 London, UK. Med Teach 2000;22:242-5.
Hammick M. Evidence shaped health professional education: Can we talk about a new paradigm? Med Teach 2012;34:435-8.
Thomson SB. Sample size and grounded theory. J Adm Gov 2010;5:45-52.
Corbin J, Strauss A. Strategies for qualitative data analysis. Basics of Qualitative Research Techniques and Procedures for Developing Grounded Theory; 2008. p. 3.
Sadeghi T, Seyed Bagheri SH. Challenges of teacher-based clinical evaluation from nursing students' point of view: Qualitative content analysis. J Educ Health Promot 2017;6:72.
Sattar K, Memon S, Siddiqui D, Mughal SA. Too early or too late adopting the best evidence medical education: A positive chang. J Postgrad Med Inst (Peshawar-Pakistan) 2017;31.
Ringsted C, Hodges B, Scherpbier A. 'The research compass': An introduction to research in medical education: AMEE guide no. 56. Med Teach 2011;33:695-709.
Wiltshier J. Evidence-Based Education: Benefits and Challenges. Second Language Acquisition-Theory and Pedagogy: Proceedings of the 6th
Annual JALT Pan-SIG Conference; 2007.
Lacasse M, Audétat M, Boileau É, Caire Fon N, Dufour M, Lafferrière M, et al
. Remediation Interventions for Undergraduate and Postgraduate Medical Learners with Academic Difficulties: A BEME Systematic Review. BEME-Best Evidence Medical Education; 2019.
[Table 1], [Table 2]