A modified tool for “reflective practice” in medical education: Adaptation of the REFLECT rubric in Persian
Saeideh Daryazadeh1, Nikoo Yamani1, Payman Adibi2
1 Department of Medical Education and Medical Education Research Center, Medical Education Development Research Center, Isfahan University of Medical Sciences, Isfahan, Iran
2 Department of Internal Medicine, Research Center of Gastroenterology, Isfahan University of Medical Sciences, Isfahan, Iran
|Date of Submission||16-Sep-2019|
|Date of Acceptance||24-Sep-2019|
|Date of Web Publication||30-Jan-2020|
Dr. Nikoo Yamani
Department of Medical Education and Medical Education Research Center, Medical Education Development Research Center, Isfahan University of Medical Sciences, Isfahan
Source of Support: None, Conflict of Interest: None
CONTEXT: Reflection is a learnable process that enhances long-life learning, clinical decision-making, and can foster empathy and professionalism. One of the methods for teaching reflection is “reflective writing” that is conducted in “reflective practice.” Some tools have been developed to assess “reflective capacity,” and The Reflection Evaluation for Learners' Enhanced Competencies Tool (REFLECT) is one of them.
AIMS: This study aimed to adapt the REFLECT rubric in Persian.
SETTINGS AND DESIGN: This quantitative study was conducted in a medical school with the participation of medical interns through census sampling, and in three stages, including translation, pilot study, and main study.
SUBJECTS AND METHODS: Persian translation was obtained by the “forward/backward translation” method. We made some changes in the tool and used it in the pilot and main study to confirm validity and reliability.
STATISTICAL ANALYSIS USED: Cronbach's alpha coefficient, Pearson correlation, and Cohen's kappa were applied for statistical analysis. Data analysis was performed using SPSS23.
RESULTS: We inserted a numerical value of 1–4 at the reflection levels, and also removed Axis 2 and an optional writing component. In the pilot study, face and content validity was confirmed involving 10 interns and five medical education specialists. Then, 67 interns participated in the main study, and we measured the reliability of the tool by internal consistency through Cronbach's alpha (0.83) and test-retest through correlation coefficient (0.89). The size of the agreement was measured to determine the inter-rater reliability by Cohen's kappa (0.84).
CONCLUSIONS: The modified REFLECT version is a valid and reliable tool that can help us to assess reflective capacity. The use of this tool is recommended for reflective practice in medicine.
Keywords: Educational assessment, medical education, medical writing, professionalism, teaching
|How to cite this article:|
Daryazadeh S, Yamani N, Adibi P. A modified tool for “reflective practice” in medical education: Adaptation of the REFLECT rubric in Persian. J Edu Health Promot 2020;9:24
|How to cite this URL:|
Daryazadeh S, Yamani N, Adibi P. A modified tool for “reflective practice” in medical education: Adaptation of the REFLECT rubric in Persian. J Edu Health Promot [serial online] 2020 [cited 2020 May 27];9:24. Available from: http://www.jehp.net/text.asp?2020/9/1/24/277356
| Introduction|| |
Today emotion in medicine is neglected, and although professional development is still important, it is not considered as an important goal in medical education. Research in pre-1960s shows that the lack of emotions has reduced doctor's effectiveness. At the end of the 1990s, reflection emerged in the literature, and “Medical Humanities” was presented as an approach to connect humanities and medical sciences,, and teaching reflection in medicine increased. Reflection means a process in a person's thoughts that related to experience and can be decomposed and interpreted to make awareness. Hence, the created understanding in this process can be used in the same condition in the future. Accordingly, reflection on past experiences will lead to deeper learning and better performance. Moreover, reflection is a base for professional development; hence, practitioners can aware of their inner values and attitudes. Besides, reflective and critical thinking can enhance “long-life learning” and improve “clinical decision-making,”, also helps physicians to choose difficult or ethical decisions when encountering complicated cases in clinical practice.
On the other hand, reflection can foster empathy and professionalism, and “medical humanities” is an approach that presented reflective writing as a valuable tool to improve “reflective capacity.” This capacity is an important ability that lets the physicians be mindful, interested, aware, and prepared to identify and correct faults, therefore fostering reflection improves professionalism, which is a core competency in medicine.,
Hence, reflection is a learnable process, so teaching methods can be used to foster it, and one of the teaching methods of reflection is reflective writing.,,, An important tip in this method is offering guidance and feedback to learners to make particular skills. Teaching reflective writing is conducted in “Reflective practice,” which is an approach in medical education and considers medical students' and practitioner concerns. These practices are “the higher-order intellectual and affective activities” in practitioners that involve “to critically analyze and evaluate their experiences to lead to new understandings and appreciation of the way they think and operate in the clinical setting.” In the traditional approach of clinical education, physicians are evaluated in skills and outcomes, and trainees gain competencies in procedures only through “repetitive guided practice.”
Assessing reflection can increase learning, and medical teachers can analyze “problem-solving” and clinical “decision-making” process of learners too. Reflective tools are used for assessing reflection levels in narratives. These tools were prepared by theories of reflection and reflective thinking , that can have a positive educational impact if we presented to learners as guidance before practice. Some studies have used these reflective tools for teaching reflection.,,
Various tools have been developed to assess reflection that addressed reflection levels and guided feedback.,,, Mezirow proposed a classification for reflective evaluation that is divided into three parts, including nonreflective, reflective, and critical reflection. This model was used for reflective journals and blogs. Besides, Boud et al. presented a conceptual framework that has been used in seven stages to evaluate the reflective process in journals and blogs. The combination of these two models can be used to create a deeper understanding of reflective writing.,, Some studies applied a combination of the Mezirow's model and the Boud et al. framework to identify the reflection process. In general, reflection divides into reflection and critical reflection and includes seven steps. Step 1 is without reflection, steps 2–4 are the reflection, and steps 5–7 are the critical reflection. Critical reflection is at a higher level of reflection. In reflection, new insights and understanding are obtained; but in critical reflection, understanding is much deeper and dilemmas that have been considered by a person before, are identified.,,,,, Tsingos-Lucas et al. used a reflective tool for assessing reflection in pharmacy students and assessed the reflective narratives of students. This tool was produced in the 7 stages and 3 levels of reflections that its details were extracted from a previous tool used in dentistry education.,,, A modified tool was provided by Kember et al., to evaluate reflection levels in students, which has four levels “habitual action, understanding, reflection, and critical reflection.” Moreover, the “Reflection Evaluation for Learners' Enhanced Competencies Tool” (REFLECT) rubric has developed and recommended by Wald et al. aimed to improve reflection, self-directed learning, and self-assessment as a qualitative and formative assessment to provide written feedback to each learner.
Among the tools reviewed, we chose the REFLECT rubric that was introduced principled and more comprehensive than others. As explained, reflection needs to be trained and should be assessed to improve professional development. There is no tool in Persian to teach and assess reflection, so we performed this study to present the modified REFLECT as a valid and reliable tool in Persian Version.
| Subjects and Methods|| |
This study performed in a quantitative methodology at a medical school and in three phases for adapting the REFLECT rubric that included translation, pilot study, and main study. The sampling method was census, and the study participants were medical students of internal medicine internships.
The Reflection Evaluation for Learners' Enhanced Competencies Tool rubric
An innovative rubric called REFLECT was developed by Wald et al. at Brown University after a comprehensive literature review and several phases, that assess reflective levels in medical students' narratives. This rubric helps to improve reflection by presenting written feedback to students and is a psychometric tool that was confirmed its “interrater reliability, face validity, feasibility, and acceptability.” This rubric was designed to analyze writing through qualitative and formative assessment that improves providing feedback to learners. The REFLECT includes four reflection levels includes (Level 1: Usual action without reflection; Level 2: Thoughtful action or introspection; Level 3: Reflection; and Level 3: Critical Reflection). In addition, the REFLECT focuses the criteria for each level and explains five writing components in a narration (”Range of writing, Presence, Description of the conflict or disorienting dilemma, Attention to emotions, Analysis and construction of meaning”). Furthermore, this tool evaluates narratives for “transformative reflection, and learning and confirmatory learning.” The process of utilizing this tool for narrative analysis involves four stages that include: (1) reading the whole reflective narrative; (2) fragmenting and analyzing the components of narration; (3) investigating the whole narrative taking into account the second stage or “Gestalt;” and (4) justifying and confirming the reflection levels and learning results that have assigned to the narration). The original version of the REFLECT in the English language was produced for formative assessment and has no scoring for reflection levels.
We selected the REFLECT rubric after reviewing the literature as a proper tool for assessing reflection in medical narratives because the levels of reflection and the writing components have described well, and we could enhance providing feedback to learners too. We received permission from one of the rubric's providers for using this tool in our study.
The modified Reflection Evaluation for Learners' Enhanced Competencies Tool
The tool was translated into Persian by three medical education specialists. We selected the final translation after agreement by the panel of researchers, then an English translator translated it back to English. We compared two English translations (original and back-translated versions), and translations were very closely matched, then Persian translation was approved.
Reflection levels and writing components of the tool were approved by five medical education specialists through reviewing the related literature in the field of reflection and narrative writing.
Ten medical interns participated in this phase (nonparticipants in the main study).
The translated tool was used in a pilot study, and some changes were made in the tool based on better understanding from the students' view and easy application for analyzing Persian medical narratives. A confirmed medical narrative was selected by a narrative analyst and gave to the medical interns along with the tool. We explained the tool and questioned each student separately about levels and components in the narrative based on the tool, and noted the ambiguous points. Then, we held a meeting with the participation of all the research members and corrected the ambiguities.
Participants included 67 medical interns.
We asked the interns to write reflective narratives about clinical encounters with their patients two times, at an interval of 2 months in test and re-test. Then, two raters assessed students' narratives using the REFLECT rubric. We measured internal consistency through Cronbach's alpha coefficients to determine reliability. We measured the correlation of scores between the writing components' and time stability in test-retest too.
The inter-rater reliability
We measured the size of the agreement between two raters by Cohen's kappa coefficient. Two raters assessed 40 Student's narratives independently, and then we measured the coefficient of agreement between them.
All analyses were performed using SPSS-23 software (IBM, Armonk, NY, USA), and significance in statistical tests was considered as P < 0.05.
Ethical approval for this study was obtained from the University Research Center with the ethics code. IR.MUI.REC.1396.3.472.
| Results|| |
We modified the REFLECT after verifying the final translation. Changes were made to the tool based on the pilot study.
The changes included
- The removal of Axis 2 about “critical reflection,” and the removal of the “optional minor criterion”
- Determining a 4-degree Likert scale in four Levels in reflection, “from habitual action to critical reflection.” Therefore, we added a numeric value in four reflection levels to assess changes in learners' reflective capacity.
Face and content validity of the tool was confirmed by five medical education specialists who were well-versed in professionalism. In addition, the tool was revised for ease of use and understanding of each component, with the comments of 10 students (6 men [60%] and 4 women [40%]) participating in the pilot study, and the necessary modifications were made.
Participants of the main study were 67 medical students (response rate 97.06%) included 32 women (47.8%) and 35 men (52.2%) from 23 to 28 years old with a mean age of 24.05 ± 1.62 years.
Kappa agreement coefficient was measured (0.84, and P < 0.000) that expressed a very good agreement between raters.
The raters assessed 67 written narratives of medical students based on the tool, and Cronbach's alpha coefficient was 0.83.
Spearman–Brown correlation coefficient was measured. Each writing components with others showed a significant correlation (P < 0.05) [Table 1].
We conducted two test sessions with a 2-month interval for medical students, and to evaluate the reliability of the tool, no educational intervention was provided for medical students. The tool stability was measured by the Spearman–Brown correlation coefficient (0.89 and P < 0.000). Therefore, the scores of the test and retest were highly correlated (P < 0.05) [Table 2].
|Table 2: Correlation between writing components and total score in test-retest|
Click here to view
The modified REFLECT in English was checked by one of the developers of the original version. A copy of the final modified REFLECT is attached [Appendix 1].
| Discussion|| |
So far, few tools developed to assess reflection. Considering that there was no tool for this purpose in Persian, we conducted this study to adapt the REFLECT rubric in this language. The modified tool includes four reflection levels and five writing components, and guides learners and educators for practice and giving feedback. The results of this study showed that the modified REFLECT is a valid and reliable tool for teaching and assessing reflection that can be used in formative and summative assessments at every level of the learning. Considering no educational intervention was provided to students, as shown in [Table 1], the correlation of writing components 1 with 5 was not significant. Because attaining high levels of reflection in Component 5 require training and practice, this should be achieved through educational intervention. Thus, in general, component 5 is less correlated with other components. However, as shown in [Table 2], the components of writing in the test and retest were correlated significantly with each other.
The REFLECT rubric in the original version was applied for formative assessment during reflection training that helped to give written feedback to learners' narratives. In our study, the numeric value was used at reflection levels to do a summative assessment, and determine the impact of teaching reflection in future educational interventions, and assess changes in learners' capacity. Furthermore, because few of Persian written narratives had “critical reflection,” we deleted the Axis 2 that contains “transformational learning,” and “confirmatory learning” which need a deep training and practicing reflective writing.
Some studies emphasized that teaching reflection through writing can make a significant improvement in “self-awareness, professionalism, and humanism,” and help students to learn empathy and promote compassion in clinical skills., There are few tools for teaching and evaluating reflective narratives, and some studies applied the REFLECT.
McNeill et al. examined how practitioners applying reflection on their practice and writing about it in e-portfolio. The researchers used a grading system to determine the level of reflection and analyzed using a qualitative method. Findings showed that practitioners presented reflection in the documents. Hoffman et al.'s study addressed the relation of reflection and professionalism errors in medical students. This study, that was done in a retrospective case–control in medical students at Indiana University showed that there is a significant correlation between them. They used a validated tool to assess reflection in medical students  that included seven reflection levels. Tracey et al. used the REFLECT rubric and investigated the application of reflective assignments in instructional designers. In general, the use of this tool and providing feedback was reported usefully for improving reflection and to address the weaknesses of the learners on the reflection levels. Patterson et al. used the REFLECT rubric to analyze reflective narratives in medical students. Assessing with this tool showed that reflective skill improved in 50% of students after teaching and practicing reflection. Huang et al. held the narrative medicine program and used the REFLECT rubric to assess students' narratives. This tool was used as an educational tool during the program, not for summative assessment. The mentioned studies did not use the REFLECT for summative assessment of learners. However, our study was intended to use this tool for summative and formative assessments.
Miller-Kuhlmann et al. compared two rubrics which had been used for the reflection assessment in previous studies. These tools included the REFLECT and the “Reflection-on-Action” which were based on the reflection theory. Both the rubrics were used to help teachers in educating and evaluating narrative writings. In this study for comparing the tools quantitatively, the REFLECT rubric was ranked from 1 to 4 in reflection levels, such as in our study. Furthermore, these tools were compared to the aspects of assessing reflective writing, strengths, and weaknesses. Both tools have advantages and challenges in education and are used to evaluate reflective narratives, so educators must choose one with considering their unique benefits and educational challenges. For example, the REFLECT rubric provides more details for presenting feedback on reflection levels, the length of time, it takes for training is longer, but using the Reflection-on-Action rubric is simpler in practice.
Considering the results of previous studies on the positive educational effects of using the REFLECT rubric in analyzing reflection levels and improving reflection in learners, we recommend applying the modified version of this valid and reliable tool to assess in the summative form besides the formative evaluation. In addition, using this tool, due to its concurrent capabilities of training and assessing, can be a novelty solution to the formal educational system to assess reflective capacity in learners which is a prerequisite for professional development.
In addition, strength point of applying the modified version is determining learners' changes in educational interventions. Using this tool is useful for universities that intend to present “reflective practice” for the first time to medical students. In Persian, we have no tool to assess reflection, so this tool can be used for teaching reflection during medical ethics or professionalism as a teaching method in the formal medical curriculum. It is recommended to use the original version of REFLECT to foster “critical reflection” and “transformative learning.” This tool can be used in reflective practice for medical students and health-care professionals.
We were restricted to find related literature about this tool because it is not widely used yet. Furthermore, the tools related to teaching and assessing reflection were limited. However, we tried to address the implications of this tool by reviewing the limited studies that had been done.
| Conclusions|| |
Improving reflection can help practitioners in clinical and ethical decision-making and problem-solving to promote clinical performance in physicians. Moreover, teaching reflection is a basis to promote professionalism. The modified REFLECT tool is a valid and reliable tool that can help us to assess reflective capacity in medical students and can promote reflection through formative and summative assessments and presenting feedback about reflection levels in narrative components. Therefore, introducing and using this tool in reflective practice can be effective in professional development.
This study comprises part of research at Isfahan University of Medical Sciences.
Financial support and sponsorship
This study was funded by the National Agency for Strategic Research in Medical Education. Tehran. Iran. Grant No. 970001, and was part of a project with the ethics code. IR.MUI.REC.1396.3.472 at Isfahan University of Medical Sciences in Iran.
Conflicts of interest
There are no conflicts of interest.
| References|| |
Coulehan J, Granek IA. Commentary: “I hope i'll continue to grow”: Rubrics and reflective writing in medical education. Acad Med 2012;87:8-10.
Bolton G. Boundaries of humanities: Writing medical humanities. Arts and Humanit High Educ 2008;7:131-48.
Sandars J. The use of reflection in medical education: AMEE guide no 44. Med Teach 2009;31:685-95.
Santen SA, Hemphill RR. A window on professionalism in the emergency department through medical student narratives. Ann Emerg Med 2011;58:288-94.
Tsingos C, Bosnic-Anticevich S, Lonie JM, Smith L. A model for assessing reflective practices in pharmacy education. Am J Pharm Educ 2015;79:124.
Tsingos C, Bosnic-Anticevich S, Smith L. Reflective practice and its implications for pharmacy education. Am J Pharm Educ 2014;78:18.
Bryan CS, Babelay AM. Building character: A model for reflective practice. Acad Med 2009;84:1283-8.
Karkabi K, Wald HS, Cohen Castel O. The use of abstract paintings and narratives to foster reflective capacity in medical educators: A multinational faculty development workshop. Med Humanit 2014;40:44-8.
Epstein RM, Hundert EM. Defining and assessing professional competence. J Am Med Assoc 2002;287:226-35.
Wald HS, Borkan JM, Taylor JS, Anthony D, Reis SP. Fostering and evaluating reflective capacity in medical education: Developing the REFLECT rubric for assessing reflective writing. Acad Med 2012;87:41-50.
Levine RB, Kern DE, Wright SM. The impact of prompted narrative writing during internship on reflective practice: A qualitative study. Adv Health Sci Educ Theory Pract 2008;13:723-33.
Brady DW, Corbie-Smith G, Branch WT. “What's important to you?” the use of narratives to promote self-reflection and to understand the experiences of medical residents. Ann Intern Med 2002;137:220-3.
Kumagai AK. A conceptual framework for the use of illness narratives in medical education. Acad Med 2008;83:653-8.
Wang W. Students' perceptions of rubric-referenced peer feedback on EFL writing: A longitudinal inquiry. Assess Writ 2014;19:80-96.
Wald HS, Reis SP, Borkan JM. Reflection rubric development: Evaluating medical students' reflective writing. Med Educ 2009;43:1110-1.
Wetmore AO, Boyd LD, Bowen DM, Pattillo RE. Reflective blogs in clinical education to promote critical thinking in dental hygiene students. J Dent Educ 2010;74:1337-50.
McNeill H, Brown JM, Shaw NJ. First year specialist trainees' engagement with reflective practice in the e-portfolio. Adv Health Sci Educ Theory Pract 2010;15:547-58.
Kember D, McKay J, Sinclair K, WongFKY. A four-category scheme for coding and assessing the level of reflection in written work. Assess Eval High Educ 2008;33:369-79.
Devlin MJ, Mutnick A, Balmer D, Richards BF. Clerkship-based reflective writing: A rubric for feedback. Med Educ 2010;44:1143-4.
Mezirow J. Transformative Dimensions of Adult Learning. Jossey-Bass, 350 Sansome Street, San Francisco, CA 94104-1310; 1991.
Boud D, Keogh R, Walker D. Promoting reflection in learning: A model. Boundaries of Adult Learn 1996;1:32-56.
Chirema KD. The use of reflective journals in the promotion of reflection and learning in post-registration nursing students. Nurse Educ Today 2007;27:192-202.
Wong FK, Kember D, Chung LY, Yan L. Assessing the level of student reflection from reflective journals. J Adv Nurs 1995;22:48-57.
Butani L, Blankenburg R, Long M. Stimulating reflective practice among your learners. Pediatrics 2013;131:204-6.
Tsingos-Lucas C, Bosnic-Anticevich S, Schneider CR, Smith L. Using reflective writing as a predictor of academic success in different assessment formats. Am J Pharm Educ 2017;81:8.
Kember D, Leung DY, Jones A, et al
. Development of a questionnaire to measure the level of reflective thinking. Assess Eval High Educ 2000;25:381-95.
Mann K, Gordon J, MacLeod A. Reflection and reflective practice in health professions education: A systematic review. Adv Health Sci Educ Theory Pract 2009;14:595-621.
Hoffman LA, Shew RL, Vu TR, Brokaw JJ, Frankel RM. Is reflective ability associated with professionalism lapses during medical school? Acad Med 2016;91:853-7.
Tracey MW, Hutchinson A, Grzebyk TQ. Instructional designers as reflective practitioners: developing professional identity through reflection. Educ Technol Res Develop 2014;62:315-34. Available from: http://dx.doi.org/10.1007/s11423-014-9334-9
. [Accessed 4 Oct 2019.]
Patterson A, Sharek D, Hennessy M, Phillips M, Schofield S. Medical humanities: A closer look at learning. Med Humanit 2016;42:115-20.
Huang CD, Liao KC, Chung FT, Tseng HM, Fang JT, Lii SC, et al.
Different perceptions of narrative medicine between Western and Chinese medicine students. BMC Med Educ 2017;17:85.
Miller-Kuhlmann R, O'Sullivan PS, Aronson L. Essential steps in developing best practices to assess reflective skill: A comparison of two rubrics. Med Teach 2016;38:75-81.
[Table 1], [Table 2]