Validating self-reflection and insight scale to measure readiness for self-regulated learning
Leila Naeimi1, Mahsa Abbaszadeh2, Azim Mirzazadeh3, Ali Reza Sima4, Saharnaz Nedjat5, Sara Mortaz Hejri6
1 Department of Medical Education, School of Medicine, Tehran University of Medical Sciences; Medical Education Development Center, Zanjan University of Medical Sciences, Zanjan, Iran
2 Department of Internal Medicine, Imam Khomeini Hospital, Tehran University of Medical Sciences, Tehran, Iran
3 Department of Medical Education, School of Medicine; Department of Internal Medicine, Imam Khomeini Hospital; Health Professions Education Research Center, Tehran University of Medical, Tehran, Iran
4 Digestive Disease Research Center, Digestive Disease Research Institute, Shariati Hospital, Tehran University of Medical Sciences, Tehran, Iran
5 Department of Epidemiology, School of Public Health, Tehran University of Medical Sciences, Tehran, Iran
6 Department of Medical Education, School of Medicine, Tehran University of Medical Sciences; Education Development Center, Tehran University of Medical Sciences, Tehran, Iran
|Date of Submission||03-Mar-2019|
|Date of Acceptance||22-Jun-2019|
|Date of Web Publication||30-Aug-2019|
Dr. Sara Mortaz Hejri
Keshavarz Boulevard, Naderi Street, Hojjatdoost Ave, Education Development Center, School of Medicine, Tehran University of Medical Sciences, Tehran
Source of Support: None, Conflict of Interest: None
BACKGROUND: Professional behavior of physicians is under scrutiny by medical associations, media, and patients; therefore, medical students are expected to be self-directed learners rather than the passive ones. One of the useful strategies for professional development and life-long learning of students is self-regulated learning. Self-regulation concept and lifelong learning commitment are in the heart of medical practice. Therefore, this study aimed to evaluate the validity of Self-Reflection and Insight Scale (SRIS) to inspect the medical students' readiness for self-regulation.
MATERIALS AND METHODS: SRIS was translated according to the Sousa and Rojjanasrirat guideline. To examine the reliability and validity evidence of the scale, 136 medical students from Tehran University of Medical Sciences completed the questionnaire. Internal consistency and intraclass correlation were used to examine the reliability evidence, as well as qualitative content validity, and confirmatory factor analysis and exploratory factor analysis (EFA) were used to examine the construct validity of the scale.
RESULTS: The content validity of the scale was verified. Cronbach's alpha and the Interclass Correlation Coefficient value for the four-factor model was 0.87 and 0.79, respectively. Goodness-of-fit indices displayed acceptable and poor values (P = 0.0001, χ2 = 373.51, df = 167, Root Mean Square Error Of Approximation = 0.096, standardized root mean square residual = 0.12). EFA was conducted; a well-structured model was achieved through the EFA. The new four-factor model was extracted as the best model by performing EFA.
CONCLUSION: SRIS Persian version is saturated with four factors and has desirable content validity and constructs reliability.
Keywords: Medical students, self-regulation, self-reflection and insight
|How to cite this article:|
Naeimi L, Abbaszadeh M, Mirzazadeh A, Sima AR, Nedjat S, Mortaz Hejri S. Validating self-reflection and insight scale to measure readiness for self-regulated learning. J Edu Health Promot 2019;8:150
|How to cite this URL:|
Naeimi L, Abbaszadeh M, Mirzazadeh A, Sima AR, Nedjat S, Mortaz Hejri S. Validating self-reflection and insight scale to measure readiness for self-regulated learning. J Edu Health Promot [serial online] 2019 [cited 2019 Sep 21];8:150. Available from: http://www.jehp.net/text.asp?2019/8/1/150/265841
| Introduction|| |
In the constantly changing world, medical doctors must continuously obtain the highest standards of patient care. Professional behavior of physicians is under scrutiny by medical associations, media, and patients. Problematic behavior demonstrated by clinicians is a serious challenge  that affects the quality of patient care, has a significant economic impact on hospital costs, is a potential threat to the educational environment, and could be harmful to well-being of the health-care team.
Given the evidence suggesting that disruptive behavior of medical doctors is associated with their previous performance as students, the crucial role of medical schools in helping medical students is evident. Medical students should be able to undertake responsibility in terms of identifying their own learning needs and learning activities. One of the useful strategies for professional development and lifelong learning of students is Self-Regulated Learning (SRL).,
In the field of educational research, SRL has been defined in different ways and many models have been developed to describe its process.,,, Moreover, a number of researchers have designed measurement instruments, in light of increasing interest in the identification and assessment of SRL and its related attributes.,,, Yet, one should notice that most of the widely used questionnaire measures other constructs, rather than SRL.,,,, Even the most verified instrument in SRL fails to assess post-action strategies such as self-reflection. However, Grant et al. designed the Self-Reflection and Insight Scale (SRIS) to measure two underlying constructs of self-regulation, namely self-reflection and insight. They developed this instrument based on their generic model of self-regulation which was conceptualized in 2001 [Figure 1]. Self-reflection refers to inspection and evaluation of the thoughts, feelings, and behavior, while insight refers to the clarity of perceptions, emotions, and behaviors. Both are considered as metacognitive features as the heart of self-regulation. The reflecting ability on thoughts, feelings, and emotions lies on the basis of self-assessment and self-critical. Reflective learning can improve professionalism, and reflective performance can stimulate self-regulation learning that leads to improved continuous performance and better management of the complex health system and patient improvement. Furthermore, students should have insight into their knowledge and performance to be effective in self-regulation.
|Figure 1: Generic model of self-regulation and goal attainment showing role of self-reflection and insight. [Published with permission from A.M. Grant]|
Click here to view
The SRIS in its original format had two subscales and 20 items. This scale has been validated in a couple of studies, and its reliability has been reported to be between 0.70 and 0.90.,,,, However, Roberts and Stark, who used this scale to measure students' readiness for self-regulating of professional behavior, have identified one additional domain in their factor analysis. They named the factors as engagement in reflection, need for reflection, and insight. They claimed that this finding is consistent with Grant's basic conceptualization. Since validity and reliability of a scale depend on the interpretation of the scores, these controversial findings on the validity of SRIS led us conduct a study in another setting to validate this instrument. Moreover, considering the curricular reform of undergraduate medical programs, it seems necessary to have an authentic tool to examine the self-regulatory behavior of medical students. Hence, to have a tool that can measure the self-reflection and insight of medical students which are central to the development of SRL in medical students, this study aimed to investigate the validity of the SRIS.
| Materials and Methods|| |
The participants included medical students of Tehran University of Medical Sciences (TUMS) who were in the clerkship training and volunteered to be enrolled in the study. The study was approved by the Ethics Committee of the TUMS (IR.TUMS.MEDICINE.REC.1395.1722). We collected informed consent before the data collection and assured the students about anonymity and confidentiality of data.
According to the study conducted by Roberts and Stark, the SRIS, a self-report 20-item tool, consists of three subscales including insight (8 items), need for reflection (6 items), and engagement in reflection (6 items). A six-point Likert scale (1 = strongly disagree, 2 = slightly disagree, 3 = disagree, 4 = slightly agree, 5 = agree, 6 = strongly agree) is used to score responses. Higher scores reflect more inspection and evaluation as well as the perception of thoughts, beliefs, feelings, and behaviors. The scale score is obtained from the sum of scores of items.
Translation of the scale
The English version of SRIS was translated into Persian, according to the guideline developed by Sousa and Rojjanasrirat. Two Iranian English professors individually translated the tool. We compared and combined the two versions to present the best wording and terms. The translated questionnaire, then, was revised by three faculty members, two PhD in medical education and one clinical specialist. They corrected vague terms and accommodated the questionnaire for the usage of medical students. Next, the Persian version was retranslated into English by two experts in the English language to ensure the semantic equivalence of the scale. We compared the retranslated version to the original tool, so a series of ambiguous word and complicated sentences were identified. As a result, we made some changes in items 2, 6, 7, and 13 in the Persian version.
Evaluating content validity of the tool
We tested the content validity of the questionnaire using a qualitative approach. We invited five faculty members (three PhDs and two clinical specialists) to discuss and express their opinion about the relevance, importance, and clarity of items in the questionnaire. The panelists decided whether each item should be retained, modified, or removed from the tool. We also asked the panel members to indicate any other items that they think should be added to the questionnaire. This effort led to modifying and restructuring several items to avoid using negative verb in the sentence. Moreover, the ordering of questions 2 and 4–6 was displaced in the Persian version, because the negative sentences were one after another which might have caused confusion for students.
In addition, we asked 10 medical students to complete the questionnaire and to highlight difficult and vague items. Based on their feedback, some minor modifications were made, and the words “self-reflection” and “insight” were defined in footnotes. In this way, the Persian version of SRIS was finalized.
Evaluating construct validity of the tool
To evaluate the construct validity of the tool, we conducted confirmatory factor analysis (CFA) and exploratory factor analysis (EFA). For this purpose, the Persian SRIS was distributed among TUMS medical students who were in the clerkship rotations during the summer 2017. For CFA, we considered maximum likelihood estimation method using LISREL8/83 software (Scientific Software International, Skokie, IL, USA). Moreover, to determine the fitness of the proposed model for the data, we computed a series of indices including Chi-square ratio, normal fit index (NFI), nonnormed fit index (NNFI), comparative fit index (CFI), goodness-of-fit index (GFI), root mean square error of approximation (RMSEA), and standardized root mean square residual (SRMR). EFA was conducted due to acceptable and weak values of goodness-of-fit indices. A well-structured model was achieved through the EFA. EFA was examined by varimax rotation method using SPSS-19 software (IBM, Armonk, NY, USA).
Evaluating reliability of the tool
Reliability of the tool was calculated by internal consistency and test–retest reliability. Internal consistency of the tool was measured by computing Cronbach's alpha coefficient. Moreover, “alpha if item deleted” for each item was calculated. To inspect test–retest reliability, a subgroup of the same medical students completed the questionnaire twice, separated by 2-week interval, and interclass correlation coefficient (ICC) was calculated. The satisfactory value of Cronbach's alpha and ICC was considered (≥0.70).
| Results|| |
One hundred and thirty-six students completed the questionnaire. The students' average age was 23.6 years (standard deviation = 1.24), and 66 students (48.5%) were female. There were no missing data. The range of students' scores was between 60 and 117 (out of 120). [Table 1] shows detailed descriptive statistics for each item, each subdomain, and the total score.
|Table 1: Statistics of ratings by clerkship students to the 20-item self-reflection and insight scale questionnaire (Persian version) on a six-point Likert scale (strongly disagree: 1 to strongly agree: 6) (n=136)|
Click here to view
Confirmatory factor analysis
The t-values obtained for this tool indicate that all paths are significant at the 0.001 level. Goodness-of-fit indices displayed acceptable and weak values (P = 0.0001, χ2 = 373.51, df = 167, RSMEA = 0.096, SRMR = 0.12, GFI = 0.78, CFI = 0.93, NFI = 0.85, NNFI = 0.92). Thus, the most effective software routes were developed to modify the model. Two covariance relationships were plotted between questions 8 and 12 and questions 15 and 18 [Figure 2]. However, drawing these two paths did not significantly improve the fit indices (P = 0.0001, χ2 = 323.21, df = 165, RSMEA = 0.084, SRMR = 0.11). The NFI, NNFI, and GFI should be ≥0.95 while it was 0.87, 0.81, and 0.93, respectively, in the given paper. The CFI should be ≥0/09, while it was 0.94 in the study. Both SRMR and RMSEA should be <0.08., The significance level of χ2 would be >0.05 as well as χ2/df value between 2 and 5, which appeared to be χ2/df = 1.95 in the current study.
|Figure 2: Three-factor confirmatory factor analysis model of the self-reflection and insight scale showing standardized factor loadings|
Click here to view
Exploratory factor analysis
The EFA was conducted with the principal component analysis method and the varimax rotation. Before the factor analysis, sampling adequacy using the Kaiser-Meyer-Oklin ( KMO) test and rejecting the null hypothesis based on the correctness of the matrix of homogeneity in the population were confirmed to be meaningful at level 0.001 by Bartlett's test of sphericity that the matrix is homogeneous, respectively (KMO = 0.844, χ2 = 1016/72, P < 0.001). In other words, implementing the factor analysis is justifiable. According to Andersson et al., the KMO values >0.6 were acceptable for factor analysis. The extracting criterion of the factors was the scree plot curve and the eigenvalue was above 1. Four factors namely insight, need for reflection, intention for reflection, and engagement in reflection were extracted [Table 1]. These four factors explained the 56.8% of the variance of the total variables. The factor matrix demonstrated that the first factor has more factor loading (18.23) and contribution than other factors.
Based on the results of EFA and cutpoint of eigenvalue, none of the items were omitted. Items which were jointly associated with a same factor and made a subscale were extracted [Table 2]. We identified four factors as follows:
|Table 2: Loadings from exploratory factor analyses: Orthogonal rotation (all factor loadings) (n=136)|
Click here to view
- Insight: Items 3, 4, 6, 9, 11, 14, 17, 20
- Need for reflection: Items 12, 15, 18
- Intention for reflection: Items 1. 2, 5, 7, 13
- Engagement in reflection: Items 8, 10, 16, 19.
The new four-factor model, as the best one, accounted for 56.8% of the total variance of the variables. [Table 3] considers the eigenvalues, the variance of each factor, and cumulative percentage of the factors. Varimax rotation was used to better differentiate the factors.
|Table 3: Eigenvalue, percentage of explanation of variance, and cumulative percentage of four factors, n=136 of self-reflection and insight scale|
Click here to view
Reliability of the tool
Twenty medical students completed the questionnaire twice. Forty questioners were collected in the first and second administrations. The Cronbach's alpha of the new four-factor scale for the whole scale was 0.87 and the subscales of insight, need for reflection, intention for reflection, and engagement for reflection had an alpha of 0.83, 0.71, 0.70, and 0.78, respectively. None of the “alpha if item deleted” values were greater than overall alpha.
The ICC value was 0.80 for the whole scale, and 0.84, 0.51, 0.58, and 0.65 the above-mentioned subscales, respectively. The related values for each subscale are presented in [Table 1].
| Discussion|| |
The current study aimed to examine the validity evidence of SRIS to assess the readiness of medical students for SRL in an Iranian university.
The content validity of SRIS was confirmed by the experts. The reliability of the tool was supported by good-to-excellent internal consistency and test–retest reliability. In Grant et al.'s investigation, the Cronbach's alpha for subscales of self-reflection and insight along with the reliability of test–retest was reported 0.91, 0.87, 0.77, and 0.78, respectively. The Cronbach's alpha reliability in the study by Roberts and Stark was reported as 0.83, 0.87, and 0.85 for the subscales, namely, engagement in reflection, need for reflection, and insight, and 0.88 for the total scale. In the study of Aşkun and Çetin in Turkey, the Cronbach's alpha for the subscales of insight and reflection and the total items was 0.65, 0.80, and 0.70, respectively. Chen et al. investigated the psychometric properties of the SRIS in China. The Cronbach's alpha was 0.83 for insight, 0.87 for self-reflection, and 0.79 for the total scale. All these differences in the coefficients could be interpreted that reliability is not feature of the tool-like validity. A tool used in different subjects can indicate wide variations in reliability.
The results of this study were congruent with that of Aşkun and Çetin. In their study, factor analysis of SRIS ver. 2008 indicated inadequate fit indices. Yet, the results were inconsistent with Chen et al.'s results done on ver. 2002 of SRIS two-factor scale. Further, their CFA displayed that two-factor model possesses goodness-of-fit indices (RSMEA = 0.057, χ2/df = 3.29, CFI = 0.98, GFI = 0.96, NFI = 0.97, and SRMR = 0.64).
In Grant et al.'s study, an EFA was conducted by using varimax rotation, generally indicated two factors, including 20 items. Subscales of self-reflection (12 items) and insight (8 items) explained 56% of the total variance, which is in line with the present study. At the present study, the four factors explained the 56.8% of the variance of the total variables.
Application of results
This study had provided robust evidence about the psychometric characteristics of an instrument, which had been developed to measure self-regulation among medical students. In addition, we have introduced the Persian version of the instrument to the Iranian community of educational health professions. We believe that students, faculty members, and program directors in different settings would benefit from this scale in different ways. These applications include facilitating SRL among students, preparing them for better management of the complicated problems, planning for curricular revisions, and monitoring curriculum changes.
Limitations of the study
One of the limitations of the present study is that it has conducted in just one university, which could have an impact on the generalizability of results. It is suggested to validate the tool in other settings and universities. In addition, this study failed to evaluate the relation of this tool with other instruments which measures self-regulation construct. Investigating this correlation as well as working more on confirming four factors related to this instrument in different fields is recommended.
| Conclusion|| |
The validity evidence of SRIS was assessed to evaluate the medical students' readiness for SRL. The results indicated that the scale is saturated with four factors in the Iranian community with acceptable reliability.
This study was extracted from PhD dissertation of the first author entitled, “The examination of the effectiveness of Holmes' reflection approach on knowledge, attitude, and professionalism performance of medical students in clerkship phase of Tehran University of Medical Sciences” (Code 21) at TUMS. We sincerely appreciate the students collaborated in the reliability and validity of the study.
Financial support and sponsorship
This article has been derived from a thesis research project and received financial support from TUMS (Grant No. 35253).
Conflicts of interest
There are no conflicts of interest.
| References|| |
Cho KK, Marjadi B, Langendyk V, Hu W. Medical student changes in self-regulated learning during the transition to the clinical environment. BMC Med Educ 2017;17:59.
Stewart K, Wyatt R, MerckGW. Unprofessional behaviour and patient safety. Int J Clin Leader 2011;17:93-100.
Hickson GB, Federspiel CF, Pichert JW, Miller CS, Gauld-Jaeger J, Bost P. Patient complaints and malpractice risk. JAMA 2002;287:2951-7.
Rawson JV, Thompson N, Sostre G, Deitte L. The cost of disruptive and unprofessional behaviors in health care. Acad Radiol 2013;20:1074-6.
Weiss KB, Wagner R, Nasca TJ. Development, testing, and implementation of the ACGME clinical learning environment review (CLER) program. J Grad Med Educ 2012;4:396-8.
Hansen AM, Hogh A, Persson R, Karlson B, Garde AH, Ørbaek P. Bullying at work, health outcomes, and physiological stress response. J Psychosom Res 2006;60:63-72.
Papadakis MA, Hodgson CS, Teherani A, Kohatsu ND. Unprofessional behavior in medical school is associated with subsequent disciplinary action by a state medical board. Acad Med 2004;79:244-9.
Roth A, Ogrin S, Schmitz B. Assessing self-regulated learning in higher education: A systematic literature review of self-report instruments. J Pers Eval Educ 2016;28:225-50.
van Houten-Schat MA, Berkhout JJ, van Dijk N, Endedijk MD, Jaarsma AD, Diemers AD. Self-regulated learning in the clinical context: A systematic review. Med Educ 2018;52:1008-15.
Pintrich PR. The role of motivation in promoting and sustaining self-regulated learning. Int J Educ Res 1999;31:459-70.
Winne PH. Student's calibration of knowledge and learning processes: Implications for designing powerful software learning environments. Int J Educ Res 2004;41:466-88.
Zimmerman B. Investigating self-regulation and motivation: Historical background, methodological developments, and future prospects. Amer Educ Res J. 2008;45:166-83.
Grant AM. Rethinking psychological mindedness: Metacognition, self-reflection, and insight. Behav Change 2001;18:8-17.
Lucy B, Valerie P, William L. Online self-regulatory learning behaviors as a mediator in the relationship between online course perceptions with achievement. Int Rev Res Open Dis 2008;9 (2).
Magno C. Developing and assessing self-regulated learning. The Assessment Handbook: Continuing Education Program. Vol. 1. 2009. Available at SSRN: https://ssrn.com/abstract=1426045
. [Last accessed on 2019 Jun 26].
Pintrich PR, Smith DA, Garcia T, Mckeachie WJ. Reliability and predictive validity of the motivated strategies for learning questionnaire (Mslq). Educ Psychol Meas 1993;53:801-13.
Weinstein CE, Palmer DR, Schulte AC. Learning and Study Strategies Inventory (LASSI). Clearwater, FL: H and H Publishing Company; 1987.
Xu X. Evaluation of the relationship between one's differentiation of cognitive processes and metacognitive self-ratings. Behav Res Methods 2009;41:244-55.
Harrington R, Loffredo DA. Insight, rumination, and self-reflection as predictors of well-being. J Appl Psychol 2010;145:39-57.
Sauter FM, Heyne D, Blöte AW, van Widenfelt BM, Westenberg PM. Assessing therapy-relevant cognitive capacities in young people: Development and psychometric evaluation of the self-reflection and insight scale for youth. Behav Cogn Psychother 2010;38:303-17.
Harrington R, Loffredo D. Insight, rumination, and self-reflection as predictors of well-being. J Psychol 2010;145:39-57.
DaSilveira A, DeSouza ML, Gomes WB. Self-consciousness concept and assessment in self-report measures. Front Psychol 2015;6:930.
Grant A, Franklin J, Langford P. The self-reflection and insight scale: A new measure of private self-consciousness. Soc Behav Pers 2002;30 821-35.
Roberts C, Stark P. Readiness for self-directed change in professional behaviours: Factorial validation of the self-reflection and insight scale. Med Educ 2008;42:1054-63.
Silvia PJ, Duval TS. Objective self-awareness theory: Recent progress and enduring problems. Pers Soc Psychol Rev 2001;5:230-41.
Sandars J. The use of reflection in medical education: AMEE guide no. 44. Med Teach 2009;31:685-95.
Hays RB, Jolly BC, Caldon LJ, McCrorie P, McAvoy PA, McManus IC, et al.
Is insight important? Measuring capacity to change performance. Med Educ 2002;36:965-71.
Silveira AC, Castro TG, Gomes VB. Scale of self-reflection and insight: New measure of self-consciousness adapted and revalidated for Brazilian adults. Dialnet Plus 2012;43 (2).
Chen SY, Lai CC, Chang HM, Hsu HC, Pai HC. Chinese version of psychometric evaluation of self-reflection and insight scale on Taiwanese nursing students. J Nurs Res 2016;24:337-46.
Aşkun D, Çetin FJ. Turkish version of self-reflection and insight scale: A preliminary study for validity and reliability of the constructs. Psychol Stud 2017;62:21-34.
Cook DA, Beckman TJ. Current concepts in validity and reliability for psychometric instruments: Theory and application. Am J Med 2006;119:166.e7-16.
Sousa VD, Rojjanasrirat W. Translation, adaptation and validation of instruments or scales for use in cross-cultural health care research: A clear and user-friendly guideline. J Eval Clin Pract 2011;17:268-74.
Severinsson E. Evaluation of the Manchester clinical supervision scale: Norwegian and Swedish versions. J Nurs Manag 2012;20:81-9.
Hooper D, Coughlan J, Mullen M. Structural equation modelling: Guidelines for determining model fit. EJBRM 2008;6:53-60.
Kline RB. Principles and Practice of Structural Equation Modeling. 4th
ed. USA: Guilford Press; 2005.
Tabachnick BG. Using Multivariate Statistics. 5th
ed. Needham Heights, MA, USA: Allyn and Bacon, Inc.; 2006.
Wheaton B, Muthén B, Alwin DF, Summers GF. Assessing reliability and stability in panel models. Sociol Methodol 1977;8:84-136.
Andersson BT, Christensson L, Fridlund B, Brostrom A. Development and psychometric evaluation of the radiographer's competence scale. Open J Nurs 2012;2:85-96.
[Figure 1], [Figure 2]
[Table 1], [Table 2], [Table 3]