|
Design and psychometric properties of willingness to mobile learning scale for medical sciences students: A mixed-methods study
Nayereh Baghcheghi1, Hamid Reza Koohestani2, Mahmood Karimy3
1 Department of Nursing, Saveh University of Medical Sciences, Saveh, Iran
2 Department of Medical Education, Social Determinants of Health Research Center, Saveh University of Medical Sciences, Saveh, Iran
3 Department of Public Health, Social Determinants of Health Research Center, Saveh University of Medical Sciences, Saveh, Iran
| Date of Submission | 17-Feb-2020 |
| Date of Acceptance | 27-Feb-2020 |
| Date of Web Publication | 30-Jun-2020 |
Correspondence Address:
Dr. Hamid Reza Koohestani
Departments of Medical Education, Social Determinants of Health Research Center, Saveh University of Medical Sciences, Saveh
Iran
 Source of Support: None, Conflict of Interest: None  | Check |
DOI: 10.4103/jehp.jehp_153_20
BACKGROUND AND AIM: Given the absence of a scale specially designed to measure willingness to mobile learning (m-learning) in medical sciences students, the present study was conducted to design and evaluate the psychometric properties of “willingness to m-learning” scale for medical sciences students.
METHODOLOGY: The study was carried out as a mixed-method study in two phases at Saveh University of Medical Sciences in 2019. Phase one was a qualitative study to elaborate on the students' perception of m-learning. Then, the statements were extracted, and statement pool was completed through reviewing the text. In the second phase, the psychometric properties including face, content, and construct validities (using explorative factor analysis), internal consistency (Cronbach's alpha), and test–retest reliability (intercluster correlation test) were measured. A total of 482 students who were selected randomly participated in the second phase. Data analysis was done with MAXQDA software (VERBI Software 2019, Berlin, Germany) for qualitative data and SPSS 19 software (SPSS Inc., Chicago, IL, USA) for quantitative data.
RESULTS: Based on qualitative content analysis and literature review, 92 statements were extracted. After checking face and content validity, 55 statements remained in the study. Construct validity of the questionnaire based on explorative factor analysis removed 10 more statements and the remaining 45 statements were categorized into nine factors, namely technophilia, perceived attraction, perceived ease, perceived conflict, self-management, attitude, behavioral intention to use, educational use, and efficacy of m-learning. Reliability of the scale was obtained as 0.95 based on Cronbach's alpha and stability was checked using test–retest method (intercluster correlation coefficient; r = 0.92).
CONCLUSION: Willingness to m-learning scale had an acceptable reliability and validity in medical sciences students. Therefore, it can be used for medical sciences students for improve learning and education.
Keywords: Medical sciences students, mobile learning, psychometric properties, scale, validation, willingness
How to cite this article:
Baghcheghi N, Koohestani HR, Karimy M. Design and psychometric properties of willingness to mobile learning scale for medical sciences students: A mixed-methods study. J Edu Health Promot 2020;9:141 |
How to cite this URL:
Baghcheghi N, Koohestani HR, Karimy M. Design and psychometric properties of willingness to mobile learning scale for medical sciences students: A mixed-methods study. J Edu Health Promot [serial online] 2020 [cited 2023 Jun 1];9:141. Available from: https://www.jehp.net//text.asp?2020/9/1/141/288334 |
| Introduction | |  |
Following the expansion of digital culture and the increase in the popularity of modern communicational technologies, new terms and concepts have also emerged. One of these is mobile learning (m-learning). The main specification of m-learning is time/place-free learning opportunity.[1],[2],[3],[4] M-learning refers to “learning any sort of knowledge, attitude, and skill using mobile technologies with no time and place limitation, which leads to behavioral change.”[2] Although m-learning is still in its development process, it is going to be an integral part of learning process and a piece in blended learning puzzle in future.[5],[6]
Clearly, given the potential capacities of mobile technologies, they can facilitate learning process in medical sciences students. Koohestani et al. reported that using mobile technologies had a positive effect on both the process and the outcome of learning in medical sciences students.[7] Purposeful use of technology for educational and learning objectives can improve interactions among students and between students and instructor.[4]
M-learning is not just about using mobile devices. M-learning if you want to be a useful teaching method, you need to pay special attention to the characteristics and desires of the learner.[8] Higher educational institutes and universities should be able to demonstrate the advantages of mobile gadgets to students and prepare the ground for the implementation of m-learning. Clearly, implementation of any system or technology entails identifying the elements effective in its acceptance in environment. To use m-learning as a new stage in e-learning, the effective factors in its acceptance need to be identified and any measure in this field should be based on complete awareness.
Taking into account the importance of m-learning and measuring willingness in medical sciences students to this learning approach, there is a need for a reliable and specially designed measurement scale. Therefore, it is imperative to design such a scale. Limiting the assessment of willingness to learning only to quantitative scales leads us to a limited perspective that is represented by the available questionnaires, while the background factors, advantages, and weaknesses remain hidden. Using a qualitative approach and through interviewing students, we can achieve a comprehensive picture of the topic; therefore, a mixed–methods approach was adopted in this study. This article is an attempt to design and evaluate the psychometric properties “m-learning willingness” scale for medical sciences students.
| Methodology | | |
The study was carried out as a mixed-method work using qualitative and quantitative methods in 2019.
Phase one: Generation of statements
To generate statements, two methods were followed. At first, using conventional content analysis, data gathering and analyzing were performed simultaneously. To achieve data abundance, 17 semi-structured, private, and face-to-face interviews were done with 17 students. The participants were selected purposefully, and selection of participants continued until data saturation. Maximum diversity in the participating students in terms of age, sex, educational level, and major was ensured. The interviews would be started with general questions such as “please tell us about your experiences about using mobile technologies for learning and educational objectives?” The interviews would be followed by more specific questions such as “what factors played a role in using mobile technology for educational purposes?” The time of each interview ranged from 30 to 45 min.
Afterward, qualitative data content analysis was performed following Graneheim and Lundman's approach.[9] To ensure data rigor, the four criteria of credibility, dependability, confirmability, and transferability or fittingness were used.[10] As the next step, more statements were extracted through literature review to make the statement pool more complete.
Qualitative phase
Face validity
To determine face validity, qualitative and quantitative approaches were followed. In qualitative approach, the students were asked to comment about difficulty, relevance, and ambiguity of each statement. In quantitative method, statement effect method was used for statement truncation, removal of improper statements, and determination of importance of each statement. The students answered the question “to what extent, each statement is essential for measuring m-learning willingness in medical sciences students?” A Likert's 5-point scale was used for each statement (5 = completely important; 4 = relatively important; 3 = moderately important; 2 = trivially important; and 1 = completely unimportant). The statements with an effect score ≥1.5 remained in the process. To obtain the effect of each statement, the following formula was used: impact score = frequency (%) × importance
Where “frequency” refers to the number of individuals who scored a statement 4 or 5 and “importance” refers to the mean score of importance of each statement.[11]
Content validity
To determine content validity, 15 faculty board members were consulted. There are two methods for content validity assessment including qualitative and quantitative methods. At first and using qualitative method, the experts were asked to check the statements in terms of grammar, wording, and place of words in the statements. Then, using quantitative method and to make sure that most relevant and current statements are used in the questionnaire, content validity ratio (CVR) and content validity index (CVI) were used, respectively.
The experts were also asked to determine whether or not a statement is necessary to represent the concept of willingness to m-learning in medical science students? The experts would choose among three alternatives “necessary,” “useful but not necessary,” and “unnecessary.” The numerical value of CVR was determined based on Lawshe table.[12] As ten experts took part in this study, when the obtained number is >0.62, a statement is acceptable at a significance level of 0.05.[13]
Content Validity Index
This index is obtained in two ways, specifically for each statement and generally for the whole scale. Fifteen experts were asked to record their opinions based on four alternatives namely “completely relevant,” “relevant,” “relatively relevant,” and “irrelevant.” A statement is acceptable when its score is higher than 79%. If the score is between 70% and 79%, the statement needs revision and if it is below 70%, it should be omitted.[14],[15]
Construct validity
To determine construct validity, explorative factor analysis method is one of the most common methods that is also used to categorize statements pertinent to a scale. According to Tabachnick and Fidell, the minimum sample size for factor analysis is 300.[16] Given that 45 statements were remained to this stage and the probable leaves, 482 questionnaires were distributed among the students. For factor analysis, a principal component analysis (PCA) with Equamax rotation was used based on the following criteria: eigenvalues >1.0 and factor loading >0.4.
Reliability
Reliability of the scale was ensured using internal consistency with Cronbach's alpha, and stability reliability was determined using test-retest on 25 medical sciences students with a 2-week interval.[13]
Ethical concerns
Obtaining permission from the officials (ethical code: IR.SAVEHUMS.REC.1397.008), giving a thorough introduction to the objectives and nature of the study, reminding the participants' right to leave the study at whatever stage, making sure of confidentiality of information, and publishing the data anonymously were among the ethical concerns respected in this study.
| Findings | | |
Generation of statements
The concept of M-learning was elaborated on through qualitative content analysis of the information collected through interviews with 17 students. Afterward, using the extracted concepts and literature review, the statement pool was prepared. The statements were omitted, mixed, and revised through consulting with experts and research team members. A questionnaire with 92 statements designed based on a 5-point scale (completely agree = 5; agree = 4; no idea = 3; disagree = 2; and completely disagree = 1) was prepared for validation. For the negative statements, the scoring is inverse. The results showed that m-learning is a relatively subjective concept and affected by several factors such as technophilia, perceived attraction, perceived ease, perceived conflict, attitude, self-management in learning, behavioral intention, educational use (function), and learning efficacy.
Results of quantitative phase
The M-learning willingness scale was designed based on a qualitative study and literature review with 92 statements. Psychometric parameter results are further discussed in the following sections.
Face validity
As all the statements had an impact score >1.5, with the modification in the eight statements based on qualitative face validity, all the 92 statements entered the content validity stage.
Content validity
Qualitative content validity
Based on the experts' opinions, 25 statements were omitted due to semantic similarity. In addition, some of the statements were edited based on the comments by experts and 67 statements remained in the scale.
Content validity ratio
To compute CVR based on the Lawshe table, the threshold was set at 0.62. Therefore, out of the 67 statements, 11 did not meet the threshold and out of these 11 statements, two remained in the scale due to the importance of the topic and consensus of opinions among the research team members. Thus, with nine statements omitted, CVI was obtained for the scale with 58 statements.
Content validity index
To determine CVI of the scale, a threshold point of 0.8 was adopted. Therefore, 55 statements met the threshold and three statements were omitted. In addition, the mean CVI of the statements was 0.97, which is very high.
Construct validity
The participants in this stage were 482 undergraduate students. Kaiser–Meyer–Olkin (KMO) test was used to examine the adequacy of sampling for explorative factor analysis (KMO = 0.921) [Table 1]. Factor analysis is recommended when KMO = 0.80 or 0.90.[17] | Table 1: Kaiser-Meyer-Olkin and Bartlett's test on willingness to m-learning scale
Click here to view |
After performing principal component analysis with varimax rotation according to the Eigen values and Kaiser Criterion (factor coefficients >0.5), nine factors with 45 items were extracted [Appendix 1].
Reliability
Internal consistency
Cronbach's alpha for the whole questionnaire (45 statements) was obtained at 0.91. [Table 2] lists Cronbach's alpha for all the subscales; clearly, all the subscales have good reliability coefficients.
Test–retest method
Intercluster correlation coefficient of the scale based on test-retest was obtained equal to 0.89. To this end, test-retest method was conducted with 25 participants with 2-week interval [Table 3].
Scoring willingness to m-learning scale
Totally, 38 statements were scored positively and seven statements were scored negatively based on the Likert's scale. The maximum and minimum scores of the 45 statements were 255 and 45, respectively.
| Discussion | | |
In general, tests and other measurement scales, regardless of usage, should meet specific standards before being used for the purpose they are designed for. There are at least four necessary standards to be met by a scale to be used for research purposes. These standards include at least one type of content validity, one type of construct validity, and two types of reliability such as internal consistency and test–retest. Therefore, validity and reliability checks are of the main steps to assess a scale.[18] Here, two methods were used to check face validity, two methods were used to check content validity, one method was used to determine construct validity, and two methods were used to check reliability.
Majority of studies have followed technology acceptance theory (1989) to study m-learning acceptance in students.[6],[19],[20] For instance, Iqbal and Qureshi reported that perceived ease of use and usefulness were the key factors in acceptance of m-learning in students.[6] Chong et al. studied the factors effective in the acceptance of m-learning and showed that perceived ease of use, quality of services, and usefulness were the factors effective in m-learning.[20] Sun et al. reported that perceived ease of use had a positive effect on the attitudes of users of social media for learning purposes.[21]
The present study showed that in addition to perceived ease of use, there were other elements such as perceived attraction, perceived conflict, and technophilia, which had a key role in students' willingness for m-learning. Although the technology acceptance theory is a key model in technology acceptance and it is still used by many studies in this field, it is relatively old and does not cover specific aspects and internal/external motivation factors in particular. Here, new aspects and fields were extracted that were not covered by previous models and scales.
”Technophilia” was the first aspect of the scale proposed here. Academic papers call the individuals with positive attitude to new technologies as “technophilia.”[22] Students' experiences as shown by qualitative phase indicated that one's intrinsic interest in digital technology and mobile is a key factor in willingness to m-learning. In other words, personal trait, “loving technology,” and willingness for innovation in this field play a key role in m-learning. This aspect is covered by three items in the questionnaire.
Results of qualitative phase showed that medical sciences students are attracted to the capabilities and attractions of mobile technology. Specifications such as entertainment, visual representation of many issues and theoretical topics, and capability to provide materials as multimedia content make learning a more attractive experience for students. This interesting feature, entertainment for learning, is a key factor in attracting students to m-learning during leisure time. Four items in the designed scale measure the perceived attraction of m-learning.
Of the other key reasons that made m-learning more desirable for students were the ease of carrying mobile phone in pocket, access to information regardless of time and place, and ease of use. Using e-books and application-based books instead of hard copy books, mainly in clinical setting, was one of the factors with a positive role in students' willingness to m-learning. Ease of using mobile internet, ease of working with mobile application, and easy communication with teachers and classmates were other factors effective in this regard. Therefore, four statements were added to the scale to measure perceived ease.
Another concept was “perceived conflict;” conflict over the value of information, inconsistency in teachers' behavioral pattern, inconsistency in classmates' behavioral patterns, and digital gap between generations in families were some of the conflicts that students encountered with in using m-learning. These affected the willingness to m-learning in students. Some of the instructors supported m-learning, whereas some did not and banned mobile in classroom under any circumstances. Many valuable things can be learned using the Internet and virtual space, while some of this content is not scientifically correct. Facing with right and wrong information on the Internet at the same time, doubts about scientific value of some of the information, and fake information and news in virtual world were some of the issues that cast a shadow of doubt about the scientific value of online information for educational purposes. Taking into account that parts of learning process take place outside classroom and at home, the parents' performance was also found as an important contributor. The results showed that the parents, in many cases, had different attitudes from students about technology and m-learning in particular, which created a conflict over subjective perceptions in students. The approach and performance of classmates about using mobile were not the same. The concept of “perceived conflict” was covered by five statements in the questionnaire.
Students' attitude to m-learning is one of the effective factors in students' willingness to m-learning. The audiences' attitude to a virtual education program plays a key role in its success. Attitudes predict well the working behaviors of individuals.[23] Attitude is defined as a positive or negative feeling about a target behavior.[24] The proposed scale covers attitude by six statements.
Capabilities and options of mobile devices enable students to manage their learning using their mobile phones. Their experiences showed that mobile technology was a catalyzer in their learning. Faster educational communication, acceleration in doing homework and other educational affairs, and repeatability of learning opportunities were some of the factors named as catalyzer of learning. Using mobile technology, students can have access to the latest and updated knowledge and take a more active role in educational class discussions. In addition, thanks to mobile phone, they did not have to carry books with themselves. This concept is covered by nine statements in the designed scale.
Another concept extracted from the interviews was behavioral intention that was covered by three statements in the questionnaire. The concept of behavioral intention represents the strength of intention and persistence in performing the target behavior.[24] The relationship between behavioral intention and behavior shows that people prefer engaging in a behavior that they intend to. Therefore, behavior always comes after and attached to behavioral intention.[24]
Mobile technologies and smartphones are widely used in educational and clinical settings by medical sciences students so that these technologies cover a wide range of educational functions in educational and clinical environments. Educational applications about diseases, physical inspections, medicines, video clips about medical sciences, filming and photo-taking capabilities in class and clinical setting, medical simulators, educational games, capability to use e-books, saving and retrieving educational content, using social media in medical fields, and all other capabilities play a key role in the learning performance of medical sciences students. The concept of educational use of m-learning was covered by five statements in the questionnaire.
According to the students' experiences, using mobile technologies had a positive effect on all learning areas including cognitive, affective, and psychomotor domains. This was effective in their willingness to use m-learning. Previous studies have supported the finding that m-learning can have positive effects on the three leaning areas including affective, cognitive, and psychomotor domains.[25] This concept is covered by three statements in the scale.
The starting point of m-learning process is the students' willingness and desire to m-learning. As far as the students show no willingness to use mobile technology as an educational scale, m-learning process does not start. Undoubtedly, one of the motivations for this research work was to use the findings in practice and implement m-learning among medical sciences students. The findings can be used in a variety of research, management, and education fields.
The results of the present study offer new helpful insights into the meaning of m-learning and provide a scale to assess the willingness to m-learning scale for medical sciences students. Educational managers look for approaches to use up-to-date technology to facilitate learning in students and provide proper services to them. The designed tool helps educational managers to guide the students in the proper process of acceptance and implementation of m-learning. In addition, this tool paves the way for further researchers in this field as the development of medical sciences needs research works in different fields. The results of each research work open new opportunities for further researchers. The results of the present work can be a starting point for other studies on the implementation of educational programs using m-learning. One of the main elements in the successful implementation of m-learning is to measure readiness and desire in students for using m-learning. In terms of education, our results can be a motivation for the development and expansion of m-learning in medical sciences students to achieve a higher quality of education and learning.
This scale was developed to measure willingness to m-learning in undergraduate medical sciences students and does not reflect a sample in all educational levels and other disciplines. Thus, the willingness to m-learning scale may not be applicable to other level of education or other students. However, this study suggests that future research should focus on the designing and validation for the measures in other disciplines and other level of education.
| Conclusion | | |
The validity and reliability of willingness to m-learning scale for medical sciences students were supported. The scale can be used for curriculum planning and improve education in medical sciences students.
Acknowledgment
The study is based on a research plan approved by Saveh School of Medical Sciences (ethical code: IR.SAVEHUMS.REC.1397.008). The authors wish to express their gratitude to all the officials and participating students for their valuable contributions.
Financial support and sponsorship
This study was financially supported by Saveh University of Medical Sciences.
Conflicts of interest
There are no conflicts of interest.
| References | | |
| 1. | Koohestani HR, Baghcheghi N, Karimy M, Hemmat M, Shamsizadeh M. Lived experiences of nursing students about ethical concerns regarding mobile learning in educational and clinical contexts. J Med Ethics Hist Med 2019;12:5.  |
| 2. | Barzegar R. From electronic learning to mobile learning: Theoretical principles. MEDIA. 2012;3:35-41. |
| 3. | Dent J, Harden RM. A Practical Guide for Medical Teachers. Amsterdam (Netherlands): Elsevier Health Sciences; 2013. |
| 4. | Gallegos C, Nakashima H. Mobile devices: A distraction, or a useful tool to engage nursing students? J Nurs Educ 2018;57:170-3. |
| 5. | Mather C, Cummings E, Allen P. Nurses' use of mobile devices to access information in health care environments in Australia: A survey of undergraduate students. JMIR Mhealth Uhealth 2014;2:e56. |
| 6. | Iqbal S, Qureshi IA. M-learning adoption: A perspective from a developing country. IRRODL 2012;13:147-64. |
| 7. | Koohestani HR, Arabshahi SK, Ahmadi F, Baghcheghi N. The experiences of healthcare professional students about the educational impacts of mobile learning. Qual Rep 2019;24:1593-609. |
| 8. | Koohestani HR, Arabshahi SK, Ahmadi F. The paradox of acceptance and rejection: The perception of healthcare professional students about mobile learning acceptance in Iran University of medical sciences. Qual Res Educ 2018;7:144-69. |
| 9. | Graneheim UH, Lundman B. Qualitative content analysis in nursing research: Concepts, procedures and measures to achieve trustworthiness. Nurse Educ Today 2004;24:105-12. |
| 10. | Guba EG. Criteria for assessing the trustworthiness of naturalistic inquiries. ETRD 1981;29:75. |
| 11. | Mousazadeh S, Rakhshan M, Mohammadi F. Investigation of content and face validity and reliability of sociocultural attitude towards appearance questionnaire-3 (SATAQ-3) among female adolescents. Iran J Psychiatry 2017;12:15-20. |
| 12. | Lawshe CH. A quantitative approach to content validity 1. Pers Psychol 1975;28:563-75. |
| 13. | Mohammadbeigi A, Mohammadsalehi N, Aligol M. Validity and reliability of the instruments and types of measurements in health applied researches. J Rafsanjan Univ Med Sc 2015;13:1153-70. |
| 14. | Ashktorab T, Baghcheghi N, Seyedfatemi N, Baghestani A. Psychometric parameters of the Persian version of the BriefCOPE among wives of patients under hemodialysis. Med J Islam Repub Iran 2017;31:20. |
| 15. | Yaghmaei F. Content validity and its estimation. J Med Educ 2003;3:25-7. |
| 16. | Tabachnick BG, Fidell LS, Ullman JB. Using Multivariate Statistics. Boston, MA: Pearson; 2007. |
| 17. | Munro BH. Statistical Methods for Health Care Research. Philadelphia (United States): Lippincott Williams and Wilkins; 2012. |
| 18. | Salaree MM, Zareiyan A, Ebadi A. Development and psychometric properties of the military nurses' job burnout factors questionnaire. J Mil Med 2019;20:645-54. |
| 19. | Zarmpou T, Saprikis V, Markos A, Vlachopoulou M. Modeling users' acceptance of mobile services. Electron Commer Res 2012;12:225-48. |
| 20. | Chong JL, Chong AY, Ooi KB, Lin B. An empirical analysis of the adoption of m-learning in Malaysia. Int J Commun 2011;9:1-18. |
| 21. | Sun Y, Liu L, Peng X, Dong Y, Barnes SJ. Understanding Chinese users' continuance intention toward online social networks: An integrative theoretical model. Electron Mark 2014;24:57-66. |
| 22. | Osiceanu ME. Psychological implications of modern technologies: “technofobia” versus “technophilia”. Procedia Soc Behav Sci 2015;180:1137-44. |
| 23. | Sohrabi Z, Koohestani HR, Baghcheghi N, Delavari S, Rezaei Shahsavarloo Z. The effects of group blogging on the attitude towards virtual education in nursing students. Med J Islam Repub Iran 2017;31:132. |
| 24. | Fishbein M, Ajzen I. Belief, Attitude, Intention, and Behavior: An Introduction to Theory and Research. Reading, MA: Addison-Wesley; 1977. |
| 25. | Koohestani HR, Soltani Arabshahi SK, Fata L, Ahmadi F. The educational effects of mobile learning on students of medical sciences: A systematic review in experimental studies. J Adv Med Educ Prof 2018;6:58-69. |
[Table 1], [Table 2], [Table 3]
| This article has been cited by | | 1 |
Design and evaluation of a smartphone app to teach students about body language and patient management |
|
| Mohamad Shooriabi, Farideh KaabOmeir, Sediqeh Modarres Mousavi, Mohammad Reza Darvishi Lardi, Amin Beshkar | | Telematics and Informatics Reports. 2022; : 100035 | | [Pubmed] | [DOI] | | | 2 |
Designing and psychometric properties of coping strategies scale for family caregivers of hemodialysis patients |
|
| Nayereh Baghcheghi, Hamid Koohestani | | Acta Facultatis Medicae Naissensis. 2021; 38(3): 257 | | [Pubmed] | [DOI] | | | 3 |
Do Medical Students’ Learning Styles and Approaches Explain Their Views and Behavior Regarding Lecture Attendance? |
|
| Ali El Mokahal,Ali Ahmad,Joseph R. Habib,Ali A. Nasrallah,George Francis,Ramzi Sabra,Nathalie K. Zgheib | | Medical Science Educator. 2021; | | [Pubmed] | [DOI] | |
|
 |
|
|
|
Related articles