Home About us Editorial board Search Browse articles Submit article Instructions Contacts Login 
Users Online: 892
Home Print this page Email this page

 



 
Previous article Browse articles Next article 
ORIGINAL ARTICLE
J Edu Health Promot 2021,  10:291

Validity and reliability of a virtual education satisfaction questionnaire from the perspective of cardiology residents during the COVID-19 pandemic


1 Rajaie Cardiovascular Medical and Research Center, Iran University of Medical Sciences, Tehran, Iran
2 Cardiovascular Intervention Research Center, Rajaie Cardiovascular Medical and Research Center, Iran University of Medical Sciences, Tehran, Iran
3 Cardiac Electrophysiology Research Center, Rajaie Cardiovascular Medical and Research Center, Iran University of Medical Sciences, Tehran, Iran
4 Cardiovascular Intervention Research Center; Cardio-Oncology Research Center, Rajaie Cardiovascular Medical and Research Center, Iran University of Medical Sciences, Tehran, Iran

Date of Submission08-Jan-2021
Date of Acceptance31-Jan-2021
Date of Web Publication31-Aug-2021

Correspondence Address:
Dr. Bahram Mohebbi
Cardiovascular Intervention Research Center, Cardio-Oncology Research Center, Rajaie Cardiovascular Medical and Research Center, Address: Niyaysh Ave, Vali-Asr Street, P.O. Box: 199691115, Tehran
Iran
Login to access the Email id

Source of Support: None, Conflict of Interest: None


DOI: 10.4103/jehp.jehp_32_21

Rights and Permissions
  Abstract 


BACKGROUND: The coronavirus disease 2019 (COVID-19) pandemic has prompted the further virtualization of medical education. The satisfaction level of specific users such as cardiology residents with virtual education can augment its quality; hence, the significance of a valid and reliable questionnaire to obtain feedback is needed. This study aimed to design and measure validity and reliability of a satisfaction questionnaire for virtual education of cardiology residents during COVID-19 pandemic.
MATERIALS AND METHODS: In this cross-sectional study, a self-administered questionnaire was developed by the faculty members of Rajaie Cardiovascular Medical and Research Center. Reliability was tested utilizing Cronbach's alpha and intercorrelation which was tested using Pearson's correlation coefficient test (ICC). Factor analysis was done by the Kaiser–Meyer–Olkin measure of sampling adequacy and Bartlett's sphericity test. The statistical analyses were performed with the SPSS software version 22.
RESULTS: The face validity index was determined via an assessment of the relevance, clarity, and simplicity of each item, and values >0.79 were accepted. The total Cronbach's alpha coefficient was calculated 0.93. Concerning test–retest reliability, the correlation between two rounds of evaluation was >80 (P > 0.001) and ICC was 0.99 (P = 0.001). The content validity evaluation yielded an index of 0.95 and a ratio of 0.91. The principal component factor analysis, conducted to investigate construct validity, generated four domains.
CONCLUSIONS: The study results confirmed the validity and reliability of the designed questionnaire to evaluate the level of satisfaction of cardiology residents with virtual learning in COVID-19 pandemic.

Keywords: Coronavirus disease 2019 pandemic, cardiology, questionnaire, reliability, residents, validity


How to cite this article:
Ghadrdoost B, Sadeghipour P, Amin A, Bakhshandeh H, Noohi F, Maleki M, Peighambari MM, Haghjoo M, Alemzade-Ansari MJ, Sadeghvashveshady M, Mohebbi B. Validity and reliability of a virtual education satisfaction questionnaire from the perspective of cardiology residents during the COVID-19 pandemic. J Edu Health Promot 2021;10:291

How to cite this URL:
Ghadrdoost B, Sadeghipour P, Amin A, Bakhshandeh H, Noohi F, Maleki M, Peighambari MM, Haghjoo M, Alemzade-Ansari MJ, Sadeghvashveshady M, Mohebbi B. Validity and reliability of a virtual education satisfaction questionnaire from the perspective of cardiology residents during the COVID-19 pandemic. J Edu Health Promot [serial online] 2021 [cited 2022 Nov 28];10:291. Available from: https://www.jehp.net//text.asp?2021/10/1/291/324904




  Introduction Top


The coronavirus disease 2019 (COVID-19) pandemic has significantly impacted educational systems the world over. Medical training with medical educationists now, faced with the unenviable task of altering hospital-based education given the contagious circumstances where clinical mentors and residents work.[1],[2]

Virtual learning has proven itself to be the solution of choice during the current pandemic; nonetheless, from the perspective of medical training, reservations have been raised vis-à -vis familiarity with technology, face-to-face interactions, and control.[3] Indeed, unless virtual learning takes heed of the attitude of its users, it cannot be evolved. Specialized medical educational programs must always reflect the ever-changing and ever-increasing demands of society, at large, and the medical community, in particular.[4] Indubitably, fulfilling such needs in the COVID-19 era, in which the hitherto practiced methods of medical training—for instance, as regards angiography and echocardiography—have been rendered well-nigh unfeasible, requires the optimal use of virtual learning.[4]

Beside several benefits that virtual learning offers to educational system, some detriment points make immediate concerns among medical students, residents, and their attends such as impossibility of face-to-face interaction between the professor and the student, less control of the professors over the learners, and unfamiliarity of some professors with the online education system.[3]

The development of virtual learning will not be successful without considering the attitude of users toward this system. Success in virtual learning in medical fields depends, to a large extent, on the satisfaction of residents and professors and their willingness to use it. The quality of specialized medical education programs has not changed in line with the changing needs of society, changing the face of disease, and changing people's expectations of the medical community and healthcare systems.[4]

The idea that virtual education was a new learning domain is crucial. It means that virtual education is a domain in its own right, and it can be combined in a course with both face-to-face and distance learning.[5] Motivation and learning behavior are two very important factors in determining students' learning achievement. Perceived usefulness, management support, self-efficacy, technical support, and training are some factors of learning satisfaction.[6]

Aspects of learner motivation and learning behavior constitute a key factor in the achievement of competency standards in educational process. Final process of motivation is completing an action that can provide satisfaction.[7] Several studies show that both intrinsic and extrinsic motivation serves to reinforce attitudes toward behavior, or motivation can make a connection between attitudes and behavior.[8],[9] Collected data from a valid and reliable questionnaire could improve quality of virtual education via promoting residents' motivation.

Research has shown that a systematic perspective toward receiving feedback from medical residents can promote higher quality standards of specialized education.[5] Given the exigencies of the COVID-19 pandemic, it seems advisable that a valid and reliable questionnaire be devised to obtain medical residents' feedback on virtual education. To that end, we endeavored to develop and validate a questionnaire on the level of satisfaction of cardiology residents with virtual learning in the COVID-19 pandemic.


  Materials and Methods Top


Study design and population

This cross-sectional study evaluated the validity and reliability of a satisfaction questionnaire for cardiology residents regarding virtual education during the COVID-19 pandemic in a tertiary teaching hospital in the Iranian capital, Tehran, in 2020.

Data collection

A self-administered questionnaire was devised by the faculty members of Rajaie Cardiovascular Medical and Research Center on the basis of scientific resources and expert opinion. The instrument consisted of three parts. The first part comprised 23 questions about various virtual training-related factors such as the quality and content of virtual education, adequacy of teaching hours, interactions between professors and residents, interactions between residents themselves, internet facilities, willingness to learn practical skills such as angiography and echocardiography, and eagerness to conduct research. This part of the questionnaire evaluated the above areas through the format of a five-point Likert scale encompassing “strongly disagree,” “disagree,” “neutral,” “agree,” and “strongly agree” to allow respondents to express how much they agreed or disagreed with a particular statement. Thus, the scores obtained in this section ranged between 23 and 115. The second part consisted of four questions each demanding a choice between two options: “face-to-face learning” and “virtual learning.” The third part comprised two visual scales to enable respondents to specify their level of satisfaction with both “face-to-face learning” and “virtual learning” on a continuum from zero, denoting minimum satisfaction, to 10, indicating maximum satisfaction.

Content validity was determined using qualitative and quantitative methods. Qualitative content validity was assessed by 14 cardiologists, who commented on the questionnaire's grammar, sentence structure, and placement of phrases in the appropriate place, while quantitative content validity was evaluated using the coefficients of the content validity ratio (CVR) and the content validity index (CVI). For CVR evaluation, the expert group was asked to rate each question on a three-part “essential” scale: “essential;” “useful, but not essential;” and “not necessary.” The acceptable CVR, which depends on the number of specialists commenting on the instrument, was considered 0.49 in the present study.[6],[7]

Construct validity was evaluated using principal component factor analysis to determine the domains of the designed questionnaire in the form of multiple-choice questions. For this purpose, a factor analysis using a varimax rotation was conducted, and the Kaiser–Meyer–Olkin (KMO) measure of sampling adequacy was employed. Interpretation optimization was ensured through the use of a Promax rotation.

External reliability was assessed by five cardiology fellows in different cardiovascular fields. The fellows measured the reproducibility of the answers to the questions at 2 weeks' intervals via the test–retest method; then, the correlations between the results were calculated. Correlations exceeding 0.7 were considered to be reliable features.

Internal consistency varies from zero to 1 which was assessed using Cronbach's alpha. Scales with internal consistency coefficients >0.7 were regarded as acceptable results.

Ethical consideration

The study was approved by the Ethics Committee of Rajaie Cardiovascular Medical and Research Center (Ethical Code Number: IR.RHC.REC.1399.100). The purpose of the study was informed to participants and written informed consent was obtained. Confidentiality was assured by informing that the information recorded was used for research purposes only and that no personal details would be recorded or produced on any documentation related to the study.

Statistical analysis

Reliability was tested for internal consistency utilizing Cronbach's alpha, and inter-item correlation was tested using Pearson's correlation coefficient. In addition, the inter-rater consistency of the raters was determined by applying the intraclass correlation coefficient (ICC), the benchmark for which was as follows: >0.75: excellent; between 0.40 and 0.75: moderate; and <0.40: poor.[8] Internal consistency as regards the interscale correlation was examined using Cronbach's alpha. Cronbach's alpha coefficient for the entire questionnaire was considered to be 0.78 (0.73–0.77). Descriptive analysis was conducted to describe the data. For the purposes of factor analysis to investigate construct validity, two tests were applied: the KMO measure of sampling adequacy and Bartlett's sphericity test. The former is a statistical measure that denotes the proportion of variance among variables likely generated by underlying factors, with values of close to 1 generally taken to indicate the possibility of the usefulness of a factor analysis with the data and values less than 0.50 taken to indicate the inadequacy of the factor analysis results. The latter compares the observed correlation matrix with the identity matrix. Overall, values >0.50 in the KMO measure of sampling adequacy and P < 0.05 in Bartlett's sphericity test are considered acceptable.[9],[10] The significance level was set at a P < 0.05. The statistical analyses were performed with the SPSS software, version 22, for Windows (SPSS Inc., Chicago, Illinois, U.S.A).


  Results Top


As a measure of internal consistency, for the first part of the questionnaire (23 questions), Cronbach's alpha coefficient for each question ranged from 0.75 to 0.91 and the total Cronbach's alpha coefficient was 0.93, indicating the goodness of the overall reliability of the instrument. Furthermore, Cronbach's alpha coefficient for reliability was 0.95, denoting the stability of the whole questionnaire. Apropos the test–retest reliability, the correlation between the two evaluation rounds was more than 80 (r > 0.80 and P > 0.001) and ICC was 0.99 (P = 0.001), which demonstrated relatively good stability for the questionnaire.

With respect to the content validity of the questionnaire, the results showed a CVI of 0.95 and a CVR of 0.91. For face validity, the relevance, clarity, and simplicity of each item were evaluated, and values >0.79 were accepted.

The appropriateness of the instrument for evaluating the intended purpose, the difficulty level of the instrument for the target group (cardiology residents), and the adequacy of the questionnaire construction, which constitute the parameters that show the content validity of a questionnaire, were given high scores by the respondents. The cardiology residents in the present study had no difficulty understanding the questions; it can, therefore, be concluded that the content validity of this questionnaire was also high.

Principal component factor analysis for investigating construct validity was carried out to determine the domains of the designed questionnaire in the form of multiple-choice questions. Kaiser's criterion (eigenvalue = 1.0), considered to determine the main factors of the questionnaire, found four domains. The values of Bartlett's sphericity test were statistically significant (χ2 = 2393.63, df = 435; P < 0.001). The KMO measure of sampling adequacy yielded a value of 0.921, indicating that the degree of common variance among the 23 items was acceptable. The results of the promax rotation, performed to optimize interpretation, are presented in [Table 1]. The first domain comprised 10 items on the quality and content of virtual education, the second domain consisted of eight items on hours/days in the week allocated to virtual education and also the facilities required for e-learning, the third domain contained three questions on practical learning for cardiology residents, and the fourth domain comprised two items on willingness to do research activity.
Table 1: Factor analysis on the validity and reliability of the virtual education satisfaction questionnaire from the perspective of cardiology residents during the COVID-19 pandemic

Click here to view



  Discussion Top


This study aimed to design a valid and reliable questionnaire to evaluate the level of satisfaction of cardiology residents with virtual education in the COVID-19 pandemic.

A viable management strategy as regards satisfaction creation in a workforce is to prevent dissatisfaction,[11] which requires the regular measurement of these concepts with the aid of valid and reliable instruments. The fact that the training of medical specialists, as valuable human resources in the healthcare system, is a significant goal pursued in medical education prompted us to design a questionnaire with a view to evaluating satisfaction with virtual education among cardiology residents.

Reliability refers to the degree of the replicability of the results obtained by a measurement instrument. However, its significant contribution to assess validity of a questionnaire notwithstanding, it deemed a sufficient condition for the validity of the questionnaire.[12] The validity of a questionnaire, on the other hand, certifies the ability of its scales to measure the intended concept.[13]

The principal component factor analysis revealed four strong, clinically relevant domains. The first domain covered the quality and content of virtual education for cardiology residents, the second domain enquired about hours/days in the week allocated to virtual education and also the facilities required for e-learning, the third domain investigated the practicality of learning for respondents, and the fourth domain determined residents' eagerness to do research activity. Factor loadings were all within the acceptable range (0.61–0.89), transcending the recommendations of other reference studies (at least 0.4 for factor loadings).[14]

The results from the current study demonstrated that our questionnaire had good reliability and validity. Cronbach's alpha coefficient was greater than 0.9, indicative of good internal uniformity. In other words, the respondents had almost the same perception of the instrument. Moreover, with respect to the reproducibility and test–retest of our questionnaire, the results indicated not only an appropriate level of reliability but also the reproducibility, stability, and internal coherence of the items. Alternatively stated, if the measuring instrument is repeated in the same conditions, similar results will be obtained. Finally, the results also showed that the questionnaire had good validity in this study population.

In summary, five distinctive attributes of virtual education are (1) many-to-many (group communication); (2) any place (place-independence); (3) any time (time-independence); (4) text-based (enhanced by multiple media); and (5) computer-mediated messaging.[15] However, since face-to-face learning also has many benefits especially for cardiovascular residents who need clinical training, awareness of their willingness and satisfaction in changing methods of training after the end of COVID-19 pandemic and even permanent changes in their educational curriculum as a combined virtual and face-to-face method, considering a suitable instrument developed in the same field in various fellowship fields of cardiology seems important. Consequently, this questionnaire as a reliable and valid instrument can be used to assess satisfaction level of cardiology residents in virtual residents. This will help clinical professionals improve virtual educational plan for better training of cardiology residents.


  Conclusions Top


The study results confirmed the validity and reliability of our Persian questionnaire, designed to evaluate the level of satisfaction of cardiology residents with virtual learning in the COVID-19 pandemic. Validated questionnaires that report medical residents' feedback on virtual education, especially in the period of the coronavirus outbreak, can assist educationists in opting for the most efficient medical training methods. We believe that the questionnaire introduced herein can help health policymakers and planners effectively devise comprehensive, yet easily comprehensible, educational programs.

Acknowledgment

The authors express their gratitude to Sara Mortaz Hejri, MD, PhD, and all other participated in the study. We would also like to express our special gratitude to deputy of research of Rajaie Cardiovascular Medical and Research Center for facilitation implementation of this study with registered number 99074.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.



 
  References Top

1.
Ahmady S, Shahbazi S, Heidari M. Transition to virtual learning during the coronavirus disease-2019 crisis in Iran: opportunity or challenge? Disaster Med Public Health Prep. 2020;14 (3):e11-e12. doi: 10.1017/dmp. 2020.142. PMID: 32375914; PMCID: PMC7264447.  Back to cited text no. 1
    
2.
Tabatabai S. COVID-19 impact and virtual medical education. J Adv Med Educ Prof. 2020;8 (3):140-143. doi: 10.30476/jamp.2020.86070.1213. PMID: 32802908; PMCID: PMC7395196.  Back to cited text no. 2
    
3.
Abbaszadeh A. Nursing teacher's perception of the challenges of clinical education and solutions. J Qual Res Health Sci 2013;2;134-45.  Back to cited text no. 3
    
4.
Johande J, Gbadamosi G. Predicting students' satisfaction through service quality in higher education. Int J Manage Educ 2013;11:107-18.  Back to cited text no. 4
    
5.
Dufrene RL. An evaluation of a patient satisfaction survey: Validity and reliability. Eval Program Plann 2000;23:293-300.  Back to cited text no. 5
    
6.
Shakibazadeh E, Sabouri M, Mohebbi B, Tol A, Yaseri M. Validity and reliability properties of the Persian version of perceived health competence scale among patients with cardiovascular diseases. J Edu Health Promot 2020;9:293.  Back to cited text no. 6
    
7.
Mohebbi B, Tol A, Shakibazadeh E, Yaseri M, Sabouri M, Agide FD. Testing Psychometrics of Healthcare Empowerment Questionnaires (HCEQ) among Iranian Reproductive Age Women: Persian Version. Ethiop J Health Sci. 2018 ;28 (3):341-346. doi: 10.4314/ejhs.v28i3.12. PMID: 29983534; PMCID: PMC6016357.  Back to cited text no. 7
    
8.
Stone AT, Bransford RJ, Lee MJ, Vilela MD, Bellabarba C, Anderson PA, Agel J. Reliability of classification systems for subaxial cervical injuries. Evid Based Spine Care J. 2010;1 (3):19-26. doi: 10.1055/s-0030-1267064. PMID: 22956924; PMCID: PMC3427957.  Back to cited text no. 8
    
9.
Santos RO, Gorgulho BM, Castro MA, Fisberg RM, Marchioni DM, Baltar VT. Principal Component Analysis and Factor Analysis: differences and similarities in Nutritional Epidemiology application. Rev Bras Epidemiol. 2019 ;22:e190041. doi: 10.1590/1980-549720190041. PMID: 31365598.  Back to cited text no. 9
    
10.
Masaeli N, Omranifard V, Maracy MR, Kheirabadi GR, Khedri A. Validity, reliability and factor analysis of Persian version of schizophrenia quality of life scale. J Educ Health Promot. 2016;5:10. doi: 10.4103/2277-9531.184547. PMID: 27512702; PMCID: PMC4959255.  Back to cited text no. 10
    
11.
Fesharaki MG, Talebiyan D, Aghamiri Z, Mohammadian M. Reliability and validity of Job Satisfaction Survey questionnaire in military health care workers. Iran J Mil Med 2012;13:241-6.  Back to cited text no. 11
    
12.
Taherdoost H. Validity and reliability of the research instrument; How to test the validation of a questionnaire/survey in a research. Int J Acad Res Manage 2016;5:28-36.  Back to cited text no. 12
    
13.
Fesharaki MG. The study of factors involved in job satisfaction of worker who works in Najmie subspecialty hospital in 2010-2011. Iran J Mil Med 2012;13:241-6.  Back to cited text no. 13
    
14.
Sadeghi-Gandomani H, Alavi NM, Afshar M. Psychometric Testing of the Persian Version of the Conditions of Work Effectiveness Questionnaire-II (CWEQ-II-PV). J Educ Health Promot. 2019 ;8:32. doi: 10.4103/jehp.jehp_285_18. PMID: 30993125; PMCID: PMC6432808.  Back to cited text no. 14
    
15.
Barbera E, Clara M, Linder-Vanberschot J. Factors influencing student satisfaction and perceived learning in online courses. Elearn Digit Media 2013;10:226-35.  Back to cited text no. 15
    



 
 
    Tables

  [Table 1]


This article has been cited by
1 Learning and quality of life in undergraduate nursing students: An exploratory study in Chile during the COVID-19 pandemic
Alejandro Almonacid-Fierro, Karla Valdebenito
Journal of Education and Health Promotion. 2022; 11(1): 216
[Pubmed] | [DOI]



 

Top
Previous article  Next article
 
  Search
 
Similar in PUBMED
   Search Pubmed for
   Search in Google Scholar for
 Related articles
Access Statistics
Email Alert *
Add to My List *
* Registration required (free)

 
  In this article
Abstract
Introduction
Materials and Me...
Results
Discussion
Conclusions
References
Article Tables

 Article Access Statistics
    Viewed751    
    Printed20    
    Emailed0    
    PDF Downloaded125    
    Comments [Add]    
    Cited by others 1    

Recommend this journal