Protocol of the TOHLA instrument: A Test of Oral Health Literacy in Adults
Mohtasham Ghaffari, Sakineh Rakhshanderou, Ali Ramezankhani, Yadollah Mehrabi, Ali Safari-Moradabadi
Department of Public Health, School of Public Health and Safety, Shahid Beheshti University of Medical Sciences, Tehran, Iran
|Date of Submission||24-Jul-2019|
|Date of Acceptance||22-Oct-2019|
|Date of Web Publication||30-Jan-2020|
Mr. Ali Safari-Moradabadi
Department of Public Health, School of Public Health and Safety, Shahid Beheshti University of Medical Sciences, Tehran
Source of Support: None, Conflict of Interest: None
BACKGROUND: A proper measurement instrument is selected based on the suggested use of the instrument, the target concept of measurement, and features of measurement (e.g. internal consistency, reproducibility, content and construct validity, responsiveness, and interpretability). Concerning the design and features of measurement, there are not any adequately specific standards for instrumentation to measure oral health literacy (OHL). The present study proposes a protocol that attempts to fill this gap by introducing the psychometrics of a standard questionnaire which measures OHL.
MATERIALS AND METHODS: The present research employs a methodological design and is carried out in Tehran, with data collected through interviews that are held face to face. The data collection procedure involves a review of the related literature, cognitive interviews, fuzzy Delphi Method, and focal groups with participants with OHL work experience for item generation. The target participants of this research are the Iranian adult population and experts working professionally in different health domains. For qualitative data analysis, the content analysis strategy and in the instrument Psychometrics COSMIN checklist will use.
CONCLUSIONS: Achievement of the present research will be used to evaluate the capability of the Iranian adult population in searching, processing, and deciding on healthcare services. This instrument will focus on evaluating both clinical and nonclinical settings. The present research can vastly improve our knowledge of the state of OHL in the Iranian adult population.
Keywords: Development, instrument, oral health literacy, protocol, psychometrics
|How to cite this article:|
Ghaffari M, Rakhshanderou S, Ramezankhani A, Mehrabi Y, Safari-Moradabadi A. Protocol of the TOHLA instrument: A Test of Oral Health Literacy in Adults. J Edu Health Promot 2020;9:11
|How to cite this URL:|
Ghaffari M, Rakhshanderou S, Ramezankhani A, Mehrabi Y, Safari-Moradabadi A. Protocol of the TOHLA instrument: A Test of Oral Health Literacy in Adults. J Edu Health Promot [serial online] 2020 [cited 2020 Jul 7];9:11. Available from: http://www.jehp.net/text.asp?2020/9/1/11/277350
| Introduction|| |
Health literacy is an important factor involved in health-related issues., Besides environmental and genetic factors, individuals' health behavior plays a pivotal role in preventing diseases and promoting health. A number of factors can cause prevalent oral diseases.,, In spite of great efforts to improve oral health, still a gap exists between oral health knowledge and practical efforts. As a result, it is maintained that a primary factor involved in oral health is OHL., OHL can be defined as “the degree to which individuals have the capacity to obtain, process, and understand basic oral health information and services needed to make appropriate health decisions.”
Worries about low levels of health literacy made investigators seek for practical methods and instruments to measure patients' health literacy skills. Formerly, researchers and clinicians viewed patient education as a sign of such skills.,,,,
Screening individuals with low levels of OHL is demanding since it is hard to gain information about one's reading ability, comprehension, listening, and other required skills. Educational performance is usually used as a proxy, yet it is usually restricted and might overestimate one's level of OHL. This is because literacy is often several grades below the educational level. So far, 19 instruments have been constructed to assess OHL in the world.,,,,,,,,,,,,,,,,,,
Among the existing instruments, 14 were about dentistry and 5 were about oral/dental health. On the one hand, 12 instruments (19 cases) which measured oral/dental health literacy were restricted to word diagnosis test (short form or rapid estimation) and reading skill. On the other hand, only one instrument among all belonged to Iran. This instrument was derived from rapid estimation of adult literacy in dentistry (REALD)-30, REALD-99, rapid estimation of adult literacy measurement (REALM)-D, OHLI, TOHLA in dentistry (ToFHLiD), and CMOHK. It was only a patchwork and its design was not based on the target concept. As there is no comprehensive instrument to measure oral/dental health literacy in Iran, the present research aims to adopt qualitative methodology to develop and validate such an instrument exclusively for Iran to fill the existing gap in literature and pave the way for a comprehensive exploration.
| Materials and Methods|| |
The procedure involved in this study consists of eleven stages, as shown in [Table 1]. The article will go on with an elaboration of the methodology (participants, materials, procedure, and statistical analysis).
|Table 1: The developmental stages of the Test of Oral Health Literacy in Adults initiative|
Click here to view
Phase I: Development of the Test of Oral Health Literacy in Adults
Three separate groups of participants are needed to develop the Test of Oral Health Literacy in Adults (TOHLA). The source of including and the participants and their characteristics vary depending on the specific stage of the project, as described below.
A wide range of experts in oral/dental health needs to be included for health education, promotion, communications psychology, and social dentistry.
General population-based interviews
A sample of about 20–30 participants should be selected along different demographic features such as age, sex, marital status, education, and expertise.
A second round of selection follows of participants meeting the above-mentioned features now included for psychometric analysis.
For interviews with experts, the target participants needed to have previous knowledge and experience of OHL. They are also supposed to be actively involved in a field with the relevant use of OHL (at least 5 years of field specialty). The inclusion criteria in the interview phase also included participants' willingness to participate and have an Iranian nationality. Those incapable of completing the questionnaires on their own are to be excluded from the study. The required age is 18 years or above.
The eleven developmental stages summarized in [Table 1] follow a procedure as:
A comprehensive review of the related literature is essential in any research project and though listed as a single activity, it enjoys a continuous nature during the research. As for the interview, the semi-structured type is held in Tehran with 17 professionals with different backgrounds and fields of expertise. They include health education, promotion, communications, psychology, and social dentistry. Items at the core of the interview guide and are as follows:
”Which skills and abilities do people need to act in ways that are beneficial for the health of oral and dental and as well as interacting with the healthcare system?”
- Define oral/dental health literacy
- What are the potential skills and capabilities one can have concerning oral/dental health literacy?
The other questions are supposed to be asked along with the primary questions.
There has been no measurement designed so far which, from the very beginning, includes a variety of potential users as a central idea. Moreover, the idiosyncrasies of the instrument were discussed in interviews too. These included the preferred methods of scoring and interpretation rules. After conduction, the interview content is transcribed and prepared for content analysis to be used to inform the later stages of the project. Exploratory, open-ended, face-to-face, interviews are used in-depth among a sample of participants with various features. The aim of the interviews is to elicit the significant aspects of OHL.
Stage 1 and 2 interviews are then transcribed whose content will be independently explored by a research team. A panel of experts, subsequently, evaluate items for completeness, ambiguity, and repetition.
Predictably, about 20–30 participants should be interviewed in-depth until data satiation occurs, i.e. “No new themes emerge.” All interviews are recorded, transcribed, and prepared for content analysis. Although there is an interview guide, participants can freely go beyond this and address other relevant issues. Then, the concepts are extracted from the primary structure of the questionnaire.
To rate items for completeness, ambiguity, and repetition, a Fuzzy Delphi Method is used. This would impede misinterpretation of initial expert opinion and provides for real reactions. Here are the steps. Primarily, three (written) Delphi rounds are involved with approximately 20 expert members of different backgrounds in health education, promotion, communications, psychology, and social dentistry.
Here is the design of the Fuzzy Delphi procedure for TOHLA:
Three Delphi rounds are followed in the Delphi procedure of TOHLA. These are outlined in [Figure 1].
a. Round (1): Issues and Interview Protocol
The extracted themes get approved by the experts, review of the related literature and general population. Thus, “TOHLA instrument” is developed.
b. Round (2): Development and Distribution of Consensus Survey
TOHLA aims to gain a consensus based on the themes proposed by the experts. Experts should agree with a statement. In Round 2, thus, many experts are involved to rate the responses of the survey.
c. Round (3): Fuzzy Delphi Data Analysis.
There are two result indicators in the FDM for decision-making. First, each expert's responses are changed into triangular fuzzy numbers to find their state of mind for each statement. Then, the defuzzification process follows to set the value that indicates respondents' extent of consensus. As the last section of data analysis, a framework is designed to help the author to continue with the development process [Figure 1].
Phase II: Psychometric evaluation of Test of Oral Health Literacy in Adults
The long version of the instrument, which is the outcome of Stage 3 and 4 will be submitted to a sample of 500 people with a variety of features. The sample size is a function of the assumption that the instrument has about 40 items. As estimates have it, at least five times as many respondents as the number of items are required for psychometric tests to make sense.
With a typical response rate of 60%–70%, this can lead to a sample of about 300–350 questionnaires, which can allow the rigorous testing of the instrument.
Data analysis in Stage 5 helps to develop a shorter version of the questionnaire using established statistical procedures as explained here in the statistical analysis section. Items that enjoy high face validity, strong correlation with the scale total which they contribute to, and an acceptable rate of reliability make up an aspect of the resultant instrument.
The short version of the measurement developed in Stage 6, should be submitted to 900 participants with a variety of features. With a response rate of 60%–70% (as formerly discussed), there will be yield a final sample size of about 540–630 questionnaires as the return rate.
The data obtained from stage 7 are further statistically analyzed to approve the results of the primary survey in Stage 6. Secondary amendments can be made to the questionnaire at this stage.
Then, the previous respondents are supposed to complete the questionnaire again. If they agree to do so, the “TOHLA instrument” is sent to them one a week after receiving the measure of Stage 6 to estimate test–retest reliability. Then, previous respondents are asked to complete the questionnaire finally after 3 months to check the responsiveness of the instrument.
The presence of outliers is checked within the data as well as the normal distribution before any further statistical procedure. Shapiro–Wilk test of normality is run for the distribution of the data. TOHLA Items are checked for floor and ceiling effects based on the criteria set by Terwee et al. Then, a factor analysis follows to test construct validity and explore the related dimensions to particular facets of participation and activity. Item-total correlations, which are corrected for probable overlap, are then calculated among items and the total score they contribute which to. To determine which items conform to a hierarchical uni-dimensional structure, Rasch analysis will follow. Other tests of validity (concurrent and discriminant) are run by estimating the Pearson's product-moment  or Spearman rank  correlation coefficients (proportional to data distribution) between the TOHLA and the other instruments. Cronbach's alpha coefficient test of internal consistency is run to test the reliability  along with the test–retest method which uses intra-class correlations. Standardized effect size is also used to check responsiveness to change. To keep confidentiality, the target data are coded analyzed through thr SPSS version 18 (IBM, Armonk, NY, USA). As for qualitative data analysis, the present study enjoys the content analysis strategy introduced by Graneheim and Lundman run in MAXQDA.
Informed consent and discontinuation/withdrawal of participants from the study
Before any special measures taken in the present research, the participants will sign a letter of informed consent. The written and oral versions of participants' data and informed consent will be provided for participants. It clearly states that they are allowed to exit the study regardless of later care and commitment whenever they wish and for any reason. Why they leave will be recorded by the researcher in a separate file.
Quality assurance procedures
To analyze qualitative data, four criteria by Lincoln and Guba will be used. These scholars maintained that a piece of research should be trustworthy. Trustworthiness involves establishing credibility, transferability, dependability, and conformability.
Test of Oral Health Literacy in Adults scoring in a focus group discussion
When the final version of the instrument is developed, the research team will form a discussion focus group with experts to score each item of the questionnaire and then its overall score. Focus group discussion (FGD) members will make such decisions as: using simple or weighted mean scores, weight of each section of instrument, multiple-choice responses for each item, direct or reversed scoring of each item, setting the overall range of score for the whole instrument and qualitative categorization of scores for each respondent (e.g. low, moderate, and high). Categorization will be done by an expert statistician who uses rock test to set the number of cutoff points experts in health literacy agreed on. The schematic structure of the research procedures is presented in [Figure 2].
|Figure 2: Test of Oral Health Literacy in Adults construction and psychometrics flowchart|
Click here to view
The study design obeys the principles of the Declaration of Helsinki  and shows to be in agreement with the ethical principles and national norms/standards for conducting medical research in Iran. To respect participants' rights, certain measures will be taken such as informed consent to take part in the study, permissions to take notes or record voices, deleting audio files after the research, anonymity of all questionnaires, confidentiality of the data, and financial and nonfinancial rights of the whole research team.
| Discussion|| |
There has been increasing research on health literacy or OHL. Yet, the main reasons for low oral/dental health literacy are: inadequate sources of knowledge about oral health, difficult guidelines on oral/dental health and inefficient dentists who are incapable of evaluating patients' literacy needs (2) in developing countries. The majority of health-related content, including drug labels, after-surgery prescriptions and guidelines on oral health in Iran are in English. That is why it is hard for patients to read and comprehend the information content. Moreover, dentists often use specialist language mostly in English, which adds to the difficulty of doctor–patient communication.
The primary instrument to measure oral/dental health is based on health literacy instruments. As an example, the REALD has its basis in the REALM  or a similar case such as the ToFHLiD has come out of ToFHLA. Original instruments elicited similar criticism on general health literacy versions as they were mainly word recognition instruments. These instruments were mostly focused on such places as clinics and healthcare centers and particular populations such as patients.,,,,,. The present research aims to offer a practical and comprehensive instrument to measure all aspects of oral/dental health literacy with a focus on appropriate content for the Iranian adult population. It takes into account settings outside clinics (i.e. home) to avoid selection bias.
The project has been approved by the Ethics Committee of the School of Public Health and Neuroscience Research Centre in Shahid Beheshti University of Medical Sciences; Approval ID: IR. SBMU. PHNS. REC.1397.051: Approval Date: 2019-01-15). The funding bodies had no role in the study design, collection, analysis, or interpretation of data; in the writing of the manuscript; or in the decision to submit the manuscript for publication.
This research has certain strengths, including the fact that it develops and evaluates a comprehensive instrument exclusively for oral/dental health literacy in Iran for the first time. To this aim, it employs qualitative methods and reviews the related literature and existing relevant instruments. It also takes into account expert viewpoints as well as those of the adult population. It employs fuzzy Delphi technique and considers expert comments to complement the questionnaire. This technique leads to better recognition besides other traditional qualitative methods. This method has been also employed successfully in a body of health-related research. The present research uses FGD or experts (in health education, social dentistry, and statistics) to score the oral/dental health literacy instrument which were mostly absent in other similar investigations.
Findings of the present research will be used to evaluate the capability of the Iranian adult population in searching, processing, and deciding on healthcare services. This instrument will focus on evaluating both clinical and nonclinical settings. It will also comprehensively evaluate oral/dental health in dentistry as well as the prevention section (oral health). This instrument can be used in developing and performing educational interventions that affect the level of oral/dental health literacy. It will also act as a valid instrument in prospective investigations to measure oral/dental health literacy of the Iranian adult population.
Financial support and sponsorship
This study was financially supported by Shahid Beheshti University of Medical Sciences Presentation at a meeting.
Conflicts of interest
There are no conflicts of interest.
| References|| |
Dadipoor S, Ramezankhani A, Alavi A, Aghamolaei T, Safari-Moradabadi A. Pregnant women's health literacy in the South of Iran. J Family Reprod Health 2017;11:211-8.
Safari Moradabadi A, Aghamolaei T, Ramezankhani A, Dadipoor S. The health literacy of pregnant women in Bandar Abbas, Iran. J Sch Public Health Inst Public Health Res 2017;15 (2):121-32.
Dadipoor S, Ramezankhani A, Aghamolaei T, Rakhshani F, Safari-Moradabadi A. Evaluation of health literacy in the Iranian population. J Family Reprod Health 2018;7:e62212.
Baur C, Comings J, Evans C, Garcia R, Horowitz A, Ismail A, et al
. The invisible barrier: Literacy and its relationship with oral health. J Public Health Dent 2005;65:174-82.
Hassani L, Aghamolaei T, Ghanbarnejad A, Madani A, Alizadeh A, Safari Moradabadi A. The effect of educational intervention based on BASNEF model on the students' oral health. J Res Health 2016;5 (4):36-44.
Ghaffari M, Rakhshanderou S, Safari-Moradabadi A, Torabi S. Oral and dental health care during pregnancy: Evaluating a theory-driven intervention. Oral Dis 2018;24:1606-14.
Horowitz AM. The public's oral health: The gaps between what we know and what we practice. Adv Dent Res 1995;9:91-5.
Horowitz AM, Kleinman DV. Oral health literacy: The new imperative to better oral health. Dent Clin North Am 2008;52:333-44, vi.
Parker EJ, Jamieson LM. Associations between indigenous australian oral health literacy and self-reported oral health outcomes. BMC Oral Health 2010;10:3.
Davis TC, Michielutte R, Askov EN, Williams MV, Weiss BD. Practical assessment of adult literacy in health care. Health Educ Behav 1998;25:613-24.
Doak CC, Doak LG, Root JH. Teaching Patients With Low Literacy Skills. 2nd
edn. Philadelphia: J.B. Lippincott; 1996.
Davis TC, Crouch MA, Wills G, Miller S, Abdehou DM. The gap between patient reading comprehension and the readability of patient education materials. J Fam Pract 1990;31:533-8.
Davis TC, Mayeaux EJ, Fredrickson D, Bocchini JA Jr., Jackson RH, Murphy PW. Reading ability of parents compared with reading level of pediatric patient education materials. Pediatrics 1994;93:460-8.
Kirsch IS. Adult Literacy in America: A First Look at the Results of the National Adult Literacy Survey. Washington: ERIC; 1993.
Health literacy: Report of the council on scientific affairs. Ad hoc committee on health literacy for the council on scientific affairs, American Medical Association. JAMA 1999;281:552-7.
Richman JA, Lee JY, Rozier RG, Gong DA, Pahel BT, Vann WF Jr., et al.
Evaluation of a word recognition instrument to test health literacy in dentistry: The REALD-99. J Public Health Dent 2007;67:99-104.
Lee JY, Rozier RG, Lee SY, Bender D, Ruiz RE. Development of a word recognition instrument to test health literacy in dentistry: The REALD-30 – A brief communication. J Public Health Dent 2007;67:94-8.
Gong DA, Lee JY, Rozier RG, Pahel BT, Richman JA, Vann WF Jr. Development and testing of the test of functional health literacy in dentistry (TOFHLiD). J Public Health Dent 2007;67:105-12.
Sabbahi DA, Lawrence HP, Limeback H, Rootman I. Development and evaluation of an oral health literacy instrument for adults. Community Dent Oral Epidemiol 2009;37:451-62.
Atchison KA, Gironda MW, Messadi D, Der-Martirosian C. Screening for oral health literacy in an Urban dental clinic. J Public Health Dent 2010;70:269-75.
Macek MD, Haynes D, Wells W, Bauer-Leffler S, Cotten PA, Parker RM. Measuring conceptual health knowledge in the context of oral health literacy: Preliminary results. J Public Health Dent 2010;70:197-204.
Macek MD, Manski MC, Schneiderman MT, Meakin SJ, Haynes D, Wells W, et al.
Knowledge of oral health issues among low-income Baltimore adults: A pilot study. J Dent Hyg 2011;85:49-56.
Wong HM, Bridges SM, Yiu CK, McGrath CP, Au TK, Parthasarathy DS. Development and validation of Hong Kong rapid estimate of adult literacy in dentistry. J Investig Clin Dent 2012;3:118-27.
Lee J, Stucky B, Rozier G, Lee SY, Zeldin LP. Oral health literacy assessment: Development of an oral health literacy instrument for spanish speakers. J Public Health Dent 2013;73:1-8.
Gironda M, Der-Martirosian C, Messadi D, Holtzman J, Atchison K. A brief 20-item dental/medical health literacy screen (REALMD-20). J Public Health Dent 2013;73:50-5.
Wong HM, Bridges SM, Yiu CK, McGrath CP, Au TK, Parthasarathy DS. Validation of the hong kong oral health literacy assessment task for paediatric dentistry (HKOHLAT-P). Int J Paediatr Dent 2013;23:366-75.
Jones K, Parker E, Mills H, Brennan D, Jamieson LM. Development and psychometric validation of a health literacy in dentistry scale (HeLD). Community Dent Health 2014;31:37-43.
Stucky BD, Lee JY, Lee SY, Rozier RG. Development of the two-stage rapid estimate of adult literacy in dentistry. Community Dent Oral Epidemiol 2011;39:474-80.
Junkes MC, Fraiz FC, Sardenberg F, Lee JY, Paiva SM, Ferreira FM. Validity and reliability of the Brazilian version of the rapid estimate of adult literacy in dentistry – BREALD-30. PLoS One 2015;10:e0131600.
Peker K, Köse TE, Güray B, Uysal Ö, Erdem TL. Reliability and validity of the Turkish version of the rapid estimate of adult literacy in dentistry (TREALD-30). Acta Odontol Scand 2017;75:198-207.
Tadakamadla SK, Quadri MF, Pakpour AH, Zailai AM, Sayed ME, Mashyakhy M, et al.
Reliability and validity of Arabic rapid estimate of adult literacy in dentistry (AREALD-30) in Saudi Arabia. BMC Oral Health 2014;14:120.
Cruvinel AFP, Méndez DAC, Oliveira JG, Gutierres E, Lotto M, Machado MA, et al.
The Brazilian version of the 20-item rapid estimate of adult literacy in medicine and dentistry. PeerJ 2017;5:e3744.
Bado FR, Ferreira FM, de Souza Barbosa T, Mialhe FL. Translation and cross-cultural adaptation of the oral health literacy assessment-Spanish to Brazilian portuguese. Pesqui Bras Odontopediatria Clin Integr 2017;17:1-10.
Naghibi Sistani MM, Montazeri A, Yazdani R, Murtomaa H. New oral health literacy instrument for public health: Development and pilot testing. J Investig Clin Dent 2014;5:313-21.
Streiner DL, Norman GR, Cairney J. Health Measurement Scales: A Practical Guide to their Development and Use. USA: Oxford University Press; 2015.
Mohamad SA, Embi MA, Nordin NJ. Determining e-portfolio elements in learning process using Fuzzy Delphi analysis. Int Educ Stud 2015;8:171-6.
Jackson JE. Principal components and factor analysis: Part I – Principal components. J Qual Technol 1980;12:201-13.
Asch DA, Jedrziewski MK, Christakis NA. Response rates to mail surveys published in medical journals. J Clin Epidemiol 1997;50:1129-36.
Shapiro SS, Wilk MB. An analysis of variance test for normality (complete samples). Biometrika1965;52:591-611.
Terwee CB, Bot SD, de Boer MR, van der Windt DA, Knol DL, Dekker J, et al.
Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol 2007;60:34-42.
Hobart J, Cano S. Rasch analysis. In: Jenkinson C, Peters M, Bromberg M, editor. Quality of Life Measurement in Neurodegenerative and Related Conditions. Cambridge, UK: Cambridge University Press; 2011.
Pearson KJ. Containing papers of a mathematical or physical character. VII. mathematical contributions to the theory of evolution.–III. Regression heredity panmixia. Philos Trans R Soc Lond 1896;195:253-318.
Spearman C. The proof and measurement of association between two things. By C. Spearman, 1904. Am J Psychol 1987;100:441-71.
Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika 1951;16:297-334.
Shrout PE, Fleiss JL. Intraclass correlations: Uses in assessing rater reliability. Psychol Bull 1979;86:420-8.
Cohen J. Statistical Power Analysis for the Behavioral Sciences. 2nd
ed. Mahwah, NJ, USA: Lawrence Erlbaum; 1988.
Schwandt TA, Lincoln YS, Guba EG. Judging interpretations: But is it rigorous? trustworthiness and authenticity in naturalistic evaluation. New Dir Eval 2007;2007:11-25.
World Medical Association. World medical association declaration of Helsinki. Ethical principles for medical research involving human subjects. Bull World Health Organ 2001;79:373-4.
[Figure 1], [Figure 2]