Review process of the health scientific journals according to explanation of experts
Hasan Ashrafi-Rizi1, Rahele Samouei2
1 Health Information Technology Research Center, Isfahan University of Medical Sciences, Isfahan, Iran
2 Social Determinants of Health Research Center, Isfahan University of Medical Sciences, Isfahan, Iran
|Date of Submission||21-Mar-2019|
|Date of Acceptance||11-Jun-2019|
|Date of Web Publication||30-Sep-2019|
Dr. Rahele Samouei
Isfahan University of Medical Sciences, Isfahan
Source of Support: None, Conflict of Interest: None
INTRODUCTION: Reviewing articles are one of the most important methods for maintaining and improving the scientific quality of research outputs, especially in the field of health and medicine, and are often accompanied with various challenges.
AIM: The current study was carried out to Health Promotion in the Review Process of the Health Scientific Journals according to Explanation of Experts.
MATERIALS AND METHODS: This study has a qualitative approach and was carried out using the content analysis method. Data were gathered through semi-structured interviews with experts with direct and related experiences with health scientific journals including editors in chief, internal managers, editorial boards, authors' council and members of national journal commission with at least 2 years of continuous related work experience in journals and review of at least 10 articles. Sampling was carried out using purposeful snowball sampling, and data were analyzed using content analysis method. Lincoln and Guba tests were used to determine the validity and reliability of the analysis based on the following four criteria – credibility, transferability, certainty, and verifiability.
RESULTS: Experts' opinions were categorized based on criteria for reviewer selection with three dimensions of technical expertise, ethical behavior, and orientation and order; reviewer selection methods including emphasis on others and emphasis on self; and review problems in the Iranian Health Science and Research Journals including incentive system, reviewer characteristics, and structural problems.
CONCLUSION: Findings of the current study are usable for all Iranian Health Scientific Journals, editors, editors in chief, and internal managers as well as lawmakers in the area of scientific research.
Keywords: Articles, health promotion, health scientific journals, review process
|How to cite this article:|
Ashrafi-Rizi H, Samouei R. Review process of the health scientific journals according to explanation of experts. J Edu Health Promot 2019;8:187
|How to cite this URL:|
Ashrafi-Rizi H, Samouei R. Review process of the health scientific journals according to explanation of experts. J Edu Health Promot [serial online] 2019 [cited 2019 Nov 18];8:187. Available from: http://www.jehp.net/text.asp?2019/8/1/187/268100
| Introduction|| |
Reviewers are the guides for authors, editorial staff, and editors for improving the quality of journals. Given the fact that determining the importance and originality of research articles is only possible through reviewing, review process is one of the essential parts of scientific journals. One of the goals of review process is to ensure that the experiences of authors are evaluated by people with similar scientific expertise who then evaluate different dimensions of the articles including content, structure, and grammar and offer constructive suggestions for improving articles' quality. However, given the important role of reviewers, finding suitable reviewers for every article is difficult. This is especially important because reviewing the articles is not possible through feeling and intuition and instead requires learning, education, and personal experience., Given the complex and sensitive nature of review process (peer-review), selecting reviewers in scientific journals are carried out through different means which include a list of possible reviewers offered by the journal, suggestions by editorial staff, searching in similar articles and publications and various other methods. Selection as a reviewer requires features such as the renowned scientific journal, past productive and useful reports, and enough experience to review scientific articles. As an expert in this field, it is also a necessary feature of the reviewers on a regular basis, with the publication of related articles, sufficient academic background, and relevant research experience. Other than the necessary characteristics of reviewers, the review process itself often faces numerous challenges due to long review process, lack of expert reviewers, and lack of necessary criteria for selecting and evaluating reviewers. In this regard, Manchikanti et al. mentioned problems such as lack of understanding among reviewers and editorial staff regarding the content of articles, delays in review process and publication, lack of transparency in the review process, and plagiarism in scientific journals. Furthermore, Vettore also mentioned problems such as delays in the review process, lack of coordination among reviewers regarding articles' strengths and weaknesses, and small effect of health-related articles in recent years on public health and medical situation.
On the other hand, due to lack of proper review and evaluation, some of the policymaking decisions are based on the results of research works which have weak practical aspects and sometimes adverse effects. As a result, improving the quality of research articles, the majority of which is carried out through improving peer review process, can result in evidence-based policymaking and therefore optimized decisions and actions. We should also remember that using the results of studies in the fields of health and medicine directly affects health and wellbeing of people. This means that ensuring the quality of research articles by authors and later by editorial staff of related journals is of outmost importance.
Previous studies on review process of articles often cover aspects such as methods for improving the quality of articles,,,, average review time, acceptance and rejection ratio of articles, reviewers' characteristics, review ethics,, effects of reviewers' demographic characteristics on review process, principles of writing review reports, and criteria and measures or review process., In fact, similar studies rarely investigate the problems of the review process from the point of view of different people. Therefore, the current study aims to use a qualitative approach to determine the criteria and methods of reviewer selection and barriers and problems of cooperation between reviewers and journals in the field of health and medicine based on opinions of experienced experts. Then, these opinions are used to provide suggestions for removing barriers of fair review process and improving the quality of published articles. The aim of the current study is to Health Promotion in the Review Process of the Health Scientific Journals according to Explanation of Experts.
| Materials And Methods|| |
This study was carried out with qualitative approach and using content analysis method. Data were gathered through semi-structured interviews with experts with direct and related experiences with health scientific journals including editors in chief, internal managers, editorial boards, authors' council and members of national journal commission with at least 2 years of continuous related work experience in journals and review of at least 10 articles. Sampling was carried out using purposeful sampling method and snowball sampling was used to identify and complete the sample. Interview questions were designed based on the main aims of the study in an interview guide. This form was first tested on two researchers who fulfilled the inclusion criteria. Then, due to the lack of need for adjustments, the same form was used for the main interviews. During the interview process, first interview questions were sent to the interviewees and an appointment for the interview was set. For those without the possibility of face-to-face interview, interviews were conducted over the phone. After the interviews, the recording was listened several times, and interview transcripts were written. Interviews were continued until no new ideas could be extracted from new interviews (data saturation). In total, 21 experts participated in the interviews. The content analysis method was used for data analysis. To this end, each transcript was read several times and each topic received a specific code. Similar codes created factors at a more abstract level which were more universal compared to the initial codes. After comparisons, similar codes were integrated, and final factors were determined. This stage was carried out manually. In order to determine the validity and reliability, Lincoln and Guba tests were used based on four criteria of credibility, transferability, dependability, and conformability. Next, the examples for each factor were determined. In order to validate the data, peer opinions were considered, and the results were shared with the interviewees. In order to increase the certainty of data gathering and interview process, details of the method was recorded. In order to improve transferability of gathered data, the phenomenon and research conditions were fully explained. In order to improve the dependability of data, research notes during interviews were presented to reviewers and details of the method were presented to peers, and their opinions on the results were sought. Participants entered the study after filling a written consent form and could leave the study at any given time.
| Results|| |
Experts' opinions regarding the criteria for selecting reviewers are presented in [Table 1]. Some of the quotes by interviewees are as follows:
|Table 1: Determining the experts' opinions regarding reviewer selection criteria for Iranian Health Scientific Journals|
Click here to view
Review knowledge and experience
“It is good to answer comments on time but it shouldn't be the most important reason. We have to work with them, some good reviewers answer late; we shouldn't complain too much.” Work field and experience: “In the journal's system, there is a place where reviewers can decline a review if they are not familiar with the field or haven't worked in any related fields and instead introduce reviewers who are familiar with the field.” Research knowledge and experience: “It is possible to have someone who isn't a faculty member but does a lot of research.” Lack of bias: “Some reviewers are too negative or are too sure of themselves and reject everything while others just offer two simple comments and accept everything.” Lack of expectations: “Sometimes doing reviews for the journal on time or any cooperation with the journal leads to future expectations like expecting to publish their articles without review or outside of the normal process.” Time management: “They should have enough time for reviews for example if we know someone is very busy and it's unlikely for them to answer, we shouldn't send them review work unless it's a very special case and we should remind them of these limited cases.”
[Table 2] shows the experts' opinions regarding methods for selecting reviewers. Some of the quotes from interviews are shown below.
|Table 2: Experts' opinions regarding methods for reviewer selection criteria for the Iranian Health Science and Research Journals|
Click here to view
Help from others
“We can also ask the authors themselves, but it's only a suggestion. Journal can select one of the suggested reviewers while adding one or two others.” Based on previous information: “Having a database of experts in different fields with contact information and the quantity and quality of their work can be helpful. Of course, paying attention to any published articles in journals or other sources can help expand this data bank.” Based on available resources: “Some researchers contact the journal themselves and express interest in doing reviews. It's possible to use these people.”
[Table 3] shows some of the experts' opinions regarding review problems with some of the quotes being shown below.
|Table 3: Experts' opinions regarding review problems in for the Iranian Health Science and Research Journals|
Click here to view
Incentive and payment system
“Most of the times there are no incentives for doing reviews. Medical journals managed to fix this problem somewhat by offering certificates for each few hours of work which helps in promotions but offering reasonable payments can be very helpful.” Reviewer characteristics: “Some of the reviewers have very busy schedules and not enough time and despite offering help for reviews don't answer which results for delays for articles.” “Some of the reviewers are biased toward the journal for example because their own article was not accepted and want to show their grudge or want to blackmail the journal to publish their own article faster.” Rules and regulations: “Since journal can only offer certificates to reviewers, some faculty members only review a limited number of articles and those not members of faculty don't care about these certificates.” Planning and implementation: “There are no differences between strong and weak reviews, there is no motivation because everything is equal and there is not even more respect for better reviewers.”
| Discussion|| |
Findings showed that professional knowledge, research knowledge and experience, review knowledge and experience, related work field and experience, and lack of bias are among the most important subcomponents of technical and professional abilities during reviewer selection. This is in agreement with results obtained by Olia and Shakiba regarding good understanding and knowledge of reviewer regarding article's scientific topic and also results obtained by Fatahi. Lack of bias, lack of expectations, and ethical credibility are among important ethical behaviors of reviewers which is in agreement with the results of Olia and Shakiba about unfair and biased acceptance or rejection of articles by some reviewer and other studies regarding ethical considerations during review process., Being up to date, time management, precise and full review of the article are among requirements for order and coordination among reviewers. The findings of the current study regarding time management are in agreements with findings by Abu Ardakan et al. and also findings by Rahimi Nejad and others regarding long review times.,,
Peer review is a process which requires simultaneous skills and abilities in different areas. Along with suitable technical and scientific abilities, reviewers should also have ethical characteristics and tempered behavior, be fair and committed to offering an acceptable review. These require having useful experiences and characteristics such as being orderly, consistent, and timely.
Findings regarding method of selecting reviewers in the Iranian Health Science and Research Journals showed that help from others, selection based on pervious information, online search, selection based on available sources and based on educational and work experience are the most important methods for selecting reviewers based on the two main components of relations and experiences. These findings are in agreement with the findings of Abu Ardakan et al. and Rahimi Nejad regarding the important role of editorial staff in selecting reviewers.
Selecting proper reviewers plays an important role in improving, optimizing, and reaching logical decisions about articles. According to pervious findings, reviewers should have the necessary characteristics. Journals and editors alone cannot handle the needs of all articles reaching the review stage. Therefore, journals should always be competent in using suggestions and various search strategies.
Findings regarding review problems in the Iranian Health Science and Research Journals showed that material and nonmaterial incentives, personal and behavioral characteristics, scientific characteristics, rules and regulations, and planning and implementation are among the main problems in the review process which can be categorized in the three main categories of incentive and payment system, reviewer characteristics and structural problems. The findings regarding financial problems are similar to the findings by Abu Ardakan et al. while findings regarding reviewer characteristics such as lack of transparency, reviewer ethics,, and understanding of review principles and regarding structural problems such as scientific problems of reviewers and lack of professional reviewers and problems of editorial staff in selecting reviewers are in agreement with the results of other previous studies.
Currently, health scientific journals are faced with various challenges and problems in regards to review of articles. Some of these problems can be solved through creating new rules and regulations especially in regards to incentive system while others are related to personal and scientific characteristics of reviewers which are less intense in areas with proper investment in cultivation and education.
The limitations of this study included busy schedules of journal editors and problems or lack of success for setting appointments for interviews.
| Conclusion|| |
According to experts the most important criteria for selecting reviewers include professional and technical ability, ethical issues, coordinated and orderly behavior and while methods for selecting reviewers included those with emphasis on others and those with emphasis on self. Furthermore, problems of reviewers in the Iranian Science and Research Journals include incentive and payment system, reviewer characteristics, and structural problems. The results of this study can be used on both education and research fields for various groups including policymakers in health-care system, researchers and stakeholders in the Iranian Health Scientific Journals. The following is suggested: other than professional and technical abilities, professional ethic, responsibility, and orderly behavior of reviewers should also be taken into consideration when selecting reviewers to ensure a proper peer review process without any lack of commitments or bias. Journals should use their own capabilities in knowing and selecting experts as well as opinions of others and searching in relevant databases. Creating motivation for a successful reviewing process requires proper, sustainable and logical payment and incentive system similar to any other service or task. Furthermore, policymaking, rules, regulations, planning, and implementation in this regard should be goal oriented and aligned with each other to achieve proper results.
We would like to thank all experts' editors in chief, journal managers, editorial boards, authors' council and members of national journal commission who answered out requests and shared their experiences and knowledge during this study.
Financial support and sponsorship
This work was supported by Health Information Technology Research Center, Isfahan University of Medical Sciences, Isfahan, Iran, with Ethical code number: Ir.mui.research.rec. 1397.352.
Conflicts of interest
There are no conflicts of interest.
| References|| |
Manchikanti L, Kaye AD, Boswell MV, Hirsch JA. Medical journal peer review: Process and bias. Pain Physician 2015;18:E1-E14.
Shahabian M. Principles of Management and Editorial of Scientific Journals. Mashhad: Abanbartar; 2015.
Olia MB, Shakiba M. How to judge research papers in a scientific and systematic way. J Shahid Sadoughi Univ Med Sci 2008;16:3-8.
Ayyoubi Ardakan M, Mirza'i A. Judges and ethics of arbitration in Iranian scientific journals. J Ethics Sci Technol 2010;5:36-47.
Fatahi R. Referees to scientific articles: Approaches, Criteria and ChallengesPresented at the Workshop of the Regional Center for Information Science and Technology and the Library and Information Society of Iran – Fars Branch. Shiraz, Iran; 26 December, 2015.
Vettore MV. The peer review process in health journals. Cad Saude Publica 2009;25:2306-7.
Innvaer S, Vist G, Trommald M, Oxman A. Health policy-makers' perceptions of their use of evidence: A systematic review. J Health Serv Res Policy 2002;7:239-44.
Ashrafi Razi H. Patient's rights to access and use health information. J Health Manage 2017;20(70):7-9.
Alipour Hafezi M. Problems and shortcomings of the arbitration process in scientific publications. Moon Book (general) 2012;177:38-43.
Motallebifard A, Navehebrahim A, Mohabbat H, Sadin AA. Diagnosis and scientific framework development of peer review: A qualitative approach. Journal of Dentistry 2013;6(22):73-83.
Lipworth WL, Kerridge IH, Carter SM, Little M. Journal peer review in context: A qualitative study of the social and subjective dimensions of manuscript review in biomedical publishing. Soc Sci Med 2011;72:1056-63.
Glujovsky D, Boggino C, Riestra B, Coscia A, Sueldo CE, Ciapponi A. Quality of reporting in infertility journals. Fertil Steril 2015;103:236-41.
Mirza'i A, Abubi Ardakan M, Gharakhani M, Sheikh Sha'ee F. Journal of sociology of Iran. Peer review in scientific journals: A case study of Iranian journal of sociology. Iran J Sociol 2006;7:147-79.
Castelo-Baz P, Leira-Feijoo Y, Seoane-Romero JM, Varela-Centelles P, Seoane J. Accessibility to editorial information in oral and maxillofacial surgery journals: The authors' point of view. J Craniomaxillofac Surg 2015;43:1078-81.
Primack RB, Maron M, Campos-Arceiz A. Who are our reviewers and how do they review? The profile and work of biological conservation reviewers. Biological Conservation 2017;(211 part A):177-82.
Alaedini F, Khoddam H, Kazemi Bajestani MR, Koshan F, Etemadi A, Keshtkar AA. Quality of published medical articles in approved medical journals by Islamic republic of Iran committee of medical journal (1983-2005). J Gorgan Univ Med Sci 2010;12:77-81.
Hariri N. Principles and Methods of Qualitative Research. Tehran: Islamic Azad University; 2006.
Abu Ardakan M, Mirzai A, Sheikh Sha'ee F. The process of reviewing articles in Iranian journals. J Sci Technol Res Inst Iran 2012;28:305-46.
Rahimi Nejad N. Understanding the Barriers and Problems in Managing and Publishing the Journals of the University of Tehran Based on the Views of their Managers and Providing Appropriate Solutions to it. Tehran: University of Tehran; 2015.
[Table 1], [Table 2], [Table 3]