GRAPHICAL ABSTRACT

INTRODUCTION

Patient safety is an essential aspect of healthcare services in various countries, including Indonesia. However, the implementation of comprehensive patient safety measures in Indonesian healthcare facilities remains challenging. This is primarily due to the lack of effective collaboration among health workers, managers, and support staff [1,2]. Previous studies have shown that hypoglycemia is a severe condition that is recognized as a leading cause of death. It is also associated with higher mortality rates, increased disease morbidity, and reduced quality of life, all of which significantly impact healthcare costs borne by both patients and the government [3,4].

Several studies have indicated that the additional costs associated with hypoglycemia in patients with diabetes mellitus (DM) range from US$1353 to US$2285. In Indonesia, the government incurred total costs of US$23 million in 2016 for treating these cases [3,5–7]. Furthermore, the incidence of hypoglycemia among ambulatory DM patients resembles the iceberg phenomenon, with many cases remaining undetected and unreported within the health system. This issue may stem from patients’ limited knowledge and awareness of the condition. Approximately 90% of DM patients have reported experiencing mild to severe hypoglycemia while managing their condition with antidiabetic agents at home. A cohort study in Indonesia found that 99.4% of patients with type 2 diabetes mellitus (T2DM) experienced at least 1 hypoglycemic event over a 4-week period (prospective study), with a hypoglycemia incidence rate of 25.7 events per patient-year [8–10].

In line with these findings, clinical pharmacists play a crucial role in ensuring patient safety. These professionals are integral to drug services and are key in preventing adverse drug events [7,11]. Despite their significant role, clinical pharmacists still require guidance to effectively review medications. Currently, in Indonesia, there is no specific tool designed to assess the risk of hospitalization due to hypoglycemia caused by antihyperglycemic medications, highlighting a critical gap that needs to be addressed. Additionally, there is a local need for a simple and efficient method to evaluate the risk profile of T2DM patients experiencing ambulatory hypoglycemia, which can be conducted by clinical pharmacists. Consequently, this study aimed to develop an instrument for clinical pharmacists to assess the risk of medication-related hypoglycemia in ambulatory T2DM patients. The instrument, named “Medication-related Hypoglycemia Risk Score Assessment Tools (HYPOGLYRISK),” is expected to significantly enhance the quality of care and patient safety.

METHODS

Study Design and Setting

The study employed an observational design that unfolded in several stages, beginning with the development of the instrument, followed by content, criterion, and construct validation. The initial stage of instrument development involved a review of the literature to explore similar studies published in academic articles. This was complemented by focus group discussions (FGDs) tailored to the local context and needs. The drafting stage of the instrument included both piloting and finalization processes. The draft instrument then underwent a content validity stage, which involved a panel of experts. The validation process continued with criterion and construct validity stages, which were designed to determine the scores for each item and to evaluate the correlation between the scoring model and disease events using a cross-sectional approach. The instrument was named HYPOGLYRISK, a medication-related risk score assessment tool. The operational definition of hypoglycemia in this study was identified as a severe condition necessitating medical intervention, specifically hospitalization.

The current study is part of a larger project entitled “Medication Safety,” conducted across 3 government hospitals in Bali, Indonesia. This multicenter study was conducted prospectively from August 2023 to June 2024. To validate information gathered through direct observation, such as histories of severe hypoglycemia, medication use, complications or comorbidities, and detailed demographic data, some retrospective data were necessary. This retrospective data were sourced from patient medical records. The specific locations for conducting this study included endocrine and internal medicine polyclinics, outpatient pharmacy installation units, and hospital medical records units. The results were presented in accordance with the Strengthening the Reporting of Observational Studies in Epidemiology guidelines for combined observational design [12].

Participants, Study Size, and Variables

Participants in this study included experts such as endocrinologists, nurses, clinical pharmacists, community pharmacists, and nutritionists participating in FGDs. Additional participants were clinical pharmacists, who validated content, and individuals with ambulatory T2DM, who validated criteria and constructs. Experts shared their insights on the incidence of hypoglycemia among outpatient T2DM patients, drawing on their specific areas of expertise. These insights were then transcribed and synthesized into items for the instrument. The group of clinical pharmacists involved in assessing content validity consisted of 5 panelists, each with over 5 years of experience serving ambulatory T2DM patients. Patients with T2DM were recruited from 3 government hospitals in Bali, Indonesia. The study included T2DM patients of all age groups who were undergoing outpatient therapy with either oral or parenteral antidiabetic agents. Exclusion criteria included patients unwilling to participate in the study, those not routinely monitored every month for 1 year, and cases with data inconsistencies between direct observations and medical records.

In the FGD and content validation stage, validation required the involvement of at least 5 experts and clinical pharmacists. Meanwhile, a minimum of 200 T2DM patients was necessary for the cross-sectional design to adequately represent the population. Data collection was carried out through direct observation at all stages. The study included 8 observers who monitored the participants. The variables examined were the incidence of hospitalized hypoglycemia in T2DM patients and the behavior of outpatient T2DM patients. Five dimensions were considered: starting medication, taking medication, adding to the medication regimen, reviewing medication use, and stopping medication use. The identification of a history of hypoglycemia hospitalization was based on direct patient recognition and confirmed through medical records, where patients were documented with the International Statistical Classification of Diseases and Related Health Problems 10th revision (ICD-10) code E16.2 and had received parenteral fluid therapy with dextrose 10% (ICD-10 code: D10). The instrument’s question items from the content validation results were utilized to assess the medication conditions of ambulatory T2DM patients, both with and without a history of hospitalization for hypoglycemia. At the cross-sectional stage, the dependent variable was identified as the presence or absence of a history of hospitalization for hypoglycemia. Meanwhile, the independent variable was a question item in the instrument. This setup allowed clinical pharmacists to map out question items necessary for analyzing patients’ medication use risks and ensured the identification of educational forms to mitigate the risk of medication-related hypoglycemia at home.

Potential Sources of Bias

The development of items in the instrument focused on the medication safety of antidiabetic drugs, tailored to the local context and needs. These items were not suitable for use in other countries without further development and revalidation. Groups with and without hypoglycemia were identified using direct observation data, which was subsequently cross-referenced with medical records to determine if hospitalization due to hypoglycemia had occurred. Patients experienced mild to moderate symptoms at home that went unreported, potentially introducing bias.

Statistical Analysis

The data presented included item development and content validation, participant recruitment flow diagrams, baseline characteristics, criterion validation, and construct validation, which included statistical analysis data and correlation diagrams. Additionally, the variables measured included the clinical pharmacist’s assessment of the developed items, the odds ratio (OR) value for each significant item, and the correlation between the scoring model and the number of items developed that were at risk in T2DM patients.

Content validity ratio (CVR) analysis was employed as the method for content validation, setting a cutoff value of 0.99 with the involvement of 5 panelists. The baseline characteristics of the patients were categorized into 2 groups: those with a history of hypoglycemia and those without. These groups were compared using chi-square analysis, with a p-value >0.05 indicating no significant difference. Each question item in the instrument was assigned a score weight. The establishment of this scoring system was defined as criterion validation. The analytical method involved calculating the OR for each item using multivariate analysis. Initially, bivariate analysis using chi-square was conducted. Items yielding a p-value <0.25 were then eligible for further evaluation through multivariate binary logistic analysis. Results showing a p-value <0.05 were considered criterion-valid, and the OR value was used to represent the magnitude of the item’s score. Furthermore, the scoring model’s construction was tested by examining the relationship between the scoring system and the number of developed items at risk in both groups of T2DM patients, using Spearman rank test analysis. The scoring system was deemed a valid construct when the p-value was <0.05 and r²≥0.70 [13].

Ethics Statement

This study was conducted from January 2022 to June 2024 as part of a broader research initiative. The Ethics Commission of the Faculty of Medicine at Udayana University, Bali, granted approval for this study, assigning it the ethical clearance number 1165/UN.14.2.2.VII.14/LT/2024. Additionally, ethical clearance was secured from the participating multicenter hospitals, under the numbers 052/EA/KEPK.RSBM.DISKES/2024, B/475/UN14.6/PT.01.04/2024, and 019/EC/KEPK-RSB/V/2023. Participants provided their consent through an approved consent form, which was translated into the local language and explained in real-time. They were fully informed about the study’s general overview, objectives, potential risks, and benefits. Confidentiality was rigorously maintained at all stages of the research. The study adhered to the principles outlined in the Declaration of Helsinki [14].

RESULTS

The experts agreed that 5 dimensions that influence medication safety: starting, taking, adding, reviewing, and stopping medications. The question items were designed to focus on the most critical components, driven by a local need to develop a rapid screening tool to assess the risk of medication-related hypoglycemia. The developed items are displayed in Table 1, which lists 10 question items compiled from the FGDs. Of these, 8 questions require a “no” response to indicate a risk to patients (items 1, 2, 3, 4, 7, 8, 9, 10), while 2 questions require a “yes” response to indicate a risk (items 5 and 6). These items were subsequently content validated by clinical pharmacists, considering that patients were the end users of the instrument. The content analysis results indicated that all question items achieved content validity, with a CVR value greater than 0.99, as shown in Table 1.

The test items were further evaluated for criteria and construction with a focus on ambulatory T2DM patients. These patients were divided into 2 groups: those with a history of hypoglycemia hospitalization and those without. Out of 400 screened ambulatory T2DM patients, 282 met the inclusion criteria. Generally, the baseline characteristics of both groups were similar, with the exception of the duration of diabetes, glucose control, comorbid conditions (neuropathy and chronic kidney disease), and the use of antidiabetic drugs (insulin and sulfonylurea), as shown in Table 2. These results were often linked to risk factors in T2DM patients, which are considered strong predictors of hypoglycemia.

Bivariate analysis revealed that all question items had a p-value of less than 0.25, and thus, all were retained for multivariate analysis. The multivariate analysis results indicated that 8 question items had a p-value of less than 0.05, as shown in Table 3. However, items 5 and 8 were deemed invalid, with p-values greater than 0.05. This finding was attributed to the interaction of questions 5 and 8 with other items that exhibited stronger associations. Consequently, since these questions were represented by other items, items 5 and 8 were excluded from the instrument.

The ORs for 8 items ranged from 2 to 3, indicating that a response indicating a problem in any of these items on the instrument increased the risk of developing hypoglycemia by 2 times to 3 times. Multivariate analysis confirmed the validity of these 8 items based on criterion validity. The total number of ORs calculated was 18. In the next phase, these OR values were utilized to develop a scoring model for the instrument. Using the median value, scores from 0 to 6 were categorized as low risk, scores from 7 to 12 as moderate risk, and scores from 13 to 18 as high risk for developing hypoglycemia.

The scoring model used in this study was subsequently validated. Moreover, the hypoglycemia risk score, which comprises 8 questions, showed a correlation with actual hypoglycemia events, as shown in Figure 1. Individuals with scores ranging from 0 to 6 exhibited fewer risk-related question items compared to those with scores in the ranges of 7 to 12 and 13 to 18. The Spearman rank test revealed significant differences (p<0.05) and demonstrated a strong positive correlation, with an r² value of +0.988, based on the scoring model. These results confirm that the scoring instrument has passed construct validity.

DISCUSSION

The FGDs with experts comprehensively discussed predictors that could influence the incidence of hospitalization for hypoglycemia in ambulatory T2DM patients. Various categories were identified, including medication-related issues, comorbidities or complications, lifestyle, risk factors, self-care, and social support. This study emphasized medication safety, structured around 5 dimensions: start, take, add, review, and stop medication. This approach aligns with the World Health Organization’s 5 Moments of Medication Safety [15,16]. The question item deemed most significant represented the curriculum conditions concerning the medication aspect of ambulatory T2DM patients. The number of synthesized questions was capped at 10 to meet local needs for a concise screening tool. However, items 5 and 8 were excluded. Item 5 pertained to the dimension of taking medication, and item 8 to reviewing medication. These items were considered invalid due to their significant interactions with other items. The remaining 10 questions were effectively condensed into 8, maintaining the integrity of the content-valid items. This reduction was endorsed by the experts participating in the FGDs, noting that the omitted items would be covered by items number 3, 4, and 6. These adjustments will be incorporated into the HYPOGLYRISK manual book.

Before starting to use an antidiabetic agent, outpatients must be informed about their medication, including at least the name of the drug, its indication, dosage, and frequency of use [17,18]. Additionally, patients must be educated about the risk of hypoglycemia associated with the medication. They should understand the definition of hypoglycemia and recognize its early signs and symptoms [9,19,20]. When patients are not informed about the implications of starting a medication, they could be 4 times more likely to suffer from this condition at home. In the “taking medication” dimension, patients or their families must know how to properly use the medication, including the correct amount, timing, and frequency of use, maintaining nutritional intake while using it, storing it in the appropriate place, and recognizing signs of damage to the pharmaceutical preparation [7,11,18]. Patients, their families, and caregivers must also understand first aid measures to take when the first signs and symptoms occur to prevent fatalities. Ignoring 2 aspects in this dimension could increase the risk of developing hypoglycemia at home by fourfold. In the “adding medication” dimension, patients must be aware of medications that could act as agonists with antihyperglycemic drugs, such as herbal and complementary medicines. However, comprehensive counseling on the use of herbal medicines and supportive therapy is not typically possible. Patients are advised not to add alternative treatment regimens other than those prescribed at health service facilities [5,6,9]. When this advice is ignored, outpatients could be 3 times more at risk of experiencing hypoglycemia. The “reviewing medication” dimension is generally considered challenging for patients. In this dimension, patients must adhere strictly to their treatment regimen. They are advised to seek support from their support system. Medications taken home must be comprehensively recorded, and technical adjustments to the dosage of antihyperglycemic drugs relative to food intake, specifically for insulin and sulfonylurea drugs, must be carefully documented in their prescription records [1,3,4]. When these steps are followed, patients are twice as safe from the risks associated with outpatient care. In the “stopping medication” dimension, it is crucial to assess patients’ knowledge and their support system’s ability to temporarily discontinue antidiabetic drugs when alarm symptoms indicative of hypoglycemia appear. Patients and their support systems are strongly encouraged to manage these risks proactively and to contact or visit the nearest emergency department relative to their daily activities location [10,15,17]. Failure of patients or their support system to manage these dimensions could result in a fivefold greater risk of severe hypoglycemia requiring hospitalization.

HYPOGLYRISK was developed to facilitate the assessment of hospitalization risk due to hypoglycemia in ambulatory T2DM patients who were discharged on antidiabetic medication. The ratio of patients to health workers in developing countries, such as Indonesia, is far from ideal [2,10]. Consequently, not all patients receive the necessary communication, information, and education, necessitating prioritization. This tool enables clinical pharmacists to effectively educate targeted patients. Ultimately, HYPOGLYRISK could become a widespread tool among pharmacists in healthcare facilities, aiming to protect DM patients from severe risks associated with home-based antidiabetic drug therapy.

This study had several limitations, including the focus of the instrument’s items on medication safety for antidiabetic drugs, tailored to the local context and needs. Consequently, these items may not be applicable in other countries. Additionally, the use of this instrument was confined to clinical pharmacists managing medication-related hypoglycemia during drug prescription services in outpatient settings. In the future, the instrument could be adapted based on local needs and epidemiological data, reflecting its dynamic nature.

Notes

Conflict of Interest

The authors have no conflicts of interest associated with the material presented in this paper.

Funding

This study was funded by the Indonesia Center for Education Financial Services (Puslapdik RI), Center for Higher Education Funding (BPPT), and Indonesia Endowment Funds for Education (LPDP) No. 00438/BPPT/BPI.06/9/2023.

Author Contributions

Conceptualization: Ikawati Z, Jaya MKA. Data curation: Jaya MKA, Yasin NM. Formal analysis: Rahmawati F, Jaya MKA. Funding acquisition: Jaya MKA. Methodology: Ikawati Z, Jaya MKA. Project administration: Rahmawati F, Yasin NM. Visualization: Jaya MKA, Rahmawati F. Writing – original draft: Jaya MKA. Writing – review & editing: Ikawati Z, Rahmawati F, Yasin NM.

Acknowledgements

Figure 1

Construct validity of Medication-related Hypoglycemia Risk Score Assessment Tools (HYPOGLYRISK).

REFERENCES

• 1. Burns A. Medication therapy management in pharmacy practice: core elements of an MTM service model (version 2.0). J Am Pharm Assoc (2003) 2008;48(3):341-353. https://doi.org/10.1331/JAPhA.2008.08514ArticlePubMed
• 2. Intergovernmental Panel on Climate Change, 2013: summary for policymakers. In: Stocker TF, Qin D, Plattner GK, Tignor MM, Allen SK, Boschung J, et al, editors. Climate change 2013: the physical science basis contribution of working group I to the fifth assessment report of the Intergovernmental Panel on Climate Change. Cambridge: Cambridge University Press; 2014. p. 3-31
• 3. Au NH, Ratzki-Leewing A, Zou G, Ryan BL, Webster-Bogaert S, Reichert SM, et al. Real-world incidence and risk factors for daytime and nocturnal non-severe hypoglycemia in adults with type 2 diabetes mellitus on insulin and/or secretagogues (InHypo-DM Study, Canada). Can J Diabetes 2022;46(2):196-203. https://doi.org/10.1016/j.jcjd.2021.09.004ArticlePubMed
• 4. Disclosures: standards of care in diabetes-2024. Diabetes Care 2024;47(Suppl 1):S309-S313. https://doi.org/10.2337/dc24-SDISArticlePubMedPMC
• 5. Quilliam BJ, Simeone JC, Ozbay AB. Risk factors for hypoglycemia-related hospitalization in patients with type 2 diabetes: a nested case-control study. Clin Ther 2011;33(11):1781-1791. https://doi.org/10.1016/j.clinthera.2011.09.020ArticlePubMed
• 6. Miller CD, Phillips LS, Ziemer DC, Gallina DL, Cook CB, El-Kebbi IM. Hypoglycemia in patients with type 2 diabetes mellitus. Arch Intern Med 2001;161(13):1653-1659. https://doi.org/10.1001/archinte.161.13.1653ArticlePubMed
• 7. Cowart K, Patel V, Bianco J, Martinez A, Castro J. Pharmacist-physician collaborative practice to improve diabetes care at Tampa General Medical Group. Clin Diabetes 2022;40(2):240-244. https://doi.org/10.2337/cd21-0080ArticlePubMedPMC
• 8. Iloh GU, Amadi AN. Epidemiology of hypoglycemia among ambulatory Type 2 diabetic patients in a primary care clinic of a tertiary hospital in Southeastern Nigeria. J Health Res Rev 2018;5(2):57-65. https://doi.org/10.4103/jhrr.jhrr_37_17Article
• 9. Lamounier RN, Geloneze B, Leite SO, Montenegro R Jr, Zajdenverg L, Fernandes M, et al. Hypoglycemia incidence and awareness among insulin-treated patients with diabetes: the HAT study in Brazil. Diabetol Metab Syndr 2018;10: 83. https://doi.org/10.1186/s13098-018-0379-5ArticlePubMedPMC
• 10. Rudijanto A, Saraswati MR, Yunir E, Kumala P, Puteri HH, Mandang VV. Indonesia cohort of IO HAT study to evaluate diabetes management, control, and complications in retrospective and prospective periods among insulin-treated patients with type 1 and type 2 diabetes. Acta Med Indones 2018;50(1):26-37PubMed
• 11. Alim U, Austin-Bishop N, Cummings G. Pharmacists in a complex chronic disease management clinic. Can J Hosp Pharm 2016;69(6):480-482. https://doi.org/10.4212/cjhp.v69i6.1612ArticlePubMedPMC
• 12. von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP, et al. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Ann Intern Med 2007;147(8):573-577. https://doi.org/10.7326/0003-4819-147-8-200710160-00010ArticlePubMed
• 13. Akoglu H. User’s guide to correlation coefficients. Turk J Emerg Med 2018;18(3):91-93. https://doi.org/10.1016/j.tjem.2018.08.001ArticlePubMedPMC
• 14. World Medical Association. World Medical Association Declaration of Helsinki. Ethical principles for medical research involving human subjects. Bull World Health Organ 2001;79(4):373-374PubMedPMC
• 15. Australian Commission on Safety and Quality in Health Care. Medication without harm – WHO globalpatient safety challenge. Discussion paper for public consultation; 2019 [cited 2024 May 22]. Available from: https://www.safetyandquality.gov.au/sites/default/files/2019-06/medication-without-harm-who-global-patient-safety-challenge-discussion-paper-for-public-consultation-pdf-1.75mb.pdf
• 16. World Health Organization. 5 moments for medication safety; 2019 [cited 2024 Jun 5]. Available from: https://www.who.int/publications/i/item/WHO-HIS-SDS-2019.4
• 17. Al Mazroui NR, Kamal MM, Ghabash NM, Yacout TA, Kole PL, McElnay JC. Influence of pharmaceutical care on health outcomes in patients with type 2 diabetes mellitus. Br J Clin Pharmacol 2009;67(5):547-557. https://doi.org/10.1111/j.1365-2125.2009.03391.xArticlePubMedPMC
• 18. Akirov A, Amitai O, Masri-Iraqi H, Diker-Cohen T, Shochat T, Eizenberg Y, et al. Predictors of hypoglycemia in hospitalized patients with diabetes mellitus. Intern Emerg Med 2018;13(3):343-350. https://doi.org/10.1007/s11739-018-1787-0ArticlePubMed
• 19. Alkhatatbeh MJ, Abdalqader NA, Alqudah MA. Impaired awareness of hypoglycaemia in insulin-treated type 2 diabetes mellitus. Curr Diabetes Rev 2019;15(5):407-413. https://doi.org/10.2174/1573399814666180806144937ArticlePubMed
• 20. Workneh Fego M, Tahir Yasin J, Mamo Aga G. Knowledge, attitude and practice towards insulin-self administration among diabetic patients attending Bedele Hospital, Southwest Ethiopia, 2019/2020. Diabetes Metabc Syndr Obes 2021;14: 1919-1925. https://doi.org/10.2147/DMSO.S279186Article

Figure & Data

References

Citations

Citations to this article as recorded by  

• Determination, Validation, and Development of Prediction Scores Related to Hypoglycemia Risk Factors Among Ambulatory Type 2 Diabetes Mellitus Patients in Bali, Indonesia
  Made Krisna Adi Jaya, Fita Rahmawati, Nanang Munif Yasin, Zullies Ikawati
  Current Diabetes Reviews.2026;[Epub]     CrossRef
• Pharmacist-led interventions for optimizing type 2 diabetes mellitus management and medication safety in Southeast Asian countries: A systematic review and meta-analysis
  Made Krisna Adi Jaya, Fita Rahmawati, Nanang Munif Yasin, Zullies Ikawati
  International Journal of Diabetes in Developing Countries.2026; 46(3): 664.     CrossRef