INTRODUCTION

Patient safety is an essential element of the quality of care [1]. The first step in monitoring and enhancing patient safety is to measure the level of patient safety [2]. Measuring the level of patient safety enables health professionals to determine the causes of problems related to patient safety, to identify patient-safety interventions to ameliorate the problem, and to evaluate the effectiveness and efficiency of such interventions. However, precisely measuring the level of patient safety can pose a challenge, due to difficulties in defining harm and developing a standardized methodology to measure harm [3].

Various methodologies, such as medical-record reviews, reporting systems, and patient interviews, have been used to measure adverse events as an indicator of patient safety [4,5]. Among these, medical-record reviews are widely used to identify adverse events, because they are a primary source of information on adverse events. In many countries, the incidence and nature of adverse events in hospitals have been analyzed using medical-record reviews [6-10]. Furthermore, changes in the incidence of adverse events and their preventability by year have been determined based on medical-record reviews at a national level [11].

Although medical-record reviews have been considered the gold standard for identifying adverse events, they are expensive and time-consuming [4,5]. The incompleteness of documentation in medical records has also been identified as a limitation of such reviews [12]. Furthermore, there have been criticisms of their reliability [13], and measures to increase reliability, such as well-organized training for reviewers, should be required prior to conducting them [14]. Because of these limitations, attempts have been made to use alternative methods of identifying adverse events and to combine 2 or more methods [4]. Administrative data, also called claims data, and the analysis thereof have emerged as an alternative for measuring adverse events [15,16]. Concerns over the validity of administrative data have been raised, but administrative data analyses are relatively inexpensive and readily available compared to other methods [15,16]. Accordingly, this strategy has been used to identify various types of adverse events, such as adverse drug events [17] and healthcare-associated infections [18]. In the case of the Republic of Korea (hereafter Korea), no study to date has investigated the scale of adverse events using representative samples from the whole nation. The Korean patient safety reporting and learning system, which was introduced by the Patient Safety Act, could be used to estimate the scale of patient safety incidents, but it has limitations for assessing the overall patient safety level due to under-reporting [19]. Although a few studies have estimated the incidence of adverse events in hospitals using medical-record reviews [14,20,21], the generalizability of these studies is limited. Considering the difficulty of conducting medical-record reviews on a large scale, using administrative data is an affordable alternative for estimating the scale of adverse events on the national level. Therefore, we identified adverse events using administrative data on the national level in Korea from 2002 to 2013, employing International Classification of Diseases, tenth revision (ICD-10) Y codes.

METHODS

Data

We used data from the National Health Insurance Service-National Sample Cohort (NHIS-NSC). The NHIS-NSC is a population-based cohort developed by the NHIS in Korea, and it provides information regarding the utilization of health insurance and health examinations for those covered by health insurance [22]. Because universal healthcare coverage was established in Korea in 1989, we expected to find a substantial volume of representative information about the utilization of health insurance from the NHIS-NSC. From a target population of 46 605 433 individuals in 2002, 1 025 340 participants (about 2.2%) were randomly selected and followed until 2013. The NHIS-NSC contains 4 databases: participants’ insurance eligibility, medical treatments, medical-care institutions, and general health examinations. We generally used the medical-treatment databases to extract information on ICD-10 Y codes from each participant in the NHIS-NSC. We considered an adverse event to have taken place if an ICD-10 Y code was present among the main diagnosis code or the secondary diagnosis codes.

Classification of International Classification of Diseases, tenth revision Y codes

ICD-10 Y codes were used to identify adverse events, referencing and adapting the Global Burden of Diseases study, in which they were utilized to detect the adverse effects of medical treatment [23]. Table 1 shows the types of adverse events assessed and their ICD-10 Y codes. We classified them into 6 types, adapting the classification used in a previous study [24]: those related to drugs, transfusions, and fluids; those related to vaccines and immunoglobulin; those related to surgery and procedures; those related to infections; those related to devices; and others. In the previous study [24], adverse events were divided into 6 types: those related to the diagnosis; those related to drugs, fluids, and blood; those related to patient care; those related to surgery and procedures; those related to infections; and others. However, adverse events related to the diagnosis or patient care were not included in this study, because no ICD-10 Y codes correspond to such events. Instead, adverse events related to vaccines and immunoglobulin and those related to devices were included in the classification, because these adverse events have specific ICD-10 Y codes. Furthermore, medication errors, such as failures in dosage during surgical or medical care (Y63), were included in the events related to drugs, transfusions, and fluids. The category of other adverse events included those that were ambiguous in their categorization, such as unspecified misadventures during surgical and medical care (Y69).

Ethics Approval

This study was approved by the institutional review board of Asan Medical Center (2016-093).

Statistical Analysis

We conducted descriptive analyses of the frequency of adverse events and their characteristics in terms of participants and medical-care institutions. Specifically, we examined trends in the number of adverse events by type from 2002 to 2013. We also examined the number of adverse events by type and according to age group, gender, type of care (inpatient vs. outpatient care), type of medical care institution (general hospital, hospital, clinic, and others), and type of medical specialty (general medical, surgical, obstetrics and gynecology, pediatrics, psychiatry, and others) in our 12-year study period. We used SAS version 9.3 (SAS Institute Inc., Cary, NC, USA) for all analyses.

RESULTS

Figure 1 and Supplemental Table 1 describe the trends in adverse events over our 12-year study period. A total of 20 817 adverse events were identified using ICD-10 Y codes, and the estimated total rate was 0.20%. The total number of adverse events increased by 131.3%, from 1366 in 2002 to 3159 in 2013, and the total rate increased by 103.9%, from 0.17% in 2002 to 0.35% in 2013. Among 20 817 adverse events, those related to drugs, transfusions, and fluids were most common (19 446, 93.4%), followed by those related to surgery and procedures (1209, 5.8%) and those related to vaccines and immunoglobulin (72, 0.3%). The number related to drugs, transfusions, and fluids increased by 115.4%, from 1283 in 2002 to 2763 in 2013, and the number related to surgery and procedures increased by 400.0%, from 75 in 2002 to 375 in 2013.

Table 2 shows the number of adverse events by type according to gender and age group. Among all adverse events, 8762 (42.1%) and 12 055 (57.9%) were identified in men and women, respectively. The 0-9 age group had the largest number of adverse events (4282, 20.6%), followed by the 60-69 age group (3345, 16.1%) and the 50-59 age group (3216, 15.4%). Adverse events related to vaccines and immunoglobulin tended to be detected more often among those in younger age groups than in older age groups. However, those related to surgery and procedures occurred more often in older age groups than in younger age groups.

Table 3 gives the number of adverse events by type according to the type of care, medical institution, and medical specialty. Of all adverse events, 20 078 (96.5%) occurred in the context of outpatient care, whereas 739 (3.5%) took place as part of inpatient care; 16 628 (79.9%) were detected in clinics, whereas 3979 (19.1%) were identified in hospitals and general hospitals. About half (10 110, 48.6%) were found in the records of general medicine. A total of 6865 adverse events (33.0%) took place under the care of surgeons. Among the 1209 adverse events related to surgery and procedures, 982 (81.2%) were found in the records of general medical practitioners, whereas 133 (11.0%) were found in the records of surgeons.

DISCUSSION

We identified adverse events based on ICD-10 Y codes in Korea from 2002 to 2013 using data from the NHIS-NSC. During our 12-year study period, a total of 20 817 adverse events were identified using ICD-10 Y codes. The average estimated total rate was 0.2%, and the estimated total rate by year showed a tendency to increase gradually. Since the sample size of the NHIS-NSC was about 2.2% of the target population, the total reported number of adverse events in Korea over 12 years is estimated to be about 9 462 273. The majority of identified adverse events were related to drugs, transfusions, and fluids, and occurred in the context of outpatient care. Approximately 80% were detected in clinics.

A strength of our current study was its comprehensiveness and ability to examine trends over time. This study included a considerable number of individuals covered by health insurance in Korea over 12 years, using data from the NHIS-NSC, which provides a substantial volume of representative administrative data reflecting the utilization of health insurance [22]. Following a previous study [23], we attempted to systematically identify all types of adverse events using ICD-10 Y codes. In this manner, the scale of events can be monitored on the national level relatively easily, compared to other methods of detecting adverse events, such as medical-record reviews.

However, it seems that the estimated total adverse event rate in the present analysis was significantly underestimated (0.2%), compared to the previous results of a systematic review of the incidence of adverse events based on medical-record reviews [6]. In that systematic review, the median overall incidence of adverse events in hospitals was 9.2% [6]. We assume that the difference in results can be attributed to the limitations of administrative data. Administrative data are inexpensive and readily accessible, but their completeness and validity may be limited, because their primary purpose is not for surveillance or research [25]. Furthermore, because of differences in coding practices between hospitals or clinics and the lack of robust risk adjustment, the comparability of the scale of adverse events between medical institutions may be limited [15,26]. Considering these limitations of administrative data, the adverse-event rate derived from other methodologies, including medical-record reviews, should be determined and compared to the results of this study. In particular, the validity of the ICD-10 Y codes for detecting adverse events should be confirmed based on an analysis using medical-record reviews. That is, it is necessary to check whether the appropriate ICD-10 Y codes are present in the records of patients whose adverse events are confirmed in medical-record reviews.

Furthermore, efforts should be made to improve the validity of the ICD-10 Y codes. In the present study, the majority of adverse events in inpatient care were seldom detected. Furthermore, those related to surgery and procedures were more often found in the records of general medical practitioners than by surgeons. Therefore, coding standards should be implemented for physicians in hospitals and surgeons, requiring them to assigning ICD-10 Y codes [27]. Inducements, such as financial incentives, to accurately assign ICD-10 Y codes would also be helpful to enhance the usability of ICD-10 Y codes for patient-safety surveillance. One option would be to include indicators of the coding validity of the ICD-10 Y codes in pay-for-performance programs.

The most-identified adverse events in our current study were related to drugs, transfusions, and fluids. A total of 19 446 adverse events in this group were identified from 2002 to 2013, and about 0.2% of hospital admissions or outpatient visits were associated with these factors. We suggest that ICD-10 Y codes are relatively more useful for identifying adverse events related to drugs, transfusions, and fluids than other types of events, such as those related to surgery and procedures or infections. In other studies, ICD-10 Y codes have been used to identify adverse drug events [26,28.29]. Wu et al. [26] reported that 0.9% of total hospital admissions were associated with adverse drug reactions and Waller et al. [28] determined that about 0.3% of hospital admissions were related to ICD-10 Y codes. Patel et al. [29] reported that 0.5% of total hospital episodes were associated with adverse drug reactions. Although the comparability of results between these studies is limited, due to the different study periods and countries involved, the estimated average rate of adverse events related to drugs, transfusions, and fluids in our study is lower than the rates found in other studies. It appears that differences in coding practices between countries are responsible for these discrepancies, and the problem of underreporting may be more prominent in Korea.

The main limitation of the present study is that the sensitivity and specificity of ICD-10 Y codes for adverse events were not evaluated. Although there is no consensus on how to identify adverse events reliably with ICD-10 Y codes, a review of the performance of ICD-10 Y codes and their refinements is required. As the validity of ICD-10 Y codes for adverse events may differ by the type of event [30], the sensitivity and specificity of ICD-10 Y codes for various event types should be determined in a future study. Furthermore, because the setting of clinical practice may affect coding practices, the validation of ICD-10 Y codes for adverse events must be performed in a variety of care settings, such as hospitals and clinics.

Another limitation was that adverse events related to the diagnosis or patient care could not be detected in this study. Because there are no ICD-10 Y codes for adverse events related to the diagnosis or patient care, it was methodologically impossible to identify them. It is particularly challenging to detect adverse events related to the diagnosis even through medical-record reviews [31], although patient interviews can be an alternative for detecting them [32].

In conclusion, our findings indicate that administrative data can be used to identify adverse events and monitor trends. Improving ICD-10 Y coding practices in Korea is necessary to precisely monitor the scale of adverse events. Because no study in Korea has yet been conducted of adverse events using representative samples, the use of administrative data may be an alternative for measuring adverse events.

ACKNOWLEDGEMENTS

Notes

CONFLICT OF INTEREST

The authors have no conflicts of interest associated with the material presented in this paper.

SUPPLEMENTAL MATERIAL

Figure. 1.

Trends in adverse events by type using ICD-10 Y codes from 2002 to 2013. ICD-10, International Classification of Diseases, tenth revision.

REFERENCES

• 1. Institute of Medicine (US) Committee on Quality of Health Care in America. Crossing the quality chasm: a new health system for the 21st century. Washington, DC: National Academies Press; 2001. p. 1-22
• 2. Kohn LT, Corrigan J, Donaldson MS; Institute of Medicine (US) Committee on Quality of Health Care in America. To err is human : building a safer health system. Washington, DC: National Academy Press; 2000. p. 1-16
• 3. Vincent C. Patient safety. 2010 [cited 2018 Jan 12]. Available from: http://file.zums.ac.ir/ebook/011-Patient%20Safety-Charles%20Vincent-1405192216-BMJ%20Books-2010-432-$71.pdfArticle
• 4. Murff HJ, Patel VL, Hripcsak G, Bates DW. Detecting adverse events for patient safety research: a review of current methodologies. J Biomed Inform 2003;36(1-2):131-143ArticlePubMed
• 5. Thomas EJ, Petersen LA. Measuring errors and adverse events in health care. J Gen Intern Med 2003;18(1):61-67ArticlePubMedPMC
• 6. de Vries EN, Ramrattan MA, Smorenburg SM, Gouma DJ, Boermeester MA. The incidence and nature of in-hospital adverse events: a systematic review. Qual Saf Health Care 2008;17(3):216-223ArticlePubMedPMC
• 7. Vlayen A, Verelst S, Bekkering GE, Schrooten W, Hellings J, Claes N. Incidence and preventability of adverse events requiring intensive care admission: a systematic review. J Eval Clin Pract 2012;18(2):485-497ArticlePubMed
• 8. Brennan TA, Leape LL, Laird NM, Hebert L, Localio AR, Lawthers AG, et al. Incidence of adverse events and negligence in hospitalized patients. Results of the Harvard Medical Practice Study I. N Engl J Med 1991;324(6):370-376ArticlePubMed
• 9. Leape LL, Brennan TA, Laird N, Lawthers AG, Localio AR, Barnes BA, et al. The nature of adverse events in hospitalized patients. Results of the Harvard Medical Practice Study II. N Engl J Med 1991;324(6):377-384ArticlePubMed
• 10. Thomas EJ, Studdert DM, Burstin HR, Orav EJ, Zeena T, Williams EJ, et al. Incidence and types of adverse events and negligent care in Utah and Colorado. Med Care 2000;38(3):261-271ArticlePubMed
• 11. Baines R, Langelaan M, de Bruijne M, Spreeuwenberg P, Wagner C. How effective are patient safety initiatives? A retrospective patient record review study of changes to patient safety over time. BMJ Qual Saf 2015;24(9):561-571ArticlePubMedPMC
• 12. Luck J, Peabody JW, Dresselhaus TR, Lee M, Glassman P. How well does chart abstraction measure quality? A prospective comparison of standardized patients with the medical record. Am J Med 2000;108(8):642-649ArticlePubMed
• 13. Forster AJ, Taljaard M, Bennett C, van Walraven C. Reliability of the peer-review process for adverse event rating. PLoS One 2012;7(7):e41239ArticlePubMedPMC
• 14. Ock M, Lee SI, Jo MW, Lee JY, Kim SH. Assessing reliability of medical record reviews for the detection of hospital adverse events. J Prev Med Public Health 2015;48(5):239-248ArticlePubMedPMCPDF
• 15. Zhan C, Miller MR. Administrative data based patient safety research: a critical review. Qual Saf Health Care 2003;12 Suppl 2: ii58-ii63ArticlePubMed
• 16. Kaafarani HM, Rosen AK. Using administrative data to identify surgical adverse events: an introduction to the Patient Safety Indicators. Am J Surg 2009;198(5 Suppl):S63-S68ArticlePubMed
• 17. Hohl CM, Karpov A, Reddekopp L, Doyle-Waters M, Stausberg J. ICD-10 codes used to identify adverse drug events in administrative data: a systematic review. J Am Med Inform Assoc 2014;21(3):547-557ArticlePubMed
• 18. van Mourik MS, van Duijn PJ, Moons KG, Bonten MJ, Lee GM. Accuracy of administrative data for surveillance of healthcare-associated infections: a systematic review. BMJ Open 2015;5(8):e008424ArticlePubMedPMC
• 19. Ock M, Kim JH, Lee SI. A legal framework for improving patient safety in Korea. Health Policy Manag 2015;25(3):174-184. (Korean)ArticlePDF
• 20. Hwang JI, Chin HJ, Chang YS. Characteristics associated with the occurrence of adverse events: a retrospective medical record review using the Global Trigger Tool in a fully digitalized tertiary teaching hospital in Korea. J Eval Clin Pract 2014;20(1):27-35ArticlePubMed
• 21. Hwang JI, Kim J, Park JW. Adverse events in Korean Traditional Medicine Hospitals: a retrospective medical record review. J Patient Saf 2015. doi: https://doi.org/10.1097/PTS.0000000000000190Article
• 22. Lee J, Lee JS, Park SH, Shin SA, Kim K. Cohort profile: the National Health Insurance Service-National Sample Cohort (NHIS-NSC), South Korea. Int J Epidemiol 2017;46(2):e15ArticlePubMedPDF
• 23. Lozano R, Naghavi M, Foreman K, Lim S, Shibuya K, Aboyans V, et al. Global and regional mortality from 235 causes of death for 20 age groups in 1990 and 2010: a systematic analysis for the Global Burden of Disease Study 2010. Lancet 2012;380(9859):2095-2128ArticlePubMedPMC
• 24. Ock MS, Lee SI, Kim Y, Lee JH, Lee JY, Jo MW, et al. Development of case review form for detecting adverse events. Qual Improv Health Care 2015;21(1):66-76. (Korean)ArticlePDF
• 25. Verelst S, Jacques J, Van den Heede K, Gillet P, Kolh P, Vleugels A, et al. Validation of hospital administrative dataset for adverse event screening. Qual Saf Health Care 2010;19(5):e25Article
• 26. Wu TY, Jen MH, Bottle A, Molokhia M, Aylin P, Bell D, et al. Ten-year trends in hospital admissions for adverse drug reactions in England 1999-2009. J R Soc Med 2010;103(6):239-250ArticlePubMedPMC
• 27. Jordan K, Porcheret M, Croft P. Quality of morbidity coding in general practice computerized medical records: a systematic review. Fam Pract 2004;21(4):396-412ArticlePubMedPDF
• 28. Waller P, Shaw M, Ho D, Shakir S, Ebrahim S. Hospital admissions for ‘drug-induced’ disorders in England: a study using the Hospital Episodes Statistics (HES) database. Br J Clin Pharmacol 2005;59(2):213-219ArticlePubMedPMC
• 29. Patel H, Bell D, Molokhia M, Srishanmuganathan J, Patel M, Car J, et al. Trends in hospital admissions for adverse drug reactions in England: analysis of national hospital episode statistics 1998-2005. BMC Clin Pharmacol 2007;7: 9ArticlePubMedPMCPDF
• 30. Ackroyd-Stolarz S, Bowles SK, Giffin L. Validating administrative data for the detection of adverse events in older hospitalized patients. Drug Healthc Patient Saf 2014;6: 101-108ArticlePubMedPMC
• 31. Graber ML. The incidence of diagnostic error in medicine. BMJ Qual Saf 2013;22 Suppl 2: ii21-ii27ArticlePubMed
• 32. Ock M. Evaluating the feasibility of introducing open disclosure of patient safety incidents [dissertation]. Ulsan: University of Ulsan; 2016. (Korean)

Figure & Data

References

Citations

Citations to this article as recorded by  

• Accuracy assessment of patient safety incident (PSI) codes and present-on-admission (POA) indicators: a cross-sectional analysis using the Patient Safety Incidents Inquiry (PSII) in Korea
  Jeehee Pyo, Eun Young Choi, Seung Gyeong Jang, Won Lee, Minsu Ock
  BMC Health Services Research.2024;[Epub]     CrossRef
• Development of the Korean Patient Safety Incidents Code Classification System
  Eun Young Choi, Jeehee Pyo, Young-Kwon Park, Minsu Ock, Sukyeong Kim
  Journal of Patient Safety.2023; 19(1): 8.     CrossRef
• Use of a hospital administrative database to identify and characterize community-acquired, hospital-acquired and drug-induced acute kidney injury
  Amayelle Rey, Valérie Gras-Champel, Thibaut Balcaen, Gabriel Choukroun, Kamel Masmoudi, Sophie Liabeuf
  Journal of Nephrology.2022; 35(3): 955.     CrossRef
• Evaluation of Factors Associated with Adverse Drug Events in South Korea Using a Population-Based Database
  Eunkyeong Choi, Siin Kim, Hae Sun Suh
  Journal of Clinical Medicine.2022; 11(21): 6248.     CrossRef
• Feasibility of Capturing Adverse Events From Insurance Claims Data Using International Classification of Diseases, Tenth Revision, Codes Coupled to Present on Admission Indicators
  Juyoung Kim, Eun Young Choi, Won Lee, Hae Mi Oh, Jeehee Pyo, Minsu Ock, So Yoon Kim, Sang-il Lee
  Journal of Patient Safety.2022; 18(5): 404.     CrossRef
• The Korea National Patient Safety Incidents Inquiry Survey: Characteristics of Adverse Events Identified Through Medical Records Review in Regional Public Hospitals
  Min Ji Kim, Hee Jung Seo, Hong Mo Koo, Minsu Ock, Jee-In Hwang, Sang-Il Lee
  Journal of Patient Safety.2022; 18(5): 382.     CrossRef
• Use of ICD‐10‐CM T codes in hospital claims data to identify adverse drug events in Taiwan
  Ya‐Fang Cheng, Chi‐Yuan Cheng, Szu‐Hsuan Wang, Yu‐Ting Lin, Tzu‐Cheng Tsai
  Journal of Clinical Pharmacy and Therapeutics.2021; 46(2): 476.     CrossRef
• Perceptions of Hospital Health Information Managers Regarding Present on Admission Indicators in Korea: A Qualitative Study
  Jee-Hee Pyo, Eun-Young Choi, Hae-Mi Oh, Won Lee, Ju-Young Kim, Min-Su Ock, So-Yoon Kim, Sang-Il Lee
  Quality Improvement in Health Care.2020; 26(1): 23.     CrossRef
• Variation between hospitals and reviewers in detection of adverse events identified through medical record review in Korea
  Sukyeong Kim, Ho Gyun Shin, A E Jeong Jo, Ari Min, Minsu Ock, Jee-In Hwang, Youngjin Jeong, Moon Sung Park, Jong Bouk Lee, Tae I K Chang, Eunhyang Song, Heungseon Kim, Sang-Il Lee
  International Journal for Quality in Health Care.2020; 32(8): 495.     CrossRef