ACM Transactions on Intelligent Systems and Technology (TIST)
Doi:
https://doi.org/10.1145/3757924
1 Abstract 1.1 Objectives Biases inherent in electronic health records (EHRs), which are often used as a data source to train medical AI models, may significantly exacerbate health inequities and challenge the adoption of ethical and responsible AI in healthcare. Biases arise from multiple sources, some of which are not as documented in the literature (e.g., bias in medical devices measurement). Biases are encoded in how the data has been collected and labeled, by implicit and unconscious biases of clinicians, or by the tools used for data processing. These biases and their encoding in healthcare records can potentially undermine the reliability of such data and bias clinical judgments and medical outcomes. Moreover, when healthcare records are used to build data-driven solutions, the biases can be further exacerbated, resulting in systems that can perpetuate biases and induce healthcare disparities. This literature scoping review aims to categorize the main sources of biases inherent in EHRs. 1.2 Methods We queried PubMed and Web of Science on January 19th, 2023, for peer-reviewed sources in English, published between 2016 and 2023, using the PRISMA approach to stepwise scoping of the literature. To select the papers that empirically analyze bias in EHR, from the initial yield of 430 papers, 27 duplicates were removed, and 403 studies were screened for eligibility. 196 articles were removed after the title and abstract screening, and 96 articles were excluded after the full-text review resulting in a final selection of 116 articles. 1.3 Results Existing studies often focus on individual biases in EHR data, but a comprehensive review categorizing these biases is largely absent. To address this gap, we propose a systematic taxonomy to classify and better understand the multiplicity of biases in EHR data. Our framework identifies six primary sources: a) bias from past clinical trials ; b) data-related biases , such as missing or incomplete information; human-related biases , including c) implicit clinician bias, d) referral and admission bias, and e) diagnosis or risk disparities bias; and f) biases in devices and algorithms. This taxonomy, illustrated in Table 1, provides a valuable tool for systematically evaluating and addressing these issues. 1.4 Conclusions Machine learning and data-driven solutions can potentially transform healthcare delivery, but not without limitations. The core inputs in the systems (data and human factors) currently contain several sources of bias that are poorly documented and analyzed for remedies. The current evidence heavily focuses on data-related biases, while other sources are less often analyzed or anecdotal. However, these different sources of bias can compound each other, leading to a cumulative effect. Therefore, to understand the issues holistically we need to explore these diverse sources of bias. While racial biases in EHR have been often documented, other sources of biases have been less frequently investigated and documented (e.g. gender-related biases, sexual orientation discrimination, socially induced biases, and implicit, often unconscious, human-related cognitive biases). Moreover, some existing studies lack concrete evidence of the effects of the bias, but rather illustrate the different prevalence of disease across groups, which does not per se prove the effect of the bias. Our review shows that data-, human- and machine biases are prevalent in healthcare and can significantly affect treatment decisions and outcomes and amplify healthcare disparities. Understanding how diverse biases affect AI systems and recommendations is critical. We recommend that researchers and medical personnel develop safeguards and adopt data-driven solutions with a “bias-in-mind” approach. More empirical evidence is needed to tease out the effects of different sources of bias on health outcomes.
