Detection of Antithrombotic-Related Bleeding in Older Inpatients: Multicenter Retrospective Study Using Structured and Unstructured Electronic Health Record Data

Background: Bleeding complications are a major contributor to adverse drug events (ADEs) among older inpatients, particularly in those treated with antithrombotic agents. Timely and accurate detection of bleeding events is essential for improving drug safety surveillance and clinical risk management. Objective: To develop and validate automated algorithms for detecting major and clinically relevant non-major bleeding events from electronic medical records by combining structured data–based rule models and a natural language processing approach, and to evaluate their performance and generalisability against a manually reviewed gold standard and an external dataset. Methods: We conducted a multicentre retrospective study using routinely collected EMR data from three Swiss university hospitals. Patients aged ≥65 years who received at least one antithrombotic agent and were hospitalised between January 2015 and December 2016 were included. To detect major bleeding (MB) and clinically relevant non-major bleeding (CRNMB) events, rule-based algorithms were developed using structured data (ICD-10-GM codes, laboratory values, transfusion records, and antihaemorrhagic prescriptions), with variables and cut-off values defined according to adapted international ISTH definitions, and expert consensus. In parallel, a supervised Natural Language Processing (NLP) model was applied to discharge summaries from one hospital. A manual review of 754 EMRs served as the reference standard for internal validation, and algorithm performance of the structured-data algorithms (SDA), NLP, and their combination (SDA + NLP) was evaluated against this manually reviewed gold standard using standard performance metrics. External validation was performed on an independent dataset from the Lausanne University Hospital to assess model robustness and generalisability. Results: Among 36 039 inpatient stays, structured data algorithms (SDA) identified 8.3% MB and 15.0% CRNMB cases. ICD-10-GM codes alone detected 28.5% of MB and 31.5% of CRNMB cases, while laboratory data contributed most to event detection (67%). Integrating SDA with NLP improved detection, identifying 12.2% MB and 27.4% CRNMB cases at one hospital. The combined model achieved the best performance (sensitivity = 0.84, PPV = 0.51, F1-score = 0.64). External validation on CHUV 2021–2022 data (n=24,054 stays) confirmed algorithms reproducibility; the prevalence of MB decreased while CRNMB increased, reflecting evolving clinical practices and antithrombotic use patterns. Conclusions: Our integrated approach, combining structured data algorithm with NLP, enhances the detection of haemorrhagic events in older hospitalised patients treated with antithrombotic agents, suggesting its potential usefulness for drug safety monitoring and clinical risk management.