BACKGROUND: Novel algorithms were developed to identify International Society on Thrombosis and Haemostasis (ISTH)-defined major and clinically relevant nonmajor (CRNM) bleeding events using electronic health record (EHR) data. OBJECTIVES: We applied and validated these EHR-based algorithms in a large US integrated health system. METHODS: We conducted a retrospective cohort and chart review study of adults with atrial fibrillation and ≥1 oral anticoagulant (OAC) prescription between 2013 and 2021. EHR-based algorithms were applied to identify ISTH-defined major and CRNM bleeding from the first OAC fill until disenrollment, death, or June 2022, whichever occurred first. Through conditional sampling, 433 patients were selected for clinician chart review. Sensitivity, specificity, positive predictive values, and negative predictive values were calculated. RESULTS: Among 62,106 adults (mean age, 72.6 years; 57.1% male), the ISTH major and CRNM bleeding event rates per the algorithms were 3.54 and 19.67 per 100 person-years, respectively, during a median follow-up of 4.3 years. Sensitivity and specificity were 0.91 (95% CI, 0.86, 0.94) and 0.92 (95% CI, 0.88, 0.95) for major bleeding, respectively, and 0.66 (95% CI, 0.54, 0.76) and 0.86 (95% CI, 0.82, 0.90) for CRNM bleeding, respectively. The positive and negative predictive values were 0.91 (95% CI, 0.87, 0.94) and 0.92 (95% CI, 0.89, 0.95) for major bleeding, respectively, and 0.52 (95% CI, 0.44, 0.60) and 0.92 (95% CI, 0.90, 0.94) for CRNM bleeding, respectively. CONCLUSION: The EHR-based ISTH major bleeding algorithm demonstrated high sensitivity and specificity, while the CRNM bleeding algorithm showed suboptimal sensitivity. The EHR-based ISTH algorithms may enhance understanding of OAC safety using real-world evidence.