Application of LLMs in CAD-RADS Classification and Patient Management.
P, T., G, M., D, C., F, C., F, S., J, U., G, S., E, S., J, B., & RA, L. (2026). Application of LLMs in CAD-RADS Classification and Patient Management.. Echocardiography (Mount Kisco, N.Y.). https://doi.org/10.1111/echo.70525
P T, G M, D C, F C, F S, J U, et al. Application of LLMs in CAD-RADS Classification and Patient Management.. Echocardiography (Mount Kisco, N.Y.). 2026; doi: 10.1111/echo.70525
P T, G M, D C, et al. Application of LLMs in CAD-RADS Classification and Patient Management.[J]. Echocardiography (Mount Kisco, N.Y.). 2026. DOI: 10.1111/echo.70525.
@article{p2026,
author = {Tarkowski P and Muscogiuri G and Casartelli D and Coraducci F and Sassi F and Usai J and Staśkiewicz G and Siek E and Byczkowski J and Licu RA},
title = {Application of LLMs in CAD-RADS Classification and Patient Management.},
journal = {Echocardiography (Mount Kisco, N.Y.)},
year = {2026},
doi = {10.1111/echo.70525},
note = {PMID: 42338008},
}
TY - JOUR AU - Tarkowski P AU - Muscogiuri G AU - Casartelli D AU - Coraducci F AU - Sassi F AU - Usai J AU - Staśkiewicz G AU - Siek E AU - Byczkowski J AU - Licu RA TI - Application of LLMs in CAD-RADS Classification and Patient Management. T2 - Echocardiography (Mount Kisco, N.Y.) PY - 2026 DO - 10.1111/echo.70525 AN - PMID:42338008 ER -
PURPOSE: To evaluate the capability of four publicly available Large language models (LLMs) to assign Coronary Artery Disease-Reporting and Data System (CAD-RADS) scores and provide patient management recommendations based on synthetic coronary CT angiography (CCTA) reports. METHODS: Four LLMs (ChatGPT 4o, Claude 3.7, DeepSeek, and Gemini 2.5 Pro) were tasked with analyzing reports and suggesting next steps. Prompts were framed from the perspective of both a cardiologist and a radiologist. Agreement with a human reference standard was assessed using weighted Cohen's kappa, Fleiss' kappa, and Krippendorff's alpha for CAD-RADS scoring, and unweighted Cohen's kappa for management recommendations. A Bayesian Wilcoxon signed-rank test was performed to assess directional bias. RESULTS: Performance variations were observed across LLMs and prompt identities. Claude-3.7 achieved almost perfect agreement for CAD-RADS scoring (κ = 0.997) regardless of prompt identity, Gemini similarly achieved almost perfect agreement (radiologist: κ = 0.962; cardiologist: κ = 0.990), ChatGPT demonstrated almost perfect agreement when prompted as a radiologist (κ = 0.896) but only substantial agreement when prompted as a cardiologist (κ = 0.715). DeepSeek showed the lowest overall performance (radiologist: κ = 0.637; cardiologist: κ = 0.768). By category, all LLMs correctly identified CAD-RADS 0, whereas higher-grade stenosis (4A/4B) remained the most challenging, with non-Claude models showing low-to-null agreement in some configurations. The LLMs' accuracy in proposing further management was considerably lower than their scoring accuracy, with CAD-RADS 3 showing the greatest variability in management recommendations across models and between human specialists. Furthermore, both CAD-RADS scoring and management recommendations varied depending on the professional identity specified in the prompt. CONCLUSION: While LLMs demonstrated reliable scoring performance for lower-grade CAD-RADS categories (0-2), agreement was substantially reduced for higher-grade stenosis categories (4A/4B) and non-diagnostic studies, which could pose risks to patients. Their current ability to generate dependable clinical management recommendations is limited.