Novel XAI algorithm development for analyzing machine learning models

Abstract

Over the years, explaining machine learning (ML) predictions has become crucial as these ML models are increasingly being deployed in high-stakes domains such as healthcare and autonomous driving. In Explainable Artificial Intelligence (XAI), SHapley Additive exPlanations (SHAP) is the widely used method for model interpretability. However, despite its high usage in the realm of XAI, it fails to differentiate between causality and correlation, often misattributing feature importances when features are highly correlated. To address this issue, we propose Causal SHAP, a novel framework that integrates causal relationships into feature attribution, while preserving all key properties of SHAP. In the framework, we combine the Peter-Clark (PC) algorithm for causal discovery and the Intervention Calculus when the DAG is Absent (IDA) algorithm for causal strength quantification to address the weakness of SHAP. Specifically in result, Causal SHAP reduces attribution scores for features that are merely correlated with the target variable, and redistribute attribution scores to features that causally contribute to the target. The effectiveness of the framework is validated through experiments on both synthetic and real-world dataset, focusing on the healthcare domain. To summarize, this study contributes to the field of XAI by providing a practical framework for causal-aware model explanations. Our approach is particularly valuable in domains such as healthcare, where the data are highly correlated, and understanding the true causal relationships is critical for informed decision-making.