Modern models can be extremely accurate, but they’re often opaque. When a network flags a patient at risk, tags a protein as disease-related, or rejects a transaction, we need to know why. This project takes a careful look at explanation methods for deep learning (like SHAP, LIME, and Grad-CAM) to see not just whether they look convincing, but whether they truly reflect the model’s reasoning.
Our focus is twofold. First, we compare explanation tools across representative tasks (tabular, text, and vision) and evaluate them on what matters in practice: faithfulness (do the explanations actually reflect what the model uses); usability (are they stable, simple, and fast); and practical impact (do they help with debugging or catching issues like data leakage). Second, we clarify the messy landscape of fidelity metrics (deletion/insertion, sufficiency/necessity, infidelity, robustness, and sanity checks): what each actually measures, where they fail, and the trade-offs involved. The outcome is a lightweight benchmark suite, reproducible reports, and clear guidelines for choosing the right explanation method and fidelity metric for a given task and constraint.
Although the emphasis is on deep neural networks, the study can be scoped to classic ML baselines or narrowed to a specific family (such as graph neural networks) for a deeper, domain-focused analysis. The outcome is a lightweight, reproducible benchmark and practical guidance for choosing the right explanation method and metric for the job.
If you want to learn more or get involved, please contact Pooya Zakeri.