Optimizing Explainable AI (XAI), Model Performance, and Fairness with Proxy Measures: A Human-Out-of-the-Loop Approach

Across high-stakes domains, the project addresses the persistent tension between transparency, predictive power, and equitable treatment. Human-in-the-loop oversight is costly, variable, and difficult to scale; instead, this study proposes a human-out-of-the-loop pipeline in which automatically computed proxy measures stand in for human judgments of interpretability and trust, while also capturing performance and fairness. The central thesis is that well-chosen proxies—validated against established criteria—can guide model selection and tuning toward principled, reproducible trade-offs.

The investigation curates and extends a library of XAI proxies: feature-importance consistency under perturbations and resampling; sparsity and monotonicity of explanations; local fidelity of surrogates to original predictions; and stability across seeds, folds, and near-neighbors. Fairness is tracked with statistical parity and equal opportunitygaps, alongside within-group calibration. Predictive quality is anchored by accuracy/F1 with calibration error and uncertainty diagnostics. These targets feed a multi-objective optimization layer—surrogate-assisted evolutionary or Bayesian search—that explores model classes, regularization, and explainer hyperparameters (e.g., SHAP kernel width, LIME segmentation, prototype budgets), yielding Pareto fronts and principled knee points. A meta-optimization step learns proxy weightings to tighten correlation with human-grounded benchmarks where available, preserving automation while checking face validity.

Evaluation spans structured/tabular settings with salient societal stakes: COMPAS for bias–fairness trade-offs and representative healthcare datasets for performance and safety. Analyses emphasize reproducibility of explanations, robustness under distribution shift, and sensitivity to reweighting protected attributes. The overarching narrative is that carefully engineered proxies can faithfully steer joint improvement of explainability, accuracy, and fairness—enabling scalable, auditable model development without continual human intervention.