Online hate speech detection is widely used by social media platforms, but current deep learning models often behave as “black boxes”, making decisions that can reinforce societal biases while offering little insight into why something is classified as hateful. To address this, we introduce Supervised Rational Attention (SRA), a method that trains models to focus on the same words that human annotators highlight when justifying their decisions.
Using English (HateXplain) and Portuguese (HateBRXplain) datasets, our approach produces 2.4× better explainability than the strongest baseline while maintaining competitive classification performance.
By aligning model attention directly with human reasoning, SRA enables transparent, faithful, and more trustworthy explanations. This is an important step toward responsible AI in sensitive applications such as content moderation.
Funding:
This research was conducted during a summer 2025 research project funded by UiO:Life Science, a cross-faculty initiative that supports educational research opportunities for students at the University of Oslo.
Abstract:
The opaque nature of deep learning models presents significant challenges for the ethical deployment of hate speech detection systems. To address this limitation, we introduce Supervised Rational Attention (SRA), a framework that explicitly aligns model attention with human rationales, improving both interpretability and fairness in hate speech classification. SRA integrates a supervised attention mechanism into transformer-based classifiers, optimizing a joint objective that combines standard classification loss with an alignment loss term that minimizes the discrepancy between attention weights and human-annotated rationales. We evaluated SRA on hate speech benchmarks in English (HateXplain) and Portuguese (HateBRXplain) with rationale annotations. Empirically, SRA achieves 2.4x better explainability compared to current baselines, and produces token-level explanations that are more faithful and human-aligned. In terms of fairness, SRA achieves competitive fairness across all measures, with second-best performance in detecting toxic posts targeting identity groups, while maintaining comparable results on other metrics. These findings demonstrate that incorporating human rationales into attention mechanisms can enhance interpretability and faithfulness without compromising fairness.
- Full paper access: https://doi.org/10.48550/arXiv.2511.07065
- Conference Information: AAAI-26 | AI Alignment Track | January 20-27, 2026 | Singapore
- Conference website: https://aaai.org/conference/aaai/aaai-26/
- Paper ID: AIA209
