We are happy to announce that our paper has been accepted to the Northern Lights Deep Learning (NLDL) Conference.
Link to paper: https://openreview.net/forum?id=8T8QkDsuO9#discussion
In recent years, the use of Large Language Models (LLMs) has soared across a wide range of applications. However, a persistent challenge that many practitioners have likely encountered is hallucinations. These can be broadly categorized into faithfulness hallucinations, where the LLM deviates from a user's provided instructions, and factual hallucinations, where the generated content diverges from verifiable real-world facts.
Our latest paper presents a method to detect both types of hallucinations using uncertainty estimates. At a high level, the approach is straightforward: we utilize an ensemble of models to predict the next token in a sequence. By measuring the agreement among ensemble members' predictions, we estimate uncertainty—consistent predictions reflect high confidence, while diverging predictions signal increased uncertainty and, consequently, a higher risk of hallucination. This uncertainty serves as a key indicator, enabling effective hallucination detection.
However, applying ensembles of LLMs in practice poses two major challenges. First, the memory requirements of a traditional ensemble scale linearly with the number of models, quickly becoming infeasible on existing hardware. Second, the cost of pretraining even a single sophisticated LLM is enormous, making the training of an entire ensemble prohibitively expensive.
Our work addresses the first issue by leveraging BatchEnsemble, a method where all ensemble members share a common set of weights while each member maintains a unique set of rank-one matrices. The ensemble is represented as the Hadamard product of the shared weights and the individual rank-one matrices, which dramatically reduces the memory footprint compared to traditional ensembles.
Secondly, to address the high cost of pretraining an ensemble of LLMs, we utilize pre-trained weights for the shared component. This approach harnesses existing knowledge, avoiding the computationally intensive process of training the entire ensemble from scratch. By tailoring the ensemble to downstream tasks, we not only enhance performance compared to a single model but also gain the capability to detect hallucinations.
A high-level overview of our method and general pipeline is illustrated in the figure below:
LoRA matrices during training and then merged. Each ensemble member is represented by an individual rank-one matrix (fast weights) that is combined with the shared weights using a Hadamard product. (Right) The ensemble generates uncertainty estimates, which serve as features for a classifier to determine whether the LLM’s prediction is correct or hallucinated.