Abstract
Scoring rules assess the quality of probabilistic forecasts, by assigning a numerical score based on the predictive distribution and the corresponding observed value. A scoring rule is proper if the expected score is optimized when the true distribution of the observation is issued as the forecast. In prediction problems, proper scoring rules encourage careful assessments and honesty. In estimation problems, proper scoring rules provide attractive loss and utility functions that can be tailored to the problem at hand. When the observation is given by an empirical distribution rather than a single value, the compatibility between the observed and the predicted distribution is instead assessed by a divergence function. We review the theory of proper scoring rules and propose a propriety condition for divergence functions such that sampling from the hypothesized distribution becomes an expectation minimizing strategy. Score divergences that derive from proper scoring rules are proper, while other commonly used divergences fail to be proper. We apply proper scoring rules to estimate and verify probabilistic weather forecasts while divergence functions are used to evaluate climate simulations.