Valeriya Naumova: A machine learning approach for adaptive parameter selection

 

Abstract: Despite a variety of methods and techniques for parameter choice, the issue of parameter and model selection, in general, still remains a challenge for many applications.  The main difficulty lies in constructing a rule, allowing to choose the parameter from a given noisy dataset without relying either on any a priori knowledge of the solution or on the noise level. In this talk, combining advances from statistical learning theory with insights from regularisation theory, we propose a novel approach to approximate the high-dimensional function, mapping noisy data into a good approximation to the optimal parameter in Tikhonov and elastic-net regularisation.  Our assumptions are that solutions of the problem are statistically distributed in a concentrated manner on (lower dimensional) linear subspaces and the noise is sub-gaussian. We provide explicit error bounds on the accuracy of the approximated parameter and the corresponding regularization solution. Furthermore, we present an efficient algorithm for the computation of an approximate optimal parameter from a given training data. We also compare our approach to the state-of-the-art parameter selection criteria and illustrate its superiority in terms of accuracy and computational time on a number of simulated and real data.