Bayesian methods in Machine Learning

Bayesian methods have recently regained a significant amount of attention in the machine community due to the development of scalable approximate Bayesian inference techniques. There are several advantages of using Bayesian approach: Parameter and prediction uncertainty become easily available, facilitating rigid statistical analysis. Furthermore, prior knowledge can be incorporated.

$Image may contain: Triangle, Slope, Parallel, Font, Circle.$

About the project

In this project we develop new models, methods and algorithms for Bayesian machine learning, in particular related to Gaussian processes, deep neural networks and computational causal inference.

Sub-projects

Sparse models for machine learning from a Bayesian viewpoint, PhD project for August Arnstad
Bayesian estimation of causal effects using directed graphical models. PhD project for Vera Kvisgaard.
A Bayesian model-averaging toolkit for causal inference with observational data under nonlinear structural equations: An application to the effect of ADHD treatment on school performance by Norwegian children. PhD project for Johan de Aguas.

Financing

Part of the activity is financed through INTEGREAT.

Selected publications

Kvisgaard, V. & Pensar, J. (2025). Bayesian Estimation of Causal Effects from Observational Categorical Data, Bayesian Analysis, Advance Publication, 1-28.
Kvisgaard, V. & Pensar, J. (2025). Bayesian IDA with Background Knowledge. In Scientific Meeting of the Italian Statistical Society (pp. 91-96). Cham: Springer Nature Switzerland.
Hubin, Aliaksandr (2025). Bayesian Generalized Nonlinear Models Offer Basis Free SINDy With Model Uncertainty. In O’Sullivan, David; Sweeney, James; Moghaddam, Shirin & Burke, Kevin (Ed.), Proceedings of the 39th International Workshop on Statistical Modelling. University of Limerick. ISSN 9781036927110. p. 415–418. doi: https:/iwsm2025.ie/wp-content/uploads/2025/07/IWSM2025_Limerick_Proceedings.pdf. Full text in Research Archive
Florian Frommlet, Jon Lachmann, Geir Storvik, Aliaksandr Hubin (2025). FBMS: An R Package for Flexible Bayesian Model Selection and Model Averaging. Preprint.
Eirik Høyheim, Lars Skaaret-Lund, Solve Sæbø, Aliaksandr Hubin (2025). Explainable Bayesian deep learning through input-skip Latent Binary Bayesian Neural Networks. Preprint
P. Denault, William R. ; Hubin, Aliaksandr; Vitelli, Valeria & Richardson, Sylvia (2024).Contributed Discussion to Sparse Bayesian Factor Analysis When the Number of Factors Is Unknown. Bayesian Analysis. ISSN 1936-0975. doi: 10.1214/24-BA1423. Full text in Research Archive
Skaaret-Lund, Lars; Storvik, Geir Olve & Hubin, Aliaksandr (2024). Sparsifying Bayesian neural networks with latent binary variables and normalizing flows.Transactions on Machine Learning Research (TMLR). 2024. doi: 10.48550/arXiv.2305.03395. Full text in Research Archive
Ellingsen, Herman; Hubin, Aliaksandr; Remonato, Filippo & Sæbø, Solve (2024).Outlier Detection in Bayesian Neural Networks: Exploring Pre-activations and Predictive Entropy. Nordic Machine Intelligence (NMI). ISSN 2703-9196. 4(1), p. 1–15. doi: 10.5617/nmi.11406. Full text in Research Archive
Sommerfelt, Sebastian & Hubin, Aliaksandr (2024). Evolutionary variational inference for Bayesian generalized nonlinear models. Neural Computing & Applications.ISSN 0941-0643. p. 21023–21040. doi: 10.1007/s00521-024-10349-1. Full text in Research Archive
Papamarkou, Theodore; Skoularidou, Maria; Palla, Konstantina; Aitchison, Laurence; Arbel, Julyan; Dunson, David; Hubin, Aliaksandr et. al (2024). Position: Bayesian Deep Learning is Needed in the Age of Large-Scale AI.Proceedings of Machine Learning Research (PMLR). ISSN 2640-3498. 235, p. 39556–39586. doi: 10.48550/arXiv.2402.00809. Full text in Research Archive
Hubin, Aliaksandr & Storvik, Geir Olve (2024). Sparse Bayesian Neural Networks: Bridging Model and Parameter Uncertainty through Scalable Variational Inference. Mathematics. ISSN 2227-7390. 12(6). doi: 10.3390/math12060788.Full text in Research Archive
Hubin, Aliaksandr; Heinze, Georg & De Bin, Riccardo (2023). Fractional Polynomial Models as Special Cases of Bayesian Generalized Nonlinear Models. Fractal and Fractional. 7(9), p. 1–23. doi: 10.3390/fractalfract7090641. Full text in Research Archive
Lachmann, Storvik, Frommlet and Hubin: A subsampling approach for Bayesian model selection. International Journal of Approximate Reasoning. ISSN 0888-613X. 151, p. 33–63. doi: 10.1016/j.ijar.2022.08.018. Full text in Research Archive
Hubin, Storvik and Frommlet: Flexible Bayesian Nonlinear Model Configuration, Journal of Artificial Intelligence Research, 2021
Hubin, Frommlet and Storvik: Reversible Genetically Modified Mode Jumping MCMC, Proceedings of 22nd European Young Statisticians Meeting, 2021
Pensar, Talvitie, Hyttinen, and Koivisto: A Bayesian approach for estimating causal effects from observational data, Proccedings of the 34th AAAI Conference on Artificial Intelligence, 2020
Viinikka, Hyttinen, Pensar, and Koivisto: Towards scalable Bayesian learning of causal DAGs, Proceedings of the 34th Conference on Neural Information Systems (NeurIPS), 2020
Hubin, Storvik and Frommlet: A Novel Algorithmic Approach to Bayesian Logic Regression (with Discussion) Bayesian Analysis, 2020
Hubin and Storvik: Combining Model and Parameter Uncertainty in Bayesian Neural Networks, arXiv preprint arXiv:1903.07594, 2019
Hubin and Storvik: Mode jumping MCMC for Bayesian variable selection in GLMM, Computational Statistics & Data Analysis, 2018
Hubin and Storvik: Estimating the marginal likelihood with Integrated nested Laplace approximation (INLA), arXiv preprint arXiv:1611.01450, 2016

Tags: data science

Published Apr. 27, 2022 11:31 AM - Last modified June 12, 2026 12:00 PM

Participants

Geir Olve Storvik University of Oslo
Ida Scheel University of Oslo
Leiv Rønneberg University of Oslo
Aliaksandr Hubin University of Oslo
August Arnstad University of Oslo
Johan Pensar University of Oslo
Johan de Aguas University of Oslo
Vera Haugen Kvisgaard University of Oslo

Detailed list of participants