Aliaksandr Hubin

• Hubin, Aliaksandr & Storvik, Geir Olve (2026). Estimating the marginal likelihood with Integrated nested Laplace approximation (INLA). Journal of Reproducible Statistics. 1(1), p. 1–13. doi: 10.68365/jrepstat.2026.9. Full text in Research Archive Show summary

  The marginal likelihood is a well established model selection criterion in Bayesian statistics. It also allows to efficiently calculate the marginal posterior model probabilities that can be used for Bayesian model averaging of quantities of interest. For many complex models, including latent modeling approaches, marginal likelihoods are however difficult to compute. One recent promising approach for approximating the marginal likelihood is Integrated Nested Laplace Approximation (INLA), designed for models with latent Gaussian structures. In this study we compare the approximations obtained with INLA to alternative approaches across a progression of models of varying complexity. In particular, we evaluate a simple linear latent model with exact analytical truth, a Bayesian linear regression model, probit and logit Bayesian regression models, a longitudinal Poisson generalized linear mixed model, and two extra simulation scenarios designed to probe the limits of the latent Gaussian approximation.

• Kvenshagen, Line Norman; Kvenshagen, Hanna Norman; Holte, Kari; Jacobsen, Anne Flem; Størdal, Ketil & Hubin, Aliaksandr [Show all 7 contributors for this article] (2026). Association between Regional Labour Analgesia and Neonatal Heart Rate Immediately after Birth. Neonatology. ISSN 1661-7800. p. 1–8. doi: 10.1159/000550615.
• Skaaret-Lund, Lars; Høyheim, Eirik & Hubin, Aliaksandr (2026). LBBNN: An R Package for Sparse and Explainable Bayesian Deep Learning with Latent Binary Bayesian Neural Networks. The R Journal. 18(3), p. 1–21. doi: https:/journal.r-project.org/articles/RJ-2026-048/RJ-2026-048.pdf. Full text in Research Archive Show summary

  The package LBBNN (latent binary Bayesian neural networks) provides a framework for doing sparse uncertainty aware and to a large degree interpretable Bayesian inference in neural network models, with potentially huge datasets, and highly overparametrized networks. Currently, most packages implementing Bayesian neural networks in R use Markov chain Monte Carlo based inference, without the possibility of GPU acceleration. Our package, using variational inference and the LibTorch backend (via the torch package), provides a more scalable and computationally efficient approach. The package integrates three different methodological works: Firstly, it takes into account the basic LBBNN model, where each weight in the network is associated with a Bernoulli latent inclusion indicator, allowing for incorporating model uncertainty and achieving substantial sparsification of the network while maintaining high predictive power. Secondly, using normalizing flows allows for modeling more complex and flexible variational distributions with statistical dependencies. And thirdly, by allowing the input variables to skip to any layer in the network, we can model different types of functions, e.g. we might only have linear connections. This allows for more explainable results. Lastly, local explanations at the prediction level with uncertainty are provided in the LBBNN package and are guaranteed to be exact for piecewise linear activations. To demonstrate how the package can be used in practice, we include experiments on both synthetic data and real-world datasets, covering both tabular and image based data.

• Vittersø, Sigurd Myhre & Hubin, Aliaksandr (2026). BGET: Bayesian Genetic Ensemble of Trees. In Bin, Riccardo De; Salucci, Clara Bertinelli & Hubin, Aliaksandr (Ed.), Proceedings of the 40th International Workshop on Statistical Modelling 28.06 – 03.07.2026, Oslo, Norway. International Workshop on Statistical Modelling. ISSN 9788255325000. p. 592–595. doi: https:/www.mn.uio.no/math/english/research/groups/statistics-data-science/events/conferences/iwsm2026/proceedings/proceedings_iwsm2026_proofonline.pdf. Full text in Research Archive Show summary

  We embed classification and regression trees in the Bayesian generalized nonlinear model framework, defining a Bayesian genetic ensemble of trees with complexity-penalizing priors on tree structures. Inference is performed via genetically modified mode jumping MCMC, which explores large spaces of tree ensembles using posterior inclusion probabilities to guide genetic operators. The approach is evaluated on the Abalone and Boston housing data and performs on par with relevant related baselines.

• Hubin, Aliaksandr; Krogdahl, Åshild; Løkka, Guro & Kortner, Trond M. (2026). Bayesian Fractional Polynomials for Optimal Dosage Estimation with Fish Nutrition Applications. In Bin, Riccardo De; Salucci, Clara Bertinelli & Hubin, Aliaksandr (Ed.), Proceedings of the 40th International Workshop on Statistical Modelling 28.06 – 03.07.2026, Oslo, Norway. International Workshop on Statistical Modelling. ISSN 9788255325000. p. 301–306. doi: https:/www.mn.uio.no/math/english/research/groups/statistics-data-science/events/conferences/iwsm2026/proceedings/proceedings_iwsm2026_proofonline.pdf. Full text in Research Archive Show summary

  The problem of optimal dosage estimation arises in diverse scientific domains, from pharmacology and toxicology to aquaculture and environmental studies. Statistical modeling of nonlinear dose–response relationships is essential to quantify biological effects and determine response-optimal levels. This paper introduces a flexible Bayesian fractional polynomial (BFP) framework for modeling such relationships, allowing for model uncertainty quantification and robust prediction through Bayesian model averaging. Extensive simulation results demonstrate that the proposed BFP approach yields accurate estimation of optimal dose levels, outperforming benchmarks significantly. The approach is demonstrated on real data from fish nutrient requirement experiments.

• Frommlet, Florian; Lachmann, Jon; Storvik, Geir Olve & Hubin, Aliaksandr (2026). FBMS: An R package for flexible Bayesian model selection and averaging with nonlinear models. Journal of Computational Science. ISSN 1877-7503. 100, p. 102947–102947. doi: 10.1016/j.jocs.2026.102947. Full text in Research Archive Show summary

  The FBMS R package provides a unified interface for Bayesian model selection and model averaging across a broad class of regression models, including Gaussian regression, generalized linear models, and models with nonlinear functional relationships, with possible extensions to mixed-effects and survival models. Users specify candidate predictors together with transformation and interaction rules, and FBMS automatically explores the corresponding model spaces, estimates marginal likelihoods, and returns posterior model and inclusion probabilities, best and median probability models, posterior modes of parameters, and predictions with associated uncertainty. The core machinery combines mode-jumping Markov chain Monte Carlo with a genetically modified extension that iteratively generates and evaluates nonlinear features in the framework of Bayesian generalized nonlinear regression, while supporting flexible prior specification (including g-priors, Jeffreys priors, and empirical Bayes constructions), subsampling for large datasets, parallel computation, and the option to supply custom likelihoods for specialized applications. As special cases, the framework covers models based on fractional polynomials, logic regression, neural networks, symbolic regression, and classical linear regression.

• Sánchez-Gordón, Mary; Colomo-Palacios, Ricardo; Gama, Kiev & Hubin, Aliaksandr (2026). Demystifying Knowledge Hiding in Software Testing: Insights from Practitioners. IEEE/ACM International Conference on Software Engineering: Software Engineering in Society Track (ICSE-SEIS). ISSN 2832-7608. 2026, p. 148–153. doi: 10.1145/3786581.3786937. Full text in Research Archive Show summary

  Software development is a knowledge-intensive and collaborativeactivity. However, practitioners may sometimes deliberately conceal or withhold information, ideas, or expertise relevant to softwaretasks, even when a teammate explicitly requests such knowledge.This is known as knowledge hiding behaviour (KHBs). This study explores what knowledge is hidden in software testing. A survey was designed based on measures of work conditions, KHBs, andknowledge related to software testing. Responses from 102 software practitioners were collected and analysed. Although only13 participants reported KHBs, these should not be overlooked. Findings highlight the contextual nature of KHBs, suggesting that exploring KHBs in software teams is a promising avenue for future research.

• Høyheim, Eirik; Skaaret-Lund, Lars; Sæbø, Solve & Hubin, Aliaksandr (2026). Explainable Bayesian Deep Learning through Input-skip Latent Binary Bayesian Neural Networks. The journal of artificial intelligence research. ISSN 1076-9757. 86, p. 1–42. doi: https:/www.jair.org/index.php/jair/article/view/20433/27320. Full text in Research Archive Show summary

  Modeling natural phenomena with artificial neural networks (ANNs) often provides highly accurate predictions. However, ANNs often suffer from over-parameterization, complicating interpretation and raising uncertainty issues. Bayesian neural networks (BNNs) address the latter by representing weights as probability distributions, allowing for predictive uncertainty evaluation. Latent binary Bayesian neural networks (LBBNNs) further handle structural uncertainty and sparsify models by removing redundant weights. This article advances LBBNNs by enabling covariates to skip to any succeeding layer or be excluded, simplifying networks and clarifying input impacts on predictions. This further allows us to learn simpler structures (e.g., linear or even constant intercept only models) when appropriate. Furthermore, the input-skip LBBNN (ISLaB) approach reduces network density significantly compared to standard LBBNNs, achieving over 99% reduction for small networks and over 99.9% for larger ones, while still maintaining high predictive accuracy and uncertainty quantification. For example, on MNIST, we reached 97% accuracy and great calibration with just 935 weights, reaching state-of-the-art for compression of neural networks. Furthermore, the proposed method accurately identifies the true covariates and adjusts for system non-linearity. The main contribution is the introduction of active paths, enhancing directly designed global and local explanations within the LBBNN framework. The latter are exact with theoretical guarantees and do not require post hoc external tools.

• Hasselgård, Ann-Margret; Torgersen, Heidi Irene Klevstuen; Melby, Ann-Cathrin; Vullum, Sylvi; Flatland, Sigbjørn & Imsland, Kristine Aasheim [Show all 11 contributors for this article] (2026). Nurses’ subjective workload assessed with the NASA-TLX and its relationship with the nursing activities score (NAS) and nurses’ demographic factors: a prospective cross-sectional study. Intensive and Critical Care Nursing. ISSN 0964-3397. 95, p. 1–8. doi: 10.1016/j.iccn.2026.104428. Full text in Research Archive Show summary

  Aim With an international shortage of experienced critical care nurses, there is an increasing focus on improving staff retention. This study examined the subjective nursing workload in Norwegian intensive care units (ICUs) and its relationship with the Nursing Activities Score (NAS) and nurses’ demographics. Method A prospective cross-sectional study utilized the National Aeronautics and Space Administration Load Index (NASA-TLX) to measure six demands of subjective nursing workload (mental, physical, temporal, effort, performance, and frustration), the NAS and nurses’ demographic data. Relationships were analyzed using mixed effect Beta regression. Results A total of 3,476 questionnaires from 12 ICUs (81.3% response rate) were analyzed. Most critical care nurses were over 46 years of age (52.3%) with most registered nurses younger than 46 years (71.6%). The mean (SD) NASA-TLX score was 48.8 (17.5). The mean (SD) NAS per shift per nurse varied from 56.4% (42.8) to 139.5% (20.7). Higher NAS scores were associated with higher NASA-TLX scores (p < 0.001) in areas of mental (p < 0.001), physical (p = <0.001), temporal (p = <0.001), and effort (p = >0.001), frustration (p = <0.001) but not with performance (p = 0.35). Age, educational level, experience and shifts were associated with all the NASA-TLX six demands, although less so with performance and frustration. Conclusion High subjective workload was consistently associated with higher NAS scores. These findings highlight the importance of integration of subjective and objective measures of workload. A multifaceted understanding of workload is essential to protect nurses’ wellbeing, support a healthy work-life balance, and address workforce challenges. Proactively monitoring both areas of workload enables management to identify areas of improvement, address staffing issues, and implement targeted interventions to promote nursing retention. Implication for clinical practice This study underscores the value of workload assessments as an effective tool for identifying staffing challenges in ICUs. Managers can implement targeted strategies, such as focusing on the needs of specific groups. In optimizing workload management, the unit can retain nursing staff.

• Olofsson, Charlott Kjærre; Hubin, Aliaksandr; Vinje, Hilde; Rachah, Amira; Reksen, Olav & Kielland, Camilla [Show all 7 contributors for this article] (2026). Machine learning for detection of subclinical mastitis: A Bayesian approach incorporating diagnostic test properties. Preventive Veterinary Medicine. ISSN 0167-5877. 249, p. 1–14. doi: 10.1016/j.prevetmed.2026.106791. Full text in Research Archive Show summary

  The large amount of data collected through automatic milking systems (AMS) may be used for early detection of intramammary infections and become instrumental for monitoring udder health in dairy herds. Machine learning (ML) techniques can aid in improving diagnostic test properties of current indicators of subclinical mastitis (SCM). In this study, we present novel customized ML models for predicting SCM from AMS data. We show how results from several diagnostic tests can be incorporated into ML model training by explicitly accounting for their sensitivity and specificity. The underlying infection status was modeled as a latent variable derived from bacteriological culture (BC) and polymerase chain reaction (PCR) results on milk samples. Model performance was evaluated using a customized log-likelihood (CLL) function, addressing uncertainty in prediction target, and compared with traditional metrics using simulated data. Our study demonstrates that incorporating prior knowledge of sensitivity and specificity of the tests directly into the likelihood function during model training enables reliable ML even in scenarios with an imperfect target variable. The customized models achieved the highest CLL scores on real data and demonstrated significantly better calibration on simulated data. At the same time, all models showed similarly near-perfect area under the curve (AUC) on simulated data. Further validation across herds is needed, but our approach shows promise for robust SCM prediction from AMS data using ML. The framework is applicable to other scenarios in veterinary epidemiology with imperfectly measured outcomes.

• Sánchez-Gordón, Mary; Colomo-Palacios, Ricardo; Gordon, Alex Sanchez & Hubin, Aliaksandr (2026). The role of voice and silence behaviors in software development: a structural equation modeling analysis. Information and Software Technology. ISSN 0950-5849. 190. doi: 10.1016/j.infsof.2025.107972. Full text in Research Archive Show summary

  Context Most software companies strive to have high-performing teams and mitigate withdrawal behaviors like being present but unproductive. In this context, psychological safety and developers’ perceived impact are suggested as potential drivers of voice and silence behaviors. However, understanding these social aspects of software development entails the incorporation of social science theories. Objective This study aims to empirically explore the relationships among software professionals’ perceived impact, psychological safety, voice and silence behaviors, burnout particularly withdrawal and performance using a theoretical model. Method A survey questionnaire was conducted, resulting in 158 valid responses from software development teams. Then, we analyzed the responses using structural equation modeling (SEM) and a novel semi-confirmatory SEM. All variables were measured using pre-validated instruments. Results The findings supported the theoretical model, showing that psychological safety was more related to silence than voice, whereas perceived impact showed a stronger relationship with voice than with silence. Furthermore, silence contributed to higher burnout, whereas voice alleviated it. In contrast, silence showed a negative association with performance, whereas voice was positively associated with it. Conclusions This study examines the direct effects of perceived impact and psychological safety on voice and silence behaviors. Additionally, it examines the direct effects of these behaviors on burnout and task performance among software professionals. Consequently, the findings offer both theoretical and practical insights.

• Lachmann, Jon & Hubin, Aliaksandr (2025). Subsampling MCMC for Bayesian Variable Selection and Model Averaging in BGNLM. In Srakar, Andrej (Eds.), 23rd European Young Statisticians Meeting 11.–15. September 2023, Ljubljana, Slovenia Proceedings. p. 57–62. Full text in Research Archive Show summary

  Bayesian Generalized Nonlinear Models (BGNLM) offer a flexible nonlinear alternative to GLM while still providing better interpretability than machine learning techniques such as neural networks. In BGNLM, the methods of Bayesian Variable Selection and Model Averaging are applied in an extended GLM setting. Models are fitted to data using MCMC within a genetic framework by an algorithm called GMJMCMC. In this paper, we combine GMJMCMC with a novel algorithm called S-IRLS-SGD for estimating the marginal likelihoods in BGLM/BGNLM by subsampling from the data. This allows to apply GMJMCMC to tall data.

• Hubin, Aliaksandr (2025). Bayesian Generalized Nonlinear Models Offer Basis Free SINDy With Model Uncertainty. In O’Sullivan, David; Sweeney, James; Moghaddam, Shirin & Burke, Kevin (Ed.), Proceedings of the39th International Workshop on Statistical Modelling. International Workshop on Statistical Modelling. ISSN 9781036927110. p. 415–418. Full text in Research Archive Show summary

  Sparse Identification of Nonlinear Dynamics (SINDy) has become a standard methodology for inferring governing equations of dynamical systems from observed data using statistical modeling. However, classical SINDy approaches rely on predefined libraries of candidate functions to model nonlinearities, which limits flexibility and excludes robust uncertainty quantification. This paper proposes Bayesian Generalized Nonlinear Models (BGNLMs) as a principled alternative for more flexible statistical modeling. BGNLMs employ spike-and-slab priors combined with binary inclusion indicators to automatically discover relevant nonlinearities without predefined basis functions. Moreover, BGNLMs quantify uncertainty in selected bases and final model predictions, enabling robust exploration of the model space. In this paper, the BGNLM framework is applied to several three-dimensional (3D) SINDy problems.

• Mathisen, Therese Fostervold; Hubin, Aliaksandr; Mountjoy, Margo & Sundgot-Borgen, Jorunn (2025). Safety of Heavy Resistance Exercise During Pregnancy and Postpartum: A Controlled, Prospective Cohort Study. Women in Sport & Physical Activity Journal. ISSN 1063-6161. doi: 10.1123/wspaj.2025-0010. Full text in Research Archive Show summary

  Objective: To address the knowledge gap on the influence of heavy free-weight resistance exercise on the health of the pregnant athlete and infant. Design: This is a prospective, controlled cohort study with two groups of pregnant women; those engaging in mainly heavy free-weight resistance exercise (REX, n = 33) and those regularly physically active (REF, n = 20). Data originate from eight questionnaires answered prospectively throughout pregnancy and 12 months postpartum. Variables cover exercise routines, pregnancy symptoms and complications, fetal growth, delivery, and mother and infant’s health. Results: Bayesian logistic regression with random and fixed effects identified group membership as a significant predictor of proteinuria, with REX having higher exposure (posterior inclusion probability [PIP] = 1.00; β1 = 197.07, 95% credible interval [CrI] [163.05, 231.08]). Further, Bayesian logistic regression with random and fixed effects confirmed a higher risk of urinary incontinence in the REF group during pregnancy (PIP = 0.98; β1 = −1.17, 95% CrI [−1.89, −0.46]). Gas incontinence was also found more common in the REF group during pregnancy (PIP = 1.00; β1 = −7.62, 95% CrI [−8.68, −6.57]) and postpartum (PIP = 0.91; β1 = −49.39, 95% CrI [−53.57, −45.43]). There were no significant differences in the postpartum period for fecal and urinary incontinence, injuries, infant health outcomes (all PIP < 0.01), or need for assisted delivery (PIP = 0.33). Conclusion: Female athletes experienced in heavy resistance exercise, who continue training during pregnancy and postpartum, do not face increased pregnancy complications. These findings provide evidence supporting safety of such exercise. Yet we highlight the need for a confirmatory study under full clinical supervision.

• Haukeland-Parker, Stacey Marie; Jervan, Øyvind; Ghanima, Waleed; Spruit, Martijn A.; Hubin, Aliaksandr & Tavoly, Mazdak [Show all 10 contributors for this article] (2025). The effect of exercise-based rehabilitation on objective physical activity levels in patients with persistent dyspnoea following pulmonary embolism. Thrombosis Research. ISSN 0049-3848. 252. doi: 10.1016/j.thromres.2025.109362. Full text in Research Archive Show summary

  Background Rehabilitation improves exercise capacity following pulmonary embolism (PE), but it is unclear whether this improvement is transferred into increased physical activity. Objectives. To explore the effect of exercise-based rehabilitation on objective measures of physical activity in patients with persistent dyspnoea following PE compared to standard care. Methods This pre-planned substudy compared an eight-week exercise-based rehabilitation program to usual care. Number of steps per day and time spent in sedentary, low, moderate and vigorous physical activity (in number of metabolic equivalent of tasks) were measured for seven consecutive days at baseline prior to intervention start and at post-intervention using accelerometery. Data were analysed with a mixed-effect generalised linear model. Results Complete data was provided by 48 patients at both timepoints (mean age 58 years, 54 % men) with 54 % randomised to rehabilitation. There were no significant differences between groups at baseline. The majority of time (717 min per day) was spent in sedentary activity and mean number of steps per day was 6701. At post-intervention, there were no significant differences between groups in any of the objectively measured physical activity parameters (p > 0.05). Conclusion In this cohort, exercise-based rehabilitation following PE did not change daily physical activity levels compared to usual care control group.

• Sánchez-Gordón, Mary; Colomo-Palacios, Ricardo; Guevara-Vega, Cathy; Quiña-Mera, Antonio & Hubin, Aliaksandr (2025). The Effect of Stereotypes on Perceived Competence of Indigenous Software Practitioners: A Study of Dress Style in Professional Photos. Empirical Software Engineering. ISSN 1382-3256. 30. doi: 10.1007/s10664-025-10675-5. Full text in Research Archive Show summary

  Context. Potential employers can easily access job candidates’ photos online and attempt to infer a candidate’s ft or alignment based on their dress style. In this context, for candidates from marginalized groups like Indigenous people, traditional clothing holds cultural signifcance as it serves as a lively expression of belonging, participation in ceremonies, and resistance. Objective. This exploratory study aims to empirically demonstrate whether dress manipulation in a picture afects the perceived competence of equally qualifed candidates for a position like a software developer in which this cue should not be crucial. Method. We conducted a quasi-experiment based on a survey. It involved job candidates (photo models) and participants (evaluators) from IT companies located in Ecuador. The analysis was performed by ftting a linear mixed-efects (LME) model based on dress style, gender and race/ethnicity of the candidates as well as evaluators’ gender and experience in hiring. Also, a thematic analysis was conducted. Results. Findings show that dress manipulation hardly infuences the evaluators’ evaluation of candidates’ competence, as no statistically signifcant diferences were found in our sample. Most of the unexplained variance (64.461%) stems from variability in scores across evaluators. Likewise, the thematic analysis revealed notable evaluator discrepancies indicating varying judgments and outcomes that suggest idiosyncrasies, which are not noise or error. Conclusions. This study demonstrates the value of contextual factors - such as gender, race/ethnicity and cultural background— in software engineering studies and calls for valuing individual software developers and their human aspects. Perceived competence extends beyond hiring situations as it can infuence initial trust and cooperative behaviors among software development team members when interacting with unfamiliar collaborators.

• Halstensen, Thor-David; Hardeland, Camilla; Ghanima, Waleed; Grøndahl, Vigdis Abrahamsen; Hubin, Aliaksandr & Jørgensen, Camilla Tøvik [Show all 8 contributors for this article] (2025). External validation of the DAYS score for suspected deep vein thrombosis. Research and Practice in Thrombosis and Haemostasis (RPTH). 9(4). doi: 10.1016/j.rpth.2025.102885. Full text in Research Archive
• Hajj, Ghadi al; Hubin, Aliaksandr; Kanduri, Chakravarthi; Pavlovic, Milena; Rand, Knut Dagestad & Widrich, Michael [Show all 11 contributors for this article] (2024). Incorporating probabilistic domain knowledge into deep multiple instance learning. Proceedings of Machine Learning Research (PMLR). 235, p. 17279–17297. Full text in Research Archive Show summary

  Deep learning methods, including deep multiple instance learning methods, have been criticized for their limited ability to incorporate domain knowledge. A reason that knowledge incorporation is challenging in deep learning is that the models usually lack a mapping between their model components and the entities of the domain, making it a non-trivial task to incorporate probabilistic prior information. In this work, we show that such a mapping between domain entities and model components can be defined for a multiple instance learning setting and propose a framework DeeMILIP that encompasses multiple strategies to exploit this mapping for prior knowledge incorporation. We motivate and formalize these strategies from a probabilistic perspective. Experiments on an immune-based diagnostics case show that our proposed strategies allow to learn generalizable models even in settings with weak signals, limited dataset size, and limited compute.

• Sommerfelt, Sebastian & Hubin, Aliaksandr (2024). Evolutionary variational inference for Bayesian generalized nonlinear models. Neural Computing & Applications. ISSN 0941-0643. p. 21023–21040. doi: 10.1007/s00521-024-10349-1. Full text in Research Archive Show summary

  In the exploration of recently developed Bayesian Generalized Nonlinear Models (BGNLM), this paper proposes a pragmatic scalable approximation for computing posterior distributions. Traditional Markov chain Monte Carlo within the populations of the Genetically Modified Mode Jumping Markov Chain Monte Carlo (GMJMCMC) algorithm is an NP-hard search problem. To linearize them, we suggest using instead variational Bayes, employing either mean-field approximation or normalizing flows for simplicity and scalability. This results in an evolutionary variational Bayes algorithm as a more scalable alternative to GMJMCMC. Through practical applications including inference on Bayesian linear models, Bayesian fractional polynomials, and full BGNLM, we demonstrate the effectiveness of our method, delivering accurate predictions, transparency and interpretations, and accessible measures of uncertainty, while improving the scalability of BGNLM inference through on the one hand using a novel variational Bayes method, but, on the other hand, enabling the use of GPUs for computations.

• Papamarkou, Theodore; Skoularidou, Maria; Palla, Konstantina; Aitchison, Laurence; Arbel, Julyan & Dunson, David [Show all 25 contributors for this article] (2024). Position: Bayesian Deep Learning is Needed in the Age of Large-Scale AI. Proceedings of Machine Learning Research (PMLR). 235, p. 39556–39586. doi: 10.48550/arXiv.2402.00809. Full text in Research Archive Show summary

  In the current landscape of deep learning research, there is a predominant emphasis on achieving high predictive accuracy in supervised tasks involving large image and language datasets. However, a broader perspective reveals a multitude of overlooked metrics, tasks, and data types, such as uncertainty, active and continual learning, and scientific data, that demand attention. Bayesian deep learning (BDL) constitutes a promising avenue, offering advantages across these diverse settings. This paper posits that BDL can elevate the capabilities of deep learning. It revisits the strengths of BDL, acknowledges existing challenges, and highlights some exciting research avenues aimed at addressing these obstacles. Looking ahead, the discussion focuses on possible ways to combine large-scale foundation models with BDL to unlock their full potential.

• Ellingsen, Herman; Hubin, Aliaksandr; Remonato, Filippo & Sæbø, Solve (2024). Outlier Detection in Bayesian Neural Networks: Exploring Pre-activations and Predictive Entropy. Nordic Machine Intelligence (NMI). 4(1), p. 1–15. doi: 10.5617/nmi.11406. Full text in Research Archive Show summary

  Describing uncertainty is one of the major issues in modern deep learning. Artificial Intelligence models could be used with greater confidence by having solid methods for identifying and quantifying uncertainty. This article proposes two alternative methods for outlier detection in Bayesian Neural Networks used for classification tasks. This is done by looking for unusual pre-activation neuron values in the last layer of a Bayesian Neural Network. The proposed methods are compared to a baseline method for outlier detection, Predictive Entropy, on three datasets: a simulated dataset, the MNIST dataset, and the Breast Cancer Wisconsin dataset. In addition, we introduce a method for separating In-Between and Out-Of-Distribution outliers. The results indicate that the proposed methods' performance depends on the dataset type. In the case of a simple simulated dataset, we see that the proposed methods outperform the baseline method. The proposed method also outperforms the baseline method on the Breast Cancer Wisconsin dataset. However, when tested on the MNIST image dataset, we observed that the baseline is better than the proposed method. As expected, for all three datasets, an OR-combination of the two methods gives slightly better POWER score than the two methods separately, but this comes at the price of also obtaining a higher FDR score.

• Digranes, Nora; Hoeberg, Emma; Lervik, Andreas; Hubin, Aliaksandr; Nordgreen, Janicke & Haga, Henning Andreas (2024). Motor effects of fentanyl in isoflurane-anaesthetized pigs and the subsequent effect of ketanserin or naloxone. Veterinary Anaesthesia and Analgesia. ISSN 1467-2987. 51(5), p. 491–499. doi: 10.1016/j.vaa.2024.07.002. Full text in Research Archive
• Skaaret-Lund, Lars; Storvik, Geir Olve & Hubin, Aliaksandr (2024). Sparsifying Bayesian neural networks with latent binary variables and normalizing flows. Transactions on Machine Learning Research (TMLR). 2024. doi: 10.48550/arXiv.2305.03395. Full text in Research Archive Show summary

  Artificial neural networks are powerful machine learning methods used in many modern applications. A common issue is that they have millions or billions of parameters and tend to overfit. Bayesian neural networks (BNN) can improve on this since they incorporate parameter uncertainty. Latent binary Bayesian neural networks (LBBNN) further take into account structural uncertainty by allowing the weights to be turned on or off, enabling inference in the joint space of weights and structures. Mean-field variational inference is typically used for computation within such models. In this paper, we will consider two extensions of variational inference for the LBBNN: Firstly, by using the local reparametrization trick (LCRT), we improve computational efficiency. Secondly, and more importantly, by using normalizing flows on the variational posterior distribution of the LBBNN parameters, we learn a more flexible variational posterior than the mean field Gaussian. Experimental results on real data show that this improves predictive power compared to using mean field variational inference on the LBBNN method, while also obtaining sparser networks. We also perform a simulation study, where we consider variable selection in a logistic regression setting with highly correlated data, where the more flexible variational distribution improves results.

• Siciliani, Daphne; Hubin, Aliaksandr; Ruyter, Bente Synnøve; Chikwati, Elvis Mashingaidze; Thunes, Vebjørn Gunnarson & Valen, Elin Christine [Show all 10 contributors for this article] (2024). Effects of dietary fish to rapeseed oil ratio on steatosis symptoms in Atlantic salmon (Salmo salar L) of different sizes. Scientific Reports. 14(1), p. 1–18. doi: 10.1038/s41598-024-68434-3. Full text in Research Archive Show summary

  Choline is recognized as an essential nutrient for Atlantic salmon at all developmental stages. However, its dietary requirement is not well defined. Choline plays a critical role in lipid transport, and the clearest deficiency sign is intestinal steatosis. The present work, aiming to find whether lipid source and fish size may affect steatosis symptoms, was one of a series of studies conducted to identify which production-related conditions may influence choline requirement. Six choline-deficient diets were formulated varying in ratios of rapeseed oil to fish oil and fed to Atlantic salmon of 1.5 and 4.5 kg. After eight weeks, somatic characteristics were observed, and the severity of intestinal steatosis was assessed by histological, biochemical, and molecular analyses. Fatty acid composition in pyloric intestine, mesenteric tissue, and liver samples was also quantified. The increasing rapeseed oil level increased lipid digestibility markedly, enhancing lipid supply to the fish. Moreover, small fish consume

• Halstensen, Thor-David; Hardeland, Camilla; Ghanima, Waleed; Grøndahl, Vigdis Abrahamsen; Hubin, Aliaksandr & Tavoly, Mazdak (2024). Development and internal validation of a simple clinical score for the estimation of the probability of deep vein thrombosis in outpatient emergency department patients. Research and Practice in Thrombosis and Haemostasis (RPTH). 8(8). doi: 10.1016/j.rpth.2024.102608. Full text in Research Archive Show summary

  Background: Wells score comprises subjective elements, making physicians reluctant to use Wells score or cause them to use it incorrectly. Objectives: To develop and internally validate a prediction score that is objective and simple for evaluating suspected deep vein thrombosis (DVT), with a safety comparable with that of Wells score. Methods: We performed a post hoc analysis using data from the Ri-Schedule study (NCT02486445) involving suspected DVT patients at Østfold Hospital’s Emergency Department, Norway (2015-2018). Candidate variables were identified through bootstrapping technique, with a confirmed DVT diagnosis as the outcome variable. Sensitivity, specificity, negative predictive value (NPV), and positive predictive values (PPV) were estimated and compared with the 2-tier Wells score. Results: Among 1312 patients (median age, 64 years [IQR, 52-73]; 55% women), 19.9% were diagnosed with DVT. Exploration of 30 variables identified tenderness along deep veins and previous venous thromboembolism as significant predictors (selection frequency >60% in 1000 bootstrapping samples). The derived score categorized 450 patients with 0 items as unlikely to have DVT, of whom 8.0% were diagnosed with DVT, compared with 8.2% in DVT unlikely category according to Wells score. Compared with Wells score, the derived score demonstrated sensitivity of 86.2 (95% CI, 81.4-90.2) vs 80.1 (95% CI, 74.7-84.8), specificity of 39.4 (95% CI, 36.4-42.4) vs 55.3 (95% CI, 52.2-58.3), NPV of 92.0 (95% CI, 89.4-94.0) vs 91.8 (95% CI, 89.7-93.5), and PPV of 26.1 (95% CI, 24.8-27.5) vs 30.8 (95% CI, 28.9-32.8). When incorporating D-dimer cutoff of <0.5 mg/mL, the derived score had sensitivity of 99.6 (95% CI, 97.9-99.9), specificity of 16.1 (95% CI, 13.1-18.4), NPV of 99.4 (95% CI, 96.0-99.9), and PPV of 22.8 (95% CI, 22.3-23.3). Conclusion: The derived DVT score, with 2 objective variables, had a comparable safety with that of the Wells score. However, an external validation is mandated prior to clinical use.

• Hubin, Aliaksandr & Storvik, Geir Olve (2024). Sparse Bayesian Neural Networks: Bridging Model and Parameter Uncertainty through Scalable Variational Inference. Mathematics. 12(6). doi: 10.3390/math12060788. Full text in Research Archive Show summary

  Bayesian neural networks (BNNs) have recently regained a significant amount of attention in the deep learning community due to the development of scalable approximate Bayesian inference techniques. There are several advantages of using a Bayesian approach: parameter and prediction uncertainties become easily available, facilitating more rigorous statistical analysis. Furthermore, prior knowledge can be incorporated. However, the construction of scalable techniques that combine both structural and parameter uncertainty remains a challenge. In this paper, we apply the concept of model uncertainty as a framework for structural learning in BNNs and, hence, make inferences in the joint space of structures/models and parameters. Moreover, we suggest an adaptation of a scalable variational inference approach with reparametrization of marginal inclusion probabilities to incorporate the model space constraints. Experimental results on a range of benchmark datasets show that we obtain comparable accuracy results with the competing models, but based on methods that are much more sparse than ordinary BNNs.

• Hubin, Aliaksandr; Heinze, Georg & Bin, Riccardo De (2023). Fractional Polynomial Models as Special Cases of Bayesian Generalized Nonlinear Models. Fractal and Fractional. 7(9), p. 1–23. doi: 10.3390/fractalfract7090641. Full text in Research Archive Show summary

  We propose a framework for fitting multivariable fractional polynomial models as special cases of Bayesian generalized nonlinear models, applying an adapted version of the genetically modified mode jumping Markov chain Monte Carlo algorithm. The universality of the Bayesian generalized nonlinear models allows us to employ a Bayesian version of fractional polynomials in any supervised learning task, including regression, classification, and time-to-event data analysis. We show through a simulation study that our novel approach performs similarly to the classical frequentist multivariable fractional polynomials approach in terms of variable selection, identification of the true functional forms, and prediction ability, while naturally providing, in contrast to its frequentist version, a coherent inference framework. Real-data examples provide further evidence in favor of our approach and show its flexibility.

• Lachmann, Jon; Storvik, Geir Olve; Frommlet, Florian & Hubin, Aliaksandr (2022). A subsampling approach for Bayesian model selection. International Journal of Approximate Reasoning. ISSN 0888-613X. 151, p. 33–63. doi: 10.1016/j.ijar.2022.08.018. Full text in Research Archive Show summary

  It is common practice to use Laplace approximations to decrease the computational burden when computing the marginal likelihoods in Bayesian versions of generalised linear models (GLM). Marginal likelihoods combined with model priors are then used in different search algorithms to compute the posterior marginal probabilities of models and individual covariates. This allows performing Bayesian model selection and model averaging. For large sample sizes, even the Laplace approximation becomes computationally challenging because the optimisation routine involved needs to evaluate the likelihood on the full dataset in multiple iterations. As a consequence, the algorithm is not scalable for large datasets. To address this problem, we suggest using stochastic optimisation approaches, which only use a subsample of the data for each iteration. We combine stochastic optimisation with Markov chain Monte Carlo (MCMC) based methods for Bayesian model selection and provide some theoretical results on the convergence of the estimates for the resulting time-inhomogeneous MCMC. Finally, we report results from experiments illustrating the performance of the proposed algorithm.

• Hubin, Aliaksandr & Bin, Riccardo De (2022). On a genetically modified mode jumping MCMC approach for multivariate fractional polynomials. In Torelli, Nicola; BELLIO, RUGGERO & MUGGEO, VITO (Ed.), Proceedings of the 36th International Workshop on Statistical Modelling. EUT Edizioni Università di Trieste. ISSN 9788855113090. p. 478–483. Full text in Research Archive Show summary

  In this work, we suggest a framework to fit fractional polynomials based on the Bayesian Generalized Nonlinear Models (BGNLM, Hubin et al, 2021). A version of the Genetically Modified Mode Jumping Markov Chain Monte Carlo (GMJMCMC) algorithm (Hubin et al, 2020) is adopted. Preliminary simulation runs show promising results in terms of identifying the data generation mechanism: The suggested approach uniformly outperforms the exist- ing Bayesian fractional polynomial framework of Saban ́es Bov ́e and Held (2011) both in terms of Power and false discovery rate (FDR). Also, the performance is on par (somewhat better) with that of frequentist fractional polynomials of Royston and Altman (1994). Still, the results indicate that further work on the priors is likely to improve the performance even further.

• Gåsemyr, Jørund Inge & Hubin, Aliaksandr (2022). Prior distributions expressing ignorance about convex increasing failure rates. Scandinavian Journal of Statistics. ISSN 0303-6898. doi: 10.1111/sjos.12588. Full text in Research Archive Show summary

  This paper deals with the specification of probability distributions expressing ignorance concerning annual or otherwise discretized failure or mortality rates, when these rates can safely be assumed to be increasing and convex, but are completely unknown otherwise. Such distributions can be used as noninformative priors for Bayesian analysis of failure data. We demonstrate why a uniform distribution used in earlier work is unsatisfactory, especially from the point of view of insensitivity with respect to the time scale that is chosen for the problem at hand. We suggest alternative distributions based on Dirichlet distributed weights for the extreme points of relevant convex sets, and discuss which consequences a requirement for scale neutrality has for the choice of Dirichlet parameters.

• Hubin, Aliaksandr; Storvik, Geir & Frommlet, Florian (2021). Flexible Bayesian Nonlinear Model Configuration. The journal of artificial intelligence research. ISSN 1076-9757. 72, p. 901–942. doi: 10.1613/JAIR.1.13047. Full text in Research Archive Show summary

  Regression models are used in a wide range of applications providing a powerful scientific tool for researchers from different fields. Linear, or simple parametric, models are often not sufficient to describe complex relationships between input variables and a response. Such relationships can be better described through flexible approaches such as neural networks, but this results in less interpretable models and potential overfitting. Alternatively, specific parametric nonlinear functions can be used, but the specification of such functions is in general complicated. In this paper, we introduce a flexible approach for the construction and selection of highly flexible nonlinear parametric regression models. Nonlinear features are generated hierarchically, similarly to deep learning, but have additional flexibility on the possible types of features to be considered. This flexibility, combined with variable selection, allows us to find a small set of important features and thereby more interpretable models. Within the space of possible functions, a Bayesian approach, introducing priors for functions based on their complexity, is considered. A genetically modified mode jumping Markov chain Monte Carlo algorithm is adopted to perform Bayesian inference and estimate posterior probabilities for model averaging. In various applications, we illustrate how our approach is used to obtain meaningful nonlinear models. Additionally, we compare its predictive performance with several machine learning algorithms.

• Hubin, Aliaksandr; Frommlet, Florian & Storvik, Geir Olve (2021). Reversible genetically modified mode jumping MCMC. In Makridis, Andreas; Milienos, Fotios; Papastamoulis, Panagiotis; Parpoula, Christina & Rakitzis, Athanasios (Ed.), 22nd European Young Statisticians Meeting – Proceedings. Department of Psychology & Department of Sociology, School of Social Science, Panteion University of Social and Political Sciences. ISSN 9789607943231. p. 35–40. Full text in Research Archive Show summary

  In this paper, we introduce a reversible version of a genetically modified mode jumping Markov chain Monte Carlo algorithm (GMJMCMC) for inference on posterior model probabilities in complex model spaces, where the number of explanatory variables is prohibitively large for classical Markov Chain Monte Carlo methods. Unlike the earlier proposed GMJMCMC algorithm, the introduced algorithm is a proper MCMC and its limiting distribution corresponds to the posterior marginal model probabilities in the explored model space under reasonable regularity conditions.

• Lison, Pierre; Barnes, Jeremy & Hubin, Aliaksandr (2021). skweak: Weak Supervision Made Easy for NLP. In Ji, Heng & Park, Jong (Ed.), Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing: System Demonstrations. Association for Computational Linguistics (ACL). ISSN 9781954085565. p. 337–346. doi: 10.18653/v1/2021.acl-demo.40. Full text in Research Archive Show summary

  We present skweak, a versatile, Python-based software toolkit enabling NLP developers to apply weak supervision to a wide range of NLP tasks. Weak supervision is an emerging machine learning paradigm based on a simple idea: instead of labelling data points by hand, we use labelling functions derived from domain knowledge to automatically obtain annotations for a given dataset. The resulting labels are then aggregated with a generative model that estimates the accuracy (and possible confusions) of each labelling function. The skweak toolkit makes it easy to implement a large spectrum of labelling functions (such as heuristics, gazetteers, neural models or linguistic constraints) on text data, apply them on a corpus, and aggregate their results in a fully unsupervised fashion. skweak is especially designed to facilitate the use of weak supervision for NLP tasks such as text classification and sequence labelling. We illustrate the use of skweak for NER and sentiment analysis. skweak is released under an open-source license and is available at https://github.com/NorskRegnesentral/skweak

• Hubin, Aliaksandr; Storvik, Geir Olve & Frommlet, Florian (2020). Rejoinder for the discussion of the paper "A Novel Algorithmic Approach to Bayesian Logic Regression". Bayesian Analysis. ISSN 1936-0975. 15(1), p. 312–333. doi: 10.1214/18-ba1141. Full text in Research Archive Show summary

  Logic regression was developed more than a decade ago as a tool to construct predictors from Boolean combinations of binary covariates. It has been mainly used to model epistatic effects in genetic association studies, which is very appealing due to the intuitive interpretation of logic expressions to describe the interaction between genetic variations. Nevertheless logic regression has (partly due to computational challenges) remained less well known than other approaches to epistatic association mapping. Here we will adapt an advanced evolutionary algorithm called GMJMCMC (Genetically modified Mode Jumping Markov Chain Monte Carlo) to perform Bayesian model selection in the space of logic regression models. After describing the algorithmic details of GMJMCMC we perform a comprehensive simulation study that illustrates its performance given logic regression terms of various complexity. Specifically GMJMCMC is shown to be able to identify three-way and even four-way interactions with relatively large power, a level of complexity which has not been achieved by previous implementations of logic regression. We apply GMJMCMC to reanalyze QTL (quantitative trait locus) mapping data for Recombinant Inbred Lines in Arabidopsis thaliana and from a backcross population in Drosophila where we identify several interesting epistatic effects. The method is implemented in an R package which is available on github.

• Hubin, Aliaksandr; Storvik, Geir Olve; Grini, Paul Eivind & Butenko, Melinka Alonso (2020). A Bayesian binomial regression model with latent gaussian processes for modelling DNA methylation. Austrian Journal of Statistics. ISSN 1026-597X. 49(4), p. 46–56. doi: 10.17713/ajs.v49i4.1124. Full text in Research Archive Show summary

  Epigenetic observations are represented by the total number of reads from a given pool of cells and the number of methylated reads, making it reasonable to model this data by a binomial distribution. There are numerous factors that can influence the probability of success in a particular region. Moreover, there is a strong spatial (alongside the genome) dependence of these probabilities. We incorporate dependence on the covariates and the spatial dependence of the methylation probability for observations from a pool of cells by means of a binomial regression model with a latent Gaussian field and a logit link function. We apply a Bayesian approach including prior specifications on model configurations. We run a mode jumping Markov chain Monte Carlo algorithm (MJMCMC) across different choices of covariates in order to obtain the joint posterior distribution of parameters and models. This also allows finding the best set of covariates to model methylation probability within the genomic region of interest and individual marginal inclusion probabilities of the covariates.

• Lison, Pierre; Barnes, Jeremy; Hubin, Aliaksandr & Touileb, Samia (2020). Named Entity Recognition without Labelled Data: A Weak Supervision Approach. In Jurafsky, Dan; Chai, Joyce; Schluter, Natalie & Tetreault, Joel (Ed.), Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics (ACL). ISSN 9781952148255. p. 1518–1533. Full text in Research Archive Show summary

  Named Entity Recognition (NER) performance often degrades rapidly when applied to target domains that differ from the texts observed during training. When in-domain labelled data is available, transfer learning techniques can be used to adapt existing NER models to the target domain. But what should one do when there is no hand-labelled data for the target domain? This paper presents a simple but powerful approach to learn NER models in the absence of labelled data through weak supervision. The approach relies on a broad spectrum of labelling functions to automatically annotate texts from the target domain. These annotations are then merged together using a hidden Markov model which captures the varying accuracies and confusions of the labelling functions. A sequence labelling model can finally be trained on the basis of this unified annotation. We evaluate the approach on two English datasets (CoNLL 2003 and news articles from Reuters and Bloomberg) and demonstrate an improvement of about 7 percentage points in entity-level F1 scores compared to an out-of-domain neural NER model.

• Hubin, Aliaksandr (2019). An adaptive simulated annealing EM algorithm for inference on non-homogeneous hidden Markov models, ACM International Conference Proceeding Series (ICPS): AIIPCC '19: Proceedings of the International Conference on Artificial Intelligence, Information Processing and Cloud Computing. Association for Computing Machinery (ACM). ISSN 9781450376334. p. 1–9. doi: 10.1145/3371425.3371641. Full text in Research Archive Show summary

  This conference is an annual forum for worldwide experts, academicians, engineers and researchers to share their new findings in related issues, explore future study direction and also promote global collaboration.

• Hubin, Aliaksandr; Storvik, Geir Olve; Grini, Paul Eivind & Butenko, Melinka Alonso (2019). Bayesian binomial regression model with a latent Gaussian field for analysis of epigenetic data. In Kharin, Y & Filzmoser, Peter (Ed.), Proceedings of Computer Data Analysis and Modeling: Stochastics and Data Science 2019. Belarusian State University Press. ISSN 9789855668115. p. 167–171. Full text in Research Archive
• Hubin, Aliaksandr; Storvik, Geir Olve & Frommlet, Florian (2018). A Novel Algorithmic Approach to Bayesian Logic Regression. Bayesian Analysis. ISSN 1936-0975. 15(1), p. 263–311. doi: 10.1214/18-BA1141. Full text in Research Archive Show summary

  Logic regression was developed more than a decade ago as a tool to construct predictors from Boolean combinations of binary covariates. It has been mainly used to model epistatic effects in genetic association studies, which is very appealing due to the intuitive interpretation of logic expressions to describe the interaction between genetic variations. Nevertheless logic regression has (partly due to computational challenges) remained less well known than other approaches to epistatic association mapping. Here we will adapt an advanced evolutionary algorithm called GMJMCMC (Genetically modified Mode Jumping Markov Chain Monte Carlo) to perform Bayesian model selection in the space of logic regression models. After describing the algorithmic details of GMJMCMC we perform a comprehensive simulation study that illustrates its performance given logic regression terms of various complexity. Specifically GMJMCMC is shown to be able to identify three-way and even four-way interactions with relatively large power, a level of complexity which has not been achieved by previous implementations of logic regression. We apply GMJMCMC to reanalyze QTL (quantitative trait locus) mapping data for Recombinant Inbred Lines in Arabidopsis thaliana and from a backcross population in Drosophila where we identify several interesting epistatic effects. The method is implemented in an R package which is available on github.

• Hubin, Aliaksandr & Storvik, Geir Olve (2018). Mode jumping MCMC for Bayesian variable selection in GLMM. Computational Statistics & Data Analysis. ISSN 0167-9473. 127, p. 281–297. doi: 10.1016/j.csda.2018.05.020. Full text in Research Archive Show summary

  Generalized linear mixed models (GLMM) are used for inference and prediction in a wide range of different applications providing a powerful scientific tool. An increasing number of sources of data are becoming available, introducing a variety of candidate explanatory variables for these models. Selection of an optimal combination of variables is thus becoming crucial. In a Bayesian setting, the posterior distribution of the models, based on the observed data, can be viewed as a relevant measure for the model evidence. The number of possible models increases exponentially in the number of candidate variables. Moreover, the space of models has numerous local extrema in terms of posterior model probabilities. To resolve these issues a novel MCMC algorithm for the search through the model space via efficient mode jumping for GLMMs is introduced. The algorithm is based on that marginal likelihoods can be efficiently calculated within each model. It is recommended that either exact expressions or precise approximations of marginal likelihoods are applied. The suggested algorithm is applied to simulated data, the famous U.S. crime data, protein activity data and epigenetic data and is compared to several existing approaches.

• Hubin, Aliaksandr & Storvik, Geir Olve (2016). On Mode Jumping in MCMC for Bayesian Variable Selection within GLMM. In Aivazian, S; Filzmoser, Peter & Kharin, Y (Ed.), COMPUTER DATA ANALYSIS AND MODELING. Theoretical and Applied Stochastics. Proceedings of the XI International Conference.. Belarusian State University. ISSN 9789855533666. p. 275–278. doi: 10.1016/j.csda.2018.05.020. Full text in Research Archive Show summary

  Generalized linear mixed models (GLMM) are addressed for inference and prediction in a wide range of different applications providing a powerful scientific tool for the researchers and analysts coming from different fields. At the same time more sources of data are becoming available introducing a variety of hypothetical explanatory variables for these models to be considered. Estimation of posterior model probabilities and selection of an optimal model is thus becoming crucial. We suggest a novel mode jumping MCMC procedure for Bayesian model averaging and model selection in GLMM.

View all works in NVA

• Bin, Riccardo De; Salucci, Clara Bertinelli & Hubin, Aliaksandr (2026). Proceedings of the 40th International Workshop on Statistical Modelling 28.06 – 03.07.2026, Oslo, Norway. International Workshop on Statistical Modelling. ISBN 9788255325000. 655 p. Full text in Research Archive

View all works in NVA

• Hubin, Aliaksandr (2026). Bayesian Model Selection and Averaging with Latent Binary Bayesian Neural Networks. doi: https:/ww2.amstat.org/meetings/jsm/2026/onlineprogram/abstract.cfm?sid=6353&tid=6357. Full text in Research Archive Show summary

  Artificial neural networks (ANNs) yield accurate predictions but are often over-parameterized and hard to interpret. Bayesian neural networks (BNNs) mitigate uncertainty by treating weights probabilistically, while latent binary BNNs (LBBNNs) handle structural uncertainty via weight sparsification. We extend LBBNNs by allowing covariates to skip to any subsequent layer or be excluded entirely, yielding an input-skip LBBNN (ISLaB) that learns parsimonious structures-from linear to intercept-only models-when suitable. ISLaB achieves extreme compression (over 99–99.9% weight reduction) with minimal loss in accuracy; on MNIST, it attains 97% accuracy and excellent calibration using only 935 weights. Moreover, it identifies relevant covariates, captures nonlinearity, and introduces theoretically grounded, intrinsic local and global model explanations without post hoc tools. The methods are available in the open-source R package LBBNN on CRAN.

• Vittersø, Sigurd Myhre & Hubin, Aliaksandr (2026). BGET: Bayesian Genetic Ensemble of Trees. doi: https:/www.mn.uio.no/math/english/research/groups/statistics-data-science/events/conferences/iwsm2026/programme_final.pdf. Full text in Research Archive Show summary

  We embed classification and regression trees in the Bayesian generalized nonlinear model framework, defining a Bayesian genetic ensemble of trees with complexity-penalizing priors on tree structures. Inference is performed via genetically modified mode jumping MCMC, which explores large spaces of tree ensembles using posterior inclusion probabilities to guide genetic operators. The approach is evaluated on the Abalone and Boston housing data and performs on par with relevant related baselines

• Hubin, Aliaksandr; Skaaret-Lund, Lars; Sæbø, Solve & Høyheim, Eirik (2026). Explainable Bayesian deep learning through input-skip Latent Binary Bayesian Neural Networks. doi: https:/www.mn.uio.no/math/english/research/groups/statistics-data-science/events/seminars/spring_2026/aliaksandr-hubin.html. Full text in Research Archive Show summary

  Modeling natural phenomena with artificial neural networks (ANNs) often provides highly accurate predictions. However, ANNs often suffer from over-parameterization, complicating interpretation and raising uncertainty issues. Bayesian neural networks (BNNs) address the latter by representing weights as probability distributions, allowing for predictive uncertainty evaluation. Latent binary Bayesian neural networks (LBBNNs) further handle structural uncertainty and sparsify models by removing redundant weights. This article advances LBBNNs by enabling covariates to skip to any succeeding layer or be excluded, simplifying networks and clarifying input impacts on predictions. This further allows us to learn simpler structures (e.g., linear or even constant intercept only models) when appropriate. Furthermore, the input-skip LBBNN (ISLaB) approach reduces network density significantly compared to standard LBBNNs, achieving over 99% reduction for small networks and over 99.9% for larger ones, while still maintaining high predictive accuracy and uncertainty quantification. For example, on MNIST, we reached 97% accuracy and great calibration with just 935 weights, reaching state-of-the-art for compression of neural networks. Furthermore, the proposed method accurately identifies the true covariates and adjusts for system non-linearity. The main contribution is the introduction of active paths, enhancing directly designed global and local explanations within the LBBNN framework, that have theoretical guarantees and do not require post hoc external tools for explanations.

• Hubin, Aliaksandr; Krogdahl, Åshild; Løkka, Guro & Kortner, Trond M. (2026). Bayesian Fractional Polynomials for Optimal Dosage Estimation. doi: https:/www.ibc2026.org/events/csprogram26. Full text in Research Archive Show summary

  The problem of optimal dosage estimation arises in diverse scientific domains, from pharmacology and toxicology to aquaculture and environmental studies. Statistical modeling of nonlinear dose–response relationships is essential to quantify biological effects and determine response-maximizing levels. This paper introduces a flexible Bayesian fractional polynomial (BFP) framework for modeling such relationships, allowing for model uncertainty quantification and robust prediction through Bayesian model averaging. Extensive simulation results demonstrate that the proposed BFP approach yields accurate estimation of optimal dose levels, outperforming standard benchmarks. The approach is further demonstrated on real data from fish nutrient requirement experiments.

• Hubin, Aliaksandr; Krogdahl, Åshild; Løkka, Guro & Kortner, Trond M. (2026). Bayesian Fractional Polynomials for Optimal Dosage Estimation. Full text in Research Archive Show summary

  The problem of optimal dosage estimation arises in diverse scientific domains, from pharmacology and toxicology to aquaculture and environmental studies. Statistical modeling of nonlinear dose–response relationships is essential to quantify biological effects and determine response-maximizing levels. This paper introduces a flexible Bayesian fractional polynomial (BFP) framework for modeling such relationships, allowing for model uncertainty quantification and robust prediction through Bayesian model averaging. Extensive simulation results demonstrate that the proposed BFP approach yields accurate estimation of optimal dose levels, outperforming standard benchmarks. The approach is further demonstrated on real data from fish nutrient requirement experiments.

• Haukeland-Parker, Stacey; Jervan, Øyvind; Ghanima, Waleed; Spruit, Martijn A.; Hubin, Aliaksandr & Gleditsch, Jostein [Show all 10 contributors for this article] (2026). The effect of exercise-based rehabilitation on objectively measured daily physical activity levels in patients with persistent dyspnoea after pulmonary embolism. Full text in Research Archive
• Hubin, Aliaksandr; Heinze, Georg; Kammer, Michael & Nold, Mariana (2025). Bayesian and Frequentist Workows in Practice, Part 2: Insights From the Kepler Task and Bacteremia Task Reanalysis with a Focus on Bayesian Model Averaging. doi: https:/roes2025.at/data/ROeS_conference_book_v1.3.pdf. Full text in Research Archive Show summary

  This presentation is a part of the session organized by TG2 of the STRATOS initiative. The talk will highlight a Bayesian analysis of two distinct datasets: open exoplanet catalogue data and bacteremia data. I will rst discuss a Bayesian approach for reanalysis of predicting blood culture outcomes in bacteremia data with substantial missingness. This task was originally dened in a frequentist paradignm, which was presented by the previous speaker in the session. For the Bayesian approach I will use the Bayesian Generalized Nonlinear Model (BGNLM), which is a generalization of symbolic regression allowing for exible interaction and non-linear projection based nonlinear eects, available in FBMS R package, where imputation is seamlessly integrated within model tting, enabling a parsimonious solution. I will be validating predictive performance through .632+ bootstrap estimates. The Kepler task was originally dened within the Bayesian paradigm in our initiative. It deals with data from an open exoplanet catalogue database. The task involves modeling planetary semimajor axes using Bayesian frameworks, exploring functional forms ranging from simple linear predictors to BGNLMs. The goal is to recover the true functional form of the law through Bayesian model averaging to compute the median probability model. Highly accurate predictions are left as a secondary goal. The next speaker will present the frequentist reanalysis for the Kepler task. Lastly, methodological challenges, particularly in Bayesian model averaging, are also discussed in this talk, while oering insights into model uncertainty and robust inference across applications, convergence may be complicated in nonlinear models with large correlations between the predictors, when the number of predictors is huge. Some frequentist approaches to handle model uncertainty will also be briey discussed.

• Mathisen, Therese Fostervold; Hubin, Aliaksandr; Mountjoy, Margo & Sundgot-Borgen, Jorunn (2025). Safety of Heavy Resistance Exercise During Pregnancy and Postpartum: A Controlled, Prospective Cohort Study. doi: https:/scs.academy/scs-annual-meeting/. Full text in Research Archive Show summary

  Objective: To address the knowledge gap on the influence of heavy free-weight resistance exercise on the health of the pregnant athlete and infant. Design: This is a prospective, controlled cohort study with two groups of pregnant women; those engaging in mainly heavy free-weight resistance exercise (REX, n = 33) and those regularly physically active (REF, n = 20). Data originate from eight questionnaires answered prospectively throughout pregnancy and 12 months postpartum. Variables cover exercise routines, pregnancy symptoms and complications, fetal growth, delivery, and mother and infant’s health. Results: Bayesian logistic regression with random and fixed effects identified group membership as a significant predictor of proteinuria, with REX having higher exposure (posterior inclusion probability [PIP] = 1.00; β1 = 197.07, 95% credible interval [CrI] [163.05, 231.08]). Further, Bayesian logistic regression with random and fixed effects confirmed a higher risk of urinary incontinence in the REF group during pregnancy (PIP = 0.98; β1 = −1.17, 95% CrI [−1.89, −0.46]). Gas incontinence was also found more common in the REF group during pregnancy (PIP = 1.00; β1 = −7.62, 95% CrI [−8.68, −6.57]) and postpartum (PIP = 0.91; β1 = −49.39, 95% CrI [−53.57, −45.43]). There were no significant differences in the postpartum period for fecal and urinary incontinence, injuries, infant health outcomes (all PIP < 0.01), or need for assisted delivery (PIP = 0.33). Conclusion: Female athletes experienced in heavy resistance exercise, who continue training during pregnancy and postpartum, do not face increased pregnancy complications. These findings provide evidence supporting safety of such exercise. Yet we highlight the need for a confirmatory study under full clinical supervision.

• Hubin, Aliaksandr (2025). Bayesian Analysis of Kepler's Third Law Discovery and Bacteremia Classification. Full text in Research Archive
• Hubin, Aliaksandr (2025). Flexible Bayesian Model Selection and Model Averaging. Full text in Research Archive Show summary

  In this talk, I will introduce a method for model selection and averaging in Bayesian generalized nonlinear regression [1]. Our approach generates nonlinear features hierarchically—similar to deep learning but with even greater flexibility—encompassing most existing statistical learning models as special cases while allowing them to be combined within a single Bayesian model averaging (BMA) framework. Model priors are designed to penalize the complexity of nonlinear features and encourage parsimony. As a result, we often obtain highly interpretable nonlinear models that recover data-generating processes for physical laws [1], logical [2] and fractional [3] polynomials, or governing equations in Sparse Identification of Nonlinear Dynamical Systems (SINDy). When no true data-generating model is feasible, BMA still accounts for model uncertainty, ensuring robust predictive performance. For Bayesian inference and posterior model probability estimation, we employ a genetically modified mode-jumping Markov chain Monte Carlo (GMJMCMC) algorithm [1,2]. To scale to larger datasets, we also provide subsampling-based MCMC [4] and an Evolutionary Variational Bayes method [5]. I will demonstrate this approach through various applications, accompanied by hands-on examples using the FBMS R package (CRAN link). Examples will include recovering physical laws and SINDy solutions, logic and fractional polynomial regressions, extensions to latent Gaussian nonlinear models, and predictive modeling tasks. References: [1] Hubin, Aliaksandr; Storvik, Geir & Frommlet, Florian (2021). Flexible Bayesian Nonlinear Model Configuration. The Journal of Artificial Intelligence Research. ISSN 1076-9757. 72, p. 901–942. [2] Hubin, Aliaksandr; Storvik, Geir Olve & Frommlet, Florian (2018). A Novel Algorithmic Approach to Bayesian Logic Regression (with Discussion). Bayesian Analysis. ISSN 1936-0975. 15(1), p. 263–311. doi: 10.1214/18-BA1141 [3] Hubin, Aliaksandr; Heinze, Georg & De Bin, Riccardo (2023). Fractional Polynomial Models as Special Cases of Bayesian Generalized Nonlinear Models. Fractal and Fractional. 7(9), p. 1–23. doi: 10.3390/fractalfract7090641 [4] Lachmann, Jon; Storvik, Geir Olve; Frommlet, Florian & Hubin, Aliaksandr (2022). A subsampling approach for Bayesian model selection. International Journal of Approximate Reasoning. ISSN 0888-613X. 151, p. 33–63. doi: 10.1016/j.ijar.2022.08.018 [5] Hauglie Sommerfelt, Philip Sebastian & Hubin, Aliaksandr (2024). Evolutionary variational inference for Bayesian generalized nonlinear models. Neural Computing & Applications. ISSN 0941-0643. p. 1–18. doi: 10.1007/s00521-024-10349-1

• Hubin, Aliaksandr (2025). Bayesian Generalized Nonlinear Models Offer Basis Free SINDy With Model Uncertainty. Full text in Research Archive Show summary

  Sparse Identification of Nonlinear Dynamics (SINDy) has become a standard methodology for inferring governing equations of dynamical systems from observed data using statistical modeling. However, classical SINDy approaches rely on predefined libraries of candidate functions to model nonlinearities, which limits flexibility and excludes robust uncertainty quantification. This paper proposes Bayesian Generalized Nonlinear Models (BGNLMs) as a principled alternative for more flexible statistical modeling. BGNLMs employ spike-and-slab priors combined with binary inclusion indicators to automatically discover relevant nonlinearities without predefined basis functions. Moreover, BGNLMs quantify uncertainty in selected bases and final model predictions, enabling robust exploration of the model space. In this paper, the BGNLM framework is applied to several three-dimensional (3D) SINDy problems.

• Hubin, Aliaksandr (2024). Structure learning in Bayesian neural networks. Full text in Research Archive
• Hubin, Aliaksandr (2024). A subsampling MCMC approach for Bayesian model selection and model averaging. Full text in Research Archive
• Hubin, Aliaksandr; Storvik, Geir Olve; Frommlet, Florian & Lachmann, Jon (2024). A subsampling MCMC approach for Bayesian model selection and model averaging. Full text in Research Archive
• Mathisen, Therese Fostervold; Sundgot-Borgen, Jorunn; Hubin, Aliaksandr & Mountjoy, Margo (2024). Kan vi anbefale kvinnelige utøvere å trene tung frivekts-styrketrening under graviditet og barseltid? Full text in Research Archive
• Denault, William R. P.; Hubin, Aliaksandr; Vitelli, Valeria & Richardson, Sylvia (2024). Contributed Discussion to Sparse Bayesian Factor Analysis When the Number of Factors Is Unknown. doi: 10.1214/24-BA1423. Full text in Research Archive Show summary

  There has been increased research interest in the subfield of sparse Bayesian factor analysis with shrinkage priors, which achieve additional sparsity beyond the natural parsimonity of factor models. In this spirit, we estimate the number of common factors in the widely applied sparse latent factor model with spike-and-slab priors on the factor loadings matrix. Our framework leads to a natural, efficient and simultaneous coupling of model estimation and selection on one hand and model identification and rank estimation (number of factors) on the other hand. More precisely, by embedding the unordered generalised lower triangular loadings representation into overfitting sparse factor modelling, we obtain posterior summaries regarding factor loadings, common factors as well as the factor dimension via postprocessing draws from our efficient and customized Markov chain Monte Carlo scheme.

• Hubin, Aliaksandr (2023). Genetically modified mode jumping MCMC approach for Bayesian multivariate fractional polynomials. Full text in Research Archive
• Mathisen, Therese Fostervold; Sundgot-Borgen, Jorunn & Hubin, Aliaksandr (2023). Tung styrketrening i svangerskap og barseltid – helt greit? Preliminære data fra en prospektiv, kontrollert kohortstudie. Full text in Research Archive
• Lachmann, Jon; Storvik, Geir Olve; Frommlet, Florian & Hubin, Aliaksandr (2022). A subsampling approach for Bayesian model selection. Full text in Research Archive
• Hubin, Aliaksandr & Storvik, Geir Olve (2022). Variational Bayes for inference on model and parameter uncertainty in Bayesian neural networks. Full text in Research Archive Show summary

  Bayesian neural networks (BNNs) have recently regained a significant amount of attention in the deep learning community due to the development of scalable approximate Bayesian inference techniques [1]. There are several advantages of using a Bayesian approach: Parameter and prediction uncertainties become easily available, facilitating rigorous statistical analysis. Furthermore, prior knowledge can be incorporated. However, so far there have been no scalable techniques capable of combining both model (structural) and parameter uncertainty. In the presented piece of research [2] we introduce the concept of model uncertainty in BNNs and hence make inference in the joint space of models and parameters. Moreover, we suggest an adaptation of a scalable variational inference approach with reparametrization of marginal inclusion probabilities to incorporate the model space constraints. Experimental results on a range of benchmark data sets show that we obtain comparable accuracy results with the competing models, but based on methods that are much more sparse than ordinary BNNs. This is particularly the case in model selection settings, but also within a Bayesian model averaging setting a considerable sparsification is achieved. As expected, model uncertainties give higher, but more reliable uncertainty measures. References: [1] Blundell, C., Cornebise, J., Kavukcuoglu, K., and Wierstra, D. (2015). Weight uncertainty in neural networks. International Conference on Machine Learning, 1613 - 1622. [2] Hubin, A., Storvik, G. (2019). Combining model and parameter uncertainty in Bayesian neural networks, arXiv preprint arXiv:1903.07594.

• Hubin, Aliaksandr & Bin, Riccardo De (2022). On a genetically modified mode jumping MCMC approach for multivariate fractional polynomials. Full text in Research Archive
• Hubin, Aliaksandr & Storvik, Geir Olve (2022). Variational Inference for Bayesian Neural Networks under Model and Parameter Uncertainty. Full text in Research Archive
• Hubin, Aliaksandr & Bin, Riccardo De (2022). On a genetically modified mode jumping MCMC approach for multivariate fractional polynomials. Full text in Research Archive
• Hubin, Aliaksandr & Storvik, Geir Olve (2021). Variational Inference for Bayesian Neural Networks under Model and Parameter Uncertainty. Full text in Research Archive Show summary

  Bayesian neural networks (BNNs) have recently regained a significant amount of attention in the deep learning community due to the development of scalable approximate Bayesian inference techniques. There are several advantages of using a Bayesian approach: Parameter and prediction uncertainties become easily available, facilitating rigorous statistical analysis. Furthermore, prior knowledge can be incorporated. However so far there have been no scalable techniques capable of combining both model (structural) and parameter uncertainty. In this paper we introduce the concept of model uncertainty in BNNs and hence make inference in the joint space of models and parameters. Moreover, we suggest an adaptation of a scalable variational inference approach with reparametrization of marginal inclusion probabilities to incorporate the model space constraints. Experimental results on a range of benchmark data sets show that we obtain comparable accuracy results with the competing models, but based on methods that are much more sparse than ordinary BNNs. This is particularly the case in model selection settings, but also within a Bayesian model averaging setting a considerable sparsification is achieved. As expected, model uncertainties give higher, but more reliable uncertainty measures.

• Hubin, Aliaksandr; Frommlet, Florian & Storvik, Geir Olve (2021). Reversible Genetically Modified MCMCs. Full text in Research Archive Show summary

  In this work, we introduce a reversible version of a genetically modified Markov chain Monte Carlo algorithm (GMJMCMC) for inference on posterior model probabilities in complex functional spaces, where the number of explanatory variables or functions of explanatory variables is prohibitively large for simple Markov Chain Monte Carlo methods. A genetically modified Markov chain Monte Carlo algorithm (GMJMCMC) was introduced in [5, 4, 2] for Bayesian model selection/averaging problems when the total number of function of covariates is prohibitively large. More specifically, these applications include GWAS studies with Bayesian generalized linear models [2] as well as Bayesian logic regressions [5] and Bayesian generalized nonlinear models [4]. If its regularity conditions are met, GMJMCMC algorithm can asymptotically explore all models in the defined model spaces. At the same time, GMJMCMC is not a proper MCMC in a sense that its limiting distribution does not correspond to marginal posterior model probabilities and thus only renormalized estimates of these probabilities [3, 1] can be obtained. Unlike the standard GMJMCMC algorithm, the introduced algorithm is a proper MCMC and its limiting distribution corresponds to posterior marginal model probabilities in the explored model spaces under reasonable regularity conditions.

• Hubin, Aliaksandr & Storvik, Geir Olve (2021). Variational Bayes for inference on model and parameter uncertainty in Bayesian neural networks. Full text in Research Archive Show summary

  Bayesian neural networks (BNNs) have recently regained a significant amount of attention in the deep learning community due to the development of scalable approximate Bayesian inference techniques [1]. There are several advantages of using a Bayesian approach: Parameter and prediction uncertainties become easily available, facilitating rigorous statistical analysis. Furthermore, prior knowledge can be incorporated. However so far there have been no scalable techniques capable of combining both model (structural) and parameter uncertainty. In the presented piece of research [2] we introduce the concept of model uncertainty in BNNs and hence make inference in the joint space of models and parameters. Moreover, we suggest an adaptation of a scalable variational inference approach with reparametrization of marginal inclusion probabilities to incorporate the model space constraints. Experimental results on a range of benchmark data sets show that we obtain comparable accuracy results with the competing models, but based on methods that are much more sparse than ordinary BNNs. This is particularly the case in model selection settings, but also within a Bayesian model averaging setting a considerable sparsification is achieved. As expected, model uncertainties give higher, but more reliable uncertainty measures.

• Lison, Pierre; Barnes, Jeremy Claude & Hubin, Aliaksandr (2021). skweak: weak supervision made easy for NLP. Full text in Research Archive Show summary

  We present skweak, a versatile, Python-based software toolkit enabling NLP developers to apply weak supervision to a wide range of NLP tasks. Weak supervision is an emerging machine learning paradigm based on a simple idea: instead of labelling data points by hand, we use labelling functions derived from domain knowledge to automatically obtain annotations for a given dataset. The resulting labels are then aggregated with a generative model that estimates the accuracy (and possible confusions) of each labelling function. The skweak toolkit makes it easy to implement a large spectrum of labelling functions (such as heuristics, gazetteers, neural models or linguistic constraints) on text data, apply them on a corpus, and aggregate their results in a fully unsupervised fashion. skweak is especially designed to facilitate the use of weak supervision for NLP tasks such as text classification and sequence labelling. We illustrate the use of skweak for NER and sentiment analysis. skweak is released under an open-source license and is available at https://github.com/NorskRegnesentral/skweak

• Hubin, Aliaksandr; Storvik, Geir Olve & Frommlet, Florian (2020). A novel algorithmic approach to Bayesian Logic Regression. Full text in Research Archive Show summary

  Logic regression was developed more than a decade ago as a tool to construct predictors from Boolean combinations of binary covariates. It has been mainly used to model epistatic effects in genetic association studies, which is very appealing due to the intuitive interpretation of logic expressions to describe the interaction between genetic variations. Nevertheless logic regression has (partly due to computational challenges) remained less well known than other approaches to epistatic association mapping. Here we will adapt an advanced evolutionary algorithm called GMJMCMC (Genetically modified Mode Jumping Markov Chain Monte Carlo) to perform Bayesian model selection in the space of logic regression models. After describing the algorithmic details of GMJMCMC we perform a comprehensive simulation study that illustrates its performance given logic regression terms of various complexity. Specifically GMJMCMC is shown to be able to identify three-way and even four-way interactions with relatively large power, a level of complexity which has not been achieved by previous implementations of logic regression. We apply GMJMCMC to reanalyze QTL (quantitative trait locus) mapping data for Recombinant Inbred Lines in Arabidopsis thaliana and from a backcross population in Drosophila where we identify several interesting epistatic effects. The method is implemented in an R package which is available on github.

• Hubin, Aliaksandr & Storvik, Geir Olve (2019). Combining Model and Parameter Uncertainty in Bayesian Neural Networks. Full text in Research Archive Show summary

  Bayesian neural networks (BNNs) have recently regained a significant amount of attention in the deep learning community due to the development of scalable approximate Bayesian inference techniques. There are several advantages of using Bayesian approach: Parameter and prediction uncertainty become easily available, facilitating rigid statistical analysis. Furthermore, prior knowledge can be incorporated. However so far there have been no scalable techniques capable of combining both model (structural) and parameter uncertainty. In this paper we introduce the concept of model uncertainty in BNNs and hence make inference in the joint space of models and parameters. Moreover, we suggest an adaptation of a scalable variational inference approach with reparametrization of marginal inclusion probabilities to incorporate the model space constraints. Finally, we show that incorporating model uncertainty via Bayesian model averaging and Bayesian model selection allows to drastically sparsify the structure of BNNs without significant loss of predictive power.

• Hubin, Aliaksandr & Storvik, Geir Olve (2019). Combining Model and Parameter Uncertainty in Bayesian Neural Networks. Full text in Research Archive
• Hubin, Aliaksandr (2019). Using node embedding to obtain information from network based transactions data in a bank. Full text in Research Archive
• Storvik, Geir Olve & Hubin, Aliaksandr (2019). Combining model and parameter uncertainty in Bayesian neural networks. Full text in Research Archive
• Hubin, Aliaksandr & Storvik, Geir Olve (2019). Combining Model and Parameter Uncertainty in Bayesian Neural Networks. Full text in Research Archive Show summary

  Bayesian neural networks (BNNs) have recently regained a significant amount of attention in the deep learning community due to the development of scalable approximate Bayesian inference techniques. There are several advantages of using Bayesian approach: Parameter and prediction uncertainty become easily available, facilitating rigid statistical analysis. Furthermore, prior knowledge can be incorporated. However so far there have been no scalable techniques capable of combining both model (structural) and parameter uncertainty. In this paper we introduce the concept of model uncertainty in BNNs and hence make inference in the joint space of models and parameters. Moreover, we suggest an adaptation of a scalable variational inference approach with reparametrization of marginal inclusion probabilities to incorporate the model space constraints. Finally, we show that incorporating model uncertainty via Bayesian model averaging and Bayesian model selection allows to drastically sparsify the structure of BNNs without significant loss of predictive power.

• Hubin, Aliaksandr & Storvik, Geir Olve (2019). Combining Model and Parameter Uncertainty in Bayesian Neural Networks. Full text in Research Archive
• Hubin, Aliaksandr; Storvik, Geir Olve; Grini, Paul Eivind & Butenko, Melinka Alonso (2019). Bayesian binomial regression model with a latent Gaussian field for analysis of epigenetic data. Full text in Research Archive
• Hubin, Aliaksandr (2019). An adaptive simulated annealing EM algorithm for inference on non-homogeneous hidden Markov models. Full text in Research Archive
• Hubin, Aliaksandr & Storvik, Geir Olve (2019). Combining Model and Parameter Uncertainty in Bayesian Neural Networks. Full text in Research Archive
• Hubin, Aliaksandr; Storvik, Geir Olve & Frommlet, Florian (2018). Deep Bayesian regression models. Full text in Research Archive
• Hubin, Aliaksandr; Storvik, Geir Olve & Frommlet, Florian (2018). Deep Bayesian regression models. Full text in Research Archive Show summary

  Regression models are used for inference and prediction in a wide range of applications providing a powerful scientific tool for researchers and analysts from different fields. In many research fields the amount of available data as well as the number of potential explanatory variables is rapidly increasing. Variable selection and model averaging have become extremely important tools for improving inference and prediction. However, often linear models are not sufficient and the complex relationship between input variables and a response is better described by introducing non-linearities and complex functional interactions. Deep learning models have been extremely successful in terms of prediction although they are often difficult to specify and potentially suffer from overfitting. The aim of this paper is to bring the ideas of deep learning into a statistical framework which yields more parsimonious models and allows to quantify model uncertainty. To this end we introduce the class of deep Bayesian regression models (DBRM) consisting of a generalized linear model combined with a comprehensive non-linear feature space, where non-linear features are generated just like in deep learning but combined with variable selection in order to include only important features. DBRM can easily be extended to include latent Gaussian variables to model complex correlation structures between observations, which seems to be not easily possible with existing deep learning approaches. Two different algorithms based on MCMC are introduced to fit DBRM and to perform Bayesian inference. The predictive performance of these algorithms is compared with a large number of state of the art algorithms. Furthermore we illustrate how DBRM can be used for model inference in various applications.

• Hubin, Aliaksandr; Storvik, Geir Olve & Frommlet, Florian (2018). Deep Bayesian regression models. Full text in Research Archive
• Hubin, Aliaksandr; Storvik, Geir Olve & Frommlet, Florian (2018). Deep Bayesian regression models. Full text in Research Archive Show summary

  Regression models are used for inference and prediction in a wide range of applications providing a powerful scientific tool for researchers and analysts from different fields. In many research fields the amount of available data as well as the number of potential explanatory variables is rapidly increasing. Variable selection and model averaging have become extremely important tools for improving inference and prediction. However, often linear models are not sufficient and the complex relationship between input variables and a response is better described by introducing non-linearities and complex functional interactions. Deep learning models have been extremely successful in terms of prediction although they are often difficult to specify and potentially suffer from overfitting. The aim of this paper is to bring the ideas of deep learning into a statistical framework which yields more parsimonious models and allows to quantify model uncertainty. To this end we introduce the class of deep Bayesian regression models (DBRM) consisting of a generalized linear model combined with a comprehensive non-linear feature space, where non-linear features are generated just like in deep learning but combined with variable selection in order to include only important features. DBRM can easily be extended to include latent Gaussian variables to model complex correlation structures between observations, which seems to be not easily possible with existing deep learning approaches. Two different algorithms based on MCMC are introduced to fit DBRM and to perform Bayesian inference. The predictive performance of these algorithms is compared with a large number of state of the art algorithms. Furthermore we illustrate how DBRM can be used for model inference in various applications.

• Hubin, Aliaksandr; Storvik, Geir Olve & Frommlet, Florian (2018). Deep Bayesian regression models. Full text in Research Archive Show summary

  Regression models are used for inference and prediction in a wide range of applications providing a powerful scientific tool for researchers and analysts from different fields. In many research fields the amount of available data as well as the number of potential explanatory variables is rapidly increasing. Variable selection and model averaging have become extremely important tools for improving inference and prediction. However, often linear models are not sufficient and the complex relationship between input variables and a response is better described by introducing non-linearities and complex functional interactions. Deep learning models have been extremely successful in terms of prediction although they are often difficult to specify and potentially suffer from overfitting. The aim of this paper is to bring the ideas of deep learning into a statistical framework which yields more parsimonious models and allows to quantify model uncertainty. To this end we introduce the class of deep Bayesian regression models (DBRM) consisting of a generalized linear model combined with a comprehensive non-linear feature space, where non-linear features are generated just like in deep learning but combined with variable selection in order to include only important features. DBRM can easily be extended to include latent Gaussian variables to model complex correlation structures between observations, which seems to be not easily possible with existing deep learning approaches. Two different algorithms based on MCMC are introduced to fit DBRM and to perform Bayesian inference. The predictive performance of these algorithms is compared with a large number of state of the art algorithms. Furthermore we illustrate how DBRM can be used for model inference in various applications.

• Hubin, Aliaksandr; Storvik, Geir Olve & Frommlet, Florian (2018). Deep Bayesian regression models. Full text in Research Archive
• Hubin, Aliaksandr; Storvik, Geir Olve & Frommlet, Florian (2017). A novel algorithmic approach to Bayesian Logic Regression. Full text in Research Archive
• Hubin, Aliaksandr; Storvik, Geir Olve & Frommlet, Florian (2017). Deep non-linear regression models in a Bayesian framework. Full text in Research Archive
• Hubin, Aliaksandr; Storvik, Geir Olve & Grini, Paul Eivind (2017). Variable selection in binomial regression with latent Gaussian field models for analysis of epigenetic data. Full text in Research Archive
• Hubin, Aliaksandr; Storvik, Geir Olve & Frommlet, Florian (2017). A novel algorithmic approach to Bayesian Logic Regression. Full text in Research Archive
• Hubin, Aliaksandr; Storvik, Geir Olve & Frommlet, Florian (2017). A novel GMJMCMC algorithm for Bayesian Logic Regression models. Full text in Research Archive
• Hubin, Aliaksandr & Storvik, Geir Olve (2017). Efficient mode jumping MCMC for Bayesian variable selection and model averaging in GLMM. Full text in Research Archive
• Skaaret-Lund, Lars; Hubin, Aliaksandr & Sæbø, Solve (2026). Sparse, uncertainty-aware and explainable deep learning via Latent Binary Bayesian Neural Networks. Norwegian University of Life Sciences, Ås. ISSN 9788257523886. Full text in Research Archive Show summary

  Neural networks can model complex and highly nonlinear relationships between input variables and the dependent variable, without requiring prior specification of these functional relationships. The Universal Approximation Theorem [Cybenko, 1989] states that a single hidden layer feedforward neural network with an appropriate activation function and sufficient number of neurons can approximate any continuous function to an arbitrary accuracy. Even though the theorem guarantees the existence of such networks, it does not say anything about how to construct them. In practice, modern neural networks have millions or hundreds of billions of parameters, depending on the architecture. This has made them highly successful in a number of different domains. The main drawbacks of neural networks are that they often require very large datasets and high-performance GPU/TPU hardware to train efficiently. In addition, the large number of parameters leads to very high memory costs and risk of overfitting if not properly handled. Moreover, their black-box nature makes it difficult to interpret how variables affect predictions, unlike regression models where the effect is directly represented through the model coefficients. Lastly, frequentist neural networks trained with maximum likelihood only provide point estimates for predictions without quantifying parameter uncertainty. These issues make them challenging to use in domains where it is important to understand why a decision was made, and the uncertainty associated with that prediction. Bayesian neural networks (BNN) [Neal, 1996] offer a conceptually different approach in that weights are treated as random variables rather than fixed values. In BNNs, the objective is therefore to infer the posterior distribution over the weights rather than trying to find a single set of optimal weight values. By sampling from the posterior distribution, BNNs give a distribution of predictions rather than a single point estimate, making them attractive in domains such as healthcare, where quantifying predictive uncertainty is essential. Moreover, BNNs are often more robust against overfitting than their frequentist counterparts. However, BNNs are more computationally and memory expensive. Markov Chain Monte Carlo (MCMC), the gold standard for inference in Bayesian statistics, is usually only feasible in very small networks, but remains unfeasible in the ultra high-dimensional space of neural network parameters. Variational Inference is much more scalable, but still requires extra parameters to represent the posterior compared to frequentist networks, where only one parameter per weight is needed. Although BNNs provide principled uncertainty estimates, they typically remain highly overparameterized and difficult to interpret. Latent Binary Bayesian Neural Networks (LBBNN) [Hubin and Storvik, 2024] are an extension of BNNs where structural uncertainty is explicitly modeled. This is achieved by introducing a latent Bernoulli variable for each weight, enabling a principled Bayesian approach to neural network sparsification, in contrast to frequentist neural networks which typically use heuristics such as thresholding small weight values. While LBBNNs have an extra parameter per weight compared to standard BNNs, the resulting networks are often very sparse without compromising predictive performance. The main limitation of LBBNNs as proposed by Hubin and Storvik [2024] is the requirement to sample the high-dimensional weight and Bernoulli variables for each layer. It is particularly challenging to sample the Bernoulli variables directly, as the binary nature of these variables renders them non-differentiable, necessitating the use of continuous relaxations. By applying the local reparametrization trick (LRT) [Kingma et al., 2015] to the LBBNN model, we sample directly in the (approximately Gaussian) pre-activation space. This leads to a gradient estimator with significantly lower variance, while avoiding the need to sample Bernoulli variables. Furthermore, this formulation allows us to apply normalizing flows [Rezende and Mohamed, 2015] to the variational distribution of LBBNNs, constructing a more flexible distribution than the commonly used mean-field Gaussian, which assumes each weight independently follows a Gaussian distribution. In Skaaret-Lund et al. [2024], we demonstrate that the normalizing flow LBBNNs are typically more sparse, have higher predictive power and are better at variable selection tasks. While BNNs and LBBNNs can improve on uncertainty quantification compared to frequentist networks, they are not fundamentally more interpretable. In Høyheim et al. [2025], we introduce the concept of input-skip, which concatenates the input to each layer. This allows the network to learn different types of functions depending on the data. For example, we show that if the data follows a simple linear function, the network effectively prunes everything aside from the linear connections coming from the last layer. In addition, we introduce the concept of active paths, which provides more interpretable and sparse networks compared to the naive approach of considering weight inclusion or exclusion independently. Finally, we developed the R package LBBNN [Skaaret-Lund et al., 2025], which unifies the approaches described above and makes LBBNNs accessible to a wider audience. To our knowledge, it is the first R package to utilize the Torch backend for efficient GPU-accelerated Bayesian neural networks in R.

• Eirik, Høyheim; Skaaret-Lund, Lars & Hubin, Aliaksandr (2024). Improving sparsity and interpretability of latent binary Bayesian neural networks by introducing input-skip connections. Norges miljø- og biovitenskapelige universitet (NMBU). Full text in Research Archive Show summary

  Being able to model natural phenomena using mathematical equations has been a major success story for researchers. Among various sophisticated methods, artificial neural networks (ANNs) have proven to be powerful models that provide highly accurate predictions when given uncertain input variables. However, a common issue with ANNs is their over-parameterization, which complicates interpretation and raises questions regarding uncertainty. Bayesian neural networks (BNNs) represent weights as probability distributions, unlike ANNs where weights are fixed quantities, thereby enabling the measurement of predictive uncertainty through credibility intervals. In addition, latent binary Bayesian neural networks (LBBNNs) incorporate structure uncertainties and sparsify models by removing redundant weights. This thesis extends the concept of LBBNN by permitting input variables to skip to any preceding layer or even be entirely excluded from the model. This approach further simplifies networks, making it easier to see how input variables affect the prediction. A complexity measure, inspired by smoothing splines, is also proposed to compute the degree of non-linearity for each input variable. This novel input-skip LBBNN implementation has been shown to reduce network density by over 99\% for small networks and over 99.9\% for larger networks, while still providing highly accurate predictions and enabling the measurement of uncertainty. It is further demonstrated that these new methods can accurately identify the true underlying input variables and measure the degree of non-linearity of a system. The main contribution of this thesis is the introduction and application of active paths. The ability to acknowledge that certain weights do not contribute to the prediction, and are thus deemed redundant, helps to provide both global and local explanations. This, in turn, serves to enhance and further sparsify models within the LBBNN framework.

• Sommerfelt, Philip Sebastian Hauglie & Hubin, Aliaksandr (2023). Combining Variational Bayes and GMJMCMC for Scalable Inference on Bayesian Generalized Nonlinear Models. Universitetet i Oslo. Full text in Research Archive Show summary

  We change the approach for computing posterior distributions in Bayesian Generalized Nonlinear Models. We replace MCMC with variational Bayes, and approximate the posterior distribution with mean-field, or through utilization of normalizing flows. Step by step, we go through the theory behind BGNLM, variational inference and normalizing flows. We also show the calculations needed to understand the new implementation, and provide a Python framework for training and testing BGNLMs. Through a series of applications we demonstrate that we are able to make accurate predictions, and get easily obtainable measures for the uncertainty of the predictions.

• Abidi, Osama; Hubin, Aliaksandr; Frausig, Jesper & Jullum, Martin (2023). Using Graph Bayesian Neural Networks for fraud pattern detection and classification from bank transactions data. Norges miljø- og biovitenskapelige universitet (NMBU). Full text in Research Archive Show summary

  The thesis aims to conduct an investigation into money laundering networks by synthesizing many earlier methods. The methodology begins by training a Bayesian graph neural network on a data set from DNB to learn the relationship between the money laundering status of a transacting party and its features and network properties. One can then get node embeddings by extracting the activation values of each prediction at the last hidden layer in the model. This will hopefully yield informative node embeddings that one could then visualize along with their predicted class and associated uncertainty by applying dimensionality reduction through Principal Component Analysis. If there are nodes that have the same predicted class and magnitude of uncertainty that cluster together, one could try retrieving their associated networks and investigate the patterns.

• Sunde, Leif-Martin Sæther; Hubin, Aliaksandr & Storvik, Geir Olve (2023). Spherical Priors for Bayesian Deep Learning. Universitetet i Oslo. Full text in Research Archive
• Ellingsen, Herman; Remonato, Filippo; Sæbø, Solve & Hubin, Aliaksandr (2023). Outlier detection in Bayesian Neural Networks. Norges miljø- og biovitenskapelige universitet (NMBU). Full text in Research Archive Show summary

  Exploring different ways of describing uncertainty in neural networks is of great interest. Artificial intelligence models can be used with greater confidence by having solid methods for identifying and quantifying uncertainty. This is especially important in high-risk areas such as medical applications, autonomous vehicles, and financial systems. This thesis explores how to detect classification outliers in Bayesian Neural Networks. A few methods exist for quantifying uncertainty in Bayesian Neural Networks, such as computing the Entropy of the prediction vector. Is there a more accurate and broad way of detecting classification outliers in Bayesian Neural Networks? If a sample is detected as an outlier, is there a way of separating between different types of outliers? We try to answer these questions by using the pre-activation neuron values of a Bayesian Neural Network. We compare, in total, three different methods using simulated data, the Breast Cancer Wisconsin dataset and the MNIST dataset. The first method uses the well-researched Predictive Entropy, which will act as a baseline method. The second method uses the pre-activation neuron values in the output layer of a Bayesian Neural Network; this is done by comparing the pre-activation neuron value from a given data sample with the pre-activation neuron values from the training data. Lastly, the third method is a combination of the first two methods. The results show that the performance might depend on the dataset type. The proposed method outperforms the baseline method on the simulated data. When using the Breast Cancer Wisconsin dataset, we see that the proposed method is significantly better than the baseline. Interestingly, we observe that with the MNIST dataset, the baseline model outperforms the proposed method in most scenarios. Common for all three datasets is that the combination of the two methods performs approximately as well as the best of the two.

• Lachmann, Jon & Hubin, Aliaksandr (2021). Subsampling Strategies for Bayesian Variable Selection and Model Averaging in GLM and BGNLM. Stockholm University. Full text in Research Archive Show summary

  Bayesian Generalized Nonlinear Models (BGNLM) offer a flexible alternative to GLM while still providing better interpretability than machine learning techniques such as neural networks. In BGNLM, the methods of Bayesian Variable Selection and Model Averaging are applied in an extended GLM setting. Models are fitted to data using MCMC within a genetic framework in an algorithm called GMJMCMC. In this thesis, we present a new implementation of the algorithm as a package in the programming language R. We also present a novel algorithm called S-IRLS-SGD for estimating the MLE of a GLM by subsampling the data. Finally, we present some theory combining the novel algorithm with GMJMCMC/MJMCMC/MCMC and a number of experiments demonstrating the performance of the contributed algorithm.

• Hubin, Aliaksandr & Aas, Kjersti (2019). FinAI: Scalable techniques to stock price time series modelling. Norsk Regnesentral. Full text in Research Archive
• Hubin, Aliaksandr (2018). Bayesian model configuration, selection and averaging in complex regression contexts. Universitetet i Oslo. Full text in Research Archive Show summary

  In this PhD thesis problems of Bayesian model selection and model averaging are addressed in various regression contexts. The approaches developed within the thesis are based on the idea of marginalizing out parameters from the likelihood. This allows to work on the marginal space of models, which simplifies the search algorithms significantly. For the linear models an efficient mode jumping Monte Carlo Markov chain (MJMCMC) algorithm was suggested. The approach performed very well on simulated and real data. Further, the algorithm was extended to work with logic regressions, where one has a feature space consisting of various complicated logical expressions, which makes enumeration of all features computationally and memory infeasible in most of the cases. The genetically modified MJMCMC (GMJMCMC) algorithm was suggested to tackle this issue. The algorithm combines the idea of keeping and updating the populations of highly predictive logical expressions combined with MJMCMC for the efficient exploration of the model space. Several simulation and real data studies show that logical expressions of high orders can be recovered with large power and low false discovery rate. Moreover, the GMJMCMC approach is adapted to make inference within the class of deep Bayesian regression models (which is a suggested in the thesis extension of various machine and statistical learning models like artificial neural networks, classification and regression trees, logic regressions and linear models). The reversible GMJMCMC, named RGMJMCMC, is also suggested. It makes transitions between the populations of variables in a way that satisfies the detailed balance equation. Based on several examples, it is shown that the DBRM approach can be efficient for both inference and prediction in various applications. In particular, two ground physical laws (planetary mass law and third Kepler’s law) were recovered from the data with large power and low false discovery rate. Three classification examples were also studied, where the comparison to other popular machine and statistical learning approaches was performed. Finally, a thorough study comparing different Bayesian approaches to genome wide association was done. It was shown that the developed in this thesis approaches can be efficiently applied to data with a huge number of covariates.

View all works in NVA