-
Mathisen, Therese Fostervold; Sundgot-Borgen, Jorunn & Hubin, Aliaksandr
(2023).
Tung styrketrening i svangerskap og barseltid – helt greit? Preliminære data fra en prospektiv, kontrollert kohortstudie.
.
-
Hubin, Aliaksandr & Storvik, Geir Olve
(2022).
Variational Bayes for inference on model and parameter uncertainty in Bayesian neural networks.
Vis sammendrag
Bayesian neural networks (BNNs) have recently regained a significant amount of attention in the deep learning community due to the development of scalable approximate Bayesian inference techniques [1]. There are several advantages of using a Bayesian approach: Parameter and prediction uncertainties become easily available, facilitating rigorous statistical analysis. Furthermore, prior knowledge can be incorporated. However, so far there have been no scalable techniques capable of combining both model (structural) and parameter uncertainty. In the presented piece of research [2] we introduce the concept of model uncertainty in BNNs and hence make inference in the joint space of models and parameters. Moreover, we suggest an adaptation of a scalable variational inference approach with reparametrization of marginal inclusion probabilities to incorporate the model space constraints. Experimental results on a range of benchmark data sets show that we obtain comparable accuracy results with the competing models, but based on methods that are much more sparse than ordinary BNNs. This is particularly the case in model selection settings, but also within a Bayesian model averaging setting a considerable sparsification is achieved. As expected, model uncertainties give higher, but more reliable uncertainty measures.
References:
[1] Blundell, C., Cornebise, J., Kavukcuoglu, K., and Wierstra, D. (2015). Weight uncertainty in neural networks. International Conference on Machine Learning, 1613 - 1622.
[2] Hubin, A., Storvik, G. (2019). Combining model and parameter uncertainty in Bayesian neural networks, arXiv preprint arXiv:1903.07594.
-
Hubin, Aliaksandr & Storvik, Geir Olve
(2022).
Variational Inference for Bayesian Neural Networks under Model and Parameter Uncertainty.
-
Lachmann, Jon; Storvik, Geir Olve; Frommlet, Florian & Hubin, Aliaksandr
(2022).
A subsampling approach for Bayesian model selection.
-
Hubin, Aliaksandr & De Bin, Riccardo
(2022).
On a genetically modified mode jumping MCMC approach for multivariate fractional polynomials.
-
Hubin, Aliaksandr & De Bin, Riccardo
(2022).
On a genetically modified mode jumping MCMC approach for multivariate fractional polynomials.
-
Hubin, Aliaksandr; Frommlet, Florian & Storvik, Geir Olve
(2021).
Reversible Genetically Modified MCMCs.
Vis sammendrag
In this work, we introduce a reversible version of a genetically modified Markov chain Monte Carlo algorithm (GMJMCMC) for inference on posterior model probabilities in complex functional spaces, where the number of explanatory variables or functions of explanatory variables is prohibitively large for simple Markov Chain Monte Carlo methods. A genetically modified Markov chain Monte Carlo algorithm (GMJMCMC) was introduced in [5, 4, 2] for Bayesian model selection/averaging problems when the total number of function of covariates is prohibitively large.
More specifically, these applications include GWAS studies with Bayesian generalized linear models [2] as well as Bayesian logic regressions [5] and Bayesian generalized nonlinear models [4]. If its regularity conditions are met, GMJMCMC algorithm can asymptotically explore all models in the defined model spaces. At the same time, GMJMCMC is not a proper MCMC in a sense that its limiting distribution does not correspond to marginal posterior model probabilities and thus only renormalized estimates of these probabilities [3, 1] can be obtained. Unlike the standard GMJMCMC algorithm, the introduced algorithm is a proper MCMC and its limiting distribution corresponds to posterior marginal model probabilities in the explored model spaces under reasonable regularity conditions.
-
Hubin, Aliaksandr & Storvik, Geir Olve
(2021).
Variational Inference for Bayesian Neural Networks under Model and Parameter Uncertainty.
Vis sammendrag
Bayesian neural networks (BNNs) have recently regained a significant amount of attention in the deep learning community due to the development of scalable approximate Bayesian inference techniques. There are several advantages of using a Bayesian approach: Parameter and prediction uncertainties become easily available, facilitating rigorous statistical analysis. Furthermore, prior knowledge can be incorporated. However so far there have been no scalable techniques capable of combining both model (structural) and parameter uncertainty. In this paper we introduce the concept of model uncertainty in BNNs and hence make inference in the joint space of models and parameters.
Moreover, we suggest an adaptation of a scalable variational inference approach with reparametrization of marginal inclusion probabilities to incorporate the model space constraints. Experimental results on a range of benchmark data sets show that we obtain comparable accuracy results with the competing models, but based on methods that are much more sparse than ordinary BNNs. This is particularly the case in model selection settings, but also within a Bayesian model averaging setting a considerable sparsification is achieved. As expected, model uncertainties give higher, but more reliable uncertainty measures.
-
Hubin, Aliaksandr & Storvik, Geir Olve
(2021).
Variational Bayes for inference on model and parameter
uncertainty in Bayesian neural networks.
Vis sammendrag
Bayesian neural networks (BNNs) have recently regained a significant amount of attention in
the deep learning community due to the development of scalable approximate Bayesian inference
techniques [1]. There are several advantages of using a Bayesian approach: Parameter and prediction uncertainties become easily available, facilitating rigorous statistical analysis. Furthermore,
prior knowledge can be incorporated. However so far there have been no scalable techniques capable of combining both model (structural) and parameter uncertainty. In the presented piece of
research [2] we introduce the concept of model uncertainty in BNNs and hence make inference in
the joint space of models and parameters. Moreover, we suggest an adaptation of a scalable variational inference approach with reparametrization of marginal inclusion probabilities to incorporate
the model space constraints. Experimental results on a range of benchmark data sets show that
we obtain comparable accuracy results with the competing models, but based on methods that are
much more sparse than ordinary BNNs. This is particularly the case in model selection settings,
but also within a Bayesian model averaging setting a considerable sparsification is achieved. As
expected, model uncertainties give higher, but more reliable uncertainty measures.
-
Lison, Pierre; Barnes, Jeremy Claude & Hubin, Aliaksandr
(2021).
skweak: weak supervision made easy for NLP.
Vis sammendrag
We present skweak, a versatile, Python-based software toolkit enabling NLP developers to apply weak supervision to a wide range of NLP tasks. Weak supervision is an emerging machine learning paradigm based on a simple idea: instead of labelling data points by hand, we use labelling functions derived from domain knowledge to automatically obtain annotations for a given dataset. The resulting labels are then aggregated with a generative model that estimates the accuracy (and possible confusions) of each labelling function. The skweak toolkit makes it easy to implement a large spectrum of labelling functions (such as heuristics, gazetteers, neural models or linguistic constraints) on text data, apply them on a corpus, and aggregate their results in a fully unsupervised fashion. skweak is especially designed to facilitate the use of weak supervision for NLP tasks such as text classification and sequence labelling. We illustrate the use of skweak for NER and sentiment analysis. skweak is released under an open-source license and is available at https://github.com/NorskRegnesentral/skweak
-
Hubin, Aliaksandr; Storvik, Geir Olve & Frommlet, Florian
(2020).
A novel algorithmic approach to Bayesian Logic Regression.
Vis sammendrag
Logic regression was developed more than a decade ago as a tool to construct predictors from Boolean combinations of binary covariates. It has been mainly used to model epistatic effects in genetic association studies, which is very appealing due to the intuitive interpretation of logic expressions to describe the interaction between genetic variations. Nevertheless logic regression has (partly due to computational challenges) remained less well known than other approaches to epistatic association mapping. Here we will adapt an advanced evolutionary algorithm called GMJMCMC (Genetically modified Mode Jumping Markov Chain Monte Carlo) to perform Bayesian model selection in the space of logic regression models. After describing the algorithmic details of GMJMCMC we perform a comprehensive simulation study that illustrates its performance given logic regression terms of various complexity. Specifically GMJMCMC is shown to be able to identify three-way and even four-way interactions with relatively large power, a level of complexity which has not been achieved by previous implementations of logic regression. We apply GMJMCMC to reanalyze QTL (quantitative trait locus) mapping data for Recombinant Inbred Lines in Arabidopsis thaliana and from a backcross population in Drosophila where we identify several interesting epistatic effects. The method is implemented in an R package which is available on github.
-
Hubin, Aliaksandr; Storvik, Geir Olve & Frommlet, Florian
(2020).
Rejoinder for the discussion of the paper "A Novel Algorithmic Approach to Bayesian Logic Regression".
Bayesian Analysis.
ISSN 1936-0975.
15(1),
s. 312–333.
doi:
10.1214/18-ba1141.
Fulltekst i vitenarkiv
Vis sammendrag
Logic regression was developed more than a decade ago as a tool to construct predictors from Boolean combinations of binary covariates. It has been mainly used to model epistatic effects in genetic association studies, which is very appealing due to the intuitive interpretation of logic expressions to describe the interaction between genetic variations. Nevertheless logic regression has (partly due to computational challenges) remained less well known than other approaches to epistatic association mapping. Here we will adapt an advanced evolutionary algorithm called GMJMCMC (Genetically modified Mode Jumping Markov Chain Monte Carlo) to perform Bayesian model selection in the space of logic regression models. After describing the algorithmic details of GMJMCMC we perform a comprehensive simulation study that illustrates its performance given logic regression terms of various complexity. Specifically GMJMCMC is shown to be able to identify three-way and even four-way interactions with relatively large power, a level of complexity which has not been achieved by previous implementations of logic regression. We apply GMJMCMC to reanalyze QTL (quantitative trait locus) mapping data for Recombinant Inbred Lines in Arabidopsis thaliana and from a backcross population in Drosophila where we identify several interesting epistatic effects. The method is implemented in an R package which is available on github.
-
Hubin, Aliaksandr
(2019).
Using node embedding to obtain
information from network based
transactions data in a bank.
-
-
Hubin, Aliaksandr & Storvik, Geir Olve
(2019).
Combining Model and Parameter Uncertainty in Bayesian Neural Networks.
Vis sammendrag
Bayesian neural networks (BNNs) have recently regained a significant amount of attention in the deep learning community due to the development of scalable approximate Bayesian inference techniques. There are several advantages of using Bayesian approach: Parameter and prediction uncertainty become easily available, facilitating rigid statistical analysis. Furthermore, prior knowledge can be incorporated. However so far there have been no scalable techniques capable of combining both model (structural) and parameter uncertainty. In this paper we introduce the concept of model uncertainty in BNNs and hence make inference in the joint space of models and parameters. Moreover, we suggest an adaptation of a scalable variational inference approach with reparametrization of marginal inclusion probabilities to incorporate the model space constraints. Finally, we show that incorporating model uncertainty via Bayesian model averaging and Bayesian model selection allows to drastically sparsify the structure of BNNs without significant loss of predictive power.
-
Hubin, Aliaksandr
(2019).
An adaptive simulated annealing EM algorithm for inference on non-homogeneous hidden Markov models.
-
Hubin, Aliaksandr; Storvik, Geir Olve; Grini, Paul Eivind & Butenko, Melinka Alonso
(2019).
Bayesian binomial regression model with a latent Gaussian field for analysis of epigenetic data.
-
Hubin, Aliaksandr & Storvik, Geir Olve
(2019).
Combining Model and Parameter Uncertainty in Bayesian Neural Networks.
-
Hubin, Aliaksandr & Storvik, Geir Olve
(2019).
Combining Model and Parameter Uncertainty in Bayesian Neural Networks.
Vis sammendrag
Bayesian neural networks (BNNs) have recently regained a significant amount of attention in the deep learning community due to the development of scalable approximate Bayesian inference techniques. There are several advantages of using Bayesian approach: Parameter and prediction uncertainty become easily available, facilitating rigid statistical analysis. Furthermore, prior knowledge can be incorporated. However so far there have been no scalable techniques capable of combining both model (structural) and parameter uncertainty. In this paper we introduce the concept of model uncertainty in BNNs and hence make inference in the joint space of models and parameters. Moreover, we suggest an adaptation of a scalable variational inference approach with reparametrization of marginal inclusion probabilities to incorporate the model space constraints. Finally, we show that incorporating model uncertainty via Bayesian model averaging and Bayesian model selection allows to drastically sparsify the structure of BNNs without significant loss of predictive power.
-
Hubin, Aliaksandr & Storvik, Geir Olve
(2019).
Combining Model and Parameter Uncertainty in Bayesian Neural Networks.
-
Storvik, Geir Olve & Hubin, Aliaksandr
(2019).
Combining model and parameter uncertainty in Bayesian neural networks.
-
Hubin, Aliaksandr; Storvik, Geir Olve & Frommlet, Florian
(2018).
Deep Bayesian regression models.
-
Hubin, Aliaksandr; Storvik, Geir Olve & Frommlet, Florian
(2018).
Deep Bayesian regression models.
-
Hubin, Aliaksandr; Storvik, Geir Olve & Frommlet, Florian
(2018).
Deep Bayesian regression models.
-
Hubin, Aliaksandr; Storvik, Geir Olve & Frommlet, Florian
(2018).
Deep Bayesian regression models.
Vis sammendrag
Regression models are used for inference and prediction in a wide range of applications providing a powerful scientific tool for researchers and analysts from different fields. In many research fields the amount of available data as well as the number of potential explanatory variables is rapidly increasing. Variable selection and model averaging have become extremely important tools for improving inference and prediction. However, often linear models are not sufficient and the complex relationship between input variables and a response is better described by introducing non-linearities and complex functional interactions. Deep learning models have been extremely successful in terms of prediction although they are often difficult to specify and potentially suffer from overfitting. The aim of this paper is to bring the ideas of deep learning into a statistical framework which yields more parsimonious models and allows to quantify model uncertainty. To this end we introduce the class of deep Bayesian regression models (DBRM) consisting of a generalized linear model combined with a comprehensive non-linear feature space, where non-linear features are generated just like in deep learning but combined with variable selection in order to include only important features. DBRM can easily be extended to include latent Gaussian variables to model complex correlation structures between observations, which seems to be not easily possible with existing deep learning approaches. Two different algorithms based on MCMC are introduced to fit DBRM and to perform Bayesian inference. The predictive performance of these algorithms is compared with a large number of state of the art algorithms. Furthermore we illustrate how DBRM can be used for model inference in various applications.
-
Hubin, Aliaksandr; Storvik, Geir Olve & Frommlet, Florian
(2018).
Deep Bayesian regression models.
Vis sammendrag
Regression models are used for inference and prediction in a wide range of applications providing a powerful scientific tool for researchers and analysts from different fields. In many research fields the amount of available data as well as the number of potential explanatory variables is rapidly increasing. Variable selection and model averaging have become extremely important tools for improving inference and prediction. However, often linear models are not sufficient and the complex relationship between input variables and a response is better described by introducing non-linearities and complex functional interactions. Deep learning models have been extremely successful in terms of prediction although they are often difficult to specify and potentially suffer from overfitting. The aim of this paper is to bring the ideas of deep learning into a statistical framework which yields more parsimonious models and allows to quantify model uncertainty. To this end we introduce the class of deep Bayesian regression models (DBRM) consisting of a generalized linear model combined with a comprehensive non-linear feature space, where non-linear features are generated just like in deep learning but combined with variable selection in order to include only important features. DBRM can easily be extended to include latent Gaussian variables to model complex correlation structures between observations, which seems to be not easily possible with existing deep learning approaches. Two different algorithms based on MCMC are introduced to fit DBRM and to perform Bayesian inference. The predictive performance of these algorithms is compared with a large number of state of the art algorithms. Furthermore we illustrate how DBRM can be used for model inference in various applications.
-
Hubin, Aliaksandr; Storvik, Geir Olve & Frommlet, Florian
(2018).
Deep Bayesian regression models.
Vis sammendrag
Regression models are used for inference and prediction in a wide range of applications providing a powerful scientific tool for researchers and analysts from different fields. In many research fields the amount of available data as well as the number of potential explanatory variables is rapidly increasing. Variable selection and model averaging have become extremely important tools for improving inference and prediction. However, often linear models are not sufficient and the complex relationship between input variables and a response is better described by introducing non-linearities and complex functional interactions. Deep learning models have been extremely successful in terms of prediction although they are often difficult to specify and potentially suffer from overfitting. The aim of this paper is to bring the ideas of deep learning into a statistical framework which yields more parsimonious models and allows to quantify model uncertainty. To this end we introduce the class of deep Bayesian regression models (DBRM) consisting of a generalized linear model combined with a comprehensive non-linear feature space, where non-linear features are generated just like in deep learning but combined with variable selection in order to include only important features. DBRM can easily be extended to include latent Gaussian variables to model complex correlation structures between observations, which seems to be not easily possible with existing deep learning approaches. Two different algorithms based on MCMC are introduced to fit DBRM and to perform Bayesian inference. The predictive performance of these algorithms is compared with a large number of state of the art algorithms. Furthermore we illustrate how DBRM can be used for model inference in various applications.
-
Hubin, Aliaksandr; Storvik, Geir Olve & Frommlet, Florian
(2017).
Deep non-linear regression models in a Bayesian framework.
-
Hubin, Aliaksandr; Storvik, Geir Olve & Grini, Paul Eivind
(2017).
Variable selection in binomial regression with latent Gaussian field models for analysis of
epigenetic data.
-
Hubin, Aliaksandr; Storvik, Geir Olve & Frommlet, Florian
(2017).
A novel algorithmic approach to Bayesian Logic Regression.
-
Hubin, Aliaksandr; Storvik, Geir Olve & Frommlet, Florian
(2017).
A novel GMJMCMC algorithm for Bayesian Logic Regression models.
-
Hubin, Aliaksandr & Storvik, Geir Olve
(2017).
Efficient mode jumping MCMC for Bayesian
variable selection and model averaging in GLMM.
-
Hubin, Aliaksandr; Storvik, Geir Olve & Frommlet, Florian
(2017).
A novel algorithmic approach to Bayesian Logic Regression.
-
Hubin, Aliaksandr & Storvik, Geir Olve
(2016).
Efficient mode jumping MCMC for Bayesian variable selection in GLM with random effects models.
-
Hubin, Aliaksandr & Storvik, Geir Olve
(2016).
On Mode Jumping in MCMC for Bayesian Variable
Selection within GLMM.
-
Hubin, Aliaksandr & Storvik, Geir Olve
(2016).
VARIABLE SELECTION IN BINOMIAL REGRESSION WITH LATENT GAUSSIAN FIELD MODELS FOR ANALYSIS OF EPIGENETIC DATA.
-
Hubin, Aliaksandr & Storvik, Geir Olve
(2016).
Variable selection in logistic regression with a latent Gaussian field models with an application in epigenomics.
-
Hubin, Aliaksandr
(2015).
Statistics for Epigenetics.
-
Hubin, Aliaksandr & Storvik, Geir Olve
(2015).
On model selection in Hidden Markov Models with covariates.
-
Hubin, Aliaksandr & Storvik, Geir Olve
(2015).
Variable selection in binomial regression with a latent
Gaussian field models for analysis of epigenetic data.
-
Hubin, Aliaksandr & Storvik, Geir Olve
(2015).
Variable selection in binomial regression with a latent
Gaussian field models for analysis of epigenetic data.
-
Hubin, Aliaksandr; Norlund, Ellen Karoline & Gribkovskaia, Irina
(2014).
Evaluating robustness of speed optimized supply vessel schedules.
Vis sammendrag
Offshore installations need supply vessel services on a regular basis.
Weather uncertainty impacts on how service is performed. We incorporate
different robustness and speed optimization strategies into
the two-phase optimization procedure for generation of supply vessel
schedules. To compare performance of these strategies by evaluating
robustness of generated schedules with different service parameters a
discrete-event simulation model is developed. Based on results from
simulation strategies for improving robustness incorporated into the
simulation model are applied to modify the schedules.
-
Ellingsen, Herman; Remonato, Filippo; Sæbo, Solve & Hubin, Aliaksandr
(2023).
Outlier Detection in Bayesian Neural Networks.
Norwegian University of Life Sciences.
Vis sammendrag
Exploring different ways of describing uncertainty in neural networks is of great interest. Artificial intelligence models can be used with greater confidence by having solid methods for identifying and quantifying uncertainty. This is especially important in high-risk areas such as medical applications, autonomous vehicles, and financial systems. This thesis explores how to detect classification outliers in Bayesian Neural Networks. A few methods exist for quantifying uncertainty in Bayesian Neural Networks, such as computing the Entropy of the prediction vector. Is there a more accurate and broad way of detecting classification outliers in Bayesian Neural Networks? If a sample is detected as an outlier, is there a way of separating between different types of outliers? We try to answer these questions by using the pre-activation neuron values of a Bayesian Neural Network. We compare, in total, three different methods using simulated data, the Breast Cancer Wisconsin dataset and the MNIST dataset. The first method uses the well-researched Predictive Entropy, which will act as a baseline method. The second method uses the pre-activation neuron values in the output layer of a Bayesian Neural Network; this is done by comparing the pre-activation neuron value from a given data sample with the pre-activation neuron values from the training data. Lastly, the third method is a combination of the first two methods. The results show that the performance might depend on the dataset type. The proposed method outperforms the baseline method on the simulated data. When using the Breast Cancer Wisconsin dataset, we see that the proposed method is significantly better than the baseline. Interestingly, we observe that with the MNIST dataset, the baseline model outperforms the proposed method in most scenarios. Common for all three datasets is that the combination of the two methods performs approximately as well as the best of the two.
-
Hubin, Aliaksandr & Aas, Kjersti
(2019).
FinAI: Scalable techniques to stock price time series
modelling.
Norsk Regnesentral.
-
Hubin, Aliaksandr
(2018).
Bayesian model configuration, selection and averaging in complex regression contexts.
Universitetet i Oslo.
ISSN 1501-7710.