larsbent

• Erdmann, Niklas; Bentsen, Lars Ødegaard; Stenbro, Roy; Riise, Heine Nygard; Warakagoda, Narada Dilp & Engelstad, Paal E. (2026). Multi-Horizon Time Series Forecasting of Non-Parametric CDFs with Deep Lattice Networks. Proceedings of the AAAI Conference on Artificial Intelligence. ISSN 2159-5399. 40(1), s. 255–264. Fulltekst i vitenarkiv Vis sammendrag

  Probabilistic forecasting is not only a way to add more information to a prediction of the future, but it also builds on weaknesses in point prediction. Sudden changes in a time series can still be captured by a cumulative distribution function (CDF), while a point prediction is likely to miss it entirely. The modeling of CDFs within forecasts has historically been limited to parametric approaches, but due to recent advances, this no longer has to be the case. We aim to advance the fields of probabilistic forecasting and monotonic networks by connecting them and propose an approach that permits the forecasting of implicit, complete, and nonparametric CDFs. For this purpose, we propose an adaptation to deep lattice networks (DLN) for monotonically constrained simultaneous/implicit quantile regression in time series forecasting. Quantile regression usually produces quantile crossovers, which need to be prevented to achieve a legitimate CDF. By leveraging long short term memory units (LSTM) as the embedding layer, and spreading quantile inputs to all sub-lattices of a DLN with an extended output size, we can produce a multi-horizon forecast of an implicit CDF due to the monotonic constraintability of DLNs that prevent quantile crossovers. We compare and evaluate our approach's performance to relevant state of the art within the context of a highly relevant application of time series forecasting: Day-ahead, hourly forecasts of solar irradiance observations. Our experiments show that the adaptation of a DLN performs just as well or even better than an unconstrained approach. Further comparison of the adapted DLN against a scalable monotonic neural network shows that our approach performs better. With this adaptation of DLNs, we intend to create more interest and crossover investigations in techniques of monotonic neural networks and probabilistic forecasting.

• Bentsen, Lars Ødegaard; Warakagoda, Narada Dilp; Stenbro, Roy & Engelstad, Paal (2024). Relative evaluation of probabilistic methods for spatio-temporal wind forecasting. Journal of Cleaner Production. ISSN 0959-6526. 434. doi: 10.1016/j.jclepro.2023.139944. Fulltekst i vitenarkiv Vis sammendrag

  Short-term wind power forecasting has become a de facto tool to better facilitate the integration of such renewable energy resources into modern power grids. Instead of point predictors, which produce single-value predictions for the expected power, probabilistic forecasts predict probability distributions over the expected power output or associated confidence intervals. In this study, three different parametric and non-parametric methods for uncertainty modelling in wind power forecasting were studied, namely quantile regression (QR), variational inference and a maximum likelihood estimation (MLE) method. Johnson’s SU distribution was studied as a novel candidate for modelling wind power, which is a transformed normal distribution that exhibits both skew and heavy tails. This was one of the first studies to provide a thorough investigation of Johnson’s SU distribution for uncertainty modelling in a complex deep learning framework for wind forecasting. It was found that Johsnon’s SU likelihood and QR-based models significantly outperformed models using Gaussian likelihoods, based on a range of quantitative metrics to evaluate probability distributions and qualitative investigation of produced forecasts. Variational inference models using Johnson’s SU likelihoods performed remarkably well, with near-perfect calibration and higher precision than models using any of the other methods for uncertainty modelling, as evaluated through the pinball loss, Average Coverage Error (ACE) and Prediction Interval Coverage Percentage (PICP) metric. With the superior performance of Johnson’s SU likelihood models, the study mainly contributes to the literature by introducing another candidate distribution for probabilistic wind forecasting, which is analytical, unbounded and easy to integrate into modern deep learning frameworks.

• Erdmann, Niklas; Bentsen, Lars Ødegaard; Stenbro, Roy; Riise, Heine Nygard; Warakagoda, Narada Dilp & Engelstad, Paal (2024). Deep and Probabilistic Solar Irradiance Forecast at the Arctic Circle. Conference record of the Photovoltaic Specialists Conference. ISSN 0160-8371. s. 1599–1606. doi: 10.1109/PVSC57443.2024.10748862. Fulltekst i vitenarkiv
• Bentsen, Lars Ødegaard; Warakagoda, Narada Dilp; Stenbro, Roy & Engelstad, Paal E. (2023). A Unified Graph Formulation for Spatio-Temporal Wind Forecasting. Energies. 16(20). doi: 10.3390/en16207179. Fulltekst i vitenarkiv Vis sammendrag

  With the rapid adoption of wind energy globally, there is a need for accurate short-term forecasting systems to improve the reliability and integration of such energy resources on a large scale. While most spatio-temporal forecasting systems comprise distinct components to learn spatial and temporal dependencies separately, this paper argues for an approach to learning spatio-temporal information jointly. Many time series forecasting systems also require aligned input information and do not naturally facilitate irregular data. Research is therefore required to investigate methodologies for forecasting in the presence of missing or corrupt measurements. To help combat some of these challenges, this paper studied a unified graph formulation. With the unified formulation, a graph neural network (GNN) was used to extract spatial and temporal dependencies simultaneously, in a single update, while also naturally facilitating missing data. To evaluate the proposed unified approach, the study considered hour-ahead wind speed forecasting in the North Sea under different amounts of missing data. The framework was compared against traditional spatio-temporal architectures that used GNNs together with temporal long short-term memory (LSTM) and Transformer or Autoformer networks, along with the imputation of missing values. The proposed framework outperformed the traditional architectures, with absolute errors of around 0.73–0.90 m per second, when subject to 0–80% of missing input data. The unified graph approach was also better at predicting large changes in wind speed, with an additional 10-percentage-point improvement over the second-best model. Overall, this paper investigated a novel methodology for spatio-temporal wind speed forecasting and showed how the proposed unified graph formulation achieved competitive results compared to more traditional GNN-based architectures.

• Bentsen, Lars Ødegaard; Warakagoda, Narada Dilp; Stenbro, Roy & Engelstad, Paal (2022). Wind Park Power Prediction: Attention-Based Graph Networks and Deep Learning to Capture Wake Losses. Journal of Physics: Conference Series (JPCS). ISSN 1742-6588. 2265. doi: 10.1088/1742-6596/2265/2/022035. Fulltekst i vitenarkiv Vis sammendrag

  Abstract With the increased penetration of wind energy into the power grid, it has become increasingly important to be able to predict the expected power production for larger wind farms. Deep learning (DL) models can learn complex patterns in the data and have found wide success in predicting wake losses and expected power production. This paper proposes a modular framework for attention-based graph neural networks (GNN), where attention can be applied to any desired component of a graph block. The results show that the model significantly outperforms a multilayer perceptron (MLP) and a bidirectional LSTM (BLSTM) model, while delivering performance on-par with a vanilla GNN model. Moreover, we argue that the proposed graph attention architecture can easily adapt to different applications by offering flexibility into the desired attention operations to be used, which might depend on the specific application. Through analysis of the attention weights, it was showed that employing attention-based GNNs can provide insights into what the models learn. In particular, the attention networks seemed to realise turbine dependencies that aligned with some physical intuition about wake losses.

• Bentsen, Lars Ødegaard; Simionato, Riccardo; Wallace, Benedikte & Krzyzaniak, Michael Joseph (2022). Transformer and LSTM Models for Automatic Counterpoint Generation using Raw Audio. Proceedings of the SMC Conferences. doi: 10.5281/zenodo.6572847. Fulltekst i vitenarkiv Vis sammendrag

  A study investigating Transformer and LSTM models applied to raw audio for automatic generation of counterpoint was conducted. In particular, the models learned to generate missing voices from an input melody, using a collection of raw audio waveforms of various pieces of Bach’s work, played on different instruments. The research demonstrated the efficacy and behaviour of the two deep learning (DL) architectures when applied to raw audio data, which are typically characterised by much longer sequences than symbolic music representations, such as MIDI. Currently, the LSTM model has been the quintessential DL model for sequence-based tasks, such as generative audio models, but the research conducted in this study shows that the Transformer model can achieve competitive results on a fairly complex raw audio task. The research therefore aims to spark further research and investigation into how Trans- former models can be used for applications typically dominated by recurrent neural networks (RNN). In general, both models yielded excellent results and generated sequences with temporal patterns similar to the input targets for songs that were not present in the training data, as well as for a sample taken from a completely different dataset.

• Bentsen, Lars Ødegaard; Warakagoda, Narada Dilp; Stenbro, Roy & Engelstad, Paal (2022). Probabilistic Wind Park Power Prediction using Bayesian Deep Learning and Generative Adversarial Networks. Journal of Physics: Conference Series (JPCS). ISSN 1742-6588. 2362. doi: 10.1088/1742-6596/2362/1/012005. Fulltekst i vitenarkiv Vis sammendrag

  The rapid depletion of fossil-based energy supplies, along with the growing reliance on renewable resources, has placed supreme importance on the predictability of renewables. Research focusing on wind park power modelling has mainly been concerned with point estimators, while most probabilistic studies have been reserved for forecasting. In this paper, a few different approaches to estimate probability distributions for individual turbine powers in a real off-shore wind farm were studied. Two variational Bayesian inference models were used, one employing a multilayered perceptron and another a graph neural network (GNN) architecture. Furthermore, generative adversarial networks (GAN) have recently been proposed as Bayesian models and was here investigated as a novel area of research. The results showed that the two Bayesian models outperformed the GAN model with regards to mean absolute errors (MAE), with the GNN architecture yielding the best results. The GAN on the other hand, seemed potentially better at generating diverse distributions. Standard deviations of the predicted distributions were found to have a positive correlation with MAEs, indicating that the models could correctly provide estimates on the confidence associated with particular predictions.

• Bentsen, Lars Ødegaard; Warakagoda, Narada Dilp; Stenbro, Roy & Engelstad, Paal (2022). Spatio-temporal wind speed forecasting using graph networks and novel Transformer architectures. Applied Energy. ISSN 0306-2619. 333. doi: 10.1016/j.apenergy.2022.120565. Fulltekst i vitenarkiv Vis sammendrag

  To improve the security and reliability of wind energy production, short-term forecasting has become of utmost importance. This study focuses on multi-step spatio-temporal wind speed forecasting for the Norwegian continental shelf. In particular, the study considers 14 offshore measurement stations and aims to leverage spatial dependencies through the relative physical location of different stations to improve local wind forecasts and simultaneously output different forecasts for each of the 14 locations. Our multi-step forecasting models produce either 10-minute, 1- or 4-hour forecasts, with 10-minute resolution, meaning that the models produce more informative time series for predicted future trends. A graph neural network (GNN) architecture was used to extract spatial dependencies, with different update functions to learn temporal correlations. These update functions were implemented using different neural network architectures. One such architecture, the Transformer, has become increasingly popular for sequence modelling in recent years. Various alterations have been proposed to better facilitate time series forecasting, of which this study focused on the Informer, LogSparse Transformer and Autoformer. This is the first time the LogSparse Transformer and Autoformer have been applied to wind forecasting and the first time any of these or the Informer have been formulated in a spatio-temporal setting for wind forecasting. By comparing against spatio-temporal Long Short-Term Memory (LSTM) and Multi-Layer Perceptron (MLP) models, the study showed that the models using the altered Transformer architectures as update functions in GNNs were able to outperform these. Furthermore, we propose the Fast Fourier Transformer (FFTransformer), which is a novel Transformer architecture based on signal decomposition and consists of two separate streams that analyse the trend and periodic components separately. The FFTransformer and Autoformer were found to achieve superior results for the 10-minute and 1-hour ahead forecasts, with the FFTransformer significantly outperforming all other models for the 4-hour ahead forecasts. Our code to implement the different models are made publicly available at: https://github.com/LarsBentsen/FFTransformer.

Se alle arbeider i NVA