Both wind and solar power, the two most rapidly expanding types of renewable power, are driven by local weather conditions, in particular wind speed, temperature, and solar irradiance. The ability to forecast these weather data, together with accurate generator models, thus enables wind and solar power production forecasts. Statistical forecasting methods can contribute to short-term forecasting, which plays a particularly important role for preventing negative impacts of large-scale renewables integration in power systems.
The goal of this thesis is to develop efficient storage systems for big weather data using the Hadoop stack [1], in particular HBase [2], and a fully distributed cluster implementation. The database will be populated with actual weather data from publicly available sources, in particular using a web service offered by the Norwegian Meteorological Institute [3]. The functionality of the big data storage should be demonstrated by a number of standard queries, such as relevant aggregations and visualizations. If time admits, the data can be leveraged to test innovative machine learning based forecasting techniques.
Description of the tasks
- Set up distributed Hadoop and HBase on cluster
- Design database
- Populate database with publicly available data
- Specify test queries and measure performance
Prerequisites
- Interest in big data management with relecant applications
- Solid Java coding skills
This project will be carried out collaboratively with Technical University of Munich (TUM), Germany with professor Hans-Arno Jacobsen from TUM as co-supervisor. The master student will be given the opportunity to visit and work closely with researchers and students from professor Jacobsen´s group on Energy Informatics. A visit can last for up to one semester and will be funded by a new project on Energy Informatics at IFI. The teaching and working language at TUM will be english. No proficiency in german is required.
References
- Apache Hadoop Video Tutorial. http://www.youtube.com/channel/UCOKFIXDVK-ntuaFMb8IPqcw
- Lars George. HBase – The Definite Guide. O’Reilly. 2011.
- api.met.no. http://api.met.no/#english
- E. Lorenz, T. Scheidsteger, J. Hurka, D. Heinemann, C. Kurz. Regional PV Power Prediction for Improved Grid Integration. Progress in Photovoltaics: Research and Applications (19), pp. 757-771. 2011.
- J. Huang et al. Wind Energy Forecasting: A Review of State-of-the-Art and Recommendations for Better Forecasts. California Renewable Energy Forecasting, Resource Data and Mapping. http://uc-ciee.org/downloads/appendixB.pdf