Forecasting of Short-Term Weather Parameters Using Attention-Based Recurrent Neural Network Model
Keywords:
Attention mechanism, Encoder-decoder architecture, Recurrent Neural Network, Short-term weather forecasting, Weather parameters interdependenciesAbstract
Short-term weather forecasting refers to the prediction of meteorological conditions over a relatively short period. The development of short-term weather forecasting is assessed as having the potential to facilitate the development of dynamic models or methods within the local weather forecasting system. In this research, we propose the incorporation of an attention mechanism into the encoder-decoder architecture of Recurrent Neural Network (RNN)-based models. The purpose of adding this attention mechanism is to enable the model to acquire knowledge regarding the interdependencies among weather parameters, thereby facilitating the capture of abrupt weather fluctuations. This research specifically focuses on the prediction of several weather parameters, including temperature, relative humidity, and wind speed. In this study, a performance comparison was conducted among several types of RNN-based models, namely Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), Bidirectional LSTM (Bi-LSTM), Bidirectional GRU (Bi-GRU), and the combination of each RNN model with an attention mechanism. The research results indicate that the model with the best performance is the Attention GRU for temperature prediction, with an RMSE value of 0.02681. For relative humidity prediction, the Attention Bi-LSTM performs the best with an RMSE value of 0.18343, and the Attention Bi-GRU achieves the highest performance for wind speed prediction with an RMSE value of 0.00395. The outcomes of this investigation demonstrate the efficacy of the attention mechanism in enhancing the accuracy of several encoder-decoder RNN models.
Downloads
References
Balasubramanian A. Weather Forecasting. Mysore: Department of Studies in Earth Science University of Mysore. 2017.
Lim B, Zohren S. Time-series forecasting with deep learning: a survey. Philosophical Transactions of the Royal Society A. 2021;379(2194):20200209.
Schultz MG, Betancourt C, Gong B, Kleinert F, Langguth M, Leufen LH, et al. Can deep learning beat numerical weather prediction? Philosophical Transactions of the Royal Society A. 2021;379(2194):20200097.
Iseh A, Woma T. Weather forecasting models, methods and applications. Int J Eng Res Technol. 2013;2(12):1945–1957.
Jaseena K, Kovoor BC. Deterministic weather forecasting models based on intelligent predictors: A survey. Journal of King Saud University-Computer and Information Sciences. 2022;34(6):3393–3412.
Abdillah MR, Sarli PW, Firmansyah HR, Sakti AD, Fajary FR, Muharsyah R, et al. Extreme Wind Variability and Wind Map Development in Western Java, Indonesia. International Journal of Disaster Risk Science. 2022;13(3):465–480.
Hou J, Wang Y, Zhou J, Tian Q. Prediction of hourly air temperature based on CNN– LSTM. Geomatics, Natural Hazards and Risk. 2022;13(1):1962–1986.
Hanoon MS, Ahmed AN, Zaini N, Razzaq A, Kumar P, Sherif M, et al. Developing machine learning algorithms for meteorological temperature and humidity forecasting at Terengganu state in Malaysia. Scientific Reports. 2021;11(1):18935.
Sahasrabuddhe DV, Jamsandekar P. Data structure for representation of big data of weather forecasting: a review. International Journal of Computer Science Trends and Technology (IJCST). 2015;3(6):48–56.
Perry GL, Seidl R, Bellv´e AM, Rammer W. An outlook for deep learning in ecosystem science. Ecosystems. 2022;25(8):1700–1718.
Hewage P, Trovati M, Pereira E, Behera A. Deep learning-based effective fine-grained weather forecasting model. Pattern Analysis and Applications. 2021;24(1):343–366.
Suleman MAR, Shridevi S. Short-term weather forecasting using spatial feature attention based LSTM model. IEEE Access. 2022;10:82456–82468.
Kreuzer D, Munz M, Schlu¨ter S. Short-term temperature forecasts using a convolutional neural network—An application to different weather stations in Germany. Machine Learning with Applications. 2020;2:100007.
De Saa E, Ranathunga L. Comparison between arima and deep learning models for temperature forecasting. arXiv preprint arXiv:201104452. 2020;.
LeCun Y, Bengio Y, Hinton G. Deep learning. nature. 2015;521(7553):436–444.
Salman AG, Heryadi Y, Abdurahman E, Suparta W. Single layer & multi-layer long short-term memory (LSTM) model with inter- mediate variables for weather forecasting. Procedia Computer Science. 2018;135:89–98.
Li Y, Zhu Z, Kong D, Han H, Zhao Y. EA- LSTM: Evolutionary attention-based LSTM for time series prediction. Knowledge-Based Systems. 2019;181:104785.
Grossberg S. Recurrent neural networks. Scholarpedia. 2013;8(2):1888.
Salehinejad H, Sankar S, Barfett J, Colak E, Valaee S. Recent advances in recurrent neural networks. arXiv preprint arXiv:180101078. 2017;.
Siami-Namini S, Tavakoli N, Namin AS. The performance of LSTM and BiLSTM in fore- casting time series. In: 2019 IEEE Inter- national conference on big data (Big Data). IEEE; 2019. p. 3285–3292.
Zaytar MA, El Amrani C. Sequence to sequence weather forecasting with long short- term memory recurrent neural networks. Inter- national Journal of Computer Applications. 2016;143(11):7–11.
Lin Y, Koprinska I, Rana M. Temporal convolutional neural networks for solar power forecasting. In: 2020 International Joint Conference on Neural Networks (IJCNN). IEEE; 2020. p. 1–8.
Xiong C, Merity S, Socher R. Dynamic memory networks for visual and textual question answering. In: International conference on machine learning. PMLR; 2016. p. 2397–2406.
Liu X, Wang Y, Wang X, Xu H, Li C, Xin
X. Bi-directional gated recurrent unit neural network based nonlinear equalizer for coherent optical communication system. Optics Express. 2021;29(4):5923–5933.
Bahdanau D, Cho K, Bengio Y. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:14090473. 2014;.
Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, et al. Attention is all you need. Advances in neural information processing systems. 2017;30.
Shahriari B, Swersky K, Wang Z, Adams RP, De Freitas N. Taking the human out of the loop: A review of Bayesian optimization. Proceedings of the IEEE. 2015;104(1):148–175.
Jin XB, Zheng WZ, Kong JL, Wang XY, Bai YT, Su TL, et al. Deep-learning fore- casting method for electric power load via attention-based encoder-decoder with bayesian optimization. Energies. 2021;14(6):1596.
Chai T, Draxler RR. Root mean square error (RMSE) or mean absolute error (MAE)?– Arguments against avoiding RMSE in the literature. Geoscientific model development. 2014;7(3):1247–1250.
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
