Single Phase Grid Connector Solar PV Systems Energy Management Using Deep Reinforcement Learning with Partial Shedding Impact
Keywords:
Deep Reinforcement Learning, Energy Management System, DQN, Solar PV, Battery Optimization, CO₂ Emission Reduction, Smart Grid, Partial Shading, Cost Minimization, Renewable Energy.Abstract
This study introduces an energy management system (EMS) for a single-phase grid-connected solar photovoltaic (PV) system that operates in partial shade using Deep Reinforcement Learning (DRL). Making use of the Deep Q-Network (DQN) process, the system learns to optimize battery charging and discharging decisions to minimize operational costs and CO₂ emissions. The U.S. Dept. of Energy's Open Energy Data Initiative provided high-resolution, minute-level data from real home load profiles for this simulation, ensuring the evaluation reflects practical usage scenarios. The DRL agent was trained and deployed in a Python-based environment with support for advanced hardware acceleration. Performance was evaluated in terms of energy cost, CO₂ emissions, energy source distribution, and savings, with the proposed DRL-based EMS showing superior results compared to traditional Fuzzy Logic, PSO-based, and GA-based EMS models. The DRL approach achieved up to 34.24% cost reduction and 41.10% CO₂ emission reduction, outperforming all baseline strategies. Statistical analysis confirmed the significance of these improvements (p = 0.000). The results demonstrate the practical potential of DRL in enabling intelligent, adaptive, and environmentally conscious energy management in modern smart grid systems.
Downloads
References
Q. Li, C. Monticelli, and A. Zanelli, “Life cycle assessment of organic solar cells and perovskite solar cells with graphene transparent electrodes,” Renew. Energy, vol. 195, pp. 906–917, 2022.
U. Otamendi, I. Martinez, M. Quartulli, I. G. Olaizola, E. Viles, and W. Cambarau, “Segmentation of cell-level anomalies in electroluminescence images of photovoltaic modules,” Sol. Energy, vol. 220, pp. 914–926, 2021.
A. Demirci, I. Dagal, S. M. Tercan, H. Gundogdu, M. Terkes, and U. Cali, “Enhanced ANN-Based MPPT for Photovoltaic Systems: Integrating Metaheuristic and Analytical Algorithms for Optimal Performance Under Partial Shading,” IEEE Access, 2025.
A. Koç, H. Yağlı, H. H. Bilgic, Y. Koç, and A. Özdemir, “Performance analysis of a novel organic fluid filled regenerative heat exchanger used heat recovery ventilation (OHeX-HRV) system,” Sustain. Energy Technol. Assessments, vol. 41, p. 100787, 2020.
J. Yang, J. Chang, X. Cao, Y. Wang, and J. Yao, “Energy, environmental, and socioeconomic potential benefit of straw resources utilization under water availability limit condition,” J. Clean. Prod., vol. 392, p. 136274, 2023.
V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, and G. Ostrovski, “Human-level control through deep reinforcement learning,” Nature, vol. 518, no. 7540, pp. 529–533, 2015.
A. N. Bishop, P. Del Moral, and A. Niclas, “An introduction to Wishart matrix moments,” Found. Trends® Mach. Learn., vol. 11, no. 2, pp. 97–218, 2018.
A. Osborne, J. Dorville, and P. Romano, “Upsampling Monte Carlo neutron transport simulation tallies using a convolutional neural network,” Energy AI, vol. 13, p. 100247, 2023.
M. Rezaeimozafar, M. Duffy, R. F. D. Monaghan, and E. Barrett, “A hybrid heuristic-reinforcement learning-based real-time control model for residential behind-the-meter PV-battery systems,” Appl. Energy, vol. 355, p. 122244, 2024.
Z. Li, H. Lei, A. Kan, H. Xie, and W. Yu, “Photothermal applications based on graphene and its derivatives: A state-of-the-art review,” Energy, vol. 216, p. 119262, 2021.
S. S. Reka, P. Venugopal, H. H. Alhelou, P. Siano, and M. E. H. Golshan, “Real time demand response modeling for residential consumers in smart grid considering renewable energy with deep learning approach,” IEEE access, vol. 9, pp. 56551–56562, 2021.
E. Koutroulis and F. Blaabjerg, “A new technique for tracking the global maximum power point of PV arrays operating under partial-shading conditions,” IEEE J. photovoltaics, vol. 2, no. 2, pp. 184–190, 2012.
B. Yang, H. Ye, J. Wang, J. Li, S. Wu, Y. Li, H. Shu, Y. Ren, and H. Ye, “PV arrays reconfiguration for partial shading mitigation: Recent advances, challenges and perspectives,” Energy Convers. Manag., vol. 247, p. 114738, 2021.
M. S. Lilburn, J. R. Griffin, and M. Wick, “From muscle to food: oxidative challenges and developmental anomalies in poultry breast muscle,” Poult. Sci., vol. 98, no. 10, pp. 4255–4260, 2019.
M. V. Da Rocha, L. P. Sampaio, and S. A. O. da Silva, “Comparative analysis of MPPT algorithms based on Bat algorithm for PV systems under partial shading condition,” Sustain. Energy Technol. Assessments, vol. 40, p. 100761, 2020.
S. S. T. Husein, “Enhanced Carrier Mobility in Hydrogenated and Amorphous Transparent Conducting Oxides.” Arizona State University, 2020.
S. Al-Dahidi, M. Madhiarasan, L. Al-Ghussain, A. M. Abubaker, A. D. Ahmad, M. Alrbai, M. Aghaei, H. Alahmer, A. Alahmer, and P. Baraldi, “Forecasting solar photovoltaic power production: A comprehensive review and innovative data-driven modeling framework,” Energies, vol. 17, no. 16, p. 4145, 2024.
V. François-Lavet, P. Henderson, R. Islam, M. G. Bellemare, and J. Pineau, “An introduction to deep reinforcement learning,” Found. Trends® Mach. Learn., vol. 11, no. 3–4, pp. 219–354, 2018.
C. Guo, X. Wang, Y. Zheng, and F. Zhang, “Optimal energy management of multi-microgrids connected to distribution system based on deep reinforcement learning,” Int. J. Electr. power energy Syst., vol. 131, p. 107048, 2021.
S. T. Spantideas, A. E. Giannopoulos, and P. Trakadas, “Autonomous Price-aware Energy Management System in Smart Homes via Actor-Critic Learning with Predictive Capabilities,” IEEE Trans. Autom. Sci. Eng., 2025.
K. Yang, M. Quddus, and C. Antoniou, “Developing a new real-time traffic safety management framework for urban expressways utilizing reinforcement learning tree,” Accid. Anal. Prev., vol. 178, p. 106848, 2022.
P. Soni and V. Dave, “Enhancing Solar PV-Fuel Cell Hybrid Systems Through AI-Driven Performance Optimization,” in Building Business Models with Machine Learning, IGI Global Scientific Publishing, 2025, pp. 137–158.
M. Mao, L. Cui, Q. Zhang, K. Guo, L. Zhou, and H. Huang, “Classification and summarization of solar photovoltaic MPPT techniques: A review based on traditional and intelligent control strategies,” Energy reports, vol. 6, pp. 1312–1327, 2020.
J. Aldahmashi and X. Ma, “Real-time energy management in smart homes through deep reinforcement learning,” IEEE Access, 2024.
R. K. Pachauri, O. P. Mahela, A. Sharma, J. Bai, Y. K. Chauhan, B. Khan, and H. H. Alhelou, “Impact of partial shading on various PV array configurations and different modeling approaches: A comprehensive review,” IEEE Access, vol. 8, pp. 181375–181403, 2020.
V. Jain, R. Singh, R. Yadav, V. K. Yadav, V. Kumar, and S. Garg, “Multi-step optimization for reconfiguration of solar PV array for optimal shade dispersion,” Electr. Eng., pp. 1–37, 2024.
M. Wu, X. Zhang, M. Wang, K. Hu, and P. Wang, “A matching scheduling control strategy based on modulation wave reconstruction for the single-phase photovoltaic cascaded multilevel inverter,” IEEE J. Emerg. Sel. Top. Power Electron., vol. 11, no. 4, pp. 3899–3909, 2023.
Kandukuri Pradeep, Dr. K. Vinay Kumar ,"Observer based quardrature signal generator for UPQC in single phase distribution system ", ITM web of confernce 50,03002(2022)-ICAECT-2022.
Kandukuri Pradeep, Dr. K. Vinay Kumar,"Single-Phase Grid-Connected Photovoltaic Systems using aDeep Reinforcement based MPPT Algorithm with Grey-Wolf Optimization under partial shading condition", Eksplorium journal Volume46,No.1,May 2025,Page 1102-1129,pISSN0854-1418,e-ISSN 2503-426X.
Single phase grid connected photovoltaic systems using an ANFIS MPPT Algorithm with grey wolf optimisation"2023 IEEE 3rd international conference on smart Technologies for power ,energy and control (STPEC)Bhubaneswar India,pp1-6.
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
