Energy management of integrated energy system in the park under multiple time scales

Linrong Wang; Xiang Feng; Ruifen Zhang; Zhengran Hou; Guilan Wang; Haixiao Zhang; Linrong Wang; Xiang Feng; Ruifen Zhang; Zhengran Hou; Guilan Wang; Haixiao Zhang

doi:10.3934/energy.2024030

AIMS Energy

2024, Volume 12, Issue 3: 639-663. doi: 10.3934/energy.2024030

Previous Article Next Article

Research article Special Issues

Energy management of integrated energy system in the park under multiple time scales

1.
Ordos Power Supply Branch of Inner Mongolia Electric Power (Group) Co., Ltd., Inner Mongolia, Ordos, 017004, China
2.
School of Control and Computer Engineering, North China Electric Power University, Baoding 071000, China

Received: 20 March 2024 Revised: 11 May 2024 Accepted: 17 May 2024 Published: 22 May 2024

Considering the problem of time scale differences among subsystems in the integrated energy system of a park, as well as the increasing complexity of the system structure and number of control variables, there may be a deep reinforcement learning (DRL) "curse of dimensionality" problem, which hinders the further improvement of economic benefits and energy utilization efficiency of park-level integrated energy systems (PIES). This article proposes a reinforcement learning optimization algorithm for comprehensive energy PPO (Proximal Policy Optimization) in industrial parks considering multiple time scales for energy management. First, PIES are divided into upper and lower layers, the first containing power and thermal systems, and the second containing gas systems. The upper and lower layers of energy management models are built based on the PPO; then, both layers formulate the energy management schemes of the power, thermal, and gas systems in a long (30 min) and short time scale (6 min). Through confirmatory and comparative experiments, it is shown that the proposed method can not only effectively overcome the curse of dimensionality in DRL algorithms during training but can also develop different energy system management plans for PIES on a differentiated time scale, improving the overall economic benefits of the system and reducing carbon emissions.
- integrated energy system,
- deep reinforcement learning,
- proximal policy optimization,
- economic performance,
- multiple time scales
Citation: Linrong Wang, Xiang Feng, Ruifen Zhang, Zhengran Hou, Guilan Wang, Haixiao Zhang. Energy management of integrated energy system in the park under multiple time scales[J]. AIMS Energy, 2024, 12(3): 639-663. doi: 10.3934/energy.2024030

Related Papers:

Abstract

Considering the problem of time scale differences among subsystems in the integrated energy system of a park, as well as the increasing complexity of the system structure and number of control variables, there may be a deep reinforcement learning (DRL) "curse of dimensionality" problem, which hinders the further improvement of economic benefits and energy utilization efficiency of park-level integrated energy systems (PIES). This article proposes a reinforcement learning optimization algorithm for comprehensive energy PPO (Proximal Policy Optimization) in industrial parks considering multiple time scales for energy management. First, PIES are divided into upper and lower layers, the first containing power and thermal systems, and the second containing gas systems. The upper and lower layers of energy management models are built based on the PPO; then, both layers formulate the energy management schemes of the power, thermal, and gas systems in a long (30 min) and short time scale (6 min). Through confirmatory and comparative experiments, it is shown that the proposed method can not only effectively overcome the curse of dimensionality in DRL algorithms during training but can also develop different energy system management plans for PIES on a differentiated time scale, improving the overall economic benefits of the system and reducing carbon emissions.

References

[1]	Feng J, Nan J, Wang C, et al. (2022) Source-load coordinated low-carbon economic dispatch of electric-gas integrated energy system based on carbon emission flow theory. Energies 15: 3641–3652. https://doi.org/10.3390/en15103641 doi: 10.3390/en15103641
[2]	Bhowmik C, Bhowmik S, Ray A, et al. (2017) Optimal green energy planning for sustainable development: A review. Renewable Sustainable Energy Rev 71: 796–813. https://doi.org/10.1016/j.rser.2016.12.105 doi: 10.1016/j.rser.2016.12.105
[3]	Li P, Wu D, Li Y, et al. (2020) A multi-objective union optimal configuration strategy for multi-microgrid integrated energy system considering bargaining game. Power Syst Tech 44: 3680–3690. https://doi.org/10.1016/p.st.20203680 doi: 10.1016/p.st.20203680
[4]	Lv J, Zhang S, Cheng H, et al. (2021) Review on district-level integrated energy system planning considering interconnection and interaction. Pro CSEE 41: 4001–4021. https://doi.org/10.3390/en20214001 doi: 10.3390/en20214001
[5]	Yu X, Xu X, Chen S, et al. (2016) A brief review to integrated energy system and energy internet. Trans China Electro Society 31: 1–13. https://doi.org/10.1016/eprint/104480 doi: 10.1016/eprint/104480
[6]	Ding T, Jia W, Shahidehpour M, et al. (2022) Review of optimization methods for energy hub planning, operation, trading, and control. IEEE Trans Sustainable Energy 13: 1802–1818. https://doi.org/10.1109/TSTE.2022.3172004 doi: 10.1109/TSTE.2022.3172004
[7]	Khodadadi A, Abedinzadeh T, Alipour H, et al. (2023) Optimal operation of energy hub systems under resiliency response options. J Electr Comput Eng 20: 23–36. https://doi.org/10.1155/2023/2590362 doi: 10.1155/2023/2590362
[8]	Song D, Meng W, Dong M, et al. (2022) A critical survey of integrated energy system: Summaries, methodologies and analysis. Energy Convers Manage 266: 58–63. https://doi.org/10.1016/j.enconman.2022.115863 doi: 10.1016/j.enconman.2022.115863
[9]	Jiang X, Sun C, Cao L, et al. (2022) Semi-decentralized energy routing algorithm for minimum-loss transmission in community energy internet. Int J Electrical Power Energy Syst 135: 35–47. https://doi.org/10.1016/j.ijepes.2021.107547 doi: 10.1016/j.ijepes.2021.107547
[10]	Yang M, Cui Y, Huang D, et al. (2022) Multi-time-scale coordinated optimal scheduling of integrated energy system considering frequency out-of-limit interval. Inter J Elect Power Energy Syst 141: 68–81. https://doi.org/10.1016/j.ijepes.2022.108268 doi: 10.1016/j.ijepes.2022.108268
[11]	Hu K, Wang B, Cao S, et al. (2022) A novel model predictive control strategy for multi-time scale optimal scheduling of integrated energy system. Energy Rep 8: 7420–7433. https://doi.org/10.1016/j.egyr.2022.05.184 doi: 10.1016/j.egyr.2022.05.184
[12]	Li X, Wang W, Wang H (2021) Hybrid time-scale energy optimal scheduling strategy for integrated energy system with bilateral interaction with supply and demand. Appl Energy 285: 458–463. https://doi.org/10.1016/j.apenergy.2021.116458 doi: 10.1016/j.apenergy.2021.116458
[13]	Li P, Guo T, Abeysekera M, et al. (2021) Intraday multi-objective hierarchical coordinated operation of a multi-energy system. Energy 228: 5–28. https://doi.org/10.1016/j.energy.2021.120528 doi: 10.1016/j.energy.2021.120528
[14]	Cheng S, Wang R, Xu J, et al. (2021) Multi-time scale coordinated optimization of an energy hub in the integrated energy system with multi-type energy storage systems. Sustainable Energy Technol Assess 47: 327–335. https://doi.org/10.1016/j.seta.2021.101327 doi: 10.1016/j.seta.2021.101327
[15]	Zhang B, Hu W, Li J, et al. (2020) Dynamic energy conversion and management strategy for an integrated electricity and natural gas system with renewable energy: Deep reinforcement learning approach. Energy Convers Manage 220: 63–75. https://doi.org/10.1016/j.enconman.2020.113063 doi: 10.1016/j.enconman.2020.113063
[16]	Xu Z, Han G, Liu L, et al. (2021) Multi-energy scheduling of an industrial integrated energy system by reinforcement learning-based differential evolution. IEEE Trans Green Commun Netw 5: 1077–1090. https://doi.org/10.1109/TGCN.2021.3061789 doi: 10.1109/TGCN.2021.3061789
[17]	Foruzan E, Soh LK, Asgarpoor S (2018) Reinforcement learning approach for optimal distributed energy management in a microgrid. IEEE Trans Power Syst 33: 5749–5758. https://doi.org/10.1109/TPWRS.2018.2823641 doi: 10.1109/TPWRS.2018.2823641
[18]	Gorostiza FS, Gonzalez-Longatt FM (2020) Deep reinforcement learning-based controller for SOC management of multi-electrical energy storage system. IEEE Trans Smart Grid 11: 5039–5050. https://doi.org/10.1109/TSG.2020.2996274 doi: 10.1109/TSG.2020.2996274
[19]	Zhang X, Liu Y, Duan J, et al. (2021) DDPG-based multi-agent framework for SVC tuning in urban power grid with renewable energy resources. IEEE Trans Power Syst 36: 5465–5475. https://doi.org/10.1109/TPWRS.2021.3081159 doi: 10.1109/TPWRS.2021.3081159
[20]	Zhu X, Yang J, Liu Y, et al. (2019) Optimal scheduling method for a regional integrated energy system considering joint virtual energy storage. IEEE Access 7: 138260–138272. https://doi.org/10.1109/ACCESS.2020.3046743 doi: 10.1109/ACCESS.2020.3046743
[21]	Li Y, Zhang F, Li Y, et al. (2021) An improved two-stage robust optimization model for CCHP-P2G microgrid system considering multi-energy operation under wind power outputs uncertainties. Energy 223: 48–60. https://doi.org/10.1016/j.energy.2021.120048 doi: 10.1016/j.energy.2021.120048
[22]	Fotopoulou M, Pediaditis P, Skopetou N, et al. (2024) A Review of the Energy Storage Systems of Non-Interconnected European Islands. Sustainability 16: 1572. https://doi.org/10.3390/su16041572 doi: 10.3390/su16041572
[23]	Rious V, Perez Y (2014) Review of supporting scheme for island power system storage. Renewable Sustainable Energy Rev 29: 754–765. https://doi.org/10.1016/j.rser.2013.08.015 doi: 10.1016/j.rser.2013.08.015
[24]	Guo M, Mu Y, Jia H, et al. (2021) Electric/thermal hybrid energy storage planning for park-level integrated energy systems with second-life battery utilization. Adva Appl Energy 4: 64–75. https://doi.org/10.1016/j.adapen.2021.100064 doi: 10.1016/j.adapen.2021.100064
[25]	Li Z, Zhang F, Liang J, et al. (2015) Optimization on microgrid with combined heat and power system. Proc CSEE 35: 3569–3576. https://doi.org/10.13334/j.0258-8013.pcsee.2015.14.011 doi: 10.13334/j.0258-8013.pcsee.2015.14.011
[26]	Zhou S, Hu Z, Gu W, et al. (2020) Combined heat and power system intelligent economic dispatch: A deep reinforcement learning approach. Inter J Electrical Power Energy Syst 120: 106016. https://doi.org/10.1016/j.ijepes.2020.106016 doi: 10.1016/j.ijepes.2020.106016
[27]	Yang HZ, Li ML, Jiang ZY, et al. (2020) Multi-time scale optimal scheduling of regional integrated energy systems considering integrated demand response. IEEE Access 8: 5080–5090. https://doi.org/10.1109/ACCESS.2019.2963463 doi: 10.1109/ACCESS.2019.2963463
[28]	Yang T, Zhao L, Liu Y, et al. (2021) Dynamic economic scheduling of integrated energy systems based on deep reinforcement learning. Power Syst Autom 45: 39–47. https://doi.org/10.7500/AEPS20200405004 doi: 10.7500/AEPS20200405004
[29]	Dong J, Wang HX, Zhou XR, et al. (2023) Low carbon economic dispatch of electricity gas heat integrated energy system considering comprehensive demand response. J North China Electr Power Univ, Nat Sci Ed 50: 81–90. https://doi.org/10.3969/j.ISSN.1007-2691.2023.03.08 doi: 10.3969/j.ISSN.1007-2691.2023.03.08

Reader Comments

Your name:*

Email:*
© 2024 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)