Processing math: 86%
Research article Topical Sections

LSTM-SAC reinforcement learning based resilient energy trading for networked microgrid system


  • Received: 24 September 2024 Revised: 25 February 2025 Accepted: 04 March 2025 Published: 25 March 2025
  • On the whole, the present microgrid constitutes numerous actors in highly decentralized environments and liberalized electricity markets. The networked microgrid system must be capable of detecting electricity price changes and unknown variations in the presence of rare and extreme events. The networked microgrid system comprised of interconnected microgrids must be adaptive and resilient to undesirable environmental conditions such as the occurrence of different kinds of faults and interruptions in the main grid supply. The uncertainties and stochasticity in the load and distributed generation are considered. In this study, we propose resilient energy trading incorporating DC-OPF, which takes generator failures and line outages (topology change) into account. This paper proposes a design of Long Short-Term Memory (LSTM) - soft actor-critic (SAC) reinforcement learning for the development of a platform to obtain resilient peer-to-peer energy trading in networked microgrid systems during extreme events. A Markov Decision Process (MDP) is used to develop the reinforcement learning-based resilient energy trade process that includes the state transition probability and a grid resilience factor for networked microgrid systems. LSTM-SAC continuously refines policies in real-time, thus ensuring optimal trading strategies in rapidly changing energy markets. The LSTM networks have been used to estimate the optimal Q-values in soft actor-critic reinforcement learning. This learning mechanism takes care of the out-of-range estimates of Q-values while reducing the gradient problems. The optimal actions are decided with maximized rewards for peer-to-peer resilient energy trading. The networked microgrid system is trained with the proposed learning mechanism for resilient energy trading. The proposed LSTM-SAC reinforcement learning is tested on a networked microgrid system comprised of IEEE 14 bus systems.

    Citation: Desh Deepak Sharma, Ramesh C Bansal. LSTM-SAC reinforcement learning based resilient energy trading for networked microgrid system[J]. AIMS Electronics and Electrical Engineering, 2025, 9(2): 165-191. doi: 10.3934/electreng.2025009

    Related Papers:

    [1] Wenlong Sun . The boundedness and upper semicontinuity of the pullback attractors for a 2D micropolar fluid flows with delay. Electronic Research Archive, 2020, 28(3): 1343-1356. doi: 10.3934/era.2020071
    [2] Keqin Su, Rong Yang . Pullback dynamics and robustness for the 3D Navier-Stokes-Voigt equations with memory. Electronic Research Archive, 2023, 31(2): 928-946. doi: 10.3934/era.2023046
    [3] Xin-Guang Yang, Lu Li, Xingjie Yan, Ling Ding . The structure and stability of pullback attractors for 3D Brinkman-Forchheimer equation with delay. Electronic Research Archive, 2020, 28(4): 1395-1418. doi: 10.3934/era.2020074
    [4] Yadan Shi, Yongqin Xie, Ke Li, Zhipiao Tang . Attractors for the nonclassical diffusion equations with the driving delay term in time-dependent spaces. Electronic Research Archive, 2024, 32(12): 6847-6868. doi: 10.3934/era.2024320
    [5] Yangrong Li, Shuang Yang, Qiangheng Zhang . Odd random attractors for stochastic non-autonomous Kuramoto-Sivashinsky equations without dissipation. Electronic Research Archive, 2020, 28(4): 1529-1544. doi: 10.3934/era.2020080
    [6] Meiyu Sui, Yejuan Wang, Peter E. Kloeden . Pullback attractors for stochastic recurrent neural networks with discrete and distributed delays. Electronic Research Archive, 2021, 29(2): 2187-2221. doi: 10.3934/era.2020112
    [7] Pan Zhang, Lan Huang, Rui Lu, Xin-Guang Yang . Pullback dynamics of a 3D modified Navier-Stokes equations with double delays. Electronic Research Archive, 2021, 29(6): 4137-4157. doi: 10.3934/era.2021076
    [8] Lingrui Zhang, Xue-zhi Li, Keqin Su . Dynamical behavior of Benjamin-Bona-Mahony system with finite distributed delay in 3D. Electronic Research Archive, 2023, 31(11): 6881-6897. doi: 10.3934/era.2023348
    [9] Shu Wang, Mengmeng Si, Rong Yang . Dynamics of stochastic 3D Brinkman-Forchheimer equations on unbounded domains. Electronic Research Archive, 2023, 31(2): 904-927. doi: 10.3934/era.2023045
    [10] Guifen Liu, Wenqiang Zhao . Regularity of Wong-Zakai approximation for non-autonomous stochastic quasi-linear parabolic equation on RN. Electronic Research Archive, 2021, 29(6): 3655-3686. doi: 10.3934/era.2021056
  • On the whole, the present microgrid constitutes numerous actors in highly decentralized environments and liberalized electricity markets. The networked microgrid system must be capable of detecting electricity price changes and unknown variations in the presence of rare and extreme events. The networked microgrid system comprised of interconnected microgrids must be adaptive and resilient to undesirable environmental conditions such as the occurrence of different kinds of faults and interruptions in the main grid supply. The uncertainties and stochasticity in the load and distributed generation are considered. In this study, we propose resilient energy trading incorporating DC-OPF, which takes generator failures and line outages (topology change) into account. This paper proposes a design of Long Short-Term Memory (LSTM) - soft actor-critic (SAC) reinforcement learning for the development of a platform to obtain resilient peer-to-peer energy trading in networked microgrid systems during extreme events. A Markov Decision Process (MDP) is used to develop the reinforcement learning-based resilient energy trade process that includes the state transition probability and a grid resilience factor for networked microgrid systems. LSTM-SAC continuously refines policies in real-time, thus ensuring optimal trading strategies in rapidly changing energy markets. The LSTM networks have been used to estimate the optimal Q-values in soft actor-critic reinforcement learning. This learning mechanism takes care of the out-of-range estimates of Q-values while reducing the gradient problems. The optimal actions are decided with maximized rewards for peer-to-peer resilient energy trading. The networked microgrid system is trained with the proposed learning mechanism for resilient energy trading. The proposed LSTM-SAC reinforcement learning is tested on a networked microgrid system comprised of IEEE 14 bus systems.



    Fluid models with delay terms are crucial for understanding and predicting the behavior of fluids where devices or control mechanisms are inserted into fluid domains to manipulate certain properties, such as temperature or velocity. For example, in a wind tunnel experiment, various control mechanisms are used to manipulate the flow of air around a model. These control mechanisms can include flaps, fans, or other devices that alter the velocity or pressure distribution in specific regions of the wind tunnel, see [1,2]. Over the years, extensive research has been conducted on specific fluid models that incorporate delay terms, such as the Navier-Stokes with delay [3], the micropolar fluid with delay [4], the Kelvin-Voigt fluid with delay [5] and the viscoelastic fluid with delay [6], and so on.

    The bipolar fluid model is a well-known incompressible non-Newtonian fluid model that was introduced in [7,8]. It is commonly used to describe the motion of various materials, including molten plastics, synthetic fibers, paints, greases, polymer solutions, suspensions, adhesives, dyes, varnishes, and more. Accounting for the effect of the history-dependent behavior of the fluid, the bipolar fluid model can be described as follows

    ut+(u)u(μ(u)e(u)2μ1Δe(u))+p=f(t,x)+g(t,ut), (1.1)
    u=0, (1.2)

    where the velocity is denoted by u=u(t,x), the pressure by p=p(t,x), the non-delay external force by f(t,x), and the delayed external force by g(t,ut) with

    ut(s)=u(t+s,x),s(h,0),t>τ.

    The rate of the deformation tensor, denoted as e(u), is a 2×2 matrix defined by its components eij(u). These components can be written as

    eij(u)=12(uixj+ujxi),i,j=1,2.

    The variable viscosity μ(u) is taken as

    μ(u)=2μ0(η+|e|2)α/2,|e|2=2i,j=1|eij|2,i,j=1,2.

    The constitutive parameters h,η,μ0,μ1 are constants that satisfy h,η,μ0,μ1>0 and 0<α<1.

    From a physical point of view, Equations (1.1) and (1.2) are often subject to (see [9,10])

    u(τ,x)=uin,τR,xΣ, (1.3)
    u(t,x)=ϕin(tτ,x),(t,x)(τh,τ)×Σ, (1.4)
    u=0,τijknjnk=0,xΣ,i,j,k=1,2, (1.5)

    where Σ=R×(K,K) for some K>0, τijk=2μ1eijxk, (n1,n2) denotes the exterior unit normal to the boundary Σ, u=0 represents the non-slip condition and τijknjnk=0 means the first moment of the traction vanishes on Σ.

    Concerning Eqs (1.1)–(1.5) without the delay term g(t,ut) in 2D domains, there have been numerous studies on the well-posedness, regularity, and long time behavior of solutions. For example, we can refer to [11,12,13,14,15,16,17,18,19] in 2D bounded domains and [9,10,20,21] in 2D unbounded ones. However, as for Eqs (1.1)–(1.5) with the delay term g(t,ut) in 2D domains, most of the existing results related to Eqs (1.1)–(1.5) concentrate on 2D bounded domains. Zhao, Zhou and Li in [22] first established the existence, uniqueness of solutions and the existence of pullback attractors. Later on, different frameworks were used to study the existence and stability of stationary solutions and the existence of pullback attractors, as seen in [23,24,25]. Recently, the authors in [26] initially paid attention to the 2D unbounded domain. They showed the existence, uniqueness of solutions and the existence of pullback attractors. Therefore, it is natural to inquire about the relation between the pullback attractors in 2D bounded domains and the 2D unbounded domain as the bounded domains approximate the unbounded domain.

    It is well known that the upper semi-continuity of attractors, as introduced in [27], is a powerful concept for describing the relationship between attractors in nonlinear evolution equations. Several researchers have also established the upper semi-continuity of attractors for various physical models, as seen in [27,28,29,30], among others. In particular, Zhao et al. [16,21] have studied the upper semi-continuity of global attractors and cocycle attractors for the bipolar fluid without delay (i.e., g(t,ut)=0) with respect to the domains from Σn to Σ. The key idea is to unify the attractors by means of a natural extension. Specifically, for any nZ+, let unX(Σn) (where X is the phase space), define

    ¯un={un,xΣn,0,xΣΣn. (1.6)

    Then one can regard the function ¯un defined on Σ as the natural extension of the function un. Moreover, one has

    unX(Σ)=¯unX(Σ)=¯unX(Σn)=unX(Σn). (1.7)

    Borrowing the previous idea, in the present paper, we aim to establish the upper semi-continuity of pullback attractors for Eqs (1.1)–(1.5). Differently from [16,21], the presence of delay introduces new challenges in the new phase space, requiring us to perform more precise estimations. Actually, the result presented in this paper generalizes the results of [16,21].

    The paper is structured as follows. In Section 2, we introduce the necessary preliminaries, and in Section 3, proceed to establish the upper semi-continuity of pullback attractors. We conclude the main result in Section 4.

    Notation. In this paper, we use the following notations: R for the set of real numbers, Z+ for the set of non-negative integers and c as a generic constant, which may vary depending on the context. If the dependence needs to be explicitly emphasized, some notations like c1 and c() will be used. O represents either Σ or Σn. We denote the 2D vector Lebesgue space and 2D vector Sobolev space as Lp(O) and Hm(O), respectively. The norms for these spaces are Lp(O) and Hm(O). Additionally, we define the following spaces: V(O) as the set of ϕC0(O)×C0(O) satisfying ϕ=(ϕ1,ϕ2) and ϕ=0, H(O) as the closure of V(O) in L2(O) with the norm H(O) and dual space H(O)=H(O), and V(O) as the closure of V(O) in H2(O) with the norm V(O) and dual space V(O). The inner product in H(O) or L2(O) is denoted by (,). The dual pairing between V(O) and V(O) is denoted by ,. The Hausdorff semi-distance between Λ1Λ and Λ2Λ is denoted by DistΛ(Λ1,Λ2), which is defined as DistΛ(Λ1,Λ2)=supxΛ1infyΛ2xyΛ.

    In this section, we begin by reformulating Eqs (1.1)–(1.5) in an abstract form. We then recall the global existence, uniqueness of solutions and the existence of pullback attractors in the channel Σ and in sub-domains Σn.

    For the purpose of abstract representation, we introduce the following operators for Eqs (1.1)–(1.5):

    Au,v=2i,j,k=1Oeij(u)xkeij(v)xkdx,u,vV(O),N(u),v=2i,j=1Oμ(u)eij(u)eij(v)dx,u,vV(O),b(u,v,w)=2i,j=1Ouivjxiwjdx,B(u),w=b(u,u,w),u,v,wV(O).

    With the help of above notation, the weak form of Eqs (1.1)–(1.5) can be expressed when O=Σ as follows (see [22,23,26])

    ut+2μ1Au+B(u)+N(u)=f(t)+g(t,ut), (2.1)
    u(τ,x)=uin,xΣ, (2.2)
    u(t,x)=ϕin(tτ)=ϕin,t(τh,τ),xΣ, (2.3)

    and in the case of O=Σn as follows

    u(n)t+2μ1Au(n)+B(u(n))+N(u(n))=f(t)+g(t,u(n)t), (2.4)
    u(n)(τ,x)=u(n),in,xΣn, (2.5)
    u(n)(t,x)=ϕ(n),in(tτ,x)=ϕ(n),in,t(τh,τ),xΣn. (2.6)

    According to the definition of the operator A, the following estimate has been established in [9,10].

    Lemma 2.1. There exists a constant c1, which is dependent only on O, such that

    c1u2V(O)Au,uu2V(O).

    From now on, we should consider X=H,V,V and spaces

    CH(O)=C([h,0];H(O)),L2X(O)=L2(h,0;X(O)),E2X(O)=X(O)×L2X(O).

    In order to ensure the global existence, uniqueness of solutions and existence of pullback attractors for Eqs (2.1)–(2.3) and (2.4)–(2.6), certain assumptions about the external forces need to be imposed. First, the time-delay function g:R×CH(O)L2(O) is required to satisfy:

    (H1) for any ξCH(O), the function tRg(t,ξ)L2(O) is measurable.

    (H2) g(t,0)=0 for all tR.

    (H3) there exists a constant Lg>0 such that for any tR, ξ,ηCH(O),

    g(t,ξ)g(t,η)L2(O)LgξηCH(O).

    (H4) there exists a constant Cg(0,2c1μ1) such that

    tτg(s,us)g(s,vs)2L2(O)dsC2gtτhu(s)v(s)2H(O)ds,

    and there exists a value γ(0,min{4c1μ12Cg,4c21μ1}) such that

    tτeγsg(s,us)2L2(O)dsC2gtτheγsu(s)2H(O)ds,

    for all tτ, u,vL2(τh,t;H(O)).

    Second, the non-delay function f satisfies:

    (H5) Assume that fL2b(R;H(O)) which means fL2loc(R;H(O)) with

    f2L2b=suptRt+1tf(s)2H(O)ds<+.

    Remark 1. According to the definitions of the spaces CH(O) and H(O), the inequality in (H3) is not directly related to the first inequality in (H4). These two inequalities are independent of each other and describe different aspects of the functions and their differences.

    Based on above assumptions, we can conclude that

    Theorem 2.2. ([26]) Let the conditions (H1)(H5) hold for the case of O=Σ.

    1) Given (uin,ϕin)E2H(Σ), there is a unique weak solution u=u(;τ,uin,ϕin) of Eqs (2.1)–(2.3) satisfying uC([τ,T];H(Σ))L2(τh,T;H(Σ))L2(τ,T;V(Σ)) and ddtuL2(τ,T;V(Σ)).

    2) The solution operators {U(t,τ)}tτ:E2H(Σ)E2H(Σ) defined by U(t,τ):(uin,ϕin)(u,ut) generate a continuous process in space E2H(Σ).

    3) The family ˆBH(Σ)={BH(Σ)(t)|tR} given by

    BH(Σ)(t)={(u,ϕ)H(Σ)×L2V(Σ)|(u,ϕ)H(Σ)×L2V(Σ)R1(t),ddtϕLV(Σ)R2(t)}

    is pullback absorbing for the process {U(t,τ)}tτ, where

    R21(t)=1+c(1+eγh)ϱγ(t),R22(t)=1+ce2γhϱγ(t)(1+ϱγ(t))+ctthf(s)2H(Σ)ds,ϱγ(t)=eγtteγsf(s)2H(Σ)ds.

    4) The process {U(t,τ)}tτ is pullback ˆBH(Σ)-asymptotically compact and possesses a unique pullback attractor A={A(t)|tR} in E2H(Σ) defined by

    A(t)=st¯τsU(t,τ)BH(Σ)(τ)E2H(Σ).

    In order to unify the pullback attractors in the channel Σ and in the sub-domains Σn, we consider the natural extensions Eqs (1.6) and (1.7) mentioned in Section 1. Then similar to Theorem 2.2, we can make slight modifications in [22] to get

    Theorem 2.3. Let the conditions {\rm(H1)–(H5)} hold for the case of O=Σ.

    1) Given (u(n),in,ϕ(n),in)E2H(Σn), there is a unique weak solution u(n)=u(n)(;τ,u(n),in,ϕ(n),in) of Eqs (2.4)(2.6) satisfying unC([τ,T];H(Σn))L2(τh,T;H(Σn))L2(τ,T;V(Σn)) and ddtu(n)L2(τ,T;V(Σn)).

    2) The solution operators {Un(t,τ)}tτ:E2H(Σn)E2H(Σn) defined by Un(t,τ):(u(n),in,ϕ(n),in)(u(n),u(n)t) generate a continuous process in space E2H(Σn).

    3) The family ˆBH(Σn)n={BH(Σn)n(t)|tR} given by

    BH(Σn)n(t)={(u,ϕ)H(Σn)×L2V(Σn)|(u,ϕ)H(Σn)×L2V(Σn)R1(t),ddtϕLV(Σn)R2(t)}

    is pullback absorbing for the process {Un(t,τ)}tτ. Note that R1(t) and R2(t) are the same in Theorem2.2.

    4) The process {Un(t,τ)}tτ is pullback ˆBH(Σn)-asymptotically compact and possesses a unique pullback attractor An={An(t)|tR} in E2H(Σn) defined by

    An(t)=st¯τsUn(t,τ)BH(Σn)(τ)E2H(Σn).

    Due to the definitions of R1(t),R2(t) and the fact

    ϱγ(t)=eγtteγsf(s)2H(Σ)ds=eγttt1eγsf(s)2H(Σ)ds+eγtt1t2eγsf(s)2H(Σ)ds+tt1f(s)2H(Σ)ds+eγt1t2f(s)2H(Σ)ds+=f2L2b1eγ, (2.7)

    we have that BH(Σ)(t) and BH(Σn)n(t) in E2H(Σ) are bounded uniformly with respect to the time t. Moreover, there exists a τ(t,ˆBH(Σ)) (independent of n) such that for ττ(t,ˆBH(Σ)),

    U(t,τ)BH(Σ)(τ)BH(Σ)(t), (2.8)
    Un(t,τ)BH(Σn)n(τ)BH(Σn)n(t)BH(Σ)(t). (2.9)

    In this section, our objective is to establish the upper semi-continuity of pullback attractors. Specifically, we aim to prove the following theorem:

    Theorem 3.1. Suppose the conditions (H1)(H5) hold for the case of O=Σ. Consider the families An={An(t)|tR} and A={A(t)|tR} as the pullback attractors for Eqs (1.1)(1.5) in the domains Σn and Σ, respectively. Then for any tR, we have

    limnDistH(Σ)×L2(h,0;H(Σ))(An(t),A(t))=0.

    To prove Theorem 3.1, it is crucial to establish the strong convergence in the space E2H(Σ) of any sequence {(u(n),ϕ(n))}n1, where (u(n),ϕ(n)) belongs to An(t), to some (u,ϕ) belonging to A(t).

    Firstly, we obtain two auxiliary lemmas.

    Lemma 3.2. Suppose the conditions (H1)(H5) hold for the case of O=Σ. Let {(u(n),in,ϕ(n),in)}n1 be a sequence in H(Σn)×L2V(Σn) and (uin,ϕin)H(Σ)×L2V(Σ) satisfying the weak convergences as n,

    (u(n),in,ϕ(n),in)(uin,ϕin)  in  H(Σ)×L2V(Σ), (3.1)
    ddtϕ(n),inddtϕin  in  L2V(Σ). (3.2)

    Then for any tτ, we obtain the following weak convergences as n,

    u(n)(t;τ,u(n),in,ϕ(n),in)u(t;τ,uin,ϕin)  in  H(Σ), (3.3)
    u(n)(;τ,u(n),in,ϕ(n),in)u(;τ,uin,ϕin)  in  L2(τh,t;V(Σ)). (3.4)

    Proof. The fundamental energy estimates for Eqs (2.4)–(2.6) can be derived using the same method as shown in [26, Equations (3.1), (3.9), (3.27)]. The following inequality holds for any τt,

    u(n)(t)2H(Σn)+ηeγttτeγsu(n)(s)2V(Σn)dsceγ(τt)(u(n),in,ϕ(n),in)2E2H(Σn)+β1eγttτeγsf(s)2H(Σn)ds, (3.5)

    where Cg and γ are the constants from (H4), β(0,4c1μ12Cgγ)andη=4c1μ12Cgγβ>0. We also have for any T>0 and τtT,

    ttTu(n)(s)2V(Σn)dsceγ(T+τt)(u(n),in,ϕ(n),in)2E2H(Σn)+ceγ(Tt)tτeγsf(s)2H(Σn)ds. (3.6)

    Furthermore, we obtain for any τt,

    tthddsu(n)2V(Σn)dsctthf(s)2H(Σn)ds+c(eγ(τt)(u(n),in,ϕ(n),in)E2H(Σn)+ϱγ(t))+c(eγ(τt)(u(n),in,ϕ(n),in)E2H(Σn)+ϱγ(t))2, (3.7)

    with ϱγ(t) given in Theorem 2.2.

    It follows from Eqs (3.5)–(3.7) that

    u(n)isboundedinL(τ,t;H(Σ))L2(τ,t;V(Σ)), (3.8)
    ddsu(n)isboundedinL2(τh,t;V(Σ)). (3.9)

    By Eq (3.8), we have the following weak star and weak convergences (for a subsequence)

    u(n)uinL(τ,t;H(Σ)), (3.10)
    u(n)uinL2(τ,t;V(Σ)). (3.11)

    By the standard method, we can demonstrate that the solution to Eqs (2.1)–(2.3) with initial data (uin,ϕin) is u. In fact, in view of Eqs (3.8) and (3.9), a subsequence can be extracted from the sequence u(n) and denoted as u(n) as well such that the following strong convergence holds

    u(n)uinL2(τ,t;H(Σr)), (3.12)

    where Σr is any bounded sub-domain of Σ. By virtue of Eq (3.12), we can apply the limit operation to Eqs (2.4)–(2.6) for u(n). This implies that u is the solution to Eqs (2.1)–(2.3) with the given initial conditions (uin,ϕin). By uniqueness we conclude that Eqs (3.10)–(3.12) hold for the whole sequence.

    Proof of Eq (3.4): It follows from Eqs (3.1) and (3.11) that Eq (3.4) holds.

    Proof of Eq (3.3): From Eq (3.10), we have for any vV(Σ) and a.e. tτ that

    (u(n),v)(u,v).

    Due to Eq (3.9), we obtain for all tτ and vV(Σ),

    (u(n)(t)u(n)(th),v)=tthddsu(n)(s),vdsvV(Σ)ddsu(n)L2(τh,t;V(Σ))h1/2. (3.13)

    In view of Eqs (3.9) and (3.13), we see (u(n),v) is uniformly bounded and equicontinuous on [τ,t]. Hence, we have for any vV(Σ) and all tτ,

    (u(n),v)(u,v).

    By taking advantage of the density of V(Σ) in H(Σ), we arrive at Eq (3.3).

    Similar to [26, Lemma 3.4] (also see [16,21]), we have the following tail estimate:

    Lemma 3.3. Let the conditions (H1)(H5) hold for the case of O=Σ and (u(n),in,ϕ(n),in)BH(Σn)n(τ). Then given ϵ>0, there are τn(ϵ)<t and rn(ϵ)>0 satisfying for any r[rn,n] and ττn,

    u(n)(t;τ,u(n),in,ϕ(n),in)2L2(ΣnΣr)ϵ.

    With the help of Lemmas 3.2 and 3.3, we will establish the key convergence of solutions via the energy method introduced by Ball [31] and developed by Moise, Rosa and Wang [32].

    Lemma 3.4. Let the conditions (H1)(H5) hold for the case of O=Σ. Then for each tR and any sequence

    {(u(n),ϕ(n))}n1with(u(n),ϕ(n))An(t),

    there is a subsequence of (using the same index) {(u(n),ϕ(n))}n1 and some (u,ϕ)A(t) such that the following strong convergence holds as n,

    (u(n),ϕ(n))(u,ϕ)  in  E2H(Σ). (3.14)

    Proof. In virtue of the invariance of the pullback attractor An, there is a sequence {(u(n),in,ϕ(n),in)}n1 with (u(n),in,ϕ(n),in)An(τ)An such that

    Un(t,τ)(u(n),in,ϕ(n),in)=(u(n),ϕ(n)). (3.15)

    Due to the compactness of An(τ), there is some subsequence (using the same index) of {(u(n),in,ϕ(n),in)}n1 and some (uin,ϕin)E2H(Σ) such that the following weak convergence holds as n,

    (u(n),in,ϕ(n),in)(uin,ϕin)inE2H(Σ). (3.16)

    For the sequence {(u(n),ϕ(n))}n1 with initial data {(u(n),in,ϕ(n),in)}n1 in Eq (3.16), the compactness of An(t) deduces that there is some (u,ϕ)E2H(Σ) satisfying the following weak convergence as n,

    (u(n),ϕ(n))(u,ϕ)inE2H(Σ). (3.17)

    As an analogy of the proof of Lemma 3.2, u is the solution to Eqs (2.1)–(2.3) with initial data (uin,ϕin). On the other hand, it follows from Eqs (2.8) and (2.9) that for any ττ(t,ˆBH(Σ)),

    An(t)=Un(t,τ)An(τ)BH(Σ)(t).

    Thanks to Eq (3.17), we have for any ττ(t,ˆBH(Σ)) that (u,ϕ)BH(Σ)(t) and thus (u,ϕ)A(t).

    The proof of Lemma 3.4 will be concluded by demonstrating that the convergence of Eq (3.17) is strong. We finish the proof in two steps.

    Step one: we show that as n,

    ϕ(n)ϕ   strongly in   L2H(Σ). (3.18)

    It follows from Eqs (2.8) and (2.9) that there is a τ(tk,ˆBH(Σ)) satisfying for any kZ+ and ττ(tk,ˆBH(Σ)),

    An(tk)=Un(tk,τ)An(τ)BH(Σ)(tk).

    Taking advantage of the definition of the pullback absorbing set ˆBH(Σ) and the diagonal argument, for each kZ+ and τ<tk, there is a (u(k),ϕ(k))A(tk) satisfying the following weak convergences (up to a subsequence) as n,

    Un(tk,τ)(u(n),in,ϕ(n),in)(u(k),ϕ(k))inH(Σ)×L2V(Σ), (3.19)
    ddtu(n)tk(;τ,u(n),in,ϕ(n),in)ddtϕ(k)  in  L2V(Σ). (3.20)

    Thus, Equations (3.15), (3.17) and (3.19) together with the fact that the limit is unique, yield

    (u,ϕ)=(u(0),ϕ(0)).

    Note that for any bounded set ΣrΣ, all the embeddings V(Σr)H(Σr)V(Σr) are compact. By Eqs (3.19) and (3.20) and Lemma 3.2, we obtain as n,

    u(n)tk(;τ,u(n),in,ϕ(n),in)ϕ(k)  strongly in   L2H(Σr).

    Consequently, for any positive ϵ, there is some n(ϵ,r,k)>0 satisfying

    u(n)tk(;τ,u(n),in,ϕ(n),in)ϕ(k)L2H(Σr)<ϵ3,nn(ϵ,r,k). (3.21)

    Also, since for each kZ+, ϕ(k)L2V(Σ)L2H(Σ) is a fixed element, there must exist some r1(k)>0 such that

    ϕ(k)L2H(ΣΣr)<ϵ3,rr1(k). (3.22)

    By Lemma 3.3, there exist r(ϵ,tk,ˆBH(Σ))>0 and τ(ϵ,tk,ˆBH(Σ))<tk satisfying for any r>r(ϵ,tk,ˆBH(Σ)) and ττ(ϵ,tk,ˆBH(Σ)),

    u(n)tk(;τ,u(n),in,ϕ(n),in)L2H(ΣΣr)hϵ3h=ϵ3. (3.23)

    Therefore, by Eqs (3.21)–(3.23), we choose r and n large enough and τ small enough so that

    u(n)tk(;τ,u(n),in,ϕ(n),in)ϕ(k)L2H(Σ)u(n)tk(;τ,u(n),in,ϕ(n),in)ϕ(k)L2H(ΣΣr)+u(n)tk(;τ,u(n),in,ϕ(n),in)ϕ(k)L2H(Σr)u(n)tk(;τ,u(n),in,ϕ(n),in)L2H(ΣΣr)+ϕ(k)L2H(ΣΣr)+u(n)tk(;τ,u(n),in,ϕ(m),in)ϕ(k)L2H(Σr)ϵ3+ϵ3+ϵ3=ϵ,

    which implies that for each kZ+, it holds as n,

    u(n)tk(;τ,u(n),in,ϕ(n),in)ϕ(k)  strongly in  L2H(Σ). (3.24)

    Particularly, taking k=0, we prove the claim of Eq (3.18).

    Step two: we aim to prove that as n, the following strong convergence holds:

    u(n)(t;τ,u(n),in,ϕ(n),in)u  in  H(Σ). (3.25)

    By Eqs (3.19) and (3.20) and Lemma 3.2, we have the weak convergence as n,

    Un(t,τ)(u(n),in,ϕ(n),in)=Un(t,tk)Un(tk,τ)(u(n),in,ϕ(n),in) (3.26)
    U(t,tk)(u(k),ϕ(k))  inE2H(Σ)×L2V(Σ). (3.27)

    As a result of Eqs (3.15), (3.17), (3.27) and the fact the limit is unique, we obtain

    (u,ϕ)=U(t,tk)(u(k),ϕ(k)),kZ+. (3.28)

    Furthermore, considering Eqs (3.27), (3.28) and the lower semi-continuity of the norm, one can conclude

    u2H(Σ)lim infmu(n)(t;τ,u(n),in,ϕ(n),in)2H(Σ). (3.29)

    Due to H(Σ) being a Hilbert space, Equation (3.25) can be inferred from Eqs (3.27), (3.29) and the following remainder

    u2H(Σ)lim supnu(n)(t;τ,u(n),in,ϕ(n),in)2H(Σ). (3.30)

    Next we concentrate our attention on proving Eq (3.30). Defining a bilinear operator

    [[,]]V(Σ):V(Σ)×V(Σ)R

    by

    [[u,v]]V(Σ)=2μ1Au,vγ2(u,v),u,vV(Σ). (3.31)

    Setting

    [[u]]2V(Σ)=[[u,u]]V(Σ).

    Thanks to Lemma 2.1, we obtain

    (2c1μ1γ2)u2V(Σ)[[u]]2V(Σ)2μ1Au,u2μ1u2V(Σ). (3.32)

    From Eq (3.32) and the condition γ<4c1μ1 in (H4), it is evident that [[]]V(Σ) defines a norm in the space V(Σ), establishing equivalence with the conventional norm V(Σ). From now on, we set

    =H(Σ).

    Multiplying Eq (2.4) by u(n)(t) yields

    ddtu(n)(t)2+γu(n)(t)2=2Γ(f(t),g(t,u(n)t),u(n)(t)), (3.33)

    where

    Γ(f(t),g(t,u(n)t),u(n)(t))=(f(t),u(n)(t))+(g(t,u(n)t),u(n)(t))N(u(n)(t)),u(n)(t)[[u(n)(t)]]2V(Σ).

    By employing the formula of constant variation, we can derive the energy equation as shown below:

    u(n)(t)2=eγ(tτ)u(n),in2+2tτeγ(tτ)Γ(f(s),g(s,u(n)s),u(n)(s))ds. (3.34)

    Thus, for each kZ+, Equations (3.26) and (3.34) mean

    u(n)(t;τ,u(n),in,ϕ(n),in)2=u(n)(t;tk,Un(tk,τ)(u(n),in,ϕ(n),in)2=eγku(n)(tk;τ,u(n),in,ϕ(n),in)2+2[G1(t,n)+G2(t,n)G3(t,n)G4(t,n)], (3.35)

    where

    G1(t,n)=ttkeγ(ts)(f(s),u(n)(s;tk,Un(tk,τ)(u(n),in,ϕ(n),in)))ds,G2(t,n)=ttkeγ(ts)(g(s,u(n)s(;tk,Un(tk,τ)(u(n),in,ϕ(n),in))),u(n)(s;tk,Un(tk,τ)(u(n),in,ϕ(n),in)))ds,G3(t,n)=ttkeγ(ts)N(u(n)(s;tk,Un(tk,τ)(u(n),in,ϕ(n),in))),u(n)(s;tk,Un(tk,τ)(u(n),in,ϕ(n),in))ds,G4(t,n)=ttkeγ(ts)[[u(n)(s;tk,Un(tk,τ)(u(n),in,ϕ(n),in))]]2V(Σ)ds.

    Limiting estimate of the first term in Eq (3.35): By Eq (2.9), it holds for any ττ(tk,ˆBH(Σ)),

    Un(tk,τ)(u(n),in,ϕ(n),in)BH(Σ)(tk),

    which suggests that the following inequality holds:

    eγku(n)(tk;τ,u(n),in,ϕ(n),in)2eγkR21(tk). (3.36)

    Limiting estimate of the term G1: We conclude from Eq (3.27) and Lemma 3.2 that the following weak convergence in L2(tk,t;V(Σ)) holds as n\rightarrow \infty ,

    \begin{eqnarray} u^{(n)}(\cdot;t-k, U_n(t-k, \tau)(u^{(n), \rm in}, \phi^{(n), \rm in})) \rightharpoonup u(\cdot;t-k, u^{(k)}, \phi^{(k)}). \end{eqnarray} (3.37)

    Since {\mathrm e}^{-\gamma(t-s)}f(s)\in L^2(t-k, t;H(\Sigma)) , Equation (3.37) indicates that

    \begin{align} \lim\limits_{n\rightarrow \infty} G_1(t, n) = \int_{t-k}^t {\mathrm e}^{-\gamma(t-s)} \big (f(s), u(s;t-k, u^{(k)}, \phi^{(k)})\big) \mathrm{d}s. \end{align} (3.38)

    Limiting estimate of the term G_4 : It is evident that \big\{\int_{t-k}^t {\mathrm e}^{-\gamma(t-s)}[\![u(s)]\!]^2_{V(\Sigma)}\mathrm{d}s\big\}^{1/2} operates as a norm in the space L^2(t-k, t;V(\Sigma)) , establishing its equivalence to the usual norm \|\cdot\|_{L^2(t-k, t;V(\Sigma))} . Then Eq (3.37) indicates that

    \begin{align} \int_{t-k}^t {\mathrm e}^{-\gamma(t-s)} [\![u(s, t-k, u^{(k)}, \phi^{(k)})]\!]^2_{V(\Sigma)}\mathrm{d}s \leqslant \liminf\limits_{m\rightarrow \infty} G_4(t, n). \end{align} (3.39)

    Limiting estimate of the term G_2 : To establish the limit of G_2 , we employ the approach of inserting the term g\big(s, u_s(\cdot; t-k, u^{(k)}, \phi^{(k)}) . As an analogy of Eq (3.18), we obtain for each s\in [t-k, t] as n\rightarrow \infty ,

    \begin{align} u^{(n)}_s(\cdot;t-k, U(t-k, \tau)(u^{(n), \rm in}, \phi^{(n), \rm in})) \longrightarrow u_s(\cdot;t-k, u^{(k)}, \phi^{(k)}) \end{align} (3.40)

    strongly in L^2_{H(\Sigma)} . It follows from the assumption (H2) and Eq (3.40) that as n\rightarrow \infty ,

    \begin{align*} &\quad \|g\big(s, u^{(n)}_s(\cdot;t-k, U(t-k, \tau)(u^{(n), \rm in}, \phi^{(n), \rm in}))\big) -g\big(s, u_s(\cdot;t-k, u^{(k)}, \phi^{(k)})\big)\| \nonumber\\ &\leqslant L_g\|u^{(n)}_s(\cdot;t-k, U(t-k, \tau)(u^{(n), \rm in}, \phi^{(n), \rm in})) -u_s(\cdot;t-k, u^{(k)}, \phi^{(k)})\|_{L^2_{H(\Sigma)}}\nonumber\\ &\quad \longrightarrow 0, \, \forall\, s\in [t-k, t], \end{align*}

    which yields that for any s\in [t-k, t] , the following strong convergence in H(\Sigma) holds as n\rightarrow \infty ,

    \begin{align*} g(s, u^{(n)}_s(\cdot;t-k, U(t-k, \tau)(u^{(n), \rm in}, \phi^{(n), \rm in}))) \longrightarrow g(s, u_s(\cdot;t-k, u^{(k)}, \phi^{(k)})). \end{align*}

    By considering the boundedness of

    \int_{t-k}^t\|g(s, u^{(n)}_s(\cdot;t-k, U(t-k, \tau)(u^{(n), \rm in}, \phi^{(n), \rm in})))\|^2\mathrm{d}s

    and applying the Lebesgue dominated convergence theorem, the following strong convergence in L^2(t-k, t;H(\Sigma)) can be shown that as n\rightarrow \infty ,

    \begin{align} g(s, u^{(n)}_s(\cdot;t-k, U(t-k, \tau)(u^{(n), \rm in}, \phi^{(n), \rm in}))) \longrightarrow g(s, u_s(\cdot;t-k, u^{(k)}, \phi^{(k)})). \end{align} (3.41)

    Then it can be inferred from Eqs (3.37) and (3.41) that

    \begin{align} \lim\limits_{n\rightarrow \infty} G_2(t, n) = \int_{t-k}^t {\mathrm e}^{-\gamma(t-s)} \big[g\big(s, u_s(\cdot;t-k, u^{(k)}, \phi^{(k)})), u(s;t-k, u^{(k)}, \phi^{(k)})\big)\big]\mathrm{d}s. \end{align} (3.42)

    Limiting estimate of the term G_3 : The limit of G_3 can be shown by using the technique of inserting the term N(u(s; t-k, u^{(k)}, \phi^{(k)}) . The estimates are essentially the same as the estimates [16, Equation (3.50)] and we omit it, thus we obtain

    \begin{align} \lim\limits_{m\rightarrow \infty} G_3(t, n) = \int_{t-k}^t{ \mathrm e}^{-\gamma(t-s)} \big\langle N(u(s;t-k, u^{(k)}, \phi^{(k)})), u(s;t-k, u^{(k)}, \phi^{(k)})\big\rangle \mathrm{d}s. \end{align} (3.43)

    According to Eqs (3.33)–(3.36), (3.38) and (3.39) and (3.42)–(3.43), we conclude that

    \begin{align} \limsup\limits_{n\rightarrow \infty} \|u^{(n)}(t;\tau, u^{(n), \rm in}, \phi^{(n), \rm in})\|^2 \leqslant {\mathrm e}^{-\gamma k}\mathcal{R}_1^2(t-k) + 2\int_{t-k}^t {\mathrm e}^{-\gamma(t-s)} \Gamma\big(f(s), g(s, u_s^{(k)}), u^{(k)}(s)\big)\mathrm{d}s. \end{align} (3.44)

    Moreover, applying the energy equation to Eq (2.1), we obtain

    \begin{align} \|u(t)\|^2 = {\mathrm e}^{-\gamma k}\|u^{(k)}\|^2 + 2\int_{t-k}^t {\mathrm e}^{-\gamma(t-s)} \Gamma\big(f(s), g(s, u_s^{(k)}), u^{(k)}(s)\big)\mathrm{d}s. \end{align} (3.45)

    Then Eqs (3.44) and (3.45) imply that

    \begin{eqnarray} \limsup\limits_{n\rightarrow \infty} \|u^{(n)}(t;\tau, u^{(n), \rm in}, \phi^{(n), \rm in})\|^2 \leqslant {\mathrm e}^{-\gamma k}\mathcal{R}_1^2(t-k) + \|u\|^2. \end{eqnarray} (3.46)

    From Eq (2.9) and the definition of \mathcal{R}_1^2(t) in Theorem 2.2, we know

    \lim\limits_{k\rightarrow \infty}{\mathrm e}^{-\gamma k}\mathcal{R}_1^2(t-k) = 0.

    Hence, the statement of Eq (3.30) follows. We prove the assertion of this lemma.

    This subsection is dedicated to establishing the proof of Theorem 3.1.

    The proof of Theorem 3.1: The approach taken to prove Theorem 1.1 involves a contradiction argument. Assume that the assertion is false, then for some t_0\in\mathbb{R}, \epsilon_0 > 0 , we can find a sequence (u^{(n)}, \phi^{(n)})\in \mathcal{A}_n(t_0), which satisfies

    \begin{eqnarray} \mathrm{dist}_{E_{H(\Sigma)}^2}\big((u^{(n)}, \phi^{(n)}), \mathcal{A}(t_0)\big) \geqslant \epsilon_0. \end{eqnarray} (3.47)

    However, according to Lemma 3.4, there is a subsequence (using the same index) of \{(u^{(n)}, \phi^{(n)})\}_{n\geqslant 1} that can be found such that

    \begin{eqnarray*} \lim\limits_{n\rightarrow \infty}\mathrm{dist}_{E^2_{H(\Sigma)}} \big((u^{(n)}, \phi^{(n)}), \mathcal{A}(t_0)\big) = 0, \end{eqnarray*}

    which obviously contradicts with Eq (3.47). The proof is complete.

    In this work, we consider the families \mathcal{{A}}_n = \{\mathcal{A}_n(t)| t\in\mathbb{R}\} and \mathcal{{A}} = \{\mathcal{A}(t)| t\in \mathbb{R}\} as the pullback attractors of the bipolar fluid with delay in the domains \Sigma_n and \Sigma , respectively. We demonstrate that the following equality holds for any t\in \mathbb{R} ,

    \begin{eqnarray} \lim\limits_{n\rightarrow \infty} \mathrm{Dist}_{E_{H(\Sigma)}^2} \big(\mathcal{A}_n(t), \mathcal{A}(t)\big) = 0. \end{eqnarray} (4.1)

    According to the definition of the Hausdorff semi-distance, Equation (4.1) shows that the pullback attractors \mathcal{{A}}_n = \{\mathcal{A}_n(t)| t\in\mathbb{R}\} in the sub-domains \Sigma_n converge to the pullback attractor \mathcal{{A}} = \{\mathcal{A}(t)| t\in \mathbb{R}\} in the entire domain \Sigma as the sub-domains \Sigma_n approach the entire domain \Sigma , demonstrating the semi-continuity of the pullback attractors in the phase space E_{H(\Sigma)}^2 .

    The authors declare they have not used Artificial Intelligence (AI) tools in the creation of this article.

    The first author is supported by the National Natural Science Foundation of China (Grant No. 12001073), the China Postdoctoral Science Foundation (Grant No. 2022M722105), the Natural Science Foundation of Chongqing (Grant Nos. cstc2020jcyj-msxmX0709 and cstc2020jcyj-jqX0022), the Science and Technology Research Program of Chongqing Municipal Educaton Commission (Grant Nos. KJQN202200563 and KJZD-K202100503). The third author is supported by the National Natural Science Foundation of China (Grant Nos. 11971356 and 11271290), the Natural Science Foundation of Zhejiang Province (Grant No. LY17A010011).

    The authors declare there is no conflict of interest.



    [1] Pudjianto D, Ramsay C, Strbac G (2007) Virtual power plant and system integration of distributed energy resources. IET Renew Power Gen 1: 10‒16. https://doi.org/10.1049/iet-rpg:20060023 doi: 10.1049/iet-rpg:20060023
    [2] Hanna R, Ghonima M, Kleissl J, Tynan G, Victor DG (2017) Evaluating business models for microgrids: Interactions of technology and policy. Energy Policy 103: 47‒61. https://doi.org/10.1016/j.enpol.2017.01.010 doi: 10.1016/j.enpol.2017.01.010
    [3] Rahman S (2008) Framework for a resilient and environment-friendly microgrid with demand-side participation. Proceedings IEEE Power Eng Soc Gen Meeting—Convers Del Electr Energy 21st Century. https://doi.org/10.1109/PES.2008.4596108
    [4] Hirsch A, Parag Y, Guerrero J (2018) Microgrids: A review of technologies, key drivers, and outstanding issues. Renewable and Sustainable Energy Reviews 90: 402‒411. https://doi.org/10.1016/j.rser.2018.03.040 doi: 10.1016/j.rser.2018.03.040
    [5] Mengelkamp E, Gä rttner J, Rock K, Kessler S, Orsini L, Weinhardt C (2018) Designing microgrid energy markets: A case study: The Brooklyn Microgrid. Applied Energy 210: 870‒880. https://doi.org/10.1016/j.apenergy.2017.06.054 doi: 10.1016/j.apenergy.2017.06.054
    [6] Shahgholian G (2021) A brief review on microgrids: Operation, applications, modeling, and control. Int T Electr Energy Syst 31: e12885. https://doi.org/10.1002/2050-7038.12885 doi: 10.1002/2050-7038.12885
    [7] Baringo A, L. Baringo L (2016) A stochastic adaptive robust optimization approach for the offering strategy of a virtual power plant. IEEE T Power Syst 32: 3492–3504. https://doi.org/10.1109/TPWRS.2016.2633546 doi: 10.1109/TPWRS.2016.2633546
    [8] Mishra DK, Ray PK, Li L, Zhang J, Hossain M, Mohanty A (2022) Resilient control based frequency regulation scheme of isolated microgrids considering cyber attack and parameter uncertainties. Applied Energy 306: 118054. https://doi.org/10.1016/j.apenergy.2021.118054 doi: 10.1016/j.apenergy.2021.118054
    [9] Kamal MB, Wei J (2017) Attack- resilient energy management architecture hybrid emergency power system for more-electric aircrafts. 2017 IEEE Power & Energy Society Innovative Smart Grid Technologies Conference (ISGT). https://doi.org/10.1109/ISGT.2017.8085993
    [10] Hussain A, Bui V, Kim H (2018) A Resilient and Privacy-Preserving Energy Management Strategy for Networked Microgrids. IEEE T Smart Grid 9: 2127‒2139. https://doi.org/10.1109/TSG.2016.2607422 doi: 10.1109/TSG.2016.2607422
    [11] Khodaei A (2014) Resiliency-oriented microgrid optimal scheduling. IEEE T Smart Grid 5: 1584–1591. https://doi.org/10.1109/TSG.2014.2311465 doi: 10.1109/TSG.2014.2311465
    [12] Li Y, Li T, Zhang H, Xie X, Sun Q (2022) Distributed resilient Double –Gradient-Descent Based Energy Management Strategy for Multi-Energy System under DoS attacks. IEEE T Netw Sci Eng 9: 2301‒2316. https://doi.org/10.1109/TNSE.2022.3162669 doi: 10.1109/TNSE.2022.3162669
    [13] Liu X (2017) Modelling, analysis, and optimization of interdependent critical infrastructures resilience. Ph. D. thesis, CentraleSupelec.
    [14] Mumbere SK, Matsumoto S, Fukuhara A, Bedawy A, Sasaki Y, Zoka Y, et al. (2021) An Energy Management System for Disaster Resilience in Islanded Microgrid Networks. 2021 IEEE PES/IAS Power Africa, 1‒5. https://doi.org/10.1109/PowerAfrica52236.2021.9543282
    [15] Gholami A, Shekari T, Grijalva S (2019) Proactive management of microgrids for resiliency enhancement: An adaptive robust approach. IEEE T Sustain Energ 10: 470–480. https://doi.org/10.1109/TSTE.2017.2740433 doi: 10.1109/TSTE.2017.2740433
    [16] Watkins CJ, Dayan P (1992) Q-learning. Machine Learning 8: 279‒292. https://doi.org/10.1007/BF00992698 doi: 10.1007/BF00992698
    [17] Nandy A, Biswas M (2019) Reinforcement learning with open AI Tensor Flow, keras using python, Springer Science (Apress.com).
    [18] Ganesh AH, Xu B (2022) A review of reinforcement learning based energy management systems for electrified powertrains: Progress, challenge, and potential solution. Renew Sust Energ Rev 154: 111833. https://doi.org/10.1016/j.rser.2021.111833 doi: 10.1016/j.rser.2021.111833
    [19] Zhang T, Sun M, Qiu D, Zhang X, Strbac G, Kang C (2023) A Bayesian Deep Reinforcement Learning-based Resilient Control for Multi-Energy Micro-gird. IEEE T Power Syst 38: 5057‒5072. https://doi.org/10.1109/TPWRS.2023.3233992 doi: 10.1109/TPWRS.2023.3233992
    [20] Wang Y, Qiu D, Strbac G (2022) Multi-agent deep reinforcement learning for resilience-driven routing and scheduling of mobile energy storage systems. Appl Energ 310: 118575. https://doi.org/10.1016/j.apenergy.2022.118575 doi: 10.1016/j.apenergy.2022.118575
    [21] Chen Y, Heleno M, Moreira A, Gel YR (2023) Topological Graph Convolutional Networks Solutions for Power Distribution Grid Planning. In: Kashima, H., Ide, T., Peng, WC. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2023, 123‒134. https://doi.org/10.1007/978-3-031-33374-3_10
    [22] Zhang B, Hu W, Cao D, Li T, Zhang Z, Chen Z, et al. (2021) Soft actor-critic–based multi-objective optimized energy conversion and management strategy for integrated energy systems with renewable energy. Energ Convers Manage 243: 114381. https://doi.org/10.1016/j.enconman.2021.114381 doi: 10.1016/j.enconman.2021.114381
    [23] Xu D, Cui Y, Ye J, Cha SW, Li A, Zheng C (2022) A soft actor-critic-based energy management strategy for electric vehicles with hybrid energy storage systems. J Power Sources 524: 231099. https://doi.org/10.1016/j.jpowsour.2022.231099 doi: 10.1016/j.jpowsour.2022.231099
    [24] Wang S, Diao R, Xu C, Shi D, Wang Z (2021) On Multi-Event Co-Calibration of Dynamic Model Parameters Using Soft Actor-Critic. IEEE T Power Syst 36: 521‒524. https://doi.org/10.1109/TPWRS.2020.3030164 doi: 10.1109/TPWRS.2020.3030164
    [25] Duan J, Guan Y, Li SE, Ren Y, Sun Q, Cheng B (2022) Distributional Soft Actor-Critic: Off-Policy Reinforcement Learning for Addressing Value Estimation Errors. IEEE T Neur Net Lear Syst33: 6584‒6598. https://doi.org/10.1109/TNNLS.2021.3082568 doi: 10.1109/TNNLS.2021.3082568
    [26] Zhang Z, Chen Z, Lee WJ (2022) Soft Actor-Critic Algorithm Featured Residential Demand Response Strategic Bidding for Load Aggregators. IEEE T Ind Appl 58: 4298‒4308. https://doi.org/10.1109/TIA.2022.3172068 doi: 10.1109/TIA.2022.3172068
    [27] Kathirgamanathan A, Mangina E, Finn DP (2021) Development of a Soft Actor-Critic deep reinforcement learning approach for harnessing energy. Energy and AI 5: 100101. https://doi.org/10.1016/j.egyai.2021.100101 doi: 10.1016/j.egyai.2021.100101
    [28] Ergen T, Kozat SS (2020) Unsupervised Anomaly Detection With LSTM Neural Networks. IEEE T Neur Net Lear Syst 31: 3127‒3141. https://doi.org/10.1109/TNNLS.2019.2935975 doi: 10.1109/TNNLS.2019.2935975
    [29] Bataineh AA, Kaur D (2021) Immunocomputing-Based Approach for Optimizing the Topologies of LSTM Networks. IEEE Access 9: 78993‒79004. https://doi.org/10.1109/ACCESS.2021.3084131 doi: 10.1109/ACCESS.2021.3084131
    [30] Zhou Y, Ma Z, Zhang J, Zou S (2022) Data-driven stochastic energy management of multi-energy system using deep reinforcement learning. Energy 261: 125187–125202. https://doi.org/10.1016/j.energy.2022.125187 doi: 10.1016/j.energy.2022.125187
    [31] Kumar J, Goomer R, Singh AK (2018) Long Short-Term Memory Recurrent Neural Network (LSTM-RNN) Based Workload Forecasting Model For Cloud Datacenters. Procedia Computer Science 125: 676‒682. https://doi.org/10.1016/j.procs.2017.12.087 doi: 10.1016/j.procs.2017.12.087
    [32] Abbasimehr H, Paki R (2022) Improving time series forecasting using LSTM and attention models. J Amb Intell Human Comput 13: 673–691. https://doi.org/10.1007/s12652-020-02761-x doi: 10.1007/s12652-020-02761-x
    [33] Liu Q, Long L, Yang Q, Peng H, Wang J, Luo X (2022) LSTM-SNP: A long short-term memory model inspired from spiking neural P systems. Knowledge-Based Systems 235: 107656. https://doi.org/10.1016/j.knosys.2021.107656 doi: 10.1016/j.knosys.2021.107656
    [34] Moghar A, Hamiche M (2020) Stock Market Prediction Using LSTM Recurrent Neural Network. Procedia Computer Science 170: 1168‒1173. https://doi.org/10.1016/j.procs.2020.03.049 doi: 10.1016/j.procs.2020.03.049
    [35] Karijadi I, Chou S (2022) A hybrid RF-LSTM based on CEEMDAN for improving the accuracy of building energy consumption prediction. Energy and Buildings 259: 111908. https://doi.org/10.1016/j.enbuild.2022.111908 doi: 10.1016/j.enbuild.2022.111908
    [36] Zhao L, Mo C, Ma J, Chen Z, Yao C (2022) LSTM-MFCN: A time series classifier based on multi-scale spatial–temporal features. Comput Commun 182: 52‒59. https://doi.org/10.1016/j.comcom.2021.10.036 doi: 10.1016/j.comcom.2021.10.036
    [37] Ariza I, Tardón LJ, Barbancho AM, De-Torres I, Barbancho I (2022) Bi-LSTM neural network for EEG-based error detection in musicians' performance. Biomed Signal Process Control 78: 103885. https://doi.org/10.1016/j.bspc.2022.103885 doi: 10.1016/j.bspc.2022.103885
    [38] Karim F, Majumdar, S, Darabi H, Harford S (2019) Multivariate LSTM-FCNs for time series classification. Neural Networks 116: 237‒245. https://doi.org/10.1016/j.neunet.2019.04.014 doi: 10.1016/j.neunet.2019.04.014
    [39] Saini KK, Sharma P, Mathur HD, Gautam AR, Bansal RC (2024) Techno-economic and Reliability Assessment of an Off-grid Solar-powered Energy System. Appl Energy 371: 123579. https://doi.org/10.1016/j.apenergy.2024.123579 doi: 10.1016/j.apenergy.2024.123579
    [40] Sharma DD (2024) Asynchronous Blockchain-based Federated Learning for Tokenized Smart Power Contract of Heterogeneous Networked Microgrid System. IET Blockchain 4: 302‒314. https://doi.org/10.1049/blc2.12041 doi: 10.1049/blc2.12041
    [41] Sharma DD, Singh SN, Lin J (2024) Blockchain-enabled secure and authentic Nash Equilibrium strategies for heterogeneous networked hub of electric vehicle charging stations. Blockchain: Research and Applications 5: 100223. https://doi.org/10.1016/j.bcra.2024.100223 doi: 10.1016/j.bcra.2024.100223
    [42] Foruzan E, Soh L, Asgarpoor S (2018) Reinforcement Learning Approach for Optimal Distributed Energy Management in a Microgrid. IEEE Trans Power Syst 33: 5749‒5758. https://doi.org/10.1109/TPWRS.2018.2823641 doi: 10.1109/TPWRS.2018.2823641
    [43] Suttan RS, Barto AG (2018) Reinforcement Learning: An introduction, the MIT Press London, England.
  • Reader Comments
  • © 2025 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)
通讯作者: 陈斌, bchen63@163.com
  • 1. 

    沈阳化工大学材料科学与工程学院 沈阳 110142

  1. 本站搜索
  2. 百度学术搜索
  3. 万方数据库搜索
  4. CNKI搜索

Metrics

Article views(237) PDF downloads(55) Cited by(0)

Figures and Tables

Figures(10)  /  Tables(2)

Other Articles By Authors

/

DownLoad:  Full-Size Img  PowerPoint
Return
Return

Catalog