In recent years, significant progress has been made in video-based person re-identification (Re-ID). The key challenge in video person Re-ID lies in effectively constructing discriminative and robust person feature representations. Methods based on local regions utilize spatial and temporal attention to extract representative local features. However, prior approaches often overlook the correlations between local regions. To leverage relationships among different local regions, we have proposed a novel video person Re-ID representation learning approach based on a graph transformer, which facilitates contextual interactions between relevant region features. Specifically, we construct a local relation graph to model intrinsic relationships between nodes representing local regions. This graph employs the architecture of a transformer for feature propagation, iteratively refining region features and considering information from adjacent nodes to obtain partial feature representations. To learn compact and discriminative representations, we have further proposed a global feature learning branch based on a vision transformer to capture the relationships between different frames in a sequence. Additionally, we designed a dual-branch interaction network based on multi-head fusion attention to integrate frame-level features extracted by both local and global branches. Finally, the concatenated global and local features, after interaction, are used for testing. We evaluated the proposed method on three datasets, namely iLIDS-VID, MARS, and DukeMTMC-VideoReID. Experimental results demonstrate competitive performance, validating the effectiveness of our proposed approach.
Citation: Hai Lu, Enbo Luo, Yong Feng, Yifan Wang. Video-based person re-identification with complementary local and global features using a graph transformer[J]. Mathematical Biosciences and Engineering, 2024, 21(7): 6694-6709. doi: 10.3934/mbe.2024293
Related Papers:
[1]
Jucheng Zhang, Minwen Ma, Huajun Li, Zhaoxia Pu, Haipeng Liu, Tianhai Huang, Huan Cheng, Yinglan Gong, Yonghua Chu, Zhikang Wang, Jun Jiang, Ling Xia .
Early diagnosis of coronary microvascular dysfunction by myocardial contrast stress echocardiography. Mathematical Biosciences and Engineering, 2023, 20(5): 7845-7858.
doi: 10.3934/mbe.2023339
[2]
Hao Dai, Qi Fu, Heng Chen, Mei Zhang, Min Sun, Yong Gu, Ningtian Zhou, Tao Yang .
A novel numerical model of combination levels of C-peptide and insulin in coronary artery disease risk prediction. Mathematical Biosciences and Engineering, 2021, 18(3): 2675-2687.
doi: 10.3934/mbe.2021136
[3]
Amparo Gil, Virginia González-Vélez, Javier Segura, Luis Miguel Gutiérrez .
A theoretical study of factors influencing calcium-secretion couplingin a presynaptic active zone model. Mathematical Biosciences and Engineering, 2014, 11(5): 1027-1043.
doi: 10.3934/mbe.2014.11.1027
[4]
H. G. E. Hentschel, Alan Fine, C. S. Pencea .
Biological computing with diffusion and excitable calcium stores. Mathematical Biosciences and Engineering, 2004, 1(1): 147-159.
doi: 10.3934/mbe.2004.1.147
[5]
Heqing Zhan, Zefeng Wang, Jialun Lin, Yuanbo Yu, Ling Xia .
Optogenetic actuation in ChR2-transduced fibroblasts alter excitation-contraction coupling and mechano-electric feedback in coupled cardiomyocytes: a computational modeling study. Mathematical Biosciences and Engineering, 2021, 18(6): 8354-8373.
doi: 10.3934/mbe.2021414
[6]
Li Cai, Qian Zhong, Juan Xu, Yuan Huang, Hao Gao .
A lumped parameter model for evaluating coronary artery blood supply capacity. Mathematical Biosciences and Engineering, 2024, 21(4): 5838-5862.
doi: 10.3934/mbe.2024258
[7]
Tahir Rasheed, Faran Nabeel, Muhammad Bilal, Yuping Zhao, Muhammad Adeel, Hafiz. M. N. Iqbal .
Aqueous monitoring of toxic mercury through a rhodamine-based fluorescent sensor. Mathematical Biosciences and Engineering, 2019, 16(4): 1861-1873.
doi: 10.3934/mbe.2019090
[8]
Shilian Xu .
Saturated lysing efficiency of CD8+ cells induced monostable, bistable and oscillatory HIV kinetics. Mathematical Biosciences and Engineering, 2024, 21(10): 7373-7393.
doi: 10.3934/mbe.2024324
[9]
Suresh Y, Kalaivani T, Senthilkumar J, Mohanraj V .
BF2 VHDR based dynamic routing with hydrodynamics for QoS development in WSN. Mathematical Biosciences and Engineering, 2020, 17(1): 930-947.
doi: 10.3934/mbe.2020050
[10]
Yong Zhu, Zhipeng Jiang, Xiaohui Mo, Bo Zhang, Abdullah Al-Dhelaan, Fahad Al-Dhelaan .
A study on the design methodology of TAC3 for edge computing. Mathematical Biosciences and Engineering, 2020, 17(5): 4406-4421.
doi: 10.3934/mbe.2020243
Abstract
In recent years, significant progress has been made in video-based person re-identification (Re-ID). The key challenge in video person Re-ID lies in effectively constructing discriminative and robust person feature representations. Methods based on local regions utilize spatial and temporal attention to extract representative local features. However, prior approaches often overlook the correlations between local regions. To leverage relationships among different local regions, we have proposed a novel video person Re-ID representation learning approach based on a graph transformer, which facilitates contextual interactions between relevant region features. Specifically, we construct a local relation graph to model intrinsic relationships between nodes representing local regions. This graph employs the architecture of a transformer for feature propagation, iteratively refining region features and considering information from adjacent nodes to obtain partial feature representations. To learn compact and discriminative representations, we have further proposed a global feature learning branch based on a vision transformer to capture the relationships between different frames in a sequence. Additionally, we designed a dual-branch interaction network based on multi-head fusion attention to integrate frame-level features extracted by both local and global branches. Finally, the concatenated global and local features, after interaction, are used for testing. We evaluated the proposed method on three datasets, namely iLIDS-VID, MARS, and DukeMTMC-VideoReID. Experimental results demonstrate competitive performance, validating the effectiveness of our proposed approach.
1.
Introduction
Myocardial ischemia (MI) causes electrical alternans [1,2,3,4,5], which refer to beat-to-beat alternans in action potential duration (APD) or/and Ca2+ transient in the cellular level [6,7,8,9] and T wave alternans (TWA) on the electrocardiogram (ECG) in the tissue level [10]. Cardiac alternans have been shown to lead fatal arrhythmia, where TWA is confirmed to have promising utility in risk prediction of sudden cardiac death (SCD) and could be considered as a strong marker of arrhythmia [11,12,13]. To make full use of TWA, making clear of the underlying mechanism forming it is essential in clinical. As T wave reflects transmural repolarization dispersion, TWA are thought to arise from repolarization differences in tissue heterogeneity [14,15], conduction velocity [16,17] and cellular alternans [3,13,18,19,20]. Cellular alternans and TWA are investigated by a plethora of studies from various perspectives [9,21,22,23,24].
With respect to cellular alternans, controversy arises from the unsolved chicken-or-egg problem: which of APD alternans and Ca2+ alternans occur first [24]? Ca2+ alternans can be induced by applying alternans AP-clamp protocol, which is membrane voltage (Vm)-driven alternans [25,26]. Fluctuation of sarcoplasmic reticulum (SR) Ca2+ ([Ca2+]sr) could trigger Ca2+ alternans, which is Ca2+-driven alternans [9,21]. The fact that alternans arising from either genesis could happen, along with the bidirectional coupling between Ca2+ transient and Vm[9,15,21,27] complicates the chicken-or-egg problem.
As to differentiate the two mechanisms, a method of constant-diastolic-interval-pacing is put forward [28]. When applying constant diastolic interval (DI), Ca2+-driven alternans could not be eliminated until the strength of calcium instability is reduced. Therefore, finding out the factors associate with calcium stability becomes vital in investigating Ca2+ alternans. Furthermore, researchers expand this constant-DI-pacing method to constant-TR-pacing method, which implements constant TR interval in one cable simulation [29]. Both of the two methods contribute to prevent alternans with cycle length (CL) larger than 250 ms. Significantly, calcium handlings could be affected whenever CL is more or less than 250 ms due to voltage-calcium coupling and deserves more investigation.
Cellular alternans could be divided into concordant and discordant [9,21,27]. When a large Ca2+ transient gives rise to a long APD, this alternans is called concordant. Correspondingly, discordant alternans indicates a small Ca2+ transients paralleled by a long APD. Due to the bidirectional coupling between Ca2+ dynamics and Vm, Ca2+-related currents, such as L-type calcium current (ICaL) and Na+-Ca2+ exchange current (INaCa) make a difference in forming the two different alternans [9,21,27]. Although the effects of the two currents on APD alternans have been investigated, how the two different alternans affect TWA forming remains unknown.
As to TWA, we mainly care about its utility and value in clinical. A lot of studies aim to build relationship between TWA and prediction of SCD [30,31]. Particularly, TWA is a strong marker of arrhythmia in patients with ischemic heart disease. Prognostic value of TWA test is also confirmed after MI [4]. The magnitude of TWA is associated with heterogeneity of repolarization, which is the key factor to predict arrhythmia [11,12]. Ischemia is considered to increase the magnitude of TWA [11]. Meanwhile, ischemia is shown to produce beat-to-beat variations in peak-systolic and end-diastolic cytosolic Ca2+ ([Ca2+]i) [6,7], alternans in amplitudes and durations of Vm[17]. Moreover, there could be 2:1 response of monophasic potential according to unipolar extracellular electrogram [17]. Various methods analyzing TWA are put forward to broaden the usage of TWA in clinical. However, there is no detailed analysis of ischemia-induced alternans forming from ionic basis to cellular level and eventually to the tissue level.
The ionic basis for cellular alternans could play an important role in explaining how TWA occur, providing valuable information in investigating risk prediction of SCD. However, the underlying mechanisms forming TWA vary from abnormal ionic currents and other electrical factors [12]. In heart failure, a steep fractional Ca2+ release determines the occurrence of alternans [32]. The balance between calcium uptake and release could be disrupted when heart rate is elevated [11]. TWA induced by myocardial ischemia is considered with calcium cycling. In all, calcium cycling and Ca2+-related currents involved in voltage-calcium coupling exert essential influence on alternans occurrence. Finding out key factors forming cellular alternans is essential, which contributes to explain the characteristics of TWA broaden its use in therapy.
To investigate ischemia-induced TWA mechanism, we simulate cellular alternans and TWA in one dimensional cable in ischemia. MI includes hyperkalemia, acidosis, and anoxia [33,34]. Hyperkalemia is shown to be related with depolarization phase [35] and QRS wave on ECG. Acidosis decreases the amplitudes of depolarizing current INa, and ICaL[34]. Anoxia reduces cytosolic ATP, resulting in opening ATP sensitive potassium current (IKATP) [36]. Acidosis along with anoxia impair sarcoplasmic reticulum Ca2+ pump (SERCA) functioning [37] and change properties of ryanodine receptors (RyRs) [38]. INaCa is reduced or even becomes to be in reverse mode [34]. In addition, diastolic [Ca2+]i and net magnitudes of Ca2+ transients are increased during ischemia [6,39].
On the one hand, we aim to simulate both concordant and discordant Ca2+-driven alternans in ischemia and analyze factors in affecting alternans forming. On the other hand, different ischemic zones and reduced conduction velocity are set to find out factors affect TWA patterns. Clinical data show TWA could be centered on the first half of T waves [40,41], however, no explanations are given in forming this alternans. We aim to build the link between cellular alternans and TWA patterns, broadening TWA utility in clinical.
2.
Methods
2.1. Impaired Ca2+ uptake pump
The Ca2+ uptake rate is decreased during ischemia [37]. A thermodynamic Ca2+ uptake pump model was used to be integrated into human ventricular cell model [42] (Ord model). Then the integrated model was applied to one dimensional cable to get ischemic p-ECG with CL of 700 ms. The uptake rate was multiplied by an appropriate coefficient of , making the amount of retaken Ca2+ minus leaked Ca2+ in integrated model almost as the same with that in original Ord model.
2.2. Two types of Ca2+ transients
Ischemia is confirmed to induce Ca2+ alternans by modulating Ca2+ dynamics. Here Ca2+ alternans patterns were mainly decided by changes in properties of RyR channels. There are three properties of Ca2+ release channel, randomness, refractoriness and recruitment [43]. Either prolonging the refractory period or increasing the frequency of spark-induced sparks (which is also called secondary sparks [43]) contributes to alternans. According to changes in the two properties, we modified the formula of Ca2+ release current, respectively. In type Ⅰ, time constants of all the RyRs (τrel) channels were increased by 80 ms to simulate prolonged refractory period. In type Ⅱ, a new gate of Or was added to the formula, controlling the occurrence of secondary sparks. The value of Or depends on [Ca2+]jsr and ranges from zero to one, representing the probability of secondary sparks occurring. The time constant of this gate Or is regulated by ICaL.
(1)
(2)
2.3. Ischemic settings
2.3.1. Transmural ischemia
We also modified formula of Ca2+ release current in Endo and M cell model, respectively. These modifications were aimed to decrease ischemic level from Endo cells to Epi cells, simulating coronary artery occlusion. Detailed transmural ischemic parameters were shown in Table 1.
During Epi ischemia, we simulated TWA induced by two patterns of Ca2+ alternans, respectively. Except for the difference in formula of Ca2+ release current, we set other ischemic parameters as the same. INa, ICaL and INaCa were reduced to 95%, 95% and 70%, respectively; [ATP]i, [ADP]i, [Pi]i and [PH]i were set 6.8 mM, 15 mM, 5 mM, and 7.1. IKATP and Ca2+ uptake current (Jup) were affected by the same ischemic conditions.
2.3.3. Endo ischemia
To investigate the effect of ischemic zone on TWA, we simulated p-ECG during Endo ischemia, where ischemic parameters are shown in Table Ⅰ and cellular alternans are induced by adding the gate Or.
2.4. p-ECG
One dimensional cable was used to simulate p-ECG in control, Epi ischemia, Endo ischemia and transmural ischemia. 60 Endo cells, 45 M cells and 60 Epi cells are connected to form this fiber. As the part of atrium was missing in our cable, P-wave representing the activation of atrium was lacking in p-ECG. The length of each cell was set as 0.01cm and p-ECG was calculated at the point 3.65 cm far from the first Endo cell [35]. Electrical coupling between neighboring cells was expressed as the following equation, where the value of D was 0.154 mm2 ms−1, representing the tensor of diffusion coefficient along the fiber [35]. In addition, Istim, Iion and Cm are current stimulus, ionic currents and membrane capacity respectively.
(3)
3.
Results
3.1. Two types of Ca2+ alternans
We simulated concordant alternans (Type Ⅰ) and discordant alternans (Type Ⅱ). Type Ⅰ is as shown in Figure 1, diastolic Ca2+ concentration alternates slightly (Figure 1a) and Ca2+ release current (Jrel) alternates in magnitude (Figure 1c). In type Ⅱ, alternans in diastolic Ca2+ concentration is evident (Figure 1b) and a large Ca2+ release occurs in every two beats (Figure 1d).
Figure 1.
[Ca2+]i, Jrel alternans and fractional Ca2+ release curve in type Ⅰ and Ⅱ Ca2+ alternans. Where asterisk (*) represents the amplitude of Ca2+ transient in one beat and hollow circle (o) marks the diastolic [Ca2+]i (1a and 1b).
To evaluate the degree of Ca2+ alternans, Ca2+ alternans ratio is introduced. Ca2+ alternans ratio is defined as the value of 1-B/A, where B and A are net magnitudes of small and large Ca2+ transients respectively [7]. The mean values of alternans ratios in two types were calculated for ten beats in stable state and given in Table 2. The value of alternans ratio in type Ⅱ is larger than in type Ⅰ, indicating that fluctuations of Ca2+ transients are more evident in type Ⅱ.
Table 2.
The difference of time duration completing 60% and 90% repolarization in two types of Ca2+ alternans.
Steep fractional Ca2+ release curve plays a key role in forming Ca2+ alternans. Here we recorded the relationship between the fractional released Ca2+ and sarcoplasmic reticulum Ca2+ concentration ([Ca2+]sr) during 100 beats, where the initial of [Ca2+]sr is 1.2 mmol/l (Figure 1e). According to Figure 1e, Ca2+ release is small with [Ca2+]sr lower than the threshold of 1.7 mmol/l. When[Ca2+]sr comes to the threshold, a small increase will result in large Ca2+ release, which indicates Or gate becomes massively open. Figure 1f shows that bifurcations exist from the beginning of recording.
3.2. Corresponding APD alternans, TWA
Two types of APD alternans are shown in Figure 2a, b. In Figure 2a, although we could not tell evident alternans in APD of type Ⅰ, partially enlarged view shows there are slight difference between APDs. In type Ⅱ, a very small Ca2+ transient causes ICaL to last longer (Figure 2d). Correspondingly, plateau phases alternate evidently (Figure 2b).
Figure 2.
APDs, ICaL and INaCa in type Ⅰ and Ⅱ Ca2+ alternans. Where the first beat is shown in red line and the second is in blue line.
During Ca2+-driven alternans, ICaL and INaCa are key factors transforming Ca2+ alternans to APD alternans. Both currents also affect repolarization durations in different phases. Here, durations of repolarization and repolarizing currents (ICaL and INaCa) are picked up and compared. We recorded the time duration completing 60% and 90% of repolarization (APD60 and APD90) in every beat and calculated the difference of them (DAPD60 and DAPD90) between every two consecutive beats. DAPD60 (DAPD90) is calculated by subtracting APD60 (APD90) of the second beats from that of the first beats.
Table 2 gives mean values of them in two types. Positive values indicate APDs are longer in the first beats than in the second beats. Table 2 shows that DAPD90 is larger than DAPD60 in type Ⅰ alternans and the absolute value of DAPD90 is smaller than DAPD60 in type Ⅱ. Where, DAPD60 and DAPD90 are mainly influenced by alternans through ICaL and INaCa. A small Ca2+ transient results in large ICaL (Figure 2c, d) and small INaCa (Figure 2e, f), having contrasting effects on the repolarization duration. Note that in the second beat of type Ⅱ alternans, although the small INaCa contributes to shorten repolarization duration, the prolongation of plateau phase plays a dominant role in the long APD. That indicates ICaL lasting longer than in the first beat. Whereas in type Ⅰ, the small INaCa plays a dominant role in all the repolarization phases in the first beat, resulting in a shorter APD. Moreover, the durations ICaL lasting for are almost the same during the two consecutive beats. The difference in durations of ICaL plays a dominant role in deciding alternans patterns.
In all, when there is a large Ca2+ transient in type Ⅱ Ca2+ alternans (Figure 1b), the corresponding APD is short (Figure 2b), which is shown as discordant alternans. Whereas type Ⅰ is concordant alternans.
T wave arises from the difference of repolarization durations between transmural cells. Repolarization starts from Epi cells to M and Endo cells. When T wave reaches to its peak (Figure 4a), the difference between transmural cells repolarization durations increases to the maximum. Figure 5a shows Epi ischemia induces larger T wave magnitudes, indicating the heterogeneity of transmural repolarization is increased. To investigate how the repolarization heterogeneity is affected by these two types of alternans, we compare the APDs in control conditions and that in ischemia (Figure 3). Repolarization durations are shortened by IKATP during ischemia. Correspondingly, the time when the magnitude of T wave reaches is earlier in all ischemic conditions than under control.
Figure 3.
APD alternans of Epi cell (a), M cell (b) and Endo cell (c), where cellular alternans are produced by adding the Or gate. The first beat is shown in red line and the second is in blue line. The blue dashed lines represent APDs in control.
Figure 5.
P-ECG and APDs during Epi ischemia. (a). Alternated p-ECG. (b). APDs of Endo cell (the 35th cell of the fiber), M cell (85th) and Epi cell (135th). (c). APDs of Epi (135th). Where Or gate was added into Epi cell models.
To compare the effect of ischemic zone on TWA patterns, we simulated p-ECG during Epi ischemia, and Endo ischemia, where cellular alternans was concordant alternans. In control conditions, T waves show very slight alternans (Figure 4a). We set initial values of transmural cell models, making them come to stable state in single cell simulation. The cable simulation with our initial values takes time to become stable and thus there is small fluctuations of T waves (Figure 4a). Figures 5–8 show that TWA patterns change with different ischemic zones and cellular alternans patterns.
Figure 6.
P-ECG and APDs during Endo ischemia. (a). Alternated p-ECG. (b). APDs of Endo cell (the 35th cell of the fiber), M cell (85th) and Epi cell (135th). (c). Alternated APD of Endo (35th) and M cells (65th). Where Or gate was added into Endo cell model.
Type Ⅰ alternans of Epi induced by ischemia could not result in obvious TWA (Figure 5a). Note that the time when J point appears is different during the two consecutive beats. In the first beat, J point appears early, which means repolarization of the cable starts early. Correspondingly, Figure 5c shows in the first beat, Epi completes depolarization phase earlier than in the second beat.
Negative T wave arises from Endo ischemia (Figure 6a). Endo ischemia reduces repolarization durations of Endo cells due to IKATP and leads Endo to complete repolarization earlier than M and Epi cells (Figure 6b). Under control, positive T wave is mainly determined by the positive vector starting from Endo to M cells. After APDs of Endo cells are reduced, the electrical vector becomes to point to Endo cell, which is opposite from the recording point of p-ECG, forming negative T wave.
TWA in type Ⅱ is mainly centered on the first half of T waves (Figure 7a). Here, TWA could be divided into two phases, separated by the point A marked asterisk. Before point A, the second T wave is higher than the previous. After crossing the point, the second T wave becomes lower than the first. Meanwhile, APDs of Epi cells during the two beats almost intersect at the point A (Figure 7b). Comparing to the first beat, plateau phase in the second beat is longer and the whole repolarization duration is shorter. Longer plateau phase of Epi cells reduce the repolarization difference between Epi cell and Endo, M cells. In addition, the total reduced repolarization duration of Epi cell increases transmural repolarization heterogeneity.
Transmural ischemia reduces APDs of transmural cells and results in APD alternans in Endo, M and Epi cells (Figure 3). Evident TWA arises from cellular alternans among the whole cable (Figure 8). The changes of TWA magnitude and morphology are owing to altered repolarization heterogeneity. Different ischemic conditions (Table 1) shorten APD of transmural cells to different degrees. Here, T wave becomes negative, indicating APDs of Endo cells are shortened the most.
3.3. TWA with slow electrical conduction
Myocardial ischemia affects electrical conduction along myocardial fiber [44]. Here we simulated the effect of reduced conduction velocity on ECG in transmural ischemia. The duration of QRS increased obviously comparing to ECG in control conditions, resulting in ST and T wave postponed (Figure 9). The pattern and degree of TWA remained unchanged with slowing conduction velocity by comparing the two T waves in Figure 9.
Figure 9.
Under transmural ischemia, comparing p-ECG in control and slow conduction velocity. Where the velocity is reduced to 0.128 mm2/ms.
Prolonging RyRs refractory would enhance Ca2+ alternans [45]. Vyacheslav et al [46] find that latency of Ca2+ release is prolonged after large Ca2+ transients and the restitution property of Ca2+ release makes a difference in forming alternans. In addition, secondary sparks are attributed to recruitment, a property of RyRs [43]. Investigators build a Ca2+ release model and prove that either increasing secondary sparks or prolonged Ca2+ release activities can lead to Ca2+ alternans [43]. In our simulations, we change the two properties separately and investigate their independent roles in contributing to cellular alternans. Although changes of either property could lead to Ca2+ alternans independently, the patterns of alternans in Ca2+ release current and Ca2+ transients are different.
Secondary sparks are the basis for forming Ca2+ waves. Propagation of Ca2+ waves is associated with Ca2+ alternans [43,47]. During Ca2+ alternans, the larger Ca2+ transient is confirmed to arise from Ca2+ waves propagation according to experimental measurements [47]. Previous studies prove the key role of [Ca2+]jsr in forming Ca2+ alternans and Ca2+ waves [47,48]. When [Ca2+]jsr is accumulated to a threshold, Ca2+ alternans will appear [48]. In alternans of type Ⅰ, we simulate the effect of Ca2+ waves propagating, which is govern by Or gate. When [Ca2+]jsr reaches a "threshold", the gate becomes completely open (Figure 1e), producing large Ca2+ transients, which are like Ca2+ waves propagating. MI is shown to increase Ca2+ sparks frequency [49], however, the existence of the Or gate has not been verified.
In type Ⅰ, duration of Ca2+ release is short and the magnitude of Ca2+ release current alternates (Figure 1c). The alternating Ca2+ release activities are governed by the gate Or. According to Eq 1, the value of Or changes with [Ca2+]jsr. Or in single cell model takes time to reach to its steady-state value and the time constant depends on ICaL, which triggers Ca2+ release. Here, Or was introduced to control magnitude of Ca2+ release current, mimicking alternating Ca2+ waves propagation.
In alternans of type Ⅱ, τrel is increased. As RyRs are activated by ICaL, the increased τrel leads to RyRs responding to ICaL in every two beats. Jrel alternates obviously and release activities in the second beat can almost be ignored (Figure 1d). The very small Ca2+ release results in small amplitude of Ca2+ transients in the second beats (Figure 1b). Slow falling phase and broadening systolic peak of large Ca2+ transients are discernible, which coincides with previous studies [6,50,51]. Wu et al. [6] attribute the slow decay to decreased Jup in ischemia. Lee et al. [51] favor that broadening peak arises from Ca2+ release. In our simulations, the velocities of Ca2+ uptake in Ca2+ alternans of the two patterns are reduced by the same ischemic conditions. However, [Ca2+]i decays slower in type Ⅱ (Figure 1a, b). Our results indicated that broadening peak is mainly affected by prolonged Ca2+ released activities. In addition, experiments show that ischemia appears to result in contrasting changes of upstroke of [Ca2+]i[6,51,52]. When the duration of ischemia is within 2.5 min, upstroke is found to be rapid [6,51]. After 4 h of ischemia, upstroke includes an initial rapid phase and a following slow phase [52]. Our upstroke (Figure 1b) slows down due to slow and sustained Ca2+ release.
4.2. The relationship between cellular alternans and TWA patterns
Ca2+ alternans of different patterns in ischemia result in different APD alternans and TWA patterns. In cellular alternans of type Ⅰ, although Ca2+-driven APD alternans are not evident, a long APD is paralleled by a large Ca2+ transient, which is referred to as "in-phase" or "concordant" [21]. Conversely, type Ⅱ Ca2+-driven APD alternans are discordant. A large Ca2+ transient arises from a large Ca2+ release, leading to reduced ICaL, increased INaCa, having contrast effects on APD. Where reducing ICaL shortens plateau phase and increasing INaCa prolongs repolarization by extruding [Ca2+]i. Previous study [9,21,27] analyzes that whether Ca2+ alternans and APD alternans are concordant or discordant depends on which one of currents between ICaL and INaCa plays a dominant role. In our simulation, the concordant alternans are due to the main effect of INaCa and the discordant alternans arise from the dominant role of ICaL.
Note that in cable simulation, APD in the first beat is longer than in the second beat (Figure 7b). In single cell simulation, the first APD is shorter (Figure 2b). During Ca2+-driven alternans forming TWA, coupling between neighboring cells also affects TWA morphology.
In control, there is slight difference in p-ECG morphology between two beats (Figure 4a). This is because the electrophysiological states of the cable are not so stable. The initial parameters of transmural cell models can make sure the cell model being in stable state during single cell simulation. Our cable simulation needs more time to come into stable with these parameters.
According to the insets in figure 5a and figure 6a, TWA can be induced by cellular alternans of type Ⅰ. In cellular alternans of type Ⅱ, TWA is obvious and its morphology is closely related with cellular alternans (Figure 7a, b). Although TWA induced by cellular alternans of type Ⅰ is not evident, the magnitudes of TWA are larger than in control. In clinical, microvolt TWA is used to predict the risk of SCD [1,4,11]: When the value of TWA is higher than 46 µv [4], the risk of SCD and cardiac morality will rise sharply. In addition, experiments show that arrhythmia will occur when ischemia-induced TWA magnitude is 100-fold larger than in control [1]. TWA magnitude could not be recognized easily in clinical and simulations, it is meaningful to detect and evaluate the extent of alternans.
T wave magnitude increases during Epi ischemia (Figures 5a, 7a). APD of Epi is shorten by ischemia, resulting in larger repolarization dispersion within transmural cells (Figures 5b, 7b). T wave becomes negative during Endo ischemia and transmural ischemia (Figures 6a and 8a, b). Note that the maximum repolarization dispersion occurs earliest in transmural ischemia than in regional ischemia. The polarization of T wave along with the time points when T wave amplitudes arrive at can be used to identify ischemic zone.
In Epi ischemia, TWA concentrates in the first half of T waves (Figures 5a, 7a). The duration when TWA occur is confirmed to parallel the vulnerable period [41]. Similar findings are reported within a few minutes of occlusion [40]. Meanwhile, the second half of T waves remain uniform during consecutive beats [40]. However, within 2 to 10 days post-MI, alternans arises during the second half of T wave [53]. Our simulations of Endo ischemia and transmural ischemia show alternans exist during the total T wave (Figures 6a and 8a, b). According to these studies, the location of alternans in T wave on ECG can be linked to ischemic time or ischemic zone.
4.3. The effect of slowing conduction velocity on TWA
Conduction velocity is reduced by ischemia [44]. The slow electrical conduction interplays with cellular alternans to lead to spatial discordant alternans [16], which has ability to induce large TWA magnitude. Previous studies show the importance of T wave magnitude in ventricular arrythmia under MI [1,54]. Here we reduced conduction velocity and investigated the effect of slow conduction on TWA magnitude. In our simulations, slowing electrical conduction along the cable postponed ST and T wave (Figure 9), however, T waves were postponed to the same extent during the two consecutive beats. Although our results favor that reduced conduction velocity has no effect on TWA pattern and magnitude, the conclusion still needs further investigation on 3D level.
Previous experiment shows activation delay increases from about 8 ms of control conditions to 35 ms in ischemic zone, which is identical during two consecutive beats [44]. Prolongation of activation time has been proven to have no effect on activation sequence and TWA magnitude [44]. Note that, delay of activation with slow conduction velocity leads to increased repolarization dispersion between ischemic zone and normal zone. The correlation between conduction and TWA magnitude should be fully investigated by simulating various reduced conduction velocity and ischemic zone.
A plethora of studies put forward that cardiac alternans in ischemia mainly arise from Ca2+ alternans [1,6,7,15,18]. Hyperkaleamia and slowed conduction play dominant roles in block, which also induce repolarization alternans [35]. As the effects of various ischemic factors on repolarization alternans are complex, cellular alternans and TWA under ischemia should be thoroughly studied.
Epi cells are proved to have a greater effect on TWA than M cells [22]. In our simulation, TWA in Epi ischemia (Figure 5a) is hardly to be detected, however, Endo ischemia leads to evident TWA (Figure 6a). Epi cells are closer to electrode than Endo cells, making bigger difference in forming TWA. However, various ischemic conditions should also be considered when analyzing the TWA magnitude. Ischemia leads to various Ca2+ alternans ratios, where the smallest Ca2+ alternans ratio is close to the boundary of ischemic zone [7]. Thus, more sever ischemia results in larger Ca2+ alternans ratio and correspondingly larger APD alternans. In addition, electrical coupling should also be considered in tissue simulation, which affects the cellular alternans pattern (Figure 7b). In our cable simulations, ischemic conditions were set more severe in Endo cells than Epi cells (Table 1). Correspondingly, in single cell simulations, ischemic Endo cell shows larger alternans in APD than ischemic Epi cell (Figure 3a, c).
5.
Conclusions
Changes of RyR properties result in different Ca2+-driven alternans and TWA patterns. Cellular alternans along with electrical coupling affect TWA morphology. Our simulations show the location of alternans in T wave can be associated with different ischemic zone. More analysis in terms of the location of TWA would provide useful insights to more detailed electrical changes during and after MI.
Reducing conduction velocity has no effect on TWA magnitude. However, slow conduction interplays with spatially concordant alternans to form SDA, indicating conduction velocity plays a key role in assisting other electrical factors to affect TWA.
6.
Limitations
In our simulation, APD alternans induced by type Ⅰ Ca2+ alternans are not obvious and thus TWA are not easy to be discernible. Changing ischemic settings or our formula of Or gate may result in various APD alternans pattern. Other ischemic factors should be added to analyze TWA pattern thoroughly in the future.
Cellular concordant alternans induced by ischemia are simulated here, however, ECG of tissue with spatially discordant alternans could also provide deeply understanding in terms of the relationship between APD alternans and TWA. We only could get more comprehensive conclusion after taking consideration of as many patterns of cellular alternans and TWA as possible.
Ischemic conditions in our simulation are set according to previous experimental data. However, our modification of Jrel is not based on experiments. We only consider if Ca2+ alternans ratios vary within reasonable range. In the future, more experimental data should be referenced and then furtherly modify the formula of Jrel.
More importantly, we only simulated p-ECG in one dimensional fiber. Investigation of TWA should be more valuable based on 3D-personalized cardiac model.
Acknowledgments
This work was supported by the Natural Science Foundation of China (NSFC) under grant number 61527811 and 61701435, the Key Research and Development Program of Zhejiang Province under grant number 2020C03016, the Zhejiang Provincial Natural Science Foundation of China under grant number LY17H180003, and the Medical Health Science and Technology Project of Zhejiang Provincial Health Commission under grant number 2020RC094.
Conflict of interest
The authors declare that there are no conflicts of interest.
References
[1]
H. Li, K. Xu, J. Li, Z. Yu, Dual-stream reciprocal disentanglement learning for domain adaptation person re-identification, Knowl. Based Syst., 251 (2022), 109315. https://doi.org/10.1016/j.knosys.2022.109315 doi: 10.1016/j.knosys.2022.109315
[2]
S. Yan, Y. Zhang, M. Xie, D. Zhang, Z. Yu, Cross-domain person re-identification with pose-invariant feature decomposition and hypergraph structure alignment, Neurocomputing, 467 (2022), 229–241. https://doi.org/10.1016/j.neucom.2021.09.054 doi: 10.1016/j.neucom.2021.09.054
[3]
H. Li, J. Xu, Z. Yu, J. Luo, Jointly learning commonality and specificity dictionaries for person re-identification, IEEE Trans. Image Process., 29 (2020), 7345–7358. https://doi.org/10.1109/TIP.2020.3001424 doi: 10.1109/TIP.2020.3001424
[4]
Y. Zhang, Y. Wang, H. Li, S. Li, Cross-compatible embedding and semantic consistent feature construction for sketch re-identification, in Proceedings of the 30th ACM International Conference on Multimedia (MM'22), (2022), 3347–3355. https://doi.org/10.1145/3503161.3548224
[5]
H. Li, M. Liu, Z. Hu, F. Nie, Z. Yu, Intermediary-guided bidirectional spatial-temporal aggregation network for video-based visible-infrared person reidentification, IEEE Trans. Circuits Syst. Video Technol., 33 (2023), 4962–4972. https://doi.org/10.1109/TCSVT.2023.3246091 doi: 10.1109/TCSVT.2023.3246091
[6]
A. Subramaniam, A. Nambiar, A. Mittal, Co-segmentation inspired attention networks for video-based person re-identification, in Proceedings of the IEEE/CVF International Conference on Computer Vision, IEEE, (2019), 562–572. https://doi.org/10.1109/ICCV.2019.00065
[7]
D. Wu, M. Ye, G. Lin, X. Gao, J. Shen, Person re-identification by context-aware part attention and multi-head collaborative learning, IEEE Trans. Inf. Forensics Secur., 17 (2021), 115–126. https://doi.org/10.1109/TIFS.2021.3075894 doi: 10.1109/TIFS.2021.3075894
[8]
A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, et al., An image is worth 16x16 words: Transformers for image recognition at scale, in 9th International Conference on Learning Representations (ICLR), (2021). https://doi.org/10.48550/arXiv.2010.11929
[9]
T. Zhang, L. Wei, L. Xie, Z. Zhuang, Y. Zhang, B. Li, et al., Spatiotemporal transformer for video-based person re-identification, preprint, arXiv: 2103.16469.
[10]
X. Liu, P. Zhang, C. Yu, H. Lu, X. Qian, X. Yang, A video is worth three views: Trigeminal transformers for video-based person re-identification, preprint, arXiv: 2104.01745.
[11]
D. Cheng, Y. Gong, X. Chang, W. Shi, A. Hauptmann, N. Zheng, Deep feature learning via structured graph laplacian embedding for person re-identification, Pattern Recognit., 82 (2018), 94–104. https://doi.org/10.1016/j.patcog.2018.05.007 doi: 10.1016/j.patcog.2018.05.007
[12]
A. Barman, S. K. Shah, Shape: A novel graph theoretic algorithm for making consensus-based decisions in person re-identification systems, in Proceedings of the IEEE International Conference on Computer Vision, IEEE, (2017), 1115–1124. https://doi.org/10.1109/ICCV.2017.127
[13]
Y. Shen, H. Li, S. Yi, D. Chen, X. Wang, Person re-identification with deep similarity-guided graph neural network, in Proceedings of the European Conference on Computer Vision (ECCV), Springer, (2018), 486–504. https://doi.org/10.48550/arXiv.1807.099757
[14]
D. Chen, D. Xu, H. Li, N. Sebe, X. Wang, Group consistent similarity learning via deep crf for person re-identification, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE, (2018), 8649–8658. https://doi.org/10.1109/CVPR.2018.00902
[15]
Y. Yan, Q. Zhang, B. Ni, W. Zhang, M. Xu, X. Yang, Learning context graph for person search, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, (2019), 2153–2162. https://doi.org/10.1109/CVPR.2019.00226
[16]
M. Ye, A. J. Ma, L. Zheng, J. Li, P. C. Yuen, Dynamic label graph matching for unsupervised video re-identification, in Proceedings of the IEEE International Conference on Computer Vision, IEEE, (2017), 5122–5160. https://doi.org/10.1109/ICCV.2017.550
[17]
L. Bao, B. Ma, H. Chang, X. Chen, Preserving structural relationships for person re-identification, in 2019 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), IEEE, (2019), 120–125. https://doi.org/10.1109/ICMEW.2019.00028
[18]
Z. Zhang, H. Zhang, S. Liu, Person re-identification using heterogeneous local graph attention networks, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, (2021), 12131–12140. https://doi.org/10.1109/CVPR46437.2021.01196
[19]
T. N. Kipf, M. Welling, Semi-supervised classification with graph convolutional networks, in International Conference on Learning Representations (ICLR), (2016). https://doi.org/10.48550/arXiv.1609.02907
[20]
P. Veličković, G. Cucurull, A. Casanova, A. Romero, P. Lio, Y. Bengio, Graph attention networks, preprint, arXiv: 1710.10903.
[21]
H. Li, S. Yan, Z. Yu, D. Tao, Attribute-identity embedding and self-supervised learning for scalable person re-identification, IEEE Transactions on Circuits and Systems for Video Technology, 30 (2020), 3472–3485. https://doi.org/10.1109/TCSVT.2019.2952550 doi: 10.1109/TCSVT.2019.2952550
[22]
H. Li, N. Dong, Z. Yu, D. Tao, G. Qi, Triple adversarial learning and multi-view imaginative reasoning for unsupervised domain adaptation person re-identification, IEEE Trans. Circuits Syst. Video Technol., 32 (2022), 2814–2830. https://doi.org/10.1109/TCSVT.2021.3099943 doi: 10.1109/TCSVT.2021.3099943
[23]
H. Li, Y. Chen, D. Tao, Z. Yu, G. Qi, Attribute-aligned domain-invariant feature learning for unsupervised domain adaptation person re-identification, IEEE Trans. Forensics Secur., 16 (2021), 1480–1494. https://doi.org/10.1109/TIFS.2020.3036800 doi: 10.1109/TIFS.2020.3036800
[24]
K. Zhou, Y. Yang, A. Cavallaro, T. Xiang, Omni-scale feature learning for person re-identification, in Proceedings of the IEEE/CVF International Conference on Computer Vision, IEEE, (2019), 3701–3711. https://doi.org/10.1109/ICCV.2019.00380
[25]
F. Yu, X. Jiang, Y. Gong, S. Zhao, X. Guo, W. S. Zheng, et al., Devil's in the details: Aligning visual clues for conditional embedding in person re-identification, preprint, arXiv: 2009.05250.
[26]
Z. Zhang, C. Lan, W. Zeng, Z. Chen, Densely semantically aligned person re-identification, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, (2019), 667–676. https://doi.org/10.1109/CVPR.2019.00076
[27]
M. Ye, H. Li, B. Du, J. Shen, L. Shao, S. C. H. Hoi, Collaborative refining for person re-identification with label noise, IEEE Trans. Image Process., 31 (2021), 379–391. https://doi.org/10.1109/TIP.2021.3131937 doi: 10.1109/TIP.2021.3131937
[28]
S. He, H. Luo, P. Wang, F. Wang, H. Li, W. Jiang, Transreid: Transformer-based object re-identification, in Proceedings of the IEEE/CVF International Conference on Computer Vision, IEEE, (2021), 14993–15002. https://doi.org/10.1109/ICCV48922.2021.01474
[29]
T. Mikolov, M. Karafiàt, L. Burget, J. Cernockỳ, S. Khudanpur, Recurrent neural network based language model, in 11th Annual Conference of the International Speech Communication Association, INTERSPEECH 2010, (2010), 1045–1048. https://doi.org/10.21437/Interspeech.2010-343
[30]
S. Hochreiter, J. Schmidhuber, Long short-term memory, Neural Comput., 9 (1997), 1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735 doi: 10.1162/neco.1997.9.8.1735
[31]
N. McLaughlin, J. Martinez del Rincon, P. Miller, Recurrent convolutional network for video-based person re-identification, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE, (2016), 1325–1334. https://doi.org/10.1109/CVPR.2016.148
[32]
D. Chung, K. Tahboub, E. J. Delp, A two stream siamese convolutional neural network for person re-identification, in Proceedings of the IEEE International Conference on Computer Vision, IEEE, (2017), 1992–2000. https://doi.org/10.1109/ICCV.2017.218
[33]
Y. Suh, J. Wang, S. Tang, T. Mei, K. M. Lee, Part-aligned bilinear representations for person re-identification, in Proceedings of the European Conference on Computer Vision (ECCV), Springer, (2018), 418–437. https://doi.org/10.1007/978-3-030-01264-9_25
[34]
X. Gu, H. Chang, B. Ma, H. Zhang, X. Chen, Appearance-preserving 3D convolution for video-based person re-identification, in Proceedings of the European Conference on Computer Vision (ECCV), Springer, (2020), 228–243. https://doi.org/10.1007/978-3-030-58536-5_14
[35]
J. Li, S. Zhang, T. Huang, Multi-scale 3d convolution network for video based person re-identification, in Proceedings of the AAAI Conference on Artificial Intelligence, (2019), 8618–8625. https://doi.org/10.1609/aaai.v33i01.33018618
[36]
P. Zhang, J. Xu, Q. Wu, Y. Huang, X. Ben, Learning spatial-temporal representations over walking tracklet for long-term person re-identification in the wild, IEEE Trans. Multimedia, 23 (2020), 3562–3576. https://doi.org/10.1109/TMM.2020.3028461 doi: 10.1109/TMM.2020.3028461
[37]
Y. Zhao, X. Shen, Z. Jin, H. Lu, X. S. Hua, Attribute-driven feature disentangling and temporal aggregation for video person re-identification, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, (2019), 4908–4917. https://doi.org/10.1109/CVPR.2019.00505
[38]
Z. Chang, Z. Yang, Y. Chen, Q. Zhou, S. Zheng, Seq-masks: Bridging the gap between appearance and gait modeling for video-based person re-identification, in 2021 International Conference on Visual Communications and Image Processing (VCIP), IEEE, (2021), 1–5. https://doi.org/10.1109/VCIP53242.2021.9675368
[39]
T. Chai, Z. Chen, A. Li, J. Chen, X. Mei, Y. Wang, Video person re-identification using attribute-enhanced features, IEEE Trans. Circuits Syst. Video Technol., 32 (2022), 7951–7966. https://doi.org/10.1109/TCSVT.2022.3189027 doi: 10.1109/TCSVT.2022.3189027
[40]
L. Wu, Y. Wang, L. Shao, M. Wang, 3-d personvlad: Learning deep global representations for video-based person reidentification, IEEE Trans. Neural Networks Learn. Syst., 30 (2019), 3347–3359. https://doi.org/10.1109/TNNLS.2019.2891244 doi: 10.1109/TNNLS.2019.2891244
[41]
V. Dwivedi, X. Bresson, A generalization of transformer networks to graphs, preprint, arXiv: 2012.0969.
[42]
T. Wang, S. Gong, X. Zhu, S. Wang, Person re-identification by video ranking, in Proceedings of the European Conference on Computer Vision (ECCV), Springer, (2014), 688–703. https://doi.org/10.1007/978-3-319-10593-2_45
[43]
E. Ristani, F. Solera, R. Zou, R. Cucchiara, C. Tomasi, Performance measures and a data set for multi-target, multi-camera tracking, in Proceedings of the European Conference on Computer Vision (ECCV), Springer, (2016), 17–35. https://doi.org/10.1007/978-3-319-48881-3_2
[44]
L. Zheng, Z. Bie, Y. Sun, J. Wang, C. Su, S. Wang, et al., Mars: A video benchmark for large-scale person re-identification, in Proceedings of the European Conference on Computer Vision (ECCV), Springer, (2016), 868–884. https://doi.org/10.1007/978-3-319-46466-4_52
[45]
Y. Liu, Z. Yuan, W. Zhou, H. Li, Spatial and temporal mutual promotion for video-based person re-identification, in Proceedings of the AAAI Conference on Artificial Intelligence, (2019), 8786–8793. https://doi.org/10.1609/aaai.v33i01.33018786
[46]
R. Zhang, J. Li, H. Sun, Y. Ge, P. Luo, X. Wang, et al., Scan: Self-and-collaborative attention network for video person re-identification, IEEE Trans. Image Process., 28 (2019), 4870–4882. https://doi.org/10.1109/TIP.2019.2911488 doi: 10.1109/TIP.2019.2911488
[47]
G. Chen, Y. Rao, J. Lu, J. Zhou, Temporal coherence or temporal motion: Which is more critical for video-based person re-identification?, in Proceedings of the European Conference on Computer Vision (ECCV), Springer, (2020), 660–676. https://doi.org/10.1007/978-3-030-58598-3_39
[48]
X. Liu, P. Zhang, C. Yu, H. Lu, X. Yang, Watching you: Global-guided reciprocal learning for video-based person re-identification, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, (2021), 13329–13338. https://doi.org/10.1109/CVPR46437.2021.01313
[49]
A. Aich, M. Zheng, S. Karanam, T. Chen, A. K. Roy-Chowdhury, Z. Wu, Spatio-temporal representation factorization for video-based person re-identification, in Proceedings of the IEEE/CVF International Conference on Computer Vision, IEEE, (2021), 152–162. https://doi.org/10.1109/ICCV48922.2021.00022
[50]
D. Chen, H. Li, T. Xiao, S. Yi, X. Wang, Video person re-identification with competitive snippet-similarity aggregation and co-attentive snippet embedding, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE, (2018), 1169–1178. https://doi.org/10.1109/CVPR.2018.00128
[51]
X. Liu, C. Yu, P. Zhang, H. Lu, Deeply coupled convolution–transformer with spatial–temporal complementary learning for video-based person re-identification, IEEE Trans. Neural Networks Learn.Syst., (2023), 1–11. https://doi.org/10.1109/TNNLS.2023.3271353
[52]
Z. Tang, R. Zhang, Z. Peng, J. Chen, L. Lin, Multi-stage spatio-temporal aggregation transformer for video person re-identification, IEEE Trans. Multimedia, 25 (2023), 7917–7929. https://doi.org/10.1109/TMM.2022.3231103 doi: 10.1109/TMM.2022.3231103
[53]
X. Zang, G. Li, W. Gao, Multidirection and multiscale pyramid in transformer for video-based pedestrian retrieval, IEEE Trans. Ind. Inf., 18 (2022), 8776–8785. https://doi.org/10.1109/TII.2022.3151766 doi: 10.1109/TII.2022.3151766
Hai Lu, Enbo Luo, Yong Feng, Yifan Wang. Video-based person re-identification with complementary local and global features using a graph transformer[J]. Mathematical Biosciences and Engineering, 2024, 21(7): 6694-6709. doi: 10.3934/mbe.2024293
Hai Lu, Enbo Luo, Yong Feng, Yifan Wang. Video-based person re-identification with complementary local and global features using a graph transformer[J]. Mathematical Biosciences and Engineering, 2024, 21(7): 6694-6709. doi: 10.3934/mbe.2024293
Figure 1. The overall architecture of our approach comprises a local branch and a global branch. The local branch primarily focuses on modeling intrinsic relationships among local regions to extract fine-grained information. The global branch is designed to capture relationships between different frames in the sequence, facilitating the learning of a global representation for the entire video sequence. DBIN is used to fuse frame-level features extracted from local and global branches
Figure 2. Diagram of the graph transformer layer using Laplacian eigenvectors. denotes positional encoding, which is added to the input node embeddings before features are fed into the first layer
Figure 3. A dual-branch interaction network based on multi-head fusion attention (DBIN), used to integrate frame-level features extracted by the local branch and the global branch, so that the final extracted features contain rich semantic information