Infrared and visible image fusion (IVIF) is devoted to extracting and integrating useful complementary information from muti-modal source images. Current fusion methods usually require a large number of paired images to train the models in supervised or unsupervised way. In this paper, we propose CTFusion, a convolutional neural network (CNN)-Transformer-based IVIF framework that uses self-supervised learning. The whole framework is based on an encoder-decoder network, where encoders are endowed with strong local and global dependency modeling ability via the CNN-Transformer-based feature extraction (CTFE) module design. Thanks to the development of self-supervised learning, the model training does not require ground truth fusion images with simple pretext task. We designed a mask reconstruction task according to the characteristics of IVIF, through which the network can learn the characteristics of both infrared and visible images and extract more generalized features. We evaluated our method and compared it to five competitive traditional and deep learning-based methods on three IVIF benchmark datasets. Extensive experimental results demonstrate that our CTFusion can achieve the best performance compared to the state-of-the-art methods in both subjective and objective evaluations.
Citation: Keying Du, Liuyang Fang, Jie Chen, Dongdong Chen, Hua Lai. CTFusion: CNN-transformer-based self-supervised learning for infrared and visible image fusion[J]. Mathematical Biosciences and Engineering, 2024, 21(7): 6710-6730. doi: 10.3934/mbe.2024294
[1] | Yijun Lou, Li Liu, Daozhou Gao . Modeling co-infection of Ixodes tick-borne pathogens. Mathematical Biosciences and Engineering, 2017, 14(5&6): 1301-1316. doi: 10.3934/mbe.2017067 |
[2] | Marco Tosato, Xue Zhang, Jianhong Wu . A patchy model for tick population dynamics with patch-specific developmental delays. Mathematical Biosciences and Engineering, 2022, 19(5): 5329-5360. doi: 10.3934/mbe.2022250 |
[3] | Guo Lin, Shuxia Pan, Xiang-Ping Yan . Spreading speeds of epidemic models with nonlocal delays. Mathematical Biosciences and Engineering, 2019, 16(6): 7562-7588. doi: 10.3934/mbe.2019380 |
[4] | Ardak Kashkynbayev, Daiana Koptleuova . Global dynamics of tick-borne diseases. Mathematical Biosciences and Engineering, 2020, 17(4): 4064-4079. doi: 10.3934/mbe.2020225 |
[5] | Holly Gaff . Preliminary analysis of an agent-based model for a tick-borne disease. Mathematical Biosciences and Engineering, 2011, 8(2): 463-473. doi: 10.3934/mbe.2011.8.463 |
[6] | Holly Gaff, Robyn Nadolny . Identifying requirements for the invasion of a tick species and tick-borne pathogen through TICKSIM. Mathematical Biosciences and Engineering, 2013, 10(3): 625-635. doi: 10.3934/mbe.2013.10.625 |
[7] | Maeve L. McCarthy, Dorothy I. Wallace . Optimal control of a tick population with a view to control of Rocky Mountain Spotted Fever. Mathematical Biosciences and Engineering, 2023, 20(10): 18916-18938. doi: 10.3934/mbe.2023837 |
[8] | Pengfei Liu, Yantao Luo, Zhidong Teng . Role of media coverage in a SVEIR-I epidemic model with nonlinear incidence and spatial heterogeneous environment. Mathematical Biosciences and Engineering, 2023, 20(9): 15641-15671. doi: 10.3934/mbe.2023698 |
[9] | Chang-Yuan Cheng, Shyan-Shiou Chen, Xingfu Zou . On an age structured population model with density-dependent dispersals between two patches. Mathematical Biosciences and Engineering, 2019, 16(5): 4976-4998. doi: 10.3934/mbe.2019251 |
[10] | Yongli Cai, Yun Kang, Weiming Wang . Global stability of the steady states of an epidemic model incorporating intervention strategies. Mathematical Biosciences and Engineering, 2017, 14(5&6): 1071-1089. doi: 10.3934/mbe.2017056 |
Infrared and visible image fusion (IVIF) is devoted to extracting and integrating useful complementary information from muti-modal source images. Current fusion methods usually require a large number of paired images to train the models in supervised or unsupervised way. In this paper, we propose CTFusion, a convolutional neural network (CNN)-Transformer-based IVIF framework that uses self-supervised learning. The whole framework is based on an encoder-decoder network, where encoders are endowed with strong local and global dependency modeling ability via the CNN-Transformer-based feature extraction (CTFE) module design. Thanks to the development of self-supervised learning, the model training does not require ground truth fusion images with simple pretext task. We designed a mask reconstruction task according to the characteristics of IVIF, through which the network can learn the characteristics of both infrared and visible images and extract more generalized features. We evaluated our method and compared it to five competitive traditional and deep learning-based methods on three IVIF benchmark datasets. Extensive experimental results demonstrate that our CTFusion can achieve the best performance compared to the state-of-the-art methods in both subjective and objective evaluations.
Lyme disease accounts for over 90
Ticks are capable of moving only very short distances independently, so their fast and large scale spatial spread cannot be attributed solely to their own mobility. Rather, large-scale changes in tick distribution arise as a consequence of the movement of ticks by the vertebrate hosts to which they attach while feeding (see, e.g., [4,5,10,15]). Among such hosts are, in the order of the distances they can move, white-footed mice Peromyscus leucopus, white-tailed deer Odocoileus virginianus, and some migratory birds. Mice can be infected by this bacterium and therefore can transmit the pathogen, and can also transport the tick nymphs. In [4], a reaction diffusion system is proposed to model the advance of the natural infection cycle mediated by the white-footed mouse. Although white-tailed deer diffusion is also mentioned in the model, since the deer cannot be infected and accordingly do not transmit the bacterium the focus of [4] is on the transmission dynamics, the role of deer diffusion in the spatial spread of the pathogen is not discussed in detail in [4]. In relation to birds, in addition to the works [3,18], there have been some works that quantitatively model the role of bird migration in the tick's range expansion, see, e.g., [23].
This paper focuses on the role of white-tailed deer in spreading the ticks. Over the past 50 years, white-tailed deer populations have undergone explosive population growth due to reversion of agricultural lands to forest and restrictions on hunting. This expanding deer population is believed to have facilitated blacklegged tick expansion throughout the Northeast and Midwest [2]. To understand this, we first point out an important difference between birds and deer in transporting I. scapularis and B. burgdorferi. On the one hand, birds carrying the infected immature stages of the tick are capable of traveling longer distances than deer. On the other hand, if immature ticks dropping from birds are to establish a new population they must survive one or two moults and then find a mate, which will be unlikely if they are dropped far from existing populations. In contrast, during fall deer will be carrying numerous already-mated female ticks, each of which becomes engorged with blood while on the deer and then falls to the ground ready to lay approximately 2000 eggs that can form the basis of a new tick population at that location. This observation seems to suggest that deer play a more important role in the tick's range expansion in regions inhabited by white-tailed deer.
In this study we use a spatial model to quantitatively investigate the role of white-tailed deer dispersal in the spatial spread of I. scapularis (and hence B. burgdorferi). Our model combines age structure with the dispersal of deer leading to a system with two time delays and spatial nonlocality resulting from the dispersal of the deer when the adult ticks are attached to them enjoying blood meals. We will begin, in the next section, with a detailed derivation of the model.
To assess the rate at which deer can transport blacklegged ticks into new areas, we develop a differential equation model with spatial effects that describes the stage-structured tick population and its transport by deer. Blacklegged ticks typically undergo a 2-year life cycle in which the larvae quest for a host (typically a small mammal or bird), and if successful feed for several days, drop back to the ground, and later moult into a nymph. The nymph then quests, feeds and moults -again typically on a small mammal. The final adult life stage (which is male or female) then quests and feeds (typically on a deer), falls to the ground when fully engorged and then produces approximately
The mouse population (which feeds the immature ticks) and the deer population (which feeds the adult ticks) are assumed to be homogeneous and constant over time in both the tick-infested and tick-free regions. Mouse home ranges are much smaller than those of deer, so the only significant movement of ticks is by deer transporting adult females while they feed. Because of this, and for simplicity, we assume that larvae and nymphs do not disperse. Since the average time a tick spends attached to a deer is around one week, the relevant deer movements are assumed to be those undertaken in the course of each deer's normal home range activity, rather than long-distance directional movements associated with natal dispersal or seasonal migration.
Consider a spatial domain
{∂L(x,t)∂t=br4e−d4τ1Af(x,t−τ1)−d1L(x,t)−r1L(x,t),∂N(x,t)∂t=r1g(L(x,t))−d2N(x,t)−r2N(x,t),∂Aq(x,t)∂t=r2N(x,t)−d3Aq(x,t)−r3Aq(x,t),∂Af(x,t)∂t=r32∫Ωk(x,y)e−d3τ2Aq(y,t−τ2)dy−r4Af(x,t)−d4Af(x,t), | (1) |
where the parameters are defined in Table 1. The table also gives the values of the parameters for the stage-structured components of the model which were all, except for
Parameters | Meaning | Value |
Birth rate of tick | ||
average time that a questing larvae needs to feed and moult | ||
average time that a questing nymph needs to feed and moult | ||
average time that a questing adult needs to successfully attach to a deer | ||
Proportion of fed adults that can lay eggs | 0.03 | |
per-capita death rate of larvae | 0.3 | |
per-capita death rate of nymphs | 0.3 | |
per-capita death rate of questing adults | 0.1 | |
per-capita death rate of fed adults | 0.1 | |
average time between last blood feeding and hatch of laid eggs | 20 days | |
average time tick is attached to a deer |
The structure of system (1) can be visualized with the help of the diagram in Fig. 1.
A very important aspect of model (1) is the term with the integral, which models the transport of adult ticks by deer. Note that
(∂∂t+∂∂a)udeer(x,t,a)=−d3udeer(x,t,a)+D∇2udeer(x,t,a),forx∈Ω,a∈(0,τ2),t>0, | (2) |
where
Adeer(x,t)=∫τ20udeer(x,t,a)da. | (3) |
Differentiating (3) and using (2) gives
∂Adeer(x,t)∂t=udeer(x,t,0)−udeer(x,t,τ2)−d3Adeer(x,t)+D∇2Adeer(x,t). | (4) |
The rate at which ticks drop off the deer after feeding is
uξdeer(x,a)=udeer(x,a+ξ,a)ed3a. | (5) |
Differentiating with respect to
∂uξdeer(x,a)∂a=D∇2uξdeer(x,a),x∈Ω. |
This is the heat equation, and its solution can be expressed in the form
uξdeer(x,a)=∫ΩK(x,y,a)uξdeer(y,0)dy | (6) |
where the Green's function
∂K(x,y,a)∂a=D∇2xK(x,y,a),K(x,y,0)=δ(x−y) | (7) |
and the boundary conditions to which the deer are subjected at
udeer(x,t,τ2)=e−d3τ2∫ΩK(x,y,τ2)udeer(y,t−τ2,0)dy. |
But
udeer(x,t,0)=r3Aq(x,t). |
Therefore
udeer(x,t,τ2)=e−d3τ2∫ΩK(x,y,τ2)r3Aq(y,t−τ2)dy. |
After the insertion of a factor
k(x,y)=K(x,y,τ2). |
The kernel
k(x,y)=K(x,y,τ2)=1l[1+∞∑n=1{cosnπl(x−y)+cosnπl(x+y)}e−D(nπ/l)2τ2]. |
If
k(x,y)=1l∞∑n=1{cosnπl(x−y)−cosnπl(x+y)}e−D(nπ/l)2τ2. |
We are interested mainly in the case when
k(x,y)=Γ(x−y),whereΓ(z):=1√4Dτ2πe−z24Dτ2 | (8) |
We point out that, just recently, starting from a version of the McKendrick-von Foerster equation without spatial effects but with temporal periodicity, Liu et al [14] also derived a periodic model with age structure for a tick population. A threshold dynamics result is obtained for the model in [14].
To prevent the tick population from increasing to unrealistic levels, density dependence is incorporated into model (1) through a simple nonlinear relationship between questing larvae and questing nymphs. The biological basis for this relationship is that there should be an upper limit to the number of larvae that the mouse population is able to feed -an equivalent relationship was assumed in [21]. This leads us to propose the following expression for the function
g(L)={NcapLcapLforL∈[0,Lcap],NcapforL∈[Lcap,∞). | (9) |
The function
g(L)=Ncapk2Lk1+k2L=NcapLk1/k2+L=NcapLh+L, | (10) |
which is smooth and yet captures the main features of the function given by (9). In the remainder of this paper, we always use (10) for
For the majority of this paper we have in mind model (1) for
Associated to (1) are the following biologically and mathematically meaningful initial conditions:
{L(x,0),N(x,0)are continuous forx∈ΩwithL(x,0)≥0,N(x,0)≥0;Aq(x,s)is continuous for(x,s)∈Ω×[−τ2,0]withAq(x,s)≥0;Af(x,s)is continuous for(x,s)∈Ω×[−τ1,0]withAf(x,s)≥0. | (11) |
Using the method of steps, one can easily see that the initial value problem (1)-(11) has a unique solution for
{L(x,t)=L(x,0)e−(d1+r1)t+br4e−d4τ1∫t0Af(x,s−τ1)e(d1+r1)(s−t)ds,N(x,t)=N(x,0)e−(d2+r2)t+r1Ncap∫t0L(x,s)h+L(x,s)e(d2+r2)(s−t)ds,Aq(x,t)=Aq(x,0)e−(d3+r3)t+r2∫t0N(x,s)e(d3+r3)(s−t)ds,Af(x,t)=Af(x,0)e−(d4+r4)t+r32e−d3τ2∫t0∫Ωk(x,y)Aq(y,s−τ2)e(d4+r4)(s−t)dyds. | (12) |
Let
Next, we prove an important property of the kernel
Proposition 1. If
∫Ωk(x,y)dx=1,forally∈Ω. |
If
∫Ωk(x,y)dx<1,forally∈Ω. |
Proof. When
∂∂a∫ΩK(x,y,a)dx=D∫Ω∇2xK(x,y,a)dx=D∫∂Ω∇xK(x,y,a)⋅ndS=0 |
where
∫Ωk(x,y)dx=∫ΩK(x,y,τ2)dx=∫ΩK(x,y,0)dx=∫Ωδ(x−y)dx=1. |
In the case of homogeneous Dirichlet boundary conditions we have
Next, we show that the solution is bounded for
∂N(x,t)∂t≤r1Ncap−(d2+r2)N(x,t). |
This implies that
lim supt→∞N(x,t)≤r1Ncapd2+r2,for allx∈Ω, |
proving boundedness of
Irrespective of the domain
{∂u1(x,t)∂t=br4e−d4τ1u4(x,t−τ1)−(d1+r1)u1(x,t),∂u2(x,t)∂t=r1Ncaphu1(x,t)−(d2+r2)u2(x,t),∂u3(x,t)∂t=r2u2(x,t)−(d3+r3)u3(x,t),∂u4(x,t)∂t=r32∫Ωk(x,y)e−d3τ2u3(y,t−τ2)dy−(d4+r4)u4(x,t). | (13) |
Consider the case
R0=br4e−d4τ1d1+r1⋅(r1/h)Ncapd2+r2⋅r2d3+r3⋅(r3/2)e−d3τ2d4+r4=Ncapb2he−(d3τ2+d4τ1)4∏i=1ridi+ri. | (14) |
From the biological interpretation of
Substituting the ansatz
{λψ1(x)=br4e−d4τ1e−λτ1ψ4(x)−(d1+r1)ψ1(x),λψ2(x)=r1Ncaphψ1(x)−(d2+r2)ψ2(x),λψ3(x)=r2ψ2(x)−(d3+r3)ψ3(x),λψ4(x)=r32e−d3τ2e−λτ2∫Ωk(x,y)ψ3(y)dy−(d4+r4)ψ4(x). | (15) |
We shall show that the dominant eigenvalue
Proposition 2. If
Proof. Let
ψ4(x)=λ+d1+r1br4e−d4τ1e−λτ1ψ1(x). | (16) |
Similarly, from the second and third equations of (15),
ψ1(x)=(λ+d2+r2)hr1Ncapψ2(x), | (17) |
and
ψ2(x)=λ+d3+r3r2ψ3(x). | (18) |
The fourth equation of (15) then yields
e−λτ1e−λτ2∫Ωk(x,y)ψ3(y)dy=2h(λ+d4+r4)(λ+d1+r1)(λ+d2+r2)(λ+d3+r3)bNcape−d4τ1e−d3τ2r1r2r3r4ψ3(x),=2h∏4i=1(λ+di+ri)bNcape−(d3τ2+d4τ1)∏4i=1riψ3(x) |
so that
e−λτ1e−λτ2∫Ωk(x,y)ψ3(y)dy=∏4i=1(λ+di+ri)R0∏4i=1(di+ri)ψ3(x). | (19) |
Integrating with respect to
f1(λ)=f2(λ) | (20) |
where
f1(x)=R0e−(τ1+τ2)x4∏i=1(di+ri), | (21) |
f2(x)=4∏i=1(x+di+ri). | (22) |
Noting that
R0e−(τ1+τ2)(Reλ)4∏i=1(di+ri)=4∏i=1|λ+di+ri|≥4∏i=1|Reλ+di+ri|, |
that is,
Finally, by the aforementioned properties of
Remark. The above argument fails in the case of homogeneous Dirichlet boundary conditions applied to a finite domain
∏4i=1|λ+di+ri|R0∏4i=1(di+ri)|ψ3(x)|=e−(Reλ)(τ1+τ2)|∫Ωk(x,y)ψ3(y)dy|≤e−(Reλ)(τ1+τ2)∫Ωk(x,y)|ψ3(y)|dy. |
Integrating over
∏4i=1|Reλ+di+ri|R0∏4i=1(di+ri)≤∏4i=1|λ+di+ri|R0∏4i=1(di+ri)≤e−(Reλ)(τ1+τ2) |
which implies that
(Fϕ)(x)=∫Ωk(x,y)ϕ(y)dy,x∈Ω | (23) |
where
In the case
L+=h(R0−1),N+=d3+r3r2A+q,A+q=d4+r4r32e−d3τ2A+f,A+f=d1+r1br4e−d4τ1L+. |
When this persistence (positive) steady state exists, it is locally asymptotically stable. The arguments are similar to those just described for studying the linear stability of the extinction steady state. Linearizing system (1) at
{∂v1(x,t)∂t=br4e−d4τ1v4(x,t−τ1)−(d1+r1)u1(x,t),∂v2(x,t)∂t=r1NcaphR20v1(x,t)−(d2+r2)v2(x,t),∂v3(x,t)∂t=r2v2(x,t)−(d3+r3)v3(x,t),∂v4(x,t)∂t=r32∫Ωk(x,y)e−d3τ2v3(y,t−τ2)dy−(d4+r4)v4(x,t). | (24) |
This linear system is the same as (13) except that
In this section, we consider
L(x,t)=φ1(x+ct),N(x,t)=φ2(x+ct),Aq(x,t)=φ3(x+ct),Af(x,t)=φ4(x+ct). |
Here,
{cφ′1(s)=br4e−d4τ1φ4(s−cτ1)−(d1+r1)φ1(s),cφ′2(s)=r1g(φ1(s))−(d2+r2)φ2(s),cφ′3(s)=r2φ2(s)−(d3+r3)φ3(s),cφ′4(s)=r32∫+∞−∞Γ(z)e−d3τ2φ3(s−z−cτ2)dz−(d4+r4)φ4(s), | (25) |
where
We are interested in traveling wave fronts that connect
lims→−∞φi=0,lims→+∞φi(s)=φ+i,i=1,2,3,4. | (26) |
Linearizing system (25) at the trivial equilibrium
{cφ′1(s)=br4e−d4τ1φ4(s−cτ1)−(d1+r1)φ1(s),cφ′2(s)=r1Ncaphφ1(s)−(d2+r2)φ2(s),cφ′3(s)=r2φ2(s)−(d3+r3)φ3(s),cφ′4(s)=r32∫+∞−∞Γ(z)e−d3τ2φ3(s−z−cτ2)dz−(d4+r4)φ4(s). | (27) |
The characteristic equation associated with (27) is
P(λ,c)=0 | (28) |
where
P(λ,c)=|cλ+d1+r100−br4e−d4τ1e−cτ1λ−r1Ncaphcλ+d2+r2000−r2cλ+d3+r3000−r32e−d3τ2e−cτ2λˉk(λ)cλ+d4+r4|, |
where
ˉk(λ)=∫+∞−∞Γ(y)e−λydy=1√4Dτ2π∫+∞−∞e−y24Dτ2e−λydy=eDτ2λ2. |
Evaluating the determinant on the left hand side of (28), we obtain
4∏i=1[cλ+(di+ri)]−r1r2r3r4bNcap2he−(d4τ1+d3τ2)eDτ2λ2−c(τ1+τ2)λ=0; |
that is
4∏i=1[cλ+(di+ri)]−R0[4∏i=1(di+ri)]eDτ2λ2−c(τ1+τ2)λ=0. | (29) |
Generically, the behaviour of solutions of (25)-(26) at
Set
H1(λ,c)=4∏i=1[cλ+(di+ri)],H2(λ,c)=R0[4∏i=1(di+ri)]eDτ2λ2−c(τ1+τ2)λ. |
Then (29) can be rewritten as
c∗=minc>0{c:H1(λ,c)=H2(λ,c)haspositiverealrootswithrespecttoλ} |
and is determined by the tangential conditions:
H1(λ,c)=H2(λ,c),and∂H1∂λ(λ,c)=∂H2∂λ(λ,c),λ>0. | (30) |
There is no explicit formula for
Based on (30), we may also numerically explore the dependence of
From the above we have seen that, for
Although we defer to another paper the proof that
(L+,N+,A+q,A+f)=(1.72×103,1.43×103,1.43×103,60.63). |
Fig. 4 shows that there is no traveling wave solution for
Next, we numerically simulate solutions of the original initial value problem (1) to observe the time evolution of solutions toward a traveling wave front. To estimate the spreading rate, we use the same approach as was mentioned in [16]. The idea is to assume some threshold population density
c=limt→∞dˆx(t)dt. |
With the model parameter values given above, Fig. 6(a)-(b) and Fig. 7(a)-(b) show the evolution of the
In this paper, based on the fact that blacklegged ticks are only capable of moving very short distances by themselves and the general belief that dispersal of ticks over appreciable distances is via transport on the white tailed deer on which the adult ticks feed, we developed a spatial differential equation model for a stage-structured tick population. In addition to well-posedness, we identified a basic reproduction ratio
We also discussed traveling wave front solutions to the model that connect
We also numerically simulated the solutions of the original initial value problem (1). The results not only demonstrate the evolution of solutions toward a traveling wave front, but also suggest that
Theoretically confirming that
In this paper, we have concentrated mainly on the case when the spatial domain is
To solve the wave equations (25) with asymptotic boundary condition (26) numerically, we truncate
{cφ′1(sj)=br4e−d4τ1φ4(sj−cτ1)−(d1+r1)φ1(sj),cφ′2(sj)=r1g(φ1(sj))−(d2+r2)φ2(sj),cφ′3(sj)=r2φ2(sj)−(d3+r3)φ3(sj),cφ′4(sj)=r32e−d3τ2∫+∞−∞k(sj−y−cτ2)φ3(y)dy−(d4+r4)φ4(sj). | (31) |
The asymptotic boundary conditions
φi(−M)=0,φi(M)=φ∗i;φi(s)=0,s<−M;φi(s)=φ∗i,s>M,i=1,…,4. |
It then follows that
f1(φ3,sj):=∫+∞−∞k(y)φ3(sj−y−cτ2)dy=∫+∞−∞k(sj−y−cτ2)φ3(y)dy=(∫−M−∞+∫M−M+∫+∞M)k(sj−y−cτ2)φ3(y)dy=∫M−Mk(sj−y−cτ2)φ3(y)dy+φ3(M)∫+∞Mk(sj−y−cτ2)dy. |
Applying the composite trapezium rule for integrals, we obtain
∫M−Mk(sj−y−cτ2)φ3(y)dy=△2[k(sj−s1−cτ2)φ3(s1)+22n∑l=2k(sj−sl−cτ2)φ3(sl)+k(sj−s2n+1−cτ2)φ3(s2n+1)], | (32) |
and
∫+∞Mk(sj−y−cτ2)dy=∫sj−M−cτ2−∞k(y)dy=12(1−∫−sj+M+cτ2sj−M−cτ2k(y)dy) |
=12{1−△2[k(sj−M−cτ2)+k(−sj+M+cτ2)+22[2n+m2−(j−1)]∑l=2k(sj−M−cτ2+(l−1)△)]}. |
Then
f1(φ3,sj)=△2n∑l=2k(sj−sl−cτ2)φ3(sl)+f2(sj) |
where
f2(sj)=△2[k(sj−s1−cτ2)φ3(s1)+k(sj−s2n+1−cτ2)φ3(s2n+1)]+φ3(M)∫+∞Mk(sj−y−cτ2)dy=△2k(sj−M−cτ2)φ∗3+φ∗3∫+∞Mk(sj−y−cτ2)dy=△2k(sj−M−cτ2)φ∗3+φ∗32{1−△2[k(sj−M−cτ2)+k(−sj+M+cτ2)]−△2[2n+m2−(j−1)]∑l=2k(sj−M−cτ2+(l−1)△)}=φ∗32{1−△2[2n+m2−(j−1)]∑l=2k(sj−M−cτ2+(l−1)△)}. |
Let
{cφ1(sj+1)−φ1(sj−1)2△=br4e−d4τ1φ4(sj−m1)−(d1+r1)φ1(sj),cφ2(sj+1)−φ2(sj−1)2△=r1g(φ1(sj))−(d2+r2)φ2(sj),cφ3(sj+1)−φ3(sj−1)2△=r2φ2(sj)−(d3+r3)φ3(sj),cφ4(sj+1)−φ4(sj−1)2△=r32e−d3τ2f1(φ3,sj)−(d4+r4)φ4(sj), | (33) |
for
{cφ1(sj+1)−cφ1(sj−1)−2△br4e−d4τ1φ4(sj−m1)+2△(d1+r1)φ1(sj)=0,cφ2(sj+1)−cφ2(sj−1)−2△r1g(φ1(sj))+2△(d2+r2)φ2(sj)=0,cφ3(sj+1)−cφ3(sj−1)−2△r2φ2(sj)+2△(d3+r3)φ3(sj)=0,cφ4(sj+1)−cφ4(sj−1)−△r3e−d3τ2f1(φ3,sj)+2△(d4+r4)φ4(sj)=0 | (34) |
for
cφ2(sj+1)−cφ2(sj−1)−2h△r1Ncapφ1(sj)+2△(d2+r2)φ2(sj)+chφ1(sj)φ2(sj+1)−chφ1(sj)φ2(sj−1)+2h△(d2+r2)φ1(sj)φ2(sj)=0. |
Thus, system (34) can be expressed as
[M1100M14M21M22000M32M33000M43M44][φ1(s2)⋮φ1(s2n)φ2(s2)⋮φ2(s2n)φ3(s2)⋮φ3(s2n)φ4(s2)⋮φ4(s2n)]+[C1C2C3C4]=0, | (35) |
where
Mii=[2△(di+ri)c0−c2△(di+ri)c⋱−c2△(di+ri)c0−c2△(di+ri)],i=1,…,4, |
M14=[0⋮0−2△br4e−d4τ1⋱−2△br4e−d4τ10…0], |
M21=[−2h△r1Ncap0⋱0−2h△r1Ncap], |
M32=[−2△r20⋱0−2△r2], |
ˆM43=[k(s2−s2−cτ2)k(s2−s3−cτ2)…k(s2−s2n−cτ2)k(s3−s2−cτ2)k(s3−s3−cτ2)…k(s3−s2n−cτ2)⋮k(s2n−s2−cτ2)k(s2n−s3−cτ2)…k(s2n−s2n−cτ2)], |
C1=[0⋮0cφ∗1],C3=[0⋮0cφ∗3],C4=[−△r3e−d3τ2f2(s2)−△r3e−d3τ2f2(s3)⋮cφ∗4−△r3e−d3τ2f2(s2n)], |
C2=[1hφ1(s2)[cφ2(s3)+2△(d2+r2)φ2(s2)]1hφ1(s3)[cφ2(s4)−cφ2(s2)+2△(d2+r2)φ2(s3)]⋮1hφ1(s2n−1)[cφ2(s2n)−cφ2(s2n−2)+2△(d2+r2)φ2(s2n−1)]cφ∗2+1hφ1(s2n)[cφ∗2−cφ2(s2n−1)+2△(d2+r2)φ2(s2n)]]. |
The algebraic system (35) can then be solved numerically using Matlab.
This work was initiated at the Current Topics Workshop: Spatial-Temporal Dynamics in Disease Ecology and Epidemiology, held at the Mathematical Biosciences Institute (MBI) at Ohio State University during October 10-14,2011. The workshop was organized by R. Liu, J. Tsao, J. Wu and X. Zou, and was funded by the NSF through the MBI. We thank Dr. G. Hickling for bringing this problem to a group discussion which stimulated this research project. We also thank those participants at the workshop who offered helpful suggestions and advice on the model, and particularly Drs. Hickling and Tsao for their valuable input during the discussions at the MBI and for providing some valuable references.
[1] |
Y. Liu, X. Chen, Z. Wang, Z. Wang, R. K. Ward, X. Wang, Deep learning for pixel-level image fusion: Recent advances and future prospects, Inform. Fusion, 42 (2018), 158–173. https://doi.org/10.1016/j.inffus.2017.10.007 doi: 10.1016/j.inffus.2017.10.007
![]() |
[2] |
H. Zhang, H. Xu, X. Tian, J. Jiang, J. Ma, Image fusion meets deep learning: A survey and perspective, Inform. Fusion, 76 (2021), 323–336. https://doi.org/10.1016/j.inffus.2021.06.008 doi: 10.1016/j.inffus.2021.06.008
![]() |
[3] |
J. Ma, Y. Ma, C. Li, Infrared and visible image fusion methods and applications: A survey, Inform. Fusion, 45 (2019), 153–178. https://doi.org/10.1016/j.inffus.2018.02.004 doi: 10.1016/j.inffus.2018.02.004
![]() |
[4] |
C. Yang, J. Zhang, X. Wang, X. Liu, A novel similarity based quality metric for image fusion, Inform. Fusion, 9 (2008), 156–160. https://doi.org/10.1016/j.inffus.2006.09.001 doi: 10.1016/j.inffus.2006.09.001
![]() |
[5] | X. Zhang, P. Ye, G. Xiao, VIFB: a visible and infrared image fusion benchmark, in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), (2020), 468–478. https://doi.org/10.1109/CVPRW50498.2020.00060 |
[6] | L. J. Chipman, T. M. Orr, L. N. Graham, Wavelets and image fusion, in International Conference on Image Processing, (1995), 248–251. https://doi.org/10.1109/ICIP.1995.537627 |
[7] |
A. V. Vanmali, V. M. Gadre, Visible and NIR image fusion using weight-map-guided Laplacian-Gaussian pyramid for improving scene visibility, Sādhanā, 42 (2017), 1063–1082. https://doi.org/10.1007/s12046-017-0673-1 doi: 10.1007/s12046-017-0673-1
![]() |
[8] |
L. Sun, Y. Li, M. Zheng, Z. Zhong, Y. Zhang, MCnet: Multiscale visible image and infrared image fusion network, Signal Process., 208 (2023), 108996. https://doi.org/10.1016/j.sigpro.2023.108996 doi: 10.1016/j.sigpro.2023.108996
![]() |
[9] | K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2016), 770–778. https://doi.org/10.1109/CVPR.2016.90 |
[10] | J. Redmon, S. Divvala, R. Girshick, A. Farhadi, You only look once: Unified, real-time object detection, in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2016), 779–788. https://doi.org/10.1109/CVPR.2016.91 |
[11] | Y. Zhang, Y. Wang, H. Li, S. Li, Cross-compatible embedding and semantic consistent feature construction for sketch re-identification, in Proceedings of the 30th ACM International Conference on Multimedia, (2022), 3347–3355. https://doi.org/10.1145/3503161.3548224 |
[12] |
H. Li, N. Dong, Z. Yu, D. Tao, G. Qi, Triple adversarial learning and multi-view imaginative reasoning for unsupervised domain adaptation person re-identification, IEEE Trans. Circuits Syst. Video Technol., 32 (2022), 2814–2830. https://doi.org/10.1109/TCSVT.2021.3099943 doi: 10.1109/TCSVT.2021.3099943
![]() |
[13] | O. Ronneberger, P. Fischer, T. Brox, U-Net: Convolutional networks for biomedical image segmentation, in Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015, 9351 (2015), 234–241. https://doi.org/10.1007/978-3-319-24574-4_28 |
[14] |
L. Tang, H. Huang, Y. Zhang, G. Qi, Z. Yu, Structure-embedded ghosting artifact suppression network for high dynamic range image reconstruction, Knowl.-Based Syst., 263 (2023), 110278. https://doi.org/10.1016/j.knosys.2023.110278 doi: 10.1016/j.knosys.2023.110278
![]() |
[15] |
H. Li, X. Wu, DenseFuse: A fusion approach to infrared and visible images, IEEE Trans. Image Process., 28 (2019), 2614–2623. https://doi.org/10.1109/TIP.2018.2887342 doi: 10.1109/TIP.2018.2887342
![]() |
[16] |
H. Li, J. Liu, Y. Zhang, Y. Liu, A deep learning framework for infrared and visible image fusion without strict registration, Int. J. Comput. Vision, (2024), 1625–1644. https://doi.org/10.1007/s11263-023-01948-x doi: 10.1007/s11263-023-01948-x
![]() |
[17] | L. Qu, S. Liu, M. Wang, Z. Song, Transmef: A transformer-based multi-exposure image fusion framework using self-supervised multi-task learning, preprint, arXiv: 2112.01030. |
[18] | K. He, X. Chen, S. Xie, Y. Li, P. Dollár, R. Girshick, Masked autoencoders are scalable vision learners, in 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), (2022), 15979–15988. https://doi.org/10.1109/CVPR52688.2022.01553 |
[19] |
S. Li, X. Kang, L. Fang, J. Hu, H. Yin, Pixel-level image fusion: A survey of the state of the art, Inform. Fusion, 33 (2017), 100–112. https://doi.org/10.1016/j.inffus.2016.05.004 doi: 10.1016/j.inffus.2016.05.004
![]() |
[20] |
H. Li, X. Qi, W. Xie, Fast infrared and visible image fusion with structural decomposition, Knowl.-Based Syst., 204 (2020), 106182. https://doi.org/10.1016/j.knosys.2020.106182 doi: 10.1016/j.knosys.2020.106182
![]() |
[21] |
Q. Zhang, Y. Liu, R. S. Blum, J. Han, D. Tao, Sparse representation based multi-sensor image fusion for multi-focus and multi-modality images: A review, Inform. Fusion, 40 (2018), 57–75. https://doi.org/10.1016/j.inffus.2017.05.006 doi: 10.1016/j.inffus.2017.05.006
![]() |
[22] |
M. Xie, J. Wang, Y. Zhang A unified framework for damaged image fusion and completion based on low-rank and sparse decomposition, Signal Process.: Image Commun., 98 (2021), 116400. https://doi.org/10.1016/j.image.2021.116400 doi: 10.1016/j.image.2021.116400
![]() |
[23] |
H. Li, Y. Wang, Z. Yang, R. Wang, X. Li, D. Tao, Discriminative dictionary learning-based multiple component decomposition for detail-preserving noisy image fusion, IEEE Trans. Instrum. Meas., 69 (2020), 1082–1102. https://doi.org/10.1109/TIM.2019.2912239 doi: 10.1109/TIM.2019.2912239
![]() |
[24] |
W. Xiao, Y. Zhang, H. Wang, F. Li, H. Jin, Heterogeneous knowledge distillation for simultaneous infrared-visible image fusion and super-resolution, IEEE Trans. Instrum. Meas., 71 (2022), 1–15. https://doi.org/10.1109/TIM.2022.3149101 doi: 10.1109/TIM.2022.3149101
![]() |
[25] |
Y. Zhang, M. Yang, N. Li, Z. Yu, Analysis-synthesis dictionary pair learning and patch saliency measure for image fusion, Signal Process., 167 (2020), 107327. https://doi.org/10.1016/j.sigpro.2019.107327 doi: 10.1016/j.sigpro.2019.107327
![]() |
[26] |
Y. Niu, S. Xu, L. Wu, W. Hu, Airborne infrared and visible image fusion for target perception based on target region segmentation and discrete wavelet transform, Math. Probl. Eng., 2012 (2012), 1–10. https://doi.org/10.1155/2012/275138 doi: 10.1155/2012/275138
![]() |
[27] |
D. M. Bulanon, T. F. Burks, V. Alchanatis, Image fusion of visible and thermal images for fruit detection, Biosyst. Eng., 103 (2009), 12–22. https://doi.org/10.1016/j.biosystemseng.2009.02.009 doi: 10.1016/j.biosystemseng.2009.02.009
![]() |
[28] |
M. Choi, R. Y. Kim, M. Nam, H. O. Kim, Fusion of multispectral and panchromatic satellite images using the curvelet transform, IEEE Geosci. Remote Sens. Lett., 2 (2005), 136–140. https://doi.org/10.1109/LGRS.2005.845313 doi: 10.1109/LGRS.2005.845313
![]() |
[29] |
Y. Liu, X. Chen, J. Cheng, H. Peng, Z. Wang, Infrared and visible image fusion with convolutional neural networks, Int. J. Wavelets, Multiresolution Inf. Process., 16 (2018), 1850018. https://doi.org/10.1142/S0219691318500182 doi: 10.1142/S0219691318500182
![]() |
[30] |
J. Ma, W. Yu, P. Liang, C. Li, J. Jiang, FusionGAN: A generative adversarial network for infrared and visible image fusion, Inform. Fusion, 48 (2019), 11–26. https://doi.org/10.1016/j.inffus.2018.09.004 doi: 10.1016/j.inffus.2018.09.004
![]() |
[31] |
H. Li, Y. Cen, Y. Liu, X. Chen, Z. Yu, Different input resolutions and arbitrary output resolution: A meta learning-based deep framework for infrared and visible image fusion, IEEE Trans. Image Process., 30 (2021), 4070–4083. https://doi.org/10.1109/TIP.2021.3069339 doi: 10.1109/TIP.2021.3069339
![]() |
[32] |
H. Zhang, J. Ma, SDNet: A versatile squeeze-and-decomposition network for real-time image fusion, Int. J. Comput. Vision, 129 (2021), 2761–2785. https://doi.org/10.1007/s11263-021-01501-8 doi: 10.1007/s11263-021-01501-8
![]() |
[33] |
H. Xu, J. Ma, J. Jiang, X. Guo, H. Ling, U2fusion: A unified unsupervised image fusion network, IEEE Trans. Pattern Anal. Mach. Intell., 44 (2020), 502–518. https://doi.org/10.1109/TPAMI.2020.3012548 doi: 10.1109/TPAMI.2020.3012548
![]() |
[34] |
L. Tang, J. Yuan, J. Ma, Image fusion in the loop of high-level vision tasks: A semantic-aware real-time infrared and visible image fusion network, Inform. Fusion, 82 (2022), 28–42. https://doi.org/10.1016/j.inffus.2021.12.004 doi: 10.1016/j.inffus.2021.12.004
![]() |
[35] | J. Liu, X. Fan, Z. Huang, G. Wu, R. Liu, W. Zhong, et al., Target-aware dual adversarial learning and a multi-scenario multi-modality benchmark to fuse infrared and visible for object detection, in 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), (2022), 5792–5801. https://doi.org/10.1109/CVPR52688.2022.00571 |
[36] | A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, et al., Attention is all you need, in Proceedings of the 31st International Conference on Neural Information Processing Systems, (2017), 6000–6010. |
[37] | H. Chen, Y. Wang, T. Guo, C, Xu, Y. Deng, Z. Liu, et al., Pre-trained image processing transformer, in 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), (2021), 12294–12305. https://doi.org/10.1109/CVPR46437.2021.01212 |
[38] | X. Zhu, W. Su, L. Lu, B. Li, X. Wang, J. Dai, Deformable DETR: Deformable transformers for end-to-end object detection, preprint, arXiv: 2010.04159. |
[39] | S. Zheng, J. Lu, H. Zhao, X. Zhu, Z. Luo, Y. Wang, et al., Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers, in 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), (2021), 6877–6886. https://doi.org/10.1109/CVPR46437.2021.00681 |
[40] | A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, et al., An image is worth 16×16 words: Transformers for image recognition at scale, in International Conference on Learning Representations ICLR 2021, (2021). |
[41] | K. Han, A. Xiao, E. Wu, J. Guo, C. Xu, Y. Wang, Transformer in transformer, in 35th Conference on Neural Information Processing Systems (NeurIPS 2021), (2021), 1–12. |
[42] | C. Chen, R. Panda, Q. Fan, RegionViT: Regional-to-local attention for vision transformers, preprint, arXiv: 2106.02689. |
[43] | V. Vs, J. M. J. Valanarasu, P. Oza, V. M. Patel, Image fusion transformer, in 2022 IEEE International Conference on Image Processing (ICIP), (2022), 3566–3570. https://doi.org/10.1109/ICIP46576.2022.9897280 |
[44] |
Y. Zhang, Y. Tian, Y. Kong, B. Zhong, Y. Fu, Residual dense network for image restoration, IEEE Trans. Pattern Anal. Mach. Intell., 43 (2021), 2480–2495. https://doi.org/10.1109/TPAMI.2020.2968521 doi: 10.1109/TPAMI.2020.2968521
![]() |
[45] |
R. Hou, D. Zhou, R. Nie, D. Liu, L. Xiong, Y. Guo, et al., VIF-Net: An unsupervised framework for infrared and visible image fusion, IEEE Trans. Comput. Imaging, 6 (2020), 640–651. https://doi.org/10.1109/TCI.2020.2965304 doi: 10.1109/TCI.2020.2965304
![]() |
[46] |
J. Liu, Y. Wu, Z. Huang, R. Liu, X. Fan, SMoA: Searching a modality-oriented architecture for infrared and visible image fusion, IEEE Signal Process. Lett., 28 (2021), 1818–1822. https://doi.org/10.1109/LSP.2021.3109818 doi: 10.1109/LSP.2021.3109818
![]() |
[47] |
Y. Zhang, Y. Liu, P. Sun, H. Yan, X. Zhao, L. Zhang, IFCNN: A general image fusion framework based on convolutional neural network, Inform. Fusion, 54 (2020), 99–118. https://doi.org/10.1016/j.inffus.2019.07.011 doi: 10.1016/j.inffus.2019.07.011
![]() |
[48] | H. Zhang, H. Xu, Y. Xiao, X. Guo, J. Ma, Rethinking the image fusion: A fast unified image fusion network based on proportional maintenance of gradient and intensity, in Proceedings of the AAAI Conference on Artificial Intelligence, 34 (2020), 12797–12804. https://doi.org/10.1609/aaai.v34i07.6975 |
[49] |
H. Li, X. Wu, CrossFuse: A novel cross attention mechanism based infrared and visible image fusion approach, Inform. Fusion, 103 (2024), 102147. https://doi.org/10.1016/j.inffus.2023.102147 doi: 10.1016/j.inffus.2023.102147
![]() |
[50] |
H. Li, X. Wu, J. Kittler, RFN-Nest: An end-to-end residual fusion network for infrared and visible images, Inform. Fusion, 73 (2021), 72–86. https://doi.org/10.1016/j.inffus.2021.02.023 doi: 10.1016/j.inffus.2021.02.023
![]() |
[51] | M. Deshmukh, U. Bhosale, Image fusion and image quality assessment of fused images, Int. J. Image Process., 4 (2010), 484–508. |
[52] |
H. Chen, P. K. Varshney, A human perception inspired quality metric for image fusion based on regional information, Inform. Fusion, 8 (2007), 193–207. https://doi.org/10.1016/j.inffus.2005.10.001 doi: 10.1016/j.inffus.2005.10.001
![]() |
[53] |
V. Aslantas, E. Bendes, A new image quality metric for image fusion: The sum of the correlations of differences, AEU-International J. Electron. Commun., 69 (2015), 1890–1896. https://doi.org/10.1016/j.aeue.2015.09.004 doi: 10.1016/j.aeue.2015.09.004
![]() |
[54] |
Z. Wang, A. C. Bovik, A universal image quality index, IEEE Signal Process. Lett., 9 (2002), 81–84. https://doi.org/10.1109/97.995823 doi: 10.1109/97.995823
![]() |
[55] | D. P. Kingma, J. Ba, Adam: A method for stochastic optimization, preprint, arXiv: 1412.6980. |
[56] |
A. Toet, The TNO multiband image data collection, Data Brief, 15 (2017), 249–251. https://doi.org/10.6084/m9.figshare.1008029.v2 doi: 10.6084/m9.figshare.1008029.v2
![]() |
[57] | X. Jia, C. Zhu, M. Li, W. Tang, W. Zhou, LLVIP: A visible-infrared paired dataset for low-light vision, in 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), (2021), 3489–3497. https://doi.org/10.1109/ICCVW54120.2021.00389 |
[58] | K. A. Johnson, J. A. Becker, The Whole Brain, 2024. Available from: http://www.med.harvard.edu/AANLIB/home.html. |
[59] |
H. Li, J. Zhao, J. Li, Z. Yu, G. Liu, Feature dynamic alignment and refinement for infrared–visible image fusion: Translation robust fusion, Inform. Fusion, 95 (2023), 26–41. https://doi.org/10.1016/j.inffus.2023.02.011 doi: 10.1016/j.inffus.2023.02.011
![]() |
1. | Xiulan Lai, Xingfu Zou, Minimal wave speed and spread speed in a system modelling the geographic spread of black-legged tick Ixodes scapularis, 2020, 269, 00220396, 6400, 10.1016/j.jde.2020.05.002 | |
2. | Ruiwen Wu, Propagation dynamics of a nonlocal spatial Lyme disease model in a time–space periodic habitat, 2023, 72, 14681218, 103843, 10.1016/j.nonrwa.2023.103843 | |
3. | Xiao-Qiang Zhao, The linear stability and basic reproduction numbers for autonomous FDEs, 2023, 0, 1937-1632, 0, 10.3934/dcdss.2023082 | |
4. | Yu-Cai Hao, Guo-Bao Zhang, Juan He, Exponential stability of traveling wavefronts for a system modeling the geographic spread of black-legged tick Ixodes scapularis, 2023, 74, 0044-2275, 10.1007/s00033-023-02014-9 | |
5. | Mingdi Huang, Shi-Liang Wu, Xiao-Qiang Zhao, The principal eigenvalue for partially degenerate and periodic reaction-diffusion systems with time delay, 2023, 371, 00220396, 396, 10.1016/j.jde.2023.06.024 | |
6. | Azmy S. Ackleh, Amy Veprauskas, Aijun Zhang, The impact of dispersal and allee effects on tick invasion: a spatially-explicit discrete-time modelling approach, 2023, 1023-6198, 1, 10.1080/10236198.2023.2285895 |
Parameters | Meaning | Value |
Birth rate of tick | ||
average time that a questing larvae needs to feed and moult | ||
average time that a questing nymph needs to feed and moult | ||
average time that a questing adult needs to successfully attach to a deer | ||
Proportion of fed adults that can lay eggs | 0.03 | |
per-capita death rate of larvae | 0.3 | |
per-capita death rate of nymphs | 0.3 | |
per-capita death rate of questing adults | 0.1 | |
per-capita death rate of fed adults | 0.1 | |
average time between last blood feeding and hatch of laid eggs | 20 days | |
average time tick is attached to a deer |
Parameters | Meaning | Value |
Birth rate of tick | ||
average time that a questing larvae needs to feed and moult | ||
average time that a questing nymph needs to feed and moult | ||
average time that a questing adult needs to successfully attach to a deer | ||
Proportion of fed adults that can lay eggs | 0.03 | |
per-capita death rate of larvae | 0.3 | |
per-capita death rate of nymphs | 0.3 | |
per-capita death rate of questing adults | 0.1 | |
per-capita death rate of fed adults | 0.1 | |
average time between last blood feeding and hatch of laid eggs | 20 days | |
average time tick is attached to a deer |