A new inconsistent context fusion algorithm based on BP neural network and modified DST

1.   Introduction

With the development and progress of information technology, such as Internet of things and big data, the context-aware computing has broad research prospects, and many continuously optimized ubiquitous computing technologies have been adopted into various application scenarios, like digital home, smart city and intelligent agriculture. As one of the core concepts of ubiquitous computing ^[1], context awareness has attracted attentions of many researchers ^[2,3,4]. It allows user-centered computing applications to provide appropriate services.

Contexts can be any information, which are used to characterize the situation of an entity. An entity is indicated to be the object that is considered relevant to the context-aware application ^[5]. With significant amounts of context information from different sources, such as different kinds of sensors, context-aware systems can adaptively adjust their services according to the continuously changing circumstances without attracting users' attention in order to improve the quality of experience (QoE) during the process of the human-computer interaction. For example, a digital home system with context-aware applications can control the air conditioning in bedroom by considering user's situation and the variation of air temperature automatically.

However, due to dynamic environments, heterogeneous context sources, limitation of context-aware computing resources and network delay, the raw context information from different sources is usually imperfect and uncertain ^[6,7], and specially the inconsistency is a kind of main uncertainty problems and should be considered very carefully ^[8,9]. The inconsistent information may affect the normal operation of context-aware applications, which may induce improper decisions or inappropriate services. Therefore, there is an urgent need to solve context inconsistency problem using effective means so as to provide appropriate context-aware services and enhance the user satisfaction ^[10,11].

To handle this problem, Dempster-Shafer theory (DST) ^[12], Bayesian network, association rules, rule-based method and other methods have been used for inconsistent context reasoning and fusion. In ^[13], Zhang et al. proposed a general context-aware application model and applied DST into the reasoning mechanism. Data collected from sensors is mapped to prior probability and then basic belief assignment (BBA) of related prior probability is calculated. According to the combination rule of DST, related BBAs are fused. A space-based context model and dynamic Bayesian network were introduced into dynamic data fusion in ^[14]. Space-based context model intuitively describes the relationship between the context attribute and the situation space. But there are two main limits of Bayesian method when it is used for the context fusion, one is the mutual exclusivity required for computing hypotheses, and the other is the inability to account for the general uncertainty. In ^[15], Al-Shargabi et al. proposed the resolving context conflict using association rules (RCCAR) algorithm to predict the valid values among conflict contexts. Lee et al. proposed a rule-based context inconsistency elimination scheme for context management in ubiquitous computing ^[16]. Based on the designed rule language, they determined the conditions that have the main influence on the corresponding activities, and used the condition contexts to process the inconsistent contexts. The method proposed in ^[17] comprehensively combined formal concept analysis, Bayesian theory, ontologies and two environment factors, i.e., time reliability and location reliability, which was beneficial for solving the context inconsistency. The quality of context (QoC) parameters were discussed as the worth of context information for a specific application in ^[18,19,20], which laid the foundation of self-adaptive inconsistent context processing algorithms. To prevent emergencies of patients in home-based healthcare, Lee et al. proposed a static evidential network for context reasoning based on Dezert–Smarandache theory (DSmT) ^[21], which is a new evidence theory popularized by DST. The context reasoning engine assigned evidence belief degree to sensor data based on the DSmT. A discounting factor was introduced into the method to express the degree of sensor data's unreliability and uncertainty. However, it did not support dynamic weighting factors.

To improve system's performance, DST or DSmT has been combined with many different methods. In ^[22], Chen et al. combined group neural network with DST in a two level fusion structure, which enhanced the anti-interference capacity of system. A data fusion method based on support vector machine (SVM), extreme learning machine (ELM) and DST was proposed for customer recognition ^[23]. Zhao et al. proposed a data fusion algorithm based on fuzzy set theory and DST, combining attribute evidence with the reliability of physical nodes in sensor networks to improve decision accuracy and reduce data redundancy ^[24]. From the perspective of QoC, we proposed an index OQoC to measure the comprehensive context quality, which is combined with DST to eliminate context inconsistency ^[25].

However, the above-mentioned methods still cannot meet the accuracy requirements of the systems in some specific scenarios. In view of this limitation, the main contributions of this paper are as follows: Firstly, by modifying the combination rule of DST, adaptive redistribution of conflicting information is realized. Secondly, a new inconsistent context fusion method by combining back propagation (BP) neural network with modified DST is presented in this paper. It can help context-aware system to take full advantage of all contexts from different sources and perform context-aware tasks effectively. Thirdly, the validity of the proposed method is verified in personal identity verification (PIV) application scenarios, optimizing the processing scheme of inconsistency for different scenarios in the field of signal processing.

The rest of the paper is organized as follows: Section 2 describes the overall framework and the specific processing steps of the proposed inconsistent context fusion algorithm. We present the performance of evaluation and experiment results, and give a comprehensive discussion in Section 3. Finally, Section 4 states the conclusion and the future outlook.

2.   System framework

In the typical smart spaces, there are different context information sources providing raw context information for context reasoning and fusion.

We can preprocess the raw context information according to the state-space-based context model, which was defined in ^[26]. For example, the raw information related to user's situation is collected by different context sources. A context state from one source can be denoted as ${S_\alpha } = \left({a_1^{}, a_{\rm{2}}^{}, \cdots, a_K^{}} \right)$, and $S_{\alpha}^{\prime}=\left(a_{1}^{\prime}, a_{2}^{\prime}, \cdots, a_{K}^{\prime}\right)$ can be defined as another context state information from a different source. Then context-aware system infers the situation by BP neural network with different context state ${S_\alpha }$ and $S{'_\alpha }$. Each neural network is responsible for the situation recognition of one context source. And we test the accuracy of each BP neural network before the context situation reasoning and define it as a control factor for combination rule. The neural network outputs are normalized as the mass functions and combined according to the modified DST rule. Finally, the result will be determined by fusion output.

The framework and specific procedure of the proposed inconsistent context fusion algorithm are shown in Figures 1 and 2, respectively, and the processing steps are explained as follows.

Figure 1.  The framework of the proposed inconsistent context fusion algorithm.

DownLoad: Full-Size Img PowerPoint

Figure 2.  The procedure of the proposed inconsistent context fusion algorithm.

DownLoad: Full-Size Img PowerPoint

2.1.   Step one: context modeling

In this step, we gather the raw context information from several context sources and preprocess the contexts according to the state-space-based context model. By modeling the context and dividing the situation space, the raw context information can be classified by BP neural networks and transformed into situation recognition information.

1) Context Attribute: A context attribute, denoted by ${a_i}$, is defined as any type of information that can help infer relevant entity's situation. Context attributes are often associated with context information sources, and context attributes from different sources have different forms and are usually inconsistent. For example, the location attribute from user's smart watch may be in conflict with the context attribute inferred from ID card recognition.

2) Context State: Context state is a collection of K context attribute values, which is the input of BP neural networks and can be used to infer the entity's situation. It is denoted by ${S_\alpha } = \left({{a_1}, {a_2}, \cdots, {a_K}} \right)$, where ${a_i}$ corresponds to the value of the i-th context attribute.

3) Situation Space: ${R_i} = \left({a_1^R, a_2^R, \cdots, a_K^R} \right)$ is a set of regions in situation space and represents the situation of interested entity (e.g., indoor or outdoor), consisting of K acceptable regions for these attributes. An acceptable region $a_i^R$ is defined as a collection of elements V which satisfies a predication P, i.e., $a_i^R = \left\{ {V\left| {P\left(V \right)} \right.} \right\}$. The special context information can be performed or associated with a certain region.

As we can see in Figure 3, ${S_\alpha }$ is a context state inside the situation subspace ${R_i}$, ${a_x}$, ${a_y}$, ${a_z}$are three different attributes, respectively, and the acceptable ranges make up the situation subspace.

Figure 3.  The diagram of the situation space.

DownLoad: Full-Size Img PowerPoint

2.2.   Step two: context reasoning

The BP neural network has been applied to solve diverse problems successfully by training with a supervised manner, which is also one of the most widely used models of the neural network. In this step, we define the context state ${S_\alpha } = \left({{a_1}, {a_2}, \cdots, {a_K}} \right)$ as the input of BP neural network. Each context source is associated with a BP neural network. So we can transform the context state into high level context information. In output layer, we set output function as "logsig" to limit the output in the range of [0, 1] and each output neuron is corresponding to an acceptable region in situation space.

First, we train the BP neural networks and test their accuracy, and then define a control factor ${C_i}$ to represent the credibility of the i-th BP neural network which equals to the correct rate of the i-th BP neural network.

Then the BP neural network is used to recognize the entity's situation.

2.3.   Step three: establish recognition framework

In this step, we use the DST to establish recognition framework ^[27,28]. There are some basic definitions in DST.

Definition 1: The recognition framework $\Theta$ is defined as an exhaustive set of mutually exclusive events.

Let $\Theta = \left\{ {{\theta _1}, {\theta _2}, \cdots, {\theta _N}} \right\}$ be a finite set of possible situations, where ${\theta _i}$ denotes a possible situation which can be recognized by BP neural network.

Definition 2: If for any subset A of recognition framework $\Theta$, we can get $m\left(\phi \right) = 0$ and $\sum\limits_{A \subset \Theta } {m\left(A \right)} = 1$, then $m$ will be called the BBA function and denoted by $m:{2^\Theta } \to \left[{0, 1} \right]$.

If $m\left(A \right) > 0$, then A is a focal element of the recognition framework $\Theta$.

Definition 3: The belief function for subset B in the recognition framework $\Theta$ is defined as:

In this algorithm, recognition framework is the exhaustive set of mutually exclusive situations and we define each situation as a focal element ${\theta _i}$.

For example, for the recognition framework $\Theta = \left\{ {A, B, C} \right\}$, it supposes that ${A_{out}}$ can be the recognition result of output neuron about situation A.

2.4.   Step four: basic belief assignment

Output neuron's recognition results of each BP neural network can be seen as a BBA of recognition framework $\Theta$, which is defined by:

Because the raw context information from different sources could be inconsistent, it is imprecise to fusion the BBA by traditional DST rule. Therefore, we propose a modified DST combination rule to enhance the accuracy of the fusion results.

2.5.   Step five: modified evidence combination

Assume that ${m_1}$ and ${m_2}$ are BBA functions of the recognition framework $\Theta$ for two sources ${S_{\rm{1}}}$ and ${S_2}$. ${\theta _1}, {\theta _2}, \cdots, {\theta _i} \cdots, {\theta _N}$ are the focal elements of ${m_1}$ and $\theta {'_1}, \theta {'_2}, \cdots, \theta {'_j} \cdots, \theta {'_N}$ are the focal elements of ${m_2}$.

The Dempster's basic combination rule ^[27] of ${m_1}$ and ${m_2}$ can be expressed as follows:

However, DST would give imprecise and counterintuitive fusion results when the context information acquired from different sources are highly conflictive.

To combine the conflict information, Murphy et al. proposed an average rule ^[29], which is defined as follows:

where $k$ is the number of BP neural networks.

Introducing control factor ${C_i}$ into the average rule, we can get the weighted average rule, which can be expressed as:

Firstly, we propose a modified weighted average rule that is defined as follows:

where ${m_ \cap }\left(X \right) = \sum\limits_{{\theta _i} \cap \theta {'}_j^{} = X} {{m_1}\left({{\theta _i}} \right){m_2}\left({\theta {'}_j^{}} \right)}$.

Then we combine Eq (6) with the self-adaptive fusion algorithm based on DST and DSmT ^[30] and get the weighted self-adaptive DST synthesis principle, which can be expressed as:

where $P\left(X \right) = \left({1 - \sigma } \right)\frac{{\sum\limits_{{\theta _i} \subset X} {C_1^n{m_1}\left({{\theta _i}} \right)} + \sum\limits_{\theta {'}_j^{} \subset X} {C_2^n{m_1}\left({\theta {'}_j^{}} \right)} }}{{C_1^n + C_2^n}}{K_{12}}$, $\sigma = \left({C_{_1}^n + C_{_2}^n} \right)/n$, and n is a factor to control the effect of ${C_{\rm{1}}}$ and ${C_{\rm{2}}}$.

As $\sigma$ changes, the fusion result would be different and the conflict information will be distributed to different hypotheses. When $\sigma$ is equal to 1, the fusion result will be the same as that of the Dempster's rule; when $\sigma$ goes to 0, it will approach to the result of Eq (6); when $\sigma \in \left({0, 1} \right)$, part of the conflict will be redistributed by control factor and the rest will be distributed averagely and then the new algorithm is the synthesis of the modified weighted average and the DST combination rules. Therefore, the system can get more preferable fusion results.

3.   Performance evaluation

In this section, the performance of the proposed algorithm is evaluated through several experiments and also compared with some other peer methods.

In order to verify the superiority of the proposed algorithm, we choose the typical scenario of multi-source context information, i.e., PIV system as the application example, and of course the proposed algorithm can also be applied to other intelligent scenarios, such as smart medical treatment and smart prison.

As we know, PIV system is of great importance to security and is wildly used in digital home and many social security scenarios. The typical identity recognition technologies usually make use of many different sensors, such as identity card, fingerprint, voice, iris and face image. In the experiments of this paper, five sensors are placed in different locations to detect and confirm user's identity. The contexts collected from which belong to the same type that has only two states to represent the user's identity is correct or not. Assume that the number of context information collected by one sensor is 8000, therefore the number of total context information provided by context sources is 40,000. To get stable results, we repeat the experiment 1000 times and get the average of context aware rates as the final results.

We choose the context aware rate ${C_{CAR}}$ ^[31,32] to evaluate the performance of several inconsistent context processing algorithms including the proposed weighted self-adaptive algorithm, the trust-worthiness based algorithm ^[33], the voting algorithm ^[34], the user feedback based context selection algorithm ^[31], and the DST algorithm ^[13].

${C_{CAR}}$ is defined as follows:

where ${C_{cor}}$ denotes the number of correct information calculated by context processing algorithms and ${C_{tot}}$ is the number of total context information provided by context sources.

Table 1 shows the abbreviations used in the performance evaluation of inconsistent context processing.

Table 1.  The abbreviations used in the performance evaluation of inconsistent context processing.

Symbols	Meaning
A-wsa	The proposed weighted self-adaptive algorithm
A-tw	The algorithm based on trust-worthiness [33]
A-vote	The algorithm based on voting [34]
A-fb	The context-aware selection algorithm based on user feedback [31]
A-DST	The DST algorithm [13]

---

 | Show Table

DownLoad: CSV

The control factor ${C_i}$ is computed by using user feedback data, and the user feedback rate is set to be a suitable value 1%. The control factor ${C_i}$ is defined as follows:

where ${T_{i - fb}}$ denotes the total number of user feedback and ${R_{i - fb}}$ denotes the number of correct context information provided by context source when user feedback occurs.

In this experimental part, we assume that the five sensors used to provide context information have different error rates, which mean the ratio of wrong context information. Therefore, we can choose the error rate to simulate different situations and approximate real application states.

The error rate of every context source can be defined as follows:

where ${n_E}$ denotes the number of error context information, and ${n_T}$ denotes the total number of context information.

Table 2 shows an example of the error rates of five context information sources in the experiments, where E_i denotes the error rate of the i-th context information source. We try to consider different combinations of error rates for five sources to cover different conditions, and of course we can choose more values to do the simulations next step.

Table 2.  $[![]!].

Experiment No.	Error rates of five context information sources (%)
Experiment No.1	${E_1} = \mathit{0} - 0.5$, ${E_\mathit{2}} = \mathit{0}.0\mathit{5}$, ${E_3} = \mathit{0}.0\mathit{5}$, ${E_4} = \mathit{0}.0\mathit{5}$, ${E_5} = \mathit{0}.0\mathit{5}$
Experiment No.2	${E_1} = \mathit{0} - 0.5$, ${E_\mathit{2}} = \mathit{0}.0\mathit{5}$, ${E_3} = \mathit{0}.0\mathit{5}$, ${E_4} = \mathit{0}.0\mathit{5}$, ${E_5} = \mathit{0}.3\mathit{5}$
Experiment No.3	${E_1} = \mathit{0} - 0.5$, ${E_\mathit{2}} = \mathit{0}.0\mathit{5}$, ${E_3} = \mathit{0}.0\mathit{5}$, ${E_4} = \mathit{0}.3\mathit{5}$, ${E_5} = \mathit{0}.3\mathit{5}$
Experiment No.4	${E_1} = \mathit{0} - 0.5$, ${E_\mathit{2}} = \mathit{0}.0\mathit{5}$, ${E_3} = \mathit{0}.3\mathit{5}$, ${E_4} = \mathit{0}.3\mathit{5}$, ${E_5} = \mathit{0}.3\mathit{5}$
Experiment No.5	${E_1} = \mathit{0} - 0.5$, ${E_\mathit{2}} = \mathit{0}.3\mathit{5}$, ${E_3} = \mathit{0}.3\mathit{5}$, ${E_4} = \mathit{0}.3\mathit{5}$, ${E_5} = \mathit{0}.3\mathit{5}$

---

 | Show Table

DownLoad: CSV

Firstly, the A-wsa algorithm is simulated with six sets of context sources, which have different error rates, to identify the best n in the experiments, n is a factor to control the effect of C_i.

Figure 4 illustrates the influence of n in different conditions. The error rates of the context information sources are listed in the legend area of Figure 4, which can be seen as different conditions.

Figure 4.  The influence of n.

DownLoad: Full-Size Img PowerPoint

From Figure 4, we can see that the A-wsa algorithm gets the best performance when n = 3. So we set n to be 3 in experiments No.1 to No.5.

In the following experiments No.1 to No.5, Figures 5–9 illustrate the performances of five inconsistent context processing algorithms under different conditions where the error rates of context information sources vary with the relation according to Table 2.

Figure 5.  ${C_{CAR}}$ with respect to error rate of ${S_1}$. (Experiment No.1: $E_{1} = \mathit 0-0.5, E_{2} = \mathit 0.0\mathit5, E_{3} = \mathit 0.0\mathit 5, E_{4} = \mathit 0.0\mathit5, E_{5} = \mathit 0.0\mathit 5$.).

DownLoad: Full-Size Img PowerPoint

Figure 6.  ${C_{CAR}}$ with respect to error rate of ${S_1}$. (Experiment No.2: $E_{1} = \mathit0-0.5, E_{2} = \mathit0.0\mathit5, E_{3} = \mathit0.0\mathit5, E_{4} = \mathit0.0\mathit5, E_{5} = \mathit0.3\mathit5$).

DownLoad: Full-Size Img PowerPoint

Figure 7.  ${C_{CAR}}$ with respect to error rate of ${S_1}$. (Experiment No.3: $E_{1} = \mathit0-0.5, E_{2} = \mathit0.0\mathit5, E_{3} = \mathit0.0\mathit5, E_{4} = \mathit0.3\mathit5, E_{5} = \mathit0.3\mathit5$.).

DownLoad: Full-Size Img PowerPoint

Figure 8.  ${C_{CAR}}$ with respect to error rate of ${S_1}$. (Experiment No.4: $E_{1} = \mathit0-0.5, E_{2} = \mathit0.0\mathit5, E_{3} = \mathit0.3\mathit5, E_{4} = \mathit0.3\mathit5, E_{5} = \mathit0.3\mathit5$.).

DownLoad: Full-Size Img PowerPoint

Figure 9.  ${C_{CAR}}$ with respect to error rate of ${S_1}$. (Experiment No.5: $E_{1} = \mathit0-0.5, E_{2} = \mathit0.3\mathit5, E_{3} = \mathit0.3\mathit5, E_{4} = \mathit0.3\mathit5, E_{5} = \mathit0.3\mathit5$.).

DownLoad: Full-Size Img PowerPoint

Figure 5 shows the performance of five context information processing algorithms in experiment No.1. It can be seen that the context aware rate of the A-wsa algorithm is almost the same as the A-vote algorithm and better than the other algorithms.

Figure 6 illustrates the context aware rate of five inconsistent context processing algorithms in experiment No.2. We can see that when the error rate of S₁ changes from 0 to 0.1, the A-wsa algorithm can get the best performance, which is almost equal to the context aware rate of the A-vote algorithm. When the error rate of S₁ is larger than 0.1, the context aware rate of the A-wsa algorithm is better than the other algorithms and keep stable around 0.99.

As shown in Figure 7, the context aware rates of five inconsistent context information processing algorithms decrease when the error rate of context information source S₁ increases. We can see that the performance of the A-wsa algorithm is better than the other algorithms as error rate of context information source S₁ increases from 0.05 to 0.5. When the error rate of context information source S₁ is 0, the context aware rate of the A-wsa algorithm is 0.9963 and which is better than that of the A-vote algorithm and the A-DST algorithm. When error rate of context information source S₁ is 0.5, the A-wsa algorithm gets a good performance and the context aware rate is 0.9629. We can see that the performance of the A-wsa algorithm is relatively steady.

In Figure 8, the performances of five inconsistent context information processing algorithms are presented. We can see that as the error rate of context information source S₁ changes from 0.05 to 0.2, the A-wsa algorithm obtains the best performance among the five algorithms. The context aware rate of the A-wsa algorithm is lower than that of the A-tw algorithm, when the error rate of context information source S₁ changes from 0.2 to 0.5. The difference of context aware rates is 0.101 when the error rate of context information source S₁ gets to be 0.5.

Figure 9 shows the results of experiment No.5 and gives a comparison of context aware rate of five inconsistent context information processing algorithms. We can see that the performance of the A-wsa algorithm is similar to that of the A-fb algorithm. Meanwhile, the A-wsa algorithm gets performance enhancement compared to other algorithms when the error rate of context information source S₁ is 0.15.

The performances of context processing in Figures 5–9 show that the advantages of the proposed algorithm in context information fusion. For instance, in Figure 7, the improvement of the context aware rate by the proposed algorithm is 2.41% compared to the A-vote algorithm, 3.15% compared to the A-tw algorithm, the A-fb algorithm, and the A-DST algorithm.

4.   Conclusions

In this paper, a new inconsistent context fusion algorithm based on BP neural network and modified DST is presented. Through modeling the raw context information collected from different sources on the basis of the state-space-based context model, the situation of entities can be recognized with the help of BP neural network. Then the recognition framework can be built based on the modification of DST. We transform the output of neural network into the BBAs of different situations which can be treated as the focal elements of the recognition framework. Through the correct rates, the fusion result is improved by adjusting the conflict redistribution on the focal elements. The experiment results show that the proposed weighted self-adaptive rule can obtain better context fusion results compared to other inconsistent context processing algorithms in most cases by setting up different error rates, so we can get the conclusion that the proposed algorithm has good accuracy and robustness.

In future's work, we plan to delve into BP neural network to improve the algorithm and apply it to more real-world scenarios to deal with the inconsistency problem of complex multi-modal contexts.

Acknowledgments

This work was financially supported by the National Key Research and Development Program of China (2018YFC0831001) and the Natural Science Foundation of Shandong Province of China (ZR2020MF139).

Conflict of Interest

All authors declare that they have no conflict of interest.