MSFSS: A whale optimization-based multiple sampling feature selection stacking ensemble algorithm for classifying imbalanced data

1.   Introduction

A matrix $A\in \mathbb{R}^{n \times n}$ is said to be stable if all its eigenvalues lie in the open left half plane, i.e., all the eigenvalues of $A$ have negative real parts. It is well-known that a matrix $A$ is stable if and only if there is a positive definite matrix $P$ such that $A^TP+PA$ is negative definite. This implies that $V(x) = x^TPx$ serves as a quadratic Lyapunov function for the asymptotically stable linear system

In this paper, we consider only real square matrices. Let $A$ be a real $n \times n$ matrix. $A \succ 0$ ($A \succeq 0$, resp.) means $A$ is a symmetric positive definite (semidefinite, resp.) matrix.

For the sake of convenience, we will adopt the concept of positive stability. A matrix $A \in \mathbb{R}^{n \times n}$ is defined as positive stable if all its eigenvalues possess positive real parts. Clearly, if $A$ is positive stable, then $-A$ is stable. Therefore, results in positive stability can be translated into terms of stability.

It is well-established that a matrix $A\in \mathbb{R}^{n \times n}$ is positive stable if and only if there exists $P \succ 0$ in $\mathbb{R}^{n \times n}$such that

In this case, $P$ is known as a Lyapunov solution for $A$ or to the Lyapunov inequality $(1.1)$. Several numerical methods have been developed to address the problem of finding such matrices $P$ ^[1,2,3].

A particular case emerges from $(1.1)$ when a positive diagonal matrix $D$ satisfies the Lyapunov inequality. If so, $D$ is called a Lyapunov diagonal solution for $A$. Furthermore, we say that $A$ is a Lyapunov diagonally stable matrix. The problem of Lyapunov diagonal stability is well investigated in the literature (^{[4,5,6,7,8,9]} and the references therein). The importance of this problem is due to its applications in, most significantly, population dynamics ^[10], communication networks ^[11], and systems theory ^[12].

Another case of $(1.1)$, known as Lyapunov $\alpha$-scalar stability, appeard in ^[13]. For a partition $\alpha = \{\alpha_1, \dots, \alpha_s\}$ of the set $\{1, \dots, n\}$, the diagonal solution $D$ has an $\alpha$-scalar structure, i.e. $D[\alpha_i] = c_iI$, $c_i\in \mathbb{R}$, $i = 1, \dots, s$, where $D[\alpha_i]$ is the principal submatrix of $D$ on row and column indices $\alpha_i$. A set $\alpha = \{\alpha_1, \dots, \alpha_s\}$, $1 \leq s\leq n$, is said to be a partition of $\{1, \dots, n\}$ if for all $i, j \in \{1, \dots, s\}$, $\alpha_i \ne \emptyset$, $\alpha_i \cap \alpha_j = \emptyset$, and $\alpha_i \cup \dots \cup \alpha_s = \{1, \dots, n\}$. We assume, without loss of generality, that these $\alpha_i$'s are taken to have contiguous indices because our results are applicable with simultaneous row and column permutations.

For brevity, if $A\in \mathbb{R}^{k \times k}$ is a Lyapunov diagonally stable matrix, we will write $A \in LDS_k$. Similarly, we write $A \in LDS^{\alpha}_k$ if $A$ is Lyapunov $\alpha$-scalar stable.

A recent generalization of Lyapunov diagonal stability to a family of real matrices of the same size, $\mathcal{A} = \{A^{(i)}\}_{i = 1}^r$, has drawn significant interest ^{[14,15,16,17,18]}. This extension studies the existence of a diagonal matrix $D\succ 0$ satisfying

$i = 1, \dots, r$. If such matrix $D$ exists, it is known as a common Lyapunov diagonal solution for $\mathcal{A}$ or to $(1.2)$. Consequently, we say $\mathcal{A}$ has common Lyapunov diagonal stability. From this definition, it is clear that common Lyapunov diaognal stability can be interpreted as simultaneous Laypunov diagonal stability for the matrices in $\mathcal{A}$. The existence of a common Lyapunov diagonal solution $D$ for $\mathcal{A}$ implies that $V(x) = x^TDx$ acts as a common Lyapunov diagonal function for the collection of asymptotically stable linear systems

An immediate observation here is that when $\mathcal{A} = {A}$, i.e., $\mathcal{A}$ is a singleton, $\mathcal{A}\in CLDS$ is equivalent to $A\in LDS$, and $\mathcal{A}\in CLDS^{\alpha}$ is equivalent to $A\in LDS^{\alpha}$. Additionally, it is worth mentioning that the cardinality of $\mathcal{A}$ is not relevant. For convenience, we shall fix it to be $r$ throughout the rest of this note.

Applications of common Lyapunov diagonal stability have been found in the fields of large-scale dynamics ^{[19,20,21,22]}, as well as in the study of interconnected time-varying and switched systems ^[18]. Beyond these practical applications, common Lyapunov diagonal stability is also a significant research topic in itself, as evidenced by works such as ^{[14,16,17,23]}.

Let $A\in \mathbb{R}^{n \times n}$ be a nonsingular matrix. In ^[24], Redheffer proved that $A\in LDS_n$ if and only if the $(n-1) \times (n-1)$ leading principal submatrices of $A$ and $A^{-1}$ have a common Lyapunov diagonal solution. This result has been restated in ^[16,23] using the notion of Schur complements. The new statement is free of the nonsingularity condition. Specifically, it was shown that a matrix $A \in LDS_n$ if and only if $a_{nn} > 0$ and the $(n-1)\times (n-1)$ leading principal submatrix of $A$ and its Schur complement have a common Lyapunov diagonal solution.

For any vectors $u, v \in \mathbb{R}^n$, when we write $u \gg v$, it means $u_i > v_i$ for all $i\in \{1, \dots, n\}$. For a matrix $A\in \mathbb{R}^{n \times n}$, the vector $u\in \mathbb{R}^n$ with $u_i = a_{ii}$, for $i = 1, \dots, n$, is denoted by ${\rm diag}(A)$. We denote the identity matrix $I \in \mathbb{R}^{k \times k}$ by $I_k$ and the matrix of all ones in $\mathbb{R}^{k \times k}$ by $J_k$.

Let $A = [a_{ij}], B = [b_{ij}] \in \mathbb{R}^{n \times n}$ and $C \in \mathbb{R}^{m \times m}$. The Hadamard product of $A$ and $B$ is denoted by $A \circ B = [a_{ij}b_{ij}]\in \mathbb{R}^{n \times n}$. The Kronecker product of $A$ and $C$ is denoted by $A \otimes C = [a_{ij}C]\in \mathbb{R}^{nm \times nm}$.

Let $n, m\in \mathbb{N}$ and for $i = 1, \dots, n$, let $m_i \in \mathbb{N}$ such that $m = m_1+\dots+m_n$, where $n \leq m$. Then, a matrix $S\in \mathbb{R}^{m \times m}$ is an $n$ by $n$ block matrix if it is partitioned into blocks that conform with $m_i$, $i = 1, \dots, n$. Moreover, we denote each $m_j$ by $m_k$ block of $S$ as $S_{jk}$. Similarly, a vector $u\in \mathbb{R}^m$ is called an n-block vector if it is partitioned into $n$ subvectors, i.e., $u^T = [u_{m_1}^T \dots u_{m_n}^T]$, where $u_{m_i} \in \mathbb{R}^{m_i}$, $i = 1, \dots, n$. Throughout this note, it is assumed that $n, m$ and all $m_i$, $i = 1, \dots, n$, are natural number with $n\leq m$ and $m = m_1+\dots+m_n$.

The Khatri-Rao product of a matrix $A = [a_{ij}] \in \mathbb{R}^{n \times n}$ and an $n$ by $n$ block matrix $B = [B_{ij}]\in \mathbb{R}^{m \times m}$ is defined as $A \star B = [a_{ij}B_{ij}]\in \mathbb{R}^{m \times m}$. Similarly, let $v = [v_i]\in \mathbb{R}^n$ and $u = [u_{m_i}]\in \mathbb{R}^m$ be $n$-block vector, then $v \star u = [v_iu_{m_i}]\in \mathbb{R}^m$.

Suppose that $\emptyset \neq\alpha \subseteq \{1, \dots, k\}$, $|\alpha|$ is the cardinality of $\alpha$ and $\alpha^c = \{1, \dots, n\} \setminus \alpha$. We denote the principal submatrix of $A$ obtained by selecting rows and columns indexed by $\alpha$ as $A[\alpha]$. Similarly, for a vector $u\in \mathbb{R}^k$, $u[\alpha]$ represents the subvector of $u$ containing only the elements indexed by $\alpha$.

Lemma 1.1. ([25,Corollary 4.2.13]) Let $A \in \mathbb{R}^{n \times n}$ and $B\in \mathbb{R}^{m \times m}$. If $A$ and $B$ are both positive semidefinite matrices then $A \otimes B \in \mathbb{R}^{nm \times nm}$ is also a positive semidefinite matrix.

Lemma 1.2. ([26,Theorem 3.1]) Let $A \in \mathbb{R}^{n \times n}$ be a positive definite matrix. If $B\in \mathbb{R}^{m \times m}$ is a positive semidefinite $n$ by $n$ block matrix with $B_{ii}\succ 0$, $i = 1, \dots, m$, then $A \star B \succ 0$.

For the remainder of this note, $Q = [q_{ij}]\in \mathbb{R}^{m \times m}$ denotes the nonzero $n$ by $n$ block matrix defined as

Additionally, for an $n$ by $n$ block matrix $B \in \mathbb{R}^{m \times m}$, we define the matrix $T(B)\in \mathbb{R}^{n \times n}$ such that $(T(B))_{ij} = (B_{ij})_{11}$.

Now, let us recall the definition of a $P$-matrix. A matrix whose principal minors are all positive is known as a $P$-matrix. A well-known characterization for $P$-matrices in the context of real matrices is given next.

Lemma 1.3. ([27,Theorem 3.3]) A matrix $A\in \mathbb{R}^{n \times n}$ is a $P$-matrix if and only if $u_i(Au)_i > 0$ for all nonzero $u\in \mathbb{R}^n$.

Motivated by Lemma $1.3$, a generalization of the concept of $P$-matrices to $P^{\alpha}$-matrices has been developed in ^[13].

Definition 1.1. Let $A\in \mathbb{R}^{n \times n}$ and $\alpha = \{\alpha_1, \dots, \alpha_s\}$ be a partition of $\{1, \dots, n\}$. Then, $A$ is a $P^{\alpha}$-matrix if there is some $k\in \{1, \dots, s\}$ such that $u[\alpha_k]^T(Au)[\alpha_k] > 0$ for all nonzero $u\in \mathbb{R}^n$.

For $A\in \mathbb{R}^{k \times k}$, $A \in P_k$ indicates that $A$ is a $P$-matrix, while $A \in P^{\alpha}_k$ means that $A$ is $P^{\alpha}$-matrix.

Using the characterization in Lemma $1.3$ and Definition 1.1, the $P$-matrix and $P^{\alpha}$-matrix properties were extended in ^[17,28], respectively, to consider a family of real matrices.

Definition 1.2. Let $\mathcal{A} = \{A^{(i)}\}_{i = 1}^r$ be a family of real $n \times n$ matrices. Then, $\mathcal{A}$ is called a $\mathcal{P}$-set and write $\mathcal{A} \in \mathcal{P}_n$ if for any family of vectors $\{u^{(i)}\}_{i = 1}^r$ in $\mathbb{R}^n$, not all being zero, there is some $k\in \{1, \dots, n\}$ such that

Definition 1.3. Let $\mathcal{A} = \{A^{(i)}\}_{i = 1}^r$ be a family of real $n \times n$ matrices and $\alpha = \{\alpha_1, \dots, \alpha_s\}$ be a partition of $\{1, \dots, n\}$. Then, $\mathcal{A}$ is called a $\mathcal{P}^{\alpha}$-set and write $\mathcal{A} \in \mathcal{P}_n^{\alpha}$ if for any family of vectors $\{u^{(i)}\}_{i = 1}^r$ in $\mathbb{R}^n$, not all being zero, there is some $k\in \{1, \dots, s\}$ such that

Theorem 1.4. ([14,Theorem 2]) Let $\mathcal{A} = \{A^{(i)}\}_{i = 1}^r$ be a family of matrices such that $A^{(i)} \in \mathbb{R}^{n \times n}$, $i = 1, \dots, r$. Then, $\mathcal{A} \in CLDS_n$ if and only if the matrix

has a positive diagonal entry for any $H^{(i)}\succeq 0$ in $\mathbb{R}^{n \times n}$, $i = 1, \dots, r$, not all being zero.

Theorem 1.5. ([17,Theorem 2.5]) Let $\mathcal{A} = \{A^{(i)}\}_{i = 1}^r$ be a family of matrices such that $A^{(i)} \in \mathbb{R}^{n \times n}$, $i = 1, \dots, r$. Then, the following are equivalent:

(i) $\mathcal{A} \in CLDS_n$.

(ii) $\{A^{(i)} \circ S^{(i)}\}_{i = 1}^r \in CLDS_n$ for all $S^{(i)}\succeq 0$, with ${\rm diag}(S^{(i)})\gg 0$ for $i = 1, \dots, r.$

(iii) $\{A^{(i)} \circ S^{(i)}\}_{i = 1}^r \in CLDS_n$ for all $S^{(i)}\succeq 0$, with ${\rm diag}(S^{(i)}) = e$ for $i = 1, \dots, r.$

(iv) $\{A^{(i)} \circ S^{(i)}\}_{i = 1}^r \in \mathcal{P}_n$ for all $S^{(i)}\succeq 0$, with ${\rm diag}(S^{(i)})\gg 0$ for $i = 1, \dots, r.$

(v) $\{A^{(i)} \circ S^{(i)}\}_{i = 1}^r \in \mathcal{P}_n$ for all $S^{(i)}\succeq 0$, with ${\rm diag}(S^{(i)}) = e$ for $i = 1, \dots, r.$

The above two theorems provide characterizations for common Lyapunov diagonal stability. Theorem $1.4$ extends Theorem 1 from ^[4], while Theorem $1.5$ is inspired by the work of Kraaijevanger ^[8]. The primary objective of our work is to offer additional characterizations that enhance and unify the existing results in the literature.

2.   Common Lyapunov diagonal stability

We begin this section with a lemma that gives a necessary condition for the common Lyapunov diagonal stability.

Lemma 2.1. ([17,Theorem 2.3]) Let $\mathcal{A} = \{A^{(i)}\}_{i = 1}^r$ be a family of real $n \times n$ matrices. If $\mathcal{A} \in CLDS_n$, then $\mathcal{A} \in \mathcal{P}_n$.

Next, we demonstrate that if a family of matrices $\mathcal{A}$ of the same size forms a $\mathcal{P}$-set, then any family of principal submatrices of $\mathcal{A}$ obtained by deleting the same rows and columns also forms a $\mathcal{P}$-set.

Lemma 2.2. Let $\mathcal{A} = \{A^{(i)}\}_{i = 1}^r$ be a family of real $n \times n$ matrices and $\emptyset \neq\alpha \subseteq \{1, \dots, n\}$. If $\mathcal{A} \in \mathcal{P}_n$, then $\mathcal{B} = \{A[\alpha]\}_{i = 1}^r \in \mathcal{P}_{|\alpha|}$.

Proof. For $i = 1, \dots, r$, let $v^{(i)} \in \mathbb{R}^{|\alpha|}$, not all being zero. Then, for each $i$, construct $u^{(i)}\in \mathbb{R}^n$ to be such that $u^{(i)}[\alpha] = v^{(i)}$ and $u^{(i)}[\alpha^c] = 0$. Clearly, not all these $u^{(i)}$'s are zero vectors since not all $v^{(i)}$'s are zero. Hence, since $\mathcal{A} \in \mathcal{P}_n$, there is some $k\in \{1, \dots, n\}$ such that

Observe that for each $i$, $u^{(i)}_k = v^{(i)}_l$ and $(A^{(i)}u^{(i)})_k = (A[\alpha]^{(i)}v^{(i)})_l$ for some $l\in \alpha$. Otherwise, the above summation equals zero. From this observation, we obtain that

Therefore, by Definition 1.2, $\mathcal{B} \in \mathcal{P}_{|\alpha|}$. □

We are now ready to present our main theorem.

Theorem 2.3. Let $\mathcal{A} = \{A^{(i)}\}_{i = 1}^r$ be a family of matrices such that $A^{(i)}\in \mathbb{R}^{n \times n}$ for $i = 1, \dots, r$. Then, the following are equivalent:

(i) $\mathcal{A}\in CLDS_n$.

(ii) $\{A^{(i)} \star S^{(i)}\}_{i = 1}^r\in CLDS_m$ for all $n$ by $n$ block matrices $S^{(i)}\succeq 0$ in $\mathbb{R}^{m \times m}$ with $S^{(i)}_{jj} \succ 0$ for all $i = 1, \dots, r$ and $j = 1, \dots, n$.

(iii) $\{A^{(i)} \star S^{(i)}\}_{i = 1}^r\in CLDS_m$ for all $n$ by $n$ block matrices $S^{(i)}\succeq 0$ in $\mathbb{R}^{m \times m}$ with $S^{(i)}_{jj} = I_{m_j}$ for all $i = 1, \dots, r$ and $j = 1, \dots, n$.

(iv) $\{A^{(i)} \star S^{(i)}\}_{i = 1}^r\in \mathcal{P}_m$ for all $n$ by $n$ block matrices $S^{(i)}\succeq 0$ in $\mathbb{R}^{m \times m}$ with $S^{(i)}_{jj} \succ 0$ for all $i = 1, \dots, r$ and $j = 1, \dots, n$.

(v) $\{A^{(i)} \star S^{(i)}\}_{i = 1}^r\in \mathcal{P}_m$ for all $n$ by $n$ block matrices $S^{(i)}\succeq 0$ in $\mathbb{R}^{m \times m}$ with $S^{(i)}_{jj} = I_{m_j}$ for all $i = 1, \dots, r$ and $j = 1, \dots, n$.

Proof. It is trivial to see that $(ii)$ implies $(iii)$ and $(iv)$ implies $(v)$. Moreover, from Lemma $2.1$, it is clear that $(ii)$ implies $(iv)$ and $(iii)$ implies $(v)$. Hence, to finish the proof, we show that $(i)$ implies $(ii)$ and $(v)$ implies $(i)$.

$(i) \Rightarrow (ii)$: Suppose that $D \succ 0$ in $\mathbb{R}^{n \times n}$ is a common Lyapunov diagonal solution for $\mathcal{A}$. Then, for $i = 1, \dots, r$, we have $(A^{(i)})^T D + D A^{(i)} \succ 0$. Let $\{S^{(i)}\}_{i = 1}^r$ be any family of positive semidefinite $n$ by $n$ block matrices in $\mathbb{R}^{m \times m}$ with $S^{(i)}_{jj} \succ 0$ for $j = 1, \dots, n$ and $i = 1, \dots, r$. Hence, according to Lemma 1.2, we have

for $i = 1, \dots, r$. Since we have

it follows from (2.1) that

Now, observe that

and

for each $i$, where $I_m \in \mathbb{R}^{m \times m}$ is the identity matrix partitioned into $n$ by $n$ blocks. Using these observations, it follows from (2.2) that

for $i = 1, \dots, r$. Clearly, the diagonal matrix $D \star I_m \in \mathbb{R}^{m \times m}$ is positive definite. Hence, $(ii)$ follows.

$(v) \Rightarrow (i)$: For $i = 1, \dots, r$, let $X^{(i)} = [x^{(i)}_{kl}]\succeq 0$ in $\mathbb{R}^{n \times n}$, not all being zero. Now, set $D^{(i)}$ to be the diagonal matrices whose diagonal elements $d^{(i)}_{kk} = \sqrt{x^{(i)}_{kk}}$ for all $i = 1, \dots, r$ and $k = 1, \dots, n$. Thus, for each $i$, we can write $X^{(i)} = D^{(i)}S^{(i)}D^{(i)}$ for some $S^{(i)} = [s^{(i)}_{kl}]\succeq 0$ in $\mathbb{R}^{n \times n}$ with $s_{kk} = 1$, $k = 1, \dots, n$. Next, let us fix $p = {\rm max}\{m_1, \dots, m_n\}$. Then, by Lemma $1.1$, $S^{(i)} \otimes I_p\succeq 0$ in $\mathbb{R}^{np \times np}$, $i = 1, \dots r$. Observe that for each $i$, $S^{(i)} \star Q \in \mathbb{R}^{m \times m}$ is a principal submatrix of $S^{(i)} \otimes I_p$, where $Q$ is a matrix defined as in $(1.3)$. Therefore, we conclude that $S^{(i)} \star Q \succeq 0$, with $(S^{(i)} \star Q)_{jj} = I_{m_j}$, $i = 1, \dots, r$, $j = 1, \dots, n$. By $(v)$, $\{A^{(i)} \star (S^{(i))} \star Q) \}_{i = 1}^r \in \mathcal{P}_m$. So, we obtain from Lemma $2.2$ that $\{T(A^{(i)} \star (S^{(i)} \star Q))\}_{i = 1}^r\in \mathcal{P}_n$. Now, let $u^{(i)}\in \mathbb{R}^{n \times n}$, $i = 1, \dots, r$, be such that $u_k^{(i)} = d^{(i)}_{kk}$, $k = 1, \dots, n$. It is clear that not all $u^{(i)}$ are zero vectors. Thus, from the definition of $\mathcal{P}$-sets, we must have

for some $q\in \{1, \dots, n\}$. Hence, it follows that

From this last inequality and by Theorem $1.4$, $(i)$ holds.

The proof is complete now. □

To demonstrate the validity of Theorem $2.3$, consider the following example.

Example 2.1. Let $n = 2$, $m = 3$, $m_1 = 2$, and $m_2 = 1$. Then, consider the family $\mathcal{A} = \{A^{(i)}\}_{i = 1}^2$, where

According to Theorem $2.3$, to show that $\mathcal{A}\in CLDS_2$ it suffices to show that $\{A^{(i)} \star S^{(i)}\}_{i = 1}^2 \in \mathcal{P}_3$ for any $2$ by $2$ block matrices $S^{(i)} \succeq 0$ in $\mathbb{R}^{3 \times 3}$ with $S_{jj}^{(i)} = I_{m_j}$ for all $i = 1, 2$ and $j = 1, 2$. Now, consider the matrices

Hence, we have

Next, for $i = 1, 2$, let $u^{(i)}$ be any vectors in $\mathbb{R}^3$. Thus, a simple calculation shows that

If at least one of $u_3^{(1)}$ and $u_3^{(2)}$ is nonzero, then $\sum_{i = 1}^{2}u_3^{(i)}(A^{(i)}u^{(i)})_3 > 0.$ Otherwise, we must have

Since $u^{(1)}$ and $u^{(2)}$ are not both zero vectors, then we must have $k\in \{1, 2\}$ such that $\sum_{i = 1}^{2}u_k^{(i)}(A^{(i)}u^{(i)})_k > 0.$ That means $\{A^{(i)} \star S^{(i)}\}_{i = 1}^2 \in \mathcal{P}_3$. Therefore, from Theorem $2.3$, this implies that $\mathcal{A} \in CLDS_2.$ In fact, we found that

is a common Lyapunov diagonal solution for $\mathcal{A}$.

We emphasize here that Theorem $2.6$ is equivalent to Theorem $1.5$ when $m = n$. Before we proceed with the presentation of further results, we cite the following two lemmas from ^[28].

Lemma 2.4. ([28,Lemma 3.2]) Let $\mathcal{A} = \{A^{(i)}\}_{i = 1}^r$ be a family of real $n \times n$ matrices and $\alpha$ be any partition of $\{1, \dots, n\}$. If $\mathcal{A}\in CLDS^{\alpha}_n$, then $\mathcal{A}\in \mathcal{P}^{\alpha}_n$.

Lemma 2.5. ([28,Proposition 4.1]) Let $\mathcal{A} = \{A^{(i)}\}_{i = 1}^r$ be a family of real $n \times n$ matrices and $\alpha$ be any partition of $\{1, \dots, n\}$. If $\mathcal{A} \in \mathcal{P}^{\alpha}_n$, then $\mathcal{A} \in \mathcal{P}_n$.

For the remainder of this paper, let $\alpha = \{\alpha_1, \dots, \alpha_n\}$ is a partition of $\{1, \dots, m\}$ such that $\alpha_1 = \{1, \dots, m_1\}, \; \alpha_2 = \{m_1+1, \dots, m_1+m_2\}, \; \dots \; , \; \alpha_n = \{m-m_n+1, \dots, m\}$. With this notation established, we now provide another characterization of common Lyapunov diagonal stability.

Theorem 2.6. Let $\mathcal{A} = \{A^{(i)}\}_{i = 1}^r$ be a family of matrices such that $A^{(i)}\in \mathbb{R}^{n \times n}$ for $i = 1, \dots, r$. Then, the following are equivalent:

(i) $\mathcal{A}\in CLDS_n$.

(ii) $\{A^{(i)} \star S^{(i)}\}_{i = 1}^r\in CLDS^{\alpha}_m$ for all $n$ by $n$ block matrices $S^{(i)}\succeq 0$ in $\mathbb{R}^{m \times m}$ with $S^{(i)}_{jj} \succ 0$ for all $i = 1, \dots, r$ and $j = 1, \dots, n$.

(iii) $\{A^{(i)} \star S^{(i)}\}_{i = 1}^r\in CLDS^{\alpha}_m$ for all $n$ by $n$ block matrices $S^{(i)}\succeq 0$ in $\mathbb{R}^{m \times m}$ with $S^{(i)}_{jj} = I_{m_j}$ for all $i = 1, \dots, r$ and $j = 1, \dots, n$.

(iv) $\{A^{(i)} \star S^{(i)}\}_{i = 1}^r\in \mathcal{P}^{\alpha}_m$ for all $n$ by $n$ block matrices $S^{(i)}\succeq 0$ in $\mathbb{R}^{m \times m}$ with $S^{(i)}_{jj} \succ 0$ for all $i = 1, \dots, r$ and $j = 1, \dots, n$.

(v) $\{A^{(i)} \star S^{(i)}\}_{i = 1}^r\in \mathcal{P}^{\alpha}_m$ for all $n$ by $n$ block matrices $S^{(i)}\succeq 0$ in $\mathbb{R}^{m \times m}$ with $S^{(i)}_{jj} = I_{m_j}$ for all $i = 1, \dots, r$ and $j = 1, \dots, n$.

Proof. It is clear that $(ii) \Rightarrow (iii)$ and $(iv) \Rightarrow (v)$. In addition, according to Lemma $2.4$, $(ii) \Rightarrow (iv)$ and $(iii) \Rightarrow (v)$.

$(i) \Rightarrow (ii)$: Let $\{S^{(i)}\}_{i = 1}^r$ be a family of positive semidefinite matrices given as in $(ii)$ and $D$ be a common Lyapunov diagonal solution for $\mathcal{A}$. Then, $D \star I_m$ is a positive $\alpha$-scalar matrix, where $I_m \in \mathbb{R}^{m \times m}$ is the identity matrix partitioned into $n$ by $n$ blocks. Thus, as we have seen in the proof of Theorem $2.3$, we have

for $i = 1, \dots, r$.

$(v) \Rightarrow (i)$: Using Lemma $2.5$, we can see that $(v)$ here implies $(v)$ in Theorem $2.3$. Therefore, $(i)$ holds. □

Now, we extend Theorem $2.6$ by considering different partitions of $\{1, \dots, m\}$. Before presenting our next result, let us set the stage first.

Let us define a bijective function $\tau: \alpha_i \rightarrow \beta_i$ that maps each element $j\in \alpha_i$ to some $\beta_i$ for $i\in \{1, \dots, n\}$. Hence, $\tau$ is a permutation of $\{1, \dots, m\}$, and $\beta$ is a partition of $\{1, \dots, m\}$. Clearly, for every $i$, the cardinality of $\alpha_i$ is the same as the cardinality of $\beta_i$. For the remainder of this section, $\beta$ denotes such partitions. In addition, construct the permutation matrix $P$ such that $P_{j\tau(j)} = 1$ for all $j = 1, \dots, m$ and zero everywhere else. For any permutation matrix $P$, we write $C_P = PCP^T$, where $C\in \mathbb{R}^{m \times m}$. Then, the following observation can be easily verified.

Observation 2.1. Let $P$ be a permutation matrix associated with some partition $\beta$. Then, we have

(1) $S\succeq 0$ ($S\succ 0$, resp.) if and only if $S_P \succeq 0$ ($S_P\succ 0$, resp.), $S \in \mathbb{R}^{m \times m}$.

(2) $D$ is $\beta$-scalar matrix if and only if $D_P$ is $\alpha$-scalar matrix, $D\in \mathbb{R}^{m \times m}$.

(3) $\{A^{(i)}\}_{i = 1}^r\in \mathcal{P}_m^{\beta}$ if and only if $\{A^{(i)}_P\}_{i = 1}^r\in \mathcal{P}_m^{\alpha}$, where $A^{(i)}\in \mathbb{R}^{m \times m}$, $i = 1, \dots, r$.

Lemma 2.7. Let $\mathcal{A} = \{A^{(i)}\}_{i = 1}^r$ be a family of matrices such that $A^{(i)}\in \mathbb{R}^{n \times n}$ for $i = 1, \dots, r$ and $P$ be a permutation matrix associated with some partition $\beta$. Then, $\{A^{(i)}\}_{i = 1}^r \in CLDS_n^{\beta}$ if and only if $\{A_P^{(i)}\}_{i = 1}^r \in CLDS_n^{\alpha}$.

Proof. The conclusion follows directly from observation $1$ and noting that

for all $i$. □

Theorem 2.8. Let $\mathcal{A} = \{A^{(i)}\}_{i = 1}^r$ be a family of matrices such that $A^{(i)}\in \mathbb{R}^{n \times n}$ for $i = 1, \dots, r$ and $P$ be a permutation matrix associated with some partition $\beta$. Then, the following are equivalent:

(i) $\mathcal{A}\in CLDS_n$.

(ii) $\{(P^T(A^{(i)} \star J_m)P) \circ S^{(i)}\}_{i = 1}^r\in CLDS_m^{\beta}$ for all matrices $S^{(i)}\succeq 0$ in $\mathbb{R}^{m \times m}$ with $S^{(i)}[\beta_j] \succ 0$ for all $i = 1, \dots, r$ and $j = 1, \dots, n$.

(iii) $\{(P^T(A^{(i)} \star J_m)P) \circ S^{(i)}\}_{i = 1}^r\in CLDS_m^{\beta}$ for all matrices $S^{(i)}\succeq 0$ in $\mathbb{R}^{m \times m}$ with $S^{(i)}[\beta_j] = I_{m_j}$ for all $i = 1, \dots, r$ and $j = 1, \dots, n$.

(iv) $\{(P^T(A^{(i)} \star J_m)P) \circ S^{(i)}\}_{i = 1}^r\in \mathcal{P}_m^{\beta}$ for all matrices $S^{(i)}\succeq 0$ in $\mathbb{R}^{m \times m}$ with $S^{(i)}[\beta_j] \succ 0$ for all $i = 1, \dots, r$ and $j = 1, \dots, n$.

(v) $\{(P^T(A^{(i)} \star J_m)P) \circ S^{(i)}\}_{i = 1}^r\in \mathcal{P}_m^{\beta}$ for all matrices $S^{(i)}\succeq 0$ in $\mathbb{R}^{m \times m}$ with $S^{(i)}[\beta_j] = I_{m_j}$ for all $i = 1, \dots, r$ and $j = 1, \dots, n$.

Proof. Clearly, the condition $(ii)$ gives $(iii)$ and $(iv)$ gives $(v)$. Moreover, $(ii)$ leads to $(iv)$ and $(iii)$ to $(v)$ by Lemma $2.4$.

$(i) \Rightarrow (ii)$: For $i = 1, \dots, r$, let $S^{(i)}\succeq 0$ in $\mathbb{R}^{m \times m}$ be given as in $(ii)$. Then, for each $i$, $S_P^{(i)}\succeq 0$ $n$ by $n$ block matrix with $(S^{(i)}_P)_{jj} \succ 0$, $j = 1, \dots, n$. Hence, it follows from Theorem $2.6$ that $\{A^{(i)} \star S_P^{(i)}\}_{i = 1}^r\in CLDS^{\alpha}_m$. Now, by observing that

for each $i$, we conclude that $\{((P^T(A^{(i)} \star J_m)P) \circ S^{(i)})_P\}_{i = 1}^r\in CLDS_m^{\alpha}$. Hence, by Lemma $2.7$, $(ii)$ follows.

$(v) \Rightarrow (i)$: From observation $1$, $\{((P^T(A^{(i)} \star J_m)P) \circ S^{(i)})_P\}_{i = 1}^r\in \mathcal{P}_m^{\alpha}$. This, by $(2.3)$, means $\{A^{(i)} \star S^{(i)}_P\}_{i = 1}^r\in \mathcal{P}_m^{\alpha}$. Finally, using Theorem $2.6$, we obtain $(i)$. This completes the proof. □

A final remark before moving to the next section is that Theorems $2.3$, $2.6$, and $2.8$ are equivalent to Theorems $4$, $9$, and $10$ in ^[29] when $\mathcal{A}$ is a singleton.

3.   Common Lyapunov $\alpha$-scalar stability

In this section, we generalize the main results of Section 2 to provide more characterizations for common Lyapunov $\alpha$-scalar stability. In this section, let $\gamma = \{\gamma_1, \dots, \gamma_s\}$ be any partition of $\{1, \dots, n\}$. Then, $\delta = \{\delta_1, \dots, \delta_s\}$, where

is a partition of $\{1, \dots, m\}$, where $\alpha = \{\alpha_1, \dots, \alpha_n\}$ is the partition of $\{1, \dots, m\}$ defined in Section 2.

Lemma 3.1. Let $\mathcal{A} = \{A^{(i)}\}_{i = 1}^r$ be a family of matrices such that $A^{(i)}\in \mathbb{R}^{n \times n}$ for $i = 1, \dots, r$ and $\gamma = \{\gamma_1, \dots, \gamma_s\}$ be any partition of $\{1, \dots, n\}$. If $\{A^{(i)} \star Q\}_{i = 1}^r \in \mathcal{P}_m^{\delta}$, then $\{T(A^{(i)} \star Q)\}_{i = 1}^r \in \mathcal{P}_n^{\gamma}$.

Proof. Let $u^{(i)}\in \mathbb{R}^n$, $i = 1, \dots, r$, not all being zero. Then, let $v = [v_{m_j}]\in \mathbb{R}^m$ be the nonzero $n$-block vector defined as follows

Then, for each $i$, choose $z^{(i)} = u^{(i)} \star v$. Clearly, $z^{(i)} \in \mathbb{R}^m$, $i = 1, \dots, r$, and not all being zero vectors since not all $u^{(i)}$ are zero. Furthermore, we have $T(z^{(i)}) = u^{(i)}$ and $T(z^{(i)}[\delta_l]) = u^{(i)}[\gamma_l]$ for $l \in \{1, \dots, s\}$. Now, because $\{A^{(i)} \star Q\}_{i = 1}^r \in \mathcal{P}_m^{\delta}$, there is some $l\in \{1, \dots s\}$ such that

Now, observe that

Consequently, it follows from $(3.1)$ that

Therefore, by Definition 1.3, the result follows. □

Lemma 3.2. ([28,Corollary 2.1]) Let $\mathcal{A} = \{A^{(i)}\}_{i = 1}^r$ be a family of matrices such that $A^{(i)}\in \mathbb{R}^{n \times n}$ for $i = 1, \dots, r$ and $\gamma = \{\gamma_1, \dots, \gamma_s\}$ be a partition of $\{1, \dots, n\}$. Then, $\mathcal{A}\in CLDS_n^{\gamma}$ if and only if there is $l\in \{1, \dots, s\}$ such that

for any $X^{(i)}\succeq 0$, $X^{(i)}\in \mathbb{R}^{n \times n}$, $i = 1, \dots, r$, not all being zero matrices.

Theorem 3.3. Let $\mathcal{A} = \{A^{(i)}\}_{i = 1}^r$ be a family of matrices such that $A^{(i)}\in \mathbb{R}^{n \times n}$ for $i = 1, \dots, r$ and $\gamma = \{\gamma_1, \dots, \gamma_s\}$ be any partition of $\{1, \dots, n\}$. Then, the following are equivalent:

(i) $\mathcal{A}\in CLDS_n^{\gamma}$.

(ii) $\{A^{(i)} \star S^{(i)}\}_{i = 1}^r\in CLDS_m^{\delta}$ for all $n$ by $n$ block matrices $S^{(i)}\succeq 0$ in $\mathbb{R}^{m \times m}$ with $S^{(i)}_{jj} \succ 0$ for all $i = 1, \dots, r$ and $j = 1, \dots, n$.

(iii) $\{A^{(i)} \star S^{(i)}\}_{i = 1}^r\in CLDS_m^{\delta}$ for all $n$ by $n$ block matrices $S^{(i)}\succeq 0$ in $\mathbb{R}^{m \times m}$ with $S^{(i)}_{jj} = I_{m_j}$ for all $i = 1, \dots, r$ and $j = 1, \dots, n$.

(iv) $\{A^{(i)} \star S^{(i)}\}_{i = 1}^r\in \mathcal{P}_m^{\delta}$ for all $n$ by $n$ block matrices $S^{(i)}\succeq 0$ in $\mathbb{R}^{m \times m}$ with $S^{(i)}_{jj} \succ 0$ for all $i = 1, \dots, r$ and $j = 1, \dots, n$.

(v) $\{A^{(i)} \star S^{(i)}\}_{i = 1}^r\in \mathcal{P}_m^{\delta}$ for all $n$ by $n$ block matrices $S^{(i)}\succeq 0$ in $\mathbb{R}^{m \times m}$ with $S^{(i)}_{jj} = I_{m_j}$ for all $i = 1, \dots, r$ and $j = 1, \dots, n$.

Proof. It is clear that $(ii)$ implies $(iii)$ and $(iv)$ implies $(v)$. Besides, from Lemma $2.4$, it is clear that $(ii)$ implies $(iv)$ and $(iii)$ implies $(v)$.

$(i) \Rightarrow (ii)$: Clearly, if $D$ is a positive $\gamma$-scalar matrix, then $D \star I_m$ is a positive $\delta$-scalar matrix. Moreover, if $D$ is a common Lyapunov $\gamma$-scalar solution for $\mathcal{A}$, then, by Lemma $1.2$, for any $S^{(i)}$'s given as in $(ii)$, we have

for $i = 1, \dots, r$. This last inequality means that $\{A^{(i)} \star S^{(i)}\}_{i = 1}^r$ has $(D \star I_m)$ as a common Lyapunov $\delta$-scalar solution.

$(v) \Rightarrow (i)$: Let $X^{(i)} = [x_{kl}]\succeq 0$, $X^{(i)} \in \mathbb{R}^{n \times n}$ $i = 1, \dots, r$, not all being zero. Next, let $D^{(i)} \in \mathbb{R}^{n \times n}$, $i = 1, \dots, r$, be a diagonal matrix such that $d^{(i)}_{kk} = \sqrt{x^{(i)}_{kk}}$ for $j = 1, \dots, n$. Thus, for each $i$, there is $S^{(i)} = [s_{kl}]\succeq 0$, $S^{(i)}\in \mathbb{R}^{n \times n}$ with $s_{kk}^{(i)} = 1$, $k = 1, \dots, n$, such that $X^{(i)} = D^{(i)}S^{(i)}D^{(i)}$. By setting $p = \max \{m_1, \dots, m_n\}$, we conclude, using to Lemma $1.1$, that $S^{(i)} \otimes I_p\succeq 0$ in $\mathbb{R}^{np \times np}$, $i = 1, \dots, r$. Since, for each $i$, $S^{(i)} \star Q$ is a principal submatrix of $S^{(i)} \otimes I_p$, then $S^{(i)} \star Q\succeq 0$, $Q$ here is the matrix in $(1.3)$. Furthermore, each diagonal block $(S^{(i)} \star Q)_{jj} = I_{m_j}$, $i = 1, \dots, r$. So, $\{A^{(i)} \star (S^{(i)} \star Q)\}_{i = 1}^r = \{(A^{(i)} \circ S^{(i)}) \star Q\}_{i = 1}^r\in \mathcal{P}_m^{\delta}$, by $(v)$. Consequently, according to Lemma $3.1$, $\{T((A^{(i)} \circ S^{(i)}) \star Q)\}_{i = 1}^r\in \mathcal{P}_n^{\gamma}$. Set $u^{(i)} = D^{(i)}e$, $i = 1, \dots, r$, where $e$ is the vector of all ones in $\mathbb{R}^n$. By the construction of these $u^{(i)}$'s, it is easy to see that not all of them are zero vectors. Therefore, there is some index $q\in \{1, \dots, s\}$ such that

Therefore, $(i)$ follows by Lemma $3.2$. This finishes the proof. □

This last Theorem can be generalized to provide more characterizations for common Lyapunov $\alpha$-scalar stability. Recall that in Section 2, we defined $\beta$ to be a partition of $\{1, \dots, m\}$ obtained from $\alpha$ through a permutation function $\tau$. Using this notation and the definition of $\delta$ above, for any partition $\gamma = \{\gamma_1, \dots, \gamma_s\}$ of $\{1, \dots, n\}$, we define another partition of $\{1, \dots, m\}$ called $\epsilon = \{\epsilon_1.\dots, \epsilon_s\}$, where

Clearly, if we replace $\alpha$ with $\delta$ and $\beta$ with $\epsilon$, Observation $1$ will hold true for a permutation matrix $P$ associated with $\beta$. Now, we have the following theorem, whose proof follows the lines of the proof of Theorem $2.8$ and is therefore omitted.

Theorem 3.4. Let $\mathcal{A} = \{A^{(i)}\}_{i = 1}^r$ be a family of matrices such that $A^{(i)}\in \mathbb{R}^{n \times n}$ for $i = 1, \dots, r$ and $\gamma = \{\gamma_1, \dots, \gamma_s\}$ be any partition of $\{1, \dots, n\}$. In addition, let $P$ be a permutation matrix associated with some partition $\beta$. Then, the following are equivalent:

(i) $\mathcal{A}\in CLDS^{\gamma}_n$.

(ii) $\{(P^T(A^{(i)} \star J_m)P) \circ S^{(i)}\}_{i = 1}^r\in CLDS_m^{\epsilon}$ for all matrices $S^{(i)}\succeq 0$ in $\mathbb{R}^{m \times m}$ with $S^{(i)}[\epsilon_j] \succ 0$ for all $i = 1, \dots, r$ and $j = 1, \dots, n$.

(iii) $\{(P^T(A^{(i)} \star J_m)P) \circ S^{(i)}\}_{i = 1}^r\in CLDS_m^{\epsilon}$ for all matrices $S^{(i)}\succeq 0$ in $\mathbb{R}^{m \times m}$ with $S^{(i)}[\epsilon_j] = I_{m_j}$ for all $i = 1, \dots, r$ and $j = 1, \dots, n$.

(iv) $\{(P^T(A^{(i)} \star J_m)P) \circ S^{(i)}\}_{i = 1}^r\in \mathcal{P}_m^{\epsilon}$ for all matrices $S^{(i)}\succeq 0$ in $\mathbb{R}^{m \times m}$ with $S^{(i)}[\epsilon_j] \succ 0$ for all $i = 1, \dots, r$ and $j = 1, \dots, n$.

(v) $\{(P^T(A^{(i)} \star J_m)P) \circ S^{(i)}\}_{i = 1}^r\in \mathcal{P}_m^{\epsilon}$ for all matrices $S^{(i)}\succeq 0$ in $\mathbb{R}^{m \times m}$ with $S^{(i)}[\epsilon_j] = I_{m_j}$ for all $i = 1, \dots, r$ and $j = 1, \dots, n$.

We remark here that Theorems $3.3$ and $3.4$ are the same as Theorems $2.6$ and $2.8$, respectively, when $\gamma = \{\{1\}, \{2\}, \dots, \{n\}\}$. Additionally, when $r = 1$, i.e., $\mathcal{A}$ is a singleton, these last two theorems reduce to the following corollaries, whose proofs shall be omitted.

Corollary 3.5. Let $A\in \mathbb{R}^{n \times n}$ and $\gamma = \{\gamma_1, \dots, \gamma_s\}$ be any partition of $\{1, \dots, n\}$. Then, the following are equivalent:

(i) $A\in LDS_n^{\gamma}$.

(ii) $A \star S\in LDS_m^{\delta}$ for all $n$ by $n$ block matrices $S\succeq 0$ in $\mathbb{R}^{m \times m}$ with $S_{jj} \succ 0$, $j = 1, \dots, n$.

(iii) $A \star S\in LDS_m^{\delta}$ for all $n$ by $n$ block matrices $S\succeq 0$ in $\mathbb{R}^{m \times m}$ with $S_{jj} = I_{m_j}$, $j = 1, \dots, n$.

(iv) $A \star S\in P_m^{\delta}$ for all $n$ by $n$ block matrices $S\succeq 0$ in $\mathbb{R}^{m \times m}$ with $S_{jj} \succ 0$, $j = 1, \dots, n$.

(v) $A \star S\in P_m^{\delta}$ for all $n$ by $n$ block matrices $S\succeq 0$ in $\mathbb{R}^{m \times m}$ with $S_{jj} = I_{m_j}$, $j = 1, \dots, n$.

Corollary 3.6. Let $A\in \mathbb{R}^{n \times n}$ and $\gamma = \{\gamma_1, \dots, \gamma_s\}$ be any partition of $\{1, \dots, n\}$. In addition, let $P$ be a permutation matrix associated with some partition $\beta$. Then, the following are equivalent:

(i) $A\in LDS_n^{\gamma}$.

(ii) $(P^T(A \star J_m)P) \circ S\in LDS_m^{\epsilon}$ for all matrices $S\succeq 0$ in $\mathbb{R}^{m \times m}$ with $S[\epsilon_j] \succ 0$, $j = 1, \dots, n$.

(iii) $(P^T(A \star J_m)P) \circ S\in LDS_m^{\epsilon}$ for all matrices $S\succeq 0$ in $\mathbb{R}^{m \times m}$ with $S[\epsilon_j] = I_{m_j}$, $j = 1, \dots, n$.

(iv) $(P^T(A \star J_m)P) \circ S\in P_m^{\epsilon}$ for all matrices $S\succeq 0$ in $\mathbb{R}^{m \times m}$ with $S[\epsilon_j] \succ 0$, $j = 1, \dots, n$.

(v) $(P^T(A \star J_m)P) \circ S\in P_m^{\epsilon}$ for all matrices $S\succeq 0$ in $\mathbb{R}^{m \times m}$ with $S[\epsilon_j] = I_{m_j}$, $j = 1, \dots, n$.

4.   Conclusions

Motivated by the work in ^[29], we have presented new characterizations for common Lyapunov diagonal stability using the Khatri-Rao product. The notions of $\mathcal{P}$-sets and $\mathcal{P}^{\alpha}$-sets have been used to formulate these results. Moreover, these characterizations have been extended to the notion of common Lyapunov $\alpha$-scalar stability. Our work here extends and broadens the scope of results in ^[17,28].

Use of AI tools declaration

The authors declare they have not used Artificial Intelligence (AI) tools in the creation of this article.

Acknowledgments

The Researchers would like to thank the Deanship of Graduate Studies and Scientific Research at Qassim University for financial support (QU-APC-2024-9/1). We would like to extend my sincere gratitude to the anonymous reviewers for their valuable comments and suggestions, which have significantly contributed to improving the quality of this paper.

Conflict of interest

The author does not have any conflict of interest.