Equivalent analysis of different estimations under a multivariate general linear model

1.   Introduction

Throughout this paper, the symbol ${\mathbb R}^{m \times n}$ stands for the collection of all $m \times n$ matrices over the field of real numbers; ${\bf{A}}^{T}$, $r({\bf{A}})$, and ${\mathscr{R}}({\bf{A}})$ stand for the transpose, the rank, and the range (column space) of a matrix ${\bf{A}} \in {\mathbb R}^{m \times n}$, respectively; ${\bf{I}}_m$ denotes the identity matrix of order $m$. Two symmetric matrices ${\bf{A}}$ and ${\bf{B}}$ of the same size are said to satisfy the inequality ${\bf{A}} \succcurlyeq {\bf{B}}$ in the Löwner partial ordering if ${\bf{A}} - {\bf{B}}$ is nonnegative definite. The Kronecker product of any two matrices ${\bf{A}}$ and ${\bf{B}}$ is defined to be ${\bf{A}} \otimes {\bf{B}} = (a_{ij} {\bf{B}})$. The vectorization operation of a matrix ${\bf{A}} = [\, {\bf a}_1, \ldots, {\bf a}_n \, ]$ is defined to be ${\rm{vec}}({\bf{A}}) = \overrightarrow{ {\bf{A}} }= [\, {\bf a}^{T}_1, \ldots, {\bf a}^{T}_n \, ]^{T}$. A well-known property of the vec operator of a triple matrix product is $\overrightarrow{ {\bf{A}} {\bf{Z}} {\bf{B}}} = ({\bf{B}}^{T}\otimes {\bf{A}})\overrightarrow{ {\bf{Z}} }$. The Moore–Penrose generalized inverse of ${\bf{A}} \in {\mathbb R}^{m \times n}$, denoted by ${\bf{A}}^{+}$, is defined by the unique solution ${\bf{G}}$ to the four matrix equations ${\bf{A}} {\bf{G}} {\bf{A}} = {\bf{A}}$, ${\bf{G}} {\bf{A}} {\bf{G}} = {\bf{G}}$, $({\bf{A}} {\bf{G}})^{T} = {\bf{A}} {\bf{G}}$, and $({\bf{G}} {\bf{A}})^{T} = {\bf{G}} {\bf{A}}$. In what follows, we denote by ${\bf{P}}_{\bf{A}} = {\bf{A}} {\bf{A}}^{+}$, ${\bf{A}}^{\perp} = {\bf{E}}_{\bf{A}} = {\bf{I}}_m - {\bf{A}} {\bf{A}}^{+}$, and ${\bf{F}}_{\bf{A}} = {\bf{I}}_n - {\bf{A}}^{+} {\bf{A}}$ the three orthogonal projectors induced by ${\bf{A}}$, respectively. Further information about the orthogonal projectors ${\bf{P}}_ {\bf{A}}$, ${\bf{E}}_ {\bf{A}}$, and ${\bf{F}}_ {\bf{A}}$ and their applications in the linear statistical models can be found, e.g., in ^{[17,20,26,27]}.

In this paper, we consider the following partitioned multivariate general linear model:

where ${\bf{Y}} \in {\mathbb R}^{n \times m}$ is a matrix of observable dependent variables that comes from an experimental design giving rise to $n$ observations, ${\bf{X}} = [\, {\bf{X}}_1, \ldots, {\bf{X}}_k \, ] \in {\mathbb R}^{n \times p}$ is a model matrix with arbitrary rank ($0 \leq r({\bf{X}}) \leq \min\{n, p\})$, ${\bf{X}}_i \in {\mathbb R}^{n \times p_i}$ and $\Theta_i \in {\mathbb R}^{p_i \times m}$ are matrices of fixed but unknown parameters, $\Theta = [\, \Theta^{T}_1, \ldots, \Theta^{T}_k \, ]^{T} \in {\mathbb R}^{p \times m}$, with $p = p_1 + \cdots + p_k$, $i = 1, \ldots, k$, and ${\rm{E}}(\overrightarrow{\Psi})$ and ${\rm{Cov}}(\overrightarrow{\Psi})$ denote the expectation and the covariance matrix of the random error vector $\overrightarrow{\Psi}$, $\sigma^2$ is an arbitrary positive scaling factor, $\pmb{\Sigma}_1 \in {\mathbb R}^{n \times n}$ and $\pmb{\Sigma}_2 \in {\mathbb R}^{m \times m}$ are known positive semi-definite matrices of arbitrary ranks ($0 < r(\pmb{\Sigma}_1) \leq n)$ and ($0 < r(\pmb{\Sigma}_2) \leq m)$, the Kronecker product $\pmb{\Sigma}_2 \otimes \pmb{\Sigma}_1$ means that $\overrightarrow{\Psi}$ has a separable covariance structure. In this study, we do not necessarily assume that the random variables have normal distributions. Only if more precise statistical properties are required, are normal or alternative distributions usually assumed and discussed (^[4,22,23]).

We now give some introductions and remarks on the backgrounds of MGLMs. Equation (1.1) is a relative direct extension of the most welcome type of univariate general linear models, which means the incorporation of regressing one response variable on a given set of regressors to several response variables on the regressors. This model is also a representative of various multivariate regression frameworks, yet, has been a core object of study in the domain of multivariate analysis and applications. In fact, MGLMs have the advantage of more complete situations that involve a number of variables, both independent and dependent, and occur in many fields of statistical sciences, such as, analysis of variance (ANOVA), analysis of covariance (ANCOVA), multivariate analysis of variance (MANOVA), analysis of repeated measurements, factor analysis models, as well as many areas of applications to describe predictive relationships of multiple responses related to a set of regressors. The increased employment of repeated measures for panel studies has led to the necessity for more researches in the modeling of this type of data; see a number of textbook and handbook literature ^{[2,6,10,14,21,29,32]} that refers to the general theory of MGLMs and their applications. Due to the matrix structures of the model equations in MGLMs, a commonly used method in the statistical analysis of MGLMs is to use the well-known Kronecker products and vectorization operations of matrices. Through the use of these operations, we can alternatively represent (1.1) in the following univariate general linear model:

As we know, there are a number of optimality criteria that can be adopted to establish estimation theory for unknown parameters in linear regression models. In comparison, the two kinds of classical and widely-used estimators are known as the ordinary least-squares estimators (for short, OLSEs) and the best linear unbiased estimators (for short, BLUEs), both of which are in fact two fundamental and orthodox estimators of unknown parameters in theory and applications of linear statistical models for regression. Below, we briefly introduce the existing definitions of the estimability, OLSEs, and BLUEs of the unknown parameters under (1.1) (^[1,26]), and then present some known exact and analytical formulas for calculating the OLSEs and BLUEs and their properties.

Definition 1.1. Let ${\mathscr{M}}$ and $\widehat{ {\mathscr{M}}}$ be as given in (1.1) and (1.2), respectively.

(ⅰ) The matrix ${\bf{K}}\Theta$ of parametric functions, where ${\bf{K}} \in {\mathbb R}^{k \times p}$, is said to be estimable under ${\mathscr{M}}$ if there exists an ${\bf{L}} \in {\mathbb R}^{k \times n}$ such that ${\rm{E}}({\bf{L}} {\bf{Y}} - {\bf{K}}\Theta) = {\bf{0}}$.

(ⅱ) The vector ${\bf{T}}\overrightarrow{\Theta}$ of parametric functions, where ${\bf{T}} \in {\mathbb R}^{t \times mp}$, is said to be estimable under $\widehat{ {\mathscr{M}}}$ if there exists an ${\bf{L}} \in {\mathbb R}^{t \times mn}$ such that ${\rm{E}}({\bf{L}}\overrightarrow{ {\bf{Y}}} - {\bf{T}}\overrightarrow{\Theta}) = {\bf{0}}$.

The definitions of the OLSE and the BLUE of ${\bf{K}}\Theta$ under (1.1) are given below.

Definition 1.2. Let ${\mathscr{M}}$ be as given in (1.1), and let ${\bf{K}} \in {\mathbb R}^{k \times p}$ be given.

(ⅰ) The OLSE of the parameter matrix $\Theta$ in ${\mathscr{M}}$, denoted by ${\rm{OLSE}}(\Theta)$, is defined to be

Correspondingly, the OLSE of ${\bf{K}}\Theta$ under (1.1) is defined to be ${\rm{OLSE}}({\bf{K}}\Theta) = {\bf{K}}{\rm{OLSE}}(\Theta).$

(ⅱ) If there exists an ${\bf{L}} \in {\mathbb R}^{k \times n}$ such that

holds in the Löwner partial ordering, the corresponding linear matrix statistic ${\bf{L}} {\bf{Y}}$ is defined to be the BLUE of ${\bf{K}}\Theta$ under ${\mathscr{M}}$, and is denoted by ${\bf{L}} {\bf{Y}} = {\rm{BLUE}}({\bf{K}}\Theta).$

Since the OLSEs and BLUEs are defined by different optimality criteria, they may have diverse expressions and mathematical and statistical properties. In fact, it has been known that the OLSEs and BLUEs of unknown parameter matrices in a given MGLM can be represented in certain analytical formulas that are composed of the given matrices and their generalized inverses in the models. Thus, the algebraic and statistical properties and performances of OLSEs and BLUEs can be easily derived from the analytical formulas. Because OLSEs are easy to compute and have many simple and nice properties in a linear regression framework, statisticians are interested in the connections between OLSEs and other estimators. Statisticians have noticed that OLSEs and BLUEs of the same unknown parameters in a linear statistical model have some essential links; in particular, the OLSEs and BLUEs of the same unknown parameters are equivalent under some rational conditions. Based on the known theory on OLSEs and BLUEs under linear statistical models, statisticians proposed and studied many inference problems from theoretical and applied points of view. Especially, they were interested in the connections between the two types of estimators under linear statistical models by means of some precise and effective matrix analysis tools. Recall that the problem of investigating twists of OLSEs and BLUEs in linear regression theory was initialized and approached in the late 1940s from theoretical and applied points of view, and many classic and novel contributions on this topic have been properly documented in the statistical literature since then; see, e.g., a recent survey paper ^[18] and the references therein.

As a new attempt to approach this kind of problems under more general model assumptions, we consider the connections between OLSEs and BLUEs under (1.1) in this current work. The study includes solving the following two problems on the relationships between the estimators of the whole and partial mean parameter matrices in (1.1):

(Ⅰ) Establish the necessary and sufficient conditions for the following equality

to hold, where ${\bf{K}}\Theta$ is an estimable matrix of parametric functions under ${\mathscr{M}}$ for ${\bf{K}} \in {\mathbb R}^{k \times p}$.

(Ⅱ) Prove the following equivalent equalities of OLSEs and BLUEs

under the assumption that ${\bf{X}}_i\Theta_i$ is estimable under ${\mathscr{M}}$, $i = 1, 2, \ldots, k$.

The equalities for estimators in (1.5) and (1.6) have many different possible interpretations from the mathematical and statistical points of view, and are not rare to see in the statistical inference of a given MGLM. In fact, there are many publications on establishing equalities between OLSEs and BLUEs under various linear statistical model assumptions. It is easy to convert these estimator equalities to certain matrix equalities that involve the given matrices and their generalized inverses in the models. Many influential and effective mathematical tools are available in order to characterize the above equalities of estimators and their covariance matrices under MGLMs, but we prefer to use the matrix rank method (for short, MRM) to characterize the equalities in (Ⅰ) and (Ⅱ). The MRM is now highly recognized as a useful tool to establish and characterize various simple or complicated algebraic equalities for matrices and their operations; see e.g., ^[35,36] and the references therein on the MRM in the investigations of the linear statistical models. In order to relate the work to existing results in machine learning, some important results can be found in ^{[7,8,15,30,39]}.

The rest of this paper is organized as follows: In Section 2, we introduce some matrix analysis tools that can be used to characterize matrix equalities that involve generalized inverses, and give a group of results related to the estimability of matrices of parametric functions under (1.1). We then show how to directly establish analytical expressions of the OLSEs and BLUEs under the assumptions in (1.1). In Section 3, we present several groups of classic and new equivalent statements for the OLSEs to be the BLUEs under (1.1) by characterizing various matrix equalities. Some conclusions and remarks are given in Section 4.

2.   Notation and preliminary results

Recall that matrix algebra is one of the most important areas of mathematics, while many matrix analysis tools play key roles in data science and statistical theory, including the field of multivariate analysis and inferences from regression regression frameworks; see, e.g., ^[5,26], among others. Specifically, the theory of generalized inverses of matrices, as ubiquitous tools to deal with singular matrices, has been widely utilized to approach various complicated theoretical and computational problems that occur in statistical analysis and inference; see, e.g., the reference books ^{[3,9,11,27,28,31]} concerning applications of matrix theory in statistics. Also recall that many problems in statistical analysis of regression models can be equivalently transformed into certain matrix analysis problems, so that people can use various algebraic tools in matrix theory to approach the statistical analysis problems and to obtain various exact and satisfactory results and facts from mathematical and statistical points of view. In the following, we introduce and explain some preliminary groundwork in linear algebra and matrix theory that will be utilized in the context of this paper. Let us first recall that block matrix and rank of a matrix are two basic concepts in mathematics that appear at the entry level of linear algebra and are easily understandable by a beginner. On the other hand, they have been taken as two irreplaceable study tools for dealing with various basic and advanced problems in theoretical and computational mathematics because they give us the ability to construct and analyze various simple and complicated matrix expressions and matrix equalities in a clear and concise way.

For the purpose of establishing and characterizing various possible equalities for estimations in the context of linear regression models, and simplifying various matrix equalities involving Moore–Penrose generalized inverses of matrices, we need to use some well-known facts on ranks and generalized inverses of matrices in the following two lemmas, and then proceed to give the proofs of the main results in this article.

Lemma 2.1 (^[19]). Let ${\bf{A}} \in {\mathbb R}^{m \times n},$ ${\bf{B}} \in {\mathbb R}^{m \times k},$ ${\bf{C}} \in {\mathbb R}^{l \times n},$ and ${\bf{D}} \in {\mathbb R}^{l\times k}.$ Then,

If ${\mathscr{R}}({\bf{B}}) \subseteq {\mathscr{R}}({\bf{A}})$ and ${\mathscr{R}}({\bf{C}}^{T}) \subseteq {\mathscr{R}}({\bf{A}}^{T}),$ then

In addition, the following results hold.

(ⅰ) $r[\, {\bf{A}}, \, {\bf{B}} \, ] = r({\bf{A}}) \Leftrightarrow {\mathscr{R}}({\bf{B}}) \subseteq {\mathscr{R}}({\bf{A}}) \Leftrightarrow {\bf{A}} {\bf{A}}^{+} {\bf{B}} = {\bf{B}} \Leftrightarrow {\bf{E}}_{ {\bf{A}}}{\bf{B}} = {\bf{0}}.$

(ⅱ) $r\!\begin{bmatrix} {\bf{A}} \\ {\bf{C}} \end{bmatrix} = r({\bf{A}}) \Leftrightarrow {\mathscr{R}}({\bf{C}}^{T}) \subseteq {\mathscr{R}}({\bf{A}}^{T}) \Leftrightarrow {\bf{C}} {\bf{A}}^{+} {\bf{A}} = {\bf{C}} \Leftrightarrow {\bf{C}} {\bf{F}}_{ {\bf{A}}}= {\bf{0}}.$

(ⅲ) $r[\, {\bf{A}}, \, {\bf{B}} \, ] = r({\bf{A}}) + r({\bf{B}}) \Leftrightarrow {\mathscr{R}}({\bf{A}}) \cap {\mathscr{R}}({\bf{B}}) = \{ {\bf{0}} \} \Leftrightarrow {\mathscr{R}}(({\bf{E}}_ {\bf{A}} {\bf{B}})^{T}) = {\mathscr{R}}({\bf{B}}^{T}) \Leftrightarrow {\mathscr{R}}(({\bf{E}}_ {\bf{B}} {\bf{A}})^{T}) = {\mathscr{R}}({\bf{A}}^{T}).$

(ⅳ) $r\!\begin{bmatrix} {\bf{A}} \\ {\bf{C}} \end{bmatrix} = r({\bf{A}}) + r({\bf{C}}) \Leftrightarrow {\mathscr{R}}({\bf{A}}^{T}) \cap {\mathscr{R}}({\bf{C}}^{T}) = \{ {\bf{0}} \} \Leftrightarrow {\mathscr{R}}({\bf{C}} {\bf{F}}_{ {\bf{A}}}) = {\mathscr{R}}({\bf{C}}) \Leftrightarrow {\mathscr{R}}({\bf{A}} {\bf{F}}_{ {\bf{C}}}) = {\mathscr{R}}({\bf{A}}).$

(ⅴ) $r(\, {\bf{A}} + {\bf{B}} \, ) = r({\bf{A}}) + r({\bf{B}}) \Leftrightarrow {\mathscr{R}}({\bf{A}}) \cap {\mathscr{R}}({\bf{B}}) = \{ {\bf{0}} \}$ and ${\mathscr{R}}({\bf{A}}^{T}) \cap {\mathscr{R}}({\bf{B}}^{T}) = \{ {\bf{0}} \}$ for ${\bf{A}}, \, {\bf{B}} \in {\mathbb R}^{m \times n}.$

Lemma 2.2 (^[24]). Given two matrices ${\bf{A}}$ and ${\bf{B}}$ of appropriate sizes, the matrix equation ${\bf{A}} {\bf{X}} = {\bf{B}}$ is solvable for ${\bf{X}}$ if and only if $r[ {\bf{A}}, \, {\bf{B}}\, ] = r({\bf{A}}),$ or equivalently, ${\bf{A}} {\bf{A}}^{+} {\bf{B}} = {\bf{B}}.$ In this case, the general solution of the equation is ${\bf{X}} = {\bf{A}}^{+} {\bf{B}} + {\bf{F}}_{ {\bf{A}}}{\bf{U}},$ where ${\bf{U}}$ is an arbitrary matrix.

We also use the following basic facts about the Kronecker products of matrices.

Lemma 2.3. Let ${\bf{A}} \in {\mathbb R}^{m \times n}$ and ${\bf{B}} \in {\mathbb R}^{p \times q},$ ${\bf{C}} \in {\mathbb R}^{n \times s},$ and ${\bf{D}} \in {\mathbb R}^{q \times t}.$ Then, $({\bf{A}} \!\otimes\! {\bf{B}})({\bf{C}} \!\otimes\! {\bf{D}}) = {\bf{A}} {\bf{C}} \otimes {\bf{B}} {\bf{D}}$. In particular, ${\bf{A}}\otimes {\bf{B}} = {\bf{0}}$ if and only if ${\bf{A}} = {\bf{0}}$ or ${\bf{B}} = {\bf{0}}.$

For convenience of representation, we denote

In this case, the model matrix ${\bf{X}}$ in (1.1) and ${\bf{I}}_m \otimes {\bf{X}}$ in (1.2) can be decomposed as

Correspondingly, the partial mean parameter matrices ${\bf{X}}_i\Theta_i$ and the partial mean parameter vectors $({\bf{I}}_m \otimes {\bf{X}}_i) \overrightarrow{\Theta}_i$ on the right-hand sides of (1.1) and (1.2) can be rewritten as

In the following, we derive some necessary and sufficient conditions for ${\bf{K}}\Theta$ to be estimable under (1.1).

Theorem 2.4. Let ${\mathscr{M}}$ and $\widehat{ {\mathscr{M}}}$ be as given in (1.1) and (1.2), respectively, and let ${\bf{K}} \in {\mathbb R}^{k \times p}$ be given. Then, the following statements are equivalent:

(ⅰ) ${\bf{K}}\Theta$ is estimable under ${\mathscr{M}}.$

(ⅱ) $({\bf{I}}_m \otimes {\bf{K}})\overrightarrow{\Theta}$ is estimable under $\widehat{ {\mathscr{M}}}.$

(ⅲ) ${\mathscr{R}}({\bf{I}}_m \otimes {\bf{K}}^{T}) \subseteq {\mathscr{R}}(\, {\bf{I}}_m \otimes {\bf{X}}^{T}\, ).$

(ⅳ) ${\mathscr{R}}({\bf{K}}^{T}) \subseteq {\mathscr{R}}({\bf{X}}^{T}\, ).$

(ⅴ) ${\bf{K}} {\bf{X}}^{+} {\bf{X}} = {\bf{K}}.$

Proof. By Definition 1.1 (ⅰ), we see that

Further, by Lemma 2.2, the matrix equation on the right-hand side of (2.8) is solvable for ${\bf{L}}$ if and only if (ⅳ) holds. The equivalence of (ⅳ) and (ⅴ) follows from Lemma 2.1 (ⅱ). Also, by Definition 1.1 (ⅱ),

and by Lemma 2.2, the equation on the right-hand side of (2.9) is solvable for ${\bf{L}}$ if and only if (ⅲ) holds. Consequently, (ⅲ) holds if and only if $r\begin{bmatrix} {\bf{I}}_m \otimes {\bf{X}} \\ {\bf{I}}_m \otimes {\bf{K}}\end{bmatrix} = r({\bf{I}}_m \otimes {\bf{X}}).$ Expanding both sides of the equality, we obtain

This fact leads to the equivalence of (ⅲ) and (ⅳ). □

Concerning the estimability of ${\bf{X}}_i\Theta_i$ in (1.1), $i = 1, 2, \ldots, k,$ we have the following facts.

Theorem 2.5. Let ${\mathscr{M}}$ and $\widehat{ {\mathscr{M}}}$ be as given in (1.1) and (1.2), respectively. Then, the following five statements are equivalent:

(ⅰ) ${\bf{X}}_i\Theta_i = {\bf{V}}_i\Theta$ is estimable in ${\mathscr{M}},$ $i = 1, 2, \ldots, k.$

(ⅱ) $({\bf{I}}_m \otimes {\bf{V}}_i)\overrightarrow{\Theta}$ is estimable under $\widehat{ {\mathscr{M}}},$ $i = 1, 2, \ldots, k.$

(ⅲ) ${\mathscr{R}}({\bf{V}}_i^{T})\subseteq {\mathscr{R}}({\bf{X}}^{T}),$ $i = 1, 2, \ldots, k.$

(ⅳ) ${\mathscr{R}}({\bf{V}}_i) \cap {\mathscr{R}}({\bf{W}}_i) = {\mathscr{R}}({\bf{X}}_i) \cap {\mathscr{R}}({\bf{W}}_i) = \{ {\bf{0}}\},$ $i = 1, 2, \ldots, k.$

(ⅴ) $r({\bf{X}}) = r({\bf{V}}_i) + r({\bf{W}}_i) = r({\bf{X}}_i) + r({\bf{W}}_i),$ $i = 1, 2, \ldots, k.$

Proof. Setting ${\bf{K}} = {\bf{V}}_i$ in Theorem 2.4, we obtain $({\bf{I}}_m \otimes {\bf{K}})\overrightarrow{\Theta} = ({\bf{I}}_m \otimes {\bf{V}}_i)\overrightarrow{\Theta} = ({\bf{I}}_m \otimes {\bf{X}}_i)\overrightarrow{\Theta}_i$. In this case, applying Theorem 2.4 to it and simplifying yield the equivalences of (ⅰ)–(ⅲ). Further, (ⅲ) is equivalent to $r[ {\bf{V}}_i^{T}, \, {\bf{X}}^{T}] = r({\bf{X}})$, where $r[ {\bf{V}}_i^{T}, \, {\bf{X}}^{T}] = r[ {\bf{V}}_i^{T}, \, {\bf{W}}_i^{T}] = r({\bf{V}}_i) + r({\bf{W}}_i) = r({\bf{X}}_i) + r({\bf{W}}_i)$ by (2.4) for $i = 1, 2, \ldots, k$, thus establishing the equivalences of (ⅲ)–(ⅴ). □

Theorem 2.6. Let ${\mathscr{M}}$ be as given in (1.1). Then, the following two statements are equivalent:

(ⅰ) All ${\bf{X}}_1\Theta_1, \ldots, {\bf{X}}_k\Theta_k$ are estimable in ${\mathscr{M}}.$

(ⅱ) $r({\bf{X}}) = r({\bf{X}}_1) + \cdots + r({\bf{X}}_k).$

Proof. From (2.5),

If (ⅱ) holds, we see from (2.10) that

Hence, (i) holds by Theorem 2.5. The equivalence of (ⅱ) and (2.11) can be established by induction. □

Theorem 2.5 (ⅳ) and (ⅴ) and Theorem 2.6 (ⅱ) can be easily verified for the given model matrix in ${\mathscr{M}}$. In particular, they are all satisfied under the condition $r({\bf{X}}) = p$.

Below, we present a new derivation for the analytical formula of the OLSE of $\Theta$ in (1.1).

Theorem 2.7. Let ${\mathscr{M}}$ be as given in (1.1), and suppose that ${\bf{K}}\Theta$ is estimable under ${\mathscr{M}}$ for ${\bf{K}} \in {\mathbb R}^{k \times p}.$ Then, the OLSE of ${\bf{K}}\Theta$ under ${\mathscr{M}}$ can be uniquely expressed as

with

In particular, the following results hold.

(ⅰ) ${\bf{X}}\Theta$ in ${\mathscr{M}}$ is always estimable, and

(ⅱ) If ${\bf{X}}_i\Theta_i$ is estimable under ${\mathscr{M}},$ then,

(ⅲ) If all ${\bf{X}}_1\Theta_1, \ldots, {\bf{X}}_k\Theta_k$ are estimable under ${\mathscr{M}},$ then,

Proof. We first decompose the matrix product $(\, {\bf{Y}} - {\bf{X}}\Theta\, )^{T}(\, {\bf{Y}} - {\bf{X}}\Theta\, )$ as

Thus,

Minimizing both sides of equality with respect to $\Theta$, we obtain the following formula

It is obvious that ${\bf{X}}\Theta = {\bf{P}}_{ {\bf{X}}} {\bf{Y}}$ is always solvable for $\Theta$, and the general solution is given by $\Theta = {\bf{X}}^{+} {\bf{Y}} + ({\bf{I}}_p - {\bf{X}}^{+} {\bf{X}}) {\bf{U}}$ by Lemma 2.2. In this case, we obtain the following three fundamental formulas

where ${\bf{U}} \in {\mathbb R}^{p \times m}$ is arbitrary. The expectation and covariance matrix of ${\rm{OLSE}}({\bf{K}}\Theta)$ are

by Lemma 2.3. Consequently, we are able to obtain (ⅰ)–(ⅲ) from (2.12) and (2.13). □

With regard to the exact expression of the BLUE of ${\bf{K}}\Theta$, the following general results were established in ^[12,13,37] through the ordinary employment of the analytical solutions of a constrained quadratic matrix-valued optimization problem in ^[34].

Theorem 2.8. Let ${\mathscr{M}}$ be as given in (1.1), and suppose that ${\bf{K}}\Theta$ is estimable under ${\mathscr{M}}$ for ${\bf{K}} \in {\mathbb R}^{k \times p}.$ Then,

The matrix equation in (2.19), called the BLUE equation, is solvable for ${\bf{L}},$ namely,

holds. The general solution of ${\bf{L}}$ and the corresponding BLUE of ${\bf{K}}\Theta$ under ${\mathscr{M}}$ can be written as

where ${\bf{U}} \in {\mathbb R}^{k \times n}$ is arbitrary. The expectation and covariance matrix of ${\rm{BLUE}}({\bf{K}}\Theta)$ are

where

Furthermore, the following statements hold:

(ⅰ) ${\bf{X}}\Theta$ in ${\mathscr{M}}$ is always estimable, and

where ${\bf{U}} \in {\mathbb R}^{n \times n}$ is arbitrary.

(ⅱ) If ${\bf{X}}_i\Theta_i$ in ${\mathscr{M}}$ is estimable, then,

where ${\bf{U}}_i \in {\mathbb R}^{n \times n}$ is arbitrary, $i = 1, 2, \ldots, k.$

(ⅲ) If all ${\bf{X}}_i\Theta_i$ in (1.1) are estimable, then,

3.   Twists of OLSEs and BLUEs under an MGLM

In this section, we solve the estimation equality problems outlined in (1.5) and (1.6), and provide a variety of algebraic and statistical descriptions of the proposed equivalent facts. For the purpose of characterizing equalities between ${\bf{L}}_1 {\bf{Y}}$ and ${\bf{L}}_2 {\bf{Y}}$, we need to use the following three manifest criteria for comparison and contrast of linear statistics:

Definition 3.1. Let ${\bf{Y}}$ be as given in (1.1), and let ${\bf{L}}_1, \, {\bf{L}}_2 \in {\mathbb R}^{k \times n}.$

(ⅰ) The equality ${\bf{L}}_1 {\bf{Y}} = {\bf{L}}_2 {\bf{Y}}$ is said to hold definitely if ${\bf{L}}_1 = {\bf{L}}_2$.

(ⅱ) The equality ${\bf{L}}_1 {\bf{Y}} = {\bf{L}}_2 {\bf{Y}}$ is said to hold with probability $1$ if both ${\rm{E}}({\bf{L}}_1 {\bf{Y}} - {\bf{L}}_2 {\bf{Y}}) = {\bf{0}}$, and ${\rm{Cov}}(({\bf{I}}_m \otimes {\bf{L}}_1)\overrightarrow{ {\bf{Y}}}-({\bf{I}}_m \otimes {\bf{L}}_2)\overrightarrow{ {\bf{Y}}}) = {\bf{0}}$.

(ⅲ) ${\bf{L}}_1 {\bf{Y}}$ and ${\bf{L}}_2 {\bf{Y}}$ are said to have the same expectation matrices and dispersion matrices if both ${\rm{E}}({\bf{L}}_1 {\bf{Y}}) = {\rm{E}}({\bf{L}}_2 {\bf{Y}})$, and ${\rm{Cov}}(({\bf{I}}_m \otimes {\bf{L}}_1)\overrightarrow{ {\bf{Y}}}) = {\rm{Cov}}(({\bf{I}}_m \otimes {\bf{L}}_2)\overrightarrow{ {\bf{Y}}})$ hold.

Our main results in the paper are presented below.

Theorem 3.2. Let ${\mathscr{M}}$ be as given in (1.1), and suppose that ${\bf{K}}\Theta$ is estimable under ${\mathscr{M}}$ for ${\bf{K}} \in {\mathbb R}^{k \times p}.$ Also, let ${\rm{OLSE}}({\bf{K}}\Theta)$ and ${\rm{BLUE}}({\bf{K}}\Theta)$ be as given in (2.12) and (2.21), respectively. Then, the following 16 statements are equivalent:

(ⅰ) ${\rm{OLSE}}({\bf{K}}\Theta) = {\rm{BLUE}}({\bf{K}}\Theta)$ holds definitely.

(ⅱ) ${\rm{OLSE}}({\bf{K}}\Theta) = {\rm{BLUE}}({\bf{K}}\Theta)$ holds with probability $1.$

(ⅲ) ${\rm{Cov}}(\overrightarrow{{\rm{OLSE}}({\bf{K}}\Theta)}) = {\rm{Cov}}(\overrightarrow{{\rm{BLUE}}({\bf{K}}\Theta)}).$

(ⅳ) ${\rm{Cov}}\{\overrightarrow{{\rm{OLSE}}({\bf{K}}\Theta)}, \, \overrightarrow{ {\bf{Y}}}\} = {\rm{Cov}}\{\overrightarrow{{\rm{BLUE}}({\bf{K}}\Theta)}, \, \overrightarrow{ {\bf{Y}}}\}.$

(ⅴ) ${\rm{Cov}}\{\overrightarrow{{\rm{OLSE}}({\bf{K}}\Theta)}, \, \overrightarrow{ {\bf{Y}}}\} = {\rm{Cov}}\{\overrightarrow{{\rm{BLUE}}({\bf{K}}\Theta)}, \, \overrightarrow{{\rm{OLSE}}({\bf{X}}\Theta)}\}.$

(ⅵ) ${\rm{Cov}}\{\overrightarrow{{\rm{OLSE}}({\bf{K}}\Theta)}, \, \overrightarrow{ {\bf{Y}}} - \overrightarrow{{\rm{OLSE}}({\bf{X}}\Theta)}\} = {\bf{0}}.$

(ⅶ) ${\bf{K}} {\bf{X}}^{+} = [\, {\bf{K}}, \, {\bf{0}}\, ][\, {\bf{X}}, \, \pmb{\Sigma}_1 {\bf{X}}^{\perp} \, ]^{+}.$

(ⅷ) ${\bf{K}} {\bf{X}}^{+} \pmb{\Sigma}_1 = [\, {\bf{K}}, \, {\bf{0}}\, ][\, {\bf{X}}, \, \pmb{\Sigma}_1 {\bf{X}}^{\perp} \, ]^{+} \pmb{\Sigma}_1.$

(ⅸ) ${\bf{K}} {\bf{X}}^{+} \pmb{\Sigma}_1 = [\, {\bf{K}}, \, {\bf{0}}\, ][\, {\bf{X}}, \, \pmb{\Sigma}_1 {\bf{X}}^{\perp}\, ]^{+} \pmb{\Sigma}_1 {\bf{X}} {\bf{X}}^{+}.$

(ⅹ) ${\bf{K}} {\bf{X}}^{+} \pmb{\Sigma}_1({\bf{K}} {\bf{X}}^{+})^{T} = [\, {\bf{K}}, \, {\bf{0}}\, ][\, {\bf{X}}, \, \pmb{\Sigma}_1 {\bf{X}}^{\perp}\, ]^{+} \pmb{\Sigma}_1([\, {\bf{K}}, \, {\bf{0}}\, ] [\, {\bf{X}}, \, \pmb{\Sigma}_1 {\bf{X}}^{\perp} \, ]^{+})^{T}.$

(xi) ${\bf{K}} {\bf{X}}^{+} \pmb{\Sigma}_1 {\bf{X}}^{\perp} = {\bf{0}}.$

(xii) ${\bf{X}}^{\perp}[\, \pmb{\Sigma}_1 {\bf{X}}, \, {\bf{0}} \, ] {\bf{F}}_{[\, {\bf{X}}^{T} {\bf{X}}, \, {\bf{K}}^{T}\, ]} = {\bf{0}}.$

(xiii) $r\!\begin{bmatrix} \pmb{\Sigma}_1 {\bf{X}} & {\bf{X}} & {\bf{0}} \\ {\bf{X}}^{T} {\bf{X}} & {\bf{0}} & {\bf{K}}^{T}\end{bmatrix} = r\!\begin{bmatrix} \pmb{\Sigma}_1 {\bf{X}} & {\bf{X}} \\ {\bf{X}}^{T} {\bf{X}} & {\bf{0}}\end{bmatrix} = 2r({\bf{X}}).$

(xiv) ${\mathscr{R}}(({\bf{K}} {\bf{X}}^{+} \pmb{\Sigma}_1)^{T}) \subseteq {\mathscr{R}}({\bf{X}}).$

(xv) ${\mathscr{R}}\!\begin{bmatrix} {\bf{X}}^{T} \pmb{\Sigma}_1 {\bf{X}}^{\perp} \\ {\bf{0}}\end{bmatrix} \subseteq {\mathscr{R}}\!\begin{bmatrix} {\bf{X}}^{T} {\bf{X}} \\ {\bf{K}}\end{bmatrix}\!.$

(xvi) ${\mathscr{R}}\!\begin{bmatrix} {\bf{0}} \\ {\bf{K}}^{T}\end{bmatrix} \subseteq {\mathscr{R}}\!\begin{bmatrix} \pmb{\Sigma}_1 {\bf{X}} & {\bf{X}} \\ {\bf{X}}^{T} {\bf{X}} & {\bf{0}} \end{bmatrix}\!.$

Proof. The equivalences of the matrix equalities and relations in (vii)–(xvi) can be proved by the algebraic tools presented in Lemma 2.1, while the details of the proofs can be found in ^[38], see also ^[16,33].

By Definition 3.1 (ⅰ), the equality ${\rm{OLSE}}({\bf{K}}\Theta) = {\rm{BLUE}}({\bf{K}}\Theta)$ in (ⅰ) holds definitely if and only if the coefficient matrices of ${\bf{Y}}$ in (2.12) and (2.21) are equal, i.e.,

By Lemma 2.2, there exists a matrix ${\bf{U}}$ such that (3.1) holds if and only if

We next simplify both sides of this rank equality. By (2.1),

Substituting (3.3)–(3.5) into (3.2) and simplifying lead to the equivalence of (ⅰ) and (xiii).

Because ${\rm{E}}({\rm{OLSE}}({\bf{K}}\Theta) - {\rm{BLUE}}({\bf{K}}\Theta)) = {\bf{0}}$, we see from Definition 3.1 (ⅱ) that ${\rm{OLSE}}({\bf{K}}\Theta) = {\rm{BLUE}}({\bf{K}}\Theta)$ holds with probability 1 if and only if

Since $\pmb{\Sigma}_2 \otimes \pmb{\Sigma}_1$ is nonnegative definite and $\pmb{\Sigma}_2 \neq {\bf{0}}$, (3.6) is equivalent to

by Lemma 2.3, thus establishing the equivalence of (ⅱ) and (viii).

It follows from (2.13) and (2.23) that (ⅲ) is equivalent to

which is further equivalent to

by Lemma 2.3, thus establishing the equivalence of (ⅲ) and (ⅹ).

By (2.12) and (2.21),

Comparing the right-hand sides of (3.7) and (3.8) leads to the equivalence of (ⅳ) and (ⅷ).

By (2.14) and (2.21),

Comparing the right-hand sides of (3.7) and (3.9) leads to the equivalence of (ⅴ) and (ⅸ).

By (2.12), (2.14), and (3.7),

Setting all sides of the equalities equal of the to zero yields the equivalence of (vi) and (xi) by Lemma 2.3. □

More results and facts associated with the equalities of OLSEs and BLUEs can be established from algebraic and statistical considerations. In particular, the matrix equalities in Theorem 3.2 (ⅷ)–(xi) correspond directly to the estimation equalities and the covariance matrix equalities. In other words, they have clear statistical interpretations and can be utilized in the corresponding statistical inference.

Let ${\bf{K}} = {\bf{X}}$ in Theorem 3.2. We obtain the following results:

Corollary 3.3. Let ${\rm{OLSE}}({\bf{X}}\Theta)$ and ${\rm{BLUE}}({\bf{X}}\Theta)$ be as given in (2.14) and (2.25), respectively. Then, the following 31 statements are equivalent:

(ⅰ) ${\rm{OLSE}}({\bf{X}}\Theta) = {\rm{BLUE}}({\bf{X}}\Theta)$ holds definitely $($with probability $1).$

(ⅱ) ${\rm{Cov}}(\overrightarrow{{\rm{OLSE}}({\bf{X}}\Theta)}) = {\rm{Cov}}(\overrightarrow{{\rm{BLUE}}({\bf{X}}\Theta)}).$

(ⅲ) ${\rm{Cov}}(\overrightarrow{ {\bf{Y}} }- \overrightarrow{{\rm{OLSE}}({\bf{X}}\Theta)}) = {\rm{Cov}}(\overrightarrow{ {\bf{Y}} }- \overrightarrow{{\rm{BLUE}}({\bf{X}}\Theta)}).$

(ⅳ) ${\rm{Cov}}(\overrightarrow{{\rm{OLSE}}({\bf{X}}\Theta)}) = {\rm{Cov}}\{\overrightarrow{{\rm{BLUE}}({\bf{X}}\Theta)}, \, \overrightarrow{ {\bf{Y}}}\}.$

(ⅴ) ${\rm{Cov}}(\overrightarrow{{\rm{OLSE}}({\bf{X}}\Theta)}) = {\rm{Cov}}\{\overrightarrow{{\rm{BLUE}}({\bf{X}}\Theta)}, \, \overrightarrow{{\rm{OLSE}}({\bf{X}}\Theta)}\}.$

(ⅵ) ${\rm{Cov}}\{\overrightarrow{{\rm{OLSE}}({\bf{X}}\Theta)}, \, \overrightarrow{ {\bf{Y}}}\} = {\rm{Cov}}\{\overrightarrow{{\rm{BLUE}}({\bf{X}}\Theta)}, \, \overrightarrow{ {\bf{Y}}}\}.$

(ⅶ) ${\rm{Cov}}\{\overrightarrow{{\rm{OLSE}}({\bf{X}}\Theta)}, \, \overrightarrow{ {\bf{Y}}}\} = {\rm{Cov}}\{\overrightarrow{{\rm{BLUE}}({\bf{X}}\Theta)}, \, \overrightarrow{{\rm{OLSE}}({\bf{X}}\Theta)}\}.$

(ⅷ) ${\rm{Cov}}\{\overrightarrow{{\rm{OLSE}}({\bf{X}}\Theta)}, \, \overrightarrow{ {\bf{Y}}}\} = {\rm{Cov}}\{\overrightarrow{ {\bf{Y}}}, \, \overrightarrow{{\rm{OLSE}}({\bf{X}}\Theta)}\}.$

(ⅸ) ${\rm{Cov}}\{\overrightarrow{ {\bf{Y}}} - \overrightarrow{{\rm{OLSE}}({\bf{X}}\Theta)}, \, \overrightarrow{ {\bf{Y}}}\} = {\rm{Cov}}\{\overrightarrow{ {\bf{Y}}}, \, \overrightarrow{ {\bf{Y}}} - \overrightarrow{{\rm{OLSE}}({\bf{X}}\Theta)}\}.$

(ⅹ) ${\rm{Cov}}\{\overrightarrow{ {\bf{Y}}} - \overrightarrow{{\rm{OLSE}}({\bf{X}}\Theta)}, \, \overrightarrow{{\rm{OLSE}}({\bf{X}}\Theta)}\} = {\rm{Cov}}\{\overrightarrow{{\rm{OLSE}}({\bf{X}}\Theta)}, \, \overrightarrow{ {\bf{Y}}} - \overrightarrow{{\rm{OLSE}}({\bf{X}}\Theta)}\} = {\bf{0}}.$

(xi) ${\rm{Cov}}\{\overrightarrow{ {\bf{Y}}} - \overrightarrow{{\rm{OLSE}}({\bf{X}}\Theta)}, \, \overrightarrow{{\rm{OLSE}}({\bf{X}}\Theta)}\} + {\rm{Cov}}\{\overrightarrow{{\rm{OLSE}}({\bf{X}}\Theta)}, \, \overrightarrow{ {\bf{Y}}} - \overrightarrow{{\rm{OLSE}}({\bf{X}}\Theta)}\} = {\bf{0}}.$

(xii) ${\rm{Cov}}\{\overrightarrow{ {\bf{Y}}} - \overrightarrow{{\rm{OLSE}}({\bf{X}}\Theta)}, \, \overrightarrow{{\rm{OLSE}}({\bf{X}}\Theta)}\} = {\bf{0}}.$

(xiii) ${\rm{Cov}}(\overrightarrow{ {\bf{Y}}}) = {\rm{Cov}}(\overrightarrow{{\rm{OLSE}}({\bf{X}}\Theta)}) + {\rm{Cov}}(\overrightarrow{ {\bf{Y}}} - \overrightarrow{{\rm{OLSE}}({\bf{X}}\Theta)}).$

(xiv) ${\bf{P}}_{ {\bf{X}}} = [\, {\bf{X}}, \, {\bf{0}}\, ][\, {\bf{X}}, \, \pmb{\Sigma}_1 {\bf{X}}^{\perp} \, ]^{+}.$

(xv) ${\bf{P}}_{ {\bf{X}}} \pmb{\Sigma}_1 = [\, {\bf{X}}, \, {\bf{0}}\, ][\, {\bf{X}}, \, \pmb{\Sigma}_1 {\bf{X}}^{\perp} \, ]^{+} \pmb{\Sigma}_1.$

(xvi) ${\bf{P}}_{ {\bf{X}}} \pmb{\Sigma}_1 = [\, {\bf{X}}, \, {\bf{0}}\, ][\, {\bf{X}}, \, \pmb{\Sigma}_1 {\bf{X}}^{\perp} \, ]^{+} \pmb{\Sigma}_1 {\bf{P}}_{ {\bf{X}}}.$

(xvii) ${\bf{P}}_{ {\bf{X}}} \pmb{\Sigma}_1 {\bf{P}}_{ {\bf{X}}} = [\, {\bf{X}}, \, {\bf{0}}\, ][\, {\bf{X}}, \, \pmb{\Sigma}_1 {\bf{X}}^{\perp} \, ]^{+} \pmb{\Sigma}_1 ([\, {\bf{X}}, \, {\bf{0}}\, ][\, {\bf{X}}, \, \pmb{\Sigma}_1 {\bf{X}}^{\perp} \, ]^{+})^{T}.$

(xviii) ${\bf{X}}^{\perp} \pmb{\Sigma}_1 {\bf{X}}^{\perp} = (\, {\bf{I}}_n - [\, {\bf{X}}, \, {\bf{0}}\, ][\, {\bf{X}}, \, \pmb{\Sigma}_1 {\bf{X}}^{\perp} \, ]^{+}\, ) \pmb{\Sigma}_1 (\, {\bf{I}}_n -[\, {\bf{X}}, \, {\bf{0}}\, ][\, {\bf{X}}, \, \pmb{\Sigma}_1 {\bf{X}}^{\perp} \, ]^{+}\, )^{T}.$

(xix) ${\bf{P}}_{ {\bf{X}}} \pmb{\Sigma}_1 {\bf{P}}_{ {\bf{X}}} = [\, {\bf{X}}, \, {\bf{0}}\, ][\, {\bf{X}}, \, \pmb{\Sigma}_1 {\bf{X}}^{\perp} \, ]^{+} \pmb{\Sigma}_1.$

(xx) ${\bf{P}}_{ {\bf{X}}} \pmb{\Sigma}_1 {\bf{P}}_{ {\bf{X}}} = [\, {\bf{X}}, \, {\bf{0}}\, ][\, {\bf{X}}, \, \pmb{\Sigma}_1 {\bf{X}}^{\perp} \, ]^{+} \pmb{\Sigma}_1 {\bf{P}}_{ {\bf{X}}}.$

(xxi) ${\bf{P}}_{ {\bf{X}}} \pmb{\Sigma}_1 {\bf{X}}^{\!\perp} = {\bf{X}}^{\!\perp} \pmb{\Sigma}_1 {\bf{P}}_{ {\bf{X}}} = {\bf{0}}.$

(xxii) ${\bf{P}}_{ {\bf{X}}} \pmb{\Sigma}_1 {\bf{X}}^{\!\perp} + {\bf{X}}^{\!\perp} \pmb{\Sigma}_1 {\bf{P}}_{ {\bf{X}}} = {\bf{0}}.$

(xxiii) ${\bf{X}}^{\perp} \pmb{\Sigma}_1 {\bf{X}} = {\bf{0}}.$

(xxiv) ${\bf{P}}_{ {\bf{X}}} \pmb{\Sigma}_1 = \pmb{\Sigma}_1 {\bf{P}}_{ {\bf{X}}}.$

(xxv) ${\bf{X}}^{\perp} \pmb{\Sigma}_1 = \pmb{\Sigma}_1 {\bf{X}}^{\perp}.$

(xxvi) $r[\, {\bf{X}}, \, \pmb{\Sigma}_1 {\bf{X}}\, ] = r({\bf{X}}).$

(xxvii) $r[\, {\bf{X}}^{\perp}, \, \pmb{\Sigma}_1 {\bf{X}}^{\perp}\, ] = r({\bf{X}}^{\perp}).$

(xxviii) ${\mathscr{R}}(\pmb{\Sigma}_1 {\bf{X}}) \subseteq {\mathscr{R}}({\bf{X}}).$

(xxix) ${\mathscr{R}}(\pmb{\Sigma}_1 {\bf{X}}^{\perp}) \subseteq {\mathscr{R}}({\bf{X}}^{\perp}).$

(xxx) ${\mathscr{R}}(\pmb{\Sigma}_1 {\bf{X}}) = {\mathscr{R}}(\pmb{\Sigma}_1)\cap {\mathscr{R}}({\bf{X}}).$

(xxxi) ${\mathscr{R}}(\pmb{\Sigma}_1 {\bf{X}}^{\perp}) = {\mathscr{R}}(\pmb{\Sigma}_1)\cap {\mathscr{R}}({\bf{X}}^{\perp}).$

Proof. The equivalences of (ⅰ), (ⅱ), (ⅵ), (ⅶ), (xii), (xiv)–(xvii), and (xxi) follow from Theorem 3.2 (ⅰ)–(xi) via setting ${\bf{K}} = {\bf{X}}$. The equivalences of the matrix equalities and relations in (xviii)–(xx), (xxiii), (xxiv), (xxvi), and (xxviii) were collected and proved in ^[38].

From (2.14) and (2.25),

Comparing the right-hand sides of (3.11) and (3.12) leads to the equivalence of (ⅲ) and (xviii).

From (2.15) and (3.8),

Setting both sides equal to zero yields the equivalence of (ⅳ) and (xix).

From (2.15) and (3.9),

Thus, setting both sides equal to zero yields the equivalence of (ⅴ) and (xx).

From (3.7),

Setting both sides equal to zero yields the equivalence of (ⅷ) and (xxiv).

From (1.1) and (3.7),

Setting both sides equal to zero yields the equivalence of (ⅸ) and (xxv).

From (xii), we obtain the equivalence of (ⅹ) and (xxi).

From (1.1), (2.15), (3.10), and (3.11),

and

where by (2.1) and Lemma 2.1(ⅴ),

Setting both sides of (3.19) equal to zero and combining it with (3.17) and (3.18) yields the equivalences of (xi), (xiii), (xxii), (xxiii), (xxvi), and (xxviii).

The equivalences of (xxiii)–(xxxi) on matrix equalities and range equalities are well known ^[25]. □

Observing from (2.7) that ${\bf{X}}_i\Theta_i = {\bf{V}}_i\Theta$, and letting ${\bf{K}} = {\bf{V}}_i$ in Theorem 3.2, we obtain the following results:

Corollary 3.4. Suppose that the partial mean matrix ${\bf{X}}_i\Theta_i$ is estimable under (1.1), and let ${\rm{OLSE}}({\bf{X}}_i\Theta_i)$ and ${\rm{BLUE}}({\bf{X}}_i\Theta_i)$ be as given in (2.16) and (2.28), respectively, $i = 1, \ldots, k.$ Then, the following 18 statistical and algebraic statements are equivalent:

(ⅰ) ${\rm{OLSE}}({\bf{X}}_i\Theta_i) = {\rm{BLUE}}({\bf{X}}_i\Theta_i)$ holds definitely $($with probability $1),$ $i = 1, 2, \ldots, k.$

(ⅱ) ${\rm{Cov}}(\overrightarrow{{\rm{OLSE}}({\bf{X}}_i\Theta_i)}) = {\rm{Cov}}(\overrightarrow{{\rm{BLUE}}({\bf{X}}_i\Theta_i)}),$ $i = 1, 2, \ldots, k.$

(ⅲ) ${\rm{Cov}}\{\overrightarrow{{\rm{OLSE}}({\bf{X}}_i\Theta_i)}, \ \overrightarrow{ {\bf{Y}}}\} = {\rm{Cov}}\{\overrightarrow{{\rm{BLUE}}({\bf{X}}_i\Theta_i)}, \ \overrightarrow{ {\bf{Y}}}\},$ $i = 1, 2, \ldots, k.$

(ⅳ) ${\rm{Cov}}\{\overrightarrow{{\rm{OLSE}}({\bf{X}}_i\Theta_i)}, \ \overrightarrow{ {\bf{Y}}}\} = {\rm{Cov}}\{\overrightarrow{{\rm{BLUE}}({\bf{X}}_i\Theta_i)}, \ \overrightarrow{{\rm{OLSE}}({\bf{X}}\Theta)}\},$ $i = 1, 2, \ldots, k.$

(ⅴ) ${\rm{Cov}}\{\overrightarrow{{\rm{OLSE}}({\bf{X}}_i\Theta_i)}, \ \overrightarrow{ {\bf{Y}}} - \overrightarrow{{\rm{OLSE}}({\bf{X}}\Theta)}\} = {\bf{0}},$ $i = 1, 2, \ldots, k.$

(ⅵ) ${\bf{V}}_i {\bf{X}}^{+} = [\, {\bf{V}}_i, \, {\bf{0}}\, ][\, {\bf{X}}, \, \pmb{\Sigma}_1 {\bf{X}}^{\perp} \, ]^{+},$ $i = 1, 2, \ldots, k.$

(ⅶ) ${\bf{V}}_i {\bf{X}}^{+} \pmb{\Sigma}_1 = [\, {\bf{V}}_i, \, {\bf{0}}\, ][\, {\bf{X}}, \, \pmb{\Sigma}_1 {\bf{X}}^{\perp} \, ]^{+} \pmb{\Sigma}_1,$ $i = 1, 2, \ldots, k.$

(ⅷ) ${\bf{V}}_i {\bf{X}}^{+} \pmb{\Sigma}_1 = [\, {\bf{V}}_i, \, {\bf{0}}\, ][\, {\bf{X}}, \, \pmb{\Sigma}_1 {\bf{X}}^{\perp} \, ]^{+} \pmb{\Sigma}_1 {\bf{P}}_{ {\bf{X}}},$ $i = 1, 2, \ldots, k.$

(ⅸ) ${\bf{V}}_i {\bf{X}}^{+} \pmb{\Sigma}_1({\bf{V}}_i {\bf{X}}^{+})^{T} = [\, {\bf{V}}_i, \, {\bf{0}}\, ][\, {\bf{X}}, \, \pmb{\Sigma}_1 {\bf{X}}^{\perp} \, ]^{+} \pmb{\Sigma}_1 ([\, {\bf{V}}_i, \, {\bf{0}}\, ][\, {\bf{X}}, \, \pmb{\Sigma}_1 {\bf{X}}^{\perp} \, ]^{+})^{T},$ $i = 1, 2, \ldots, k.$

(ⅹ) ${\bf{V}}_i {\bf{X}}^{+} \pmb{\Sigma}_1 {\bf{X}}^{\perp} = {\bf{0}},$ $i = 1, 2, \ldots, k.$

(xi) ${\bf{X}}^{\perp}[\, \pmb{\Sigma}_1 {\bf{X}}, \, {\bf{0}} \, ] {\bf{F}}_{[\, {\bf{X}}^{T} {\bf{X}}, \, {\bf{V}}_i^{T}\, ]} = {\bf{0}},$ $i = 1, 2, \ldots, k.$

(xii) ${\bf{X}}^{\perp} \pmb{\Sigma}_1 {\bf{W}}_i^{\perp} {\bf{X}}_i = {\bf{0}},$ $i = 1, 2, \ldots, k.$

(xiii) $r\!\begin{bmatrix} \pmb{\Sigma}_1 {\bf{X}} & {\bf{X}} & {\bf{0}} \\ {\bf{X}}^{T} {\bf{X}} & {\bf{0}} & {\bf{V}}_i^{T}\end{bmatrix} = r\!\begin{bmatrix} \pmb{\Sigma}_1 {\bf{X}} & {\bf{X}} \\ {\bf{X}}^{T} {\bf{X}} & {\bf{0}}\end{bmatrix} = 2r({\bf{X}}),$ $i = 1, 2, \ldots, k.$

(xiv) $r\!\begin{bmatrix} \pmb{\Sigma}_1 {\bf{X}} & {\bf{X}} \\ {\bf{W}}_i^{T} {\bf{X}} & {\bf{0}}\end{bmatrix} = r({\bf{X}}) + r({\bf{W}}_i),$ $i = 1, 2, \ldots, k.$

(xv) ${\mathscr{R}}[({\bf{V}}_i {\bf{X}}^{+} \pmb{\Sigma}_1)^{T}] \subseteq {\mathscr{R}}({\bf{X}}),$ $i = 1, 2, \ldots, k.$

(xvi) ${\mathscr{R}}\!\begin{bmatrix} {\bf{X}}^{T} \pmb{\Sigma}_1 {\bf{X}}^{\perp} \\ {\bf{0}}\end{bmatrix} \subseteq {\mathscr{R}}\!\begin{bmatrix} {\bf{X}}^{T} {\bf{X}} \\ {\bf{V}}_i\end{bmatrix}\!,$ $i = 1, 2, \ldots, k.$

(xvii) ${\mathscr{R}}\!\begin{bmatrix} {\bf{0}} \\ {\bf{V}}_i^{T}\end{bmatrix} \subseteq {\mathscr{R}}\!\begin{bmatrix} \pmb{\Sigma}_1 {\bf{X}} & {\bf{X}} \\ {\bf{X}}^{T} {\bf{X}} & {\bf{0}}\end{bmatrix}\!,$ $i = 1, 2, \ldots, k.$

(xviii) ${\mathscr{R}}(\pmb{\Sigma}_1 {\bf{W}}_i^{\perp} {\bf{X}}_i) \subseteq {\mathscr{R}}({\bf{X}}),$ $i = 1, 2, \ldots, k.$

Proof. The equivalences of (ⅰ)–(ⅴ) follow from Theorem 3.2 (ⅰ)–(ⅵ). The equivalences of the matrix equalities and relations in (ⅵ)–(xviii) were collected and proved in ^[38]; see also ^[33]. □

Concerning the equalities between the OLSEs and the BLUEs of the whole and partial mean parameter matrices in (1.1), we have the following results:

Theorem 3.5. Suppose that all ${\bf{X}}_i\Theta_i$ in (1.1) are estimable, $i = 1, \ldots, k.$ Then, the following 18 statistical statements are equivalent:

(ⅰ) ${\rm{OLSE}}({\bf{X}}\Theta) = {\rm{BLUE}}({\bf{X}}\Theta)$ holds definitely $($with probability $1).$

(ⅱ) ${\rm{Cov}}(\overrightarrow{{\rm{OLSE}}({\bf{X}}\Theta)}) = {\rm{Cov}}(\overrightarrow{{\rm{BLUE}}({\bf{X}}\Theta)}).$

(ⅲ) ${\rm{Cov}}(\overrightarrow{ {\bf{Y}}} - \overrightarrow{{\rm{OLSE}}({\bf{X}}\Theta)}) = {\rm{Cov}}(\overrightarrow{ {\bf{Y}}} - \overrightarrow{{\rm{BLUE}}({\bf{X}}\Theta)}).$

(ⅳ) ${\rm{Cov}}(\overrightarrow{ {\bf{Y}}}) = {\rm{Cov}}(\overrightarrow{{\rm{OLSE}}({\bf{X}}\Theta)}) + {\rm{Cov}}(\overrightarrow{ {\bf{Y}}} - \overrightarrow{{\rm{OLSE}}({\bf{X}}\Theta)}).$

(ⅴ) ${\rm{Cov}}(\overrightarrow{{\rm{OLSE}}({\bf{X}}\Theta)}) = {\rm{Cov}}\{\overrightarrow{{\rm{BLUE}}({\bf{X}}\Theta)}, \, \overrightarrow{ {\bf{Y}}}\}.$

(ⅵ) ${\rm{Cov}}(\overrightarrow{{\rm{OLSE}}({\bf{X}}\Theta)}) = {\rm{Cov}}\{\overrightarrow{{\rm{BLUE}}({\bf{X}}\Theta)}, \, \overrightarrow{{\rm{OLSE}}({\bf{X}}\Theta)}\}.$

(ⅶ) ${\rm{Cov}}\{\overrightarrow{{\rm{OLSE}}({\bf{X}}\Theta)}, \, \overrightarrow{ {\bf{Y}}}\} = {\rm{Cov}}\{\overrightarrow{{\rm{BLUE}}({\bf{X}}\Theta)}, \, \overrightarrow{ {\bf{Y}}}\}.$

(ⅷ) ${\rm{Cov}}\{\overrightarrow{{\rm{OLSE}}({\bf{X}}\Theta)}, \, \overrightarrow{ {\bf{Y}}}\} = {\rm{Cov}}\{\overrightarrow{{\rm{BLUE}}({\bf{X}}\Theta)}, \, \overrightarrow{{\rm{OLSE}}({\bf{X}}\Theta)}\}.$

(ⅸ) ${\rm{Cov}}\{\overrightarrow{{\rm{OLSE}}({\bf{X}}\Theta)}, \, \overrightarrow{ {\bf{Y}}}\} = {\rm{Cov}}\{\overrightarrow{ {\bf{Y}}}, \, \overrightarrow{{\rm{OLSE}}({\bf{X}}\Theta)}\}.$

(ⅹ) ${\rm{Cov}}\{\overrightarrow{ {\bf{Y}}} - \overrightarrow{{\rm{OLSE}}({\bf{X}}\Theta)}, \, \overrightarrow{ {\bf{Y}}}\} = {\rm{Cov}}\{\overrightarrow{ {\bf{Y}}}, \, \overrightarrow{ {\bf{Y}}} - \overrightarrow{{\rm{OLSE}}({\bf{X}}\Theta)}\}.$

(xi) ${\rm{Cov}}\{\overrightarrow{ {\bf{Y}}} - \overrightarrow{{\rm{OLSE}}({\bf{X}}\Theta)}, \, \overrightarrow{{\rm{OLSE}}({\bf{X}}\Theta)}\} = {\rm{Cov}}\{\overrightarrow{{\rm{OLSE}}({\bf{X}}\Theta)}, \, \overrightarrow{ {\bf{Y}}} - \overrightarrow{{\rm{OLSE}}({\bf{X}}\Theta)}\} = {\bf{0}}.$

(xii) ${\rm{Cov}}\{\overrightarrow{ {\bf{Y}}} - \overrightarrow{{\rm{OLSE}}({\bf{X}}\Theta)}, \, \overrightarrow{{\rm{OLSE}}({\bf{X}}\Theta)}\} + {\rm{Cov}}\{\overrightarrow{{\rm{OLSE}}({\bf{X}}\Theta)}, \, \overrightarrow{ {\bf{Y}}} - \overrightarrow{{\rm{OLSE}}({\bf{X}}\Theta)}\} = {\bf{0}}.$

(xiii) ${\rm{Cov}}\{\overrightarrow{{\rm{OLSE}}({\bf{X}}\Theta)}, \, \overrightarrow{ {\bf{Y}}} - \overrightarrow{{\rm{OLSE}}({\bf{X}}\Theta)}\} = {\bf{0}}.$

(xiv) All ${\rm{OLSE}}({\bf{X}}_i\Theta_i) = {\rm{BLUE}}({\bf{X}}_i\Theta_i)$ hold definitely $($with probability $1),$ $i = 1, 2, \ldots, k.$

(xv) All ${\rm{Cov}}(\overrightarrow{{\rm{OLSE}}({\bf{X}}_i\Theta_i)}) = {\rm{Cov}}(\overrightarrow{{\rm{BLUE}}({\bf{X}}_i\Theta_i)})$ hold, $i = 1, 2, \ldots, k.$

(xvi) All ${\rm{Cov}}\{\overrightarrow{{\rm{OLSE}}({\bf{X}}_i\Theta_i)}, \ \overrightarrow{ {\bf{Y}}}\} = {\rm{Cov}}\{\overrightarrow{{\rm{BLUE}}({\bf{X}}_i\Theta_i)}, \ \overrightarrow{ {\bf{Y}}}\}$ hold, $i = 1, 2, \ldots, k.$

(xvii) All ${\rm{Cov}}\{\overrightarrow{{\rm{OLSE}}({\bf{X}}_i\Theta_i)}, \ \overrightarrow{ {\bf{Y}}}\} = {\rm{Cov}}\{\overrightarrow{{\rm{BLUE}}({\bf{X}}_i\Theta_i)}, \ \overrightarrow{{\rm{OLSE}}({\bf{X}}\Theta)}\}$ hold, $i = 1, 2, \ldots, k.$

(xviii) All ${\rm{Cov}}\{\overrightarrow{ {\bf{Y}}} - \overrightarrow{{\rm{OLSE}}({\bf{X}}\Theta)}, \, \overrightarrow{{\rm{OLSE}}({\bf{X}}_i\Theta_i)}\} = {\bf{0}}$ hold, $i = 1, 2, \ldots, k.$

Proof. The equivalences of (ⅰ)–(xiii) follow from Corollary 3.3. The equivalences of (xiv)–(xviii) follow from Corollary 3.4. If (ⅰ) holds, then ${\mathscr{R}}(\pmb{\Sigma}_1 {\bf{X}}) \subseteq {\mathscr{R}}({\bf{X}})$ holds by Corollary 3.3 (xxviii), and therefore the rank equality in Corollary 3.4 (xiii) holds as well, so that (ⅰ) implies (xiv)–(xviii). On the contrary, adding both sides of the $k$ equalities in (xiv) and combining the equality with (2.18) and (2.31), we obtain (ⅰ). □

4.   Concluding remarks

Because OLSEs and BLUEs of unknown parameters in MGLMs can all be represented in certain exact and analytical formulas, it is definitely possible to obtain a number of clear and significant facts and results regarding the two fundamental and orthodox types of estimators by means of various mathematical analysis tools. In light of this fact, we described and studied in the preceding sections a group of research problems on establishing connections between OLSEs and BLUEs of unknown parameters in MGLMs through the well-organized employment of some exact algebraic methods and techniques in matrix theory and obtained a variety of algebraic and statistical interpretations for the equivalences of OLSEs and BLUEs under MGLMs. Unquestionably, this study shows that the equivalence problems of OLSEs and BLUEs under MGLMs are not isolated facts, but have diverse intrinsic links from algebraic and statistical points of view. Also, notice that the results in the previous sections are given in clear and analytical forms, so that they are easy to understand under various assumptions, and thereby enable us to recognize and use these versatile equivalent facts in many different situations when estimating parametric spaces in MGLMs and describing various mathematical and statistical properties of the estimators. Thus, it is technically necessary to collect these equivalent algebraic and statistical facts together and take them as a certain theoretical foundation for OLSE and BLUE problems on MGLMs. This work also illustrates that although OLSEs and BLUEs are classic objects of study, we are still able to propose various new and deep statistical inference problems on these objects and derive many novel and profound results under general assumptions by making use of various influential and effective matrix analysis tools and skilled partitioned matrix calculations. Hence, the contributions in this paper are closely related to the current research on linear statistical inference in its broadest sense. Finally, we believe that the resultant approaches to the equivalences of OLSEs and BLUEs provide significant advances to algebraical methodology in the statistical analysis of MGLMs and will enable methodological improvements and advances in the field of multivariate analysis.

Author contributions

Bo Jiang: conducted the formal research and investigation process; Yongge Tian: provided the ideas formulation and methodology for the research goals of the article and wrote the initial draft. All authors have read and approved the final version of the manuscript for publication.

Use of AI tools declaration

The authors declare they have not used Artificial Intelligence (AI) tools in the creation of this article.

Acknowledgments

We would like to express our sincere thanks to the anonymous reviewers for their helpful comments and suggestions on an earlier version of this paper.

Conflict of interest

The authors declare no conflict of interest.