An effective treatment of adding-up restrictions in the inference of a general linear model

1.   Introduction

Throughout, let ${\mathbb R}^{m \times n}$ stand for the collection of all $m \times n$ matrices over the field of real numbers; ${\bf A}^{\prime}$, $r({\bf A}),$ and ${\mathscr R}({\bf A})$ stand for the transpose, the rank, and the range (column space) of a matrix ${\bf A} \in {\mathbb R}^{m \times n}$, respectively; and let ${\bf I}_m$ denote the identity matrix of order $m$. Given an ${\bf A} \in {\mathbb R}^{m \times n}$, the Moore–Penrose generalized inverse of ${\bf A}$, denoted by ${\bf A}^{+}$, is defined to be the unique solution ${\bf G}$ satisfying the four matrix equations ${\bf A} {\bf G} {\bf A} = {\bf A}$, ${\bf G} {\bf A} {\bf G} = {\bf G}$, $({\bf A} {\bf G})^{\prime} = {\bf A} {\bf G}$, and $({\bf G} {\bf A})^{\prime} = {\bf G} {\bf A}$. Further, let ${\bf P}_{ {\bf A}}$, ${\bf E}_{ {\bf A}}$, and ${\bf F}_{ {\bf A}}$ stand for the three orthogonal projectors (symmetric idempotent matrices) ${\bf P}_ {\bf A} = {\bf A} {\bf A}^{+}$, ${\bf E}_{ {\bf A}} = {\bf A}^{\perp} = {\bf I}_m - {\bf A} {\bf A}^{+}$, and ${\bf F}_{ {\bf A}} = {\bf I}_n - {\bf A}^{+} {\bf A}$, which will help in briefly denoting calculation processes related to generalized inverses of matrices. Further information about the orthogonal projectors ${\bf P}_ {\bf A}$, ${\bf E}_ {\bf A}$, and ${\bf F}_ {\bf A}$ with their applications in the linear statistical models can be found, e.g., in ^[5,9]. Two symmetric matrices ${\bf A}$ and ${\bf B}$ of the same size are said to satisfy the inequality ${\bf A} \succcurlyeq {\bf B}$ in the Löwner partial ordering if ${\bf A} - {\bf B}$ is nonnegative definite. For more results on the Löwner partial ordering of real symmetric matrices and its applications in statistical analysis, see e.g., ^[9,17].

Consider a general linear model

where it is assumed that ${\bf y} \in {\mathbb R}^{n \times 1}$ is a vector of observable random variables, ${\bf X} \in {\mathbb R}^{n \times p}$ is a known model matrix of arbitrary rank ($0 \leq r({\bf X}) \leq \min\{n, p\})$, $\pmb{\beta} \in {\mathbb R}^{n \times 1}$ is a vector of fixed but unknown parameters, $\sigma^2$ is an arbitrary positive scaling factor, and $\pmb{\Sigma} \in {\mathbb R}^{n \times n}$ is a known nonnegative definite matrix of arbitrary rank ($0 \leq r(\pmb{\Sigma}) \leq n$), for example $\pmb{\Sigma} = {\bf I}$.

Below, we present some background details of the work. For a variety of reasons, statisticians may meet with the situation where certain restrictions are imposed on the unknown coefficient vector $\pmb{\beta}$ and the observed random vector ${\bf y}$ in (1.1). For example, it is a regular case to add a system of linear matrix equations ${\bf B} \pmb{\beta} = {\bf c}$ to the unknown parameter vector under the assumption in (1.1), and there have been plenty of approaches and discussions addressing how to carry out statistical inference under linear models with restrictions to their unknown parameters. In addition to the situations imposing restrictions on unknown parameters, it is necessary to take into account as well the situations of adding certain limitations and restrictions upon observed random variables in the model from theoretical and applied points of view. Under the model assumption in (1.1), one of such considerations is assuming that the observed random vector ${\bf y}$ satisfies a consistent linear matrix equation

where it is assumed that ${\bf A} \in {\mathbb R}^{m \times n}$ is a known matrix with rank$({\bf A}) = k \leq \min\{m, \, n\}$ and ${\bf b} \in {\mathbb R}^{m \times 1}$ is a known vector with ${\bf b} \in {\mathscr R}({\bf A})$. A matrix equation as given in (1.2) is usually called adding-up restrictions to ${\bf y}$ in (1.1) in the literature. Clearly, the adding-up restrictions include ${\bf w}^{\prime} {\bf y} = a$ or 1 etc. as its special cases, where ${\bf w}$ is a column vector. This kind of plausible-sounding restrictions do exist in statistical practice, which were noticed and approached in certain fields of applied statistics and attracted sort of consideration. For example, economists explored some situations of the specific kind where certain adding-up restrictions appeared, in which they presented fitting descriptions of how to think the restrictions, and gave some of their solutions to a number of corresponding estimation and inference problems. We refer the reader to ^[3,4,10,13] for more information on the appearance of such kind of adding-up restrictions. However, the general situation depicted in (1.2) has not been properly approached in the statistical literature, yet the very process of mathematical and statistical approaches of the adding-up restrictions remains hidden.

For the purpose of making inference in the contexts of (1.1) and (1.2), we merge the two model equations in the following form

In this case, we can substitute (1.1) into (1.2) to lead to ${\bf A} {\bf X} \pmb{\beta} + {\bf A} \pmb{\varepsilon} = {\bf b}$, and rewrite (1.3) in the following equivalent form

Given the model equations as such in (1.3) or (1.4), we are confronted with the task of how to properly merge the adding-up restrictions in the estimation and inference procedure of the unknown parameter vector $\pmb{\beta}$. There have been several attempts in the past to solve the problem of choosing possible merge procedures. Unfortunately, when faced to such a question, professional statisticians have no definitive answers or rather they have different methods which have been suggested and implemented in the literature, none of them being completely convincing in the sense that it has been shown to be better than the others. The purpose of this article is to focus attention on dealing with the adding-up restrictions in (1.3) with some new thoughts and methodologies. The author will offer a feasible algebraic method to reconcile the second adding-up restrictions with the first regression equation in (1.3), and then use the method to solve some basic estimation and inference problems associated with ${\mathscr N}$.

The rest of this paper is organized as follows. In Section 2, we introduce some basic formulas, facts, and results in matrix theory, as well as two groups of existing results related to the ordinary least-squares estimators (OLSE) and the best linear unbiased estimators (BLUEs) of unknown parametric vectors under (1.1). In Section 3, we show how to transform ${\mathscr N}$ in (1.3) into two kinds of linear models with implicit and explicit restrictions to the unknown parameter vector $\pmb{\beta}$ respectively via certain suitable equivalent explanation of the adding-up restrictions. In Sections 4 and 5, we presents the description of the estimability of unknown parametric vector ${\bf K} \pmb{\beta}$ under the transformed models, and give the definitions and the derivations of analytical expressions of the OLSEs and BLUEs of ${\bf K} \pmb{\beta}$ through the transformed models. Section 6 gives some concluding remarks and a group of research problems concerning general linear models with adding-up restrictions.

2.   Some preliminaries

In this section, we introduce some fundamental formulas and facts about matrix operations that have related applications to statistics, especially linear statistical models. It was properly known that the theory of generalized inverses of matrices is a major and dependable source of methods and techniques that was brought into the theory of linear statistical models for regression in 1950s, and thereby it played a key role for carrying out statistical estimation and inference in a wide variety of situations; see e.g., ^[2,9,14]. In this section, we shall present a group of well-known formulas, facts, and results in linear algebra and matrix theory, which we shall use as resource to simplify various matrix expressions that involve generalized inverses of matrices.

Lemma 2.1 (^[7]). Let ${\bf A} \in \mathbb R^{m \times n},$ ${\bf B} \in \mathbb R^{m \times k},$ and ${\bf C} \in \mathbb R^{l \times n}.$ Then$,$

${\rm (a)}$ $r[{\bf A}, \, {\bf B}] = r({\bf A})+ r({\bf E}_{ {\bf A}} {\bf B}) = r({\bf B}) + r({\bf E}_{ {\bf B}} {\bf A}).$

${\rm (b)}$ $r\begin{bmatrix} {\bf A} \\ {\bf C} \end{bmatrix} = r({\bf A}) + r({\bf C} {\bf F}_{ {\bf A}}) = r({\bf C}) + r({\bf A} {\bf F}_{ {\bf C}}).$

In particular$,$

${\rm (c)}$ $r[{\bf A}, \, {\bf B}] = r({\bf A}) \Leftrightarrow {\bf E}_{ {\bf A}} {\bf B} = {\bf 0} \Leftrightarrow {\mathscr R}({\bf B}) \subseteq {\mathscr R}({\bf A}).$

${\rm (d)}$ $r\begin{bmatrix} {\bf A} \\ {\bf C} \end{bmatrix} = r({\bf A}) \Leftrightarrow {\bf C} {\bf F}_{ {\bf A}} = {\bf 0} \Leftrightarrow {\mathscr R}({\bf C}') \subseteq {\mathscr R}({\bf A}').$

Lemma 2.2 (^[8]). Let ${\bf A} \in {\mathbb R}^{m \times n}$ and ${\bf B} \in {\mathbb R}^{m \times k},$ Then$,$ the linear matrix equation ${\bf A} {\bf X} = {\bf B}$ is solvable for ${\bf X}$ if and only if $r[{\bf A}, \, {\bf B}] = r({\bf A}),$ or equivalently$,$ ${\bf A} {\bf A}^{+} {\bf B} = {\bf B}.$ In this case$,$ the general solution of the equation can be written in the parametric form ${\bf X} = {\bf A}^{+} {\bf B} + (\, {\bf I}_n - {\bf A}^{+} {\bf A} \, ) {\bf U},$ where ${\bf U} \in {\mathbb R}^{n \times k}$ is an arbitrary matrix$.$

There were different established inference theories and methods that we can adopt to estimate the unknown parameter vector $\pmb{\beta}$ in (1.1), the two best-known tools were OLSEs and BLUEs. We turn now to reviewing some basic definitions and existing facts in linear model theory regarding the estimability, as well as the OLSEs and BLUEs of a given unknown parametric vector under (1.1), see e.g., ^{[1,9,11,12,18]}.

Definition 2.3. Let ${\mathscr M}$ be as given in (1.1) and let ${\bf K} \in {\mathbb R}^{k \times p}$ be given. The vector ${\bf K} \pmb{\beta}$ of parametric functions is said to be estimable under ${\mathscr M}$ if there exists an ${\bf L} \in {\mathbb R}^{k \times n}$ such that ${\rm E}({\bf L} {\bf y} - {\bf K} \pmb{\beta}) = {\bf 0}$ holds for all $\pmb{\beta}$ in ${\mathscr M}$.

Definition 2.4. Let ${\mathscr M}$ be as given in (1.1)$,$ and let ${\bf K} \in {\mathbb R}^{k \times p}$ be given.

${\rm(a)}$ The OLSE of the parametric vector $\pmb{\beta}$ under (1.1), denoted by ${\rm OLSE}_{ {\mathscr M}}(\pmb{\beta})$, is defined to be

The OLSE of ${\bf K} \pmb{\beta}$ under (1.1) is defined to be ${\rm OLSE}_{ {\mathscr M}}({\bf K} \pmb{\beta}) = {\bf K}{\rm OLSE}_{ {\mathscr M}}(\pmb{\beta}).$

${\rm(b)}$ The BLUE of the vector of parametric functions ${\bf K} \pmb{\beta}$ under (1.1)$,$ denoted by ${\rm BLUE}_{ {\mathscr M}}({\bf K} \pmb{\beta})$, is defined to be linear statistic ${\bf L} {\bf y}$, where ${\bf L}$ is a matrix such that ${\rm {\rm Cov}}({\bf L} {\bf y} - {\bf K} \pmb{\beta}) = \min$ in the Löwner partial ordering subject to ${\rm E}({\bf L} {\bf y} - {\bf K} \pmb{\beta}) = {\bf 0}.$

As we know that the concepts of OLSE and BLUE have a long history and deep roots in parametric regression analysis, both of which have many nice and optimal algebraic and statistical properties, and therefore are the most welcome linear statistical inference techniques in parametric regression theory and the related applications. The conventionality of OLSEs/BLUEs under linear regression models really attracted statisticians' attention in the development of regression theory, and numerous formulas and facts regarding the OLSEs/BLUEs of $\pmb{\beta}$ and ${\bf K} \pmb{\beta}$ under (1.1) were established via various precise and analytical algebraic operations of the given vectors and matrices and their generalized inverses. Specifically, the results in the following two lemmas were highly recognized in the domain of linear statistical models.

Lemma 2.5. Let ${\mathscr M}$ be as given in (1.1)$,$ and let ${\bf K} \in {\mathbb R}^{k \times p}$ be given$.$ Then$,$ the general expression of OLSEs of $\pmb{\beta}$ in ${\mathscr M}$ can be written as

where ${\bf v} \in \mathbb R^{p \times 1}$ is arbitrary$;$ and the OLSE of ${\bf K} \pmb{\beta}$ under ${\mathscr M}$ can be written as

Lemma 2.6. Let ${\mathscr M}$ be as given in (1.1)$,$ ${\bf K} \in {\mathbb R}^{k \times p},$ and suppose ${\bf K} \pmb{\beta}$ is estimable under ${\mathscr M},$ namely$,$ ${\mathscr R}({\bf K}^{\prime}) \subseteq {\mathscr R}({\bf X}^{\prime}).$ Then$,$ the BLUE of ${\bf K} \pmb{\beta}$ under ${\mathscr M}$ can be written as

where ${\bf P}_{ {\bf K}; {\bf X}; \pmb{\Sigma}}$ is the solution of the matrix equation

This equation is always solvable for ${\bf G},$ that is$,$ ${\mathscr R}([\, {\bf K}, \, {\bf 0} \, ]') \subseteq {\mathscr R}([{\bf X}, \, \pmb{\Sigma} {\bf X}^{\perp}]').$ In this case$,$ the general solution of (2.5) can be expressed as

where ${\bf U} \in \mathbb R^{k \times n}$ is arbitrary$.$ Moreover$,$ the following results hold$.$

${\rm(a)}$ $r[{\bf X}, \, \pmb{\Sigma} {\bf X}^{\perp}] = r[{\bf X}, \, \pmb{\Sigma}]$ and ${\mathscr R}[{\bf X}, \, \pmb{\Sigma} {\bf X}^{\perp}] = {\mathscr R}[{\bf X}, \, \pmb{\Sigma}].$

${\rm(b)}$ The product ${\bf P}_{ {\bf K}; {\bf X}; \pmb{\Sigma}} \pmb{\Sigma}$ can be uniquely written as ${\bf P}_{ {\bf K}; {\bf X}; \pmb{\Sigma}} \pmb{\Sigma} = [{\bf K}, \, {\bf 0}][{\bf X}, \, \pmb{\Sigma} {\bf X}^{\perp}]^{+} \pmb{\Sigma}.$

${\rm(c)}$ The expectation and covariance matrix of ${\rm BLUE}_{ {\mathscr N}}({\bf K} \pmb{\beta})$ are given by

${\rm(d)}$ The matrix ${\bf P}_{ {\bf K}; {\bf X}; \pmb{\Sigma}}$ is unique if and only if $r[{\bf X}, \, \pmb{\Sigma}] = n.$

${\rm(e)}$ ${\rm BLUE}_{ {\mathscr M}}({\bf K} \pmb{\beta})$ is unique if and only if if and only if ${\bf y} \in {\mathscr R}[{\bf X}, \, \pmb{\Sigma}]$ holds with probability $1.$

Apparently, the representations and derivations of the OLSEs and BLUEs in the above two existing lemmas do not require to specify the mathematical forms of the distributions of the error term vector $\pmb{\varepsilon}$, and thereby we are able to actively utilize the clear and exact expressions of the OLSEs and BLUEs to solve many algebraic and computational problems in the context of (1.1).

3.   Transformations of ${\mathscr N}$ into linear models with implicit and explicit restrictions to unknown parameters

Because the adding-up equation is directly put on the random vector ${\bf y}$ in (1.1), where no information of $\pmb{\beta}$ explicitly appears in the equation, we should be careful to include the equation in the estimation and inference procedure of (1.1). In other words, we have to seek some alternative methods to approach estimation and inference problems of unknown parameters in the model. To this purpose, we show in this section how to make use of the complete information associated with the adding-up restrictions and to convert (1.3) into certain ordinary linear models with implicit and explicit restrictions to the unknown parametric vector $\pmb{\beta}$, respectively.

Given that ${\bf y}$ in (1.1) is a random vector, we then take the expectation and covariance matrix of both sides of the equation ${\bf A} {\bf y} - {\bf b} = {\bf 0}$ with respect to ${\bf y}$ to obtain

Since the matrix $\pmb{\Sigma}$ in (1.3) is positive semi-definite, it is easy to verify that the matrix equality ${\bf A} \pmb{\Sigma} {\bf A}' = {\bf 0}$ is equivalent to $\pmb{\Sigma} = {\bf F}_{ {\bf A}} \pmb{\Sigma} {\bf F}_{ {\bf A}}$. The adding-up equation in (1.2) thereby suggests the following facts

This treatment is by no means profound and difficult to understand under the assumption in (1.3), and thus we can view them as the best mathematical interpretation that can be given about the adding-up restrictions. Recognizing the key role of (3.2) in the interpretation of (1.2), we are able subsequently to deal with the adding-up equation in carrying out inference under (1.3). There are basically two algebraic methods to merge the adding-up equation into (1.1) via (3.2). Below, we perspicuously illustrate the algebraic processes.

$\rm(I)$ Firstly, substituting the first equation into the second equation in (1.3) and noting (3.2), we can equivalently rewrite (1.3) in the following implicitly restricted linear model

$\rm(II)$ Also replacing ${\bf A} {\bf y} = {\bf b}$ and $\pmb{\Sigma}$ in (1.3) with (3.2) produces with probability 1 the following explicitly restricted linear model

Apparently, the two alternative forms in (3.3) and (3.4) are in compliance with ${\mathscr N}$ in (1.3), and thereby they can conveniently help solve various estimation and inference problems under ${\mathscr N}$. It is easily seen that (3.3) and (3.4) are nothing but two ordinary linear statistical models with implicit and explicit restrictions to unknown parameters in the models. This fact enables us to adopt different approach processes to carry out common estimation and inference under (1.3) via (3.3) and (3.4), and thereby this alternative way in fact makes clear insights actionable into the connotation of the model in (1.3).

As a classic subject of study in regression analysis, there has been some general discussion regarding estimation and inference problems of a linear statistical model with implicit and explicit restrictions to unknown parameters in the model; see, e.g., ^[19] and references therein. In light of the existing theory pertaining to this topic, we are now able to make statistical inference of (1.3) via the two alternative forms in (3.3) and (3.4) through the well-organized employment of ordinary theory and methodology of dealing linear regression models under various assumptions.

4.   Estimation results under ${\mathscr N}_a$

For convenience of representation, we adopt the notation

in the sequel. We first describe the consistency problem in the context of (3.3). Note that the matrix equality

holds from the definition of the Moore–Penrose generalized inverse. Therefore, it turns out under the assumptions in (3.2) that

These two equalities imply $[\widehat{ {\bf X}}, \, \widehat{ \pmb{\Sigma}}][\widehat{ {\bf X}}, \, \widehat{ \pmb{\Sigma}}]^{+} \widehat{ {\bf y}} = \widehat{ {\bf y}}$ holds with probability 1, or equivalently,

holds with probability 1. In view of this fact, we adopt the following definition.

Definition 4.1. ${\mathscr N}_a$ in (3.3) is said to be consistent if (4.1) holds with probability 1.

The OLSEs and BLUEs of unknown parameters in a given linear statistical model were recognized as two principal estimations in the domain of linear regression models, which were deeply approached and utilized in the development of statistical science. In the following, we introduce the definitions of the OLSEs and BLUEs of vectors of parametric functions, and then presents the exact and analytical formulas for calculating the OLSEs and BLUEs under (3.3).

Definition 4.2. Let ${\mathscr N}_a$ be as given in (3.3)$,$ and let ${\bf K} \in {\mathbb R}^{k \times p}$ be given. The vector ${\bf K} \pmb{\beta}$ of parametric functions is said to be estimable under ${\mathscr N}_a$ if there exists an ${\bf L} \in {\mathbb R}^{k \times (n+m)}$ such that ${\rm E}({\bf L}\widehat{ {\bf y}} - {\bf K} \pmb{\beta}) = {\bf 0}$ holds for all $\pmb{\beta}$ under ${\mathscr N}_a$.

Definition 4.3. Let ${\mathscr N}_a$ be as given in (3.3)$,$ and let ${\bf K} \in {\mathbb R}^{k \times p}$ be given.

${\rm(a)}$ The OLSE of the parametric vector $\pmb{\beta}$ under (3.3), denoted by ${\rm OLSE}_{ {\mathscr N}_a}(\pmb{\beta})$, is defined to be

The OLSE of ${\bf K} \pmb{\beta}$ under (3.3) is defined to be ${\rm OLSE}_{ {\mathscr N}_a}({\bf K} \pmb{\beta}) = {\bf K}{\rm OLSE}_{ {\mathscr N}_a}(\pmb{\beta}).$

${\rm(b)}$ The BLUE of the vector of parametric functions ${\bf K} \pmb{\beta}$ under ${\mathscr N}$, denoted by ${\rm BLUE}_{ {\mathscr N}_a}({\bf K} \pmb{\beta})$, is defined to be a linear statistic ${\bf L}\widehat{ {\bf y}}$, where ${\bf L}$ is a matrix such that ${\rm {\rm Cov}}({\bf L}\widehat{ {\bf y}} - {\bf K} \pmb{\beta}) = \min$ in the Löwner partial ordering subject to ${\rm E}({\bf L}\widehat{ {\bf y}} - {\bf K} \pmb{\beta}) = {\bf 0}.$

Now applying the above definitions to ${\mathscr N}_a$ in (3.3), we obtain the following results.

Theorem 4.4. Let ${\mathscr N}_a$ be as given in (3.3)$,$ and let ${\bf K} \in {\mathbb R}^{k \times p}$ be given$.$ Then$,$ ${\bf K} \pmb{\beta}$ is estimable under ${\mathscr N}_a$ $\Leftrightarrow$ ${\mathscr R}({\bf K}^{\prime}) \subseteq {\mathscr R}({\bf X}^{\prime}).$ In particular$,$ ${\bf X} \pmb{\beta}$ is always estimable under ${\mathscr N}_a.$

Proof. It follows from ${\rm E}({\bf L}\widehat{ {\bf y}} - {\bf K} \pmb{\beta}) = {\bf 0}$ $\Leftrightarrow$ ${\bf L}\widehat{ {\bf X}} \pmb{\beta} - {\bf K} \pmb{\beta} = {\bf 0}$ for all $\pmb{\beta}$ $\Leftrightarrow$ ${\bf L}\widehat{ {\bf X}} = {\bf K}$ $\Leftrightarrow$ ${\mathscr R}({\bf K}^{\prime}) \subseteq {\mathscr R}(\widehat{ {\bf X}}^{\prime})$ $\Leftrightarrow$ ${\mathscr R}({\bf K}^{\prime}) \subseteq {\mathscr R}({\bf X}^{\prime})$ by Lemma 2.2.□

Referring to Lemmas 2.5 and 2.6, we obtain the following two results about the OLSEs and BLUEs under (3.3).

Theorem 4.5. Let ${\mathscr N}_a$ be as given in (3.3) and suppose ${\bf K} \pmb{\beta}$ is estimable under ${\mathscr N}_a.$ Then$,$ the OLSE of $\pmb{\beta}$ under ${\mathscr N}_a$ can be written as ${\rm OLSE}_{ {\mathscr N}_a}(\pmb{\beta}) = \widehat{ {\bf X}}^{+}\widehat{ {\bf y}} + {\bf F}_{ {\bf X}} {\bf v},$ where ${\bf v} \in \mathbb R^{p \times 1}$ is arbitrary$;$ and the OLSE of ${\bf K} \pmb{\beta}$ under ${\mathscr N}_a$ can be uniquely written as

Theorem 4.6. Let ${\mathscr N}_a$ be as given in (3.3) and suppose ${\bf K} \pmb{\beta}$ is estimable under ${\mathscr N}_a.$ Then$,$ the BLUE of ${\bf K} \pmb{\beta}$ under ${\mathscr N}_a$ can be written as

where ${\bf P}_{ {\bf K}; \widehat{ {\bf X}}; \widehat{ \pmb{\Sigma}}}$ is the solution of the matrix equation ${\bf G}[\widehat{ {\bf X}}, \, \widehat{ \pmb{\Sigma}}\widehat{ {\bf X}}^{\perp}] = [{\bf K}, \, {\bf 0}].$ This equation is always solvable for ${\bf G},$ that is, ${\mathscr R}([{\bf K}, \, {\bf 0}]') \subseteq {\mathscr R}([\widehat{ {\bf X}}, \, \widehat{ \pmb{\Sigma}}\widehat{ {\bf X}}^{\perp}]').$ In this case$,$ the general solution of the matrix equation can be expressed as

where ${\bf U} \in \mathbb R^{k \times (n+m)}$ is arbitrary$.$ Moreover$,$ the following results hold$.$

${\rm(a)}$ $r[\widehat{ {\bf X}}, \, \widehat{ \pmb{\Sigma}}\widehat{ {\bf X}}^{\perp}] = r[\widehat{ {\bf X}}, \, \widehat{ \pmb{\Sigma}}\, ]$ and ${\mathscr R}[\widehat{ {\bf X}}, \, \widehat{ \pmb{\Sigma}}\widehat{ {\bf X}}^{\perp}] = {\mathscr R}[\widehat{ {\bf X}}, \, \widehat{ \pmb{\Sigma}}].$

${\rm(b)}$ The product ${\bf P}_{ {\bf K}; \widehat{ {\bf X}}; \widehat{ \pmb{\Sigma}}}\widehat{ \pmb{\Sigma}}$ can be uniquely written as ${\bf P}_{ {\bf K}; \widehat{ {\bf X}}; \widehat{ \pmb{\Sigma}}}\widehat{ \pmb{\Sigma}} = [{\bf K}, \, {\bf 0}][\widehat{ {\bf X}}, \, \widehat{ \pmb{\Sigma}}\widehat{ {\bf X}}^{\perp}]^{+}\widehat{ \pmb{\Sigma}}.$

${\rm(c)}$ The expectation and covariance matrix of ${\rm BLUE}_{ {\mathscr N}_a}({\bf K} \pmb{\beta})$ are given by

${\rm(d)}$ The matrix ${\bf P}_{ {\bf K}; \widehat{ {\bf X}}; \widehat{ \pmb{\Sigma}}}$ is unique if and only if $r[\widehat{ {\bf X}}, \, \widehat{ \pmb{\Sigma}}] = m + n.$

${\rm(e)}$ ${\rm BLUE}_{ {\mathscr N}_a}({\bf K} \pmb{\beta})$ is unique if and only if $\widehat{ {\bf y}} \in {\mathscr R}[\widehat{ {\bf X}}, \, \widehat{ \pmb{\Sigma}}]$ holds with probability $1.$

5.   Estimation results under ${\mathscr N}_b$

In what follows, we denote $\widetilde{ {\bf A}} = {\bf A} {\bf X}$ and $\widetilde{ \pmb{\Sigma}} = {\bf F}_{ {\bf A}} \pmb{\Sigma} {\bf F}_{ {\bf A}}$. Recall a well-known fact that the matrix equation $\widetilde{ {\bf A}} \pmb{\beta} = {\bf b}$ is solvable for $\pmb{\beta}$ if and only if ${\bf b} \in {\mathscr R}(\widetilde{ {\bf A}})$. By Lemma 2.2, the general solution of $\pmb{\beta}$ and the corresponding ${\bf K} \pmb{\beta}$ can be written in the following parametric forms

where $\pmb{\gamma} \in \mathbb R^{p \times 1}$ is arbitrary. Substitution of (5.1) into (3.4) yields

where ${\bf z} = {\bf y} - {\bf X}\widetilde{ {\bf A}}^{+} {\bf b}$. This is a new linear model with the unknown parameter vector $\pmb{\gamma}$. Hence, the estimablility, the OLSE, and the BLUE of the vector of parametric functions ${\bf K} {\bf F}_{\widetilde{ {\bf A}}} \pmb{\gamma}$ can be obtained from various existing results as follows.

Definition 5.1. Let ${\mathscr N}_b$ be as given in (3.4), and let ${\bf K} \in {\mathbb R}^{k \times p}$ be given$.$ The vector ${\bf K} \pmb{\beta}$ of parametric functions is said to be estimable under ${\mathscr N}_b$ if there exist ${\bf L} \in {\mathbb R}^{k \times n}$ and ${\bf c} \in {\mathbb R}^{k \times 1}$ such that ${\rm E}({\bf L} {\bf y} + {\bf c} - {\bf K} \pmb{\beta}) = {\bf 0}$ holds under ${\mathscr N}_b$.

Lemma 5.2. Let ${\mathscr N}_b$ be as given in (3.4)$,$ and let ${\bf K} \in {\mathbb R}^{k \times p}$ be given$.$ Then$,$ ${\bf K} \pmb{\beta}$ is estimable under ${\mathscr N}_b$ if and only if ${\mathscr R}({\bf K}') \subseteq {\mathscr R}({\bf X}').$

Theorem 5.3. Let ${\mathscr N}_b$ be as given in (3.4)$,$ let ${\bf K} \in {\mathbb R}^{k \times p}$ be given$,$ and suppose ${\bf K} \pmb{\beta}$ is estimable under (3.4)$.$ Then$,$ the OLSE of $\pmb{\beta}$ under ${\mathscr N}_b$ can be written as

where ${\bf u} \in \mathbb R^{p \times 1}$ is arbitrary$.$ The OLSE of ${\bf K} \pmb{\beta}$ under ${\mathscr N}_b$ can be uniquely written as

Proof. According to Lemma 2.5, the OLSE of $\pmb{\gamma}$ in (5.3) can be written as

where ${\bf u} \in \mathbb R^{p \times 1}$ is arbitrary. Substitution of this formula into (5.1) gives the OLSE of $\pmb{\beta}$ in (3.4):

establishing (5.4) and (5.5).□

Theorem 5.4. Let ${\mathscr N}_b$ be as given in (3.4) and suppose ${\bf K} \pmb{\beta}$ is estimable under (3.4)$.$ Then$,$ the BLUE of ${\bf K} \pmb{\beta}$ under ${\mathscr N}_b$ can be written as

where ${\bf P}_{ {\bf K} {\bf F}_{\widetilde{ {\bf A}}}; {\bf X} {\bf F}_{\widetilde{ {\bf A}}}; \widetilde{ \pmb{\Sigma}}} = [\, {\bf K} {\bf F}_{\widetilde{ {\bf A}}}, \, {\bf 0} \, ][\, {\bf X} {\bf F}_{\widetilde{ {\bf A}}}, \, \widetilde{ \pmb{\Sigma}} {\bf E}_{ {\bf X} {\bf F}_{\widetilde{ {\bf A}}}}\, ]^{+} + {\bf U}_1 {\bf E}_{[\, {\bf X} {\bf F}_{\widetilde{ {\bf A}}}, \, \widetilde{ \pmb{\Sigma}} {\bf E}_{ {\bf X} {\bf F}_{\widetilde{ {\bf A}}}}\, ]},$ and ${\bf U}_1 \in \mathbb R^{k \times n}$ is arbitrary$.$ Moreover$,$ the following results hold$.$

${\rm(a)}$ $r[{\bf X} {\bf F}_{\widetilde{ {\bf A}}}, \, \widetilde{ \pmb{\Sigma}} {\bf E}_{ {\bf X} {\bf F}_{\widetilde{ {\bf A}}}}] = r[{\bf X} {\bf F}_{\widetilde{ {\bf A}}}, \, \widetilde{ \pmb{\Sigma}}]$ and ${\mathscr R}[{\bf X} {\bf F}_{\widetilde{ {\bf A}}}, \, \widetilde{ \pmb{\Sigma}} {\bf E}_{ {\bf X} {\bf F}_{\widetilde{ {\bf A}}}}] = {\mathscr R}[{\bf X} {\bf F}_{\widetilde{ {\bf A}}}, \, \widetilde{ \pmb{\Sigma}}].$

${\rm(b)}$ The product ${\bf P}_{ {\bf K} {\bf F}_{\widetilde{ {\bf A}}}; {\bf X} {\bf F}_{\widetilde{ {\bf A}}}; \widetilde{ \pmb{\Sigma}}} \widetilde{ \pmb{\Sigma}}$ can be uniquely written as ${\bf P}_{ {\bf K} {\bf F}_{\widetilde{ {\bf A}}}; {\bf X} {\bf F}_{\widetilde{ {\bf A}}}; \widetilde{ \pmb{\Sigma}}}\widetilde{ \pmb{\Sigma}} = [{\bf K} {\bf F}_{\widetilde{ {\bf A}}}, \, {\bf 0}][{\bf X} {\bf F}_{\widetilde{ {\bf A}}}, \, \widetilde{ \pmb{\Sigma}} {\bf E}_{ {\bf X} {\bf F}_{\widetilde{ {\bf A}}}}]^{+}\widetilde{ \pmb{\Sigma}}.$

${\rm(c)}$ The expectation and covariance matrix of ${\rm BLUE}_{ {\mathscr N}_b}({\bf K} \pmb{\beta})$ are given by

${\rm(d)}$ The matrix ${\bf P}_{ {\bf K} {\bf F}_{\widetilde{ {\bf A}}}; {\bf X} {\bf F}_{\widetilde{ {\bf A}}}; \widetilde{ \pmb{\Sigma}}}$ is unique if and only if $r\begin{bmatrix} {\bf X} & \widetilde{ \pmb{\Sigma}} \\ \widetilde{ {\bf A}} & {\bf 0} \end{bmatrix} = r(\widetilde{ {\bf A}}) + n.$

${\rm(e)}$ ${\rm BLUE}_{ {\mathscr N}_b}({\bf K} \pmb{\beta})$ is unique if and only if $\begin{bmatrix} {\bf y} \\ {\bf b} \end{bmatrix}\in {\mathscr R}\begin{bmatrix} {\bf X} & \widetilde{ \pmb{\Sigma}} \\ \widetilde{ {\bf A}} & {\bf 0} \end{bmatrix}$ holds with probability $1.$

Proof. According to Lemma 2.6, the BLUE of ${\bf K} {\bf F}_{\widetilde{ {\bf A}}} \pmb{\gamma}$ under (5.3) is given by

Substitution of this BLUE into the equality in (5.2) gives the BLUE of ${\bf K} \pmb{\beta}$ under (3.4)

as required for (5.6).

Result (a) follows from Lemma 2.6(a). Result (b) follows from Lemma 2.6(b). Result (c) follows from (5.6).

It can be seen from (5.6) that ${\bf P}_{ {\bf K} {\bf F}_{\widetilde{ {\bf A}}}; {\bf X} {\bf F}_{\widetilde{ {\bf A}}}; \widetilde{ \pmb{\Sigma}}}$ is unique if and only if ${\bf E}_{[\, {\bf X} {\bf F}_{\widetilde{ {\bf A}}}, \, \widetilde{ \pmb{\Sigma}} {\bf E}_{ {\bf X} {\bf F}_{\widetilde{ {\bf A}}}}\, ]} = {\bf 0}$, i.e., $r[\, {\bf X} {\bf F}_{\widetilde{ {\bf A}}}, \, \widetilde{ \pmb{\Sigma}} {\bf E}_{ {\bf X} {\bf F}_{\widetilde{ {\bf A}}}}\, ] = n$. Also see from (a) and Lemma 2.1(b) that $r[\, {\bf X} {\bf F}_{\widetilde{ {\bf A}}}, \, \widetilde{ \pmb{\Sigma}} {\bf E}_{ {\bf X} {\bf F}_{\widetilde{ {\bf A}}}}\, ] = r[\, {\bf X} {\bf F}_{\widetilde{ {\bf A}}}, \, \widetilde{ \pmb{\Sigma}}\, ]$ $= r\begin{bmatrix} {\bf X} & \widetilde{ \pmb{\Sigma}} \\ \widetilde{ {\bf A}} & {\bf 0} \end{bmatrix} - r(\widetilde{ {\bf A}})$, so that Result (d) follows.

It can be seen from (5.6) and that ${\rm BLUE}_{ {\mathscr N}_b}({\bf K} \pmb{\beta})$ is unique if and only if ${\bf E}_{[\, {\bf X} {\bf F}_{\widetilde{ {\bf A}}}, \, \widetilde{ \pmb{\Sigma}} {\bf E}_{ {\bf X} {\bf F}_{\widetilde{ {\bf A}}}}\, ]}({\bf y} - {\bf X} \widetilde{ {\bf A}}^{+} {\bf b}) = {\bf 0},$ i.e.,

holds with probability 1 by Lemma 2.1(c). In this situation, it is necessary to simplify the rank equality by removing the generalized inverses on both sides of (5.7). In fact by Lemma 2.1(b) and elementary block matrix operations,

So that (5.7) is equivalent to $r\begin{bmatrix} {\bf y} & {\bf X} & \widetilde{ \pmb{\Sigma}} \\ {\bf b} & \widetilde{ {\bf A}} & {\bf 0} \end{bmatrix} = r\begin{bmatrix} {\bf X} & \widetilde{ \pmb{\Sigma}} \\ \widetilde{ {\bf A}} & {\bf 0} \end{bmatrix}$, i.e.$,$ $\begin{bmatrix} {\bf y} \\ {\bf b} \end{bmatrix}\in {\mathscr R}\begin{bmatrix} {\bf X} & \widetilde{ \pmb{\Sigma}} \\ \widetilde{ {\bf A}} & {\bf 0} \end{bmatrix}$ holds by Lemma 2.1(c), as required for Result (e).□

6.   Conclusions

The author proposed and investigated some estimation and inference problems regarding a linear statistical model with adding-up restrictions to the observable random variables in the model, and obtained a group of formulas and facts about estimations and inferences in the context of (1.3), including the computational processes to derive the OLSEs and BLUEs under the model assumptions via a series of careful algebraic operations of the given vectors and matrices. Based on the findings obtained in the preceding sections, there is no doubt to say that this approach clearly demonstrates a normal procedure of dealing with adding-up restrictions in the context of linear statistical models. Hopefully, this study will add to what is out there with regard to the subject, and also is compatible with the previous contributions.

Given the resolutions to the adding-up restrictions, we may say some additional remarks pertaining to further research problems under the model assumption. Recall that the OLSEs and BLUEs are defined by different optimality criteria in mathematics and statistics. Therefore, their expressions and properties are not necessarily the same, and thereby it is natural to seek possible connections between these estimation results. It is, in fact, a subject area in regression analysis is to characterize relationships between different OLSEs and BLUEs, which has deep roots with strong statistical explanation and usefulness in the domain of linear statistical models and applications; see, e.g., ^{[6,15,16,20,21]} and references therein for the background and study of this subject. Based on the exact and analytical expressions of OLSEs and BLUEs obtained, we can consider in depth various additional topics in the statistical inference of general linear statistical models with adding-up restrictions. Particularly, it is natural to speculate on the relationship between the OLSEs and BLUEs under the two models in (1.1) and (1.3), and therefore we put forward the following five clear and reasonable equalities between the OLSEs and BLUEs under the two models in (1.1) and (1.3):

${\rm (a)}$ ${\rm OLSE}_{ {\mathscr M}}({\bf K} \pmb{\beta}) = {\rm OLSE}_{ {\mathscr N}}({\bf K} \pmb{\beta})$,

${\rm (b)}$ ${\rm OLSE}_{ {\mathscr M}}({\bf K} \pmb{\beta}) = {\rm BLUE}_{ {\mathscr N}}({\bf K} \pmb{\beta})$,

${\rm (c)}$ ${\rm BLUE}_{ {\mathscr M}}({\bf K} \pmb{\beta}) = {\rm OLSE}_{ {\mathscr N}}({\bf K} \pmb{\beta})$,

${\rm (d)}$ ${\rm BLUE}_{ {\mathscr M}}({\bf K} \pmb{\beta}) = {\rm BLUE}_{ {\mathscr N}}({\bf K} \pmb{\beta})$,

${\rm (e)}$ ${\rm OLSE}_{ {\mathscr N}}({\bf K} \pmb{\beta}) = {\rm BLUE}_{ {\mathscr N}}({\bf K} \pmb{\beta})$.

The five equalities describe the direct relevances of the four estimators under the two models, and thus it would be of interest to make a deep-going study of the equalities from theoretical and practical points of view. As a matter of fact, these kinds of equalities were properly considered with clear objective intention in the statistical inference of linear statistical models in the past several decades. Unquestionably, the equalities in (a)–(e) not only can be classified as certain statistical inference problems, but also can be alternatively converted to certain matrix equality problems in term of the clear and analytical expressions of the OLSEs and BLUEs obtained in the preceding sections. In order to resolve these proposed equivalence problems, we need to prepare a complex of preliminary methods and techniques in matrix algebra, including many formulas and facts about generalized inverses of matrices and the matrix rank methodology. Nevertheless, this sort of future investigations together with the current theoretical and methodological advances in this research area will definitely provide considerable insight into the intrinsic natures hidden behind the adding-up restrictions, so that we believe that the algebraic treatments presented in this article will sufficiently prompt other similar studies regarding different kinds of regression models with adding-up restrictions to observable random variables under various assumptions. The five equalities describe the direct relevances of the four estimators under the two models, and thus it would be of interest to make a deep-going study of the equalities from theoretical and practical points of view. As a matter of fact, these kinds of equalities were properly considered with clear objective intention in the statistical inference of linear statistical models in the past several decades. Unquestionably, the equalities in (a)–(e) not only can be classified as certain statistical inference problems, but also can be alternatively converted to certain matrix equality problems in term of the clear and analytical expressions of the OLSEs and BLUEs obtained in the preceding sections. In order to resolve these proposed equivalence problems, we need to prepare a complex of preliminary methods and techniques in matrix algebra, including many formulas and facts about generalized inverses of matrices and the matrix rank methodology. Nevertheless, this sort of future investigations together with the current theoretical and methodological advances in this research area will definitely provide considerable insight into the intrinsic natures hidden behind the adding-up restrictions, so that we believe that the algebraic treatments presented in this article will sufficiently prompt other similar studies regarding different kinds of regression models with adding-up restrictions to observable random variables under various assumptions.

Acknowledgments

The author would like to express his sincere thanks to the handling editor and anonymous reviewers for their helpful comments and suggestions.

Conflict of interest

The author declares that there is no competing interest.