Hybrid principal component regression estimation in linear regression

Jian-Ying Rong; Xu-Qing Liu; Jian-Ying Rong; Xu-Qing Liu

doi:10.3934/era.2024171

Electronic Research Archive

2024, Volume 32, Issue 6: 3758-3776. doi: 10.3934/era.2024171

Previous Article Next Article

Research article

Hybrid principal component regression estimation in linear regression

Jian-Ying Rong ^{1
,
,},
Xu-Qing Liu ^{2
,
,}

1.
Department of Quality Education, Jiangsu Vocational College of Electronics and Information, Huai'an 223003, China
2.
Faculty of Mathematics and Physics, Huaiyin Institute of Technology, Huai'an 223003, China

Received: 29 January 2024 Revised: 27 May 2024 Accepted: 27 May 2024 Published: 07 June 2024

In this paper, the principal component regression (PCR) estimators for regression parameters were studied in a linear regression model. After discussing the advantages and disadvantages of the classical PCR, we put forward three versions of hybrid PCR estimators. For the first two versions, we obtained the corresponding optimal solutions under the prediction error sum of squares (PRESS) criterion, while for the last one we offered two methods for obtaining the solution. In order to examine their practicality and generalizability, we considered two real-world examples and conducted a simulation study, which took into account varying degrees of multicollinearity. The numerical experiment revealed that the new estimators could substantially improve the least squares (LS) and classical PCR estimators under the PRESS criterion.
- hybrid PCR,
- linear regression,
- PCR,
- weighted PCR (WPCR),
- WPCR with nonnegative weights
Citation: Jian-Ying Rong, Xu-Qing Liu. Hybrid principal component regression estimation in linear regression[J]. Electronic Research Archive, 2024, 32(6): 3758-3776. doi: 10.3934/era.2024171

Related Papers:

Abstract

In this paper, the principal component regression (PCR) estimators for regression parameters were studied in a linear regression model. After discussing the advantages and disadvantages of the classical PCR, we put forward three versions of hybrid PCR estimators. For the first two versions, we obtained the corresponding optimal solutions under the prediction error sum of squares (PRESS) criterion, while for the last one we offered two methods for obtaining the solution. In order to examine their practicality and generalizability, we considered two real-world examples and conducted a simulation study, which took into account varying degrees of multicollinearity. The numerical experiment revealed that the new estimators could substantially improve the least squares (LS) and classical PCR estimators under the PRESS criterion.

References

[1]	S. Wang, Theory of Linear Models and its Applications, Anhui Education Press, Hefei, 1987.
[2]	R. H. Myers, Classical and Modern Regression with Application, 2nd edition, Higher Education Press, Beijing, 2005.
[3]	C. R. Rao, H. Toutenburg, Linear Models: Least Squares and Alternatives, Springer, New York, 1995. https://doi.org/10.1007/978-1-4899-0024-1_2
[4]	A. E. Hoerl, R. W. Kennard, Ridge regression: biased estimation for nonorthogonal problems, Technometrics, 12 (1970), 55–67. https://doi.org/10.1080/00401706.1970.10488634 doi: 10.1080/00401706.1970.10488634
[5]	A. E. Hoerl, R. W. Kennard, Ridge regression: application for non-orthogonal problems, Technometrics, 12 (1970), 69–82. https://doi.org/10.1080/00401706.1970.10488635 doi: 10.1080/00401706.1970.10488635
[6]	X. Q. Liu, F. Gao, Linearized ridge regression estimator in linear regression, Commun. Stat.-Theor. M., 40 (2011), 2182–2192. https://doi.org/10.1080/03610921003746693 doi: 10.1080/03610921003746693
[7]	X. Q. Liu, F. Gao, Z. F. Yu, Improved ridge estimators in a linear regression model, J. Appl. Stat., 40 (2013), 209–220. https://doi.org/10.1080/02664763.2012.740623 doi: 10.1080/02664763.2012.740623
[8]	X. Q. Liu, P. Hu, General ridge predictors in a mixed linear model, Statistics, 47 (2013), 363–378. https://doi.org/10.1080/02331888.2011.592190 doi: 10.1080/02331888.2011.592190
[9]	X. Q. Liu, H. Y. Jiang, Optimal generalized ridge estimator under the generalized cross-validation criterion in linear regression, Linear Algebra Appl., 436 (2012), 1238–1245. https://doi.org/10.1016/j.laa.2011.08.032 doi: 10.1016/j.laa.2011.08.032
[10]	A. F. Lukman, K. Ayinde, B. M. G. Kibria, E. T. Adewuyi, Modified ridge-type estimator for the gamma regression model, Commun. Stat.-Simul. C., 51 (2022), 5009–5023. https://doi.org/10.1080/03610918.2020.1752720 doi: 10.1080/03610918.2020.1752720
[11]	B. M. G. Kibria, More than hundred (100) estimators for estimating the shrinkage parameter in a linear and generalized linear ridge regression models, J. Economet. Stat., 2 (2022), 233–252.
[12]	Z. Y. Algamal, A. F. Lukman, B. M. G. Kibria, A. O. Taofik, Modified jackknifed ridge estimator in bell regression model: theory, simulation and applications, Iraqi J. Comput. Sci. Math., 4 (2023), 146–154. https://doi.org/10.52866/ijcsm.2023.01.01.0012 doi: 10.52866/ijcsm.2023.01.01.0012
[13]	W. F. Massy, Principal components regression in exploratory statistical research, J. Am. Stat. Assoc., 60 (1965), 234–256. https://doi.org/10.1080/01621459.1965.10480787 doi: 10.1080/01621459.1965.10480787
[14]	K. J. Liu, A new class of biased estimate in linear regression. Commun. Stat.- Theor. M., 22 (1993), 393–402. https://doi.org/10.1080/03610929308831027 doi: 10.1080/03610929308831027
[15]	K. J. Liu, Using Liu-type estimator to combat collinearity. Commun. Stat.- Theor. M., 32 (2003), 1009–1020. https://doi.org/10.1081/STA-120019959 doi: 10.1081/STA-120019959
[16]	X. Q. Liu, Improved Liu estimator in a linear regression model, J. Stat. Plan. Infer., 141 (2011), 189–196. https://doi.org/10.1016/j.jspi.2010.05.030 doi: 10.1016/j.jspi.2010.05.030
[17]	J. Y. Rong, Adjustive Liu-type estimators in linear regression models, Commun. Stat.-Simul. C., 39 (2010), 1162–1173. https://doi.org/10.1080/03610918.2010.484120 doi: 10.1080/03610918.2010.484120
[18]	A. Ullah, S. Ullah, Double k-class estimators of coefficients in linear regression, Econometrica, 46 (1978), 705–722. https://doi.org/10.2307/1914242 doi: 10.2307/1914242
[19]	D. M. Allen, Mean square error of prediction as a criterion for selection of variables, Technometrics, 13 (1971), 469–475. https://doi.org/10.1080/00401706.1971.10488811 doi: 10.1080/00401706.1971.10488811
[20]	D. M. Allen, The relationship between variable selection and data augmentation and a method for prediction., Technometrics, 16 (1974), 125–127. https://doi.org/10.1080/00401706.1974.10489157 doi: 10.1080/00401706.1974.10489157
[21]	X. Q. Liu, J. Y. Rong, X. Y. Liu, Best linear unbiased prediction for linear combinations in general mixed linear models. J. Multivariate Anal., 99 (2008), 1503–1517. https://doi.org/10.1016/j.jmva.2008.01.004 doi: 10.1016/j.jmva.2008.01.004
[22]	H. Woods, H. H. Steinour, H. R. Starke, Effect of composition of Portland cement on heat evolved during hardening. Ind. Eng. Chem., 24 (1932), 1207–1214. https://doi.org/10.1021/ie50275a002 doi: 10.1021/ie50275a002
[23]	T. Kunugi, T. Tamura, T. Naito, New acetylene process uses hydrogen dilution, Chem. Eng. Prog., 57 (1961), 43–49.
[24]	G. C. McDonald, D. I. Galarneau, A Monte Carlo evaluation of some ridge-type estimators, J. Am. Stat. Assoc., 70 (1975), 407–416. https://doi.org/10.1080/01621459.1975.10479882 doi: 10.1080/01621459.1975.10479882

Reader Comments

Your name:*

Email:*
© 2024 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)