Condition numbers of the generalized ridge regression and its statistical estimation

Jing Kong; Shaoxin Wang; Jing Kong; Shaoxin Wang

doi:10.3934/math.2024205

AIMS Mathematics

2024, Volume 9, Issue 2: 4178-4193. doi: 10.3934/math.2024205

Previous Article Next Article

Research article Special Issues

Condition numbers of the generalized ridge regression and its statistical estimation

Jing Kong ¹,
Shaoxin Wang ^{2
,
,}

1.
Qufu Mingde School, Qufu 273165, China
2.
School of Statistics and Data Science, Qufu Normal University, Qufu 273165, China

Received: 28 November 2023 Revised: 31 December 2023 Accepted: 10 January 2024 Published: 15 January 2024
MSC : 15A12, 15A60, 65F35

In this paper, we considered the condition number theory of a new generalized ridge regression model. The explicit expressions of different types of condition numbers were derived to measure the ill-conditionness of the generalized ridge regression problem with respect to different circumstances. To overcome the computational difficulty of computing the exact value of the condition number, we employed the statistical condition estimation theory to design efficient condition number estimators, and the numerical examples were also given to illustrate its efficiency.
Citation: Jing Kong, Shaoxin Wang. Condition numbers of the generalized ridge regression and its statistical estimation[J]. AIMS Mathematics, 2024, 9(2): 4178-4193. doi: 10.3934/math.2024205

Related Papers:

Abstract

In this paper, we considered the condition number theory of a new generalized ridge regression model. The explicit expressions of different types of condition numbers were derived to measure the ill-conditionness of the generalized ridge regression problem with respect to different circumstances. To overcome the computational difficulty of computing the exact value of the condition number, we employed the statistical condition estimation theory to design efficient condition number estimators, and the numerical examples were also given to illustrate its efficiency.

References

[1]	A. Hoerl, R. Kennard, Ridge regression: biased estimation for nonorthogonal problems, Technometrics, 12 (1970), 55–67. https://doi.org/10.1080/00401706.1970.10488634 doi: 10.1080/00401706.1970.10488634
[2]	P. Hansen, Rank-deficient and discrete ill-posed problems: numerical aspects of linear inversion, Philadelphia: Society for Industrial and Applied Mathematics, 1998. https://doi.org/10.1137/1.9780898719697
[3]	W. van Wieringen, Lecture notes on ridge regression, arXiv: 1509.09169.
[4]	G. Golub, C. van Loan, Matrix computations, 4 Eds., Baltimore: Johns Hopkins University Press, 2013.
[5]	J. Rice, A theory of condition, SIAM J. Numer. Anal., 3 (1966), 287–310. https://doi.org/10.1137/0703023
[6]	S. Wang, L. Meng, A contribution to the conditioning theory of the indefinite least squares problems, Appl. Numer. Math., 177 (2022), 137–159. https://doi.org/10.1016/j.apnum.2022.03.012 doi: 10.1016/j.apnum.2022.03.012
[7]	S. Wang, H. Yang, Conditioning theory of the equality constrained quadratic programming and its applications, Linear Multilinear A., 69 (2021), 1161–1183. https://doi.org/10.1080/03081087.2019.1623858 doi: 10.1080/03081087.2019.1623858
[8]	Z. Xie, W. Li, X. Jin, On condition numbers for the canonical generalized polar decompostion of real matrices, Electron. J. Linear Al., 26 (2013), 842–857. https://doi.org/10.13001/1081-3810.1691 doi: 10.13001/1081-3810.1691
[9]	I. Gohberg, I. Koltracht, Mixed, componentwise, and structured condition numbers, SIAM J. Matrix Anal. Appl., 14 (1993), 688–704. https://doi.org/10.1137/0614049 doi: 10.1137/0614049
[10]	F. Cucker, H. Diao, Y. Wei, On mixed and componentwise condition numbers for Moore-Penrose inverse and linear least squares problems, Math. Comp., 76 (2007), 947–963. https://doi.org/10.1090/S0025-5718-06-01913-2 doi: 10.1090/S0025-5718-06-01913-2
[11]	Y. Wei, D. Wang, Condition numbers and perturbation of the weighted Moore-Penrose inverse and weighted linear least squares problem, Appl. Math. Comput., 145 (2003), 45–58. https://doi.org/10.1016/S0096-3003(02)00437-X doi: 10.1016/S0096-3003(02)00437-X
[12]	D. Chu, L. Lin, R. Tan, Y. Wei, Condition numbers and perturbation analysis for the Tikhonov regularization of discrete ill-posed problems, Numer. Linear Algebra, 18 (2011), 87–103. https://doi.org/10.1002/nla.702 doi: 10.1002/nla.702
[13]	H. Diao, Y. Wei, S. Qiao, Structured condition numbers of structured Tikhonov regularization problem and their estimations, J. Comput. Appl. Math., 308 (2016), 276–300. https://doi.org/10.1016/j.cam.2016.05.023 doi: 10.1016/j.cam.2016.05.023
[14]	L. Meng, B. Zheng, Structured condition numbers for the Tikhonov regularization of discrete ill-posed problems, J. Comput. Math., 35 (2017), 169–186. https://doi.org/10.4208/jcm.1608-m2015-0279 doi: 10.4208/jcm.1608-m2015-0279
[15]	N. Higham, Accuracy and stability of numerical algorithms, 2Eds., Philadelphia: Society for Industrial and Applied Mathematics, 2002. https://doi.org/10.1137/1.9780898718027
[16]	C. Kenney, A. Laub, Small-sample statistical condition estimates for general matrix functions, SIAM J. Sci. Comput., 15 (1994), 36–61. https://doi.org/10.1137/0915003 doi: 10.1137/0915003
[17]	A. Laub, J. Xia, Applications of statistical condition estimation to the solution of linear systems, Numer. Linear Algebra, 15 (2008), 489–513. https://doi.org/10.1002/nla.570 doi: 10.1002/nla.570
[18]	C. Kenney, A. Laub, M. Reese, Statistical condition estimation for linear least squares, SIAM J. Matrix Anal. Appl., 19 (1998), 906–923. https://doi.org/10.1137/S0895479895291935 doi: 10.1137/S0895479895291935
[19]	M. Baboulin, S. Gratton, R. Lacroix, A. Laub, Statistical estimates for the conditioning of linear least squares problems, In: Parallel processing and applied mathematics, Berlin: Springer, 2014,124–133. https://doi.org/10.1007/978-3-642-55224_13
[20]	A. Farooq, M. Samar, Sensitivity analysis for the generalized Cholesky block downdating problem, Linear Multilinear A., 70 (2022), 997–1022. https://doi.org/10.1080/03081087.2020.1751033 doi: 10.1080/03081087.2020.1751033
[21]	A. Farooq, M. Samar, H. Li, C. Mu, Sensitivity analysis for the block Cholesky downdating problem, Int. J. Comput. Math., 97 (2020), 1234–1253. https://doi.org/10.1080/00207160.2019.1613528 doi: 10.1080/00207160.2019.1613528
[22]	A. Laub, J. Xia, Fast condition estimation for a class of structured eigenvalue problems, SIAM J. Matrix Anal. Appl., 30 (2009), 1658–1676. https://doi.org/10.1137/070707713 doi: 10.1137/070707713
[23]	H. Diao, H. Xiang, Y. Wei, Mixed, componentwise condition numbers and small sample statistical condition estimation of Sylvester equations, Numer. Linear Algebra, 19 (2012), 639–654. https://doi.org/10.1002/nla.790 doi: 10.1002/nla.790

Reader Comments

Your name:*

Email:*
© 2024 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)