Investigating difficulties and enhancing understanding in linear algebra: Leveraging SageMath and ChatGPT for (orthogonal) diagonalization and singular value decomposition

1.   Introduction

Linear algebra is a branch of mathematics that deals with linear equations, linear functions, and their representations in vectors and matrices. This involves the study of vector spaces, linear transformations, matrices, determinants, eigenvalues, and eigenvectors. Linear algebra is a fundamental tool in many areas of mathematics, including geometry, calculus, optimization, and numerical analysis. It also has practical applications in various fields such as physics, engineering, computer science, and economics ^{[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20]}.

Similar to other fields of mathematics, algorithmic thinking in linear algebra is a crucial skill for understanding and mastering the topics. Often also called computational thinking, algorithmic thinking is the process of breaking down complex mathematical problems into smaller, more manageable parts, and then solving them systematically using a sequence of steps or algorithms. This process involves identifying patterns, constructing algorithms, and developing logical and analytical skills to solve problems ^{[21,22,23,24]}. Teaching and learning linear algebra for both mathematics and non-mathematics majors provides an opportunity for cultivating and embracing problem-solving, logical reasoning, and computational thinking skills, which are essential in various areas of science and engineering requiring solid knowledge of linear algebra, such as operational research, computer science, data science, and machine learning, among others ^{[3,20,25,26,27,28,29,30,31,32,33]}. Strengthening computational thinking among future teachers will also be useful when they eventually train future student generations ^[34,35].

In addition to being able to manipulate a matrix algebraically and perform matrix operations, some essential concepts in linear algebra include but are not limited to understanding vector spaces and subspaces, applying linear transformation, working with norms and inner products, finding least-squares solutions, and the ability to perform matrix factorization. In this study, we focused on the latter. Occasionally also called decomposition, factorization in linear algebra refers to the process of decomposing a matrix into several, often simpler, matrices that can be analyzed and manipulated more easily. These simpler matrices may have special properties or structures that make them easier to analyze or implement in computations. Mastering algorithmic thinking in performing matrix factorization is essential for every linear algebra learner because this topic not only requires other basic concepts in linear algebra, but it is also an essential building block for understanding other more complicated topics in linear algebra.

There are various types of matrix decompositions, such as LU decomposition, QR decomposition, diagonalization, orthogonal diagonalization, singular value decomposition (SVD), and eigendecomposition. The latter refers to the factorization of a matrix into its canonical form, where the matrix is represented by its eigenvalues and eigenvectors. Each type of decomposition has its own specific properties and applications. For example, LU decomposition is useful for solving systems of linear equations, QR decomposition is used for least-squares problems, SVD is used for data compression and feature extraction, and eigendecomposition is used for analyzing the behavior of linear operators. Although matrix diagonalization only applies to square matrices and orthogonal diagonalization only applies to a special class of symmetric square matrices, SVD can be implemented for any size of the rectangular matrix, which does not necessarily have to be square.

In short, SVD is a factorization method in linear algebra that decomposes a real- or complex-valued matrix into three components: a diagonal matrix of singular values and two unitary (orthogonal) matrices. Mathematical applications of SVD encompass the calculation of the (Moore-Penrose) pseudoinverse, approximating a matrix, and determining the rank, range, and null space of a matrix. SVD is a widely utilized technique in data analysis and machine learning for reducing the dimensionality of data and extracting important features. SVD is also used in various applications in science and engineering, including signal and image processing, text mining, data least-squares fitting, process control, and recommendation systems ^{[36,37,38,39,40,41,42,43,44,45,46]}. The singular values obtained from SVD represent the importance of each feature in the original data and can be used to reconstruct the data with different levels of accuracy.

A common curriculum in linear algebra outlines that students ought to know how to calculate both matrix diagonalization and orthogonal diagonalization before being introduced to SVD. However, we observed that students often encountered some difficulties with these decompositions. For orthogonal diagonalization, several students often forget to normalize the associated eigenvectors to become unit vectors. In other cases, when one eigenvalue yields two associated eigenvectors, some students also forget to transform the set of eigenvectors into an orthonormal set, which can be easily done by orthogonal projection via the Gram-Schmidt process, keeping one of them while projecting the other.

For SVD, a common mistake is in finding the singular values of a matrix. Let $A$ be a matrix. Instead of finding the eigenvalues of $A^T A$, some students calculated the eigenvalues of $A$, took their square root, and designated them as singular values. Another difficulty occurs when the matrix does not have a full rank, for which the students need to find the missing one or more orthonormal eigenvectors in one of the matrices by implementing the orthogonal property or cross product vector operation. In the absence of these eigenvectors, the resulting factorization is called "reduced SVD" instead of (full) SVD.

To the best of our knowledge, this issue of common mistakes and learners' struggles in understanding (orthogonal) diagonalization and SVD has not been fully addressed in the body of published literature. The closest article that is tangentially related to our study is Yildiz Ulus' (2013) study, in which the author investigated teaching diagonalization using advanced calculators and observed that such technological tools are beneficial for learners' acquisition of algorithmic mathematical knowledge for this particular topic of linear algebra ^[47].

Although Lazar (2012) argued in his master's thesis that a solid understanding of fundamental concepts in linear algebra is essential for mastering SVD, the author did not inquire the participants in his study regarding the difficulty or common mistakes they encountered when studying the topic ^[48]. Finally, Zandieh et al. (2017) explored student learning in linear algebra, where they touched briefly on symbolizing the diagonalization equation $A = PDP^{-1}$ but did not discuss students' difficulty in acquiring algorithmic thinking skills ^[49].

This study attempts to fill this gap by investigating common mistakes and challenges when students learn linear algebra, particularly (orthogonal) diagonalization and SVD. Furthermore, because teaching and learning in contemporary mathematics cannot be dismantled by utilizing technological tools, we are also interested in investigating whether computer software or another newly arrived technology can assist and enhance students' learning, instead of disrupting and marring it. The scope of our computer algebra system (CAS) is SageMath, which has a static nature, and we will consider ChatGPT as an artificial intelligence (AI)-assisted tool, which features dynamic interaction output. In the following paragraphs, we provide a brief overview of the static CAS SageMath and dynamic ChatGPT.

What is a CAS? A CAS is a software program that allows the manipulation and computation of mathematical expressions and symbols, including algebraic equations, calculus, and other mathematical functions. It is designed to perform symbolic manipulation, numerical computations, and graphics, as well as to provide tools for solving equations, manipulating matrices and vectors, and performing other mathematical operations. CASs are commonly used in scientific research, engineering, and education, and they can be used to solve complex mathematical problems that may be too difficult or time-consuming to solve by hand ^{[50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65]}. Examples of popular CASs include (wx)Maxima ^{[66,67,68,69,70]}, Wolfram Mathematica ^{[71,72,73,74,75,76]}, Maplesoft Maple ^{[77,78,79,80,81,82]}, and MathWorks MATLAB ^{[83,84,85,86,87]}.

What is SageMath? SageMath (also known as Sage) is a free open-source mathematical software system that uses Python as its primary programming language. It aims to provide an alternative to commercial mathematical software systems, such as Wolfram Mathematica and Maplesoft Maple, while also providing an interface to other popular mathematics software systems, such as MathWorks MATLAB and GAP, the latter being a system for computational discrete algebra. SageMath has a wide range of capabilities, including algebraic and numerical computations, graphics, symbolic manipulation, and combinatorics. Its development is community-driven and it has been released under the GNU General Public License ^{[89,90,91,92,93]}.

Remarkably, SageMath offers robust support not only for linear algebra but also for a wide range of related subjects, boasting numerous built-in functions and capabilities dedicated to matrix operations, vector manipulations, and linear transformations. SageMath provides a variety of algorithms and methods for solving linear algebra problems, such as performing matrix operations, solving linear systems of equations, finding eigenvalues and eigenvectors, and computing matrix decompositions (e.g., LU, QR, SVD). SageMath is a powerful tool for symbolic linear algebra, which allows exact computations with variables and expressions. Despite being static in nature, we are convinced that SageMath can be useful for teaching and learning linear algebra, not only in terms of assisting computational tasks but also with regard to check whether our hand calculations were correct. In addition, the time saved on computations can be channeled toward other purposes, such as understanding deeper mathematical concepts in linear algebra or exploring various problems within the subject matter.

What is ChatGPT? The Chat Generative Pre-trained Transformer, that is, ChatGPT, is a language model developed by OpenAI, a USA-based AI research laboratory consisting of a team of researchers and engineers dedicated to creating safe and beneficial AI. Although OpenAI was founded in 2015, ChatGPT, as one of its products, was launched as a prototype in November 2022. Amazingly, it reached one million users within five days after its launch. An AI tool built on top of the unsupervised transformer language model GPT-3, ChatGPT was trained on a large dataset of text and could generate responses to questions, write coherent paragraphs, and even conduct a conversation with users. Its purpose is to assist with various tasks, such as answering questions, providing information, and generating text in a conversational manner.

Interestingly, as an AI language model, ChatGPT has been trained on a large corpus of texts, including mathematical concepts and problems. Thus, it is capable of solving mathematics problems, at least that is what it claims. ChatGPT admits that its ability to solve mathematical problems may be limited by its training data, although it can certainly provide assistance and guidance on various mathematical topics. Specifically, when asked whether it can solve problems in linear algebra, ChatGPT claimed that it "possesses knowledge and understanding of linear algebra concepts and can provide solutions to problems in this field." During the past few months, the number of published articles related to ChatGPT and its capabilities has increased steadily, including many that appear in various preprint servers. The following examples cover only a few articles on ChatGPT related to mathematics.

Recently, Frieder et al. (2023) investigated the mathematical capabilities of ChatGPT by asking a wide range of questions, and although it understood the question, it often failed to provide correct solutions. They concluded that the mathematical abilities of ChatGPT were significantly lower than those of average mathematics graduate students ^[94]. Shakarian et al. (2023) also evaluated ChatGPT for mathematical word problems and discovered that its performance was dramatically altered based on the requirement to show its work. When the chatbot was not required to reveal the complete solution, it failed 84% of the time, whereas it failed by only 20% when revealing the detailed solution was requested ^[95]. Azaria (2022) tested the numerical literacy of ChatGPT and noticed that when it comes to using number digits, ChatGPT is rather biased, with $7$ being the most frequent digit generated by the machine, which also turns out to be many people's favorite number ^[96]. Borji (2023) focused on ChatGPT's failures in mathematics, including in areas of arithmetic, logic, and reasoning ^[97].

Certainly, ChatGPT is not the only chatbot available on the market. There are numerous alternatives to ChatGPT and the AI race among different companies is becoming fierce. Some examples include but are not limited to Google Bard AI, Microsoft Bing Chat, Amazon Codewhisperer, Github Copilot, Chatsonic, Character AI, Quora Poe, etc. Some early findings suggest that when it comes to solving mathematics problems at the high school level, the Vietnamese students still performed better than both ChatGPT and Bing Chat ^[98]. Another study from Vietnam on the mathematics test for its national high school graduation examination indicated that Google Bard's performance was lagging behind its competitors, that is, ChatGPT and Bing Chat ^[99].

In terms of output production, SageMath and other CASs tend to be static, whereas ChatGPT and its competitors are dynamic. The latter can provide a step-by-step explanation of a solution to a particular problem. For matrix factorization in linear algebra, it is essential for learners to understand not only the technical details of a calculation but also to grasp the algorithm behind any particular problem task. Certainly, matrix factorization is a broad topic in itself, and any attempt to discuss other types of matrix factorization should be addressed separately elsewhere. Presently, our focal point for the topic of "matrix factorization" covers only diagonalization, orthogonal diagonalization, and SVD. Based on the literature mentioned earlier in this introduction, we consider the following research questions:

● What are some common mistakes and difficulties that students encounter when learning matrix factorization in linear algebra?

● How can a static CAS such as SageMath assist students in learning linear algebra, particularly in (orthogonal) diagonalization and SVD?

● Can we rely on the dynamic ChatGPT to better understand (orthogonal) diagonalization and SVD?

The remainder of this article is organized as follows. After this introduction, Section 2 features some common mistakes that students often make when learning (orthogonal) diagonalization and SVD. Section 3 continues with SageMath and its ability to assist students in learning (orthogonal) diagonalization and SVD. Section 4 features some examples in which we asked the chatbot to solve problems related to (orthogonal) diagonalization and SVD. We also discuss where ChatGPT makes mistakes and encounters troubles in completing assigned tasks. Section 5 provides some applications of (orthogonal) diagonalization and SVD in life sciences and engineering. Finally, Section 6 discusses the results and concludes the study.

2.   Common mistakes in matrix factorization

2.1.   Matrix diagonalization

In this section, we consider some common mistakes that students encounter when learning to diagonalize both non-symmetric and symmetric matrices. We also consider similar aspects of SVD. For the former, if $A$ is an $n \times n$ square matrix, then $A$ is diagonalizable whenever it possesses $n$ linearly independent eigenvectors. Usually, the students did not encounter difficulties in diagonalizing a non-symmetric diagonalizable matrix provided they understood the procedure for finding it. A typical algorithm starts with finding the eigenvalues and their associated eigenvectors, constructing a diagonal matrix $D$, constructing an invertible matrix $P$, finding its inverse $P^{-1}$, and expressing its diagonalization, that is, $A = P D P^{-1}$. The following example illustrates this algorithm.

Suppose that there exists a $3 \times 3$ matrix $A$ with integer entries, given as follows:

To diagonalize a matrix, we must determine its eigenvalues and their corresponding eigenvectors. The former can be obtained by solving the characteristic equation $\det \left(A - \lambda I \right) = 0$ or $\left(\lambda - 1\right) \left(\lambda - 3 \right)\left(\lambda - 1 \right) = 0$, which gives $\lambda_1 = 1$, $\lambda_2 = 3$, and $\lambda_3 = 1$. Note that some students might attempt to express the characteristic equation in cubic form, that is, $\lambda^3 - 5\lambda^2 + 7\lambda - 3 = 0$, and solve this for $\lambda$. Although in other cases, this might be a necessary step and even inevitable, we do not have to do this particular step in this specific example. Up to this point, we can form a diagonal matrix $D$ by placing the eigenvalues along the diagonal:

However, this was not the only option. Depending on the choice of eigenvalues designated as the first, second, third, and so on, we might obtain a different expression for $D$. Coincidentally, both the algebraic multiplicity and geometric multiplicity for the eigenvalue $\lambda = 1$ from the matrix in this example are equal, that is, two. In this case, we are guaranteed to obtain three linearly independent eigenvectors and thus, the matrix is diagonalizable.

Let $\mathbf{p}_1$, $\mathbf{p}_2$, and $\mathbf{p}_3$ be the corresponding eigenvectors, we found that

An invertible matrix $P$ can then be formed by stacking these eigenvectors as its columns:

Its inverse $P^{-1}$ is given by

Finally, we can write a diagonalization of $A$ by expressing it as $P D P^{-1}$ and verify that the products are reduced to the original matrix $A$:

2.2.   Orthogonal diagonalization

The following example illustrates an orthogonal diagonalization for a symmetric matrix, whereby one of the eigenvalues has both algebraic and geometric multiplicities of two; thus, the resulting eigenvectors are not orthogonal. Consider the following $3 \times 3$ matrix $B$:

This matrix has a characteristic polynomial of $\lambda^3 - 9\lambda^2 - 21\lambda + 245$, which yields three real-valued eigenvalues $\lambda_1 = -5$ and $\lambda_2 = 7 = \lambda_3$ upon solving the characteristic equation. A diagonal matrix $D$ is given by

Let $\mathbf{q}_1$, $\mathbf{q}_2$, and $\mathbf{q}_3$ be the eigenvectors corresponding to these eigenvalues, we have

A common mistake we often encounter is that some students construct an invertible matrix $P$ directly from these eigenvectors without normalizing them, and then they find its inverse by simply transposing it, that is, $P^T = P^{-1}$. However, the correct step would be to construct an orthonormal set of eigenvectors that form the column of $P$. We observe that $\mathbf{q}_1$ is orthogonal to both $\mathbf{q}_2$ and $\mathbf{q}_3$. However, $\mathbf{q}_2$ and $\mathbf{q}_3$ are not orthogonal because they originate from the same eigenspace. Only the eigenvectors associated with the distinct eigenvalues are orthogonal. Applying the Gram-Schmidt process to the set $\left\{\mathbf{q}_2, \mathbf{q}_3\right\}$, we can obtain a new set of orthogonal eigenvectors $\left\{\mathbf{q}_2, \mathbf{q}^{\prime}_3\right\}$, where

Angle brackets denote the usual inner (dot) product. By normalizing these vectors, we obtain an orthonormal set of eigenvectors and construct matrix $P$ accordingly. Because $P$ is now an orthogonal matrix, its inverse is simply its transpose, given as follows:

We can now express an orthogonal diagonalization of $B$ and again confirm that the right-hand side will be reduced to the original matrix upon multiplication and simplification, as shown by the following computaton:

2.3.   SVD

Although matrix diagonalization and orthogonal diagonalization can be implemented in a square matrix, SVD encompasses both the square and rectangular matrices. Let $M$ be an $m \times n$ real-valued matrix with rank $r$; then, an SVD of $M$ is given by $M = U \Sigma V^T$, where $U$ is an $m \times m$ orthogonal matrix, $\Sigma$ is an $m \times n$ rectangular "diagonal" matrix with non-negative real numbers on the diagonal, and $V$ is an $n \times n$ orthogonal matrix. The columns of $U$ and $V$ in such a decomposition are called the left- and right-singular vectors of $M$, respectively. Regarding complex-valued matrices, $U$ and $V$ are complex unitary matrices, and instead of $V^T$, we would have the conjugate transpose of $V$, that is, $V^{\ast}$. In general, $\Sigma$ is composed of matrix blocks and admits the following form:

where $D$ is an $r \times r$ diagonal matrix for $r \leq \min\{m, n\}$. The numbers of (zero) rows and columns in the second row and column of the block are $(m - r)$ and $(n - r)$, respectively. For $r = m$, $r = n$, or $r = m = n$, some or all of the zero blocks would disappear. Furthermore, the diagonal entries of $D$ are the first $r$ singular values of $M$, that is, $\sigma_1 \geq \sigma_2 \geq \cdots \geq \sigma_r > 0$.

Similar to obtaining matrix diagonalization and orthogonal diagonalization, constructing an SVD of a matrix requires a step-by-step algorithm and an understanding of (orthogonal) diagonalization. For square matrices with full rank, many learners usually encounter no difficulty when constructing an SVD of a matrix, although one must be careful when calculating eigenvalues and the corresponding eigenvectors of a symmetric matrix $M^T M$ instead of the original matrix $M$. However, for rectangular matrices or those with rank deficiencies, many students often could not complete the construction of an SVD, partly because one or more singular values can be zero. The following example illustrates the construction of an SVD of a square matrix with rank deficiency. Let

be a matrix with rank $r = 1$; the eigenvalues of $A = M^T M$ are given by $\lambda_1 = 50$ and $\lambda_2 = 0$, which gives only one singular value $\sigma_1 = 5 \sqrt{2}$ because the second eigenvalue is zero. Thus, $D = \sigma_1$ and matrix $\Sigma$ contain one block of zeros in each row and column below and to the right of $D$, respectively, given as follows:

By finding the corresponding eigenvectors of $\lambda_1$ and $\lambda_2$, we obtain $\mathbf{v}_1$ and $\mathbf{v}_2$ and set up an orthogonal matrix $V$:

The first column of matrix $U$, that is, $\mathbf{u}_1$, can be calculated using the following formula:

The second column of matrix $U$, that is, $\mathbf{u}_2$, must be orthonormal to $\mathbf{u}_1$:

An SVD of $M$ can be expressed as follows:

3.   (Orthogonal) diagonalization and SVD using SageMath

To use SageMath online, we can utilize Sage Cell Server or SageMathCell, accessible online at the URL https://sagecell.sagemath.org/. In addition to an open-source, scalable, and easy-to-use web interface for SageMath, this cell server also allows the embedding of SageMath computations into any webpage. Without loading up any program, this is one way to conduct one-off computations using SageMath, with the idea of accessing the computation in the cloud as simply as possible, as long as one has an internet connection.

3.1.   Matrix diagonalization using SageMath

The SageMath commands for a matrix diagonalization discussed in Subsection 2.1 is presented in Appendix 7.1. Observe that the presence of the "print" command might be excessive for first-time readers who are new to SageMath. However, the expressions inside the double quotation marks, as well as an empty vertical spacing, might be omitted if one wishes.

After constructing matrix $A$, we asked SageMath to display it, finding an expression for its characteristic polynomial using "A.charpoly()", calculating its eigenvalues using "A.eigenalues()", and acquiring the corresponding eigenvectors using "A.eigenvectors_right()". The outputs for the first four essential commands are given as follows:

Matrix A =

[1 2 2]

[0 2 1]

[0 1 2]

Characteristic polynomial of A: p(x) = x^3 - 5*x^2 + 7*x - 3

Eigenvalues of A = [3, 1, 1]

Eigenvalue, eigenvector, and geometric multiplicity:

[(3, [(1, 1/2, 1/2)], 1), (1, [(1, 0, 0), (0, 1, -1)], 2)]

Observe that the command "A.eigenvectors_right()" provides information not only about eigenvectors but also on the associated eigenvalue and their geometric multiplicity.

The next step is extracting the eigenvalues of $A$ using the command "A.eivenvalues()[n]", which correspond to $\lambda_{n + 1}$, where $n = 0, 1, 2$. Once we have obtained the eigenvalues, we can construct a diagonal matrix $D$ manually by inserting each value of $\lambda_n$. Alternatively, we can also construct $D$ directly using the command "diagonal_matrix(A.eigenvalues())", as shown in our code. The outputs of these two essential commands are given as follows:

Extracting eigenvalues:

lambda1 = 3

lambda2 = 1

lambda3 = 1

Diagonal matrix D =

[3 0 0]

[0 1 0]

[0 0 1]

To construct an invertible matrix $P$, we also need to extract the associated eigenvectors, that is, using the command "A.eigenvectors_right()[m][1]", where $m = 0, 1$, which corresponds to the first and second sets of the outputs in the same command in the absence of both square brackets. The second square bracket [1] indicates the second entry of each output, that is, the eigenvector(s). We can multiply by 2 to obtain the eigenvector corresponding to $\lambda_1 = 3$ with integer entries, which is given by $\mathbf{p}_1 = (2, 1, 1)$. Because the eigenvalues $\lambda_2 = 1 = \lambda_3$ have geometric multiplicities of 2, we again need to extract by adding the brackets [0] and [1], which correspond to $\mathbf{p}_2$ and $\mathbf{p}_3$, respectively. Furthermore, because these vectors appear as row columns, we need to apply a transpose command to construct $P$, that is, "P = matrix([p1, p2, p3]).tranpose()". The outputs for eigenvector extraction and $P$ are given as follows:

Extracting eigenvectors:

p1 = (2, 1, 1)

p2 = (1, 0, 0)

p3 = (0, 1, -1)

Invertible matrix

P = [ 2 1 0]

[ 1 0 1]

[ 1 0 -1]

The final two steps are to determine the inverse matrix $P^{-1}$ and confirm that the diagonalization process is correct. The former can be achieved using the command "P.inverse()", whereas the latter is performed by simply taking the product of $P D P^{-1}$, that is, using the command "P*D*P.inverse()", which should result in product simplification and a return to the original matrix $A$, thus confirming that the diagonalization is indeed correct. The outputs are given as follows:

Inverse of P, P^(-1) =

[ 0 1/2 1/2]

[ 1 -1 -1]

[ 0 1/2 -1/2]

Calculate PDP^(-1) =

[1 2 2]

[0 2 1]

[0 1 2]

= A

3.2.   Orthogonal diagonalization using SageMath

The SageMath commands for an orthogonal diagonalization of the matrix example considered in Subsection 2.2 are given in Appendix 7.2. Similar to the previous example, we use standard SageMath commands to display the matrix, find its characteristic polynomial, compute its eigenvalues, and construct a diagonal matrix $D$. We have the following outputs:

Matrix A =

[ 3 -4 -4]

[-4 3 -4]

[-4 -4 3]

Characteristic polynomial of A: p(x) = x^3 - 9*x^2 - 21*x + 245

Eigenvalues of A = [-5, 7, 7]

Extracting eigenvalues:

lambda1 = -5

lambda2 = 7

lambda3 = 7

Diagonal matrix D =

[-5 0 0]

[ 0 7 0]

[ 0 0 7]

We also calculated the corresponding eigenvectors $\mathbf{q}_1$, $\mathbf{q}_2$, and $\mathbf{q}_3$, and extracted each of them accordingly, that is, using the following commands:

The first square bracket indicates the set of solutions containing the eigenvalue, eigenvector(s), and geometric multiplicity. Because there are two distinct eigenvalues, the entry in the first set of square brackets only takes a value of either 0 or 1. The entry inside the second set of square brackets is always 1, because we would like to extract the eigenvector. The entry in the third set of square brackets indicates which eigenvector to extract. For the first case, there is only one; thus, the value is 0. For the second case, because there are two eigenvectors, the values 0 and 1 are taken for the second and third eigenvectors, respectively. We have the following outputs:

Eigenvalue, eigenvector, and geometric multiplicity:

[(-5, [(1, 1, 1)], 1), (7, [(1, 0, -1), (0, 1, -1)], 2)]

Extracting eigenvectors:

q1 = (1, 1, 1)

q2 = (1, 0, -1)

q3 = (0, 1, -1)

However, before constructing an orthogonal matrix $P$, we must ensure that the set of eigenvectors is orthonormal. To check the orthogonality between vectors $\mathbf{q}_m$ and $\mathbf{q}_n$, we use the command

where $m, n = 1, 2, 3$ and $m \neq n$. If the result is zero, then both vectors are orthogonal to each other; otherwise, they are not orthogonal. We observed that $\mathbf{q}_2$ and $\mathbf{q}_3$ are not orthogonal. To obtain an orthogonal matrix, we apply the Gram-Schmidt process, using the command

After obtaining the new eigenvector $\mathbf{q}_3^{\prime}$, we verified that it is now orthogonal to $\mathbf{q}_2$. We obtain the following outputs:

Checking orthogonality:

<q1, q2> = 0

<q1, q3> = 0

<q2, q3> = 1

q2 and q3 are not orthogonal; apply the Gram-Schmidt process:

q3' = (-1/2, 1, -1/2)

Check that now q2 and q3' are orthogonal: <q2, q3'> = 0

The next step was to obtain an orthonormal set of eigenvectors by normalizing this orthogonal set of vectors. The norm of any vector can be calculated using the command "sqrt(qn.inner_product(qn))". Thus, the unit eigenvectors $\mathbf{p}_n$ can be obtained using the command "qn/sqrt(qn.inner_product(qn))", where in both instances, $n = 1, 2, 3$. The outputs are given as follows:

Normalize all orthogonal eigenvectors:

p1 = (1/3*sqrt(3), 1/3*sqrt(3), 1/3*sqrt(3))

p2 = (1/2*sqrt(2), 0, -1/2*sqrt(2))

p3 = (1/6*sqrt(6), -1/3*sqrt(6), 1/6*sqrt(6))

We can now construct an orthogonal matrix $P$, which is also invertible. The commands are similar to those in the previous example. We also further verified that $P^{-1} = P^T$ and the calculations for both $PDP^{-1}$ and $PDP^T$ reduce to the original matrix $A$, which confirms that an orthogonal diagonalization for $A$ is indeed correct. Readers are presented the following outputs:

Construct an invertible and orthogonal matrix P:

P =

[ 1/3*sqrt(3) 1/2*sqrt(2) 1/6*sqrt(6)]

[ 1/3*sqrt(3) 0 -1/3*sqrt(6)]

[ 1/3*sqrt(3) -1/2*sqrt(2) 1/6*sqrt(6)]

Inverse of P, P^(-1) =

[ 1/3*sqrt(3) 1/3*sqrt(3) 1/3*sqrt(3)]

[ 1/2*sqrt(2) 0 -1/2*sqrt(2)]

[ 1/6*sqrt(6) -1/3*sqrt(6) 1/6*sqrt(6)]

Calculate PDP^(-1) =

[ 3 -4 -4]

[-4 3 -4]

[-4 -4 3]

= A

Transpose of P, P^T =

[ 1/3*sqrt(3) 1/3*sqrt(3) 1/3*sqrt(3)]

[ 1/2*sqrt(2) 0 -1/2*sqrt(2)]

[ 1/6*sqrt(6) -1/3*sqrt(6) 1/6*sqrt(6)]

Calculate PDP^T =

[ 3 -4 -4]

[-4 3 -4]

[-4 -4 3]

= A

3.3.   SVD using SageMath

The SageMath commands for constructing an SVD of the matrix discussed in Subsection 2.3 are displayed in Appendix 7.3.

After constructing matrix $M$, we calculate a symmetric matrix $A$, compute its eigenvalues, take the square root, and obtain the singular values of $M$. This process can be achieved using the following commands:

Matrix $\Sigma$ is constructed manually using the command

We have the following ouputs:

Matrix M

= [ 2 -2]

[-3 -4]

[-4 -3]

Matrix M^T M =

[29 20]

[20 29]

Eigenvalues of M^T M = [49, 9]

Singular value of M:

sigma1 = 7

sigma2 = 3

Matrix Sigma =

[7 0]

[0 3]

[0 0]

We then calculate the eigenvectors of $A$, normalize them, and construct an orthogonal matrix $V$, which can be achieved using the following commands:

We have the following outputs, leading to $V$:

Eigenvalue, eigenvector, and geometric multiplicity:

[(49, [(1, 1)], 1), (9, [(1, -1)], 1)]

Eigenvectors of M^T M:

v1 = (1/2*sqrt(2), 1/2*sqrt(2))

v2 = (1/2*sqrt(2), -1/2*sqrt(2))

Matrix V =

[ 1/2*sqrt(2) 1/2*sqrt(2)]

[ 1/2*sqrt(2) -1/2*sqrt(2)]

To construct an orthogonal matrix $U$, we require three linearly independent orthonormal vectors $\mathbf{u}_1$, $\mathbf{u}_2$, and $\mathbf{u}_3$. In this example, both vectors $\mathbf{u}_1$ and $\mathbf{u}_2$ are calculated using a formula that involves a singular value $\sigma_i$, the original matrix $M$, and the eigenvectors of $A$, that is, $\mathbf{v}_i$, $i = 1, 2$, given by

The SageMath commands for this computation are straightforward:

However, because the third singular value $\sigma_3$ does not exist, we must find $\mathbf{u}_3$ using other techniques. To ensure its orthogonality with $\mathbf{u}_1$ and $\mathbf{u}_2$, $\mathbf{u}_3$ can be calculated using either a dot or cross product operation. In this example, we implemented the latter, that is, "u3 = u1.cross_product(u2)". As always, we should not forget to verify that these vectors are orthogonal, and we can do this by simply calculating the dot product between the two vectors "um.dot_product(un)", where $m, n = 1, 2, 3$, but $m \neq n$. We present the following outputs:

Left singular vectors of M:

u1 = (0, -1/2*sqrt(2), -1/2*sqrt(2))

u2 = (2/3*sqrt(2), 1/6*sqrt(2), -1/6*sqrt(2))

u3 = (1/3, -2/3, 2/3)

Checking orthogonality

u1.u2 = 0

u1.u3 = 0

u2.u3 = 0

Because vectors $\mathbf{u}_i$, $i = 1, 2$, and $3$ appear as row vectors, matrix $U$ is constructed by taking the transpose of these row vectors. Finally, we confirm that the construction of our SVD is correct because the product of each matrix returns to the original matrix $M$, that is, $M = U \Sigma V^T$. The outputs are given as follows:

Matrix U =

[ 0 2/3*sqrt(2) 1/3]

[-1/2*sqrt(2) 1/6*sqrt(2) -2/3]

[-1/2*sqrt(2) -1/6*sqrt(2) 2/3]

SVD of M = U*Sigma*V^T =

[ 2 -2]

[-3 -4]

[-4 -3]

= M

We observed that some students did not know how to find the third vector $\mathbf{u}_3$. Very often, they simply ignored and abandoned it entirely, constructing a factorization using the first two vectors in matrix $U$ instead, that is, $\mathbf{u}_1$ and $\mathbf{u}_2$. In this situation, they were confused between "reduced SVD" and "(complete, non-reduced) SVD". The former depends on the rank of $M$, that is, $r$, and instead of employing $\Sigma$, we admit the diagonal matrix $D$ instead. Thus, the reduced SVD for $M$ in the example considered in this subsection is given by

This reduced SVD is beneficial for calculating the pseudoinverse $M^{+}$ (also called the Moore-Penrose inverse) of the matrix $M$. For the particular $M$ considered in this subsection, its pseudoinverse is given by

SageMath can certainly compute the pseudoinverse of a matrix in a matter of seconds using the command "print(M.pseudoinverse())". However, it is important to exercise discretion and only use this step to check the final calculation, rather than as a substitute for an appropriate algorithm to find it.

4.   (Orthogonal) diagonalization and SVD in ChatGPT

We used the version from March 14, 2023, for any responses generated by ChatGPT in this study.

4.1.   Matrix diagonalization in ChatGPT

When asking the chatbot to provide an example of matrix diagonalization without specifying any size of the matrix, ChatGPT provided an answer by selecting a simple $2 \times 2$ matrix, as illustrated in the following example. We input an inquiry "Provide an example of a matrix diagonalization." ChatGPT responded as follows:

Sure, here's an example of matrix diagonalization:

Suppose we have the following matrix $A$:

We want to diagonalize $A$, which means finding a diagonal matrix $D$ and an invertible matrix $P$ such that $A = PDP^{-1}$, where $D$ has the eigenvalues of $A$ along the diagonal.

To diagonalize $A$, we first find the eigenvalues of $A$. The characteristic polynomial of $A$ is:

Solving for the roots of the characteristic polynomial, we get $\lambda = 3$ and $\lambda = 2$. These are the eigenvalues of $A$.

Next, we find the eigenvectors corresponding to each eigenvalue. For $\lambda = 3$, we solve the equation $\left(A - 3I \right) \mathbf{x} = \mathbf{0}$:

This gives us the equation $x_2 = 0$ and $x_1$ can be any nonzero value. So an eigenvector corresponding to $\lambda = 3$ is $\left[1 \quad 0 \right]^T$.

For $\lambda = 2$, we solve the equation $\left(A - 2I \right) \mathbf{x} = \mathbf{0}$:

This gives us the equation $x_1 + x_2 = 0$, or $x_1 = -x_2$. So an eigenvector corresponding to $\lambda = 2$ is $\left[-1 \quad 1 \right]^T$.

We can normalize these eigenvectors to get:

We can then form the matrix $P$ by stacking the eigenvectors as columns:

We can also form the diagonal matrix $D$ by putting the eigenvalues along the diagonal:

Finally, we can verify that $A = PDP^{-1}$:

which is equal to $A$. So we have successfully diagonalized $A$.

As we observed in this example, ChatGPT came up with an excellent diagonalization example with a step-by-step explanation that can help readers better understand the proper algorithm. Although the calculation for finding $p^{-1}$ was not shown, the machine assumed that the readers have working knowledge of it, particularly for an easy, simple $2 \times 2$ matrix where the computation is relatively straightforward. As some examples in the following subsection illustrate, ChatGPT encountered difficulty in (orthogonally) diagonalizing a larger matrix size, such as a $3 \times 3$ matrix.

4.2.   Orthogonal diagonalization in ChatGPT

When we asked ChatGPT to produce an example of a diagonalization of a $3 \times 3$ symmetric matrix, it gave the following response:

Sure, let's consider the symmetric matrix:

To diagonalize this matrix, we need to find a matrix $P$ and a diagonal matrix $D$ such that:

where $P^T$ denotes the transpose of $P$. First, we need to find the eigenvalues of $A$ by solving the characteristic equation:

where $\lambda$ is the eigenvalue and $I$ is the identity matrix. This gives us:

We observed that ChatGPT made calculation mistakes starting with the second expression, where it wrote $5$ instead of $5 - \lambda$. The subsequent terms, including the final expression for the characteristic equation, are undoubtedly incorrect. These are represented by the underlined boldface terms. The correct characteristic equation is $\lambda^3 - 15 \lambda^2 + 10 \lambda = 0$. Interestingly, when we asked ChatGPT to orthogonally diagonalize the given $3 \times 3$ symmetric matrix $A$ or find its characteristic equation, it still produced an incorrect answer although it differed from the original version. Both queries yielded $\lambda^3 - 15 \lambda^2 + \underline{\mathbf{16 \pmb{\mathsf{ λ}} - 33}} = 0$. In all three cases, both the cubic and quadratic terms were correct, whereas both the linear and constant terms were incorrect, as indicated by the underlined boldface terms.

We continue with what ChatGPT offered after this computational step.

This polynomial has a repeated root at $\lambda = 0$, and distinct roots at $\lambda = 3$ and $\lambda = 12$. Therefore, the eigenvalues of $A$ are $\lambda_1 = 0$, $\lambda_2 = 3$, and $\lambda_3 = 12$. Next, we need to find the eigenvectors of $A$ for each eigenvalue. For $\lambda_1 = 0$, we have:

Reducing this matrix to row echelon form, we get:

which gives us the eigenvector $v_1 = [-2, \ 1, \ 0]$. For $\lambda_2 = 3$, we have:

Reducing this matrix to row echelon form, we get:

which gives us the eigenvector $v_2 = [-1, \ -2, \ 1]$. For $\lambda_3 = 12$, we have:

Reducing this matrix to row echelon form, we get:

which gives us the eigenvector $v_3 = [1, \ 2, \ \underline{\mathbf{1]}}$.

ChatGPT stopped at this stage, even without completing the final component for the third eigenvector, as indicated by the red color. In other instances, ChatGPT could continue the process by constructing a diagonal matrix $D$, an invertible matrix $P$, and finalizing the answer with " Finally, we can check that $A = PDP^{-1}$." However, this did not occur in this example.

Interestingly, however, ChatGPT obtained one correct eigenvalue, that is, $\lambda_1 = 0$, although one entry in the row echelon form matrix was incorrect, that is, $a_{23} = 0$ instead of 1. The correct associated eigenvector is $\mathbf{v}_1 = \left(3, \ 0, \ -1 \right)$. Furthermore, contrary to other positive integer eigenvalues obtained by ChatGPT, $\lambda_2$ and $\lambda_3$ of matrix $A$ in this example are not integers, but irrational numbers instead, although both are still positive. Arranged ascendingly, they are

A possible choice for the corresponding eigenvectors would be

If we use the eigenvalues and associated eigenvectors delivered by ChatGPT to construct the original matrix $A$, the resulting matrix is neither symmetric nor has all integer entries. Instead, we obtain the following matrix:

To further test its capability, we investigated ChatGPT to orthogonally diagonalize a particular $3 \times 3$ symmetric matrix. This is the same matrix $B$ that we explored in Subsection 2.2:

Unfortunately, ChatGPT provided an incorrect answer for this matrix diagonalization. Its computation for the characteristic polynomial was erroneous: $\lambda^3 - 9 \lambda^2 + 24 \lambda$, which results in a bogus set of eigenvalues: $\lambda_1 = 0$, $\lambda_2 = 3$, and $\lambda_3 = 6$. The calculation was continued to seek the corresponding eigenvectors $\mathbf{v}_1$ and $\mathbf{v}_2$, but ChatGPT was unable to complete the calculation for $\mathbf{v}_3$. The computation stopped abruptly with some missing entries in the matrix equation. By further analyzing each step, we discovered that neither the row-reduction process of the augmented matrix for the first and second eigenvalues nor the obtained eigenvector was correct. Requesting ChatGPT to "regenerate response" was also ineffective because it once again produced inaccurate and partial replies. From these examples, we observe that the current version of ChatGPT does not seem ready for the (orthogonal) diagonalization of a simple matrix beyond a size of $2 \times 2$.

4.3.   SVD in ChatGPT

We asked ChatGPT to "provide an example of a singular value decomposition." Interestingly, ChatGPT provided a relatively advanced example of a $3 \times 3$ matrix and utilized the matrix algebra library numpy in Python, which SageMath also employs for as its primary language solver. The chosen matrix is given by

In the previous two cases, ChatGPT explained the algorithm to perform an (orthogonal) diagonalization of a matrix. In this case, it directly provided a Python code and its corresponding output. Furthermore, although the original matrix contains integer entries, the matrices in its SVD consist of real numbers, which can be difficult to check manually.

Unfortunately, ChatGPT got it wrong for its own given example, as shown in Figure 1. It dispensed singular values of $\sigma_1 = 24.57$, $\sigma_2 = 9.49$, and $\sigma_3 = 3.63$, whereas the correct singular values are $\sigma_1 = 25.58$, $\sigma_2 = 11.36$, and $\sigma_3 = 3.13$. Although the error for the first singular value was less than 4%, the errors for the second and third singular values were approximately 16%, which is too large to be considered acceptable. Using the SageMath commands A = matrix(RDF, [[4, 11, 14], [-1, 1, 17], [0, 12, 5]]) and A.SVD(), we obtained the following output:

(

[-0.7020788202506024 0.21713691817311598 -0.6781864706124737]

[-0.5829773893610252 -0.7221727564849129 0.3722954113130941]

[-0.4089287146304667 0.6567481012970376 0.6336081105805151],

[25.575254913906992 0.0 0.0]

[ 0.0 11.363233131054077 0.0]

[ 0.0 0.0 3.1278217497152005],

[-0.08701136700816689 0.13998836518016763 -0.9863226042353253]

[ -0.5166317611363659 0.8401931429966347 0.16482446980522636]

[ -0.8517749969308255 -0.5239071865088518 0.000783918281394777]

),

where the first, second, and third matrices correspond to $U$, $\Sigma$, and $V^T$, respectively. Here, RDF stands for Real Double Field, which is an approximation to a real number using double-precision floating point numbers.

We further asked ChatGPT to "provide an example of an SVD where the matrix has integer singular values." ChatGPT responded with the same wording but chose a different matrix, and this time it was a $4 \times 2$ matrix, given as follows:

However, the result was still incorrect and the computation stopped nearly at the end of the answer with the message "Error in body stream" and an offer to "Regenerate response." See Figure 2. Requesting another response multiple times did not yield any results. For the given matrix, the correct singular values are not integers, although they can be expressed analytically, that is, $\sigma_{1, 2} = \sqrt{9 \pm \sqrt{65}} \approx 4.13$ and $0.97$, respectively.

Using a new chatbot and asking for an identical inquiry, we obtained an improved result, even though there were still some conceptual mistakes in the answer. See Figure 3. Another $3 \times 3$ matrix was selected, but this time it was symmetric:

Instead of calculating singular values, ChatGPT computed the eigenvalues of matrix $A$. Coincidentally, they are identical for this particular example, that is, $\lambda_1 = 5 = \sigma_1$, $\lambda_2 = \lambda_3 = 2 = \sigma_2 = \sigma_3$. This is a rather rare finding, as it occurs only for positive-definite and symmetric (or Hermitian, for complex-valued) matrices. Even though matrices $U$ and $V$ appeared to be correct, matrix $\Sigma$ did not because ChatGPT displayed the square root of the eigenvalues instead of correctly calculating the singular values.

5.   Applications

There are many applications of (orthogonal) diagonalizations and SVD in various scientific, engineering, and computational fields. Their significance provides valuable tools for solving complex problems and understanding the underlying structures of various systems. In this section, we only provide brief coverage of applications in life sciences and engineering.

5.1.   Applications in life sciences

The study of eigenvalues, eigenvectors, and (orthogonal) diagonalization has useful applications in the solutions of systems of ordinary differential equations as well as in discrete dynamical systems, for which such models appear not only in biology but also other branches of science and engineering. In particular, in the field of population ecology and dynamics, with applications such as the predator-prey system ^[17] and competing species model ^[100], mathematical modeling often involves the use of matrices ^[101].

When solving a coupled system of differential equations, diagonalization allows for the transformation of a system of linear differential equations into a set of decoupled equations, which are easier to solve. Observe that the system $\mathbf{x}' = A \mathbf{x}$ admits the solution $\mathbf{x}(t) = \mathbf{x}(0) \, e^{A \, t}$, where $e^{A \, t}$ is the matrix exponential. If $A$ is diagonalizable, that is $A = P D P^{-1}$, then the matrix exponential can be calculated according to

where $e^{D \, t}$ is a diagonal matrix with entries $e^{\lambda_i t}$, $i = 1, 2, \cdots, n$, on its diagonal ^[102].

Matrix diagonalization also finds applications for investigating sex-linked genes, such as recessive color blindness, for which it would be easier to describe the situation mathematically using a matrix model. One such model that groups the population based on sex, that is males and females, can be described by $\mathbf{x}^{(n)} = A^n \, \mathbf{x}^{(0)}$, where $\mathbf{x}^{(n)}$ denotes the proportion of color blindness-related genes in the male and female populations of the $(n + 1)$st generation, and $A$ denotes a coefficient matrix related to the model. To find the proportions of genes for color blindness in a particular population for multiple generations, we need to calculate the limit of $\mathbf{x}^{(n)}$, which also requires calculating the limit of the power matrix on the right-hand side, that is, $A^n$, which will be easier if we know an expression for its diagonalization ^[103].

Diagonalization and SVD play significant roles in biomedical signal processing by enabling dimensionality reduction, noise reduction, feature extraction, and improvement of the signal analysis and interpretation. Diagonalization techniques, such as principal component analysis (PCA), are widely employed to process and analyze biomedical signals, such as EEG (electroencephalogram), ECG (electrocardiogram), and fMRI (functional magnetic resonance imaging) data ^{[104,105,106,107]}. By extracting meaningful features and reducing noise, these methods significantly contribute to diagnosis and our comprehension of brain activity and cardiac patterns. Through these applications, diagonalization plays a vital part in advancing our understanding of biological processes, disease diagnosis, and the development of personalized healthcare solutions.

In the field of evolutionary biology, two crucial symmetric matrices underlie the understanding of microevolutionary change. The first matrix describes the individual fitness surface, referred to as the matrix of nonlinear selection gradients $\gamma$. The second matrix affects a multivariate response to selection, known as the genetic variance-covariance matrix $G$. By applying (orthogonal) diagonalization to both $\gamma$ and $G$, biologists not only gain deeper insights into the form and strength of nonlinear selection, but they also assess the availability of genetic variance for multiple traits. This powerful technique enhances our understanding of evolutionary processes, shedding light on the interplay between selection pressures and genetic variation in shaping the traits of organisms ^[108].

Eigenanalysis and SVD find valuable applications in population genetics, enabling the study of genetic variation and evolutionary processes within populations. SVD, in particular, plays a prominent role in analyzing high-dimensional biological data, such as gene expression profiles, protein-protein interaction networks, and genomic sequencing data. By identifying significant patterns, correlations, and reducing noise, SVD helps extract biologically relevant information from intricate datasets. Additionally, SVD aids in identifying population substructure and detecting signatures of natural selection, further enhancing our understanding of genetic diversity and evolutionary dynamics. Moreover, in the realm of data visualization, SVD proves particularly advantageous in simplifying data complexity and fostering a clearer visualization of biological datasets. This capability empowers researchers to gain valuable insights into the intricate relationships and processes underlying complex biological systems ^{[109,110,111]}.

In summary, both (orthogonal) diagonalization and SVD are powerful and versatile tools with diverse applications in biology and life sciences. Their capacity to analyze, integrate, and extract crucial information from complex datasets plays a pivotal role in advancing our understanding of biological systems, disease mechanisms, and personalized healthcare solutions. These techniques are indispensable in the field of biosciences, making significant contributions to our knowledge and applications across various areas of biology and life sciences. As researchers continue to harness their potential, the impact of orthogonal diagonalization and SVD is likely to extend even further, driving innovation and discoveries in the quest to unravel the complexities of life.

5.2.   Applications in engineering

Diagonalization and SVD play a pivotal role in electrical engineering, particularly in power system analysis, load flow calculations, and fault diagnosis. These techniques empower engineers to efficiently study and optimize the behavior of electrical power systems, allowing them to gain a comprehensive understanding of power grid dynamics and optimize power distribution. By offering valuable insights into stability, dynamic responses, and control strategies, diagonalization and SVD serve as versatile tools that significantly enhance power system analysis. Moreover, they contribute to ensuring the reliable, secure, and efficient operation of electrical power systems, facilitating the seamless integration of renewable energy sources and driving advancements in smart grid technologies. The integration of diagonalization and SVD in electrical engineering proves essential for promoting sustainable and resilient power systems, which will foster a more efficient and eco-friendly energy landscape ^{[112,113,114,115,116]}.

In mechanical engineering, (orthogonal) diagonalization and SVD have strong applications, particularly in mechanical and structural analysis. Diagonalization techniques are employed to analyze the dynamic behavior of mechanical systems, enabling engineers to analyze the vibration modes, modal frequencies, and dynamic responses of such systems. It facilitates understanding the system behavior, structural integrity, structural design optimization, stability, and performance of mechanical components and systems. SVD finds application in structural analysis for the dimensionality reduction of large-scale data, allowing engineers to extract dominant modes and efficiently study complex structures. Additionally, SVD is used in model reduction to create reduced-order models, facilitating faster and more accurate simulations of mechanical systems. These techniques contribute significantly to enhance mechanical engineering design, optimization, and reliability, paving the way for innovative and efficient mechanical systems ^{[117,118,119]}.

(Orthogonal) diagonalization and SVD play a crucial role in control engineering, offering powerful tools to analyze the dynamic behavior of linear systems and design effective controllers. Specifically, diagonalization serves as a key technique for stability analysis in linear control systems. By transforming the system's transfer function into a diagonal form, engineers gain valuable insights into stability and response characteristics, facilitating the design and optimization of control systems. Through diagonalization, engineers can easily determine the system's stability based on the eigenvalues of the diagonal matrix. Moreover, both diagonalization and SVD are indispensable in controller design. By diagonalizing the system's state-space representation, engineers can thoroughly understand the system's dynamics and efficiently optimize controller parameters. This enables the design of feedback controllers that stabilize the system and meet stringent performance specifications, leading to robust and reliable control solutions ^{[120,121,122,123,124,125]}.

(Orthogonal) diagonalization and SVD find diverse and valuable applications in image and signal processing, as we have seen in Subsection 5.1. In image processing, diagonalization techniques are utilized for dimensionality reduction, denoising, and feature extraction. By transforming image data into their principal components, diagonalization enables efficient compression and denoising while preserving essential image features. SVD plays a pivotal role in image compression and data analysis, allowing for efficient representation of images with reduced storage requirements. Moreover, SVD is employed in image and signal denoising, where it separates noise from the underlying signal, improving the quality and clarity of images and signals. Additionally, SVD is used in feature extraction, pattern recognition, and image registration, aiding in tasks such as object detection, face recognition, and medical image alignment. These techniques have proven indispensable in various image and signal processing applications, contributing to advancements in computer vision, medical imaging, multimedia processing, and many other fields ^{[126,127,128,129,130,131,132,133]}.

Linear algebra serves as a vital bridge between theoretical mathematical concepts and their practical applications, particularly in data science and machine learning. A solid understanding of this subject opens doors to comprehending complex machine learning algorithms more effectively. SVD stands out as a prominent technique commonly used for dimensional reduction in data science. Moreover, SVD finds extensive use in matrix factorization and data compression. In collaborative filtering and recommendation systems, SVD uncovers latent features and predicts missing values in sparse datasets, while also contributing to techniques like latent semantic analysis for natural language processing tasks. These techniques greatly aid in data preprocessing, enhancing the performance, interpretability, and scalability of machine learning models, thus making them invaluable tools in diverse applications. Similarly, diagonalization finds application in various machine learning algorithms, enabling PCA for dimensionality reduction. By transforming data into principal components, diagonalization enables efficient data representation and visualization, preserving vital information while reducing computational complexity. Together, these linear algebra techniques play pivotal roles in the success of data science and machine learning, empowering researchers and practitioners to tackle complex real-world problems with greater clarity and efficiency ^{[134,135,136,137,138,139,140,141,142]}.

In summary, mastering (orthogonal) diagonalization and SVD provides students with immense benefits, enabling efficient problem-solving in diverse science and engineering domains. These techniques enhance computational efficiency, reduce data complexity, and provide deeper insights into various scientific and engineering phenomena. By harnessing their power, innovative solutions are unlocked, driving advancements and revolutionizing multiple disciplines. Indeed, (orthogonal) diagonalization and SVD continue to inspire innovation, deepen our understanding of complex phenomena, and pave the way for future advancements in science and engineering.

6.   Discussion and conclusion

This study admits several limitations. First, we only focused on the topics of (orthogonal) diagonalization and SVD, whereas algorithmic thinking in matrix factorization encompasses a broader range of topics. For example, we did not consider decompositions related to solving systems of linear equations, such as LU, QR, and Cholesky decompositions. Other eigenvalue-based decompositions are related to our study, but we did not cover them in this article; they include Jordan, Schur, Takagi, and QZ decompositions, among others. Second, we particularly selected SageMath to obtain a better understanding of matrix factorization instead of another CAS. The main reason is that it is a free and open-source CAS, and as stated on its website, its mission is to "create a viable free open source alternative to Magma, Maple, Mathematica, and Matlab" ^[143]. Additionally, we can perform our computation directly on the server without the hassle of installing the program on our computer, as long as we have an internet connection.

Third, the outcome that ChatGPT presented disappointingly contains numerous mistakes. On the one hand, this can be confusing for many students who are academically weak and just willing to accept what is handed out by the chatbot. On the other hand, this can be an excellent opportunity for further discussion on cultivating critical thinking among learners. Additionally, because our study was conducted at the initial stage of ChatGPT development, the responses considered in this discussion might no longer be relevant in the future, thus making it challenging for other researchers to replicate a similar study. However, we are aware that updated versions will be released in the future and the responses will be improved accordingly. (In fact, at the time of this writing, ChatGPT Plus subscribers can already enjoy the more advanced version of the chatbot, that is the GPT-4 model.) Thus, we can be confident that incorrect answers related to matrix factorization in particular or other mathematics problems in general will eventually be minimized ^[94].

Many mathematics educators and linear algebra instructors have likely observed that while some students grasp the subject with ease, others struggle to establish connections between different concepts, leading to a sense of confusion and disorientation. This feature makes teaching linear algebra challenging for undergraduate students. To investigate the first research question, we identified some common mistakes and difficulties that students encountered in particular topics of linear algebra, that is, (orthogonal) diagonalization and SVD. Indeed, these topics can be challenging for many learners because they require previous, more basic knowledge of linear algebra, such as proficiency in performing elementary row operations and finding orthogonal projections. At the same time, a lack of understanding of matrix diagonalization will lead to more difficulties in the subsequent topics, such as orthogonal diagonalization and SVD, as well as other topics related to diagonalization, which are usually not covered in a standard linear algebra course but have some applications in data science or machine learning.

Learners of linear algebra should be aware of the fact that not all square matrices can be diagonalized. Having a misconception that every matrix can be diagonalized can lead to confusion in subsequent topics. To master computational thinking in the area of solving diagonalization problems, in addition to understanding an algorithmic procedure to diagonalize a particular matrix, students should have an intuitive feeling for less concrete objects, such as eigenvalues and their associated eigenvectors. This also means that they should know how to find the characteristic equation of a matrix, compute the eigenvalues, and find the corresponding eigenvectors via elementary row operations. Learners who struggle with row reduction usually do not go far from figuring out how to express eigenvectors in a simple way. Certainly, those who struggle with this more basic matrix algebra should practice sufficiently until they are comfortable with the computational process.

Another difficulty arises when students learn orthogonal diagonalization for symmetric matrices. Although the procedure is similar to previously learned matrix diagonalization, some students often forget to normalize the obtained eigenvectors and check that these eigenvectors must be orthonormal to each other. The absence of this step and simply establishing that $P^{-1} = P^T$ will not yield a correct orthogonal diagonalization, that is, $PDP^T$ will not return to the original matrix $A$. Further difficulty will occur when one needs to construct a set of orthogonal eigenvectors, whereas the resulting eigenvectors are not orthogonal because they arise from the same eigenspace. Connecting to another concept of vector projection and an orthogonalization procedure of the Gram-Schmidt algorithm requires a solid understanding of related concepts, in addition to proficiency in the calculation of the orthogonal projection itself. By becoming competent in these computational thinking skills, any learner in the field of linear algebra will be able to solve (orthogonal) diagonalization problems.

When learning SVD, a common mistake is the confusion between singular values and eigenvalues. Even if learners are aware of the definition of singular values as the square root of eigenvalues, many of them often forget that they should compute the eigenvalues of the symmetric matrix $M^TM$ instead of the original matrix $M$. A small mistake in this early step of the algorithm will not lead to a correct SVD, even though they understand the procedure—and implement it correctly—required to calculate an SVD of a matrix. Another difficulty is in constructing matrix $\Sigma$, particularly when $M$ is neither square nor possesses a full rank. Indeed, we must correctly determine the number of blocks of zeros that we should embed along the diagonal matrix $D$ to construct $\Sigma$. An easy rule to remember is that $\Sigma$ must be the same size as $M$. If $M$ has more rows than columns, the square orthogonal matrix $U$ will be larger than the square orthogonal matrix $V$, and vice versa. A challenge in constructing $U$ occurs when either $M$ has a deficient rank or $m > n$, where $m$ and $n$ denote the number of rows and columns, respectively. When we attempted to express a decomposition in the absence of the missing eigenvector, we obtained a reduced SVD instead of a full, non-reduced one. Solid knowledge of both inner (dot) and cross products, as well as not forgetting to normalize the vectors, will be sufficient to master algorithmic thinking skills when attempting to find an SVD of a matrix.

This leads us to discuss the second research question, that is, how can a static CAS such as SageMath assist learners in mastering topics in linear algebra that require algorithmic thinking skills, particularly (orthogonal) diagonalization and SVD? Using a CAS for assisting computational processes, particularly in linear algebra, can be a tremendous help for many of us, including both learners and teachers. SageMath in particular, being more powerful than a graphic calculator, is easier to use owing to its offer web browser cell access to users with an internet connection. Similar to other symbolic computation-type CASs, SageMath can assist many of us in solving many mathematical problems, aiding us to investigate whether our calculations done by pen and paper are correct, which would save a lot of time on lengthy and complex calculations that are otherwise too difficult or time-consuming to perform by hand.

As we have shown in Section 3, despite its static nature, SageMath can assist us in finding the (orthogonal) diagonalization and SVD of a given matrix. Calculating the characteristic polynomial, eigenvalues, and eigenvectors is straightforward and quick. This is helpful for a larger-sized matrix, where obtaining eigenvalues and eigenvectors may take a significant amount of time. Without worrying about this step, learners in the subject of linear algebra can directly focus on the main business of constructing matrices for (orthogonal) diagonalization and SVD. Once we have all of the matrices, SageMath can help us to quickly determine whether those matrices are correct by multiplying the products in the (orthogonal) diagonalization or SVD. Any correct factorization that does not have to be unique will return the product to the original matrix. The static nature of SageMath also demands that learners carefully observe outputs. For example, the command A.eigenvectors_right() gives an output comprising of an eigenvalue, eigenvector(s), and its geometric multiplicity in that particular order. Constructing a diagonal matrix $D$ and invertible matrix $P$ also requires hand intervention, which requires some dynamics from the users' side. This interaction between static SageMath and dynamic learners makes the CAS a powerful tool for learning mathematics. Furthermore, the interactive feature of SageMath allows users to experiment with distinct matrices, or matrices of different sizes or characteristics. Although we do not really feature the visualization tools of SageMath in this study, its ability to present plots or graphs will assist learners further in explaining more difficult concepts encountered in mathematics.

As knowledge improves and technology progresses, the presence of a static CAS will increasingly be accompanied by the emergence of dynamic chatbots that make use of the power of AI, such as ChatGPT, Google Bard, and Bing Chat. Although ChatGPT is not primarily designed for responding to inquiries related to mathematical problems, some attempts in this study and other researchers suggest that it has the capability to handle mathematical problems. It is just a matter of time before the responses will improve and get better. This triggers us to address the third and final research question, that is, whether we can rely on ChatGPT, despite its dynamic nature, in understanding linear algebra, particularly (orthogonal) diagonalization and SVD. The short answer is no. A quick and simple reason for this argument is that ChatGPT often provides an incorrect answer. In our study, except for the first inquiry, all other responses from ChatGPT contained mistakes in one way or another. If this is the case, should we simply abandon ChatGPT and other similar AI-generated chatbots when it comes to learning mathematics or understanding linear algebra, perhaps going all the way to forbidding the students to access it entirely during the course of their study? How can we strike a balance between utilizing AI-generated chatbots such as ChatGPT in learning mathematics and understanding linear algebra, while still ensuring that students develop a deep and thorough understanding of the subject matter without solely relying on the technology?

Although many would not agree, we have some reservations regarding abandoning or forbidding ChatGPT, not only in terms of learning mathematics but also for other subjects. Whether we like it or not, this "technological train" has left its compound and accelerated ever since, reaching all corners of the earth where everyone has access to the Internet. Its dynamic features have captivated many researchers. And, this is just the beginning. Since its release at the end of the last year, it has disrupted many industries, and academia and education are no exception. Thus, instead of abandoning or forbidding its use, we should embrace and integrate it into our teaching and learning. It is similar to the period when the calculator or CAS infiltrated the traditional chalk-and-board classrooms. Although in the beginning there was some resistance toward integrating technology in mathematics teaching and learning, nowadays, it is standard practice, and aspiring educators are even encouraged to do so ^{[144,145,146,147]}. After all, this is one way to connect the divide between digital natives and digital migrants ^{[148,149,150,151]}.

This early stage of ChatGPT development provides a tremendous opportunity for developing algorithmic thinking skills among learners, especially when the output contains many mistakes. When inquired, instead of providing an answer in one go, ChatGPT released the answer letter by letter, literally like someone else typing out the answer from the other side of the computer. This dynamic feature makes ChatGPT superior to static CASs and attractive to many new users. In our study on matrix factorization, ChatGPT demonstrated a proper step-by-step algorithm on how to diagonalize or find an SVD of a given matrix. In the absence of specifying a particular matrix, it can even provide its own choice of matrix example, usually with a relatively small matrix size.

As observed in Section 4 on orthogonal diagonalization, ChatGPT shows how to calculate the characteristic polynomial of a matrix and solve the corresponding characteristic equation to find eigenvalues, albeit with a computational error. This provides an opportunity for learners to critically examine the provided answers and identify where mistakes occur. The same principle can also be applied to other mistakes when finding the associated eigenvectors, and when it abruptly stopped without completing the calculation, thus making the results incomplete. For the SVD, the first two attempts provided examples in which a detailed algorithm appeared to be eschewed. Instead, ChatGPT employed a matrix algebra library in Python to perform the task. It then provided the final answers of the three matrices involved in the decomposition, that is, $U$, $\Sigma$, and $V$. Our third attempt seems to be an improvement, where ChatGPT showed a step-by-step algorithm for computing singular values, finding eigenvectors, constructing an orthogonal matrix $V$ that contains right singular vectors on its columns, forming matrix $\Sigma$, and calculating another orthogonal matrix $U$ that contains left singular vectors on its columns. The fact that ChatGPT is confused between eigenvalues and singular values, we can glean a valuable lesson from this output because this issue is also commonly discovered among linear algebra learners, as addressed in Section 2.

From our brief coverage of the applications of (orthogonal) diagonalization and SVD, it becomes evident that students stand to gain immense benefits when they master algorithmic thinking skills and develop a solid understanding of these topics. As they delve into fields like control engineering, image processing, data science, and beyond, their proficiency in (orthogonal) diagonalization and SVD empowers them to efficiently tackle complex problems and optimize system behavior. Not only do these techniques enhance computational efficiency and reduce data complexity, they also provide deeper insights into the underlying structures of various scientific and engineering phenomena. By harnessing the power of (orthogonal) diagonalization and SVD, students and researchers alike can uncover hidden patterns, streamline analyses, and unlock innovative solutions across diverse disciplines, propelling advancements in science and engineering to new heights.

The versatility and power of (orthogonal) diagonalization and SVD have revolutionized various fields of science and engineering. Their ability to efficiently extract crucial information, reduce data dimensionality, and optimize system behavior has made them indispensable tools in linear algebra, signal processing, control engineering, data science, and more. From enabling stable control of complex systems to enhancing image and signal analysis, these techniques have proven invaluable in the advancement of research and technology. Moreover, their applications in population genetics, structural analysis, and biomedical signal processing have further extended their impact, contributing to breakthroughs in life sciences and healthcare. With the potential to solve intricate problems and reveal hidden patterns, (orthogonal) diagonalization and SVD continue to drive innovation, deepen our understanding of complex phenomena, and pave the way for future advancements in science and engineering.

In conclusion, we have considered the relationship between algorithmic thinking skills in linear algebra and some common mistakes among learners when they study particular topics such as (orthogonal) diagonalization and SVD, as well as how technological tools such as the static CAS SageMath and dynamic AI ChatGPT can contribute to enhancing algorithmic comprehension of these topics. Understanding a procedure for matrix factorization and the ability to perform the computational process accurately requires a solid understanding of other topics in linear algebra, including but not limited to solving a system of linear equations using elementary row operations, finding eigenvalues and the corresponding eigenvectors, orthogonality and vector projection (the Gram-Schmidt process), and dot (inner) and cross products.

We observed that some common mistakes that students make can be eliminated with more practice and by strengthening the basic concepts in linear algebra and their interrelationships. Although static by nature, utilizing CAS SageMath can provide tremendous help in verifying calculation results done by hand, handling a larger matrix size, checking whether the obtained factorizations were correct, and other required computational activities for understanding the materials. Its free and open-source characteristics, together with a web-based interface using cells and Python, are the primary strengths of SageMath in comparison to other costly CASs such as Maple, Matlab, and Mathematica. The current stage of ChatGPT, despite its dynamic and attractive features, can be used for complementing the study, but neither as a primary tool nor to be relied upon, owing to its numerous output mistakes. This has exciting future implications, particularly in the field of education, where updated versions of ChatGPT will be improved and released in the future. One question remains as to how to utilize a chatbot effectively for teaching and learning mathematics without compromising academic integrity. Regarding this, we also hope that a sequence of follow-ups of this study from other researchers will appear accordingly.

Use of AI tools declaration

The author declares that he has used the AI tool of ChatGPT in the creation of this article. The tool was used to obtain responses to inquiries related to matrix (orthogonal) diagonalization and singular value decomposition, which can be found in Section 4.

Acknowledgments

This research was supported by the National Research Foundation (NRF) of South Korea, as funded by the South Korean Ministry of Science, Information, Communications, and Technology (MSICT) through Grant No. NRF-2022-R1F1A1-059817 under the scheme of Broadening Opportunities Grants–General Research Program in Basic Science and Engineering.

Conflict of interest

The author declares that he has no conflict of interest to disclose.

A.   SageMath code

This appendix provides SageMath code for matrix diagonalization, orthogonal diagonalization, and SVD.

A.1.   Matrix diagonalization

A = matrix([[1, 2, 2], [0, 2, 1], [0, 1, 2]])

print("Matrix A = ")

print(A)

print()

print("Characteristic polynomial of A: p(x) =", A.charpoly())

print()

print("Eigenvalues of A =", A.eigenvalues())

print()

print("Eigenvalue, eigenvector, and geometric multiplicity:", )

print(A.eigenvectors_right())

print()

lambda1 = A.eigenvalues()[0]

lambda2 = A.eigenvalues()[1]

lambda3 = A.eigenvalues()[2]

print("Extracting eigenvalues:")

print("lambda1 =", lambda1)

print("lambda2 =", lambda2)

print("lambda3 =", lambda3)

print()

D = diagonal_matrix(A.eigenvalues())

print("Diagonal matrix D = ")

print(D)

print()

r1 = A.eigenvectors_right()[0][1]

p1 = 2*r1[0]

r2 = A.eigenvectors_right()[1][1]

p2 = r2[0]

r3 = A.eigenvectors_right()[1][1]

p3 = r3[1]

print("Extracting eigenvectors:")

print("p1 =", p1)

print("p2 =", p2)

print("p3 =", p3)

print()

P = matrix([p1, p2, p3]).transpose()

print("Invertible matrix P = ")

print(P)

print()

print("Inverse of P, P^(-1) = ")

print(P.inverse())

print()

print("Calculate PDP^(-1) = ")

print(P*D*P.inverse())

print(" = A")

A.2.   Orthogonal diagonalization

A = matrix([[3, -4, -4], [-4, 3, -4], [-4, -4, 3]])

print("Matrix A = ")

print(A)

print()

print("Characteristic polynomial of A: p(x) =", A.charpoly())

print()

print("Eigenvalues of A =", A.eigenvalues())

print()

lambda1 = A.eigenvalues()[0]

lambda2 = A.eigenvalues()[1]

lambda3 = A.eigenvalues()[2]

print("Extracting eigenvalues:")

print("lambda1 =", lambda1)

print("lambda2 =", lambda2)

print("lambda3 =", lambda3)

print()

D = diagonal_matrix(A.eigenvalues())

print("Diagonal matrix D = ")

print(D)

print()

print("Eigenvalue, eigenvector, and geometric multiplicity:", )

print(A.eigenvectors_right())

print()

q1 = A.eigenvectors_right()[0][1][0]

q2 = A.eigenvectors_right()[1][1][0]

q3 = A.eigenvectors_right()[1][1][1]

print("Extracting eigenvectors:")

print("q1 =", q1)

print("q2 =", q2)

print("q3 =", q3)

print()

print("Checking orthogonality:")

print(" <q1, q2> =", q1.inner_product(q2))

print(" <q1, q3> =", q1.inner_product(q3))

print(" <q2, q3> =", q2.inner_product(q3))

print()

print("q2 and q3 are not orthogonal; apply the Gram-Schmidt process:")

print()

q31 = q3 - q2.inner_product(q3)/q2.inner_product(q2)*q2

print("q3' =", q31)

print()

print("Check that now q2 and q3' are orthogonal: <q2, q3'> =", q2.inner_product(q31))

print()

print("Normalize all orthogonal eigenvectors:")

print()

p1 = q1/sqrt(q1.inner_product(q1))

print("p1 =", p1)

print()

p2 = q2/sqrt(q2.inner_product(q2))

print("p2 =", p2)

print()

q32 = 2*q31

p3 = -q32/sqrt(q32.inner_product(q32))

print("p3 =", p3)

print()

print("Construct an invertible and orthogonal matrix P:")

P = matrix([p1, p2, p3]).transpose()

print("P = ")

print(P)

print()

print("Inverse of P, P^(-1) = ")

print(P.inverse())

print()

print("Calculate PDP^(-1) = ")

print(P*D*P.inverse())

print(" = A")

print()

print("Transpose of P, P^T = ")

print(P.transpose())

print()

print("Calculate PDP^T = ")

print(P*D*P.transpose())

print(" = A")

A.3.   SVD

M = matrix([ [2, -2], [-3, -4], [-4, -3] ])

print("Matrix M = ")

print(M)

print()

A = M.transpose()*M

print("Matrix M^T M = ")

print(A)

print()

print("Eigenvalues of M^T M =", A.eigenvalues())

print()

print("Eigenvalue, eigenvector, and geometric multiplicity:", )

print(A.eigenvectors_right())

print()

sigma1 = sqrt(A.eigenvalues()[0])

sigma2 = sqrt(A.eigenvalues()[1])

print("Singular value of M:")

print("sigma1 =", sigma1)

print("sigma2 =", sigma2)

print()

Sigma = matrix([[sigma1, 0], [0, sigma2], [0, 0]])

print("Matrix Sigma =")

print(Sigma)

print()

w1 = A.eigenvectors_right()[0][1][0]

v1 = w1/w1.norm()

w2 = A.eigenvectors_right()[1][1][0]

v2 = w2/w2.norm()

print("Eigenvectors of M^T M:")

print("v1 =", v1)

print("v2 =", v2)

print()

print("Matrix V =")

V = matrix([ v1, v2 ])

print(V)

print()

u1 = 1/sigma1*M*v1

u2 = 1/sigma2*M*v2

u3 = u1.cross_product(u2)

print("Left singular vectors of M:")

print("u1 =", u1)

print("u2 =", u2)

print("u3 =", u3)

print()

print("Checking orthogonality")

print("u1.u2 =", u1.dot_product(u2))

print("u1.u3 =", u1.dot_product(u3))

print("u2.u3 =", u3.dot_product(u2))

print()

UT = matrix([u1, u2, u3])

U = UT.transpose()

print("Matrix U =")

print(U)

print()

print("SVD of M = U*Sigma*V^T =")

print(U*Sigma*V.transpose())

print(" = M")