Types of matrix factorization

Linear Algebra

Matrices can be factorized in multiple ways.

In this section, we list some factorizations that you might encounter in research literature, especially in machine learning.

Prerequisites

To understand the various types of matrix factorization, we recommend familiarity with the concepts in

Follow the above links to first get acquainted with the corresponding concepts.

LU factorization

LU Factorization is a subtle extension of the Gaussian elimination method that we studied earlier.

Just as in Gaussian elimination for solving \( \mA \vx = \vy \), we factorize the coefficient matrix \( \mA \) such that \( \mA = \mathbf{L}\mathbf{U} \). Here, \( \mathbf{L} \) is a lower triangular matrix with 1's along the diagonal and \( \mU \) is an upper triangular one. So, effectively, the Gaussian elimination step is equivalent to arriving at the matrix \( \mU \).

Any square matrix with non-zero determinant has a LU factorization, which is effectively the same condition as what we would have for Gaussian elimination.

LDL factorization

LDL Factorization is a further extension of LU, in that \( \mA = \mathbf{L} \mD \mU \). Here, \( \mU \) also has 1's along the diagonal and \( \mD \) has the pivots.

Cholesky factorization

Cholesky Factorization is a decomposition that applies to symmetric positive definite matrices.

Any symmetric positive definite matrix \( \mA \in \real^{n \times n} \) can be factorized as \( \mA = \mathbf{G}\mathbf{G}^T \), where \( \mathbf{G} \) is a lower triangular matrix with positive diagonal entries.

QR factorization

Any rectangular matrix \( \mA \in \real^{m \times n} \) can be factorized as \( \mA = \mQ \mathbf{R} \), where \( \mQ \in \real^{m \times m} \) is an orthogonal matrix and \( \mathbf{R} \in \real^{m \times n} \) is an upper triangular matrix.

Where to next?

In this section, we have defined the various ways to factorize a matrix.

To contiue your study, you may choose to explore other advanced topics linear algebra.

Already feeling like an expert in linear algebra? Move on to other advanced topics in mathematics or machine learning.

Please share

Let your friends, followers, and colleagues know about this resource you discovered.

Let's connect

Please share your comments, questions, encouragement, and feedback.