Matrix Factorization (Step-by-Step for Students)

Did you know that you can factor a matrix?

Jenn (B.S., M.Ed.) of Calcworkshop® teaching matrix factorization

Jenn, Founder Calcworkshop^®, 15+ Years Experience (Licensed & Certified Teacher)

Just like in algebra, we learned how to express a polynomial in terms of a product of two or more factors. We can similarly define a matrix as a product of two matrices.

Defining Matrix Factorization

Matrix Factorization, or matrix decomposition, is the process of taking a matrix and decomposing it into a product of two triangular matrices.

Explaining Triangular Matrices

Now a triangular matrix is a special kind of square $n \times n$ matrix. A lower triangular matrix is when all the entries above the main diagonal are zero, and an upper triangular matrix is when all the entries below the main diagonal are zero.

Lower Triangular Matrix

$[\begin{array}{llll} * & 0 & 0 & 0 \\ * & * & 0 & 0 \\ * & * & * & 0 \\ * & * & * & * \end{array}]$

Upper Triangular Matrix

$[\begin{array}{llll} * & * & * & * \\ 0 & * & * & * \\ 0 & 0 & * & * \\ 0 & 0 & 0 & * \end{array}]$

Understanding LU Decomposition

And LU Decomposition is a factorization of a nonsingular square matrix A (i.e., A is invertible) such that $A = L U$ where $L$ is a lower triangular matrix and $U$ is an upper triangular matrix.

Applications of Matrix Factorization

But why do we care? What’s the point of factoring a matrix?

This procedure is motivated by the fairly common industrial and business problem of solving a sequence of equations, all with the same coefficient matrix, as well as in engineering applications, partial differential equations for various force functions, and is even considered a better way to implement Gauss elimination (REF).

Step by Step LU Decomposition

Alright, now it’s time to see how LU decomposition works. To fully understand these steps, sometimes called the matrix factorization algorithm, it is best illustrated through an example.

Find the LU factorization of the matrix $A = [\begin{array}{ccc} - 2 & 1 & - 3 \\ 6 & - 1 & 8 \\ 8 & 3 & - 7 \end{array}]$

First, we will find the Upper Matrix (U) using row operations. But there’s a bit of a twist… we do not care about getting ” $1 s$ ” along the main diagonal. All we want to achieve are the appropriate zeros in the lower triangle of the matrix.

$[\begin{array}{ccc} - 2 & 1 & - 3 \\ 6 & - 1 & 8 \\ 8 & 3 & - 7 \end{array}] \sim \begin{aligned} 3 R_{1} + R_{2} \to R_{2} \\ 4 R_{1} + R_{3} \to R_{3} \end{aligned}$

$[\begin{array}{ccc} - 2 & 1 & - 3 \\ 0 & 2 & - 1 \\ 0 & 7 & - 19 \end{array}] \sim - 7 R_{2} + 2 R_{3} \to R_{3}$

$[\begin{array}{ccc} - 2 & 1 & - 3 \\ 0 & 2 & - 1 \\ 0 & 0 & - 31 \end{array}]$

$U = [\begin{array}{ccc} - 2 & 1 & - 3 \\ 0 & 2 & - 1 \\ 0 & 0 & - 31 \end{array}]$

Next, we will use our row operation command lines from our U-matrix and our always important Identity matrix to find our lower triangular matrix $(L)$ .

Our first command was $3 R_{1} + R_{2} \to R_{2}$ , so we will replace the second row with those coefficients from the command line in the appropriate column.

$E_{1} = [\begin{array}{ccc} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{array}] \underset{3 R_{1} + 1 R_{2} \to R_{2}}{\Rightarrow} [\begin{array}{ccc} 1 & 0 & 0 \\ 3 & 1 & 0 \\ 0 & 0 & 1 \end{array}]$

Our second command was $4 R_{1} + R_{3} \to R_{3}$ , so we will replace the third row with those coefficients from the command line in the appropriate column.

$E_{2} = [\begin{array}{ccc} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{array}] \underset{4 R_{1} + 1 R_{3} \to R_{3}}{\Rightarrow} [\begin{array}{ccc} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 4 & 0 & 1 \end{array}]$

Our third command was $4 R_{1} + R_{3} \to R_{3}$ , so we will replace the third row with those coefficients from the command line in the appropriate column.

$\begin{array}{r} E_{3} = [\begin{array}{ccc} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{array}] \underset{- 7 R_{2} + 2 R_{3} \to R_{3}}{\Rightarrow} [\begin{array}{ccc} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & - 7 & 2 \end{array}] \end{array}$

After completing each elementary matrix, we can say $E_{3} E_{2} E_{1} A = U$ . Therefore, to find the lower triangular matrix, L, we just need to compute the inverses of the elementary matrices using the strategies we discovered in our lesson on Inverse Matrices, such that:

$\begin{aligned} L & = E_{1}^{- 1} E_{2}^{- 1} E_{3}^{- 1} \end{aligned}$
$\begin{aligned} E_{1} & = [\begin{array}{lll} 1 & 0 & 0 \\ 3 & 1 & 0 \\ 0 & 0 & 1 \end{array}] \\ E_{1}^{- 1} & = [\begin{array}{ccc} 1 & 0 & 0 \\ - 3 & 1 & 0 \\ 0 & 0 & 1 \end{array}] \end{aligned}$
$\begin{aligned} E_{2} & = [\begin{array}{lll} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 4 & 0 & 1 \end{array}] \\ E_{2}^{- 1} & = [\begin{array}{ccc} 1 & 0 & 0 \\ 0 & 1 & 0 \\ - 4 & 0 & 1 \end{array}] \end{aligned}$
$\begin{aligned} E_{3} & = [\begin{array}{ccc} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & - 7 & 2 \end{array}] \\ E_{3}^{- 1} & = [\begin{array}{ccc} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 7 / 2 & - 1 / 2 \end{array}] \end{aligned}$

Therefore,

$\begin{aligned} L & = E_{1}^{- 1} E_{2}^{- 1} E_{3}^{- 1} \\ = [\begin{array}{ccc} 1 & 0 & 0 \\ - 3 & 1 & 0 \\ 0 & 0 & 1 \end{array}] [\begin{array}{ccc} 1 & 0 & 0 \\ 0 & 1 & 0 \\ - 4 & 0 & 1 \end{array}] [\begin{array}{cccc} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 7 / 2 & - 1 / 2 \end{array}] \\ = [\begin{array}{ccc} 1 & 0 & 0 \\ - 3 & 1 & 0 \\ - 4 & 7 / 2 & - 1 / 2 \end{array}] \end{aligned}$

And we can verify our decomposition by checking $A = L U$ .

$\begin{aligned} L U & = A \\ = [\begin{array}{ccc} 1 & 0 & 0 \\ - 3 & 1 & 0 \\ - 4 & 7 / 2 & - 1 / 2 \end{array}] [\begin{array}{ccc} - 2 & 1 & - 3 \\ 0 & 2 & - 1 \\ 0 & 0 & - 31 \end{array}] \\ = [\begin{array}{ccc} - 2 & 1 & - 3 \\ 6 & - 1 & 8 \\ 8 & 3 & - 7 \end{array}] \end{aligned}$

Not bad, right?

I should point out that there are actually two methods for finding LU factorization, and we will compare and contrast their similarities and differences in our video lesson.

LU Factorization in Solving Linear Systems

Okay, so while it’s great to know that we can decompose a matrix into a product of two triangular matrices, LU factorization is most useful when solving systems of linear equations.

Consider the matrix equation $A \vec{x} = \vec{b}$ . Now, if $A$ is an invertible matrix, then we can let $A = L U$ . Thus the matrix can be rewritten as $L U \vec{x} = \vec{b}$ . This is important because we can make the following substitutions:

$L U \vec{x} = \vec{b}$
$L (U \vec{x}) = \vec{b}$
$L y = \vec{b}$ where $y = U \vec{x}$

Solving Systems of Linear Equations using LU Factorization

Consequently, if we solve for $L y = \vec{b}$ for $y$ and then solve $y = U \vec{x}$ for $x$ we will have solve the system of linear equations!

Example of LU Factorization in a Linear System

Alright, let’s see this in action to help us make sense of this process.

We’ll use LU factorization to solve the linear system in the following format:

$\begin{aligned} A \vec{x} & = \vec{b} \\ A & = [\begin{array}{ccc} 3 & - 1 & 2 \\ - 6 & 3 & 1 \\ 9 & - 1 & 1 \end{array}] \\ \vec{b} & = [\begin{matrix} 1 \\ 3 \\ 6 \end{matrix}] \end{aligned}$

First, we will find our upper triangular matrix (U) and lower triangular matrix (L) using the steps seen above.

$\begin{aligned} U & = [\begin{array}{ccc} 3 & - 1 & 2 \\ 0 & 1 & 5 \\ 0 & 0 & - 15 \end{array}] \\ and \\ L & = [\begin{array}{ccc} 1 & 0 & 0 \\ - 2 & 1 & 0 \\ 3 & 2 & 1 \end{array}] \end{aligned}$

Next, we solve the system $L y = \vec{b}$

$[\begin{array}{ccc} 1 & 0 & 0 \\ - 2 & 1 & 0 \\ 3 & 2 & 1 \end{array}] [\begin{array}{l} y_{1} \\ y_{2} \\ y_{3} \end{array}] = [\begin{array}{l} 1 \\ 3 \\ 6 \end{array}] \Rightarrow \begin{aligned} y_{1} = 1 \\ - 2 y_{1} + y_{2} = 3 \\ 3 y_{1} + 2 y_{2} + y_{3} = 6 \end{aligned}$

By using forward substitution, we can solve for the missing y-variables

$\begin{matrix} y_{1} = 1 \\ - 2 y_{1} + y_{2} = 3 \to - 2 (1) + y_{2} = 3 \to y_{2} = 5 \\ 3 y_{1} + 2 y_{2} + y_{3} = 6 \to 3 (1) + 2 (5) + y_{3} = 6 \to y_{3} = - 7 \end{matrix}$

Thus, $y = [\begin{matrix} 1 \\ 5 \\ - 7 \end{matrix}]$

Now, it’s time to find the original system, so we solve $\vec{u} = y$

$[\begin{array}{ccc} 3 & - 1 & 2 \\ 0 & 1 & 5 \\ 0 & 0 & - 15 \end{array}] [\begin{array}{l} x_{1} \\ x_{2} \\ x_{3} \end{array}] = [\begin{matrix} 1 \\ 5 \\ - 7 \end{matrix}] \Rightarrow \begin{aligned} 3 x_{1} - x_{2} + 2 x_{3} = 1 \\ x_{2} + 5 x_{3} = 5 \\ - 15 x_{3} = - 7 \end{aligned}$

This time we will use backward substitution to solve of $x$ .

$\begin{aligned} 3 x_{1} - x_{2} + 2 x_{3} & = 1 \\ \to 3 x_{1} - (\frac{8}{3}) + 2 (\frac{7}{15}) = 1 \\ \to x_{1} = \frac{41}{45} \end{aligned}$

$\begin{aligned} x_{2} + 5 x_{3} & = 5 \\ \to x_{2} + 5 (\frac{7}{15}) = 5 \\ \to x_{2} = \frac{8}{3} \end{aligned}$

$\begin{aligned} - 15 x_{3} & = - 7 \\ \to x_{3} = \frac{7}{15} \end{aligned}$

$Hence, \vec{x} = [\begin{matrix} 41 / 45 \\ 8 / 3 \\ 7 / 15 \end{matrix}]$

Next Steps

In this video lesson, you will:

Learn the two methods for LU factorization
Use matrix decomposition to solve matrix equations $Ax = b$

Get ready for a fun learning experience!

Video Tutorial w/ Full Lesson & Detailed Examples

matrix-factorization-example

Get access to all the courses and over 450 HD videos with your subscription

Monthly and Yearly Plans Available

Get My Subscription Now

Still wondering if CalcWorkshop is right for you?
Take a Tour and find out how a membership can take the struggle out of learning math.

4.9 / 5

1,128 Reviews

Kevin S.10-30-24 - CO, United States

I had never even heard of discrete math before I had to take it for my Bachelor's in Cybersecurity. I had actually hoped that my math days were done! I suffered through class not understanding any of the presentations until I stumbled across Calcworkshop. I used the Calcworkshop lessons more than I used my instructor's lectures and my book. I ended up with a high B (just 12 points away from an A) thanks to Jenn. Without her easy to understand style of teaching, I'm not sure I would have even passed the class. Thank you so much!!!

Richard10-29-24 - Norway

Very well structured and site. It is well structured giving a good overview of the different topics regarding all mathematics where you easily can choose which path is applicable to your specific needs.The video lectures are superb with introduction to the topic followed by a detailed theoretically explaination and what I personally get most out of is; heaps of examples where you are taken and shown, step by step, the prosess of solving the problem. They also includd practice sheets and solutions to test yourself after each chapter. There is a lot to undertake on this site, but it provides an easy and understandable interface to map out what you need to do. The site has been very helpfull for me and I highly recommend it.

RobWP10-26-24 - United States

I am taking this course for mainly linear algebra review. It is a course that I have had but fairly long ago. I'm currently working in masters in data science and linear algebra namely transformations is something that has come up and I thought I need a good review on. I so far what I have liked about the course is, the common sense approach. Her approach makes it easy to remember the concepts because of her common sense approach. I like the fact that she gives information that is not really in any textbooks. I can vouch for that because not only have I used textbook recently to review I have had a formal class in the past that used textbooks. She goes over things thoroughly and for the price it's very much worth it. I have had many quarters of formal calculus and differential equations courses and I am kind of anxious to look to see what her approach to calculus is. I can't really answer how this class would be for a novice or someone who has not had linear algebra the past, but I really don't think they need to worry about that. Her method I feel will make a novice be successful and to understand the material. She is very easy to understand. Gives good examples but doesn't beat a dead horse either. What I like about her introductory remarks is that she says what linear algebra is all about and she points out what the core of the linear algebra is about and what you need to know. Having taken it before I would have to agree and she is dead on spot.

Robbin03-18-25 - CO, United States

Very well organized, helpful and comprehensive. It is great also to have access to prior course material so you can easily go back and refresh if you need to. Jenn is very clear and the number of examples leaves one comfortable with each topic. Couldn't be happier.

RCJK03-07-25 - ID, United States

I had to take Calculus I and II as accelerated classes while working full time. It was my first time taking this courses and was very overwhelming. Calcworkshop really made learning the subjects easier for me by having all the subjects easy to find and easy to understand explanations. I received a B in both classes which I am extremely happy with doing the classes in double time.

Jacqueline S.03-07-25 - SC, United States

Jenn is awesome!Her breakdown of the topic is beyond what you would expect. She is showing you how to arrive at the answer instead of doing a simple equation. Most people need to know the why it give confidence and structure to the learning process. This is what Jenn is great at, she knows how to deliver the information to a non math mind in a way that instills knowledge that you can forever draw from as you move forward. It's really well thought out. She is a true master teacher. Thanks for all that you do for us! Hugs n Blessings... JS

Rose M.03-07-25 - CA, United States

Best instructor ever, I have never had Math explained in a way that was so interesting. Her lectures and instruction is explained perfectly, and I will continue to use this for all my future math courses.

A Reviewer03-07-25 - United States

I needed help with college precalculus, they provided more than enough help that allowed me to succeed and get the scores I needed.

Jujuana03-05-25 - MO, United States

I love the videos the way each lesson is explained step by step. I love that she has examples that you can go over again and again in order to understand the way algebra works. I was lost because my instructor was flying through each lesson and I was totally lost. I attached my syllabus and they helped me stay focus on what I was working and give me a guide to go back and refresh what I went over in class.

Stacey01-31-25 - AZ, United States

Hello, I highly recommend getting a subscription to Jenn and her Calcworkshop teaching math service. I did a general, "go from bad to good at math", Google search and found out about Calcworkshop through responses to the topic on reddit. I've just started watching her videos and found them to be extremely helpful. I'm going back to college after a long absence and math was never my strong suit. So I'm starting at the very bottom of her available subjects and going from there. Just watching a couple of the videos has taken me from being dragged along not understanding to following along just fine in my math class. I'm moving from the "I'll never get this", to the "I look forward to understanding this", mindset in approaching math. This for me is huge, and even though I have a long way to go, I'm confident I will get there thanks to Jenn. Totally worth it!

Kathy W.01-14-25 - WA, United States

Calcworkshop is an outstanding source to learn math and understanding math concepts. Jenn is an outstanding master teacher. Every lesson is clearly presented making understanding easy to grasp. Practice lessons with accompanying answers are given for each section. I am 82, a retired teacher and lifelong learner. Learning calculus has always been on my bucket list. Calcworkshop is helping me reach my goal.

Cheryl01-14-25 - NY, United States

Calc workshop is the absolutely best resource for math and I have seen almost everything that exists out there. I have even recommended it to my daughter's school so that all the kids could use the videos during the summer months for the subject they plan to study in the fall. They are guaranteed to get a A if they have watched the videos and completed the problems before taking the class. Students who are weak in areas can access lower level classes and those who seek to advance above grade level can do so. The videos and exercises are crystal clear and the excellent presentations raise the bar for teachers everywhere. I have always believed that one great math teacher is important in the education process. The videos, exercises and problems are excellent. It is not possible to have ANY knowledge gaps. I have never seen anything on the market that even begins to compete with it!

Justin N.12-22-24 - OK, United States

Calculus can be a scary subject. I know I was scared. I had been out of any math class for over 23 years. I had taken an 8 week Calculus 1 class and felt like I was overloaded. Going into Calculus 2 I found CALCWORKSHOP.com. I used the training videos and material to really dive back into Calc 1 and prepare for Calc 2. Long story short with the material provided and ample study time, I finished Calc 2 with a 95%. Thanks Jenn!

Shelby12-10-24 - OH, United States

CalcWorkshop was a wonderful resource as someone going onto a college calculus 1 class after not having taken a math course in a year! Jenn always explained things in a logical order that built upon previous lessons, and the way she explained topics was always easy to understand. Calcworkshop saved this semester for me, and I couldn't be more grateful to have found this site. :)

Jordan Q.12-09-24 - AR, United States

Anyone struggling with math at any level would greatly benefit from this product. I signed up for it immediately after getting a D on my first exam in calculus 2 and ended up getting A's and B's on the remaining exams, getting a B overall in the class. Jenn breaks down the topics, without skipping any steps, in a way that makes them very easy to understand. I highly recommend this product!

N8Booker11-28-24 - MA, United States

I'm in school to get my degree Networking and Cybersecurity and I'm in my first year. I've been away from school for quite a long time so the concepts that are being taught although not foreign to me, have so much dust on them from a lack of being used in real world applications regularly.I stumbled upon CalcWorkshop by pure chance and it has been a game changer for me. Jen filled in the gaps in between what my Professor is teaching at a rapid fire pace and where my brain grasps and applies it. If not for Jen's approach, I'd have failed my test I took miserably!Am I grateful? It goes beyond words and not I recommend her workshops to everyone!

Kevin S.10-30-24 - CO, United States

Richard10-29-24 - Norway

RobWP10-26-24 - United States

RCJK03-07-25 - ID, United States

Jacqueline S.03-07-25 - SC, United States

See more reviews on Shopper Approved