From the series: Differential Equations and Linear Algebra
Gilbert Strang, Massachusetts Institute of Technology (MIT)
The shortest form of the solution uses the matrix exponential y = eAt y(0). The matrix eAt has eigenvalues eλt and the eigenvectors of A.
OK. We're still solving systems of differential equations with a matrix A in them.
And now I want to create the exponential. It's just natural to produce e to the A, or e to the A t. The exponential of a matrix. So if we have one equation, small a, then we know the solution is an e to the A t, times the starting value.
Now we have n equations with a matrix A and a vector y. And the solution should be, at time t, e to the A t, times the starting value. It should be a perfect match with this one, where this had a number in the exponent and this has a matrix in the exponent.
OK. No problem. We just use the series for e to the A t. We plug in a matrix instead of a number. So the identity, plus A t, plus 1/2 A t squared, plus 1/6 of A t cubed, forever. It's the same. It's the exponential series. The most important series in mathematics, I think.
And it gives us an answer. And that answer is a matrix. Everything here, every term, is a matrix.
OK. Now, is that the right answer? We check that by putting it into the differential equation. So I want to put that solution into the equation. So I need to take the derivative.
The derivative of this is the derivative of-- that's a constant. The derivative of that is A. The derivative of this is 1/2. I have an A squared, and I have a t squared. The derivative of t squared is 2t, so that'll just be a t. The 2 and the 2 cancel.
OK. Now I have A cubed here. t cubed? The derivative of t cubed is 3t squared, so I have a t squared. And the 3 cancels the 3 and the 6, and leaves 1 over 2 factorial, and so on.
And I look at that. And I say it's very much like the one above. Look. This series is just A times this one. Multiply the top one by A. A times I is A. A times A t is A squared t. Term by term, it just has a factor A.
So it's A e to the A t, is the derivative of my matrix exponential. It brings down an A. Just what we want. Just what we want.
So then if I add a y of 0 in here, that's just a constant vector. I'll have a y of 0. I'll have a y of 0 here. When I put this into the differential equation, it works. It works.
Now, is it better than what we had before, which was using eigenvalues and eigenvectors? It's better in one way. This exponential, this series, is totally fine whether we have n independent eigenvectors or not. We could have repeated eigenvalues.
I'll do an example. So for with repeated eigenvalues and missing eigenvectors, e to the A t is still the correct answer. Still the correct answer. But if we want to use eigenvalues and eigenvectors to compute e to the A t, because we don't want to add up an infinite series very often, then we would want n independent eigenvectors.
So what am I saying? I'm saying that this e to the A t-- All right, suppose we have n independent eigenvectors. And we know that that means, in that case, a is V times lambda times V inverse. And we can write V inverse because the matrix V has the eigenvectors.
This is the eigenvector matrix. If I have n independent eigenvectors, that matrix is invertible. I have that nice formula. And now I can see what is-- e to the A t is always identity plus A.
I'm now going to use the diagonalization, the eigenvectors, and the eigenvalues for A. So I'm doing the good case now, when there are a full set of independent eigenvectors. Then the A t is V lambda V inverse t. That's right, that's I, plus A t, plus 1/2 A t squared. Right?
So I need A squared. So everybody remembers what A squared is. A squared is V lambda V inverse, times V lambda V inverse. And those cancel out to give V lambda squared V inverse, times t squared, and so on.
You remember this A squared, so I'll take that away. And look at what I've got. Look what I've got it. Yes. Factor V out of the start, and factor V inverse out of the end.
And in here I have V times V inverse is I, so that's fine. V times V inverse, I have a lambda t. V and a V inverse, so I have a 1/2 half lambda squared t squared. And so on, times V inverse.
This is all just what we hope for. We expect that a V goes out at the far left at the front. This V inverse comes out at the far right. And what do you see in the middle?
You see-- so this is now my formula for e to the A t, is V. And what do I have there? I have the exponential series for lambda t. So it's e to the lambda t V inverse.
And what is e to the lambda t? Let's just understand the matrix exponential. When the matrix is diagonal, the best possible matrix, this will be V. What does my matrix look like? V inverse.
If I'm looking at this, looking at this. Lambda is diagonal. All these matrices are diagonal with lambdas. So that'll be e to the lambda 1t down to e to the lambda nt.
I'm not doing anything brilliant here. I'm just using the standard diagonalization to produce our exponential from the eigenvector matrix and from the eigenvalues. So I'm just taking the exponentials of the n different eigenvalues.
So e to the A t. This would lead to e to the A t y at 0, would be-- y of 0 is some combination. And then there's an e to the lambda 1t coming from here. And there's an x eigenvector x1, plus C2 e to the lambda 2t x2, so on.
That's the solution that we had last time. That's the solution that using eigenvalues and eigenvectors.
Now. Can Can I get something new here? Something new will be, suppose there are not a full set of n independent eigenvectors. e to the A t is still OK. But this formula is no good. That formula depends on V and V inverse.
And suppose we have an example. So all that is very nice. That's what we expect.
But we could have a matrix like this one. A equals-- well, here's an extreme case.
What are the eigenvalues of that matrix? It's a diagonal matrix. The eigenvalues are 0 and 0. The eigenvalue of 0 is repeated. It's a double eigenvalue.
And we hope for two eigenvectors, but we don't find them. That has only one line of eigenvectors. It only has an x1 equals 1, 0, I think. If I multiply that A, times that x1, gives me 0 times x1. That's an eigenvector.
Well, because the eigenvalue is 0, I'm looking for the null space. There is in the null space, but the null space is only one-dimensional. Only one eigenvector. Missing an eigenvector.
Still, still, I can do e to the A t. That's still completely correct. That series will work.
So to do this series I need to know a squared. So I'm actually going to use the series, but you'll see that it cuts off very fast. a squared, if you work that out, it's all 0's.
So our e to the A t is just I, plus A t, plus STOP. A squared is all 0's. A cubed is all 0's. So the matrix e to the A t is identity, a times t. a is this, times t is going to put a t there.
There you go. That's a case of the matrix exponential, which would lead us to the solution of the equations. Of course, it's a pretty simple exponential.
But it comes from pretty simple equations. The equations dy dt, that system of two equations, with that matrix in it. Our system of equations is just dy1 dt, I have a 1 there so it would be a y2. And dy2 dt is 0 on the second row.
Well, that's pretty easy to solve. In fact, this tells you how to solve-- you could naturally ask the question, how do we solve differential equations when the matrix doesn't have n eigenvectors?
Here's an example. This matrix has only one eigenvector. But the equation that we just solved by, you could say, back substitution. This gives Y2 equal constant. And then that equation, dy1 dt equal that constant, gives me y1 equals t times constant. That's what I'm seeing.
Oh. Yeah. Are you surprised to see a t show up here? Normally I don't see a t in matrix exponentials. But in this repeated case, that's the t that we're always seeing when we have repeated solutions.
Everybody remembers that when we have second-order equations, and we have the two exponents are the same. So we only get one solution of that, e to the st. And we have to look for another one. And that other one is? te to the st. It's that same t there.
OK. There is an example of how a matrix with a missing eigenvector, the exponential pops a t in. The exponential pops a t in.
And if I had two missing eigenvectors, then in the exponential. Shall I just show you an example with two missing eigenvectors?
Let a be-- well, here it would be 0, 0, 0, 0, 0, triple 0, with, let's say. There's a matrix with three 0 eigenvalues, but only one eigenvector. So it's missing two eigenvectors. And I would, in the end, in e to the A t here, I would see probably 1, 1, 1, t, t, and probably I'll see a 1/2 t squared there.
A little bit like that. But one step worse. Because the triple eigenvalue, well, that's not going to happen very often in reality. But we see what it produces. It produces a t squared as well as the t's.
OK. So, the x matrix exponential gives a beautiful, concise, short formula for the solution. And it gives a formula that's correct, even in the case of missing eigenvectors.
1.1: Overview of Differential Equations Linear equations include dy/dt = y, dy/dt = –y, dy/dt = 2ty . The equation dy/dt = y *y is nonlinear.
1.2: The Calculus You Need The sum rule, product rule, and chain rule produce new derivatives from the derivatives of xn , sin(x ) and ex . The Fundamental Theorem of Calculus says that the integral inverts the derivative.
1.4b: Response to Exponential Input, exp(s*t) With exponential input, est , from outside and exponential growth, eat , from inside, the solution, y(t), is a combination of two exponentials.
1.4c: Response to Oscillating Input, cos(w*t) An oscillating input cos(ωt ) produces an oscillating output with the same frequency ω (and a phase shift).
1.4d: Solution for Any Input, q(t) To solve a linear first order equation, multiply each input q(s) by its growth factor and integrate those outputs.
1.4e: Step Function and Delta Function A unit step function jumps from 0 to 1. Its slope is a delta function: zero everywhere except infinite at the jump.
1.5: Response to Complex Exponential, exp(i*w*t) = cos(w*t)+i*sin(w*t) For linear equations, the solution for f = cos(ωt ) is the real part of the solution for f = eiωt . That complex solution has magnitude G (the gain).
1.6: Integrating Factor for a Constant Rate, a The integrating factor e-at multiplies the differential equation, y’=ay+q, to give the derivative of e-at y: ready for integration.
1.6b: Integrating Factor for a Varying Rate, a(t) The integral of a varying interest rate provides the exponent in the growing solution (the bank balance).
1.7: The Logistic Equation When –by2 slows down growth and makes the equation nonlinear, the solution approaches a steady state y( ∞) = a/b.
1.7c: The Stability and Instability of Steady States Steady state solutions can be stable or unstable – a simple test decides.
2.1: Second Order Equations For the oscillation equation with no damping and no forcing, all solutions share the same natural frequency.
2.1b: Forced Harmonic Motion With forcing f = cos(ωt ), the particular solution is Y *cos(ωt ). But if the forcing frequency equals the natural frequency there is resonance.
2.3: Unforced Damped Motion With constant coefficients in a differential equation, the basic solutions are exponentials est . The exponent s solves a simple equation such as As2 + Bs + C = 0 .
2.3c: Impulse Response and Step Response The impulse response g is the solution when the force is an impulse (a delta function). This also solves a null equation (no force) with a nonzero initial condition.
2.4: Exponential Response - Possible Resonance Resonance occurs when the natural frequency matches the forcing frequency — equal exponents from inside and outside.
2.4b: Second Order Equations With Damping A damped forced equation has a particular solution y = G cos(ωt – α). The damping ratio provides insight into the null solutions.
2.5: Electrical Networks: Voltages and Currents Current flowing around an RLC loop solves a linear equation with coefficients L (inductance), R (resistance), and 1/C (C = capacitance).
2.6: Methods of Undetermined Coefficients With constant coefficients and special forcing terms (powers of t , cosines/sines, exponentials), a particular solution has this same form.
2.6b: An Example of Method of Undetermined Coefficients This method is also successful for forces and solutions such as (at2 + bt +c) est : substitute into the equation to find a, b, c .
2.6c: Variations of Parameters Combine null solutions y1 and y2 with coefficients c1(t) and c2(t) to find a particular solution for any f(t).
2.7: Laplace Transform: First Order Equation Transform each term in the linear differential equation to create an algebra problem. You can then transform the algebra solution back to the ODE solution, y(t) .
2.7b: Laplace Transform: Second Order Equation The second derivative transforms to s2Y and the algebra problem involves the transfer function 1/ (As2 + Bs +C).
3.1: Pictures of the Solutions The direction field for dy/dt = f(t,y) has an arrow with slope f at each point t, y . Arrows with the same slope lie along an isocline.
3.2: Phase Plane Pictures: Source, Sink Saddle Solutions to second order equations can approach infinity or zero. Saddle points contain a positive and also a negative exponent or eigenvalue.
3.2b: Phase Plane Pictures: Spirals and Centers Imaginary exponents with pure oscillation provide a “center” in the phase plane. The point (y, dy/dt) travels forever around an ellipse.
3.2c: Two First Order Equations: Stability A second order equation gives two first order equations for y and dy/dt . The matrix becomes a companion matrix.
3.3: Linearization at Critical Points A critical point is a constant solution Y to the differential equation y’ = f(y) . Near that Y , the sign of df/dy decides stability or instability.
3.3b: Linearization of y'=f(y,z) and z'=g(y,z) With two equations, a critical point has f(Y,Z) = 0 and g(Y,Z) = 0. Near those constant solutions, the two linearized equations use the 2 by 2 matrix of partial derivatives of f and g .
3.3c: Eigenvalues and Stability: 2 by 2 Matrix, A Two equations y’ = Ay are stable (solutions approach zero) when the trace of A is negative and the determinant is positive.
5.1: The Column Space of a Matrix, A An m by n matrix A has n columns each in R m . Capturing all combinations Av of these columns gives the column space – a subspace of R m .
5.4: Independence, Basis, and Dimension Vectors v 1 to v d are a basis for a subspace if their combinations span the whole subspace and are independent: no basis vector is a combination of the others. Dimension d = number of basis vectors.
5.5: The Big Picture of Linear Algebra A matrix produces four subspaces – column space, row space (same dimension), the space of vectors perpendicular to all rows (the nullspace), and the space of vectors perpendicular to all columns.
5.6: Graphs A graph has n nodes connected by m edges (other edges can be missing). This is a useful model for the Internet, the brain, pipeline systems, and much more.
6.1: Eigenvalues and Eigenvectors The eigenvectors x remain in the same direction when multiplied by the matrix (A x = λx ). An n x n matrix has n eigenvalues.
6.2: Diagonalizing a Matrix A matrix can be diagonalized if it has n independent eigenvectors. The diagonal matrix Λis the eigenvalue matrix.
6.3: Solving Linear Systems d y /dt = A y contains solutions y = eλt x where λ and x are an eigenvalue / eigenvector pair for A .
6.4: The Matrix Exponential, exp(A*t) The shortest form of the solution uses the matrix exponential y = eAt y (0) . The matrix eAt has eigenvalues eλt and the eigenvectors of A.
6.4b: Similar Matrices, A and B=M^(-1)*A*M A and B are “similar” if B = M-1AM for some matrix M . B then has the same eigenvalues as A .
6.5: Symmetric Matrices, Real Eigenvalues, Orthogonal Eigenvectors Symmetric matrices have n perpendicular eigenvectors and n real eigenvalues.
7.2: Positive Definite Matrices, S=A'*A A positive definite matrix S has positive eigenvalues, positive pivots, positive determinants, and positive energy vT Sv for every vector v. S = AT A is always positive definite if A has independent columns.
7.2b: Singular Value Decomposition, SVD The SVD factors each matrix A into an orthogonal matrix U times a diagonal matrix Σ (the singular value) times another orthogonal matrix VT : rotation times stretch times rotation.
7.3: Boundary Conditions Replace Initial Conditions A second order equation can change its initial conditions on y(0) and dy/dt(0) to boundary conditions on y(0) and y(1) .
8.1: Fourier Series A Fourier series separates a periodic function F(x) into a combination (infinite) of all basis functions cos(nx) and sin(nx) .
8.1b: Examples of Fourier Series Even functions use only cosines (F(–x) = F(x) ) and odd functions use only sines. The coefficients an and bn come from integrals of F(x) cos(nx ) and F(x) sin(nx ).
8.1c: Fourier Series Solution of Laplace's Equation Inside a circle, the solution u (r , θ) combines rn cos(n θ) and rn sin(n θ). The boundary solution combines all entries in a Fourier series to match the boundary conditions.
8.3: Heat Equation The heat equation ∂u /∂t = ∂2u /∂x2 starts from a temperature distribution u at t = 0 and follows it for t > 0 as it quickly becomes smooth.