From the series: Differential Equations and Linear Algebra
Gilbert Strang, Massachusetts Institute of Technology (MIT)
Linear equations include dy/dt = y, dy/dt = –y, dy/dt = 2ty. The equation dy/dt = y*y is nonlinear.
OK. Well, the idea of this first video is to tell you what's coming, to give a kind of outline of what is reasonable to learn about ordinary differential equations. And a big part of the series will be videos on first order equations and videos on second order equations. Those are the ones you see most in applications. And those are the ones you can understand and solve, when you're fortunate.
So first order equations means first derivatives come into the equation. So that's a nice equation that we will solve, we'll spend a lot of time on. The derivative is-- that's the rate of change of y-- the changes in the unknown y-- as time goes forward are partly from depending on the solution itself. That's the idea of a differential equation, that it connects the changes with the function y as it is.
And then you have inputs called q of t, which produce their own change. They go into the system. They become part of y. And they grow, decay, oscillate, whatever y of t does. So that is a linear equation with a right-hand side, with an input, a forcing term.
And here is a nonlinear equation. The derivative of y. The slope depends on y. So it's a differential equation. But f of y could be y squared over y cubed or the sine of y or the exponential of y. So it could be not linear. Linear means that we see y by itself. Here we won't. Well, we'll come pretty close to getting a solution, because it's a first order equation. And the most general first order equation, the function would depend on t and y. The input would change with time. Here, the input depends only on the current value of y.
I might think of y as money in a bank, growing, decaying, oscillating. Or I might think of y as the distance on a spring. Lots of applications coming.
OK. So those are first order equations. And second order have second derivatives. The second derivative is the acceleration. It tells you about the bending of the curve.
If I have a graph, the first derivative we know gives the slope of the graph. Is it going up? Is it going down? Is it a maximum?
The second derivative tells you the bending of the graph. How it goes away from a straight line. So and that's acceleration. So Newton's law-- the physics we all live with-- would be acceleration is some force. And there is a force that depends, again, linearly-- that's a keyword-- on y. Just y to the first power.
And here is a little bit more general equation. In Newton's law, the acceleration is multiplied by the mass. So this includes a physical constant here, the mass.
Then there could be some damping. If I have motion, there may be friction slowing it down. That depends on the first derivative, the velocity.
And then there could be the same kind of forced term that depends on y itself. And there could be some outside force, some person or machine that's creating movement. An external forcing term.
So that's a big equation. And let me just say, at this point, we let things be nonlinear. And we had a pretty good chance. If we get these to be non-linear, the chance at second order has dropped. And the further we go, the more we need linearity and maybe even constant coefficients. m, b, and k. So that's really the problem that we can solve as we get good at it is a linear equation-- second order, let's say-- with constant coefficients. But that's pretty much pushing what we can hope to do explicitly and really understand the solution, because so linear with constant coefficients. Say it again. That's the good equations.
And I think of solutions in two ways. If I have a really nice function like a exponential. Exponentials are the great functions of differential equations, the great functions in this series. You'll see them over and over. Exponentials. Say f of t equals-- e to the t. Or e to the omega t. Or e to the i omega t. That i is the square root of minus 1.
In those cases, we will get a similarly nice function for the solution. Those are the best. We get a function that we know like exponentials. And we get solutions that we know.
Second best are we get some function we don't especially know. In that case, the solution probably involves an integral of f, or two integrals of f. We have a formula for it. That formula includes an integration that we would have to do, either look it up or do it numerically.
And then when we get to completely non-linear functions, or we have varying coefficients, then we're going to go numerically. So really, the wide, wide part of the subject ends up as numerical solutions. But you've got a whole bunch of videos coming that have nice functions and nice solutions.
OK. So that's first order and second order. Now there's more, because a system doesn't usually consist of just a single resistor or a single spring. In reality, we have many equations. And we need to deal with those.
So y is now a vector. y1, y2, to yn. n different unknowns. n different equations. That's n equation. So here that is an n by n matrix. So it's first order. Constant coefficient. So we'll be able to get somewhere. But it's a system of n coupled equations.
And so is this one with a second derivative. Second derivative of the solution. But again, y1 to yn. And we have a matrix, usually a symmetric matrix there, we hope, multiplying y.
So again, linear. Constant coefficients. But several equations at once. And that will bring in the idea of eigenvalues and eigenvectors. Eigenvalues and eigenvectors is a key bit of linear algebra that makes these problems simple, because it turns this coupled problem into n uncoupled problems. n first order equations that we can solve separately. Or n second order equations that we can solve separately. That's the goal with matrices is to uncouple them.
OK. And then really the big reality of this subject is that solutions are found numerically and very efficiently. And there's a lot to learn about that, a lot to learn. And MATLAB is a first-class package that gives you numerical solutions with many options.
One of the options may be the favorite. ODE for ordinary differential equations 4 5. And that is numbers 4, 5. Well, Cleve Moler, who wrote the package MATLAB, is going to create a series of parallel videos explaining the steps toward numerical solution.
Those steps begin with a very simple method. Maybe I'll put the creator's name down. Euler. So you can know that because Euler was centuries ago, he didn't have a computer. But he had a simple way of approximating. So Euler might be ODE 1. And now we've left Euler behind. Euler is fine, but not sufficiently accurate.
ODE 45, that 4 and 5 indicate a much higher accuracy, much more flexibility in that package. So starting with Euler, Cleve Moler will explain several steps that reach a really workhorse package.
So that's a parallel series where you'll see the codes. This will be a chalk and blackboard series, where I'll find solutions in exponential form. And if I can, I would like to conclude the series by reaching partial differential equations.
So I'll just write some partial differential equations here, so you know what they mean. And that's a goal which I hope to reach.
So one partial differential equation would be du dt-- you see partial derivatives-- is second derivative. So I have two variables now. Time, which I always have. And here is x in the space direction. That's called the heat equation. That's a very important constant coefficient, partial differential equation.
So PDE, as distinct from ODE. And so I write down one more. The second derivative of u is the same right-hand side second derivative in the x direction. That would be called the wave equation.
So this is like the first order equation in time. It's like a big system. In fact, it's like an infinite size system of equations. First order in time. Or second order in time. Heat equation. Wave equation.
And I would like to also include a the Laplace equation. Well, if we get there. So those are goals for the end of the series that go beyond some courses in ODEs. But the main goal here is to give you the standard clear picture of the basic differential equations that we can solve and understand.
Well, I hope it goes well. Thanks.
1.1: Overview of Differential Equations Linear equations include dy/dt = y, dy/dt = –y, dy/dt = 2ty . The equation dy/dt = y *y is nonlinear.
1.2: The Calculus You Need The sum rule, product rule, and chain rule produce new derivatives from the derivatives of xn , sin(x ) and ex . The Fundamental Theorem of Calculus says that the integral inverts the derivative.
1.4b: Response to Exponential Input, exp(s*t) With exponential input, est , from outside and exponential growth, eat , from inside, the solution, y(t), is a combination of two exponentials.
1.4c: Response to Oscillating Input, cos(w*t) An oscillating input cos(ωt ) produces an oscillating output with the same frequency ω (and a phase shift).
1.4d: Solution for Any Input, q(t) To solve a linear first order equation, multiply each input q(s) by its growth factor and integrate those outputs.
1.4e: Step Function and Delta Function A unit step function jumps from 0 to 1. Its slope is a delta function: zero everywhere except infinite at the jump.
1.5: Response to Complex Exponential, exp(i*w*t) = cos(w*t)+i*sin(w*t) For linear equations, the solution for f = cos(ωt ) is the real part of the solution for f = eiωt . That complex solution has magnitude G (the gain).
1.6: Integrating Factor for a Constant Rate, a The integrating factor e-at multiplies the differential equation, y’=ay+q, to give the derivative of e-at y: ready for integration.
1.6b: Integrating Factor for a Varying Rate, a(t) The integral of a varying interest rate provides the exponent in the growing solution (the bank balance).
1.7: The Logistic Equation When –by2 slows down growth and makes the equation nonlinear, the solution approaches a steady state y( ∞) = a/b.
1.7c: The Stability and Instability of Steady States Steady state solutions can be stable or unstable – a simple test decides.
2.1: Second Order Equations For the oscillation equation with no damping and no forcing, all solutions share the same natural frequency.
2.1b: Forced Harmonic Motion With forcing f = cos(ωt ), the particular solution is Y *cos(ωt ). But if the forcing frequency equals the natural frequency there is resonance.
2.3: Unforced Damped Motion With constant coefficients in a differential equation, the basic solutions are exponentials est . The exponent s solves a simple equation such as As2 + Bs + C = 0 .
2.3c: Impulse Response and Step Response The impulse response g is the solution when the force is an impulse (a delta function). This also solves a null equation (no force) with a nonzero initial condition.
2.4: Exponential Response - Possible Resonance Resonance occurs when the natural frequency matches the forcing frequency — equal exponents from inside and outside.
2.4b: Second Order Equations With Damping A damped forced equation has a particular solution y = G cos(ωt – α). The damping ratio provides insight into the null solutions.
2.5: Electrical Networks: Voltages and Currents Current flowing around an RLC loop solves a linear equation with coefficients L (inductance), R (resistance), and 1/C (C = capacitance).
2.6: Methods of Undetermined Coefficients With constant coefficients and special forcing terms (powers of t , cosines/sines, exponentials), a particular solution has this same form.
2.6b: An Example of Method of Undetermined Coefficients This method is also successful for forces and solutions such as (at2 + bt +c) est : substitute into the equation to find a, b, c .
2.6c: Variations of Parameters Combine null solutions y1 and y2 with coefficients c1(t) and c2(t) to find a particular solution for any f(t).
2.7: Laplace Transform: First Order Equation Transform each term in the linear differential equation to create an algebra problem. You can then transform the algebra solution back to the ODE solution, y(t) .
2.7b: Laplace Transform: Second Order Equation The second derivative transforms to s2Y and the algebra problem involves the transfer function 1/ (As2 + Bs +C).
3.1: Pictures of the Solutions The direction field for dy/dt = f(t,y) has an arrow with slope f at each point t, y . Arrows with the same slope lie along an isocline.
3.2: Phase Plane Pictures: Source, Sink Saddle Solutions to second order equations can approach infinity or zero. Saddle points contain a positive and also a negative exponent or eigenvalue.
3.2b: Phase Plane Pictures: Spirals and Centers Imaginary exponents with pure oscillation provide a “center” in the phase plane. The point (y, dy/dt) travels forever around an ellipse.
3.2c: Two First Order Equations: Stability A second order equation gives two first order equations for y and dy/dt . The matrix becomes a companion matrix.
3.3: Linearization at Critical Points A critical point is a constant solution Y to the differential equation y’ = f(y) . Near that Y , the sign of df/dy decides stability or instability.
3.3b: Linearization of y'=f(y,z) and z'=g(y,z) With two equations, a critical point has f(Y,Z) = 0 and g(Y,Z) = 0. Near those constant solutions, the two linearized equations use the 2 by 2 matrix of partial derivatives of f and g .
3.3c: Eigenvalues and Stability: 2 by 2 Matrix, A Two equations y’ = Ay are stable (solutions approach zero) when the trace of A is negative and the determinant is positive.
5.1: The Column Space of a Matrix, A An m by n matrix A has n columns each in R m . Capturing all combinations Av of these columns gives the column space – a subspace of R m .
5.4: Independence, Basis, and Dimension Vectors v 1 to v d are a basis for a subspace if their combinations span the whole subspace and are independent: no basis vector is a combination of the others. Dimension d = number of basis vectors.
5.5: The Big Picture of Linear Algebra A matrix produces four subspaces – column space, row space (same dimension), the space of vectors perpendicular to all rows (the nullspace), and the space of vectors perpendicular to all columns.
5.6: Graphs A graph has n nodes connected by m edges (other edges can be missing). This is a useful model for the Internet, the brain, pipeline systems, and much more.
6.1: Eigenvalues and Eigenvectors The eigenvectors x remain in the same direction when multiplied by the matrix (A x = λx ). An n x n matrix has n eigenvalues.
6.2: Diagonalizing a Matrix A matrix can be diagonalized if it has n independent eigenvectors. The diagonal matrix Λis the eigenvalue matrix.
6.3: Solving Linear Systems d y /dt = A y contains solutions y = eλt x where λ and x are an eigenvalue / eigenvector pair for A .
6.4: The Matrix Exponential, exp(A*t) The shortest form of the solution uses the matrix exponential y = eAt y (0) . The matrix eAt has eigenvalues eλt and the eigenvectors of A.
6.4b: Similar Matrices, A and B=M^(-1)*A*M A and B are “similar” if B = M-1AM for some matrix M . B then has the same eigenvalues as A .
6.5: Symmetric Matrices, Real Eigenvalues, Orthogonal Eigenvectors Symmetric matrices have n perpendicular eigenvectors and n real eigenvalues.
7.2: Positive Definite Matrices, S=A'*A A positive definite matrix S has positive eigenvalues, positive pivots, positive determinants, and positive energy vT Sv for every vector v. S = AT A is always positive definite if A has independent columns.
7.2b: Singular Value Decomposition, SVD The SVD factors each matrix A into an orthogonal matrix U times a diagonal matrix Σ (the singular value) times another orthogonal matrix VT : rotation times stretch times rotation.
7.3: Boundary Conditions Replace Initial Conditions A second order equation can change its initial conditions on y(0) and dy/dt(0) to boundary conditions on y(0) and y(1) .
8.1: Fourier Series A Fourier series separates a periodic function F(x) into a combination (infinite) of all basis functions cos(nx) and sin(nx) .
8.1b: Examples of Fourier Series Even functions use only cosines (F(–x) = F(x) ) and odd functions use only sines. The coefficients an and bn come from integrals of F(x) cos(nx ) and F(x) sin(nx ).
8.1c: Fourier Series Solution of Laplace's Equation Inside a circle, the solution u (r , θ) combines rn cos(n θ) and rn sin(n θ). The boundary solution combines all entries in a Fourier series to match the boundary conditions.
8.3: Heat Equation The heat equation ∂u /∂t = ∂2u /∂x2 starts from a temperature distribution u at t = 0 and follows it for t > 0 as it quickly becomes smooth.