4.3 Least Squares Approximations - MIT （4.3最小二乘近似-麻省理工学院）.pdf

218 Chapter 4. Orthogonality 4.3 Least Squares Approximations It often happens that Ax Db has no solution. The usual reason is: too many equations. The matrix has more rows than columns. There are more equations than unknowns (m is greater than n). The n columns span a small part of m-dimensional space. Unless all measurements are perfect, b is outside that column space. Elimination reaches an impossible equation and stops. But we can’t stop just because measurements include noise. To repeat: We cannot always get the error e Db Ax down to zero. When e is zero, x is an exact solution to Ax D b. When the length of e is as small as possible, bx is a least squares solution. Our goal in this section is to compute bx and use it. These are real problems and they need an answer. The previous section emphasized p (the projection). This section emphasizes bx (the least squares solution). They are connected by p DAbx . The fundamental equation is still AT T Abx DA b. Here is a short unofficial way to reach this equation: T T T When Ax Db has no solution, multiply by A and solve A Abx DA b: Example 1 A crucial application of least squares is fitting a straight line to m points. Start with three points: Find the closest line to the points .0; 6/; .1; 0/, and .2; 0/. No straight line b D C CDt goes through those three points. We are asking for two numbers C and D that satisfy three equations. Here are the equations at t D 0; 1; 2 to match the given values b D6; 0; 0: t D 0 The first point is on the line b DC CDt if C CD 0 D6 t D 1 The second point is on the line b DC CDt if C CD 1 D0 t D 2 The third point is on