6: Orthogonality and Least Squares

Understanding Linear Algebra · LibreTexts

0%

Contents

    Text

    Size
    100%
    Theme
    Typeface
    Line length
    Spacing

    From LibreTexts, under CC BY 4.0, as the source states it; its equations are written in TeX, which the PDF shows typeset. This page at LibreTexts.

    We introduced vectors as a means to develop visual intuition about our basic questions concerning linear systems. For example, vectors allow us to reinterpret questions about the existence of solutions to linear systems as questions about the span of a set of vectors. Questions about the uniqueness of solutions led to the concept of linear independence.

    In this chapter, we will begin to think of vectors as geometric objects that have lengths and that form angles. In some cases, this will simplify our search for solutions to a linear system. Perhaps more importantly, we will be able to measure the distance between vectors. This means that if a system \(A \mathbf{x}=\mathbf{b}\) is inconsistent, we can look for \(\widehat{\mathbf{x}}\), the vector for which \(A \widehat{\mathbf{x}}\) is as close to \(\mathbf{b}\) as possible. This leads to the method of least squares, which underpins regression, a key tool in data science.

    • 6.1: The Dot Product
    • 6.2: Orthogonal Complements and the Matrix Tranpose
      This section introduces the notion of an orthogonal complement, the set of vectors each of which is orthogonal to a prescribed subspace. We'll also find a way to describe dot products using matrix products, which allows us to study orthogonality using many of the tools for understanding linear systems that we developed earlier.
    • 6.3: Orthogonal bases and projections
    • 6.4: Finding Orthogonal Bases
    • 6.5: Orthogonal Least Squares
      n this section, we'll explore how the techniques developed in this chapter enable us to find the line that best approximates the data. More specifically, we'll see how the search for a line passing through the data points leads to an inconsistent system Ax=b. Since we are unable to find a solution x, we instead seek the vector x where Ax is as close as possible to b. Orthogonal projection give us just the right tool for doing this.

    This page titled 6: Orthogonality and Least Squares was last modified on Wed, 18 Mar 2026 00:23:08 GMT and is shared under a CC BY 4.0 license and was authored, remixed, and/or curated by David Austin via source content that was edited to the style and standards of the LibreTexts platform.