solnsets.xml

<?xml version="1.0" encoding="UTF-8"?>

<!--********************************************************************
Copyright 2017 Georgia Institute of Technology

Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License, Version 1.3 or
any later version published by the Free Software Foundation.  A copy of
the license is included in gfdl.xml.
*********************************************************************-->

<section xml:id="solution-sets">
  <title>Solution Sets</title>

  <objectives>
    <ol>
      <li>Understand the relationship between the solution set of <m>Ax=0</m> and the solution set of <m>Ax=b</m>.</li>
      <li>Understand the difference between the solution set and the column span.</li>
      <li><em>Recipes:</em> parametric vector form, write the solution set of a homogeneous system as a span.</li>
      <li><em>Pictures:</em> solution set of a homogeneous system, solution set of an inhomogeneous system, the relationship between the two.</li>
      <li><em>Vocabulary words:</em> <term>homogeneous</term>/<term>inhomogeneous</term>, <term>trivial solution</term>.</li>
    </ol>
  </objectives>

  <introduction>
    <p>In this section we will study the geometry of the solution set of any matrix equation <m>Ax=b</m>.</p>
  </introduction>

  <subsection xml:id="solnsets-homog">
    <title>Homogeneous Systems</title>

    <p>The equation <m>Ax=b</m> is easier to solve when <m>b=0</m>, so we start with this case.</p>

    <definition>
      <idx><h>System of linear equations</h><h>homogeneous</h></idx>
      <idx><h>System of linear equations</h><h>inhomogeneous</h></idx>
      <idx><h>Homogeneous</h><see>System of linear equations</see></idx>
      <idx><h>Inhomogeneous</h><see>System of linear equations</see></idx>
      <statement>
        <p>A system of linear equations of the form <m>Ax=0</m> is called <term>homogeneous</term>.</p>
        <p>A system of linear equations of the form <m>Ax=b</m> for <m>b\neq 0</m> is called <term>inhomogeneous</term>.</p>
      </statement>
    </definition>

    <p>A homogeneous system is just a system of linear equations where all constants on the right side of the equals sign are zero.</p>

    <bluebox>
      <idx><h>System of linear equations</h><h>trivial solution</h></idx>
      <idx><h>System of linear equations</h><h>nontrivial solution</h></idx>
      <idx><h>System of linear equations</h><h>homogeneous</h><h>trivial solution</h></idx>
      <idx><h>Trivial solution</h><see>System of linear equations</see></idx>
      <idx><h>Nontrivial solution</h><see>System of linear equations</see></idx>
      <p>A homogeneous system always has the solution <m>x=0</m>.  This is called the <term>trivial solution</term>.  Any nonzero solution is called <term>nontrivial</term>.</p>
    </bluebox>

    <observation xml:id="solnsets-nontrivial">
      <idx><h>System of linear equations</h><h>nontrivial solution</h><h>and free variables</h></idx>
      <p>The equation <m>Ax=0</m> has a nontrivial solution <m>\iff</m> there is a free variable <m>\iff</m> <m>A</m> has a column without a <xref ref="defn-pivot-pos" text="title">pivot position</xref>.</p>
    </observation>

    <example xml:id="solnsets-eg-no-nontrivial">
      <title>No nontrivial solutions</title>
      <statement>
        <p>What is the solution set of <m>Ax=0</m>, where
        <me>A = \mat{1 3 4; 2 -1 2; 1 0 1}?</me></p>
      </statement>
      <solution>
        <p>We form an augmented matrix and row reduce:
        <me>\amat{1 3 4 0; 2 -1 2 0; 1 0 1 0}
        \quad\xrightarrow{\text{RREF}}\quad
        \amat{1 0 0 0; 0 1 0 0; 0 0 1 0}.
        </me>
        The only solution is the trivial solution <m>x=0</m>.
        </p>
      </solution>
    </example>

    <observation xml:id="solnsets-no-augment">
      <p>When we row reduce the augmented matrix for a homogeneous system of linear equations, the last column will be zero throughout the row reduction process.  We saw this in the last <xref ref="solnsets-eg-no-nontrivial"/>:
      <me>\amat{1 3 4 0; 2 -1 2 0; 1 0 1 0}</me>
      So it is not really necessary to write augmented matrices when solving homogeneous systems.</p>
    </observation>

    <p>
      When the homogeneous equation <m>Ax=0</m> does have nontrivial solutions, it turns out that the solution set can be conveniently expressed as a span.
    </p>

    <specialcase hide-type="true">
      <title>Parametric Vector Form (homogeneous case)</title>
      <idx><h>Parametric vector form</h><h>of a homogeneous equation</h></idx>
      <p>
        Consider the following matrix in reduced row echelon form:
        <me>A = \mat{1 0 -8 -7; 0 1 4 3; 0 0 0 0}.</me>
        The matrix equation <m>Ax=0</m> corresponds to the system of equations
        <me>
          \syseq{x_1 \+ \. - 8x_3 - 7x_4 = 0;
          \. \+ x_2 + 4x_3 + 3x_4 = 0\rlap.}
        </me>
        We can write the parametric form as follows:
        <me>\syseq{
        x_1 = 8x_3 + 7x_4;
        x_2 = -4x_3 - 3x_4;
        x_3 = x_3;
        x_4 = \. \+ x_4\rlap.}
        </me>
        We wrote the redundant equations <m>x_3=x_3</m> and <m>x_4=x_4</m> in order to turn the above system into a <em>vector equation</em>:
        <me>x = \vec{x_1 x_2 x_3 x_4} = x_3\vec{8 -4 1 0} + x_4\vec{7 -3 0 1}.</me>
        This vector equation is called the <term>parametric vector form</term> of the solution set.
        Since <m>x_3</m> and <m>x_4</m> are allowed to be anything, this says that the solution set is the set of all linear combinations of <m>\vec{8 -4 1 0}</m> and <m>\vec{7 -3 0 1}</m>.  In other words, the solution set is
        <me>\Span\left\{\vec{8 -4 1 0},\;\vec{7 -3 0 1}\right\}.</me>
      </p>
    </specialcase>

    <p>Here is the general procedure.</p>

    <bluebox>
      <title>Recipe: Parametric vector form (homogeneous case)</title>
      <idx><h>Parametric vector form</h><h>of a homogeneous equation</h></idx>
      <idx><h>System of linear equations</h><h>parametric vector form of</h><see>Parametric vector form</see></idx>
      <p>Let <m>A</m> be an <m>m\times n</m> matrix.  Suppose that the free variables in the homogeneous equation <m>Ax=0</m> are, for example, <m>x_3</m>, <m>x_6</m>, and <m>x_8</m>.
      <ol>
        <li>Find the reduced row echelon form of <m>A</m>.</li>
        <li>Write the parametric form of the solution set, including the redundant equations <m>x_3=x_3</m>, <m>x_6=x_6</m>, <m>x_8=x_8</m>.  Put equations for all of the <m>x_i</m> in order.</li>
        <li>Make a single vector equation from these equations by making the coefficients of <m>x_3,x_6,</m> and <m>x_8</m> into vectors <m>v_3,v_6,</m> and <m>v_8</m>, respectively.</li>
      </ol>
      The solutions to <m>Ax=0</m> will then be expressed in the form
      <me> x = x_3v_3 + x_6v_6 + x_8v_8</me>
      for some vectors <m>v_3,v_6,v_8</m> in <m>\R^n</m>, and any scalars <m>x_3,x_6,x_8</m>.  This is called the <term>parametric vector form</term> of the solution.
        </p>
        <p>In this case, the solution set can be written as
        <m>\Span\{v_3,\,v_6,\,v_8\}.</m>
        </p>
    </bluebox>

    <p>
      We emphasize the following fact in particular.
    </p>

    <bluebox xml:id="solnsets-are-spans">
      <idx><h>Solution set</h><h>of a homogeneous system is a span</h></idx>
      <p>
	The set of solutions to a homogeneous equation <m>Ax=0</m> is a span.
      </p>
      </bluebox>

    <example xml:id="solnsets-eg-line">
      <title>The solution set is a line</title>
      <statement>
        <p>Compute the parametric vector form of the solution set of <m>Ax=0</m>, where
        <me>A = \mat{1 -3; 2 -6}.</me>
        </p>
      </statement>
      <solution>
        <p>We row reduce (without augmenting, as suggested in the above <xref ref="solnsets-no-augment"/>):
        <me>\mat{1 -3; 2 -6} \quad\xrightarrow{\text{RREF}}\quad
        \mat{1 -3; 0 0}.
        </me>
        This corresponds to the single equation <m>x_1 - 3x_2=0</m>.  We write the parametric form including the redundant equation <m>x_2=x_2</m>:
        <me>\syseq{x_1 = 3x_2; x_2 = x_2\rlap.}</me>
        We turn these into a single vector equation:
        <me>x = \vec{x_1 x_2} = x_2\vec{3 1}.</me>
        This is the parametric vector form of the solution set.
        Since <m>x_2</m> is allowed to be anything, this says that the solution set is the set of all scalar multiples of <m>{3\choose 1}</m>, otherwise known as
        <me>\Span\left\{\vec{3 1}\right\}.</me>
        We know how to draw the picture of a span of a vector: it is a line.  Therefore, this is a picture of the the solution set:
        <latex-code>
              <![CDATA[
\begin{tikzpicture}
  \path[clip] (-6,-2) rectangle (6,2);
  \draw[grid lines] (-6,-2) grid (6,2);
  \draw[thick] (-6,0) -- (6,0);
  \draw[thick] (0,-2) -- (0,2);
  \draw[seq4] (-6,-2) -- node[pos=.4, above left=1pt, whitebg, thin border]
   {$Ax=0$} (6,2);
  \draw[vector, seq-green] (0,0) -- (3,1);
\end{tikzpicture}
              ]]>
        </latex-code>
        </p>

        <figure>
          <caption>Interactive picture of the solution set of <m>Ax=0</m>.  If you drag <m>x</m> along the line spanned by <m>{3\choose 1}</m>, the product <m>Ax</m> is always equal to zero.  This is what it means for <m>\Span\{{3\choose 1}\}</m> to be the solution set of <m>Ax=0</m>.</caption>
          <mathbox source="demos/Axequalsb.html?lock=true&amp;x=3,1&amp;mat=1,-3:2,-6&amp;range2=5&amp;closed=true" height="600px"/>
        </figure>
      </solution>
    </example>

    <p>Since there were <em>two</em> variables in the above <xref ref="solnsets-eg-line"/>, the solution set is a subset of <m>\R^2</m>.  Since <em>one</em> of the variables was free, the solution set is a <em>line</em>:
        <latex-code>
              <![CDATA[
\begin{tikzpicture}
  \path[clip] (-6,-2) rectangle (6,2);
  \draw[grid lines] (-6,-2) grid (6,2);
  \draw[thick] (-6,0) -- (6,0);
  \draw[thick] (0,-2) -- (0,2);
  \draw[seq4] (-6,-2) -- node[pos=.4, above left=1pt, whitebg, thin border]
   {$Ax=0$} (6,2);
  \draw[vector, seq-green] (0,0) -- (3,1);
\end{tikzpicture}
              ]]>
        </latex-code>
</p>




    <bluebox>
      <idx><h>System of linear equations</h><h>nontrivial solution</h><h>finding</h></idx>
      <p>In order to actually <em>find</em> a nontrivial solution to <m>Ax=0</m> in the above <xref ref="solnsets-eg-line"/>, it suffices to substitute any nonzero value for the free variable <m>x_2</m>.  For instance, taking <m>x_2=1</m> gives the nontrivial solution <m>x = 1\cdot{3\choose 1} = {3\choose 1}.</m>  Compare to this <xref ref="finding-actual-soln"/>.</p>
    </bluebox>

    <example xml:id="solnsets-eg-plane">
      <title>The solution set is a plane</title>
      <statement>
        <p>Compute the parametric vector form of the solution set of <m>Ax=0</m>, where
        <me>A = \mat{1 -1 2; -2 2 -4}.</me>
        </p>
      </statement>
      <solution>
        <p>We row reduce (without augmenting, as suggested in the above <xref ref="solnsets-no-augment"/>):
        <me>\mat{1 -1 2; -2 2 -4} \quad\xrightarrow{\text{RREF}}\quad
        \mat{1 -1 2; 0 0 0}.
        </me>
        This corresponds to the single equation <m>x_1 - x_2 + 2x_3 = 0</m>.  We  write the parametric form including the redundant equations <m>x_2=x_2</m> and <m>x_3=x_3</m>:
        <me>\syseq{x_1 = x_2 - 2x_3; x_2 = x_2; x_3 = \. \+ x_3\rlap.}</me>
        We turn these into a single vector equation:
        <me>x = \vec{x_1 x_2 x_3} = x_2\vec{1 1 0} + x_3\vec{-2 0 1}.</me>
        This is the parametric vector form of the solution set.
        Since <m>x_2</m> and <m>x_3</m> are allowed to be anything, this says that the solution set is the set of all linear combinations of <m>\vec{1 1 0}</m> and <m>\vec{-2 0 1}</m>.  In other words, the solution set is
        <me>\Span\left\{\vec{1 1 0},\;\vec{-2 0 1}\right\}.</me>
        We know how to draw the span of two noncollinear vectors in <m>\R^3</m>: it is a plane.  Therefore, this is a picture of the solution set:
        <latex-code>
              <![CDATA[
\begin{tikzpicture}[myxyz]
  \path[clip, resetxy] (-6,-4) rectangle (6,4);

  \def\v{(1, 1, 0)}
  \def\w{(-2, 0, 1)}

  \node[coordinate] (X) at \v {};
  \node[coordinate] (Y) at \w {};

  \draw (0,0,-3) -- (0,0,0);

  \begin{scope}[x=(X), y=(Y), transformxy]
    \path[clip] (-5, 0) -- (-5, -5) -- (5, -5) -- (5, 0) -- cycle;
    \fill[seq4!30, nearly opaque] (-3,-2) rectangle (3,2);
    \draw[step=1cm, seq4, very thin] (-3,-2) grid (3,2);
  \end{scope}

  \begin{scope}[transformxy]
    \fill[white, nearly opaque] (-3, -3) rectangle (3, 3);
    \draw[grid lines] (-3, -3) grid (3, 3);
    \draw[->] (-3,0) -- (3,0);
    \draw[->] (0,-3) -- (0,3);
  \end{scope}

  \begin{scope}[x=(X), y=(Y), transformxy]
    \path[clip] (-5, 0) -- (-5, 5) -- (5, 5) -- (5, 0) -- cycle;
    \fill[seq4!30, nearly opaque] (-3,-2) rectangle (3,2);
    \draw[step=1cm, seq4, very thin] (-3,-2) grid (3,2);
  \end{scope}

  \draw[->] (0,0,0) -- (0,0,3);

  \draw[vector, seq1] (0,0,0) -- \v;
  \draw[vector, seq2] (0,0,0) -- \w;
  \draw[thin, densely dotted] \w -- \projxy\w;

  \node[seq4, anchor=south east] at (-3,-2.5,.5) {$Ax=0$};

  \point at (0,0,0);

\end{tikzpicture}
              ]]>
        </latex-code></p>

        <figure>
          <caption>Interactive picture of the solution set of <m>Ax=0</m>.  If you drag <m>x</m> along the violet plane, the product <m>Ax</m> is always equal to zero.  This is what it means for the plane to be the solution set of <m>Ax=0</m>.</caption>
          <mathbox source="demos/Axequalsb.html?lock=true&amp;closed=true&amp;x=0,0,0" height="600px"/>
        </figure>
      </solution>
    </example>

    <p>Since there were <em>three</em> variables in the above <xref ref="solnsets-eg-plane"/>, the solution set is a subset of <m>\R^3</m>.  Since <em>two</em> of the variables were free, the solution set is a <em>plane</em>.</p>

    <p>
      There is a natural question to ask here: is it possible to write the solution to a homogeneous matrix equation using fewer vectors than the one given in the above recipe?  We will see in <xref ref="param-vect-form-lin-ind"/> that the answer is <em>no</em>: the vectors from the recipe are always linearly independent, which means that there is no way to write the solution with fewer vectors.
    </p>

    <p>
      Another natural question is: are the solution sets for inhomogeneuous equations also spans?  As we will see shortly, they are never spans, but they are closely related to spans.
    </p>

    <p>
    There is a natural relationship between the number of free variables and the <q>size</q> of the solution set, as follows.
    </p>

    <bluebox xml:id="soln-sets-dim-hom"><title>Dimension of the solution set</title>
    <idx><h>Line</h><h>dimension-1 solution set</h></idx>
    <idx><h>Plane</h><h>dimension-2 solution set</h></idx>
    <idx><h>Space</h><h>dimension-3 solution set</h></idx>
    <idx><h>Dimension</h><h>of a solution set</h></idx>
    <p>The above examples show us the following pattern: when there is one free variable in a consistent matrix equation, the solution set is a line, and when there are two free variables, the solution set is a plane, etc.  The number of free variables is called the <em>dimension</em> of the solution set.
    </p>
    </bluebox>

<p>We will develop a rigorous definition of dimension in <xref ref="dimension"/>, but for now the dimension will simply mean the number of free variables.  Compare with this <xref ref="pivot-cols-dim"/>.</p>

    <p>Intuitively, the dimension of a solution set is the number of parameters you need to describe a point in the solution set.  For a line only one parameter is needed, and for a plane two parameters are needed.  This is similar to how the location of a building on Peachtree Street<mdash/>which is like a line<mdash/>is determined by one number and how a street corner in Manhattan<mdash/>which is like a plane<mdash/>is specified by two numbers.</p>


  </subsection>


  <subsection>
    <title>Inhomogeneous Systems</title>

    <p>
      Recall that a matrix equation <m>Ax=b</m> is called <term>inhomogeneous</term> when <m>b\neq0</m>.
    </p>

    <example xml:id="solnsets-eg-line2">
      <title>The solution set is a line</title>
      <statement>
        <p>What is the solution set of <m>Ax=b</m>, where
        <me>A = \mat{1 -3; 2 -6} \sptxt{and} b = \vec{-3 -6}?</me>
        (Compare to this <xref ref="solnsets-eg-line"/>, where we solved the corresponding homogeneous equation.)
        </p>
      </statement>
      <solution>
        <p>We row reduce the associated augmented matrix:
        <me>\amat{1 -3 -3; 2 -6 -6} \quad\xrightarrow{\text{RREF}}\quad
        \amat{1 -3 -3; 0 0 0}.
        </me>
        This corresponds to the single equation <m>x_1 - 3x_2 = -3</m>.  We can write the parametric form as follows:
        <me>\syseq{x_1 = 3x_2 - 3; x_2 = x_2 + 0\rlap.}</me>
        We turn the above system into a <em>vector equation</em>:
        <me>x = \vec{x_1 x_2} = x_2\vec{3 1} + \vec{-3 0}.</me>
        This vector equation is called the <term>parametric vector form</term> of the solution set.
        We write the solution set as
        <me>\Span\left\{\vec{3 1}\right\} + \vec{-3 0}.</me>
        Here is a picture of the the solution set:
        <latex-code>
              <![CDATA[
\begin{tikzpicture}[thin border nodes]
  \draw[grid lines] (-6,-2) grid (6,3);
  \path[clip] (-6,-2) rectangle (6,3);
  \draw (-6,0) -- (6,0);
  \draw (0,-2) -- (0,3);
  \draw[seq2] (-6,-2) -- node[pos=.6,below right=1pt,whitebg] {$Ax=0$} (6,2);
  \draw[seq4] (-6,-1) -- node[pos=.7,above left=1pt,whitebg] {$Ax=b$} (6,3);
  \point (a) at (-3,0);
  \draw[vector, seq1] (a) -- (0,1);
  \draw[->, vector, seq3] (0,0) -- node[pos=.3,above] {$p$} (a);
  \point at (0,0);
\end{tikzpicture}
              ]]>
        </latex-code></p>

        <figure>
          <caption>Interactive picture of the solution set of <m>Ax=b</m>.  If you drag <m>x</m> along the violet line, the product <m>Ax</m> is always equal to <m>b</m>.  This is what it means for the line to be the solution set of <m>Ax=b</m>.</caption>
          <mathbox source="demos/Axequalsb.html?x=-3,0&amp;mat=1,-3:2,-6&amp;lock=true&amp;closed=true" height="600px"/>
        </figure>
      </solution>
    </example>

    <p>In the above <xref ref="solnsets-eg-line2"/>, the solution set was all vectors of the form
    <me>x = \vec{x_1 x_2} = x_2\vec{3 1} + \vec{-3 0}</me>
    where <m>x_2</m> is any scalar.  The vector <m>p = {-3\choose 0}</m> is also a solution of <m>Ax=b</m>: take <m>x_2=0</m>.  We call <m>p</m> a <term>particular solution</term>.
    </p>

	<p>In the solution set, <m>x_2</m> is allowed to be anything, and so the solution set is obtained as follows: we take all scalar multiples of <m>{3\choose 1}</m> and then add the particular solution <m>p = {-3\choose 0}</m> to each of these scalar multiples.  Geometrically, this is accomplished by first drawing the span of <m>{3\choose 1}</m>, which is a line through the origin (and, not coincidentally, the solution to <m>Ax=0</m>), and we <em>translate</em>, or push, this line along <m>p = {-3\choose 0}</m>.  The translated line contains <m>p</m> and is parallel to <m>\Span\{{3\choose 1}\}</m>: it is a <em>translate of a line</em>. 
        <latex-code>
              <![CDATA[
\begin{tikzpicture}[thin border nodes]
  \draw[grid lines] (-6,-2) grid (6,3);
  \path[clip] (-6,-2) rectangle (6,3);
  \draw (-6,0) -- (6,0);
  \draw (0,-2) -- (0,3);
  \draw[seq2] (-6,-2) -- node[pos=.6,below right=1pt,whitebg] {$Ax=0$} (6,2);
  \draw[seq4] (-6,-1) -- node[pos=.7,above left=1pt,whitebg] {$Ax=b$} (6,3);
  \point (a) at (-3,0);
  \draw[vector, seq1] (a) -- (0,1);
  \draw[->, vector, seq3] (0,0) -- node[pos=.3,above] {$p$} (a);
  \point at (0,0);
\end{tikzpicture}
              ]]>
        </latex-code>

	</p>


    <example xml:id="solnsets-eg-plane2">
      <title>The solution set is a plane</title>
      <statement>
        <p>What is the solution set of <m>Ax=b</m>, where
        <me>A = \mat{1 -1 2; -2 2 -4} \sptxt{and} b = \vec{1 -2}?</me>
        (Compare this <xref ref="solnsets-eg-plane"/>.)
        </p>
      </statement>
      <solution>
        <p>We row reduce the associated augmented matrix:
        <me>\amat{1 -1 2 1; -2 2 -4 -2} \quad\xrightarrow{\text{RREF}}\quad
        \mat{1 -1 2 1; 0 0 0 0}.
        </me>
        This corresponds to the single equation <m>x_1 - x_2 + 2x_3 = 1</m>.  We can write the parametric form as follows:
        <me>\syseq{x_1 = x_2 - 2x_3 + 1; x_2 = x_2 \+ \. + 0; x_3 = \. \+ x_3 + 0\rlap.}</me>
        We turn the above system into a <em>vector equation</em>:
        <me>x = \vec{x_1 x_2 x_3} = x_2\vec{1 1 0} + x_3\vec{-2 0 1} + \vec{1 0 0}.</me>
        This vector equation is called the <term>parametric vector form</term> of the solution set.
        Since <m>x_2</m> and <m>x_3</m> are allowed to be anything, this says that the solution set is the set of all linear combinations of <m>\vec{1 1 1}</m> and <m>\vec{-2 0 1}</m>, <em>translated</em> by the vector <m>p = \vec{1 0 0}</m>.  This is a plane which contains <m>p</m> and is parallel to <m>\Span\left\{\vec{1 1 1},\;\vec{-2 0 1}\right\}</m>: it is a <em>translate of a plane</em>.  We write the solution set as
        <me>\Span\left\{\vec{1 1 1},\;\vec{-2 0 1}\right\} + \vec{1 0 0}.</me>
        Here is a picture of the solution set:
        <latex-code>
              <![CDATA[
\begin{tikzpicture}[myxyz]
  \path[clip, resetxy] (-6,-4) rectangle (6,4);

  \def\v{(1, 1, 0)}
  \def\w{(-2, 0, 1)}
  \def\pp{(1, 0, 0)}

  \node[coordinate] (X) at \v  {};
  \node[coordinate] (Y) at \w  {};
  \node[coordinate] (P) at \pp {};

  \begin{scope}[x=(X), y=(Y), , xshift=-.8cm, yshift=-.5cm, transformxy]
    \path[clip] (-5, 0) -- (-5, -5) -- (5, -5) -- (5, 0) -- cycle;
    \fill[seq4!30, nearly opaque] (-3,-2) rectangle (3,2);
    \draw[step=1cm, seq4, very thin] (-3,-2) grid (3,2);
  \end{scope}

  \draw (0,0,-3) -- (0,0,0);

  \begin{scope}[transformxy]
    \fill[white, nearly opaque] (-2, -3) rectangle (4, 3);
    \draw[grid lines] (-2, -3) grid (4, 3);
    \draw[->] (-2,0) -- (4,0);
    \draw[->] (0,-3) -- (0,3);
  \end{scope}

  \draw[->] (0,0,0) -- (0,0,3);

  \begin{scope}[x=(X), y=(Y), xshift=-.8cm, yshift=-.5cm, transformxy]
    \path[clip] (-5, 0) -- (-5, 5) -- (5, 5) -- (5, 0) -- cycle;
    \fill[seq4!30, nearly opaque] (-3,-2) rectangle (3,2);
    \draw[step=1cm, seq4, very thin] (-3,-2) grid (3,2);
  \end{scope}

  \draw[vector, seq1] \pp -- ($(X) + (P)$);
  \draw[vector, seq2] \pp -- ($(Y) + (P)$);
  \draw[thin, densely dotted] ($(Y) + (P)$) -- (-1, 0, 0);
  \draw[vector, seq3, opacity=.5] (0,0,0) -- node[below right, thin border] {$p$} \pp;

  \node[seq4, anchor=south east] at (-2,-2.5,.5) {$Ax=b$};

  \point at \pp;
  \point at (0,0,0);

\end{tikzpicture}
              ]]>
        </latex-code></p>

        <figure>
          <caption>Interactive picture of the solution set of <m>Ax=b</m>.  If you drag <m>x</m> along the violet plane, the product <m>Ax</m> is always equal to <m>b</m>.  This is what it means for the plane to be the solution set of <m>Ax=b</m>.</caption>
          <mathbox source="demos/Axequalsb.html?lock=true&amp;closed=true&amp;range1=3&amp;x=1,0,0" height="600px"/>
        </figure>
      </solution>
    </example>

    <p>In the above <xref ref="solnsets-eg-plane2"/>, the solution set was all vectors of the form
        <me>x = \vec{x_1 x_2 x_3} = x_2\vec{1 1 0} + x_3\vec{-2 0 1} + \vec{1 0 0}.</me>
    where <m>x_2</m> and <m>x_3</m> are any scalars.  In this case, a particular solution is <m>p = \vec{1 0 0}</m>.
    </p>

    <p>In the previous <xref ref="solnsets-eg-plane2"/> and the <xref ref="solnsets-eg-line2"/> before it, the parametric vector form of the solution set of <m>Ax=b</m> was exactly the same as the parametric vector form of the solution set of <m>Ax=0</m> (from this <xref ref="solnsets-eg-plane"/> and this <xref ref="solnsets-eg-line"/>, respectively), plus a particular solution.
    </p>

    <bluebox xml:id="solnsets-translate-span" type-name="Key Observation">
      <title>Key Observation</title>
      <idx><h>Parametric vector form</h><h>particular solution</h></idx>
      <idx><h>Particular solution</h><see>Parametric vector form</see></idx>
      <idx><h>System of linear equations</h><h>particular solution of</h><see>Parametric vector form</see></idx>
      <idx><h>Solution set</h><h>translate of a span</h></idx>
      <p>
        If <m>Ax=b</m> is consistent,  the set of solutions to is obtained by taking one <term>particular solution</term> <m>p</m> of <m>Ax=b</m>, and adding all solutions of <m>Ax=0</m>.
      </p>

      <p>In particular, if <m>Ax=b</m> is consistent, the solution set is a <em>translate of a span</em>.</p>
    </bluebox>

    <p>
      <idx><h>Parametric vector form</h><h>of an inhomogeneous equation</h></idx>
      The <term>parametric vector form</term> of the solutions of <m>Ax=b</m> is just the parametric vector form of the solutions of <m>Ax=0</m>, plus a particular solution <m>p</m>.
    </p>

    <p>It is not hard to see why the <xref ref="solnsets-translate-span"/> is true.  If <m>p</m> is a particular solution, then <m>Ap=b</m>, and if <m>x</m> is a solution to the homogeneous equation <m>Ax=0</m>, then
    <me>A(x+p) = Ax + Ap = 0 + b = b,</me>
    so <m>x+p</m> is another solution of <m>Ax=b</m>.  On the other hand, if we start with any solution <m>x</m> to <m>Ax=b</m> then <m>x-p</m> is a solution to <m>Ax=0</m> since <me>A(x-p)=Ax-Ap=b-b=0</me>.
    </p>

    <remark>
      <p>Row reducing to find the parametric vector form will give you one particular solution <m>p</m> of <m>Ax=b</m>.  But the <xref ref="solnsets-translate-span"/> is true for any solution <m>p</m>.  In other words, if we row reduce in a different way and find a different solution <m>p'</m> to <m>Ax=b</m> then the solutions to <m>Ax=b</m> can be obtained from the solutions to <m>Ax=0</m> by either adding <m>p</m> or by adding <m>p'</m>.</p>
    </remark>

    <example xml:id="solnsets-eg-point">
      <title>The solution set is a point</title>
      <statement>
        <p>What is the solution set of <m>Ax=b</m>, where
        <me>A = \mat{1 3 4; 2 -1 2; 1 0 1} \sptxt{and} b = \vec{0 1 0}?</me></p>
      </statement>
      <solution>
        <p>We form an augmented matrix and row reduce:
        <me>\amat{1 3 4 0; 2 -1 2 1; 1 0 1 0}
        \quad\xrightarrow{\text{RREF}}\quad
        \amat{1 0 0 -1; 0 1 0 -1; 0 0 1 1}.
        </me>
        The only solution is <m>p=\vec{-1 -1 1}</m>.
        </p>

        <p>According to the <xref ref="solnsets-translate-span"/>, this is supposed to be a translate of a span by <m>p</m>.  Indeed, we saw in the first <xref ref="solnsets-eg-no-nontrivial"/> that the only solution of <m>Ax=0</m> is the trivial solution, i.e., that the solution set is the one-point set <m>\{0\}</m>.  The solution set of the inhomogeneous equation <m>Ax=b</m> is
        <me>\{0\} + \vec{-1 -1 1}.</me>
        Note that <m>\{0\} = \Span\{0\}</m>, so the homogeneous solution set is a span.
        <latex-code>
              <![CDATA[
\begin{tikzpicture}[myxyz, x={(-.3cm,-.5cm)}, z={(0cm,1.3cm)}, thin border nodes]
  \path[clip, resetxy] (-6,-4) rectangle (6,4);

  \def\v{(-1, -1, 1)}

  \begin{scope}[transformxy]
    \fill[white, nearly opaque] (-3, -3) rectangle (3, 3);
    \draw[grid lines] (-3, -3) grid (3, 3);
    \draw[->] (-3,0) -- (3,0);
    \draw[->] (0,-3) -- (0,3);
  \end{scope}

  \draw[vector, seq3] (0,0,0) -- \v;

  \draw[thin, densely dotted] \v -- (-1, -1, 0);
  \point[fill=seq-violet, "$Ax=b\,$" {above, seq-violet}] at \v;
  \point[fill=seq-blue, "$Ax=0$" {below right, seq-blue}] at (0,0,0);

\end{tikzpicture}
              ]]>
        </latex-code></p>
      </solution>
    </example>

    <p>See the interactive figures in the next <xref ref="solnsets-ss-and-cs"/> for visualizations of the <xref ref="solnsets-translate-span"/>.</p>

    <bluebox xml:id="soln-sets-dim-inhom"><title>Dimension of the solution set</title>
    <idx><h>Line</h><h>dimension-1 solution set</h></idx>
    <idx><h>Plane</h><h>dimension-2 solution set</h></idx>
    <idx><h>Space</h><h>dimension-3 solution set</h></idx>
    <idx><h>Dimension</h><h>of a solution set</h></idx>
    <p>As in this <xref ref="soln-sets-dim-inhom"/>, when there is one free variable in a consistent matrix equation, the solution set is a line<mdash/>this line does not pass through the origin when the system is inhomogeneous<mdash/>when there are two free variables, the solution set is a plane (again not through the origin when the system is inhomogeneous), etc.
    </p>
    </bluebox>

	<p>
	Again compare with this <xref ref="pivot-cols-dim"/>.
	</p>

  </subsection>

  <subsection xml:id="solnsets-ss-and-cs">
    <title>Solution Sets and Column Spans</title>
    <idx><h>Column space</h><h>versus the solution set</h></idx>
    <idx><h>Solution set</h><h>versus the column space</h></idx>

    <p>To every <m>m\times n</m> matrix <m>A</m>, we have now associated two completely different geometric objects, both described using spans.
    <ul>
      <li>
        <p>The <term>solution set</term>: for fixed <m>b</m>, this is the set of all <m>x</m> such that <m>Ax=b</m>.
        <ul>
          <li>This is a span if <m>b=0</m>, and it is a translate of a span if <m>b\neq 0</m> (and <m>Ax=b</m> is consistent).</li>
          <li>It is a subset of <m>\R^n</m>.</li>
          <li>It is computed by solving a system of equations: usually by row reducing and finding the parametric vector form.</li>
        </ul>
        </p>
      </li>
      <li>
        <p>
          The <term>span of the columns of <m>A</m></term>: this is the set of all <m>b</m> such that <m>Ax=b</m> is consistent.
          <ul>
            <li>This is always a span.</li>
            <li>It is a subset of <m>\R^m</m>.</li>
            <li>It is not computed by solving a system of equations: row reduction plays no role.</li>
          </ul>
        </p>
      </li>
    </ul>
    </p>

    <bluebox><p>Do not confuse these two geometric constructions!  In the first the question is which <m>x</m>’s work for a given <m>b</m> and in the second the question is which <m>b</m>’s work for some <m>x</m>.</p></bluebox>

    <example hide-type="true">
      <title>Interactive: Solution set and span of the columns (1)</title>
      <figure>
        <caption>Left: the solution set of <m>Ax=b</m> is in violet.  Right: the span of the columns of <m>A</m> is in violet.  As you move <m>x</m>, you change <m>b</m>, so the solution set changes<mdash/>but all solution sets are parallel planes.  If you move <m>b</m> within the span of the columns, the solution set also changes, and the demo solves the equation to find a particular solution <m>x</m>.  If you move <m>b</m> outside of the span of the columns, the system becomes inconsistent, and the solution set disappears.
        </caption>
        <mathbox source="demos/Axequalsb.html?mat=1,3:2,6&amp;x=1,1" height="600px"/>
      </figure>
    </example>

    <example hide-type="true">
      <title>Interactive: Solution set and span of the columns (2)</title>
      <figure>
        <caption>Left: the solution set of <m>Ax=b</m> is in violet.  Right: the span of the columns of <m>A</m> is in violet.  As you move <m>x</m>, you change <m>b</m>, so the solution set changes<mdash/>but all solution sets are parallel planes.  If you move <m>b</m> within the span of the columns, the solution set also changes, and the demo solves the equation to find a particular solution <m>x</m>.  If you move <m>b</m> outside of the span of the columns, the system becomes inconsistent, and the solution set disappears.
        </caption>
        <mathbox source="demos/Axequalsb.html" height="600px"/>
      </figure>
    </example>

    <example hide-type="true" xml:id="solnsets-line-plane">
      <title>Interactive: Solution set and span of the columns (3)</title>
      <figure>
        <caption>Left: the solution set of <m>Ax=b</m> is in violet.  Right: the span of the columns of <m>A</m> is in violet.  As you move <m>x</m>, you change <m>b</m>, so the solution set changes<mdash/>but all solution sets are parallel planes.  If you move <m>b</m> within the span of the columns, the solution set also changes, and the demo solves the equation to find a particular solution <m>x</m>.  If you move <m>b</m> outside of the span of the columns, the system becomes inconsistent, and the solution set disappears.
        </caption>
        <mathbox source="demos/Axequalsb.html?mat=1,0,-1:0,1,1:1,1,0" height="600px"/>
      </figure>
    </example>

  </subsection>

</section>