2.8 Subspaces of R
Column Space
Proof (Proof of Theorem 1). Suppose \(B\) is the RREF of \(A\), then there’s a sequence of elementary matrices, \(E_1,\dots,E_p\), where \(E_p\cdots E_1 A=B\). Let \(E=E_p\cdots E_1\), so we have \(EA=B\).
Denote the columns of \(A\) and \(B\) as \(\mathbf{a}_k\) and \(\mathbf{b}_k\), respectively. The linear relationship in the columns of \(A\) is preserved in the columns of \(B\) because \[\begin{align} EA &= E\begin{bmatrix}\mathbf{a}_1 & \cdots & \mathbf{a}_n\end{bmatrix} \\ &= \begin{bmatrix}E\mathbf{a}_1 & \cdots & E\mathbf{a}_n\end{bmatrix} \\ &= \begin{bmatrix}\mathbf{b}_1 & \cdots & \mathbf{b}_n\end{bmatrix} \end{align}\] Thus, if \(\mathbf{a}_k = \sum_{j\neq k} c_j\mathbf{a}_j\), \(\mathbf{b}_k=E\mathbf{a}_k=E\sum_{j\neq k} c_j \mathbf{a}_j=\sum_{j\neq k} c_j E\mathbf{a}_j=\sum_{j\neq k} c_j\mathbf{b}_j\).
Conversely, since \(E\) is invertible, we can prove the linear relationship in the columns of \(B\) is preserved in the columns of \(A\) by replacing \(E\) in the above proof with \(E^{-1}\) and swapping \(A\) and \(B\).
Therefore, we have proved the linear relationship is the same in both \(A\) and \(B\).
Remark. It’s easy to show that every non-pivot column can be written as a linear combination of the prior pivot columns by the definition of RREF, so we can safely remove the non-pivot columns from the set of the columns without changing the span. Thus, we have completely proved Theorem 1.
Although the pivot columns of \(A\) form a basis for \(\operatorname{Col} A\), it’s not the only basis that can be formed from the columns. If we do row reduction in a different order of the columns, the pivot columns may be different.
Remark. It may be a bit confusing why we need to prove both directions here. We can formalize this proof using sets. Let \(U\) be the set that represents the linear relationship in \(A\), then \(U=\{(k,c_1,\dots,c_n):\mathbf{a}_k=c_1\mathbf{a}_1+\dots+c_{k-1}\mathbf{a}_{k-1}+c_{k+1}\mathbf{a}_{k+1}+\dots+c_n\mathbf{a}_n\}\). In the same way, we define \(V\) to be a set containing the linear relationship in \(B\).
The direction that is directly written in the above proof shows that for every combination \(t\in U\), \(t\in V\), so \(U\subseteq V\). The opposite direction shows that \(V\subseteq U\), so together we show that \(U=V\).
Null Space
Definition 1 (Null Space) The null space of a matrix \(A\) is the set \(\operatorname{Nul} A\) of all solutions of the homogeneous equation \(A\mathbf{x}=\mathbf{0}\).
Proof (Proof of Proposition 1). This is how we solve equations.
Proof (Proof to Proposition 3 (Informal)). The solution to the system of linear equations is also the solution to each linear equation.
Proof. Since \(A\mathbf{x}=\mathbf{0}\), by the rule of matrix multiplication, \(\operatorname{row}_i(A)\mathbf{x}=0\) for all \(1\leq i\leq m\). Thus, each vector \(x\) in \(\operatorname{Nul} A\) is orthogonal to the row space.
The first sentence is a special case of the second.
Remark. Let’s construct a basis for the null space of a row of \(A\).
The null space of the \(i\)-th row is \(\{(x_1,\dots,x_n):r_1x_1+\dots+r_nx_n=0\}\), where \(r_j\) is the \(j\)-th column of the \(i\)-th row of \(A\).
Set \(x_2=1\) and \(x_3,\dots,x_n=0\), we have \((-\frac{r_2}{r_1},1,0,\dots,0)\) as one vector in the basis.
Similarly, we can obtain the other \(n-2\) vectors in the basis.
\[(-\frac{r_3}{r_1},0,1,0,0,\dots,0)\] \[(-\frac{r_4}{r_1},0,0,1,0,\dots,0)\] \[\cdots\] \[(-\frac{r_n}{r_1},0,0,0,0,\dots,1)\]
Linear Transformation
Remark. Consider a linear transformation \(T: \mathbb{R}^n \to \mathbb{R}^m\), where \(T(\mathbf{x})=A\mathbf{x}\). \(\operatorname{Col}A\) is the range of the transformation, and \(\operatorname{Nul}A\) is the set of \(\mathbf{x}\in\mathbb{R}^n\) whose image under transformation \(T\) is \(\mathbf{0}\). Thus, for every \(T(\mathbf{u})=\mathbf{v}\) and \(\mathbf{p}\in\operatorname{Nul}A\), \(T(\mathbf{u}+\mathbf{p})=\mathbf{v}\).