Change of Basis

Inspiration

Let's say we want to answer this question: What do you get when you project the vector $\begin{bmatrix}3 \\ 2 \\ 1\end{bmatrix}$ onto the plane $x - y + z = 0$? That seems very difficult. Let's ask a new question. What do you get when you project the vector $\begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix}$ onto the $xy$ plane? Well that's easy, it's just $\begin{bmatrix}1 \\ 2 \\ 0\end{bmatrix}$. If we can change our bases so that the first question becomes like the second question, then we can solve the problem. Before we do that, we have to have some background.

Background

Here's a quick analogy. Let's say someone tells you they have 10 dogs. That's a lot of dogs! Well, it turns out that person is kinda weird and meant $10_2$ dogs = 2 dogs. That's a normal amount. In linear algebra, the same confusion can occur. In our basis, when we say $\begin{bmatrix}2 \\ 1 \end{bmatrix}$, we mean $2\begin{bmatrix}1 \\ 0 \end{bmatrix} + 1\begin{bmatrix}0 \\ 1\end{bmatrix}$. Our vector equals 2 times the first basis vector + the second basis vector. Someone else could say $\begin{bmatrix}2 \\ 1\end{bmatrix}$ and mean $2\begin{bmatrix}7 \\ 3 \end{bmatrix} + 1\begin{bmatrix}-1 \\ 1\end{bmatrix}$. In order to clear up the confusion, we say this: $$\begin{bmatrix}2 \\ 1\end{bmatrix}_{\mathcal{E}}\text{ which means the vector in our standard basis}$$ $$\begin{bmatrix}2 \\ 1\end{bmatrix}_{\mathcal{B}}\text{ which means the vector in } \mathcal{B} = \left\{\begin{bmatrix}7 \\ 3\end{bmatrix}, \begin{bmatrix}1 \\ -1\end{bmatrix}\right\} $$

Converting Between Bases

We're given this point: $$\begin{bmatrix}2 \\ 1\end{bmatrix}_{\mathcal{B}} \mathcal{B} = \left\{\begin{bmatrix}7 \\ 3\end{bmatrix}, \begin{bmatrix}1 \\ -1\end{bmatrix}\right\} $$ We want to understand it in our basis. $$\begin{bmatrix}2 \\ 1\end{bmatrix}_{\mathcal{B}} = 2\begin{bmatrix}7 \\ 3\end{bmatrix} +1\begin{bmatrix}1 \\ -1\end{bmatrix} = \begin{bmatrix}7 & 1 \\3 & - 1\end{bmatrix}\begin{bmatrix}2\\1\end{bmatrix} = \begin{bmatrix}15\\5\end{bmatrix}$$ We can rewrite this generally with two bases $\mathcal{B}$ and $\mathcal{C}$. $$x_{\mathcal{C}} = \begin{bmatrix}[b_1]_{\mathcal{C}}& [b_1]_{\mathcal{C}}\end{bmatrix}x_{\mathcal{B}} $$ $$x_{\mathcal{C}} = \underset{C \leftarrow B}{\mathcal{P}} x_{\mathcal{B}} $$

Some Practice

$$\text{Let } \mathcal{B} = \left\{\begin{bmatrix}7\\5\end{bmatrix}, \begin{bmatrix}-3\\-1\end{bmatrix}\right\}, \mathcal{C} = \left\{\begin{bmatrix}1\\-5\end{bmatrix}, \begin{bmatrix}-2\\2\end{bmatrix}\right\}$$ Find the change of coordinate matrix from $\mathcal{B} \to \mathcal{C}$ and vice-versa.
The first thing we should realize is this is a different type of problem. We have both bases described using a third basis: the elementary basis. So we have to compute: $$\underset{B \leftarrow E}{\mathcal{P}}*\underset{E \leftarrow C}{\mathcal{P}} \text{ and its inverse}$$ $$\underset{E \leftarrow C}{\mathcal{P}} = \begin{bmatrix}1 & -2\\-5 & 2 \end{bmatrix}$$ $$\underset{B \leftarrow E}{\mathcal{P}} = \underset{E \leftarrow B}{\mathcal{P}}^{-1} = \begin{bmatrix}7 & -3 \\ 5 & - 1\end{bmatrix}^{-1} = \frac{1}{8}\begin{bmatrix}-1 & 3 \\ -5 & 7\end{bmatrix}$$ $$\underset{B \leftarrow C}{\mathcal{P}} = \begin{bmatrix}-2 & 1\\-5 & 3\end{bmatrix}$$

Change of Basis Formula

$$[T]_{\mathcal{C}} = \underset{\mathcal{C} \leftarrow \mathcal{B}}{P}[T]_\mathcal{B}\underset{\mathcal{B} \leftarrow \mathcal{C}}{P} = \boxed{\underset{\mathcal{B} \leftarrow \mathcal{C}}{P}^{-1}[T]_\mathcal{B}\underset{\mathcal{B} \leftarrow \mathcal{C}}{P}}$$ A good justification is to think big-picture about the problem. We want to input a point in C coordinates and output T(x) also in C coordinates. So, we convert into B coordinates, apply the transformation (still in B coordinates), and then convert back into C coordinates.

Back to our original problem

Let's rewrite this problem again, so we don't have to scroll up.
Let's say we want to answer this question: What do you get when you project the vector $\begin{bmatrix}3 \\ 2 \\ 1\end{bmatrix}$ onto the plane $x - y + z = 0$?
The elementary basis has the $\begin{bmatrix}0\\0\\1\end{bmatrix}$ as the normal vector to its plane, and it has $\begin{bmatrix}1\\0\\0\end{bmatrix}$,$\begin{bmatrix}0\\1\\0\end{bmatrix}$ as the other vectors that comprise the vector space. If we consider a point $\begin{bmatrix}x\\y\\z\end{bmatrix}$, the projection onto the plane is $\begin{bmatrix}x\\y\\0\end{bmatrix} = \begin{bmatrix}1&0&0\\0&1&0\\0&0&0\end{bmatrix}\begin{bmatrix}x\\y\\z\end{bmatrix}$.

In the same way, if we consider a coordinate system based around that plane, with $b_1 = \begin{bmatrix}1\\1\\0\end{bmatrix}, b_2 = \begin{bmatrix}0 \\ 1\\ 1\end{bmatrix}$, our new "x" and "y" vectors, and $b_3 = \begin{bmatrix}1\\-1\\1\end{bmatrix}$, then the projection of $\begin{bmatrix}a\\b\\c\end{bmatrix}_{\mathcal{B}} = \begin{bmatrix}a\\b\\0\end{bmatrix}_{\mathcal{B}}$

Here is our plan: we will convert a point in our elementary basis to $\mathcal{B}$, calculate the projection, then convert back into $\mathcal{E}$ $$y = \underset{E \leftarrow B}{\mathcal{P}}T\underset{B \leftarrow E}{\mathcal{P}}*x$$ $$y = \underset{E \leftarrow B}{\mathcal{P}}T\underset{E \leftarrow B}{\mathcal{P}}^{-1}*x$$ As we showed before, $\underset{E \leftarrow B}{\mathcal{P}}$ is just $\begin{bmatrix}b_1]_{\mathcal{E}} & [b_2]_{\mathcal{E}} & [b_3]_{\mathcal{E}}\end{bmatrix} = \begin{bmatrix}1&0&1\\1&1&-1\\0&1&1\end{bmatrix}$ We have our final answer $$y = \begin{bmatrix}1&0&1\\1&1&-1\\0&1&1\end{bmatrix}\begin{bmatrix}1&0&0\\0&1&0\\0&0&0\end{bmatrix}\begin{bmatrix}1&0&1\\1&1&-1\\0&1&1\end{bmatrix}^{-1}\begin{bmatrix}x_1\\x_2\\x_3\end{bmatrix}$$ $$y = \dfrac{1}{3}\begin{bmatrix}2&1&-1\\1&2&1\\-1&1&2\end{bmatrix}x$$ Let's plug in $x = \begin{bmatrix}3\\2\\1\end{bmatrix}$, we get $$y = \begin{bmatrix}\frac{7}{3}\\\frac{8}{3}\\\frac{1}{3}\end{bmatrix}$$ We can do two things to check this is a valid point. First, we can plug our point into the plane and we see that it works. Second, notice that $\frac{2}{3}\begin{bmatrix}1\\-1\\1\end{bmatrix} + y = x$, meaning x is directly above y, which is what we want.