The Gram-Schmidt Process
Orthonormal Basis
A set of vectors in is an orthonormal set if it is an orthogonal set of unit vectors. It is an orthonormal basis for a subspace of if it spans .
Let be an m x n matrix. Suppose its column vectors in form an orthonormal set. It is equivalent to because the -entry of is for and is an orthonormal set if and only if for and for .
Here are some nice properties of : For any vector in ,
1.
2.
(Note: If we view be a linear transformation from to , then this linear transformation preserves the inner product and norm.)
Proof:
1.
By definition, and . Hence,
2. Using (1),
In particular, when , is an n x n square matrix such that , which means is invertible and . Such a square matrix is called an orthogonal matrix. From above, we know that the linear transformation preserves the inner product and norm. Moreover, it also preserve volume because . (Why?)
If is an orthonormal basis for a subspace of , then for any in , then it is very convenient to write down its orthogonal projection onto :
If , then .
Proof: Easy
The Gram-Schmidt Process
Given a basis for a subspace of , we have a simple way called the Gram-Schmidt process to produce an orthogonal or orthonormal basis for the subspace. The idea is as follows:
Suppose you are given a basis of a subspace of . Start with and let it be the first one in the orthogonal basis that we are going to produce and call it i.e. .
Let . We consider the orthogonal decomposition and get the component in i.e.
Then is an orthogonal set.
Let . We consider the orthogonal decomposition and get the component in i.e.
Then is an orthogonal set. We repeat this procedure. At the ith step, we let and
After p steps, we obtain the orthogonal basis for . If you want to make it an orthonormal basis, then you can simply normalize each vector to , where for .
Remark: If you change the order of the vectors in the given basis, you will get a different orthogonal basis from the Gram-Schimdt process.
Example: Find an orthonormal basis for in .
You can try this online Gram-Schimdt calculator.