Let's look at splitting methods a little
bit more generally. Okay,
so far the driving example has always
been A u equals f.  We were trying to
solve u and the choice of u and f came
from the fact that traditionally when
solving Poisson equations, you use u for (the vec) the u function that they're trying
to solve for and f for the force
vector, for the force function. Now we like
using x and y so let's actually continue
now our discussion with x and y.  And so
far, we've taken our matrix A and we have
partitioned it into the diagonal minus
negative of the lower, strictly lower
triangular part minus the transpose of that.
And now we want to do it more generally,
where actually the matrix does not
necessarily need to be symmetric.  And
therefore, we are going to replace this
with a U. Okay? So that is the negative of
the strictly upper triangular part of the
matrix. And we can use U because we
won't be confused with the vector u that
we had before, because we've switched
here to the vector x in that position.
Alright?  We all aboard?  So we can
then come up with different splitting
methods.  But actually we can get away
from all of these details altogether and
simply say, "Well, let's do an analysis of
what happens when you take your matrix A
and you split it into a matrix M and a
matrix N."  And your method then becomes
you solve with M,
in this way, as you've seen before, except
now using x and y instead of u and f.  And
this was the general formulation of
splitting method where for the Jacobi
iteration M was the diagonal and N was everything else, well the
negative of everything else. For Gauss-Seidel M was the lower triangular
matrix. N was the transpose of the
strictly lower triangular parts,
except negated, etc., etc., etc. Okay? So
the question now is, "How can we show that
the sequence of vectors x^(k) actually
converges to the answer?"  And for that,
we're going to do a little bit of theory.
Now we know that this is true.  But we
also know that the actual solution must
satisfy this right here. Why?  Because we
want A times x to be equal to y.  If we
plug in M minus N for A, then we can
bring the minus N times x to the other
side.  And therefore this is actually
equivalent to stating this right here.
What you can then do is say, "Okay, let's
take this equation and subtracted it off of
that equation." Happily the y's cancel
each other, and what you get is that M
times the quantity x^(k+1) minus the
actual solution is equal to N times x^(k)
minus the actual solution. Okay?  And this is
good because now we're talking about how
close is x^(k+1) to x relative to how
close x^(k) was to x.  And then it actually
becomes beneficial to bring M to the
other side. So we get that x^(k+1)
minus x is equal to
M inverse times N times x^(k) minus x.
Ah, and then we can take norms. Okay?
Because we can say, "Oh, we know that the norm of this is just equal to the norm
of this."  And then if we use a submultiplicative matrix norm or a
subordinate matrix norm, or whatever you
want to call it, then we know that this
is less than or equal to M inverse N
times x^(k) minus x.  This obviously is less
than or equal to M inverse N squared
times x^(k-1) minus x because we can
just telescope this out. And if you do
that many, many times, then you can bring
the original vector x_0 into the picture.
And you can see that this right here
should be M inverse N to --- good question to
what power?  Well if you look at it, the
power that you raise this to increases
every time and this here decreases every
time so this plus this has to be kept
constant.  This plus this power is k plus
1 so 0 plus this power has to be k plus
1.  So this is raised to the k+1
power.  Now what does this mean? If for
this matrix norm, the norm of M inverse N
is less than 1, then we know that raising
that to the kth power and letting k go to
infinity makes a coefficient go to 0. And
therefore we know that our sequence of
vectors converges to the right answer. But actually it's a
little bit simpler than that.  If for any
matrix norm, M inverse N in that norm
is less than 1, then we know that in that
norm, it doesn't need to be the same norm
that we chose here, in that norm this
quantity goes to 0.  And then by
equivalence of norms, we know that every
other norm that quantity goes to 0 as well.
And therefore, if we can find some norm for which we can prove that M inverse N in
that norm is less than 1, then we know
that's our splitting method actually
converges.
