[noise]
Welcome. [vocalized-noise] In the [vocalized-noise]
[noise] last [vocalized-noise] session, we
[vocalized-noise] ah try [noise] to look into
[vocalized-noise] the matrix representation
[noise] of Jacobi [noise] or Gauss-Seidel
[noise] method [noise] or ah [noise] any [vocalized-noise]
ah basic [vocalized-noise] iterative solver
method, [noise] we classified [noise] Jacobi
and [noise] Gauss-Seidel as [vocalized-noise]
basic iterative solvers [noise] [vocalized-noise].
Ah And from the matrix [noise] representation,
we [vocalized-noise] ah also found out the
[noise] iteration matrix G [noise]. And observed
[noise] that for ah [vocalized-noise] certain
property of G, [noise] there is matrix [noise]
norm of G must be less than 1 [vocalized-noise].
If we start [noise] with any guess value,
the iteration should converge [noise] to the
[vocalized-noise] exact solution of A x is
equal to b, [noise] [vocalized-noise] and
that is because [noise] the error between
the guess value and the [vocalized-noise]
ah [noise] [vocalized-noise] actual solution,
[noise] which is uh [noise] maybe at [vocalized-noise]
[noise] k-th iteration level, this is x k
minus x [vocalized-noise] x star, [noise]
x star being the actual solution, [noise]
this error [vocalized-noise] will reduce in
the next iteration level, [noise] as it is
multiplied them [noise] it by [vocalized-noise]
[noise] the iteration matrix G [noise].
So, if [vocalized-noise] if this [noise] multiplication
[noise] reduces the [noise] length of the
[vocalized-noise] error vector, [vocalized-noise]
then after [vocalized-noise] number of iterations
[noise] [vocalized-noise] the reductions [noise]
will [vocalized-noise] be multiplied on each
other, and then it will [noise] go to an infinite
[noise] small value, [vocalized-noise]. And
therefore, [vocalized-noise] [noise] ah this
should converge [noise] to the exact solution
[vocalized-noise].
Now, [vocalized-noise] there is another ah
issue, which we are discussing that, [vocalized-noise]
when the iterations [noise] [vocalized-noise]
converge [noise] to the exact solution, [noise]
we can see [noise] that [vocalized-noise]
[noise] there is practically [noise] no difference
between the guess an [noise] updated value,
[noise] [vocalized-noise] they also converge
[vocalized-noise]. So, we will also look in
[vocalized-noise] in the convergence from
the perspective that the [vocalized-noise]
[noise] guess an updated [noise] value [vocalized-noise]
ah mean [noise] practically same, [noise]
they are also converging, [noise] [vocalized-noise]
and what are the requirements [noise] for
them [vocalized-noise].
So, we [noise] [vocalized-noise] start with
[noise] ah [noise] assume that convergence
[noise] is [vocalized-noise] achieved at [vocalized-noise]
k plus 1th step [noise] [vocalized-noise].
The convergence step then is [noise] x k plus
1 [noise] is equal to [noise] G x k plus f
[vocalized-noise] [noise].
Now, [noise] the step [noise] before that,
which is [noise] k-th [noise] iteration [noise]
step, [noise] here it is not converge. Convergence
means [noise] the [vocalized-noise] difference
[noise] from the exact solution and [noise]
the [vocalized-noise] [noise] ah [noise] actual
[noise] solution [noise] is [vocalized-noise]
of the [vocalized-noise] of a very small [noise]
number, [noise] and we [noise] [vocalized-noise]
make a [vocalized-noise] [noise] criteria
for that, say this number is10 to the power
minus 8 [noise]. When the difference is less
than 10 to the power minus 8, [noise] it will
tell that [noise] [vocalized-noise] it has
converged [vocalized-noise].
So, let us assume that this [noise] convergence
has been done in [noise] k-th [noise] iterations
[vocalized-noise] k plus 1th [noise] iteration
step [noise] [vocalized-noise]. And in k-th
[noise] iteration step, though it is [noise]
not converged, but [noise] [vocalized-noise]
the ah [vocalized-noise] relation of the guess
and [noise] updated value [noise] is similar,
x k is equal to [noise] G x k minus 1 [noise]
ah plus f [vocalized-noise] ah so [noise]
on, we can come down to [vocalized-noise]
first iteration [noise] step, [noise] which
is x 1 is equal to G x 0 minus f [vocalized-noise].
Using the above relationships, [noise] [vocalized-noise]
we can write [noise] x [noise] k minus 1 is
equal to G of x [vocalized-noise] x k plus
1 minus k is equal to G of x k minus 1. By
using [vocalized-noise] these two, [noise]
we can write this [noise] x k plus 1 [noise]
is equal to [noise] G of this [noise] [vocalized-noise].
Now, again using [noise] ah the next one x
k, [noise] we can write that [noise] ah x
k minus 1 is equal to G of x k [noise] [vocalized-noise]
ah um [vocalized-noise] minus 2 sorry [vocalized-noise]
[noise] G of x [noise] k minus 2 plus f [noise]
[vocalized-noise]. So, if we [noise] um [vocalized-noise]
ah [vocalized-noise] if we subtract from [noise]
these two, we will get [noise] x k minus x
k minus 1 [noise]. So, this [noise] minus
this, [noise] this [noise] minus this [noise]
[vocalized-noise] is minus x [vocalized-noise]
k minus 1 is equal to G of x k minus 1 minus
x k minus 2 [noise] [vocalized-noise].
So, [noise] the [vocalized-noise] if now [noise]
[vocalized-noise] this we substitute by G
f [noise] x k minus minus x k minus 2, [noise]
we will get G square [vocalized-noise] x k
minus minus x k minus 2 and [vocalized-noise]
so on, [noise] we will get [noise] G [noise]
[vocalized-noise] to the power k [noise] x
1 [noise] minus x 0, [noise] [vocalized-noise]
till we come to [noise] this equation [noise]
[vocalized-noise]. So, the difference is also
being [noise] multiplied by G to the power
k, [noise] reference from the [noise] guess
and [noise] updated value [noise] [vocalized-noise]
[noise].
So, we [vocalized-noise] write [noise] x k
plus 1 is [noise] equal to [noise] [vocalized-noise]
minus x k's [noise] [vocalized-noise] so on,
it is G to the power k x 1 minus x 0 [noise].
And the [noise] first iteration step is [noise]
x 1 is equal to G x 0 [noise] minus f [vocalized-noise]
[noise]. So, [noise] substituting x 1, [noise]
so what we will do, we will [noise] substitute
x 1 [noise] into the [noise] ah relationship
a [noise] [vocalized-noise]. We can write
that [noise] this combination [vocalized-noise]
G x k [noise] minus 1 x k [noise] etcetera,
[vocalized-noise] G to the power k [noise]
[vocalized-noise] I minus G x 0 [noise] plus
f [noise] [vocalized-noise] [noise].
Now, [noise] [vocalized-noise] I minus G is
a [noise] [vocalized-noise] the initial requirement
was that I minus G must be a non-singular
matrix, so if it is a non-singular matrix
[vocalized-noise] [noise]. ah For any x 0,
[vocalized-noise] which is multiplied by G
to the power k [noise] this should will be
a very small number, [vocalized-noise] if
G also [vocalized-noise] again has a matrix
norm [vocalized-noise] [noise] less than 1
[noise] [vocalized-noise].
So, we [noise] can say that [noise] for convergence,
[noise] x k [noise] plus 1 [noise] and x k
we have practically same value [noise]. And
therefore, [noise] that [noise] difference
is [noise] [vocalized-noise] infinite small,
[noise] which will need [noise] that [noise]
x k plus 1 minus x k is less than epsilon,
[noise] ah we got x k plus 1 is minus x k
is [noise] G to the power k I minus G [noise]
x 0 plus f [vocalized-noise] [noise]. x 0
can be ah [noise] assume that I minus [noise]
G is a [noise] non-singular matrix, [noise]
which is requirement [noise] for a Gauss-Seidel
step [noise] [vocalized-noise]. x 0 is a [noise]
[vocalized-noise] is an arbitrarily chosen
vector, so x 0 can be anything. Whatever be
the value of x 0, this value has to be less
than a very small number epsilon, [noise]
[vocalized-noise] this value has to be less
than ah [vocalized-noise] a value [noise]
has to be [noise] an infinite small value
[vocalized-noise].
And [noise] that needs that [vocalized-noise]
the convergence [noise] of [vocalized-noise]
[noise] k plus 1-th step [noise] which is
x k plus [noise] 1 minus x k will be same
convergence [noise] in that term, [vocalized-noise]
[noise] will happen [noise] [vocalized-noise]
only if the norm [vocalized-noise] matrix
norm [noise] of G k plus 1 goes to 0 [noise].
Then this value if if G k plus 1 is [noise]
if this happens, [noise] then this value [vocalized-noise]
should also [noise] go to 0 [noise] or should
be less than epsilon [noise]. And this is
[noise] again possible, [noise] if matrix
norm [noise] of G is less than 1 [noise] [vocalized-noise].
So, [vocalized-noise] [noise] we can look
into two steps things, [noise] one is that
[noise] if [noise] modulus of [noise] [vocalized-noise]
if matrix [noise] norm of G [noise] is less
than 1, [noise] then the solutions will converge
in a sense that [noise] the difference between
two [noise] guess an updated will be [vocalized-noise]
is essentially 0, very small number [noise].
The solution [vocalized-noise] the [vocalized-noise]
the [noise] iterations will converge [noise]
to a value [vocalized-noise]. And the converge
value [noise] will be the exact [noise] solution
[noise]. So, both things are [noise] satisfied,
[noise] [vocalized-noise] if [noise] the matrix
norm [noise] of G is less than 1 [noise] [vocalized-noise].
So, A x is equal to b, [noise] [vocalized-noise]
ah we will [noise] look into convergence and
[vocalized-noise] accuracy [noise] of the
[vocalized-noise] iterative method [noise].
A x is equal to b, we have the iterative [noise]
step [vocalized-noise] x k plus 1 is equal
to G x k plus f [noise] [vocalized-noise].
If [noise] matrix norm of G is less than 1,
[noise] then ah [vocalized-noise] if [vocalized-noise]
[noise] sorry [noise] if matrix norm of [noise]
this is not G k plus 1, [noise] if matrix
norm of [noise] the if [noise] [vocalized-noise]
G is less than 1, matrix norm of G to the
power k [noise] plus 1 is basically multiplying
[noise] the matrix [noise] several times,
this goes to 0 [noise] [vocalized-noise].
And [noise] once this happens, [noise] the
equat[ion] [vocalized-noise] iterations converge
for some k [noise] for which [vocalized-noise]
the difference between guess an updated value
is very small [noise] [vocalized-noise]. Also
[noise] the converge solution x k [noise]
becomes practically same as the [noise] exact
solution x star, because these two differences
are [noise] small [vocalized-noise]. So, therefore,
[noise] if [noise] the first thing happens,
[noise] if the first thing happens, [noise]
this will happen for this particular criteria,
[noise] and which [vocalized-noise] for which
this should also happen, [noise] or if the
reverse happens like [vocalized-noise] the
solution [noise] practically converges to
the exact solution, [vocalized-noise] ah we
will see that the [vocalized-noise] the iteration
[vocalized-noise] guess an updated value [vocalized-noise]
has also converge to the [noise] same value
[noise] [vocalized-noise].
So, if [noise] the iterations [noise] at all
converge, [noise] if we [vocalized-noise]
[noise] come into a stage [noise] that the
x [vocalized-noise] x value is not being [noise]
updated guess an iterations [noise] are being
same, [vocalized-noise] then the iterations
have converged to the right solution [noise].
And that is the [vocalized-noise] Robustness
of this particular technique, Gauss-Seidel
[noise] or ah Jacobi or we will see successive
[noise] over relaxation [noise] as a [vocalized-noise]
class of these methods [noise] [vocalized-noise].
This this class of technique [noise] that
if we can have converging [vocalized-noise]
solution [vocalized-noise] converging iteration,
[vocalized-noise] the [vocalized-noise] [noise]
the final result must be [noise] same [noise]
as the [noise] [vocalized-noise] actual result
[noise]. So, we can [noise] sum it up as,
[noise] [vocalized-noise] if the iterations
converge, [vocalized-noise] they will converge
[noise] to the exact [noise] solution [noise]
[vocalized-noise].
Now, [noise] the questions [noise] come that
[noise] under which case [noise] iterations
will converge, [noise] of course we know that
by now that mod G ah [noise] matrix norm [noise]
of G should be less than 1 [noise] [vocalized-noise].
And [noise] when matrix norm [noise] of G
will be less than 1, we have to look into
the A matrix, because [vocalized-noise] we
are doing very [noise] ah [vocalized-noise]
two steps to get G matrix from [vocalized-noise]
we we are doing one type of splitting of A
in [noise] Gauss [noise] Jacobian, [noise]
we are doing another type of splitting of
A in [vocalized-noise] ah Gauss-Seidel, but
this is basically splitting of the matrix
A [noise] [vocalized-noise].
So, [noise] through splitting of the matrix
A, [noise] we are getting [noise] ah the iteration
matrix G [vocalized-noise]. The iteration
matrix G has a [vocalized-noise] ah norm less
than 1, [vocalized-noise] [noise] this is
our requirement [noise] for convergence of
[noise] Gauss-Seidel Jacobi methods [noise]
[vocalized-noise]. And this requirement [noise]
is satisfied [noise] based on what [noise]
based on how is the A matrix [noise]. So,
you have to [noise] look into the A matrix,
and see that in under which cases [noise]
splitting of A matrix [noise] [vocalized-noise]
gives us a G [vocalized-noise] which has [noise]
matrix norm less than 1 [noise] [vocalized-noise].
Another thing [noise] we can also see that
[noise] the [noise] value x k plus 1, [noise]
if we go to the [noise] [vocalized-noise]
may be [noise] we can go to the [noise] previous
slide [noise] and [vocalized-noise] [noise]
or here [noise] we can see also, [noise] [vocalized-noise]
that [noise] this value [noise] x k plus 1
minus x k [noise] is [noise] something is
[noise] this is [noise] like this value [noise]
x k plus 1 [noise] minus x k [noise] is [noise]
G to the power k into [noise] something [noise].
Similarly, x star [noise] minus x k [noise]
is also G to the power k into something [noise]
[vocalized-noise]. So, all this is [noise]
some initial value multiplied with G to the
power k [vocalized-noise].
So, [noise] [vocalized-noise] what is G, if
[noise] G is a very small number [noise] [vocalized-noise]
within very less number of steps, [noise]
G to the power k will be a [vocalized-noise]
infinite small number [noise]. If G has to
be [noise] less than 1, but G [vocalized-noise]
G is [vocalized-noise] close to 1, say [noise]
for example G is 0.9, then it will take large
number of steps [noise] [vocalized-noise].
Consider to the case, G is 0.1 [noise] [vocalized-noise]
ah matrix norm of G is 0.1, it will take less
number of [vocalized-noise] steps [vocalized-noise].
So, how first will be the convergence that
will also depending on, [vocalized-noise]
how is the G matrix [noise]. And how is the
G matrix, that depends on [noise] how is the
A matrix, [noise] because G is coming [noise]
through splitting of A [noise] [vocalized-noise].
So, we will look into [noise] [vocalized-noise]
the [noise] properties of A matrix [noise]
[vocalized-noise] for which [noise] these
things [noise] will heard [vocalized-noise].
Matrix A [vocalized-noise] [noise] [vocalized-noise]
is called [noise] ah weakly diagonally dominant
matrix, [noise] we will look into [noise]
few definition, [noise] and the definitions
of [noise] [vocalized-noise] diagonal domine
[noise] diagonally dominant [noise] matrices
[vocalized-noise]. A weakly diagonally [noise]
dominant matrix [noise] if [noise] the [noise]
all the [vocalized-noise] any diagonal term
[noise] we considered [noise] any diagonal
term [vocalized-noise] that is greater than
the [noise] all of diagonal terms [noise]
of that particular [noise] sum of all of diagonal
terms of that particular row [noise] in their
absolute [noise] form greater than equal to,
then we called weakly diagonal [noise]. They
may be [vocalized-noise] greater than the
[noise] sum of off diagonals, or may be is
equal to the sum of off diagonals [noise]
[vocalized-noise].
We call it [noise] strictly diagonally [noise]
dominant, if [noise] all the diagonal terms
is greater than [noise] equal to sum of [noise]
the of diagonal terms [noise] of that particular
row, [noise] except the [vocalized-noise]
diagonal term of [noise] [vocalized-noise]
all all diagonal terms [vocalized-noise] [noise].
We call it [noise] it to be irreducibly [noise]
diagonally dominant, [noise] if [noise] for
all j, [noise] it is weakly diagonally dominant,
[noise] but there is at least one row [noise]
or at least one j [noise] [vocalized-noise]
for which the [noise] diagonal dominance is
there [noise].
So, [vocalized-noise] [noise] for all j, [vocalized-noise]
absolute value of [vocalized-noise] the diagonal
term is greater [noise] than equal to [vocalized-noise]
sum of the [noise] [vocalized-noise] absolute
value of the off diagonal [noise] terms [vocalized-noise].
ah But, there is at least one j [noise] for
which [noise] at least one row for which [noise]
the diagonal term [vocalized-noise] is [noise]
absolute value of diagonal term is [vocalized-noise]
[noise] greater than this is not greater than
equal to, [vocalized-noise] [noise] this should
be [noise] greater than [noise] [vocalized-noise]
greater than [noise] the sum of [vocalized-noise]
[noise] ah [vocalized-noise] [noise] off [noise]
diagonal terms [noise] [vocalized-noise].
So, [noise] and the requirement is that [noise]
[vocalized-noise] ah another thing is that
irreducible [noise] or strictly diagonally
dominant [noise] matrices [noise] show non-zero
pivots, [noise] at least in [vocalized-noise]
in a permuted form, [vocalized-noise] and
hence [vocalized-noise] non-singular; [vocalized-noise]
not in permuted from, [noise] also [vocalized-noise]
a non-permuted form [noise] also [vocalized-noise].
Strictly [noise] diagonal matrix are reduced
[noise] severally [noise] matrix [noise] as
[vocalized-noise] [noise] show non-zero pivots
[noise]. So, the [noise] permuted form [noise]
is not important here [noise] [vocalized-noise].
And therefore, [noise] these matrices [noise]
strictly diagonally dominant or [noise] irreducibly
diagonal [vocalized-noise] dominant matrices
[vocalized-noise] are [noise] non-singular
also, [noise] [vocalized-noise] where we discussing
this, because it has a relation. Remember
[noise] at the [vocalized-noise] beginning
when [vocalized-noise] is introducing Gauss-Seidel,
ah [vocalized-noise] I was introducing Jacobi,
I said that this is only valid for diagonally
dominant matrices [noise] [vocalized-noise].
Now, we will see [vocalized-noise] that only
for diagonally dominance [noise] or irreducibility
diagonally dominant matrix, [vocalized-noise]
the [vocalized-noise] G will be such that
[noise] that the [vocalized-noise] matrix
norm of G is less [noise] than 1, and that
is why these methods will be valid [noise]
[vocalized-noise].
If A is strictly [noise] diagonally dominant
[noise] or irreducibly diagonally dominant
matrix, [vocalized-noise] then the [noise]
associated Jacobi or Gauss-Seidel iterations
[noise] converge for [noise] any x 0 [noise].
This is the theorem for [noise] convergence
of [vocalized-noise] iteration [vocalized-noise].
When the Jacobi and Gauss-Seidel [noise] iterations
converge, when [noise] G has a [noise] matrix
norm [noise] less than 1 [noise]. What is
G, [noise] G is the [vocalized-noise] G is
a split [vocalized-noise] comes from [noise]
splitting of A [noise].
And this condition if we look into A [noise]
comes as if A strictly diagonally dominant
or [vocalized-noise] or an irreducibly [vocalized-noise]
diagonally dominant matrix, [vocalized-noise]
then associated [vocalized-noise] Jacobi [noise]
or Gauss-Seidel converge that means, G [noise]
will have matrix norm [noise] less than [noise]
[vocalized-noise] 1 [noise]. So, ah [noise]
for which type of matrices, [noise] this will
converge, [noise] for example the first matrix
5 0 4, 1 3 2, 2 6 8 [noise] is A matrix [noise]
[vocalized-noise]. So, these are the A matrices
[noise] [vocalized-noise].
Now, if I [noise] look here, this is a diagonally
dominant line, 5 [noise] is greater than sum
of this [vocalized-noise]. This is ah weakly
diagonally dominant 1 plus 2 is equal to 3,
[noise] 2 plus 8 is equal to 6 [noise] [vocalized-noise].
However, [vocalized-noise] [noise] this becomes
an [noise] irreducibly [noise] diagonal [noise]
dominance is there [noise] irreducibly diagonally
dominant matrix [noise] [vocalized-noise].
And this is the [noise] diagonally dominant
matrix [noise] [vocalized-noise]. So, for
these two, [noise] this ah A x is equal to
b can be solvable [noise] using Jacobi or
Gauss-Seidel [noise] [vocalized-noise].
If [noise] in case, this would have been 4
[noise] in case instead of 5, this is 4, [noise]
[vocalized-noise] we could not have been able
to solve this equation, [noise] because then
[vocalized-noise] it would have been a weakly
diagonally dominant [noise]. So, for [noise]
weakly diagonally dominant [noise] matrix,
[vocalized-noise] [noise] Gauss-Seidel or
Jacobi cannot work. You can ah [vocalized-noise]
try, you will will discuss about writing our
own program, but you can try also [vocalized-noise]
ah this [vocalized-noise] by hand, even ah
paper, pencil, you can try few steps, and
see what is happening to this [noise]. You
will see that the values are not converging
x k [vocalized-noise] plus 1 minus x k, if
if it is ah weakly diagonally dominant [vocalized-noise]
x k [vocalized-noise] [noise] plus 1 minus
x k, [vocalized-noise] it is not [noise] reducing,
it is increasing or remaining constant at
a I value [noise] [vocalized-noise].
If it is not a diagonally dominant matrix
[noise] like if we look into the first row
2 plus 4 is equal to 6, [noise] which is [vocalized-noise]
5 plus 4 [noise] is equal to 9, which is greater
than 2, [vocalized-noise] then it cannot be
solved. Also [noise] [vocalized-noise] if
we think of [noise] doing a row permutation
of this [noise] matrix, like I will ah permute
this rows, I will again [noise] permute this
and this, [noise] if we think of a row [noise]
permuted form [noise] of this matrix, [noise]
is it does not remain a diagonally dominant
matrix [vocalized-noise].
However, the solutions will remain same, [noise]
because row permuted form and the [vocalized-noise]
actual form should give us same solution [vocalized-noise].
But, if we do row permutation, [noise] the
matrix losses diagonal dominance and [noise]
we cannot solve it using [noise] Jacobi or
Gauss-Seidel [noise] [vocalized-noise]. Though
row permutations [vocalized-noise] remain
the solutions in solutions exist, [vocalized-noise]
and solutions are same with a diagonally dominant
matrix or [vocalized-noise] irreducibly diagonally
dominant matrix, [vocalized-noise] and the
row permutated matrix [noise] [vocalized-noise]
form of that matrix solutions are same. However,
[vocalized-noise] the row promoted from, as
it does not remain diagonally dominant, cannot
be solved using [vocalized-noise] Jacobi or
Gauss-Seidel method [noise] [vocalized-noise].
Why, because of the fact that G has been changed
[noise].
So, in case, we get a matrix which is solvable,
but does [noise] [vocalized-noise] it is not
in a diagonal dominant form, [vocalized-noise]
we can try row permutations, and give it to
a diagonal dominance form. And only in the
diagonal dominant form, [vocalized-noise]
the matrix can be solved. This cannot be solved
[noise] using a [vocalized-noise] Jacobi or
Gauss-Seidel method, [noise] because this
this is though it [vocalized-noise] this [noise]
these two matrices are permutated form [noise]
of each other [vocalized-noise]. However,
2 plus 2 is equal to [vocalized-noise] 5,
which is greater than 1, [noise] so this is
not a diagonally dominant form, [vocalized-noise]
so it cannot be something (Refer Time: 18:55)
[noise] [vocalized-noise].
Here comes [noise] another [noise] important
definition is that the maximum modulus of
eigenvalues of A is called spectral radius
of A or row A [noise]. And this is [noise]
given as [noise] any for any matrix norm,
we can take 1, 2, 3 [noise] up to p or infinite
[noise] matrix norm [vocalized-noise]. Matrix
norm of matrix A to the power k to the power
1 by k and as limit, k goes to infinity, this
is row of A [noise]. So, [vocalized-noise]
maximum and this is same as the maximum modulus
of eigenvalues of A spectral radius.
So, any matrix raised to a [vocalized-noise]
high power, and then we take [vocalized-noise]
matrix norm, and then ah [noise] take say
[vocalized-noise] that [vocalized-noise] root
of that matrix 2 to the power, it was raised
[vocalized-noise] with the the [vocalized-noise]
limit is the [noise] spectral radius or the
largest [vocalized-noise] eigenvalue of A
[noise]. This is a [noise] definition [noise]
[vocalized-noise].
So, let [noise] A is equal to M minus N [noise].
M, N pair is called regular splitting, if
M is non-singular [noise] and M inverse and
[vocalized-noise] ah N are [noise] non-negative
[noise] [vocalized-noise]. Then A is equal
to M N is called [vocalized-noise] a regular
splitting [noise] of matrix A [noise]. A non-negative
matrix means all [noise] elements of that
matrix are [noise] non-negative [noise].
Let [noise] M and N [noise] be a regular splitting
of matrix A [noise]. Then row M inverse N
[noise] is less than 1, if A is non-singular
and A inverse is ah [vocalized-noise] [noise]
non-negative [noise] [vocalized-noise]. So,
if M and N are [noise] [vocalized-noise] regular
splitting of A, [noise] then rho inverse [noise]
[vocalized-noise] N is ah rho [vocalized-noise]
rho of M inverse N less than a 1 [noise] is
non-singular, [vocalized-noise] if A is non-singular
and A inverse is non-negative [vocalized-noise]
[noise].
And the iteration step in that case, [noise]
the iteration step x k plus 1 is equal to
M inverse N x k plus M inverse b will converge
to rho of [vocalized-noise] M in [vocalized-noise]
converge, [noise] if this rho M inverse N
(Refer Time: 21:08) [noise]. This is basically
the G matrix right [vocalized-noise] [noise]
this is basically [noise] the G matrix [noise].
This will converge, [noise] if [vocalized-noise]
the matrix norm of [noise] G G to the power
k is [noise] very small or [vocalized-noise]
ah [vocalized-noise] matrix norm of G is less
than 1, which is spectral radius of G is less
than 1 [noise]. So, if spectral radius of
G is less than 1, x k plus 1 G x k plus f
[noise] will [noise] converge [vocalized-noise].
And this will happen, [vocalized-noise] if
M N [noise] leaner regular splitting of matrix
A that means, M is non-singular [noise] and
M inverse and N are non-negative [vocalized-noise].
And now, there is there was a scientist named
[noise] Greshgorin, [noise] [vocalized-noise]
who [noise] ah [vocalized-noise] found out
the [noise] theorem [noise] on ah uh the [vocalized-noise]
diagonal ah dominance of a matrix and its
eigenvalues. And [vocalized-noise] as a [vocalized-noise]
corollary of the theorem, [noise] we can say
that this regular splitting with as above
[noise] conditional [noise] spectral number
row [noise] I mean M inverse N is less than
1, [vocalized-noise] can only be [noise] obtained
[vocalized-noise] for irreducibly or [vocalized-noise]
strictly diagonally [noise] dominant matrices
[noise]. This is comes from [vocalized-noise]
ah Greshgorin [vocalized-noise].
So, if the if [vocalized-noise] the matrix
A is strictly [vocalized-noise] diagonally
dominant or [vocalized-noise] irreducibly
diagonally dominant, [vocalized-noise] [noise]
we can [noise] get a regular splitting of
M and N [noise] following our [vocalized-noise]
[noise] ah [noise] Jacobi or Gauss-Seidel
method [vocalized-noise]. And then, [noise]
we will also see [noise] rho of M inverse
N, where M is [vocalized-noise] M inverse
[noise] is and N are non-negative, [vocalized-noise]
[noise] rho of M inverse N [vocalized-noise]
is less than 1 [noise]. A inverse [noise]
will also be non-negative, and A is non-singular
[noise] in that case, [noise] [vocalized-noise]
if we have [vocalized-noise] dominant or [noise]
diagonally dominant matrix [noise] [vocalized-noise].
So, rho G is [noise] less than 1, [noise]
and [noise] diagonally dominance, [noise]
they are actually [noise] related [noise].
And this relation [noise] comes through [noise]
Greshgorin theorem [vocalized-noise] [noise].
I [vocalized-noise] I am not going [noise]
into detail of [noise] Greshgorin [noise]
theorem [noise] here, [noise] but this [noise]
[vocalized-noise] discusses [vocalized-noise]
how should be the ah [vocalized-noise] depending
on the eigenvalues of the matrix, [vocalized-noise]
how should be the [vocalized-noise] diagonally
dominant diagonal term and off diagonal term
[noise] and their [noise] arrangements [noise]
[vocalized-noise].
So, now we what we got [noise] is that if
the matrix [noise] is [noise] non-singular,
[noise] A inverse is non-negative, [noise]
if it can have [noise] a regular splitting,
[vocalized-noise] [noise] which is [noise]
for a irreducibili[ty] [vocalized-noise] diagonally
dominant or strictly [noise] diagonally dominant
matrix [vocalized-noise]. Then G should have
[noise] [vocalized-noise] ah [vocalized-noise]
[noise] convergent [noise] ah [vocalized-noise]
G should have a spectral radius [noise] which
is less than 1, [noise] therefore [vocalized-noise]
matrix norm of G must be less than 1, [noise]
and [vocalized-noise] matrix norm of [noise]
G to the power k [noise] [vocalized-noise]
must go to 0 [noise] [vocalized-noise].
So, let G be a [noise] square matrix, [noise]
such that [noise] rho of G is less than 1
[vocalized-noise]. Then [noise] [vocalized-noise]
I minus G is [noise] non-singular [noise]
and iteration [noise] G x k plus 1 [vocalized-noise]
is equal to G x [vocalized-noise] x k plus
1 is equal to [noise] G x k plus f, [vocalized-noise]
[noise] this iteration [vocalized-noise] also
[noise] converges for every f and x 0 [vocalized-noise]
[noise]. And its converse is [noise] also
true [noise] [vocalized-noise] [noise].
So, if G is a square matrix, [noise] so so
that [noise] rho of G [noise] is less than
1 obviously, [noise] I minus G [noise] will
be [noise] non-singular, [noise] [vocalized-noise]
and the [noise] iterations will converge,
[vocalized-noise] and that converge statement
is also true [vocalized-noise] [noise]. And
this [vocalized-noise] happens, if [noise]
and only if [noise] A is a diagonally dominant
[vocalized-noise] or [noise] irreducibility
diagonally dominant matrix. So, this [vocalized-noise]
this say these are the cases [noise] in which
we will have [noise] [vocalized-noise] convergence
of the iterative methods. And this is very
important Gauss-Seidel and Jacobi are very
robust methods, [vocalized-noise] but only
for the matrices, which is domin[ant] diagonally
dominant or irreducibly diagonally dominant
[vocalized-noise] [noise].
Now, you can see that the [noise] convergence
depends on G [vocalized-noise] rho of G has
to be less than 1, spectral radius we [vocalized-noise]
made things [noise] ah much simpler and much
easier to quantify at this stage [vocalized-noise].
Instead of ah looking into the matrix norm
of G, [noise] we can just consider the [noise]
spectral radius of G or the largest eigenvalue
of G [noise]. If that is less than 1, then
the equation system will converge, and that
should be done [noise] for any ah [vocalized-noise]
diagonally dominant or irreducibly diagonally
dominant matrix [vocalized-noise] [noise].
Now, the question comes is that, how first
will they converge, what will be the number
of steps, when we ah [vocalized-noise] Rana
Gauss-Seidel Jacobi program, how many iterations
will it need. If it needs a very large number
of iterations, there is a [noise] ah much
not much need of running the iterative methods,
because [vocalized-noise] [probabe/probably]
probably the [vocalized-noise] [noise] direct
solvers can give us faster solution, but if
it less takes less number of iterations, we
can do it [vocalized-noise].
So, how to see what is the rate of convergence
or how fast do they converge, how many steps
needed for convergence of an iterative method
[vocalized-noise]. It is also important to
see, which will do in the later section al[so]
[noise] also. That if we can improve the rate
of convergence, if we can do certain things,
so that the [noise] convergence rate improves
[vocalized-noise] or if we can if we [vocalized-noise]
play with the splitting of the matrix, [noise]
so that we can get [noise] faster convergence
[noise] [vocalized-noise]. So, we ah [vocalized-noise]
[noise] before going into this, we will [noise]
ah try to define [noise] two more ah [vocalized-noise]
parameters regarding convergence [noise] [vocalized-noise].
One is the convergence factor [noise]. Let
the error at k-th step be d k, which is x
k minus x star; x star [noise] is x [noise]
exact solution. The difference between the
ah [vocalized-noise] Gauss Vector and the
Solution Vector is the error, we define it
as d k [noise] [vocalized-noise]. d k is [noise]
G to the power k into d 0 ah initial [vocalized-noise]
error into G to the power k, you have seen
that earlier [noise] [vocalized-noise].
The convergence factor which is defined [noise]
by rho is given as [noise] rho is equal to
limit k goes to infinity [noise]. So, this
rho is not spectral radius, rho of a matrix
is the spectral radius, simple rho is the
convergence factor. There are two rows [noise]
must not be convergence [vocalized-noise]
[noise]. Row limit k goes to 0 d to [vocalized-noise]
d k ah [noise] d [vocalized-noise] ah by d
0 1 [noise] [vocalized-noise] 1 by k ah to
the power 1 by k [noise]. For faster convergence,
the convergence factor must be solved must
be small [vocalized-noise] [noise]. So, how
should we determine convergence factor, [noise]
[vocalized-noise] we after k-th iteration
[vocalized-noise] after a large k is a large
number iteration, we see what is the [noise]
ah [vocalized-noise] solution, ah what is
the error, and the ratio between these two
errors [noise] to the power 1 by k is the
convergence factor [noise].
If the factor is small that means, its the
[vocalized-noise] the [vocalized-noise] [noise]
the error is reducing in a first way, so convergence,
so we need less number of iteration steps
or convergence factor [noise]. However, if
we look into the previous slide, [noise] this
this definition depends on the term d 0 right.
We have to start with a d 0, and see what
is happening [vocalized-noise] [noise].
[vocalized-noise] So, a better definition
[noise] is given [noise] which in which, [noise]
[vocalized-noise] we do not need to a ah have
[vocalized-noise] we do not have to need ah
we we do not need ah to depend on x 0, we
call the general convergence factor. And it
is [dif/defined] defined as independent of
initial guess [noise] phi general convergence
factor is limit k goes to 0, maximum of x
0, which belongs to r to the power n d k by
d 0 to the power 1 by k [noise] [vocalized-noise].
And this is maximum [vocalized-noise] d k
is G to the power k d 0 maximum of matrix
norm of G to the power k [vocalized-noise]
sorry vector norm, but these are [vocalized-noise]
ratio of vector norm G to the power k d 0
by d 0 1 by k [noise] [vocalized-noise].
Now, by definition, [noise] this is the definition
of [noise] ah [noise] say vector norm of any
vector B [noise] is equal to maximum of [noise]
x 0 belongs to R [noise] [vocalized-noise]
matrix norm of any matrix B is maximum of
x 0 belongs to R [vocalized-noise] B x 0 by
[noise] x 0 [noise] [vocalized-noise]. So,
this is [noise] the definition of matrix norm
of G to the power k [noise] [vocalized-noise].
So, we get limit k goes to infinity, matrix
norm of G to the power k to the power [noise]
1 by k, [noise] [vocalized-noise] which is
nothing but spectral radius.
So, if we try to see maximum ah [vocalized-noise]
of the convergence factor maximum value of
the convergence factor [noise] for [vocalized-noise]
for any guess x any gauss x 0, we will see
that is the general convergence factor, which
is spectral radius [noise] [vocalized-noise].
So, spectral radius [noise] now [vocalized-noise]
the smaller the convergence factor, faster
is the convergence [vocalized-noise]. Therefore,
smaller the spectral radius, faster should
be the convergence. We need to have a matrix
G, whose largest eigenvalue is a small number,
[noise] therefore we should get first convergence
[noise] [vocalized-noise].
And there is another term called convergence
rate [noise]. For faster convergence, convergence
rate must be high, so it is kind of [vocalized-noise]
has an inverse relation with this vector ah
convergence factor [vocalized-noise] [noise].
If the convergence factor is small, then ah
[vocalized-noise] convergence is faster. If
the convergence rate it [vocalized-noise]
rate is high, convergence factor [vocalized-noise]
ah is faster. And convergence rate is defined
as minus log of rho ah tau is equal to minus
log of the [vocalized-noise] logarithmic of
convergence [vocalized-noise] factor, which
is minus logarithm of [noise] equivalent to
general convergence factor and minus log of
row G [noise] [vocalized-noise].
Smaller the spectral radius of G, [noise]
higher is the convergence rate, and therefore
faster the convergence had the convergence
rate. So, less number of iterations will be
needed. So, we need to see that and that will
give an idea, that how we can increase [noise]
increase the [noise] improve the [noise] ah
convergence criteria [vocalized-noise] through
splitting, which which one will give us faster
convergence Jacobi Gauss-Seidel or some gradient
of it, [vocalized-noise] for which, we get
smaller spectral radius of G.
If that spectral radius of G is equal to 1,
there is no convergence [vocalized-noise]
it is skilled right, I minus G is singular,
it cannot converge [noise] [vocalized-noise].
If spectral radius of G is less than 1, that
will this will converge [noise]. And as small
as it will be in spectral radi[us] this this
is the [vocalized-noise] modulus of the eigenvalues,
[vocalized-noise] so this cannot be [vocalized-noise]
negative. So, as small as it be as [vocalized-noise]
it is a [vocalized-noise] in between 0 to
1, as small as it be faster [noise] will be
the convergence [vocalized-noise] [noise].
So, by in next class, we will look into some
matrices and look into the spectral radius
of this ah of the G associated G matrix. And
see for Jacobian Gauss-Seidel, how is the
convergence rate. And we will also see [vocalized-noise]
if we can do something with the matrix or
we can design a ah better matrix solver, which
we will have [noise] less [vocalized-noise]
condi[tion] smaller [vocalized-noise] ah spectral
radius, [noise] so that the matrix ah [vocalized-noise]
the solver [vocalized-noise] convergence in
a faster way than Jacobi and Gauss-Seidel.
We will look into this in the next class [noise].
Thank you. [noise] [vocalized-noise]
