in the last lecture we were looking at how
to analyse convergence of non-linear procedures
for solving nonlinear algebraic equations
iterative procedures and we said that in general
we could write any iterative method for solving
nonlinear algebraic equations as 1 equation
i want to solve for f of x=0 x belongs to
rn and f is a nx1 vector this is nx1 function
vector 
any iterative method to solve this problem
numerically can be written as xk+1=g of xk
so the old guess generates a new guess and
this process is continued till differences
between 2 successive solutions become negligible
or norm of f of x goes close to 0 if you look
carefully this is a nonlinear difference equation
the index here is iteration index k so the
guess is generated from the old guess g is
the transformation i showed you that all the
methods that we are looking at iterative methods
can be expressed in this form now just like
we had conditions for analysing linear difference
equations
earlier we had looked at equations of this
type=b xk and for this particular case we
had derived necessary and sufficient condition
for norm xk to go to 0 as k goes to infinity
in this case we had a very very powerful result
that is spectral radius of b is strictly less
than 1 this was the situation for the linear
difference equation we had got this kind of
a generic form by analysing iterative methods
for solving linear algebraic equations
we could derive a very very powerful result
here based on the eigen value of matrix b
we wanted all eigen values of matrix b to
be inside the unit circle now coming to nonlinear
equations it is not possible to prove so strong
result we can only give sufficient conditions
it is not possible to come up with necessary
and sufficient conditions for a general nonlinear
difference equation of that form we have to
come up with some kind of local condition
these local conditions i described through
contraction mapping theorem or contracting
mapping principle which forms the foundation
of analyzing iterative schemes and 1 special
that we saw was the operator g
g is something g maps a ball around x not
of radius r to where r was a special radius
it should be >= a certain number that we had
defined yesterday so if is a mapping which
maps a ball of radius r*itself and if g is
a contraction map 1 simple way of finding
out whether g is contraction map over u was
to see whether dou g/dou x was strictly <1
or <= theta which is <1 for all x 
if the partial derivative of g with respect
to x has any induced norm strictly <1 everywhere
then we know that map g is the contraction
if map g is contraction then in neighborhood
of x not of radius r we were assured of existence
of a solution we are assured that any sequence
starting from any point in this region would
converse to the solution so solution of this
problem is x*=g of x* x* is the solution and
if this condition is met everywhere in this
ball then it is sufficient condition to say
that any sequence generated by this difference
equation will converse to this solution
solution is x*=g(x*) just to draw the parallel
i am writing this just to draw parallel we
had a sufficient condition here that if norm
of b is strictly <1 then also this condition
holds that xk goes to 0 as k goes to infinity
so we said this is the weaker condition than
this necessary and sufficient condition but
this condition helped us to analyse to come
up with diagonal dominance and all kinds of
other theorems which were used to analyse
iterative schemes
likewise analogous to this when i come here
this contraction mapping principle tells us
very very important things 1 is that if g
is a contraction map if its local derivative
has if you g (x) to be dx then local derivative
of g with respect to x will be matrix b and
any induced norm of matrix b being strictly
less than 1 is the condition that we are looking
for there so they coincide this particular
equation only difference there was the solution
the point where we wanted to reach was 000
origin
in this case we want to reach a solution x*=g(x*)
it is possible to make everything in terms
of 000 if you redefine or shift the origin
to x* then you can make the 2 problems almost
equivalent but that is not important it is
just matter of shifting the origin what is
important is that there is an analogous sufficient
condition here for nonlinear difference equations
it does not help us here to look at the spectral
radius of this matrix
it does not help here the reasons which are
difficult to explain as a part of this course
but we have to use only norm and any induced
norm if any induced norm is strictly < 1 in
some region then you are guaranteed that there
exist a solution to this difference equation
in that region the solution is unique and
the third point which was very very important
start from any initial guess you will converse
to that solution
start from any initial guess in that region
you will converse to the solution x*=g(x*)
so these are very very important findings
of this particular theorem in general it is
more difficult to apply this theorem for a
complex real problem nevertheless it gives
us some insights for example you can try and
make the sufficient conditions meet by ensuring
that dou g/dou x has induced norm <1 you can
try to do this
if there is some problem in solving some nonlinear
equations we can these are sufficient conditions
remember that if this conditions are violated
even then the conversions can occur these
are not necessary conditions but this happens
convergence will occur just like in this case
when we were talking about linear algebraic
equations if norm of these <1 spectral radius
is <1 it is a sufficient condition
but if norm of b is >1 even then convergence
can occur because convergence depends upon
the spectral radius spectral radius can be
<1 similarly contraction mapping principle
gives us a sufficient condition for convergence
it is not a necessary condition if you meet
the sufficient condition you are guaranteed
to converge so this gives at least some handle
to understand how the convergence occurs from
that view point this is important
those of you who are solving large algebraic
equations as a part of your research m tech
or phd and hit into problems you should look
at the norm of the jacobian i mean at least
that much you should remember look at the
norm of the jacobian i try to see whether
you can make the norm of the jacobian <1 you
have good chances of convergence just to illustrate
this idea of contraction map i just give you
1 example here
i want to solve simultaneously these are 2
nonlinear algebraic equations which i want
to solve simultaneously if i write this – this=0
and this-this=0 then this is f(x)=0 there
are 2 functions f1zy and f2zy i want to find
out a solution for this particular problem
i am formulating an iteration scheme here
zk+1=1/16-1/4 yk square and 
i have just formed 1 iteration scheme this
is not the only way to form iteration scheme
i am showing you 1 possible way of forming
the iteration scheme this is a jacoby type
iteration scheme what would be the gauss-seidel
kind of iteration scheme if i were to use
zk+1 here it will become gauss-seidel type
iteration scheme this is the jacoby type iteration
scheme now what i am going to do here is
i have this scheme which is yz=g(yz) where
g i this right hand side function 
i am considering this unit ball let us say
x not my initial guess is 0 1 no no my initial
guess 
is x0=0 0 and i am considering this unit ball
of radius 1 in the neighborhood of 0 0 so
i am looking at
now what is 
this infinite norm i am taking some point
xi and some point xj x here is x consist of
y and z x is the vector consisting of 2 elements
y and z now i am looking at this 
what is the infinite norm infinite norm is
maximum of the absolute value of the elements
what i am doing is i am taking xi-gxz it has
2 elements i am just taking the maximum of
these 2 absolute values will be the norm i
am just using definition of infinite norm
nothing else
just this is definition of infinite norm 
so you can show that this is <= 
max of i am skipping in between steps you
should fill them up just go back and look
at why this step comes from this 
you can prove this in equalities that is this
particular difference infinite norm of this
difference is <xi-xj 1/2*xi-xj actually the
contraction constant is half i just wanted
to show that in this particular case you can
show that
i am using here the fact that the elements
are drawn from the unit ball so that is why
these types have been written and essentially
using this inequalities what you can show
is that gi-gj uing these inequalities you
can also do analysis using the derivative
of this and taking this infinite norm you
can also do analysis using derivative of this
right hand side jacobian matrix and infinite
norm of the jacobian matrix that analysis
is also possible
in this particular case we have found that
if we apply g on any xi and xj then this inequality
holds if this inequality holds what it means
is that this constant on the right hand side
is <1 so this is strictly <1 so this g map
is a contraction if g map is a contraction
i am guaranteed that there exist a solution
in this unit ball the solution is unique and
starting anywhere in this unit ball this is
in reference to the infinite norm
it will be a square it will look like a square
we have seen this how does the unit ball look
like in different norms starting from any
initial guess within this the iterations will
converge to the solution so this we are guaranteed
because we are able to prove this in equality
here for this particular x=g(x) what is important
here is that just looking at or just developing
this inequality this is infinite i am guaranteed
that a solution exist in the ball
i am guaranteed that i start from anywhere
and i will reach the solution and this iteration
scheme is going to work that is what i know
from this analysis just do not bother about
these in between steps assume that this sequence
is true because our aim is not to do this
algebra you can work on this algebra later
more important is that by doing this algebra
i can show that infinite norm of gi-gj/xi-xj
for any i j i can prove this
i take any 2 points in this ball apply g on
both the points the new points will have a
distance which is closer than the original
2 points that is the main thing if that happens
we are assured that the solution exist we
are assured that starting from x not we will
reach the solution moreover from any initial
guess in this region if we start we will still
reach the solution that is the important point
it is difficult to do this analysis for a
very large scale nonlinear system
nevertheless it is important to get this insight
that how does 1 look at analysis of convergence
of iterative schemes for solving nonlinear
algebraic equations because most of the times
you will be actually dealing with nonlinear
algebraic equations large scale in your computation
work because most of the chemical engineering
problems 999% of them are nonlinear problems
reactions of heavy transfer occurs and turbulence
and always things will make the life very
very complex
we have to work with a set of nonlinear algebraic
equations what is it that governs the convergence
we can get some clues if you can show that
the iteration scheme that you have formed
actually is a contraction map difficult to
show in general for large scale system but
this does give you insight which is very very
important that is what you should carry i
want to stop here i do not want to get into
too much details
in the notes i have given some more detailed
discussion on newton’s method so there are
special theorems for convergence of newton’s
method and more than the proof and the theorem
statement i have tried to give some qualitative
insights as to how to interpret those theorems
i have not included the proof the proof can
be found in any of the text books on nonlinear
systems like one of the very well known textbooks
so you can find proofs there but the interpretation
is quite important as to how do you make convergence
occur so typically if you have formed iteration
scheme in this case i worked with i did not
take a derivative but you could also try to
see for this particular system you can work
this out
you can try to see whether dou g/dou x infinite
norm if this is strictly <1 in the region
where you are trying to operate or trying
to solve the problem or dou g/dou x 1 norm
is strictly <1 if these conditions are met
then we are guaranteed that the solution exist
and we will reach the solution these are some
why infinite norm and why 1 norm because they
are easy to compute infinite norm and 1 norm
are easy to compute
other norms like 2 norms will require eigen
value computation other than that 1 norm and
infinite norm are easy to compute so you can
quickly make a judgment what is going wrong
when you are solving the problem this brings
us to an end of methods for solving nonlinear
algebraic equations we have looked at different
concepts we have looked at how to solve them
using different algorithms we just briefly
touched upon idea of condition number
also we very very briefly touched upon the
idea of convergence of iterative schemes we
have not gone deep into it but at least you
know about what is the tool or what is the
machinery that is used for actually looking
at this problem let us move on to solving
ordinary differential equations initial value
problems now what i want to do next is before
i proceed again we go back to our global diagram
so our global diagram was so we have this
original problem then we use approximation
theory to come up with transformed problem
so we have been calling it transformed computable
forms and then we said there are 4 tools 1
is ax=0 this tool set which we will be using
and the other tool set was f(x)=0 so solving
nonlinear algebraic equations solving linear
algebraic equations this is the second tool
set that we have
the third tool set that i am going to look
at is od-ivp because in many cases the transformed
problem is an od initial value problem i talk
about a method later on how do you transform
a boundary value problem into initial value
problem actually not just one initial value
problem a series of initial value problems
which are then solved iteratively the fourth
tool is stochastic methods but we are not
going to get into this
so right now we have done this how to solve
ax=b we looked at many many methods we looked
at many issues that are associated with this
we have looked at f(x)=0 and now we are moving
to od-ivp all these after all is going to
give us approximate solution 
this is going to give an approximate solution
to the original problem so moving on to solving
ordinary differential equations initial value
problem
general form that the types of equation that
i am going to look at is of this type dx/dt=f(x
t) where f is the function vector 
and x belongs to rn what i am given apart
from this differential equation model i am
also given 
initial condition at time=0 now before i move
on let me explain one notational difference
that we will have in this case if you are
dealing with vectors we will have to deal
with 3 different attached indices with the
vector
suppose x is my vector here i-th element of
the vector will be given by xi this notation
we have been using even earlier bracket k
will indicate k-th iteration now additional
complexity comes in we have time so time will
come here so there are 3 things attached to
the vector in some cases you will have i-th
component of the vector you will have time
t appearing here and you may have k-th iteration
in some cases we do not need i and k we just
might work with x t x t means 
vector x at time t so now a third dimension
comes into picture here when you write in
the notation sometimes there are schemes which
are iterative and you will need index sometimes
you need to prefer to i-th component so you
need xi and t is time now what kind of equations
i am worried about what kind of equations
i am going to look at
you might say that well what is written here
is only a first order vector matrix equation
dx/dt=f(x) i am writing only a first order
equation only first order derivatives and
in your engineering problems you often come
across models which are second order third
order fourth order and when you did your first
course in the differential equations you had
n-th order differential equations and then
you had methods of solving n-th order differential
equation
so why am i doing things only for the first
order differential equation though the difference
here is the vector differential equation earlier
we were looking at scalar differential equation
what i am going to show that any n-th order
differential equation can be converted into
n first order differential equations so this
form which i have written here is very very
generic so let us begin by looking at this
conversion let us say you have this
let us say i have this differential equation
in the scalar variable y so y is a scalar
y is some mass fraction or some temperature
or whatever is the case you have some differential
equation let us say this is n-th order differential
equation in general nonlinear differential
equation we do not know i am just writing
a generic form could be anything this is in
one variable an independent variable is time
what i am going to do now i am going to define
new state variables so my state variable and
what i am given together here to solve this
problem say initial value problem so what
do i need to solve this problem i need a differential
equation and i need the initial conditions
initial conditions are given for y(0) dy/dt
at 0 so we are given initial condition we
are given initial y0 initial derivatives up
to order n-1 these are required to solve this
differential equation
with this differential equation together with
this initial condition will be initial value
problem solving ordinary differential equation
initial value problem this is what i get now
what i am going to do now is to start defining
a new set of variables
my new variable x1t-yt x2t=dy/dt x3t=d2y/dt
square up to xnt=dn-1/dt n-1-th derivative
i am defining new variable x1 to xn now you
can see that these variables are related to
first order differential equations i can very
easily say that dx1/dt=x2 dx2/dt=x3 so i have
such n-1 equations 
this is my equation number 1 equation number
2 and this is my equation number n-1 i have
n-1 such relationships between the variables
all of them are first order differential equations
the last 1 is now just the equation that we
have so the last equation n-th equation 
this is dxn/dt this is nothing but d/dt of
d n-1y this is my definition this is = fx1
x2… xnt i have an n-th order differential
equation which got converted into n first
order differential equations this is my first
equation second equation n-1-th equation and
the last equation came from the original n-th
order differential equation
x1 x2 x3 …xn are the new state variables
that we have defined so what i have actually
done is a scalar n-th order differential equation
i have converted into n first order differential
equations in new variables so if i have n-th
order equation i can convert it into n first
order equations if i have 2 simultaneous equations
1 n-th order in 1 variable other m-th order
in other variable first 1 will give me n first
order equations
second 1 will give me m first order equations
you can stake them together into a bigger
vector you will still get this form so this
is the very very generic form i am not doing
any compromise any n-th order equation or
any set of n-th order equations n-th m-th
order equations can be combined into finally
this form this is the very very generic form
so do not worry about why are we looking at
only first order vector differential equation
so all the advanced books on nonlinear differential
equations will worry about this generic form
because anything can be converted to the generic
form that is the first thing to understand
so all the methods that we will develop are
for this if you have n-th order equations
you know how to convert them into n-th first
order equations and write it like this so
what will be f(x) in this particular case
what will be the f vector
let us go back and write that 
in this particular case my f vector after
a transformation actually
my equations are d/dt of x1 x2 x3 …xn=x2
x3 …xn and f(x1 x2 …xnt) this is my f(x)
this is the transform problem this is my f(x)
and i am given the initial condition so i
am given initial condition x not which is
whatever this is y0 dy0/dt all these are given
to me this is my x not this is given to me
this is my f(x) the original equation will
appear as 1 scalar nonlinear function in a
function vector this my function vector
this is a transform problem i do not have
to worry about n-th order equations i am not
going to do separate methods in the first
course of differential equation you have second
order differential equations 1 chapter on
second order differential equations then you
will look at n-th order equations we are not
going to separate we are just going to look
at n differential equations which are coupled
if you are trained to solve dynamic simulation
of a chemical plant there will be 1000s of
differential equations which are solved simultaneously
together in fact they might be differential
and algebraic equations not differential equations
so we are worried about right now to begin
with solving large number of differential
equations simultaneously together in 1 shot
that is my aim this form is very generic applicable
to any set
other way of getting these kind of equations
we have already seen where do you get these
kind of equations in problem discretization
where did you find them finite difference
method orthogonal collocations of partial
differential equations that involve time and
space we discretize in space we got differential
equation in time we got n differential equations
they were first order if those are all second
order you can convert them into 2 first order
equations
all that is possible that is not difficult
so converting n-th order equation into first
order equations is not a problem we are going
to look at the generic form this could be
arising from any of the sources this could
be arising from the 1 which we have done right
now it could be arising from discretization
of a pde it might be arising from some other
context we already have studied about in what
context this kind of problems will come
we will look at only how to solve this abstract
form of vector differential equation the other
thing which you might worry about is that
where does this time t come into picture most
of the times the differential equations that
you get an exercise that i have given you
to solve differential equations for 1 particular
system and i had given you a program which
solves differential equations for a cstr i
suppose you remember to submit assignment
soon
that equation is of this form dx/dt=f(x u)
there are some free variables x are dependent
variables and there are some free variables
like feed flow coolant flow coolant temperature
inlet concentration all these are this u variables
so in that particular problem cstr problem
x corresponds to concentration of a and temperature
and u corresponds to inlet flow rate cooling
water flow rate inlet concentration cooling
water temperature at inlet and so on
so these are the free variables but if you
go back and look at the problem statement
these manipulated variables or input variables
have been defined as a function of time this
is sinusoidal this is whatever we have defined
these as some functions of time once these
are given as functions of time we can substitute
them here as some function of time and then
once 
these are specified functions of time then
only we can solve the initial value problem
for those specified functions of time this
problem has been transformed to dx/dt=f(x
t) because u will be function of only time
some specified function of time a ramp function
step function sinusoidal function or whatever
whatever you want to study the dynamics of
the particular system you are specified this
free inputs and then this becomes a problem
which again is the generic form
so this parameter or these input variables
we assume that we already know them and then
we want to solve the problem for the known
inputs how does the dynamics evolves in time
that is what we want to solve that is why
we are looking at in general dx/dt=f(x t)
how this is specified as a function of time
let us not worry about that right now it could
be an operator who is giving these values
it could be a controller which is finding
out these values
it could be some environmental conditions
which define the cooling water inlet temperature
we do not bother about that right now we want
to solve the problem when this is specified
how do you actually find out x as a function
of time i want to find out given these input
trajectories in time i want to find out x
trajectory that is concentration trajectory
starting from time 0 to whatever final time
you want and temperature trajectory as solution
of this problem is going to be not 1 vector
when you are solving nonlinear algebraic equations
you got 1 vector as a solution the fixed point
now the solution is going to be a trajectory
in time trajectory in time over the finite
if we are solving over a finite time or whatever
t goes to infinity if you want to look at
now linear differential equations of this
type you probably have already looked at in
some other course wherever we need them we
will visit them
those of you who have not done the other course
on analytical methods in chemical engineering
i will briefly mention those results which
we need here we are going to look at the problem
when this f(x) on the right hand side is nonlinear
not when it is linear that is very very crucial
we will use the results for linear later on
to get some insights into the convergence
properties under what conditions the methods
that you have proposed will converge
that is why we will use some linear system
results but in general what we are going to
look at is methods for solving nonlinear ordinary
differential equations given initial conditions
how do you get trajectories in time or it
could be trajectories in space we have seen
that for example method of lines for converting
laplace equation you discretize only in 1
spatial direction the other 1 is stated as
a differential equation so you get instead
of differential equations in time or space
you want to integrate the differential equations
so t here in general need not be time alone
t here is treated as independent variable
in some context it could be space so maybe
i should write a generic form that neta so
neta is some independent variable it could
be time on space depending upon the context
and initial condition at neta=0 is given and
you want to integrate this set of differential
equations
the way that we are going to proceed will
briefly peak into the issue of existence of
solution very very briefly and then move on
to the different methods of doing numerical
integration again what is going to help us
taylor series approximation and polynomial
approximations we are going to meet our old
friends taylor and weierstrass again and use
them repeatedly to solve these problems
what i want to stress here is that the same
ideas are used again and again to form the
solution methods there are few fundamental
ideas which if you understand those ideas
and if you know how to apply them you can
almost do everything from scratch same idea
is repeatedly used if you get this viewpoint
then i think you have learnt a lot next class
onwards we will begin with how to solve ordinary
differential equations and algorithms
and then finally we will move on to the convergence
properties under what conditions these converge
try to get some insights into relative behavior
of different methods and so on
