Hello Internet Oscar Veliz again this
time with a video on Steffensen's method
for finding roots and Aitken's
accelerator let's start with Aitken's
Delta squared method it was first
published in 1927 and is known as a
convergence accelerator the main idea is
that given a sequence of numbers with
linear convergence you can use three
consecutive numbers to predict the next
one let's look at an example using fixed
point iteration to create our sequence
of numbers if you're not familiar fixed
point iteration I suggest watching my
video on this topic fixed point iteration
iteration gives us this sequence of
numbers and if you subtract the root
from each X we can compute the error
eventually our sequence of errors will
approach zero now let's evaluate the
error ratio at each step eventually
these error ratios become approximately
the same number this is the basis of
Aikens Delta squared method given a
linearly convergent sequence in this
case we're approaching a root when n is
large our error ratios will be about the
same for simplicity we'll call these
consecutive numbers a b and c and our
formula will look like this and given
this we can actually solve for r
here's how we'll do that starting with
this formula you can remove the
denominators then expand them since they
have r squared in both of them we can
subtract those out then we factor out an
r from the right side and move
everything with r term to the left
everything else to the right afterwards
we factor out r again and rearrange it
so that looks like this now let's just
simply divided by that term and you have
an approximation for r but we're not
quite done yet we can rearrange this
form to make future math easier first we
complete the square in the numerator and
then move the terms around afterwards
we'll do a double negation on part of this
so that we can turn this into a
subtraction of fractions for the term on
the left we can factor out an a and for
the term on the right we can simplify
that to be b minus a squared this will
give us this form for r we know that r
is approximately equal to this
expression what that expression is
actually equal to we'll call p-hat if
you substitute our x's back into this
expression there's something special
about the numerator and the denominator
the numerator is the forward difference
and the denominator is the second order
central difference this is where we get
Aitken's Delta squared method from
let's look at an example using three of
the numbers we computed from fixed point
iteration we can compute Delta X and
Delta squared X and plug them into Aitken's
Delta squared method when we do this we result in the value of 1.625
now if we had kept iterating using fixed
point iteration we would eventually have
reached the value of 1.625 this is the
power of Aitken's Delta squared method
it's known as an accelerator it can
predict later values using three of the
current ones Steffensen's method was
first published in 1933 in a paper
called "remarks on iteration" here he's
referring to fixed point iteration
he writes "our principal object being to
show how the process of iteration which
is often futile in its primitive form may
be improved by a suitable combination of
three consecutive values" does that sound
familiar
"we shall therefore use as a working
formula the approximation this function
where X sub v may be any element of the
sequence and that looks a lot like
Aitken's delta squared method Steffensen's
method says that given a function and a
fixed point iteration version of that
function with an error threshold and a
starting value we said a equal to the
starting value and then b and c are two
consecutive values of fixed point
iteration then we compute p-hat
given our Aitken's formula and repeat
the process with a equal to p-hat until
F of p-hat is less than some epsilon in
absolute value let's continue to use our
example from earlier we started with the
value of 2 did two iterations of fixed
point iteration and computed p-hat as
1.625 which was the value of x5
now with Steffensen's method will restart
using x5 so then we did two more
iterations a fixed point iteration and
compute a new p-hat this time is the
value of x13 then we restart again
using x13 doing two more iterations of
fixed point iteration and plugging it into
Aitken's Delta squared method which
gives us the value of x27 and here
we'll meet our threshold and stop
there's another version of Steffensen's
method that I'll simply refer to as
version 2.0 that looks like this you'll
see this very commonly in the literature
this example is from Kumar et al. these
two versions are actually the same thing
they're both derived from Aitken's Delta
squared method you'll see it either as
the version on the left or the version
on the right and we'll actually show that
these are the same thing by showing that
the numerators for both these are the
same
and that the denominators for both of these
are the same first the numerators you
know that from fixed-point narration x_n+1 is equal to g(x_n) so
if you do substitution we're able to get
this form for the equation on the left
we also know that our sequence of x's is
eventually going to reach the root so we
can say that when n is large g(x_n) minus x_n is going to be equal to 0
since we're at the root and the function
at that root is also going to equal to 0
therefore g(x_n)  minus x_n is
equal to f(x_n) and if we substitute
back this shows their numerators are
equal to each other visually they look
like this where when you have a very
large value of n you're converging on
the root they have the same root now we
need to show that the denominators are
equal to each other first we know this
from earlier and should be fairly
intuitive to show that x_n+2 is
equal to g(x_n+1) let's start
with the equation on the right if we
substitute in what we know for f(x) we
get this form which we can simplify a
little now if we evaluate the composite
function of f(g(x_n)) we get this
form which we can simplify again now if we
do our substitution and rearrange this
is our equation on the left so we've shown
that they're equal to each other let's
do a bit more cleanup in this form we
aren't doing fixed point iteration so we
don't necessarily need a p-hat variable we
can simply say that now we're just
computing x_n+1 if we do a bit
more math we can say that the term on
the bottom is actually a very good
approximation of the derivative which we
can also simplified to this now our function
actually looks like Newton's method now
about that order Newton's method we know
has a quadratic convergence but so does
Steffensen's method and we can show this
using our order formula if you look at
our examples of p-hats that we computed
and take a look at the errors meaning
the p-hat minus the root lets
substitute in our first three errors
and the result is an order of about 1.92
if we substitute in our last 3 years
let me get in order for about 2.24
showing that this is converging
relatively quickly some notes on
Steffensen's method we achieve quadratic
convergence without a derivative
although we can still diverge or divide
by zero the method also works best when our
error ratios are approximately equal to
each other so if we're too far away
might have a tough time depending on the
version of Stefan's is method you use
you might get a different numeric
sequence although it's doing the same
thing not that every iteration of Steffensen's requires two new function calls
either two new g(x) calls or two new f(x)
calls compared with Secant method which
only means one new function call even
though it has a lower order you might
actually converge faster depending on
how complicated your function is the code
that I used to be hosted on github as
always thank you for watching the links
to all the papers I discussed will be in
the description box below the video I had a
lot of difficulty accessing Steffensen's
and Aitken's original papers so I needed
help from my librarian if you have any
suggestions for future videos or
questions or comments on this one please
be sure to leave them in the comments
