Hello friends; welcome to the lecture 4.4
in the course on introduction to time-frequency
analysis and wavelet transforms. In the previous
lecture, we learnt the concept of analytic
signals instantaneous frequencies and also
studied the limitations of instantaneous frequencies.
The concept of instantaneous frequency is
good. In general, the physical concept is
good, but the mathematical definition itself
has the limitation. So, you should not get
confused between a concept and the definition.
The definition that we had for the instantaneous
frequency as a change Ð the derivative of
the phase; has its limitations in the sense
that, if a signal has more than one frequency
at a given time, then the definition gives
me observed results. And, it gives me very
good results when the signal has a single
frequency at a given time instance. The frequencies
are allowed to vary from time to time. But,
at any given time instant, it should have
only single frequency and we call such signals
as mono component signals. We are going to
at a later stage learn the technique of empirical
mode decomposition; which is also known as
a Hilbert-Huang transform.
But, for now, to keep the flow of things intact,
we will move on to the duration-bandwidth
principle, which is a very fundamental result
as we have seen in the previous lectures as
well in the form of illustrations. Today,
we are going to formally prove the duration-bandwidth
principle. And, in order to derive the duration
bandwidth principle, we have to introduce
a concept called time-frequency covariance.
Now, this is not surprising; again, recalling
the analogy of the energy densities with the
probability densities. In fact, constantly,
if you keep reminding yourself of this close
analogy with the probability theory or the
random variables theory, then a lot of these
definitions become obvious and easy to understand.
So, when we talk of time-frequency covariance,
we are essentially talking about the covariance
of frequency and time in a sense that, we
are asking how frequency of a signal changes
with time. So, we are not really asking how
this concept of frequency or this general
frequency, dimension of frequency changes
with time and so on. What we are asking is
how the frequency of a signal changes with
time; whether there is a dependence of the
frequency of the signal on time. Even before
we dwell into the definitions of time-frequency
covariance and so on, intuitively itself,
since time-frequency covariance is going to
measure how dependent the frequency is on
time for a constant frequency signal, you
should expect the covariance to be 0; which
means frequency is independent of time. And,
we will see that when we discuss a couple
of examples here at theÉ At the bottom, I
am given you a couple of examples here. So,
the definition of time-frequency covariance
is fairly straightforward; it is along the
same lines as the covariance in probability
theory or covariance for random variables.
Once again, I should tell you that, you should
not treat time and frequency as random variables
and so on. So, we denote the covariance with
sigma subscript t omega denoting that, we
are computing the time-frequency covariance
as in the case of random variables; the covariance
is defined as the average of the product less
the product of averages. Let me bring up the
analogy of random variables for you here;
that is, the definition of covariance for
random variables.
When I have two random variables Ð continuous
valued random variables: X and Y; and, when
I want to measure the dependence of Y on X
or X on Y, whichever way I am looking at it;
then, a standard measure of dependence is
the covariance. We have learnt this in the
context of deterministic signals as well.
When we spoke of cross-covariance function,
we did remarked there that, the cross covariance
function is a measure of linear dependence;
and that, it is based on the concept of covariance,
which measures linear dependence between two
variables. Of course, there we talked about
deterministic signals; now, we are looking
at random variables. So, how is the covariance
between two random variables defined as? It
is defined as expectation of X minus mu X
times Y minus mu Y; where, mu X and mu Y are
the expected values of X and Y. And, it is
fairly easy to show that, this expression
can be rewritten as the expectation of the
product less the product of expectation, so
that whenever the X and Y are 0 mean variables,
the expectation of the product itself is the
covariance. So, this is the definition that
you see on the slide as well here.
Instead of omega I, I have phi dot of t in
the equation. Remember we are supposed to
evaluate the dependence of frequency on time
or the influence of time on frequency. Therefore,
I have to use instantaneous frequency. Very
often when I write omega of t, it should be
understood that, I am looking at instantaneous
frequency. So, the I need not be there all
the time. So, this analogy really helps, but
again you should tell yourself that, here
I am looking at random variables; these are
different. Here time and frequency are not
random variables or you should not draw any
such observed conclusions. So, the normalized
version is often used; but, the only difference
between the normalized and the un-normalized
version is that, it is independent of the
choice of units that you have for time and
omega. That is the advantage of working with
normalized version.
Now this is also the practice in random variables,
where I use correlation instead of covariance.
There also correlation is defined in an exactly
similar manner. However, in the theory of
random variables, the correlation can be shown
to be bounded in magnitude above by unity.
That is not necessarily the case here. So,
the advantage of working with correlation
in random variables is that, one Ð it is
independent of the choice of units for X and
Y Ð random variables; and, two Ð that is
the bounded measure. Whereas, here the advantage
is that, it is independent of the choice of
units that you make for t and omega; but,
otherwise, it need not be bounded. So, that
is it. So, that is the basic definition of
time-frequency covariance. Again, the purpose
of this measure is to tell me how dependent
the frequency of the signal is on time.
Let us look at these two examples here. I
have a complex exponential. Once again, this
expression is familiar to us. And, it is fairly
straightforward to see that, the frequency
of the signal is constant; and that, it is
equal to omega naught. Therefore, the average
also is omega naught; average frequency is
also omega naught. And, the average of the
product can be worked out here by evaluating
the integral, which is fairly straightforward
to evaluate. I get here the average to beÉ
that is, the average of the product to be
omega naught times average time; and, plugging
in this result into the definition of covariance
gives me 0, because here I have average of
the product being omega naught times average
of time t; and, average of omega itself is
omega naught. So, both these terms are identical.
And therefore, covariance evaluates to 0,
which is what we expect; that is, there is
no dependence of frequency on time for this
signal.
Now, when will look at a chirp, you should
recognize that this chirp has a linear modulation.
This is the example that we have looked at
in the previous module as well. The phase
has a quadratic dependence on time. Therefore,
the instantaneous frequency has a linear dependence
on time. Hence, we say that, it is a linearly
frequency modulated signal. And then, of course,
you have an amplitude modulation as well.
In this case, you will have to work out the
integral; you will have to do a bit of algebra
and evaluation of the integrals that finally
leads to these values of expressions for covariance
and correlations. So, covariance is given
by beta by 2 alpha; and, the correlation itself
is given by beta by square root of alpha square
plus beta square. So, if you look at the correlation,
when beta goes to 0É What does it mean when
beta goes to 0? That means I am taking of
this quadratic dependence on time from the
phase. The moment I take off this quadratic
dependence, then it goes back to the case
of no-frequency modulation. Remember Ð when
phase is simply linear function of time, then
the frequency is going to be constant. So,
when beta goes to 0, the correlation goes
to 0; once again indicating clearly that,
there is no dependence of frequency on time.
So, this is the beauty of this measure.
So, let us look at the central topic of this
lecture, which is a duration-bandwidth principle.
The duration bandwidth principle is something
that we have seen earlier as well; and, it
places fundamental limitations on what we
can do in time-frequency analysis, because
it says that, if I try to localize the energy
of a signal in time, then the localization
of energy in frequency is affected. Fundamentally,
what it says is I cannot really localize the
energy of a signal in time and in frequency
with arbitrary fineness. So, that is the basic
problem that I have in the time-frequency
analysis. Now, here we have a formal statement
of the duration-bandwidth principle, which
states that, the product of the duration Ð
sigma t and the bandwidth Ð sigma omega is
bounded below by half clearly telling me that,
whenever the duration of the signal becomes
small, then its bandwidth increases. And,
we have seen this through few illustrations
in the previous lectures as well. But, it
is very important to understand the duration-bandwidth
principle, the assumptions that go into it,
the setting in which it is derived, so that
we do not have any misconceptions or misinterpretations.
Quite often, this duration-bandwidth principle
is called the uncertainly principle for signals,
because the expression here in equation 4
has a very strong similarity with the one
that you see in quantum mechanics in the form
of HeisenbergÕs uncertainty principle. There
the term uncertainty is appropriate, because
the HeisenbergÕs uncertainty principle is
derived in a probabilistic framework. There
is nothing probabilistic here; pretty much
like what we said earlier for time-frequency
covariance. The expressions for the covariance
look very similar, but that does not make
time and frequency random variables; or, that
does not mean that, we are working in a probabilistic
framework. Nevertheless, this term has stuck
on and people have been using widely this
phrase called uncertainty principle for signals.
So, you should remember that, there is nothing
uncertain about things here; it is all about
the product of duration and bandwidth; and
that, there are four quantities involved in
deriving this relation. Two Ð being the densities
in time and frequency; and, the other two
Ð being the duration and bandwidth. And,
very importantly, this is not some frequency;
it is a Fourier frequency; which means this
bandwidth itself is defined with respect to
a Fourier transform. If you are working with
some other transform, this duration-bandwidth
principle or any of these things may not even
arise at all. So, more rigorous statement
of the duration-bandwidth principle is given
in equation 5 that involves a covariance between
time and frequency. I have givenÉ We have
already discussed this covariance between
time and frequency in the previous slides.
So, let us see how to derive this duration-bandwidth
principle. The derivation is fairly straightforward.
The fundamental result that we use is that
of SchwarzÕs inequality. And, we apply it
to the product of the square duration and
square bandwidth. We know from definition,
sigma square t, sigma square omega are given
by this product of integrals assuming that,
the average-time and average-frequency is
0. This is not a major assumption that is
going to spoil the result here. So, without
loss of generality, you can assume this. It
is just that, the math becomes convenient
by assuming this; otherwise, you can also
derive this by assuming nonzero mean time
and mean frequencies.
So, let us get back here. So, I have the product
of this square duration and square bandwidth
given by these integrals. And, I can rewrite
this integral here Ð omega square times modulus
x of omega square d omega as integral x dot
of t square dt. And, that is using the property
of Fourier transforms. You can refer to the
table of properties of Fourier transforms
in any standard book. And, it will tell you
that, multiplication of X of omega with j
omega will correspond to taking the derivative
in time. Using that property, we have rewritten
this integral in terms of time-domain variables.
And now, we invoke the SchwarzÕs inequality,
which says that, the product of integrals
Ð which integrals here? Integral mod f of
x square dx and integral mod g of x square
dx; that is greater than or equal to the squared
modulus of integral f star of x g of x dx.
This is a very standard inequality that is
prevalent everywhere in functional analysis
and so on.
Now, we apply this to the product of the square
duration and bandwidth, and we get this inequality.
So, on the left-hand side here, if I replace
f of x with t times x of t and x itself with
t; please note that, x here is a dummy variable;
do not get confused with the x here and x
of t here. And, I apologize for any confusion
that you may have. That is it. So, you apply
the left-hand side to the problem of interest;
where, now, we have sigma square t times sigma
square omega. And, on the right-hand side,
now, I have t times x star of t. Remember
it says f star of x; and, our f of x here
is t times x of t. So, time is a real valued
quantity. So, there is no conjugate; conjugate
and the number itself are identical; variable
itself are identical. So, I have t times x
star of t in place of f star of x. And, in
place of g of x, I have x dot of t. So, that
is what I have here.
Now, what remains is the evaluation of this
integral. The integrand itself now can be
written in terms of amplitude and phase; where,
we are invoking the complex representation
for x of t. Remember Ð x of t is written
as a of t times e to the j phi of t. So, I
substitute that representation here for x
of t and I get t A dot times A plus j times
t times phi dot times A square, which I have
intentionally rewritten in this form. All
this derivation is borrowed from CohenÕs
book. So, if you have any confusion, you can
refer to the text book by Cohen. So, now,
if you look at this first term, it is a perfect
integrand. Therefore, the integral of this
term is going to turn out to 0 when you integrate
from minus infinity to infinity. And, the
second one here evaluates to minus half assuming
that, we have normalized the signal to have
unit energy. So, the second term is minus
half with that normalization assumption. And,
the third term is nothing but j times the
covariance. So, you can see when I integrate
j times t phi dot of t by definition integral
t times phi dot of t assuming that, average
time and average frequency as 0, is nothing
but the covariance itself.
Therefore, I have this result here. Sigma
square t time sigma square omega is greater
than or equal to minus half plus j sigma t
omega mod square, that is, a magnitude square
of this. The inner term here is a complex
number. And, the moment I take the magnitude
square, I get 1 over 4 plus covariance square
t omega leading to this result that we have
stated earlier. So, this is a more rigorous
statement. This is the complete statement
of the duration bandwidth principle. Normally,
this covariance square of t comma omega is
omitted from the general statement of the
result. Is that right? It is right whenever
you are looking at signals that have constant
frequencies, because when signals have constant
frequencies like a sine wave and so on; then,
the covariance is 0. We have seen that in
the example.
However, the issue is not whether covariance
is 0 or not, the main point is that, the product
of duration and bandwidth is bounded below
by a finite number. Consequently, whenever
sigma t increases, sigma omega false down;
and, whenever sigma t decreases, sigma omega
increases. So, that is the most important
point that you have to remember rather than
really worrying about whether this 1 overÉ
It is the right-hand side is half or square
root of 1 over 4 plus covariance square t
comma omega. Now, when does the equality occur?
The equality occurs for a GaussianÕs signal;
you can see the worked out example in CohenÕs
text and also for a chirp; that is, the weaker
version equality occurs for a Gaussian signal;
the product of sigma t sigma omega evaluates
to half for a Gaussian signal and for a chirp
when you are looking at the stronger version.
Both these examples are worked out in CohenÕs
text. And, I strongly recommend you look up
the workings of that example.
So, that brings us to the close of this module.
I just want to make a few remarks again with
respect to the duration-bandwidth principle.
What it says is a signal with narrow bandwidth
has a longer duration and vice versa. And,
or, that the effective bandwidth and duration
of a signal cannot be both arbitrarily small.
Now, this principle applies to any signal
and its modification. What we mean by modification
is in short-time Fourier transform for instance,
I window the signal. So, the duration-bandwidth
principle again applies to this windowed signal
as well. There you are looking at the duration
of the windowed signal and the bandwidth of
the windowed signal. So, you have to be careful
in asking Ð to which signal does this duration
bandwidth principle apply. And, we will derive
the lower bound for the local quantities in
time-frequency analysis. That is what we mean
by localize in joint time-frequency analysis;
I will be looking at conditional bandwidth
and conditional duration and I will have to
re-derive the bounds for the product of the
conditional duration and conditional bandwidth.
We will understand this better when we talk
of short-time Fourier transforms and CWT transform
and so on.
Now, there are two common misinterpretations
or wrong statements given for duration-bandwidth
principle. One Ð that time and frequency
cannot be made narrow. That is a very weird
statement; there is no sense in that. Or,
that energy densities in time and frequency
cannot be measured with arbitrary accuracy.
There is absolutely no statement here with
respect to measurements here. It has got nothing
to do with your ability to measure; it is
just got to do with the spread of the energy
densities in time and frequency. How you measure
is not dictated by the duration-bandwidth
principle, nor does it place any limitations
on it. There is also sometimes this statement
called delta t times delta omega is greater
than or equal to half; where, delta t and
delta omega are the resolutions in time and
frequency. That is also wrong. The sigma t
and sigma omega are not the resolutions in
time and frequency; delta t is the sampling
interval that is dictated by the sampling
rate; and, delta omega is in turn dictated
by the sampling rate as well, because we know
in DFT, the frequency resolution is limited
by the number of observations as well. So,
delta omega is dictated both by the sampling
rate and the number of observations. That
has got nothing to do with the duration and
bandwidth. So, if you see statements like
delta t and delta omega greater than or equal
to half; where, delta t and delta omega are
being referred to as resolutions in time and
frequency, that statement is incorrect; the
correct statement is sigma t times sigma omega
is greater than or equal to half.
So, with those remarks, we will close this
module. And, once again, I welcome you to
refer to CohenÕs book and also work out a
few examples in the time-frequency toolbox
frame work for you to understand how this
duration-bandwidth principle works.
We will meet again in the next lecture.
Thank you.
