ALBERTO DEL PIA: In
this video, we're
going to see a few examples of
linear-programming problems.
Even if it's not really
the focus of this course
to see real-world examples
of optimization problems
and model them as
linear-programming problems,
I still want to show
you a few examples here.
We start with a
production problem
which will be really
straightforward to write down
as a linear-programming problem.
What we're going to
do in this example
is first we state
the problem in words,
and then we write it down as
a linear-programming problem.
So in our production problem,
we have m different goods
and m different materials.
We also know exactly
the available amount
of every material
that we have, and this
is encoded in the number bi
for i that goes from 1 to m.
Therefore, b1 will be
the available amount
of the first material, b2
of the second material,
and so on and so forth.
Next, we know the
connection between the goods
and the materials.
Namely for every good, we
know exactly the amount
of material required
to produce such a good.
In order to produce
the j-th good,
we need aij units
of the material i.
So also, all this aij
form the data that
is given to us in this problem.
The last amount of data
that we need is the revenue.
So we know that selling
a unit of the good j
will result in revenue of cj.
The question is what
is the optimal strategy
in producing this good?
Namely, how much of each good
should we produce in order
to maximize the total revenue?
Now every time we have an
optimization problem stated
in words, the first thing
that we should do in order
to translate it into a
linear-programming problem
is to write down the
decision variables.
So the decision
variables normally
are exactly the things you
should decide in your problem.
In our case, we need to
decide how much of each good
we should produce, and so we
introduce the decision variable
xj that represents exactly
the amount of the j-th good
that we produce.
And at this point, writing
down the formulation
is straightforward.
The first thing we should look
at is the objective function.
So we want to maximize
the total revenue.
Now for the good j, the revenue
is cj times xj, so the revenue
per unit times the
units produced.
And so, of course, we
sum over all the goods.
Then what are our constraints?
Well, first we know
that we can produce
only non-negative quantities,
so we have xj greater than
or equal to 0 for every j.
And the only
remaining constraint
regards the available
amount of the materials.
In fact, for every material, we
cannot use more than bi units
of such a material.
So how many units of material
i are used in our production?
Well, this is the sum of
how much of this material
is used to produce each good.
So we sum over all the
goods, and each good
requires exactly aij times
xj units of this material.
So in total, this is our
linear-programming problem.
We maximize our objective
function subject
to the constraints.
As you can see,
all the functions
that we've written
here are linear,
and so this is a
linear-programming problem.
In the next example, we're going
to see the multiperiod planning
of electric-power capacity.
This optimization problem will
be definitely more complicated
to state than the
previous one, but I
promise it's going to still
fit in just the one slide.
So our task here is to plan
the electricity capacity
for the next capital T
years, and this year's
will be denoted by year 1,
year 2, until year capital T.
We already know the
demand for electricity
that we are required
to meet at every year,
and this is encoded
in the number dt.
So dt represents
exactly the demand
in megawatts during year t.
And again, t ranges
from 1 to capital T.
So the demand in
year 1 would be d1,
in the year 2 is d2, and so on
and so forth until d capital T.
In order to meet such
a demand, we already
have some power plants, and
these are all oil power plants.
We also know how many
megawatts these oil
plants will produce
in each year,
and this is encoded in et.
So in year t, our oil plants
will produce et megawatts
for every t that goes
from 1 to capital T.
In general, this
capacity will not
be sufficient to
meet the demand.
So what can we do about it?
Well, we can build
new power plants,
and these can be either coal
plants or nuclear plants.
Building these plants has,
of course, different costs.
And also here we
know exactly how much
it costs to build coal
plants or nuclear plants.
So the cost is expressed
in cost per megawatt
that such a plant will produce.
In particular, the cost per
megawatt for coal plants
is ct and for
nuclear plants is nt.
Note that also here
we have a subscript t,
and this is because these costs
are not the same every year.
Now, the difference in
coal and nuclear plants
is not just in the
cost per megawatt
in constructing such
a plant, but it also
lies in how long
this megawatt will
be available-- so essentially
how long these plants will
last.
And we already know that
coal plants last for 20 years
while nuclear plants
last for only 15 years.
Therefore, even
though let's say n1
could be, for example,
smaller than c1,
then it's not necessarily
true that it's better for us
to build nuclear
plants only in year 1
because the nuclear
plants produced in year 1
will stop working in year 15
while the coal plants will
work until year 20.
To conclude the description
of this optimization problem,
there's only one more
constraint that we have.
In fact, for political reasons,
we know that no more than 20%
of the total capacity in each
year should ever be nuclear.
Of course, our task is to
find the least cost capacity
expansion plan which
must meet the demands.
Once again, the very
first thing to do
is to decide which are
the decision variables.
Also in this
problem, we're going
to use the most natural ones.
We have to decide every
year the amount of coal
and nuclear capacity to
bring on line in such a year,
and this will be denoted
by xt for the coal
and yt for nuclear.
At this point, we
can easily write down
the objective function.
So remember, we want to
minimize the total cost.
Now, what is the cost in year t?
Well, we need to pay ct times
xt to bring on line all the coal
capacity and nt yt to bring on
line all the nuclear capacity.
And, of course, we need
to sum over all t's.
At this point before
getting to the constraints,
I want to introduce some
more decision variables.
I'm going to talk more about
these decision variables
once we have fully written down
our linear-programming problem.
What you need to
know right now is
that it's not really
necessary to introduce
this auxiliary variables, but
it will make our life easier.
So my additional decision
variables are wt and zt,
and they represent the total
coal and nuclear capacity
available in the year t.
Our first constraints
then will be
the ones that link our
new additional variables
with the original variables.
So wt is the total coal
capacity available in year t.
And so how can it be written
in terms of the x variables?
Well, wt will be
equal to xt, which
is the coal capacity
which we just brought
on line in the same year.
But remember, the
coal plants last
for a total number of 20 years.
Therefore, in year t we also
have available the capacity
which was brought on line
one year ago, two years ago,
and so on and so forth
until 19 years ago,
so for a total of
the last 20 years.
This is why wt can then be
written as the sum of xs for s
that ranges from the max
of 1, t minus 19, and t.
I'm sure at this
point you're wondering
why there is this strange max.
It would have been kind of
more intuitive to simply have s
that ranges from t
minus 19 until t,
and this matches exactly the
discussion that we just had.
However, remember that our time
horizon starts only in year 1.
Therefore, we don't want
to consider any xs with s
equal to 0, minus 1, minus 2,
minus 3 because those decision
variables don't exist and
also don't make sense.
In a sense, it's just like
having all of them being
equal to 0 since there was
no coal plants before year 1
because we only had oil plants.
So essentially when we sum
the last 20 xs for the last 20
years, we can just stop
at 1 in case t minus 19
is a negative number or 0.
For example, this
constraint for t equal to 3
will be simply w3 equal
to x3 plus x2 plus x1.
Now similarity, nuclear
plants last for 15 years,
and so we can write zt equals
the sum of the ys for s
that goes from max 1,
t minus 14, until t.
Once again, except for
this technical case
where the 1 comes into
play, in total I'm
summing the very
last 15 variables ys,
and this is because nuclear
plants last for only 15 years.
So now we can get back to the
constraints of the problem.
And now that we have introduced
our new variables wt and ct,
it will be very simple
to write them down.
So the first constraint says
that the available capacity
must meet the forecasted demand.
Therefore, for every year t
that ranges from 1 to capital T,
the demand is dt, and
the available capacity
is the available
capacity in oil plants
plus nuclear plants
plus coal plants.
And so we can simply
write wt plus zt plus et
greater than or equal to dt.
Another constraint is
that no more than 20%
of the total capacity
should ever be nuclear.
In order to write this
constraint, for every year
from 1 to capital
T, we simply write
the ratio of the
total nuclear capacity
over the total capacity.
So this is zt over
wt plus zt plus et.
And this must be at most
0.2 because it's 20%.
Note that this
inequality is not linear
because the function
on the left hand
is not a linear inequality.
However, it can be
easily transformed
into a linear constraint by
multiplying both the right hand
and left hand by the
denominator wt plus zt plus et.
In this way, the denominator
will vanish on the left hand,
and it will be multiplied
by 0.2 on the right end.
Now zt will appear
both on the left hand
and on the right hand.
So part of it can be simplified,
and what you obtain is this
inequality 0.8zt minus 0.2wt
is less than or equal to 0.2et.
You might be wondering why
I suddenly brought the 0.2et
on the right end
of this inequality,
and the reason is that it is a
standard notation to have all
the variables on the left
hand and all the numbers
on the right hand.
And remember that zt and
wt are decision variables.
These are things we
will have to decide.
On the other hand,
et is data, is
a specific number for every
t that is given in input.
At this point, we can write
down the full linear-programming
formulation.
We have to minimize our
objective function subject
to all the constraints
we have so far written.
The first one is the
equality constraints
that comes from the definition
of wt in terms of the x.
The second comes from the
definition of zt in terms
of the y's.
Then we have the twin equalities
we have just written down.
And we shouldn't forget that all
our decision variables should
be non-negative
because we can only
bring on line a non-negative
number of megawatts.
At this point, as
promised, I want
to spend a couple
of minutes regarding
the additional variables
that we defined.
So the reason is that I'm
sure you're wondering,
well, what if I didn't
know I should introduce
those type of variables?
In fact, I kind of defined them
a little bit out of the hat.
Well, now if you think about
it a little bit more deeply,
they're not really
out of the hat
too much because
when we wrote down
the main inequalities
of the problem, which
are these two over
here, these quantities
are those that appear exactly
in these inequalities.
And so it kind of makes sense
to think about those first
before writing down
these inequalities.
In any case, even if you
didn't think about this,
it doesn't really matter
because we can, at this point,
write down another
linear-programming problem that
doesn't have these
new variables and that
is equivalent to this one.
How can we do that?
Well, look, wt is equal to this
sum because of this constraint,
and zt is equal to this sum
because of this constraint.
Well, that's obvious.
That's exactly how
we defined them.
So now what we can do is
simply trash these equality
constraints and then
replace wt and zt
with the corresponding sum in
the remaining inequalities.
So then we will no
longer have wt here
and no longer have zt
here, and instead we'll
have a sum of x and
here a sum of y.
Similarly, also here we only
will have y and x variables.
And so in our formulation,
these variables will completely
disappear and end up with a
linear-programming problem that
only contains the
variables x and y's.
The last thing I want to discuss
is what about non-negativity?
So here we quickly wrote that
all our decision variable
should be non-negative.
However, if you think a little
bit more deeply about this,
we could definitely avoid
writing wt and zt over here
because what is really
important is that x and y's are
greater than or equal to 0.
Then the non-negativity
of w and z
immediately follows from
these equality constraints.
In fact, each wt is the sum
of a bunch of x variables.
And since they're
all non-negative,
then so will be wt as
well and similarly for zt.
And this concludes our example
on multiperiod planning
of electric-power capacity.
The next example of a
linear-programming problem
that we're going to see
is a scheduling problem.
In this example,
we have a hospital
that needs to make a
weekly night-shift schedule
for its nurses.
And we know exactly
for every night
of the week what is
the demand for nurses,
and this is encoded
in this quantity
dj for j that goes from 1 to 7.
So in day 1, the demand is d1.
Day 2 is d2 and so on and
so forth over the week.
And the next week, everything
repeats with the same d1,
d2, d3, until d7 and
so on and so forth.
An important constraint
in this problem
is that every nurse needs to
work for exactly five days
in a row on the night shift.
And, of course, the
question is to find
the minimal number of nurses
the hospital needs to hire.
As always, the
first thing to do is
to decide what are the
decision variables?
Probably the most intuitive way
of defining decision variables
is to have a
variable x1, x2, x3,
until x7, so seven variables.
And each one should
represent how many nurses
work in that specific day.
I encourage you to
try to write down
a linear-programming
problem that
uses these decision variables.
You will soon realize
that it's very difficult,
if not impossible, to
write down the constraint
that every nurse should be
working five days in a row.
Instead, we define a
different decision variable
xj which represents instead
the number of nurses
that start their week on day j.
So they're going to be working
on day j, j plus 1, j plus 2,
j plus 3, and j plus 4--
of course, modulo 7.
So at this point,
we can immediately
write the formulation.
So we want to hire the minimum
possible amount of nurses.
Every nurse must start
in a specific day.
Therefore, the total
number of nurses hired
is exactly x1 plus x2
plus x3 until plus x7.
So let's now talk
about the constraints.
We need to write down that
in every day of the week
we have enough nurses.
Let's start, for
example, with the day 7.
So we need at least d7
nurses working on that day.
And so which nurses will
be working in that day
or, well, in that night?
Well, those that start
their week on the same day
plus those that started
their week one day
before, two days before,
three days before,
and four days before.
Now the ones that started five
days before, of course they
will not be working on the day
7 because they have completed
their shift on the day 6.
Therefore, the constraint
corresponding to day 7
is this one, x3 plus x4
plus x5 plus x6 plus x7
greater than or equal to day 7.
Now similarly, you can
obtain all other days.
For example, what are the nurses
that will be working on day 1?
Well, they will be the nurses
that started on the same day,
so x1, plus those that started
in the previous day, which
is day 7, plus day 6 plus
day 5 plus day 4 of the week.
Next, the number of
nurses hired in each day
must be for sure greater
than or equal to 0,
but it also must be
an integer number.
We definitely cannot hire
0.3 nurses on a specific day.
It must be an integer.
Therefore, this problem is not
a linear-programming problem.
It's very similar to a
linear-programming problem.
The only problematic
constraint that
is not allowed in
linear programming
is exactly this
integrality constraint.
Due to this
integrality constraint,
this is called an
integer-linear-programming
problem.
These problems are much
more difficult to solve
than a linear-programming
problem,
both from a
theoretical perspective
and from a computational
perspective.
As you can imagine,
integrality constraints often
arise in a real-world
problem, and there
are several techniques to
deal with these problems.
We're not going to talk
about it in this course,
but we have a full
course devoted
to this type of optimization
problem, which is the course
728, Integer Optimization.
In any case, I
still want to make
a couple of interesting
observations
about this
integer-linear-programming
problem that we have just
seen without the need for you
to take 728.
The most natural
thing that we can
do in this
integer-linear-programming
problem is just to ignore
the integrality constraints.
Well, why?
Because in this way, we obtain
a linear-programming problem,
and this is called
the LP relaxation
of the original
integer-linear-programming
problem.
So here is our problem,
and as you can see,
we just trashed this
integrality constraint.
Now the first observation
that we can make
is that the optimal cost of
the linear-programming problem
will be less than or
equal to the optimal cost
of the original problem.
The reason is that
any feasible point
for the
integer-linear-programming
problem of course
will be also feasible
for the
linear-programming problem
because it just has
fewer constraints.
Therefore, the
linear-programming problem
has a physical
region that is larger
than that of the original
integer-programming problem.
Since we are minimizing the
same objective function,
then the optimal cost of the
linear-programming problem
can only become smaller.
The next observation is
that if it just so happens
that the optimal solution to
the LP relaxation is integer,
then it is also an
optimal solution
to the original problem.
So why is this true?
Let's call x star the
optimal solution to the LP.
Since x star is feasible to
the LP and is also integer,
it is feasible to the
integer-linear-programming
problem.
But then why is it optimal?
Well, because it's
already the best
solution among all the
feasible solutions to the LP.
Therefore, it must be the best
solution also among the smaller
set of the feasible solutions to
the integer-linear-programming
problem.
So this gives us
a nice way to try
to solve
integer-linear-programming
problems, namely trash the
integrality constraint.
Solve the
linear-programming problem.
You just might be lucky and
obtain an integral solution,
in which case you
have completely
solved your
integer-linear-programming
problem.
Of course, in general, you
will not be that lucky,
and this is where
things become harder.
However, we can make a very
simple observation that
doesn't work for any
integer-linear-programming
problem, but it works for the
example we are talking about.
Namely what you could do is
obtain a feasible solution
to the original problem
by rounding every xj
that you obtained upwards.
For sure you obtain
a feasible solution
to the original problem
because by rounding
you obtain integer variables.
So you satisfy the
integrality constraint,
and you will still satisfy
all the inequality constraints
because you're just making
larger the left-hand side.
Of course, this solution
is not necessarily optimal,
and it's also really hard to
say how good the solution is.
So it's definitely
not a satisfactory way
to deal with our
scheduling problem,
but at least it's a way.
If you want to
know more about how
to deal with
integer-programming problem,
you should, of course, take 728.
The next example is
about choosing paths
in a communication network.
In this problem, a communication
network is given to us,
and the communication network
looks something like this.
The way you should
think about this
is that you have a
bunch of, let's say,
offices of your company,
and these are formally
called the nodes of the
communication network.
Now in this picture, these
are the circles with a number
inside.
Then you have a number
of communication links
that connect your nodes.
Now in this picture,
these are the arrows.
For example, here we have
an arrow from 1 to 4,
and this represents
the fact that you
can transmit data from
your node 1 to your node 4.
Formally, we're going to denote
our communication network by G,
which is a pair, and A
where N is the set of nodes
and A is the set of
communication links.
To denote a link, we're
going to use this notation.
i, j in parenthesis means that
this is the link starting in i
and ending in j.
In this problem, then we're
given a communication network,
but this is not the only
data that we're given.
In fact, for every
link i, j, we also
are given a number
uij that represents
the maximum amount of
bits per second that
can be sent along such a link.
Moreover, in order to send this
amount of data, we have to pay,
and we're given
for every link cij
the cost per bit transmitted
along such a link.
And this is encoded in cij.
Then we are given values
bkl for every pair
kl of different nodes.
And bkl represents
the amount of data
that is generated at node k and
that should be sent to node l.
So how can we transmit the data?
Well, we can either
transmit it directly
from k to l along a direct
link kl as long, of course,
as it exists, or we can
instead use a sequence of links
that starts in k and ends in l.
Moreover, the data
from k to l doesn't
need to travel
along the same path,
but it can be split
and be transmitted
along different paths.
What is then our objective?
As you can imagine, it is
to send all the data that
is generated in our
communication network while
at the same time
minimizing the total cost.
As always, to model this
problem as a linear-programming
problem, we first have to
decide our decision variables.
For this problem, our decision
variables are x, i, j, k, l.
They indicate the
amount of data that
traverses the link ij that
is generated in the node k
and is destined to node l.
Once we have defined
these decision variables,
it's really easy to write
down the objective function,
as always so far.
So how much do we have to
pay to send the data xijkl?
Well, this is data that
traverses the link ij,
and the cost per bit in
such a link is given by cij.
Therefore, it's going
to be cij times xijkl.
Of course, we now need
to sum over all links ij
and over all origins and
over all destinations,
and this gives us our objective
function to be minimized.
Let's now discuss
the constraint.
Our first constraint
is the easiest.
In fact, we can only send
a non-negative amount
of data along every link.
Therefore, we have
non-negativity
on all our variables.
Then we have the
capacity constraints.
Remember that for every link
ij, no more than uij bits
can be sent per second.
Therefore, we only
need to compute
how many bits per second we
are sending along such a link.
In order to do so, we take
our xijkl and sum over all k
and over all l.
And this inequality
holds for every link ij.
We are only left with one type
of constraint to write down,
and these are
constraints that are
very common in this type
of network problems,
and they're called
flow-conservation constraints.
Think about the data generated
in k with destination l.
In this problem, we have to
decide how this data flows,
essentially, from k to l.
So essentially what we need
to do in these constraints is
guarantee that the only
type of data with origin k
and destination l
is generated in k
and is absorbed,
essentially, in l.
And along any other
node of the network,
no such data is ever
generated or absorbed.
In order to write down
these inequalities,
we need to write
down what is called
the net flow at every
node i, and this
will be denoted by bikl.
This represents this quantity
that we just discussed,
namely the amount of data from
k to l that is generated in i.
Formally, this is
the flow that exits
i minus the flow that enters i.
So how is this quantity defined?
Well, it is, of course, 0 if
i is different from k or l.
On the other hand,
if it is equal to k,
then bkl is generated
in such a node.
And if i is equal to l, then
bkl is absorbed in that node,
and therefore we're
going to write minus bkl.
We're now ready to write a
flow-conservation constraint.
Essentially to do
so, we're going
to have to write the same
net flow using our variables
x instead.
So the definition of
net flow, once again,
is the flow that exits i
minus the flow that enters i.
So let's start.
Write down the
flow that exits i.
So if it exits i, it will
correspond to a link ij in A.
And so everything
we need to do is
to sum over all these links ij
for any j the quantity xijkl.
Then from this quantity, we need
to remove the flow that enters
i, and this is very similar.
However, such a flow
will enter through a link
of the type ji for any possible
j, and we're summing xjikl.
In this way, we have written
our net flow at node i
with respect to the flow with
origin k and destination l.
Earlier on, we wrote
down the same net flow
that we should achieve given
the data of the problem.
And so we have to enforce
equality between these two
quantities.
How many inequalities
do we have?
Well, we will have one for
every i and for every kl.
And that's all.
We are ready to write
down our full formulation,
and I'm just pasting here
our objective function
together with all
the constraints
that we have written so far.
Now in the real-world
application
that we were thinking
about in this example,
we had a bunch of different,
let's say, offices,
and we wanted to
transmit data among them.
Now, of course there can be many
different real-world problems
that actually fit the same type
of linear-programming problem.
Another example is
whenever you want
to transport several different
commodities from their origins
to their destination
through your network,
and this problem is known as
the multicommodity-flow problem.
And every commodity
corresponds to the traffic
in each origin-destination pair.
In a simpler version
of this problem,
we don't distinguish between
different commodities,
and this is called the minimum
cost network flow problem.
In this problem, instead
of having all the bkl
data in input, we only are given
the amount bi for every node,
and this represents the
external supply or demand
at each node i.
If bi is positive, then i
is called a supply node.
And if bi is negative, then
it's called a demand node.
Then the problem is,
of course, to transport
the material from the supply
nodes to the demand nodes
at minimum cost.
In turn, the network-flow
problem contains a bunch
of other problems as
very special cases--
for example, the
shortest-path problem
which finds the shortest
path to go from an origin
to a destination point; the
maximum-flow problem, which
is the problem of sending
the maximum amount of flow
possible, again, from an origin
node to a destination node;
and the assignment
problem that finds,
for example, the best-possible
assignment of workers to jobs.
Therefore, all this
network problem
can be solved via a
linear-programming problem,
and they can be solved
efficiently in this way.
However, it's important to
note that this problem has such
a special structure and are so
important that people have been
studying them and developing
for them algorithms
that works specifically just
for one of these problems.
Of course, such an
algorithm can take advantage
of the very special
structure of this problem,
so in general it will be much
faster than solving the problem
with linear programming.
In UW Madison, we have
a course dedicated
to this type of
algorithms, and this
is it our 425
course, Introduction
to Combinatorial Optimization.
And this is all for this video.
In the next video,
we're going to look
at a class of nonlinear
optimization problem that
can be formulated as linear
programming problems.
