Thank you for your
patience, everyone.
I'm going to be very brief.
I'd like to present
to you Dr. Weintraub.
He's been
a tremendous colleague.
This marks my 24 months here.
I've worked with Dr. Weintraub
in the ETR (unintelligible),
and working with him
has simply been remarkable.
And without further
delay, Dr. Weintraub.
(applause)
Thank you, Murray.
So, once again, it's a pleasure
to talk to this group.
And hopefully we'll be able
to bring on everybody
from the CTR, if--
I'll be covering some
of the same ground
at the CTR annual meeting.
Our focus will be
a little bit different.
So if this doesn't
work out, I don't know
if they're even hearing me
right now, are they hearing me?
They don't even
know (unintelligible).
You'll have other opportunities.
So I wanted to talk particularly
about comparative effectiveness
and big data.
And really to show
our capabilities.
Because our colleagues
(unintelligible)
said to me the other day,
we're really capable of anything
analytically, at this point.
This is some pretty
complicated data analyses.
Once presented, once understood,
it's actually very simple
to understand the material.
And that's always our goal,
that we can present
and bring forward
complicated data sets,
complicated ideas
structured in a way
that makes it highly available
to clinical audiences
and allows us
to make good decisions.
Okay, so now--
All right, so I can just
advance it with--with this guy?
Yup.
Left click is advance.
All right.
We'll see what happens.
There you go.
Okay.
Okay.
Scroller is much too sensitive.
And I've had
this problem before.
I'll have to have Eric back up
because he knows how
to deal with the mouse.
Okay, so do we need
comparative effectiveness
to improve quality?
What's this all about?
We've been comparing diagnostic
and therapeutic strategies
for decades.
However, randomized trials
don't really
meet all of our societal needs.
And why is that?
Most randomized trials,
the vast majority
of randomized trials,
are conducted
for registration purposes,
for new pharmaceuticals
or new devices for the FDA.
And so the kinds of questions
that we've come up
with all the time,
in serving therapeutic
and diagnostic strategies
that are of great
societal interest,
may not have been
adequately addressed.
There are some gaps
in our knowledge base
that we really need to address.
And that's why you've heard,
over the last year,
so much--over the last
several years,
so much concern
about comparative effectiveness.
So what kind of studies
do we have that allow us
to address the issue
of comparative effectiveness?
Well, of course,
the gold standard
remains randomized
controlled trials.
But then there
are observational studies.
They can come
from clinical databases,
or administrative databases,
or a mix between them,
which is what
I'll show you today.
And then, within each of these,
you have cohort studies
where you get a group
of patients together
and follow them forward
and look at outcomes,
or case control studies, which
you start out working backwards.
You take people who have
some kind of outcome
and people who don't,
and then look back
at risk factors.
Other kinds of studies,
in part, systematic reviews
in their analysis
of simulations and qualitative,
separate qualitative
search simulations.
(unintelligible)
All right, so what
are the advantages
of randomized trials?
There's really only one.
This is the only method
that can overcome
treatment selection bias.
You'll say, well,
is it worth it?
Is it worth it, all the great
expense of randomized trials,
and all the time it takes,
just to overcome
this one little thing
called treatment selection bias.
Well, there are many
clinical trialists who believe
that observational studies
are completely worthless,
and we should only
do randomized trials
because treatment selection bias
is so great that we can never--
that we can never overcome it.
So it's a debate that
goes on and on and on
without resolution.
And we'll address
a little bit more of that,
as you'll see
as we (unintelligible).
We accomplish certain things,
very real things,
with non-randomized approaches.
There are disadvantages
of randomized trials.
They lack generalizability,
and also I'll show you
graphically how that can work.
They're very expensive.
They become outdated.
They're crossovers.
This nonblinding of studies--
we really like it
when they're blinded,
but strategies and studies,
usually do not blind them.
There's limited power
to look at subgroups.
Even with really
big mega trials,
there's often very limited power
to look at subgroups.
And then you have problems
finding groups to be compared.
Well, you have that in anything.
You think, "Well, I'll compare
this group to that group.
So this could be an example."
We actually were part
of an NIH grant here
to look at use of genetics
and picking antiplatelet agents,
P2Y12 agents,
and we spent two years
on the strategy group
for the trial trying to figure
out what the design was.
Never could figure it out.
Finally we had
to abandon the trial.
So it's not always
straightforward
to find what you want to study.
Asking good questions is always
the most important of things.
So let's look at
generalizability a little bit.
So I do not have
a pointer on this, do I?
No.
No, okay.
All right, so, the problem
of generalizability.
You want to look at your
population distributions
to really understand outcome
in this population.
How do you do it?
The whole principle that we use
in doing research is we sample.
We don't look
at the entire population.
In principle what our
presidential election is,
where you're looking
at everybody.
But we don't do that.
We sample.
And so how do you draw a sample
that'll allow you to look
at the whole population?
In a randomized trial,
you look at--
if you draw from sample 1,
sample 1 may tell you
about the mean,
but it doesn't tell you much
about this subgroup
that's in gray.
But what happens
if you draw a sample 2?
You can't say
much about the mean.
We can't say anything at all
about the group that's in gray.
You would say,
"We'd never do that.
We're not going to draw
a sample like that."
But if people
don't like the result
of your randomized trial,
they will accuse you
of it anyway.
They'll say, "You wanted
to look at this group,
but you got that group."
You get at that better
with observational studies.
It's really a realistic
ability to go wider
and look at the full spectrum
of your population,
including the ability to look
at subgroups of interests.
So, there are advantages
and disadvantages
of registry
and administrative data
for comparative effectiveness.
Once you get out
of observational studies,
you have very
large sample sizes.
Ability to look at subgroups.
You have real world patients,
much more contemporary data.
But there are disadvantages.
Treatment selection bias,
as we discussed.
Data quality in observational
studies, in principle,
you could set up
an observational study
just like a randomized trial
collect data
with the same integrity.
But in practice, people
generally don't do studies
in dealing
with data quality issues.
(unintelligible)
Outcomes in randomized trials,
the ones you're really
most interested in,
you're going to adjudicate
by a committee.
That virtually never happens
in observational studies.
And when they're really big,
it's not practical.
Uncertain definitions
of the treatment groups.
This goes on especially
in pharmaceutical
observational studies
in which the exposure period
may be very uncertain.
And people are endlessly
arguing about whether
you have an adequate exposure
to one pharmaceutical
versus one that you're
trying to compare to.
And then the covariate
and outcomes data
of interest
may not be available.
And this is really
much more of a problem
with the covariates
in observational studies
than in randomized trials
because, in principle,
randomized trials
can make your groups
look exactly the same.
That's what
randomized trials do.
By overcoming treatment
selection bias,
you're--the descriptions
of your two groups
should be--should be the same.
The larger
the randomized trial,
(unintelligible)
to being true.
But the covariates
are really very important
in observational studies
because you use
the characteristics
of the patients of the study
to correct for differences
between the groups.
And the outcomes, the things
you're most interested in,
are very often not available.
So what do you do?
So we have this problem with
these large national databases,
such as the Cardiovascular
Data Registry,
the STS database,
the actual outcomes data
you want are not available.
And so you have to figure out
how you're gonna handle that.
And I'll have quite
a bit of information
about that and
the kinds of approaches
that people are taking.
Okay, so now we're
just comparing
administrative data
and registry data.
Administrative data
are readily available.
Registry data require
prospective collection.
So, in principle,
we like registry data.
It's clinical data.
It takes a lot
more work to get it,
to put it together,
and it's much more expensive.
Analysis of administrative data
are relatively inexpensive,
and relatively easy
within administrative data,
actually, to have
long-term outcomes.
But there are disadvantages
to administrative data as well.
They lack clinical description.
It may lack critical variables.
And generally,
if you have problems
with quality of data
in registries,
the quality of data
in administrative databases
is even worse.
Now let's move on
to the kinds of approaches
people take to overcome
treatment selection bias.
That's a theme
that'll sort of run
throughout as I
go through examples.
The first thing that
people try to do,
looking at observational data,
is to have very rigorously
defined groups.
That's all very nice,
but then you're
really very limited
into the patient section that
you're going to look at.
And then there's standard
multivariate analysis
such as logistical regression
and Cox model,
which use statistical
approaches to try
and account for differences
in the covariate data
between your patients.
So it's a method of,
statistically,
of overcoming treatment
selection bias.
That's sort of been
overcome or replaced today,
in many of these studies,
by propensity score analysis.
The propensity score is--
if you compare
one form of treatment
and another,
you create a model
to see the propensity
for one treatment
compared to the other.
And I'll show you
examples of that,
it'll become more clear
as I work through this.
And most propensity scores,
you use
a statistical approach
called logistic regression.
Then how do you use it,
if you have the propensity.
One is to define subgroups
by the propensity score.
It's a fairly common approach.
Another is just to use it
as the covariate
in the multivariate analysis.
Well, I've always felt that
that didn't make much sense
because if you got
the covariate data,
it doesn't make much sense
to form another score
that will replace
the covariate.
So it doesn't add any
information.
So I've never been
much of a fan of it.
And then there are matched
groups by propensity.
That's historically, or at least
for the last number of years,
been the most common approach.
You'll see many papers
in literature,
and it really looks nice
because when you do that,
it looks like you've
got randomized trials.
You've got the same number
of patients in both groups
or sometimes two to one,
but you have--
and all the covariates
look exactly the same.
To some extent,
that's been replaced
by inverse probability
weighting.
Now the advantage of inverse
probability weighting,
is it's a statistical
approach that allows you
to take two groups
and essentially line them up
so they look--
they look the same.
The advantage of inverse
probability weighting
is that you don't lose
any of your patients.
You can keep all your patients.
Now you may say,
"Well, that's all very nice,
but if you have one group over
here, and one group over there,
they can be outliers.
Both groups don't belong
in this analysis."
You can trim it as well,
so you can overcome
that particular problem.
With that being said, with all
these methods,
I'm unconvinced that any of them
add anything to simple
multivariate analysis.
They look great.
They really look great.
But do they overcome
treatment selection bias?
And they really don't,
in the end of the day,
because you can
only correct for what?
You can only correct
for the things you measure.
You can't correct for the things
that you don't measure.
Now (unintelligible)
to get away from that,
is use what's called
an instrumental variable.
So it's a different
kind of approach
to overcoming treatment
selection bias.
An instrumental variable
is a variable
that cleanly
separates your groups
but does not predict outcome.
Economists really
like instrumental variables.
You'll find biostatisticians
and epidemiologists
don't do this very much.
There is one that works--
one instrumental variable
that does work.
Who can tell me what that is?
(unintelligible)
No.
Good guess, often used.
But who's to say
that because you have
different geographical locations
that it overcomes
all the differences
in your covariates?
Randomization.
Randomization is
an instrumental variable.
Cleanly separates
your groups and by itself,
randomization by itself
does not predict outcome.
To my mind,
as an epidemiologist--
you know what
epidemiologists do,
they criticize everyone
else's methods.
Because as an epidemiologist,
I can say
that randomization is
the only one that works.
Okay, pharmacologic versus
strategy trials.
When we--when we use
observational studies,
when we use randomization.
Strategy RCTs,
such as PCI versus CABG,
CABG versus medicine,
PCI versus medicine.
They suffer from crossovers,
small sample sizes,
unblinded design,
and the data become obsolete.
That creates a place
for registry studies
and comparative effectiveness
using nonrandomized data.
On the other hand,
pharmacological RCTs
have limited crossover,
large sample sizes,
blind design, and less
problem with obsolescence.
The other problem with--
the problem you have
with registries look
at pharmaceuticals,
as I said, you have problems
defining exposure.
People are on
and off the product.
So there's been less
of a place, in my mind,
for comparative effectiveness
studies
using observational data
to look at pharmaceuticals.
However there really is
a place for very large datasets
to look at safety, including
safety of pharmaceuticals.
Many, many studies
in pharmacological epidemiology
with just that subject.
So a great source of data
for all these kinds of studies
are these national registries.
The STS and the NCDR.
The NCDR was formed
in the 1990s,
and now has grown to half
a dozen or so of registries
and we participate
in almost all of them here.
And there are now
hundreds of papers
that have come out
of these registries.
So, 2,000 hospitals,
16-plus million records.
It's a large database.
The CathPCI, we'll show
you data from CathPCI.
Largely,
its 15 million-plus records.
And that has a penetration
of over 90 percent
of the cath labs.
I don't know exactly,
95 percent, something like that,
of the cath labs in the country
participate in the NCDR.
So that really solves
the problem of generalizability.
It's essentially everybody.
Staying with the STS,
Society of Thoracic Surgeons,
for the CABG database.
Virtually 100 percent
penetration.
And some are absolutely
100 percent of penetration.
So the ICD registry
is a mandated registry.
That's 100 percent
of penetration
for the Tavr Program,
percutaneous aortic
valve replacement.
That's a mandated registry.
There's 100 percent.
And you'll be seeing
some very interesting
comparative effectiveness
studies coming out
of that as well
over the next couple years.
So here's one of my favorite
comparative effectiveness
studies
done with the NCR,
and what we did with this
is just 59 institutions,
not that big, 14,000 patients.
And we collected
some prospective data
on use of closure devices
after cardiac catheterization.
So we didn't just
use the registry,
we had to collect
additional data as well.
But not really a lot.
It was nonrandomized.
And we didn't do
specific statistics.
So very simple
multivariate analysis.
(unintelligible)
And the most serious
of events was bleeding
and one of the devices,
VasoSeal, was demonstrated
to have a high risk
complication odds ratio--
(unintelligible)
Now to these closure devices.
(unintelligible)
This is what I wanna point out.
One thing is when you
put a study up,
that you're in the middle of,
a research study,
you never wanna give all your
information on how you do it.
Okay, so these closure devices
are not such great
treatment selections bias.
Why use one closure device
versus another,
and so here
the treatment selection bias
that you have
in nonrandomized studies
is not felt to be
such a big problem.
The VasoSeal was off
the market very rapidly
after the publication
of this study.
This is comparative
effectiveness really done right.
We served a societal need
with this,
without having to mount
a massive randomized trial.
Now the other thing
about closure devices
is they were approved by the FDA
based on efficacy alone,
with just a couple
hundred of patients.
But you can't get its safety
with a couple hundred patients,
but we could with 15,000.
It allowed us to make
what I think was
a very good (unintelligible).
Here's a little bit
more data from the study,
and we'll go into that.
Here's another study,
somewhat larger in scope.
Also from the NCDR.
Comparing bare-metal
and drug-eluting stents.
So it's led
by Pam Douglas from Duke.
And what's interesting
about this study,
this is the first comparative
effectiveness study
done with these databases
in which
the NCDR was linked
to administrative databases.
It was linked
to Medicare database
to allow us to look at outcomes.
So here are the data.
It's a huge study.
Drug-eluting stents:
218,000 people.
Bare-metal stents:
45,000 people.
And to bring these
into alignment,
they used inverse
probability weighting.
So you get to use
all the patients,
and essentially
bring them into alignment
so they look the same.
On the left, you'll see
the unadjusted analysis,
on the right,
the IPW adjusted analysis.
And there were small differences
between the groups,
like the age is a little bit
of a difference here.
But essentially it went away
with the adjustment.
That's what IPW does.
It essentially makes
the groups look the same.
So that's all very nice.
What did they find?
So here are the results,
30-month event rates.
And they found that there
was a mortality advantage
with bare-metal stents
and but no difference with
the graph for vascularization.
(unintelligible)
Opposite of clinical trials.
Any clinician that looks
at this, you know,
first clinician that
looks at this says,
"I don't believe this.
Is this right?"
So how does this compare
with the clinical trials?
So here we go.
Here's the meta analysis
of 21 clinical trials comparing
drug-eluting
and bare-metal stents,
looking at mortality.
Which is what--so--so--
if there's a takeaway,
if there's one takeaway
from the day,
that's very important to us
as we go forward,
we consider developing
these kinds of studies
at Christiana
and across our partner
institutions in the CTR,
size does not overcome bias.
I'll say that one again.
Size does not overcome bias.
And so what're
you gonna believe?
And so at the end of the day,
when they're going
in opposite directions,
the randomized trials,
even though it's a much smaller
number of patients,
9,000 versus 300,000,
randomized trials will trump
the observational data.
That didn't deter us, of course.
So we went ahead
with ASCERT.
I know Zugui--I missed it
last week because I was out.
Zugui--or the week before--
Zugui presented
interesting data on ASCERT.
Probably pretty technical.
This one would've been better
to have this one first
and his after, but okay.
You know, it is what it is.
And this is going to be a much
less technical presentation.
It'll cover some,
but not all
of the same ground.
So this is the final results
of the ASCERT study.
This was supported
by a Great Opportunity grant
from the National
Institutes of Health.
And here we--because we're
comparing PCI and CABG,
we used two--both
of the big databases.
We used the NCDR database,
and we used the STS database,
and had 600 sites.
So in clinical trials, if you go
to the national means,
you will see
the clinical trials,
they're very proud
to put up pictures
of all their sites,
50 or 100 sites, something.
But none of them have maps
that look like this.
With 600 sites.
Okay, so our purpose
initially was to compare
long-term mortality of CABG
and percutaneous
coronary intervention
in patients with stable
ischemic heart disease.
We've linked both
of the databases
to CMS 100 percent
denominator file,
so we've linked both of them
to MEDPAR,
and then we look at
the propensity for CABG
using logistic regression,
and then we brought the groups
into balance
using inverse probability.
(unintelligible) all
the other methods as well.
Uh, and then we did
sensitivity analysis,
which I'll--I'll
ultimately get to it.
So this is just
a consort diagram
of how we get to
our study population.
And here's the propensity
for CABG.
Now this is--we're beginning
to see this kind of figure
more and more, and really,
everybody that does
propensity analysis
should have a figure
something like this,
in which you look at
the propensity for--
for CABG and the propen--
uh,
and, uh,
the PCI group in green
and the CABG group.
This is propensity for CABG,
they have
a much higher propensity
based on the covariance for CABG
in the group that
actually got CABG
than the group who got the PCI.
You look at this, you say,
"We can't compare those.
Those groups are
so very different,
we can't possibly, uh,
compare."
That's one view.
The other group is, "Wow,
you really define
the differences
between people who got CABG
and people who got PCI,
so you accounted for
all the differences,
therefore all I have to do is
correct for that statistically
when you compare them."
So you decide.
Here--here are the baseline
data.
Unadjusted and IPW Adjusted.
And there were, uh,
statistically significant
differences
between the groups
before correction
that largely went away
after correction.
Of course, you can have
very small differences
between--between groups
and still have it be
statistically significant,
so 73.1 versus 74.5 in age,
highly statistically
significant.
The p-values here,
they don't mean a thing.
When the groups are so large,
the power doesn't mean anything.
You've got
statistical significance
when they're
very small differences.
But sometimes,
the differences are much larger.
So if you look at
3 Vessel Disease,
32.1 in the PCI group,
80 in the CABG group.
And again, it gets corrected
by IPW.
Now you may say,
"I don't believe that.
Those differences are so great,
represent such difference
in--in your groups
that I don't believe that
this kind of correction works."
And so that's a reason for using
other statistical approaches.
(unintelligible)
So here are the data,
unadjusted data
and the adjusted data
on the right,
I mean, IPW.
And there's really not that much
difference after you--
after we've adjusted
this (unintelligible).
Sort of two different--
two different scales.
This allows us to blow up
the differences.
And if you look here, it's the
most sensitive one over here,
If I initially have
higher mortality
with--with CABG
than with PCI--
CABG in the red,
PCI is in green.
And initially you had, like,
initial loss with CABG.
At one year,
there's no difference,
but then you have increasing
difference over time
so that by four years,
there's a significant difference
between them,
with the mortality of CABG
at 16.4 percent
and PCI at 20.8 percent.
Are you gonna believe it?
Maybe.
Maybe.
Okay, that's how it fits in
with other data,
and I'll show you that
in just a second.
We also did a matched analysis.
If you don't like IPW,
you can also do a matched.
So what happens in the matched--
here's the unadjusted,
and that's the same
as you saw before
for the baseline data,
and here is, uh, using matched.
It's important to see
here it comes out
to the same numbers,
43,000 in both groups,
and again, you find no, uh,
there's very little difference
between the two groups
after they're corrected.
And here we have
on top of each other
all of the different approaches,
all the different
survival methods,
and it made no difference.
Initially you have that
(unintelligible) with CABG.
At one year there's no
difference in mortality
between the groups.
Then at four years,
there's a mortality advantage
with CABG compared to PCI.
We also look through all
of the, uh, the subgroups.
Now when we set this up,
we were sure,
we were just absolutely sure
that we were going to define
those subgroups
in which there'd be
an advantage to PCI
and those subgroups
in which there'd be
an advantage to CABG.
Didn't happen.
Turns out that in
all subgroups examined
there was a mortality advantage
to, uh, PCI.
A couple of interesting
things here.
In particular, I'd ask you
to look at,
at diabetes,
this group here in blue,
and in dia--
in patients with diabetes,
randomized trials have
fairly consistently shown
a survival advantage
to CABG compared to PCI
in multi vessel disease.
And while we do--we found
a survival advantage overall,
you can see that there's
much greater survival advantage
in insulin-dependent
patients with diabetes
compared to patients
without diabetes.
So sort of directionally
was consistent with--
with, uh,
the randomized trial data.
Now, could there be a, um,
a confounder
that explains the differences
between our groups?
So this is another very
important kind of analysis
that should be done
in comparative effectiveness.
To see if you can come up
with a confounder
that would explain the
differences between two groups.
And so a confounder,
what does a confounder do?
A confounder predicts outcome
and has to have different
prevalence in your groups
under comparison.
The different prevalence
of a confounder
between PCI and CABG
that would predict outcome
and then account for
the differences between those.
Is there a potential
confounder like that?
Well, there's been a lot
of interest
in one in particular,
and that's frailty.
So could there be a difference
in prevalence between frailty?
We avoid doing CABG
in patients that are frail
and tend to do more of them
in patients--
tend to do more PCI
in frail patients.
And could that explain the
differences that we're seeing?
Now, it doesn't have to be
one confounder,
it could also be a composite.
It could be a set of things.
But if we just use frailty
as a consideration,
could that happen?
Suppose you've had an incidence
of frailty,
of 30 percent
in the PCI group,
but only 10 percent
in the CABG group.
Then you start off here
with PCI at 30,
then go up to the green line,
and then from there go over
to the x-axis.
So, if there was
a three times difference
in the prevalence of frailty
between the groups,
10 and 30, you'd only need
a hazard ratio for frailty
of about 2.4 to explain
the mortality differences
between the groups.
Could that happen?
Well, you know,
that really happened.
So those who really loved
the study and said,
"This is definitive results
that you'd never find that--
you'd never find a confounder
that powerful,"
and those who hated the results
of ASCERT and said,
"I don't believe it,"
said, "See?
We told you you could
find a covariant
that could explain this."
So you decide.
We also looked at
composite outcomes
of death, MI, and stroke,
and essentially mimicked
the results.
I'm not gonna take you
through that one in detail.
Okay, so all observational
studies could have
treatment selection bias.
You can approach but not
fully resolve--
resolve this with
our statistic approaches.
We may not have all the data
that (unintelligible).
We did miss some data on GFR
and EF in particular,
and that's really important.
Remember I started
at the beginning,
observational studies,
having the covariant data
available,
is much more important
than it is in randomized trials.
ASCERT's limited
to patients over 65
that were linked to Medicare.
And it actually fits in,
ASCERT fits in very well with
other data in the literature.
Here's data from the New York
State Database.
Um, here we're looking at
3 Vessel Disease on the left,
2 Vessel Disease on right,
survival on the top
and frequency of death
or MI on the bottom,
and in all of these
you can see that
there's an advantage to CABG
compared to PCI.
Well, the same problem
could apply.
That is, that you may have
a covariant that explains it.
So if you have a bias
and the same bias applies
in another study,
it doesn't help you at all
to have another study.
So how about
randomized trial data?
Here's a meta-analysis
of 10 randomized trials
that (unintelligible)
published,
looking at death or MI
on the right
and mortality on the left.
You see that their trend for
mortality is (unintelligible)
with CABG (unintelligible)
is going the other way,
they're increasing
while they're going down,
and see that there's
a mortality advantage for CABG
compared to these
(unintelligible).
A trend of great interest,
there's an interaction.
So there's a difference
in the mortality advantage,
depending on age.
Over the age of 65,
which is the ASCERT population,
that's where there's actually
a mortality advantage to CABG.
So actually
the randomized trial data
and the observational data
are really--
are really quite consistent.
Now, it's interesting,
you know,
that's different than
Pam Douglas observed
in comparing drug-eluting
and bare-metal stents.
That'd be a whole subject
discussion
of why I think that is.
The reasons to think
that that's consistent.
So here's a, um,
a meta-analysis,
a more recent meta-analysis
comparing CABG and PCI,
again showing you
an advantage to CABG.
And then here is--
here is, uh,
the data on myocardial
infarction,
which also favors CABG.
So it goes the other way.
And ASCERT found exactly
the same thing as well.
And so here's a study
on the same subject.
I'm happy to take you
through it in detail,
but essentially
it's also consistent.
Frankly, this is a better study,
you know, but they--
they beat us by a week
to the journals.
Yeah, a week.
But the editor told me
that ours was a better study,
but "we're gonna take
the other one anyway."
What can you do?
Anyway.
So, survival was similar
in the two arms at one year.
Survival was better in the CABG
than PCI at four years.
Consistent across subgroups.
Consistent with clinical trial
and observational studies.
It allows us, I think,
in each (unintelligible)
to actually make
a causal inference
concerning all the data,
that there was a mortality
advantage in most patients
with multi vessel disease
long-term
with CABG compared to PCI.
ASCERT really showed us
how to do it.
But we can only see part of it.
I'm just gonna very briefly take
you through the economic data.
I suspect that Zugui
did this,
but this is more for overview
than for understanding
all of the methods,
because...
remember what
we're really interested
is not the clinical alone,
we're interested in the value,
after all,
where do we work?
Value Institute.
And so all--I think you all now
know Zugui's study
has been accepted
by theJournal
of the
American College
of Cardiology.
One of the hardest things
to do in doing
cost-effectiveness analysis--
I could make a case this is
actually the hardest part--
is determining
long-term survival,
determining survival
after the study period.
I don't think there's
a good way of doing it.
Zugui has made himself
an expert on the subject,
but, you know,
sort of pick your poison.
It's extremely difficult
to do.
And this is a problem
that doesn't get solved,
by the way,
with randomized trials.
Randomized trials,
if you have an observation
of five years,
you've got a lot of data.
You still gotta--
you gotta create
some kind of model somehow
to predict survival long-term
if you really want to do
cost-effectiveness
(unintelligible).
So we looked at the influence
of stroke,
myocardial on survival,
and of course we looked at
mortality within the trial,
and then we used
various approaches
from Framingham
and US Life Tables
to try and estimate
what survival would be
depending on age
and on gender
and whether people had
a previous event.
And from that, you can get
to life years lost or gained,
so if we look at it
at a lifetime,
the quality adjusted life years
gained with CABG
is (unintelligible).
Now, you hear people say,
"Boy, you know,
if you give that very expensive
CABG cancer drug,
you only gain four months
of life."
And I kind of listen
to that,
because most of the things
that we do,
you have a relatively
small amount of gain,
or if you give a cancer drug
and you gain two months of life,
and it costs $400,000
for a life year
that would be $2.4 million.
It starts to be
an awful lot of money.
So the small gain is
not the important part.
It's a couple things,
one is--
'cause some people gain a lot,
some people gain nothing,
some people gain a lot.
And also, is it cost effective?
So you really need to tie
the cost
and the clinical outcomes
together.
So went through a very
complicated process,
and Zugui really led
the effort on this.
Look at costing
in which we had to take
our hospitalizations,
apply them with DRG,
and then come up with costings
based on the prospective
payment system
from Medicare.
It's a very complicated
process.
And then on top of that,
the estimates
of physician cost as well,
another complicated process,
which is not depicted
really in detail.
And then you still only have the
cost within the trial period.
After that, you want to try
and estimate what's cost
going to be lifetime.
So, uh, in this study
we used 2004
as our base year data,
and per capita Medicare
expenditures $5,200.
Now you may say, "I don't
believe that at all.
These patients may veer
from that significantly."
So that's another problem
that you don't have
a perfect way
of estimating costs beyond
the observation period.
Another very important thing
for cost-effectiveness analysis
is to discount,
and the discount rate
most people would say
should be the same between
(unintelligible) clinical
 and economic outcomes.
Why do you wanna do that?
Why do you wanna discount?
And the idea is to bring
everything to present value,
so you're going to value things
in the future
less than you are now.
So if I was gonna
charge you five dollars,
would you rather pay me now
or pay me in one year?
Would you rather pay me
in one year or in ten years?
Rather pay me
in ten years.
You discount that five dollars.
That's the principle
of discounting.
(unintelligible).
Discount rate (unintelligible).
Would that be restricted
to the US?
Well, this is a US study,
so this is restricted
to the US.
(unintelligible)
Yeah, (unintelligible) is very,
very, very common.
You'll see debate about that.
The other thing is,
once you do
a sensitivity analysis
around your--
around your discount rate,
and then you can have
endless debate about whether
international studies,
you use the same discount rate
everywhere.
Some people maybe get that
in another country,
you should use just
(unintelligible) discount rate.
I mean, once you get to
international studies,
that's a whole 'nother
extremely complicated subject.
So then you do the basis
for doing
a cost-effectiveness analysis.
You know, so how does CABG
compare to PCI?
Remember, you have a mortality
advantage at 30 days
to PCI compared
to the CABG,
and CABG is still more
expensive.
So CABG is then dominated
by PCI at 30 days,
but what happens lifetime,
what lifetime,
you'd never make up
the initial cost
of the CABG,
it's just more expensive.
Uh, but what you gain
in quality-adjusted life years,
uh, and so you end up with
incremental cost-effectiveness
ratio of $50,000
of quality-adjusted life.
And here we plot it.
So this is the
cost-effectiveness plane,
with which you look at clinical
efficacy on the x-axis,
and (unintelligible)
on y-axis.
Everything is in the first
quadrant,
so (unintelligible)
it's better outcomes
at higher cost.
This can also be expressed
in a cost-effectiveness
acceptability curve,
so whatever you believe
the willingness-to-pay
threshold is.
If you believe it's $20,000,
the probability of CABG
being cost effective,
it's gonna be
essentially zero,
or if you believe it's $80,000,
there's a very
high probability that CABG
is going to (unintelligible)
cost effectiveness.
This is essentially
the same data
that's in the cost-effectiveness
plane,
it's just displayed
in a different way.
On top of this, we did
probabilistic sensitivity
analysis.
And Zugui has really become
an expert also
on a couple different
kinds of things.
One, the kind of costing
we do;
two, the estimation of life
years;
three, the, um,
probabilistic sensitivity
analysis;
and four, the application of
inverse probability weighting
to this kind of study.
And I believe that this paper
will be the first
in the literature
that essentially has
all of those things,
especially the inverse
probability weighting.
So if you wonder
if we could push
the (unintelligible) boundaries,
we can push
the (unintelligible) boundaries.
When you do sensitivity
analysis,
you have to pick
ranges for variables
that seem reasonable
based on the literature
and some (unintelligible)
the ranges that are used.
And then you end up
with something like this.
Now, you look at this and say,
"Oh my gosh,
what am I supposed to do
with this?"
Well, this is actually
quite interpretable.
The (unintelligible) in
the cost-effectiveness plane
with the differences in
quality-adjusted life years
on the x-axis,
the difference in cost
on the y-axis.
We see compared to the first one
I showed you,
where all the dots seemed
together,
this should splay out
a lot more.
And, then, these
flying saucers here,
give you your,
your lines of uncertainty.
So, the higher
your confidence interval,
going from 50 to 95,
you include more and more
patients.
This is really very believable.
This is believable
for the difference in efficacy,
the distribution,
the difference in efficacy,
and the difference in cost.
This is as fine
an analysis of this type
as you could possibly
hope for.
So, all right, so then
in conclusion,
in less than one year,
CABG was more costly,
less effective.
In the long run,
longer than one year,
CABG offers longer
quality-adjusted life years
but at higher cost
using thresholds
that are often used,
and remember such thresholds
are not scientific numbers,
they're numbers that may or may
not be agreed to by society,
but almost
any reasonable numbers,
CABG turns out to be
cost-effective
compared to PCI.
So then what would we say
overall?
Observational studies provide
real-world outcomes,
greater generalizability
than randomized trials.
We can get the best
of both worlds
by linking clinical databases
and administrative databases,
allowing us to look
at very large samples
from tremendous power.
Administrative databases
supplement clinical databases
well by allowing us
to look at resources
and, uh, cost data.
So we can do both clinical
and comparative effectiveness
studies.
Now we'll actually still have
the big problem
of observational studies
remains
of treatment selection bias,
but clearly, I think,
both randomized trials
and observational studies
have critical roles to play.
And what I can say about us is
we're capable of any of this.
These are as complicated
analyses as you're gonna find,
and we're absolutely capable
of doing any of them right here.
Thanks very much!
(applause)
I'd love to take some questions.
Jen.
In regards to
(unintelligible) matching,
I happened to match
all your equations
and to have some
(unintelligible) to match.
If you--yes.
Yes, that's why--
that's why many people believe
that you should use
inverse probability weighting
rather than matched.
You'll notice that we lost
actually the majority
of our patients
by using a matched approach.
But it's good to do it
both ways.
Do you think cost-effectiveness
analysis like this,
are you going to do
(unintelligible) as well?
Or just like, or not,
just like using
(unintelligible),
something like that?
Uh, Zugui, what do you
think about propensity--
propensity matching
on the cost?
I don't know what to make
out of that.
That's an interesting question.
Propensity matching
on the cost.
You really can--you remember
you're doing propensity matching
on--it's clinic--
it's covariance,
you're not matching--
you're not doing it on outcome.
Cost is really not it.
(unintelligible)
I can't hear you.
(unintelligible)
So you can,
and people do.
So meta-analyses,
you see meta-analyses
on observational studies
as well as meta-analyses
on randomized trials.
(unintelligible)
So--so--and people do do that.
People do do that,
but if you look at
the total number of patients
that have been randomized
in, uh, in the strategy
of PCI versus CABG ever,
it's only, like,
20,000 patients,
so you still run out of data,
and if you look at
more contemporary data,
looking at drug-eluting stents
versus CABG,
that's a tiny amount of data.
Five thousand.
(unintelligible)
Right.
So we do the same--
we did the same thing
on cost-effectiveness analysis
of using both IPW and matched.
So if you can do, you know,
the people that--
there are people that can only
the clinical propensity studies
analytically,
but if you can do
the cost effectiveness,
you gotta do the clinical first.
You can do
the cost effectiveness,
you really can do any of them.
Okay.
Thank you.
(applause)
