MICHELLE BONNER: And welcome
to the webinar, BJA Technical
Assistance Webinar
Series: Budget Advocacy
in the 21st Century.
This is the second of a three
part series on budget advocacy
and this session is entitled
Data Collection and Analysis
for Budget Advocacy.
Today we have with us two
presenters, Thomas Maher,
who is the executive director
of the North Carolina Indigent
Defense Services,
and Geoff Burkhardt,
who is a attorney and project
manager at the American Bar
Association.
And they have come to
talk about the topic
that we're highlighting today.
The topics are related to
the survey that many of you
participated back in
the summer of 2014.
The American
University and NLADA
sent out a survey on the 10
principles of a public defense
delivery system.
And the questions were
designed to ask about not only
your awareness of the
principle, but also
if you needed any
assistance in better
cheering to the principals.
And the response that we
received overwhelmingly
showed that offices wanted
more information about budget
advocacy, and they also
wanted more information
about not just the collection
and analysis of data,
but then how to use that
data and analysis to meet
their goals in particular
with regard to funding.
And so we designed this series
to address those concern.
The first part of
the three part series
address cultivating
relationships.
This one's on data
collection and analysis,
and the third one, which
will be on July 30th
addresses the issue
of how to then use
that data and analysis to
advocate for your increased
budget.
So today's topic
on data analysis
will address several
of the 10 principles
that you had questions about.
Of course the advocacy
for independence,
that's certainly a
goal that many of you
have, and data can certainly
help in that effort.
And then also in the
analysis of case loads
and workloads
principles two and five,
you'll be able to better discern
what those rules actually are
after collecting information.
Another principle that I
think is very interesting
to measure is the defense
counsel's ability, training,
and experience and
whether that matches
the complexity of the case.
You can really go
deep in data and do
an analysis of that to
determine how much more
professional development
your attorneys
may need as a result of that and
that also relates to principle
10, which goes to the
supervision of your staff
for quality and
efficiency of the service
that they provide
for your client.
And going back to that
topic that most of us
focused on with the
survey is on budget.
And so data collection certainly
helps with principle eight
in terms of parity
between defense counsel
and our prosecution with
respect to resources.
And so we hope to be able
to provide some information
that you will be able to use
to increase those resources.
So the three part
series again is
offered by American
University and NLADA
and these webinars
and the information
provided with the webinars
will be archived and found
on American
University's website.
It is www.righttocounselta.org.
And that information, you'll
see that further along
during this webinar.
So without further ado, let's
get into some of the questions
that we hope to answer
with this webinar.
What data should you collect
for purposes of budget advocacy
and how should you
collect this data
and how can you
creatively use the data
you're already collecting
in support of your budget
initiative?
We have with us two experts in
data collection and analysis.
One a defender chief,
another an attorney,
an expert in data collection
and analysis project.
And he'll be able to
break that down for you
during his presentation.
We'll start with
Tom Maher executive
director of North Carolina
Office of Indigent Defense
Services.
And I think that he has much
enlightening information
that will help
offices, especially
offices that are just starting
down this road of data
collection and analysis.
Tom, I will turn it over to you.
TOM MAHER: Thank you.
OK.
Hope everyone can hear me.
Welcome.
Thanks for spending
the afternoon.
As Michelle said,
I am Tom Maher.
I'm the director of the
North Carolina Office
of Indigent Defense Services.
We are a statewide
agency responsible
for public defenders, private
lawyers throughout the state
of North Carolina.
And today I'm going to talk
about basically three things.
I'm going to talk about the
history of North Carolina's
development using
data, I'm going
to talk some about
the lessons I've
learned from our use
of data, and then
I'm going to spend
some time talking
through a recent
example of using data
as a campaign for funding.
In this case, the campaign
was to restore funding
for the private
assigned council.
In North Carolina,
our history I guess
can be summarized that we
started before IDF in what
I view as a primordial soup.
We were created in 2000 and
began operations in 2001.
And now, as an
agency, we actually
have a fairly robust
ability to use data.
We have a research department.
We work on what we call the
systems evaluation project.
We produce reports,
reports, and reports.
And one of the things that I've
learned since I've been here--
and I've been here six years--
is the importance of data,
both in terms of management
but also advocacy.
So back before IDS, North
Carolina's Indigent Defense
System was run through
our administrative office
of the courts.
And there was no
centralized management.
There was no real oversight.
Basically judges
appointed lawyers
and they were allowed to
pay whatever they wanted to
on an hourly basis.
And everything was basically
counted by the number
of case files closed each year.
So there was no effort
to look at cases
in any systematic fashion.
There was no ability to figure
out where the money was really
going, let alone
what was happening
to the clients in the system.
In 2000, IDS was created.
And fortunately, the
creators recognized
that having reliable
statistical information
was an important part
indigent defense.
And so that's part of
our statutory mission.
We've actually got a
research department.
I put department in
quotes, because some of you
might know her as Margaret
Gressens, if you've
spoken to her, or John
King, her assistant.
So it's a two-person department.
But they are focused on
meeting our research needs.
And we've been able
to do a lot of things.
We have, for example,
begun working
on what we call the systems
evaluation project, which
is not just about how much
we spend or how many cases
we resolve, but what's actually
happening in the system.
And the notion, which you
can spend a whole day on,
is identifying the goals
and objectives of the system
and then identifying
the concrete indicators
that you can measure to see if
you're achieving those goals.
And we've been able to take
a fair amount of progress
on that.
So now, in IDS,
basically we routinely
use data to analyze issues.
For example, we did a report
on the outcomes and potentially
capital cases.
We were able to look at
the fact that in most cases
the difference between a
case to proceed capitally
and a case that does not
proceed capitally is the cost.
It costs about
three times as much.
And for the most part, the
outcomes are about the same.
And so there was a
powerful message there
about district
attorneys basically
driving costs by their decision
to charge a case capitally.
We have a few counties
that pay flat fees per case
in lower level cases.
They've been doing
that historically.
It allows us to look at
the outcomes in those cases
and see what we viewed as
a very concerning trend
that the outcomes
in those counties
were not as good for clients
as in counties that had lawyers
who were paid by the hour.
And of course, that
relates to the whole notion
of pushing lawyers to
resolve cases too quickly.
And thanks to work with the NLDA
and open sites on [INAUDIBLE],,
as I said, we've been working on
this systems evaluation project
and have been able to work with
several other jurisdictions
and starting to really develop
the indicators that measure
the outcomes in the cases.
So I've been fortunate
to work in an environment
where research is
kind of a given.
And it's really
been very useful.
I will say that my background
is not in research.
I was a trial lawyer.
I loved being in court.
For me, evidence was evidence
about a specific client.
It was not data
research evidence.
And so coming here,
it took me a while
to learn the importance of it.
And I've been here
long enough, I think,
that I understand it now.
And one of the things for us
and I think most people running
an office, there are really--
think about why data matters.
And I would say, not to
be first and foremost,
but of some
important is funders.
We all get funded somehow.
In our case, it's the state
legislature or the General
Assembly that decides
how much money we
get, and to a certain extent,
dictates how we spend it.
And their basic question
is, where's the money going.
What are we getting for
what we're spending?
But people who run their
office also matter.
If you're running an office
or supervising an office,
data matters to you
because you want
to know how the system is
operating in your office.
If you spend time with data, it
can become a useful management
tool.
We had a chief public defender
in one of our larger offices
who actually reduced
caseloads for people
handling certain felonies and
looked at it and tracked it.
We realized that
they were actually
able to dispose of more
cases because they spent less
time juggling too many cases.
So there was a useful example
of actually looking at the data
and recognizing that
reducing the current workload
might increase their
productivity while helping
the clients.
And then, of course, ultimately
the reason we do any of this
is for clients.
The main goal we have in the
systems evaluation project
is to identify
indicators that tell us
that we're achieving
for the clients
the objectives and goals
we want to achieve.
And data really helps you
understand that in something
other than an anecdotal way.
Now, as I said, I have not
spent my time as a researcher.
I came into this without
a research background.
And maybe that makes
it useful for me
to share some of
the lessons I've
learned as a non-research person
about data and the benefits
and pitfalls.
One of the things I've learned
is to not reinvent the wheel.
We are in an age where there's
a lot of research being done out
there.
I think in indigent
defense, there's
more being done than there would
have been 10 or 15 years ago.
There's a fair amount of
available research and data.
So before you take off
on a project on your own,
I would strongly
suggest you look
at what's already out there.
We're currently in a very
tough legislative session.
There are concerns about
how we handle capital cases.
We have a capital
defender's office
that covers cases
in various counties
with full time attorneys.
And there's concerns about that.
And quite frankly, even with
our research capability,
we can't do a cost effective
or quality analysis
for an entire office.
But going on to the web, I
was able to very quickly find
that in Texas they had a
regional public defender.
It operates very much
the way ours does.
They had done a study that
showed both quality and cost
effectiveness.
And while it wasn't perfectly
tuned to North Carolina,
it was a useful tool
in terms of educating
both ourselves about
how to do a study
and responding to at least
some legislative concerns
about a capital
defender's office.
And all it took was
a Google search.
There are, as some of
you know, reports out
there Texas once
again has some, I
think Minnesota may have some--
about caseloads.
One of the things I always hear
when I'm talking to legislators
is we need to be more efficient.
Well, I know the hours
our people spend on cases
is at least for private
assigned counsel.
And one of the things
in looking at that
and then looking at studies
like this is the reality
is we're too efficient.
If you look at some of the
research that's been done,
we've kind of gotten used
to very low hours on cases.
And the research is
very powerful in helping
us understand that maybe
we're spending too little time
and what we need
is more resources.
Once again, we didn't
have to do that study.
There are studies out
there about compensation
for public defenders
and district attorneys
that help show, for
us, where we fit
in the scheme but also the value
of paying people well enough
to stick around so
you don't lose people
that you've trained and invested
in and have developed skills.
So the first thing
I would say, even
if you've got a
research department, is
don't reinvent the wheel.
Discover existing data before
you start collecting data.
I think most--
I would say at this point, all
court systems collect data.
And you should educate
yourself about that data.
We have a system
that we call ACIS.
It's a combined infraction and
criminal information system
that the clerks put in.
And there is a huge
wealth of data in there
if you learn how to read it.
And the simple thing is
to get yourself educated
the way the clerks are.
Get their manual.
You can go to their
training, find out
how they're putting
data in, what it means,
and what's available.
And I think learning
what it means is crucial.
For example, in North
Carolina, case files
do not equate to a case.
We define cases by
all charges that
are disposed of at the same
time in front of the same judge.
But the court system
doesn't group them that way.
So when you look
at their numbers,
they're very different
from our numbers.
And you need to understand
why that's true.
If you're looking at, for
example, number of clients
who get active
sentences, for us,
all sentences that
involve any active time
are lumped together.
But a significant number of
those, as we've looked into,
are time served sentences.
And the time served plea
presents real issues,
but they're different
issues than somebody being
sentenced to go into prison.
So you have to
understand what's there.
And then I think
you can find it.
There's a lot of data
already available.
Analyzing data.
I spend more time than I ever
thought I would in law school
looking at spreadsheets
and analyzing data.
And one of the
things, of course,
you're looking for is trends.
Are caseloads going up?
Are active sentences
going up or down?
And it's very important to spend
some time understanding what
happens in trends because
they can be misleading.
A change in procedure
or even in staffing
can change disposition
tracks, particularly
in small jurisdictions.
North Carolina, we
have 100 counties.
Some are large like
Mecklenburg where Charlotte is.
Some are so small they hold
Superior Court four times
a year.
And you have to understand
the law of small numbers.
So for example, several
years ago, North Carolina
started allowing prosecutors
to appeal a suppression motion
when a district court, which
is our low level court,
grants a suppression order.
Well, that adds a
lot of time and delay
to a driving while
impaired case.
And it may explain when you see
trends of caseloads going up
or cases aging differently,
that without knowing
the change in the law
you wouldn't understand.
It's also true
that, particularly
in small jurisdictions, a change
in staffing can cause a trend.
I remember going to a
modestly sized county where
we were working with contracts.
And I'd noticed
that the caseloads
seemed to spike, a huge
increase in the disposition.
So I asked a local lawyer,
why has this gone up.
Well, it turned out the
answer was too fairly
elderly assistant DAs retired.
They hadn't been
working very hard.
And a whole bunch of cases that
had been kind of sitting around
suddenly got disposed.
So the trend was very
misleading if you
were to predict that in the
future there'd be more cases.
It really was the result
of a few individuals.
So look behind the trends.
Collecting new data.
I will say, we have, as I
said, a research department.
And they would love to
collect more and more data.
There's no such thing as
too much data for them.
But of course, if
you are a defender,
you have other things
to do than report data.
So when you're going to ask
people in the defense community
to collect new data, understand
that that's a time commitment.
That's using their resources.
And our mantra is collect
what you need which
may be less than you want.
I mean, think through
what you really
need them to do
because you're asking
them to take on a burden.
Educate them about the
importance of the data.
We, for example,
would love to have
our assistant public
defenders collect hours.
They don't like doing that.
But there is a
real value in being
able to show that the amount of
resources needed for the work.
And if you don't
record the hours
or you don't record
all your hours,
you end up undermining the claim
that you need more resources.
So you have to
educate them about why
taking the time to
record this data
will help them in
the long run and help
the system in the long run.
And then I personally think you
need to use what you collect.
Because if you're going to
ask people to take the time
and then you end up not
using it or not letting
them know how you
use it, you kind of
undermine your ability
going forward in the future.
When I first came
here, we had started
asking lawyers who are the
private lawyers when they
filled out a fee
application to put down
the date of their first
interview with a client
because we wanted to be able
to check and make sure people
were seeing their
in custody clients.
The truth is, we haven't used
that as well if we could have.
And I think it probably
makes it harder
for us to go back
to them and ask them
to provide additional date.
Presenting data, and I think
this is really important.
I cannot express to you how
important it is that simple is
better than complex even if
it does not paint the whole
picture.
I spend a significant
amount of time
trying to understand the nuances
of data, different case types,
why a trend may or may not be
a trend, things of that nature.
And so for example, I tend
to look at things like this,
very complex spreadsheets.
If you go to a
legislature with that,
they are not going to
get anything from that.
What you really need is a
very simple presentation.
Pie charts work very well.
I recently put together
a whole notebook
for a legislature
and the only thing
he was interested in was some
map that I had hand colored
showing counties we cover.
So think hard about what
the essential narrative
or story from your data and
how to present it in a way
that somebody who is not steeped
in all the nuances, who doesn't
have the time you
have, and who has
to juggle a lot of other issues,
is going to understand it.
And run it by other people.
Make sure that somebody
who hasn't put together
the data the way you have
understands the narrative
that you're telling.
Because otherwise all
of your hard work really
doesn't go very far.
I'm going to talk some
now about advocating using
data for increased PAC rates.
And for us, PAC means a
Private Assigned Counsel.
As I said, North Carolina
has a statewide system.
We have public defenders
that cover about a third
of the counties.
And even in those
counties, their conflicts
are covered by private lawyers.
And in the remaining,
all the work
is done by private lawyers.
And up until recently, most
of them were paid by the hour.
They'd be assigned a case and
they'd be paid an hourly rate.
And we were fortunate that
our hourly rates were fairly
healthy compared to most.
We were paying $75 an
hour in state court
for non-capital work
and $95 for capital.
Bad times came.
A much more conservative
legislature came.
And in 2011, we had
to reduce the rates
for our private
assigned counsel.
And so we've gone from a system
where about 2/3 of the work
was done by people who were
paid an hourly rate to cover
overhead and encourage them
to do the work to a system
where they were losing
as much as $20 an hour
from what they were being
paid, which is very difficult.
Because their overhead
hadn't gone down.
Their costs hadn't gone down.
That was coming straight
off their ability
to pay themselves.
And so we've been working on the
message to restore those rates.
And the message was that those
rates were not sustainable.
We knew we had a
legislature that
wasn't going to pay the
lawyers what they're worth
just out of a desire to be
good to lawyers, that they had
to understand that the system as
a whole required healthy rates,
and that if they didn't restore
them the system would not work.
And we also made it
clear that we weren't
asking for what we need
is a raise in the rates
but as restoring them.
And language can
be very important
as you advocate
on budget matters.
So we started off, as I said,
looking for our available data.
And in our situation, as
we get fee applications,
they're entered into
our financial system
and checks are cut.
So we know the names of
lawyers and the number
of fee applications
they're doing each year.
And we were able to look at
the rate at which lawyers
who had been doing work
stopped taking cases
because their names disappeared.
We would end up seeing that
they were no longer doing
fee applications.
And of course, that
happens every year.
People retire.
They take a different job.
They move.
Well, we could look at the rate
and all things being equal,
you would expect the
rate to stay the same.
And so we were able to analyze
going through all of our data
to see which attorneys
were no longer taking cases
and analyzed that the rate of
people dropping off the list
was, in fact, increasing.
And in fact, it was
increasing a fair amount.
And we went to our courts
commission recently--
and this is a slide
we presented--
we were able to show that
the rate of people leaving
had increased between
a third and 50%.
And so if we were losing 100
lawyers a year in the past,
now we were losing 130 or 150.
So this was kind of available,
quantitative data we had.
But by itself, it wasn't
going to be enough.
So we then moved
on to collecting
some additional data.
We did it through
Survey Monkey, which
you don't have to be
a brilliant researcher
to use Survey Monkey.
You don't have to
have a lot of money.
You do have to give
some thought to it.
And what we tried to do was to
collect both quantitative data
on the impact of the rate cuts
on attorneys and their clients,
because ultimately this
is about the clients.
But we also wanted
some qualitative data.
So on quantitative,
we were looking
at number of attorneys who
were losing support staff
or reducing their indigent
caseloads, on attorneys who
believed that
quality had declined
due to the loss of
experienced attorneys.
And then we wanted qualitative
data on the impact.
And by qualitative
data, I basically
mean stories, narratives.
Because hard data by itself, in
my view, is really important.
It's hard to get
anywhere without it.
But you also need a
story to go with it
to make people care about
it and put it in context.
So we did the survey.
We had about 400
of our attorneys
respond, which is a
very high response rate.
And we were able to produce a
report which we distributed.
And I've got it in just a
couple of pages here to show,
for example, we had
pages about the rate
at which people are
dropping off of the rosters
or planning to, losing support
staff so that they had less
resources to provide
representation,
the number of attorneys
who believe just
watching on a daily basis
that quality had gone down,
that was about 80%.
And we were able to
put this together.
And we also asked the
attorneys to basically give us
quotes or stories about
the impact on them,
the impact on other
attorneys in their district,
and the impact on their clients,
and the importance of what
we do for our clients.
And we were able to
get some good quotes.
In here, for
example, the attorney
talking about the clients were
facing financial distress,
homelessness, lack of
resources, and all the things
that our client base faces,
and that the importance
of effective representation.
Because ultimately
if your funders
don't care at least some level
about the quality of what
you about your clients, they're
not going to care as much about
whether the system works.
So we were able to put
this report together.
We have this report.
Anybody who goes to
our website can see it.
And we wrote it in a way
that it's got a lot of data,
a lot of charts, a
lot of pie charts,
but also quotes
and stories so it
would appeal to both
the quantitative side
and the qualitative side.
We were able to send
it to all legislators
and make a presentation to
our courts commission, which
is a commission that meets when
the General Assembly is not
in session, and had our
presentation on this.
They were issued a report with
a number of recommendations.
And fortunately for us,
one of the recommendations
is that they should
provide increased funding
for the rates paid by us
for our private counsel.
Now, we haven't gotten it yet.
We're still in the
legislative session.
You know, there's always a lot
of people asking for money.
But it certainly gave us
some traction with them.
And it was, I think,
the combination
of hard data and some actual
narrative to go with it.
So in closing, I'd
share, I guess,
a few more lessons that I've
learned through all this.
One is that it's a lot
easier to collect data
the participants see of value.
Our lawyers on this particular
survey saw the value.
They knew we were
using this to advocate
for resources they needed.
So we didn't have to
convince them to do that.
That will not always be true
if you don't spend some time
to educate people.
So you want enough people to
spend enough time providing you
data you need to kind of
sell them on why it matters.
The results you get are not
always going to be simple.
We looked at this.
And we saw that some people
said we're losing lawyers
and some said there's
too many lawyers.
We have these for
profit law schools.
There's many young
lawyers out there.
So you had to kind of
analyze it carefully.
In our case, I think
what we learned
is that we were losing
experienced attorneys
and being replaced by
far too many attorneys
without experience who weren't
getting enough cases to make
a living.
And that the story,
while more complex,
still was a story of a real
threat to indigent defense.
The other thing I learned,
and this can be painful,
is that when you've got
a mixed delivery system
and you are speaking for the
entire delivery system, when
you advocate for
one aspect, you're
going to bother
people in the others.
And we were doing
a very forceful job
of advocating for the
private assigned counsel.
We had tried to educate the
public defenders that this
was something that
benefited everybody,
that a public defender
system shouldn't be compared
to an overly cheap
private system,
that the clients needed
to benefit from this.
But even with that, we got
a fair amount of pushback
from some of the public
defenders about the fact
that we were advocating
for the private bar.
And that's just a
reality that people
tend to view the world through
the work they specifically do.
And you need to be aware
of that when you're
advocating for
part of the system
and not necessarily
for the whole system.
So we've had a lot of
complaints on that.
We tried dealing with that.
In closing, I will say that
data can be a powerful component
of efforts.
In fact, I spend a lot of
time in the legislature.
I spend a lot of time
answering questions.
I brainstorm ahead of
time, just like I'm
preparing an oral argument.
Because you are, in a sense.
And if you can't
answer data questions,
you end up undercutting
your effectiveness.
And so it's not just
affirmatively making the story.
It's being prepared to respond
when questions come up.
A lot of that data's out there.
You don't necessarily have
to start from scratch.
Take time to read it.
Quite often the data
is complex and you've
got to spend a
fair amount of time
to think about how to put it
into a simple, believable,
and non-misleading story.
And then ultimately, once
you've done all that,
you have to spend some
time kind of pairing it
with powerful stories
to really get anywhere
with an advocacy campaign.
The data is crucial.
But by itself, it doesn't
really tell the complete story.
Thank you.
MICHELLE BONNER: Thank
you very much, Tom.
And that old adage, never
a stat without a story,
never a story without
a stat certainly
does hold true in terms of
effective advocacy using data.
And I do have a
couple of questions
for you, if you don't mind.
You have a research department
in North Carolina, Indigent
Defense Services.
Did that research department
start with the office itself?
Or was that something that
you had to advocate for?
TOM MAHER: That started
with the office.
And I think we were fortunate
that when IDF was created,
the legislature and those
who were advocating for it
had a fair amount of knowledge
about what a good indigent
defense system should
look like and had support.
So that part of the mission was
to do this kind of research.
And Margaret, who
is our researcher,
was hired very early
on in the process.
So that was kind of a
given when IDS was created.
It was something that
was built into us.
We had a study commission.
The reports on our
website, it talked
about the reasons to have an
independent agency and data
was one of them.
Because quite frankly, the
administrative the office
of the courts had not really
used or collected data.
And the result was they
didn't manage indigent defense
in a way that satisfied anybody.
MICHELLE BONNER: So even though
you do have this research
department, you're
pretty savvy in research
as well although your
background didn't lend itself
to that initially.
So was it a very steep
learning curve for you
in terms of being
able to look at data
and analyze data to use
it towards your ends?
TOM MAHER: I'd like to
say no, it was all easy.
But it wasn't.
And I will say one
of the benefits
of having Margaret here is
that I spent time with her.
When she would produce a report
or a spreadsheet or analysis,
I would sit down and have her
walk me through it so that I
understood what they were
doing and what it meant,
what the question actually
was that was being answered
and what the answer was.
Once you've done that
a number of times,
you start to understand,
OK, what data really means
and what it doesn't mean.
I've also been
fortunate, Margaret
with NLADA's assistance,
has been working
with other jurisdictions.
And I've sat in
through meetings.
And one thing you learn is kind
of the differences in systems
and how important it
is to understand that.
We criminalize a huge amount
of conduct in North Carolina.
And it means a lot of
charges get dismissed.
So it looks like we're
winning a lot of cases.
Well, those cases are cases
that might not be brought,
say, in Texas or Tennessee.
So you end up
learning by doing it.
I find it very rewarding.
It's not just
interesting, but helps me
understand the system better.
And I think can do a better job.
But I have to make the
time to kind of let
myself get educated.
It doesn't just
come instinctively.
MICHELLE BONNER:
So for those of us
who are not as fortunate
to have a Margaret
presence in their office to
help walk them through it,
do you have any
suggestions for folks
to find some expert assistance
to help them with that
walk through a few Times
TOM MAHER: Yeah.
I think there's a number
of places you can turn.
If you've got local
colleges and universities,
there's often
research departments,
professors who are actually
interested in what we do,
who would probably spend
time with you educating you.
Sometimes you just find that
somebody in your office, maybe
a support staff member who
knows the case management
system well, who can
help you design queries.
We do our capital and appellate
assignments and fees in house.
We have an access database.
And I've had people
help me build queries
so I can, on an ongoing
basis, get data.
Don't overlook the
support staff that
put the data into your system.
They may actually understand
it a whole lot better
than the lawyers
who are reading it.
MICHELLE BONNER:
One more question.
You talked about the existing
data that people already have.
And my question to you
is, how difficult or easy
was it for you to get that
data from other stakeholders,
like courts or prosecutors
or law enforcement.
TOM MAHER: It ranges.
For example, for
us, and I imagine
this is true in a lot of
states, the court systems
generate reports all the
time that you can just
go on the website and find.
So if I wanted to see were case
dispositions slowing or going
faster, caseloads
going up or down,
I just went to the
existing reports.
I didn't have to ask.
For the large data that we
often work with from our court
system, first of all, you
need somebody like Margaret
to do it.
And then you do have
to kind of negotiate
with whoever has
the data the format
you're going to get it in.
But it's all public data.
In general, you're
going to get it.
I think the one time
we ran into a problem
is we were trying to
get data from our DMV
to look at driving while
license revoked cases
and they wanted money.
And one thing we don't
have is much money.
So that was the one time we
haven't been able to get it.
But you have to understand
how much you're getting
and how you're going to use it.
And we're lucky that we actually
have like a data warehouse
where we can do that.
MICHELLE BONNER: Great.
Wonderful.
And you know how much I love the
private assigned counsel survey
report.
And I strongly suggest
to those who are tuned
in to check out that report.
I believe we can probably set up
a link on the right to counsel
[? TA ?] website as well.
It's a great example of
using relatively low cost
or free services
of Survey Monkey
but turning that
data and information
into a very professional
and very persuasive piece.
So thank you very much for that.
Now we're going to switch to
Geoffrey Burkhart, the attorney
and project director at the
American Bar Association.
Geoff has been involved in
some really exciting workload
projects.
And a big component, if not
the component of the workload
projects, of course, is
collecting and analyzing
data to produce those reports.
So Geoffrey Burkhart, I will
turn it over to you now.
GEOFFREY BURKHART: Great.
Thanks, Michelle.
So I want to pick
up with something
that Tom mentioned that in a lot
of cases we're too efficient.
And that's a problem in a lot
of public defender offices
where people have prized
efficiency over effectiveness
for quite a long time.
While you have the
number of attorneys
in a lot of officers
staying pretty steady,
the cases have gone up and
up and up over the years.
So the effect of
that a lot of times
is systemic ineffectiveness.
You can't possibly represent
hundreds and hundreds
of clients effectively.
So that's where the
workload studies come in.
So in the United States,
there are more than 15,000
public defenders.
So that number is
based on a study
by the Bureau of Justice
Statistics back in 2007.
So the data's kind of old at
this point, speaking of data.
We should have a new study
fairly soon and a better
picture-- actually
thanks in part to NLADA--
of what that looks like.
But that's roughly the number
of public defenders [INAUDIBLE]
in the United States.
Now those defenders handle over
5.5 million cases each year.
So again, that's based on the
Bureau of Justice Statistics
numbers.
So quite a number of cases.
And this is some
quick and dirty math,
that basically averages out to
about 360 cases per attorney.
Again, it's very quick
and dirty math and that's
a little bit older
data at this point.
And we have a little
bit of anecdotal data,
though, that in a
lot of jurisdictions
that number is much--
oh, let me go back.
In a lot of jurisdictions,
that number is much higher.
If you look at Rhode
Island-- and this
is based on the
Norm Lefstein's book
Securing Reasonable
Caseloads-- in Rhode Island
the average attorney
was handling
a mix of felony and misdemeanor
cases, 1,756 cases a year.
And that's not a typo,
1,756 cases a year.
When you think
about that number,
try to imagine what kind
of client communication
you could do in
that amount of time.
What of legal research could
you do in that amount of time
with that many cases?
What kind of factual
investigation could you do?
What kind of sentencing
investigation
could you possibly do?
Really, you get
to a system where
you get a lot of
[INAUDIBLE] just by default.
So it doesn't have to do
with any individual attorney.
It doesn't matter how good
of an attorney you are.
When you have that
many cases, when
you're saddled with
that kind of caseload,
you just can't be effective.
So we know the causes of
these types of caseloads,
these enormous
caseloads that are
so common in public
defender systems.
One is inadequate funding.
And I don't think that
needs much explanation.
Chronically, and this has
been the case since not too
long after Gideon,
public defender offices
have been inadequately funded.
Lack of independence
of the other, and this
is the first of
the 10 principles.
A lot of times you have judicial
or political interference
of some sort that kind of binds
the hands of public defenders.
That ties your hands in terms
of regulating your caseload,
among other things.
One of the other things
that affects your caseload
is the lack of control.
So you don't control intake
as public defender, usually.
The prosecution,
on the other hand,
they can kind of turn
that valve on or off,
take more cases or fewer cases.
And they can actually not just
determine the number of cases,
but also how big that case is.
They control the way that
they actually charge.
So what is the
solution to all this?
Well, the best solution
we've found so far
is workload studies.
And that's what I'm going
to talk about today are
these workload studies.
[TRAIN WHISTLE] Oh, sorry.
I hear a train.
I hope you can
hear me over that.
The first one, and
workload studies
have been going on
for quite a while.
But the ones we
started to do, they're
a little bit more modern.
They're slightly
different than ones
you may have heard of
that were done previously
by, say, The Spangenberg
Group or the National
Center for State Courts.
So today I want to try to
demystify workload studies just
a bit.
Now the workload studies that
we're doing have two parts.
The first is a time study.
And really, the
time study is just
a way to figure out what is,
what's going on on the ground.
During the time study,
we have public defenders
track their time in
fractions of an hour.
And it's somewhat
comparable to what
attorneys have been
doing for decades
in the private practice.
So again, the time study,
when you think of time study,
think of just what
is, what's actually
happening on the ground.
The second part is
the Delphi process.
And that is what should be.
In other words, what are
our workload standard.
How much time should attorneys
be putting into cases in order
to be reasonably effective?
So if you have attorneys putting
in very little time because
of the very large
caseloads, that
will show up in the time study.
And if you have many
more hours than should
be put in order to be
reasonably effective,
that's what you're going to
find in the Delphi process.
You really need both.
You need what's
going on right now
and what should be in
order to show that gap.
So I'm going to start
with the time study.
The time study,
we have attorneys
tracked along two axes.
One is the case type.
And case type is exactly
what it sounds like.
It could be homicide.
It could be sex offenses.
It could be misdemeanors.
Depending on the
states, you know, here
in Illinois we have class
1, 2, 3, 4, and class x.
It can be divided along those.
Other states have A,
B, C, D. Other states
still, like
Louisiana, I know they
don't have any kind of
strict classification
system in that sense.
But you can divide it
according to roughly
in terms of sentencing ranges.
But in case, what we
want to do with the case
type is figure out some kind
of proxy for difficulty.
We know that some
felonies may be--
generally felonies
are going to be more
difficult than misdemeanors.
We know that there may be an
individual misdemeanor that
takes a considerable
amount of time
and maybe more so
than in some felonies.
But generally, we
want to have some sort
of proxy for difficulty.
And these tend to be
pretty good stand ins.
Generally your homicide is
going to be a little bit more
difficult, a little
bit more time consuming
than your sex offense.
Generally a sex offense will
be a little bit more time
consuming than a low
level felony, and so on.
So those are the case types.
The case tasks are also, I
think, fairly common sense.
What are you doing?
That's basically it.
Is it discovery?
Are you spending time in court?
Is it client communication?
Is it legal research?
So we want to capture data
along both of those axes.
Are you in a homicide case?
And if you're in not homicide
case, what are you doing?
Are you doing something
with discovery right now?
So we want to track that.
If you're in a misdemeanor
case and you're
doing time in communication,
we want to track that.
Breaking those down
a little bit more,
what we did in the
Missouri study--
which we're kind of replicating
right now with some refinements
both in Rhode Island
and Tennessee--
in Missouri we had
eight case types.
And again, it's really just
kind of proxies for difficulty.
So we have murder cases,
sex felonies, AB felonies,
CD felonies,
misdemeanors, and then
after that, juvenile and
then probation violation
and an appeals and
post-conviction.
This is going to
vary a little bit.
And again we're working in
Tennessee and Rhode Island
right now.
And they do vary a little bit.
I believe in Tennessee we're
looking at roughly 12 case
types.
Same thing for Rhode Island.
So a little bit more than
what we had in Missouri.
But in any case,
generally you're
going to have something that
looks something like this.
It will be fairly similar.
You're going to
have probably murder
cases and a separate case site.
Sex offenses, we
divided out of Missouri
because they tend to be kind
of a strange animal in terms
of the amount of
research that's involved
and forensics and
things like that.
And then after that,
we're going to break it
down further to either
best 1 and 2, best 3 and 4.
Or here in Missouri, it was
A, B, C, D. And eventually
once you get past
misdemeanors, you
have kind of an
assortment of other things
that will pop up
in your caseload.
So juvenile cases take a
slightly different amount
of time than other cases,
probation violation, appeals
and post-conviction petitions.
Also in other states, if you
have abuse and neglect cases.
There's something that
are slightly different,
that you wouldn't handle in
your normal criminal kind
of breakdown.
So Missouri, we stuck
with eight case types.
Now, could you do more?
Yeah.
Yeah.
You could really fine tune
this and break it down to.
you know, here in
Illinois, we could probably
do 100 different case types.
The problem is at the
end of the day you
wouldn't have a lot of data.
I mean, how many times does
fishing without a license
really come up?
You might get one case
that comes up every 20
years or something like that.
You just wouldn't get
a lot of data on it.
So at the end of the day we want
a lot of data points on this.
Moving over to the case
tasks, they broke it down to--
we stuck with 14 case
tasks in Missouri.
And again, this
will vary slightly
depending on what you're
doing in your office.
So in Rhode Island
right now, we're
looking at about 12 case tasks--
in Tennessee, I
believe it was 13--
that will be somewhere
in this realm.
And again, this is
just trying to show
exactly what you're doing.
Now, could you break
this down even more?
Absolutely.
You know you can
really fine tune
this and have 1,000
different case
tasks if you really wanted to.
Again, we want
enough data though
at the end of the day
to show what's going on.
So there's a little
bit of a balance
that's taking place here.
Now, there are three things
that time-keeping requires.
People, so you need attorneys
tracking their time.
You need clerical
staff especially
if you have attorneys who are
not so technologically savvy.
You need clerical
staff and IT folks
who can work with them a little
bit to kind of get up to speed.
Hardware, and again this is just
computers, tablets, and phones,
and then software.
And I want to focus on
software a little bit more
because I think
people and hardware is
fairly self-explanatory.
The software, a lot
of people aren't
familiar with what
kind of software
you really need in
order to track time.
So what sort of
software should you use.
Well, there are two
different ways to go here.
A lot of folks will use
commercial software systems.
There are two that
tend to be pretty
popular among public defenders.
A lot of folks will use
Defender Data, which
is created by Justice Works.
Defender Data is used
in parts of Texas.
It's used throughout
the federal system.
So all federal defenders
are using that right now.
It's used in parts of
California and Utah.
So it is fairly common.
One of their
competitors is Justware.
They also have time
training software.
So if you're fortunate
enough to already
have one of these
systems, you likely
have some sort of time tracking
already built into your system.
You want to talk to the folks
with that software provider
and ask them about not
just turning it on,
but how do you tweak it.
They can give you something
that's sort of prefab.
But you might want to be
able to tweak it in order
to do a workload study.
The other thing you
can go with is kind
of a more homegrown system.
And we're used to
working with these.
With our Missouri project,
Missouri had their own system
that they built in house.
And they had a very good
IT staff to help build it.
Same thing with Rhode Island.
There's a guy named Dan Bright
out of the Rhode Island office
who built their system.
It's very good.
And same thing with Nashville.
They have some
folks in Nashville
who built their own system.
Knoxville, again, actually we're
working in Knoxville as well.
They are using Defender Data.
So we do have states that
have sort of this mix.
The point at end of
the day is that it's
possible to do time tracking
with either your own system
that you build in house or
with a commercial system.
Next thing you want
to consider, cost.
And that's a huge concern.
It's definitely worth
shopping around.
And the cost will have to
balance with some other things,
like mobility, adaptability,
compatibility, ease of use.
You know a lot of these
homegrown systems,
if you have a great
IT person, you
can certainly build some
decent time tracking software.
If you're given the
time and the resources
and a very good IT person,
you can do it fairly easily.
Now will it be it as easy to use
as something like Defender Data
where they have
full time staff who
have spent a lot of time really
kind of fine tuning this?
And they work with
public defender offices
across the country.
Well, maybe, maybe not.
And that's something you
need to really research.
How good is your IT staff?
Are they able to do this?
Talk to the folks
at Defender Data.
Talk to the folks at Justware.
Talk to your IT staff.
See what this would really
take to get something
that one, is cost effective
but two, is easy to use
and possibly mobile.
A lot of public
defenders like to be
able to actually
track their time
on smartphones or on tablets.
And that's something
that you have
definitely with
Defender Data I believe
Justware has it as well.
Increasingly, IT
people in-house are
able to do that,
although it tends
to be a little bit trickier.
But something to keep in mind.
So the next few slides I'm going
to go through fairly quickly.
I don't think you're going to
be able to see too much of this.
I just want to give you a quick
idea of what some of these time
tracking softwares look like.
So this is Missouri's.
They built theirs in-house.
And again, basically,
you have drop down menus
that you can pull and pull down
and click on create a time log.
And again, I don't want to get
into too much detail on these.
Your time tracking
software is going
to vary based on your commercial
system or your in-house system.
I just want to give you
some quick screenshots
to give you an idea of
what they can look like.
So this is an example of
Defender Data's fairly clean
screen.
Again, if you have
a good IT person
and with a decent amount
of time and resources,
you may be able to replicate
something like this.
If you already have Defender
Data, talk to the folks there.
You may be able to
just turn this on.
And then talk to them
about tweaking it
a little bit for a time study.
And then this is Nashville's.
This is another
homegrown system.
So how long must defenders
keep track of time?
Well, short answer
is permanently,
for the rest of your career.
I mean, this is something that--
public defenders have
been an exception
to this for years and years.
And I know it's difficult
in a lot of offices.
And hopefully during the
question and answer session,
we can address a
little bit about ways
to kind of overcome
resistance to this.
Nobody wants the add, again,
it is one more thing to do.
And I think Tom alluded
to this a little bit.
But public defenders, they're
already doing a ton of work.
They're already overworked.
That's why we're
doing these studies.
You're asking them to
do yet one more thing.
And that's a difficult sell.
And that goes back
to yet another thing
that Tom said, the importance of
educating your public defenders
about data and how you're going
to use it, why it's important.
But again, the short answer,
you have to do this permanently.
And one, this is
going to happen.
Legislatures
increasingly are starting
to require public defenders
and other assigned counsel
to do this anyway.
It's much better to get
out in front of this thing.
Instead of having this
dictated from above,
as a public defender,
get in front of it,
start tracking it on your own.
Then you can kind
of set your terms.
You can set how you
are tracking this.
We're in the 21st century.
It's going to happen
at some point.
But the other reason
is that this is
a very big effect on funders.
Missouri, we did
a workload study.
When we went to the legislature,
for the state legislature,
and said, our public
defenders statewide
are tracking their time
in fractious of an hour
and they've committed to
doing that for the rest
of their professional
careers, the first thing
that the legislators said was,
prosecutors aren't doing that,
right?
No, that's right.
Prosecutors aren't doing that.
One of your goals
as a public defender
is to be one of the most
responsible, transparent
agencies in your
state, in your county.
And this is one of
the ways to do that.
Now for purposes of the workload
studies that we've been doing,
we then take six
months of that data
in order to get a decent sense
of how public defenders are
spending their time.
So the short answer is, again,
you're tracking it permanently.
But we take six months of data
for purposes of our studies.
Now before we start
that six month
clock running, however, lots of
times we give a little bit of--
three or four months,
sometimes a little bit longer,
in order for public
defenders to get up to speed.
A lot of times this is very new.
I think the transition
with the younger attorneys
tends to be a little bit easier
because they are a little bit
more tech savvy.
But that's not always the case.
Sometimes you have
some older attorneys
who definitely are as well.
But in any case,
a lot of times you
have some-- there's a
little bit of a learning
curve in terms of tracking
time, just using the software.
And also, how do I
fit this into my day?
My already very busy day?
And making sure that you have
public defender compliance.
So we have a little
bit of a learning curve
before you hit that
six month period.
So I'm going to move now
to the Delphi process.
Basically, the
Delphi process, we're
leaving the world of
what is, the time study,
behind right now.
And we're moving onto the
world of what should be,
the Delphi process and
workload standards.
So this is completely
separate from your attorneys
tracking time.
Basically, we want
at the end of the day
to figure out how much time
should your attorneys spend
on cases on average in order
to be reasonably effective.
Well, and we can
talk about it maybe
in the question and answers.
But we've attempted this as
public defenders for decades.
In 1973, the [INAUDIBLE]
standards came about.
There were a lot of
problems with those.
And maybe we'll talk
about that a little bit.
But we've been trying
to get a good way to get
a handle on what should be.
What hours should attorneys
be putting into cases?
So the Delphi process seems
to be the best so far.
And this is something that we
stood up for funding challenges
in Missouri.
And we're going to be using
in Rhode Island and Tennessee.
But this is a way to
show how much time they
should be putting in.
And this is through
a multi-round survey.
It is a technique
that was developed
by the Rand Corporation
back in the '50s and '60s.
It's been used by
many businesses.
It's been used by the military.
So it's a tried and true
technique to basically distill
expert knowledge.
So if you have a lot
of experts on a topic,
how do you take their
collective knowledge
and really distill that down
into some sort of number?
So that's essentially what we're
doing with the Delphi process.
This is important.
We're using the same
case types and case tasks
that we did with the time study.
So you're tracking exactly
the same sort of information.
We need some sort of apples
to apples comparison.
If you broke down your case
types like we did in Missouri,
we had eight case types
for the time study,
you're going to use those
exact same ones with the Delphi
process.
And same thing with
the case tasks.
And essentially, you're
asking your expert panel.
And again, your expert panel,
we're taking state experts
and this is a mix of public
defenders and private defense
attorneys, generally
people who've
been practicing for 25 or
30 plus years in this area.
And we're asking them some
variation on the same question.
And that question is, how much
time on average is reasonably
required to perform
this case task--
so for instance, in person
client communication--
for this case type--
for instance, homicide-- with
reasonable effectiveness.
So you're asking some variation
on all those questions.
So in Missouri, again,
we had eight case types
and we had 14 case tasks.
If you multiply that
out, that's 114--
if you think about it
like a grid like this--
case types and case tasks.
You get 114 data points then.
It's a lot of questions.
You're asking a lot
of your expert panel.
Now, I said-- let me
back up real quick--
basically we do this and
it's an anonymous survey.
So when they're doing
this, they don't
know what the other experts are
answering for the first round.
It's just a survey.
And Tom had mentioned
using Survey Monkey.
You can use something like
that, basically any way
to get the survey
out in front of them
to have them answer
these questions.
We then do it, using the
Delphi process, a second round.
In the second
round, you're asking
the exact same questions.
The only difference
is that now you're
giving them information
that helps aid them in terms
of answering that question.
So you're asking the
exact same question,
but you're also telling them,
when you're answering this,
here was the range.
Here was the range of
answers and maybe you're
here in the middle.
Maybe you're way
off to the side.
But here's what the range was.
And here's what the median was.
And here's where you sit.
Now, do you want to
keep with your answer?
Do you want to
stick with it or do
you want to move a little bit
based on what this range is
from the other experts?
So we give them a little
bit of information.
The final round that we
do then is in person.
And you actually get all
these experts in a room.
And we did this in Missouri.
And we'll be doing this in
Rhode Island and Tennessee.
And they really duke it out.
You get a bunch of
public defenders
who've been doing
this 20, 30 years
and have very strong opinions
about how much time you
should be spending on a case,
they really duke it out.
But at the end of the
day, we get answers.
We get these standards.
And this is based not
only on their experiences,
public defenders and
private attorneys,
but also we're giving them
standards to work with.
So we're giving them ABA
Defense Function standards,
providing defense services,
the 10 principles.
So they have something to go on.
So based on these principles
and based on your experience,
how much time should be spent?
Now at the end of
the day, this is what
we ended up with in Missouri.
So for the Delphi
process-- now, this
is how many hours should
be put in in order
to deliver reasonably
effective assistance of counsel
according to prevailing
professional norms.
And this is just an average.
So the Delphi panel in
Missouri said, on average, you
should be spending 106.6 hours
on your murder and homicide
case.
You should be spending 47.6
hours on your A/B felony cases,
25 hours on your
C/D felony cases.
Now again, this is
everything from they're
covering all the
hours that it takes
to do research, all the
hours for investigation,
client communication,
researching sentencing, and so
forth.
So I'm going to come back to
these numbers in a minute.
But remember, the workload
studies have two portions.
You have the time study,
which is a measure of what is.
And you have your
Delphi process which
is a measure of what should
be, your workload standards.
Now here's what happens when
we compared those in Missouri.
Again, this is the
number of hours,
if you look at the time study,
that attorneys were putting in.
If you look at the Delphi,
this is the number of hours
that experts through
this rigorous process
said that they should be putting
in to be reasonably effective.
Now, when you look at
the top, there's a gap.
You see in murder and homicide,
there's a definite gap there.
But as you go down, the gap
gets much larger and larger
as a percentage.
So if you go down to
misdemeanors, on average,
these attorneys were
spending 2.3 hours
on a misdemeanor case.
And again, you think about that,
what kind of an investigation
can you do in 2.3 hours?
What kind of legal
research are you doing?
How much time are you
meeting with the client?
I mean, court time
and all this--
it's unbelievable
that somebody really
could be effective in all
these cases in 2.3 hours.
The Delphi panel said that it
should be much closer to 12.
Now that's a huge leap.
We're talking roughly five
times the amount of time
that was being spent.
Now again, that gap isn't
as big when you go up
to that murder and homicide.
And intuitively,
this makes sense.
You have offices that
are kind of triaging.
If you have just a
few murder cases,
you're going to push your--
especially if you have death
penalty cases--
you're going to push your
resources to those death
penalty cases.
You're going to push your
resources to those murder
cases.
Because those folks, the
penalties are huge there.
We're talking in some cases
about somebody's life.
So of course, of
course then the gap's
not going to be as
big in those areas.
Because as a public
defender, it would
be irresponsible to not really
push for resources to really
effectively represent somebody
charged with a homicide.
Now unfortunately we see this.
The NACDL had a great
study a few years back,
Minor Crimes, Massive Waste,
that talked about misdemeanors.
Now you may have just a few
people in this homicides,
and of course you're going
to push your resources there.
But down at the bottom
of this pyramid,
you have a lot of people
charged with these misdemeanors.
Many times, more charged
with misdemeanors
than you do with
these homicides.
And a lot of times those
are the ones being ignored.
And that's exactly
what we see here.
So I'm going to go
ahead and wrap it up.
So that's the gist of
the workload studies.
And again, this is kind
of a 30,000 foot view.
And if anybody has any
questions about this,
don't ever hesitate to
email me, give me a call.
I know my number was on the
front of this presentation.
I'm happy to talk more about
it and the studies we're doing.
And we're happy to work in
additional jurisdictions.
We have a few more coming up.
And I'll turn it back to
Michelle for questions.
MICHELLE BONNER: Thank
you very much, Geoff.
And I would like to
remind all the attendees
as well that if you would
like to ask any questions,
you can type some questions
at the bottom of the screen.
And we will get to
them, time permitting.
And if there are
additional questions,
you can always reach out.
You see there on the
screen Preeti Menon, who
is the senior policy
associate at Justice Programs
Office of American University.
You can always email
additional questions to her.
And she will make sure that
the panelist receive them.
And they'll be able to respond.
So Geoff, I do have a
few questions for you.
Thank you so very much
for that presentation.
I think any opportunity to talk
about the workload studies,
which I think is very important
and groundbreaking work
in a lot of ways in data
collection and analysis
for indigent defense.
I think it's
critically important.
So thank you very much for that.
I do have some questions
about timekeeping to start,
because these studies do
involve those two parts.
So have you heard about in
terms of defenders maybe--
not all the offenders.
There may be some
defenders not wanting
to track time or
do time keeping.
GEOFFREY BURKHART: Sure.
And this is common.
And it's understandable.
You're asking already
overworked defenders
to do yet one more thing.
And one more thing that
I think a lot of them
see as kind of
putting a microscope
on them and their work.
And also, a lot of times
learning new software, which
nobody wants to do.
So this is something
that we face.
And it does tend to
be a little bit easier
with newer defenders.
You bring somebody
new into the office,
and frankly they
don't know any better.
This is their reality.
So if they come into
the office and you say,
welcome to the public defender
office, we keep track of time
and how we do it, it's just
one more training that they do.
So with the younger attorneys,
generally, or newer attorneys,
we tend not to have
as much resistance,
although it happens there too.
A lot of the
resistance tends to be
attorneys who have been there
for five, 10, 15, 20 years.
And again, that's not always
the case either though.
There is some variation.
At the end of the
day, there are kind
of two things that I think
are important to keep in mind.
One is going back to what
Thomas said about educating them
about the importance of data.
Why become a public defender?
Did you become a
public defender to be
kind of a speed bump along
the way to a guilty plea?
Well, nobody does that.
Nobody wants to do that.
Did you become a public defender
to be a part of assembly line
justice?
No, absolutely not.
Nobody signed up for that.
But in too many jurisdictions,
we bend toward that.
And I think seeing that
larger picture that a workload
study like this and tracking
your time can really help
tends to persuade.
And in addition to that,
though, at the end of the day,
if you make it part of the
job, it's part of the job
and you've got to do it.
And it's tough.
And there's definitely
a transition period.
And that's why we build in a
series of months for people
to adjust to the software
and complying to it.
But if somebody just didn't file
motions on a case, how would
you address that?
So there's two pieces to this.
One is doing that sell
and convincing them
of the importance of data.
And it is important.
And it can make a huge
difference in your career
and in your office.
But two is ultimately,
as a public defender,
as a chief public defender
or as a supervisor,
you have to ensure compliance.
You have to work with defenders
to educate them, but also
say, look, this is part of your
job just like filing motions,
just like visiting your client.
You've got to do it.
MICHELLE BONNER:
So, following up
on that question really quick.
You talked about the
resistance and software.
And so did you or
your teams look
at other ways of
collecting time,
say through existing
programs they already
had like payroll system?
Because some of those actually
allow you to track time.
There are some
cost efficient apps
that help people
track time that way.
Did you can look at alternatives
to more customized software
for offices?
GEOFFREY BURKHART: Yeah.
We've looked into it.
And we're kind of constrained
by a couple of things.
The main thing is
that whatever it is,
it has to be able to track time
according to both case type
and case task.
If it can do those two
and it can track time
in fractions of an
hour, and usually we
recommend about 10 or 15
minutes for the minimum time.
But offices have done six
minutes, seven minutes.
As long as they
can do those things
and you can extract the data
once you have all these folks
track time, that works.
That should be fine.
MICHELLE BONNER: Great.
We have a question from
one of the attendees,
and I'll read that to you,
from Catherine Johnson.
The question is, a criticism
to the Delphi process
is that when it's done by
public defense attorneys,
it's self-serving
and subjective.
Any advice on whether to include
other partners-- prosecutors,
judges, private bar--
to insert more objectivity
into the process?
GEOFFREY BURKHART: Yeah.
That's a great question.
And that's something
that we're definitely
trying to refine and do more
of, build in more neutrality
and objectivity.
Again, our goal at
the end of the day
is to be the clearest,
most transparent,
most responsible agency
in your jurisdiction.
So how do you do that then?
And we don't want allegations
of just kind of cherry picking
defenders.
Frankly, I think when you
look at most jurisdictions,
if you practice in an area,
in a state or a county,
for a significant amount of
time and you ask the judges,
you ask the prosecutors, who are
the top defense attorneys here,
generally people
are going to agree.
I think generally, folks
having a fairly good sense
of who those true experts are.
But having said that,
one of the things
that we're working
on now in both Rhode
Island and Tennessee,
is they have kind of--
you know, we're toying with
the terminology of this--
kind of like super luminaries
or kind of a selection panel
where basically you
have people who--
they have to have some sort
of experience with defense
because defense is definitely
a different activity
from prosecution.
I know there's a lot more
similarity in some civil areas,
but in criminal areas,
I think the duties
are fairly different.
So you have to have somebody
both from the selection
panel and also the
expert panel who
does have some familiarity with
what goes into public defense.
But if you can get, say, three
or five folks on a selection
panel who are people who
have done private defense,
a lot of judges
served as prosecutors
and then later on did
some private practice
as criminal defense attorneys.
And now they're on the bench.
And if they are a very
respectable judge, somebody
that you can work with and
somebody who you can definitely
rely on, that's a great person
to do include in that selection
panel.
So then that panel can
select anywhere from-- we
aim for roughly 40 people
for a state in order
to, on that expert panel,
usually roughly half
and half public defenders and
private defense attorneys.
So again, that kind
of selection panel,
it adds a little bit of an
extra layer of objectivity.
Another thing we do
is we always have
an outside agency working with
us on these Delphi processes.
So it's not just the public
defender saying, here's
what those numbers are.
In Missouri, we had
an accounting firm
that we worked with, RubinBrown,
which they're fantastic.
We're working with
[? Crosslin ?] and Associates
in Tennessee, another
accounting firm, BlumShapiro
in Rhode Island.
But also, Tom, I think you
had mentioned and Michelle
you'd mentioned working
with academia too.
The Texas workload study, they
worked with Dottie Carmichael
at Texas A&M. So
those are a couple
of ways to build
in some objectivity
to the Delphi process.
And we're working on additional
ways to refine it though, too.
MICHELLE BONNER: I think
with the Delphi process,
and the time keeping, I think
on both ends, we also may have,
on the one hand with
the time keeping,
people may be overestimating
how much time they spend
on certain cases for
one, subconsciously
for fear or thinking,
wow, I should
be spending more time on this.
And maybe I do.
But in reality, they
actually spend less time.
And then with the experts, on
the other hand, because they
are experts, they may
report the amount of time
that they think should be
spent on a particular case.
But in actuality, with
somebody with less experience,
it could be more time.
So I see the numbers that
you put up with regard
to the comparison with the
time study versus Delphi,
where actually although we
see the stark differences,
the differences could even
be starker giving those
human tendencies.
GEOFFREY BURKHART: I think
that's absolutely right.
I mean, I think at
the end of the day,
this is a good,
conservative estimate that--
not just an estimate.
That's the wrong word.
I mean, this is really a
pretty finely tuned thing.
But even then, you're
going to have human error.
And that's the truth
of whatever you're
going to do in this area.
But part of that is going
back to training before we
do that six-month
time study, is really
training the [INAUDIBLE]
defenders exactly how
we want them to track time.
And we want them to be
as accurate as possible.
And a lot of times, when you're
talking about human error,
a lot of this is the
law of large numbers.
You're not just looking at
one attorney and their time.
But you're looking
at system wide.
And if you start to
see big outliers,
so-and-so says that
they're spending 10 hours
at the jail when everybody else
is only spending five or four,
you have an outlier.
And then you need to do
a little bit of research
and see why is that.
Are they genuinely spending
that much time and why is that?
Or is there a mistake in
terms of time tracking?
So again, that's why we
have that learning period
before we jump into
the six month study.
The Delphi process is
a little bit tricky
because you're not working--
really, you want
workload standards.
We're not dealing
with what is reality.
We're dealing with what
absolutely should be in order
to be effective.
And the Delphi process
is a very rigorous.
And it's been used in
businesses, in academia,
in the military.
But yeah, and again,
you rely in part
on the law of large numbers.
So you may have some outliers
in terms of these numbers.
But basically, when
you have 40 experts
and you have hundreds
of years of experience,
and you're distilling all that
knowledge into these numbers,
you should have a pretty
accurate sense of how much time
should be spent.
MICHELLE BONNER: Great.
Thank you.
Thank you.
I want to also thank Catherine.
And I hope we answered
your question.
But I wanted to let everybody
know that after the webinar,
the PowerPoints
will be available
and also available at the
righttocounselta.org web site.
Geoff, I wanted to
ask you same question
that I asked Tom in terms of
getting up to speed on this.
I know that your
background is you're
a criminal appellate attorney.
Did you have a large
statistical background
that enabled you to get
up to speed on this?
Or how difficult was it for you?
GEOFFREY BURKHART: Well,
I had-- my dirty secret
is that I went to grad
school before law school
and was once upon a time
going to be a sociologist.
So I have a master's
degree in sociology.
And I do have a little
bit of background in this
before I come into it.
But frankly, that had
been years and years.
I'd gone to law school and
practiced as an attorney
both at a firm and
as a public defender
before coming here or
working on workload studies.
So there was a little
bit of a learning curve.
And I think having somebody--
I mean Margaret Gressens
in Tom's office,
Andy Davies in New York,
a lot of these offices
do have some IT folks
in-house which is fantastic.
But a lot of people don't.
It's a little bit tricky
to get up to speed,
but it's definitely possible.
I think it's something
that attorneys
can learn and should learn.
And if you're just interested
in statistics generally,
and then how some of
these numbers work,
and the law of large
numbers, and things
like that there's a great
book called Naked Statistics
that really breaks it
down into very simple,
great for laypeople.
So if you want to
kind get up to speed
on basic statistical and
basic kind of big data stuff,
great place to start.
MICHELLE BONNER: Great.
I do have a question
for both of you.
I think really both of
you touched on this.
And the question is, what
are some of the incentives
that you have used to encourage
people to submit data,
submit the information.
Because like you said,
people are very busy.
It's hard enough to
do the work itself
without having to say, OK,
let me input this information.
How have you incentivized
or made it easier
for people to do that?
And this question
is for both of you.
TOM MAHER: I'll
take this is Tom.
For us, it's worked best
when the work relates,
they can see incentive.
So for example, for the
private time council,
they understood the
research we were doing
and the report we were
doing was directly related
to increasing their resources.
So we didn't really
have to pitch it.
I think if we had not turned
around and produced the report,
we would have lost
credibility the next time
we wanted to do that.
We've never been
in a position where
we've had to kind of give
outside incentive, do a survey
and win a iPad type stuff.
I think as a state agency, it'd
be very hard for us to do that.
So we do our best
to educate people
that the data is
going to be used
and it's going to be
used in a way that
will have at least a reasonably
direct impact on their work
and what they're doing
for their clients.
GEOFFREY BURKHART: My answer
is fairly similar to Tom's.
We've never used incentives.
And part of that is it's
cost prohibitive, I think,
for a lot of offices.
But in addition to that,
we're asking people
to track time permanently.
So if you take away those
incentives at some point,
do they stop tracking time?
We didn't want to
get into that mess.
A lot of it is,
as Tom was saying,
just educating about
why we're doing this
and what this can be used for.
And in the case of
workload studies,
again, it can be used for both
working with the legislature
or your funding source to
get additional funding,
but also it can be used
for case management
within the office
and managing attorney
caseloads within the office.
I think educating
attorneys on that is
the first thing you want to do.
And then the
incentives, we haven't
messed with that at all.
I think something that both
in terms of the learning curve
and in terms of an
additional time component,
I think when DNA
kind of first hit
the scene with public defenders,
and all of a sudden everybody
has to learn a little bit
more about forensics, things
that they hadn't studied
since high school, again,
you're asking attorneys
to do a lot more.
You're asking them to
learn about science, which
is for a lot of attorneys,
they avoided med school
for that very reason.
And here it is, forensics
[INAUDIBLE] It's more time.
But most agencies, you didn't
start giving incentives.
You said, this is
part of your job now.
And it's tough and
it's a learning curve,
but here's why we're doing it.
We're doing this
for the clients.
And that's the exact same thing
with these workload studies.
MICHELLE BONNER: I
like the similarity
in both of your answers in terms
of not only educating people
as to why you're doing it but
that's a very good starting
point before you even
start collecting data.
Because I think you have to
know what the end game is,
what your goal is before
you know what type of data
to collect.
Correct?
TOM MAHER: Yes.
I mean, I think, for example,
if you're doing a Survey Monkey,
you need to sit down and think,
what is this data going to be,
how am I going to organize it,
what am I trying to gather,
and what my end product be.
Or you may end up
with a lot of data
that you're not able to
really use or organize.
So you have to have kind of
your eye on the prize of what
are we expecting to
learn, what are we
expecting the deliverable to be
for all the work you're doing.
MICHELLE BONNER: Exactly.
Exactly.
And Geoff, you made
a very good point
about having-- you can have
too many data points, which
basically just neutralizes
or waters down anything
that you're trying to do.
And I think that's
something certainly
that we should all keep
in mind when we're trying
to collect and analyze data.
GEOFFREY BURKHART: Yeah.
You put it in a good way.
It's almost like
maps, in a sense.
You could have a map as big as
the territory if you wanted.
We could have a map of Illinois
that is as large as Illinois.
But it's not usable.
Or on the opposite extreme, you
could have one that's very tiny
and doesn't show too much.
You really have to
strike a balance where
you have enough data, it's not
too much data, not too little.
And it has a pretty accurate
picture of what you're doing.
And it's a tricky
balance to strike.
MICHELLE BONNER: Great.
Well, I do want to thank both
of you very, very, very much.
It's been not just
informative, but I
think it's been a very
energizing and very
inspirational conversation.
It's one of those,
for many of us,
uncharted territories
that I'm hoping
that more and more offices
will look to and realize
how much data they're
already collecting
and take it from there.
There is one thing I would
like to push in furtherance
of our embarking on
this journey together,
which is this basic
data tool kit.
It's the basic
data every defender
program needs to track.
This is a tool kit that
was produced by NLADA,
Marea Beeman, our director
of justice systems evaluation
and research initiative
wrote in late 2014.
There will be a link to the tool
kit on the righttocounselta.org
website.
But also, I'd say
check it out because it
is a good, basic start
for folks in terms of,
for example, what
types of categories
you can start collecting.
And this is a lot of work that
was done with Tom's office
and others in terms of helping
to discern what those data
points are and what those
starting data points are
that a lot of people
are already collecting.
And it is a basic tool
kit to help walk you
through how to start collecting
and analyzing data in a very
targeted and concerted way.
We also hope to have a
link on the same website
to Tom's private assigned
counsel survey that
can give people
an example of how
to use Survey Monkey,
for example, to package
the data in such a way to
be effective and persuasive.
The Missouri workload project,
is that online as well, Geoff?
GEOFFREY BURKHART: It is.
If you go to
indigentdefense.org,
and it's on the website.
MICHELLE BONNER: And we'll
try to put a link up to that
as well so people can look
at that report in more detail
and see how that Delphi
process worked for Missouri
and look at and see how
it might work for you.
Again, thank you so very much
for attending the webinar.
We do have a third webinar
scheduled for July 30.
And that's the Session Three
on persuasive budget advocacy.
We've talked about
cultivating relationships
with other stakeholders and
also within the defender
community in Session One.
And today we talked about
data collection and analysis
and began to have a conversation
about how to use that data.
And then in the
third session, we
will focus more on the
advocacy portion of that.
Once you have that
data analyzed and you
know where you want
to sell, how do you
sell this information
to get what you
want in your budget advocacy?
So join us on July
30 at 2:00 PM.
And I hope to see you then.
Meanwhile, you can
always check us out
at the website
www.righttocounselta.org.
Thank you so very much.
GEOFFREY BURKHART:
Thank you, Michelle.
TOM MAHER: Thank you.
