I worked as a mathematician and then as a
quant in finance - I saw the worst of finance.
I went into data science and I was struck
by what I thought was essentially a lie – namely,
that algorithms were being presented and marketed
as objective fact. A much more accurate description
of an algorithm is that it’s an opinion
embedded in math.
An algorithm is a very general concept - it’s
something that we do actually in our heads
every day.
To build an algorithm we need only two things,
essentially: a historical data-set and a definition
of success. So I build an algorithm to cook
dinner for my family. The data that I use
on a daily basis is the ingredients in my
kitchen – sometimes the time I have, the
ambition I have for that dinner. And then
I assess the dinner after the fact – was
it a success? I define that because I’m
the one who is building the meals – I’m
in charge, I have the power (there’s always
a power element here) and I’m in charge;
I get to decide a meal is successful if my
kids eat vegetables. My kids, if they were
in charge, would have defined it differently.
And it matters because we optimise over time;
we optimise to success. The succession of
meals that I cook from month to month, is
a very different path of meals than if my
son were in charge. So we do that every time
we build algorithms – we curate our data,
we define success, we embed our values into
algorithms.
So when people tell you algorithms make things
objective, you say ‘no, algorithms make
things work for the builders of algorithms.’
In general, we have a situation where algorithms
are extremely powerful in our daily lives
but there is a barrier between us and the
people building them, and those people are
typically coming from a kind of homogeneous
group of people who have their particular
incentives - if it’s in a corporate setting,
usually profit - and not usually fairness
towards the people who are subject to their
algorithms
So we always have to penetrate this fortress.
We have to be able to question the algorithms
themselves, especially when they are very
important to us.
We have to inject ethics into the process
of building algorithms and that starts with
data scientists agreeing and signing a Hippocratic
oath of modelling. We have to stop blindly
trusting algorithms to be fair - they are
not inherently fair, they are inherently picking
up whatever bias we’ve given them – and
start looking into what they are actually
doing.
