Good morning good afternoon good
evening everyone my name is Fredrik Winsnes
and then would be net hope and the
network solution Center today we are
very pleased to approach a very popular
topic artificial intelligence and
machine learning and particularly in the
health space we have a large group of
experts with us today and we're going to
hear about this topic from multiple
different perspectives before we get
started I just want to go over some
general housekeeping rules we are going
to be quite a large crowd today so we do
want to make this as interactive as
possible please open up the chat window
in WebEx you do that by clicking the
little bubble the third icon from the
right under the slides in follow along
there post your questions in the chat
window you can post them to all
attendees or you can post them to me
privately and I'll rebroadcast them to
everyone but we will reserve some time
towards the end of the hour today for a
facilitated Q&A session and as always we
are recording this session today and
after conclusion of the webinar we will
be posting to both the recording and the
slides to the net up solution Center you
know they're posting that link to the
chat window in a little while and at the
very end when we close down the webinar
you will see a webinar satisfaction poll
presented to you in your browser if you
couldn't could take a couple minutes to
answer those questions that would be
very helpful
I'll help us improve this webinar series
over time so with that I want to pass it
on to our moderator as on-air would so
she's with Catholic Relief Services so
over to you
Sonia thank you very much Frederick and
thank you very much to not hope for
hosting this now about
my great pleasure to introduce you to
our webinar and Frederick said it's an
artificial intelligence and machine
learning and health program and we have
a very diverse speaker panel for you we
will ask each speaker a question and
they will answer this question in a bit
more detail and that gives you also time
to think about questions for our panel
expertise and then we have time for you
all to ask questions and we have some
general Q&A lined up for the the end of
our webinar so let me introduce you to
our speaker panel we have Toby Norman
he's the CEO of sim print technology we
also have dr. anna shankar he's the
research senior research scientist at
the harvard school of public health and
stephen helen director of ICT for D and
G is at Catholic Relief Services and
Nilsa Jota
he's the UN AI subject matter expert
also a faculty member at the university
of california and an IBM master inventor
if you can see a lot of expertise so to
bring your questions up in the chat
window and without further delay I like
to pass on to our first speaker and Toby
will be sharing with us some insights on
what are the main opportunities and
risks of deploying artificial
intelligence into digital health care so
we over to you great thanks very much
can you hear me clearly yeah we can
thank you really
so I'm good morning good afternoon
everyone so I'm Toby Norman co-founder
and CEO of simple technology to start us
off I'm going to lay out some of the big
picture opportunities and risks that
were seen in the application of AI and
machine learning models to digital
healthcare challenges rather than just
looking at a single application I'm
really going to try and focus today on
sort of a couple of the big tensions we
see in the space
specifically three tensions learning
versus bias data versus privacy and
customization versus vendor locking to
give very quick background context the
lens the we view this sector is through
biometrics for patient identification
and verification so our team at sim
Prince is a nonprofit technology company
from the University of Cambridge with
really mission to radically improve
transparency and effectiveness in global
health livery using tools like AI
biometrics GPS mapping so in real terms
what that means for example we work with
Brax frontline healthcare workers in
Bangladesh to link mothers and children
to the healthcare records through
fingerprints enabling continuity of care
for patients as they move from health
workers to facilities but also enabling
brac to track who they reach into
they've missed and I think one of the
really fun parts about a job is working
as a technology partner we get a front
row seat into many organizations and
programs about how they're actually
applying artificial intelligence versus
potentially just writing about it so if
we go to the next slide I think one of
the first challenge is to pull out in
the application of AI for digital
healthcare is the challenge between
learning versus learning from our biases
and so for example I'm sure most people
on the call are familiar really be a
central attraction of AI for many
organizations is the ability to have
algorithms learn from data to make
decisions and the opportunity here is
huge you know for example we see teams
like google deepmind release
applications like streams to monitor and
predict when patients with liver disease
or kidney disease deteriorate and allow
doctors and nurses to get to them in
time to prevent a patient crashing
similarly we have partners in the field
who are using tools like super set to do
predictive analytics on community health
worker visits something we're allowing
them to use past performance in terms of
visit coverage to predict which areas of
health workers that which in the future
and more closer to our field we're using
tools like image recognition and
approaches like convolutional neural
to do machine driven biometric image
recognition the flipside to this
capacity of a I've learned is of course
it is entirely determined by the type of
data we train these models on what these
algorithms learn and the risk is that
they learn from our pre-existing biases
so for example there is a study released
by MIT earlier this year run by joy
Boylan we need the research on the the
right hand side of that photo the showed
for example that across some of the top
machine learning algorithms put up by
IBM put out by a number of Chinese
companies Microsoft and others the their
phase classification software was nearly
90 percent at the 99% accurate with
white males but error rates ballooned up
to over 35% with black females based on
the quality of the data sets
similarly there was review in nature
released earlier this year and I can
share the link in the side panel looking
at both bias trends in terms of racism
and sexism in machine learning
algorithms and three criticisms bias in
the traditional sense these algorithms
not trying to be biased but based on the
quality of the data that we train them
on this can can introduce our own biases
into machine learning so sort of that
old adage garbage in garbage out holds
very true for AI and particularly as we
think about applying it for digital
healthcare applications we need to think
very hard about the balanced about
datasets to make sure algorithms are
learning from our biases if you click to
the next slide one of the seconds I
think real opportunities but real risks
with machine learning applied to digital
health care is the balance between the
incredible access to data that we have
now versus the incredible breaches of
privacy of that data potentially makes
us susceptible to for example
conceptually a artificial intelligence
machine learning is a pretty old concept
computer scientists have been theorizing
about it for decades however for a long
time there have been very few real-world
applications that have taken off I think
as many of you probably know two things
have really driven what some call the
end of the AI when
first was the access to cloud computing
to serve cheap distributed processing
power and the second is the explosion of
data that we've experienced really in
the past decades so for example the IDC
estimates now that globally
by the end of 2025 there's going to be
163 zettabytes which is 10 times more
data in the universe in our human
universe that exists today that's 10 to
the power of 70 bytes if you look at all
of human history at the moment we have
an explosion of data and access to data
that we've never seen anything
comparable to before now for digital
healthcare applications this is a huge
opportunity for example when I started
my doctoral research with frontline
field workers most of the work in
Bangladesh was entirely paper records
which means by the time aggregate
numbers reach the Ministry of Health or
reach the headquarters
you've lost 99 percent of that data
you're only taking the very top level
aggregate statistics from your community
health workers or from your facilities
well that's exciting it also exposes us
to huge risks you know I think many
people probably where there's been giant
security breaches over the past couple
years the US government has seen major
hacks the Philippines government closer
to our sector red rose was attacked by a
competitor but it's not just security
breaches
for example the Cambridge analytic on
Facebook scandal wasn't a security
breach it was a privacy breach and so as
we think through some of the potential
applications of AI thinking through what
are the privacy standards that we will
apply is a really funny challenge for
example we typically stand behind gdpr
of the new European general data
protection regulation is one of the
stricter standards the teams can
potentially follow but it's definitely
not perfect there are limitations in
this area and program out how do you get
genuine formed consent say from a
frontline patient you might have five to
seven years of education across their
life it's a really complex issue and
something we could take in a little
further
moving to the next slide but find me one
of the tensions I think we want to pull
out as we dive into specific examples of
AI and machine learning is a huge
opportunity there is the bill
customized algorithms to a degree that
we've never been able to do before and
so today it is easier than it has ever
been to actually build machine learning
models yourself with very small
technical teams large players like
Google have opened up their models like
tensorflow
and built a range of tools both for
computer based applications but also
through tensorflow light phone based
applications and on the hardware front
as you know the real trend now is
towards edge or on device machine
learning which means you're able to
upload your custom data set to a cloud
hosted by Google or one of the other
large providers and then push that on to
frontline mobile devices which many
working in digital healthcare in
developing countries is really the key
because that allows you to apply machine
learning models offline in real time on
device without having to be connected to
a cloud one of the challenges that opens
up though is the particularly the
digital healthcare space
interoperability and standards is a
really thorny problem to begin with much
less before now we start having custom
models with signatures and embeddings
that are unique to certain projects that
mean it was very difficult to transfer
that data and use those models between
projects so to give a concrete example
in the biometric worlds you know if one
vendor comes in and stores a number of
biometric images in their proprietary
format for projects they're doing on
confidential HIV testing and using the
biometric is a unique way to link
patients to record if that can't be
transferred to another vendor that
Ministry of Health or that hospital is
potentially locked into that vendor
going forward and at the moment there
aren't standards that we agreed
internationally for the use of AI in
this space so if we go to the next slide
I think there's a number of really
exciting the conclusion really is the
for every complex problem there is an
answer that's clear simple I'm
potentially wrong I think AI has a huge
number of potential applications and
opportunities in this space will it come
some tensions I'd be careful of the
cynics either too far on the left or the
right to say
we shouldn't use it versus its going to
solve all our problems and just probably
somewhere in the middle but as you think
about the application of AI and your own
problems I would keep a close eye on
these three tensions so figure out
what's the best way forward to use this
as a tool to solve challenges but
definitely not as a silver bullet so let
me pause there but I'm happy to dig into
any of these further during the Q&A
excellent thank you very much Toby that
was a great start and it's my pleasure
now to introduce you to our next speaker
dr. Anna Shankar he's the senior
research scientist at Harvard School of
Public Health and I know over to you
great thank you so much great to be with
everyone today I'm going to be
presenting some work to you today
related to use of AI machine learning
for frontline health workers and a lot
of the work is based on data collected
to this platform it's called the open
smart register platform this is a
scalable open-source platform enables
frontline workers to digitally register
and track their clients and as you know
and currently if most frontline workers
are using paper registers transition to
this type of platform really enables a
lot of the AI and ml algorithms to be
used most effectively as Toby was
alluding to and you can go to smart
register dot org to really learn more
information about the platform itself
next slide so some of the challenges
really for AI and machine learning
particularly in public health I want to
highlight a few of these the first one
is of course the use case itself like
what are you going to use AI and ml for
in fact so really defining that well is
important many times you hear sort of
general ideas that G we want to use AI
to improve health worker performance or
to improve Diagnostics but you really
have to formulate the question in a
clear way what particular type of
diagnostic do you want to improve and
using what kind of information and that
brings me to the second issue which is
that you've got to decide what
the type of data that you need to put
into any sort of algorithm it's not the
case you can just gather a whole lot of
information and assume the right
information is going to be in there and
the key there really is identifying what
is the right information that's likely
to in fact enhance the output of the
algorithms themselves and this really
brings me to the third point which is
the assumption that somehow AI or
machine learning can clean up bad data
or bad quality data and that is not
correct if you have bad quality data of
course a signal-to-noise ratio is not
going to be good and your machine
learning AI algorithms are really not
going to produce something that's going
to be used
going to be very useful to then I want
to highlight this issue related to the
human gap which is essentially the
willingness of frontline health workers
and their supervisors and perhaps
government or other officers or
clinicians to trust the results that
you're getting from these algorithms so
often these algorithms use some advanced
statistical techniques various types of
advanced analytical processes and if
these cannot be explained carefully and
so people understand what's going on and
why that result is a reliable result
they may not be willing to accept
recommendations from that and not trust
the conclusions and keep in mind a lot
of the recommendations from AI they are
probabilistic sorts of recommendations
so yes sometimes they're not going to be
correct so how do you work with that
kind of information in the context of
clinical care and ultimately I come down
to this issue which as humans help other
human being technology is just a tool
meaning that you're not going to get
some great change all of a sudden in
your health program by using AI
ultimately you really have to use the
sort of fusion between high quality
human resource implementation on the
field linked with the actual inferred
itself so I'm going to walk you through
a use case here next slide so basically
we were doing some work in Indonesia
related to active screening for
tuberculosis and this involved frontline
health workers going to a health clinic
and just asking other people not just
their patients with other persons who
were there at the clinic if they had
various signs or symptoms of
tuberculosis and here I've listed the
symptoms there that these people were
asking these people might be persons
accompanying a patient a relative could
be a worker in the health clinic
actually and just asking these questions
and then deciding are you a suspect TB
patient and should you be referred to a
more expensive assessment like using a
gene expert or a x-ray and this work I'm
going to really present to you now is
done by an analyst his name is Ali Sept
Ian Dury
and I've listed there the reference at
the bottom so next slide so we had this
initial information of all this data
from around 4,000 persons or so maybe
close to 5,000 person so the first thing
is actually going through the data
preparation and cleaning standardization
there's absent data missing data you
have to decide are you going to impute
that and so forth and I only emphasize
this to understand that really there is
quite a lot of data preparation that
goes into any sort of machine learning
algorithm then you want to select
potentially additional data that you
might want to use it might be
interesting in this case we selected
clinic location you can see there in the
table that for this particular analysis
using these three different ml
approaches X eBoost
SVM support vector machines and just
actually standard logistic regression
that when compared to the w-h-o scoring
approach for those symptoms I showed you
before the WTO scoring approach had an
accuracy of around 30% a sensitivity
which means a proportion of actual true
TB cases detected was around
91% in the specificity which is the
false cases so the false positives you
end up getting or really is around 20%
so it means that actually the WH out
score really doesn't correctly identify
persons who don't have tuberculosis and
that has implications for costs of
course using the AI approaches what we
the main thing you can see is you're
able to preserve the sensitivity and the
specificity goes up so you can see
you're talking about changes from let's
say 20% specificity with the w-h-o
algorithm up to around 40% or more using
these different ml approaches the next
slide so the implication of that is that
in fact these frontline workers equipped
with this more advanced machine learning
algorithm can in fact increase their
specificity by 20% or more and this ends
up they save time and costs from doing
unnecessary more advanced tuberculosis
assessments because they have actually a
better assessment of those patients who
are more likely to actually be positive
for tuberculosis and I've shown you
another display there on the bottom of a
data visualization and data reduction
technique that's common now in machine
learning AI approaches it's called
uniform manifold approximation and
projection you can see on the left there
each dot is a person and I encoded each
color there is a clinic there's a
basically around ten clinics there and
then on the right there you can see that
all the yellow dots are the true TB
cases so you can see in fact they're
just from the visualization itself that
many of these TB cases true TB cases are
showing up at specific clinics so again
that just sort of confirms why in fact
the specificity is increased by using
the other machine learning approaches
next slide so I just want to really
finish with this slide emphasizing that
certainly AI and machine learning can
have very important applications to
increase workforce productivity
identifying false or bogus Jaidev for
instance is very useful
Peter LaBelle Doughty doughnut systems
of working on that quite extensively you
can think of ways to improve worker
productivity I showed you an example of
that and also of course a lot of
applications to improve client health
itself by identifying specific point
health seeking behavior which clients
are likely to come back for example for
a health visit which clients are likely
to be compliant with their treatments
and so forth and what types of
interaction interaction for counseling
sessions might be effective so we really
have to define what is the use case so
there's certainly a lot of potential I
think we're going to be discussing more
of those hopefully on the question and
answer thanks so much I think I'm going
to end there thank you very much Anna
for sharing this interesting case study
with us I also like to take this moment
to mention that we will share these
slides afterwards with all the
participants so you don't have to be
busy taking notes and we also share the
comments and a question look afterwards
as well so this could keep your
questions and your comments coming and
thank you also for posting resource
links I think that's very interesting
and helps a more diverse overview and
now it's my pleasure to introduce you to
my colleague Steve Helen and Steve will
be sharing some use cases of a MRC
learning in the global programs at cap
relief services Steve over to you
in Malawi we are using machine learning
algorithms to predict food insecurity
and by extension to influence
trishing outcomes so Malawi is a country
where eighty-four percent of the people
live in rural areas and rely on
subsistence agriculture and this this
characteristic makes the population
particularly vulnerable to
weather-related shocks going back to
2015 there were devastating floods that
displaced hundreds of thousands of
people since then there have been severe
cyclical droughts and more recently
there is crop destruction due to the
spread of the fall army worm pest two
years ago we partnered with Cornell
University to develop a low burden high
frequency data collection protocol
across tens of thousands of households
to track shocks household
characteristics things like demographics
asset and location and food security and
we measure that using what's called the
coping strategy index that looks at
activities that households might be
compelled into such as skipping meals
were sending their children out to beg
we apply two different machine learning
algorithms to predict this household
level vulnerability and we validate the
models in the field and ultimately we're
able to predict with high confidence the
specific households that would be at
risk of food shortages a full one to two
months in advance and this is an
improvement and in extension to what's
built through services like fews net
which are providing a prediction and
monitoring around a famine at a broader
geographic area so it's kind taking that
as one of our inputs but really
detailing down to this specific
household level where we'll see the
vulnerability this early warning
information is now regularly shared with
the local village development committees
so that they can plan and target
responses more effectively since we've
implemented this in Malawi we're now
process of extending this to Madagascar
so we've been through the first few
months of this high frequency data
collection we've put in place a slightly
different algorithm that's more
contextually appropriate to that
location in this specific needs there
some of the challenges that were we're
facing with this and to kind of fall
into two categories number one is that
the technology still requires a high
degree of customization to a plan to
introduce case this requires specialized
skills
it doesn't replicate easily and this
manifests itself by you know I gave two
examples Malawi and Madagascar we have a
portfolio of about 500 projects so we
have these point examples of this being
used effectively but the technology is
not yet easily replicable to use at
scale as a normal part of our project
operation I believe that over time that
will evolve and I think one of our roles
as the international development
community is to influence some of the
technology partners to begin to make
more of the AIA tools more accessible to
non specialists and what I would call
the democratization of AI tools this is
the second barrier that we're seeing is
I'd say within our organization most of
our staff don't know we're not aware of
what's possible with artificial
intelligence or simply how to get
started so we need to do a much better
job at socializing what's possible
explaining what resources are available
and you know if I can offer some
commentary on how to get started as a
NGO practitioner first is to get a good
expert at this point where the tools are
not as easily accessible
we found that establishing a partnership
whether it's with a specialized
technology provider
or university or simply bringing
somebody one board that has the this
specific skillset can lead to some early
successes and then the second focus is
to lay the groundwork to use AI as a
strategic capability that achieves
specific outcomes at CRS were
approaching this in several ways
number one we a couple months ago formed
a task force so this is spearheaded by
our monitoring and evaluation unit in
partnership with our Information
Technology Group that's that will be
doing things like creating guidance
selecting tools curating trainings
socializing the art of what's possible
and establishing support models and
partnerships and if I can leave with
become one closing comment at this point
we are still very much in a learning
phase I'm striving not to overlook your
biases some of the concerns that Kobe
caught out and really keeping an eye on
the fact to make sure that the results
are impactful so that technology itself
does not become self justifying thank
you very much Steve
and that is I think a very good
transition to our next speaker Neal we
like to ask you what are the key
questions you should ask yourself before
starting an AI based initiative and also
what are the small things you can do to
make a difference Thank You Neal thank
Sonia and a good day or good evening to
everyone wherever you are um just to I
guess build off I think what Steve was
saying um really to get started it's not
about coding a lot of people that's
their first reaction to doing something
with AI or machine learning that it's
like a hundred percent coding and that's
not the case as you've actually heard
from Toby and Anuj and Steve
data is really important I've been good
actually having the data and having good
data is key as well as having the
subject matter experts to actually train
your solution but that being said I walk
it back to even an earlier standpoint
that you have to really think about the
problem that you want to solve and I
think that's where a lot of people
struggle they they think that everything
has to be so technically oriented how am
I going to be able to do everything need
to find someone good and I'll use law as
an example because it's a very
slow-moving area and nothing's really
broke in and really slow to adopt things
but you hear a lot of the buzzword of
legal tech but if you look at all the
companies doing legal tech very few or
successful in more so because yes really
smarter technologists guys trying to do
something but they understand the pain
points of the industry the the actual
the legal technologies are actually the
most successful actually ones that are
started by loyal or lawyers paralegals
people that actually have a strong
technical background they just know
where the problems are trying to
actually solve those so I'm a firm
believer that if you get a really smart
group of technologists together they're
going to think of the cool ideas like
the self-driving car and not saying
there's not valuing like that but if you
gave some physicians or nurses or
clinical researchers so basic
understanding of the technology and then
put them in a room together think about
what they could actually do so to really
get started you ask yourself what is the
problem that I want to solve see do you
have the right subject matter experts to
help you then identify those
opportunities and then start looking at
okay
who should I partner up with much like
Steve was talking about that to bring
that technical expertise help you build
a solution they can help me see do I
actually have the right data the good
data can i manufacture the data
training solution now I know that people
say that that sounds kind of easy it's
not AI is not like a magic solution that
just happens to know stuff the data is
always a challenge especially in health
care because a lot of people
unfortunately they don't share
information so it could be very hard to
access what you need the second is how
do you actually free up subject matter
expert time so I remember with IBM
Watson when we were teaching it about
cancer research and trying to help
develop some new target proteins
well the biggest challenge is actually
training Watson understand the space and
while we had access to the data having
access some of our experts was more of a
challenge because they're very busy
people they're working on real-world
problems trying to help people and so
every how we take away from them doing
that train is the challenging so takes
with a long process that being said
there's real value in trying to do this
type of work now people always think
kind of big bang well pink pie in the
sky let's try and you know find a cure
for cancer let's try and put an end to
you know sickle cell anemia these are
great ideas but they're very large very
effort intensive and could take years
potentially even decades I think
actually a lot of low-hanging fruit out
there that's often gets overlooked that
we could actually do with AI machine
learning and I guess you heard a couple
of good examples from the other
panelists firm believer that small
changes actually make a difference so
this is kind of your first foray and
trying to think about this technology
don't think big bang think small think
incremental right there's a Spanish
company called ivy health that they're
very much about preventive medicine and
prior focus on wellness
they could try and go and say well hey
could I create like II totally
personalized routine around nutrition
so Fitness mindfulness all these things
it would take forever instead of looking
at small incremental things and say well
we have all these exercise videos right
it's not to try making one size at all
could I tailor that using a eyewear
basically I know about the person about
their current physical condition the
goal you know maybe some of the
restrictions that maybe took them doing
this exercise you know twelve you know
three sets 12 reps each maybe they can
only start off by doing you know one set
three reps but let them build up over
time so to really create that you know
personalized trainer for you but take it
stepwise and once you get that then you
go a little bit more you go with more
you'll find it over time these small
changes will really add up and make a
difference so the best advice I can give
you is think about the problem you want
to solve
don't worry too much for the technology
wedge but the problem solves the
opportunity have out of that find a good
partner and then see she has the data
and their expertise to actually create a
solution and thanks fall start small and
build on thanks thank you very much meal
very good advice and also thank you to
all the speakers to very much sticking
to your time that we have plenty of time
for discussion now which is fantastic
because we have a lot of questions
coming in we'll just go a bit
chronologically one of the first
questions maybe I'll ask Toby first
because you mentioned it in your
presentation and are you aware of any
audit processes designed to prevent the
ethical risk yeah it's a good question I
mean it certain my knowledge there are
no comprehensive audits to prevent all
the ethical risks so if you look in the
common panel I think Merricks shade the
USAID document there's a good section in
there
looking to the ethical risks so if
you're thinking about deploying an AI
into one of your projects I'd definitely
take a look at that the frameworks are
still quite nation I would still argue I
think the strictest framework currently
is Europe's GDP our standard which many
of us probably familiar with now because
it's hit now every email list and every
group that we're part of but
particularly some of the practices that
they use like data protection impact
assessments I think is a really solid
approach to try and think through
preemptively some of the risks our
sherilyn fenn wants to see examples we
post the ones we do it's imprints
publicly please feel free to sort of
steal that format and steal that
template what you're really looking to
do here is just work through
methodically and intelligently what are
the genuine risks that having this type
of data collection which you'll need to
train a model and then the deployment of
that AI model could potentially have and
talk with people both within your
organization but really critically
outside of your organization as well
both with experts which I'm sure you can
get through communities like this but
also thinking through how it might be
perceived at the field level and though
will hopefully give you the strongest
possible audit in terms of potential
risks on your project excellent thank
you very much Toby any of the other
panelists like to chip in on preventing
the ethical risk yeah this is one
resource I would add there was a body of
work that came out just soon I think
within the past year called the Toronto
declaration that really called out some
good good perspectives about how to
prevent discrimination when machine
learning or artificial intelligence is
applied and thinking about things
focused on human rights and equality and
offering some good guidance and I think
that that provides at least the
foundation upon which may be more
deliberate methods to audit or review or
certify algorithms could be based upon
excellent thank you very much Steve we
will share the link as well in our
summary my next question maybe I can
address it too I know what are the what
are some of the most promising data
sources for AI and machine learning in
the health space and what challenges do
you see in accessing or sharing such
data I think some of that second part we
might have already answered but please
yeah that's a very good question and I
was I would say a huge part of the
answer as well you know it depends on
what you want to do with the data
there are several databases in public
health that that are really open for
people to look at a lot of those
databases relate to households survey
data so for instance the demographic and
health survey data the UNICEF multiple
indicator cluster surveys these you can
you can access if you're looking to look
at the use machine learning to identify
sort of general predictors of disease
patterns in communities and in
individuals and households there's also
data that would be available there's
some biological health data warehouses
his yo net is one example of that and
there are also ways to access data from
social media from Twitter and so forth
some of these platforms actually will
allow you to access a certain percent of
the tweets and so forth other than that
certainly there are many other sources
that certain consortia are collecting
for instance there are some groups that
are
using personal health personnel
information trackers like the Fitbit or
ACTA graphs and I think getting in touch
with some of those groups and creating
some collaborations would enable you to
access more of that sort of information
also and also one thing that's really
growing very quickly in terms of
information sources are with the
deployment of universal healthcare
national health insurance systems as a
big source of data again there's lots of
security health confidentiality issues
there but if you can develop a
collaboration with a government that
would be a source of a lot of
information and certainly some of the
work done with Open SRP or openmrs with
collaborators in-country if you're able
to create links with that those would be
some other large-scale sources of
information but I think the the open
ones that I mentioned earlier would be
the easiest ones to get started with
excellent thank you very much for that
advice and next question is from Fred
Wright please go ahead
oh yeah there's been a lot of talk about
it we'll be responsible with the data
and the risk involved and everything but
did you can you point to some resources
that are available to educate staff on
how to handle data responsibly
I because anyone could jump in yeah yeah
so one thing I would do this is on a
Rosh Hashanah car I would definitely
recommend there are several exams on the
web that people can take related to
confidential and ethical use of data and
I can i'll send through some of the
links for those they don't have those
handy now but I think having your staff
go and read that information and to take
those online exams is extremely useful
there's also quite a lot of training I
think in education that needs to be do
needs to be done with frontline health
workers and government staff so the
actual confidentiality in fact of
paper-based information is not as high
as one might expect and I think one of
the nice things about this information
becoming electronic is its really
raising these issues around confidential
confidentiality and ethical treatment of
persons and their information because
the ability to share it so easily has
really prompted people to examine that
but in fact data confidentiality and
many frontline health systems even on
the paper-based systems has not been
what it should be so developing training
overall not only with your staff but
also with any persons that their
engagement in engage with I think would
be extremely useful and there are some
resources for that as well in some of
the links for the online trainings
there's a lots of resource materials you
can download Johns Hopkins University
has a lot of resource materials related
to research in human populations as part
of the part of the trainings I think
what I'll put some of these links in I
think with with the slides that you're
going to be available from this they're
very helpful so I'm here back to you
excellent thank you any of the other
panelists like to chip in on
how to train your partners or colleagues
on responsible data handling just a
quick comment so Toby Norman's begin it
would maybe split these into two
considerations so I think there's data
security and in terms of how you manage
your keeping the data that you're
collecting secure and you probably want
to think about that in the different
levels of your organization so for
example for your moderators for your
technology or IT team they're a data
security courses I think or sir and the
good me have a whole bunch if you can
afford it those with external
consultancies who will come in and run
things like penetration testing to make
sure the sort of the system is secure
and also do general staff education
that's quite critical because really
from a security point the vast majority
of data breaches are actually human
error more anything than known it's
people responding to spear phishing
attempts or phishing attempts giving
away passwords and things voluntarily
sometimes it's a technical security
breach but a lot of the time it's do
that for the field level staff there are
things you can enforce for example if
the frontline workers are using mobile
devices there are things that you can
employ like two factor authentication
which could be considered best standard
the one caveat I would add to that as
often you're going to be tend to be
working in low internet or connectivity
settings which can make some of those
things more difficult to do and making
sure that the level of sort of security
training you do is appropriate to the
field level the second area I think is
equally difficult to challenge which is
rounds through privacy so it's thinking
through very carefully which data do we
collect in the first place and how do we
you know actually train users to get
genuine and informed consent for during
that there's a really natural tension
between I think the researcher I come
from a research background and many of
us who want to collect all the data all
the time
immediately versus actually the
principle of data minimization which is
to collect as little data as possible to
begin with because that's the most safe
and the most secure and the most privacy
respecting route and again there are
external consultants who come and train
those a couple web courses will put up a
couple links it's not an easy thing to
do but
usually this is something that I very
rarely he rolled out perfectly
immediately something you want to
iterate on and make sure you have the
appropriate level of training for the
appropriate level of staff thank you
very much my next question is or
actually the next question that mine
personally is for Neal and more often
than not investment in the capacity of
you that is not prioritized in AI or
machine learning initiative the where
can a I be most impactful and how would
you advise to go about ensuring
sustainability and in country over ship
of AI initiative that's a great question
and that's it's a difficult challenge I
mean the digital age there really aren't
any boundaries anymore and I think it
winds up boiling down to having a common
set of best practices best practices
ethical standards one of the big
challenges that I see and a lot of
people tend to talk about is how do we
handle things like with the
infrastructures that exist or the
resources or there's a talent gap and
actually kind of do some of these things
and it's this knotty solution it
requires commitment investment and help
I can point to the Commonwealth of the
Northern Mariana Islands actually a u.s.
territory they were very focused on the
garment industry until the WTO relaxed
the regulations about 10 years ago and
suddenly other areas all became much
cheaper and their economy fell apart
they don't have the infrastructure to
have massive tourism so the question
becomes what could they do and one of
the things we're looking at well we
learn a little bit of technology and try
and become a tech hub right people in
Japan South Korea China very familiar
with that area vacation of those spots
but goodness
was could we actually learn like basic
skills like web that kind of stuff and
the truth is if you have unique
opportunity to leapfrog and actually go
into the AI and watching and the IOT and
try and become a tech hub but one they
don't have obviously the skills in-house
so because they set up programs
education they're actually actively
working to try and do that actually
partnering with some organizations
around the world including the Bay Area
for upskilling internships and stuff
there's also the question of internet
speed and telecom but there's enough
traction and from the first steps that
they're doing that South Korea Telecom
is actually now said they're going to
bring 5g to the islands so so what
they're really doing is like to call Joe
Cisco models or looking to say how can
we build our own ecosystem and find the
right partners and I think to try and
tackle some of these issues keep things
in country in-house or sustainable at
least we start thinking in terms of
ecosystems we can't do everything
ourselves there's too many moving pieces
I'm a big believer about not reinventing
the wheel we have to think about what is
it what's the outcomes we're looking for
and what are the components that are
going to drive that and then more
importantly who are the strategic
partners which actually work with to do
something and I think particularly in
health care we look at some of these
things we may not have all the pieces
but maybe we could find the partner and
it has the data we can find those
partners or the technical skills that we
can less you look in different areas to
actually say what can we piece together
and for like the low middle income
countries there's actually a good
opportunity for them to solve some
pressing problems by building their own
ecosystems they may not have all the
resources today but there are now
opportunities to pull those partnerships
to bridge those gaps wonderful oh thank
you very much Neil the next question is
for onwards I think
more about question around definitions
and some of exams were shared what's the
real difference between artificial
intelligence and old-school statistical
method to be able to comment on them oh
yes that's a very good very good
question
so the there's sort of two things there
the one is that the the the old-school
statistical approach is typically ask
you to to formulate and greater detail
what are the other variables or the
predictors or the things that you think
are going to be associated with a
particular outcome you know survival or
being positive for some infection or
whatever it is and so they're more
geared to be hypothesis driven so a lot
of the regression analyses for examples
are like that
and also the the most of them are most
of these statistical methodology the old
school ones are either based on simple
classification system approaches or a
regression approach of some sort the AI
approaches tend to be a bit more complex
and being able to look at more advanced
probabilistic decision making an
approximation approaches and typically
you're not being asked there to define
as well a a priori a particular
hypothesis but instead looking for
patterns in the data that might be
associated with some particular outcome
or even looking at natural clusters in
the data itself now having said that I
will say that there's not often that
much you don't always get that much of a
improvement in insight using some of the
more advanced AI approaches compared to
some of the old-school approaches so I
always recommended
get your data together if you want to do
data mining whatever it is start with
some of the simple approaches whether
they're the machine learning or the
old-school one start with those first
and then you can move on to the more
advanced approaches but that's a very
good question thank you very much and
our last question I like to address to
Steve can you please expand impedence
how to engage Iranian staff we're
interested in using AI and machine
learning so I would say definitely
engage and not bring in um you know I
really believe that what we're seeing in
these early days with AI and machine
learning is quite analogous to what we
saw 15 years ago with the start of the
mobile revolution that this technology
paradigm will be transformative to our
work in the Ada and development sector
and trying to rein it in I think is the
wrong approach but rather to engage and
the one one example I can sure that we
we kicked off about a year ago was
developing a some guidance to use within
our organization that really tried to
surface three things one was to explain
through some examples the art of what's
possible applying these techniques into
the sectors that we work in this kind of
effective ways to enumerate some of the
resources are available so areas of
expertise tool sets and so forth and the
third is to start to develop some
thinking around biases and areas to be
cautious of and we hear this was
packaged in the form of you know like a
two-page document and circulated broadly
within our organization to all of our
field practitioners all of our program
managers technologists M&E staff and
others had a chance to see that and
that's something that I think is a way
to start to open that conversation about
this technology trend which I think will
really continue to grow
in an important and impact in the work
that we're all focused on I think thank
you very much
we're known merely at the end of our
webinar so I would like to thank all our
panelists for their fantastic
contribution and I have a final question
for each of you so I would like to ask
first Toby and then Neal
what is your key trend you're seeing in
a use of AI for health program to be
would you mind answering sure so you're
hearing yes yeah sorry so I think the
key trends that I'm seeing is really use
of AI to figure out where health visits
are happening and where they're going to
be happening I think that's really
exciting but we've got to think through
the tensions we talked about you know do
we have biases in the data in terms of
what we're expecting to see are we just
having our highest performers generate
the most data and thus the models are
biased towards those I think we've got
to think through you know security and
privacy very very carefully and then
finally long term but particular because
this is you know it's a new field it's a
sexy field a lot of funding it's
gravitating to this field I think we're
not engaging enough with questions about
future systems interoperability which is
we think about these tools not just
deployed in pilots but potentially at
large scales potentially with national
government initiatives we're going to
have to confront these questions so
that's the trends and those are the
three things I would think through
before engaging significantly with the
AI based projects Thank You Toby and
Neil do you have any final trends you'd
like to share with them I do there's two
things that I'm seeing out in the field
so one is there's a big push towards
personalized medicine that's you know
even with the pool that we're using
today they're still too abstracted and
so there's more focus on using AI to
rules to help drive that to much lower
level even to the level now where
they're using some AI tools for genomic
sequencing to figure out you know
there's four pharmaceuticals that might
work for the diagnosis which one would
might be the best at what dosage based
on your genetic the second trend that
I'm seeing is focus on try actually
create and use AI tooling to try and
solve more of the common problems or at
least assist you are just a time on
those common problems free up doctors
nurses and researchers to work on the
more complex cases maybe the more you
know infrequent or rare cases that have
a much more like fatal or long-term
damage to people so essentially treat up
more time to do that type of work
wonderful thank you that pretty much
concludes our ena thank you very much to
our speakers and also to met hope for
facilitating this webinar for us we just
have one more announcement I'd like to
highlight that we currently have an open
call for speakers for our next ICT
foodie conference which will take place
in Kampala on April 30th to May 3rd and
the co first speaker and end of this
month and particular AI machine learning
again will be very important topics we
like to discuss so please consider
applying as a speaker and then we will
meet here of our next ICT for the
webinar again December the 12th when we
will talking about digital financial
tools for monitoring one I thank you
very much for joining and looking
forward to reconnect next month
thank you very much Sonia I just want to
thank you so never be all the presenters
and ask everyone to please answer this
short webinar satisfaction poll did
you'll see in your on your screen as you
exit the
the webinar today thanks everyone and
we'll be back in touch soon take care
have a great rest of your day so bye-bye
you
