I've built a recommendation engine before as part of a large organization and worked
through all types of engineers and accounting for different parts of the problem.
It's one of the ones I'm most happy with because ultimately,
I came up with a very simple solution that was easy to understand from all levels,
from the executives study, engineers and developers.
Ultimately, it was just as efficient as something really complex,
and they could have spent a lot more time on.
Back in the university,
we have a problem that we wanted to predict algae blooms.
This algae blooms could cause a rise in
toxicity of the water and it could cause problems through the water treatment company.
We couldn't like predict with our chemical engineering background.
So we use artificial neural networks to predict when these blooms will occur.
So the water treatment companies could better handle this problem.
In Toronto, the public transit is operated by Toronto Transit Commission.
We call them TTC. It's one of
the largest transit authorities in the region, in North America.
And one day they contacted me and said, "We have a problem."
And I said, "Okay, what's the problem?"
They said, "Well, we have complaints data,
and we would like to analyze it, and we need your help."
I said, "Fine I would be very happy to help."
So I said, "How many complaints do you have?"
They said, "A few." I said,
"How many?" Maybe half a million.
I said, "Well, let's start working with it."
So I got the data and I started analyzing it.
So, basically, they have done a great job of keeping
some data in tabular format that was unstructured data.
And in that case, tabular data was when the complaint arrived,
who received it, what was the type of the complaint,
was it resolved, whose fault was it.
And the unstructured part of it was the exchange of e-mails and faxes.
So, imagine looking at
how half a million exchanges of e-mails and trying to get some answers from it.
So I started working with it.
The first thing I wanted to know is why would people complain
and is there a pattern or is there some days when there are more complaints than others?
And I had looked at the data and I analyzed it in all different formats,
and I couldn't find the impetus
for complaints being higher on a certain day and lower on others.
And it continued for maybe a month or so.
And then, one day I was getting off the bus in Toronto,
and I was still thinking about it.
And I stepped out without looking on the ground,
and I stepped into a puddle, puddle of water.
And now, I was sort of ankle deep into water,
and it was just one foot wet and the other dry.
And I was extremely annoyed.
And I was walking back and then it hit me,
and I said, "Well, wait a second.
Today it rained unexpectedly,
and I wasn't prepared for it.
That's why I'm wet, and I wasn't looking forward."
What if there was a relationship between
extreme weather and the type of complaints TTC receives?
So I went to the environment Canada's website,
and I got data on rain and precipitation,
wind and the light.
And there, I found something very interesting.
The 10 most excessive days for complaints.
The 10 days where people complain the most were the days when the weather was bad.
It was unexpected rain,
an extreme drop in temperature,
too much snow, very windy day.
So I went back to the TTC's executives and I said,
"I've got good news and bad news."
And the good news is,
I know why people would complain excessively on certain days.
I know the reason for it. The bad news is,
there's nothing you can do about it.
