Welcome to "This is My Architecture"
I am Kiriyama from Amazon Web Service
Today, we have Mr. Mizutani from Cookpad
Thank you for coming today
Thank you
First, tell us about Cookpad
Cookpad is a platform for sharing
cooking recipes and ideas
We have around 100 million users 
around the globe
So protecting the information of these users is
extremely important for us
My work involves protecting user information
and protecting this architecture itself
So your role is to protect 
the Cookpad service and the security of the users
Yes, exactly
Could you explain what you mean
by that this architecture is 
a tool to accomplish the purpose?
It was structured to monitor security logs
It was structured to help 
those who monitor security,
or those who respond to incidences
For instance, a member of 
the internal security operation center
uses this architecture
So this solution was made 
for the security members
How does it work?
This architecture consists of three parts
The first is where the log is collected
Next, the log is retained, then analyzed 
These are the three parts
Regarding the first log collection
It is equipped with various functions 
for log collection
From the EC2 instance, 
Fluentd collects syslog messages
and saves them in S3 via Firehose
For several managed services like Cloudtrail,
logs can be sent directly to S3
So logs are saved by using these functions
From third party services, 
logs are collected using Lambda
Then logs are saved in S3 via here
I can see that different types of logs are handled
What about the second part?
The second part is  log retention
All logs are saved in S3
Using S3's life cycle policy,
we control the entire log retention period
All logs are stored in S3
and managed by the S3's life cycle policy
What about the last part?
The last part is the log analysis
The short term objective is
to find alerts in logs and notify them
using lambda function
For alerts that we find,
we notify Slack
However, as a mid- to long-term objective,
in order to be able to search the log
Lambda function sends the log to Graylog
Graylog is an open source log search engine
It uses Amazon Elasticsearch Service 
as a back-end.
The ultimate goal is to make it possible 
to do long-term search for these logs
It is equipped with functions for auditing
This is made possible by Amazon Athena
Thank you
I was able to understood the big picture
Of these, which part is most innovative?
Security engineers need to respond 
to alerts as quickly t as possible
Based on SLA in a security service 
that is managed generally,
Regarding the time to process the log,
we want to keep it around one to two minutes
However, if a system that 
processes logs real-time
tries to build a system 
to handle a significant amount of logs,
it tends to get extremely costly
But this architecture has 
the characteristic of achieving both
the low cost and decreased latency period, 
or log processing time.
You mentioned low cost
Where does the cost merit lie in this architecture?
This architecture alone handles 
around 300 million logs per day
This is equivalent to 270GB
The cost of the Lambda function 
that handles these logs
is kept as low as about two dollars per day
Around two dollars per day is incredible
Compared to the general log monitoring solutions,
what is exceptional about this solution?
Regarding general log management tools,
in general, before collecting logs
a log schema needs to be defined
But for this architecture, 
in the log collection part,
we can collect without defining the log schema
This is because when S3 saves the log
it can save it without concerning schemas
By not having a schema, 
various logs are easier to handle
Finally, are there any plans
for future expansion of functions?
Yes
As we have collected security logs,
we want to create 
a system that detects abnormalities
using Amazon SageMaker
That is amazing
Thank you for sharing
Thank you so much
Today we had Mr. Mizutani from Cookpad
discuss security monitoring solutions
Thank you for watching 
"This is My Architecture"
