In this tutorial we want to explore the possibilities
to create scatter plot charts using Vega visualizations
that has been introduced in Kibana 6.2.
If you are running and older version of Kibana
you can checkout the link in the description
to Yuri's standalone Vega visualization plugin.
We want to visualize requests from a request
log.
In each request there is also a bytes and
an extension field, that we will use later
on.
To create a new Vega visualization just select
the Vega visualization type from the "Other"
categories when creating a new visualization.
Vega visualizations are using the Vega visualization
grammar to express how a graph should be drawn.
Let's start with a fresh configuration that
only has the schema definition that any Vega
visualization needs to have.
Since we want to draw a scatter plot chart
we should specify "mark: point", so that each
document will be drawn as an individual point.
Next we need to tell Vega where to load its
data from.
Therefore we will use the "data" configuration
and specify an "url" object.
If we specify an object to the "url" parameter,
we will actually load data from Elasticsearch
and be able to configure the request done
to Elasticsearch here.
By specifying "%context%: true" we'll tell
the Vega visualization to take the filters
from the dashboard it's embedded into account.
With "%timefield%" we can specify the field
name of the main time field of our documents.
In the rest of the "url" object we'll configure
the request, that will be executed against
the "_search" API of Elasticsearch.
First we specify the index.
The "body" object will be send as the request
body to Elasticsearch and can contain any
valid Elasticsearch Query DSL syntax.
We could use aggregations here, but for a
scatter plot chart we want to visualize individual
documents, so we just set the "size" parameter
to 10000 to get as many documents as we can
and we can use the "_source" parameter to
tell Elasticsearch in which field of the documents
we are interested, so it won't return the
full source document.
We specify only the "@timestamp", "bytes"
and "extension" field here, so we can draw
a scatter plot over the time on the x-axis,
with the bytes of each request drawn on the
y-axis.
To distinguish between different extension
types, let's give each extension a specific
color and shape.
We need to tell Vega now, where in the response
the actual array of data points can be found.
Therefore we'll use the "format.property"
key and set it to "hits.hits", which is the
path within an Elasticsearch response, that
contains the array of documents.
In our example the timestamp field would return
values in a string format, which can't be
properly used by Vega for time dimensions.
Vega allows us to transform each document
before visualizing it.
The transform array can contain a list of
transformations.
In our case we just need one.
We use the "toDate" method on the @timestamp
field to convert its string value into a proper
time value for Vega.
Using the "as"key we can specify under which
name we later want to access this value.
In this case we give it the name "time".
The "encoding" key now holds the actual information
how we want to encode this data in the chart.
Let's first configure the x-axis.
We want the previously calculated time field
to be drawn on it, so we specify "field: time"
to use it.
We set the type of the axis to "temporal"
to tell Vega, that it should be handled as
a time axis.
Just to show some more configuration options,
let's also disable the title below the x-axis.
Next we need to configure the y-axis.
As mentioned earlier, we want to draw the
actual transferred bytes on the y-axis, so
we specify the "_source.bytes" field and we
set the type to "quantitative".
We can also specify an axis label in our case
"Transferred bytes".
We are now ready to draw the first version
of our scatter plot chart.
Be aware since we are drawing individual documents
this might take a few seconds to draw.
And voilà ... our first scatter plot chart
has rendered.
To make the chart a little bit more interesting,
let's give each request and individual color
and shape depending on its extension, that
was requested.
Let's make the "color" dependent on the _source.extension
field and specify a type of "nominal".
If you want to color data points depending
on a numeric value, you could also use a type
of "quantitative" to get a color scale instead
of distinct colors.
And voilà and even more colorful scatter
plot chart.
By default Vega will use the field name as
a label for the legend.
We can easily change this by specifying a
different legend title.
Last but not least, let's also use shapes
besides colors to distinguish extensions from
each other.
I hope you had fun learning how you can use
Vega visualizations to configure scatter plot
charts in Kibana.
You can find a link in the description to
a blog post containing the configuration for
this chart.
See you the next time.
