>> This team and I'm
here with Xiaoying.
She is a Principal PM with the
Customer Service Insights Crew.
She has can be walking us through
the CSI overview with
the key insights,
and providing the
standalone experience,
and then also providing
the embedded experience with the
customer service hub. Xiaoying.
>> Thanks Leslie. Hey
everyone. This is Xiaoying.
I'm so glad to take this
time to walk you through
more deeper into the
Customer Service Insights,
and also to share with you
what's new in this month.
So here you see the Customer
Service Insights demo app.
You can go there by,
go to your address bar and
inputs, csi.ai.dynamics.com.
You are going to see by
default it's loading
a sample environment with
all the sample data
that you can play with.
The goal for this product is
to provide supervisors and
service managers a
better idea and get
deeper actionable insights
into their case data.
What it does is that it will
pull all the cases from
the case entity or
any other custom entities from
Dynamics or CDS database,
and then it will run
through the analytics
and AI pipeline to generate
all of those insights.
The first thing that you
want to do in addition to
just exploring the sample data
is to create your own workspace.
The workspace in the CSI is
a concept that connect to
your own environment and
consists of a set of
reports as you can see
in the left sitemap.
To do that actually is first
through forwards in the header.
The first icon on the right top
is a list of my workspaces.
Here, you can see I've already
created multiple workspace and each
of them will connect
to a CRM environment.
To create a workspace,you just
Click the "Create Workspace" button,
and then go to the Wizard.
It actually only requires two clicks,
one is to choose
Dynamics and then you
can choose your customer
service environment,
and then the pipeline
will kickoff and
provision all the reports for
you for that environment.
Also here you can see for
each of the workspace,
we have two type of the view models.
One is that if you create a
workspace, you are the owner.
As you can see here,
it's simply attached with the owner.
The other is that you can also
share the workspace with the others,
or other people can share
the workspace with you.
For the people who get the workspace
shared by the other people,
you're the viewer
for those workspace.
It means that you
cannot change anything,
you can only see the exact
same analytics numbers
send in charts with the other person.
Here you can see I have a co-worker
share his workspace with me,
so the name is tacked here.
To share the workspace is
also very straight forward.
So just go to click the "Share
button" and then input
your e-mail address.
Like here I can say I want
to share that with Leslie.
Let me try somebody in my
team in this environment.
Yeah, because this is a
separate demo environment,
so not all the Microsoft
people are here, sorry Leslie.
So here I can add another person
into the e-mail address and then
share that workspace with them.
As the owner, you can
also manage all access or
revoke any excess you've already
shared the workspace
with the other people.
This is a very quickly to go through
how to set up the workspace,
and after that once it's set up,
as you see here a set of
reports and what comes the
first is the new homepage.
This is something that
currently in preview.
That's why you see
the pre-attack here,
but we're going to join
pretty soon in this week.
What it does is that,
we heard a lot of customers
complain to us that
although there are a lot of
different products even for
the customer servicing
insights before we provide
a set of very comprehensive
analytics PowerBI reports to them,
but for a lot of
customers supervisors,
it's very hard for
them to quickly get
the idea with so many
overwhelming formation,
what things they want
to put their focus.
So this page is to adjust
that problem and highlight
all the key insights
that we want or we
think will be very important or
interesting for the Supervisors,
or Service Managers
to take a quick look.
In front of this page, once they
identify any topics that
they're interested,
they can drill through
to get more details.
Speaking of that,
here you'll see that
this page shows the topics that,
this way in a simple environment,
we have 177 topics discovered
from all of those cases.
In the topic here is a concept in
Customer Service Insights that
using the natural language
understanding AI model
to automatically group
similar cases together.
So here, 177 topics means we
find across all the cases here,
we can find a 177 group
for each of the group,
they have similar cases.
So falling by that,
this page highlight
the topics to watch
and come with the top three topics
that the Supervisors or Services
Managers may want to
take a look first.
The ranking here is actually
based on the calculation
across different metrics.
Here we can see all the influencer
for the ranking including
the volume drivers,
the negative CSAT impact,
and emerging topics
and new cases, etc.
For the first one, it indicates
the first one why we show that as
the first topic that worth
watching because it's the
number 6 in the volume driver.
Number 18 negative CSAT
impact for the CSAT score.
Also the number 2 as the
emerging topic means that
it's comparing to the
volume changes over time.
It's actually showing this
is something new that
actually is going to
trendy up very quickly.
So that's why I can
suggest that this as the one
worthy taking a look first.
As a user for this report,
you can click the
link for this topic,
which will navigate you to
the more detailed analytics
page for this particular topic.
In this page, it shows
more KPI numbers for
this particular topic talking
about the payment issues,
and we can find the total numbers
and the numbers between the
resolution, time escalation.
How many of them are compliant
regarding to this SLA,
the average time, etc.
Also it calculates the overall impact
of what a CSAT and resolve time.
The impact here is calculated
to see the contribution
for all the cases under
this topic regarding to
a matrix comparing to
all the cases loaded
from the data source.
For example here,
the average CSAT impact
was positive 0.2 percent.
It means that for all the cases under
this topic talking about
the payment issues,
they have the positive 0.2
percent of the impact for
the CSAT score in general.
It means that it helps
the overall average CSAT score
to be up by 0.2 percent.
If it's negative, it means that
it will actually lower
down the CSAT score.
Below that, it provides
more breakdown,
including to provide the chart to
breakdown and the analyze
the drivers across the
channel and product for CSAT,
and also for the resolution time.
From the Supervisor and
Service Manager perspective,
we still calculate for each
of the combination of
a channel product,
how much the impact regarding
to the CSAT scores or
the resolution time.
So the Service Managers and
Supervisors can use that as
a guidance to identify
which one they can go
deeper and to follow up first because
the top impact means that
if the problem for sample
here on the phone channel
and for all cases from
the phone channel,
and talking about the product
for Contoso kids rain boots,
the CSAT actually has
very negative impact to the
overall average CSAT score.
So if the Supervisors can solve
the problem for this channel,
this product first, we
are going to the CSAT,
then the overall CSAT scores will be
higher much faster or
more efficient compared
to the other channels or the
product regarding to the CSAT for
this problem is similar to the
resolution time drivers well.
Also it provides the rankings
across the top agents
for different metrics,
including agents with most active
cases so the Supervisor can see
who are those top agents need
more help to balance
those active cases.
Agents with the
longest resolved time,
it means that those
agents may also need
help or maybe more training as well,
and who can help them.
Also the agent with the lowest
average CSAT also provide a view
for the top regions that need
help regarding to
improving their CSAT.
In this whole dashboard actually,
it also provides the interactive
experience because these reports
is behind the scene is
implemented through the PowerBI.
So all the PowerBI default experience
also applies here as well.
For example, let's say we want to
focus to improve the CSAT
for the phone channel and
the kids rain boots item,
we select this one.
Here we can see, in addition to
the general top rankings across
all the cases in this topic,
now because we just
select one of them,
all the data are filtered
based on this item,
and we can get deeper insights
for different metrics.
For example here, agents
with longest resolved time,
we saw although in general
actually Cat didn't
do very well in spanned most
of the time for all the cases,
but actually, for this
channel products,
he doesn't have any problem for that.
Instead, Tom actually, he's doing
better than Cat and
the other people here,
but actually for this particular one,
certainly he needs more help.
So we can spend more time
to help him regarding to,
how to handle the costumer inquiries
through the phone channel
for this particular product.
They'd be, in this case,
Cat can help Tom to share the
experience and help him to improve.
In addition to these charts
for the topic details,
it also provides this
Sankey chart that shows
the journeys across all the
cases under this topic.
For example, here we
can see the proportion
between different
channels, and certainly,
from this chart, we can
see the phone channels
contains most of the cases
for this particular topic.
It means that a
customer is more likely
to talk about these payment
issues over the phone.
Then we can also get a view about for
all the cases across
multiple channels,
how many of them are escalated?
It seems the frontier
agent did a good job,
then most of them are not escalated,
and then we can see
the proportion between
the resolved and active.
Let's get back, well,
actually, one of the example.
By the way, for the supervisors
and service managers who want to
get better idea about which
cases belong to this topic,
they can also go to the cases list
to review all of those cases.
We'll get there later.
Let's get back to the homepage
first and we'll get
back to that later.
Continue to the new homepage.
In addition to the topics to watch,
it also provides multiple cards
focusing on different
perspective and metrics,
for different interests from
the supervisors or
the service managers.
For example, if the supervisors,
they would like to learn more and
focus more on how to improve
the resolution time,
the resolve time from the team,
we have a cart here to highlight
which topics has the most impact
for the resolution times.
We'll list the top three topic
and shows what is the
shortest and longest,
so the people can compare
between each of the top topics,
and also shows the average.
Still, for each of the topic on
the [inaudible] on each card,
people can [inaudible]
by click the link.
If you're not sure how to use
the information from the cart,
we have the icon here which
you can click through and
see the explanation why
this one is important and
what is the recommendation.
Also, each of the cart is
associated with a [inaudible]
dashboard in general,
and users can also click
this icon to open the
corresponding dashboard.
Like this one for
the resolution time,
we have a resolution dashboard
here to show the general
analytics insights
information about focusing on
the resolution time so that
people can click this link and
go to the resolution time dashboard.
The other card is very
similar to this one.
The topic impacting
CSAT and the peak time.
This one is to help the supervisors
to make the staffing decision.
One of the cards here,
which is slightly
different from the other,
is to provide a more direct,
actionable suggestion for the
supervisors or the service team
to see how they can improve further
about the team's productivity.
Here, the only actionable
insights we have,
suggestion we have today,
is to ask people to
consider automating some of
the topics based on the calculation
from the analytics data.
To automate those topics with
a chat bot like the
Power Virtual Agent.
In the future, the plan is for
the customer service insights,
we're using AI and BI to
provide more and more,
this direct,
actionable suggestions for
supervisors and service managers.
For example, let's take a look,
one of them is use the
customer promo codes,
so people still can click the link,
and drill through that,
and get some idea
about to get all the
information about this topic,
and to review the cases,
and make the final call.
After to that, they can
click the Automate button
because it's a simple dashboard,
so all the action is disabled.
But let me switch to another one
so you can see how that works.
Here, it's in another
workspace I created,
although it doesn't contain
a lot of data to showcase,
but we can click this
Automate button,
which will actually create a topic in
the Power Virtual Agent if you're
also a Power Virtual Agent user.
What it does is that it will
auto-fill the name for
this topic and also pick
the top three most relevant case
titles as the trigger phrases
where the users can start with
and take those trigger phrases,
or add more trigger phrases to
compose a topic for automation.
Also, as I just mentioned
that for the case list,
here, once you create
your own workspace,
that connect to your own environment,
all of those cases are
listed here under the topic,
it's linked back to Dynamics.
Here, I click one of them.
It opens the case forum in Dynamics,
so supervisors can take
a further look about the
details for each of the case.
In addition to that, you can
see for each of the cases,
there are some other actions
you can take for a topic.
Here, the name is auto-generated by
the AI model after
passing through all of
those case titles and identify
the key words from each
of the case title and
compose that automatically.
But if this case topic
title doesn't make sense,
you can also rename that by
simply click the button and
change to the name that
works better for you.
In addition to that,
this action actually, from
the model perspective,
it will take that as
a implicit feedback and
the model will consider
this new topic to influence the topic
clustering next time when the
workspace starts refreshing.
The other is that for the case title,
speaking of feedback, users
can also provide thumbs up,
thumbs down feedback for each of
the case to tell whether a case
should be belong to this topic
or actually shouldn't
belong to this topic.
Those feedback will help to improve
the model clustering
for next time as well.
If the case for
the supervisor believe that it
should belong to another topic,
we can also move the
case to another topic.
Cool. We've spent a lot of time
talking about the topics and cases.
Also, let's quickly go to
the other dashboard so we
can get a better view.
Actually, let me switch to
the sample workspace which
contains more interesting data.
For the following dashboard,
KPI summary, new cases,
customer satisfaction,
and resolutions, it follows
a very similar pattern in
the UI and the layout.
On the left side is the list of
the drivers from the topics
generated through the AI model.
For example, here,
the KPI summary shows
two type of the drivers list.
One is the volume driver.
It shows the top 20 topics that
impacts the volume and also
the emerging topics do
the top 20 topics that shows
a larger volume change.
Those topics here, it focus
on the new topics that
haven't included in the case
volume driver list yet.
On the right side, it shows
a bunch of the charts
and metrics for people to visualize.
On the top of the report,
it also has a consistent list of
the future so people can choose
between different time periods.
Here, today, we support last one day,
seven days, and 30 days,
and then people can also
filter between the product,
channel, business unit,
and assigned team.
All of those values
are generated based on
the data values stored
with customers case data.
At the end, all of these dashboards
are implemented using Power BI.
It also provides the
interactive experience.
For example, if we want
to see more details
regarding to the first
promo code topic,
we can click this topic and
all the dashboard will be
filtered and focused on the
cases under this topic.
The New Cases here is
to help the supervisors
who want to understand better
about the incoming topics.
Let me refresh that again,
interesting I don't know
what's wrong with this chart.
But the New Cases is to
showcase the incoming cases,
if the supervisors want to
focus on the incoming cases
and to make some staffing decisions
based on the incoming traffic.
We found that the most
interesting chart
on the report is the Case Timing,
which shows the average traffic
of the incoming cases by hour.
So from a supervisor perspective,
you can use this chart
to decide across
different channels which time,
which hour is the peak hour that
may need more agents to help.
In addition to that because of
the interactive experience we
can further filter down
like for the promo code,
we can select that
and it will show in
the chart about the peak hours
focusing on this particular topic.
So here we can tell for example
here on the web channel,
actually 1:00 PM is the peak hour,
so it means that for this
particular time or this time range,
we need more agent with
the skills answering
the promo code questions at 1:00 PM.
So here you can see that
we're using this experience,
the Customer Service Insights
not only to provide all of
those analytics and also help
you to understand more details,
what skills that agents need
in order to help people
at the right time.
For the customer satisfaction
is using the CSAT score,
a data attribute from the
case entity by default,
and it does the analytics and
show the top drivers
on this dashboard
regarding to which top
20 topics that has
bigger impact to the
average CSAT score.
In addition to that,
it's still showing
the other metrics numbers regarding
to the CSAT and breakdown
between different channels.
Again, users can use this
interactive experience to go through
and identify if there's
any interesting information
from this topic.
The last dashboard
is the Resolutions,
again it shows the top 20 topic
as the resolution time driver.
So supervisors can use
the Impact Indicator to see
which one they would
like to focus most,
and also using the interactive
experience to identify for
each of the agent who may need
more help for each of the topic.
So for example, if we select
the first agent here,
the topic drivers
list will be updated,
and we can use this list
of the content to compose
a personalized training
material to coach this agent
to do a better job regarding
to resolving the cases.
The Topics page simply provide
a list of all the topics
because the other topic drivers
list only shows the top 20,
but you can find all the topics
generated by the models here and also
sorted by the total cases
and go through that
for each of the topic.
So those are all the dashboards
in the Customer Service Insights,
and in addition to that there
are also multiple settings
that as a workspace owner you can
configure to generate better result.
Let's take a quick look.
So for the settings, we
have the data mapping,
title cleaning, topic granularity,
and you can also export the data.
For the data mapping, it's usually
used for those customers who are not
using a default case
entity and a default data
attribute that we're using in
Customer Service Insights.
So let me switch to my own workspace
because all the actions are disabled
here in sample environment.
To do the data mapping is
also very straightforward,
just click the button
and it will retrieve and
list all the entities in
the connected environments.
Customers can choose any of
the entity here including
the custom entities,
and then go to the
next page to choose
the data attribute that
map to the original ones.
For example here, the case title is
used by default for
the topic clustering,
and if you find there's
any other fields like
the description works better to
describe the problem for a case,
you can also change the mapping
here by selecting this description,
and a similar idea for
the other ones you can
change those mappings.
Then case title cleaning and topic
granularity are also the settings
that help to influence
the topic clustering.
For the case title cleaning,
we have this option because we
find for some of the customers,
their case titles are not
only talking or focusing
on a brief discussion
of the user problems,
instead when they say,
"I see the patterns like this."
So they have those tags and sections
to capture the other information
whether it's escalated,
what is the priority,
or which tier is handling that, etc.
So in this case because
those informations are not really
talking about the problem,
including those information will
actually decrease the quality
for the topic clustering.
So in order to solve that problem,
it's better to use this option to
identify usually where
those sections are,
whether it's before
the summary core part
of the case title
like in this pattern,
or whether it's mostly
in this pattern,
or if you're not sure you
can choose before and after,
the tags of before and
after the initial summary.
So the Customer Service
Insights will do
the smart intelligence to
capture the most important
information from the case title.
Then to provide the diameters,
like here is the
brackets in a pipe here,
so we specify those ones and
then the service will know
which diameters to look at in order
to extract the right information
from the case title.
All of these changes won't
change the original case title,
it's just to tell the
model which part to ignore
when it's processing all
of those case titles for
the topic clustering.
Another setting for topic
is the Topic Granularity,
and here you can see actually we have
an example chart using the simple
data here to showcase the idea.
For the Topic Granularity it
indicates the scope for a topic,
like for some of the business cases
people want to see the
problem at higher level,
in some of the cases
people want to see
very specific issues
under a bigger group.
The Topic Granularity will provide
the five levels of the granularity,
so based on your needs you can
choose either if you want
to see a bigger group,
you can choose that
to be more general,
or if you want smaller topics
to focus on specific issues,
you can choose to be more specific.
The middle one is
coming by default and
actually for most of
the cases the middle
one should be good enough
after a lot of experiment
from our AI team,
but still will provide
this flexibility for
customers based on their
different business needs.
Export Data is a preview
feature today that allows
the customers to export the AI
generated data from the dashboard.
To do that is also
very straightforward,
what it does is to click the
button to generate a URL and
this URL will point to an update,
like a daily refresh CSV file which
contains all the information
including the raw data that
you're going to see
from the dashboard like
the casing formation and also
the topics for each of the case.
So to get more information,
you certainly can click the link
to check out our documentation.
So we've spent a lot of time talking
about and walking through
the CSI external app.
Another thing as Leslie mentioned
earlier in this session
is that for April we
also provide the embedded
report experience
for the customers who are
using Customer Service Hub.
So for those customers after we
released this feature in
the Customer Service Hub,
they don't need to go to
the external experience,
they can just simply get
the reporting experience
from within the CSH that
they're using every day.
To do that, it's also very simple,
so let's go there.
First, it requires a little
bit admin setup and to
install the Customer Service Insights
reports into the
Customer Service Hub.
What you need to do as an
administrator is go to
the Customer Service Hub and click
the "Service Management" area,
and there you're going to find in
sitemap entry settings for
Analytics Insights which is
new with the latest release.
Here, we can choose between
the analytics report,
which is another set
of the BI analytics,
and the insights which contains
all the reports I've just showed you
from the Customer Service Insights.
To do that, you just simply
click "Get Started",
and then it will open another page
to show the progress
of the installation.
>> Xiaoying, are there
any prerequisites for being
able to install this,
or is it just a license?
>> Yeah, it's just a license.
Thanks for bringing it up Leslie.
So for the Customer Service Insights,
it requires a valid Customer
Service Insights license,
but for all the Customer
Service Enterprise customers,
it means that who
have the license for
Dynamics 365 Customer
Service Enterprise,
the CSI license has already
being seeded there.
So for all the Customer
Service Hub customers,
I believe they all have
their Customer Service
Enterprise license at least,
so they'll be able to
use this functionality.
>> Awesome.
>> So here you can see
the installation is in
progress and you can
safely close this page and
come back later from
the settings page to look into
the installation progress.
To save the time, I
don't want to wait here.
I have another environment
that has already set up.
Once the installation is done,
this go to "Settings" button
will light up and it will
help you to go back
to the settings page
where you're going to see
the "Insights" section is
updated was a couple of link.
The first one will go
back to the service area,
which I'm going to show later,
where you can find
the embedded reports.
For now to share the reports
or change any settings,
we haven't done that yet in
the "Customer Service" hub
with this preview release,
so we provide a link here which
will point you directly to
the share module in the "Customer
Service Insights" standalone app.
You can share the
reports here and then
the people again going back
with the embedded reports.
It's the same for the
settings as well,
it will point to the standalone
app settings dialogue.
Once the setting changes,
it will impact the embedded report
in "Customer Service" hub as well.
So in general, once the report
is created and set up in
the "Customer Service" hub for the
same user they are going to see
this workspace is created in
the standalone app as well.
It's in sync.
Another thing to mention is
that the "Customer Service
Insights" today is created per user.
So even if after the admin has
installed the "Customer Service
Insights" and also
with the installation,
it automatically creates a
workspace for the administrator.
The admins do need to share
the workspace with the
reports to other people
in order for the other folks in
a team to look into the reports.
Otherwise, for the other people,
if the reports have
been shared with them,
they are going to see
and experience with
one button on the page to provision
their own workspace
and also for the data.
So here you can see
once it's installed,
there is a section for "Analytics
Insights" and the report
is enabled by the first
section in the page.
This is the analytics reporting for
this item and we can
take a quick look.
This one contains
all the BI analytics
and it shows the summary
between the case and agent.
Those BI analytics report
contains even more,
a bunch of the BI
analytics charts and KPIs.
Then the insights reports are
consistent with what I just
showed you earlier in
the standalone app.
Another thing to mention
is that speaking
of all the data in the "Customer
Service Insights," because
it's the per-user model,
each of the user can create
their own workspace without
the administrator to do
the first round setup.
For each of the user,
once they create the workspace,
that data they are going to see is
based on their security role for
the access to the CDS entity.
For example, if user A
can see case A record in
entity but not case B,
then this report for that
person will only show
the data like analytics generated
from the case A but not case B.
Here you can see the
corresponding [inaudible]
that's exactly the same
way as what we just saw in
the standalone app and also
there the button for a user
to share and change the settings
or open the standalone.
So all of those buttons will
open the standalone app in
the other web browser to make
the sharing or settings changes.
I think that's the all of it.
Leslie, do you have
any other questions
or I'll let you to take it over?
>> Actually, I do have one question.
From a traffic standpoint,
is there a minimum
that you need to have
in order to start
gathering this data?
>> For generating the reports,
there is no minimum.
In order to generate
more useful topics it
still depends on the data,
but usually you'll find if you
have more than 500 topics,
then it will start generating
more useful topics.
Because usually more data means that
there can be more similar cases
that can be grouped into a topic.
Also speaking about that for
each of the topic the definition
is that the AI model will
group similar cases.
This AI will group similar cases into
different groups and
for each of the group,
if it contains three
cases or more cases,
then that will be
considered as a topic.
So for each of the topic,
you will find at least three
similar cases or more.
>> That can start trends, right?
>> Right, yeah.
>> Awesome.
>> Also related to that limit,
although it won't impact,
there's no minimum limit.
But for the "Customer Service
Insights," actually,
there's an up limit.
For now, we only load the case data
created within the past 60 days
or under one million as the up
limit, whichever hit first.
So for example, if you have fewer
than one million of cases
created in the past 60 days,
then "Customer Service
Insights" will load
all the cases created
from the past 60 days.
But if you have more than one million
of the case is created
in the past 60 days,
then we just only load the
first one million of the cases.
>> That's helpful to know.
What do you do with the rest of them,
do you archive those?
>> So for the rest of
the cases we don't
load that to generate
the analytics today,
but we're still open
for the feedback.
Actually, we've already heard some
customers feedback regarding to
the up limit and we're looking for
the improvements in the future.
>> Well, Xiaoying, thank you
so much for your time and
providing this rich
information and deep dive.
We really appreciate it.
If those that are viewing this
have any questions or if there's
an area that you would
like us to go into
a deeper dive, please comment below.
We're happy to come back and revisit
a certain area and
go into more detail.
Xiaoying, do you have anything else?
>> Nope. So for any of
you if you're interested
in trying the "Customer
Service Insights," please go to
aka.ms/csinsights to get more
information and get started,.
>> Thank you. With that,
we will be back with
more updates when we hit our next
release. Thank you, everyone.
>> Thank you.
