OK. Thank you so much, everyone, who
has joined us. My I am
based in Nairobi. I am part of the AfricaR
leadership team and we are so grateful you
have joined us today. We are facilitating
this tutorial as AfricaR. If you have been
following the useR workshops you will notice
different groups have been facil theying tutorials
and we are so glad North Africa is facilitating
this. It is Egypt R and Nigeria and we are
grateful for this opportunity. We are going
to be speaking or rather going to be hearing
about Periscope and CanvasXpress, creating
an enterprise data visualization. The Periscope
tool is a kit built to address the need of
developers to enterprise grid Shiny application.
We are going to here from a team from Aggregrate
Genius. And I will pass it over to the leader
of the team for further introductions.
>> Thank you so much for having us here. We
really appreciate it. I will just start with
the housekeeping items and then we can move
on. I want to remind everybody that there
will be a large number of people on the phone
lines. Keep yourself muted unless you are
asking a question directly. We have a whole
team of people helping with this and I really
appreciate all their help. Omnia Safaan is
the person to direct questions you have during
the session that you would want to have answered.
We will try to answer as many of them live
as possible. I am not directly monitoring
questions or I wouldn't get the presentation
done. Please send the questions there. We
have a closed captioning there thanks to useR.
If you would like to view the captions, you
can turn them on in Zoom with the closed captioning
button. The tutorial is being recorded and
shared and edited so the live pieces are not
a part of the recording that is distributed.
If that makes you more comfortable interacting
during that part, feel free, but you will
have this for your reference. I will introduce
myself. My name is Connie. I am the CEO of
aggregate Genius. We are on the West Coast
of candidate and completely remote. My background
is a chemist originally. We run a small company
that provides enterprise-grade Shiny services
as well as Full-Stack analytics development.
We have a couple of our colleagues here on
the call to help us out. Todd Brett and Kaelen.
They are fielding questions behind the scene
and I appreciate them being here and I appreciate
the North Africa team for sponsoring this
is supporting us. We will get started right
away. I have a lot of content but not a lot
of slides so you will be thrilled, hopefully,
that most of this is interactive. We are going
to create an enterprise-grade big data application
in a few hours. Luckily, it won't take a whole
day. This has been sponsored by useR 2020
and was going to be a live workshop. Although
I can't meet you in person as intended due
to COVID good to meet you all and I look forward
to interacting with you during the session.
I will get started. I wanted to touch briefly
on the problem domain. Enterprise-grade Shiny
applications. Why a Shiny application? It
is an invaluable resource in addition to charts
and static records on analysis and datasets.
Our world is becoming more visual and it is
a nice way to see what you have done when
doing an analysis and let your users interact
with it it. From an enterprise perspective
-- what does that mean? When you are in a
big company there is competing interests and
they want to make everybody happy. They need
to keep their employees happy, stakeholders
happy and they still need to make money but
there is kind of some issues you have take
something like a Shiny application and bring
it into an enterprise. When you are an enterprise
in a big company, you have to have applications
that if they are going to be used by a large
number of users or your customers, they have
to be scalable, performant, safe and consistent.
This is your face to your customers and stakeholders.
They need to be professional. We are used
to applications developed by developers, people
are Computer Science Degrees, user interface
degrees, human interaction degrees that do
studies and know how to build applications
that meet a lot of these enterprise needs.
But Shiny application are developed by analysts.
Largely, people focused on science and analytics
and insights and results. We want to get those
numbers out. And the charts and show people
the cool stuff we have done with all of the
great algorithms we have. We can do that with
a Shiny application. But, you know, and this
works fairly well and we can put some really
great applications together, but we also have
another problem that has crepted up and that
is datasets get bigger and bigger which is
fantastic but we are relying more and more
on visualizations. Not just applications but
visualizations and big visualizations and
big data. We are going to kind of address
both of those issues today to some extent.
I am not providing you with a pansea of all
your problems but hopefully I can help you
get started. I get asked what does a Shiny
enterprise app look like? I have seen Shiny
apps and apps on the web but what does it
mean to be an enterprise app? Give me an example.
What makes it an enterprise app beyond the
fact a company sponsor it? Usually, enterprise
app have complications. Usually the dataset
is really big. I will show you an application
where the dataset is very wide and very tall.
The users want to do things like filter and
segbrate on both rows and columns. They want
to do interactive segregation and handle datasets
from different sources and those datasets
if they are from different sources an enterprise
they usually have different results and formats.
It is one of those you are bringing a lot
of fruit together to make a fruit basket and
not bringing oranges together to make oranges.
Then of course you have the access of non
developers using the app. And quality assurance
and things like that. The app I am showing
to show you is called single cell viewer.
It is out there in public. This application
needs to visualize a large number of interactions
on the charts and has a wide variety and number
of charts. It is also allows the user to do
a small but very heavy duty analysis. There
is something called differential expression
in the biological world and this allows the
user to do differential expression analysis
which is unusual because on the fly analysis
on a large dataset can be difficult because
the user has to wait for it. You have to balance
the time it takes to get the analysis back
versus the user experience on the front end.
And then of course there is that balance I
was mentioning. That performance. Not only
with something like an interactive analysis
but you have to balance performance of your
charts. The user may be willing to wait a
few seconds for a chart, in fact, we will
show that today but they also are not willing
to wait five minutes for a chart to show up
generally. This is kind of, I wanted to show
you what an enterprise Shiny app looks like
before getting started. This is that single
cell viewer. I will include the link to this
in the resource slide. You can go on and try
it. There is a version that is hosted on a
smaller server. You can see on this plot what
you are looking at is a dataset composed of
cells and genes. There is a lot of charts.
You can imagine this dataset has about 20,000
cells and each cell has quite a large number
of genes. And so you can see that there is
a large number of charts with a large number
of points on the charts. That's kind of what
I wanted to show you. This is the interactive
analysis of the app. You can see it coming
back in this table below. . This is what I
mean when I say an enterprise Shiny application.
This is used by internal and external stakeholders
and used by a large number of people at a
time so of course there are deployment concerns.
The idea is to give you a jump start on making
an application that can visualize large datasets
as well as, you know, gave you a framework
for building an application that could become
this big. Once you see the framework and how
it is used I think you will use it for everything.
I love it and I did develop it so I guess
I am a little bias there. One of the things
this session will do is probably introduce
you to a charting package you may not be familiar
with. It is called CanvasXpress. I want to
provide you a good option for visualizing
large number of points. CanvasXpress was developed
in 2009 by Isaac new house at bristol myers
Squibb. One of the packages didn't scale.
It was released as an open source in 2011.
It is an R package and it has been around
for awhile. We are going on six years on this
one. The package itself, the core of this
package is written in JavaScript. The package
gives you live plots by design. By design
it wants the useer to interact with the plots
once you make them. It is not a charting package
where you make a static chart and then show
it to the user. You send the data to the web
page and it builds the chart and it can do
a lot of things on that chart but you don't
have to write as developer. Things that users
expect nowadays I would say five years ago
they didn't expect this. There are a wide
variety of charts. It is not a limited package.
It is very expansive and new charts and new
chart types are being added to it continually.
Just giving you an idea of what is there to
look at. And in general, this package provides
much better scaling than packages based on
SVG graphics. You can think of SVG graphics
in packages like D3 and Plotly. It works differently
kind the scenes. For example, one million
point heat map in Plotly takes about 50 seconds
to render. In CanvasXpress it takes about
8 seconds. Now that still is a significant
amount of time when you are looking at an
application but you also have to remember
you have to build in time not only for that
50 or 8 second but sending the chart over
the internet and the data points and things
like that. Kind of gives you know an idea
of what does a one million point data heat
map look like? Like this. We will build a
big chart with some nice size data for this
tutorial. I hope you enjoy learning a new
package and have somewhere to go if you have
large datasets you want to visualize. The
other reason that CanvasXpress was created
and has been focused on were the last couple
of yours to add features is the reproducible
research. As you might imagine, at an enterprise
and especially a pharmaceutical company, being
able to produce your research, the scientific
principle of reducibility is really, really
important. These charts actually build in
full audit capability. The data, configuration
and every modication of the chart can be captured
so when somebody saves the chart somebody
else can go back and customize it, see what
somebody did to change it and I will show
you how to do that. That's really, really
nice in an image. It is all captured in the
PMG. If you are somewhere where you do research,
doing science, you are doing analysis, chances
are you probably want to know when your user
says I got this crazy chart from your data
where it came from and what did they do and
what did they change and how did it get there
from what you originally showed in the application.
When you provide users with that ability to
customize their charts you will get the questions
that are hard to answer like wait I got this
chart and you look at it and like wow. Wonder
how you got that. Needless to say this gives
you the capability to go back and say you
change this and set this setting and changed
that and filtered your data this way. Yes,
you can filter it right on the chart. So,
we are going to get to the introductive portion
soon. I want today give you background on
CanvasXpress visualizations. Since it was
designed for very large datasets in the genomics
field, it applies to any field but the data
format is tabular but three dimensional. Think
of a data table but a data table with extra
information. It actually utilizes both the
column and row names to cross reference the
datasets. You will see this when we live code.
The charts are built using named options which
I will show you. The order is your personal
preference or style. Luckily, there is a ton
of extra information at the website. CanvasXpress.org.
There is examples there. Since this is a JavaScript
package, you have write Java application or
Python applications and want to bring in the
JavaScript library separately it is not limited
to R. If you have pipelines that are hybrid
at your enterprise, you use R for research
and analysis and also use it for, you know,
and use something different to build your
enterprise applications which is still fairly
common. You can still build these charts without
R but you can build them in R. That's great
but you will see this website is super rich.
Full API documentation. It has a ton of examples
and each example has JavaScript, JSON and
R code that goes with it. You can just grab
it right out of the web page and drop it right
into RStudio and create the same chart and
explore the data. That's really, really helpful
resource that I kind of point you to. There
is a couple of terms since the data format
is three dimensional and tabular and uses
row and columns. Variables are the rows. Row
names as the variable names. Samples are the
columns. If you are not a biologist that takes
time to get used to. Annotations is extra
information. You can add extra information
about the rows, variables or columns or samples
of the dataset. For instance, maybe your variables
have additional information. We will talk
a little bit about why it is done this way
when we get into the code. You can either
have annotations or not. That extra or extended
information is up to your data. And we are
at the point of one of our first live coding.
I would like to step out of the presentation
mode and interactive live with you guys and
show you how to create a CanvasXpress visualization
and give you a tour of what the visualizations
have to offer. Before I jump into that, I
want to ask our organizers if there are questions
I should answer at this point.
>> No questions that this point.
>> Connie: Thank you very much. Let me stop
my sharing and I am going to reshare. I will
also pause my video so that I can share my
whole screen. Is there a setting you can set
that will let me share my whole desktop?
>> I am not sure. What are you trying to do?
>> I would like to share my whole desktop.
Sometimes the Zoom that is allowed or not
allowed and it is not allowing me to share
my desktop.
>> Sorry. I don't know about that. Let me
see. I hate to have to switch back and forth
because this will pop-up in a new window.
For now we can work in our studio but will
you look during it?
>> Yeah, let me look at it.
>> I am going to share my RStudio. OK. Here
we go. If you are coding along, I suggest
you go ahead and create a new project. We
will go ahead and make a new project. I always
work in a project and I will kind of walk
you through a little bit of my thought process.
Just to give you a heads up, if you are a
new Shiny developer or not that experienced,
maybe some best practice would make sense.
Usually when I am working on a project, I
also connect it up to GitHub. Let's do this.
The nice thing about RStudio projects is they
set your working directory right into your
project directory. Everything is relative
to that directory. Let's make ourselves a
new script. We will just work in R to start
with. Bring in our libraries. And we are going
to use the data package to get our data for
this tutorial. And we will use dplyr. If you
look at the package itself you will find there
are datasets available. We will take a look
at each of these. What I will do is -- if
you wonder where it came from I will leave
this up here but you can refer since we have
brought this package into your environment,
we can refer to this dataset name as gse.
Jebe gene set expression dataset and the attribution
and information where it comes from is available
in the help from CanvasXpress.data. You can
always take a look at the dataset and where
it came from the papers and things like that.
It is basically a cervical cancer gene expression
dataset. Let's take a look at it. There are
three pieces that are exposed in this package.
Three datasets similar to information that
we might get in an enterprise. Is my screen
and the sizing OK?
>> Yes.
>> OK. Fantastic. I just want to make sure
that everybody can see what I am doing here.
I am going to go ahead and view this dataset.
This dataset is the main gene expression dataset.
You will notice the samples and columns and
they are named and they are actually cells.
The rows here are named and they gene references.
That's the first dataset. We are going to
view the other datasets too. For each of those
genes, of course, there is gene information,
so you can see here for this gene name, there
is a symbol which is the more usual way that
you would see a gene, and there is the description.
So this gives you the information on the genes.
And then let's take a look at the third dataset.
And this one is actually, probably, the most
interesting one but it and it is the smallest
one. This is the sample annotation. This,
for each cell, it tells you a sample type,
a cell type, a stage of cancer because this
is a cancer dataset, the age of the person
is exposed in groups, and their HPV status.
Human papilloma virus. We have missing data
in here. This is typical of bilogical data.
This is the annotation on the samples. The
extra information on each of these samples
and the extra information on the genes. And
I am going to stop and see if there are questions
on the dataset before I get any further. Anybody?
Nope? OK. I will expect that you will be quick
on the draw Omnia.
>> Sorry. A question. Someone is asking about
your Twitter handle.
>> Oh, my Twitter handle. We have a Twitter
account. [Laughter] It is @aggregategenius.
I will share details and anybody who wants
to connect with me on LinkedIn can because
that's the one I mainly have time for. I don't
have a terribly active Twitter handle. These
three datasets, one question I get often is
why not just put this all in one data frame?
You could take this expression data and instead
of having these gene references you could
add the gene details as a set of columns and
add the sample notations as a set of rows.
Well, the reason is that does not scale. So
when you are working with a dataset, let me
show you the dimensions here. The dimensions
of the expression dataset. This dataset is
about 1.5 million items which is only 22,000
rows by 66 columns. That is one 1.5 million
data points. This is set as a matrix. It numeric
matrix. Not a mixed type matrix. That's really
important when you think about performance.
If I add a column to this matrix that has
non-numeric data in it it suddenly becomes
a data frame which is under the hood a list
of lists and a list of different kinds of
lists because some fields are numeric and
some are character. That becomes much slower
for pretty much any analysis to handle whereas
a numeric matrix we have made very efficient
ways to deal with all numeric matrices for
a long time. This is why it is separated this
way and why we are going to use the datasets
like this to chart. You will find that, in
general, if you work with big datasets, you
will want to keep your numeric data separate.
You can have sparse matrices that are numeric
but you will want to find a way for them to
be separate from the character data. The character
data is super important. That's what tells
us the information but you want to be able
to process it in a reasonable time. These
are the datasets we are going to use. I will
leave them up here so we can refer to them.
Let's just make a simple bar chart. I am going
to go ahead and take our sample annotation
because that's the one that is most humanly
readable and understandable. I am going to
group it by age because, you know, let's see
what the age groups look like and maybe let's
see how many samples we have in there. I will
start with this. And you can see that I get
a tibal down here because I am using the dplyr
tools and I have our age groups and these
are the number of samples or cells, right,
that we have in each of these groups. I am
going to change it into a wide data frame.
Like I mentioned with CanvasXpress, you do
need to think about what are your samples
and what are your variables. Our variable
here is frequency. It needs to be in a row
and our samples, each of these are age categories.
You can think of them as samples. We want
to transpose this. You have to love the dplyr
-- well, this is actually the McGrider pipe.
This came before dplyr but you can pipe, if
you are not aware of this, you can pipe anything
that comes out using this pipe into other
functions. I am going to pipe it into transpose
and turn it back into a regular data frame
and I will tell you why in a minute. Here
we go. Now we have a wide data frame where
we have age and frequency are rows and the
columns don't have names yet but we can see
our categories, samples are going to be the
age. Let's go ahead and fix the column names
and assign this to a variable. OK. We will
go ahead and fix the column names. Waiting
for that function I am pipe in for column
names. I don't know what time it is in your
part of the world. I think we have people
here from every continent but it is morning
so I am still working through my coffee and
my typeability. I am going to set the column
names and get rid of that age row and that
way we clean that up. We can look at the column
name version, obviously, then we have a duplicate
row because we don't need that age know any
more. What I will do is I will just go ahead
and remove it. Now we have a great dataset.
Very small. Very, very small. We are starting
small. Don't worry. Let's make a chart. It
is super easy to make a chart so we are going
to send canvas express our data and tell it
what kind of graph we want. I just want to
see a bar chart. And there we go. We have
a nice bar chart. I will show you how to customize
this in our next hands-on but what I wanted
to do was show you the basic functions on
the chart. We have a chart that we would expect
from, you know, gg plot or whatever your normal
favorite package might be but if you are new
to CanvasXpress let me show you some of the
cool things on this chart. This is an HTML
widget and CanvasXpress exposes a lot of things.
We already have tool tips. You don't have
to worry about hovering or set handlers to
deal with them. We have the ability to export
it as a PMG. If you push this button, the
camera, it will save it as a PMG on your computer.
We can resize it and drag the edge and make
it smaller or bigger. This is not a function
of the viewer. This is whether you open it
out into a web page or not. If you push the
button it works exactly the same way and all
of the charts on the web work the same way.
You can view your data table. Hey, you can
save the data. When you view the data table,
you can filter the data, you can transpose
it, you want to look at it differently, sometimes
it is easier to look at it different ways.
So there is a lot of really good built-in
functionality. If you right-click on this
you will find there is a ton of other things.
You can save it as JSON. You can edit it in
JavaScript if you are a JavaScript guru. You
can show the R code or the metadata. I will
talk about reproduce and replay in a minute.
You can download it here. You can, you know,
do a lot of things to this chart. So, this
is worth exploring and your users can do it.
Your users can change the axis, they can change
a title, they can change what it looks like.
So they can add a correlation line to a scatter
plot. This one doesn't make sense to do that
so only some of the options only apply to
certain chart types. You can resort the data.
You can switch out some of the data. This
is really, really powerful for your end users
especially those scientific users. There is
becoming more of a line between business and
science. There is less of a line. There is
a blur. Your business users have become much
more adapt at analytics and they want to do
these things and transform the data and customize
it. What if they just don't like pink? You
know, I mean that can be a huge problem for
a static report. Somebody is coming back to
you I want it in green. I need it this big.
Can I have my axis set at 0? I need a different
title. If you have a static report and chart
you are rerunning things all of the time to
provide those customizations. This provides
you with a lot of that on-chart functionality
for your dataset and for your users. I am
going to get more into that as we customize
the charts. But I need to pop out to the slides
briefly. I want to take questions at this
point because this is the start. I would suggest
you save this filled. We will build the charts
we put in our app during this live sessions
and drop them into the app. You might as well
save your file and in your project and that
way, you know, you can use it later drop it
into your app. Any questions?
>> I did want to follow up. I cannot figure
out why you can't share your desktop. You
absolutely should be able to but for some
reason you can't.
>> No worries.
>> We have one question with regard to menu
translation. You were showing menus earlier.
>> Yes. What is the question?
>> Can the menus be translated? I am presuming
to another language.
>> That's a really good question. I have not
been asked that before. I am assuming you
mean the on chart menu. I will ask that and
answer it in the frequently asked questions
when that is over. I will send that out as
an answer. Todd, will you please write that
down? I will ask Isaac about that. In my mind,
that should be possible but I don't know if
it is currently part of the capability. OK.
I am going to stop my sharing and come back
to the dreaded slides. Are you seeing my slides?
>> Not yet.
>> There we go. OK. You see the slides. There
is a little lag there. We have seen a basic
chart. You know, not very exciting at this
point. Bar chart, it's pink, it has some numbers
on it. But we have seen the dataset and what
I wanted to do was show you kind of -- do
a quick rundown of the most commonly used
options and that's more for your reference
and you will see them used. We will go back
and customize that chart and make it look
awesome and make some more charts because
we certainly don't just want to have a bar
chart in our application. You have to get
your data into the chart. That's put in with
three different variables. Data which is the
main dataset of the application and smpAnnot
and bar an varAnnot. There is a ton of visualizations
and charts you can make. The graphOrientation.
Do you want it horizontal or vertical? Things
you always need to put in is title, subTitle.
Tell your user what they are looking at. The
theme. There is about 10 predefined themes.
If you don't want to have to set all of the
different colors and stuff each time you can
use one of the predefined themes. Datapointsize.
Transparency is always a big one. There is
a special option that's called scaleFontFactor
and then you can move your legend around.
It is always a good idea to have a legend.
There is more. There is a saveFilename. When
the user downloads your chart it should have
a good file name. The program and the package
will assign a file name but honestly, we were
using that and we will set a custom file name
for ourselves but the chart we just made,
it makes more sense to have that be, you know,
frequency_age or maybe the name of the dataset_frequency.
It makes sense oo to the user so they don't
have all these downloads and don't know what
they mean. Really quick on the slides and
we will finalize this chart. After this hands-on
we will have a quick break. Let's go make
all of our charts. I will show you some of
the more advances features and where to go
and look for help, and that. I am going to
stop sharing and reshare. We are back where
we started from. I am going to share quickly.
I want to show you on the website where to
go to look for help. If we go to CanvasXpress,
this is the website. Are you seeing the website?
This is the CanvasXpress main website. If
you hit the home page it looks like this.
There are a ton of examples. The biggest place
that I tend to go, even I do, and I maintain
the CanvasXpress R package but I go out and
look at the examples. They are by chart type.
If you want a contour chart, a dot plot, facet
chart, right? You are looking at all of the
different chart types and they have a ton
of examples. Usually, you can find an example
that is similar. Maybe you want to label a
point like this. This is called a decoration.
You can decorate your chart. And you would
find a chart and then you can view the R code
for it here with this button or scroll down
here. It will talk about the dataset, the
configuration, and here is the R code for
this chart. You will see the decorations and
how to add addition things like a citation,
how to set the different axis. This is a very,
very information rich place to go and look
for help when you are going to create a new
chart. And you will see here, obviously, these
are same functionalities we have on our charts.
The functionality shown in the chart depends
on the type. For this chart there is a lasue
and that's to allow you to select points without
a scare. I just selected these three points.
But, you know, there are tons of examples.
This isn't just one scatter2D. You can go
see other plots that are richer and based
on different datasets. This is the place where
I would send people to go for help and this
is where I go for help. I don't remember everything
about the package. There is a documentation
session as well. And then there is the APIs.
This will walk you through some of the documentation
and then there is the API. The API itself
is all of the options that are generally exposed.
If you are looking for I need to set that
save file name, how do I do that? Which charts
is it used in? This is the option name and
then you would set it to a value, a string
value. Maybe you need to tag your chart. Maybe
you want to change the team. Here is the themes.
And we will do this. I wanted to show you
the website just briefly so that you know
where to go on the website for help. The website
is the latest and greatest version. The R
package trails behind just slightly. Our team
maintains the R package and werun tests on
it and make sure everything is still working
and put out those updates. That is, you know,
and you are welcome to check out the package
yourself and contribute if you find the time
to. This is where you can go for had help.
Let's make some more interesting charts. Let's
go ahead and customize a bar chart and make
a better chart. Let's clear this off. Some
tips for using RStudio are larger datasets
is to make sure you don't build up a ton of
plots especially if they are really big in
the viewer. You don't want to crash your session.
I would just suggest if you are done with
charts you clear them out and start again.
We have our base chart. We made this chart.
This is not the chart I would want to show
to my users. I like vertical bar charts so
let's change it to vertical. And then you
will notice, OK, great. I have got a vertical
chart. But I don't like those labels, so I
am walking you through my customization pipeline.
I start with get a chart and then I make it
look how I want. I am going to rotate those
labels 45 degrees. You will notice these are
the samples. Remember, we talked about these
are our categories, these are our columns,
and so this is the sample label and we will
rotate it 45. Looks great. Let's turn that
legend off because I am not actually showing
anything exciting in that legend. This is
a frequency chart. And I am going to stick
a title on. All charts should have a title.
Let's take a look at it. Great. I have got
my title. A couple other things, let's add
a subtitle. And this is to show you how to
make it smaller if you like. Let's give the
user some idea of where this came from. And
I am going to show you. The subtitles is the
font size as the title but for this chart
let's make it smaller. Subtitle is our item.
And I am going to do the scale -- if I can
type -- scalefontfactor is equal to half the
size. This is a relative size. You can say
cut it down to 50%, right? And then I don't
particularly like pink in my charts so I am
going to change the theme. Just to show you
what the different themes look like you can
explore them on the web. It let's you change
the theme on all of the charts. I like this
one for this type of chart. I don't know how
to pronounce it. Paultol. It looks like it
belongs in a magazine or on the news. You
can see it changed the align on the title
and subtitle. I like the blue. I am going
to make this a reasonably robust chart. One
of the things I noticed when I started the
chart is it starts at six and goes to 21.
I don't want to mislead my users on the height
of these bars so I am going to going to show
the 0 value on that axis. I am setting the
minimum X to be 0. This is a more comparable
chart. Either way. I just wanted to show that
you have all that capability.
Now we have a customized chart. I am going
to assign it it to an object just to show
you what it looks like. This is -- I mentioned
this is an HTML widget. The plot itself is
just a list. All HTML widgets look like this.
They have hooks and X variable. This is where
theidate is set held -- the data is held and
all the configurations. Shows the legend is
false. These are our options. There are a
couple that might be added in like is R. This
chart is coming from R. Then the data. We
only have data in the main data frame which
is Y and you will see here the bars frequency.
That's our only row and our samples. These
are the columns and then the data itself.
If you are curious you can take a look at
that. That's what is in here. It is not an
image. It is interactive here and it will
be interactive when we put it in our Shiny
application. That is the customized bar chart.
I will stop here and ask questions because
we are going to make two more charts that
are much more exciting. Any questions from
participants?
>> Just one that's not a direct question but
we have had some people with challenges getting
CanvasXpress on to R 4.0.2. I know we regularly
maintain and update this. Is there anything
you are aware of?
>> I am running 4.0.2. I have not encountered
challenges installing or running CanvasXpress
on 4.0.2 so it might be something else.
>> OK. Thank you very much.
>> We will take it offline so maybe at the
end or somebody can help directly.
>> We have a new question, how can one reorder
the bars according to frequency?
>> You can sort them on the chart and then
there is also an option to sort your data
in this configuration file. I need more coffee.
You can sort your data in the configuration
and send it how you want it to show. You can
reorder your data before you send it to the
chart which makes a lot of sense for larger
datasets versus ordering it once it is in
the chart. Something to remember and this
is a great question. Once you put the data
into this structure, everything that happens
to it happens in your browser. It is rendered
in the browser. Even more is actually in a
browser. The RStudio application itself is
not special. It is actually really just running
an inhouse browser and that's why when you
pop it out to a window with this button it
pops it out to a browser and it is exactly
the same. If you are going to do things like
analysis, like clustering, you can do it inside
CanvasXpress. You can do sorting, clustering,
filtering. You can cut your data. But it is
all done the web page. As your charts get
larger and larger you want to do those types
of analyses and manipulations before you actually
make your chart if you can but yes, you absolutely
are resort it.
>> We have two more questions if you have
time.
>> One can you change the color manually?
>> Yes. Absolutely. I will show how to set
the color on the next chart. We will set a
color by factor and on the heat map we will
set the color scale.
>> And should there be any issue with integrating
CanvasXpress plots in mark down?
>> No, it works anywhere on HTML widget works.
You can use a Plotly or a CanvasXpress chart.
You can drop it straight into a web page.
You can drop it on our R markdown, put it
in a notebook or R app.
>> Thank you. That appears to be all.
>> Let's do a little analysis and principle
component analysis. Prcomp is the expression
that does that. Let's do it on the expression
because I would like to make another interesting
but smaller chart. We run that analysis and
we will make a chart. If we take a look at
this pca result. If you haven't done a principle
component analysis before there is a rotation
field and the rotation field is your actual
principle component. I don't want to view
it there. This is the principle component
analysis for your samples. These are the principle
components. PC1-3 are the most interesting.
Let's make a 3Dchart. You can make a 2D with
the first two principle components but let's
make something more exciting. We will make
a principle analysis chart with the first
three. I will just send all the rotation data,
which is that data frame, to CanvasXpress
and I am going to set the graph type to be
scatter3D. And the nice part is you can start
with just that. This is my 3D part of my principle
component analysis. Not terribly interesting
other than it is 3D and rolls around. I think
this would entertain a fifth grader. Let's
make it better. Let's give it some additional
information. This is just numbers. Let's give
it that annotation. Let's annotate those variables.
The varAnnot and that's the GSE9750 and the
sample annot. Let's color these points by
sample_type. Now I have done something and
made it much more interesting. The sample
type is here. You can see they are clustered
by sample type which is actually the interesting
part of this chart and doing this chart. Principle
components are clustered by sample type. This
is not a chart that is quite ready to show
anybody. Let's go ahead and make it look even
better. Let's give it a title first. And there
is a reason I am going and setting them up
at a time. Let's make them look like we want
to and then put them in a Shiny app and make
them interactive. I want to make those axis
ticks so these numbers, I am not sure if you
can see my mouse but I would like to make
these numbers smaller. Axis tick. Let's make
them half the size they are now. And let's
add some transparency. Stop showing the legend
title. I am going to take the title off the
legend because we have it in our title up
here and I made these small. That looks much
better. Now I have transparent dots so we
can see through and a 3D chart that makes
a lot of sense. I don't like the lines. Let's
make the outline width smaller.
Let's make it half of what it is. I can take
it all the way off. Let's move that legend
around. I am going to move the legend to the
bottom. The idea is just to get you started
on all of this. I will move that legend to
the bottom. I can make it have three columns.
There we go. I like that chart. That chart
works for me. OK. We have got our second chart.
This is our second chart. Let's make a third
chart. Let's make a big data chart. Let's
make a big heat map. We will make that 1.5
million heat map. OK. Let's go ahead and make
this heat map. We are going to take a look
at the expression data itself. What I want
to do -- let's just start out with what you
might try. Let's just throw it right in a
heat map. What happens? A lit built of data
manipulation and wrangling lesson here. I
am going to make a heat map. This might be
the initial chart I go ahead and make. You
have to wait a few seconds and I go and look
and say that does not look very good. You
can see the scale goes from 0 to 80,000 on
the value for the expression which makes sense
but most are blue. Most of the data is close
to 0. What is going on in this dataset? I
am going to be like what is the data actually
look like? What's the distribution? I could
have told that from all of the blue but that's
the histogram of that. What if I wanted to
take a look at one gene? Let's just take a
look at the first gene. OK. That looks normally
distributed. That looks decent and doesn't
mean they are all that way but let's look
at the first column. That is not normally
distributed. This is the one sample. We have
one gene here and this is one sample. You
can pick any number to look at. When I look
at this data from an analysis standpoint,
I go I need to log transform this data, so
let's make the data log transformed. And again,
I use that McGrid of pipe. Let's take a look
at that log transform data and see if it looks
better. Oh, way better. The dataset looks
much better. Let's take a look at the chart
with this data instead of using the raw expression
data let's take a look at the chart. OK. Much
better on the heat map. We can see the differentiation
of the expression on the heat map and now
we need to customize it. You can see we have
the tool tips. Here is it that gene reference.
The 204649_AT is the gene reference. I will
leave that up to you to change that into a
gene symbol. There is the sample ID and then
the log value. This looks much better. This
you can tell is actually something. Now let's
go ahead and make this chart even better and
more readable. We have these labels on this
axis, these are the labels for all of the
genes and nobody is going to be able to read
these no matter what we do, so let's take
those off. Those are, if you remember, the
variables. I am going to show those as false.
And let's get a title on this so we don't
forget that we log the data. OK. Because that's
important. And let's also recolor this. You
might notice that it goes from blue through
yellow and it is kind of rainbow-y to red.
Usually on gene expression you are looking
for yes/no. Expressed less or more than 0.
So let's take that color spectrum and change
it. This kind of goes back to that earlier
question. Let's go to color spectrum. I am
getting a little bit of beeping from somebody
just as a heads up. Let's go from blue which
will be negative through white to red. Like
the red genes that show up, let's take a look
at this chart, the red genes that show up
are the ones that are positive and the blue
ones are toward the negative side. You can
see here now the mean expression is about
a 4. You can play with this and play with
the data but this is your big data heat map.
You may or may not want to show the sample
names as well. We can turn them off. And then
you just have, for instance, here you are.
This is the chart with no sample names. We
have this heat map and big data heat map.
And so this we will eventually drop into your
Shiny application. I am going to save this
and I am going to stop sharing. Are there
any questions at this point?
>> Not at this time.
>> We just got one. Do you have an option
to facet plots based on groups or something
similar to GGplot facet chart?
>> Yes, and the user can do it right on the
screen. Let me reshare. We have the big chart
up and I don't have the facets built but let
me show you the CanvasXpress website and show
you because it is really, really awesome.
Let's take a look at a scatter plot. I want
to look at a more rich scatter plot. Let's
say we have got this scatter plot. This is
the typical cars dataset and you will see
the cylinders, 6, 4, and 8 and regression
lines on here. You can go ahead this chart
and if you would like to you can facet it
by the -- where is it? Switching gears. You
can segregate it by cylinder. What if I I
want thee chart where each cylinder is on
their own chart? I can do this right on the
chart. You can preset it and segregate it
initially but your user are also do this which
is really, really awesome. If you look at
the single cell viewer app you can short where
it shows all of the genes and cells by gender.
One of the annotations is to look at genes
for females and males. You can bring it back,
remove the segregation, and bring it back
to the original. Everything that is on chart
you can use it to set it up for the user initially
how you want. You can set it up with a correlation
on it like in this instance or you can remove
it. You can let them correlate it. You can
let them set it up with the segregation initially
or not. And they can correlate it by cylinder.
This is all available to you. It is really,
really rich and we won't cover everything
by far. Any other questions before we take
a quick break and get into the application
part? OK. I am going to take that as a no.
>> One question. How easy flexible is CanvasXpress
when it comes to mapping text, point, et cetera?
>> I think you are meaning the decorations.
>> And there is a follow-up statement to clarify.
I am just wondering if it is possible to disable
the interactivity?
>> Yes, it is. We will disable a part of that
interactivity with our Shiny application.
I will show you why you want to do that. But
you can disable all interactivity if you want.
But I don't know that that's really the point
of the library, so, yes, you can. You can
disable things like resizing or scrolling
in certain directions, disable to ability
to zoom or zoom in certain ways. Yes, it is
extremely rich. The decorations I would point
you to the documentation. It's interesting
to add that. It also depends on whether your
chart is due to change or not change. We add
a lot of, you know, reference lines, you know,
for instance let's say you want to add a line
at a particular value, you can certainly do
that. You know, those types of customizations
are all there. Let me share my screen. I can
show you a chart that has a bunch of those
right here. So when you look at this chart,
the lines here are a decoration at .5 and
minus .5. You can take a look at the R code
to create those decorations. They are basically
a list that tells the line where it is, the
axis .5, if it was a slanted line, it would
have X and Y, the color of the line can be
set with color names like red, yellow, blue.
Or it can be set with RBG value including
transparency which is RBGA. You can see you
can customize colors to your liking. You can
set-up this decoration and this has two lines
as decorations. And if you had something that
was reactive, then you could move the line.
Let's say maybe your X value is based on something
the user sets and you would set this to be
your reactive value. We will get into that
in the second half. OK. I am going to share
the slides because they have the countdown.
Let's have a quick break. And come right back.
Here we go. I had it set for a 10 minute break
but I think we will run close on time. Is
10 minutes OK? Do you want to shorten it to
5?
>> I think it is OK.
>> Take a brain break. Get up and move around.
I think that's really important in our view
COVID world where we all sit in front of screens.
I will be back in 10 minutes. If you have
questions, please, still send them to Omnia
and thank you team for reviewing the questions
and helping with what you can. I will take
more questions throughout the session and
at the end there will be a half an hour where
I stay and answer any unanswered questions
or I can do any small deep dives if we need.
So don't worry. Hopefully your question will
get answered and if not like the question
on the languages, I will try to put together
some of the questions we don't get to and
send them out with the recording and the slides.
OK. I will be back shortly.
[10-minute break]
>> OK. I believe we are at the end of the
our break. I think we are just going to one
break in the interest of time. I just took
a couple of slides out in the interest of
time. You will get them in the deck but you
will notice this deck is not very slide heavy.
Are the organizers ready to start again? We
are going to jump right back into a topic
that's usually much more development focus
than not but I want to provide you with information
about scope in a Shiny application. Since
we have talked about enterprise applications
I want to talk a little bit about scope. There
are three main scope in a Shiny application.
There is global which is sibyl to all scopes
and the UI and only present where the global.R
exist. Not present in a 2-file app. We have
something that's called session global. It
is accessible to all server sessions. This
becomes important when you deploy your app.
That means that all of the user sessions,
which are green here, are attached to this
session global scope. That means that those
are shared across all the sessions. There
is no differentiation between user sessions.
It is outside of that server function in a
2-file app if you are used to 2-file apps.
Then there is session-local. That is the suf
the user changes. If they change a widget
you want them changing their own widget value.
If you have multiple sessions and put your
widgets in if one changes it they will all
see it. We talk about it because the Periscope
package is setup to make it obvious where
you are putting things. The global scope is
meant for what? What would you put there?
What goes there in a Shiny app? Things that
go into the global are things like library
calls. If you use dplyr every, put it in the
library scope. You don't need a library call
inside the UI or the server file. Things that
are static. Static datasets. Things that nobody
can change. Anything that is a connection
pool. If you have a database that you are
connecting to, you have to be careful in the
scope for environmental pollution. If you
name a variable X and put it in global everything
is X. You have to be careful because you can
can data and security issues. Session global
is a similar type of scope. You would put
common functions, might put connections, if
you need to access them in your UI file you
would want to put it in global but if it is
just behind the scenes it would be in session-global.
Session-local is where most things go. Most
of the code goes there. It is a reactive context.
This is one session of the application. One
browser. A lot of people say one user but
a user can have multiple browser sessions.
It is one window into your application. You
should always use the most restricted scope
possible. If you can do it in session-local
you should be doing it there. I am going to
introduce the package and show you why we
talked about it. Periscope is an R package
built by scratch. I am the author of it and
and I do the maintenance of it. It was initially
developed for internal use and it has been
released publicly and a lot of people really
like this. It is framework for developing
these Shiny applications in an enterprise.
But it is not the solution for all of your
problems. A Shiny app development is still
a Shiny app development. It is not a substitute
or replacement for properly planning an action
but it helps you out. It reduces the amount
of coding, the boilerplate, for common things.
How do I style that button? Those are the
boilerplate/wheels on the bus items. It creates
a consistent UI experience. You may or may
not like the UI. It gives you the dashboard-ish
UI that works across a lot of applications.
When you are in an enterprise or have someone
with hundreds of Shiny applications developed
by many people you need the consistent user
experience. When they go into an app they
expect certain things to be there. It will
also help with scaling and scoping concerns
for multi-user apps. If you have any sort
of group size, a lot of times you will find
that you make it, it works great on your machine,
you send to your friend, group, whatever,
PhD cohorts, your colleagues, and two people
try it at once and something goes way wrong.
That's that scaling and scoping concern. This
package makes it obvious where things go and
it keeps your scopes as safe as possible.
You know when you are making something global
and you know when you are making it local
to the user session. That's the biggest thing
it does behind the scenes that really you
will use it and you won't care about except
you really do care about it if you have spent
hours and hours trying to track down something
happens this solves that. Give it a try. It
is really flexibility. It is not prescriptive.
The content is what you focus on. It is totally
documented. There is R help, vignettes, and
example apps. It will create an empty framework
and get you running very fast. It is actively
maintained and updated. We are constantly
adding things and improvements to it. I will
just give you a quick run down of conveniences.
We will use the Shiny module if we have a
chance to put in a button. Each module has
a dedicated vignette but the things that are
built-in that you don't have to worry about,
you can just start your app, and it is there
and works is the busy indicator, the tooltips,
there is alerts that show up in the app in
standard locations, there is URL parameter
fetching, there is a reset button. If your
organizations requires logging actions you
can log your user's actions. A lot of places
require that audit capability and for things
like financial applications that's really
important. And it can work on devices so it
is already responsive. You can have a left,
right sidebar app, both or none. The sidebars
are up to you. That's one of the more recent
features. It is similar to programming traditional
Shiny applications. We tried to make this
as particular as possible. And what will happen
is we are going to put all of your content
under what's called the program subdirectory.
This is critical. Files above that program
directory are part of the framework that run
the application. Each framework file has a
header saying use this or don't use this so
that way you know what is going on. The UI
content is divided by application. You will
see UI body and UI sidebar and UI sidebar
right. You can imagine where your stuff goes
by the name. And then the server content.
This is that scope that we talked about. There
is a global.R and that is where your global-global
stuff goes. There is server-dash-global.r.
and there is server-local where the majority
of the code goes. This is all reactive context
in the file. This makes it obvious where you
are putting things in the scope. Usually you
won't even tend to think terribly much about
it. What I want to do is show you the example
app. Sometimes coding by example is much more
obvious and usable. I want it talk about that
file layout. I removed the slides. Just skipped
past them on the file layout but they will
be in the final deck. Just in expedeancy of
time. Let me show you the sample app and how
to create it. This is the part where we might
have to do a bunch of switching between the
shares because I can't share my desktop. Let's
clear out our viewer. While you are working
with Shiny apps and what not, and for any
of you that have seen Jenny Braun's talk you
will notice I restart my auth fairly often.
It cleans out the gremlins. If you are using
the latest RStudio you won't tend to find
much but if you are still on 1.2 you will
find restarting R helps with the stability.
We have a new R session. Made our chart.R
and let's close down the datasets. Leave the
chart.R up. Let's make a new application.
I am just going to show you here. Take a look
at Periscope and the documentation. Some people
don't know this trick. This is the main documentation
page for the package. You can always look
at the index. The index is my favorite page.
I wish it went to this to start with. It gives
you all of the help pages and functions in
this package and directs you to the vignettes
which are here and you can view them here.
This is what we will do now. I will not show
any more of that but the function we will
use is called create new application. This
creates a templated framework application.
I am just going to show it here. We will create
a sample app. There is an action here that's
false by default that creates a sample application.
We are just going to go ahead and do that.
Periscope. Now you can library Periscope.
We will call this one sample so we know the
difference between it and our real app. And
we will give it a location. And we will say
create us a sample app. That was easy. Let
me show you what actually happens. It created
it inside our live tutorial because this is
our project folder. Since this is relative,
it created this folder name there. And you
will see this sample app folder. Let me walk
you through it it. When you go in here, there
are some web resources, Shiny is a web application
so if you have custom JavaScript, custom CSS
you put them in here. And then there is the
files you might expect. If you open this global.R
you will notice this says do not edit this
file. This is part of the framework.
This is a reminder. As you use this package
you will get much more familiar with where
to go to edit the files. If you get into one
of these and it says do not edit this file,
I suggest not editing it unless you know what
you are doing. The framework is intended to
be used inside this program directory. You
will see here is the user directory that will
hold the logs. In the program director this
is where you will go all of the time in this
framework. There is presetup a data folder.
You know, in this sample app it has a reference
dataset called example.csv. There is a function
folder. It is a place where you add those
extra files that you are going to reference
to make your plots or use your helpers. Everything
is not inside your code. You will want to
functionalize things and organize your code.
I think organizing your code is super, super
important. This is a true global you will
be using. This is where the application is
setup. Basically you can set a title and title
info and how much you want to log. Here is
that server global scope and you will notice
here in this example application, this is
where we read in our static dataset. Things
like states. Nobody is going to add a steak
in the middle of your application or countries
in the world. Thing like this that are static
is exactly what this scope should be used
for. If you put it in the local scope, it
will read it in every time a user connects.
This becomes really important as your data
gets bigger. This is a static dataset. It
should be red once no matter how many people
read my app. Nobody is changing it. That's
the important part. Then we have the server
local. This is your reactive location. You
can read in your helpers here. These are our
functions. We are just going to put them right
here. They are local functions and that's
fine. Or you can put them in the server global
scope. These are the scoped items. As you
put code into these files, you know what scope
you are putting it in. Server local is one
instance. Server global is everybody's instances
and so is global. Then we have got the UI
which is divided by location in the application.
We have body and things you stick here go
in the body. Let me launch this cyber. We
named it sample app. Since it isn't a subfolder,
you actually have to provide an option to
run app. You provide that subfolder name.
If you were in that subfolder inside the sample
app, let's say you are working on code and
you had changed your working directory, you
can always see the working directory here,
then you would just run app without the option.
We can pop it up. Now you may notice this.
In your RStudio it can pop your application
up in this viewer which surprised me the first
time it happened to me especially in the newer
viewer. You can always pop it out to a web
page here although I am going to show you
a few things in the viewer. I don't suggest
you view your application in the viewer unless
it is smaller. You will want to, before you
send your application anywhere, make sure
it is viewed in a browser. You can always
set it. Let me show you how to make it pop
out. There is hidden options somewhere in
the interface and I always have to dig for
them. You can always make it pop out by using
launch.browser as true. You guys can't see
it but let me reshare. It popped out into
this browser. Do you see my browser?
>> Yes.
>> OK. Good. I am not talking to myself. I
was a little worried I might have been on
mute. This is a browser. You will notice right
off the bat a number of things. When you saw
it in the app it was in device mode. You can
always take a look at the device mode or expand
it out. We have our sidebar automatically
and it has a couple of tabs in it. We have
our reset button and it will even give you
a warning and say hey, the application will
reset in five seconds if you don't cancel
it. I will cancel that. And then this application
actually is a documentation item. If you open
these boxes you will notice that you can activate
a number of the items inside the application.
This working indicator up here, I am hoping
you can see it, is tied in automatically.
You don't have to do anything special or call
anything for it but I give you a button so
you can see it. You can set alerts in the
application. You can set alerts over here
on this tab. You can set alerts on this tab.
And you can set alerts in the body. Some of
these alerts can be set to be closeable or
not closable. This one is closeable and I
guess they are all closeable on the example.
Probably because I didn't want you building
up a bunch of alerts. You can automatically
close them, let the user close them or not
let the user close them. You will see tooltips.
Here is a tooltips. All of the functionality
of the framework is showcased to you and there
is a lot of explanation here. These buttons
are a module and you can see that, you know,
how to make it easily downloadable. There
is a downloadable table module that makes
it easier to show a table that is downloadable
by default. Uses DT under the hood and gives
you a presetup table. You don't have to worry
about how to add the sorting or put the search
bar on there. Then I have to put a button
in and tie it in. Those things have been taken
care of. The idea is not to invent the wheel.
You want to focus on the science and the analytics.
It handles lattice plots and provides you
with the ability to download them as well.
We have something called a downloadable plot.
It is for these graphics that don't have built-in
download capability. Things like ggplot or
lattice. It will reformat it into different
values. You will notice you can download either
the plot or the actual data table. So these
buttons handle both data and plots in the
same button so your user can be like I want
it as give me the text file. Of course, it
handles HTML widgets. Any HTML widget will
drop into your Shiny application. It will
show you vertical logging and how to set it
up. You will see we requested a reset, canceled
it, pushed this button a number of times.
You can log everything a user does here for
your audit capability. This is the sample
application. I really -- let me reshare again.
I really suggest you take a look at it and
the code for it especially if you are a new
Shiny developer. I tried to make it very explicit
where some of these items come from. Box is
not special to Periscope. You will notice
some of the functions are called out with
double colons instead of library calls just
to make it very explicit where some of these
things come from. You will see this is the
UI body file where the elements are created
and here is where I make boxes and put in
them the various things. You can go through
this application and kind of trace out where
the things go especially if you are new to
Shiny. It will help. And I mentioned that
we have to register those UI items here. This
is the one main difference from a regular
Shiny app. Instead of just creating the UI
items you just have to put them in a list
in the order they show up in the app and that's
because it is dynamically laid out. I want
I wanted to take the time to show this because
I think it is a useful tool and then we will
go back and make our app. Any questions at
this point?
>> There is one. Could you just comment briefly
on the difference between Periscope and gallum?
>> Gallum is intended to package up a Shiny
application. It is an interesting concept.
I was very interested to talk to the owner
at the last RStudio conference. It is an interesting
concept and I would love to think about packaging
Shiny apps as packages like he does. I think
it could be a good compliment but this is
much more focused on the Shiny application
itself sed of the wrapping around it. That
was what I took away from my initial exploration.
This is more what goes inside Golem if you
are using Golem to wrap your application up.
For those that aren't familiar that is G-O-L-E-M
if you want to look it up. Let me go back
to the slides. I wanted to talk briefly about
planning a Shiny app. I have jumped in with
this example application but really, Shiny
application development is less like R programming
and more like enterprise or building Facebook
or an app on our phone or something like that.
You actually need to plan it and that gets
more important as it gets bigger. Planning
a Shiny application doesn't have to be complicated.
I use this example on the right with my teams
and my teams are used to this and they call
it Connie draw. If you are a whiz at layout
packages and PowerPoint maybe you will do
it in PowerPoint or Word but the idea is you
kind of need to say look, I have a screen
and I am going to show the user things. I
am going to put the controls here and these
are the kinds of control I need and these
are the kinds of data and visualizations I
need. I want tables and charts and I want
them to kind of look like this. Basic planning
is necessary and a lot of times my planning
looks like this. You do need to think about
your data. Where are you going to read it?
How is the accessed? Is it static? Is it big?
Is it going to change? Do your users actually
change it? How are you going to get it into
your application? You do need to think about
these things. You need to think about your
user. That's actually really important. Why
are they using your app? Are they using it
to explore? To create reports that are given
it their boss? Are they using to to find,
you know, efficiencies in marketing result/campaigns?
Why is the user using your app? And then,
you know, that kind of goes in concert with
what controls are needed. And then you pick
your tools depending on your app. And again,
you have a toolkit. Maybe sometimes you need
a hammer, maybe sometimes you need a screw
driver. Those are, you know, the tools that
you can pick from. The charting libraries
are, you know, a whole ton of them. There
is a whole ton of different table formats
and widgets. Maybe your application is so
cool and has interesting charts that you need
TensorFlow in there and custom 3D visualizations.
The sky is the limit on content but you do
need to plan ahead. That's really what I am
saying here. Plan your layout and styling.
A couple of notes on reactive programming
in Shiny modules. There is a different in
programming Shiny versus R. R is a functional
language and we are used to go line by line
and executing results. But in Shiny the user
is not going to go in the order you expect.
The user is going to go in their very own
order and that is tough. It works much more
like a modern application. It is reactive.
User does something, something else happens
but the trick is not just one thing happens,
lots of things happen. As the app gets bigger
and bigger more things happen and you have
to keep track of what's happening. Even though
R is a functional language, the pieces of
Shiny are going to trigger other things to
happen. There are context like the server
local file that is reactive so that everything
goes in there can change and trigger something
else to happen. That means if you change your
reactive variable, something else will probably
happen. Anything that depends on it unless
you isolate it. It is not just input and output
in session. There is whole blocks of code
that can change. Just keep in mind it is not
just observes and you can restrict it with
isolates but you have to plan ahead for if
you change something and something else depends
on it it will change. The order of that change
is not what you would expect all of the time.
I don't say that as a dig to R but when you
trigger an event and the light stops at a
road you know certain things are going to
happen. Cars are going to stop, pedestrians
are going to walk but pedestrians are individuals
so sometimes a pedestrian might walk faster
than a car stops and get hit at the intersection.
You have to be Cognizant there is a whole
chain of events going to happen and if you
need them to happen in a specific order you
may need to make them happen in a specific
order. And I will briefly mention Shiny modules.
They are pieces you don't have to write over
and over again. It gives you an UI and server
function and you will use it. This is no different
than a button. My action button and then,
you know, or, you know, an HTML output and
then you render the HTML output. Modules are
similar. They did change in Shiny 1.5. When
you are programming in the latest version
of Shiny, keep an eye out for packages that
have moved to the call server paradigm. That's
a little different and you will have to read
up on it. Periscope is using the old functionality
right now. It is not they don't go backwards
and forwards. At some point we will have to
switch over to the new functionality but then
everybody has to use the new functionality.
That's a fairly new change. That's all I wanted
to say on that. Let's make the app. Any quick
questions before I switch gears into the app?
>> Nope.
>> Don't worry. Let me share my RStudio. Let's
create an empty framework. OK.
Let's make this false and we have made our
new application. I have to give it a new name
or it will override it. Let's make this our
Shiny app. We won't overwrite your existing
files. If we go to the live tutorial, we now
have Shiny app. Let's launch it and see what
happens. It should launch and has nothing
in it which is what I would expect. It is
all here and ready to go. Let's put some stuff
in it. We know we want R3 charts. Let's go
ahead and put our charts in it. We go to the
program directory. We go to the UI body. What
I am going to do is -- first you have to setup
my imports because I want to be able to use
my libraries. In global.R we are going to
set our library calls. CanvasXpress. And one
of my favorite functions if you haven't used
it yet is glue. We have our four depenancies
and let's give it a nice title. There is the
title. That's where your title is set. We
have set our title. We have our added our
libraries. Let's go ahead and add some things
into the body. Let's put our three charts
in. Canvas express has a canvas express output
and a render CanvasXpress function. Just like
a lot of other packages. Let's do a two-column
layout with the two small charts on the left
and the one large heat map on the right. We
will make a column. Let's make it smaller
than -- whoops. And we are going to add a
part to the top left. We will make the PCA
chart after that. Let's make another column
that will hold our big data chart. Our width
is 12 overall, so we have 7 left here. If
you start your app up and you are like nothing
happen, you have to put the elements in the
add UI body function. This is the one odd
thing you have to do that's different than
a normal application. This has added those
front-end components. Now we need to go add
the charts. This is that CanvasXpress output.
This is paired like in a normal Shiny app
with the CanvasXpress chart output. We will
put the server local because each user gets
their own chart. This is where, and this is
where it will start to look really familiar,
output $. We are going to render it. OK. This
is the active expression that will hold our
chart for chart bar 1. Let's add the skeleton
to 2. Chart two is PCA. These are the same
IDs. This is a standard Shiny programming.
Chart 3 is going to be our heat map. We have
added our spots to put our things in here.
Let's go and put R charts into these spots.
We have our charts already built. This is
fantastic. Let's put the PCA chart in. It
is the most concise. Just to show you that
it does work to just drop it in. Here is the
PCA calculation and here is where I create
the chart. The render should also result in
the output that you are expecting. When you
have render something, you would want that
to be the output. This is the output of what
comes out of CanvasXpress. I am going to launch
this because I want to see that my chart showed
up in it. It did! OK. I have an app with one
chart. OK. Not terribly exciting. Let's add
the other chart to the left. We have created
this chart with this data so we are going
to go ahead. It is always so satisfying when
you can drop it in because you created your
plots. The Shiny application is not where
you want to be, you know, trying to customize
your charts, you know, ultimately. In this
one, we assigned it to a plot item. You will
find that this is very useful. Maybe you want
to do a, you know, if-else statement, you
want to change this for whatever reason, you
don't have to have this to be the last call
in the output and the render. You can create
it into a variable and just return that at
the end. We will add that and launch our app.
And now we have both charts in. OK. And then
let's put in our big data chart. I am going
to show you a trick with this. OK. OK. We
are going to put our chart in here. You will
notice it takes a second or two to load up.
I am going to address about the question of
can we turn off some of the interactivity.
You might notice it is too short. You wanted
it to be tall. We will fix that. CanvasXpress
let's you grab your chart like this and take
the mouse wheel and zoom in and out. This
is great if you want to look in detail about
what's going on. You will notice each step
has to redraw this chart. That is called zooming
disable. The user will still be able to go
and select an area and be like what gene was
that?
>> If you get zoomed in or you can hit escape
and it goes back to the initial review. This
is a 1.5 million point chart. That means not
just 1.5 million spots are on this chart.
There is 1.5 million numbers, 4.0.22. There
is labels and colors so 1.5 million potential
other colors. It is not the only option. I
love Plotly and program in Plotly. Used to
be a ggplot user so I completely respect those
libraries but I wanted to provide you with
another option that gives you that tooling
or toolkit and different screw driver.
>> The default height is 400 pixels. Let's
make it double the other charts. 850 pixels.
It will fill up the whole right of the screen
and be twice as tall. When we set-up the charts
we set them up to have control. What if I
want to do PCA on something other than sample
type? Let's add scrolls. We want to put them
on the sidebar. There is advance and basic.
Let's do a bar chart choice and name the variable
something you will remember later. I am calling
this bar factor choice. I am going to give
it a default value which is going to be -- I
have a typo there. We have got that selectize
input. If you copy and paste you have to be
careful. We will make this one the PCA and
a color choice. This is PCA color and we will
give it a new ID. PCA color choice. You may
have a lot of choices for your user and we
will give this a PCA factor and it is still
going to be the column names. We will set
to a different default. Now we have to add
them to the sidebar. Make a list and bar choice
and PCA color. Now, they don't do anything
yet but if we start up our app, we can them
there in the sidebar. Let me run the app.
When you are working with controls, if you
have got big charts, like we have this is
a tip and you can see here we have got our
big heat map chart here. You may want to take
and comment it out and do a block comment
on it while working with the controls. That
is just an old school tip. We are going to
work on the reactivity for these two charts
right now so they are tied in. You will see
they have the items but they don't do anything.
Let's go make do something. You can always
add a to-do tag. If you do it is a special
tag and you can find them all later. Let's
tie those choices in. That is what makes an
app. You can do something. Let's go up to
the PCA chart. We have tied in what we are
going to color by. We were default coloring
by the sample type which is fixed. We want
to give the user a chance to do the color
choice. OK. I am going to make a color choice.
Let's get the user's color choice. I could
directly use the input here but I am not going
to do that. The reason is I am going to set
it to a variable. That input is called input
$. We named it PCA color choice. I will find
out soon enough. You can always come back
and refer to it here. It's this name. The
ID. Not the variable you set it to. PCA color
choice. I have my color choice. I could just
run it like this. I want to show you why I
don't directly use it first. One of the things
the user can do is delete the value. That
can in certain instances cause you issues
with your charts or your analysis or things
like that. Let's make a PCA one. In this chart,
it just uncolors it which is not horrible
but the title is still wrong. It could crash
your application depending on what you are
relying on this for. This input is important
for me to check. Almost all inputs are going
to be important to check. We want to say,
hey, if somebody has deleted this, I don't
even want to show this chart. If it's null
or NA. Sometimes it is one or the other. I
always check for both. Then I am going to
move this here into this if-else and I am
going to have this enclose the whole chart.
If I return null the chart does not show.
This is a great way to be a little bit more
safe in your code and have fewer problems
with your Shiny app as you debug it. Now what
I have here is I have my data. If you haven't
chose a cull or cleared it out or it is the
initial application it will not make the chart.
If you have done that -- I have gotten too
efficient. If you haven't changed it and it
is not null it will set it down here as color
choice. I am going to change this title because
it is wrong to use glue and this color choice.
Glow if you are not familiar with it, is a
string replacement type of tool and let's
you inside these brackets use any R code you
want. I could do analysis in this bracket
if I wanted. I tend to output variables with
it. There is my title. You may or may not
be able to refresh your application and see
your changes. The reason I tell you that is
the viewer does not pick up all changes. That
show up in this reactive context. It is not
specific to Periscope. It is just in general
Shiny app. We want to make this not null and
not NA. I would have expected my chart to
be here but it didn't show up. Here we go.
Now, if we delete this -- we can check for
blank string and suppress the app to see if
it picks it up. No, it is still not picking
it up. I guess I will show you one more debugging
trick and that's called browser. Browser takes
your application and stops it right here.
Maybe I want to check the value of this. More
color choices. This needs to be and. That's
why. I think I mentioned it is morning here.
So if it is not null, and it is not NA, and
it is not equal to the blank string we don't
need to drop into the browser but I will show
what happens. You can see we have come into
the browser here. On my console, or in the
environment, you will see that functions value.
If you want to inspect your data, right where
this browser is, that's what you can do and
you can start to run through line by line.
This isn't a debugging course but this is
probably going to be your best friend at some
point or another while making your Shiny app.
Let's take that out so we don't stop our application.
You can continue. But every time it hits that
reactive code it will stop there if you will
put browser. We have got our application now
with sample type and I have it properly setup
so it is not going to refresh it. You will
see it as it not turning back into that pink.
It leaves it at the old values until I pick
a new one. Here is color by stage. Cell type.
HPV status and our cell type. Everything that
depends and see this color choice depends
on input will change. I am going to pause
here before I hook up anything else. This
is how you go about taking three charts, dumping
them into an app and adding those controls.
We will add a control up here and that will
be used in the groupby. Let me frays a second.
It is a lot of information.
>> We have one question going back to the
color choice stuff. Somebody wants to know
if we can use require color choice?
>> Yes, absolutely. And you can, yeah, you
can use require. You can use any way of checking
these as you want. I point this out not because
this is necessarily terribly efficient but
maybe you need a value that's greater than
5. You may depend on other values that aren't
-- let's say the user changes their sample
type but they also have to have not picked
the same value for something else. The reason
I point this out is because it is a great
way to do all of that checking you need to
do. Maybe you are checking if it is positive
or negative before you try and log something
that gives you an error. You are going to
find you do a lot of checking on these inputs
as your apps get more complicated. That answers
that.
>> Thank you. And I know this is a very complicated
topic, all things considered overall, but
can you just elaborate a little bit on whether
Periscope provides functions to facilitate
deployment? Docker image is a specific example.
If you could just touch on that.
>> I am going to touch on deployment when
I come back so I am going to hold on that
one but Periscope is creating a Shiny app
and that's its purpose. It is similar to the
question about Galem. Once you have the Shiny
app you have to deploy it. It doesn't facilitate
any type of deployment but it is the same.
It will deploy any way else that you deploy
your applications. Let's hook in the rest
of this and if anyone wants to stay I can
hook in more things. You can focus on your
charts and hooking in your actual functionality
versus how do I put the label on my item,
how do I add a tool tip, how do I, you know,
make it downloadable? I will share the code
for the sample application as well if you
are not coding along. Let's look at this first
chart and then we can go a little further.
I want the user to set the value. We will
reaggregate by the value they chose for the
bar chart factor and use it down here. So,
let's get the -- I feel like I am running
out of time. I will leave this as an exercise
if you guys because it is very similar to
this exercise. I want to show the user a tooltip.
Let me show you how to use some of the other
functions of the application. I am going to
leave this one unhooked up for the moment.
Let's go back and add a tooltip to this element.
I want to show you how to do that. I think
that's really important live. We want a hover
thing for the user. When we go into the sidebar
for that PCA factor we want to add a tool
tip. I am going to call this a tag list. When
you put lots of things together in the UI
it is called a tag list. That's the Shiny
container for a whole bunch of stuff. We are
basically going to go ahead and make a tag
list and we going to add the tooltip. This
is a Periscope function. We are going to say
let's give it an ID.
Everything has to have an ID. This is PCA
tooltip. It has the optional label. We can
replace the label with this. Bar chart. This
one is PCA factor. Then we will give it the
tex we want to tooltip to pop up. That's the
tooltip itself. It is not very big. After
the tooltip comes the select input. Since
we are using the tooltip, when we are putting
the label here, we want to set this label
to null. We don't want two labels. Let me
show you the application. You will see there
is your tool tip and crore awesome explanatory
text. This gives you a function you can add
a tooltip anywhere you like. Then I also want
to show you how easy it is to add a download
button. What you have want to let your users
download this PCA dataset and don't want them
to have to find it. You can always view the
full PCA data here in the CanvasXpress widget
but your userss are not familiar with that
or you are using another plotting library.
Let's put a button for them to download the
data. Let's add a download file button here.
We will put it after this. I am not going
to add a label for it at the moment. Let's
just add that download button. Here is the
download file button. See this module here?
And this is the one that you give it an ID.
PCA download and then you are going to give
it download type. How do we want to let them
download it? If you send a table, it will
format it into the various download types.
Let's make a list and we are going to 'give
it a CSV and TSV. You should probably use
the labels on these. Download types. I like
to do this because when I come back later
I want to know why I set that value. It is
CSV by default. We have created the download
file button and you can even set hover text
right on the button. That's the tooltip. If
we refresh the app here and when I tell you
add a new UI widget and you will need to restart
as it you will see the two values. If you
are not familiar with modules this is why
I wanted to show it to you. In the reactive
context, in the users server local file, you
are going to call the module. Again, this
is the original implementation of module.
Call module.Output. You put the module name.
It seems odd. You don't actually put it in
quotes but you put the model name which is
the code name, right? You give it the ID you
are going to give. We called it PCA_download.
OK. We give it the log so you can pass it
the user action log. It is right here in this
context for you. And then we have going to
pass it the required module parameters. The
file name root. When the user downloads it
it will be named PCA_analysis and csv and
tsv. Where is it going to get the data? The
data is going to come from reactive or regular
functions. If you have a dataset that doesn't
change it would just be regular. Csv and we
will give it a data function and tsv. That's
the module call. I said go get it from this
function and this function. You can have different
functions for different formats. Maybe you
want to filter had data for different formats.
You can set your csv to be a different download
than your tsv you have wanted. I need a function
that makes this data available. I am going
to use something called a reactive variable.
It is a like a function with no parameters.
Where does that data come from? That data
comes from this. This is the analysis. We
are going to put this here. This only recalculates
if something changes. This only recalculates
once for the user session. You could use this
PCA data down here instead of this call. This
is replaced by the function bracket. This
is calling that reactive function. I am going
to put that down here. We are going to just
expose the rotation. Were, this is a PCA analysis
with a lot of things in it. The function,
I think will just return the rotation data.
This function here is going to just return
the rotation from the analysis because we
want to give the user a table. We don't want
to try to have them download. This will provide
data for the chart and download. Here is our
download button. CSV. It asks where I want
to share it. This is the unfortunate part
of not being able to share my desktop. I will
put it in the same folder as the tutorial.
It saved this file here. PCA_analysis and
here is the data and all 66 principle components
the user can download. We are showing the
first three on our chart. That is our application!
Unfortunately, since we are so close on time,
I am going to restart my application and show
it to you all in one. We comments out that
big chart and, of course you will note it
if you watch the app. You can float it back
up. You will see it here in just a second.
Gives it about five seconds to render. We
have our PCA factor hooked up so we can change
our factor here and recolor it by whatever
we want and interact with this one. This one
we can no longer zoom with our mouse wheel.
You can turn off the toolbar, turn off resizing,
this chart is resizable by the user. You may
or may not want to allow that inside your
Shiny app. You can turn off zooming in your
certain directions. But the user can still
zoom but we just turned off the scrolling
reel. You have these rich charts and you have
started to tie-in the items that your user
needs to interact with. I will leave it to
you to tie in age into this but you will tie
it into more than just the chart name. It
is part of how the data is recalculated abut
I think you can all probably handle that.
We will wrap up the official talk here. Let
me return to the slides. Some debugging items
that we actually already got to. Deployment,
you need to think about where you will deploy
it. RSConnect was mentioned and that is where
most will deploy their applications because
it handles the scaling nicely. Remember, if
you are using the free Shiny server, it is
single threaded. If you have 10 users connecting
to one application you are using one thread
for 10 users and they have all to wait for
each other. As long as you scope it, they
will not interact with each other's sessions
but keep that in mind. Depends on your platform
but this is not a package that actually deals
directly with deployment but it is compatible
with different types of deployment. There
are other toolkits that deal with deployment
specific. I will give you the slides and these
have those resources on them that we talked
about. CRAN is the primary repository. You
are welcome to check out the GitHub repository
but GitHub until it has gone through testing
and through CRAN and tagged you may notice
differences in GitHub just like any other
package in development and it may or may not
work from GitHub. Please use the CRAN package
when developing Shiny apps and you will find
they work the best. That single cell viewer
I showed you has a pre-print publication if
you want to read about the science and why
it was created and you can check out the code
and take a lookt an enterprise app code and
try it out on hosted server at Periscope-apps.org.
CanvasXpress is out on CRAN and you can check
it out at CanvasXpress.org. I have a couple
block posts on scoping and one on CanvasXpress
visualizations and welcome to check out our
company as well. We are Aggregate Genius.
This wouldn't be possible without Isaac, brisol
Meyers Squibb and useR! 2020 and AfricaR,
EgyptR, R Ladies and Algiers. I am going to
stick around for questions but I did get the
warning I was very, very close to time.
>> Thank you so much. That was amazing. I
love how your code is clean. Your code is
so clean. Like wow. When I grow up I want
to write such good code. Thank you so much.
This has been amazing. Speaking personally,
I love your mode of presenting. It is so engaging
and the energy from beginning to the end is
the same. Like I can't imagine almost going
to three hours. Thank you so much. We are
so grateful. To the participants who joined
us, thank you so much. I won't even say, I
will say I know you have learned so much.
And the team is very accessible. Maybe you
can share an email, Dr. Connie, where people
can talk to you. But, yeah, if you have questions,
stick around and ask. I am grateful for the
engagement. I think that's all from us. We
are very happy to have -- we are grateful
that you people decided to apply. Imagine
organizing something and no one applies. We
are grateful for the curiosity and hoped we
lived up to. What will happen is we are going
to send -- I am not sure about today since
it is end of day, but we will share the material
with you on email and the recording once it
is edited. I hope if you want to have your
face shown you covered it but if you still
have a concern you can email us and we will
know what to do from there. Yeah, I think
that's it from my end.
>> Excellent. I am share agslide with my email.
You are welcome to reach out to me on email
or find me on LinkedIn. Twitter I occasionally
look at. It is one more thing I tend not to
get to because I am so busy but feel free
to reach out to me directly at any time.
>> Dr. Connie, I didn't know the pipe could
work with other functions. For me, every day
is such a learning process. Thank you so much
for insisting on that because that's something
I think we need to know. I didn't know the
dplyr function.
>> If you use dplyr it is so useful because
you can send it to almost anything. It has
to be a function that returns the value to
keep sending.
>> Yeah, it is but what I am saying is I never
thought you could use it outside of the platform.
Thank you so much. That was insightful.
>> I will speak around for questions. Feel
free to unmute and I don't know if we have
a pile up of questions.
>> Absolutely. We are running late. Someone
may ask a question that you could benefit
from. If you are afraid or not willing to
ask questions publicly, you can easily DM
on the chat anyone.
>> We did have one question that slid by going
all the way back to CanvasXpress.
>> No worries.
>> And that is reference to 4 cat with regard
to functions like FCT and reorder. They were
curious whether CanvasXpress provided similar
functionality.
>> I am familiar with it but it is more of
a data expression language. I have not used
it. CanvasXpress is focused on visualization
and I think you can use any package to do
that.
>> I have used four cats to reorder things.
I imagine you can and then pipe to CanvasXpress.
I am not sure if the reorders would hold but
they might.
>> Unless they are held in a different structure.
You could always reorder and if you need to
change it back into a standard data frame.
There is no reason a tibal can't go into the
CanvasXpress. In my mind unless it is unusual
under the hood it should be fine.
>> Anything else? You are probably madly coding.
>> You can ask me to unmute you if you are
not unable to mute yourself. We will stick
around for a few minutes. Sometimes you need
to gather your thoughts but shell, if that's
OK with you?
>> Sorry. I was just distracted for a minute.
>> We can stick around for questions for the
next 10 minutes and if nobody asks or wants
to come up that's fine.
>> This is your opportunity to go around the
moderator. Thank you for everybody's questions.
Great ones and definitely stuff to follow
up with.
