Vallia good to see you, um
there will be folks that are likely
going to file in as we
continue our discussion I do have the
chat
window open if you have a comment or
want to weigh in or a question
or want to share a link whatever please
make sure that you
put that in the chat window I am um
I would like to ask everybody to kind
of
you know weigh in if i missed something
in the chat let me know um i will try to
check it periodically
um but i'm going to lean on my
colleagues Suzanne and Mitzi
a little bit to help me kind of watch
that chat window if you all would
um so so um i want to go ahead
and and kind of give you all a plan for
the next hour and let you know
what we're going to talk about so as
as you know this is a workshop on on
questioning
in the classroom but but not necessarily
in the classroom
physically but also in the virtual
classroom and so we're going to touch on
both of those
ideas but there's going to be kind of
three parts
to our discussion the first part i'll
talk a little bit about what we
currently do in our classes how we kind
of
currently think about questioning our
students the second part i'll offer some
research based
recommendations on maybe what we could
do
to to maybe mix it up a little bit and
then i'm going to open it up to the
group
because we have got a breadth of
experience and a wealth of experience
here
and there's no reason not to tap that
and so i would like to make sure that we
um hear from everybody and and hear
about your experiences and how you
question students in your classrooms
both virtual and physical
that sound good all right good deal
we'll go ahead and get started then
so i do want to mention that we're not
going to focus a whole lot on
writing assessment questions this is
this isn't really a discussion on how to
write a test
this is mostly about uh formative
assessment how we question in the
classroom how we
uh bring in discussion how we ask and
answer questions things like that
and and you'll get a better feel for
that here in a moment
at various points, I'm going to share my
screen
let me make sure that I have got what I
need
ready to go here i've got my trusty
folder that i created
for quick access so um i want to kind of
bring up
one article or one publication that was
written by Christine Chen back in 2007
she um she did a lot on
it was mostly k-12 but she did a lot on
the analysis of question types she was
looking at science courses
in k through 12 trying to understand
better
how teachers tend to ask questions and
the types of questions
that they ask um it's it's a really long
paper and
i'm not going to open it up and walk you
through it but i i do want to point out
and if you'd like links to it i can
certainly
um or i can certainly uh give you well
actually instead of just
telling you to do that i can go ahead
and do that very quickly because i made
a folder after all i might as well use
it right
so let's see here
so I don't there's
sorry all right so Christine Chen this
is uh
types of questions um if you wanted to
look at that there's a really good table
that is it's like i said it's a pretty
big
paper but if you look there is a table
that is on page
it's 9 of 29 in the pdf
um and there's two
basic types of questions that i have
seen asked in in the ten or so years
that i've been observing
stem classes and even outside of stem
for that matter
um and those two are pumping
and verbal clothes and both are really
designed
to encourage discussion it's really
they're both really designed to try and
get
students talking unfortunately often the
way faculty
use these types of questions though is
is
you know to ask a question so they can
take a sip of coffee
you know we want to be a little more
intentional
in our questioning now verbal clothes
and pumping questions are great
i i don't uh i recommend you continue to
use those
but but but we need to step it up a
little bit
and so we're going to talk about that
specifically here in a moment but
this is a really neat table whenever i'm
i'm working with STEP fellows
and i've done some trainings for
graduate students
and undergraduate TA's and undergraduate
LA's
we actually use this table as an
exercise where we talk about the things
that we already have in our tool kit
and maybe some of the things we'd like
to add to our toolkit so i recommend
that you
you know if you want to read the paper
that's great but this table is a really
good place to start
because it helps you think about what
you're currently doing and maybe where
you'd like to go
for the next steps um that's
um that's all all we're going to do with
with this particular paper i just wanted
to make you aware of it
but my next question is so why do we ask
questions at all and this is fairly
rhetorical because i think everybody
knows why we ask
questions right um but i liken
asking questions in class to
athletic training now i'm not going to
go too
too deep into this idea of athletic
training because some of you may not do
it and i'm
i i'm i'm now very nervous by saying
that because we now
officially have a coach a former coach
in this conversation so
Dr. Lumpkin please bear with me um
so let me ask you this if you've ever
run a 5k or wanted to prepare for a 5k
um did you just throw on a t-shirt and
shoes and run out there and run three
miles
probably not um that that's you could
try it
um it may not go well but
that's not how we typically would do it
um probably what you do is you probably
start off by walking around the block
right or you probably find some some
shoes that are
aren't going to kill your feet right
that's probably the first step then
you're going to walk around the block
maybe walk to the end of the street
you're going to do that a few times um
you're
you it's probably going to be a month
before you start jogging
around the block right you've got to
you've got to get your body used to that
impact you got to get used to
you know oh wow breathing that's
important for this whole running thing
right
um you're probably do that a dozen times
or more
and then eventually you might get to
running a half mile
and then you're slowly going to build up
and build and build and build probably
over the course of several months before
the big day
right so what why do i bring that up
think about the way a student prepares
for an exam or the way we hope they
prepare for an exam and the way we
should prepare them for exams
we're not going to walk into class one
day and
plop this test down on their table it's
worth 30 percent of their grade and say
okay good luck
well we might actually some some faculty
do that
right and and we have to think about
have we
prepared them for that big assessment
that big summative assessment
and so we don't often tr follow the
training module or
or model i should say of of kind of that
your your couch to 5k
so to speak so what we often do is we
we talk at them for 50 minutes or 80
minutes
and we you know sprinkle them with
pumping
or verbal closed questions like i'll you
know i'll make an example
um you know from my discipline which is
biology i'll say
so dna stands for deoxyribonucleic
acid right and everybody fills that in
and then i feel really good about myself
because i ask questions right
well that's not a question that's that's
that's me
trying to convince myself that i'm
asking questions
that's okay to get the ball rolling
perhaps but it's not going to get them
where you want them to be
because what ends up happening is that
we often ask all these
low ball kind of low level questions and
i'll describe specifically what those
questions are here in a moment
but we do that over and over convincing
ourselves that we have
you know we've given them lots of
questions and we've prepared them
and then we sit down and we write this
assessment
that's worth 30 percent of their grade
and then
we hand it to them and then they they're
like bunnies and headlights they look at
this thing they're like
where did this come from so
what i want to encourage is that we take
the amount of time
to write questions for class
that we that we put into writing our
exams
you know when we sit down we write an
exam we really think through the exam
you know how do we you know we try to
match it with our objectives
with our student outcomes our learning
outcomes you know we were thinking about
you know how we want to
uh phrase the question but then when
we're in class
we just kind of willy-nilly ask a
question you know and it's it's and it's
not planned
uh it's not um you know necessarily well
thought out
and so what i want to do is encourage
everyone
to start thinking about the questions
that you ask in class be very
intentional
about the questions that you ask in fact
so intentional that you literally spend
a powerpoint slide
to create a series of questions
in between your your your content
transfer right
so every five maybe ten slides
put in a blank slide that says
and it doesn't say ask a question here
no it actually has the question on it
because if we ask a question if we say
ask a question here that means that we
have to in in the moment come up with a
good question right
odds are you may not come up with the
best question
because we need to be very intentional
in how we ask those questions
so i want to pause there for a moment
before i continue with this idea of of
how we intentionally ask questions
and kind of open it up to the group and
and hear what you think so far and kind
of hear of your experiences
ken i would add that i think it's more
important than ever to be intentional
and think through our questioning
um i don't know if any of you all are
teaching face-to-face classes but in my
two this week i have found that between
the protocols and the mask
and you know i'm i'm not thinking as
quickly on my feet as i have in the past
and you know i think we have to give
ourselves grace with that but i love the
idea of
thinking before the moment can and
having them on the slide
in notes that we we have because whereas
i might have been able to do that more
easily in past semesters just on the fly
i'm i'm struggling a bit to do and i
know it's just the first week we'll get
used to these routines but
i do think what you're saying is more
important than ever
thanks, Mitzi i i i appreciate that
because
you know yes we're experts in our
content but that doesn't mean that we're
not
a little bit distracted by what's going
on in the classroom whether it's right
now
during all this craziness or just
you know that the guys are out weed
eating you know outside our classroom
and that's distracting or you know
you hear a cell phone go off and that's
distracting you know it's just
if you have a plan and you can stick to
it then you're far less likely to to
miss that opportunity for asking
questions so yeah Mitzi thank you
any other thoughts um yes
i um started adding those slides
last semester within my power points
because
when i prepared my lectures i maybe
thought of those questions
but when i was lecturing i forgot to ask
the questions
so i started incorporating those and i
found that very useful
uh not just for my lecture but also it
helps the student
to understand the type of
analysis i want to i want them to make
or
type of questions um i will ask
so it was very helpful I'll say for me
and and also for them thanks Alexandra
yeah so Alexandra is
one of our pedagogical specialists in
the STEM Teaching, Engagement, and
Pedagogy (STEP) Program and
i'm really honored to work with her and
we talked a lot about that last year
about
you know creating questioning and
opportunity for questioning in the class
yeah thank you i just i just wanted to
add um
the two classes that i teach are back to
back
and so i know i often find myself in the
second class
like did i talk about this already did i
ask this question already because you
know you may or may
not have so i do think having that on
the slide will be super helpful so i i
really like that idea
awesome yeah thanks so and i want to
point out what type of questioning that
is
so there's really kind of there's two
big types of questioning uh and in fact
if uh if you're new faculty
and you had the uh honor of watching Dr.
Lumpkin talk about this
uh during new faculty orientation she
talked a little bit about the
types of questioning uh one's diagnostic
and then uh
the other two types that i'm going to
focus on are formative
and summative and
summative is kind of how we've done it
for a long time right
or a lot of the way a lot of faculty do
it uh and where
you just ask this big giant you have
this test with all these
uh big giant questions that are worth a
ton of points and
and uh it's at the end of a module
that's two or three weeks long and
that's fine and i know that a lot of
people do that but
um but i recommend some formative
questioning before we get there the
beauty of formative questioning is it is
low risk and high reward
because if you're if you're careful and
intentional about the types of questions
that you're asking
in class you can really
bump up the rigor in those questions you
know we talked a little bit about verbal
close and we talked a little bit about
pumping
and those are really low ball questions
they're underhanded questions
and um and it's fine to when you're when
you're
teaching teaching your your child to to
knock it out of the park you want to
give them a
a slow underhanded pitch but eventually
you're going to have to start throwing
some heat
so that they're used to the big day and
so um sorry for all the sports analogies
y'all
i don't know where that's coming from
but um
i i think that if if we're intentional
in our questioning
we can start thinking about bloom's
taxonomy and i'm going to talk about
that here in a second and what that is
and how we can think about our questions
and relate them directly to bloom's
taxonomy and even
help us write our questions with that so
formative questioning is really
important
so that's a perfect way to segue into
this idea of
using bloom's taxonomy i'm going to go
ahead and open up
this slide here i'm sure you have seen
this before
um and you've probably seen it in
various ways i'm gonna share my screen
everybody should see a big multi-colored
triangle on your screen is that what
you're seeing
great thank you for the nods that helps
all right so this is bloom's taxonomy uh
this is
noth this is not new this is like from
1956
so this is nothing new and we've been
using this
in bagley development since 1956
well not that i remember that but you
get my point um
you'll see this if you google bloom's
taxonomy you're going to see it in a
pyramid you'll see in a square
you'll see in a table you'll see it in a
sphere and a circle in all kinds of ways
shapes or forms
this has been researched ad nauseam
it still holds today you may see some
slightly different words in these very
tax not
various taxonomic levels so for instance
where you see remember and understand
here at the bottom
sometimes you'll see comprehension or
comprehend
recall things of that nature but i want
to
point out the the the reason i like it
in a triangle
is because it kind of it emphasizes
um this idea of a foundation right
because
you know if we think of like maslow's
hierarchy of needs and we think of
pyramids and we think of you know good
structures
triangles and pyramids are a great way
to think about that because the base is
really important
so that's where we come up with this
idea of remembering and recalling facts
sure there there is a time where lecture
is absolutely appropriate
sometimes you just have to get the
information out there so they've got
some
some working knowledge of the content of
your course and
in the context of your course but our
goal
shouldn't be to stick around the lower
levels to remember and the understand
um if you're teaching a 1000 level
course sure you might spend a little bit
more time there
but if you're teaching two thousand
three thousand four thousand level
courses and certainly graduate level
courses you don't want to stick around
the bottom of that pyramid for long
the idea is to move our students up that
pyramid
we want to get them if we can get them
to create or in
in some cases the way you might see this
this pyramid you might see synthesize up
at the top or near the top
this is this is kind of graduate level
or senior level type stuff where we want
to see
students start to create some original
work or some some predictive analyses
and things of that nature
so but we have to if we're again we're
if we're going from couch to 5k
we've got to start down here we've got
to get the first
you got to go buy a decent pair of
tennis shoes and then we got to get out
and we got to walk around the block
first so this is where remember and
understand
come in so when you're creating your
slides
for your intentional questions consider
a three-part question
on that slide the first question could
be a remember
you know could be a basic definition
type question
do you know what this word means and
then maybe the second question
is moving more towards apply so how
would you take that definition and apply
it to this formula or apply it to this
theory or apply it to this exercise
and then maybe further down the road in
that class
or that class period or maybe even the
course itself
move them towards analyze evaluate
create start asking them to
maybe predict what a change
in that thing that's that the topic of
discussion
might result in you know if you know if
if if you're talking about positive
integers how does it change if you
change it to a negative you know
everybody can find an example in your
discipline
and so um this is just one way to do it
now
what i like about this particular um
visual is that off to the side here it
really defines
um what is in each of these sections
and it also includes verbs
so i want to point out these verbs when
when we're writing learning objectives
when we're writing questions for our
students
whether it be formative questions for
class class time
or summative questions summative
questions for an exam
reference these verbs this it's kind of
a cheat sheet in a way
it really helps you lock in on the type
of question that you're asking
if you want to ask them if you want to
get them started with
remember say define pcr
so you know everybody's everybody's
probably heard now of pcr
so pcr is actually a a term for
um you know if we're thinking about
covid pcr as a type of testing right
stands for polymerase chain reaction and
we're not going to talk
any more about that but define pcr
um that's that's a pretty low ball
question right because that just means
do they recall
have they heard it before and then i
might say well how would we apply
pcr preliminary chain reaction what what
do we use that for
and then you know so i could say
see here use pcr to describe
how it would be helpful in determining
if someone is coveted positive
well that's a that's a much bigger
question right
and then i might say eventually in a
class i might say all right now we're
going to design
a pcr experiment so that you can test
for uh for covid or for the presence of
some
gene or organism right so i took the
same idea
i started off with very low ball what
what is pcr
then i said well what we use pcr for
and now i want you to come up with an
idea on how to
use pcr to answer your own question
all in the same slide now you know you
may need to spread that
out through a class so maybe have the
the lowball question in one slide
and then have the middle level question
somewhere down the down the road in the
in the lecture and then at the end of
class maybe say okay
here's what i want you to think about
for next class period
get with your you know have your
students
you know start a group me or something
like that because they do that anyway so
you might as well
just acknowledge it um you know talk
about this
and and we're going to talk about it
next class period on how you're going to
create this sort of experiment
so let me show you
another uh great resource
for um for this these verbs these blooms
taxonomy verbs so this actually comes
from
linda nilsson's book
this is i mean if you google bloom's
verbs you're going to get back a lot of
options but this is a book called
teaching at its best it's i don't even
know how
old this book is now at this point it's
been around for a long time
um but it is an amazing resource but
this is a
a bunch of those verbs and i have to be
honest with you i have a cheat sheet
well i don't have it right here
these days but when i was teaching full
time in biology
i actually had a printout of this in a
sheet protector taped
on my wall next to my desk and when i
was writing a quiz
i could just reach is a cheat sheet i
totally looked over there and i said
okay i need to write a good question i
want to get them towards application
all right um interpret ooh that's a good
one
so then i would design a question around
the word interpret
it's it's such an awesome and simple
reference
that i really encourage you to use it
and so literally print the sucker off
put it in a in the top drawer of your
your desk
and reference it as often as you can
so this is how you can be sure that
you're covering
these various blooms taxonomic levels
right
so i think i saw a chat come in i want
to go ahead and look at that
let's see here yes okay you get a um
a cheat sheet a copy of this cheat sheet
yeah you can i will um i will do that
this is
this is actually not a pdf this is
actually a screenshot i
actually scanned this page from the book
but
um i will i will
actually Suzanne might have one that's
handy
maybe not but um yes
i will i will get one as we go to our
question uh
and kind of the the crowdsourcing
component i will grab that and i will
put it in there
but yeah you could literally bloom uh
google bloom's verbs
and you'll get a thousand hits on this
so
thank you for that question um
so let me pause there and let's talk for
a moment
about this idea of creating questions
using these blooms verbs
what your experience has been with using
bloom's verbs awesome Mitzi thank
you so let's let's hear hear from
everybody
so i'm gonna demonstrate something here
because we're going to talk about that
here in a moment but i know that
Alexandra offered up her experience from
you know a few moments ago and so i know
that Alexandra can answer this question
and so which is which is why I'm about
to ask Alexandra this question
so so alexander
exactly see i i know that alexander
would you kind of describe
your process for creating the questions
that you said you started to implement
ah that's a good question
um well i actually like what you said i
use my cheat sheet
um because it says it's very useful
uh i think that sometimes you can you
think you're being creative but you're
really not creating
enough to bring those questions um the
other thing i do
is i try to build on concepts like you
were
actually explaining so i
used a very basic concept let's say
a microorganism so salmonella causes
diarrhea
so what if and then i try to build
a case a more complicated case so we all
know that you can get diarrhea from
salmonella from e coli but
then i try to build a case so i add
something
different so what if that salmonella is
in
eggs or white food or what is
what if the conditions change so i just
try to beat that case
and then ask the question write it on a
powerpoint
and with that i forced them to analyze
so i don't know if that helps and
actually i i want to be honest with you
all
i i was struggling with these things
when i was teaching and um i learned
this with ken
so it's been a very good tool
to to help these students
analyzing it more on theoretical
concepts
yeah well would you say that the
students
as you are introducing more rigorous
questions
are you seeing that the students are
able to rise to the challenge
absolutely absolutely um yes and
and i saw that actually change
uh during my lectures so when i was
lecturing i usually
i teach microbiologists like any biology
class is very straightforward
is is black or white um
but when you when you don't ask them
questions what the students do is just
memorize
but if they have a different scenario
then they infor they are forced to
analyze so these
questions are a very good tool
that we can offer them to
to train them in analyzing situations
rather than memorizing
and that's something actually i had to
learn i had to learn how to
help them analyzing because you
you as the expert on the field you're
intrinsically able to analyze but you
really don't you're not aware that they
can't or they
they don't know how to do it so
yeah and you bring up a really good
point how many of your by a show of
hands or clappy hands and the Zoom
thingy um how many of your students ask
for a test review
like pretty much everybody right well
the reason they are asking for a test
review because they have no
idea what kind of questions you're gonna
ask on an exam
right they are they are they're asking
for anything
any clue that you're gonna that you can
provide ahead of time
of what's gonna be on that exam
how many of you hate that question do
you are going to have a test review
but if you're asking questions
all the time they are far less likely to
ask for a review because they're going
to have a pretty good idea
as to what what you're going to ask on
the exam
if you're pushing them and increasing
the rigor as alexander just described
getting them to analysis i mean that's
that's way up the bloom's pyramid right
if you're getting them to analysis the
exam
they'll be fine they're going to feel
prepared
because they probably are prepared
because you have been formatively
assessing them and they've been
formatively assessing themselves
throughout the class module before they
get to that exam
the the request the request for the
review is almost
always because they have they have never
really been
asked rigorous questions and they're
scared to death about that exam that's
coming up
so consider that now i will i will give
you a little a tip
um and this is something i did back when
i was teaching cell biology
um so um this is a nice way of providing
them with a review so i do
uh various techniques and and and um
strategies one of them happens to be a
muddiest point
and so i will ask them uh you know
please you know i would pass out index
cards or i would go
get colored sheets and i would slice
them up and you know into
kind of cheap index cards and i would
pass them around
and i'd say please write your muddiest
points on
this card and then we would do a series
of switching things around and so and so
forth and i would take some and i'd
answer them and we had a lot of ways of
answering those questions but it's
basically where the student
says i'm completely lost on this topic
you know
and and it's very low risk they don't
have to raise their hand and say i
don't know what i don't understand how
pcr works right
they can just say what the heck is pcr
please explain this again
all right that's that's telling me i
didn't do a good enough job of
explaining it
so i do this over the course of of
of a module we'll say for exam one
and what i did and i wouldn't tell them
that i was doing this
but i would i would pick up the muddiest
point cards
at the end of every class meeting that i
did a muddiest point i wouldn't do it
every class period but i would do it
probably once a week and i would take
those cards and i would literally
type them out into a form and do a
do a word document and then a week
before the exam
i would pass that word document out and
i would say
if you can answer these questions you're
in good shape for the exam
so i basically you know hopefully we
addressed their muddiest point questions
in class
because that also encouraged them to
come to class right
there um but it encouraged them to come
to class because they know their money's
points are going to be answered
but they also know they eventually
learned that those muddies points were
going to show up
in their review that they could then
answer
as a study guide prior to the exam so
that's just one way that you can use a
review
that's not really the creation of a
separate review if that makes sense um
i want to kind of switch gears a little
bit and talk specifically about
something that i just demonstrated with
Alexandra
i just cold called Alexandra right
and the word cold calling has gotten a
bad rap
for a long time right um because it's
seen as putting a student on the spot
it's seen as
you know you know it's gonna make them
it will embarrass them it puts them in
an awkward position
and for years and years and years we
were told oh never never cold call
anybody because
it's a bad teaching practice elise
dalamore
she's actually a communications expert
um i believe she's at northeastern
she was at northeastern she i'm not sure
where she is now but
she's kind of the um the cold calling
expert
a lot of her research is specifically on
cold calling in fact i'm going to go
ahead and share my screen again
i've got a paper up here for you to take
a look at
so this is one of her older papers this
is a 2012 paper
and essentially are you all able to see
that
okay cool so this is the impact of cold
calling on student voluntary
participation
i'll go ahead and i'll go ahead and
bring the abstract for you
to go ahead and look at it but basically
cold calling leads to voluntary
participation that's what she was able
to determine
now let's let's talk about this idea of
cold calling for a second
yes there is a punitive way to cold call
and it's not
it's not good you know it can actually
lead to students shutting down
if we're if we're not careful but what i
did a few minutes ago
is i opened up the the the discussion
for anybody to weigh in
and Alexandra weighed in and she
described what she was doing right
and i know that she's doing this because
we she and i have talked about this for
a year
right and so when it came time to talk
specifically or when i asked questions
again
a few minutes later it was silent right
nobody wanted to volunteer
well i just happened to know that
alexander would know the answer to my
question
and so then i singled her out and sure
it was a little awkward she was a little
nervous at first
but i had i promise i did not know that
she would actually have the cheat sheet
right there and i was like
oh sweet she's got her cheat sheet handy
but um but i knew that she would know
the answer it just so happened that she
had the cheat sheet handy
um you can do that in class as well
so let's say for instance you want to
try a think pair share
okay so you ask a question because
you've got your slides that are up
and you ask a question and you say okay
everybody i want you to go and take a
couple of minutes and turn to your
neighbor six feet away
and see what they think about this
question now
a quick recommendation on how to um
deploy a think pair share be very
intentional in what you want them to do
during that think pair share i'm i'm not
a big fan
of saying okay you guys discuss this
question
it's okay it's it's it's all right but
it's not very direct
um i'm a stem guy so maybe i'm a little
too direct but a lot of times what i'll
say is
i want you guys to discuss this question
and i want you to defend your answer to
one another
oh that's that's very specific isn't it
so why
why is your answer correct now if they
both have the same answer
then you know talk talk amongst
yourselves as to why that answer is
correct
so um so during the think pair share
now right now it's a little bit weird
because it's
difficult to walk around our classroom
and almost maybe not even allowed
but you that doesn't mean you can't
still kind of move to the side
and listen to what's going on it's very
likely that you're going to hear
exactly what you want to hear in that
discussion and so if you're at the front
of the room
excuse me and you hear um mary and steve
talking about the answer to that
question and
you are hearing exactly what you were
hoping to hear
you can always walk over and say mary
you are right on the money on that
question
you know what here in a few minutes i'm
going to ask if you would wouldn't mind
kind of explaining how you came to that
conclusion or how you came to that
answer
and i'm going to ask you to go ahead and
talk in front of the whole class
and explain how you got there what you
have just done
is you have prepared that student for a
cold call
you have now she's gonna freak out for
just a few minutes
but what she's gonna do is she's gonna
you know zone in
lock in on how she wants to answer your
question
so that she's ready for it when it comes
it when it comes to her
and she's going to be prepared she's
going to be confident she will know that
her answer is correct because i told her
her answer was correct
and she's going to then hear her voice
in that classroom
the other students are going to hear her
voice in that classroom they're going to
see her confidence they're going to see
that she
you know had you know went into this
with with clear confidence and a clear
response
and so when i when i'm when i bring
everybody back together as the whole
group i can say
all right everybody so um i was
listening into marrying steve's
discussion
and i would just i i would like to hear
what mary had to say about
uh this particular topic because it was
something that that i was hoping that
y'all would think about and she really
did i wanna
so i see i'm building her up i'm
building her up building her up and then
then all of a sudden she's now she feels
like a million bucks
as she sits there and says well here's
what we discussed and here's how we
we can came to a conclusion
other students see that and they're like
cool that that wasn't horrible
she handled that but what it does is it
also
says i'm gonna do that again so you will
step you put this
you build this into the fabric of your
course you make sure that students are
aware
from day one that you that this is going
to be a discussion based course
you're going to ask questions and you're
going to expect them to answer it
now what elise dalamore determine is
that you only have to do this a few
times
you only need to cold call a few times
because what ends up happening
volunteerism increases they start to
hear their voice in class
they start to see other people speak up
confidence grows
students start to realize that you've
got a very comfortable classroom
environment
it's okay if we're not quite 100 correct
because nobody's going to say no you're
wrong
everybody's you know if if they had the
answer to all the questions we wouldn't
need to be in the class
right so the beauty of this is that it
slowly builds rapport in your classroom
that's the key to cold calling building
rapport
building a comfortable environment in
your classroom
and so i want to draw your attention to
another a more recent dalmore
publication
and this one is really cool
so she actually determined in a 2019
paper
that how cold calling actually levels
the field in terms of gender equity
so this is a 2019 paper and let me
reduce the size a little bit so i can
pull up the abstract and basically i
will draw your attention
to this basically results indicate
in high cold calling classes that women
answer the same number of volunteer
questions as men
that's pretty cool huh additionally
increased cold calling did not make
either group
uncomfortable differences were observed
between men and women in low
cold calling environments where women
answered fewer questions
so basically by by cold calling
you can be intentional about who you're
calling on as well as what the question
is
but we actually see that there's an
increase in volunteerism
and success with both women and men in
your classroom
so cold calling actually makes your
class more equitable
i mean how cool is that it's 2019 paper
and so
um i think that that
tells me this is something we're talking
about we're doing in class
but again i will stress that it's really
important that we
we build this into the fabric of our
course from day
one don't don't save it till the middle
of the semester when they're used to
being lulled into this kind of
spoon feeding process where they can
just passively sit in class
and not engage because when you start
cold calling it's going to be like whoa
where'd that come from
you know we don't want that to happen we
want to introduce them
introduce it early get them used to it
you know start that that couch to 5k
early right so i'm going to take a
glance at the chat here for a second and
i'm going to
offer some uh offer for the next
eight minutes i'm gonna offer some tips
on how you do this online
and then in the last 10 minutes i want
to kind of open it up to the group so
let me take a glance at the chat here
for a second
come on chat open up
there it goes all right
please please jump in uh first of all i
totally agree with kent what ken just
said about cold calling
i've now taught my two classes for this
semester they're both graduate classes
at night
and i was i was totally unprepared
for the hesitancy of these are graduate
students
you can imagine it will be for
undergraduates to speak when wearing a
mask
and i i did none of us want to be in
masks
we all are but it there was a great deal
of hesitancy
and as i began to ask questions because
i'm a big believer in questions
i think the cold calling and the
affirmation that it's okay to have a
muffled conversation
to a class as a whole and to a class and
a think pair share or talk to your
neighbor about
a question that was phrased to the old
class and i could just see a measurable
change over again it was a two and a
half hour class period
we've got to reaffirm to these students
that we're not
that abnormal i mean the new normal on
this campus and
elsewhere is wearing the mask and so
what we have to do
is we have to say you can have a class
you can interact
and meet your classmates in this format
and we've got to affirm it
and affirm it and reinforce it and they
don't have to be in each other's face
they can be
a reasonable distance apart and they can
still
talk with each other they can still
answer questions you can still cold call
and i think we have to build their
confidence even more so than what ken
just said
under the current circumstances so i
encourage everyone to follow what ken
said
and then maybe even do it a little bit
more to
to break people through now just read
one comment i'm setting up meetings with
my students and
i've already had heard from one student
i was concerned about
coming to class but you assured
reassured me that it's going to be all
right
and i think that has to do with the
distancing i think it has to do with the
mask
and it has to do with them wanting to be
college students and
and continuing to learn during a
pandemic
Angela thank you so much for that i mean
that that couldn't be more timely i
appreciate you mentioning the mask thing
because yeah it's it's
it's noticeably weird right so i'm sure
it does have an effect thank you for
that
i can can i also point one other thing
out that's um
a little bit indirectly related but but
maybe not
please um you just did something and i
think we
we love to do this at the TLPDC which is
to talk about the way that we're
we're actually leading a workshop and
point out a strategy
and I feel a lot of pressure in teaching
in an online context
to to manage everything
at one time and to keep my eyes on the
screen and and
also to keep my eyes on the chat and
it's impossible
um for me at least i can't do it and you
were really intentional about saying
hey i'm going to stop now i'm going to
pause and i'm going to take a look at
the chat
and see what questions we have there and
see what's happening there
so it's another way to prompt questions
and and also to take a break break and
cut yourself some slack as the
instructor and say i'm going to have a
look at this now let's see what great
questions were in
our comments were in the chat so i love
the way you did that and i just
wanted to point that out thanks for
mentioning that Suzanne but you'll also
notice that it left some silence
and some space for Angela to jump in
right so so that's that's another reason
that that worked out really well
awesome thank you Suzanne so
um i want to offer
something that is a fairly recent
discovery
um you know on how to do this
in a distance modality uh specifically
in a synchronous
distance modality i'm not sure if
anybody is doing that right now but
um so I'm part of my gig here
in the TLPDC is that I'm working with
learning assistants
which are undergraduate students that
are embedded in live courses
and i'm working specifically with two
different uh departments
but this example is over in calculus so
I'm working with Dr. Brock Williams who's
you know associate chair over there
professor you know mathematician
extraordinaire member of the teaching
academy
he's a rock star teacher right um
we have been so over the summer we were
we implemented la's
and what we were finding is that we had
these
um students where they're they were not
engaging
the the the cameras were off the mics
were off
and even we would ask you know pumping
questions and verbal closed questions
and
uh we would go into breakout rooms and
we do all kinds of stuff and
and the la's were prepared and ready to
go and yet when they go in the breakout
rooms they would have these
black boxes with their name across it
and so we're we're really trying to
figure out how
we're going to get them going so we came
up with this idea
and we it was kind of a an experiment at
first
and the increase
in uh engagement has
tripled and i'm going to go ahead and
explain to you what we did
this is a method that's all carrot and
no stick okay
um so what we did
was um so it's set up so there's a
certain number of
learning assistants to uh to to groups
right so in this particular group we
have you know about 10 students for
every learning assistant
you don't have to have learning
assistance to do this
what you can do is you can create
breakout groups and if you haven't um if
you don't know how to do that
our colleague here in the TLPDC Erika
Brooks-Hurst has has done some
YouTube videos so I point you in that
direction where she describes how you do
that
um we can also talk offline if you need
some help on that but anyway you can
create breakout groups in in Zoom
and I suggest pre-assigning
certain groups to certain students or
certain certain students to certain
groups so
these groups kind of start and they
maintain as the same group
and what we did was during the
during the class, Dr. Williams would
actually have a moment where he'd say
okay i'm done talking
we're going to ask a question now and
you're going to solve a problem
so we're going to go to break out groups
we're going to spend five minutes you're
going to solve this problem
go bam and then everybody shoots into
their breakout groups
and then their assignment is is that
they
they need as a group they need to come
up with the answer this question because
when they come back into class
he is now um i i will uh tell you uh Dr.
Williams is a big Dungeons and Dragons
fan and he's got all these dice
so he loves to roll dice to to select
different groups randomly right so um
when they come back into the main group
he rolls his
20-sided dice and selects a group
so group eight all right group eight
that's awesome so group eight you're
gonna uh need to answer this question
and then he rolls a dice for an
individual
within that group so it's random so
let's say
Suzanne awesome so you're you're gonna
need to answer this question
so again they the students know about
this well in advance
this is part of the fabric of the course
and so that student then gets to
weigh in on the question now it's up to
you to determine
what is a quality answer is it did they
set the problem up correct did they
you know get the right answer did you
know you can
create your own rubric for what is an
acceptable answer or what is considered
correct right
but here's here's the thing the biggest
concern with the groups
was that the students were not engaging
you had some groups that were
or some students that were some green
students that weren't
here's the all carrot no stick part
so if the student gets it right
everybody in that group gets a bonus
point
now it could be towards the exam towards
the final grade towards the quiz grade
towards the participation rate it
doesn't matter
because points are king right the
students want to see points
if they get it wrong no points are
conferred but they don't lose anything
either now again what is wrong
we have to be careful about what is
wrong you know we we often in the teal
pdc will you will use this term kernel
of correctness
where we you know identify something
about what they said that is correct
and we kind of draw it out of them and
if we do that
and then they're on board and they can
answer the question then
it's correct right because we want them
to continue to respond
so if it's correct then boom everybody
that
um that group gets a gets gets a point
so what does that do that actually
raises raises student accountability
within that group to make sure that
everybody is on the same page right
so if i'm a student so if i'm in a group
with Suzanne and Mitzi
and I'm the guy that doesn't quite
understand what we're talking about but
Mitzi and Suzanne
absolutely understand what's going on
they're gonna make sure
that i understand because when it comes
time to share with the group
their extra point rides on me being able
to answer that question
right and so i because i'm the slacker
you know
slacker guy that doesn't know the answer
and i always have my camera off
you know they're going to make sure that
i'm on it
it benefits Suzanne and Mitzi too
because
they have the opportunity to explain
this concept
in a couple of different ways to make
sure dummy ken gets it right
and so um it's it's a win-win-win all
the way around
and we saw we went from in in so i
you know as a co-host in these these
classes i can kind of jump around in the
various groups
and it was amazing to see
cameras on students engaged
the black boxes were no more they were
gone microphones were on there's
chatting chatting chatting
and a byproduct of this was that the las
now are competing with each other
as to whose group is better and and it's
it's
it's awesome and and so like i said you
don't have to have la's to do this
you can put the accountability of the
students and then you can go in and you
can jump from group to group and kind of
see
make sure that the accountability is
working but they are accountable to each
other
and to themselves and so you very you
have to spend very little time
you know making sure that they're
staying on task so that's
that's just one example and if you
wanted some more details on how we did
that
um i'm i'm happy to share them with you
but it's turned out so well that we're
actually talking about trying to write
this up so
um i'm i'm really excited about that but
in our remaining time we've got about
eight minutes left or so
i want to open it up for your questions
and for
your uh your comments and your
experiences and things like that so i
will hush and i want to turn it over to
y'all
do some of these techniques work in
asynchronous
classes that's a really good question
hey so we we like to recommend
asynchronous because it's a lot easier
to deploy
and and and so on and so forth they do
work
in asynchronous classes as long as you
have some sort of substantive
interaction with the students either
through a discussion group
or a live chat or
an asynchronous posting type process
it does work but if if you treat your
distance course like a correspondence
course
then it's less likely to be that
effective
in fact we we really recommend against
the idea of treating your distance
asynchronous course as a correspondence
course there should be some form
of interaction um and but yes it
it will work if you build it into that
interaction
thank you patty and just uh
hit on a little bit of how i'm doing my
asynchronous course
uh i have so what i'm trying to do is
you know essentially add in a slide with
a question a series of questions and i
kind of do it the
low ball medium and then a pretty tough
question that's going to have discussion
and then because thankfully i have a
still a face-to-face time where i have
uh
discussion time with my students the
students broke into three parts
i can still have those questions
answered because i have tried already
this year to get them to use the
discussion board
and that is not working they are not
they're not utilizing the discussion yep
yeah so thanks darren um did i lose your
audio
were you still okay okay good
because i'm getting the little bars that
says connecting to audio sorry
um so yes if you have an asynchronous
course where it's
somewhat hybrid where you're delivering
content perhaps in a semi-flipped mode
and the content is going in an
asynchronous
method where they watch the videos ahead
of time and then class
is treated as kind of a time to to do
application
then that class time is a really good
way to be able to spend
uh that um that time actually asking
them questions and deploying some of
these ideas
but yeah the the thing with the
discussion is that
you have to a a stagnant discussion is
forgotten very quickly
um you have to it's like if you're a
Twitter person or any kind of social
media person which i'm not so
if you are um i will defer to my
communications experts
but um if if you're not constantly
updating that feed
people eventually start to ignore what
you're saying and so discussions are
really
important if you're going to do it you
have to make sure that you're constantly
feeding that discussion there's a lot of
care and feeding to discussion that has
to be done
so um i see some things coming in
through the the chat here
um let me see here some students gave me
feedback last semester that they
really disliked being asked questions at
8 a.m in the morning
any suggestions for getting around
student fatigue and then uh
Suzanne weighed in uh I think your
energy will influence the class
and the early expectation that they will
be participating despite the hour of the
day
yeah eight o'clock is is difficult right
and especially depending on the the
concept or the the the topic of
conversation because
i'm currently one of my learning
assistants courses is an 8 a.m chemistry
class
no human should be subjected to 8 am
chemistry right
but the the the fact is is that that 8
a.m is still when we're teaching
and so yes Suzanne's comment of if we
walk in there
excited to be there and we um uh
you know excitement is contagious right
maybe contagious is not the right word
to use these days but
um it's it's really important that we
model how to be excited and interested
in the material
so if you walk in there with your coffee
and you're like
all right y'all we're gonna balance some
equations today
um then they're gonna be like okay i'm
gonna sit here and watch the youtube
right um but if you walk and say all
right everybody let's go ahead and get
started here's
you know what we talked about last last
time here's what we're going to do today
you know have some energy you know if if
you don't leave your classroom exhausted
you're probably not doing it right
right so so thank you for that uh
that topic and for the response Suzanne
other thoughts
and I just have a question so you
suggested for
online classes that one of the options
is to
break them into the rooms and etc but if
you don't do that
what other suggestions are there for
questioning or to engage students so if
you don't break them into breakouts
right so um is it a synchronous course
or an asynchronous course
synchronous so synchronous so um
well we tried in in calculus we tried
not breaking them out
um and the reason we ultimately did
breaking
we broke them out is because we weren't
getting the engagement we wanted
and so it's kind of that problem
that we have with face-to-face courses
when we're in a large lecture hall and
nobody wants to speak up because it's
intimidating
well now nobody wants to speak up
because they're they're get you know
they got this camera right in their face
and it's really easy to not speak up
because all i have to do is
turn off my camera right then i'm just
uh i'm just an anonymous participant
you can you can beg them until you're
blue in the flat to fade
blue in the face to ask them to turn
their cameras on they will not do it
so that's why we actually implemented um
breaking them out into groups so we
tried several ways to try and get them
engaged when they're in the whole group
and we had
we we failed at every turn so yeah
i don't have a whole lot of successful
ideas on how to do that i have a lot of
failed ideas on how to do that but but
not many successful ideas on how to do
that
in one group so i if if there's a way to
group them up and put them in
in breakout groups i i would really try
to figure out a way to make that work
again can i ask something else please
value
so based on your experience
and the other participants have you seen
any difference
if we just put up a question in a
PowerPoint slide or if we use any
polling system
does it make it easier for the students
to engage because it's kind of
anonymous equally effective or
any suggestion yeah absolutely polling
software is fantastic
a lot of times faculty will not use
polling software because it's an extra
cost or it's
you know it's not approved by the
university or all kinds of various
reasons
i want to show you something very
quickly and this might answer
Alexandra's question so even if you
don't have polling software
there's something called abcd cards and
this is
it's a free app it's from west
university of western washington
so if you wanted to ask a question on
the screen
and use if you make it in a multiple
choice format
you can say okay everybody here's the
question of the day um
or of this ten point or uh ten slide
question
formative question let me know which
what you think it is
and then they click on d and they can
literally
hold it right here in front of them in
front of themselves oh there we go
they can hold it for themselves so
they're not like everybody's not looking
around to see what the answer is right
but you can see what the answer is and
you can see
if you see a bunch of blue in the room
that tells you if you see a homogenous
you know room of blue that tells you
that that
you know they're on on target and you
can move on
with the next question now if you see a
rainbow of fruit flavors if you see red
yellow
green blue and it's very heterogeneous
that tells you that there's some
confusion
so you might want to back up and
rephrase your question or review that
question
or have them break into peers and say
okay now turn to your neighbor
and get to a consensus let's find out
what the correct answer is
this is actually recommended so eric
missouri did a lot of work on peer
instruction
physicist so valia and giannis you might
be familiar actually i know giannis
knows eric missouri how cool is that
um but anyway he did a lot of work on
peer instruction hey honest
did a lot of work on peer instruction
and this is a really good way to assess
the room
and so um alex if you're still in the
room
you can you can try that method to see
how that works
because now they're still gonna have to
turn their camera on
and ooh i just thought of a side hustle
we can come up with an
app that turns their the black box
into a color so they don't have to show
their face they can change the color
anyway just an idea but um
that's the abcd cards is a way of doing
polling volume
where you don't have to use software to
do it um
but yeah absolutely the polling software
so you know various publishers have
you know top hat and pearson mcgraw-hill
they all have some sort of pulling as
part of their
their thing so yeah i totally encourage
polling it's a really good way
to to assess your students poll
everywhere is
free and if you have a class below 40
everybody can be
independent and it's anonymous
absolutely thank you for that Angela
and Zoom has some polling built into it
where you can actually use the built-in
uh utility to pull within Zoom and and
it's
really pretty easy to use so i encourage
you to try that as well
so i i do want to make sure that that
i'm respectful of your time
it is a couple minutes after we went
over um
thank you so much for coming to uh to
this session
and contributing uh got a bunch of great
ideas from everybody
i'm happy to share anything that that i
have included in this
this powerpoint or i'm no powerpoint in
this presentation
but um if you have questions or comments
or you want to reach out and discuss
this some more
please do so um i really appreciate all
of your feedback
and i i guess we'll kind of end it there
cool
