Hi I'm Shaozhang
Hi I'm Tianyang. In this video we'll give
you a brief introduction to our final year
project Ar Robotics. There're numbers of
high abstraction topics in robotics
related study which is hard for students
to understand. Traditional education
presented by textbook or whiteboard cannot
present a 3d content clearly. Augmented
reality is a tool which bridges the real
and virtual world. Benefit from its strong
expressiveness, real-time interaction
and multiple platform support, this
technology can provide 
opportunities to improve the interaction
between the learners and learning
materials which produce more effective,
active and motivative learning. The
content presented by AR is in three
dimensions. It's vivid, intuitive which
can help student to understand some
abstract concepts visually. The
visualization and the interactivity of
AR make the teaching progress really
attractive. As a result enhancing
students willing to study. In addition,
some results points out that AR has the
potential to improve students spatial
thinking ability. Due to those reasons, we
aim to develop an AR application that
assist student in their studies of
Robotics about spatial description about
transformation metrics and Denavit and Hartenberg
convention aka DH notation. This
project is separated into two sections.
The first section mainly contains 3d
thinking training and spatial
description through the transformation
matrix, which is the fundamental
knowledge of robotics.
The second part is designed for learning
of an important concept in the study of
the Robotics: DH notation, which is a
popular method developed for solving the
forward or configuration transformation.
Technically, we choce Unity 3d as the
development engine. We use Vuforia, the
AR software development kits to enable
the creation of AR application on mobile
devices. And, Blender is our 3d modeling
tool to polish the representation of
abstract content. Vuforia is the augmented
reality SDK that enables app developers
to create immersive AI experience. The
Vuforia sdk leverages computer vision
technology to identify and track image
targets and 3d objects in real time.
This functionality enable AR development
agencies to Orient and place virtual
objects, including 3d models and the
other content in relation to the real
world environment. 3d models and digital
information can then be overlaid on top
of real world sense and viewed in
relation to the environment through an
AI enabled smartphone or tablet. In the
development, we use the image targer feature
of Vuforia. In brief, the basic working
principle of it is Vuforia will
convert the image into grayscale form
and detect the feature points. After the
image is saved into the database, when
the camera capture an image has a
similarity higher than a certain level,
the engine will match the image with the
original image in database. Unity 3d is
our development engine. Unity 3d
provides industry-leading developer
experience combined with tools build for
AR and VR creaters and numerous AR and VR
partners. With its intuitive and tools,
Unity makes it easy to start a
working prototype and bypassing time on
low-level programming. We use blender as
our modeling tool. It is the free and
open source 3d creation suite. It
supports the entire of the 3d
pipeline: modelling, rigging, animation
simulation and rendering. Now we will
introduce the most significant content
involved in our application and followed
by the development history for our user
interface and user experience. In spatial thinking part, user 
will be provided with the coordinate system model.
Users need to rotate the model according
to the given information. And through play the
animation in AR, user will know if they
made the rotation correctly. The
interaction between the real world
model and virtual environment can help
students to improve their spatial thinking
ability. This section provide a
introduction about transformation matrix,
which is an important concept about
spatial description to the users. As you
can see, there are two coordinate systems
which present users input and the correct
transform separately. Through comparison
between the two coordinate systems, the user
can get a feedback whether they have
input a correct transformation or not.
With the visualization of coordinate
systems and transform, we hope users to
get a deeper understanding on the
meaning of transformation matrix. DH
notation is a very popular method to solve
forward kinematic problems. In this
section we'll provide students with
basic background information about the
DH notation as well as the essential
definitions. The definition of DH
parameters are represented in animations.
Each part of the animation represents the
specific parameter, so that students can
understand the specific geometric
meaning of that parameter. The virtual
representation of DH notation makes the
invisible content more intuitive. At the
end of this part users are expected to have an
enhanced understanding of the DH
notation. After understanding the concept
of the DH notation, we provide users
with a specially designed exercise
based on a real tutorial question from
Unit TRC4800 : Robotics. In order to
solve this kind of question. Users are
expected to set our coordinate system on
each joint. After that, user need to fill our DH
table. Instead of putting the exact value
inside the table, we asked user to select
the corresponding axis for that
value. For example, a is the normal
distance between the pervious z-axis
to the new z-axis. After confirming the
selection we can see if the answer is
correct or not. The main purpose of this
part is to show how the Dh notation can
be implemented in forward kinematics
question. We hope with the help of AR
technology the concept of the DH 
notation can be easier for students to
understand. After achieving the
fundamental functionality of the
application our team spent more than one
semester to improve the user interface
and user experience. In the following
part, we will introduce some significant
improvement in our application and
discuss the consideration behind it. The
instruction block has been added design
background and border to make all
instruction blocks look neat and uniform.
Some arrow icons are made to lead the
instruction block to the corresponding
button. When we build up the model for
coordinate system
the thinkness of axis were increased to
make it more visible. the end edge of the
pink track is thinner to indicate the
direction of rotation. The coordinate system
before rotation is more transparent.
What's more, the labels of axis are
facing to screen all the time. We render
different colors for different links in
the robotic arm to make users understand
the structure of it better. To make the
input field more user friendly, there are
two main modification made. Firstly the
input field was separate into 16
individual input fields.
This makes the input numbers looks
neatly arranged. Additionally it is easy
for user if they want to change one of
the numbers. Secondly
bounce and highlight effects were made
for the input field when it is chosen
to show users which blocks they're editing.
When the application was tested on an Android
device, we found that once the user press
the input field, a default keyboard of
Android would present,
it blocked most of the screen and
the way to input the number is too
cumbersome.
We imitates the numberpad on the
keyboard to create the mini keyboard
shown in the right figure. The keyboard
provides all basic functionality need to
import in the application. Additionally
when user click anywhere is not an input
field, the keyboard will disappear
automatically.
We asked one of the students who have
finished Unit TRC 4800 before,
to try our application and gave some
comments. In his first time using,
although he had few questions on how to
use the application
he understood the content and our aim of
the project really fast. In general, he said
that it is a really good supporting tool
which could help students to learn TRC
4800 better. Here is a
feedback from him. Hello, after using the
program I think it's very useful for
people studying robotics to visualize
those fundamental concepts like the DH
notation but then I guess if you let
normal people who have no background
in engineering to use this software, I
think it will be a bit overwhelming 
in terms of all the theory and thought
loaded into the application. We also
invited some students who don't have any
background in Robotics to test the
application. They really enjoyed the AR
experience and pointed out that the
application is very interesting and
using AR to visualize some abstract
content could be very helpful. However
the concept of the DH notation is hard to
understand in a short period of time,
which makes it difficult to comment on
how far the application can help students
on these topics. OK, I really like
about this. I'm someone who really
can't visualize things in my head so
when you're doing anything in a math
class or they're like ok picture how
something moves or changes or even just
picture a graph it's just not there so
when you have something like this it's
really need especially to see different
angles to see how it changes. I think
something like this would be really
helpful in general for people to really
understand the concept how it visualizes
and how it actually applies because you
can kind of start learning about how you
can make the numbers work out on paper
but you can really understand better
when you can see how it works.
We also want to have feedback from
someone who is authoritative in Robotics.
So, we visited lecturer of Unit TRC
4800
Dr. Chao Chen. In the process, he seems to be
very interested in our idea and his
comment is mostly postive. Also, he seems
to really like the 3d coordinate in
particular. We asked him to try all
functions to see if there is anything incorrect
or not clear enough. He also gave
us some advice based on his own
experience. My general comment for Shaozhang
and Tianyang's final year project
is that the work is very and
noble and very practical as well. So this
is an excellent work. In my opinion, it is
really good for demonstration the user
interfaces still need be improved for
this software to be really tested in
theatre need be more user friendly
there's all my comment.
After evaluating those feedbacks we
realize that the subsection of the
content in our application seems to
have a relatively large gap for some of
the students. From transformation matrix
to DH notation, some students cannot
understand the connection between those
two concepts clearly. In the future we
can add some intermediate content so that
students have a smoother learning curve. In
addition, we can cooperate with the lecturer
of Unit TRC4800 to design a
special tutorial which implements our
application to support students.
Subsequently the capability of our
application can be validated in terms of
how far our application can help
students in their study. Reffering to the
result, we can improve our application to
make it fit the unit syllabus better. In
conclusion we have finished all the
functionality in the application, and the
general feedback from different
background students and the lecture of
TRC 4800 is mostly positive. However, due
to the time limitation we cannot hold a
rigorous experiment on the users
performance with and without the support
of our application. But still, we think
the goal of the project is achieved to
the greatest
extent. That's all of our project. Thanks
for your watching!
