Hi welcome to this course on autonomous
cars 101 this course is meant for people
that know how to use ROS and have
interest in starting with autonomous
cars in this course we are going to
learn how to use the sensors available
in in any standard autonomous car how to
navigate using GPS also generates
algorithms for obstacle detection and a
security system implementation and
finally we are going to use the can bus
protocol as all the cars usage in the
industry and at the end we are going to
do a micro project where you'll have an
objective moving your car along the road
to get to the gas station ready so the
first step is obviously going to Robot
Ignite Academy and login to the web page
then select the course that you want in
this case we're going to select ROS
autonomous vehicles 101 and you'll get
something like this you have a notebook
where all the details and all the
explanations are you have an IDE where
you generate your code then for weapons
and the simulation
so let's start with a simple exercise
which is moving the car this is a very
good exercise to get a feel of how a
simulation world works in this case how
this car handles so this command in
particular is it's called unsafe test
because it's not using all the security
systems that are already online in this
simulated car in this simulated car do
you have
obstacle detection and if we ought to
use the standard topic where you where
you send them the moving commands it
would stop when it detected some
obstacle also it has a system that if
you don't publish with a certain rate it
will stop the car so for those two
reasons
we are using this unsafe just to move
the car around so you can turn become
you can go backwards and turn backwards
you can stop and you can increase or
decrease the speed so we can increase
the speed and stop it
that's it so play around with the car
and when you're ready let's go to the
next unit welcome back again so this
unit is about sensors so you're going to
learn how to use the laser how to use a
camera and how to use the GPS let's have
a look on how to to see all this
information the best way to see this
information is with Argos but first we
have to know which topic has that
information so let's have a look let me
see there we have it
so now in this case we have the laser
data in this cat vehicle front laser
points
let's have a look there we have it it's
the basic sensory messages laser scan
message nothing strange about it and you
will ask okay how can I see this laser
data of course you can do a roster pic
echo which is useless in this case
because you won't have the idea of what
the car is really seeing so let's
execute our list
just type Rosslyn RVs are Vince and wait
a few seconds
there you have it the first thing you
have to do as you might well know if the
leg the base the fixed spring in this
case we're going to select that cat
vehicle autumn then that's at some let's
add a robot model to have some sense of
reference of where the car is and then
let's add the lasers there we go then we
select the topic and we should see
something hmm I can't really see
anything well I you may see some tiny
tiny dots there let's make them a bit a
bit bigger much better now so as you can
see these are three boxes that the car
has just in front and if we move let's
move as we did and cat vehicle test
package then CD unsafe
so if for example we turn you can see
here how this dated changes in real time
let's stop the car
and that's basically it's about lasers
let's go to the next set of sensors hmm
cameras this is my favorite the same
thing where do we have camera data it's
very important just because then you
have to select the topic with that data
let's have a look let's have a look
camera you have all these topics with
data advanced cameras in this case we
are going to use this image roll front
it's the front camera that it's around
in this tiny white box here in the real
car you have it not here but in the
windshield but more or less it is the
same location this one is a bit more
it's better in visualization but
actually you'll see model is the same so
let's let's have a look so
you can use our fists to to visualize of
course how just you have to select here
image and then select the blah blah blah
but we're going to do it in a different
way let's close this let's save it for
later and let's execute this our katti
image view this is a dedicated program
just for viewing images which comes in
handy there we have it so we select this
filters all the topics and just shows us
the topics related to images and we wait
there we go
so these this is the images that are
captured by the camera in front of the
car so if we move the car for example
right now it's moving around
you can
there we go marital dating
where is the carnival here's a card so
as you can see the updates it seems
quite slow we just lowered the frequency
rate to make the simulation go much
smoother in reality you'll have a higher
framerate so don't worry about that
there we go
and finally we're going to get we're
going to visualize the GPS data
the GPS base data is published in two
topics in this fixed topic and this
fixed velocity topic it's also published
in other topics but we'll see it more in
depth in unit 4 I think so let's have a
look at this in this case we are going
to use artists but we're going to use it
in a special way there are two ways of
visualizing GPS data one is generating
markers to see waypoints or seeing the
GPS data that it has in particular but
there's a very good way of doing it
which is with this and our via satellite
plugin so you have an example here of
how to do it manually we're going to use
an already predefined file so that we go
faster and you can see the results
immediately so we fired up our again we
check this path that I that would give
you here it is o at opt ross indigo cher
cat vehicle Arvest config and select the
cat vehicle GPS so here we can open this
one thanks look how to T
and can feel cool
you have to predefine
this one has all the visualization of
the lasers lighter also and so on and
this one has now I don't want to there
we have it
so what could what should you see here
so here if we zoom out really cutaway we
have a tile a map tile that is fence
from this maybe we can sleep better here
yeah from this website you can go to
this website sign in and get your your
access token that you'll replace instead
of the one that it's placed here you'll
place your own if you want to do it
manually but everything else you have to
leave it just as it as it is because
what it does is get the x y&z let's say
latitude longitude and altitude of the
of the fix topic which you're selecting
here and it's putting that data and it
puts the car exactly in the place where
the fixed data of the GPS is giving this
means that this car based on its GPS
sensor that it's brittle in this case
it's based in Germany
it's based in Germany because we started
the simulation with those initial values
but you can change those values option
obviously in any simulation that you
generate and then
this makes now we can move the car look
okay
we are moving the car around and it's
moving in the map and the laser data
obviously is updating and if we use the
camera which we could see it also so now
you have a car that it's moving with GPS
data in the real location where it
should move isn't that amazing
the cool thing of this is that you could
use these same tools for a real car that
it has GPS topics published and laser
told and I think that's quite it so
congratulations you Morrel your you have
more or less the idea of how to work
with the sensors in this car and in any
car that has rows and topics published
so if there's if you don't have any
doubts
let's continue
Unit two talks about the GPS navigation
it's quite straightforward because we
want this courses to be very practical
we start with this exercise this
exercise it's meant to get you used to
using the the GPS data in this case you
have to do an action client that what it
does is given a waypoint
it returns the distance from the that
Waypoint that means that if you move it
should return as feedback the distance
updated and it updates because it's
reading all the time the GPS data and
it's comparing it with the Waypoint
you gave him so this exercise is just to
get you working on that matter at the
end of this exercise you should have an
algorithm of some kind that makes the
car more or less go to the Waypoint
based on the distance so if if it
detects that the distance is getting
lower then it has to change direction
until it sees it detects that the
distance increases it's very basic but
it's meant to it's meant to that what I
said that you get used to using GPS data
let's have a look so here you have the
more detailed the exercise and then you
have to bear in mind this which is quite
important when you publish you will
publish in this topic this topic if is
using them let's say the note that it's
reading this topic is using obstacle
detection and a system security system
that we
talk about a bit later which is the
Deadman switch this means that if you
don't publish these the message that
movement methods the twist messages with
a certain rate in this case five Hertz
the car will stop moving bear that in
mind when you generate the algorithm to
move your car because maybe you think
you just publish and the car will
continue moving forever and no you have
to publish it with a frequency of around
five words so have a try with with this
exercise pause the video and when you
finish let's have a look on the solution
of a possible solution or more or less
what it should give you yes so I have
prepared a pair of strips which are
essentially the action valiant and the
action server that we are going to
launch so let's let's close things that
we don't need
because we are we are not over exerting
the system and then Roslin you will have
to do that I suppose you did more or
less the same then GPS action server and
then we do the same thing on the sky and
GPS action climbs as you can see I have
a client that what returns is the
distance so now if I move the car let's
say Rose large website launch can't be a
cool tip and I'm going to use the unsafe
just to go faster
and I start to move around let's go back
see what happens
in this case we finished that's because
I got the Waypoint in the place so as
you can see it was giving a feedback of
the distance and when it got to the
distance more or less at a distance of
around less than three meters which is
the distance it stopped and gave me a
result that's essentially what you have
to do just that you have to do an
algorithm that moves the car around
based on that information and that's all
so did you manage was difficult okay so
let's go to the next unit this unit
loads a different simulation this
simulation is the exact same car and
world just that this car doesn't have
the obstacle detection and also the
Deadman switch system this implies that
it doesn't work this car won't move and
what you're going to learn in this unit
is how to do an obstacle detection and
also how to implement the security
system that has this Deadman switch
protocol also you learn how you
transform the twist commands into joint
commands that move the car and they will
move the car using the Ackermann
steering model which is widely used in
any car nowadays really I'm going to
explain more or less what the ideas so
first have a look at this diagram this
is that diagram of how more or less the
information is transferred
through our system until it the car
moves the first step is the keyboard
telling this case or any commander that
it's published then this command bail is
read by this obstacle stopper cat
vehicle which is the first barrier that
sells security barrier this node what it
does is read from this distance
estimator this this this topic that
generates this distance the laser
distance which gives you the lowered
distance to any object so for example if
you have no objects around except one
just in front at 3 meters it will give
you 3 meters
based on this information and that cats
vehicle a CMD valve that you are sending
it filters it
it's a distance from 4 to the object and
also some suit to the object is less
than 3 meters or 5 meters or whatever
you you you state it inside it
transforms this command via and command
velocity methods into a zero command
velocity so the car doesn't move
if there is no object nearby then it
just copies the methods that you sent
then the next step is that and final
step is this command Ville - gazebo so
on this one what if what does is reads
from this command failsafe which it's
safe because now there's no obstacles
and what it does is record at what
frequency it's receiving this with this
messages if this frequency is not high
enough then the Deadman switch comes
into place there are many many many
protocol
of this stem and switch but in this case
is implemented that it will stop the car
until it gets the next message and what
it does this is it gets this commander
safe and send the the commands to the
joints and moves the car based on this
commander using the Ackermann steering
model yeah
so the exercise tree the publisher that
sends commands to the joints of the car
and wheels and we give you some
information about it so you have to
publish here where you have to publish
the move the car is around here so the
front left steering position controller
so this is for steering the MER wheels
and this is for moving the wheel so make
that wheel turn the one from the back
yeah so in this case it's only velocity
controller and here we say that you can
move the car as you wish you don't have
to use the Ackermann steering model but
it's highly advisable that you learn how
to use it because in all the the
simulations and in all the car systems
the real systems there is some version
of the Ackermann steering model
implemented if you don't know how this
acumen steering works have a look at
this video is really nice it's you have
a very good idea of how this is
implemented the basics at least so have
a time for the video and when you come
back I'll explain more or less the code
that use you should use and the details
around it
okay welcome back so the first thing is
generating this command valve to
receiver which what it does is reads
from the command valve safe and
publishes into that the controllers that
move the wheels it has some data for
example the dimensions of the car and
you say why do I need the dimensions of
a car because Ackerman's theory model is
based on how wide the car is and what
the distance between the front wheels
and the rear wheels it also had make
some calculations and so on and start
the timer this is very important for the
Deadman switch implementation then we
have this publish method which evaluates
first if the message the where is it the
CAD vehicle commander safe when was the
last one sent and this is how you
implement the Deadman switch if it's too
old then we don't move it's just you're
generating this float64 so here when the
messages is too old you just generate a
message that sends a command that makes
the car stop that's it then you have two
cases one is when you're steering and
the other one is when you're not this is
the angular velocity set this angular
velocity is the angle in radians is just
how much you want to turn the wheels so
you make some calculations and here is
where the Ackermann steering model come
essentially what it does is the inner
wheel let's have a look to simulation so
if we are going to turn let's say to the
right to the right this inner wheel has
to turn a bit more than the outer wheel
in this way it turns better is
essentially that also the real wheel
might have to go a bit faster than the
other one or the other way around
depends on how to turn so this is what
you're doing in this code as you can see
here the turning and then to the rear
you see is that slight is a slight
difference in to this dear reader the
left on the right see this ones are the
rears so they will go a bit faster
depending on which and in which
direction you're turning and this one
also this is the steering how much you
steer the wheels in the case that it
doesn't move it's going straight forward
it's really easy because they do exactly
the same and they move exactly the same
yeah and that essentially is the led
around here then the next step is the
obstacle stopper so this one what it
does is detect an obstacle in this
exercise you just have to read from this
this is the estimator you're not reading
from the lasers directly you're reading
from a note that it's pre-made and gives
you the closest object and
distance so the same thing try it and
then come back and see more or less my
explanation on some details welcome back
again as you can see it's a very very
simple algorithm there's nothing more to
it just you you publish in the command
failsafe and you read from this one
you read also from the distance
estimator and it's based in this
callback this means that when you get
messages from this one is when you
publish here so you'll publish more or
less at the same rate in the in both
topics and essentially what you seek
what you're trying is you see if the
distance is slower than a value and
that's it if it's lower then it's zero
the linear if it's not then it's not
that's it and that's it
congratulations now you have you know
how to visualize the sensors how to
navigate with GPS and how to implement
some very simple obstacle detection and
security systems if you ready yes
continue so this is unit four and we are
talking about the can bus the come bus
is a protocol that was generated it was
created by a bush a company Bosch and it
was around late seventies and it's a
protocol that was generated to multiplex
signals in a few electrical wires this
was created for the necessity in for
many other reasons but one reason was
because they they started to put
electronics in the cars and they saw
that they needed so many cables and that
the cable management was becoming a
monster
so they generated this this protocol
that makes transfer information through
two three wires very few wires are
needed to transfer a lot of information
it also has other features like security
features and so on to validate that the
messages sent are correct and they are
not corrupted and so on for security
reasons basically so if you know how the
canvas works just jump to the exercise
the first exercise you 40.1 if you don't
stay a bit a bit longer so essentially
the canvas messages have three parts the
identifying the data length code and the
data that you have colors here in green
yellow and red and the topics that we
are going to use that has it have this
can codification have the exact same and
variables so you'll have a variable that
has been amplifier you'll have a
variable that says how long the DLC code
and then you'll have the data
all of them will be coded in integers
and so on so in data letting bits of
data as it would be in the can the real
can so you learn that now using the
campus you will publish in the topics in
some topics that are real let's say
steering throttle pedal brake pedal and
gear selection and this will be
transforming to can information and
transmit it through the canvas to the
car systems and the car will return data
in can protocol also and you'll have to
transform that information to to
information that you can understand and
publish in the topic but that's the real
way of doing things it's not through
topics that majority vast majority
information you it will be transmitted
through canvas so let's have a look on
how this system works
and for this you have to execute the RTT
graph you'll get something in something
like this as you can see this is quite
complex and I think we have to talk
about this so this region around here
it's the simulation or it would be a car
and this region would be the bus driver
device we we have made it easier to
understand with these diagrams so what
you have in reality is this you have a
car and you have this device and this
device would communicate would would
communicate through USB with a node in
this case this canvas the b/w that will
then convert that information that both
scanned information to topics there you
have it but but that information is in
can bus standards so it will just
transform it to topics but it still
encoded this now the vehicle DB w node
will be the one in charge of
transforming this of reading this data
this is the output and this is the input
so it will publish here and it will read
from here in the simulation you have the
exact same thing the only difference is
that the device the real device it
doesn't exist and it's simulated also so
we have simulated that part also but
essentially it's the same you just
publish some topics that this vehicle
DBW
we'll read and then it will code it to
canvas protocol and from there it will
be exactly the same thing
why have I explained this here so you
have the RTT graph it's exactly to see
exactly what I what I'll show you you
have to select all the nerves to see
everything all the topics all the nerves
everything why did I explain this to you
because I want you to be aware that all
the things that you do from now on have
to be having in mind that there is no
topics there is no twist message that
you publish and the car moves you will
have to publish into topics that like
steering or like how much I press the
pedal because that's real information
and that information then will be
transmitted through the scan to the
different devices in the car that will
interpret that data and will press the
throttle on the wheels and so on yeah so
this exercise this exercise is meant to
make you aware of that make you aware
that you have to publish in these topics
for example vehicle break command you
see now it has this name space vehicle
this means that this would be a topic
that the vehicle would have the real one
would have so you have to publish here
and this will be transformed into
information for the canvas yes I have
some explanations around here on how to
publish how to get these commands
published speakers there you can't
publish them and that's it
this node that transforms this
information to both kind of canvas needs
this message being published before
which is vehicle enable so you have to
send a message through this topic to
enable the publishing of this this is a
security measure so that you don't
publish steering or throttle messages by
very accident and then there are some
special things that you have to bear in
mind when you publish this kind of
information for example in the buskin
protocol the the break command needs to
know which type of pedal is it which
type of data you're sending because you
can send how much your how much force
you're exerting over the pedal the
percentage of of how much you're
pressing the pedal or just a number so
in this case and in the throttle case
use this pedal command type 1 which
makes you send any number and it's much
easier but you can you can dig in and
see in the messages which type here if
you if you look for the break command
messages you'll have a good idea of how
it works let's let's have a look for
example
Ross message show baby messages break
commands there we go do it a bit bigger
so in this case you see that for
publishing this type messages you need
some variables here but also you have
some commands and it depends on how for
example in the pedal type you can select
if you want toll K if you want to send
in the pedal command it's okay it's
force or percentage or just the pedal
information or none if you don't want to
to move the pedal yeah
this is one thing then also the the gear
messages it's quite arbitrary because
it's it's numbers it's not something
that you could understand or you could
imagine that it's one two three four
five four gears so the same thing here
you go here and you select gear command
and as you can see here the gear command
the gear you can select 0 1 which is
Park reverse neutral low and so on and
that's what it have a try it's to
publish in theory is just publishing
into topics which would wouldn't be that
difficult but these this tiny details
like this for example you have to be
sure that it's enabled otherwise you'll
start publishing and it won't publish
anything because you didn't enable it
and you didn't wait for the unable to to
get that data so you have to do some
try-catch system that makes it that to
make sure that you publish that data at
the understood so pause the video and
we'll come back when you finish when you
need some some extra information
but the back here you have an example of
this car control so how do you move it
how do you publish this this case we did
a class with publishing this then we
enables enable then it makes a loop
which runs and tries and tries until
until it works now when you have it
enabled just wait for the physics to be
up and running this is not necessary
because do you have it already running
but for simulations it's it's good thing
to have and then we publish so we have
different methods to publish different
things so we're going to you have it all
discs here but for example the throttle
command the throttle command it's
getting this throttle command from these
DB this package DB w MKZ messages and
let's see for example here you put the
throttle value the pedal type which is
one and the rest you can put it by
default so false falls in zero and
that's it you do the same thing more or
less with the brakes and more or less
the steering has a bit but more or less
is exactly the same you say the angle
and that the lawsuit in which you're
moving you want to move the steering
wheel and the pier is just selecting the
gear so if you want to go forward then
let's see go forward drive number four
if you want to go reverse - that's quite
it
yeah
so if all went well you should have
something similar to this so have you
have a testing system to do this which
is I'm sorry we go this is an example of
how it should more or less work your
system as you can see it's not moving
because it's detecting an obstacle if I
remove the the obstacle it will start to
move again okay this is just an example
of how how it should work
as you can you might have seen it at
first it made some tries because it's
since it's trying to send the enable but
as you can know as you may know the
topics not always work the first time so
you have to try and try some until it
works and until the the channel is open
and then a few more tries to to be sure
and when you have essentially what we're
doing it here is reading the topic so if
the topic received that message then
it's good to go yeah okay
congratulations this is their main
exercise if you want to do an extra
exercise to know more about the can
protocol and so on keep on reading so
this you need C++ knowledge because we
we will we will be going over some C++
code and to generate your own coding
Python or in C++ you'll need to know how
to do this yeah
this exercise has the objective to show
the basics of campus the first things
you have to do is subscribe to the
canvas topics this is the output and
this is the input let's say then
essentially what you're going to do is
read and know what kind of data is
running there so we're going to decode
the encoded data and that's what the
exercise is all about is making a
decoder of both channels so that you
have a control of what is being sent and
you see the break commands you'll see
the throttle commands you'll see also in
V R X which is the input the GPS data
and you'll see that it has different
morphology so just just a few tips this
these are the IDs and as you can see you
have to transform them from X format to
decimal format first to know what type
of messages are sent this is one of the
first steps I would recommend then when
you have more idea then you have to dig
in in how the data is encoded and how
its how it transforms the twist messages
or the sardor the steering the throttle
messages into this integrated coding
messages because if we have a look for
example I don't know let's have a look
it was done
and we do loss topic first topic echo -1
days
yeah for example this is an example of
how the data is encoded in this topic as
you can see it makes no sense
absolutely no sense it says the data
which is 255 255 and an ID and DLC so
the first exercises know what this
stands for
is it the throttle is it the brake is it
the gear what is it and then the next
step is decoding this data to know what
what is it yeah so you have to have a
look to this BB can so let's have a look
and I'll give you there PD can yeah so
you have this extra Doc's that we we
copied here and the this is the code of
in the node that it's already running
and you can have a look inside and see
okay so I'll have a look for example
this page and you see dance okay thanks
yeah there we go you have here the
defined structures of these messages for
example this message break command has
this PCM D which is the data more or
less the data and you see that it's 16
bits of information and you you may see
if you know about can protocol is that
all the messages sent in this version of
can are 64 bits and nothing else so all
the messages have to have these 64 bits
yes
okay so good luck with this this part
this is tough but I'm sure you can do it
further references you have information
about canvass books the original
protocol from Bosch and the canvas
hardware used in this car in particular
so you can you can have all the
information available yeah okay and
that's quite it the final unit talks
about the micro project which
essentially is using so using the data
from the books can you see that this is
vehicle GPS fix is not fixed its vehicle
GPS so this is the the data the GPS data
that went through the canvas and was
decoded and published and this is the
topic that you would have in real life
then you have some information some
topics that you would have because it's
the image and the lasers which would go
in their own channel maybe and then but
in this case of this car go like that
and then you have how to control how to
move the car and that's it so with these
three rules you have to move the car
throughout the environment so here we
have environment let's make it a bit
bigger and you have the environment and
you have to move the car from here to
the gas station around here yeah so good
luck with it use the laser the obstacle
detection use the GPS anything you want
cameras you can recognize for example
the line and try to follow the line
anything be created and now yeah
now it's finished what to do now well
what you can do now is if you want to
develop simulations and go further in
the simulation development or use this
simulation the same simulations that we
used here for free go to Ross
development studio and there you you can
use them as much as you want and
practice and generate your own
simulations if you want to learn more
then go to remote ignite Academy to
learn more about Ross Ross navigation we
have our pad for example we have open
yarn we have topics about TF and each
month we have new and new topics so
you're more than welcome there
then certifying Ross also you can do
online exams and in behalf of all the
concert team and I myself Minnesota
legal we thank you and hope to see you
soon
