Welcome to the 2016 Nasa ames Summer Series?
Humans Are Evolutionary Successful
Species due To our inherent Drive To Explore and Migrate
This in Part is due To our Ability
To invent tools and adapt them
for the Purpose of Surviving new Environments
Today's Talk Entitled Planetary Exploration Reinvented will Be given by Dr.. Terry Fong
he Received a bachelor's in Science and Aeronautics and Astronautics from mit and
Then Followed With an Ms in Aeronautics and Astronautics also at mit
He then came to ames
from 1991 To 1994 and
Decided to Go and do A Phd
so he Went and Started A Phd and received A phd from
in Computer science From Carnegie Mellon University
During That Time he, also
Co-Founded A Company Fourth Planet
Producer of interactive Tools for Real-Time Information
Visualization
in
2004 he Came back to ames so all of you that are here for the first Time Maybe
We'll See most of You back Here?
He has Numerous Publications and Awards Please Join, Me in Welcoming Dr.. Terry Fong
Thanks very Much Jacob good Morning
It Still is Morning I think I'm Really glad to see so Many People Here
Quick Show of Hands How Many of You are here Just for the summer at least for now
Wow so that's Almost all of you that's great
That Means I can go ragging all the people who are here all the Time They Didn't come to see me
Although They were Encouraged to
Let's See as
Jessica Said There will Be some Questions at the end but
Because You get to grill me at the end I'm going to start Off by Grilling all of you to start with so
Quick Show of Hands How Many of you were Born After December 1972
Wow Almost all of You again Can Any of you tell me why December in 1972 and Specifically December 7 1972 Is important
Not yet
Well so stay tuned I'm not gonna answer that right away
But you know Just Keep it in mind next Question how Many of you Work in Planetary Exploration in our Space sciences?
so only A few so
Hopefully that Means A lot, of want to tell you will be new to you and you
Won't just go to sleep and then you know come Apologize Afterwards so Let me Tell you what i'm going to
Describe Today Is some of the Work we've Been Doing here at
Nasa ames for the past decade in my group the intelligent Robotics Group to try to come up with new tools new
Techniques New Ways of Doing Planetary Exploration A lot of what we Care about at
Nasa Across the board of Course is Learning more About the Universe Learning more About the solar System
Understanding you Know
Differences of The Moon Mars and other Places Compared to the Earth and Doing so Really helps us better understand how
We all Came to be here and to, also think about the future of how
We Can Perhaps expand Human Presence From Where we are Today to Where We Would like to be in the Future?
so Maybe
Well Start With Is trying to answer the Question I just Asked you about why December
1972 Is so important and the Reason Is that was the very last Time that Humans went Beyond Low-Earth Orbit
That was and Still Is today the state of The Art of Human Planetary Exploration
1972 December 1972 Is when Apollo 17 went to the Moon and in Particular
Image to me Really so sort of summarizes Where, we are Still more Than 44
Well Coming up on 44 Years Later in Terms of the State of the Art of Human Planet Exploration
This is a picture of Jack Schmitt
Astronaut Geologist on the Surface of the Moon With his car the Lunar Roving Vehicle
Doing Field Work With Handheld tools
Shovels
Sample Collection Bags
Walking Inside out of a pressure Suit on the Surface of the Moon and if You think about it you know that's Both Really
Exciting the fact that you know
We did get out Beyond Low-Earth Orbit that There was?
Exploration of the Surface of the Moon at the Same Time it's a little Sad if You think about it that was 44 Years ago
And A lot has changed since then which means that Part of the Question Is well can, We do things Differently Can?
We do things better or perhaps are There other Ways to think about Planetary Exploration
in Particular a Lot has evolved, over the past Forty some Years in Terms of our Ability to use tools
Especially Things like Orbiters You See in The Top Row There are There a number of Different
Satellite Systems That Have Been Back to the Moon
The Japanese The Indians
Nasa have sent Probes to Orbit the Moon Collect High-Resolution images and to use other Instruments to measure the Properties of the Lunar Surface
We've learned how to live and Work in Space the Space Station has Been Up there from for more Than a decade now?
Where People
Routinely Conduct Experiments that do Work Onboard Space Station They learn A lot about what it's like to live and Work in A really
Unnatural Environment for Humans
and of Course Nasa has Been very Fortunate
Over the past Few Decades to be able to send you know our
Robotic
Explorers far Beyond
Earth Orbit We've Sent Landers Such as Phoenix Tomorrow as we've used Mars Rovers to Explore the surface of the Third?
Sort of The Fourth Planet
And Over the past you know A few Years we've Been Doing a lot of Experimentation on Space Station With Robots like Robonaut 2 as
Well As here on Earth With a variety of Robots to Understand how Robots can be used Productively for
Planetary Exploration
But you Know if We look at where nasa Is headed, we're Trying to Embark upon a Journey to mars
We're Trying to go from Where
We are Today which is A lot of Work on the Earth Or in Earth Orbit to a future Where?
We do have humans you know on the Moon again at Mars in Mars Orbit on the?
Surface of Mars and Being able to Carry out These Missions
Requires Us to do A Lot of development a lot of Thinking of how, do We Really Make that possible These Missions as
We're Looking to go Further and Further away from Earth Are More Complex, They're Longer Duration They, Require New Technology that Require our new tools
A variety of Different things, We have to, worry About include robotics
Deep-Space Habitation Spacesuits
Communications for Paulson's Lots of Different Things that We today still don't quite have all the answers and so one of the Challenges
for all of you if you're interested is to try to Help nasa
Answer Some of The Questions of How
Do you create the tools and techniques the Methods that We need for doing future Exploration at least Future
Human Exploration of Deep Space
So what I'm going to talk to you about Today are Three of the things that we've Been Doing here at
Nasa ames in My group to Really Try, to
Expand and Try To reinvent in Some way the way That Planetary Exploration can be Conceived of can be performed
Three Parts so Three-Part Act Today The first one in Terms of Robots for Human Exploration how Can
We use Robots to Improve the way that humans live and Work in Space?
Number Two Is an Interesting Area Called neo Geography There's Been A real Revolution in the Past couple decades of how
We think About using Maps and images and?
Combining Maps and images Together so I'll talk a little bit About that and the third Is in Terms of Exploration Ground
Data System Which is a fancy way of Saying of how do you organize?
The Information that You're using to Plan and Carry out Exploration Missions
Through Software in Particular
Do you do that on a laptop is it on A tablet is it with a giant ground Control Team like we've Routinely Done Here
At nasa so we'll go to each of These in a little bit and Hopefully this will give you
Some you know Insight of Some of the Different ways to
Perhaps Explore in The Future so We start off With the Robots for Human Exploration
This is the Topic that's Been Near and Dear to my heart for Many Years Now and
The Reason That it's Really I think Interesting Is this whole Notion of how you combine Humans and Robots Into an effective Team
Nasa has Had A long History of using Robots for deep Space Exploration we've Had a long History
Although Some Of it is Quite historical now of using Humans to do Exploration in Space
But the Question Looking Forward Is how do you combine Humans and Robots Together are There effective Ways to create Teams
Robots That Can Complement and Supplement the Activities of Humans you know and how, do you do that so
One of the things that we've Been Doing here at ames is trying to look at
The the whole trade Space you know how, do you combine Humans and Robots in an effective Manner
And i'd like to point out to people that that Human-Robot Teaming Is not just what you might See in The Movies it's not Just
You know Luke Skywalker and R2D2 Just Being Closely you know Walking Hand in Hand Or Hand and Gripper or Whatever
But it's a Broader
Set of
Configurations it's the Idea that you Can Have Robots Working before
Humans Or Robots Working in Parallel or supporting Humans and of Course Robots Working Afterwards so the whole Notion of before in
Parallel or Supporting and After IS an Interesting twist
On the idea of Human Robot Teaming it's not Just about Hand in Hand it's really the, whole Idea of Looking at how
They Can be Complementary over A Broad Range of Space and Time
And not Going off to Lunch
Let me go back Up here
so
One of the things that
We did A few Years ago was Conduct an Experiment Called the Robotic recon Experiment This was an Experiment that
We ran to understand a little bit Better how Having Robots Working Ahead of Humans might Really Improve the overall
Productivity of The Exploration Make it more productive Make it more Effective
To do this you know, We set UP an Experiment Where
We we tested Exploration of an Unknown Area I'll get?
Into The Details in A minute here Both With and Without the Benefit of Having Robots Working in Advance and We try to
Use This Experiment to Better understand what are the requirements
The Kinds of Things That Are needed to Carry out This kind of Joint Human and Robot Activity in Terms of the
Instrumentation The Robots Have to Carry the Communications and Navigation the Planning how
Do We understand the Coordination between Robot Activity and Human Activity
All These Kinds of
Questions Are Things that We tried to look at
And if Any of you are Really interested in in the the Detailed results There's a nice Paper that my
Deputy Wrote Maria Bullets published Back in 2011
Happy To give you the reference Afterwards or you Can Look at it on YouTube so you try to scribble it Down right now
But to motivate this Let me Tell you a little bit about why?
Recon Or Scouting in General Is useful and to do that I'm going to tell you a little bit About
Apollo 17 as I mentioned Apollo 17 Happened in December 1972 it was the the last Apollo Mission and it was the only Mission?
Where One of the astronauts was a trained scientist jack Schmitt
Geologist By Training
Was kind of A last-Minute Replacement but he, was a member of the Crew?
Who Had Training a, background in Field Geology that Is he was used to going out and doing Field Work
Walking Around
Trying To
Understand Multiple
Hypotheses at the Time About the way that the environment was?
Constructed What are the Different geologic units how do They fit Together or Where do they come from and
One of the things that that jack did was he, was Part of A number of Sorties A number of Eevee Activities
On the surface of the Moon the second of Those Eevee A2?
Started Off From the Landing Site and They went out Along
You Follow the blue lines Towards the South Massif and Then they Worked Their Way back out on the Upper Part There to?
What was Termed Station Four iT's a location called Shorty crater you can see by the map here it's About 3/4 away Through
The Sortie About 75% of The Time Through and at That Location Jack Schmitt
Walking Around Trained Field geologists Discovered
Orange Soil
Actually Turns out to be This Pyroclastic Material
You Can See here There's This volcanic material in Orange it was Really exciting it was Perhaps the most exciting Discovery of Apollo 17
But Had Happened Three-Quarters of the Way Through This Traverse and as you Can Imagine they, were Running Short on Time They
Were Running Short on Oxygen They, Couldn't Stay There very Long
And so they Quickly grabbed some samples then They went back and you think about it perhaps the most important Scientific to govenor Paulo
17 and Didn't Really have a whole Lot of Time to Study it
So you think About how Could you you know perhaps do Better Than that and of Course the obvious answer as
Well We Had Known that Shorty Crater was an Important place Maybe?
We would have Gone There first or maybe Would have Sped up the Traverse so we Could Spend more Time There?
But the only way you can do that Is by Having
Better Information so you need to do Scouting you need to do Recon to Make that Determination
So for us it was an Interesting Question, okay so if We want to think about reimagining
Apollo 17 How
Do We do this and how Would you carry this out by using a robot that was a Background for this Experiment
We did Back in
2009
Where We went Through a sequence of Steps, we did A pre Recon sort of Planning Phase for this this Combination?
Robot Followed By Human Mission
The First Phase in Terms of Pre Recon Looked at using Satellite images, we, did some Planning with a, geologic map
We developed to try to lay out Where We Would want to do Scouting Ahead of Time Scouting Carried out by a planetary Rover?
In the Second Phase and what you see here is the K10 Planetary Rover
Under Control by a ground Control Team
Followed Up by
Some Secondary Planning With The for a human Mission and then Ultimately Carried out as a Simulated
Astronaut Mission This Is an Experiment that We Carried out Over the Course of A number of Months in 2009?
We did This at a place Called Black Point Lava flow
How Many of You Have ever Heard of Black Point Lava flow
Six A one Maybe two people Three People so black Point Lava Flow IS 65 Kilometers North of Flagstaff it's in Arizona
It's what we consider to be a planetary analog that it Is have some Characteristics that are similar
In This case to a Feature on the Moon Called the straight Wall it's A large Lava Flow of Fairly Old Lava Flows
Basaltic
Volcanic Rocks A lot of Different
Geologic Units that Is Different Areas That Have very Different Characteristics and We were Just Didn't studying This because it
Was A very large Structure 15 Kilometers wide east to west
About Five Kilometers North To South A large Area to cover if You're going to try to do Exploration
In the Style that, was Done During Apollo Which I said Is still the State of The Art Today for Human Exploration
Arizona Desert To Simulate A scouting Mission on the Moon the Robots Known as Kate in Black and Kate in Red Are
Using Their Onboard Cameras and 3D Laser Scanners to Take images and Map the Terrain, we're Looking at using A
Smaller Robot like Kate N to Explore the Area Ahead of Time to Make
The Astronauts Time More efficient on the Moon the Data is transmitted to Mission Managers at the ames Research Center
Where The Robots are Remotely Controlled?
Robotic Scouting Missions to the Moon will Provide Astronauts a lunar Roadmap that will Improve the Quality and Amount of science Data
Collected during Their Stay on the Lunar Surface
Information Gathered From the K10 will Be used to Plan a simulated Astronaut Mission to the Moon this August and that this Week at nasa
Or at least that was this Week at. Nasa Back in 2009
The Video Is interesting
In A couple Respects one is you saw a Planetary rover that
Was being Interactively Controlled by a science Operation Team they, were using a number of Instruments Onboard the Robot
Cameras 3D Scanning Lidar To Better Understand the Environment
But the Primary Purpose of Course was not moving the Robot from point A to point B
I mean that's A sort of a secondary Effect
Roboticists Or Operators Care About you know Making Sure The Robot is Safe
but the Primary Purpose
Was of Course to use a robot to Gather Information that's Necessary for Improving the Planning of what Comes next
Which is the Human Mission and so one of the key Questions was you know how
Should You Carry out That Mission and What sort of Data Should you collect
Scouting is a non-Trivial thing the goal Is not to go out There and Spend Every Possible Hour Doing a super Detailed
Comprehensive
Study of Environment Because you just Never Have the Time in fact you don't have the resources Either and so the Question is how Can
You be smart About going to Different Locations Together the most Important Information that will be
You know Useful for Planning what Comes next
One of things, We did of Course
We use the number of Different Instruments on that Robot as I said it Had laser Scanners as Cameras. We also happen to have A
Panoramic Image here Called Gigapan which, allowed Us to create very High-Resolution Panoramas and We collected A lot of Recon Data?
Eight-And-A-Half Gigabytes of Data Over 52 Hours of Remote Operations and you Can See here it, was Spread out over A
Fairly large Area We Had an Area to the west and blue There
in an Area to the the only Considerably the North Section and very A
Fairly Large Amount of Data Collected in The Center Here A lot of Data
This Data, was important Because it Really Helped complement some of the Data that
We start Out With As I told You Earlier We Began This whole Experiment by Starting off With Satellite Imagery The kind of Imagery?
We would get Today if We were to go back to the Moon or go to mars
This Is an Example of Orbital Data This Comes from a digital globe it
Was a quick Bird Image at the Time 60 Centimeters per Pixel you can, do Better Than that Today Those are Commercially Available
State-Of-The-Art
Orbital Image and if I tell you a little bit more About this image this is such as
Well Here's The Location and Here's the Time of day and you do a little bit of math based on where the sun
Was you Could Probably Look at this Area Highlighted in Red and Figure out?
Oh this Area Where you see some Dark Area This Kind of dark
Shadow is Actually A cast Shadow you Could infer that on the Bottom of the Image
That Section of the Ground is higher than what's inside the Box and so the Shadows Cast That's Falling Into That Area and so what
You're Really Looking at here Is not a area That has Different color Materials but rather is a Basin Where Something Is lower
That's The Kind of Thing that you Can you know glean from this Image and of Course That might Help you in Terms of Planning
for Navigation Because you know
Well I don't want to start on the Bottom Edge and Just drive Straight Cuz I'm Probably Fall off of a Cliff
But it doesn't Tell you more than that it doesn't tell you in Particular you know
Why are There some Differences in There and why
Is that Area Which kind of Looks a little bit Like I
Guess North America in White you know what Is that why Is it so white how
Is that Different from This very very Deep Black Area and if in Fact That black Area is Shadow what's in There
And so one of the Frustrations We have of using even This?
High-Resolution Satellite Imagery Is that it doesn't Give us enough Information to Really Plan Surface Activities well
Contrast This Kind of Data Which Comes from Satellites from from Orbit With This Kind of Data Which Is what we gathered With our Robot
This is Information Gathered With Imagers
We have for example This Top Panorama inside the exact Same Area at the Same Basin and Now you can See
Well yes it is a wall
So we Definitely do not want to drive Off of that
but more Interesting of Course is you Can zoom in and Take a look at you know from an Oblique Angle and Understand
Well in This case you know the wall looks like this
We could Look down Close to the ground and very High-Resolution because here of Course you Can Have?
High-Resolution Imaging on the Surface and Determine Whether or not it's important to go to this Area or or was Just a feature to Avoid
The Other of The interesting Thing of Course Is that if you do have these Sensors that are you know on the Surface you can
Have very Close-Up very Extremely High-Resolution Detailed Measurements That Are impossible to Acquire from Orbit This Is an example of of
A camera image that we also managed to?
Image With our 3D Scanning laser System Here very High Resolution a three Millimeter Depth Resolution
The Kind of Information That Makes it Really I think
Effective if You're Trying to Plan Whether or not you Should to go to an Area
Click Samples Or you know, Merely Take a look at it
you Know from a Distance
What We did After That Robotic Mission of Course is We We took a look at?
How, we could use it Information and Plan A follow-Up human Mission the Mission that We Carried out at that Time?
Was done With some of our Friends at
Nasa Johnson who Back in The 2009 Period Word of Developing A vehicle Called the Space Exploration vehicle This
Was meant to be I know the the fard?
Descendant of the Lunar Roving Vehicle That I start off With Talking About from Apollo 17 in this Case
Here it's A vehicle Where the idea
Is that you Keep the Spacesuits Outside of The vehicle you can See in The Bottom image There These Spacesuits They're mounted
Through A
Mechanism Would Call A suit Port and so the astronauts Would live Inside This Vehicle and a nice Shirtsleeve Clean
Environment When They need to go outside and do Field Work they, Would step, Into the Spacesuits and Detach and and be able to then
Work Outside The vehicle This helps you know Minimize The amount of
Dust and Other
Other Materials That you might bring in and Contaminate the the Clean Living Environment it also allows Us to be very
efficient of being able to
Quickly go in and out of a place Which is you know Comfortable for living to have to go Work on the Outside
Rather Than Go Through A very Long Process and Airlocks and all These kinds of Things
What We did during our Experiment was We Divided Up A?
Set of Astronauts Or at least Simulated Astronauts Here I shouldn't Say Similarly because Actually two
Of our our Test subjects Mike Bernhardt and Andy Thomas are in Fact Astronauts have Been on the Space Station
We combined Them Along With?
Brent Gary and Jake Bleacher who are Field geologist Practicing geologists That Work for nasa They Routinely go out and do Field Work
We had Them Carry out A number of Different Traverses to do Sampling do Field Work and?
What we did of Course is We took?
The Data From The Robotic Recon Mission that We Had Carried out Ahead of Time and We use that Information
And We gave them information only to one of the Crews and the other Crew, We said well?
We're Gonna Try to pretend like you're Just Apollo you only have the benefit of?
Satellite Imaging
Orbital Data and Then We'll see what are the effects of working With and Without surface Information
So here's A short Video just give you an Idea of what the Space Exploration vehicle Looks like large Six Wheeled Vehicle
You Can See here the the astronauts are now Into Their Simulated Spacesuits in There They're Going out to do Field Work
What was Interesting of Course? Is that you know our Focus was on?
Using Robots
We didn't Try to optimize the?
Use of Hand tools and so the tools that they use were very similar to what was used back During Apollo 17
This Idea of using Shovels and
Collection Bags To collect Samples
It's good and it's just some Time We move on Here
Some of the results that We got from from this Study Is that in The Area that the western Area
The Pre Recon Plan That was designed and Again This is based Purely on Satellite Information
Was designed to be very Apollo like that Is if You Think of what was Done During apollo?
They Tried to do A very Rapid Area Coverage because you only Had a single visit you had to try to maximize the Area Covered
Trying To visit you know as Many Different geologic units as possible Because you, don't have a chance to go back There
Well We found of Course is then Maybe this is all Common sense? Is that with the Benefit of?
Surface Recon Information our Plan was Significantly Different Because?
We decided that you know the things Which Look Radically Different from Orbit the things Which Appeared to be you know Incredibly
Important To go Look at this Unit versus Another Turns out well Actually They're very similar on the ground so we don't really need to?
That and as A result our pre Recon Plan and our Post Recon Plan are Significantly Different
so one of the Impacts of
Having Scout Information Is that We were able to you know Really Improve the Prioritization and the Targeting of the Work being Done by Humans
Seems you know, Fairly Self-Evident the more Information you have the Better but you know other Interesting results were that
Well Just Having Information Is not enough because part of the Question is how do you Coordinate
The Activities Done By humans versus Robots I mean if you're going to send a robot out for Scouting in Advance Where
Do You go how, do you carry out that that's Scouting and then how
Do you Take that Information and then pass it over to
The Humans do you Provide Humans for example with the Data that
Was collected by the Robots Because of Course The Robots Are Going to Locations that Maybe you're not going to send Humans Is that Helpful
Or not
It was a really Interesting Experiment Because it Really Started Opening our eyes This whole Notion of A coordination between Humans and Robots
Something that it was not Really, Evident to us as, was A key Driver but Turns out to be extremely Important
So that was a robotic Recon Experiment you know After We finished that one of Course, We were sitting Around Thinking?
Well what's A natural Follow-Up and Someone in my group Said, well you know We Should Just Write a proposal to do the Opposite?
We should go right A Robotic FolloW-Up Experiment and so we did?
and it's of Course a nice Thing to do if you're out There writing grants you know you do some Work and then your second
Grant you Just Take what you just did and you Flip it on its Head and yeah you can go get some more Funding to
do that
so, we did A Sex Pyramid called the Robotic Follow-Up Experiment and This
Was meant to Look at Solving the Problem that We have when, we're out doing Field Work and that We Never ever Have enough Time?
If You're in the Field
Oftentimes You you run out of Resources or you're your field Experiment Just run Short
and you get Back Home and you think
Well if I only Had more Time I could have done more Observations I've Collected more Samples I could have Done Additional Work
Now Now that I'm sitting at Home Sitting in my office Boy if I just gone to this Location
But you can't and so One Question was well what if you?
Combine Human Activity With Robots and You leave the Robots Behind so that after you get home you can
Actually use the Robots to Follow Up and do the things that you Would have liked to have done if You could have Stayed in
The Field Longer
Now of Course why is follow up useful?
I'll Show you this Slide and you Probably Recognize this Since I showed it A few Minutes ago
Except before it Said why is recon useful?
Paulla 17
Same Sort of Problem you know as you recall
Out here at Shorty Crater not enough Time to really collect all the samples You want not enough Time to do all the Detailed Fieldwork
Well what if you?
Had Left the Lunar Rover and Vehicle Behind but it was A
Self-Driving Car Today and it Could Carry out its Own Set of Activities
Well That's fine you know Humans Can Go home leave the Robots Behind and then you Could use Those Robots to do some Detailed
Systematic Work Afterwards
There's an Interesting Idea something Which I fully believe is likely to happen in The Future Because these Days When
We build Things?
They all have Software in Them Whether it's A?
It's a tool or a vehicle Or a robot it's very Likely in the Future that as Humans go to Planetary
Environments Whether that's the Moon or mars There leave things Behind and then Afterwards, we're Going to Operate them Robotically
So to try to understand what are the benefits of this, We went out to a Different analog site This
Was a place in the Canadian Arctic called hotend crater
Hotend Crater Is
About 74
Degrees North it's in The High Canadian Arctic and Here's a trivia bit of Trivia Knowledge for all of you
What's the largest uninhabited Island on Earth and the answer Is devon Island Which is where hotend crater is Located?
Anybody Asks you you know you know where Should you go to get away from everybody this is a great place for that
A devon Island Is an interesting place Because it's Snowbound Most of the Year There's About a six-Week Period in the Summer
Where the snow Is all gone at least on the Surface There is Permafrost A lot of Interesting of?
Subsurface Features That Persist Year-Round Because of the very Cold
Arid Climate and We went There Because A hot and Crater this is a picture I bet Here
In The far North Here Is an Analogue for one of the most interesting Places on the Moon Called Shackleton Crater
They're Both Polar Impact Structures About 20 Kilometres in Diameter
Hotend Crater Has Subsurface, Water ice Shackleton crater We believe in Many Places has Subsurface Water Ice as well?
They're Remote
Isolated difficult Access Interesting Location for Us to go try to do some Experimentation and
So what we did Back in 2009 Is we Carried out A crew Mission by first Having Humans go Explore very much an
Apollo Style They used This This Humvee as a Proxy for a future Vehicle and
They Carried Out A couple Different surveys one in Terms of Geologic mapping which is a a
Classical Way that Fuel geologists Go out to try to Document the history of an Area Trying to examine
The Structural Geometry The Major units of an Area and the Second was A geophysical survey using
Handheld Or These Hand Deployed Ground-Penetrating Radar to examine The 3D Subsurface Structure
Here Are Some example images These are Collected by Mark Helper, one of our Field geologists who?
We sent Up to hotend Crater he Collected These images With a handheld Camera and
Identified Afterwards Different Contacts Between Different Carbonates you can See a Variety of Different sediments and
Different views of the Haughton Impact Structure
Here Are Some examples of the the Ground-Penetrating radar, Data This, was Collected by Assam Peggy who at the Time Worked at Jpl and
Was able to identify as the the Presence of subsurface Ice wedges Locations Where the Ice and the Subsurface is Pushing Up in
Little Peaks Towards the Surface
Based on that, We then Sat Down and Said, well how can We follow up with this with a robot you know where Should?
We send Robots To do Additional Work Complimentary Work to what was done by our Humans and We?
Decided to create A number of Different Plans in These Various Sites
Some of These Are
Traverses Where You're Going from Point to Point to Point others Are Systematic surveys Where you might be Following A?
Raster Or A lawnmower Kind of Pattern to get Detailed Information about that site
And here's an Image or sort of a movie of one of our Rovers this is the T10 rover that
We sent to hotend Crater it Turns out it was the Same River that, was in Black Point Lava Flow in Arizona as?
Well but this Time We Added some Different Instrumentation on the Bottom of the Chassis There's A Ground-Penetrating Radar?
This was meant to complement the Work that was done With a handheld tool There's also an x rf Device on the back
There's a high
Resolution 3D Scanning Ladder on the Top this is A
Gigapan System that We're Using for Panoramic Imaging and We use These to?
Conduct The Follow-Up Work Now One of the interesting Things About This of Course is that
We were Trying to understand the Impact of using Robots so we Didn't want to just have the Robots
Replace Humans
We wanted to Try to Understand how the Robots Could Function in Ways That are Complementary so this Robot for example Is very Autonomous?
It's able to drive and Navigate From Point to Point by Itself all the
Steering of the Wheels you're Seeing here be done Fully Autonomously by the Robot it's Making its Own Decisions of how to Drive
Which Places to Avoid
you Know when to acquire images based on High-Level Guidance Provided by the Planning Team
Some of The results from this is that
We we found That using Robots and in Sort of A follow up Mode was very useful because you Could verify and amend?
the Data That
Was Collected by humans in Some Places you were able to go back and Can ferm some of the hypotheses that
The Human Team Had When they, were Carrying out That Work by themselves?
And in Particular in Terms of the geophysical survey, We were able to Correlate and very
Very High-Resolution Way
Surface and Subsurface Features Because the Robots were able to Acquire A lot more Detailed Information in A very Precise
Manner you Know Robots of Course Are very very, Easy to track, We use Positioning systems, we know Exactly Where They were?
But Harder to do that with with Humans
However, We we learned A number of interesting things one Is that it's Really Non-Trivial
Trying to Plan The Coordination between Human Activity and Robot Activity if You send Humans out and Tell them
Well Robots Going to come Along Afterwards it's Really Hard for those Humans to think
Well
Can the Robot Actually Come here or go to that Location
That's far away that I'd like to send it Because one of the Problems that We have as Humans?
Is it's difficult for us to understand in Detail what are the performance limits what are the Capabilities of Those Robots you know Can
We Make them go to the place Where We really want to?
That's an Important Thing Because of Course if you're Depending upon Somebody Else
Well There's A?
Human Teammate Or a Robot to Follow up after you you need to understand what Their
Capabilities Are and You have to Take them Into Consideration when you're Planning because otherwise you know
When you hand off A Plan and Say hey go do this you know at the end of the day
You're not going to get the results that you wanted so
One of the things that We learn from
This Work of Having Robots Working Before and After Is that is Really
Critically Important To Think About Human Robot Teaming and Thinking about what that means?
Coordination is Clearly Important Understanding the Capabilities of each Whether the human or the Robot is important
Understanding you know How do you transfer Information the way that humans Acquire Data With our senses and
Interpret that Is very very Different than the way that Robots do that
These are The kind of issues that I think are Critically Important if We really want to depend upon Robots
For Future Planetary Exploration When They're Working you know before Or in Support or after Human Teammates?
Any Case that, was Part One Talking About Robots for Human Exploration
I
Think When
We get to the end of This Talk of Any of you have Questions about that you know Please Keep in Mind this this whole?
Idea of Robots Before and Pair Alone After it's a really
Interesting Area and I think That's Fundamentally something that's Going to Help reinvent the way that humans Explore Planets
But Let me Switch Now and Talk about a different Set of tools Another way of Thinking of how Can, we?
Change the way that we're Doing Planetary Exploration and That's in Terms of a whole demand that's
Over the Past decade Been Called neo Geography or you know sort of Reinvention of the way that We think about the use of Maps?
How Many of you in the Past Say six Months, or so has has Relied upon a Digital map Google Maps or bing
Or what's on your Phone I think it's like every Single Person in This Room
Now what's Fascinating is if You go back and you ask you know like your Parents your
Grandparents That Same Question and the answer will Probably be, no, We Didn't do that or at best, We used a paper map
Some of you are Old Enough I'm going to Date Myself now
to know
What a triptych Is this is A thing that Came from the triple a people like a triple a what's the triple A
Well that's a whole Nother Question?
Or a topic of Discussion but The point Is that over the past decade we've Come to Rely Increasingly upon the use of Geographic Information
Systems That Are online that Are Real-Time that have a lot to do with Mapping
And so what's important for for Us here at
Nasa is to Understand how Can
We do the Same for Planetary Exploration I mean Here on Earth We Really have you know gleaned A lot of Benefits from?
The Existence of Gps and Satellite Data and Online and Maps how can, We do the Same for Planetary Exploration
Well Pry Doesn't Surprise you that over the past Few Decades Nasa and other Space Agencies Have Gotten Increasingly good at Acquiring
High-Resolution Data From Orbit you Know as I mentioned for the Moon we've Had A lot of Orbiters Acquire Information, over the past several Years
And the Question is then Based on that Information how Can you use that to Improve the way that you Carry out Scientific
Investigation How to use that to Improve the way that you Plan Future Exploration Missions Whether Those are Orbital Missions or ones on the Surface?
I
Think One of the Reasons why is it important Is that We have you know?
Found Ways of Acquiring More and More Planetary Data and at the Same Time it's not Just an exciting Thing it's, also a real Problem
Over the past Few Decades the Number of Images
The Moon and Mars has Grown at Greater Than Exponential rate
in Terms of The amount of Data we Acquire
We are Really great at getting that Information
What we're Not so great at Is being able to find Ways of Processing and using that Information Because
We have A lot of Data?
And it's not Just the Number of images acquired it has to do with the Size and Scale of These images
We have Really Really Big Images These Days?
Images from example From the Mars
Camera Called High Rise Which is Carried by the mars Reconnaissance Orbiter
20 K by 40 K images are Routine
These Are Huge Pieces of Data and if You think about it trying to Process These in A manual
Manner of the Way that we've Done for Many decades the way that's traditional in The Space science World
Which Is Human intensive Cartography is Really?
Really Limiting There's only so much that humans Can do no Matter how Many Humans you might Have
Because Really There aren't that Many Skilled Cartographers in The World to be able to Process this Data
And so the Question Is how Can you Make use of This in A faster way
Well One way of doing that Is doing Automated Stereo Processing
We can use Computer?
Systems To Take Pairs of images and We Can Reconstruct 3d Terrain by Processing These nasa
Has done This for a Long Time Although Traditionally With With Manual tools
We can now Apply computer Software to do the same sort of Thing?
Here in My group We've developed a map
Processing Pipeline That Takes Data From Lots of Different sources Runs it Through A Variety of Computer Vision algorithms that
We may run on Say The supercomputer Here and I'Ll put that in Different Ways?
We've created Lots of Models of Mars?
We've Worked with A number Different Imagers we've Done The Same Thing of the Moon and some of the Work we've done is Actually With?
Historic Data Where We've Taken Scans of?
The Original Apollo Films From the Apollo Panoramic and Metric Camera and Created These High-Resolution 3D Models
We've used up to create These high Resolution 3D Maps of the Moon for example
We've created a digital Elevation map A 3D Terrain Model of A large Mosaic of 4,000 images They've Been all Registered and adjusts it together
and then of Course you Can, Reconstruct This and Project us back out to the Moon
and the Reason why this is exciting
Is that Once You have this Data in Digital format is you can not only?
Process it but then you Can Visualize it in Ways That are Really Different
One of the great tools that it was created, over the past decade was a tool called
Google Earth I'm sure Most of you Have Probably played with it at. Some Point in your life
I'm not sure How Many Though are aware that Google Earth Actually has a mars Mode and a moon Mode
If You go back to your Desktop Computer or your Laptop and you launch it
After This Talk Click on the Toolbar There's a little Saturn Icon if you click on the Saturn Icon you Can Flip to these other
Modes and You Can find
Data That We created and Then Work With Google to push out to the the Broad public of Both Mars and the Moon
Earth to the Moon in Google Earth go to the top toolbar and select Moon
Now you'll be able to Explore the Moon and Moon related Content in the Left Panel Layers
With historical Charts You Can Explore actual Planning Charts of the Moon From the Apollo Missions
These High-Resolution Maps, were used for Astronaut Training and by Mission Control during the Lunar Missions
Clicking The Human Artifacts Layer Displays Those objects Humans Have Left on the Moon including 3D Models of Spacecraft in
the Left Panel You'll See links to the six Apollo Landing sites on the Moon
Double-Click Apollo 11 and Zoom in and See it in More Detail
Once you arrive you Can
Watch Video Clips of Neil Armstrong's first Steps on the Moon and Many other significant Mission Moments
view 3D Models of Mission Spacecraft Like The Apollo 11 Lunar Module Eagle and
Sea, Panoramic Imagery Taken by the astronauts Themselves of the Moon Surface
And Show you one of the other Interesting Things that's Embedded Within
Moon and Mars and Google Earth and that is guided tours so if you want to go and Explore the Moon
You don't have to just Look at it by Navigating Yourself but you Can
Actually Take A tour With
For example Jack Schmitt Jack Schmitt Apollo 17 Astronaut and geologist who on the Apollo 17 Mission in December of?
1972 Crew of Apollo 17, was an Aggregate of Two Backup Crews
Apollo 14 and 15 I replaced Joe Engle on that
and gene Cernan Would be the Commander and Ron evans
Was going to Continue as the Command Module Pilot gene Cernan got out of the Spacecraft furs and
I
Followed Him very Fairly Quickly Afterwards
We were in A valley Deeper than the Grand Canyon of the colorado the Mountains on either Side Were 2,100 Meters or about
7,000 feet high the sun, was as Brilliant as Any sun that you Can Imagine even More
Impressive was the
Earth Which was Hanging over one of the Mountains and Stayed at Hanging over that Mountain the South Massif and that was Really A?
Magnificent sight for Me and that's what I remember as being Sort of my first real Impressions of the Valley of Taurus-Littrow
So the interesting Thing of Course is the fact that
We have These tools Now that Allow Anyone not Just you Know A trained Planetary scientist not Just someone who Works at
Nasa or Another Space Agency but Anyone
You your Best friend your Grandmother to go Explore These These other Environments These These Different Planets
We've also done some Work not just With Google but With microsoft microsoft Had a another Piece of Software Called World wide Telescope
That We Worked to Help Ad Planetary Data to?
A couple Years Ago been Working With Chris kemp from nasa, We started Thinking how
We could bring Together The Features and the Functionality that We Had in World wide Telescope?
To some of the Planetary Datasets and We Really Wanted to Focus on a unique asset the high-Rise Imagery that mars Had and so
We started working and Collaborating to Figure out how, We could bring to the public and get It in Their Hands?
The High-Rise Imagery is One-Of-A-Kind Camera That's Onboard one of the Satellites That Goes Around Mars and Takes Really
HiGh-Resolution Images of The Planet Surface
We have complete Base Maps of Mars as, well as very High Resolution?
Data That are Actually higher
Resolution Than Most of the Satellite Data that is Publicly Available over Earth and so
We Can Actually See Mars in Better Detail in Some Areas Than, We can See Earth?
This Project Leverages
Several Teams Within Nasa and Then, also of Course Teams at microsoft and
Really it's Bringing Together Some of The Kind of Cutting-Edge Technologies in Both Institutions
To do Something which has not Been Possible before
People Can go for the Walk on Mars Now and Actually See the craters See the Cliffs and and get
Appreciation of the Scale in A way that they could never have Done before
Mars is A big Place by making this raw
Data it's Full High-Resolution Data Available Worldwide Telescope, we're Just Gonna Open all of that up to
Classrooms Across America Across The World and and Connect With the public in A way we've Never been, able to before
So if We Can Really put Projects like Worldwide Telescope in Datasets like the Mars High-Rise Dataset I think
We could inspire the next Generation of Americans like apollo?
Inspired This Generation of Americans to Really be Innovators the Thought leaders and Be leaders in the World
The interest of Time I'm going to skip Ahead and Just Take Us to the last but I want to tell you about Which Is
Exploration Ground Data System
You all know you've Probably Seen on TV that then when nasa operates Missions We typically Have large Control Rooms of very excited People?
Trying to Figure out how, do We Operate a Robot or a spacecraft how
Do We Work with Humans on the Space Station These are large Teams?
One of the key Questions Is how do you coordinate the Activity of Those Teams on the Ground
Supporting Those Things Which are not on the ground Things are in Space Things on our other Planets?
Traditionally We spent A lot of Time Creating One-off Systems that
Is will create A mission and will staff Up a whole Team and Build A very Expensive very large
Software System That's used Just for that single Mission and that's Been an effective Way for us because
We don't have that Many Missions I mean They're There Frankly a number of Missions out There that that Take A?
Decades Or Maybe a whole Career to put Together
But if You Think about it the World doesn't Operate that Way Anymore
Software Is Something that Changes very Rapidly
Whether it's on your Phone on your Desktop your laptop it's something Which is Distributed
We don't have large systems in our Offices in Fact most of Us May not even Use an Office
We may exist by working in Cafes and so the Question is how do we?
Try to Catch Up in Terms of the World of Software for the way that We Operate These Missions so here we've Been?
Been Trying to Figure out How to create a Web-Based systems to Help Plan
Monitor and
Ultimately Explore Whether, we're Using Robots or humans to Carry out Activities
and This of Course as I was Saying Just with the Robots before is the kind of Thing that could be Done before so in
A planning Phase during the Mission and when you're Carrying something out?
Human Or Robot and Afterwards the Idea that you can use the same sort of Software to support your Analysis
We've Been Looking at a number of Different?
Exploration Missions of These Future Mission Concepts Some of Those Involving Humans
In Say the Space Exploration Vehicle in The Center There some using Robots as You've Seen before with With the K10
Planetary Rovers and Other other Cases That we've Tested out Over the past several years Nasa Carries out
oh
I'm pretty much a yearly Basis A number of Different of These
Planetary Analog Field Campaigns, We tried to support These in Fact There's one That Just Wrapped up last Week That?
Was at the crater of the Moons in idaho where, We were Trying to Use our Software to support?
Exploration
The System We've Developed Here and my group is called the Exploration ground Data System and like all good nasa Projects has a nice Acronym
XGds Which is Only Meaningful to the Developers
But xgds is a system it's a web-Based System that allows us to combine
These Maps and Data like I just showed you in the Previous segment here in A way that allows People to Interactively Browse
Without A whole Lot of Training and People use that to Carry out These Exploration Missions
For Those of you who are the Geeks in The Room and Really
Don't know About the Details Underneath it it's a web-Based System it uses Jango and my Sequel
Pulls Together A lot of Different Kind of Data Interfaces Different user Interfaces Including Google Earth
Openstreetmap Web Browsers
and all kinds of Data if You're interested I can Point you to a couple of Papers about this
Or if You're Just in Coming to Work on this Come See me Afterwards and we Can Talk about that
But what, We use this tool for Is?
to have a nice Way to Quickly and Rapidly and Flexibly Support Different
People who Care About science Operations for Fuel Exploration Missions
We serve Maps We serve Data That We Use for Planning Purposes for Laying out Traverses?
We Can track in Real Time People and Human and People in Human People and Robots as There as they are Acquiring Data
We can represent the Data Products that Are Acquired Either real live raw. Data or the derived Products after Processing
We Can Show raster Plots so you Can Actually Look at?
Signals Over Time Information That's very Important if You're Trying to look at
Time Varying signals Or things Which are Really driven by where you are at a Certain moment
We can turn That Information Into Raster Maps and Show a Different?
Coloring of
Various Parameters of The Environment vary Characteristics Change over Spatial
Regions We can
Log Different Information and Real Time so this Is sort of like a Stenographer
Function Or you're Trying to Log Information and We Can cross link Those with Images and Geographic Data
and of Course then you Can Take all Information and Provide tools to help People Browse this Information These are Kinds of
Flexible tools that We all are used to today in our everyday life but Which we've Never Had before?
in Terms of Real-Time Mission Operations is Especially for Exploration Missions
We've done This Work and Never Drew Projects one of Those the Pavilion lake Research Project
Was an Interesting Project that's Led by Darlene Lim here at. Nasa ames Where They've used These One-Man submersibles to map out
Microbial Light Formations in A lake Loop North of Vancouver and Canada is Actually a Video of darlene
Having A fun day at Work where She's out to doing some Exploration in This One-Man Submersible
These Are Kind of images that are Collected by
The Pilots as They're Flying Around Underwater and Pavilion Lake Here and They're Making Real-Time Observations
Commentary That is Streamed up along With This High-Definition Video that we're Recording
and then Putting Into our Exploration ground Data System and in an effort to try to understand the the Formation of
These Really Interesting
Areas Within Fidelium Lake
We've used all the Information put it together Into These Different tools and That's Really Helped us Figure out how, we Can Better support?
Distributed Science Operations as we're Looking for Future Missions
The Kind of Thing that We Feel is important to be more Flexible more Extensible and Ultimately More
Reusable if We're Gonna Carry out Missions
That Are not Just One-Offs and Missions that are not Just you know
One Per Career but Missions That Happen Routinely Throughout the lives of the People Involved
I'm just going to wrap up right now Since I see every run a little bit Short
Of Time Here and Just Point out to you that I've Told you about Three Different Ways
Of Perhaps Reinventing Planetary Exploration the idea of using Robots With Human Explorers Robots That Work Before in Parallel or after
Very Important Interesting Challenges in Terms of how do you core Human and Robot Activity Secondly and the fact that Today
The World Relies Upon
Geography in Ways That we'd never Thought
Were going to be critical to our everyday lives but
Fundamentally Are Also Important I think for Planetary Exploration and Finally This Whole Notion that it's Important to think About how
We reinvent Software to Make it more Flexible more Distributed more WeB-Based?
Lighter weight to Really support the Way that
We Can Carry out Exploration tasks but in Essence all that's Really?
Just meant to be a starting Point for trying to reinvent the Way that
We explore outer Space?
It's something that Would Challenge all of you each and Every one of you to think about how Can
We take the tools Today and Make them better how Can, We create new tools and?
Go Further Than we've Ever gone Before but With that I'll wrap Up and
Turn it Back Over to Jacob
So, We have Time For A few Questions if You have a question please raise your Hand Wait for the Microphone and Ask One Question
Only Thank you
Hi There Thanks for the Amazing Talk
Can you actually Say any Kind of Difference Main Differences for
A
Combine Misha Robotic Human
Between The Moon and Mars
So i think that one of the things the nasa Cares About Today is is Understanding how, do we get to Mars
The Agency Is
Focused Really on This whole Journey to Mars and That Involves The development of tools and Systems that can be tested out on the Moon?
And used Ultimately on Mars and One of the Areas that the Agency Really Is Putting a lot of Effort Into right Now is Understanding
How do Robots
you know Support That
One of the things that, We do know is that
We are Going to need Robots to carry out A lot of Activities when humans aren't present Because
We can't Keep Humans in Space Indefinitely and Then Even When humans are Present and Something I didn't Talk About Today?
Is that We have to be able to use these Robots to Help support Humans a space Station for example right now
Exists Because not Only The People on board the Space Station but for the fact that
We have A very large Mission Control Team that's in Continuous tight Communications With the People on the Space Station?
Well Think of what happens if You Take Those People and You put Them far out at mars?
Perhaps 20 To 40 Minute Round-Trip Communication Delay and They're No longer Tightly Coupled
Well in Those Situations when you need to take Care of The vehicle Basic Housekeeping and Things like that
You're Going to need Robots to Help support you know the humans that are Trying to live and Work There I
Guess you Just
Kind of morgan From Florida Tech I guess you just sort of Talked a little bit About that but how
do you See Robots Working in the Future to like build Habitation for Humans and and how, do you See
Human and Robotic Teams Working Together From Like Long Distance
Environments Yeah that's a great Question I think that if We are going to send
To live in Some Place Whether that's on the Moon or Mars or other Destinations We have to find Ways of Building the Infrastructure and
Maintain it Even When They're not present
And in Particular part of that Is that, We can't Just Launch Every Single Thing that?
We need from Earth it's Incredibly difficult to to launch
Large Structures is Incredibly difficult to supply all the Information all the Infrastructure and all the consumables that you need to Continuously
You know, Keep Those things Running and so one of the things that, We have to do is Figure out how?
Do We use Robots for that are They Just to assemble things do They
Support Other Things Such as collecting Resources and Processing Those Resources
Ames for Example Is working on A developing a future Planetary Mission called resource Prospector Where we're Interested in Going to the Moon and
Looking and Characterizing the Presence of Hydrogen
In This in the Subsurface and if We Can Really Determine that yes This is a resource
We can mine and then Process, well now We don't need to bring Fuel Along from earth Or?
We don't need to to bring Water, we Can actually?
Mine it and use it on the on the Moon and of Course that's not something that
We would Rely on Humans to do We Carry that Up here Lee Robotically
Those Are The Kinds of Things I think are Really important as, we're Looking Towards the Future
Ok, so please Join, Me in Thanking Dr.. Fung for an excellent Seminar
you
