>>I'm worried about where we are right now.
I just feel like our present Internet is destroying societies
and democracies and economies.
I think it's bringing down civilization.
It's bad.  We really screwed this thing up.
[MUSIC]
>>Hi, everyone, welcome to Behind the Tech.
I'm your host, Kevin Scott,
Chief Technology Officer for Microsoft.
In this podcast, we're going to get behind the tech.
We'll talk with some of the people who've made
our modern tech world possible
and understand what motivated them
to create what they did.
So join me to maybe learn a little bit about
the history of computing
and to get a few behind-the-scenes insights
into what's happening today.
Stick around.
[MUSIC]
>>Hello, and welcome to the show.
I'm Christina Warren
senior cloud advocate at Microsoft.
>>And I'm Kevin Scott.
>>And today we have an amazing guest.
>>Indeed, we do.
>>Okay, but before we hear from him
I wanted to ask you a question first, Kevin.
All right, so you have a ton of interests,
you're into cooking and photography
and there's a long list of stuff.
Do you ever feel like there isn't enough time in the day
to do all the stuff that you want to do?
Like, you know, to pursue your passions?
>>Yeah, this is sort of the curse of my life.
I wish there were more hours in the day.
>>Same.  But if you could stop and spend time
during the day to do just the thing that you're most passionate about
at this moment, what would that be?
>>It would definitely be making something
and you know with me,
what it is that I would make changes over time.
So, at one point in time, it might be
a dish to share with someone together at the dinner table.
At another point in time,
it might be a piece of furniture.
Right now, interestingly enough, I'm sort of obsessed
with machining.
So, I would go to my metal shop
and like fabricate something mechanical.
>>I love it, that sounds really cool.
I think for me, I don't know, it would be boring
I'd probably just write.
Continue to just focus on writing.
>>I think that sounds great.
That's what I should be doing given that
I'm writing a book right now.
I should like you know go into a nice quiet place
and try to pound out the last chapter of my book.
>>Okay, but you know, maybe do some machining, too.
Maybe machine something to --
machine your own typewriter or something, I don't know.
>>Yeah, maybe.  Maybe.  (Laughter.)
>>Well, today's guest, Jaron Lanier, is kind of beyond belief
in regard to the things that he's accomplished.
Like, how does he find the time?
Speaking of not having enough hours in the day?
>>Yeah.  Jaron is awe inspiring in the
breadth of his intellectual interests.
When I first met him,
I wasn't really prepared
for the full brunt of his intellect, really.
Like, I knew just from things that I'd read from him
and from admiring him
like all the way back in the '90s when you know sort of
there was a VR boom happening at the time
and, like, he was frequently on TV and, you know,
sort of during the rest of the tech boom
conveying the promise and possibility of virtual reality.
But I think the first time I met him, like,
I learned that he's got this, like,
crazy musical instrument collection.
And, like, we share this interest in classical piano.
And he is, unlike me, like, I am a rank amateur.
And, you know, so I spend most of my time
listening and very little of my time playing.
Like, Jaron's an accomplished performer.
He's actually a composer.
Like, he's done performances with folks like Philip Glass.
He's recorded records.
He's been a session musician.
And, like, this is on top of him being, like,
one of the best computer scientists in the world.
Like, one of the best writers in the world.
One of the best philosophical thinkers about
our digital modernity.
You know, and, and, and --
Like, I'm not even covering everything.
So, it's unbelievable.
Like, I'm totally jealous.  (Laughter.)
>>I am, too.  I am, too.
Okay, so, without further delay, let's meet our next guest.
>>I'm delighted to introduce our guest, Jaron Lanier.
Jaron is a scientist, musician, and author,
best known for his work in virtual reality
and his advocacy of humanism and sustainable economics
in a digital context.
His 1980 startup, VPL Research,
created the first commercial VR products
and introduced avatars, multi-person virtual world experiences
and protypes of major VR applications
such as surgical simulation.
Some say his mind is as boundless as the Internet.
Welcome, Jaron.
>>Hey, I am so happy to be here.
Thank you for having me here.
>>Awesome.  So, let's talk a little bit about your history.
You are, you know, I say this to many of my guests
because I have interesting guests,
but you -- you have one of the most interesting
careers and, like, set of life experiences
of anyone I've ever met.
So, like, let's start with you as a kid.
Tell us a little bit about how you grew up.
>>Oh, my.  Okay, well, my parents were both
refugees from anti-Semitic violence.
My mother was a Holocaust survivor.
My dad's family was mostly wiped out in pogroms
in the Ukraine.
>>And your mother, like, was in a concentration camp.
>>Yeah.  She was taken at 13, and --
well, we can talk about those stories,
but they're truly just horrible beyond understanding.
And my parents met in the bohemian New York of the 1950s.
And I was born in '60 in Harlem Hospital.
And they immediately fled.
And I think the idea -- although I never really got
a clear answer from them
is that they wanted to be as far from
civilization as possible.
But not so far that they weren't next to
a good university.  (Laughter.)
>>Right.
So, they ended up in southern New Mexico.
And I would catch a bus across the border every day
because there were better schools in Mexico.
>>What was the strength of the schools back then?
Because, like, obviously, you've had this career
in science and music and technology.
Did that sort of get sparked there?
Well, my mom came from an educated family in Vienna.
She was a prodigy pianist and that sort of thing.
She had very high standards.
And when I was a kid,
the Mexico school system was about two years ahead
for any given grade in terms of its curriculum
compared to the Texas or New Mexico schools.
So a lot of the families that cared about education
sent their kids across the border to Mexico.
And this was like just one continuous place.
There wasn't like some big prison wall
between the two.
It just was a continuation.
So, she died in a car accident.
I was about nine.
And many years later, it turned out
that there had been a mechanical flaw
in that model of car.
That's likely to have been the cause.
So, there's a layer of tragedy in that.
And that she had deliberately gotten a Volkswagen
out of a sense of wanting to find reconciliation
with humanity, you know?
>>Right.
>>This was, of course, devastating,
but devastating on some levels that were unusual in that era
because she was also the family breadwinner.
My dad was always kind of the multi-career
slightly weird artist type.
He had all these little careers.
He designed windows for Macy's
and was an architect for a while.
And he was a science fact writer for
Hugo Gernsback, the great --
>>Oh, interesting.
>>The great -- yeah, he would write the science fact columns
and Fantastic and Amazing and Astounding in the '50s.
So he knew all the golden-age science fiction writers.
He was in their circle of friends.
But none of that was particularly lucrative for him.  (Laughter.)
And my mom was kind of a systems thinker
and figured out how to play the stock market.
And she would do it with phone calls from the desert
in New Mexico.
Which nobody did at that time.
>>Wow.
>>I mean, these days, that's normal.
In those days, it was highly innovative.
And especially for a woman.
And I suspect a lot of the people at trading desks in New York
didn't know she was a woman.
I think she figured out just ways to do it.
At any rate, we suddenly ended up quite impoverished.
Meanwhile, I was hit by my mother's death
very, very, very hard.
I think she held me very close
because of her background.
>>But you went from there to, like,
right around the time you were 12 years old or so
you were taking college courses?
>>Well, so, what happened afterwards,
my dad was kind of backed into a corner.
And what he did is he got certified to teach school
so he could get a job
because it was the only thing he could see
that he could do out there.
And he bought a piece of cheap desert land
and we move onto the land in tents.
And we gradually started building a house.
He let me design it.
It was this crazy thing with geometry
and it lasted for like 30 years then collapsed.
So, don't let your 11-year-old --
I think by that time, I was about 11.  (Laughter.)
Don't let your 11-year-old design your house.  (Laughter.)
Although, that --
>>But it's somewhat remarkable that an 11-year-old
could design a house that would stand for 30 years.
>>Yeah.  Well, I let me daughter, at 11,
design part of our new house.
But I checked her work.  (Laughter.)
Let's just say, like, I think it's great.
Let your kids design the house.
But in terms of actually living in the house,
check their work.
Anyway, it happened by luck that this place
we were in the desert
was the perfect place for me.
One of our near neighbors was Clyde Tombaugh,
who discovered the planet Pluto.
And was the head of optics research at
White Sands Missile Range.
And he started showing me how to make telescopes and lenses,
which is the background that I used
to be able to make virtual reality headsets later on
when I needed to be able to do that.
So, I learned optics as a kid
directly from somebody who was as much a master as exists.
>>And had you been a precocious kid before that?
>>My mom had made a demand that I would be so.
Like, this wasn't an option.
I was informed, you know, you have to have your concert
at Carnegie Hall
and you have to -- I expect a Nobel Prize
and not one of those sissy ones in economics or something.
Like, a real one.
And, like, that was kind of the expectation.
Like, there wasn't really negotiating room on that.
That was how I was raised.
And I think a lot of Jewish kids of that generation
were raised that way.
It was compensatory for all that had just happened
and there's just this expectation.
This is what you're going to do.
How different I'd be if my mom hadn't pushed me as much,
I really just don't know.
>>So, you were 11 or 12 years old, like,
when you're sort of learning about optics
and, like, getting into science.
Like, this is in the early '70s, right?
>>Yeah, so --
>>And so, like, the personal computer revolution
is like a decade away at this point.
>>Well, it's -- yeah.
This is before you could make your own computer.
This is before the little Altairs and all the --
>>And so were you toying around with, like,
computers at that point?
Like the big mainframes of the day?
>>So, what happened was in order to design this crazy house,
which was a mixture of geodesic domes
and these other geometric shapes,
I had to learn trigonometry.
And so I did.  I just forced myself to learn it.
And it was really tedious calculating all these angles
and lengths for this thing, you know?
And I was really interested in computers.
When I was 14, I went to a summer course in chemistry.
They had like this chemistry summer camp
at the university there at New Mexico State.
And that was great.
And I did all of the things that chemistry professors
worry a 14-year-old would do
in a well-equipped chemistry lab.  (Laughter.)
And I was directed to do those things
in the empty lot next door
rather than in the building.
And I used to visit and there were still
a few pock marks and whatnot.  (Laughter.)
But I think now it finally has a building on it.
But, you know, I learned to make, you know,
flavors and explosives
and I was just fascinated by the geometry of molecules,
which was a little like the house I had worked on and all that.
So, the summer came to an end,
and it just seemed somehow absurd to go to school.
So, by age, I would have been going to high school.
So, I just -- I never went to high school.
I just never -- I just skipped it.
And I sort of enrolled in --
So the question is:  How did I get into the college?
And I'm not sure.
I just sort of signed up for courses.
And either it's possible I might have falsified a document or two
or the might have forgotten to check.
I'm not sure.
And they've had me back to the campus,
and I think by now whatever happened would be forgiven.
But at any rate, I just started attending.  (Laughter.)
>>And which school was this?
>>This is New Mexico State University.
And as it happened --
>>And how old were you?
>>Well, I was 14.
I would have been 14, yeah.
And it happened by wonderful coincidence that
because New Mexico State
was supporting the White Sands Missile Range,
it had one of the earliest good computer science departments.
Like, actually, way ahead of a lot of the fancy places.
Way ahead of someplace like Caltech,
as I learned later.
So there were actually good computers,
good computer scientists
and an excellent math department.
So I was like in heaven.
Like, I mean, I remember I would like just haunt
the basement of the math building
programming at night.
>>What was the first substantial program
that you remember writing?
>>A psychedelic graphics thing.
And in those days -- so, the way you programmed in those days
was on decks of cards.
>>Yeah.
>>And so this is something that's hard to describe these days.
But what would happen is you'd have these
stacks of cards.
And you'd have to take them to this window in a place
where some graduate student would take them
and then run your thing and then give you
another stack of cards that came out.
And one of the things about southern New Mexico is
it can get quite windy.
So, there was -- as you approached this cinder block building,
with your stack of precious --
>>And order matters.
>>Well, listen, there would be like these clouds
of these punchcards just flying everywhere.
So there was a certain degree of cross-pollination
between people's programs, I suppose.
But it was actually -- I mean, I remember seeing
actual code tornadoes made of punchcards
in those days, so --
>>Wow, that's incredible.
>>Yeah, so when we talk about cloud computing
like we had fast cloud computing back then. (Laughter.)
Low latency.
As the '70s progressed, there started to be
a few options for real-time computer graphics.
And that really turned me on
and we started to be able to do some very simplistic --
like even like just a biplot of just moving
like a rectangle of stuff on a screen
was still kind of challenging.
But we could start to do it.
By the way, do you know how the biplot was invented?
>>I think I do, but why don't you tell the audience?
Larry Tesler, one of the original Xerox PARC people,
invented the biplot
which is just moving a rectangle of pixels
around on a screen.
And he'd originally done it to control
those little colored cards that fans hold up at the Stanford stadium.
So, it actually started as a stadium ritual
before it was applied to --
>>That, I actually did not know.
>>Yeah.  (Laughter.)
>>That's fascinating.
>>I that sort of, like, yeah, the antiquities department here.
So, there was a good library at NMSU,
which was -- like kind of became my comfort zone in a way.
And you could just like get lost in these stacks
of all different kinds of journals
and crazy art books and everything.
And there, I discovered Ivan Sutherland's work.
And Ivan Sutherland was the founder of computer graphics
and the founder of graphical interaction on computers
and user interface design with graphics.
And many other things.
He's kind of the father of a lot of the experience of modernity.
And he's still with us.
He's teaching at a little school in Oregon these days.
And I keep up with him once in a while.
And but I was -- he had described computer graphics.
And there was like this picture of a cube
just painted by a computer.
And when I was a kid,
this was before we could get a computer graphics machine
that happened like a year or two after I arrived.
I would like run down and people --
just strangers on the street, say,
"Look at this, you can make images with computers."
It's like so excited.
>>That's awesome.
>>I was like -- because there was no Internet,
you couldn't like just reach
you had to just attack strangers on the street
with weird journal articles.  (Laughter.)
So, Ivan in something like '65 had proposed
a head-tracked computer graphics headset.
And he actually built one in '69.
And thought was -- really, really turned me on
because ever since my mom had died,
I'd had this sort of feeling of incredible isolation from people.
I was super socially awkward
and a very weird kid.
And I just always had this sensation that people were like
these -- like the stars.
Like, you can see the stars,
you know there must be interesting things in the stars,
but they're too far away to reach.
And just people felt that way to me.
>>Yeah.
And I always imagined that if there was some kind of new medium
somewhere -- some way of sharing dreams,
maybe it would be like a starship, you know,
where you could reach those distant stars.
And to me, when I read about Ivan's headset,
I thought, okay, if you could network these things
maybe you could have that shared dream thing.
So, that was how I got into this whole virtual reality
you know, whatever it is.
This crazy adventure.
>>Yeah, I mean, so, like, you are, I think rightfully so,
credited with being the father of virtual reality.
Like, I -- and I remember, like, the first time
I was aware of you was in the '90s.
Like, I saw you on, like, I forget what the name of this --
there was a technology network that Leo Laporte was at.
And I think Leo was interviewing you.
And I was, like, oh, my God, like,
this is the most unusual and incredible thing I've ever seen.
And it was you.
And so, like, when -- when did, like, this whole notion
of virtual reality crystallize for you?
Like, when did you coin the name?
>>Well, the name -- okay, so the deal is,
Ivan Sutherland called the thing you saw through the headset
the virtual world.
And he got that from an art theorist
named Susanne Langer,
who was writing about virtual worlds
in the '40s and '50s.
And I thought, well, if there were going to be a net --
if there as a network or social version of it
to share a virtual world, maybe that would be virtual reality.
And so I started, like, writing little things about it
in zines and all kinds of stuff.
And that probably started, like, in the late '70s
or something.
But I didn't actually have any way to do it.
>>Right.
>>And so what happened next -- and, by the way, in those days
it was almost impossible to explain to somebody what this was.
Like, if you try to explain how quantum computing works today
you get a feeling for what it was like
trying to explain virtual reality in the late '70s
or the early '80s, it's just really hard.
>>My sense was, even like the first time that
I saw you being interviewed,
that it -- like, in the '90s, like, after you know,
the Internet has started to sort of take off
with these, you know, like the graphical Web browser,
that it was still difficult to explain virtual reality.
>>Yeah.
>>Like, it was an -- you know, like, it's --
it's not the easiest thing in the world even now,
but, like you know, there are far more, you know,
examples and, you know, like, there's a richer continuum
I think of different types of experiences
that people are trying to build.
>>Well, people can actually try it now.
>>Right.
>>It's not rarified, so --
>>Like, you don't have to go into a room full of
a million dollars' worth of equipment.
>>Right, exactly.
It's seems kind of almost mundane these days,
which is probably good.
It's probably a stage it has to go through.
>>But it -- but it's sort of, you know, I --
I always find things like these sort of disruptive innovations
you know, so sometimes, like, when you look at startups,
you know, you sort of see a company that's doing something
and, like, they might be a few years early.
And just being a few years early is enough to, like, kill them.
>>Oh, yeah.
>>And, like, you were 30 years early maybe?
And, like you know, I -- it's almost incomprehensible to me,
like, how you maintained the intellectual stamina
for, like, all this time to -- and, like, you're still --
you're still pushing on it today.
Like, how did -- like, what made you sort of stick with it that long?
It was so hard.
>>Right.  It was really hard.  It was a very, very hard
area to innovate in back then.
I had good fortune in the early '80s.
I made some money in video games.
So I had one of the top ten games in '83?  Yeah.
And so I suddenly had some capital.
And so a bunch of buddies and I just invested this money
from video games in the first VR company,
which we incorporated in '84.
Although, we'd been sort of doing garage experiments
for some years before that.
And that was called VPL Research.
If it had more of a ring to it,
it probably would be better known these days, but --
yeah, and VPL was quite an entity.
It actually -- even though it was always a small company,
it had a kind of a feeling
of being a larger, more influential company.
Everybody knew about it, and it was like one of the things
in the Valley for a while.
And, yeah, of course, it was way too early,
and I --
The story of VPL is still emotional for me,
because I still feel some guilt about --
I wonder in my head if I could have done things differently
to keep it going.
A lot of people thought it was going to become
one of the big Silicon Valley companies.
You know, and a lot of -- there were just a lot of challenges
and difficulties that mounted up simultaneously.
And it just became too hard a thing to keep going.
And but there were just wonderful people.
And we did a lot of stuff.
I mean, I'm still reading today about these new innovations
that are actually things we'd already done back then.
>>Like, give us an example.
>>Well, I just got a pitch in the mail from somebody who is --
has a new way of visualizing tumors in VR
that's supposed to help radiologists.
And so I -- and they've just gotten all this money
and all this research.
And I read the papers and looked through it,
and it's almost the same thing as
a project that we had done in the '80s
with visualization for radiologists.
Of course, it's cheaper and higher resolution
and more responsive and all that.
Like, everything's better now than it could have been back then.
Vastly less expensive than it was back then.
But it's sort of like a history that's forgotten.
I think the right attitude to have about that
is that the purpose of history is to enhance the present.
So if people want to experience the myth that what they're doing
is entirely new and if that helps them,
then the better use of history is to be forgotten.
If there are some lessons we learned or some inventions
or something that can help them,
the better use of history is to be remembered.
But I think, obviously, forgotten has to be part of life.
We can't live -- we can't live in a way that sort of
subdues the present moment for us.
It has to be fresh.  It has to be its own thing.
So I'm fine with the new waves of VR
not knowing about the old waves of VR, you know?
To a degree, I'd like it if they remember it once in a while,
but it's not important to me that they remember everything.
>>Yeah.  Well, it really is, I think, an incredible thing that you --
you know, that you all did so much stuff in the '80s.
And, like, you had thought about --
like, you'd had this clear vision, even before VPL of,
you know, what could be.
And, like you know, you're still,
even to this day,
like, we just launched the second version of the HoloLens
at the Mobile World Congress in Barcelona.
And, you know, in a sense, like that's just another
waypoint along, you know, this sort of vision
that you had decades ago.
>>Yeah, I still have a triptych of drawings at home,
meaning three drawings that are meant to be seen together.
Showing two people, first in natural reality,
then in mixed in reality, then in virtual reality.
But the term "mixed reality" is mine as well
from back then.
It predates augmented reality
by at least a decade.
And we did have kind of a roadmap for these things
and a vision of what they could mean to people.
And I mean, the first arriver -- the first arrivers
often can see more clearly
because there's just less clutter and less crap, you know?
So, I mean, I think Ted Nelson did the first design
for a digital network.
And in many ways, I think it's more insightful and decent
and reasonable than many of the ones that came later.
And many other examples.
I think the generation of Turing and Wiener
and these first people
had a sense of what computation was all about
that in a way was kind of deeper
than a lot of what came later.
I like reading computer science conference proceedings
from before I was born.
Like, the ones in the '50s are fascinating
because it's people coming upon this thing really afresh.
And having to think about it without preconceptions.
And before there was all this money to be made and all,
and it's amazing.
I mean, it's actually -- a lot of the old stuff reads
as quite radical today.
>>Yeah, well and even the stuff that doesn't read as radical
it's, you know like, if you try to put yourself in context,
I remember I was -- when I was working on my PhD,
there was a question about Dijkstra's Algorithm
on my, like, doctoral qualifying examples.
And, like, I remember, like in that moment
being irritated at, like why --
why is this relatively simple thing, like, enshrined
as one of the most important accomplishments in computer science?
Because it looks so simple.
And then, you know, but you think about it and, like,
not knowing that it existed
like, it wasn't obvious at all.
And I think sometimes even, like, I think you're totally right,
like, in your point about, you know, like, some of these older things
reading as very radical today.
But, like, I think even the simple things
that look simple from our perspective today
were quite radical, like back then.
>>Yeah, I think we're --
I think computer science these days suffers from
a couple of problems.
One is just there's such a profit motive.
There's just so much money to be made
and people are so attracted to those --
to the things that are probably going to make money.
That's one issue.
But then another issue is that there are just so many people in it
and so many strong personalities
and so much baggage on everything
that it's kind of like trying to navigate in some giant city
with no infrastructure or something.
Like, you're -- it's just at every turn,
you're just facing other people
who are trying to go somewhere else.
You don't have that.
It felt more like this open wilderness.
I mean, even, like when I used to write video games,
when we wrote our operating system
for virtual reality in our applications,
we did everything.  We did everything.
And we did it in a way that would --
was completely unique to its time.
We had a different architecture than anyone would use today
in order to get the efficiency we needed.
We had a high-level incremental compiler,
that's unlike anything else I'm aware of that was amazing.
We'd actually see something that looked like code
or graphical programming,
but at the bottom, it was instantly turning into op codes
that would never crash.
And it was like this whole really interesting way of doing code
that I've never seen anywhere else.
And these days, you can't do that.
These days, it's all about all the pre-existing tools
in libraries that you learn.
So, you're kind of more in this downstream position.
So, it's both better and worse in a way.
>>Yeah, and it's sort of funny.
Like, this has come up in a bunch of the conversations
that I've had on this podcast about
like, one of the interesting things with contemporary
computer science and software engineering
is that the level of abstraction at which people are operating
is so much higher now
because like you aren't, you know, like
writing the operating system kernel
or the code generator for the compiler or whatnot.
Like, there's some people who do that, but you know,
you can do an awful lot by just sort of assembling
like a bunch of the building blocks that are
already there for you.
And it's, you know, in a sense,
almost miraculous.
Like, I even look over the course of my career
you know, just sort of thinking about, you know,
the startup that I did in 2007.
I went from Google where, you know, we had had to build
all of this distributed computing infrastructure from scratch
because it didn't exist.
And, like, four years later, when I did the startup,
like, there were all these open-source things
that were replicating some of those things.
And so, like, I had a little bit of a start but, like,
I was still in '07 we were building our own data centers
and, like, racking computers and whatnot.
And if, like, you were doing that today,
you build it on a public cloud
and, you know, not have to, you know, sort of, you know,
spend a bunch of your startup's money
you know, sort of standing up a bunch of physical infrastructure.
You know, so on the one hand, it's great.
Like, you're just sort of putting this incredible amount of power
into the hands of, like, individual creators.
Like, fewer people can make bigger things
inside of, you know, some sort of constraints.
But, like, the downside is, like you know, if you're not curious
about what's on the other side of these abstraction boundaries
like, you can limit yourself in, like really interesting ways.
And like, that's sort of the thing.
I mean, like, you weren't --
When you were developing this stuff,
you didn't feel constrained at all, right?
You were just --
>>Not at all.
I remember feeling like even the built-in architecture in some
chip, in a CPU, bothered me
because it was enforcing structure on me.
And, like, you know, like, I would hate that.
I would resist it.
I would say, "These people at Intel, they're jerks."
Like, why would they make this -- (Laughter.)
You know, it has to be something, you know?
Yeah, I think if you live inside an abstraction
for long enough,
it becomes like a religion for you.
It becomes the framework within which your mind functions, right?
There was a thing that's just been --
there's a controversy in the last week in AI
where somebody's saying that we must not understand
when I say "somebody," there's a certain group of people
who are arguing with another group of people.
And the debate is since there are mechanisms
in our machine learning implementations
like back propagation that don't seem to have
direct analogs in biological neurology,
maybe we don't understand biological neurology or something.
And I'm, like, what do you mean?
This is just this code we made.
What does it have to do with anything?
It doesn't even work that well, if we're honest.
Like, why is this even -- how can this even be a controversy?
But there's this thing where you start to believe
in your abstraction so much
that they loom larger than they should.
>>Yeah, and I -- like, I couldn't not more strongly
agree with that point.
Like, and I think, you know, it's sort of especially interesting right now
in all of these discussions about AI.
You know like, these machines, these machine learning systems
are far less mysterious than some people think they are.
I mean, it's like all the way at the bottom,
it's just a bunch of things doing linear algebra.
>>And there's nothing wrong with that.
>>There's nothing wrong with that.  (Laughter.)
>>It's all great.
>>But like, it doesn't -- and, you know, it's trying to emulate
some aspects of intelligent behavior.
But, like, that doesn't necessarily mean that --
I mean, like, the connections I think are, you know,
sort of arbitrary between, you know, biological intelligence
and, like, a supervised machine learning algorithm.
>>I've always been bothered by not --
I don't have any --
I'm not opposed to the research program of the algorithms
that are called AI,
but I have a problem with the sort of concept
and the culture of the term "AI."
And a little later, when I was a teenager,
my main mentor early on was Marvin Minsky
who kind of invented a lot of the mythology of AI.
About this idea that we're building these little creatures
in the computer.
And I just find it -- I'm actually thinking
of writing a book that might be called something like
"AI is not a thing."
Because what happens is we choose some bundle
of algorithmic techniques and say, "This is AI,"
but it's kind of arbitrary.
Sometimes things go in, sometimes they're taken out.
And then we allow ourselves to believe in it
like a monster or a god or something.
AI will take your job.  AI will do this.
AI has done that.
Whereas, in fact, there's not any particular thing that's AI.
It's just another example of people coming up with ways
of using computation to do things.
And this whole storytelling I think makes us a little nuts, you know?
>>Yeah, you should definitely write that book.  (Laughter.)
I've, like, I'm writing a book right now.
And, like, I say something to that effect in one chapter of my book,
but like, you are like, the far more eloquent writer.
And I think, like, an entire book on that topic would be amazing.
Just totally amazing.
>>You know, maybe it'll happen.
>>Yeah, in your copious free time.
>>Yeah, that's an issue.
>>Yeah, it is an issue.  (Laughter.)
So like, after VPL, like, you went on to do
a whole bunch of other things.
And I don't want to, like skim past
like, anything super important.
But, like, where I want to get to before we run out of time here
is you have become, like, quite the you know, sort of
technical philosopher for modern digital life.
You've written these amazing books.
Like, you are promoting a bunch of super interesting ideas.
Like you like, recently in some of the work that you're doing,
doing at Microsoft,
you've sort of coined the phrase "Data Dignity."
Which is, like, a really interesting concept.
So, like, I'd love for you to talk about, like, why it is
that you took on this responsibility
for like, being the, you know, sort of the human intellectual
thinking about our digital world.
>>I'm not sure if I coined "Data Dignity," by the way.
I think either Glen Weyl or maybe even Satya Nadella did.
I'm not sure.
"Digital Dignity" was a term was going to be the title of
"Who Owns the Future"
but the editor didn't like it.
>>Ah, I gotcha.
>>So, it turned into Who Owns the Future.
At any rate, so there's a --
this is a whole long tale as well.
In the '80s and '90s, there were a couple of really
vociferous, intense movements within hacker culture
within technical culture
about what networking should be like
whenever it really comes about.
One of them was this idea that everything
should be open and free.
And that was started from a number of sources.
One of them was a guy who's a friend of mine,
Richard Stallman back in Boston.
And there were a few other source points for that as well.
And then another was this kind of intense libertarian philosophy
that government shouldn't be involved
we should leave everything to entrepreneurs.
And in the late '80s and early '90s,
I ended up spending time with somebody named Al Gore,
who's at that time a senator from Tennessee.
He eventually became Vice President.
And he was really interested in these things.
And he came up with this idea of throwing some government money
at people with nascent packet switch networks
to bribe them to become interoperable,
and that was the Internet.
So, that was funded by the Gore bill.
And so we used to debate like what this thing should be.
And because of the extremely intense --
those two dogmas, there was this feeling, well,
it'll be minimalist.
It won't have accounts, for instance.
It won't represent people.
That'll be left to private industry.
There won't be any persistent data on it.
That'll be left to private industry.
There won't be any transactions on it.
That'll be left to private industry.
And on and on and on.
There won't be any memory on it.
There won't be any contextualization on it.
That'll be left to private industry.
And I remember saying to him, you know,
we're creating this gift of many hundreds of billions of dollars
to persons unknown because there will be
natural network monopolies that form
to fill these obviously needed functions.
But, whatever, that was -- there's just this feeling
that that was the better way to do things.
And since the experiment wasn't run the other way,
we don't know.
But then everything should be free,
I think set us down a terrible path.
Because it feels all socialist at the first --
you know, it feels like this friendly socialist lefty thing.
But since it's being mixed with this libertarian philosophy,
you end up with only one possible business plan,
which is advertising.
So, everything feels free, but actually the money is made
by third parties who want to influence the users using user data.
And it ends up -- it starts cute,
and ends up evolving into
this sort of monstrous universal behavior modification scheme.
Anyway, this is the stuff I talk about all the time,
where I think we've gone wrong.
And we've created a network that's more about deception
than it is about reality.
>>So, what do you think we can do about that?  (Laughter.)
>>Well, we're kind of in a pickle now,
to use an expression from when I was a kid.
It's a little -- it's tricky.
I mean, there are a lot of schools of thought about it.
I think we can't combine socialism and libertarianism
in the awkward way we did
and expect to get anything useful.
And I think we should just choose one of them.
And I personally think we're better off choosing markets.
I'm worried about where we are right now.
I just feel like our present Internet is destroying societies
and democracies and economies.
You know, I think it's bringing down civilization.
It's bad.  We really screwed this thing up.
>>So, you've been working on a bunch of concrete things
to try to figure out like how to introduce
these new incentive structures.
Can you elaborate on that a little bit more?
>>Yeah, well, the problem is how to get from here to there.
I kind of have in my head an image of what a society
would be like with paid data.
There's a few things to say about it.
One is there are a lot of people out there
who pretend to be making a living online, but aren't.
Because they're fakers.
It's all a big illusion.
It's what we used to call Horatio Alger illusion.
Where you create this illusion that there's this
way of making a living,
when, in fact, there isn't.
It's only for a token small number of people.
However, there's another population of people out there
who are making a living,
but not within the rules dictated by a central hub,
but as actors.
Like, for instance, there are tens of millions, maybe --
well, we don't know the total number,
but at least 50 million people in the world
who are actually making a living delivering online video lessons
and counseling and guidance and --
you know, this is anything from legal consulting
to religious training to yoga teachers
to musical instrument teachers.
All those people have sort of cobbled together something
that has to fight against the grain of everything.
Because there's no --
>>There's no infrastructure to support them.
There's no infrastructure.  So, each one of them
has had to invent their own infrastructure
by cobbling together little pieces from the different
digital companies.
And that population interests me.
In a way, I see them as the future.
Those are the people who don't have to worry about their jobs
being taken by robots.
Unless -- I mean, they could be.
All we'd have to do is create some machine learning thing
that steals all their data
and makes a fake clarinet teacher without paying them
for their data, just stealing their value.
And that's what we've done in so many other areas.
So, the future I would see is to, first of all,
try to support --
to identify those groups and support them.
And also identify those communities that are trying to create
new structures to help people cooperate in decentralized ways.
And here, the blockchain community,
not the get-rich-quick blockchain,
but the other blockchain,
the blockchain of people who are interested
in new ways of cooperation
that can be mediated by networks.
Those people could be really important and helpful.
I think we need to invent new structures.
The reason that we treat data as being worthless
even though the companies that collect the data
become the most valuable ones in the world
is that there's no collective bargaining
for people whose data is taken.
So, out in any other economic example,
in order to have a productive economy,
you have to have some -- you have to invent some kind of structure
so that people can cooperate and not have it
not be the Hobbesian race to the bottom
where each person is against each other person.
And if you believe more in capital than labor,
you call that a corporation
or a legal partnership or something,
so these people are incentivized to cooperate
instead of try to kill each other.
If you believe in labor over capital,
you call it a union
and you call it collective bargaining.
But on the Internet, the difference is academic.
And I was playing around with terms like "unorp"
and "corpian" and they're terrible.
So, we just came up with a very --
my research partner, Glen Weyl, and I came up with a term "MID."
Actually, my wife came up with that.
Mediator of individual data.
So, you'd have something that's a way for people
to band into a group
so as to not have the value of their data descend to zero
through interpersonal competition.
But, instead, have a degree of local cooperation.
And so we need to create those things.
And MIDs can serve another function here.
I'm talking fast because I know we're almost out of time.  (Laughter.)
>>It's okay.
>>But one of the things that's really terrible
about what's happened in the world
is we've been petitioning tech companies
to become the arbiters of culture and politics.
But the thing is, do we really want tech companies
to become the new de-factor government?
Is that what we want?
I don't think so.
So the MIDs could also become brands in themselves
where people who have bonded together to create a MID
not only are collectively bargaining
for the value of their data,
but the MID itself has become
a channel, if you like, like a guilder union
or like a corporation or a brand.
That represents a certain thing.
It might say whatever data comes through here
is scientifically rigorous and has been checked.
Or whatever data comes through here
is fashionista approved
and is very beautiful.
Or whatever data comes through here
is guaranteed to be really amusing
and suitable for your whole family or whatever.
You know, like what it creates is these in-between-size
structures that can take on this function
of quality maintenance, you know?
Because you don't want a centralized source
being the maintainer of quality.
That's a recipe for some kind of dysfunction,
too much centralized power.
So the MIDs both solve the economic problem
and the quality problem.
And we need to start creating them.
So, there are fledgling attempts to create them.
Right now, they have no infrastructure tools to help them along.
I'd like to change that.
And that's just one little corner of the problem.
I'm also just trying to --
Honestly, I'm just trying to get the tech companies
to see the light.
And, here, you know, some of them are
better than others.  (Laughter.)
>>Yeah.
So, let's switch, you know, switch a little bit into, you know,
like, all of these other interests that you have.
Like, I think one of the fascinating things about you
that folks underappreciate is
that you are a composer and a musician.
And you have one of the largest -- maybe the largest
collection of musical instruments in the world
in your home.
[MUSIC]
So your mother was a piano virtuoso.
So, but, like, how have -- has this, like, remained a thread
in your life all these years?
>>Yeah.  After she died, I feel like music is my main connection to her.
You know?  And I still play the piano.
But not so much straight classical playing anymore.
I have my own style, and it's pretty unusual.
[MUSIC]
I started just learning new instruments,
and I have this voracious --
perhaps not always healthy need
to always be learning a new instrument.
And so whether it's the largest instrument collection,
I'm a little doubtful of
because there are some pretty big instrument museums.
But in terms of playable collection,
I'm pretty sure it is.
And I don't know how many there are
but there are a lot of instruments.
And I do run -- I can play them.
And I have a --
>>And we're talking like hundreds, if not thousands of --
>>Certainly in the thousands, yeah.
>>Yeah.  (Laughter.)  Which is, you know,
sort of a mind-boggling, interesting thought
in and of itself, that there are, like, 1,000 --
you know, thousands of distinct instruments
that one could collect.
>>Well, they're the best user interfaces
that have ever been created.
They're the ones that support peak human performance
like no other invention ever.
And they're profoundly beautiful
and each one has a story.
And each one is kind of a form of time travel
because you learn to move and breathe
like the people who originally played it
wherever it's from.
So, it's a kind of a cultural record
that's unlike any other one.
It's a haptic record, if you like.
Yeah, so --
>>I mean, I've always been fascinated with piano
and I think, you know, the reason is
it's always struck me that a piano
is not too dissimilar from a computer.
It's like this complicated machine
that requires, you know, some non-trivial degree of mastery
to get anything out of it.
And like sometimes that struggle to achieve mastery
is like this sort of long, isolating, you know, activity.
You know, like, I've read biographies of famous pianists
and you know, like, some describe it as, like you know,
you just sort of sit alone in this room and, you know, like,
struggle against this machine
to, like get it to bend to your will.
And I'm, like, holy -- holy crap, like,
that's sort of what, you know, you do sometimes as a programmer.
>>Well, I've experienced feeling alone with a computer
but I've never experienced feeling along with a piano.
>>Which is interesting.  Even when your practicing?
>>Yeah, pianos are a big mysterious
because they're sort of the button box
that transcends button-boxness.  You know?
They have some kind of a life in them
that they shouldn't have.
I think that's one of the reasons that they're so provocative
to computer scientists.
Of course, the piano led to the computer
pretty directly because they --
around Mozart's time, somebody was --
made non-deterministic player piano
which is what inspired the Jacquard loom,
which inspired the Babbitt calculator
which inspired Turing, et cetera.
So you can blame the piano for all this if you want to.
>>So, what has been your favorite performing experience
over all of the years?
>>Oh, performing?
>>Yeah.
>>I've had the good fortune to perform
with a lot of interesting people.
Although I was living as a professional musician
for a while in the '90s,
but I've never been like a major one.
I've never been a major star or anything.
But as a side man, I've had incredible performance experiences.
I think my favorite one was when I toured with Yoko Ono
and her Plastic Band.
And Yoko and I would do these duets
that were --
a number of people said they were the strangest thing
ever on stage.
And I think we got there.
I think we did -- John Perry Barlow called it the
heavyweight championship of weirdness.
And I think we got -- yeah, that was good.
And I played with Ornette Coleman,
this wonderful -- the father of free jazz.
And I've done a lot with Philip Glass,
including just recently.
In theory, we're doing a new record together, we did one --
>>Oh, that's so awesome.
>>-- in the '90s.
I just did a show with Philip where I brought the
pedal steel guitar into the minimalist music aesthetic for the first time
and it worked great.  It was so fun.
And I just did a thing with T-Bone Burnett a while ago
for a Sarah Bareilles record that was really, really cool.
And I've played with all kinds of people.
I've played with George Clinton and P Funk.
I've done more than you'd imagine,
but always kind of in the background as a performer.
I've done some solo stuff, too,
but as a performer,
I just -- you can only do so much.
I've had this career in computer science
and another one as a writer.
And at a certain point, you know, you can't really
do everything always for every --
and have a family, you know?
So, unfortunately, I haven't been doing as much lately,
but still a little bit now and then.
>>Yeah, so, like, even though, like,
so the writer, philosopher, computer scientist, composer, performer
like, you still have other interests, which, you know,
I guess your mother really did --
>>She did a number on me.  (Laughter.)
>>Well, she certainly convinced you that, like,
you had to, you know, sort of exhibit excellence
in a bunch of different domains.
But, like, maybe one of the most interesting things
that you and I have chatted about
is the Neural Information Processing Symposium
is like the big deep learning conference for the field.
And the best paper at this year's symposium
was this thing called Neural Ordinary Differential Equations.
>>Right.
>>Which is this -- you know, the short idea
is that neural networks are usually these
sort of layers of connected artificial neurons
and you know you're trying to, you know,
sort of figure out, like the activation weights
you know, between the connections of all of these layers.
And, like, which non-linear functions you're using
to, you know, sort of accumulate the weights
into a network node and, you know like,
whether or not you should use things like dropout
to, like, impose some sort of memory loss
as you're doing training and blah, blah, blah, right?
And, like, there's back propagation and all this other stuff.
And so, like, this paper is sort of interesting in that
it sort of models a bunch of the interior state
of these deep neural networks as a system of
ordinary differential equations.
And it was sort of a sensational result
because you know, like it has some like really big
performance implications for training deep neural networks.
And, like, the amount of computation that's required
to train them is one of the, you know,
sort of big things on people's minds.
>>Sure.
You know, so, like, that's great.
But, like, when you saw it, the connection you made
was to quantum field theory.
>>Yeah, so here -- I don't -- it might be a bit premature
to speak about this, but one of my --
one of the dimensions in my life is I have
a lot of physicist friends
And I've done sort of weird projects with theoretical physicists
from time to time.
And this goes way -- it's all --
this all started because my first serious girlfriend
was the daughter of the head of the physics department at Caltech
so when I was a teenager, I was sort of hanging out there
and I got an informal chance
to learn directly from Feynman and Gell-Mann
and all these amazing people, and --
>>Yeah, which is insane.
>>Yeah.  Ever since then, I've had this kind of --
so, one of my best friends is this guy named Lee Smolin.
who co-founded the quantum gravity approach
to trying to find a unified theory,
which is probably -- since string theory kind of burned itself out
it might be one of the most prominent ones now
as an alternative.
Anyway, I had -- so, there's this thing in physics
where we've always had pretty --
we want really simple equations to describe
let's say fields.
And that we want them to have emergent behavior
that's complicated enough to be reality.
But the thing is, we've kind of burned --
It's kind of like in a house where you put all your mess in one room
to pretend the house is clean.
So like in string theory,
they tried to simplify some stuff
and they ended up with this insane --
>>Room full of mess.
>>Really messy room of, like, you know, an unbounded
number of possible --
like this was a really messy room,
the sort of range of possible string theories.
And so when you try to --
it's kind of like playing whack-a-mole.
When you try to -- you often end up creating this big mess
in another corner when you're cleaning up one part of it.
>>Yeah, I think it's fantastic how broadly your mind wanders.
And with that, I think we should wrap up, thank you somuch, Jaron, for being on the podcast.
>>Delighted to be here, thank you for having me.
>>Awesome.  (Music.)
>>Well, thanks for joining us for Behind the Tech.
You just heard Microsoft CTO Kevin Scott
speaking with Jaron Lanier.
So, what really struck me about that conversation
was how far ahead some of the visions for virtual reality
were, you know, decades ago,
and how similar those visions are
to what we're actually seeing in the market now with
both VR and with augmented reality.
>>Yeah, it's this really fascinating thing
with true visionaries like Jaron.
He saw this thing like way, way, way, way before anyone else did.
And it's not just the vision of sort of seeing this thing
that one day might be,
But just the -- his consistency and tenacity over time
to sort of stick with the vision.
It's not like he's wavered.
Like, he's been doing this for almost four decades now.
And like he's had this vision
and he's kept pushing, you know, episodically
for like this very, very long period of time.
And, like, I find that almost as amazing
as the vision itself.
Just the willingness to believe in something for that long
and to just push against it as hard as you can.
>>Yeah, it's so interesting to me.
You know, when he was talking about how
he's getting pitches for certain uses of VR.
And he's, like, "Oh, yeah, I had a paper, you know,
kind of predicting that 30 years ago."
And he was right.
And as you said, he continued to push
and be committed to that.
Which is just kind of incredible.
>>Yeah.  You know, and sometimes
the frustrating thing with technology
is timing matters way more than you would
like to think.
>>Yeah.
>>Like, unfortunately, like, vision and persistence aren't enough.
Like sometimes like the technology that you need
and like the set of conditions in the ecosystem you need to exist
in order for something to become broadly adopted
by a whole bunch of people
is just -- just isn't there.
And it's really interesting.
Like it's sort of -- you can almost see it right now.
That like, you know, mixed reality, augmented reality,
you know, like the whole grand, you know --
the whole grand virtual reality vision, like,
might actually be within reach now.
But it's like taken all of that time.
>>Yeah, no, it's so true.
I mean, as you said, you know,
Microsoft just showed off the new version of the HoloLens
and we -- it feels so much closer.
And, yet, it's just still somewhat, you know --
you can kind of see what Jaron's vision
has been all this time.
And we seem to just be within grasp.
It's really exciting.
>>The other thing that's sort of
fascinating about his vision for VR
is on the one hand, I think it is like a very deeply technical thing,
but I think, you know, as you heard in the conversation
the thing perhaps even more than the technology
that motivated it is like, this very you know, humane
desire that he had, like, to connect with other people.
>>Yes.
>>And, like, that's something that, you know, you don't always get
from folks who are trying to do something with, like,
deeply, deeply, deeply technical technology.  (Laughter.)
>>No, you're exactly right.
It kind of reminds me a little bit of, you know,
Tim Berners-Lee and the World Wide Web.
Which was a similar thing, and that's, you know,
celebrating an anniversary right now, too.
And it's, like, you're right,
a lot of times it's rare
to see these intersections between these
highly technical things
and these also highly social and personal
and deeply connective things.
>>But sometimes, like, those are the things that,
like, have the biggest impact on the world
is, like, you've got this desire to, you know,
sort of facilitate more of our own humanity
to, like, empower and ennoble, like individuals and groups.
And like, those technologies can be really, really
profoundly transformative.
>>I mean, I would actually argue, I think
that what you just described
is kind of the basis for the most transformative technologies.
>>Yes.
>>Whether we're talking about radio or television
or transistors or anything else.
>>Or the PC.
>>PC, absolutely, is finding a way to facilitate humanity.
>>Yeah, for sure.
>>All right, so I think we're out of time for this episode,
but we are going to meet another icon on our next show.
>>That's right.  I'll sit down with Reid Hoffman,
investor, author, and entrepreneur.
Someone I consider a true friend.
[MUSIC]
Be sure to join us next time on Behind the Tech.
And, please, help spread the word.
Tell your friends, your colleagues,
and all of the geeks and non-geeks you know.
See you next time.
(Music.)
