Hello, and welcome back to our second
video lecture for
week one of CS631 "Advanced Programming
in the UNIX Environment". In our first
segment, we discussed all the what,
why, and how of the class; we introduced
the course syllabus
and I warned you about all the code you'd
have to write for this class.
In this segment, we'll briefly review the
history of the UNIX operating system.
There will be intrigue, lawsuits,
differences of opinions, and one or two
holy wars,
although we leave the most sensitive
political battle - vi vs. emacs -
unaddressed so as not to offend anybody. [cough] vi [cough]
The UNIX operating systems family has
come a long way from a test platform for
Ken Thompson's "Space Travel" game running
on a PDP-7 to the most widely used
server operating system.
Its history begins at AT&T Bell Labs in
New Jersey where Dennis Ritchie and Ken
Thompson
- shown here working on the PDP 11 -worked together with Dough McIllroy
and Joe Ossana a replacement for
the MULTICS operating system.
The C programming language was developed
in parallel by Dennis Ritchie
for and on UNIX deriving from the B
programming language developed by Ken
Thompson for the new OS they wrote.
Dennis Ritchie describes the history of
the language and the operating system
at the link shown here in the slides.
Eventually sometime around 1973 or so
UNIX itself
was rewritten in C and... wait a second!
If it was rewritten then what was it
originally written in?
The answer is that early on UNIX was
written in assembly for the specific
target platform and hardware
and only as Ken Thompson and Dennis  Richie
developed C as a systems programming
language
did they then decide to rewrite the OS
to take advantage of this new high-level
programming language.
This way, the operating system became
portable, meaning it was no longer tied
to the hardware in question and could be
recompiled for other platforms.
In 1975, Ken Thompson was
on sabbatical from Bell Labs and went to
the University of California in Berkeley
as a visiting professor.
Since its parent company AT&T was
prohibited from selling the operating
system,
Bell Laboratories licensed it together
with the complete source code to academic
institutions and commercial entities.
This, one might argue, ultimately led
directly to the very notion of
"Open Source" when the Computer Systems Research Group or
CSRG at UC Berkeley extended the
operating system with their patch sets.
Graduate students Chuck Haley and Bill
Joy, who, in 1982 would go on to co-found
Sun Microsystems,
added new tools and other software and
eventually began distributing these
as the Berkeley Software Distribution or
BSD.
During that time, two main lineages of
UNIX developed:
the BSD derived or influenced systems as
well as those deriving for what became
"System V"
of UNIX, one of the first commercial
versions of the OS.
As development on the different
operating systems continued, the folks
over at Berkeley added the TCP/IP stack
funded by a DARPA research grant.
This stack is still the defecto sample
implementation of TCP/IP and can be found
in a number of other operating systems.
In the meantime, a company called BSDi
began selling their version of UNIX
based on the Berkeley Software
Distribution, called it BSD/OS
and branding it as UNIX. They even had a
telephone hotline called 1-800-ITS-UNIX.
Now Bell Labs had licensed the OS and the
source code to Berkeley but BSDi was
selling a product derived from this code
and was subsequently sued by Unix
Systems Laboratories (USL),
a subsidiary of AT&T Bell Labs at that
time.
Now BSDi claimed that they could not
possibly be at fault because
they got their code from UC Berkeley
and so USL said "Cool beans, we'll sue UC
Berkeley as well!"
But now things got interesting: the BSD
patches had always been licensed under
the so-called BSD license,
which, in a nutshell, said you can do
whatever you want with this code -
including selling it as closed source -
but you need to give us credit.
Just say your product includes code
developed by us and everything's peachy.
Sounds pretty simple, right? And the BSD
patches included so much cool stuff!
You had TCP/IP, NFS, vi, 
and much much more. Many of the
commercial unix providers at the time
had by then incorporated these patches
as had USL in the product they then
licensed and sold.
Only USL appears to have forgotten to
include
in their copyright notices the
disclaimer that the code did
originate at least in part from BSD,
which according to the license term they
were obligated to do.
So UC Berkeley, ever the spunky
californians,
when threatened with a lawsuit said: "You
know what? We're suing you!"
After some time then the case was
settled: UC Berkeley
would rewrite the bits that were
encumbered by the old AT&T
license, leading to a code base without
any proprietary code known as 4.4BSD-Lite. At this point in time
there were about six files that still
retained
encumbered code out of 18,000 files or
so.
So eventually that was rewritten and the
4.4BSD-Lite
release of BSD became the new
unencumbered version.
At this time, there had already been born
two descendants of BSD:
NetBSD was first released in march of
1993,
with a focus on portability and
technical correctnes; and FreeBSD,
first released in december of 1993, with
a focus on the new i386 platform.
Meanwhile, over in Finland, a young
computer science student named Linus
Torvalds had been playing around with
Minix,
- a unix-like operating system created by
Andrew Tanenbaum -
and he had just finished his own OS
kernel termed "Linux",
which he released on the internet in
1991.
Linus had chosen the GNU Public Picense
for his kernel.
This license originated from the "GNU's Not UNIX"
project, an effort created in the early
80s by Richard Stallman at MIT
to develop a completely free unix-like
operating system.
The new project had written a compiler,
an editor
(emacs, of course), various utilities, and
all the other things,
but they had been lacking a kernel up
until now. An operating system without a
kernel
is not really an operating system, just
like a kernel
without the remainder of the operating
system is not an operating system either.
So the announcement of the Linux kernel
under the GPL
allowed the new project to create,
finally,
a complete operating system: "GNU/Linux",
which is, in fact, the correct name for
the operating system
that nowadays everybody and their
brother and sister everywhere refers to
as just "Linux".
Now in contrast to the BSD license ,the
GPL
provides additional restrictions on the
recipient of the software.
It seems counterintuitive in that it was
a free
software license, but it did add a
restriction; an important one.
That is, you are free to use the code in
any way you saw fit,
but if you made any changes to the code,
those changes had to be released under
the same terms
those of the GPL. That is in contrast to
the BSD license,
which merely states you can do whatever
you want -
including make modifications, keep those
modifications to yourself,
and then sell the resulting product ,so
long as you acknowledge where the original code came from.
The birth of this new operating system,
the new "Linux", despite a more restrictive
license that normally might make
businesses hesitant to adopt it
at the time that the USL vs. UC Berkeley/BSDi lawsuit was ongoing
may have directly led to broader
adoption of Linux over the BSD variants
and possibly caused Linux's market
dominance nowadays.
Of course we don't know for sure, since
we can't go back in history and
evaluate what ifs at that time,
but, be that as it may, throughout the 90s
and early 2000s
many of the commercial unix versions
lost market share.
The number of interesting developments
continued onwards from there.
We'll see over here in the slides a
whole bunch of them
listed by year. Several items that
may be of interest. Of course that's not
an exhaustive list.
The Darwin operating system, for example,
was derived from NeXT,
using the Mach microkernel with code from
the userland coming from Free- and NetBSD.
It was born around 2000.
This, of course, is no surprise in that
Steve Jobs, who had
left Apple and then worked for NeXT
had been using this kernel and upon his
return to Apple
began working on this operating system
that would later on develop
into MacOS X.
Solaris, the operating system that
followed Sun Microsystems's SunOS
after merging a number of BSD patches
and a number of System V derived
features
developed a number of other ground-breaking features,
including ZFS, a very advanced file
system with novel ideas,
dtrace, and containers, when at that time
containerization was really not yet
in widespread use. Android, a linux variant,
and iOS, effectively a version of Darwin
and thus BSD
derived, ended up on our mobile devices.
So throughout the last 50 years, we've
seen a perhaps surprising number of unix
systems.
Some of them are all-caps UNIX systems,
which derive directly from the AT&T code,
some are trademark Unix versions, meaning
they have undergone certification to meet the Unix Specification. This
trademark certification is expensive, so
not many companies would do that and
every time you make a change, you would
have to undergo the same certification
again.
Therefore, a number of operating systems
providers - and especially open source projects -
would not undergo the certification and
not become a trademark Unix
even if they are a so-called genetic
Unix.
And then there are so called "unix-like"
operating systems meaning those are
operating systems that share no lineage
or code, but they look and behave
just like a unix system.
So in this slide, you see basically a
small selection of unix systems,
which I'm primarily showing to remind
people that there are more unix flavors
than merely linux.
The interesting thing is that even
though these different variations
of unix do behave by and large
in the same way  - meaning if you
are able to use one operating systems
you should be able to
quickly and easily adapt to the other,
and if you could write code for one
operating system you should be
able to quickly adapt your code for the
others.
Again, the difference to linux here
is that all of these are separate
operating systems,
while all the different linux
distributions that we see are just that:
linux distributions,
versions of packaging linux.
But realistically, out there in the
so-called mystical "real world" that we
occasionally refer to,
you will only come upon a shorter number
of all these different
operating systems. So over here is a list
that divides them and they groups them
together
primarily under Linux, BSD, and "other",
even though as you can tell the "other"
category has overlapped with the
first two. So it's important again to
note that linux is a unix-like operating system which
just happens to come in a surprisingly
large number of distributions,
whereby different projects or companies
have bundled different pieces of
software
together. One company may add
a web server, or a database server,
another company may focus on desktop
usage,
and still pick and patch together the
same pieces, the kernel, and other
libraries and tools,
bundle them together, perhaps offer
support and call it a distribution.
This is notably different from the other
operating systems which remain whole
units
and cannot be split up or recombined.
We do not have a number of distributions
of NetBSD; there is only one NetBSD.
We do not have a number of variations a
FreeBSD, OpenBSD, or DragonFlyBSD.
Those are all coherent units that are
one and the same.
Now the market share for commercial unix
platforms outside of linux, the BSDs and
the mobile platforms has increasingly
shrunk over the last few years,
so you are unlikely to encounter the
range of
operating system variants I showed in
the previous slide,
but nevertheless, some of them are still
out there,
working hard.
Now our reference platform for this
class is NetBSD.
NetBSD, then, is a direct true genetic
Unix,
even though it does not hold the Unix
trademark; the open source NetBSD
foundation does not have
the monetary resources to undergo a
certification
for their product every time they make a
new release.
But as a whole operating system it
provides not just a kernel,
but system libraries and various
utilities that all are developed
together
and provide a coherent self and a
coherent
entire OS image.
Being a complete operating system, it
also includes some additional
information,
such as the summary of the Unix history
as it relates to the BSDs.
You can find this history under '/usr/share/misc'.
That may be fun for you to browse
through this tree;
as you can see we have here a shortened
version of the unix history as we just
rehashed,
by identifying the different lineages
and showing release dates
as they correlate between the different
BSD variants. As we scroll through here,
we identify how DragonFly splits off
FreeBSD; we saw earlier OpenBSD
splitting of NetBSD,
and then we move forwards through time
as the different releases are made,
towards the end of the file.
We then have an itemized list of all the
release dates
for the different versions, including
additional history information about the
different branches and the operating
systems,
and it may be useful for you to just
poke around and see
where all these things end up.
But to really understand how widely used,
how diverse
the term "Unix" is, let's have a look at
the complete family tree here:
We see the timeline of most unix
versions as it split off the original
systems
from Bell Labs.
As we scroll to the right, we can see the
birth of the Berkeley Software
Distribution with its various releases
alongside several commercial unix
distributions.
Over here, in 1984,
we see Minix being born;
down at the bottom, splitting off of the
initial unix versions.
And as you progress through the mid 80s,
you will notice that there's a lot of
variation,
a lot of code flowing back and forth
between the different operating systems.
Notably, most OS vendors are importing
the BSD patch sets to get all the
features developed in Berkeley.
Now over here, in 1991, we see the birth
of the linux kernel,
branching off of the minix timeline.
In 1993, we see the first appearance of
NetBSD,
followed soon after by the birth of
FreeBSD
in December of the same year.
Disagreements with the NetBSD developer
group leads to the fork of OpenBSD
off the NetBSD timeline in 1995.
In the late 90s, Apple begins developing
MacOS X
or "MacOS 10", which is officially
released for consumer products around
2001.
MacOS X Server was released before that.
Now as we scroll through the early 2000s -
a time that doesn't feel all that long
ago to me, but that
I guess by now actually is - we notice
that there's distinctly less
cross-pollination between the different
operating systems,
and fewer commercial versions appear to
have survived.
In 2008, Android appears,
a Linux version for mobile devices, while
Apple's "iPhone OS"
later becomes "iOS", as we all know.
You see it over here in the timeline,
developing first as "iPhone OS", then
becoming "iOS".
We noticed some code sharing, in parts
based on the liberal
BSD license, as we see over here
NetBSD code flowing into a few other
products,
such as the recent Minix release around
2012.
In a freebsd feeding back into different
products
such as JuneOS  and PCBSD
and other products ,but as we are
catching
up with 2019 and 2020,
we are left with but a handful of popular unix versions.
The final releases that we see over here
remain NetBSD, FreeBSD,
then we have macOS,  we have OpenSolaris,
we have Linux, of course, Android, HP-UX,
but that is about it, which is a sparse
field compared to what it looked like in
the 80s.
Now lest you think the lineage of linux
by itself is any less nuts, here's a
quick look at how the different linux
distributions
developed over time: it looks just as
bonkers as the regular unix timeline
graph, doesn't it?
We identify a few major lineages here as
well:
there's Debian, which, as we follow this
brownish line over to the right,
leads to Ubuntu and all of the variants.
You see a number of short-lived projects
before we take a look at Slackware Linux,
one of the oldest distributions
alongside Debian,
which then also develops or forks off
the SuSE variant of linux.
At the bottom of our graphic we see
RedHat Linux, which nowadays
enjoy enjoys significant popularity in
the server market.
Anyway, as we've seen, the history of unix
and linux as one version of unix-like
operating systems
is diverse; it is no surprise in that
nowadays we find unix just about
everywhere.
Unix runs on your desktop, your laptop,
your servers;
it powers the public clouds provided by
Amazon and Google; it runs on your TV,
your phone, your watch, your stereo, your
car navigation system -
meaning you may at times have to pull
over off the road to install a software
update -
your thermostat, your refrigerator, your
toaster, etc etc.
This is not only fascinating, but it also
has a number of implications:
on the one hand, and this is why this
class is particularly relevant and
hopefully of interest to you,
it means that if you understand unix, you
will better able to understand how all
of these things work;
on the other hand, this also means that
your fridge now has a CVE, that your
thermostat runs a web server that can be
hacked.
We're running a general purpose
operating system on the internet of
things without much consideration
for how to manage these systems. But do
take a look
at the printed manuals of some of your
devices; I'm sure you will find a number
of copyright disclaimers in them
saying "This product includes software
derived from software contributed to
Berkeley by..."
or "This product includes software
written by the Regent of the University
of California..."
or something similar. it's a world full
of unix.
Unix everywhere.
Okay, phew, this concludes our brief
history of the Unix operating system.
In our next segment, we'll take a closer
look at the features and properties of
the system,
of the features of the C programming
language, and we'll discuss the Unix
program design and philosophy.
We'll also finally get to write some
code, as we explore these features,
so please make sure to keep your eyes
open for the next video lecture.
Until next time - cheers!
