- [John Lakos] So, welcome,
my name is John Lakos.
I work at Bloomberg in New York City
and I have a presentation on modules.
I'm not going to tell you
what I wrote the slides.
I'll leave that as a surprise.
The good news is that
Nathan Sidwell presented
these state of the art
of where we are with
modules right now as of last week,
and I was at the meeting
and I am pleased to report
that we are making progress.
These slides are a little bit older.
But the good news also is
that the requirements for modules,
in my mind, haven't changed
in a very long time.
I'll explain how long that's been,
and so I'm going to give you
some background to begin with
as to what we should expect from modules.
Then we'll take the last little bit
to look at what the requirements
are specifically for the upcoming
incorporation into the standard.
Then I will take questions.
So with that said,
I'm going to get started.
I'm going to go really quickly,
I have to put this copyright notice up.
Here is the abstract
but you're already here
so this is for the viewing audience.
What's the problem?
Well we've got this large-scale
c plus plus software thing,
and it's got both logical
and physical design.
You might have noticed that
I broke a book back in 1996.
Good news, I'm committing
right here and now,
to having the next version of that book
done this year and in
fact I've been spending
a lot of time at this
conference finishing up the
figures and whatevers.
So I'm still promising
by the end of this year,
Large-Scale C++ Process and Architecture,
is going to be in hard copy.
So having said that again, (laughs)
(audience applause)
Thank you.
And it's mostly about modules, by the way.
Not kidding, it really is.
This is the stuff that I put up
in every talk which is that we are going
to talk about logical
and physical designs.
But specifically modules
are really physical designs.
So that is what this talk is about.
It's about physical design,
and if you recall the 1996 book,
it was largely about physical design.
The middle part the sort of beefy part
was about that and the
notion of a component
and a module, for all intents and purposes
are synonymous and I've been using
components for 25 years,
No, more, 30, long time.
So modules are due.
It's true that back in 2007
some people came to me and
said, "Hey, we'd like to
work with you on modules."
I said, "We don't need
modules, we've got components.
Why would we need modules?"
Well time has passed
and now we need modules.
Alright, so we're just
going to review basic
component design and I'm just going to
put this up really quickly
without saying all the words
because I know you guy can read.
But this is what we're going to go through
and I want to do it because you can look
at all the syntax that
you want but if the thing
doesn't do the what you need
it to do architecturally
then it really doesn't matter.
So I'm putting this up for you to read,
this is the quick version
and then I'm going
to move on, so this is
what we're going to do.
Alrighty, so here's the outline.
I have four parts to this.
If I had four hours I would
do parts three and four.
But, I don't so were going
to do parts one and two.
That's it.
So here, this is a
component, it's a logical and
physical thing.
I've drawn so many of these recently,
you can't even imagine.
Okay, logical is classes and function.
Physical is files and libraries.
Here is my iconography.
I have this nice stack of physical things.
Those things are modules, by thew way.
Those fun things right
here, that's a module.
You'll notice that the bobbed corners mean
the generic module
opposed to a module that
has interesting logic in it.
Because then they would
be rounded bubbles.
But if they have the bobbed corners,
their just like, yeah it's a module.
It says component but
when you see component
just replace it with module.
It's too tedious for me to go through
and change all my slides to modules.
That would require work.
So when I say a component
I'm talking about a module.
Now this is the old style component before
we had modules so we
called them components.
They had a .h file and a .cpp file
and along with them, so
this is the implementation
and then we had the
header and along with them
we had a single stand
alone test driver file.
When we have modules,
I hope nothing changes.
I hope this is still true.
There will be a few little
syntactic naming things
in whatever the change
but basically I hope
this is still true.
The thing is if it's not true,
I won't be able to use modules and
that would be unfortunate.
So a module should be able to do pretty
much but not everything
a header file can do.
The only thing that we
don't want a module to do
is something that is absolutely wrong.
So the analogy would be,
if there's something
that I don't particularly
agree with, maybe it's a political thing
maybe I'm in favor or not in
favor of the second amendment.
Something that smart people can discuss.
But I don't think we
want to keep polio around
anymore, so we're going
to get rid of those
horrible, horrible things
that just don't belong
for any reason whatsoever.
Chemical records, no, we're not going
to have any of that stuff.
So modules should
preclude those things that
are so horrible that we just
can't have them anymore.
We know what they are,
I don't want to say them
because somebody might argue with me.
So I'm just going to say,
we're not going to have them.
(audience laughing)
But those things are really bad,
they pass a threshold,
that if you want to do that
then you really need to
have your head examined.
So this is the fundamental unit of design.
What is it? It's a module.
So here we have a module and they're four
properties to a module.
Those four properties have
existed since the dawn of time,
or at least as long as I've
been programming the C++.
The first one is that the
header file is included
as the first substantive line of code.
Quickly, does anybody have
a reason or know a reason
why I would want to do that?
Why would I want to
include the header file in
my C++ file as the first
substantive line code?
Say?
Uncovered other dependencies.
Sounds like a good enough
answer given my time frame.
The idea is that we want,
when we include something
we want to make sure
that the include file
compiles an isolation
and if we make is the
first line in it's own
.cpp file then we know that
there's nothing else there.
So every time that we write something,
it compiles an isolation even
if the .cpp would otherwise be empty.
So a component has a .h and at least one
but we'll say for now, one .cpp file.
It turns out that modules have a .h and
it can have more than one module .cpp file
but it can't have more
than one interface file.
So again everything seems
to be mapping exactly
on to what we've done forever and ever
in components, that's great.
All logical constructs having
external linkage defined
in a .cpp file are declared in the
corresponding .h file.
This may seem like second
nature to some of you.
Turns out that as recently as May,
I learned something
about that and I've been
thinking about his for a long time and I
want to think Chandler for
clearing this up for me.
Because I learned that every,
every definition in C++
is also a declaration.
Now that is meaningless but I learned it.
So you're hearing it now but forget it
because it's not useful.
So I'm telling it, but just forget it.
But I'll talk about that in a minute.
A declaration typically
introduces a name into a scope.
But hyper technically every
definition is a declaration
but it's not a useful
thing to say because then
what's the point.
So I'm going to use the term
declaration definition both.
When I say it's a
definition only that's a lie
but it's a lie only if you
are talking in C++ grammar.
What I mean is the definition only doesn't
introduce a name into a scope.
What do I mean by that?
For our purpose, a declaration introduces
a name into a scope.
A declaration can introduce something else
into a scope, but don't worry about that.
There are the things
that make it impossible
to have an conversation in a elevator.
For our purposes, so.
Let's look at what this means.
Int a; is that declaration
or a definition?
It's a declaration and a definition.
Right, I put that in
file scope, it's both.
Extern int a; declaration or definition?
Excellent.
Declaration Only.
Extern int a = 0;
(silence ensues)
So if you don't say anything,
(silence)
It doesn't have a definition but what
works out to happen kinda sorta some
implementation, maybe
is you'll get something.
Maybe.
Look at that.
Okay.
Ah.
So this is the lie.
It's really a declaration
and a definition.
However if you don't have the declaration
it won't compile, so even though it's a
declaration and definition, it isn't.
(audience laughs)
So I'm going to stand-by
it's a definition only.
If anybody has a problem with it,
argue with Chandler not me.
Alrighty.
So is this external or internal linkage?
At file scope, it's external linkage.
What about this?
And this?
And this?
So that's interesting, right?
Used to be a long time
ago, when I had more hair.
It used to be internal and problematic and
all kinds of things because at one point
it was the C front
preprocessor kind of thing.
But now it's for real.
It's just, we inline it and compile it,
so people have to do more work.
What about this?
And what about this?
And this?
(silence)
(silence)
What about this?
This?
And this?
This?
Trick question.
(audience laughs)
This?
This?
This?
And this?
And this?
(silence)
Doesn't compile, why?
Why doesn't it compile?
So what if we get rid of that?
Then it's internal linkage.
Right?
If you know this without
checking the standard,
like if you know all of
this stuff like totally.
Then you are better than everybody because
this is really hard stuff sometimes.
But it doesn't matter,
what if I get rid of that?
Very good.
Alright, you now understand
what I mean by those terms.
Actually let me go back
and explain with you.
All constructs having
external linkage defined in
the .cpp file are declared
in the corresponding .h file.
What that means is, if you have something
that is a definition in your component
and it has external
linkage, which means it
could affect something
outside your module.
Then it must be declared in your module
and the current everything
that we're talking
about right now in modules, does that.
That will not happen otherwise.
That's good news.
Every construct with external linking, ah-
Every construct having
external or dual bindage
declared in a .h file,
if defined at all, are
defined within the component.
What the heck is bindage?
Can you find that in the standard?
No. I made it up.
So I'll tell you what it means.
So let's hear some examples of bindage.
Anything that the compiler
does to bind things,
if the compiler does it,
it's internal bindage.
If the linker does it exclusively,
it's external bindage,
and if either the compiler
or linker could do it,
then it is dual bindage.
So what's an example of internal bindage?
Anybody?
How about a typedef?
If I have a typedef declaration,
the binding of the name
to it's structural definition
has internal bindage,
because it's being bound,
the name is being bound into
it's definition by the compiler.
Does that make sense?
Okay. It's not linkage.
It's not the same thing
as linkage. It's bindage.
It's tool based and if the
linker is going to do it
for example, I have an
function, a plain old
C style function.
The linker deals with that, right?
You got a name, you declare
it in the header file,
the client uses it, you
get an undefined symbol,
then somewhere you have
a definition, there's
a symbol definition.
The .Os get mixed together
and then the linker comes
along and puts the two
together and binds them.
That's external linkage.
Now, bindage. (background soft speaking)
Thank you. My buddy Pablo
is keeping me honest.
Too much coffee, so bindage not linkage.
Linkage is a standard term,
it has standard meaning.
It has nothing to do with tools.
Bindage has everything
to do with tools and
nothing to do with standard.
Yet the two rule together
very nicely it turns out.
Can you tell me two
constructs in the language
that typically, and by
typically I mean always,
have external
and internal dual bindage?
What's a construct other then Pablo,
because I know he knows.
(soft muttering in background)
Templates, Templates is one.
What is the other one?
- [Voice] - Backrows.
- Backrows do not have
anything to do with this.
They're not part of C++ ,
they're a pre processing step,
leave them out.
If they had anything they would
have preprocessor bindage.
(swallowing sound)
Not types.
Some kind of function.
- [Voice in Audience] Inline.
- Inline functions. Inline
functions and templates
behave the same way when
you look under the hood.
Right? Okay.
So there are instances
of the function in every
.O and then the linker
comes along and deals
with unless the inline
function gets inline directory.
By the way, templates and
inline functions can be inline.
The reason for having
the inline keyword is not
what everybody thinks these days.
It's to enable it to live in the header so
that it can be seen by
the other compiler without
violating the one definition.
Contrary to popular belief,
if you put the word inline
in front of a template
function the compiler is more
likely to inline it then if you don't.
So it does mean something
and the person I learned
that from is sitting
in the front row here,
Pablo, and he told me
about that in about 2004.
That was before I wrote
these slides, by the way.
What was the point of this? Oh yes.
All constructs having external
or dual bindage declared
in a .h, if defined at all,
are defined within the component.
What does that mean?
That means if I declare
it and the linker could
get involved, which is
pretty much everything but
the forward class declaration.
I say forward class
declaration for a reason.
A forward declaration
implies the definition is
coming in the translation edict.
If it turns out that's
not happening, then we
don't call it a forward
class declaration, or
a forward anything declaration, we call it
a local declaration.
Local declarations that are not
forward declarations are bad things.
So if you declare a
class, there are issues
with doing that, but if you declare a
function locally, that's a problem,
because now what can
happen you no longer have
a neighborhood for the client to say,
"Oh look, I'm using this
in the context of this
header file, it doesn't
match, compiler tell me."
or, "I'm defining this
in the context of this
header file but it doesn't
match, compiler tell me."
What you have, is you
have a link time error.
Link time errors are brittle
and bad and we won't do them.
Then finally, we use components
pretty much exclusively
to gain access to functionality.
Meaning that we don't try to
use declarations, in general.
When we need to use
functionality at all, instead
of repeating it pound, include or import
what it is that we need.
Nothing's changed in a very long time.
These slides were written a long time ago.
Does anyone have a
guess as to how long ago
that I wrote these slides?
I'm not going to tell you now, anyway.
This is my iconography.
Just putting it up here.
These things are modules,
but they were written as components.
You'll notice there are classes and
you'll notice that one
of the components has
two classes and one of the classes has
an underscore.
Underscore means that it's
private to the component,
but it's just a convention today,
but it works just fine.
We like that.
We don't like making
the class a nested class because
all kinds of bad things
happen when you nest
classes in classes.
All kinds of weirdness and friendships,
and strangeness and
diagrams don't look good
and you have to think really
hard and the indentation
is horrible, so this is good.
Turns out that when we
get our modules together,
this will all be implemented naturally
just by accident.
Total accident.
Pure coincidence, but this is the way that
I've done things since 1990 something.
So Is-A, A polygon is a shape.
Well normally we wouldn't
have a value semantic type
implementing an abstract interface.
What does that even mean?
But that's another talk.
But for the purpose of intuition,
let's assume that shapes
and abstract interface,
polygon is what you think and we have a
Uses-in-the-interface
relationship that says that
polygon uses point in the interface.
For example, that of vertex.
Something uses something,
function uses something
in the interface.
If it names it as part of
it's signature or return type.
We look at PointList and PointList_Link,
both of them use point in
their interface clearly.
Now even though PointList_Link is not
accessible programmatically from outside.
Which is what we mean,
nonetheless, it still uses
Point in it's own interface.
Then we have this notion of
Uses-in-the-Implementation,
a polygon might have as
a data member a PointList
and so it's not
programmatically accessible.
If it's not programmatically accessible,
I can take that out and
put something else in
and everything would work just fine.
Except the client, the
client might have included
polygon, and polygon
might include PointList.
The client might go, "Oh
look, I have access to
a PointList, so I'm
going to use a PointList
somewhere else."
That's called an transitive include.
If we change the PointList
to do something else,
to be something else, let's say I use a
Std vector, not I don't have my PointList.
So I take the PointList
pound include out of
my header file for
polygon and what happens?
I break my client, my
client was relying on
that implementation detail in a very
nefarious insidious nasty way.
It would be nice if
modules took care of that.
They do.
PointList uses PointList_Link
in it's implementation.
No where outside of this module can you,
if you have access of PointList would be
be able to know that the
PointList_Link is being used.
But wouldn't it be nice
where we had a view
where that was visible and we could say
that it wasn't?
In other words, if there
were a function that
actually gave you the
PointList_Link for some
purpose inside the component
but not for external use.
Let's say you wanted to do that.
Wouldn't it be nice,
if we could expose only
the subset of methods
we want poured it out
externally?
Turns out, that's an
incredibly useful thing
to do for many, many reasons.
It's not currently proposed.
It's not currently blocked.
Write your congressperson.
Before modules are done,
that ability must exist.
Finally, Uses in name only, which is kind
of a strange thing
because it doesn't imply
physical dependency.
So shape, being an abstract
class has something
called origin and I can
get the origin of the
shape abstractly and
it's returned by value.
But just because it's returned by value,
does not mean that you
need to know it's size
because it's an abstract interface and
in fact this is a general property.
Just because something
is used, passed in by
value or returned by
value, you as a client
don't need to know it's size unless you
call it a function.
This is news, I'm sure.
But it's okay.
So we use this special symbol, this is
Uses in name only, and Uses in name only
means that I can build the entire thing in
test it without ever having
known the definition.
Without ever including the header.
Then we have this thing
called, Depends-On.
Depends-On is the fat
arrow, when it goes from
box to box.
Boxes are physical entities,
the inheritance arrow
is bubble to bubble.
Those are logical entities.
So a polygon depends on shape.
A polygon because of this
Uses-in-the-Interface,
a polygon, we know this just
by the symbols by the way.
It's explained in the 1996 book but it's
also explained in the
2018 book, so either one.
You can have this explained
and why it's true.
Those logical symbols
apply physical dependencies
between the physical
entities that contain them.
So this is good to know.
Level numbers, once you have
those physical dependencies
disregard the logical, we go up a level.
We can assign level numbers.
If something doesn't depend on anything
else it's a level one.
Locally, by depends on, I mean locally.
Because shape depends
on Point in name only,
it doesn't actually depend on it.
It's also on level one.
However we have something
else that depends
on Point, that's at level two and
polygon although it
depends on Point and shape,
it also depends on PointList,
so it's at level three.
I hope that makes sense.
Essential physical design
rules, there are two.
Somebody tell me quick, we have 35 mins.
What are the two most
important design rules?
Yes!
No cyclic physical dependencies.
What's the other one?
Also very important.
Really?
Really?
No long-distance friendship.
No one should be able to
from the outside reach
into a module and grab
something that's supposedly
and Imp detail of a module.
Does anyone disagree with that?
That's been a rule forever.
Go look at the 1996 book.
Those are the two most
important rules, and
the third one, the third
most important rule
is no transitive include
because it's brittle
and it breaks large systems.
One, two and three. Modules
address all of them.
When do you put two public classes in
the same component?
It's an important design consideration.
All of this is covered in
the 2018 book, by the way.
So there are four.
One of them is friendship of course.
You have a iterator and
it's container and they
go in the same component.
Another one is cyclic dependency.
Almost never happen,
but they're rare cases,
with templates that depend on each other.
They do need to do that.
So there are reasons
for doing that, but they
are rare and they're
specialized and they're
typically compiled
time-ish like parsing tree
thingys, but they exist, very rare.
Very rare in my experience. Super rare.
Single solution, what does that mean?
Before we have variatic
templates and those things
we would have approximations
to them that had
multiple syntactical
elements, that together,
although they didn't depend on each other,
together they conspired to
give you a single solution.
Don't confuse it with
hierarchal solutions.
Those I have a point, a coordinate, a box.
A box collection and a garage.
Now you might say, "Wait a minute. How can
I possibly do anything
with a box collection,
if I don't have a box? So I'm going to put
the box in with the box collection."
What's wrong with that?
Somebody.
- (softly whispered speaking)
- You can use the box by itself without
the box collection and that's a reasonable
thing to do.
So just because you have
something complex that needs
something simple, that's
not a reason to put the
simple thing in with the complex thing.
Because the simple thing
is hierarchically usable
somewhere else.
Does that make sense?
So this is a bad idea.
Instead, we'll use this
think independently,
and maybe we'll use those
two things independently,
and maybe these three
things independently,
maybe all four of those
things independently.
We don't need a garage.
Even though all of these things can be put
in a garage module, we're
not going to do that.
We're going to put them
in separate modules.
There's no charge for
having separate modules.
They do not get charged by the module.
In fact, there's an extraordinarily strong
argument to use class
categories, something
that Herb Sutter has been
talking a lot about recently.
He calls it something else,
but these class categories
which have existed for and we've used at
Bloomberg extensively,
were clarified around
2011 and they are in our taxonomy.
You can go to Bloomberg
and see our taxonomy.
They are different categories of things
and anyway but let me move on.
So this is fine.
The last one is Flea on an Elephant.
What does that mean?
If you have a flea and you put it in with
the elephant, no one's going to care,
the example of this might be for example,
if you have a logger and
you need a scoped guard
to activate the logger in Maine, say
logger create, whatever it is.
That's a little bit of
sugar that gets something
going, it doesn't change the dependency,
it doesn't change anything
other then provide
a needed piece in a welcome usage example,
in the module, every module should have a
usage example.
Maybe multiple usage
examples, but at least one.
On the other hand, if you have a flea,
and you try to stuff a elephant in there.
People will notice and
we need to be careful,
this can be abused.
This is no good.
Of course, we don't do that.
But as Pablo suggested,
an abuse alert might
be Appropriate because you can rationalize
almost anything.
You can well it's not
a flea and a elephant,
it's not a elephant and I flea, maybe
it's a dog and a goat
or something like that.
No, it's really got to be
a flea and an elephant.
If it isn't, don't do it.
So now I'm going to quickly
talk about the other
side of the physical
coin which is Insulation.
So encapsulation, an implementation detail
of a component , type,
data or function, that can
be altered, added or removed
without forcing clients
to rework their code is
said to be encapsulated.
Technically encapsulated
means the functions in the
data are in the same capsule.
That the data is private,
is data hiding, but
people leave that out and say that's
encapsulated to mean it's hidden.
It's together and the data is hidden.
An implementation detail of a component,
type, data or function,
that can be altered,
added or removed without
forcing clients to
recompile is said to be insulated.
What that means is that it's not the same,
it's not necessarily a
stronger condition, but
it's a stronger condition.
It's saying that, that
I can actually make this
change and my clients
don't even have to compile.
People have heard of the pimple idiom.
The pimple idiom is
designed to insulate the
pimple from the client.
In 1996 we called that the
fully insulated concrete class.
Go look at the book.
But I later found out it's called pimple.
Not sure when the term was coined.
So here is a very rare case where I have
dependencies pointing up.
I don't know why they are
pointing up but they are.
And I was too lazy to change this slide.
It's an old slide, by
the way, did I mention
it's an old slide?
So, the dependencies
are pointing up but it
seems to work better,
so here's the client.
Oh I know why, because
this is includes and
people think of includes like that.
Think of this as one
piece of paper, maybe.
So this is a library
component, so I have a
client and a library component and the
client component depends
on the library component
in a very specific way.
It includes its header in it's cpp file.
That's what's happening.
Now, depending on the
library component it might
depend on something like an Imp Detail D.
So D is an Imp Detail,
but it's included in
the header file.
Which means if I make the
change to the header file
of D, that will flow right through to A,
because it will flow
through C's header file
to A, that means I need to recompile C and
I need to recompile A.
Okay?
Whereas, D is encapsulated
by C and what we really
mean is the Use Of D is encapsulated by C.
The Use of D, because D in
not contained, it's used.
This is important for
hierarchal, we used to
be successful, we can't
hide the implementation
details of the implementation details of
the implementation details
of what we're building.
So they all have to be
publicly accessible.
But their use is encapsulated.
On the other hand, this guy
is included in the .cpp file.
So that changes, it
flows through the here.
That needs to recompile and we're done.
It does not affect the client.
The client does not have to recompile.
Do you see the difference
between when we say
module, right, the code is modularized.
It's hidden, it's encapsulated
but is it insulated?
Who knows whether it's insulated or not?
Is it totally insulated,
partially insulated,
not insulated one tiny
little stinking bit.
What do you think?
(soft voices in audience)
(audience laughs)
I have a vote for partially insulated.
When I go to modules,
when I take something that
module compatible, instead
of a component, and
I move it to a module, we
know what happens with the
private classes that had
the extra underscore.
They won't be visible
because they're not exported.
We know the header file
was used, won't be visible
because it won't be accessible either.
It will be completely local to the module,
because it's not exported.
But what about something that is exported?
What?
What about templates?
The question is what is being
hidden, what implementation
details can a module hide?
So if I have, for example, a box
and a box consists of two points, and
the point is private to the module,
it's just used as data members of the box.
If I do the header file
thing, I have to know
the size of the points but
if I do the module thing,
do I have to know the size of the points?
Yes. So the sad fact is
that no matter what we
do with modules, it will
have absolutely nothing at
all to do with insulation,
which means pimple lives.
The same way it always did.
Anything you ever did to
try to reduce compile time
compilation in the traditional
way will be left alone.
Now that's not to say that
we can't have pre compiled
modules the same way we have
pre compiled header files.
But that's not part of the standard.
The point is that modules
give you architecturally,
is no different from that
perspective then what header
files give you.
I just want to be clear on that.
From an architectural point of view.
From a build point of view,
that's a different story.
So, the Use of E is insulated by C.
That's important, we
can take the thing out,
we can put something
else in and life is good.
Criteria for having a
include in a .h file,
a header file must be self-sufficient with
respect to compilation.
I think we can agree on that.
You shouldn't have to
include something before
you include a header file.
I'm including this header
file, it should compile.
Yes?
Okay, no one would disagree with that.
So there are five reasons.
Is-A, if something is a
something else then you know
for a fact that you're
going to need to know what
that thing is.
Is-A is one of the strongest dependencies.
So Is-A in fact is the
one thing that we can say,
" If I'm a polygon and I
know that polygon is the
shape, I don't have to
include shape. I know polygon
is going to include shape."
That's a given.
That's the one exception,
every other kind of Uses,
you include it. if you use it,
you include it or import it.
Has-A, if I has a something
then I'm going to need
to put it in the header file
because that's necessary.
In modules that won't be necessary if
it's not used elsewhere,
so that's a good thing.
What about, what's another one?
There are five.
So it turns out, not Uses.
So here we have an example.
I have const Point rep, I do not include,
for this reasoning, I
do not include Point .h,
I say class Point.
I don't need to include it.
It's not necessary.
What about that point, do
I need to include it here?
So I already said no,
somebody wasn't listening.
No, you don't need to
include it and I know
you don't believe me and I want you to go
right outside and check, or if you have
something portable, check it now.
No, you don't.
You don't need to include it just
because it's declared there.
It's only if you call the function.
Alright, so inline.
If I have a body of an
inline function and you
happen to be using a
compiler that moves into
templates or whatever, anything
that's in the header file
or whatever, that could
be looked at to see
if it makes sense.
Of course, you want to
make sure that's included
so you can see it.
Enum, until recently,
C++ 11 recently,
Enums had to be included.
Now we can actually, we
can do something where we
allied the numeration.
That is a whole can of
worms of it's own and
I'm not going to go into
it now but warned if
you're using one of those
new fangled enumerations
that allow you to inline
the enumerators and
you think you are doing someone a favor,
be very careful.
I'll just say be very
careful because it's tricky.
And the fifth one, is a typedef.
Typically like string,
those kinds of things we
can't forward declare
them, we're not supposed to
forward declare standard templates because
they can put in op-shul
arguments and then it's
not longer compatible.
So, there's strong
suggestions not to do that.
So, the bottom line is,
there are a few routine
cases that,
I learned a new word recently, called
quotidian, so quotidian
cases, everyday cases,
where we do this and
there's some other strange
ones like covariant
return types and whatever.
These are the examples
where we do it but if it's
not one of these don't
do it unless you have a
good reason, the real
reason to do it is to
makes sure header file
compiles in isolation and
if you're not doing it for that reason,
then don't do it.
Any questions on this?
Alright, here's some questions.
(soft talking in audience)
The question is, where do we have it?
Is here. This one.
Yes, right here.
Repeat the question.
The question is, repeat the question.
(soft questioning from the audience)
This one, yes, I can do this.
Check it.
If you don't use it, if
you use it, include it.
Remember, if you use it, include it.
If you use it, import it. Same thing.
Question, I'll repeat it.
If you have an inline
function body, obviously
you're going to pound
include the header file
you can't forward declare
the inline function body.
Okay.
So back to these questions.
Let's pick one.
How do we extract component
dependencies effectively?
How does one do that?
I was doing this back in the
early 90s before we had client.
How did I do it?
- (soft voice in audience)
- What?
Remove one header compiler,
that sounds very painful.
Say it again, inline-
Analyze the header, but
I didn't have client
back in the 90s, how did I do it?
Grep. I grepped for what?
Grep free include, so basically
if I followed the rules
I told you about that
modules will enforce,
by the way by accident.
If we follow those rules,
all we need to do is look at
the import statements of
the modules and we can
infer the envelope of physical dependency.
Who knew.
That can be done 10,000 times more faster
then running a client program.
Which means it's scalable.
Which means we can do it
for the largest projects
very quickly using a prototype conscript.
Right, tiny, tiny script.
Alright, so why would we
put a include directive
in an include, just checking.
Does anyone know what
the reason, The reason.
- [Man] It compiles.
- There you go, so it compiles.
Include order is independent,
we get rid of include
order dependency.
And finally what's the
cost/benefit associated
with insulation?
If insulation is so
good why don't we always
insulate our code?
Okay, one it's ra-bose.
What's another reason?
Inheritance, I'm not sure
I understand that one.
Indirection, and what's
the result of indirection?
Performance.
So often if you fully insulate
something, it's going to
be slower but there are
awesome partial insulation
techniques where you can
find the sweet spot and
insulate just that and if
you think about a stack,
where's the one place
you want the insulation?
The part of push that has
to grow the container.
So if you have one function that says,
"I'm not big enough, find
me some more space and
make it private and put all
that code in that C++ file
and all the inclusions
associated with it."
You get the insulation
and when it turns out that
was a bad idea and you want to change it.
None of your clients have to recompile and
everything else runs as fast as possible.
It's an awesome, awesome technique.
Partial insulation technique.
Strongly recommend it.
Lakos, 2018.
Alright, so let's get
to the part that we are
supposed to be talking about.
I'm not as far ahead as I wanted to.
So I'm just going to tell
you about the requirements.
I've been putting them
in the talk as we went.
It's a critically needed
language feature now.
It wasn't in my opinion
15 years ago, because
we had components, but
enough has been put into the
header files that we
need to worry about it.
There's a lot of stuff
we want to clean up.
As recently as last week, the
orange-strew-strip informed
us at the module meeting
that was here that people are
receptive to the idea that
modules are not going to
be one hundred percent
compatible with the worst
of the worst code that
has ever been written.
So isn't going to be the goal.
Absolutely promise you that
it's not Bloomberg's goal,
to take existing legacy code
and jam it into modules.
That is not the purpose,
we have no interest in
doing that, what we have
interest in doing is taking
modules and making capable
of doing all the awesome
work that we want to
do and translating that
which is out of date or
really badly designed into
something that is viable.
So if we make modules too
restrictive so that we
can't do what we need to
our jobs, we've failed.
If we also make modules
so general, that the worst
of the worst code gets
in and thereby harms the
greater society, if you
will, like say oh sure,
long distance friendship,
sacred dependencies,
yeah, we're not even
going to help you with
transitive includes.
No that's your problem.
We have again failed.
So we need to have all
of the major design rules
that have been proven over
20 years to say enough.
(loud popping noise)
Enough is enough and we're
going to make that change.
So only those very very
well understood changes,
where we're not going
to allow things that are
absolutely wrong, those
things we're going to fix.
Other then that, anything
that is plausible,
anything that we need to do
our jobs, anything that even,
doesn't even matter whether
I agree or not, as long as
an argument to do it,
that there is a business need
somewhere to do it, then
yes it should be supported.
But it is absolutely no
reason to support it other
then it is a bug and should be fixed,
we don't need to support it.
Now the thing is the other
thing, if you went to
Nathan Sidwell's talk
earlier this week and you
can catch it on the video,
because we don't have
a time travel machine
yet or you would have
already seen it.
That talk, talks about the
latest and greatest enhancements
a little bit more about the
syntax, a little bit more
about what it looks like
to write modules and those
kinds of things.
Very up to date with the proposal.
This talk, and this
particular slide, this paper
that I wrote in 2017,
my goodness, about 20 months ago.
I wrote this very quickly
and it's uncharacteristic
that I actually publish stuff like this.
This is the actual paper business
requirements for modules.
This hasn't changed in my
mind in many, many years.
It's not about performance,
it's about providing
a better architectural
piece for C++ development.
The performance will come.
I look at performance
as small rocks compared
to the big rocks of having
the right properties.
If we worry about performance now and
we don't get the priorities right,
we've lost an opportunity that comes once
in a generation.
So it is my goal to make
sure that we preserve
all of the good properties,
much the way that we
wanted to get concepts
right, we really need
to get modules right
and we're moving in the
right direction.
I question strongly whether we're getting
there fast enough to have it in 2020.
We will have it proven
it's skill in 2023 and
Bloomberg has made monumental
commitment to make sure
that happens by doing our
own prototyping ourself.
There's been a lot of
work that has been done
as I said, this is paper from 2017.
A lot of work has been done by then but
I'm going to encourage
everybody that we don't want to
run ahead and do modules
so quickly to get the
performance benefit that
we miss out on the other
opportunities.
So there are four points
that I want to make.
There's purely additive, which
is what modules need to be.
There's hierarchical
incremental and interoperable,
by purely additive, I mean,
I should not have to change
one line of my legacy code
base in order to start
using modules and integrate it.
If I add modules it should
affect none of my legacy
code base, that's hierarchical.
It should be incremental,
any group in my company
could start to use modules
and it wouldn't be a
problem with anybody else
who's writing new code.
They can use it or not.
And finally, interoperable,
if I have a client
and they want to consume
modules but they also want
to include old header files and let's say,
both of them depend
something on a lower level.
There's no one definition
violation at all.
Those are requirements.
So logical versus physical encapsulation.
Right now if we have private
data we have to pound
include the private data's header file.
That needs to be addressed in modules.
So that we don't transitive includes.
That's very important.
Another one is, with
the advent of contracts,
which is something that
Bloomberg has worked
very, very hard to make sure happen.
The level of which we do defensive checks,
precondition checks, post
condition checks, whatever,
should be able to be
controlled by the module
and not be affected by the
client consuming the module.
This is an important
property that does not exist
for inline functions and
templates in standard
translation units.
Modules need to fix that.
We need to address that.
It is not fully addressed yet.
We need to make that happen.
Many people try to, in
the object oriented world,
try to things like
procedural interfaces where
I can give a client a view of my system.
Wouldn't it be nice if
we had a modular view,
that said, "Here is my subsystem,
it's under construction,
you didn't pay me enough,
I'm going to give you a
subset of the functionality.",
and I want to do that
at a per member function basis
so that I can have a poor
man's encapsulation.
So that when I finally
come time and I figured out
what I'm doing over here, then I can say,
"Okay, here's the whole
thing. It's stable."
So I'm providing what's
called a stable view but
over here I have some
people that are much more
tolerant that are closer
to me that I can trust.
I can give them a different
view of my system.
Then that one fine day,
there's somebody up there
wants the work that
their doing and the work
that their doing, and
they're sitting up here.
Now we have to unite two views.
Imagine you have std
vector, one has a push back,
one has pop back, but they
don't have the other function.
The guy up there that
includes both views gets both
functionality, get the
union of the capabilities.
This is the kind of
functionality that modules can
provide, and will stop
people from wrapping things.
Has anyone ever tried
wrap something, you lose
performance, it's a big
pain in the butt, it's a
maintenance nightmare,
it's a lot of crufty code,
there's no reason for that.
So by just having views,
we will be able to achieve
something that people try
to achieve in practice.
That is why I mentioned the
C procedural interface thing
which I got ahead of myself a bit on.
So, real world scenario,
just to run through it.
I might have something
that looks like this,
this is mostly for the
audience at home so they
can read all the text.
But you can imagine that
I might come along and I
might want to add a system
that is a module based
system that uses a library
module base system.
Later, maybe I want to have
a client that uses that
library module based
system and also some legacy
header and both of them
use some legacy header.
This should be possible.
This was actually in
the talk as described.
I mean in the paper when
I wrote it, so I'm putting
this up there but since
I'm low in time and I want
to answer questions.
A year ago, there was a
lot emphasis on caching and
we know from experience with
the template repositories,
that if we put caching
into the standard, we will
be doing ourselves a huge disservice.
We want to maintain
embarrassingly parallel nature
of individual translation
units so that anybody,
if I happen to have 10,000
machines and I want to
build each module
independently, I will get
maximum performance gain by
doing that opposed to forcing
the lowest level to create
some artifact and then have
the second level create some artifact.
In fact, what we're getting
is build time levelization of
our system and you can just
imagine what would happen
if the system weren't leveliazable.
Oh my goodness.
Okay, we don't want to do that.
We want to make sure that
everything is buildable in
an embarrassingly parallel way.
Once we have that, the natural
consequence the compile
developers will make it fast.
They can't help it, they will.
That will happen.
But there's an order to things.
Big rocks first, then small
rocks, then sand, then water.
I'm not going to say
performance is water but
it's not a big rock.
There are many different
competing ideas around this.
What I'm basically saying
is there's a lot going on.
I have about eight minutes
left, so I'm just going
to quickly review what's
going on with modules
in big font.
We do want to reduce compilation time.
We do want to reduce but
not eradicate macros.
That's a misconception.
Macros are necessary for
prototyping and language features
and for things like loggers
and for things like assertion
levels and such, it was
necessary to have macros in order
to do that.
It will continue to be
necessary to have macros to do
prototyping and I do not
know, today, to write a
logger without a macro and I'm
pretty sure it can't be done.
We have a very nice logger
that you can look at,
it's the B-A-L-L logger and it's available
opensource and I suggest
you take a look at it.
It is our style of programming an it has
been around for 15 years
and it's battle tested,
and it's awesome but it uses macros.
We can't write module systems
that doesn't allow that
to be modularized.
It can not happen.
We can't use it, it's just
a fact, it's not like we're
being sniggly, that is
throughout our code base.
We have to be able to do that.
So I put this thing up
here, reminding us what
additive, hierarchical, incremental and
interoperable mean but
you've already seen it,
so you can go to the video.
I wanted to remind you of these things.
I already said this stuff so
I don't need to repeat it.
So again, this is the idea that over time,
we'll add things, the
red things are modules,
and that can happen
and that can happen and
then I can do that, and then
I can do that if I want.
Then I can decide that I
didn't want to and then later
I can go do that if I want to.
There should be no reason
that I can't go back
and forth and do that.
So in other words, purely
additive, incremental,
hierarchical, and physically
modular are the things
I'm trying to get at.
We don't want to do this
prematurely, requiring
centralized repositories is a bad idea.
I already said that and
I'm still going to argue it
may be too soon even with
all the progress to get this
into C++ 2020, but I will
try and keep an open mind
and what I'm really trying
to do is ensure that we
don't make any missteps.
So if we put something
in that's too small but
we've got it right, I don't
have a problem with that.
But a big problem is if we
try to put something that is
too small and we don't have a plausible,
rest of the puzzle pieces to make it work.
And with that, I have five minutes left.
I'm sorry.
So I'll just go with questions.
But this is actually
one of the engineering
requirements in modules.
I'll leave that out.
I'll just leave it with questions.
(audience applauds)
- Questions?
- [Man] You didn't say
how old your slides are.
- Oh! So fair enough, so
does anyone want to guess,
when the first part of the talk?
I presented those slides somewhere.
Does anyone want to guess?
And by the way, when I
presented those slides,
I actually had older slides
that were very similar,
but those particular slides,
with a couple of updates.
When do you think the date
was that I presented them?
It was ACCU.
(Audience quietly makes guesses)
96, nice guess.
No, not quite that long ago.
I remember the date.
It was part of a larger talk.
Alright , the answer is 2009.
So, these are nine year old slides.
Nothing's changed.
Just changed component to module.
It's the same concept,
excuse the overload.
Okay, thanks for asking.
- [Audience Member] So you
mentioned that it's not possible
to write a logger without macros.
So would there be some sort
of construct that could be
added to the language so you
could write like a logger
without macros?
- So the question, which
you just said so on the mic.
The question being, could we
write, add something to the
language, I presume other than
macros, to allow something
to be written and yes, are lazy
expressions I've been told,
that are key to doing that,
but then you'd still need to
have some zero cost way of
putting it all together.
So if you combined cons-tex-ter
with a lazy evaluation
with a this, that and the other thing.
But the real problem is, when
we try to write something.
If we're not going to log
it, it needs not to be there.
So the evaluation of the
perimeters to the logging function
can't be evaluated until
you've decide you are going
to log it, and that's a run
time thing and that's why
macros important.
If that makes sense, configuring
the syntax so that it's
as easy as possible to do
that, but of course if we were
to come up with a dedicated
syntax for that purpose or
something similar to that or
piece parts that would allow
us to assemble that, then
yes of course it can be done.
- [Male Audience Member] So
you mentioned getting rid of,
at least exposing transitive dependencies,
which is a big design goal with modules
and thank you of that.
That's awesome.
But does that basically
mean that the module exposes
a much wider interface to the
linker then it does to the
users of the module?
You have a whole bunch
of un-utter-a-ble things.
- So the point is that what
we get, is we kind of get
three different views.
We used to have, you either
didn't get to see it at all
because the client compiler
didn't get to see it or
you get everything.
So what we're going to have
is an in-between, where
the client compiler can see
the code but the client can't.
The client can't create a
new one, so I can't create an
instance of Point, but I can use the box,
which itself has access to.
So, indirectly, I can use
it, but that solves the
transitive include problem.
Did I, I did it?
- [Audience Member] Yes.
- Okay, good.
Good question, thank you
for clarifying that for me.
I think you did that for
my benefit and not yours.
(laughing)
Any other questions?
Well all right then.
We have a minute and 25 seconds
left over, that's a first.
Well thank you all for coming, enjoy!
(Audience clapping)
