Okay, so I'm gonna go over the basics of building neural networks in Simbrain.
I'm gonna start by opening up a network window.
So, we can start by just creating neurons, and to do that you could either press this button here.
And as you see, they sort of growing a line out of the first one you make.
You could click where you want the first one to be, or the next one to be.
So wherever you click that's where the neuron will show up, and then you could sort of make little lines that way.
You could also use the P key for "put", so I'm going to do Command + A to select all and delete.
(repeating "P" with key presses)
Click!
Now to select neurons, you simply click on them.
You could also... left-click drag to make a lasso,
And select them that way.
You could also shift,
To add or remove from a selection.
And then there are a few key commands for neurons. If you just press the letter n, it'll select all neurons.
You could also do... Command or Ctrl + A, depending on your system,
To select everything, which includes all the neurons in this case.
Another cool feature is... smart copy and paste.
So I'm gonna... get rid of these, so I select them and press the delete key.
You could also delete, by the way, by selecting and pressing this button. This is the delete neuron button.
So for smart copy and paste, I'm going to select these,
Command or Ctrl + C to copy,
And then, paste.
And then I could move them to create...
A difference vector between the original copied neurons and the new pasted neurons.
And then when I paste again,
It'll use that same vector to create the new ones, so you can quickly create a bunch of neurons this way.
To show you in the opposite direction, I'll select these,
Command or Ctrl + C, Command + V, to paste.
And then I'm going to put them-- make this sort of... displacement vector go this way,
And then if I do Command + V again, it'll create them in that direction.
And I could even repeat the process and get a really big network here.
Like this.
Again, N selects neurons.
So N to select all those and--
Oh! This button randomizes them. It produces random activations.
You could also press the R key to randomize the activation.
So that's a good key sequence to know: N and then R,
To randomize the activations of all the neurons... in a network window.
Also, a quick reminder: for networks this big, you could use... trackpad gestures to zoom in.
And actually, there's something called semantic zooming, so once you zoom in past a certain scale,
The actual activations of the neurons will show up.
And there's also tooltips that give you more detail:
The neuron ID, and the... activation to I think about eight digits of precision.
This button recenters is the network.
And again, the trackpad gesture or your mouse wheel will zoom in.
And to pan, you hold down the Command or Ctrl key and left drag.
And I think I mentioned this in last video, but if you just move anything a little bit, it'll automatically rescale.
To turn that off, just go to view, auto zoom, turn that off.
And then you could zoom in and... Command + left drag... to move around.
Okay, but I'm gonna leave it on. And again, N and R to change the activations, to randomize them.
Here's another one: N and C to clear.
So those are two good key sequences: N and R to randomize, and N + C to delete, or to clear activations.
To change the properties of the neuron,
You select the neuron or neurons you are interested in, and double click on one of them.
And that will bring up the neuron dialog. You could also go to edit, and--
The edit menu, and then edit selected neurons.
You could also use Command + E.
Those are all different ways to bring up the neuron dialog
Again, it'll apply to whatever was selected. And to know how many you've selected, or many it's about to apply to,
You could use the menu. So N, edit says, "Okay, these 42 selected neurons are what you're about to edit".
And here, you see there's lots of different neuron... activation rules that have different properties.
So for example, the binary rule:
If the... activation coming in is above this threshold, it'll take this value.
Otherwise, it'll take this value.
You could also label neurons,
And... in general, they're just quite a bit you could do here, with all these different... settings
So I'll just set these all to binary,
And now you'll notice that if I do N and R, it randomizes them to the binary values,
So randomization is sensitive to the neuron type.
Okay.
And of course, depending on what you set here,
The Neuron will behave differently. But we haven't really connected them together,
So that we could observe their dynamics much. We'll do that in a second.
Labeling, often you'll want to do that per neuron, so...
It's pretty easy to do.
Okay, one last feature. So, all right, I'm gonna do N and delete to get rid of these neurons,
And show you one last way to make neurons, and that's the add neurons dialog.
You go in here, and you could set how many neurons you want, set what properties you want the neurons to have,
How you want them to be laid out. We'll talk more about layout in a future video.
Maybe do that.
And you can't put them in a group, and that's also something we'll talk about in a future video, in more detail.
So let's talk about connecting neurons together now. N, delete
Start over. And I'm just gonna press P a few times to get a few neurons going.
Now to connect neurons, there's a number of ways to do it,
And the simplest way is to use the 1, 2... keypad trick.
So, first we need to designate a set of source neurons, and then we need to designate a set of target neurons.
So we could make these the source neurons and these the target neurons.
To set the source neuron, you just select them and press the number 1,
And a red box appears around the source neurons.
The target neurons are just selected as normal.
So red is source neurons, and green is target neurons.
And to connect them you could just press the number 2.
And so you can get pretty quick at this.
Let's do it another example, so I'll make a few here,
Move them to the side,
And... 1, 2.
1, 2.
1, 2. 1, 2.
And in my experience, people get the hang of this pretty fast.
So now we have two little networks. They will propagate activation if I select this.
Now here's a new keyboard command: if you press the up key,
It will increment it, and the down key will decrement it.
Okay, so I'm gonna add some activation.
And now another keyboard command: space
Will propagate the activity.
So if I select these and press the up key a few times. Space, and it propagates the activity.
All the things I've just showed you, you could also do with...
Menu commands or toolbar commands, for example...
For example, like a double click here, set the activation manually to 1,
And then press the step button here, to... iterate the simulation.
But the key commands are much faster.
You should note that when you press the spacebar, you're only updating the network.
You won't be updating any other workspace components.
If I have another network here, and I press the spacebar here, notice that the...
Iterations are going up here, but not over here.
Whereas if you press this, it's iterating everything in the network, or in the workspace
I should also mention that when making these connections, you don't have to use the key commands.
So you can select what you want to be your source neurons, and either right click...
And go to set source neurons, or we'll go up to the edit menu and set source neurons.
Ok, then you can go here, and you can right click and say connect neurons, all to all.
This big dialog shows up, and you just press ok.
And there's lots more to discuss there, but you should know that's available.
Also, if you want to get rid of these things,
You could just go to edit, and clear source Neurons.
Or, you could just click outside anywhere, and press the 1 key again, and that's pretty fast.
So if I select that as source, or designated source, by pressing 1,
I could just click outside and press 1 again to unselect.
So the weights of your neural network can be edited in much the same way that you edit the neurons.
You select them either by clicking on them, but if you see, it's kind of hard. You could zoom in if you want.
And, you know...
Do it that way.
Hold down shift.
But an easier way is to just lasso across them, or across the lines that connect to them.
So, sort of like the axons.
All right, so you just lasso across these, and you could select them.
You could also sometimes do things like pull one out of the way and lasso the area to get a particular group.
There are some special commands in there, so for example,
You could right-click and select, and select incoming weights, for example.
Select outgoing weights on a bigger network, sometimes that's useful.
To select all the weights in a network, you can just press W, so that's good to get used to.
So N selects neurons, W selects weights.
And a few things that are common with--
 between neurons and... synapses.
So just as with neurons, you could press R to randomize.
So too with weights. Press W and press R, and that will randomize them.
And you could also use this toolbar command.
The strengths of the synapses...
Correspond to the size of these disks.
And red represents a positive weight and excitatory synapse.
And blue represents an inhibitory... synapse or negative weight.
And as you could see, the tooltip will show you the exact value.
You could also press W and C to get all the weights to zero.
All right, now no more activation will flow.
So just as you could do N, R, and C to randomize then clear, you could W, R, and C to randomize and clear.
You could also use the up and down buttons.
Just as you could increase and decrease the activation that way,
You could increase or decrease synapse strengths like this.
All right, so...
This is interesting, because it's-- these are linear neurons and negative valued weights.
The activation changes sign at each iteration, in this little chain.
Okay, you could also edit then-- the synapses in the same way you could edit the neurons.
So for neurons, you select... and then either double click,
Or edit,
Or Command + E.
With weights, you either double click,
Or... edit,
Or Command + E.
And again, you have all these things you could set, including a... local update rule.
And for each of these you'll see different settings.
Alright.
So let's do a little demo here. I'm gonna add some neurons.
I'm gonna add nine of them, in a grid, and I'm gonna make these Naka-Rushton neurons.
Actually, just straight decay neurons. How about that?
Let's set them to decay slowly. So the decay fraction I'm setting at 0.01. They will lose...
.01 of their activation at each update, and I'm gonna set the upper... bound to 10,
And the lower bound to negative 10 on these.
So we can confirm that we have these decay neurons with a upper bound of 10 and a lower bound of negative 10.
And now I'm gonna fully interconnect these. And to do that, I'm gonna press N to select all the neurons,
1 to designate them all as source neurons,
And then I just do nothing else, because all the neurons are now source... neurons, as indicated by the red,
And target neurons is indicated by the green.
So I just press 2 and they all connect.
And I could press 1, sort of unselect-- click outside of the network,
And click and press 1 to take away the source indicators.
I know that was very fast, so W, delete, and then,
N, 1, 2, to fully interconnect
So you can make recurrent networks very fast in this way.
Okay,
Now, let's check out some dynamics.
So W, or some dynamics on the weights this time, so W and C to clear them,
And then Command + E to edit them, and I'm gonna designate these as hebbian,
And maybe I'll give them a slightly slower learning rate of 0.01
So now if I do N and R to randomize the neurons,
Notice I randomize between negative 10 and positive 10, because of the upper bound I set on them.
And now if I just press run,
We could see that the weights slowly change in accordance with the hebbian update rule.
Okay, so that's it for just real simple...
Editing and building neural networks. They're very fast to connect together using these various... commands.
You know, you could very quickly create all kinds of networks...
With all kinds of properties. Now this is called...
We call these loose neurons and loose synapses
But we also have provisions in Simbrain for dealing with much larger networks.
If you make really large simulations using these kinds of loose elements, then pretty soon it gets pretty ugly.
For example, if I added 80 neurons and fully interconnected those with the 1, 2 trick,
You could see it's-- really, you're not seeing much. There's a little bit of lag now
It's just not an ideal way to do things,
So we've done-- let me get rid of this, so I'm gonna click A, to select all, and delete.
And even-- you're gonna see it deletes kind of slow now, because there's just a lot of stuff in there.
And so we now have, with Simbrain 3.0,
And I have to credit Zach Tosi with being the main person to help design this,
So we had a long sort of... process of... working out this interface,
Is neuron groups and synapse groups.
So a neuron group-- to create a neuron group, you just go to insert,
Insert neuron group,
And you could do bare neuron group. And we could make this pretty big, say 100.
And so now we've got this higher level of structure, a whole group of neurons.
It's got various performance features, and they interact-- in the place you interact with it is with this little...
Tab here. We call this an interaction box. So you could double click on that
You could also drag and, just as a reference, let me put a loose neuron down.
So you could drag this guy around.
And now these kind of become the basic units, right? You can copy and paste these.
And now we can connect these with something called a synapse group,
And we do it using the same 1, 2 kind of framework. There's still key commands,
But it's easiest to just do the 1, 2 trick, so I select this and I press 1 we get this...
Red selection box around the interaction box of this neuron group.
And now we click this one to designate it as the target.
So the one with a red... box around is is the source neuron group,
The one that is selected in green is the neuron,
The target neuron group, and so this is just as the same as with loose neurons.
And now if we press 2, a dialog will come up asking us what kind of synapse group we want to create.
And there's lots of stuff in here, which again, we'll talk about at a later date.
I click ok,
And... a synapse group is created. So all of the individual connections--
These are now actually fully interconnected, and all of the individual connections are hidden.
But if I double click on the synapse group-- we can get rid of this now,
We see various properties of it.
For example, we could see the distribution of weight strengths. They're all of strength 1.
And we could, you know, if I want to make it 50/50,
Half excitatory, half inhibitory, I can get kind of fancy with a kind of...
Distribution I want on these.
Okay, I click apply there.
Apply, apply.
And so now we have a, you know, more interesting distribution of synaptic strengths.
So I've now set those, and if I double click it should reflect the changes.
All right, so let's watch some activation flow through this.
So to do that, I'm gonna change these to all be...
Let's try Naka-Rushton neurons this time.
I'm gonna go into this wand mode, and I'm gonna kind of paint some activation on there,
And I'll go back to regular mode.
And then I'm just gonna press the spacebar,
And you can see...
The activation has flowed through, and as it decays, it decays here as well.
You could make very large simulations this way.
It's sort of going up to the next level in the complexity of a neural network simulation.
Right, so I could delete this, delete this.
Now the last thing I want to show you is that,
In addition to sort of handcrafting neural networks in this way, there are a lot of built-in...
Network types, and group types.
So there's different kinds of neuron groups you could put in, and different kinds of...
Sort of sub networks you could put in.
For example, a backprop network.
you could also right click where you want one of these, and add one.
A competitive network.
Ooh, that's, I think, a bug. It shouldn't have shown up that far away.
What else...
A self-organizing map.
Okay, and so these all have special properties.
We could have networks within networks, or groups within networks.
Let me just delete these, just to give you a sense of it.
So for a backprop network, it's got multiple layers that you could separately move around and adjust.
You could double-click on any of the layers,
and maybe you want this to have a...
Grid layout, for example.
Didn't look that nice.
You could adjust the connections by clicking on those.
And of course, you could adjust the backprop network as a whole. This is where you would train it,
And again, this is all material for a future video.
So, using these tools, you could build very complicated networks very fast.
You can connect the loose neurons to these...
Sub networks.
You can connect the output of a sub network to loose neurons, or to other sub networks.
And, I mean, this isn't much of a network we've got here, but it does illustrate... that...
You could very quickly build very complicated networks in Simbrain.
If you've got any questions, post them in comments. I'm happy to answer.
I hope you enjoyed it!
