Hey it's Andrew Huang! there are some people who believe that the development of artificial
Intelligence and machine learning will lead to humanity's demise which maybe so maybe no, I'm asking the same question
I'm always asking
Which is: will it music? this video is a collaboration with a team at Google called magenta whose focus is on exploring
How we can create art and music with machine learning
They got in touch a little while ago and showed me a bunch of things they were working on and asked if there was anything
I would want to do with it, and I was really drawn to a project of theirs called N-SYNTH, which stands for neural synthesizer
It's super exciting to me because it's an entirely new form of synthesis rather than manipulating sounds like most
synthesizers N-SYNTH manipulates data
it analyzes audio files learns how all that data relates to itself and then
Produces new audio files from scratch and this opens up some interesting
Possibilities so for instance if we have this vibraphone sound and this guitar sound
We can ask and synth to generate a new sound that is 50% of each
It's interpolating between the raw data of both sounds at the level of individual samples and when I say samples
I don't mean like this is a drum sample or this is a flute sample I
Mean like the building blocks of digital audio
You know how a video is made up of different frames for instance these are all the frames
You just saw of me saying the word frames in this video
You're seeing 24 frames per second, and it makes for a pretty smooth recreation of all of my movement
With audio, it's similar
You need all these tiny little
Individual samples of sound that get strung together to make what you hear except with audio you need a little bit more right now
You're hearing
48,000 samples per second our ears just need that much more resolution to perceive digital audio with a quality that's close to real life and
Synth contextually generates one sample at a time based on the previous few thousand samples as well as all the audio
It's been trained on and n synth sounds a little grittier because they opted for a lower sample rate of
16,000 samples per second because that is still a ton of sample
To generate so you can play with n synth right in your browser
I'll put a link in the description since I had access to magentas resources though
I wanted to do some experimenting the incent algorithm was trained on pitched material. It was trained on notes
300,000 notes from a thousand different instruments. I like to mess with stuff so when the folks at Google asked
If there was anything I'd like to do I said two things what if we feed it?
Percussion and what if we feed it a bunch of completely random sounds that people send me on Twitter
And that's what we did I sent in about a hundred drum sounds from my own sample packs
And I also got about a hundred submissions from people on Twitter, and I didn't give people any guidelines
We ended up with sounds from instruments household objects voices animals
The magenta team crossed every combination of two drum sounds and every combination of two random Twitter sounds leading to just shy of
9,000 new sounds and yes there will be a sample back
The algorithm did a really good job with the drums. I just got a whole bunch of variations on drum sounds
Where things really got interesting is when we combined two sounds that were completely different from each other? Here's a Stylophone?
And a 3d printer. This is what they sound like crossed
What if we take this baby goat and?
combine that with that 3d printer
Oops just summoned Satan here's one more example. We've got a fountain
Frost with a string scrape
This combination is really interesting
Doesn't that just sound like an electric guitar this type of sound happened fairly often
And I'm wondering if it's a byproduct of ensign having to deal with much more complex sounds than single notes dealing with a lot more
Frequencies at once but still trying to interpret that in a pitched way because it was trained on pitched material
Maybe I'm gonna end this video by sharing a piece of music
I made using only these sounds that ends synth generated the drum ones and the Twitter ones
I started by just trying to listen to them all and I think I got through almost a
Thousand before giving up on that idea so I made some selects
I organized them in Ableton based on how they sound and then I made this track basically by just
Moving them around putting them in different orders stacking them in different combinations
listening the whole time
Seeing what worked and how things flowed and I ended up using somewhere between two and three hundred of these
Sounds in the final piece if you're interested
I talk more about my process in a video on natin friends channel
Which I'll link to I named this track rainbow gram
Which is what they call all these cool-looking sound visualizations on the N synth website
That's a whole other tangent that we're not going to take right now. Here's the music
