Welcome to the ZKM Kubus.
My name is Gero Koenig.  
We present to you today »Chordeograph Augmented Reality«.
I would like to thank the Hertz Lab.
I would like to thank the Hertz Lab.
Ludger Brümmer,
Ludger Brümmer,
Dorte Becker and Sophie Hesse, who did the press work,
Götz Dipper, Yannick Hofmann 
and Hans Gass, who set up the light for tonight,
and all the other ZKM employees who made this evening possible.
I warmly welcome Clarence Barlow,
who came here for the presentation this evening
and Gottfried Düren from the Artoxin Gallery in Munich,
who supports this project.
Tonight we will present three experimental setups.
The first will be a solo performance
followed by an interactive performance,
where you are cordially invited to participate.
After a small break we present
the concertante analogue version of Chordeograph.
Afterwards there will be a talk between
Clarence Barlow, Gottfried Düren and me.
You are welcome to ask questions and participate in a discussion.
I start with a small introduction to »Chordeograph Augmented Reality«.
How can you make a concert interactive or even participative?
Based on this question I would like
to share the results of my work with you tonight.
An important goal of the development was
to achieve the quality of an acoustic instrumental sound.
You see here an instrument with copper-wound steel strings 
and a spruce wood soundboard typical for piano construction.  
The score is a user interface placed as
transparent video directly on the strings.
By movements of your arms and hands
you can navigate in the instrumental sound space.
The progression possibilities are displayed in the form of nodal points. 
So it is not like in the traditional concert setting
where it is necessary to sit as still and quiet as possible.
Something completely different happens here:
You have the possibility to navigate in a sound space.
Another goal of the development is to augment
the feedback between composer, performer and listener.
That means the possibility to have different interactions.
Another point of this work is that a static tempered sound system,
as given by the piano, has been overcome.
That means the deconstruction of a culture
that is relevant for us as listeners,
namely, I call it the black and white culture
that we all have internalized more or less deeply.
To show you how the interaction works, we look at an example first.
At first there are two dimensions of navigation.
One is the orientation from right to left,
as you navigate between different strings, 
so you choose which strings can be agitated to sound.
The navigation direction from top to bottom determines the pitch.
A further dimension of navigation is that from front to back.
I'll just handle it.
The range was still active when you stopped.
You have to wait until the hands are down.
Exactly this shows us already an important principle of navigation.
When the hands are down the navigation ranges go down
and then practically the triggering of tonal events is finished.
If you raise only one hand,
you can trigger all nodes that appear in the score by this hand.
At the same time you let the other arm hang loose.
If you raise both hands,
the tonal nodes are split up between right and left hand.
Then there is a directional orientation
through triangles visualized in the score.
If the triangle points to the right,
the tonal node can be trigered by the right hand
and vice versa for the left hand.
A further dimension of navigation is that from front to back.
I would like to explain this with a second example
where this dimension is used to create different
beat rhythms between strings.
Another approach of this navigation is
to be able to continuously modulate
between very different states of sound.
between very different states of sound.
From simple resonances up to constellations with maximum tonal complexity.
From simple resonances up to constellations
with maximum tonal complexity.
This is followed by two more examples.
Furthermore you can see something about the timing here.
The timing follows in principle the idea,
that the change between different sound constellations
The timing follows in principle the idea,
that the change between different sound constellations
takes place in a similar way as birds switch
the location in the context of forage.
takes place in a similar way as birds switch
the location in the context of forage.
Studies have shown, if the time interval between
the finding of two grains exceeds a certain time threshold,
the birds fly away even if further grains could be found.
This means, a change between sound constellations takes place
when a certain time interval is exceeded between
the triggering of two sound events in the score.
At the same time it is determined for the interactive progression,
how many sound events at least have to be triggered.
So these two criteria together make the dynamics
of the progression which we will see here.
So first an example of simple tonal relationships.
We can conclude this little introduction herewith
and start a performance on this basis.
