Performance Publica
Iteration 1 - Hemisphere
Format for Performance
Presented during the Hemisphere residency of the House of Music Hungary
WHAT:
Performance Publica is a type/format for performances where the audience becomes the performers, which means it is about and from the audience. It is established through an auto-generative-dynamic-feedback system that constantly co-creates with the audience. The common image of people and space and the sounds they emit are changing the system, which itself expands them on its own as well. Through its technical nature, it enhances the beholder's capabilities to create together with space, light, and sound.
The idea is to have a leveling, approaching co-creation of each part involved, the beholders in the audience, the space, and the system itself. The system is not just a tool as of its autonomous parts, and the audience is not directed by anything at the same time.
As much as it is a format, it is also open-source research into this new format itself. Everyone is wanted and welcomed to contribute, during and after the performances, to expand the techniques and content for the goal of making art that is about you, the beholders.
WHERE:
​
The first iteration of Performance Publica was presented in Budapest at the House of Music Hungary inside the Sound Dome. It was shown there as part of the Hemispheres Budapest/Berlin Project. The Project was a residency of the House of Music Budapest which commissioned pieces for their spatial audiovisual system (The Sound Dome) to be presented there every Tuesday for a year. Further, the Spatial Sound Syndicate facilitated accommodation and also the possibility to engage with their own spatial sound system, which is based on 4Dsound technology. It premiered on the evening of 28.03.23, together with nine other pieces from students and professors of the UDK SoundS and the Liszt Academy Electro-acoustic-Music Master programs.
On this note, special thanks go out to Orhun Caglidil, Saeed Alizadeh, Mariano Rosales, Jan Thoben, Boris Hegenbart, Vazul Mandli, Vito Villems, Alois Jang, Yuna Lee Pfau, Ákos Lovász,
Ádám Kalamár, Dániel Németh, Péter Tóth
and the team of the House of Music Hunagry for making it happen and the abundance of support and valuable feedback.
HOW:
​
Performance Publica is realized in a space with the audience performing with a system. The system called "Publica" is based on Max/Msp language and combines video grabbing, a game of life circuit that uses a logic equation developed by Orhun Caglidil, matrix analysis, column scanning, dependent randomness generators, a clocking system with two different cycles, pixel change detection, a circular buffer of varying length, dynamic volume fading and limiting, spectrogram to 3D position data, Ircam's Spatialiser, a harmonic delay using pixel change as chordal data and many more different data processing circuits.
This results in a dependency of the system and the audience; still, if one decides not to move, they create an input of position data which basically means it's impossible not to perform in front of the Publica system. Very broad outcomes are possible, and the limit in these mostly lies in the performers will to collaborate.
WHY:
​
The research began in summer of 2022 with the artist statement:
“Art is not about the entity created by artists but first the reactive process of each individual who can perceive the artwork and finally the combined processes of each of these observers.” Ruben Bass
Or elaborately:
“Art is about the collective added metaphysical value to an entity by its beholders. The processes had by and with the entity inside the beholders are at some point greater than what the artists could ever initially imagine; as the collective thoughts, reactions, associations, felt emotions, and memories re- experienced, at some point become greater in amount and in each of the criteria’s factors than the creation process of the entity and the entity
itself.” Ruben Bass
As a piece of art is observed by many individuals, the value and reason why it would be art become far greater than any “genius-individual” could ever imagine, as the collective experience will always be more significant in volume and factors than the individual one.
If the personally added metaphysical value is seen as the main factor and core difference between art to other fields, it becomes of utmost strategical relevance. To create metaphysical value, as many personal subjective processes as possible must be enabled. Therefore, the main objective becomes to create as much openness for personal experience as possible.
It could be argued that the most relevant factor of art is the process of the beholders / “Betrachter” rather than the product of the artist. With this perspective on art, a need for a different form can be derived. A form that enables the beholders to take an immediate part in the very core process and thus the identity of art itself and make this process more accessible. A moment where you are in dialogues with the processes itself and able to impact/change what you experience.
Suppose you are interested in a deeper conceptual and theoretical exploration of Performance Publica. In that case, you can read the Paper on it here:
Move as you wish -
double click to enlarge
Move as you wish -
double click to enlarge
THE FOUR MAIN SECTIONS OF THE SYSTEM:
The visual part receiving the camera input analyses the image regarding changes happening and sends this change to other sections, and finally transforms the image depending on changing rules.
The audio part receives the microphone input of what is happening in the space, recording it by overdubbing it into a circular buffer, which loops the recorded material. Further, different circuits apply pitch shifting and delay to it, and the sound is spatialized in a 30by30 meter virtual space.
The clocking system has two cycles: recording and generation. These cycles change the behavior of all other parts. The recording cycle makes the audience able to see themselves while being able to record input into the system. The generation cycle is when the system generates with the rules determined in the Conway section and expands the image and sound with its own generation. The length of both cycles varies according to movements and sounds produced in the space.
The data processors/scalers are not a coherent system but rather many more small circuits with essential functions like cleaning the data or randomness generation depending on the audience. There are too many different ones to list here.
THE FUTURE OF THE PROJECT
The first iteration of Performance Publica during the Hemispheres residency is, of course, only one of the endless possibilities to approach the idea of the format.
Everyone is invited to contribute and practice Performance Publicas. Therefore, the performancepublica.com platform is in the making, which shall aid as a future hub for exchanging, explaining, and downloading different iterations for everyone to engage with.
Move as you wish -
double click to enlarge
THE SIGNAL FLOW:
The camera filming everything is mirrored (because of image warping so that all surface of the dome shows the audience moving in the space) and is sent to the Conway logic circuit, which then determines through changing rules if a pixel will continue to remain on screen or if it will turn to black (0), resulting in broadly varying generative patterns being formed. The random generators which determine the numbers of the Conway logic equation are impacted by the sound and image present in the room at the time. This collective image of the audience, space, and system is then analyzed for at which position the changes are occurring. These positions are then formatted around the +-XYZ axes and sent to Ircams Spat which simulates a virtual acoustic space which has speakers placed exactly the same way as the real Sound Dome in the House of Music. This makes it possible to move the sound through space inside the Sound Dome by the audience and the system simultaneously in co-creation.
At the same time, the microphone records into a circular buffer loop which has varying lengths depending on the number of generation cycles already completed by the system. In this way, the audience can layer sounds further and further, increasing length until the buffer is cleared so that the constant recording of the space does not turn into extremely crowded sounds. So new ideas can be developed with the system together. The circular buffer
During this process, many other circuits analyze the change of sound and image and generate dynamic time for the clocking circuit, seeds for the randomness generators, chordal information for the harmonic delay, spectral analysis, volume limiting, dynamic volume fading… Most of the circuits are connected to at least two other circuits, so there is a lot of codependency already in the system itself.
As mentioned, there are two cycles of the system determined by the clocking circuit. The recording cycle is the time when audio and video pass into the buffer and into the Conway logic circuit. When the recording cycle ends, the generation cycle starts, the Conway part generates new changes to be seen on the screens, and the harmonic delay generates different pitch allocations to be heard, while Spat changes the positions of the sounds heard dynamically according to changes seen on the screens as well.
Explore the Max patch on the left and move freely.
THE PROCESS:
A rough out(time)line of the 600+hours process of research and experiments made.
​
RESEARCH PHASE:
​
The main initial research question was how to turn the audience into performers as it is them who are the most significant factor in art. This research question was broadly elaborated on in the paper about Performance Publica, and it fuelled the process in every part to come.
As of the perspective that the process of beholders is truly what art is in the end, In its most ideal future version, the system would be a true beholder itself. To approach this technical level which would require not yet developed fast learning AI, experiments with autonomy are made for now. This meant using some form of auto-generative system, which means that even from a very tiny input, a large complex outcome is expected and possible. The research (and code experiments) led to the exploration of the thoughts of Heinz von Förster, Stephen Wolfram, and John Conway about cellular automata. Especially John Conway’s Game of Life and the possibility to take an image of the audience as input which then would be extended by the system, offered autonomy and the ability to create for the envisioned system itself.
PRACTICAL PHASE:
During the process, many things have been tried and considered (mostly) in Max/MSP. Initial concepts involved typical generative methods such as random generators, impulse expansion, and shift registers.
Further down the process, an application called ZigSim which can stream all data from all sensors of a smartphone to Max, was thoroughly explored. It was mostly used in conjunction with the generative methods explored before and would control their parameters. After some time, it would turn out that such an approach would not work with many people due to network issues and further problems, such as delivering the app to everyone without lots of delay to a performance.
Exchange with Orhun and others led towards further elaboration on the term of the collective image. In a way, technology could actually surpass and extend human perception in creating a collective image of the space and the people. The system would be capable of detecting changes of everyone moving together in one image stream. This opened up the possibility to make the space performable in moving sound through it by using the data of changing pixels - movement -. John Conway’s use of cellular automata in his Game of Life, which takes a set of pixels and then generates and expands on this initial collective image by rules, served as a ground idea. If the system itself were creating these rules dynamically, it would have its own autonomy and a way to move as well, resulting in the system being a co-creator rather than just being a mere tool. This core idea was then expanded and elaborated upon through 25+ different iterations, which over time, implemented the parts described above in “the signal flow”.
ADAPTION PHASE:
When the artists arrived in Budapest and got within contact and context of the 4d sound system of the SSS and the Sound Dome system of the House of Music, the question immediately became how to make the most of the space. This meant how to create maximum performability of the space. Initial Iterations of Publica were extended by considering the input situation, which involved the possibility of grabbing the image of the space from the security cameras of the Sound Dome. As of the nature of the Dome structure and how it distorts images to wrap them into a sphere and as of having the camera in one position over the door, it became inherent that the image would have to be mirrored so that the sphere above the performers-audience could see their movements mirrored and expanded in every corner.
Further experiments at the SSS with their native non-linear archive, called Holos, were conducted. The experiments revealed possibilities of two feedback systems playing with each other and might have been the first case of such ever happening (in a spatial sound system). Surprisingly, Holos is in many ways similar to Publica, as it is also recording into a buffer that then gets fed back into the room, recalling events that happened minutes and sometimes longer ago. At this point, it should be noted that Publica was created to make faster dynamic changes to make the impact of the performers felt immediately. In this way, the two systems covered the short- and long-term reciprocation of happenings in the room, which made for an interesting couple. As the computer which hosts Publica was left alone in the space with Holos overnight, the two systems started to feed into each other and created an atmosphere that woke everybody up in the morning.