Zangezi is a Russian Cubo-Futurist poem/play written by the poet Velimir Khlebnikov 101 years ago in 1922. The story revolves around Zangezi, a prophet who speaks in the language ZAUM, a Russian word that is translated as “beyondsense.” Zangezi speaks with and can understand the birds, the gods, the stars and speaks in those languages as well as in poetic and also what K. called “ordinary” language. The Immersive Arts Space is working on this text because as a futurist work, Zangezi deals with is a fundamental human question which is increasingly becoming a problem for machines – what is the basis of language? Is language only about meaning based on syntax? Is it about predictable sequences? Rules and probabilities? Or is there something else going on that is universal and cosmic about language not as words and their meanings but as an act of sound?
Indeed, Khlebnikov, who had a mystical belief in the power of words, thought that the connections between sounds and meaning were lost during mankind’s history, and it was up to those in the future to rediscover them. 101 years later we ask how we might be able to approach Zangezi’s operations on multiple levels – cosmological, political, historical, technological – in a moment where we are increasingly surrounded by machines that produce something that appears like human language but in which there is no speaker, no body and no sound.
Zangezi-Experiments, a multimedia stage play, will be performed at the MESH Festival. Before, fragments of the piece were recreated as a work in progress for the REFRESH#5 2023 festival, that took the form of a theatrically staged reading within the technical machine of the Immersive Arts Space itself.
Credits: Chris Salter (Direction, Sound Design) Corinne Soland (performer) Luana Volet (performer) Antonia Meier (performer) Stella Speziali (Development Meta Human) Valentin Huber (Visual Design/Unreal Engine) Pascal Lund-Jensen (Sound Design), formerly Eric Larrieux Sébastien Schiesser (Light Design, show control) Ania Nova (Russian voice over)
reconFIGURE is a real time installation exploring how diverse human bodies are reimagined by computing machines. The visitors’ frontal image is first captured by a camera, and using machine learning, transformed from a 2D image into an animated 3D avatar in a matter of seconds. Appearing in the projection on the wall and an unsettling sonic atmosphere, these doppelgaengers gradually seem to move like strange creatures in an aquarium, where hidden forces throw them around like rag dolls. As the visitors’ encounter their re-animated selves on the screens, they bear witness to the mutations generated by the machine processes. Thus, through the transformation of bodies by way of increasingly black boxed AI systems, reconFIGURE probes how we will negotiate the truth between our own image and that image as reimagined by machines.
reconFIGURE probes how computational systems capture, represent and transform human bodies [Hansen 20121; Hayles 20002]. In art theory and criticism, this theme has long been examined through concepts such as “technical exteriorisation” [Wegenstein 2010] and the “cyborg” [Haraway 20133]; “data bodies” [Salazar 20234]; and (3) the “undigital image” where a disconnect is felt between a subject and their body through computational image making [Zylinska 20205]. At the same time, the tension between a body and its captured other marks reconFIGURE as an encounter with digital “doppelgängers.” A term originally used by 18th century German writer Jean Paul, the doppelgänger is a ghostly counterpart for a living person, seen as an omen or sign of death
Aesthetically, reconFIGURE aims to highlight exactely this uncanny situation – the unsettling feeling that feels oddly familiar [Freud 2017]6 when confronting a digital other. To achieve this, we structure the experience in two parts: First, capture and transformation of the visitors’ body image and second, the reanimation of the image in an audio-visual environment that evokes the sense of bodies suspended in outer space or a liminal zone between life and death. This confrontation with one’s doppelgänger has an intimate quality yet, the doppelgänger quickly disappears from the display, reappearing in a larger projection in the room.
reconFIGRUE has already been part of the following exhibitions, conferences and festivals:
Credits: Chris Elvis Leisi: Experience Animation Florian Bruggisser: Volumetric Capturing, Machine Learning Pascal Lund-Jensen: Sound Design Martin Fröhlich: Scenography Chris Salter: Project lead Kristina Jungic: Exhibition Production
Mark N.B. Hansen (2012). Bodies in Code: Interfaces with digital media. London: Routledge. ↩︎
N. Katherine Hayles (1997). How We Became Posthuman. Chicago: University of Chicago Press ↩︎
Donna Haraway (2013). Simians, cyborgs, and women: The reinvention of nature. London: Routledge ↩︎
Constanza Salazar (2023). “Challenging the “Data Body” in New Media Art, 1990s–Present.” Afterimage, Vol. 50, Number 2, 93–111. ↩︎
Joanna Zylinska (2020). AI art: machine visions and warped dreams. London: Open Humanities Press. ↩︎
Sigmund Freud (2017). “The Uncanny.” In Romantic Writings. London: Routledge: 318-325. ↩︎
Changing Matters is a project that explores the concept of a playful and abstract virtual representation of oneself. This is achieved through the use of a camera that tracks the movements of the user and translates them onto an avatar. The virtual environment is further enhanced with sound that complements the user’s movements and empowers them with control. Therefore, experience encourages you to experiment with the possibilities of the virtual body’s interaction with the immersive sound and landscape.
The project utilizes SARMotion, a software developed by Florian Bruggisser, along with Unity data that is processed using Max/MSP and Ableton Live for sound generation.
Changing Matters is a graduation project by Lorenz Kleiser (BA Game Design) and Floris Demandt (MA Composition and Theory).
The project will be showcased and can be experienced at the upcoming conference REFRESH#5.
The LabInsights is a recurring event hosted by and in the Immersive Arts Space. At least once a year we open the doors to visitors and participants and showcase the current projects being developed in the lab. The informal event aims to demonstrate the state of the projects and at the same time involve students, artists and visitors to test the expereiences and give feedback.
Upcoming LabInsights will be posted on the website and our social media channels.
The next LabInsights will take place on April 4, 2024. This year visitors are invited to experience XRconFIGURE with Virtual Reality Googles and learn more about our latest work-in-progress projects and research such as Puppets, Probing XR’s Futures and Realtime Stable Diffusion. In an informal setting and with an aperitif it is a great opportunity to meet and exchange about current and new developments taking place in the Immersive Arts Space.
At the LabInsights, hosted on 4th may 2023, the following projects were presented: reconFIGURE (Chris Elvis Leisi, Florian Bruggisser, Chris Salter), Changing Matters (Lorenz Kleiser, Floris Demandt), Digital Gold VR experience (Chris Elvis Leisi) and The Feeling Machine (Manuel Hendry, Norbert Kottmann).
On June 29th and 30th 2022, the Immersive Arts Space presented new developments from current projects. Visitors were invited to experience our project Shifting Realities (Florian Bruggisser, Martin Fröhlich, Valentin Huber, Norbert Kottmann, Eric Larrieux, Chris Elvis Leisi, Oliver Sahli, Stella Speziali) with or without Virtual Reality goggles and to learn more about Digital Gold (Chris Elvis Leisi, Florian Bruggisser, Kristina Jungic, Christian Iseli) in our smartphones.
During REFRESH #4, from 10th to 11th November 2021, team members offered insights into the projects Shifting Realities (Oliver Sahli, Martin Fröhlich, Eric Larrieux and many more), Neural Volumetric Capture (Florian Bruggisser), Digital Twins (Stella Speziali) and cineDesk (Norbert Kottmann, Valentin Huber).
The Immersive Arts Lecture Series for Spring semester 2023 investigates artistic and design experiences in which humans, media and architectural spaces shape one another within the context of the built environment. Guest lecturers who come from the fields of digital arts, architecture, theater studies and the visual arts, will focus on specific artistic works from their practice, as way to open up critical discussion around emerging paradigms of bodily experience in spatial computing environments, human-machine interactions in architectural spaces, and immersive experiences between human, technical and natural systems.
Atmospheric Forest visualizes the complex relations between the forest, climate change and atmosphere. Trees are not only oxygen generators, but they breathe as well, and emit large amounts of volatile organic compounds into the air – we recognize it as a habitual scent of the forest. Scientists have long known about the link between a fragrant forest and the warming climate, but are uncertain about the impact and scale. Atmospheric Forest VR artwork is the outcome of a three-year artistic research project on Pfynwald, an ancient Swiss Alpine coniferous forest suffering from drought due to climate change; the scientists from WSL Research Institute have turned this forest into a “living observatory”.To create the Atmospheric Forest VR artwork, the artists scanned the Pfynwald creating a virtual point cloud environment, and visualized the data provided by scientists. The data sets collected during one growing season included measurements of volatile emissions, resin pressure in pine tree trunks and changing weather. The artists transformed these data into animated particle flows, revealing the complex interaction between the forest ecosystem and atmospheric processes.
As part of the Lecture Series, Atmospheric Forest will be presented on the evening of 08.03.23 in the Immersive Arts Space.
Rasa Smite is an artist, network researcher and cultural innovator, working with science and emerging technologies since 90s. She is founding director of RIXC Center for New Media Culture in Riga, curator of its annual festivals, and a chief-editor of Acoustic Space – peer-reviewed publication series. She holds a PhD in sociology of culture and media (from Riga Stradins University, 2011), and MA in visual arts (from Arts Academy of Latvia, 2000). Currently she works as associate professor in New Media Art programme at Liepaja University. She is author of the books – “Creative Networks. In the Rear-View Mirror of Eastern European History”, published by Amsterdam Institute for Network Culture (2012), and “Talk to Me. Exploring Human-Plant Communication” (Published by RIXC, 2014). She is also author of numerous articles and co-editor of the Acoustic Space series issues.
ON ANIMATE
Animate is an Extended Reality-based theater work (XR) at the crossroads of performance, radio play and installation focused on a near-future Canada radically transformed by climate change. Developed by a team of Canadian and German artists and researchers, the work premiered in Summer 2022 in large theater festivals in Germany and is preparing for a 2023 international tour in Europe, Canada and Asia. The story “Animate” focuses on two characters, Daniel and Laurie, who are fleeing a near future of climate disasters. As part of its dramaturgical strategy, Animate harnesses a recent XR technology to explore this potential mixing of real and the simulated in worn AR: live video “passthrough” which takes a real time video feed from the tiny cameras attached to a head mounted display (HMD) and which allows the embedding of computer-generated 3D objects into the video feed of the real environment. Animate aims to use emerging XR technologies to explore the thematic of climate change through the sensorial-aesthetic possibilities of “spatial computing” in which “human interaction with a machine in which the machine retains and manipulates referents to real objects and spaces” shifts older understandings of the virtual as abstracted from the lived, moving body.
Christopher Salter is an artist, Professor for Immersive Arts and Director of the Immersive Arts Space at the Zurich University of the Arts (ZHdK). He is also Professor Emeritus, Design and Computation Arts at Concordia University in Montreal, former Co-Director of the Hexagram network for research-creation in arts, cultures and technology and Co-Founder of the Milieux Institute at Concordia. His artistic work has been seen all over the world at such venues as the Venice Architecture Biennale, Barbican Centre, Berliner Festspiele, Wiener Festwochen, ZKM, Kunstfest Weimar, Musée d’art Contemporain, Muffathalle, EXIT Festival and Place des Arts-Montreal, among many others. He is the author of Entangled: Technology and the Transformation of Performance ( 2010), Alien Agency: Experimental Encounters with Art in the Making (2015) and Sensing Machines (2022), all from MIT Press.
ON DEGREES OF LIFE
Degrees of Life is a responsive environment exhibited in February 2022 at ZentrumFokusForschung in Vienna. The project explores the interaction between humans and living systems at an architectural scale, developing interactive environments within an architectural space that learn, grow, and decay in relation to human presence and behavior. The space reflects on the concept of biomediality and biofacts, the possible applications of living technologies, and human sensory interfaces in architecture. Degrees of Life is the result of a larger artistic research context called Co-corporeality that weaves together architectural design, sensor systems, machine learning, and microbiology. The exhibition pursues the idea of interactive architecture as a living system, in which physical presence and new modes of observation are intertwined with tangible forms of computation.
Daniela Mitterberger received her architectural master degree with honors at the Academy of Fine Arts Vienna. Within her studies in Vienna, at the Hong Kong University (HKU) and the Royal Danish Academy of Fine Arts in Copenhagen she developed a deep seated interest in the topics of bodies in architecture, human body as architecture and the definition of technology and tools within the field of architecture. Daniela’s work ranges from digital experiments to analog production, combining the fields of media culture, philosophy and narration with architecture. Currently, she is a Ph.D. researcher and A&T Ph.D. Fellow at the Chair of Architecture and Digital Fabrication (Prof. Fabio Gramazio, Prof. Matthias Kohler) focusing on intuition in digital design and robotic fabrication.
A DATA-DRIVEN DRAMA
While progressively churning all aspects of life into data, the digital escapes conventional material methods of observation and measurement. Digital materiality has been addressed by numerous media, communication and infrastructure studies, often focusing on a single effect like their carbon footprint, as the principal measure of anthropogenic pollution in the ongoing climate crisis. My research is concerned with expanding the perspective on digital materiality. Such a perspective should be able to include unlike dimensions in a perfromance: the design of graphical interfaces, internet users labour, their emotional states, purchase decisions, rare mineral mining and the labour of assembling electronic devices, automated collection of user data, financial transactions within AdTech markets, energy consumption and carbon-footprint of these processes. In this talk, I will focus on reading digital materiality through the operations of internet-based platforms and their immersive grip on users. In particular, I will propose performance-based methods to experience these operations through staging. The convergence of space, time, networks and bodies in the performative medium relates and renders experienceable the otherwise incommunicable material entanglements. If we are, as Chun suggested, characters in a ‘big data’ drama, then exposing the drama is to expose the roles and scripts that are in place, enabling them to be questioned and possibly undone.
Selena Savić is a researcher and trained architect. Her research interests revolve around the mixture of computational processes with the built environment, exploring ways to communicate communication processes. After her PhD at EPFL and a postdoc at ATTP, TU Vienna, she joined the IXDM at the FHNW Academy for Art and Design where she is currently Head of the Make/Sense PhD programme. She edited two books (Ghosts of Transparency, 2019 and Unpleasant Design, 2013) and writes about computational modeling, feminist hacking, and post human networks in the context of design and architecture.
AMBIVALENT PRACTICE OF IMMERSIVE INSTALLATION AND PERFORMANCE
Since the early nineties, Kurt Hentschlager creates media installations and live shows, most always audiovisual and immersive in nature. The ever-deepening effects of digital technology on individual and collective consciousness, and a subsequent reassessment of concepts of nature and artifice are among the subject matters that inform his practice. His shows have characteristically been visceral and challenging, with extreme perceptual effects, composed from projections, light, sound and fog. These works physiologically and emotionally affect the viewer’s experience. His approach is interdisciplinary and experimental fusing both analogue and digital media. His artistic process is rooted in an early and continuing focus on human interaction with architectural space, light and sound, loosely within the concept of the “Gesamtkunstwerk”. A recurring topic in his more recent works such as as Sub, FeedX, Zee and others is the human body with its perceptual brain “apparatus”, especially the way that the brain processes the world, colored always by cultural background, habits, imagination, subjectivity and individual psychology.
New York-based Austrian artist Kurt Hentschläger creates audiovisual performances and installations. He began to exhibit his work in 1983, and since has been working predominantly with time-based media, light and sound. The immersive nature of his work reflects on the metaphor of the sublime and the human condition in the 21st century. His current work continues to investigate human perception and the impact of new technologies on both individual and collective consciousness. Between 1992 and 2003 he worked collaboratively as the artist duo “Granular-Synthesis”. Selected presentations include the Venice Biennial, the Venice Theater Biennial, Stedelijk Museum Amsterdam, Fondation Beyeler Basel, MAC – Musée d’Art Contemporain Montreal, MAK – Museum of Applied Arts Vienna, ICC Tokyo, Arte Alameda Mexico City, MONA – Museum of Old and New Art, Hobart, Tasmania.
This lecture series investigates the concept of immersion in art and design from aesthetic, historical and political perspectives. Guest lecturers come from digital arts, curating, media studies, and the histories of science and technology from the ZHdK, Royal Academy of Fine Arts Antwerp, Rensselaer Polytechnic Institute (New York), Haus der Elektronischen Künste (HEK, Basel) and the Technical University in Dresden. Topics will include contemporary artistic practices in multi-sensory immersion, extended reality and history of VR and the arts, and art and artificial intelligence.
THE ARTS OF IMMERSION: Sensing, Bodies and Responsive Environments
The history and practices of “immersion” in the arts has long focused on the senses being transformed through melding them with technologies embedded into the actual physical world. As the French actor and theater writer Antonin Artaud wrote in 1938, the theater would be a virtual reality (réalité virtuel) – a doubling or “stand in” for reality. But now, the next wave of immersion seeks the opposite: to capture the senses in order to render a synthetic world that is “realer” than the physical one. In the words of computer graphics pioneer Ivan Sutherland who invented the first head mounted artificial reality display in 1965, such an “ultimate display” would need to “serve as many senses as possible.” Thus, contrary to the idea that the senses are simply to be replaced by the prosthetics of artificial sensors, a different story seems to be emerging. Our senses are needed to drive and feed ever-new immersive experiences by being interfaced to the simulated world that we increasingly inhabit. Using my own artistic work as well as historical examples, this talk will give a critical historical and practice-based introduction and overview of the HS 2022 Immersive Arts Lecture Series.
Christopher Salter (USA/CH) is an artist, Professor of Immersive Arts and Director of the Immersive Arts Space, ZHdK. He is also Professor Emeritus, Design and Computation Arts at Concordia University in Montreal and from 2014-2022, was Co-Director of the Hexagram network for Research-Creation in Media Arts and Technology, also in Montreal. His work has been seen all over the world at such venues as the Venice Architecture Biennale and Barbican Centre among many others. He is the author of Entangled (2010), Alien Agency (2015) and Sensing Machines (2022), all published by the MIT Press.
BREAKING THE FIFTH WALL: Immersion between live performance and virtual spaces
How can virtual performances draw spectators and performers into the virtual while connecting them in an immersive experience? Immersion is of all times and of all arts. Where different arts come together, a complete experience emerges, often bridging the traditional boundaries between spectators, medium and character. These performances balance on the border between performance, video and installation art. They appeal to all of the senses and explore spatial relationships, placing the spectator at the center of this sensory game. The fourth wall – the imaginary wall at the end of the stage between the audience and the performance space – has come down. But what if we take the spectator one step further? What if the spectator becomes the story, as if we, the creators, were the directors of one’s life? And what are the parameters that we must define, control and manipulate in order to provide such a complete experience? In both a presentation of his own artistic work, as well as compelling examples of other digital artists, researcher and director Kristof Timmerman attempts to map out the complexity and enormous potential of live performances within virtual environments.
Kristof Timmerman (BE) is a designer and director of digital performances and installations, working in the field of live, interactive digital environments and virtual reality. He worked for several theater companies, including the experimental CREW. In 2006 he founded the digital artist collective studio.POC Kristof is the chair and coordinator of MAXlab, the research group on the interaction between art and digital technology at the Royal Academy of Fine Arts Antwerp. He is also a teacher and frequently consulted coach for VR productions and digital storytelling.
VIRTUAL WORLDS-INTERACTIVE ENVIRONMENTS: Art as an Immersive Experience
Mobile and extended reality technologies are opening new spaces and interaction possibilities for us. The immersion in virtual worlds by means of VR or the inclusion of the outside world in AR, in which the real world is overlaid with virtual images both open new perceptions, creating an alterity of our living world. Immersive image worlds are conquering the art world and these experiences are becoming a new field of action, as the exhibitions of the Japanese collective teamLab or the enterprise Superblue, a new branch of the New York based Pace Gallery, show. Sabine Himmelsbach will speak about these current developments and present examples from her own curatorial work at HEK (House of Electronic Arts), Works discussed range from interactive installations to AI, game environments or video installations which challenge, provoke, and explore how technology is representing, influencing, and changing our world.
Sabine Himmelsbach is director of HEK (House of Electronic Arts) in Basel. Trained in art history, she was project manager for the Steirischer Herbst Festival in Graz and in 1999 became exhibition director at the ZKM | Center for Art and Media in Karlsruhe. From 2005–2011 she was the artistic director of the Edith-Russ-House for Media Art in Oldenburg, Germany. In 2022 she curated Earthbound – In Dialoge with Nature for the European Capital of Culture Esch-sur-Alzette in Luxembourg. As a writer and lecturer, she is dedicated to topics related to media art and digital culture.
CONTESTING A “NEW MEDIUM”: Virtual Reality as Cultural Probe
The Art and Virtual Environments Project, produced in Canada at The Banff Centre for the Arts between 1991-1994, is one of the earliest large scale artistic initiatives employing Virtual Reality. Now twenty-eight years old, its organizational structure and political context, as well as the efforts, only partially realized, to integrate critical-theoretical concerns from the humanities and social sciences within an intensive experimental development and exhibition-driven production cycle are important for contemporary initiatives. From his perspective as one of the organizers, media theorist and musician Prof. Michael Century reconstructs the intent and actualization of the project’s three-part design: a transdisciplinary three-month residency of artists, engineers and critical theorists, with parallel rapid prototyping using low-end VR equipment establishing hardware and software requirements for a high-end production lab; an onsite and virtual seminar and publication informing an open call for full-scale commissions; a final production and exhibition phase including 10 completed artworks and critical analysis. Century will expose the challenges encountered and successes achieved in this effort at transdisciplinary creative and intellectual production, set against shifting Canadian policy priorities in the late millennium and the turn away from the immersion-intensive mythos of early VR to the network-centric concerns of the early internet.
Michael Century, pianist and composer, is Professor of New Media and Music in the Arts Department at Rensselaer Polytechnic Institute, which he joined in 2002. Prior to joining Rensselear, he worked as an new media researcher, inter-arts producer, and arts and technology policy advisor (Banff Centre for the Arts (1979-93), McGill University (1998-2002), Government of Canada (1993-98)). He is the author of Northern Sparks: Innovation, Technology Policy and the Arts in Canada from Expo 67 to the Internet Age.
THE SMARTNESS MANDATE: AI and Ubiquitous Computing’s Impact on Art and Design
Smart medicine. Smart homes. Smart cities. Smart Grids. This talk traces a unique genealogy of the natural and human sciences in relationship with art and design to demonstrate how the long historical imperative to make our world ever smarter through ubiquitous computing, artificial intelligence, and perpetual “learning” has become the new logic of our present. As Halpern, a historian of science with broad interests in the social-cultural impacts of technology will demonstrate, the mandate for ubiquitous and immersive “smartness” has changed the way we understand the very nature of society, economics, culture and environment for both better and worse. In other words, the smartness mandate is central to how we design and envision the future.
Orit Halpern is Lighthouse Professor and Chair of Digital Cultures and Societal Change, Technische Universität Dresden. Her work bridges the histories of science, computing, and cybernetics with design. She is the author of Beautiful Data (Duke UP 2015) and The Smartness Mandate with Robert Mitchell (forthcoming). As part of her work, she is also interested in digital cinema and multimedia documentary, architecture and design, contemporary art practice, animation, and literature.
As of July 2022, Christopher Lloyd Salter, until now Professor of Design and Computation Arts at Concordia University in Montreal, will take over the Immersive Arts professorship from Christian Iseli, who will retire at the end of the Spring semester.
Christopher Lloyd Salter, an internationally known artist and the codirector of the Hexagram network for arts, culture, and technology, is the author of many publications, among which the acclaimed book Entangled: Technology and the Transformation of Performance (MIT Press 2010).
Christian Iseli has been teaching and researching at the Zurich University of the Arts ZHdK since 1995. He holds a professorship for Immersive Arts, heads the Immersive Arts Space and teaches in the MA Film program. After studying history, German and English literature at the University of Bern, Iseli was a director of documentary films and worked in editing and cinematography on feature films and documentaries.
