Changing Matters is a project that explores the concept of a playful and abstract virtual representation of oneself. This is achieved through the use of a camera that tracks the movements of the user and translates them onto an avatar. The virtual environment is further enhanced with sound that complements the user’s movements and empowers them with control. Therefore, experience encourages you to experiment with the possibilities of the virtual body’s interaction with the immersive sound and landscape.
The project utilizes SARMotion, a software developed by Florian Bruggisser, along with Unity data that is processed using Max/MSP and Ableton Live for sound generation.
Changing Matters is a graduation project by Lorenz Kleiser (BA Game Design) and Floris Demandt (MA Composition and Theory).
The project will be showcased and can be experienced at the upcoming conference REFRESH#5.
Friendly Fire is an interactive hommage to Joseph Weizenbaum’s psychotherapy simulation «Eliza» (1966), the world’s first computer chatbot. Weizenbaum was shocked by how quickly and deeply users became emotionally involved with Eliza – and how unequivocally they treated it as if it was human. As a German-born Jewish scientist who fled the Nazis, Weizenbaum worried about dehumanization. He argued that certain human activities – like interpersonal relations and decision-making – should remain inherently human, and that the uncritical adoption of AI could erode whole aspects of human life. Weizenbaum died in 2008. In 2022, the chatbot application «ChatGPT» was released and set a record for the fastest growing software ever, reaching 100 million users in just two months. «Friendly Fire» uses ChatGPT for a creative re-imagination of Weizenbaum’s work, exploring the fears and desires that artificial «intelligence» triggers in us.
Credits:
Manuel Flurin Hendry (Artistic Direction, Production) Norbert Kottmann (Programming) Paulina Zybinska (Programming) Meredith Thomas (Programming) Florian Bruggisser (Programming) Linus Jacobson (Scenography) Marco Quandt (Light Design) Domenico Ferrari (Audio Design) Martin Fröhlich (Projection Mapping) Stella Speziali (Scanning and Motion Capture) Hans-Joachim Neubauer, Anton Rey, Christopher Salter (Project Supervision)
Supported by Migros Kulturprozent, Speech Graphics Cooperation by Immersive Arts Space, Institute for the Performing Arts and Film, ZHdK Film
Friendly Fire was developed out of the artistic-scientific project The Feeling Machine, in which Manuel Flurin Hendry explored the question of artificial emotionality. “The Feeling Machine” is the PhD thesis of Manuel Hendry, which is part of the PhD program of the Filmuniversität Babelsberg Konrad Wolf and the ZHdK. Since 2022 the project is developed artistically-technologically in the Immersive Arts Space.
Find more information and by now six conversations of Stanley with different people here
With the help of the so-called pre-trained Large Language Model (LLM), a speech algorithm, the linguistic simulation was developed. In months of development within the Immersive Arts Space and with the help of our scientific collaborators Norbert Kottmann, Martin Fröhlich and Valentin Huber, Stanley was born. In the early stagesa physical mask was developed with which the audience can converse directly in oral language in a museum-like situation. This being will be programmed as Unreal MetaHuman with the Omniverse Avatar Cloud Engine (ACE). Its physical form will be generated with projection mapping.
Stanley and The Feeling Machine were part of the exhibition “Hallo, Tod! Festival 2023” and on display for a chat in the cemetary Sihlfeld in Zurich. Read more about it in an article by the Tagesanzeiger
The LabInsights is a recurring event hosted by and in the Immersive Arts Space. At least once a year we open the doors to visitors and participants and showcase the current projects being developed in the lab. The informal event aims to demonstrate the state of the projects and at the same time involve students, artists and visitors to test the expereiences and give feedback.
Upcoming LabInsights will be posted on the website and our social media channels.
The next LabInsights will take place on April 4, 2024. This year visitors are invited to experience XRconFIGURE with Virtual Reality Googles and learn more about our latest work-in-progress projects and research such as Puppets, Probing XR’s Futures and Realtime Stable Diffusion. In an informal setting and with an aperitif it is a great opportunity to meet and exchange about current and new developments taking place in the Immersive Arts Space.
At the LabInsights, hosted on 4th may 2023, the following projects were presented: reconFIGURE (Chris Elvis Leisi, Florian Bruggisser, Chris Salter), Changing Matters (Lorenz Kleiser, Floris Demandt), Digital Gold VR experience (Chris Elvis Leisi) and The Feeling Machine (Manuel Hendry, Norbert Kottmann).
On June 29th and 30th 2022, the Immersive Arts Space presented new developments from current projects. Visitors were invited to experience our project Shifting Realities (Florian Bruggisser, Martin Fröhlich, Valentin Huber, Norbert Kottmann, Eric Larrieux, Chris Elvis Leisi, Oliver Sahli, Stella Speziali) with or without Virtual Reality goggles and to learn more about Digital Gold (Chris Elvis Leisi, Florian Bruggisser, Kristina Jungic, Christian Iseli) in our smartphones.
During REFRESH #4, from 10th to 11th November 2021, team members offered insights into the projects Shifting Realities (Oliver Sahli, Martin Fröhlich, Eric Larrieux and many more), Neural Volumetric Capture (Florian Bruggisser), Digital Twins (Stella Speziali) and cineDesk (Norbert Kottmann, Valentin Huber).
The Immersive Arts Lecture Series for Spring semester 2023 investigates artistic and design experiences in which humans, media and architectural spaces shape one another within the context of the built environment. Guest lecturers who come from the fields of digital arts, architecture, theater studies and the visual arts, will focus on specific artistic works from their practice, as way to open up critical discussion around emerging paradigms of bodily experience in spatial computing environments, human-machine interactions in architectural spaces, and immersive experiences between human, technical and natural systems.
Atmospheric Forest visualizes the complex relations between the forest, climate change and atmosphere. Trees are not only oxygen generators, but they breathe as well, and emit large amounts of volatile organic compounds into the air – we recognize it as a habitual scent of the forest. Scientists have long known about the link between a fragrant forest and the warming climate, but are uncertain about the impact and scale. Atmospheric Forest VR artwork is the outcome of a three-year artistic research project on Pfynwald, an ancient Swiss Alpine coniferous forest suffering from drought due to climate change; the scientists from WSL Research Institute have turned this forest into a “living observatory”.To create the Atmospheric Forest VR artwork, the artists scanned the Pfynwald creating a virtual point cloud environment, and visualized the data provided by scientists. The data sets collected during one growing season included measurements of volatile emissions, resin pressure in pine tree trunks and changing weather. The artists transformed these data into animated particle flows, revealing the complex interaction between the forest ecosystem and atmospheric processes.
As part of the Lecture Series, Atmospheric Forest will be presented on the evening of 08.03.23 in the Immersive Arts Space.
Rasa Smite is an artist, network researcher and cultural innovator, working with science and emerging technologies since 90s. She is founding director of RIXC Center for New Media Culture in Riga, curator of its annual festivals, and a chief-editor of Acoustic Space – peer-reviewed publication series. She holds a PhD in sociology of culture and media (from Riga Stradins University, 2011), and MA in visual arts (from Arts Academy of Latvia, 2000). Currently she works as associate professor in New Media Art programme at Liepaja University. She is author of the books – “Creative Networks. In the Rear-View Mirror of Eastern European History”, published by Amsterdam Institute for Network Culture (2012), and “Talk to Me. Exploring Human-Plant Communication” (Published by RIXC, 2014). She is also author of numerous articles and co-editor of the Acoustic Space series issues.
ON ANIMATE
Animate is an Extended Reality-based theater work (XR) at the crossroads of performance, radio play and installation focused on a near-future Canada radically transformed by climate change. Developed by a team of Canadian and German artists and researchers, the work premiered in Summer 2022 in large theater festivals in Germany and is preparing for a 2023 international tour in Europe, Canada and Asia. The story “Animate” focuses on two characters, Daniel and Laurie, who are fleeing a near future of climate disasters. As part of its dramaturgical strategy, Animate harnesses a recent XR technology to explore this potential mixing of real and the simulated in worn AR: live video “passthrough” which takes a real time video feed from the tiny cameras attached to a head mounted display (HMD) and which allows the embedding of computer-generated 3D objects into the video feed of the real environment. Animate aims to use emerging XR technologies to explore the thematic of climate change through the sensorial-aesthetic possibilities of “spatial computing” in which “human interaction with a machine in which the machine retains and manipulates referents to real objects and spaces” shifts older understandings of the virtual as abstracted from the lived, moving body.
Christopher Salter is an artist, Professor for Immersive Arts and Director of the Immersive Arts Space at the Zurich University of the Arts (ZHdK). He is also Professor Emeritus, Design and Computation Arts at Concordia University in Montreal, former Co-Director of the Hexagram network for research-creation in arts, cultures and technology and Co-Founder of the Milieux Institute at Concordia. His artistic work has been seen all over the world at such venues as the Venice Architecture Biennale, Barbican Centre, Berliner Festspiele, Wiener Festwochen, ZKM, Kunstfest Weimar, Musée d’art Contemporain, Muffathalle, EXIT Festival and Place des Arts-Montreal, among many others. He is the author of Entangled: Technology and the Transformation of Performance ( 2010), Alien Agency: Experimental Encounters with Art in the Making (2015) and Sensing Machines (2022), all from MIT Press.
ON DEGREES OF LIFE
Degrees of Life is a responsive environment exhibited in February 2022 at ZentrumFokusForschung in Vienna. The project explores the interaction between humans and living systems at an architectural scale, developing interactive environments within an architectural space that learn, grow, and decay in relation to human presence and behavior. The space reflects on the concept of biomediality and biofacts, the possible applications of living technologies, and human sensory interfaces in architecture. Degrees of Life is the result of a larger artistic research context called Co-corporeality that weaves together architectural design, sensor systems, machine learning, and microbiology. The exhibition pursues the idea of interactive architecture as a living system, in which physical presence and new modes of observation are intertwined with tangible forms of computation.
Daniela Mitterberger received her architectural master degree with honors at the Academy of Fine Arts Vienna. Within her studies in Vienna, at the Hong Kong University (HKU) and the Royal Danish Academy of Fine Arts in Copenhagen she developed a deep seated interest in the topics of bodies in architecture, human body as architecture and the definition of technology and tools within the field of architecture. Daniela’s work ranges from digital experiments to analog production, combining the fields of media culture, philosophy and narration with architecture. Currently, she is a Ph.D. researcher and A&T Ph.D. Fellow at the Chair of Architecture and Digital Fabrication (Prof. Fabio Gramazio, Prof. Matthias Kohler) focusing on intuition in digital design and robotic fabrication.
A DATA-DRIVEN DRAMA
While progressively churning all aspects of life into data, the digital escapes conventional material methods of observation and measurement. Digital materiality has been addressed by numerous media, communication and infrastructure studies, often focusing on a single effect like their carbon footprint, as the principal measure of anthropogenic pollution in the ongoing climate crisis. My research is concerned with expanding the perspective on digital materiality. Such a perspective should be able to include unlike dimensions in a perfromance: the design of graphical interfaces, internet users labour, their emotional states, purchase decisions, rare mineral mining and the labour of assembling electronic devices, automated collection of user data, financial transactions within AdTech markets, energy consumption and carbon-footprint of these processes. In this talk, I will focus on reading digital materiality through the operations of internet-based platforms and their immersive grip on users. In particular, I will propose performance-based methods to experience these operations through staging. The convergence of space, time, networks and bodies in the performative medium relates and renders experienceable the otherwise incommunicable material entanglements. If we are, as Chun suggested, characters in a ‘big data’ drama, then exposing the drama is to expose the roles and scripts that are in place, enabling them to be questioned and possibly undone.
Selena Savić is a researcher and trained architect. Her research interests revolve around the mixture of computational processes with the built environment, exploring ways to communicate communication processes. After her PhD at EPFL and a postdoc at ATTP, TU Vienna, she joined the IXDM at the FHNW Academy for Art and Design where she is currently Head of the Make/Sense PhD programme. She edited two books (Ghosts of Transparency, 2019 and Unpleasant Design, 2013) and writes about computational modeling, feminist hacking, and post human networks in the context of design and architecture.
AMBIVALENT PRACTICE OF IMMERSIVE INSTALLATION AND PERFORMANCE
Since the early nineties, Kurt Hentschlager creates media installations and live shows, most always audiovisual and immersive in nature. The ever-deepening effects of digital technology on individual and collective consciousness, and a subsequent reassessment of concepts of nature and artifice are among the subject matters that inform his practice. His shows have characteristically been visceral and challenging, with extreme perceptual effects, composed from projections, light, sound and fog. These works physiologically and emotionally affect the viewer’s experience. His approach is interdisciplinary and experimental fusing both analogue and digital media. His artistic process is rooted in an early and continuing focus on human interaction with architectural space, light and sound, loosely within the concept of the “Gesamtkunstwerk”. A recurring topic in his more recent works such as as Sub, FeedX, Zee and others is the human body with its perceptual brain “apparatus”, especially the way that the brain processes the world, colored always by cultural background, habits, imagination, subjectivity and individual psychology.
New York-based Austrian artist Kurt Hentschläger creates audiovisual performances and installations. He began to exhibit his work in 1983, and since has been working predominantly with time-based media, light and sound. The immersive nature of his work reflects on the metaphor of the sublime and the human condition in the 21st century. His current work continues to investigate human perception and the impact of new technologies on both individual and collective consciousness. Between 1992 and 2003 he worked collaboratively as the artist duo “Granular-Synthesis”. Selected presentations include the Venice Biennial, the Venice Theater Biennial, Stedelijk Museum Amsterdam, Fondation Beyeler Basel, MAC – Musée d’Art Contemporain Montreal, MAK – Museum of Applied Arts Vienna, ICC Tokyo, Arte Alameda Mexico City, MONA – Museum of Old and New Art, Hobart, Tasmania.
This lecture series investigates the concept of immersion in art and design from aesthetic, historical and political perspectives. Guest lecturers come from digital arts, curating, media studies, and the histories of science and technology from the ZHdK, Royal Academy of Fine Arts Antwerp, Rensselaer Polytechnic Institute (New York), Haus der Elektronischen Künste (HEK, Basel) and the Technical University in Dresden. Topics will include contemporary artistic practices in multi-sensory immersion, extended reality and history of VR and the arts, and art and artificial intelligence.
THE ARTS OF IMMERSION: Sensing, Bodies and Responsive Environments
The history and practices of “immersion” in the arts has long focused on the senses being transformed through melding them with technologies embedded into the actual physical world. As the French actor and theater writer Antonin Artaud wrote in 1938, the theater would be a virtual reality (réalité virtuel) – a doubling or “stand in” for reality. But now, the next wave of immersion seeks the opposite: to capture the senses in order to render a synthetic world that is “realer” than the physical one. In the words of computer graphics pioneer Ivan Sutherland who invented the first head mounted artificial reality display in 1965, such an “ultimate display” would need to “serve as many senses as possible.” Thus, contrary to the idea that the senses are simply to be replaced by the prosthetics of artificial sensors, a different story seems to be emerging. Our senses are needed to drive and feed ever-new immersive experiences by being interfaced to the simulated world that we increasingly inhabit. Using my own artistic work as well as historical examples, this talk will give a critical historical and practice-based introduction and overview of the HS 2022 Immersive Arts Lecture Series.
Christopher Salter (USA/CH) is an artist, Professor of Immersive Arts and Director of the Immersive Arts Space, ZHdK. He is also Professor Emeritus, Design and Computation Arts at Concordia University in Montreal and from 2014-2022, was Co-Director of the Hexagram network for Research-Creation in Media Arts and Technology, also in Montreal. His work has been seen all over the world at such venues as the Venice Architecture Biennale and Barbican Centre among many others. He is the author of Entangled (2010), Alien Agency (2015) and Sensing Machines (2022), all published by the MIT Press.
BREAKING THE FIFTH WALL: Immersion between live performance and virtual spaces
How can virtual performances draw spectators and performers into the virtual while connecting them in an immersive experience? Immersion is of all times and of all arts. Where different arts come together, a complete experience emerges, often bridging the traditional boundaries between spectators, medium and character. These performances balance on the border between performance, video and installation art. They appeal to all of the senses and explore spatial relationships, placing the spectator at the center of this sensory game. The fourth wall – the imaginary wall at the end of the stage between the audience and the performance space – has come down. But what if we take the spectator one step further? What if the spectator becomes the story, as if we, the creators, were the directors of one’s life? And what are the parameters that we must define, control and manipulate in order to provide such a complete experience? In both a presentation of his own artistic work, as well as compelling examples of other digital artists, researcher and director Kristof Timmerman attempts to map out the complexity and enormous potential of live performances within virtual environments.
Kristof Timmerman (BE) is a designer and director of digital performances and installations, working in the field of live, interactive digital environments and virtual reality. He worked for several theater companies, including the experimental CREW. In 2006 he founded the digital artist collective studio.POC Kristof is the chair and coordinator of MAXlab, the research group on the interaction between art and digital technology at the Royal Academy of Fine Arts Antwerp. He is also a teacher and frequently consulted coach for VR productions and digital storytelling.
VIRTUAL WORLDS-INTERACTIVE ENVIRONMENTS: Art as an Immersive Experience
Mobile and extended reality technologies are opening new spaces and interaction possibilities for us. The immersion in virtual worlds by means of VR or the inclusion of the outside world in AR, in which the real world is overlaid with virtual images both open new perceptions, creating an alterity of our living world. Immersive image worlds are conquering the art world and these experiences are becoming a new field of action, as the exhibitions of the Japanese collective teamLab or the enterprise Superblue, a new branch of the New York based Pace Gallery, show. Sabine Himmelsbach will speak about these current developments and present examples from her own curatorial work at HEK (House of Electronic Arts), Works discussed range from interactive installations to AI, game environments or video installations which challenge, provoke, and explore how technology is representing, influencing, and changing our world.
Sabine Himmelsbach is director of HEK (House of Electronic Arts) in Basel. Trained in art history, she was project manager for the Steirischer Herbst Festival in Graz and in 1999 became exhibition director at the ZKM | Center for Art and Media in Karlsruhe. From 2005–2011 she was the artistic director of the Edith-Russ-House for Media Art in Oldenburg, Germany. In 2022 she curated Earthbound – In Dialoge with Nature for the European Capital of Culture Esch-sur-Alzette in Luxembourg. As a writer and lecturer, she is dedicated to topics related to media art and digital culture.
CONTESTING A “NEW MEDIUM”: Virtual Reality as Cultural Probe
The Art and Virtual Environments Project, produced in Canada at The Banff Centre for the Arts between 1991-1994, is one of the earliest large scale artistic initiatives employing Virtual Reality. Now twenty-eight years old, its organizational structure and political context, as well as the efforts, only partially realized, to integrate critical-theoretical concerns from the humanities and social sciences within an intensive experimental development and exhibition-driven production cycle are important for contemporary initiatives. From his perspective as one of the organizers, media theorist and musician Prof. Michael Century reconstructs the intent and actualization of the project’s three-part design: a transdisciplinary three-month residency of artists, engineers and critical theorists, with parallel rapid prototyping using low-end VR equipment establishing hardware and software requirements for a high-end production lab; an onsite and virtual seminar and publication informing an open call for full-scale commissions; a final production and exhibition phase including 10 completed artworks and critical analysis. Century will expose the challenges encountered and successes achieved in this effort at transdisciplinary creative and intellectual production, set against shifting Canadian policy priorities in the late millennium and the turn away from the immersion-intensive mythos of early VR to the network-centric concerns of the early internet.
Michael Century, pianist and composer, is Professor of New Media and Music in the Arts Department at Rensselaer Polytechnic Institute, which he joined in 2002. Prior to joining Rensselear, he worked as an new media researcher, inter-arts producer, and arts and technology policy advisor (Banff Centre for the Arts (1979-93), McGill University (1998-2002), Government of Canada (1993-98)). He is the author of Northern Sparks: Innovation, Technology Policy and the Arts in Canada from Expo 67 to the Internet Age.
THE SMARTNESS MANDATE: AI and Ubiquitous Computing’s Impact on Art and Design
Smart medicine. Smart homes. Smart cities. Smart Grids. This talk traces a unique genealogy of the natural and human sciences in relationship with art and design to demonstrate how the long historical imperative to make our world ever smarter through ubiquitous computing, artificial intelligence, and perpetual “learning” has become the new logic of our present. As Halpern, a historian of science with broad interests in the social-cultural impacts of technology will demonstrate, the mandate for ubiquitous and immersive “smartness” has changed the way we understand the very nature of society, economics, culture and environment for both better and worse. In other words, the smartness mandate is central to how we design and envision the future.
Orit Halpern is Lighthouse Professor and Chair of Digital Cultures and Societal Change, Technische Universität Dresden. Her work bridges the histories of science, computing, and cybernetics with design. She is the author of Beautiful Data (Duke UP 2015) and The Smartness Mandate with Robert Mitchell (forthcoming). As part of her work, she is also interested in digital cinema and multimedia documentary, architecture and design, contemporary art practice, animation, and literature.
You are kindly invited to the opening reception of Atmospheric Forest by artists Rasa Smite and Raitis Smite (RIXC Center for New Media Culture Riga, Latvia), featuring an immersivescreening and VR experience, and apéro, taking place on Wednesday, March 8, 2023 in the Immersive Arts Space.
The VR Experience event is following the public lecture by Rasa Smite On Atmospheric Foresttaking place a day before – on Tuesday, March 7th 2023.
The Immersive Arts Practices aims to provide a general introduction to the current use of doubles, twins and avatars in films, games, the arts, and social media. Using the technologies that are available in the Immersive Arts space, the course also provides a basic understanding of creating 3D representations of humans and integrating avatars in digital media environments and immersive experiences, such as virtual and augmented reality (VR, AR) photogrammetry and 3D scan, volumetric capture, motion capture, and projection mapping. In small interdisciplinary groups, the participants then develop their own projects and implement their double, twin and avatar within the range of their own capacity and the available technologies.
In painting, photography and film, landscapes are transformed into dream images and stylized into archetypes. Landscapes are contemporary witnesses of collective longings and dystopias. With immersive technologies such as video mapping, virtual reality and spatial audio, these become multimodal experiences. In interdisciplinary groups the module aims to work on scenes for spatial or object projections in the Immersive Arts Space and test possible interactions with the motion capture system. Getting to know the workflows and experimenting with immersive media are the main focus. In this short week, experimental installations, sketches and small projects will be created, exploring diverse approaches and perspectives on digitally designed landscapes through immersive media.
Team: Eric Larrieux, Valentin Huber, Stella Speziali, Thomas Isler, Miriam Loertscher
