“Extended reality” (XR) devices like Apple’s recently announced Vision Pro or Meta’s Oculus Quest 3 enable new possibilities for mixing the real world with a computationally generated one, promising to “change interaction as we know it.” What are we to make of this claim?
To tackle this question, Probing XR’s Futures brings together designers, scholars, artists and curators during a 4-year SNF-funded research project. Its opening workshop on Wednesday, 19 June 2024, will bring together project partners, collaborators, and interested researchers at the Immersive Arts Space to take a first stab at core issues relating to XR from contrasting perspectives. These perspectives include design fiction, sensory ethnography, video analysis, critical disability studies, STS and more.
The opening workshop at the IAS offers an apt opportunity to engage interdisciplinary conversation on design fiction, bodily experience, and critical inquiry, while providing a forum for instructive exchange between upcoming scholars and established researchers. Confirmed participants include Sabine Himmelsbach (HeK Basel), David Howes (Concordia), Lorenza Mondada (Basel), Pilar Orero (Barcelona), Andreas Uebelbacher (Zugang für Alle, Zurich).
Workshop participation is upon invitation. For those interested in joining, please contact Joëlle Kost at IAS.
Where: ZHdK | Immersive Arts Space | 1.J30 When: 19.06.2024
The IASpace team conducts practice-based research on the field of interaction between art, design and digital technology. The team members have their professional roots in film, game design, interaction design, music, computer science and engineering, and often members have a background in more than one discipline. The IASpace is part of the research cluster of the Digitalization Initiative of the Zurich Higher Education Institutions (DIZH).
“Extended reality (XR) devices like Apple’s recently announced Vision Pro or Meta’s Oculus Quest 3 enable new possibilities for mixing the real world with a computationally generated one, promising to “change interaction as we know it.” Yet, there is little research on exactly how XR might reshape bodily subjectivity and experience. Probing XR’s Futures utilizes a critically-historically informed, practice-based design approach to examines how XR technologies reimagine bodily subjectivity, interaction and experience, on the one hand, and how bodily experience could reimagine XR, on the other. The 4-year project employs critical, creative, conceptual and empirical approaches to address three questions: How is everyday interaction in XR achieved? How will XR change interaction and what social reciprocity and mutual access will be enabled? What concrete effects and forms of discipline will be enacted on disabled bodies interacting in XR? The objective is to use design fiction, a design research method that prototypes objects and scenarios to provoke new ways of thinking about the future, as a form of critical inquiry to probe the present and future of social interaction in XR in three different settings and contexts: the lab, public space and in collaboration with disabled researchers and communities. Situated at the Immersive Arts Space at the Zurich University of the Arts, the project is at the interdisciplinary intersection of Critical VR studies, Science and Technology Studies (STS) and experimental media design. It will constitute one of the first in the context of Swiss and German speaking design research to develop alternative thinking and experimental aesthetic-design analysis, reflection and critique of XR directly in situated action and use with the general public.
Team: Christopher Salter (Project Lead) Puneet Jain (PhD Candidate) Eric Larrieux (Researcher) Chris Elvis Leisi (Researcher) Oliver Sahli (Researcher) Philippe Sormani (Senior Researcher) Stella Speziali (Researcher)
Project Partners: Andreas Uebelbacher (Access for All Foundation) John David Howes (Concordia University Montreal, Sociology/Anthropology) Sabine Himmelsbach (Haus der elektronischen Künste Basel, HeK) Pilar Orero (Universitat Autònoma de Barcelona, Transmedia Research Group) Lorenza Mondada (Universität Basel, Institut für Französische Sprach- und Literaturwissenschaft)
Funded by the Swiss National Science Foundation SNSF (01.11.2023 – 31.10.2027)
At the Zurich Art Weekend, the Immersive Arts Space will provide insights into two ongoing research projects that deal with the transfer of the visitor’s body into the digital domain in different ways. The multiuser mixed reality experience Doppelgänger invites visitors to a confrontation with their own self as a three-dimensional double in a real environment. In the installation A Hero’s Return, the viewers’ bodies provide the starting point for generative heroic images in real time.
The interactive installation A Hero’s Return transforms visitors’ poses into a digitally generated alter ego. The creation of the character and its background is realized in real-time by artificial intelligence models. This installation prototype explores the capabilities of these models (such as Stable Diffusion) and focuses on the technical and aesthetic possibilities in combination with the performative interaction with visitors. The installation also incorporates popular film representation techniques: a virtual camera performs a circular tracking shot around the scanned visitor’s body, reminiscent of scenes from numerous action hero films.
Credits: Artists: Martin Fröhlich, Stella Speziali Thanks to: Ege Seçgin Produced by: Immersive Arts Space – Zurich University of the Arts
Doppelgänger is a multiuser mixed reality experience that let’s the participants confront their own, computer generated 3D double in the real surrounding. While co-present with other participants, they are also co-present with their own double while keeping it very personal and intimate to engage participants in interacting with their ‘mirror image’. While the participants get the feeling to be in control over their ‘mirror image’, this changes rapidly after a short time and the double, now free from the strings of its original, starts to invade the participants personal space, leading to a disturbing and perhaps even a bit uncanny play between the two of them.
Before the experience begins, a frontal 2D image of the participants is taken and quickly transformed into an animated 3D reconstruction with Nevo (Neural Volumetric Capture), a software developed at the IAS. After this ‘set up’ process, the participants put on the XR head mounted display and the experience starts. When they put on the headset, they see the real environment captured by a camera on the headset, but in this familiar space they see their double in front of them. The experience has a fixed duration and is interactive, combined with a dramaturgy that is divided into three main parts.
Credits: Artists: Chris Elvis Leisi, Christopher Salter Machine Learning (NEVO): Florian Bruggisser Sound: Pascal Lund-Jensen Produced by: Immersive Arts Space – Zurich University of the Arts
A glocal network exploring bodies in the age of computer mediated reality
BODIES-MACHINES-PUBLICS brings together a “glocal” (global and local) network of cultural and research partners in Switzerland, India and Chile that seek to provide time and space for artists to develop prototypes and works exploring the sensitive body and technical mediation in relation to public space. The project is a two-year collaborative initiative between NAVE (Chile), KHOJ Studios (India), Immersive Arts Space/ZHdK (Zurich, Switzerland) and Kornhausforum (Bern, Switzerland) and is supported by the Pro Helvetia Synergies Program.
People increasingly interact with technologies such as wearable sensors, VR/AR headsets or other data gathering systems on an intimate, bodily level which successfully blurs the lines between the physical and the digital; the body and its interaction with its environment. At the same time, the involuntary data collection and machine-led decisions that arise from these sensing technologies exacerbates historical inequalities, particularly affecting marginalized groups. Addressing the capture of human motion, thoughts and experience through new technologies is therefore an ongoing challenge that requires new kinds of creative and imaginative practices.
We are interested in the contribution of artists who are processually exploring the relationship between existing power structures using both digital and physical technologies and the possibility of challenging them through artistic-led creation, where the human body lies at the center of the exploration. Proposals for installations, and/or performative interventions indoors and outdoors are particularly welcome.
CALL FOR PROPOSALS
For 2024, the BODIES-MACHINES-PUBLICS project invites Swiss artists together with Chilean visual, performing and media artists, designers, architects or researchers working between art, technology and science for a joint research residency at the Immersive Arts Space and Kornhausforum Bern in Switzerland. Artists for the residencies can apply through an open call from May 8, 2024 to July 07, 2024. The residency will last four weeks, two weeks in each institution (Immersive Arts Space and Kornhausforum) between September 5 and October 5, 2024. The aim of the residency is to develop prototypes and works that can be publicly exhibited. In each venue, technical and production support will be provided as well as travel, accommodation, artist and production fees. The selected artists will also be involved in internal and public workshops at the venues and will present their research work in progress to local artistic and research communities. Full details of the open call can be found here.
Immersive Arts: Performative Perspectives on Immersion
The immersive arts lecture series for Spring Semester 2024 investigates questions of immersion from the perspectives of the performing arts including dance, theater and even real time animation. Our exciting list of five guest lecturers includes Prof. Juergen Hagler (the head of the Ars Electronica animation jury), Arnd Wesemann (journalist and long-time editor of tanz), Corinne Soland (performing artist in film, theatre and interactive media arts), Dr. Ramona Mosse (head of the theater program at the ZHdK) as well as Gilles Jobin and Susana Panadés Diaz (world renowned choreographer and collaborating dance artist working with real time motion capture).
The pubic lecture series take place at the Kino Toni (room 3.G02, Pfingstweidstrasse 96) from 17:15-18:30h. They will be also available via live stream.
5th March 2024 | 17:15-18.30 | Prof. Juergen Hagler
THE ART OF PERFORMANCE AND EXTENDED ANIMATION
Performance is a word that evokes a multitude of interpretations. In the simplest terms, performance describes the act of executing a task or function, but it also encompasses the staging and presentation of the act itself. As a form of artistic expression, such an act can be a play, a piece of music, a dance choreography, or in the case of this lecture, live animation. In contrast to animated films, performative acts are typically unique, and though they can be repeated, each differs in some way. Immediacy and unpredictability, among many other qualities, are the building blocks of performance in this context. But all animations, whether pre-recorded or live, are essentially a composition of static elements brought to life as a performative act.
Juergen Hagler (AT) studied art education, experimental visual design, and cultural studies at the University for Art and Design in Linz, Austria. He currently works as a professor of Computer Animation and Animation Studies in the Digital Media department at the Hagenberg Campus of the University of Applied Sciences Upper Austria. Since 2014 he a leading researcher of the research group Playful Interactive Environments with a focus on the investigation of new and natural playful forms of interaction and the use of playful mechanisms to encourage specific behavioral patterns. Since 2009 he is the curator of the Ars Electronica Animation Festival and initiator and organizer of the symposium Expanded Animation
19th March 2024 | 17:15-18.15 | Arnd Wesemann
PLAYFUL SIMULATIONS
It’s funny: All theatres, next to their museum function, assert a claim to political realism. For example, a theatre that only would like to see poverty represented on the stage by the poor would therefore be a Poor Theatre. Likewise, a theatre that would only allows digital artificiality would therefore be called an Artificial Theatre. Are either of these true? Can a theatre become a place where even non-human sensors, data and artificial intelligence play a role? Or isn’t the theatre the place where precisely everything plays a role, in this case, even technologies. In this sense, Theatre would therefore be the place where we would no longer blindly trust the calculations of models or the actions of autonomous software. This talk proposes therefore the question: Don’t we owe technology one thing above all else: a playful approach to it? Raise the curtain on a theatre of true virtuality which will allow us to experience all kinds of possibilities.
Arnd Wesemann (DE) is long-time editor of the German magazine tanz. He studied Applied Theatre Studies, a then newly formed professional education program of the University of Giessen, graduating in 1987. In 1990 he became a free-lance writer on advanced theatre and arts phenomena, publishing essays and reports on contemporary theatre, dance and New Media events in magazines such as tanz, Theater der Zeit and the Swiss-printed magazine Musik & Theater. He has been a regular contributor to the Süddeutsche Zeitung, reporting about new tendencies in European theatres. In the field of electronic media, dance and the respecting performing arts, he freelanced for Die Zeit, Frankfurter Rundschau and Screen Multimedia. In October 1994 he published a monograph on the Flemish avant garde stage director Jan Fabre with Fischer-Verlag and the book Data dummies & Net nomades with Fannei & Walz, Berlin. Since 1997 he is editing Europe’s Leading Dance Magazine, tanz, now in his 25th year and now the Global dance platform tanz.dance. He has been lecturing, teaching on Dance und New Media as well as serving on numerous juries and panels.
2nd April 2024 | 17:15-18:30 | Corinne Soland
PERFORMING 3D CHARACTERS FOR DIFFERENT MEDIA FORMATS
In the world of performing arts, actors and performers are not exclusively in demand for theater, film or circus. Characters are also brought to life in video games, interactive installations and virtual reality films. Corinne Soland leads us through movement exercises to understand the process of training locomotion for “the Volume” – the capture stage. What are differences in Game movement versus Narrative 360° Stories? The Immersive Arts Space researcher gives insights on how to teach performance with MoCap technologies and if (& how) this technology can shape and shift a performance. With a focus on acting, the lecture provides key concepts and current topics of Corinnes research at the IASpace in chosen past and current projects as well as accumulated knowledge of production at the intersection with performers’ needs in the Volume.
Corinne Soland (CH) is a performing artist in film, theatre and interactive media arts. After completing a Motion Capture training with Performance Captured Academy (PCA UK) in 2019, Corinne has recorded movement for both 2D and 3D programs. Selected projects include Kusunda VR (NowHere Media, GER/INDIA), The Conspiracy (Third Party Film, USA), Amazing Monster! (wowl!, SmallCreative CH/FR) and LimitsVR (Tell Me the Story Prod. CH/FR). A freelance actor and voice over talent for both animation as well as live-action projects, Corinne’s latest roles are Dimitri in the TV Series “Guetnachtgschichtli” (SRF) and Anna in Neumatt (SRF, Zodiac, Netflix). Since 2019 Corinne is part of the Immersive Arts Space team as a research associate. Corinne teaches students coming from different disciplines how to translate their movement into a characters’ body narrative through the MoCap technology in Workshops and ZHdK seminars.
16th April 2024 | 17:15-18:30 | Dr. Ramona Mosse
THE NETWORKED STAGE: BETWEEN FUTURITY, AGENCY, AND DRAMATURGY
Theatres of the future are frequently invoked in the German-speaking cultural landscape to mark a crisis of institutions and to question theatre’s mandate in a publicly funded system. Yet, in this talk, Ramona Mosse would like to coin the futurity of theatre along a different trajectory and map the digital dramaturgies of the networked stage, i.e. a radically expanded theatre in which digital technologies have reshaped the space of performance and in which our established concepts of agency and authorship no longer hold. What is at stake, then, in such a hybrid future of theatre that had been so frequently called upon during the COVID pandemic? How does the reality differ from the manifestoes for a digital theatre future, spurred on by the 772% increase in digital theatre ticket sales (ETC Digital Theatre Study) during the pandemic? To explore these questions, Mosse will use the founding of and artistic programming at the HAU4, the digital stage of the HAU Hebbel-am-Ufer Theater in Berlin, as a case study. In particular, its resident artist group Interrobang moves into focus with their recent work “Commune AI” (2023), in which a digital audience sets out to build a virtual community, moderated by an AI.
Ramona Mosse (DE/CH) is the Head of Theatre at the Zurich University of the Arts. She has taught at the Free University Berlin, the Goethe University Frankfurt, Bard College Berlin, and Columbia University. In her research, she explores digitality in performance, genre and media in/of the theatre, and contemporary political performance practices. From 2021-2022, she headed up the VolkswagenFoundation funded research project Viral Theatres, which documents the aesthetic shifts of pandemic theatre-making.
30th April 2024 | 17:15-18:30 | Gilles Jobin and Susana Panadés Diaz
COSMOGONY: NEW AVENUES BETWEEN XR AND DANCE
Cie Gilles Jobin is focusing on technology and innovation for real time performances at a distance. Creating multiple groundbreaking projects Gilles Jobin is recognized as one of a leading creative force in the XR world. Gilles is regularly invited as an international guest lecturer on dance and technology in Europe, Asia and Americas. Geneva’s company studio is equipped for motion capture and digital creation and structured as an affordable research and production center for motion capture and digital technology from the point of view of the performing arts. In this talk, Jobin and his main dancer/collaborator Susana Panadés Diaz will discuss the company’s artistic process working with technologies like Motion Capture and Virtual Reality.
Gilles Jobin (CH) is a choreographer living and working in Geneva whose productions have been performed all over the world since 1995. Based in London from 1997 to 2004 he created the groundbreaking pieces A+B=X (1997), Braindance (1999), The Moebius Strip (2001) and Under Construction (2002), relying on choreographic language outside of established aesthetic frameworks that included forays into visual arts and live art. In 2003 he created TWO-THOUSAND-AND-THREE for the 22 dancers of the Ballet of the Grand Théâtre de Genève for a performance that “transcends both classical and contemporary dance” (Libération). His creations are being presented in major film festivals such as the Sundance Film Festival (2018 and 2020) and the Venice Film Biennial (2018 and 2020), leading dance festivals such as the Lyon Dance Biennial (2018) or BAM (2019) in New York or museums such as Haus For Electronishe Kunst / HEK in Basel. With the same creative team and dancers behind Magic Window (2019) and Dance Trail (2020), two augmented reality dance pieces, followed by La Comédie Virtuelle a multiuser VR piece and La Comédie Virtuelle – live show a real time multiuser performance in VR. In 2021 he created Cosmogony, a dance piece motion captured live in the company’s studios and streamed in real time globally.
Susana Panadés Diaz (CH) studied classical and contemporary dance at the Institut del Teatre in Barcelona, before entering the P.A.R.T.S.programme in Brussels, directed by Anne Teresa de Keersmaeker. She worked with Caterina Sagna’s dance company in 2001 on the creation of Sorelline. From 1999 to 2004 she worked as a performer with Fabienne Berger in: Azur Blues, Natal, Océane Lili and Avril en mai. Since 2005 she has worked with Cie Gilles Jobin as a performer in a variety of projects. She was part of the cast for The Moebius Strip (Pachuca 2007) and A+B=X (Paris 2010) re-runs and has also worked as artistic assistant and multimedia manager for the company. Susana Panades Diaz is Cie Gilles Jobin’s main dance artist. She is an expert in digital motion capture. Member of the Cie Gilles Jobin since 2005, Susana is extremely familiar with the company’s dance style. Her role is to resolve issues related to the optimisation of motion capture from the point of view of the performers.
Zangezi is a Russian Cubo-Futurist poem/play written by the poet Velimir Khlebnikov 101 years ago in 1922. The story revolves around Zangezi, a prophet who speaks in the language ZAUM, a Russian word that is translated as “beyondsense.” Zangezi speaks with and can understand the birds, the gods, the stars and speaks in those languages as well as in poetic and also what K. called “ordinary” language. The Immersive Arts Space is working on this text because as a futurist work, Zangezi deals with is a fundamental human question which is increasingly becoming a problem for machines – what is the basis of language? Is language only about meaning based on syntax? Is it about predictable sequences? Rules and probabilities? Or is there something else going on that is universal and cosmic about language not as words and their meanings but as an act of sound? Indeed, Khlebnikov, who had a mystical belief in the power of words, thought that the connections between sounds and meaning were lost during mankind’s history, and it was up to those in the future to rediscover them. 101 years later we ask how we might be able to approach Zangezi’s operations on multiple levels – cosmological, political, historical, technological – in a moment where we are increasingly surrounded by machines that produce something that appears like human language but in which there is no speaker, no body and no sound.
The Immersive Arts Space recreated fragments of the play as a work in progress for the REFRESH#5 festival, that took the form of a theatrically staged reading within the technical machine of the Immersive Arts Space itself.
Credits: Chris Salter (Direction, Sound Design, Performance) Corinne Soland (performer) Stella Speziali (Development Meta Human) Valentin Huber (Visual Design/Unreal Engine) Norbert Kottmann (Visual Design/Unreal Engine) Martin Fröhlich (Show control) Eric Larrieux (Sound Design) Sébastien Schiesser (Light Design) Ania Nova (Russian voice over)
