“Extended reality” (XR) devices like Apple’s recently announced Vision Pro or Meta’s Oculus Quest 3 enable new possibilities for mixing the real world with a computationally generated one, promising to “change interaction as we know it.” What are we to make of this claim?
To tackle this question, Probing XR’s Futures brings together designers, scholars, artists and curators during a 4-year SNF-funded research project. Its opening workshop on Wednesday, 19 June 2024, will bring together project partners, collaborators, and interested researchers at the Immersive Arts Space to take a first stab at core issues relating to XR from contrasting perspectives. These perspectives include design fiction, sensory ethnography, video analysis, critical disability studies, STS and more.
The opening workshop at the IAS offers an apt opportunity to engage interdisciplinary conversation on design fiction, bodily experience, and critical inquiry, while providing a forum for instructive exchange between upcoming scholars and established researchers. Confirmed participants include Sabine Himmelsbach (HeK Basel), David Howes (Concordia), Lorenza Mondada (Basel), Pilar Orero (Barcelona), Andreas Uebelbacher (Zugang für Alle, Zurich).
Workshop participation is upon invitation. For those interested in joining, please contact Joëlle Kost at IAS.
Where: ZHdK | Immersive Arts Space | 1.J30 When: 19.06.2024
The Immersive Arts Space conducts practice-based research on the field of interaction between art, design and digital technology. The team members have their professional roots in film, game design, interaction design, music, computer science and engineering, and often members have a background in more than one discipline. The IASpace is part of the research cluster of the Digitalization Initiative of the Zurich Higher Education Institutions (DIZH).
“Extended reality (XR) devices like Apple’s recently announced Vision Pro or Meta’s Oculus Quest 3 enable new possibilities for mixing the real world with a computationally generated one, promising to “change interaction as we know it.” Yet, there is little research on exactly how XR might reshape bodily subjectivity and experience. Probing XR’s Futures utilizes a critically-historically informed, practice-based design approach to examines how XR technologies reimagine bodily subjectivity, interaction and experience, on the one hand, and how bodily experience could reimagine XR, on the other. The 4-year project employs critical, creative, conceptual and empirical approaches to address three questions: How is everyday interaction in XR achieved? How will XR change interaction and what social reciprocity and mutual access will be enabled? What concrete effects and forms of discipline will be enacted on disabled bodies interacting in XR? The objective is to use design fiction, a design research method that prototypes objects and scenarios to provoke new ways of thinking about the future, as a form of critical inquiry to probe the present and future of social interaction in XR in three different settings and contexts: the lab, public space and in collaboration with disabled researchers and communities. Situated at the Immersive Arts Space at the Zurich University of the Arts, the project is at the interdisciplinary intersection of Critical VR studies, Science and Technology Studies (STS) and experimental media design. It will constitute one of the first in the context of Swiss and German speaking design research to develop alternative thinking and experimental aesthetic-design analysis, reflection and critique of XR directly in situated action and use with the general public.
Team: Christopher Salter (Project Lead) Puneet Jain (PhD Candidate) Eric Larrieux (Researcher) Chris Elvis Leisi (Researcher) Oliver Sahli (Researcher) Philippe Sormani (Senior Researcher) Stella Speziali (Researcher)
Project Partners: Andreas Uebelbacher (Access for All Foundation) John David Howes (Concordia University Montreal, Sociology/Anthropology) Sabine Himmelsbach (Haus der elektronischen Künste Basel, HeK) Pilar Orero (Universitat Autònoma de Barcelona, Transmedia Research Group) Lorenza Mondada (Universität Basel, Institut für Französische Sprach- und Literaturwissenschaft)
Funded by the Swiss National Science Foundation SNSF (01.11.2023 – 31.10.2027)
At the Zurich Art Weekend, the Immersive Arts Space will provide insights into two ongoing research projects that deal with the transfer of the visitor’s body into the digital domain in different ways. The multiuser mixed reality experience Doppelgänger invites visitors to a confrontation with their own self as a three-dimensional double in a real environment. In the installation A Hero’s Return, the viewers’ bodies provide the starting point for generative heroic images in real time.
The interactive installation A Hero’s Return transforms visitors’ poses into a digitally generated alter ego. The creation of the character and its background is realized in real-time by artificial intelligence models. This installation prototype explores the capabilities of these models (such as Stable Diffusion) and focuses on the technical and aesthetic possibilities in combination with the performative interaction with visitors. The installation also incorporates popular film representation techniques: a virtual camera performs a circular tracking shot around the scanned visitor’s body, reminiscent of scenes from numerous action hero films.
Credits: Artists: Martin Fröhlich, Stella Speziali Thanks to: Ege Seçgin
doppelgaenger:apparatus is a multiuser mixed reality experience that enables participants to confront their computer generated 3D doubles in a real physical surrounding. Enabled by a custom designed AI-based process, a frontal 2D image of the participants is taken and quickly transformed into an animated 3D reconstruction. From the moment the visitors put on the headset, they see the real environment captured by a camera on the headset. Gradually, this familiar space shfits, with the appearance of the visitors’ double standing before them. Co-present with other participants, visitors intimately interact with their ‘mirror image’ while observing and interacting with others doing the same. While at first the participants get the feeling of controlling their ‘mirror image’, this rapidly shifts as their doubles, free from the constraints of the original bodies, start to invade the participants’ personal space, leading to a disturbing and uncanny play between them. Historically, a doppelgänger was seen as a ghostly counterpart for a living person, seen as an omen or sign of death. Freud later argued that encountering one’s double produces an experience of “doubling, dividing and interchanging the self.” This installation thus takes the next step in exploring how new digital sytems increasingly produce uncanny tensions between our bodies and their captured other.
Video Copyright ZHdK 2024
Credits: Artist and Development: Chris Elvis Leisi Artist and Sound: Christopher Lloyd Salter Machine Learning: Florian Bruggisser
Next presentation: MESH Festival, 16. to 20. October at HEK in Basel. More information [here]
Recent exhibition: REFRESH x FANTOCHE, 04.-08. September 2024 from noon to 8pm.
A glocal network exploring bodies in the age of computer mediated reality
BODIES-MACHINES-PUBLICS brings together a “glocal” (global and local) network of cultural and research partners in Switzerland, India and Chile that seek to provide time and space for artists to develop prototypes and works exploring the sensitive body and technical mediation in relation to public space. The project is a two-year collaborative initiative between NAVE (Chile), KHOJ Studios (India), Immersive Arts Space/ZHdK (Zurich, Switzerland) and Kornhausforum (Bern, Switzerland) and is supported by the Pro Helvetia Synergies Program.
People increasingly interact with technologies such as wearable sensors, VR/AR headsets or other data gathering systems on an intimate, bodily level which successfully blurs the lines between the physical and the digital; the body and its interaction with its environment. At the same time, the involuntary data collection and machine-led decisions that arise from these sensing technologies exacerbates historical inequalities, particularly affecting marginalized groups. Addressing the capture of human motion, thoughts and experience through new technologies is therefore an ongoing challenge that requires new kinds of creative and imaginative practices.
Residency 2024
The project starts this year with the combined residencies at the Immersive Arts Space and Kornhausforum. The two part residency starts on 9th September at the IAS and moves then into the Kornhausforum until 5th October 2024, where there will be a public presentation of the projects. Two artists – one from Switzerland, and another one from Chile – will share the spaces and work on their projects with the aim to exchange artistically as well as culturally.
The Swiss artist is Ludovica Galleani d`Agliano, who graduated in 2023 from Interaction Design (ZHdK) with her project Syntia_CAM. In the month long residency she aims on developing Syntia_CAM into a live performance and therefore interactive installation between the main characters and audiences.
Javier Muñoz Bravo is a composer within the field of instrumental and electronic music. Javier was trained at The University of Chile (Santiago) and Institut de recherche et coordination acoustique/musique (Ircam) in Paris. Today, Javier is based in Paris and works together with choreographer Stefanie Inhelder and the cie glitch company creating immersive dance performances. During the residency the duo will develop Une saison en métavers – an immersive piece, with a female singer in combination with AI driven visuals, motion capture and live electronics. https://javiermunozbravo.com/
Immersive Arts: Performative Perspectives on Immersion
The immersive arts lecture series for Spring Semester 2024 investigates questions of immersion from the perspectives of the performing arts including dance, theater and even real time animation. Our exciting list of five guest lecturers includes Prof. Juergen Hagler (the head of the Ars Electronica animation jury), Arnd Wesemann (journalist and long-time editor of tanz), Corinne Soland (performing artist in film, theatre and interactive media arts), Dr. Ramona Mosse (head of the theater program at the ZHdK) as well as Gilles Jobin and Susana Panadés Diaz (world renowned choreographer and collaborating dance artist working with real time motion capture).
The pubic lecture series take place at the Kino Toni (room 3.G02, Pfingstweidstrasse 96) from 17:15-18:30h. They will be also available via live stream.
THE ART OF PERFORMANCE AND EXTENDED ANIMATION Prof. Jürgen Hagler | March 5th, 2024
Performance is a word that evokes a multitude of interpretations. In the simplest terms, performance describes the act of executing a task or function, but it also encompasses the staging and presentation of the act itself. As a form of artistic expression, such an act can be a play, a piece of music, a dance choreography, or in the case of this lecture, live animation. In contrast to animated films, performative acts are typically unique, and though they can be repeated, each differs in some way. Immediacy and unpredictability, among many other qualities, are the building blocks of performance in this context. But all animations, whether pre-recorded or live, are essentially a composition of static elements brought to life as a performative act.
Jürgen Hagler (AT) studied art education, experimental visual design, and cultural studies at the University for Art and Design in Linz, Austria. He currently works as a professor of Computer Animation and Animation Studies in the Digital Media department at the Hagenberg Campus of the University of Applied Sciences Upper Austria. Since 2014 he a leading researcher of the research group Playful Interactive Environments with a focus on the investigation of new and natural playful forms of interaction and the use of playful mechanisms to encourage specific behavioral patterns. Since 2009 he is the curator of the Ars Electronica Animation Festival and initiator and organizer of the symposium Expanded Animation
PLAYFUL SIMULATIONS Arnd Wesemann | March 19th, 2024
It’s funny: All theatres, next to their museum function, assert a claim to political realism. For example, a theatre that only would like to see poverty represented on the stage by the poor would therefore be a Poor Theatre. Likewise, a theatre that would only allows digital artificiality would therefore be called an Artificial Theatre. Are either of these true? Can a theatre become a place where even non-human sensors, data and artificial intelligence play a role? Or isn’t the theatre the place where precisely everything plays a role, in this case, even technologies. In this sense, Theatre would therefore be the place where we would no longer blindly trust the calculations of models or the actions of autonomous software. This talk proposes therefore the question: Don’t we owe technology one thing above all else: a playful approach to it? Raise the curtain on a theatre of true virtuality which will allow us to experience all kinds of possibilities.
Arnd Wesemann (DE) is long-time editor of the German magazine tanz. He studied Applied Theatre Studies, a then newly formed professional education program of the University of Giessen, graduating in 1987. In 1990 he became a free-lance writer on advanced theatre and arts phenomena, publishing essays and reports on contemporary theatre, dance and New Media events in magazines such as tanz, Theater der Zeit and the Swiss-printed magazine Musik & Theater. He has been a regular contributor to the Süddeutsche Zeitung, reporting about new tendencies in European theatres. In the field of electronic media, dance and the respecting performing arts, he freelanced for Die Zeit, Frankfurter Rundschau and Screen Multimedia. In October 1994 he published a monograph on the Flemish avant garde stage director Jan Fabre with Fischer-Verlag and the book Data dummies & Net nomades with Fannei & Walz, Berlin. Since 1997 he is editing Europe’s Leading Dance Magazine, tanz, now in his 25th year and now the Global dance platform tanz.dance. He has been lecturing, teaching on Dance und New Media as well as serving on numerous juries and panels.
PERFORMING 3D CHARACTERS FOR DIFFERENT MEDIA FORMATS Corinne Soland | April 2nd, 2024
In the world of performing arts, actors and performers are not exclusively in demand for theater, film or circus. Characters are also brought to life in video games, interactive installations and virtual reality films. Corinne Soland leads us through movement exercises to understand the process of training locomotion for “the Volume” – the capture stage. What are differences in Game movement versus Narrative 360° Stories? The Immersive Arts Space researcher gives insights on how to teach performance with MoCap technologies and if (& how) this technology can shape and shift a performance. With a focus on acting, the lecture provides key concepts and current topics of Corinnes research at the IASpace in chosen past and current projects as well as accumulated knowledge of production at the intersection with performers’ needs in the Volume.
Corinne Soland (CH) is a performing artist in film, theatre and interactive media arts. After completing a Motion Capture training with Performance Captured Academy (PCA UK) in 2019, Corinne has recorded movement for both 2D and 3D programs. Selected projects include Kusunda VR (NowHere Media, GER/INDIA), The Conspiracy (Third Party Film, USA), Amazing Monster! (wowl!, SmallCreative CH/FR) and LimitsVR (Tell Me the Story Prod. CH/FR). A freelance actor and voice over talent for both animation as well as live-action projects, Corinne’s latest roles are Dimitri in the TV Series “Guetnachtgschichtli” (SRF) and Anna in Neumatt (SRF, Zodiac, Netflix). Since 2019 Corinne is part of the Immersive Arts Space team as a research associate. Corinne teaches students coming from different disciplines how to translate their movement into a characters’ body narrative through the MoCap technology in Workshops and ZHdK seminars.
THE NETWORKED STAGE: BETWEEN FUTURITY, AGENCY, AND DRAMATURGY Prof. Dr. Chris Salter | April 16th, 2024
Theatres of the future are frequently invoked in the German-speaking cultural landscape to mark a crisis of institutions and to question theatre’s mandate in a publicly funded system. Yet, in this talk, Ramona Mosse would like to coin the futurity of theatre along a different trajectory and map the digital dramaturgies of the networked stage, i.e. a radically expanded theatre in which digital technologies have reshaped the space of performance and in which our established concepts of agency and authorship no longer hold. What is at stake, then, in such a hybrid future of theatre that had been so frequently called upon during the COVID pandemic? How does the reality differ from the manifestoes for a digital theatre future, spurred on by the 772% increase in digital theatre ticket sales (ETC Digital Theatre Study) during the pandemic? To explore these questions, Mosse will use the founding of and artistic programming at the HAU4, the digital stage of the HAU Hebbel-am-Ufer Theater in Berlin, as a case study. In particular, its resident artist group Interrobang moves into focus with their recent work “Commune AI” (2023), in which a digital audience sets out to build a virtual community, moderated by an AI.
Ramona Mosse (DE/CH) is the Head of Theatre at the Zurich University of the Arts. She has taught at the Free University Berlin, the Goethe University Frankfurt, Bard College Berlin, and Columbia University. In her research, she explores digitality in performance, genre and media in/of the theatre, and contemporary political performance practices. From 2021-2022, she headed up the VolkswagenFoundation funded research project Viral Theatres, which documents the aesthetic shifts of pandemic theatre-making.
COSMOGONY: NEW AVENUES BETWEEN XR AND DANCE Gilles Jobin and Susana Panadés Diaz | April 30th, 2024
Cie Gilles Jobin is focusing on technology and innovation for real time performances at a distance. Creating multiple groundbreaking projects Gilles Jobin is recognized as one of a leading creative force in the XR world. Gilles is regularly invited as an international guest lecturer on dance and technology in Europe, Asia and Americas. Geneva’s company studio is equipped for motion capture and digital creation and structured as an affordable research and production center for motion capture and digital technology from the point of view of the performing arts. In this talk, Jobin and his main dancer/collaborator Susana Panadés Diaz will discuss the company’s artistic process working with technologies like Motion Capture and Virtual Reality.
Gilles Jobin (CH) is a choreographer living and working in Geneva whose productions have been performed all over the world since 1995. Based in London from 1997 to 2004 he created the groundbreaking pieces A+B=X (1997), Braindance (1999), The Moebius Strip (2001) and Under Construction (2002), relying on choreographic language outside of established aesthetic frameworks that included forays into visual arts and live art. In 2003 he created TWO-THOUSAND-AND-THREE for the 22 dancers of the Ballet of the Grand Théâtre de Genève for a performance that “transcends both classical and contemporary dance” (Libération). His creations are being presented in major film festivals such as the Sundance Film Festival (2018 and 2020) and the Venice Film Biennial (2018 and 2020), leading dance festivals such as the Lyon Dance Biennial (2018) or BAM (2019) in New York or museums such as Haus For Electronishe Kunst / HEK in Basel. With the same creative team and dancers behind Magic Window (2019) and Dance Trail (2020), two augmented reality dance pieces, followed by La Comédie Virtuelle a multiuser VR piece and La Comédie Virtuelle – live show a real time multiuser performance in VR. In 2021 he created Cosmogony, a dance piece motion captured live in the company’s studios and streamed in real time globally.
Susana Panadés Diaz (CH) studied classical and contemporary dance at the Institut del Teatre in Barcelona, before entering the P.A.R.T.S.programme in Brussels, directed by Anne Teresa de Keersmaeker. She worked with Caterina Sagna’s dance company in 2001 on the creation of Sorelline. From 1999 to 2004 she worked as a performer with Fabienne Berger in: Azur Blues, Natal, Océane Lili and Avril en mai. Since 2005 she has worked with Cie Gilles Jobin as a performer in a variety of projects. She was part of the cast for The Moebius Strip (Pachuca 2007) and A+B=X (Paris 2010) re-runs and has also worked as artistic assistant and multimedia manager for the company. Susana Panades Diaz is Cie Gilles Jobin’s main dance artist. She is an expert in digital motion capture. Member of the Cie Gilles Jobin since 2005, Susana is extremely familiar with the company’s dance style. Her role is to resolve issues related to the optimisation of motion capture from the point of view of the performers.
Zangezi is a Russian Cubo-Futurist poem/play written by the poet Velimir Khlebnikov 101 years ago in 1922. The story revolves around Zangezi, a prophet who speaks in the language ZAUM, a Russian word that is translated as “beyondsense.” Zangezi speaks with and can understand the birds, the gods, the stars and speaks in those languages as well as in poetic and also what K. called “ordinary” language. The Immersive Arts Space is working on this text because as a futurist work, Zangezi deals with is a fundamental human question which is increasingly becoming a problem for machines – what is the basis of language? Is language only about meaning based on syntax? Is it about predictable sequences? Rules and probabilities? Or is there something else going on that is universal and cosmic about language not as words and their meanings but as an act of sound?
Indeed, Khlebnikov, who had a mystical belief in the power of words, thought that the connections between sounds and meaning were lost during mankind’s history, and it was up to those in the future to rediscover them. 101 years later we ask how we might be able to approach Zangezi’s operations on multiple levels – cosmological, political, historical, technological – in a moment where we are increasingly surrounded by machines that produce something that appears like human language but in which there is no speaker, no body and no sound.
Zangezi-Experiments, a multimedia stage play, will be performed at the MESH Festival. Before, fragments of the piece were recreated as a work in progress for the REFRESH#5 2023 festival, that took the form of a theatrically staged reading within the technical machine of the Immersive Arts Space itself.
Credits: Chris Salter (Direction, Sound Design) Corinne Soland (performer) Luana Volet (performer) Antonia Meier (performer) Stella Speziali (Development Meta Human) Valentin Huber (Visual Design/Unreal Engine) Pascal Lund-Jensen (Sound Design), formerly Eric Larrieux Sébastien Schiesser (Light Design, show control) Ania Nova (Russian voice over)
