Gilles Jobin and Susana Panadés Diaz | 30th April 2024 | 17:15-18:30
Shifting Realities
Shifting Realities was an overarching interdisciplinary research project, that started in Spring 2021. The goal is to explore the interplay of real and virtual experiences as well as the interaction of VR-users and by-standing spectators in shared experiences. Combining these different perspectives creates scope for new practices and forms
of expression in Extended Reality (XR).
The research is structured in four binary focus areas (or pair of opposites):
• Virtual vs. augmented environments
• Virtual vs. real spaces, objects or humans
• Binaural vs. spatial audio
• Storytelling vs. game mechanics
The research focuses at the interfaces defined by the pairs of opposites above. Through them, differences in perspective, shifts in perception, and the extent of interactivity can be shaped and controlled. The primary goal is the development of prototypical unique solutions in limited subareas.
In a first phase, basic concepts of reality shifts were explored and developed. They all depict situations, in which the users find themselves in transitional states between virtuality and reality. The aesthetics to create such transitional settings are defined by rough 3D scans, sketchy point clouds created in real time by depth cameras or the Oculus passthrough feature. The resulting imperfection of such low-end processes induces an awareness of our fluctuating and at times uncertain perception while experiencing mixed reality content.
In a second phase, three prototypical experiences were developed. Two are device-based with mixed reality components displayed in VR goggles:
• PASSING THROUGH THE REAL and
• FRAGMENTS OF REALITY.
One is installation-based with spatial augmented reality contingent on projection mapping:
• FRAGMENTS OF A CONVERSATION (watch video).
For the first time, the three experiences were presented in preliminary versions at the LabInsights in June 2022.
Team:
Researchers: Florian Bruggisser, Martin Fröhlich, Valentin Huber, Norbert Kottmann, Eric Larrieux, Chris Elvis Leisi, Oliver Sahli, Stella Speziali. Production manager: Kristina Jungic; Chief Technician: Sébastien Schiesser. Principal Investigator: Prof. Christian Iseli
Dreaming of Time and Space
Installation and immersive performance by Eric Larrieux (MA Electroacoustic Composition) / October 6th – 9th 2021
Focal Field
Jordan Juras & Davide Luciani are Artists in Residence 2021 at the Immersive Arts Space and ICST.
Focal Field explores the spectrum of presence felt in augmented sonic reality – from the intimacy of closeness to the abstracted relation between disembodied voice, sound, and space. Perceptual relations between the real and virtual are tightened by situating augmented reality sound within synthetic reverberant architectures, and reinforcing localisation through light. The installation engages with our necessity to rationalise any aural manifestation. These fields of fragmentation – between the spectrums of connection and disconnection – have led the duo to draft a sound space where the voice, the Phonè, and its experience become the object and subject of aesthetic speculation.
The presentations took place on Thursday, October 22nd 2021.
Immersive Landscapes in Art and Film
In the Z-Module “Immersive Landscapes in Art and Film” in September 2021, students from the fields of design, film, theatre and fine arts dealt with photogrammetry, virtual reality and 3D sound. Within two weeks, the groups developed three different immersive landscapes:
– Using virtual reality glasses, an actor could discover a walk-in visual scenery by means of the sense of touch and change it by interacting with virtual objects.
– Real hands became moving digital sculptures in a planetary virtual reality landscape using photogrammetry and motion capture.
– White rigid bodies became transformers of an abstract soundscape via motion capture in a participatory performance in immersive arts space.
In addition to engaging with transdisciplinary art and film projects, the students were able to test and evaluate the latest ‘interactive experience’ with Birdly at Somniacs in a field trip.
Teaching Staff:
Miriam Loertscher (Institute for Performing Arts and Film)
Thomas Isler (Department Fine Arts)
Jyrgen Ueberschär (Departement Fine Arts)
Valentin Huber (Department Performing Arts and Film/ Immersive Arts Space)
Stella Speziali (Immersive Arts Space)
Eric Larrieux (Immersive Arts Space)
Power and Presence
Diploma project, MA Game Design, by Oliver Sahli
Power and Presence explores meaningful and empowering interaction in virtual reality and how it can be implemented as game mechanics without breaking the feeling of being in another world. A critical analysis of game design theories and how they need to be applied to VR is demonstrated through a game that uses phonetic interaction.
Oliver Sahli, research associate at the Immersive Arts Space and graduation student in Master in Game Design, showcased his project Power and Presend within the diploma exhibition of the ZHdK in June 2021.
Virtual Real World
Diploma project, MA Game Design, by Chris Elivis Leisi
In today’s VR games, the body often serves as the controller. However, when the player enters the virtual world, the connection to the physical environment is often lost. This master’s thesis deals with immersion mechanics in VR and reveals the potentials that arise when one’s own home can be integrated into the virtual world as a play area.
Chris Elvis Leisi, research associate in the Immersive Arts Space and graduate student in Master in Game Design, exhibited his graduation project Virtual Real World within the diploma exhibition of the ZHdK in June 2021.
black box fading
Diploma performance, MA Transdisciplinarity by Melody Chua
black box fading, a performance for human, sensor-augmented flute (chaosflöte), and improvisation machine (AIYA), is an immersive experience that draws upon the performance interplay between human and machine to craft a narrative that manipulates perceptions of human-machine agency and human-machine interactions in a neosurrealist environment. The work is a hybrid setting between performance and installation, and between virtual reality and live events, where live reactive sounds and visual projections, shifting perceptions of space and scale, and unconventional 360° editing techniques contribute to the sensation of continuously negotiable dynamics between human and machine as well as the disruption of traditional performance hierarchies.
Crew
Melody Chua – concept, instrument, performance
Valentin Huber: cinematographer 360° camera
Eric Larrieux: sound engineer
Sébastien Schiesser: technical manager, IAS
Chimaera
Diploma project, Master Transdisciplinarity, by Bojan Milosevic
The use of human data in combination with computer algorithms creates a kind of post-human entity. In form of an improvisational dance performance, Chimaera explores the interaction between a performer and his avatar – a human-machine hybrid. The performance took place in March in the Immersive Arts Space and was streamed live.
Crew
Bojan Milosevic (project leader, audio and video coding)
Petra Rotar (dance, choreography)
Carmen Stüssi (dramaturgy)
Patrick Müller (mentoring)
Tobias Baumann (support motion capture)
Eric Larrieux (support 3D sound)
Schiesser Sébastien (IAS technician)
Martin Fröhlich (support projection mapping
NEVO | Neural Volumetric Capture
NEVO is a processing pipeline that combines multiple machine learning models to convert a two-dimensional images of a human into a volumetric animation. With the help of a database containing numerous 3D-models of humans, the algorithm is able to create sketchy virtual humans in 3D. The conversion is fully automatic and can be used to efficiently create sequences of three-dimensional digital humans.
Group members: Florian Bruggisser (lead), Chris Elvis Leisi. Projects: Shifting Realities Associated events: REFRESH X FANTOCHE, LAB Insights