Floating in Dancing Lights is a mesmerizing dance performance featuring a dancer and a flying swarm of objects, illuminated by means of Spatial Augmented Reality. The performance represents the tentative culmination of an internal project of the Immersive Arts Space which offers a do-it-yourself framework for designing and building remotely controllable helium drones.
Floating in Dancing Lights was part of the artistic research project Helium Drones. The performances premiered at the REFRESH#3 conference in September 2020.
Crew: Martin Fröhlich (project lead), Max Kriegleder (robotics, motion control), Roman Jurt (rapid prototyping, construction), Serena Cangiano (ideation).
Performance: Denise Lampart, (choreography), Naomi Khamihigashi (dancer), David Peña (assistant to choreographer), Ben Vorhar (costume design), Luca Magni, Eric Larrieux (sound).
The performance A Day at the Beach explores the use of 3D audio and projection mapping to achieve a sense of immersion without isolating the participants from the real world and thus enabling an imaginary fantasy world to come to life. In in order to transport the participants into this virtual world, multiple levels of 3D audio and projection mapping are employed, both directly onto umbrellas, as well as throughout the room itself. The goal is to create exploratory, interactive sonic worlds, where the participants’ behaviour (relative to each other and the environment) shape the sonic and visual experience. These environments are best experienced from directly underneath the umbrellas.
At the REFRESH conference, A Day at the Beach demonstrated some early results of the ongoing research project The Umbrella Project. – Watch the teaser video.
Crew: Eric Larrieux (project lead), Stella Speziali, Martin Fröhlich, Corinne Soland, Mariana Vieira Grünig
cineDesk is a versatile previsualization tool that provides real-time simulations for the development of film scenes, VR experiences and games in 3D spaces. In the preproduction process of films, cineDesk primarily supports scene blocking and lighting previsualization. Directors, cinematographers and production designers can collaboratively explore how space, props and acting, are translated into cinematic sequences. The positions and movements of the virtual actors and the virtual camera in the 3D space are rendered in real time into 2D videos with the help of the Unreal game engine. This allows staging and visualization ideas to be explored interactively as a team. And, by means of virtual reality goggles, the virtual 3D space can be entered and experienced directly.
cineDesk, is a further development of the Previs Table, which goes back to cooperation with Stockholm University of the Arts. The actual research focuses on advanced features and multiple applications (e.g. for gaming, stage design, architecture etc.)
The Umbrella Project is a research project that explores the use of 3D audio and projection mapping to achieve a sense of immersion without isolating participants from the real world, essentially, enabling an imaginary fantasy world to come to life in our own. We employ multiple levels of 3D audio and projection mapping (both directly within and on the umbrella, as well as throughout the room itself) in order to transport the participant into this virtual world.
The end goal of the project was to create a series of navigable compositions in the form of exploratory sonic worlds, as well as intereactive experiences where the participants’ behaviours (relative to each other and the world) shape the sonic and visual environment. Furthermore, we were investigating sonic and visual paradigms where the umbrellas can function both as objects existing in and can interact with the virtual world, as well as being windows onto these other worlds.
Naturally, these environments are best experienced from directly underneath the umbrella, where one can best appreciate the various levels of MR.
The interactive performance was part of the REFESH #3 conference and demonstrates some early results of the ongoing artistic research project The Umbrella Project.
The performance with a dancer and a swarm of helium drones, illuminated by means of projection mapping, premiered at the REFRESH #3 conference. The concept originates from the artistic research project Helium Drones.
The goal of is to develop a simple production pipeline to create photorealistic digital humans for Virtual and Augmented Reality applications. The project includes a workflow optimization from 3D capturing to body and face rigging and real time animation by means of motion and performance capture. One of the prototypes is the digital twin of Stella Speziali, a research associate of the Digital Human Group.
The interdepartmental professorship in Immersive Arts promotes, explores and develops the interaction between art, design and digital technology. The overarching structure of the chair enables mobility and permeability between the study programs of the ZHdK while encouraging a focus on inter- and transdisciplinary approaches. And, as a part of the research cluster of the Digital Initiative of the Zurich Universities (DIZH) the chair pursues networked cooperative projects with the other Zurich universities.
The practice-based exploration of Spatial Augmented Reality – or tracking based projection mapping – is a major research focus at the Immersive Arts Space. Currently, the optimization of latency and the development of high-precision projection mapping on fast moving objects or humans has a high priority.
The aim of the project is to develop a versatile co-location multi-user system, with which several users can dive into virtual worlds together in the same room. The system can be
The goal of this field of is to develop versatile and unique co-location multi-user experiences, in which several users can share virtual worlds in the same room. The concept gives room for multiplayers games and interactive experiences in which the users feel closer to their fellow players through increased physical presence experience higher levels of immersion.
Virtual Production is a central field of competence of the Immersive Arts Space. Its focus is primarily on collaborative simulations in virtual 3D spaces, which find concrete applications especially in film, but also in games, production design and architecture. An important project in this area is the pre-visualization tool cineDesk. In addition, methods for film shoots in virtual spaces, based on the Unreal game engine and projection technology are being advanced. An important achievement this area is the completed project Virtually Real, funded by the Swiss National Science Foundation SNSF.
The acquisition of photorealistic 3D models of rooms, objects and human beings is a main focus of this group. Laser scanning, photogrammetry as well as neural volumetric capture processes are used to reach these goals. A central aspect is the optimization of real-time animation by means of motion and performance capture. The implementation of digital avatars has become an important aspect of VR and AR projects in research and teaching.
The integration of 3D-Audio in all central activities is an essential feature of the Immersive Arts Space. Spatial audio concepts are currently applied in a highly differentiated way through artistic research components of The Umbrella Project.
Based on the concept of immersion, Immersive Arts deals with mediated and performative processes of involvement in which the boundaries of the medium or the work dissolve and the viewers or users seem to become one with their mediated environment. In the 1990s, the term immersion was primarily used in connection with game theory, virtual reality and other technology-supported art forms. Yet it also refers to a long tradition in art history of creating illusions 1). In this context, the so-called perceptual immersion (also called spatial immersion or presence) is prioritized. The intensity of the immersive experience is governed by the realism of the imagery and sound, and by the potential to physically interact with the mediatized environment 2).
Immersive Arts is located at the intersection formed by the convergence of film, games, music, sound design, and the visual and performing arts and is dedicated to the transdisciplinary artistic exploration of state-of-the-art technology. This contemporary horizon of experience in this field draws upon virtual reality, augmented reality, real-time simulations and the mixing of media and performative practices to enable technology-based research and teaching. Immersive Arts is related to and overlapping with the fields of Digital Art, Virtual Art, Interactive Art, Sound Art, Pervasive Games, Immersive Theatre and Expanded Cinema.
1) Cf. Grau, Oliver. Virtual Art – From Illusion to Immersion. MIT Press, 2003. 2) Another primary form is story immersion (also: narrative immersion), which can become apparent, for example, when consuming a film or a novel. Empathy and imagination play a pivotal role in this experience. The terminology here can be traced back to media psychology. On story immersion see among others: Lu, Thompson, Baranowski, Buday, & Baranowski, 2012 as well as Suckfüll & Scharkow, 2009. On perceptual immersion or presence see Lombard & Ditton, 1997, Kaplanis, Bech, Jensen, & van Waterschoot, 2014.
