Digital Twins

Digital Twin «Stella», screenshot by Tobias Baumann, ZHdK ©2020

The goal of is to develop a simple production pipeline to create photorealistic digital humans for Virtual and Augmented Reality applications. The project includes a workflow optimization from 3D capturing to body and face rigging and real time animation by means of motion and performance capture. One of the prototypes is the digital twin of Stella Speziali, a research associate of the Digital Human Group.

Crew:
Florian Bruggisser (lead), Patxi Aguirre, Tobias Baumann, Stella Speziali


Educational Projects

The Immersive Arts Space hosts classes and diploma projects from all departments of the ZHdK and conducts its own educational courses, such as the Z-module ‘ Illuminated Flying Objects’ in 2020 (see below). In 2021 a lecture series focuses on the topic of immersion and practical workshops introduce key activities of the IASpace. The launch of the transversal minor program Immersive Arts is planned for autumn 22.


Immersive Arts Chair

The interdepartmental professorship in Immersive Arts promotes, explores and develops the interaction between art, design and digital technology. The overarching structure of the chair enables mobility and permeability between the study programs of the ZHdK while encouraging a focus on inter- and transdisciplinary approaches. And, as a part of the research cluster of the Digital Initiative of the Zurich Universities (DIZH) the chair pursues networked cooperative projects with the other Zurich universities.


Immersive Arts Research

The IASpace team conducts practice-based research on the field of interaction between art, design and digital technology. The team members have their professional roots in film, game design, interaction design, music, computer science and engineering, and often members have a background in more than one discipline. The IASpace is part of the research cluster of the Digital Initiative of the Zurich universities (DIZH).

Five research groups focus on the following topics:

Spatial Augmented Reality

Photo by Davide Arrizoli, ZHdK ©2019

The practice-based exploration of Spatial Augmented Reality – or tracking based projection mapping – is a major research focus at the Immersive Arts Space. Currently, the optimization of latency and the development of high-precision projection mapping on fast moving objects or humans has a high priority.

Group members: Martin Fröhlich (lead) Stella Speziali, Eric Larrieux. Projects: Presence & Absence, Helium Drones. Associated events: Floating in Dancing Lights, Dancing Digital, Twin Lab. Associated educational courses: Illuminated Flying Objects, Immersive Landscapes. Videos: Projection Mapping, Presence & Absence, Helium Drones, Floating in Dancing Lights, A Day at the Beach.


Co-experiencing Virtual Reality

Photo by Chris Elvsi Leisi, ZHdK ©2020

The aim of the project is to develop a versatile co-location multi-user system, with which several users can dive into virtual worlds together in the same room. The system can be used for many different scenarios: as a multiplayer game in which the users feel even closer to their fellow players through increased physical presence, as a virtual film set to try out camera movements or interactions of actors without much effort, or as an interactive experience that transports the user into a new dimension of immersion.

Group members: Chris Elvis Leisi, Oliver Sahli. Projects: Home in the Distance VR, Batvision, OEXR (in cooperation with ZHAW). Associated event: Virtual Echololocation (@ Refresh #3). Associated educational courses: Mulitiuser VR courses, BA Game Design. Videos: Multi-user VR, Virtual Echolocation


Virtual Production

Photo by Andreas Birkle, ZHdK ©2020

The current focus of the Virtual Production group is on a fundamentally new version of the pre-visualization tool cineDesk, a further development of the Previs Table, which goes back to our partnership with Stockholm University of the Arts. The new development includes advanced features and multiple applications (e.g. for gaming, stage design, architecture etc.)

Group members: Valentin Huber, Norbert Kottmann. Project: cineDesk. Associated educational courses: Previsualization courses, MA Film. Videos: Virtual Production, cineDesk


Digital Humans

Screenshot by Tobias Baumann, ZHdK ©2020

The Digital Humans group conducts basic research and development in the area of scanning humans and generating photorealistic digital avatars. A central aspect is the optimization of real-time animation by means of motion and performance capture. The implementation of digital avatars will become a central aspect of VR and AR projects in research and teaching.

Group members: Florian Bruggisser (lead), Patxi Exequiel Aguirre, Tobias Baumann, Stella Speziali. Project: Digital Twins


Spatial Audio

Photo by Stefan Dux, ZHdK ©2020

The integration of 3D-Audio in all central activities is an essential feature of the Immersive Arts Space. Spatial audio concepts are currently applied in a highly differentiated way through artistic research components of The Umbrella Project.

Group lead: Eric Larrieux, Associated projects: The Umbrella Project. Associated events: A Day at the Beach. Associated educational courses: Illuminated Flying Objects, Immersive Landscapes. Video: A Day at the Beach.


Immersive Arts

Based on the concept of immersion, Immersive Arts deals with mediated and performative processes of involvement in which the boundaries of the medium or the work dissolve and the viewers or users seem to become one with their mediated environment. In the 1990s, the term immersion was primarily used in connection with game theory, virtual reality and other technology-supported art forms. Yet it also refers to a long tradition in art history of creating illusions 1). In this context, the so-called perceptual immersion (also called spatial immersion or presence) is prioritized. The intensity of the immersive experience is governed by the realism of the imagery and sound, and by the potential to physically interact with the mediatized environment 2).

Immersive Arts is located at the intersection formed by the convergence of film, games, music, sound design, and the visual and performing arts and is dedicated to the transdisciplinary artistic exploration of state-of-the-art technology. This contemporary horizon of experience in this field draws upon virtual reality, augmented reality, real-time simulations and the mixing of media and performative practices to enable technology-based research and teaching.
Immersive Arts is related to and overlapping with the fields of Digital Art, Virtual Art, Interactive Art, Sound Art, Pervasive Games, Immersive Theatre and Expanded Cinema.


1) Cf. Grau, Oliver. Virtual Art – From Illusion to Immersion. MIT Press, 2003.
2) Another primary form is story immersion (also: narrative immersion), which can become apparent, for example, when consuming a film or a novel. Empathy and imagination play a pivotal role in this experience. The terminology here can be traced back to media psychology. On story immersion see among others: Lu, Thompson, Baranowski, Buday, & Baranowski, 2012 as well as Suckfüll & Scharkow, 2009. On perceptual immersion or presence see Lombard & Ditton, 1997, Kaplanis, Bech, Jensen, & van Waterschoot, 2014.


Dreams & Dystopia

Photo by Thomas Isler ©ZHdK 2020

Interdisciplinary workshop with BA students (Z-module), Aug/Sept. 2020 

The two-week workshop Dreams & Dystopia (Immersive Landscapes II) focused on the creation of cinematic landscapes (artificial, urban or natural spaces) for three-dimensional media such as spatial projection or virtual reality. Artistic approaches from painting, photography, film and CGI were taught and the differences worked out interactively with the students.

Teaching staff:
Thomas Isler, Department of Fine Arts
Miriam Loertscher, Department of the Performing Arts and Film
Jyrgen Ueberschär, Department of Fine Arts,
Valentin Huber, Stella Speziali, Immersive Arts Space


state:lucid

Photo by Urs Beglinger, ZHdK @2020

Dance, motion-tracking, live projections, sounds and a lot of heavy fog: by means of advanced technology, “state: lucid” combines installational and performative elements with a holistic, immersive experience.

The premiere of Robi Voigt’s diploma project state: lucid took place on November 1st, 2020, with further performances on the following two days, in cooperation with the Swiss Digital Days.

Crew

Robi Voigt – concept, installation, staging, video
Mira Studer – dance, choreography
Stella Speziali – interaction design
Friederike Helmes – costumes, lighting, collaboration installation
David Eliah Bangerter – composition, sound
Stefanie Olbort – dance, choreography
Line Eberhard – the eye from outside

Technical support: Martin Fröhlich, Tobias Baumann, Eric Larrieux, Viktoras Zemeckas, Hans-Jürg Hofman, Matthias Röhm, Lukas Keller, Thomas Utzinger, Michel Weber, Lucien Sadkowski, Dominik Fedier.
 


Floating in Dancing Lights

Photo by Regula Bearth © ZHdK 2020

REFRESH#3, Sept 17, 2020, 19:00, Immersive Arts Space

Floating in Dancing Lights is a mesmerizing dance performance featuring a dancer and a flying swarm of objects, illuminated by means of Spatial Augmented Reality. The performance represents the tentative culmination of an internal project of the Immersive Arts Space which offers a do-it-yourself framework for designing and building remotely controllable helium drones.

Crew:
Martin Fröhlich (project lead), Max Kriegleder (robotics, motion control), Roman Jurt (rapid prototyping, construction), Serena Cangiano (ideation).

Performance:
Denise Lampart, (choreography), Naomi Khamihigashi (dancer), David Peña (assistant to choreographer), Ben Vorhar (costume design), Luca Magni, Eric Larrieux (sound).

Floating in Dancing Lights is part of the artistic research project Helium Drones. – Watch the teaser video.