Floating in Dancing Lights

REFRESH#3, Sept 17, 2020, 19:00, Immersive Arts Space

Floating in Dancing Lights is a mesmerizing dance performance featuring a dancer and a flying swarm of objects, illuminated by means of Spatial Augmented Reality. The performance represents the tentative culmination of an internal project of the Immersive Arts Space which offers a do-it-yourself framework for designing and building remotely controllable helium drones.
In a workshop prior to the conference REFRESH #3, a group of students will build their own helium drones and explore different aspects of the framework – from sculpting their own balloon shapes to choosing an appropriate propulsion technology to designing a scenography and light setting with the help of projection mapping. Curated by Martin Fröhlich and Denise Lampart, some of the results of the workshop will become part of the performance.

Crew:
Martin Fröhlich (project lead), Max Kriegleder (robotics, motion control), Roman Jurt (rapid prototyping, construction), Serena Cangiano (ideation).

Performance:
Denise Lampart, (choreography), Naomi Khamihigashi (dancer), David Peña (assistant to choreographer), Ben Vorhar (costume design), Luca Magni, Eric Larrieux (sound).

Floating in Dancing Lights is part of the artistic research project Helium Drones.

For tickets, please refer to the Refresh-website: https://refresh.zhdk.ch


The Umbrella Project

Photo by Stefan Dux, © ZHdK 2020

The Umbrella Project is an ongoing research project that explores the use of 3D audio and projection mapping to achieve a sense of immersion without isolating participants from the real world; essentially, enabling an imaginary fantasy world come to life in our own. We employ multiple levels of 3D audio and projection mapping (both directly within and on the umbrella, as well as throughout the room itself) in order to transport the participant into this virtual world.

The end goal of the project is to create a series of navigable compositions in the form exploratory sonic worlds, as well as intereactive experiences where the participants’ behaviours (relative to each other and the world) shape the sonic and visual environment. Furthermore, we are investigating sonic and visual paradigms where the umbrellas can function both as objects that exist in (and can interact with) the virtual world, as well as windows into these other worlds.

Naturally, these environments are best experienced from directly underneath the umbrella, where one can best appreciate the various levels of MR.

At the REFRESH conference, there will be short performance demonstrating some of the early results as a teaser of what is to come: Friday, Sept 18th, 19:30h at the Immersive Arts Space.

Crew:
Eric Larrieux (lead), Stella Speziali, Martin Fröhlich, Corinne Soland, Mariana Vieira Grünig


Kusunda

Filmmaker in Residence Gayatri Parameswaran

KUSUNDA is a virtual reality documentary experience about what it means to lose a language and what it takes to keep one alive.

Narrated by two of its co-creators — 86-year-old Kusunda shaman Lil Bahadur and his 15-year-old granddaughter Hema — the experience contrasts two generations set apart by their lifestyles and brought together by the struggle for their indigenous identity.

In the VR experience, you join Hema as she reminds her grandfather of his forgotten mother tongue. You navigate by speaking words in the endangered Kusunda language and join an audible fight against its extinction.

Most of Lil Bahadur’s story happens in the past — especially his life in the forest as part of a hunter-gatherer group. The recreation of this non-existent past lends itself naturally to the use of animations within virtual reality.

With the help of the motion capture facilities at the Immersive Arts Space at the Zürcher Hochschule der Kunst (ZHdK), actors were recorded recreating these sequences from Lil Bahadur’s past. This not only simplifies and speeds up the process of character animations but also offers unique possibilities for the documentary storytelling form.

Crew at ZHdK:
Cast: Offir Limacher, Johannes Voges, Ferhat Türkoğlu, Liliana Heimberg, Corinne Soland
Mocap coaching and production: Corinne Soland
Mocap recording: Tobias Baumann
IASpace producer: Kristina Jungic
Further ZHdK support: Chantal Haunreiter, Martin Fröhlich, Stella Spezialli.

ZHdK Residency
The residency at the Zurich University of the Arts has been made possible by the Ernst Göhner Foundation, Switzerland, as well as by additional support of the ZHdK film program (led by Sabine Boss) and by the Immersive Arts Space (led by Prof. Christian Iseli).

Further crew members:
Co-creators: Gyani Maiya Kusunda, Hema Kusunda, Lil Bahadur Kusunda; Storytelling/Production coordination: Felix Gaedtke, Gayatri Parameswaran; Executive Producer: Rene Pinnell; Associate Producer: Mia von Kolpakow; Co-Producers: Emma Creed, Aliki Tsakoumi, Sönke Kirchhof, Philipp Wenning, Kuan-Yuan Lai; Lead Developer: Tobias Wehrum; Art Director, 3D designer & animator: Moritz Mayerhofer (and team); Volumetric Video post-processing: INVR.SPACE; Photogrammetry post processing: realities.io; AI speech recognition: Valerio Velardo; Sound designer: Mads Michelsen; Project website: Tom Lutherburrow | Nepal Production team: Direction/Production: Felix Gaedtke, Gayatri Parameswaran; Line Producer Nepal: Deepak Tolange, Sandeep Bhaju; Volumetric Video & Photogrammetry: Felix Gaedtke, Gayatri Parameswaran; DoP and Drone pilot: Aditya Thakuri; Sound recordist: Mia von Kolpakow; Kusunda linguistics researcher: Uday Raj Aale; Driver: Dharmendra Shakya


Virtual Production

Interview with Christian Iseli about the Immersive Arts Space with a special focus on virtual production and film. Recorded for the 2020 online edition of NIFFF (Neuchâtel International Fantastic Film Festival), aired on July 6th, 2020. © ZHdK 2020

BATVISION

© ZHdK 2020

«Have you ever wondered how a bat perceives the world?» BATVISION offers the chance to playfully explore how bats use echo-location to detect their surroundings. The VR-experience visualizes the bat’s auditory sensation and makes it more tangible. Surrounded by complete darkness, the virtual world only becomes visible when the users start shouting. BATVISION simulates the ultrasonic navigation of bats, enables new forms of perception and raises the awareness for an endangered species.

BATVISION is a collaboration between the ZHdK Industrial Design program and the Immersive Arts Space. In their bachelor thesis Eliane Zihlmann and Raffaele Grosjean developed the concept of the VR experience and the associated hardware design. They were supported by IASpace staff members Oliver Sahli (programming, visual implementation, interaction control), Chris Elvis Leisi (implementation of multi-user functionality) and Florian Bruggisser (3D scanning and point-cloud processing).


Co-located Multi-User VR

Photo by Stefan Dux © ZHdK 2020

The aim of the project is to develop a versatile co-location multi-user system, with which several users can dive into virtual worlds together in the same room. The system can be used for many different areas. As a multiplayer game in which the users feel even closer to their fellow players through increased physical presence. As a virtual film set to try out camera movements or interactions of actors without much effort. Or as an interactive experience that transports the user into a new dimension of immersion.

An iPad also serves as a window into the virtual world. The user can use this virtual camera to move through the room and in some cases even interact with the virtual world. The virtual world can also be placed on an interactive screen. This gives outsiders the opportunity to change the entire scene.

The co-location multi-user system can be used wherever the headsets can connect to a network thanks to the Mobile Vr-Headsets. The system is developed by IA-Space staff members Chris Elvis Leisi and Oliver Sahli.


ZFICTION.20 – Virtual Production

Theater der Künste, Zurich (postponed due to corona crisis)

With real-time simulations, Virtual Production enables playful access to fictional narrative forms, can make new worlds visible and promotes creative collaboration. The increasingly cost-effective tools lead to a democratization of previously exclusive work processes and are now also coming within the reach of European and Swiss productions.

The first edition of the feature film conference ZFICTION presents the versatile promises of the new tools for discussion. Filmmakers and researchers* will explore the question of how virtual production will change the near future of fictional storytelling in film and what new challenges this will bring.

More info: www.zfiction.ch
Contact: zfiction.info@zhdk.ch


Presence&Absence

Augmented projection mapping with virtual characters

Photo by Davide Arrizoli, ZHdK ©2019

This artistic research project focuses on the interplay of presence and absence of real dancers and virtual characters. It is based on augmented projection mapping, motion capture and movable stage elements. Dancers disappear behind stage elements while their avatars are projected on these elements. When the dancers step from behind the elements, their virtual characters vanish immediately. This principle is varied, when the body of a dancer is only partially hidden by an element. In this case, the spectators witness a figure who is half avatar and half human being.

Further variations of presence and absence are made possible with stage elements that allow dancers to walk or jump through the walls made out of elastic ribbons. Thus, the avatars appear immediately when the dancers are covered by the ribbons and vice versa.

Photo by Christian Iseli, ZHdK ©2019

The technical setup includes a motion capture system with a large tracking space that also covers non-visible areas of the stage. Furthermore, a projection mapping system with multiple projectors and a performative 3D mapping software is needed as well as game engine that guarantees real time performance of up to eight virtual characters.

#Keywords: Motion capture, projection mapping, virtual characters, real time rendering, game engine, modular stage elements, dance performance.

The project ‘Presence and Absence’ is connected to the workshop and performance Dancing Digital.

Live-Performce ‘Dancing Digital’, Sept. 26th, 2019. Photo by Davide Arrizoli, ZHdK ©2019

Research team:
Visual artist: Tobias Gremmler
Set designer: Mariana Vieira Gruenig
Augmented projection artist: Martin Fröhlich
Motion capture & Unity: Tobias Baumann, Norbert Kottmann, Chris Elvis Leisi, Oliver Sahli
MoCap coaching: Corinne Soland
Project manager: Kristina Jungic
Performers: Chantal Dubs, Aonghus Hode, Svenja Koch,Lucas del Rio Estevez, Johannes Voges
Project lead: Christian Iseli

Immersive Landscapes

Photo by Christian Iseli © ZHdK 2019

Interdisciplinary workshop with BA students , September 2019 (Z-module)

In painting, photography and film, landscapes are often transformed into dream images and stylized into archetypes. Landscapes are contemporary witnesses of collective longings and dystopias. With the new possibilities of virtual reality and immersive media, they can become overwhelming media experiences. In the workshop, real landscapes or self-built miniature landscapes were captured and prepared for VR experiences or spatial projection mapping. In both formats, the results appeared in the three-dimensional space.

Project lead: Thomas Isler, Miriam Loertscher
Lecturers: Jyrgen Ueberschär, Norbert Kottmann, Simon Peter Pfaff, Martin Fröhlich
In cooperation with the Immersive Arts Space.
Students: Giulia Hess, Yangzom Sharlhey, José Manuel Zacate Lizárraga, Aylin Cagri Acikel, Nemo Bleuer, Suphansa Buraphalit, Yvonne Haberstroh, Sonjoi Nielsen, Vanja Victor Tognola, Danuka Ana Tomas, Flavia Trachsler.