SNSF-Project: Virtually Real (2018-20)

At the shoot of the short film LUX (Director: Wendy Pillonel, DoP: Ramón Königshausen) at the ZHdK studio. (Photo by Christian Iseli, ZHdK 2019)

The research project “Virtually Real – Aesthetics and the Perception of Virtual Spaces in Film” is the first IASpace project to be financed by third-party funds. The project concerns itself with the increasing virtualisation of film production, focussing on the transition to 3D recording of real environments and objects using laser scanning and photogrammetry. For a comparative study, short feature films will be recorded both virtually (in previously scanned 3D spaces) and conventionally (in the corresponding real spaces). The film variants are used to investigate the effects on perception and changes in work processes.

Studio shoot in front of green screen with real-time display of the scanned location (compare top center with the monitor on the bottom right). Project: LUX by Wendy Pillonel. (Photo by Christian Iseli, ZHdK 2019)

Project lead and principal investigator: Prof. Christian Iseli. Co-applicant: Dr. David Weibel (Institute of Psychology, University of Bern). Researchers: Tom Gerber, Wendy Pillonel, Miriam Loertscher, Martin Fröhlich, Valentin Huber. Project partners: Michael Schaerer, Max Rheiner. Industry partners: InstaLOD GmbH, Stuttgart; Leica Geosystems, Heerbrugg. Funded by the Swiss National Science Foundation (SNSF) / Call: Digital Lives

 


Home in the Distance (2019)

Writer and director: Andreas Dahn, Filmmaker in Residence

Still image from Home in the Distance (2019)

Home in the Distance is based on a short story, written by director and vfx artist Andreas Dahn when he was about 17 years old. In his residency at the Immersive Arts Space he turns his story into a 3D animated short film.

In contrast to the traditional animation and vfx workflow, which is heavily based on storyboarding and previsualization, the conscious decision was made to leave out these steps, in order to find out if new technologies make it possible to create a more emotional and spontaneous film experience.  

The character was sculpted by hand with dough and photogrammetry was used to create the virtual model. Rigging was done with Human IK in Autodesk Maya. Students from the 3D Akademie Stuttgart helped to model the set and props. In addition, an actor`s performance was motion-captured with OptiTrack in the IASpace at the Zurich University of the Arts (ZHdK). The camera work will be done with a VR tool, developed by Mirko Lempert (Stockholm University ot the Arts) and Simon Alexandersson (Royal Institute of Technology, Stockholm) and rendered in realtime in Unity.

Andreas Dahn (writer and director) on the left, with Pascal Holzer (actor) on the right, during MoCap recording.

Crew at ZHdK
Cast: Pascal Holzer
Mocap coaching: Corinne Soland
Mocap recording: Norbert Kottmann
Further ZHdK support: Valentin Huber, Robin Disch, Marco Quandt, Lucien Sadkowski, Andreas Birkle, Chantal Haunreiter, Claudia Hürlimann, Thomas Gerber, Stefan Jäger,  Martin Fröhlich

ZHdK Residency
The residency at the Zurich University of the Arts has been made possible by the Ernst Göhner Foundation, Switzerland, as well as by additional support of the ZHdK film program (led by Sabine Boss) and by the Immersive Arts Space (led by Prof. Christian Iseli).

Crew at 3D Akademie Stuttgart
Props & set modeling & texturing: Gerrit Gaietto, Katharina Rodak, Kimberly Niesner Pierre Urbanek (Head of 3D Akademie)

Further crew members
Unity VR support: Mirko Lempert, Simon Alexanderson
Junior producer: Jana Günther
Script: Aimée Torre Brons
Screenplay translation: Karen Ma
Special thanks: David Maas, Renate Schirrow, Ella Steiner, Felix Bucella, Alireza Sibaei, Astrid Weitzel 


Dancing Digital (2019)

Dancing Digital performance on September 26th, 2019
(Photo by Davide Arrizoli / ZHdK ©2019)

«Dancing Digital» is an exploratory workshop connecting real dancers and virtual characters. The four-day workshop focuses on the interplay of presence and absence and will result in a prototype performance in the Immersive Arts Space.

Together with visual artist Tobias Gremmler, choreographer Nadav Zelner and set designer Mariana Vieira Gruenig, students of the Contemporary Dance program at the Zurich University of the Arts will explore a modular stage design that allows for a high degree of artistic versatility.

The prototype presentation took place during the REFRESH #2 conference on Thursday, September 26th.

Dancing Digital performance on September 26th, 2019
(Photo by Davide Arrizoli / ZHdK ©2019)

Dancers: Freeda Electra Handelsman, Rabii Hadane, Aonghus Hode, Pornpim Karchai, Francesca Lapadula, Liza Lareida, I-Fen Lin, Rozemarijn Louwerse, Lara Müller, Stefanie Olbort, Elena Paltracca, Lucas del Rio Estevez, Kristin Tims, Suzanne Vis
Visual artist: Tobias Gremmler 
Choreographer: Nadav Zelner
Assistant choreographer: Denise Lampart
Set designer: Mariana Vieira Gruenig
Augmented projection artist: Martin Fröhlich 
Motion capture: Tobias Baumann, Norbert Kottmann, Corinne Soland, Benjamin Thoma
Sound: Eric Larrieux, Hans-Jürg Hofmann
Production manager: Kristina Jungic 
Project lead: Christian Iseli


Helium Drone Project (2019)

The Helium Drone project seeks to explore the aesthetic properties of installations based on floating devices that can autonomously navigate in space. In its complete state it will be a flocking swarm of diverse drones in different forms and shapes, each behaving in its individual way and interact with each other and the spectators on the ground. It will use the tracking system for location and navigation and the projection mapping system to wrap them into dynamically created ‘skins’.

A technological groundwork is laid out, with the goal to design a framework to quickly prototype and build helium drones with different shapes and propulsion concepts that integrate easily with the current technical setup of the IASpace. This involves the research for suitable materials and processes to build the floating volume, design of mechanical structures, electronics and network protocols, the evaluation and testing of motor and servos and the integration of motion and sensor data for autonomous flying capabilities (among many other things).

Involving many different skills, from designing and building physical structures to programming behaviors and creating interactive textures that are projected onto the creatures, it will be attractive to a wide spectrum of disciplines and a playground to experience multidisciplinary teamwork. There will also be an open source toolkit with instructions that can be made accessible to the public.

CREW

Project lead, motion tracking, projection mapping: Martin Fröhlich
Robotics, motion control, sensor integration: Max Kriegleder
Micro-controller, networks, protocols: Joel Gähwiler
Materials, rapid prototyping, construction: Roman Jurt
Theory: Serena Cangiano


TwinLab (Pilot, 2018)

TwinLab Performance, October 25th 2018. Photo by Regula Bearth, ZHdK ©2018

For several years now, the ZHdK has maintained a lively exchange with cultural and educational institutions in Hong Kong. On the occasion of an official visit to the ZHdK from the head of government of Hong Kong, Chief Executive Carrie Lam in January 2018, temporary partner labs were set up in both cities. The goal was to promote interdisciplinarity within a framework of research and development into technology-supported arts.
The resulting TwinLab project is premised upon the availability of the same technical infrastructure at both locations. The cooperation can thus be based primarily on data exchange and be sustainable. Real-time transmissions allow for synchronous experiments and productions.

The first TwinLab project “The Hidden Formula – The Heavenly Palace” was developed in cooperation with the experimental theatre group Zuni Icosahedron which is based in Hong Kong and the two performances could be experienced simultaneously in both locations. The movements of the dancers were recorded at both locations using motion capture technology to create real-time visualizations in a video projection. Despite the geographical distance of over 9000 kilometres and a six-hour time difference, the motion data was transmitted in real time, thus enabling virtual interaction.

Based on Plato’s cave parable, the ZHdK performance “The Hidden Formula” explored mechanisms of suppression and restriction as well as the human will to be free. The piece was made in close dialogue with “The Heavenly Palace” by Zuni Icosahedron, which refers to the 16th century Chinese novel A Journey to the West.
The live performance of “The Hidden Formula” took place on 25 October at the ZHdK in Zurich and simultaneously at the Community Center in Hong Kong and was broadcast live in Zurich’s Bahnhofshalle as part of the Swiss Digital Day.

For data transmission, the TwinLab project relied on the findings and technological developments of the SNF research project “Telematic Performance”, which is housed at the Institute for Computer Music and Sound Technology (ICST), ZHdK.

Cast & Crew of the ZHdK
Dance: Sophie Bertschy, Alex Ferro, Denise Lampart, Cary Shiu
Choreography: Denise Lampart
Visual Art: Tobias Gremmler
Music: Thierry de Mey
Digital set artist: Martin Fröhlich
Technical crew: Simon Broggi, Martin Fröhlich, Norbert Kottmann, Eric Larrieux, Olav Levrik, Hansruedi Näf, Marc Nathmann, Matthias Röhm, Marco Quandt, Viktoras Zemeckas and many others.
Production management: Andreas Birkle, Kristina Jungic
ZHdK producers: Chantal Haunreiter, Christian Iseli
Project steering: Hartmut Wickert
Project lead: Christian Iseli

Cast & Crew Zuni Icosahedron, Hong Kong
Dance: Chang Yu-chau, Rady Nget
Artistic direction: Danny Yung, Mathias Woo
Digital art: Dan Fong, Benny Woo
Lighting design: Mak Kwok Fai
Music: Steve Hui
Technical assistance: Dan Fong, Benny Woo, Lee Kin Tak, Tso Chi Yan, Ha Yan Pui, Ko Man Kit and many others
Production: Carmen Cheng, Chun Yin Chow, Satina Shum, Leung Kin Kai
Project management: WyWong Yuewai, Cedric Chan


Happy Robots (Pilot, 2018)

Motion Capture Shooting for the film HUMAN RESOURCES. (Photo by David Oesch, ZHdK 2018)

Students from the ZHdK departments of Film, Contemporary Dance, Game Design, Interaction Design, Sound Engineering and Composition (for Film, Theatre and Media), together with students studying Animation at the Lucerne University of Applied Sciences and Arts, realised individual projects using the “Motion Capture” technique. Their projects were linked together in a common story world.
The Happy Robots pilot project was presented on 22 September 2018 as part of ZHdK Highlights. In addition to seeing films and dance performances, visitors could immerse themselves in the world of Happy Robots through gaming and an augmented reality exhibition.
For the individual projects within the Happy Robots story world, please refer to the to the special Showcase page.
As a pilot project, Happy Robots helped evaluate and plan for the ZHdK-wide Immersive Arts Space, through which the artistic examination of new technology in research and teaching is promoted.

MoCap explorations. (Photo by Martin Fröhlich / ZHdK 2018)
Project lead: Michael Schaerer
Lecturers: René Bauer, Sven Bergmann, Georg Bringolf, Manuel Gerber, Tom Gerber, Olav Lervik, Julian Oberbeck, Max Rheiner, Michael Schaerer, Christian Schwaller
Technical support & programming: Martin Fröhlich, Norbert Kottmann, Marco Quandt, Simon Broggi, Ignasi Duelo, Simon Pfaff, Luca Tomasi
Production manager IASpace@Highlights: Andreas Birkle
Production assistant: Kristina Jungic
 
Dance rehearsal with live visuals. (Photo by Martin Fröhlich, ZHdK 2018)

Virtual Production (Pilot, 2018)

Virtual Production Workshop at  the IASpace 2018. (Photo byDavid Oesch, ZHdK)

Another IASpace pilot project focuses on the question of virtual production. Spaces and objects are captured as 3D models with laser scanning and photogrammetric measurements, which are then used for real-time simulations. Creative teams can thus pre-visualize and test audiovisual ideas before the actual film production begins. Real-time simulations are an important part of the future direction of IASpace.

A proven expert in previsualization and virtual production, the film producer Mirko Lempert joined the IASpace team  in the autumn semester of 2018. Mirko was recruited as Artist in Residence at the ZHdK. He also is an assistant professor at Stockholm University of the Arts.

The ‘Previs Table’ is used as a communication tool in pre-production and enables real time previsualizations of film scenes. (Bild: David Oesch, ZHdK)

Crew
Project lead: Christian Iseli
Team: Mirko Lempert (University of the Arts, Stockholm), Simon Broggi, Martin Fröhlich, Valentin Huber, Norbert Kottmann, Simon Pfaff, Max Rheiner, Michael Schaerer

3D model of the Toni areal based on photogrammetry and 3d laser scanning. The model is used for real time simulations for lighting or scene blocking. (Screen shot by Valentin Huber, ZHdK 2018)