Designing Enactive Co-Presence with Humanlike Characters in Cinematic Contexts

Talk by Pia Tikka (Enactive Virtuality Lab, Tallinn University)
16:30-17 pm with open Q&A session afterwards at the Immersive Arts Space

The scope of technologies available to filmmakers is expanding and apparently opening new avenues of storytelling. My focus is on the application of new findings in the fields of psychophysiological tracking and machine learning in order to create virtual characters, whose behavior resembles that of humans in the most natural ways. 

In this talk Pia Tikka will share some recent updates in this fast developing domain and discuss their possible applications to the co-presence of human participants and humanlike virtual characters in narrative contexts. This  implies a range of multidisciplinary challenges. The core research question is what types of roles can the filmmaker give to machine learning and psycho-physiological tracking in the process of creating humanlike behaviors in narrative settings. The discussion draws from the holistic embodied approach to the mind,  which in my view provides useful explanatory frames for my claims. The talk aims to inspire discussions related to the use of adaptive artificial characters in the future of virtual storytelling.

Pia Tikka is a filmmaker and EU Mobilitas+ Research Professor at the Baltic Film, Media, and Arts School (BFM), Tallinn University. Her filmography includes international film productions, feature films, and interactive VR installations. The founder of the NeuroCine research group and Enactive Virtuality Lab, she has published on the topics of neurocinematics and enactive media, and written the book Enactive Cinema: Simulatorium Eisensteinense (2008). She is Adjunct Professor of New Narrative Media at University of Lapland, fellow in the Society for Cognitive Studies of the Moving Image, and member of the European Film Academy. Currently, she focuses on biosensor-driven virtual storytelling.

Lab Insights

Shifting realities and hands-on experiences in the Immersive Arts Space. (Norbert Kottmann © ZHdK 2022)

On June 29th and 30th, the Immersive Arts Space presented new developments from current projects. Visitors were invited to experience shifting realities with or without Virtual Reality goggles and to learn more about digital gold in our smartphones.
More about the featured research project Shifting Realities.
More about Kamituga | Digital Gold.


Book Launch: Conflict Minerals INC.

Thursday, June 30th, 2022, 16:30 | Immersive Arts Space, ZHdK

The term “conflict minerals” regroups artisanal tin, tantalum (coltan), tungsten and gold originating from war zones in Central Africa. In his book, Christoph N. Vogel tells the story of how well-intended efforts to solve a global problem have led to white-washing and abetting the continued exploitation of Congo’s resource wealth.

Conflict Minerals INC focuses on a topic that is also addressed by a a joint production of the Immersive Arts Space and the Department of Geography of the University of Zurich. The exhibition Kamituga | Digital Gold allows insights behind the shiny surface of the mobile tech industry and invites the visitors to engage with the concrete challenges and living conditions of artisanal gold miners in the region of Kamituga (Democratic Republic of Congo). [more]

No registration is needed. | Location: Immersive Arts Space, 1.J30, entrance via gate to film studio. > See map here.

Immediately after the book launch, visitors can take part in the LabInsights: The Immersive Arts Space presents new developments from current projects. [more]


Lecture by Takashi Ikegami

Tuesday, June 28th, 2022, 17:15–18:30 | Kino Toni, ZHdK

Abstract:
Even now, I am often asked as a scientist”Why do you also do art projects? I wonder why. When I do research on complex systems, there is a lot of fascinating data that I can’t reproduce (I forget the initial values and parameters) when I do computer-based simulation experiments. Science is about reproducibility, and if you do it 10 million times, you want the same event to happen 9999 times; if it happens once in 10,000 times, that’s not science, that’s art. I consider that once-in-a-lifetime interest to be art. This talk will introduce a series of works created since 2005 with a range of collaborators including the composer Keiichiro Shibuya, sound artist evala, and photographer Kenshuu Shintsubo, as well as recent works based on a custom designed android called Alter in order to discuss the meaning, possibility and challenges of the intersection between the arts and the sciences.

Takashi Ikegami is Professor in General Systems Science in the Graduate School of Arts and Sciences at The University of Tokyo and principle investigator of the Ikegami Lab. He received his PhD in physics from The University of Tokyoand is internationally known for his contributions to the development of complex systems science and artificial life. Some of his results have been published in Life in Motion (Seidosha, 2007) and Between Man and Machine (Kodansha, 2016). He has also been active 2005 in the arts with works such as “Filmachine” (with Keiichiro Shibuya, YCAM, 2006), “Mind Time Machine” (YCAM, 2010), “Long Good bye” (with Kenshu Shimpo, Japan Alps Festa, 2017), “Offloaded Agency” (Barbican, 2019), among many other.

No registration is needed.
Kino Toni, ZHdK, Pfingsweidstrasse 96, 8005 Zürich > See map here.


Exhibition: Planet Digital

February 11th – June 6th  2022| Museum für Gestaltung, Zürich

 Overview of a complete 3D model of an artisanal gold mine in Kamituga based on a 3D scan with a smartphone. 3D scan: Gabriel Kamundala, 3D model: Chris Elivis Leisi, ZHdK ©2022

The exhibition Planet Digital featured a joint production of the Immersive Arts Space and the Department of Geography of the University of Zurich. The exhibition segment KAMITUGA | DIGITAL GOLD allowed insights behind the shiny surface of the mobile tech industry and invited the visitors to engage with the concrete challenges and living conditions of artisanal gold miners in the region of Kamituga (Democratic Republic of Congo). About seven percent of global gold is used for technological purposes, as a chemically stable and exceptional conductor. It is found in all our smartphones, tablets and computers.

The starting point for all digitally-based experiences were the photographs, videos and 3D scans that PhD candidate Gabriel Kamundala recorded in Kamituga in the summer of 2021 using a latest-generation smartphone. In this way, a ubiquitous device, itself containing gold and rare minerals, provided insights into the living conditions of artisanal mine workers and, at the same time, drew attention to the many challenges of the mobile tech industry.

Teaser video: Kamituga Dgital Gold (Alan Sahin / Christian Iseli © ZHdK 2022)

Visitors also encountered texts about the global context. With the help of QR codes, they could view supplementary information on their smartphones. Thus, awareness of both how our digitalized lives are dependent on rare minerals and of the problematic interrelation of supply chains in the mobile tech industry was promoted.

[ Planet Digital website ] [Arte report about the exhibition]

Public Lecture Series (Autumn 2022)

This lecture series investigates the concept of immersion in art and design from aesthetic, historical and political perspectives. Guest lecturers come from digital arts, curating, media studies, and the histories of science and technology from the ZHdK, Royal Academy of Fine Arts Antwerp, Rensselaer Polytechnic Institute (New York), Haus der Elektronischen Künste (HEK, Basel) and the Technical University in Dresden. Topics will include contemporary artistic practices in multi-sensory immersion, extended reality and history of VR and the arts, and art and artificial intelligence.

Christopher Salter | September 27th, 2022 | 17:15-18:30
THE ARTS OF IMMERSION: SENSING, BODIES AND RESPONSIVE ENVIRONMENTS 

Kristof Timmerman | October 11th, 2022 | 17:15-18:30 
BREAKING THE FIFTH WALL: IMMERSION BETWEEN LIVE PERFORMANCE AND VIRTUAL SPACES

Sabine Himmelsbach | October 25th, 2022 | 17:15-18:30 
VIRTUAL WORLDS-INTERACTIVE ENVIRONMENTS: ART AS AN IMMERSIVE EXPERIENCE

Michael Century | November 8th, 2022 | 17:15-18:30 
CONTESTING A “NEW MEDIUM”: VIRTUAL REALITY AS CULTURAL PROBE

Orit Halpern | November 22nd, 2022 | 17:15-18:30 
THE SMARTNESS MANDATE: AI AND UBIQUITOUS COMPUTING`S IMPACT ON ART AND DESIGN

The lecture series will take place at the Kino Toni (3.G02), ZHdK. See on the [here]


REFRESH#4

The REFRESH#4 conference on Design, Arts and Technology took place from November 10th to 14th 2021. The fourth edition focussed on the topics «Immersive Journalism», «Immersive Play», «Digital Identities», «Digital Doubles» and «Techno-Biological Futures». The conference featured keynote speeches including panel discussions with national and international guests as well as a large exhibition have been part of the conference. 

Photographs of the conference have been published and can me viewed in this gallery. All photo credits are with Regula Bearth, ZHdK 2021.

Recap Video by Marco Bach ZHdK©2021

The Immersive Arts Space hosted the Lab Insights and a VR exhibition. Team members offered insights into the projects Shifting Realities (explorations within the intersection of reality and virtuality), Neural Volumetric Capture (experimental methods for capturing photorealistic 3D models), Digital Twins (development of animated digital avatars based on photogrammetry and Meta Humans) and cineDesk (a real-time collaborative simulation tool for the development of film scenes, VR experiences and games in 3D spaces). 

Credits:
Multiuser VR: Chris Elvis Leisi, Oliver Sahli
Helium Drones/ SAR: Martin Fröhlich, Max Kriegleder,
Joel Gähwiler, Roman Jurt, Mariana Grüning
Volumetric Capture: Florian Bruggisser
cineDesk: Norbert Kottmann, Valentin Huber
Sound Design: Luca Magni
Lights: Sébastien Schiesser
Project Lead: Martin Fröhlich
Production: Kristina Jungic
Head of Unit: Christian Iseli


Emerging Talents: Stella Speziali & Chris Elvis Leisi

ZHdK Talent Talk in the context of Swiss Digital Days

Stella Speziali is research associate in the Immersive Arts Space and will talk about her research project on the topic of digital humans.

Thanks to technological advancements the creation of virtual characters is more accessible than ever. New time-saving platforms have been created, thus democratising the access to high-fidelity, real-time, fully-rigged, diverse, portable, 3D human characters.
With My Digital Twin, I wish to understand better the creation and deployment of digital humans with different critical approaches. Furthermore, I am ethically questioning their applications in entertainment, marketing, art, and beyond.

Chris Elvis Leisi, research associate in the Immersive Arts Space and a graduate of the ZHdK Game Design program, talks about his graduation project Virtual Real World.

In today’s VR games, the body often serves as the controller. However, when the player enters the virtual world, the connection to the physical environment is often lost. This master’s thesis deals with immersion mechanics in Virtual Reality and reveals the potentials that arise when one’s own home can be integrated into the virtual world as a play area.

The ZHdK talent talk took place on November 4th at the Kino Toni.
Speaker: Stella Speziali (en), Chris Elvis Leisi (de)
Host: Christian Iseli