Immersive Imaginaries: Aesthetics/Politics/Practices

This lecture series investigates the concept of immersion in art and design from aesthetic, historical and political perspectives. Guest lecturers come from digital arts, curating, media studies, and the histories of science and technology from the ZHdK, Royal Academy of Fine Arts Antwerp, Rensselaer Polytechnic Institute (New York), Haus der Elektronischen Künste (HEK, Basel) and the Technical University in Dresden. Topics will include contemporary artistic practices in multi-sensory immersion, extended reality and history of VR and the arts, and art and artificial intelligence.

Christopher Salter | September 27th, 2022 | 17:15-18:30
THE ARTS OF IMMERSION: SENSING, BODIES AND RESPONSIVE ENVIRONMENTS

Kristof Timmerman | October 11th, 2022 | 17:15-18:30
BREAKING THE FIFTH WALL: IMMERSION BETWEEN LIVE PERFORMANCE AND VIRTUAL SPACES

Sabine Himmelsbach | October 25th, 2022 | 17:15-18:30
VIRTUAL WORLDS-INTERACTIVE ENVIRONMENTS: ART AS AN IMMERSIVE EXPERIENCE

Michael Century | November 8th, 2022 | 17:15-18:30
CONTESTING A “NEW MEDIUM”: VIRTUAL REALITY AS CULTURAL PROBE

Orit Halpern | November 22nd, 2022 | 17:15-18:30
THE SMARTNESS MANDATE: AI AND UBIQUITOUS COMPUTING`S IMPACT ON ART AND DESIGN


THE ARTS OF IMMERSION: Sensing, Bodies and Responsive Environments

The history and practices of “immersion” in the arts has long focused on the senses being transformed through melding them with technologies embedded into the actual physical world. As the French actor and theater writer Antonin Artaud wrote in 1938, the theater would be a virtual reality (réalité virtuel) – a doubling or “stand in” for reality. But now, the next wave of immersion seeks the opposite: to capture the senses in order to render a synthetic world that is “realer” than the physical one. In the words of computer graphics pioneer Ivan Sutherland who invented the first head mounted artificial reality display in 1965, such an “ultimate display” would need to “serve as many senses as possible.” Thus, contrary to the idea that the senses are simply to be replaced by the prosthetics of artificial sensors, a different story seems to be emerging. Our senses are needed to drive and feed ever-new immersive experiences by being interfaced to the simulated world that we increasingly inhabit. Using my own artistic work as well as historical examples, this talk will give a critical historical and practice-based introduction and overview of the HS 2022 Immersive Arts Lecture Series.

Christopher Salter (USA/CH) is an artist, Professor of Immersive Arts and Director of the Immersive Arts Space, ZHdK. He is also Professor Emeritus, Design and Computation Arts at Concordia University in Montreal and from 2014-2022, was Co-Director of the Hexagram network for Research-Creation in Media Arts and Technology, also in Montreal. His work has been seen all over the world at such venues as the Venice Architecture Biennale and Barbican Centre among many others. He is the author of Entangled (2010), Alien Agency (2015) and Sensing Machines (2022), all published by the MIT Press.


BREAKING THE FIFTH WALL: Immersion between live performance and virtual spaces

How can virtual performances draw spectators and performers into the virtual while connecting them in an immersive experience? Immersion is of all times and of all arts. Where different arts come together, a complete experience emerges, often bridging the traditional boundaries between spectators, medium and character. These performances balance on the border between performance, video and installation art. They appeal to all of the senses and explore spatial relationships, placing the spectator at the center of this sensory game. The fourth wall – the imaginary wall at the end of the stage between the audience and the performance space – has come down. But what if we take the spectator one step further? What if the spectator becomes the story, as if we, the creators, were the directors of one’s life? And what are the parameters that we must define, control and manipulate in order to provide such a complete experience? In both a presentation of his own artistic work, as well as compelling examples of other digital artists, researcher and director Kristof Timmerman attempts to map out the complexity and enormous potential of live performances within virtual environments.

Kristof Timmerman (BE) is a designer and director of digital performances and installations, working in the field of live, interactive digital environments and virtual reality. He worked for several theater companies, including the experimental CREW. In 2006 he founded the digital artist collective studio.POC Kristof is the chair and coordinator of MAXlab, the research group on the interaction between art and digital technology at the Royal Academy of Fine Arts Antwerp. He is also a teacher and frequently consulted coach for VR productions and digital storytelling.


VIRTUAL WORLDS-INTERACTIVE ENVIRONMENTS: Art as an Immersive Experience

Mobile and extended reality technologies are opening new spaces and interaction possibilities for us. The immersion in virtual worlds by means of VR or the inclusion of the outside world in AR, in which the real world is overlaid with virtual images both open new perceptions, creating an alterity of our living world. Immersive image worlds are conquering the art world and these experiences are becoming a new field of action, as the exhibitions of the Japanese collective teamLab or the enterprise Superblue, a new branch of the New York based Pace Gallery, show. Sabine Himmelsbach will speak about these current developments and present examples from her own curatorial work at HEK (House of Electronic Arts), Works discussed range from interactive installations to AI, game environments or video installations which challenge, provoke, and explore how technology is representing, influencing, and changing our world.

Sabine Himmelsbach is director of HEK (House of Electronic Arts) in Basel. Trained in art history, she was project manager for the Steirischer Herbst Festival in Graz and in 1999 became exhibition director at the ZKM | Center for Art and Media in Karlsruhe. From 2005–2011 she was the artistic director of the Edith-Russ-House for Media Art in Oldenburg, Germany. In 2022 she curated Earthbound – In Dialoge with Nature for the European Capital of Culture Esch-sur-Alzette in Luxembourg. As a writer and lecturer, she is dedicated to topics related to media art and digital culture.


CONTESTING A “NEW MEDIUM”: Virtual Reality as Cultural Probe

The Art and Virtual Environments Project, produced in Canada at The Banff Centre for the Arts between 1991-1994, is one of the earliest large scale artistic initiatives employing Virtual Reality. Now twenty-eight years old, its organizational structure and political context, as well as the efforts, only partially realized, to integrate critical-theoretical concerns from the humanities and social sciences within an intensive experimental development and exhibition-driven production cycle are important for contemporary initiatives. From his perspective as one of the organizers, media theorist and musician Prof. Michael Century reconstructs the intent and actualization of the project’s three-part design: a transdisciplinary three-month residency of artists, engineers and critical theorists, with parallel rapid prototyping using low-end VR equipment establishing hardware and software requirements for a high-end production lab; an onsite and virtual seminar and publication informing an open call for full-scale commissions; a final production and exhibition phase including 10 completed artworks and critical analysis. Century will expose the challenges encountered and successes achieved in this effort at transdisciplinary creative and intellectual production, set against shifting Canadian policy priorities in the late millennium and the turn away from the immersion-intensive mythos of early VR to the network-centric concerns of the early internet.

Michael Century, pianist and composer, is Professor of New Media and Music in the Arts Department at Rensselaer Polytechnic Institute, which he joined in 2002. Prior to joining Rensselear, he worked as an new media researcher, inter-arts producer, and arts and technology policy advisor (Banff Centre for the Arts (1979-93), McGill University (1998-2002), Government of Canada (1993-98)). He is the author of Northern Sparks: Innovation, Technology Policy and the Arts in Canada from Expo 67 to the Internet Age.


THE SMARTNESS MANDATE: AI and Ubiquitous Computing’s Impact on Art and Design

Smart medicine. Smart homes. Smart cities. Smart Grids. This talk traces a unique genealogy of the natural and human sciences in relationship with art and design to demonstrate how the long historical imperative to make our world ever smarter through ubiquitous computing, artificial intelligence, and perpetual “learning” has become the new logic of our present.  As Halpern, a historian of science with broad interests in the social-cultural impacts of technology will demonstrate, the mandate for ubiquitous and immersive “smartness” has changed the way we understand the very nature of society, economics, culture and environment for both better and worse. In other words, the smartness mandate is central to how we design and envision the future.

Orit Halpern is Lighthouse Professor and Chair of Digital Cultures and Societal Change, Technische Universität Dresden. Her work bridges the histories of science, computing, and cybernetics with design. She is the author of Beautiful Data (Duke UP 2015) and The Smartness Mandate with Robert Mitchell (forthcoming). As part of her work, she is also interested in digital cinema and multimedia documentary, architecture and design, contemporary art practice, animation, and literature.


Kamituga | Digital Gold @ Zurich meets Berlin

Beneath the surface of the global tech industry

Screenshot from the scan of a artisanal mine in Kamituga

The exhibition Kamituga | Digital Gold which was originally created for the exhibition Planet Digital at the Museum für Gestaltung (February to June 2022) will be showcased in an adapted version during the Festival Zurich meets Berlin. From November 3rd to 7th 2022 projects from ETH, University of Zurich, ZHdK and ZHAW will be displayed at the Museum für Naturkunde in the heart of Berlin!

Furthermore, a panel discussion with Gabriel Kamundala (PhD student, Department of Geography UZH) and Florian Bruggisser (Research Associate, ZHdK) hosted by Gayatri Parameswaran (KUSUNDA, NowHere Media) will take place on Saturday, November 4th. During the panel the experts will give insights into the creation of the exhibition and try to give outlooks on what can be done to create a more sustainable, more transparent and social chain of trade and consumption.

For more info go to
www.digital-gold.ch | Zurich meets Berlin | Berlin Science Week


Designing Enactive Co-Presence with Humanlike Characters in Cinematic Contexts

Talk by Pia Tikka (Enactive Virtuality Lab, Tallinn University)
Tuesday, 18th October 2022 @ Research Academy

The scope of technologies available to filmmakers is expanding and apparently opening new avenues of storytelling. My focus is on the application of new findings in the fields of psychophysiological tracking and machine learning in order to create virtual characters, whose behavior resembles that of humans in the most natural ways. 

In this talk Pia Tikka will share some recent updates in this fast developing domain and discuss their possible applications to the co-presence of human participants and humanlike virtual characters in narrative contexts. This  implies a range of multidisciplinary challenges. The core research question is what types of roles can the filmmaker give to machine learning and psycho-physiological tracking in the process of creating humanlike behaviors in narrative settings. The discussion draws from the holistic embodied approach to the mind,  which in my view provides useful explanatory frames for my claims. The talk aims to inspire discussions related to the use of adaptive artificial characters in the future of virtual storytelling.

Pia Tikka is a filmmaker and EU Mobilitas+ Research Professor at the Baltic Film, Media, and Arts School (BFM), Tallinn University. Her filmography includes international film productions, feature films, and interactive VR installations. The founder of the NeuroCine research group and Enactive Virtuality Lab, she has published on the topics of neurocinematics and enactive media, and written the book Enactive Cinema: Simulatorium Eisensteinense (2008). She is Adjunct Professor of New Narrative Media at University of Lapland, fellow in the Society for Cognitive Studies of the Moving Image, and member of the European Film Academy. Currently, she focuses on biosensor-driven virtual storytelling.

Lab Insights

Shifting realities and hands-on experiences in the Immersive Arts Space. (Norbert Kottmann © ZHdK 2022)

On June 29th and 30th, the Immersive Arts Space presented new developments from current projects. Visitors were invited to experience shifting realities with or without Virtual Reality goggles and to learn more about digital gold in our smartphones.
More about the featured research project Shifting Realities.
More about Kamituga | Digital Gold.


Public Lecture Series (Autumn 2022)

This lecture series investigates the concept of immersion in art and design from aesthetic, historical and political perspectives. Guest lecturers come from digital arts, curating, media studies, and the histories of science and technology from the ZHdK, Royal Academy of Fine Arts Antwerp, Rensselaer Polytechnic Institute (New York), Haus der Elektronischen Künste (HEK, Basel) and the Technical University in Dresden. Topics will include contemporary artistic practices in multi-sensory immersion, extended reality and history of VR and the arts, and art and artificial intelligence.

Christopher Salter | September 27th, 2022 | 17:15-18:30
THE ARTS OF IMMERSION: SENSING, BODIES AND RESPONSIVE ENVIRONMENTS 

Kristof Timmerman | October 11th, 2022 | 17:15-18:30 
BREAKING THE FIFTH WALL: IMMERSION BETWEEN LIVE PERFORMANCE AND VIRTUAL SPACES

Sabine Himmelsbach | October 25th, 2022 | 17:15-18:30 
VIRTUAL WORLDS-INTERACTIVE ENVIRONMENTS: ART AS AN IMMERSIVE EXPERIENCE

Michael Century | November 8th, 2022 | 17:15-18:30 
CONTESTING A “NEW MEDIUM”: VIRTUAL REALITY AS CULTURAL PROBE

Orit Halpern | November 22nd, 2022 | 17:15-18:30 
THE SMARTNESS MANDATE: AI AND UBIQUITOUS COMPUTING`S IMPACT ON ART AND DESIGN

The lecture series will take place at the Kino Toni (3.G02), ZHdK. See on the [here]


REFRESH#4

The REFRESH#4 conference on Design, Arts and Technology took place from November 10th to 14th 2021. The fourth edition focussed on the topics «Immersive Journalism», «Immersive Play», «Digital Identities», «Digital Doubles» and «Techno-Biological Futures». The conference featured keynote speeches including panel discussions with national and international guests as well as a large exhibition have been part of the conference. 

Photographs of the conference have been published and can me viewed in this gallery. All photo credits are with Regula Bearth, ZHdK 2021.

Recap Video by Marco Bach ZHdK©2021

The Immersive Arts Space hosted the Lab Insights and a VR exhibition. Team members offered insights into the projects Shifting Realities (explorations within the intersection of reality and virtuality), Neural Volumetric Capture (experimental methods for capturing photorealistic 3D models), Digital Twins (development of animated digital avatars based on photogrammetry and Meta Humans) and cineDesk (a real-time collaborative simulation tool for the development of film scenes, VR experiences and games in 3D spaces). 

Credits:
Multiuser VR: Chris Elvis Leisi, Oliver Sahli
Helium Drones/ SAR: Martin Fröhlich, Max Kriegleder,
Joel Gähwiler, Roman Jurt, Mariana Grüning
Volumetric Capture: Florian Bruggisser
cineDesk: Norbert Kottmann, Valentin Huber
Sound Design: Luca Magni
Lights: Sébastien Schiesser
Project Lead: Martin Fröhlich
Production: Kristina Jungic
Head of Unit: Christian Iseli


Shifting Realities

FRAGMENTS OF A CONVERSATION, interactive installation, @LabInsights, June 2022 (Norbert Kottmann ©ZHdK 2022)

Shifting Realities is an overarching interdisciplinary research project, that started in Spring 2021. The goal is to explore the interplay of real and virtual experiences as well as the interaction of VR-users and by-standing spectators in shared experiences. Combining these different perspec­tives creates scope for new practices and forms
of expression in Extended Reality (XR). 

The research is structured in four binary focus areas (or pair of opposites):
Virtual vs. augmented environments
Virtual vs. real spaces, objects or humans
Binaural vs. spatial audio
Storytelling vs. game mechanics

PASSING ­THROUGH THE REAL, Mixed Reality experience @LabInsights, June 2022 (Norbert Kottmann © ZHdK 2022)

The research focuses at the interfaces defined by the pairs of opposites above. Through them, differences in perspective, shifts in perception, and the extent of interactivity can be shaped and controlled. The primary goal is the development of prototypical unique solutions in limited subareas. 

In a first phase, basic concepts of reality shifts were explored and developed. They all depict situations, in which the users find themselves in transitional states between virtuality and reality. The aesthetics to create such transitional settings are defined by rough 3D scans, sketchy point clouds created in real time by depth cameras or the Oculus passthrough feature. The resulting imperfection of such low-end processes induces an awareness of our fluctuating and at times uncertain perception while experiencing mixed reality content. 

In a second phase, three prototypical experiences were developed. Two are device-based with mixed reality components displayed in VR goggles:
• PASSING ­THROUGH THE REAL and
• FRAGMENTS OF REALITY.

One is installation-based with spatial augmented reality contingent on projection mapping:
• FRAGMENTS OF A CONVERSATION.

For the first time, the three experiences were presented in preliminary versions at the LabInsights in June 2022.

FRAGMENTS OF REALITY, Mixed Reality experience, @LabInsights, June 2022 (Screenshot by Oliver Sahli ©ZHdK 2022)

Team:
Researchers: Florian Bruggisser, Martin Fröhlich, Valentin Huber, Norbert Kottmann, Eric Larrieux, Chris Elvis Leisi, Oliver Sahli, Stella Speziali. Production manager: Kristina Jungic; Chief Technician: Sébastien Schiesser. Principal Investigator: Prof. Christian Iseli

PASSING ­THROUGH THE REAL, Mixed Reality experience @LabInsights, June 2022 (Screenshot by Chris Elivis Leisi © ZHdK 2022)