On this page you can find current and upcoming projects. We develop and host a variety of projects, such as graduation student projects, internal research projects and collaborations with external partners.
Conversations with Puppets
Part of the exhibition “The Museum of the Future – 17 Digital Experiments”
29th August 2026 untill 1st February 2026, Museum für Gestaltung Zürich
About the exhibition
The exhibition ‘The Museum of the Future – 17 Digital Experiments’ examines the potential, areas of application, forms of content expansion and implementation options relating to digitality in forward-looking museum and educational work. This year, a major exhibition (17 exhibited projects) will make the ‘state of the art’ accessible to the public with several installations, a website and a broad-based educational program (analogue and digital).
About the project
The Immersive Arts Space` contributing project is called ‘Conversations with Puppets’. It offers an innovative and interactive experience in which the visitors become an integral part of Fred Schneckenburger’s puppet cabaret. After World War II, the versatile artist Fred Schneckenburger founded the “Puppencabaret Fred Schneckenburger” in Zurich and enjoyed success throughout Europe with his sketches. His expressionistic, surreal stick puppets were characterized by their idiosyncratic and unique design. In terms of content, his pieces criticized the hypocrisy and contradictions of the conservative bourgeoisie, with stereotypical Swiss characters in the form of puppets.
Technical architecture
Three of the original stick puppets – Kaspar, der Polizist and die Moral/das Fräulein – from the collection of the Museum für Gestaltung Zürich have been photogrammetrically captured and digitized. 3D artists worked out the details of the textures and cloths to recreate and animate the physical appearance of the puppets surfaces. In addition, Motion Capture recordings were used to add more puppet-like movement to the avatars.
Large Language Models (LLMs) and generative AI form the basis of the interaction and conversations (speech to text, text to text and text to speech). In order to create a character prompt for each puppet, the original audio recordings of the pieces in which the specific puppets appeared, were transcribed and analyzed. The result of the analysis were distillations of the most characteristic words and/or expressions of the puppets, which were then inserted in the prompt. In addition, to the character references the prompts were further developed to fit the role of each puppets within the exhibition and its dramaturgy.
Further, the interaction is supported by different systems, aiming at a flawless, humanlike interaction. SAR motion, a system internally developed by Florian Bruggisser, detects via a camera the movement and body of a visitor. This triggers the interaction and the puppet starts to talk.
Team:
Florian Bruggisser: Software Engineering & Machine Learning
Oliver Sahli: System Integration
Chris Elvis Leisi: Interaction & Experience Design
Johannes Köberle: 3D Artist
Corinne Soland: Motion Capture
Pascal Lund-Jensen: Audio Engineering
Stella Speziali: 3D Artist, Animation
Sébastien Schiesser: Technischer Support
Kristina Jungić: Projektkoordination
Prof. Christopher Salter: Projektinitiator/ Leiter IAS
Photo credits: Chris Elvis Leisi, ©ZHdK 2025
A Hero’s Return
The interactive installation A Hero’s Return transforms visitors’ poses into a digitally generated alter ego. The creation of the character and its background is realized in real-time by artificial intelligence models. This installation prototype explores the capabilities of these models (such as Stable Diffusion) and focuses on the technical and aesthetic possibilities in combination with the performative interaction with visitors. The installation also incorporates popular film representation techniques: a virtual camera performs a circular tracking shot around the scanned visitor’s body, reminiscent of scenes from numerous action hero films.
Credits:
Artists: Martin Fröhlich, Stella Speziali
Thanks to: Ege Seçgin
doppelgaenger:apparatus
doppelgaenger:apparatus is a multiuser mixed reality experience that enables participants to confront their computer generated 3D doubles in a real physical surrounding. Enabled by a custom designed AI-based process, a frontal 2D image of the participants is taken and quickly transformed into an animated 3D reconstruction. From the moment the visitors put on the headset, they see the real environment captured by a camera on the headset. Gradually, this familiar space shfits, with the appearance of the visitors’ double standing before them. Co-present with other participants, visitors intimately interact with their ‘mirror image’ while observing and interacting with others doing the same. While at first the participants get the feeling of controlling their ‘mirror image’, this rapidly shifts as their doubles, free from the constraints of the original bodies, start to invade the participants’ personal space, leading to a disturbing and uncanny play between them. Historically, a doppelgänger was seen as a ghostly counterpart for a living person, seen as an omen or sign of death. Freud later argued that encountering one’s double produces an experience of “doubling, dividing and interchanging the self.” This installation thus takes the next step in exploring how new digital sytems increasingly produce uncanny tensions between our bodies and their captured other.
Credits:
Artist and Development: Chris Elvis Leisi
Artist and Sound: Christopher Lloyd Salter
Machine Learning: Florian Bruggisser
Next presentation: MESH Festival, 16. to 20. October at HEK in Basel. More information [here]
Recent exhibition: REFRESH x FANTOCHE, 04.-08. September 2024 from noon to 8pm.
Zangezi
Zangezi is a Russian Cubo-Futurist poem/play written by the poet Velimir Khlebnikov 101 years ago in 1922. The story revolves around Zangezi, a prophet who speaks in the language ZAUM, a Russian word that is translated as “beyondsense.” Zangezi speaks with and can understand the birds, the gods, the stars and speaks in those languages as well as in poetic and also what K. called “ordinary” language. The Immersive Arts Space is working on this text because as a futurist work, Zangezi deals with is a fundamental human question which is increasingly becoming a problem for machines – what is the basis of language? Is language only about meaning based on syntax? Is it about predictable sequences? Rules and probabilities? Or is there something else going on that is universal and cosmic about language not as words and their meanings but as an act of sound?
Indeed, Khlebnikov, who had a mystical belief in the power of words, thought that the connections between sounds and meaning were lost during mankind’s history, and it was up to those in the future to rediscover them. 101 years later we ask how we might be able to approach Zangezi’s operations on multiple levels – cosmological, political, historical, technological – in a moment where we are increasingly surrounded by machines that produce something that appears like human language but in which there is no speaker, no body and no sound.
Zangezi-Experiments, a multimedia stage play, was performed at the MESH Festival. Before, fragments of the piece were recreated as a work in progress for the REFRESH#5 2023 festival, that took the form of a theatrically staged reading within the technical machine of the Immersive Arts Space itself.
Credits:
Chris Salter (Direction, Sound Design)
Corinne Soland (performer)
Luana Volet (performer)
Antonia Meier (performer)
Stella Speziali (Development Meta Human)
Valentin Huber (Visual Design/Unreal Engine)
Pascal Lund-Jensen (Sound Design), formerly Eric Larrieux
Sébastien Schiesser (Light Design, show control)
Ania Nova (Russian voice over)
NEVO | Neural Volumetric Capture
NEVO is a processing pipeline that combines multiple machine learning models to convert a two-dimensional images of a human into a volumetric animation. With the help of a database containing numerous 3D-models of humans, the algorithm is able to create sketchy virtual humans in 3D. The conversion is fully automatic and can be used to efficiently create sequences of three-dimensional digital humans.
Group members: Florian Bruggisser (lead), Chris Elvis Leisi. Projects: Shifting Realities Associated events: REFRESH X FANTOCHE, LAB Insights
cineDesk
cineDesk is a versatile previsualization tool that provides real-time simulations for the development of film scenes, VR experiences and games in 3D spaces. In the preproduction process of films, cineDesk primarily supports scene blocking and lighting previsualization. Directors, cinematographers and production designers can collaboratively explore how space, props and acting, are translated into cinematic sequences. The positions and movements of the virtual actors and the virtual camera in the 3D space are rendered in real time into 2D videos with the help of the Unreal game engine. This allows staging and visualization ideas to be explored interactively as a team. And, by means of virtual reality goggles, the virtual 3D space can be entered and experienced directly.
cineDesk, is a further development of the Previs Table, which goes back to cooperation with Stockholm University of the Arts. The actual research focuses on advanced features and multiple applications (e.g. for gaming, stage design, architecture etc.)
Developers: Norbert Kottman, Valentin Huber, videos: cineDesk, Scene-Blocking, research field : Virtual Production
> cineDesk has its own website. Please visit www.cinedesk.ch