Sonified Body

AI-based instrument transforming dance into sound

Sonified Body is a dance-based instrument that uses unsupervised AI to create a personalised interface in total sympathy with the human body as it already exists.

Conventional interfaces are designed as layers of abstract entities understood primarily through metaphor. We propose an alternative: We define a bespoke interactional language from a person’s natural vocabulary of movement.

Rather than navigating the abstract space of a user interface, we navigate the embodied space of your body.

Rather than thinking and planning, we improvise, guided by sensual exploration.

Photo of dancer Catriona Robertson working with Tim Murray-Browne in a studio at the Centre for Contemporary Arts, Glasgow. Photo by Alan Paterson.

Sonified Body results from art-led research into Negotiated Interaction between human and computer.

It speculates on a future where computers let humans be human.

It reclaims AI as a weapon wielded by states and corporations to categorise and manipulate people. It reframes it as a gateway to bring embodied individual expression into the digital sphere.

The Sonified Body System

Sonified Body is a negotiated interface that transforms the movement of a dancer into sound.

An unsupervised AI model seeks out the shared space where a person’s existing movement language overlaps with the computer’s capability to sense that movement.

We use the AI’s internal representation to drive a range of synthesisers.

The AI interprets the body holistically without reducing it to explainable components. Its interpretation is 16 dimensional, entangled, and unexplainable. This means any movement of the body can affect every parameter of the interpretation. And while consistent in behaviour, there is no logical reasoning that will explain how the interface interprets movement. Like singing or riding a bike, it must be learnt first-hand through play and embodied exploration.

A dancer also considers the body holistically. They weave together body and mind to attune for sensitivity, intuition, inhibition and action. This allows movement to be improvised in continual resonance with the dynamics of the space. The complex relationship between body and sound can be explored and understood at an embodied level.

Between sessions, the system can be retrained based on the dancer’s evolving language of movement. The updated model will define a completely new landscape of sound to explore. This is not a loss because the dancer relies on knowledge of their body rather than knowledge of the system