Emergent Interfaces: Vague, Complex, Bespoke and Embodied Interaction between Humans and Computers
This paper gives the philosophical details behind Sonified Body. I explain what I mean by supplanting abstract manipulation with embodied resonance. Published in the peer reviewed journal Applied Sciences.
Human-computer interaction is currently dominated by a paradigm where abstract representations are manipulated. An alternative is that of emergence and resonance. This requires us to let go of ideas of the need for our interactions with computers to be precise and explainable. It makes little sense in traditional computing tasks originating in the office, but may open new opportunities for embodied activities like creative expression, dance and social connection. In Sonified Body, we explore this approach by using AI to devise an interface based on an individual's existing vocabulary of movement, and test out its artistic potential with some dancers in the studio.
abstract
Abstract
Most human–computer interfaces are built on the paradigm of manipulating abstract representations. This can be limiting when computers are used in artistic performance or as mediators of social connection, where we rely on qualities of embodied thinking: intuition, context, resonance, ambiguity and fluidity. We explore an alternative approach to designing interaction that we call the emergent interface: interaction leveraging unsupervised machine learning to replace designed abstractions with contextually derived emergent representations. The approach offers opportunities to create interfaces bespoke to a single individual, to continually evolve and adapt the interface in line with that individual’s needs and affordances, and to bridge more deeply with the complex and imprecise interaction that defines much of our non-digital communication. We explore this approach through artistic research rooted in music, dance and AI with the partially emergent system Sonified Body. The system maps the moving body into sound using an emergent representation of the body derived from a corpus of improvised movement from the first author. We explore this system in a residency with three dancers. We reflect on the broader implications and challenges of this alternative way of thinking about interaction, and how far it may help users avoid being limited by the assumptions of a system’s designer.
bibtex
@article{murraybrowne2021emergent-interfaces, author = {Murray-Browne, Tim and Tigas, Panagiotis}, journal = {Applied Sciences}, number = {8531}, title = {Emergent Interfaces: Vague, Complex, Bespoke and Embodied Interaction between Humans and Computers}, volume = {11}, year = {2021} }
reference
T. Murray-Browne and P. Tigas, “Emergent Interfaces: Vague, Complex, Bespoke and Embodied Interaction between Humans and Computers,” Applied Sciences, 11(18): 8531, 2021.