Interacting with machines without losing what makes us human
Right now, I'm using AI to explore new ways of thinking about human-computer interaction, replacing concepts of control and manipulation with resonance, sensing and belonging.
My research explores how interaction with technology shapes how we see ourselves.
We are at risk of turning into the machines that we create.
Any interface defines potential: what actions the person can take, and how the system can respond. The actions the interface cares about are a tiny fraction of what that person can do elsewhere. In this way, the system defines a model of what a human is, a vastly simplified model. For example, my laptop is designed for a human with fingers, eyes and ears but is indifferent to my nose, the expression on my face, the scars on my skin, my need for warmth. The question is: when I use my laptop, do I start to conceive of myself as this model.
I create art that explores the impact of interactive system design and speculates on how it could be different. If familiar technology limits which parts of being a human are considered relevant then how can we invent new forms of interaction that unleash everything else? If consumer technology embodies values of passive consumption and predetermined expression, then what would systems look like that embrace activism and open-endedness? (I write about this in depth in a recent paper.)
I approach interaction from an embodied perspective by which I mean the non-verbal parts of our intelligence that do not rely on the manipulation of symbolic abstractions. Things like dance, music, social dynamics, beauty, riding a bike - basically that which is difficult to put into words or to model in a formal process. I often collaborate with dancers to create systems that respond to the moving body. Indeed, much of this research has developed through collaborations with other artists on their own journey.
I'm interested in how far such a system can encapsulate a philosophical concept and bring it to life in the minds of its participant. Do ideas infused into the nature of a system become internalised by those who use it?
For example, Cave of Sounds is an ensemble of new instruments that invites anybody to come and play music with strangers. It embodies the idea that the value in a musical encounter emerges through participation. There is no scoring for playing to a prescribed standard. There are real people around whose reactions one can gauge. Compare this to a social network, where the myriad of human responses is often reduced to a single bit of information – the "like" button – and the quantified system of status that ensues.
The disproportionate amount of power wielded by the dominant tech companies is a familiar subject. But the abstractions the systems define - users, posts, content – are often so transparent and ubiquitous that we don't even realise they are there mediating how we think about the system and our role within it. To me, the value of interactive art is in showing how interactivity exists today and probing how it could be different.
The ultimate hidden truth of the world is that it is something we make and could just as easily make differently.
I began this journey when I completed my PhD exploring interactive music systems. I was looking for the interactive equivalent of a musical experience but kept hitting upon a paradox familiar to those designing for creative interaction. For the participant using a system to be creative, they need agency over the outcome of the interaction. But any agency the designer grants to the participant is less agency that designer has over the experience and aesthetic.
What I found interesting is that sometimes people blame the system and other times they blame themselves. Who they blame depends on how the system is presented and framed, and the role they assume. This is significant for creative expression, but also for the everyday technology that mediates our lives.
The interplay between roles, agency, empowerment and enjoyment is central to how I think about our interactions with technology. For example, in Cave of Sounds, visitors are invited to participate as musicians, and the piece is built to support that intimidating prospect while avoiding any fakery. On the other hand, in Post-Truth and Beauty, visitors are invited to participate as observers. They must interact by moving to explore an unfamiliar world of sounds suspended around them, but the piece is framed to avoid suggesting they are in a creative role. However, Movement Alphabet introduced a human performer as the mediator between participant and system. This let us escape the roleplay of human-machine interaction entirely and rest on the richness of human interaction.
Recently, I've been exploring alternatives to the paradigm of command and manipulation as the basis for interacting with computers. Consider the swiping of abstract entities such as icons and windows, or the typing of commands and the expectation of results. Compare this to embodied interaction at its best, a deep conversation or the improvisation between two dancers. The lack of designed abstraction is what keeps it open ended, what keeps people free.
Drawing influence from the writings of the psychiatrist and philosopher Iain McGilchrist, I'm applying AI to explore alternative approaches to devising human-computer interaction rooted in resonance, sensing and belonging. You can see the results in Sonified Body. I've written more about this in a recent journal paper on emergent interfaces.