Rewilding Human-Computer Interaction

It’s about letting nature take care of itself, enabling natural processes to shape land and sea, repair damaged ecosystems and restore degraded landscapes. Through rewilding, wildlife’s natural rhythms create wilder, more biodiverse habitats.

Sometimes it seems the internet is a magnet for our worst instincts and the worst people. Sometimes it seems to transform even the best people into the worst.

And so we find ourselves polarised across two unappealing options. We have content moderation: the widespread censorship of human expression by algorithms governed by corporations. The other option is what we might call let it rip: just putting up with toxic spaces filled with bullying, abuse, propaganda and spambots.

But this is a false dilemma. There is a third option: tapping into the human superpower of self-organising into mutually supportive communities that cooperate and get on. The same one that made free society possible in the physical world.

Why is this not happening in online spaces? I suspect the design of online platforms may be blocking it, and the more they try to fix us the more they end up blocking our capacity to self-regulate. Let’s explore how to recover the third option.

User behaviour is not human behaviour

If you're the head of moderation at Facebook reading this, you might think “you have no idea what we're dealing with.” You would be correct. However, if your primary experience of people using technology is through what you see of them on the internet then you’re only ever seeing them through the digital prism. So I may levy the same critique: “you have no idea what we're dealing with”. All you’re seeing are people who have already been Facebooked. You can’t see what you’ve done to them because you only meet them after you’ve done it.

There is an inverse perspective available if you watch the physical human interacting with the system rather than their digital artefacts. Before the internet and its world of analytics, this was the only option.

In my work, it remains the only option. I create digital interactive art: gallery installations with bespoke software connecting human movement to light, image and sound. As an artist, my canvas is the unfolding interaction between person and system. For example, Post Truth and Beauty is an audio-luminescent sculpture that changes in response to the viewer’s head position. Cave of Sounds is an ensemble of eight digital musical instruments are exhibited in the gallery for visitors to play.

Touring the world, I’ve watched thousands of people confronted with unfamiliar interactive technology created by me and other artists. In the gallery, I see people exist as humans before they become users. We're both on the same side of the digital prism. They approach new technology with curiosity, playfulness, kindness and compassion. In this atmosphere, strangers connect. Communities emerge spontaneously.

But with interactive art, there's also the potential to leave people feeling somewhat humiliated. Interactivity invites you to give something of yourself, to put yourself on the line. If it doesn't work out as promised, many are quick to blame themselves – especially when others around seem to succeed with little difficulty.

Digital interaction creates a greater opportunity for this mismatch between promise and reality. To design an interactive system is to construct a space of actions and consequences. The design constructs a division of agency: who is responsible for what. People interacting with it need to figure out what lies with them and what lies with the system. But the opaque nature of digital technology makes this difficult to ascertain. Think of the mysterious relationship between the content of a social media post and its number of likes. Compare that to an ingredient and the flavour of a dish.

When people interact with technology, they become vulnerable. It's hard to see this when all you can see are the digital effects of their behaviour. In my experience, we over-estimate how much agency technology gives us, and under-estimate how much agency we need for us to function like the human beings we know from the physical world.

So this is the position from the human side of the interface. Rather than starting from the premise that users are problematic beings that need fixing, what happens if we start from the premise that humans are generous, hospitable, sensitive beings with a remarkable ability to figure out how to get on with each other?

To be clear, I'm not saying that the nice people I meet in the gallery are representative of the whole population. My purpose here is not to try to convince you of the innate ability humans to get along. I'm taking that as read.

My purpose is to explore how digital interaction blocks these innate abilities. And how we might fix that.

The fragility of designed interaction

The light switch, Twitter, the welfare system, the layout of Ikea. What do these all have in common? They are all systems infused with interaction design. Someone has designed how you interact with them. They thought about who you are, what outcomes you (or they) might want and designed a range of actions and consequences to lead you there.

In many ways, interaction design is the fundamental craft of my artwork. But it’s a paradoxical term because when you try to design interaction, you realise that you can't. You can design an interactive system, and you can design the context to cue the person. But the interaction is determined by the combination of human and system. You don’t get the same control over the human as you do the system. You can herd them, corral them, seduce them, manipulate them, threaten them. But humans will always respond in diverse ways, many of which are off the chart of your expectations.

For example, one instrument in Cave of Sounds is a sphere that fits neatly in your palm. Its creator, Panagiotis Tigas, called it Sonicsphere. When you shake it, it triggers notes and pitch-bends them as you rotate it. It sits on a plinth with the minimal starter instructions “Spin ’n’ Shake”.

It seemed simple enough, until one day I noticed someone spinning circles with their body in front of the plinth. She had only seen the empty plinth while someone else was playing with the sphere. It was a fair conclusion based on the other instruments in Cave of Sounds, but not one that I had considered.

What is obvious to me is not obvious to them. If I want to get someone to do something specific, I have to make it incredibly basic. This lesson, I have to keep learning over and over. Even though I know it, I somehow keep being surprised by it.

As the interface designer, the more vested I become in a particular outcome, the more I need to understand the audience in order to invite, bargain or manipulate them into making the decisions that construct the experience I have in mind.

The invasive nature of User Experience Design

And so it is that we go from interface design, where we design the potential actions and responses between human and system, to interaction design, where we design how someone is led on a journey through narratives of action and response, to experience design, where we design how this journey affects how they feel and impacts upon their life. The more exacting the designers intentions, the more closed the range of potential interactions.

In my gallery work, I find the most exciting parts are the unexpected. Human behaviour can always be surprising. But for an artwork to be enhanced rather than broken by people’s spontaneity, it needs to be open. This gets lost when the unfolding dynamic between human and system has been completely mapped out.

In Cave of Sounds, we did what we could to leave open the emergent dynamics of strangers playing music together. No conductor. No guidelines. No expert musicians. No victories or level advancements.

In the hundred hours or so I must have spent observing people with that work, there are four or five moments where a sense of the sublime has moved through me. In those moments, it suddenly seems everyone in the room knows each other. As if we’ve always known each other and just pretend otherwise in the real world. Experiences like this are more than sentimental. They form part of the glue holding humanity together. And I’m not experiencing them on the internet anymore.

Open-endedness is not incompatible with intense experience design. There is a pleasure in planning and executing interaction, in immersing someone along a journey in which they an active participant. If done with care, remarkable things can emerge.

For example, Movement Alphabet is a work I created with dance artist Jan-Ming Lee. We created portraits of people’s physicality by mapping movements of their body into digital paint strokes. We would place nine of them in a grid to make a Movement Portrait.

We intended Movement Alphabet as a gallery installation and were invited to debut it at Tate Modern. But we knew that if we're to ask someone in the middle of a busy gallery to move their body, what they do tends to be stilted, contrived and unlike any movement they do in real life. It’s like trying to walk in front of someone talking about how you walk.

So to help someone move in a way that is open and authentic, we created a 15 minute experience. Each participant is barefoot, blindfolded and personally led by a performer on a journey of whispered stories inviting heightened awareness of the surroundings. When the blindfold is removed, they are in a safe translucent pod, just the two of them. Here, the performer invites them to share memories while mirroring their body to encourage and amplify their movements.

The impact was beyond what I had imagined. People shared deeply personal stories. Crawling as a toddler. Giving birth. The death of a spouse. Afterwards, they were given their printed portrait, seated on a sofa in a quiet corner and given a brief questionnaire. Most sat for 10 to 20 minutes afterwards.

This level of experience design puts someone in a vulnerable position. That vulnerability is what makes such a personal and moving experience like this possible. But it places upon us a duty of care and respect. Throughout this experience, there is a person looking after you, listening, responding, empathising. The human-to-human experience brings a depth of emotional experience that, again, is open and unpredictable.

In everyday interaction, experience design still has its place, for example in specific tasks like filling out a tax return, buying a coffee or playing a game. These tasks are things we take on for moments of time to achieve specific ends. They are contained, with a start and an end and the opportunity to stop and walk away. However, our current relationship with technology has evolved far from the task-oriented roots of the offices that computers emerged from. When an operating system or a social media platform becomes the primary means through which we manage our relationships, communities and politics, then things get a little weirder. Experience design is no longer a transient play we step into but something invasive and potentially oppressive.

Walled gardens

The trend in consumer tech over the past decade has been to lock things down. When Apple launched the iPhone, they decided that no app could run on anyone’s iPhone without their prior approval. This policy is known – tellingly, for our rewilding metaphor here – as the Walled Garden.

Apple did not create the first walled garden. For example, AOL had a walled garden version of the internet in the 90s. But AOL’s attempts crumbled in the face of the open internet. Apple’s succeeded. The App Store demonstrated that people will put up with a walled garden if you have a product that is special enough. Also, controlling those walls is incredibly profitable.

There are pros and cons to walled gardens. The pros are sometimes simpler to argue than the cons. Locking down iOS lets Apple enforce standards of safety, security, data protection, accessibility and usability. The obvious con is that they also use this leverage to monopolise markets. Maybe this is an acceptable trade-off.

But it's the con less easily articulated that I want to focus on here. A walled garden defines a central authority that determines how we can use the technology that mediates our everyday life. Even if today it is used to protect us from things we don’t want, it holds the potential to revoke freedoms we do want. The reason we may still feel free is because that authority chooses not to impose themselves. The tech commentator Ben Thompson describes this well: we rely on policy rather than capability. Policy can change in an instant. The longer we participate in such a system, the more entangled it becomes with our lives, and so the harder it is to stop.

It’s easy not to notice what is lost until those in power are forced into a polarising decision. For example, how far to censor social media content. In these moments, the power held behind the curtain is revealed. In this moment, the loss of freedom may feel sudden but the power to take it away has grown incrementally in the preceding years.

A moment like this happened in the aftermath of the 2021 Capitol Storming.

Some on the left fought to permanently censor the former president from using Twitter because what he says is too dangerous. Some on the right argued for government intervention into a successful corporation because it is at odds with their values. Strange times indeed.

Perhaps you think this is just a mob of woke snowflakes so complacent about the freedoms they inherited without lifting a finger. Or maybe you think this is the provocateur of an armed and delusional militia ready to start a civil war.

I'm not here to argue one side. I want instead a third position. The fact we've even found ourselves debating this suggests something's gone seriously wrong with how we have enmeshed technology into our lives.

The Garden. The Wild.

Let's draw an analogy between the ecosystem of human society and that of green space.

A garden is planned. What goes where is organised according to someone's vision. Gardeners support the plants growing where they are intended, and mercilessly kill the ones that are not.

When we find beauty in a garden, it tends to be there by design. I think one reason gardens are relaxing is that they signal both natural abundance and a sense of human order. Prosperity and peace. When the plants go out of control, it signals neglect, decay. A breakdown of order.

The wilderness, on the other hand, has evolved to self-regulate without gardeners. The plants (and animals) are always out of control, or more, they control each other. Balance emerges not from a centralised planner but through mutual adaptation. We don't get the neat lines of the garden, or its overabundance of flowers. A different kind of beauty emerges out of the complexity, an awe which signals the power and vibrancy of nature. A deterioration of the wilderness can often be traced back to an over-reach of human intervention. Too much order. Here, decay signals human society divorced from its place in the natural environment.

Evolution is a slow process. This leaves wild ecosystems sensitive to disruption from sudden external shocks. For example, fertilizer run-off can lead to algae bloom which then blocks light to the plants in the water which stops them from releasing oxygen which poisons and suffocates the fish.

Rewilding

It seems our ancestors, subjected to the brutality of unbounded nature, grew quite fond of taming it. But for many today, with so little natural environment left, the sentiment has inverted.

As described in the quote introducing this essay, rewilding is an approach to conservation that aims to restore an earlier natural ecosystem. At its, I see a humility that acknowledges how hard it is to design balanced ecosystems. Our ability to reason about a complex system will always be limited. We can create complex models of an ecosystem's many self-regulating processes, but to make any model tractable, we have to introduce reductive assumptions.

Rewilding instead invites processes that have evolved over billions of years around the material reality rather than the abstract models of a human designer.

What does it signal when seeds grow through the cracks of the city? The planner sees neglect and decay, a problem to be fixed. But the rewilder sees the beauty of nature persevering, and sighs with relief.

I think we all have planner and rewilder within us. But sometimes we get stuck in one mindset.

Humans are wild. Human-Computer Interaction is designed.

The basic insight is, again, seemingly straightforward:
that every community practices the design of itself.

This is the community notice board at Tchaiovna, my local teahouse in Glasgow. It’s chaos. The posts are spilling out of the frame and over each other. Each post has a completely different personality. There is no consistent design language. Some of these are barely readable.

It feels like a community of people.

Now compare this to the London House/Flat/Spare Room To Rent group on Facebook.

The Facebook group is certainly more efficient. You can easily scroll through pictures of possible rooms you might live in. But the design of each post is uniform. The personality has been squeezed out of the medium. The person behind the post, who we may be sharing a home with, is only revealed through the text and photos they have chosen.

When humans connect and respond to each other in wild spaces, the space becomes filled with vague and ambiguous communications incidental to the message. There is facial expression, tone of voice, gesture. There is where you stand, objects you've left around, how you dress, moments you pause, what you look at, moments you remain present but silent. Some of these signals are sent and received consciously, some unconsciously. Some we'd hide if we could.

The nature of analog communication means we cannot help but embed our signal within a wealth of contextual information. This mess of information spills into the physical materials we use to communicate. When I receive a letter from my mum, I feel her presence in her handwriting on the envelope before I open it. That feeling of someone’s personality unavoidably bleeding into analog media was a starting point for Movement Alphabet, mentioned above, where we transformed someone's body movements into a digital calligraphy.

When two humans are face-to-face, feedback loops of communication emerge. We signal attention through unconscious mirroring, from body poses right down to tiny twitches of the eyes. This back and forth creates resonance, a feeling of connection. That facilitates empathy compassion, presence, intimidation in some ambiguous mixture. Much like a wild ecosystem, these systems of interpersonal communication will always remain more complex than any model we can invent.

Digital spaces tend to be sparse as they are designed onto a backdrop of nothingness. Whereas the structure of a building defines a space by working with the constraints of physical reality, Facebook defines a space by defining the digital reality. Modes of communication are invented rather than uncovered. This leaves us without the mess of accidental signals by which we get to know each other in reality.

Wildness can spill into digital space. For example, here is a Geocities page from the 90s, The Fold.

I get a feel for this author’s personality before I’ve even read anything. Not saying he would, but let’s suppose on digging deeper I find he starts rambling about QAnon. This would fill out a picture of him that I’d already started forming. Maybe I’d find it easier to respond with compassion rather than confrontation.

However, today’s trend in web design is for clean and streamlined content. Most platforms don’t even let you choose the colour of your text. This lets them solicit content that is both easy to read and atomic enough to be rebundled in a feed somewhere.

Compare the above 90s Geocities page to this 20s Medium post by Benjamin Reinke What Are The Simpsons.

The article is neat and clean. It looks more professional than the Geocities page. You might even describe Medium as a tool that democratises “good” design. But if you post on Medium, you have no option but to have your message formatted like every other person posting on Medium. As with Substack, Facebook, Twitter. You can’t even change the colour. “Good” design is no longer a meaningful signal.

However well designed these sites are, they homogenize the personality of those using them, squeezing it all into the linear sequence of text and images that makes up the content. (Benjamin’s other articles include 19 Tips on How To Tell Your Parents You’re Pregnant At 22 and Can Zebras Swim? (The Truth).)

When we design interaction, we're usually trying to help someone achieve their goals as efficiently as possible. It's tempting to strip away the fluff and focus on the essential actions and views. But every decision we save the user from is one less opportunity for a person to reveal themselves. The messy bits of us, the parts that reveal us as human, get filtered out. There is less of what we might call expressive bandwidth.

As with the App Store, constraints are a mixed bag. Just as Apple is quick to point out the threat of bad apps, designers have quickly pointed out to me that Medium is accessible to people using screen-readers in a way that Geocities was not. I don't disagree with them. But just as those making wild spaces accessible need to begin from the premise of preservation, so designers need to address issues of accessibility and malware while preserving expressive bandwidth.

Extremism as an artefact of limited opportunities for expression

The uniform presentation of tweets, posts, messages and articles has increased the efficiency of communication. It enables us to keep in touch in remarkable new ways, and perhaps allows outsiders a greater chance to influence public debate. But it has also revealed a world of aggression, addiction, groupthink, manipulation, bullying and a lack of empathy. It seems to have unleashed the worst parts of human nature.

But what if these behaviours are not human nature at all, but are actually unhuman artefacts resulting from the idealised and incomplete worldview of such efficient communication? Stripping away expressive bandwidth leaves us trying to say so much with so little. Those clues to our broader story in which empathy might take root become lost.

This is not to say online platforms aren't filled with creative expression. Where there are humans, there is expression. We have things to say and identities to project. This becomes even more visible in tightly constrained mediums. The 160 character limit of early SMS messages led to new forms of compressed writing. SndsGdMtUAt8? We can celebrate these artefacts as evidence of cultural invention much as an economist might celebrate the use of cigarettes as a prison currency as evidence of monetary invention. It doesn't mean putting people in prison is good for economic innovation. Reducing all human communication to SMS messages would likewise be a disaster for expression.

Today's digital communication platforms offer more expressive bandwidth than SMS. But they share the binary division between what is the person and what is the medium. You can’t hear the shake in someone's voice when they are upset. What you write inside your post cannot escape its borders. The ink cannot drip down onto someone else's post. I can’t scribble over it in a moment of rage.

When disruptions to the medium are no longer an option then nothing is signalled by their absence. When there’s no way my nervousness might leak into my voice, then there’s no way for me to speak in a confident voice either. If I post a disagreeable note on the physical noticeboard, anyone could deface it or tear it down. So when someone instead posts a counterargument, they signal not only their disagreement, but their belief in discussion over censorship.

When destructive acts are made impossible, we lose the sense of community that emerges from people continually choosing not to do those acts.

In analog media, the boundary between content and medium is much more permeable. But digital platforms tend to be designed with explicit boundaries between content and medium. This goes right down into the language of the web which splits pages into content (HTML) and style (CSS).

If our only option for expression is content creation, and its boundaries are immovable, then how does someone push against the boundaries? How does someone needy for attention respond? By making their content more extreme.

In one sense, the platforms are not entirely to blame. The more we hold them responsible for what we find on them, the more clearly they need to distinguish between them and their users, less we confuse who is to blame. But in another sense, the platforms are entirely to blame. Because the more efficient they try to make our interactions, the more they collapse our complex inner worlds into humanless digital abstractions.

The most extreme case is the Like button, where someone’s reaction is reduced to a single bit of information. I remember when it was introduced in 2009. It didn’t seem a good idea back then either.

What if, instead of a like button, there was a long roll of paper where you might draw a response? Instead of 🙂🙂🙂🙂🙂 you'd need to draw

Someone else might take the time to handwrite how they felt. We'd feel something of their personality just in how they wrote. We might notice if they were a child. We sense how much time they invested. Maybe it’s just a scrawl.

I actually already do this by having a comment book at gallery installations. The result looks like this.

Before even reading the words, it’s heartening to see all the different personalities spread on the page. Also, handwritten criticism is somehow easier to read.

Of course, every now and then someone draws a penis, maybe something worse. Here’s an example from Cave of Sounds in Rome in 2014.

We survive graffiti in physical space. In digital space, it's so much easier to exert control. But what if the ease at which regulation can be introduced atrophies the processes of self-regulation? Perhaps by trying to fix everything through platform design, we neglect the chance for self-regulation to emerge.

Human nature is not the problem. Human nature is the solution.

This hypothesis of Rewilding Human-Computer Interaction speculates that the solution to our online woes may not more authoritarian design that attempts to regulate out our problematic human nature. It may be the opposite: more open design that imposes less to welcomes in more human nature, tapping into our innate capacity for self-organisation.

If it holds then there are specific actions we can take right now. Stop criticising platforms for their failures to police content. To do so is to position the platform as dictator of its users. Instead, criticise platforms for their design. Look at how they block the formation of communities that can police themselves. Look for how shitty behaviour emerges from design decisions. Pay particular notice to when platforms claim to represent us through digital abstractions that reveals little of the depths of who we are: retweets, like buttons, follows, swipe rights.

To let humanity leak into digital space, we need to increase our bandwidth with computers, to let in more noise.

Human nature is not the problem. Human nature is the solution.