This paper gives the philosophical details behind Sonified Body. I explain what I mean by supplanting abstract manipulation with embodied resonance. Published in the peer reviewed journal Applied Sciences.
Human-computer interaction is currently dominated by a paradigm where abstract representations are manipulated. We propose an alternative paradigm of emergence and resonance, which can be realised through unsupervised machine learning. This shift requires us to release the requirement that interactions with computers be precise and explainable, and instead embrace vagueness, ambiguity and embodied cognition. This makes little sense in traditional computing tasks originating in the office, but it offers new opportunities for embodied activities like creative expression, dance and social connection.
In Sonified Body, we explore this approach by using a Variational Autoencoder to devise an interface based on an individual's existing vocabulary of movement, and test out its artistic potential with some dancers in the studio.
There are broader political implications to this approach. It may help to reduce the potential of interfaces to uniformly impose the mindset of its designer onto its users.