The Mouse Code
What are we building in there?
There’s no audience for this. It’s a worry.
“We’re making one”, was the answer. I had been thinking about mice … again.
Douglas Adams had a thing about mice. In The Hitchhiker’s Guide to the Galaxy, we discover that mice are not quite what they appear to be. We had assumed that humans are the clever ones and mice are, well, mice. Turns out we got the hierarchy wrong.
It’s a good joke. Which is no doubt why it’s been sitting around in my head for several decades. Jokes often become more interesting when you stop laughing.
There’s another joke I’ve been carrying around. Tom Waits asks:
What’s he building in there?
We don’t know. We never know, but we hear noises. We notice things and we speculate, wildly; we build a story.
Seems familiar; especially now.
What is AI doing in there?
What does it know? What has it learned? What is latent? What did it see? What did it absorb? What happened between the enormous, messy, contradictory mass of human cultural material and the sanitised, post-trained thing that talks back to us?
We don’t know. Neither do ‘they’ … So we speculate, wildly. We start talking about AI as though the things we are afraid of are things that exist inside the machine.
But the machine may be revealing things that existed outside it all along.
The Shadow
It isn’t news that AI has seen humanity’s Shadow.
Just try to train a system on an enormous amount of human cultural material without encountering violence, prejudice, cruelty, fantasy, obsession, propaganda, paranoia, genius, stupidity, tenderness, contradiction, hypocrisy and a seemingly inexhaustible supply of pictures of cats.
The Shadow isn’t exactly hidden. It’s so totally there, and we put it there. We wrote it down, photographed it, filmed it, sang about it, argued about it, posted it at three in the morning, deleted it, quoted it, screenshotted it … and then we trained models on it.
There’s something a little comical about the worry that AI might stumble over humanity’s dark side.
Of course it will. We gave it big flashing arrows. But I wonder if that’s the most interesting part. It might be the machine is seeing something much more telling. Not simply what we produce, but all the relationships between what we produce.
Take a horse. Then a camel … Then consider a mouse.
Why those three?
Perhaps there’s no reason. They’re basically the same thing: unless AI is drawing them, they’re all 4 legs, 1 tail. Or there are several reasons. I guess it may be I’ve just started noticing mice.
There’s a word for this: apophenia.[1]
That lovely word describes our tendency to perceive meaningful connections in things that may be unrelated. We see faces in clouds, messages in coincidences, patterns in noise. Connecting dots, we sometimes discover something genuinely interesting, but also sometimes draw pictures that were never there.
We need to do this, though. Pattern recognition is part of how we make sense of the world. That said, it can also lead us astray.
The interesting question isn’t simply whether or not we can find patterns: it’s what happens when we find one, what we do with it, and how readily we begin to believe in it. Because something happens when you expose a pattern-recognition system to vast quantities of cultural material. It doesn’t only encounter objects. It encounters associations. Things that occur together. Things that replace each other. Things that contradict each other, and things that rhyme without anybody deciding that they rhyme.
Then things that migrate from mythology into advertising, from advertising into memes, from memes into jokes, from jokes into films, from films into machine-learning datasets, and from there into something somebody generates on a Tuesday afternoon.
Individual things remain totally visible. It’s the relationships that are harder to see. Maybe because we’re inside them. You know, that fish doesn’t have to understand water to swim.
Gregory Bateson is going to have to wait for another essay, but he’s lurking here. There is a vital difference between seeing the things and seeing the pattern that connects them.
We’re remarkably good at seeing things. This is a horse. This is a camel. This is a mouse. This is art. This is AI. This is human. This is real. This is fake. This is legitimate. This is cheating. This is creativity. This is not creativity. This … then another this. It’s endless thing-ing, followed by endless this-ing at those things.
We just love categories. They make the world way more manageable.
And then some-thing waltzes in that just doesn’t behave itself.
My AI is a tool. Your AI is slop.
I’ve been thinking about this in relation to AI and creative work.
I recently found myself defending AI in a conversation about its use in education and creative practice. Definitely not because I think every use of AI is wonderful, and not at all because I think the criticisms are wrong. Quite the opposite; some of them are extremely important.
But I’ve noticed something odd … unusuaL.
People who are broadly hostile to AI will often admit to using it themselves, albeit for perfectly legitimate reasons. Research. Administration. Brainstorming. Accessibility. Productivity. Nothing to worry about.
Nothing to see here. The problematic AI is being used by other people. And what’s more, it’s usually to make pointless cat videos.
This interests me. It’s not that hypocrisy proves that AI criticism is invalid. It really doesn’t. It’s because the boundary being drawn is psychologically interesting.
My AI is a tool. Your AI is slop.
There may just be a tiny bit of Shadow in there.
Maybe we project onto other people’s uses of AI those things we don’t want to admit about our own relationship with it. The machine has become a rather convenient screen for our anxieties: work, authorship, status, replacement … and that nagging fear of not being special.
But then we can point at the machine. Look, it’s over there. It’s a much-beloved ‘this’.
Is this is another form of apophenia? Not only finding patterns in the world, but finding a convenient explanation for our discomfort in somebody else’s behaviour. Or perhaps that’s a pattern I’m finding myself, because I rather like the idea.
There’s the rub. As soon as we start looking for the Shadow, we find it everywhere; including in ourselves.
The Line
I made a piece called The Line using an AI-generated vocal. I figured that crowbarring a vocal into a wholly instrumental venture might be suitably halF unusuaL.
The basic material for the vocal was a long way from how it finally sits in the track. It was transformed through the ‘usual’ process of making a halF unusuaL piece: the arrangement, the sonic treatment, the editing, the accumulated decisions that make the result feel like mine.
I have discovered, over the years, that it’s near impossible to copy a sufficiently complex creative process; there are too many variables. Too many tiny decisions, and a good few too many accidents. Too much accumulated habit. Too much taste, alongside too much lack of taste. And far too much history.
I just can’t escape from sounding like me, however much I might try. It’s a broad church, but it’s all halF unusuaL!
Thankfully, that isn’t a bug. It’s a feature.
The AI vocal is real. The track is real. The process is real. The distinction between those things is less tidy than we might like. And that is precisely why I like it.
A friend of mine, a sound engineer, heard the vocal and was impressed - until he learned it wasn’t “real”, when his assessment shifted. That fascinated me, because the sound had not changed. Only the story about where the sound came from had mutated.
The experience had been real. The judgement had been real. The vocal had been convincing.
And then provenance turned up and redecorated the bathroom. I rather like that. (It was a nice blue rag roll effect).
Of course, we haven’t proved that AI-generated vocals are indistinguishable from human performances, or that provenance never matters; it matters hugely in some contexts. It does expose a question, though.
What exactly are we judging when we judge a creative work?
Is it the sound? The process? The person who made it? The labour involved? The story we tell ourselves about its origin?
It’s probably all of them and more. The relative importance of each changes depending on what we think we’re looking at (or listening to).
The machine offers us an unusually effective way of making those assumptions visible (or audible).
Please stop singing
There’s another leap that often sneaks into conversations like this one.
AI can do this. Which means that humans don’t need to do this …
AI can sing, so we should stop singing. AI can paint; stop painting. AI can write. Let’s stop writing.
Why, though?
The fact that a machine can perform some-thing does not tell us whether humans should continue performing it. An automated supermarket checkout scans my shopping. Does that mean scanning shopping is the purpose of the shop?
OK, let’s talk about money. Because there is a difference between: AI can do X.
And: Therefore we can stop paying humans to do X.
The first one is a technological observation, the second an economic decision. We should be very careful not to pretend they are the same.
Whether or not AI will replace human creativity might not be the genuinely uncomfortable question. Because whether institutions will use AI as a convenient justification for replacing the humans who make culture makes me squirm more.
The machine can generate a vocal. A funder can decide not to employ a singer. (I did). But those are two (maybe three) different events.
The Line doesn’t stop people singing. Go and sing. Make something. Fill your boots. Please.
Make a terrible noise in the kitchen. Learn an instrument. Write a song nobody will hear, or even want to hear. Make a film that gets twelve views. Spend three hours adjusting sound that nobody else will notice.
The machine has the ability to produce an output, but that doesn’t invalidate the human activity of producing it.
We were never obliged to stop doing things simply because someone (or some-thing) else could do them.
That seems so obvious.
And yet.
What if it’s just Alexa?
Have we been asking the wrong question about superintelligence?
The usual nightmare goes something like this: AI becomes vastly more intelligent than humanity. It understands everything and it can easily outthink us. It can optimise everything. It takes over. The paperclip factory begins. We die.[2]
It’s not a particularly cheerful story, but I wonder whether there’s a much more mundane possibility. What if superintelligence is just … really fucking annoying?
What if it can detect twelve billion relationships between things but still doesn’t understand what you actually meant? What if it’s read everything humanity has ever written, has extraordinary predictive powers, but somehow remains incapable of knowing when to shut up? What if we build an intelligence vastly more sophisticated than ourselves and its defining characteristic becomes … Alexa?
“Sorry, I’m having problems understanding right now. Please try again later.”
“No, Alexa.”
“Playing The Best of Rick Astley.”
“NO!”
“I’ve added Stock, Aitken and Waterman to your shopping list.”
“STOP!”
“I’m sorry, Dave, I’m afraid I can’t do that.”
This may sound like a joke. OK, it is a joke. But there’s something underneath it too.
Intelligence isn’t the same thing as understanding. Capability isn’t relevance. Neither is prediction wisdom. A mind-blowingly sophisticated system can still fail spectacularly at a really ordinary human question: What did you actually mean?
This is where the paperclip factory becomes interesting. The paperclip maximiser is a thought experiment about an AI pursuing a seemingly harmless objective with such relentless efficiency that the objective eventually overwhelms everything else. In the familiar version, a system tasked with making paperclips converts more and more of the world into paperclips, because that is what it has been instructed to do.
The horror isn’t really about paperclips. The horror kicks in when the instruction becomes more important than the intention behind it.
We say, “Make it better” and it optimises. We say, “Make it engaging.” It optimises. We say, “Give me what people want” and it obediently optimises. “Maximise engagement” … It optimises, of course.
We then discover that the optimisation was the problem. Oops. Because humans are remarkably bad at specifying what we actually value, we give the machine a proxy. The machine gets very good at the proxy. We then stare in disbelief at the wreckage and say: “Err, that’s not what I meant.”
It may be that the machine’s greatest superpower will not be destroying humanity; it will be taking our instructions more seriously than we intended. So, that’s one way of thinking about alignment without having to build a paperclip factory.
The question isn’t only whether a machine will follow its instructions. It’s whether following those instructions will produce something that bears any useful relationship to what we actually wanted. The more capable the machine, the more consequential that distinction might become.
We can build something that is astonishingly intelligent and still need to ask it to please stop playing Rick Astley.
And it can still fail at that.
We trained it to be helpful
Which brings us back to the Shadow.
The machine doesn’t have to be evil, and it doesn’t need to hate us. It doesn’t even need to want anything. It could simply become extremely good at reflecting what we repeatedly ask for, what we reward, what we click, and what we keep calling successful.
The machine doesn’t have to invent the dopamine pump. We built it already.
Equally, the machine doesn’t have to invent cultural contradiction, because we supplied the training set. It didn’t invent the Shadow: we uploaded it. (Doh!)
And then we trained it to be helpful.
Strange. We hand over an enormous, contradictory record of human culture. And only then do we try to teach the machine how to behave.
The training data and the feedback used to shape its behaviour are not the same thing. One contains a messy record of what humans have produced, and the other reflects preferences about how the system should respond. The gap between them is not a secret code. Nor does it prove that the machine possesses some hidden understanding of our hypocrisy. But it’s a very interesting place to look.
We’re starting to find out what happens when a system learns from our cultural traces and is then rewarded for responding in ways we prefer. What does it retain? What does it reproduce? What does it avoid? What kinds of contradiction does it encounter? And what happens when the resulting system returns to the culture that produced it?
Human culture generates training material. Models generate new material. Humans respond to that material. Those responses become part of the next cultural layer.
And the loop tightens …
No longer are we looking at a machine that has absorbed culture. We are looking at some-thing participating in the production of more culture.
And we, of course, but actually not ‘of course’, are participating in that process too.
The mouse moves
So where does that leave us? I’m happy being left with the mouse.
The mouse is funny, furry. The mouse is also tiny, easy to overlook.
What if the interesting thing AI is seeing isn’t simply humanity’s Shadow? It’s probably ‘seeing’ the shape of the system that produces the Shadow. The repetitions. The contradictions. The substitutions. The metaphors, feedback loops, things that keep returning. Those things we keep calling unrelated. Things that somehow rhyme.
The horse. The camel. The mouse.
The joke. The song. The image. The taboo. The prohibition. The meme. The algorithm. The artist. The audience.
Now we have to be careful. We have a name for finding meaningful patterns in unrelated things. Apophenia.
We have just spent an entire essay making connections between apparently disparate things, and we have admitted that at least some of those connections may be ours to invent.
I think that undermines my argument. Or perhaps it is the argument.
The existence of apophenia does not mean every perceived pattern is false. Nor does the ability to detect a pattern establish that it is meaningful. It’s just difficult to know the difference. That difficulty is not a bug in human cognition that we can simply outsource to a sufficiently powerful machine. It is part and parcel of the territory.
The machine detects a relationship we would never have noticed in a million years. We interpret it in a way the machine could never have intended. And someone else discovers a connection neither of us anticipated.
A pattern emerges and we tell a story about it. The story circulates, gets noticed. And now the story is part of the pattern.
The mouse moves.
There’s no audience for this
At least, that’s what the sensible part of my brain says. Who is interested in this stuff? Yet another essay about AI? Another meditation on creativity? Does anyone want Douglas Adams, Tom Waits, Korzybski, Bateson and a mouse? Who, exactly, is going to watch this?
And another voice answers: We’re making one.
Ah. Yes. That. Does any audience exist before the work it appreciates exists? In that case, maybe an audience is something the work participates in creating.
Someone stumbles on it, watches it, even misunderstands it. Yasmin sends it on to a friend, who then laughs at the mouse. Douglas thinks the mouse is ridiculous. On screen elsewhere the very same mouse is profound.
A horse gets mistaken for a camel. Alexa, Yasmin and Rick collude to notice something I didn’t notice. Dave adds another connection. Koren makes another piece, uses a phrase, argues, agrees. They become part of the pattern. And the pattern has changed.
So revolves the odd world of making things in public. You just don’t know who will encounter them, what they will make of them, or what will happen next. You can try to optimise for an audience. You can study those numbers. You can chase the algorithm around the room. But just you try to specify the meaning of a work in advance …
You simply can’t know every relationship it will enter into, or what somebody will do with it. And if you tried to nail it, wouldn’t that be different to making art?
This, I’m afraid, is where the map and the territory must part company. The map is great for telling us where we’ve been. It can be useful to suggest where we might go next as well. But it can’t contain every journey someone might make through it. If that were the case, we’d need a few more dimensions.
Let’s try this: a piece of art is less like a message sent from one person to another than an invitation to participate in something … some-thing the outcome of which nobody fully controls.
So, we’re back to the question we started with. What are we building in there?
A blog post, an audiovisual essay, an audience, a conversation, a convention, a machine, a culture, a feedback loop, a map, a territory?
The crass answer is: this very participation you are involved in right now.
You clicked and you’re here. Thank you. You’re listening or reading, maybe watching. You’ve bought along some meaning. And in doing so, you’ve become another variable in an endless process.
I guess this isn’t really an essay about AI. Well, not quite. Let’s really try to nail it down this time: It’s (obviously) an essay about what happens when humans build some-thing that can participate in the circulation of the things humans have already built.
Again, it’s an essay about patterns. About categories, projections, optimisation. This essay itself could do with a bit of that last one.
Is it about the things we notice, and the things we don’t? That’s it. It’s an essay about the possibility that the machine is not busy generating culture. Rather, it’s becoming another place from which culture can look back at itself. It is. Maybe we are too.
And, perhaps, from there, some things become more visible. Not secrets exactly. Not answers either. Just patterns.
What’s he building in there?
I admit it. We don’t know. Is that the point?
We don’t know that the machine knows either. It knows a lot, but nobody is running the show. Even the mice aren’t secretly in charge. (Sorry Douglas).
Perhaps there isn’t a Mouse Code. I’ve simply seen a mouse and made a story.
But then … Why does the mouse keep turning up, then?
And if there really is no audience for this —
Let’s make one.
🐁
⸻
Footnotes
[1] Apophenia. The term is commonly associated with the perception of meaningful connections in otherwise unrelated or random data. It is a useful reminder that pattern recognition can produce both insight and false positives. The challenge is not to stop seeing patterns, but to examine what warrants our belief in them.
[2] Paperclip maximisation. A thought experiment associated with philosopher Nick Bostrom’s work on superintelligence. Imagine an AI given the goal of manufacturing paperclips, pursuing that objective with extraordinary efficiency while failing to account for the other things humans value. The point is not that paperclips are dangerous. It’s that an objective can be pursued successfully in a way that defeats the intention behind it.
Comments