Brave

I just finished watching Brave. I was ordered to watch it, as all Disney films Must Be Known, but had been putting it off for a while. It’s pretty much what one could expect of a Disney film - the princess has duties, but wants to live her own life, and over the course of the film both she learns the value of tradition, and her parents learn the value of letting teenagers have their way. Standard fare. With very pretty landscapes and scottish accents.

The film itself is quite good, with a decent plot and character growth. The main character, Merida, is stupid. I know she’s a teenager princess who’s been brought up to not have to worry about anything (apart from behaving like a princess), but I find it hard to watch such recklessness. The film starts by her running off into the woods as a toddler to find a lost arrow. Though I suppose that’s on the parents, as it was their idea. She gets a day off each year to do as she wishes, which is spent galloping of into the wild on her carthorse to free climb wet cliffs. To be fair, this too can be blamed on her father, who says that is something that Is Done, and it’s also something I’d have done at her age. Anyhoo - the film starts by establishing that she’s not a staid, respectable girl. Which I suppose is fine? Who am I to criticize? My real issue is how she reacts to the main conflict. Not that she runs away from home after quarreling with her mother - that’s totally normal, if suboptimal. My problem is that when she finds a witch, she asks for a spell to “change her mother”. That’s all the instructions given. No more. No extra questions, no clarification, no attempts to establish the trustworthiness of said witch. And once she gets the spell, in the shape of a fruit tart, she gives it to her mother to eat. WHO DOES THAT?! She has no idea what it will do, other than change her mother. She didn’t even specify what aspect was to be changed! Or direction! What if the spell changed her to be even more convinced that Merida should quickly choose a husband? That would totally fit the request. Of course the spell does what it’s supposed to do, i.e. changes Merida’s mother. The rest of the story is how to revert the changes.

Matthew Yglesias wrote a article about how Terminator is actually a decent into to AI safety. While the specifics might not be - it doesn’t seem likely that time travel will save the world - Skynet is a useful anchor to start talking about the general problems. Ditto 2001: A space odyssey. They depict certain failure modes, albeit with extra bits to look nice. It’s like the difference between reading The Plague and the Myth of Sisyphus - the same ideas can be found in both, though one of them is more indirect in it. Fiction, especially Science Fiction, is good at packaging real world issues into stories which are widely consumed. Ideas which if presented in a scientific paper might be more rigorous, but a lot less known.

Brave seems to also be an AI cautionary tale. Or rather the old classic of being careful what you wish for. Which is a large section of alignment problems. Also a popular source of bugs, especially when one has only just started programming. In this case the witch even ends up in a very computer like form, with Merida’s later interactions with her in the form of an automated cauldron replier.

Merida’s case is an over simplified one. Seeing as it’s supposed to be obvious to everyone that such wishes are very dangerous. Especially as it turns out that, spoiler alert (hehe), the main baddie also didn’t think through the ramifications of wishing for the strength of 10 men… There are two main takeaways from the stupidity of Merida and the previous customer. The first, obvious lesson is to be very careful when asking for magic spells. There is the basic phrasing which is always crucial - you want to make sure that the granter understands exactly what you want. Of course, that doesn’t suffice. You also have to make sure that you completely understand how the spell will work. Even if these can be achieved, you still have to be able to assess the granter’s moral alignment, so you don’t just monkey paw. The witch in the film was by no means malicious. At least she didn’t seem so. Just in a sense amoral and detached. You paid your price and got what you wished for. She even set up the cauldron to give the extra info that she forgot to provide earlier. This, of course, didn’t change anything. The road to hell and all that.

The second lesson is more meta. Merida behaved rashly. That is undeniable. Maybe understandable - being a princess maybe she was used to people doing what she wanted, not what she asked for? But stupid, even if she didn’t know better. I fear that if an AI was created that in some way could do stuff that would be indistinguishable from magic, whoever would claim a boon of it would probably also fail. Not in such an obvious way. But it seems inconceivable that Merida is obviously stupid but that a decently intelligent human (as I assume such would be the one making the wish) is the pinnacle of precision. Especially seeing that my job consists of trying to get computers to do what I want… This has all been described a lot better than I can in the sequences.

My basic point is that yes, Merida was stupid. But that I’d probably be just as stupid, albeit in a lot more intelligent and precise way. That while I’m far from being so absolutely irrational as to wish for something to “change my mother” and then give her some random pie from a cauldron of toxic goop, I’d probably pat myself on the back for fixing all loopholes, just to see it blow up in my face in an unforeseen way. Maybe the Luddites were right…?