AI, the Genie

When talking about AI tools, they're often called "assistants". That word conjures up someone imminently capable, who will do their best to fulfil the requests made of them in letter and spirit. This is aspirational. Kent Beck, legendary programmer and co-creator of the original Agile manifesto, has picked a more apt metaphor: That of a genie.

In mythology, genies have to grant the wishes of their captors. But they're more than happy to grant them in letter only, and cause much mischief within those parameters. In coding tasks, that manifests itself in obviously nonsensical behaviour like, "I can make the tests pass by removing them!"

Armed with this metaphor, we can make better choices in using these tools: How do we instruct the genie so that it can't mess things up for us? How can we design a system where unleashing a genie behind the scenes ultimately creates something helpful and aligned? How can we build mechanisms of trust and verification into the system?

The genie won't go back in the bottle, so we might as well make the most of the wishes it grants us.