AI Doesn’t Learn

AI doesn’t learn. That might sound contradictory. Isn’t it called machine learning? What I’m talking about here concerns large language models (LLMs). In their creation, learning takes place. But once they’re done, they remain static and unchanged, unless the creators deliberately add on additional training runs.

It’s a bit like the main protagonist in Christopher Nolan’s Memento or, if you’re in the mood for lighter fare, Adam Sandler’s 50 First Dates. The characters have long-term memories until a traumatic incident, but cannot form new ones, and their short-term memories regularly reset.

Why it matters

If we want to treat AI like a capable personal assistant, we’d like it to learn and improve at its tasks over time. We want to avoid repeating ourselves when we give feedback.

Current AI systems have to rely on emulating learning. There’s ChatGPT’s memory function, for example. During the conversation, it will identify and store salient information about you and your preferences in a database. And then, behind the scenes, whenever you ask it a question, it’ll use that information to provide additional context to the underlying language model.

Higher-level learning and insight

These clever tricks allow the AI tool to build a repository of information that’s highly relevant to what you use it for. However, a higher-level feedback loop around growth and mastery is missing: If you repeatedly assign the same task to a human, they’ll become more proficient and efficient over time. They might even identify shortcuts or ways to automate or eliminate the task altogether. Assign the same task repeatedly to an AI tool, and it’ll work diligently on it the same way it’s always worked. That’s fine for small rote jobs, but it will be a significant handicap for the larger jobs we’d like AI agents to handle.

I’m curious whether simple tricks similar to ChatGPT’s memory will enable a fake-but-useful version of this higher-order learning, or whether we need a completely different AI paradigm for that.