User Stories Revisited
Here's one reason AI initatives end up in Proof-of-Concept Purgatory, never to see the light of the real world: Not enough high-bandwidth communication and iterative development between the users and the builders.
Of course this afflicts projects of all types, but at least in recent years, "normal" software projects did quite okay. Take, for example, this explanation by the makers of Linear (a beautifully made project management and issue tracking software) on why they recommend tracking "issues" instead of "user stories" (the agile concept of writing the things you want to work on from the point of view of the user):
To quote,
User stories evolved over twenty years ago as a way to communicate what a customer wanted into product requirements that a software team could deliver. Fast forward to today and a lot of things have changed about how we build software. Customers are tech-savvy enough to articulate basic product requirements. We’ve developed standards for common features such as shopping carts, todo lists, and notifications so there is no need to explain how they should work. The best product and engineering teams understand their users deeply and are familiar with how their product should work.
Source: Write issues not user stories - Linear Method
If a client wants a web shop, they'll talk about the shopping cart, and everyone has a shared mental picture of how that should generally behave. Some discussion for full alignment are in order, but you won't see wild swings that miss the mark completely.
Not so with AI. There has not been enough time for shared norms to emerge where everybody agrees what terms mean, how systems should behave, how AI capabilities should be presented. I've seen this directly affect project outcomes when talking to people who participated in AI initiatives: Things get lost in communication and now there is a wide gap between what the users wanted and what was delivered. In essence, this cartoon is relevant again. In a concrete example, the users needed a tool that gets a good draft going, with lots of iterations. What was delivered was an overengineerd tool that tries (in vain) to get a perfect version out in the first shot (to save precious processing time). The users assumed it was obvious that a tool would work the way they expected. The engineers assumed their way was the best.
The way out? Communication. To the point that it feels like too much communication. Exactly how will the tool fit into the existing workflow and the larger ecosystem of its company? How will results be delievered? Do we need speed, or accuracy? What are the thresholds for each? Lots of questions, and you can't, for the time being, assume that some things are too obvious to mention.
User stories, as they were originally conceived, were about realizing this need for thinking from the users' point of view. Communication is important, but certain decisions cannot be made by the user. Instead, they require that the designers and engineers understand, at a deep level, what the users want to accomplish. That's then the one decision that takes care of a thousand little decisions down the road. What does decidedly not work is a user asking for a feature without revealing their intent, and engineers rushing ahead with an implementation without pushing to know that intent.
