AI Product Decisions Need AI Experts In The Room

I'm reminded again how unintuitive at times the complexity landscape is with machine learning and AI. Reviewing the use cases and user stories I've come up with in a workshop with a fintech founder, lots of open questions remain. They're ultimately driven by what the product should do, but small nuances can require completely different machine-learning approaches.

With non-AI tech, we've built up enough intuition so that even non-technical folks will have a good sense of what's a complex feature and what's an easy feature. With AI, it's still quite elusive. In our example (with details omitted because it's confidential), the AI is supposed to do some sort of classification. But whether those categories are pre-defined with simple rules, or have to be learned from labelled data, or should, in fact, be discovered fully autonomously, makes a world of difference.

A bad engineer would just say yes to the first thing the client described and start building (and billing). What's needed is a tough conversation where the product side and the engineering side hash out which combination of solution and cost is satisfactory: "We can build it like so, which will take 6 months, or we can build it like so, where it can do this but not that, but it'll be done in 2 months."

Back in the world of agile user stories, it is said that stories must, among other things, be negotiable. The "why" is given, but not the "how". For anything involving AI, this makes all the difference.