Tackling AI Projects
Right before the (long, in Canada) weekend, let’s start talking about how you can tackle an AI Project in a way that does not end up in tears of frustration (or just lack of interest).
I’d hope to write a number of posts about this topic. For now, let’s start with one big high-level question: How much AI are we talking about?
Little AI, Lots of Other Stuff
It might very well be that AI plays a small but important role in a much larger workflow, where the majority of steps are handled by traditional software or more classical machine learning as opposed to generative AI. An example would be any pre-existing tool that incorporates some AI into it as a way to speed up some steps, but isn’t strictly necessary for the product to function. Like a todo-list app with an AI feature that breaks bigger tasks into smaller ones for you. Neat, but not essential.
Lots of AI, Little Other Stuff
Or you have something in mind where the AI does all the heavy lifting, and the other surrounding stuff is minimal skeleton that lets you get to the AI with as little distraction as possible. The extreme example of this are the chat interfaces to large language models, especially in their original form without extra frills: You get a text box and a send button and that’s all you need to get to the latest super powerful LLM.
Why does it matter? Because it determines where our risks lie and where efforts need to be focused. In the first case, we have more software engineering to do and (comparatively) less AI engineering. It’ll still be important, but we won’t have to squeeze as much out of the model as we’d have to do in the other case. If your product is the AI model and little else, it better be really good. In this case, you’ll spend a lot of effort on AI engineering.
Somewhere in the middle
Unless you already have an established non-AI product and want to sprinkle a bit of it in there, chances are you’re not in the “Little AI” situation. But unless you’re building the next big foundation model, you’re probably also not in the “Little Other Stuff” camp. In that case, you’re looking at a complex workflow that also requires a fair amount of AI that’s not optional. An example here would be a complex industry-specific document review tool. It needs a lot of scaffolding and supporting workflows, data storage, user management etc., but it also needs an appropriately engineered AI solution.
The right skillset for the right situation
What’s the right team for these scenarios? Here’s my take:
Software Engineers with a bit of AI training will handle Little AI just fine
No need to hire Prompt Engineers for $400,000/year. Just have your engineers take a stab. In this scenario, the AI is likely just one more external tool that gets incorporated, like any other third-party tool they’re already using.
ML Scientists can build the little stuff around their AI models
When it’s all about the model and you just need basic scaffolding to let it shine, your ML Scientists already have a great ecosystem of tools they can use. Gradio, Streamlit, and other tools let you hook an AI model up to a nice web interface in no time.
A mixed team for the complex middle
As much as I’m against artificial skill boundaries (as we’ve unfortunately seen with the Front End / Back End split), I believe that projects that have both complex “traditional” software needs and complex AI model requirements need a mix of two roles. Someone to handle the AI engineering and someone to handle “everything else”. At the very least, you’d want a two-person team with a strong software engineer and a strong AI engineer.
So, that’s it for the start. Once you’ve got an idea what sort of project you’re looking on and who the right people are for it, you can go to the next step. More next week!
