Quantum Won’t “Save” AI

I've seen an uptick in commentary and headlines along the lines of, "Oh well, current large language model progress is plateauing, so we won't have Artificial General Intelligence next month; but with quantum computing, we'll soon have it, because... quantum" (waves hands).

I've worked in the quantum computing sector and am still in contact with my former colleagues (👋 shoutout to the 1QBit team!) to say with reasonable confidence: Quantum computing won't do anything meaningful for the sort of AI a business would care about, and certainly not for large language models / generative AI, for the foreseeable future.

Yes, important and exciting work is happening. Progress is steady. Multiple players are advancing the state of the art, and I'm certain that great things will come of that.

No, none of this matters for AI systems that work at the scale of a GPT-5.

Quantum computing is not a drop-in replacement for classical computing, where you just replace a conventional CPU with a quantum processing unit (QPU) and off you go. Instead, it's specialized hardware designed to solve incredibly narrowly defined problems, such as factoring a large number or determining the ground-state energy of a spin glass. The latter is what the D-Wave quantum annealing hardware is designed to do. If you do some clever math, you may be able to cast other problems you actually care about in those terms, particularly in scheduling and optimization. None of these use cases matters for training a gigantic machine learning model. (There is a quantum algorithm for solving linear equations, but its requirements in terms of the number of qubits are beyond ridiculous for current quantum hardware.)

In a way, the computational paradigms behind AI and quantum are opposed to each other; on the AI side, we're dealing with staggeringly large models with billions of parameters, on the quantum side, we're (currently) dealing with, at best, dozens of usable qubits.

It's almost as if, now that the irrationally exuberant hype is wearing off, certain tech influencers (and CEOs of quantum hardware companies?) latch onto the next topic for their hype. Blockchain. VR. AI. Quantum. All of these have kernels of usefulness that are at risk of being crowded out by undifferentiated hype.

Instead of dreaming about living in the Star Trek universe with sentient androids, holodecks, and faster-than-light travel, let's focus on solving actual problems with existing and proven solutions.

1000x Faster Monte Carlo Simulations

I've written before about using the right, simple, tool for solving a problem, rather than going after the shiny new thing.

One such example: On a previous project, we achieved great success using relatively simple machine-learning models to achieve massive speedups in the complex simulations that a large insurance company or financial institution would run to manage the risk of their portfolio.

Massive here means that, instead of spending 80 hours for a complete run, it now takes a couple of minutes. This is, of course, a massive unlock. You can either save the time and use it elsewhere or spend the same amount of time doing a much more thorough analysis. These sorts of risk calculations are often required by regulators, with hefty penalties if reporting doesn't happen on time.

Despite this success, the technique, as far as we can tell, is not widely adopted. That's why we've decided to run a short, free webinar on the topic. It will take place this October at 10 a.m. Pacific Time, which corresponds to 1 p.m. Eastern Time and 7 p.m. Central European Time.

Who is this for?

• People interested in applying machine learning to financial and other statistical simulations

• Insurance analysts, quants, and actuaries tired of long runtimes

• Risk modelers who want to integrate machine learning into existing workflows

• Analytics and data science teams pushing against time, compute, or compliance pressure

Check out the event page and register for free here and tell your friends in finance and insurance.

Big Consulting Agile

I've now heard this story from multiple independent sources, working at completely different companies:

1. Leadership brings in a big consulting company to "help with efficiency"

2. The consultancy introduces by-the-book Scrum, a popular agile framework: Two-week iterations, story point estimates, and all the roles and ceremonies associated with it

3. The consulting company collects a fat check and leaves

4. Employees are unhappy with the overhead and heavy-handed processes, and efficiency does not, in fact, increase

The problem: Neither of these companies was a traditional software company. They were a research-first hardware company and a large "legacy" industrial company, respectively. Work there just does not fit neatly into two-week increments of small, estimable user stories. In the case of the former company, the fellow I talked to complained:

"Now I can't just go read a research paper. No, I have to write a user story first about what I'm researching. Then I have to give an estimate for how long it'll take me to read that paper, and every morning during standup, I have to say that I'm still working my way through the paper."

Doesn't that just sound like the opposite of agility?

In the case of the industrial company, the lament can be summarized as, "Everything we do is on a large scale with complex interlocking processes; nothing there can get done in two-week increments."

Now, with AI, many companies are in danger of repeating the mistake of using the wrong methodology to explore it, by going too wide too soon, and adopting a top-down mandate driven directly from the C-suite, supported by a one-size-fits-all playbook courtesy of the Big Expensive Consulting Co.

Companies would do well to remember Gall's Law, which states that anything complex that works must have gradually evolved from something simple that worked. This goes for adopting agile methodologies as much as it goes for integrating AI into the company. Small pilot, learn what's required for your company specifically to make it work, and don't expect much value from an off-the-shelf, by-the-book transformation, whether it's agile or AI.

Lessons from Harvey AI

An anonymous person posting on the social media platform Reddit claims to be a former employee of the Legal Tech startup Harvey AI. They allege that the tool has low internal adoption, is favoured more by leadership and procurement than by those doing the actual work, wasn't built in close collaboration with actual lawyers, plus a number of other criticisms around the product’s quality.

While Harvey's CEO responded and countered these claims, there has been a lot of schadenfreude from others in the legal tech industry, as well as plenty of piling on from AI skeptics. While I'm in no position to judge who's right and who's wrong, we can still extract some lessons, based on the complaints levelled by the anonymous Redditor and the other practitioners.

Biting off more than you can chew

It seemed to me, back in 2023, that Harvey was starting with an overly broad mission: essentially feeding a large amount of legal documents to an AI and having it become proficient at writing legal documents to the point where you could replace, if not your senior lawyers, at least a bunch of your paralegals. Yet, even if a large language model is fine-tuned with incredibly industry-specific material, it only delivers value when plugged into a concrete workflow aimed at solving a particular problem. Lawyers (presumably) don't just want a ChatGPT that's aware of how lawyers write. They want tools that tackle specific tasks, such as drafting and reviewing contracts.

From the observed criticism, I get the impression that Harvey is sort of "bleh" at a lot of lawyer-like tasks, but not amazing at any one of them. If that's true, then it's no surprise that adoption is lacking.

There was a sort of irrational exuberance in the air right around GPT version 3.5, where it seemed the winning formula would be to take an off-the-shelf language model, finetune it with proprietary industry-specific data, and instantly get an expert that could handle any task in that industry. By now, we know that this isn't quite the case, as in the recent MIT study about enterprise AI pilots.

What we must realize is that AI doesn't let us skip proper product development. AI might enable previously unthought-of capabilities inside a product. However, the rest of the product still requires solid engineering, user experience design, and all the other pesky things that are hard work, requiring human insights.

Why 95% of AI Initiatives Fail And Why Yours Doesn’t Have To

You've probably come across the striking headline that "95% of enterprise generative-AI pilots" fail, with failure defined as "no measurable P&L (profit and loss) impact".

Read the full article here if you're curious about the research methodology and exact findings. Here, instead, let us focus on takeaways.

What goes wrong

There are a lot of reasons mentioned in the report. A few standout ones:

• Poor integration into actual business workflows

• Unclear success metrics

• Top-line hype instead of concrete use-cases

Incidentally, we've written about all these before (check out our archive) or, in particular, these:

• AI Project Outcomes, Part 2

• We've been here before

• AI Adoption for the Skeptical

It's a nice validation of our thinking.

How to get it right

To distill the whole article—with the pitfalls and the things that those who succeed with AI are doing right—into a single sentence, I'd say:

• Start one pilot focused on a single, measurable back-office process and define the P&L metric before building.

No sweeping, company-wide digital transformation, no press-release-driven bravado, no chasing after shiny objects. Just one area where your well-paid knowledge workers (engineers, lawyers, copywriters, you name it) waste time on a back-office process that's not part of their value creation chain. Declare what success looks like and then go build and iterate.

Finally, the researchers found that your success rate increases dramatically if you bring in a specialized partner who can help you bridge the tech-business gap, rather than going it alone. If that sounds intriguing, hit reply and let's start a conversation.

We Need to Talk About Workslop

We've got a strong contender for word of the year: Workslop. Defined in this article by a research team at BetterUp Labs and the Stanford Social Media Lab, it refers to "AI generated work content that masquerades as good work, but lacks the substance to meaningfully advance a given task."

The article is well worth a full read. The pernicious thing about workslop is that, at first glance, it appears to be of high quality. A well-structured report, polished slides, etc. However, because it's not carefully reviewed and crafted, it actually creates more work for the other people in the organization, who now have to review and redo the sloppy work.

My own takeaways:

• This is what you get when you vaguely demand that your employees use AI to boost productivity, without clear goals or measures on what that would entail.

• It's also what you get when "visible activity" is valued above concrete outcomes—what Cal Newport calls pseudo-productivity. You reward people only for the number of emails they send? You'll get a whole lot of rapidly generated and ultimately useless emails.

• Finally, the ease with which AI generates polished outputs can paper over processes that are inherently inefficient and shouldn't exist in the first place. Yes, thanks to AI you can rapidly generate all sorts of reports—if only at low quality—but is anyone even going to read them? (They'll probably read the AI-generated summary 😛)

What to do? As is often the case, the answer was there all along, made more acute by the generative AI revolution: empower people to own outcomes instead of outputs and hold them accountable for them. For example, don't ask for a report or presentation that compares different solution providers. Ask for a decision on which solution provider to pick. The person working on that task might ask AI to create an initial overview, but because they'll have to defend their choice, they won't just send it off to some poor coworker to sift through.

In short, don't just yell at your workers to stop generating AI workslop. Let them own the outcomes of what they create, and the problem will take care of itself.

AI Advice Must Come From the Trenches, Not the Sidelines

There's a common saying that I dislike: "Those who can't do, teach." If teaching isn't informed by practice, it tends to drift into the theoretical. Especially in fast-moving fields such as AI, with numerous pitfalls and nuances, only those who also practice are qualified to teach and advise.

How else can you help a company pick the correct path to pursue with their AI initiative if you haven't walked that path before? In slower-moving fields, you can rely on a wealth of accumulated knowledge, slowly absorb it, and then disseminate it: You don't have to lay brick or cut wood to be a successful architect; you can learn the relevant material properties from a textbook. After all, wood, stone, and concrete, among others, don't undergo dramatic progress every few months.

But with your AI projects and initiatives, you'll want to get advice directly from those who are building and doing, not just theorizing. This, of course, goes against the common industrial-scale consulting model: A hyper-effective sales team creates FOMO (fear of missing out) and/or promises untold riches if only you buy their services, then hands you off to their engineers who're now tasked with doing the impossible (or delivering something over-built). Or, a seasoned strategy consultant draws up all sorts of ares where your company would be guaranteed to benefit from AI, but their suggestions are completely unburdened by actual feasibility because that person has never gotten their hands dirty.

So when you're shopping around for advice on AI, seek out the nerds and hackers and doers. You might not get as polished a PowerPoint deck, but you might just save yourself a lot of headache and wasted effort.

When are you ready for AI?

It might be sooner than you think.

I've heard some department heads say that they want to explore AI, but only after they're ready. It's wise not to rush headlong into an AI initiative—that's precisely how you get it to fail and join the 95% of AI projects that fail. However, there's also such a thing as waiting too long for a perfect world that never comes. Here then are a few checks for whether you're ready to start properly exploring AI:

• You think the way your company does things could be improved, but at least there is a well-defined way your company does things. If everything is a hot mess, throwing AI at it will give you a white-hot mess.

• Data isn't perfect, but it's available. Modern AI can handle messy data. What it can't handle is an absence of data. And even there, you might need less data than you thought. LLMs have already been trained on the entire available written word, so getting them to understand your specific data might not require gigabytes of it.

• You know what you want to achieve. Nothing leads to project failure quicker than a lack of clarity on this point. There are dozens of ways a given problem can be tackled with (or without) AI, and each way has its own set of tradeoffs. Only absolute clarity on what "solved" looks like can the tradeoffs be matched to the requirements.

How did you score?

• 3 - Absolutely ready. Don't wait for perfect. Just start implementing your initiative. In-house, or with a trusted partner.

• 2 - Almost ready. You just need a little nudge to remove the last hurdle before you can start implementing AI. Perhaps a quick call with us can help you get unstuck.

• 0-1 - Not quite ready. There's still a lot of groundwork to lay before you can start fully adopting AI. These steps might be projects in and of themselves, such as building a solid data pipeline, mapping your business processes, and defining success criteria for an AI initiative. You might also benefit from ongoing coaching in this topic. Most importantly, you'd want to pick an area where we can work iteratively, in small steps, with low-hanging fruit and easy wins available.

---

At AICE Labs, we'd love to help you do AI right, no matter what stage of your journey you're in. Whether it's a custom project right away or providing you with the advice to get you there, we've got the experience to get you unstuck.

Problems Vs Solutions

Common advice for founders and other problem solvers is to fall in love with a problem, not a solution. This is meant to keep people from building "solutions" that nobody wants. These days, that means avoiding the temptation to shove AI into just about everything.

On the other hand, I find this advice incomplete. It's solid when technological progress has been slow and steady. But when novel technology arises, the game changes. You will want to take a really close look at new tools and ask: Where could I apply this?

Think about other monumental technological achievements, like electricity. It was so different from everything that came before that it made perfect sense to ask: What are all the marvellous things we could do with this?

Same with AI. It really is a monumental shift in what computers are capable of doing. Why not brainstorm a long list of ways to apply this in our lives and business? The initial advice, to avoid obsessing over the cool tech of the solution rather than the actual problem, is still sound. Once you have your brainstormed list of AI use cases, ask critically whether it tackles a real problem: Does it cost you time, money, energy, or peace of mind? Have you tried solving it before but ran into challenges that AI could overcome? If not, move on, because a use case that's not for a painful problem isn't a real use case. But if yes, dig deeper and define what "solved" would look like, and then go try solving it (or hit reply and chat with us about solving it).

That’s Not an Agent

There are two places where I've seen people misuse the term "agent". One of them is benign, the other not so much.

First, the benign version. Talking with potential clients, they're genuinely curious about AI but aren't necessarily familiar with all the fine distinctions. So they have an idea for where AI might help them, and they call that solution an "agent". That's not the place to barge in with a "Well, actually... ". What matters more is their intent and the problem they're facing, as well as what "solved" would look like for that problem. Once we design and present a solution, we'll explain that the final product may or may not end up being an agent. What matters is that the problem gets solved.

Now for the not-so-nice version: Folks who sell something software-related, knowing full well that it's not actually an agent, but they call it that to tap into hype and fear. I've seen simple automations ("Post a message to the team chat when a user files a bug") described as "our customer support agent". Ouch. If it's not a large language model (or multiple, at that) embedded in a system with a feedback loop, autonomously invoking tools to achieve an outcome, it's not an agent.

Why does it matter there, and not in a client conversation? Because if we're selling a service and positioning ourselves as experts, we have to be precise in our communications. We have to stand for what we advertise. You get what we say you get, and it won't be dressed up in colourful, hyped-up language.

Needless to say, if you're looking for someone to blow smoke and sound fancy, you can go somewhere else. But if you're after someone who'll solve challenging problems with what’s appropriate instead of what’s hip with the tech influencers, we're right here.

Don’t Distrust The Simple Approach

Phew, it's been a while. Summer, start of school, travels. Anyway.

I've recently come across multiple situations where simple is best, but gets overlooked in favour of something complex:

• I've had discussions about diet and exercise. Simple (eat less, move more) is best, but people don't trust it, so they have a 12-step exercise routine and a complicated schedule of exactly which food group is verboten at what time of day.

• I've had finance folks reach out about investing. Simple (index funds matched to your risk tolerance) is best. Still, people don't trust it, so they want a complicated, actively managed portfolio that gets adjusted every time the US president sends a tweet.

• I've chatted about strategy and consulting with a friend. For exploratory work and new initiatives, the best approach is to just start and iterate on the feedback. But, of course, that just seems too simple, so instead we ask a big consulting company to make a deck with 60 slides, complete with SWOT analysis, 2x2 matrices, stakeholder personas, ROI projections, a RACI chart, a change management framework, risk register, industry benchmarking, and an executive summary that uses 'synergy' unironically.

We're all smart people here, so we have domain experience that's genuinely complex. That can bias us to distrust simple solutions. What we should adopt is a mindset that distrusts complexity and isn't ashamed to select and defend the simple approach.

Do You Have Experience With…?

It's a running gag among programmers that job descriptions, often created without input from technical team members, will ask for five years of experience in a technology that hasn't been around for even three years yet. And recently, nowhere has the fallacy in that been more apparent than with generative AI. In a sense, we're all newbies here. By the time you've become proficient in working with one particular model, the next one gets released. If we take this narrow, "HR needs to check the boxes"-style view of skill, then everybody is a bloody beginner at this.

This applies not just to individual job seekers, but consultants and their companies as well. How many years of GenAI-productization experience does Delloite have? Accenture? AICE Labs, for that matter? In every case the answer is, "as long as those things have been around, which isn't really that long".

Explicit experience, measured in years, with exactly one piece of technology or its subdomains is a poor measure of the likelihood that the hire will get you what you need. What changes it the new and shiny tool they get to wield. What stays the same is their methodical approach (or lack thereof...) to the underlying challenges. At the end of the day, it's engineering: Solving complex challenges with the best tools available under multiple competing constraints. Once you've got a knack for that, the actual tools become much more fluid, and checking how much time a practitioner has racked up in tool A versus tool B becomes much less relevant.

For instance, take someone with twenty years of programming experience but no prior JavaScript knowledge, who has deeply internalized the principles of good coding, then run them give them a one-hour overview to the language. Then pit them against a programming novice who spent three months in an intensive JavaScript bootcamp. I'd bet money the veteran will write better JavaScript.

With AI, we certainly have lots of new kids on the block who poured hours into prompting and vibe coding tutorials. They'll all be outperformed by those with solid engineering principles.

---

A quick personal note, it's the end-of-summer, almost-back-to-school chaos, so while I try, just for myself, to keep posting regularly, it's a bit more challenging than usual. :)

LLM Recommendations

Here's another area where large language models can do a fantastic job: Content recommendations.

It's no secret that the recommendation algorithms of YouTube, Spotify, TikTok, etc have come under scrutiny. One issue is that they're blindly optimized to drive engagement, which has been shown to lead viewers down rabbit holes of increasingly "extreme" content. TikTok, for example, is frighteningly good at sensing if content pushes your buttons, and before you know it, your feed is nothing but that, dialled up to 11 by the content creators vying for relevance on the platform.

But even if the algorithms were mostly harmless in their recommendations, they're also exceptionally bland. I have yet to make mind-blowing discoveries purely via the "you might also like" feature. These features are, for the most part, just presenting you with the average stuff people like you would listen to or watch. That inevitably pulls it into mediocrity and the least common denominator.

Recently, I tried out just asking ChatGPT. I told it about the artists and styles I generally like, and that I needed music for a road trip. We went back and forth with it, putting forth albums to listen to, and I told it what I liked/didn't like about each one. We ended up in pretty unexpected places, and I discovered several new bands that I'll keep following.

Now, music picking isn't the most world-changing use case, but the implications are larger. The lack of real understanding of traditional recommender algorithms means that their usefulness is limited, leading you either down rabbit holes or in circles. With a better understanding of the underlying subject, a recommender can unearth gems that are just what you wanted.

AI Coding Sweet Spots

It's no secret that I, along with many other seasoned software engineers, dislike the unbridled "vibe" approach to coding with AI. I have, however, found it immensely useful in several cases. Here are a two:

Fixing small issues in open source projects

While a recent study among AI users on open source projects found that it does not actually make them faster, on the whole, those findings are extremely context-dependent. Here's an example from my own experience: An extension to my code editor of choice (VS Code, currently) was lacking a feature that I really wanted. In the past, I would have filed a feature request with the tool's maintainer and hoped for the best. But today, I just downloaded the code, asked Claude Code to implement the feature, tested it (obviously!) and submitted the changes for the maintainer to review.

AI, in this case, rapidly sped up the part where you slowly familiarize yourself with an existing codebase to find the place(s) where a change must be made. Given that I have no intention to become a permanent maintainer of that tool, that upfront time investment would be prohibitive if it weren't for AI.

Understanding and Improving Legacy Code

This is an area I'm pretty excited about. The world is replete with legacy code (and one can argue that vibe coding is only going to add to that pile). First, a definition. Legacy code isn't code that's old. Instead, according to the guy who wrote the book on legacy code (Working Effectively with Legacy Code by Michael Feathers):

Legacy code is code without tests

Simple as that. Why is that? Without tests, you can't safely change or extend the code, and you lack a true documentation of the code's intent. And that's when you're in a case of "well, this works for now but we better don't touch it because Bob wrote this five years ago and has since left the company and we don't know how it works".

In that book, Feathers describes strategies for slowly turning legacy code into well-tested code. These steps are somewhat mechanical and deterministic but not quite as straightforward as automated refactorings. That makes them excellent for AI coding agents:

• AI can scan a large codebase and figure out dependencies that span many files. If you ask the right questions, it'll do the tedious "look in this file, find a function being used, so go look for the file where that function is defined, find another rabbit hole to go down" etc etc

• AI can write, or at least suggest useful tests. Once you have tests, you can move on to more aggressive refactorings and code improvements

• The existing code and its behaviour are a narrow source of truth that keeps the AI honest: If all you do is add a test, that new test must pass. If all you do is a structural change (aka refactoring), the existing tests must pass.

I'd be curious to see how far one could push autonomous agents to overhaul a legacy codebase; there are probably too many idiosyncrasies in each case to allow for a fully automated one-click solution. Still, it's a great chance where the costs of continuing to tolerate a legacy codebase might finally be larger than the costs of tackling them (and all without a full from-scratch rewrite!)

This was a longer and more technical post than usual, but I wanted to provide some positive counterweight to my bashing of vibe coding. The tools are what they are, and it's up to us to use them to good effect.

Fast Fashion, Fast Software

OpenAI's CEO, Sam Altman, recently tweeted (X'ed? 🤷‍♂️) that we were about to enter the "Fash Fashion" era of Software. The implication: With tools like Lovable, Bolt, Replit and co, people will just quickly vibe-code a piece of software, only to discard it soon thereafter.

And I'm sitting here and don't want to live in that world. It's bad enough that clothes these days barely last beyond the third wash. But what does that even mean for software? With clothes, the idea is that it is so ridiculously cheap to make them that there's no point caring for them and maintaining them, especially when they'll go out of fashion in a second. But with software? Unless we're talking about disposable entertainment like the umpteenth clone of Candy Crush, these things hold a lot of our data, which we'd then have to migrate. I don't want my to-do app to disintegrate once I add one too many items, or my CRM to implode when I want to import a contact with a spécial chäracter.

Vibe Coding was never meant for things that seriously see the light of day. It's neat to be able to throw together quick one-offs when they're truly one-offs, like a one-time script for a one-time maintenance task.

In my view, the integration of AI on the user-facing side makes it even more important that the backend has rock-solid engineering.

The other part where the analogy breaks down is in terms of quantity. People passionate about fashion own countless garments, for each mood and occasion. Fine. But would install twenty different to-do apps and use alternate between their usage? That makes no sense.

So instead of fast fashion, we should look at, of course, slow fashion. Buy less, but better. Same with software. Build it well so you don't have to throw it away.

Outrunning a Bear

A friend and reader of the newsletter asked me whether I'm using AI much for the writing of these daily posts. As a general rule, I don't (maybe a topic for another post as to why) but today I'm feeling cheeky (and a bit off thanks to a stomach bug), so what follows below is 100% generated by Claude Opus 4.1 with no edits on my part. Enjoy this one, inspired by our recent travels in the Canadian Rockies.

Why Your AI Strategy Doesn't Need to Outrun the Bear

Remember that old joke about bear encounters? "You don't have to outrun the bear, just your slowest friend."

After dodging imaginary bears on vacation last week (everyone had bear spray, nobody saw bears), I realized this perfectly captures what I'm seeing with AI adoption right now.

Most companies are paralyzed, waiting for the "perfect" AI strategy. They're trying to outrun the bear.

But here's the thing: You don't need to be OpenAI. You don't need a $10M AI transformation. You just need to be moving faster than your competition—who, let's be honest, are probably still debating whether ChatGPT is a fad.

The bar is surprisingly low:

• While they're forming committees, you're running small experiments

• While they're waiting for perfect, you're shipping good enough

• While they're protecting old processes, you're questioning why those processes exist

The bear isn't AI replacing your business. The bear is standing still while everyone else starts jogging.

And unlike the wilderness, in business you can actually help your "friends" run faster too. Rising tide, boats, etc.

P.S. - If you're the slowest friend reading this, hit reply. Let's get you moving. No bear spray required.

---

Just a few closing thoughts:

• It's not bad, but it's cliche. "Here's the thing" and all

• The title is too long and clunky

• It tries hard to be friendly and ethical, so in the end, it's about helping your slow friend run faster instead of letting the bear eat them

• Only a single em-dash. Take that, ChatGPT!

• That final analogy about "the bear is standing still while everyone else starts jogging" makes no sense

Feel free to hit reply and let me know your own thoughts about where it lands and where it doesn't.

Almost There But Not Quite

Looking at the steep mountain faces on my recent vacation to the Canadian Rockies, I got a strange thought: If you could teleport yourself to anywhere along the mountain to a (relatively) secure spot, but you didn't have any rock climbing experience, you'd be just as stuck thirty meters off the ground as you'd be 30 meters from the top. For your stated goal of making it up the mountain alive, it's actually irrelevant how close to the top you'd be. Yet the providers of the teleportation device would tout that their's gets you 90% there while the competition only gets you 85% there. Irrelevant. It only becomes relevant if you're a seasoned rock climber with the right equipment and with the route ahead within your capabilities. Then, yes, it totally makes a difference how close to the top you can get teleported.

Such are the dangers of vibe coding and other vibe disciplines. If the AI gets you an app that's almost there, and you don't have that engineering background, then it doesn't matter how "almost" it is. You'll still need to determine what's missing and how to fill those gaps. Better to focus on your area of expertise and use AI to accelerate you on that journey. In short, don't use it to shore up your weaknesses. Use it to double down on your strengths.

Exponential vs S-Curve

GPT-5 has been out for a few days now, and apart from marketing hype, the response has been a resounding "meh". Some say it's great for their particular use case, others say it's mediocre at best.

In the endless cycle of whose company's AI model is the current best, we can get the impression that there are huge strides being made. The whole "accelerationist" movement tells us that we can expect an exponential growth of model capabilities, just because the early steps (GPT-1 to 2 to 3 and 4) were so monumental. They'd tell us that, before we'd know it, the AI would design better versions of itself and then we'd really be off to the races, towards the so-called singularity, super-human intelligence and, depending on your mood, annihilation or abundance for all.

Well, just like many other promises of endless growth, this one doesn't quite seem to pan out as well. Instead, progress levels off to incremental gains and diminishing returns. Just like medicine in the 1900s has made tremendous strides and made it look like life expectancy was on an exponential growth curve didn't mean that life expectancy would grow indefinitely (don't tell Peter Thiel or Ray Kurzweil I said that, though), there are natural limits and constraints.

So, what does that mean? It means it's crunch time for engineers. We can't just sit around with the same old systems and same old prompts and just wait for better models. Models will keep getting better, but not at a rate that excuses laziness. Now's the time to tweak the prompts, the model selection, the vector databases and the scaffolding. Now's also the time to be less forgiving of products and tools that seemed to bank too hard on vast improvements in bare LLM capabilities. If it's nowhere near useful right now, don't let them tell you to "just wait until GPT-6 hits".

It's okay for bare LLM progress to slow down. It's not like in classical software engineering we write low-performance software and then say, "oh well, we'll just wait for the next version of PostgreSQL to make our bad queries execute faster". (Though there was that glorious time in the 90s where CPU speed doubled every time you blinked...)

Long story short, GPT-5 was underwhelming but that fact itself is also underwhelming. Lets just get back to the actual work of engineering working solutions to real problems.

The Eagle’s Call

Back from vacation, here's a nature-inspired newsletter edition with a fun nature fact: The sound many movies use for a bald eagle's cry is actually the cry of the red-tailed hawk (Red-tailed hawk - Wikipedia). You know, the high-pitched, long and echo-y scream. Real eagles calls sound more like dolphin chatter. Bald eagle - Wikipedia

Because movies use the wrong call so much, those of us who don't live in an area with abundant bald eagles end up thinking that that's their real sound. Now, the consequences for this misunderstanding are benign, but it points to a certain danger when presented with plausible but wrong information which AI has a chance to amplify. The analogy goes like this:

1. I have no idea what a bald eagle sounds like because I've never heard one in the wild.

2. I come across a movie with the wrong eagle sound.

3. It sounds plausible enough. Powerful and piercing. I now think that that's what eagles sound like.

With AI:

1. I have no idea how to write good marketing copy because I'm not a marketing specialist.

2. I ask AI to write me some good marketing copy.

3. To me, it sounds good enough. Punchy and engaging. I now think AI makes marketers and copywriters obsolete.

Just because the AI result seems good to me doesn't mean it's actually good (unless it is meant purely for my own consumption). It takes an expert at a given craft to judge whether the result is truly good. Which reiterates the point: In the hands of an expert, AI can be a great productivity boon. In the hands of a hack, it can be a dangerous delusion.

The Review Trap

Consider this scenario: An AI tool generates some output. Maybe it's code. Perhaps it's a marketing communication to be sent out. A human reviews it before it gets sent out. This works and makes sense in scenarios where reviewing is faster than creating. Luckily, there are many such scenarios. Writing a good joke can take a comedian hours or days. Judging whether a joke is funny happens instantly. The same dynamics apply to writing and many creative endeavours. In such a scenario, the AI does not need to get it 100% right in one shot. The review will be brief, and if the output is 90% accurate, the subsequent changes will be rapid.

But there are scenarios where this dynamic doesn't apply, and that's reviews themselves. To judge whether an AI-generated review is correct, you have to do the whole review yourself. Otherwise, how can you be sure the AI didn't miss something critical? So in this scenario, where you want to use AI-generated reviews (or summaries, or style-transformations, ...), you're not better off unless you have complete trust in the tool's accuracy. That immediately sets a much higher bar for AI tools used in these scenarios.

In such a scenario, your current best bet is to use the AI review as a jumping-off point for your own thorough review: Let's say you use GitHub's Copilot Review functionality to provide an initial review of suggested code changes. Great. Take a look at those and then check the code thoroughly yourself, looking specifically for subtle things the AI might have missed. Just don't trust them blindly. And when thinking about AI use-cases in your org, pay attention to which scenario we're dealing with: Generation or review, and don't fall in the trap of automating something that needs to be double-checked by a human anyway.

P.S.: It's a beautiful summer here, and I'm off on another camping trip, returning in mid-August with more AI-related posts.