Lessons from Harvey AI
An anonymous person posting on the social media platform Reddit claims to be a former employee of the Legal Tech startup Harvey AI. They allege that the tool has low internal adoption, is favoured more by leadership and procurement than by those doing the actual work, wasn't built in close collaboration with actual lawyers, plus a number of other criticisms around the product’s quality.
While Harvey's CEO responded and countered these claims, there has been a lot of schadenfreude from others in the legal tech industry, as well as plenty of piling on from AI skeptics. While I'm in no position to judge who's right and who's wrong, we can still extract some lessons, based on the complaints levelled by the anonymous Redditor and the other practitioners.
Biting off more than you can chew
It seemed to me, back in 2023, that Harvey was starting with an overly broad mission: essentially feeding a large amount of legal documents to an AI and having it become proficient at writing legal documents to the point where you could replace, if not your senior lawyers, at least a bunch of your paralegals. Yet, even if a large language model is fine-tuned with incredibly industry-specific material, it only delivers value when plugged into a concrete workflow aimed at solving a particular problem. Lawyers (presumably) don't just want a ChatGPT that's aware of how lawyers write. They want tools that tackle specific tasks, such as drafting and reviewing contracts.
From the observed criticism, I get the impression that Harvey is sort of "bleh" at a lot of lawyer-like tasks, but not amazing at any one of them. If that's true, then it's no surprise that adoption is lacking.
There was a sort of irrational exuberance in the air right around GPT version 3.5, where it seemed the winning formula would be to take an off-the-shelf language model, finetune it with proprietary industry-specific data, and instantly get an expert that could handle any task in that industry. By now, we know that this isn't quite the case, as in the recent MIT study about enterprise AI pilots.
What we must realize is that AI doesn't let us skip proper product development. AI might enable previously unthought-of capabilities inside a product. However, the rest of the product still requires solid engineering, user experience design, and all the other pesky things that are hard work, requiring human insights.
