Jev: Sometimes, new models are exciting

As you'll have noticed, I don't usually write about new AI models, as most of the time it's an endless stream of incremental improvements. This week, OpenAI's Astra is on top, next week it'll be Anthropic's Fable, then it'll be Gemini and after that maybe even Grok.

But this week, a former OpenAI employee came out of nowhere, announcing his company TypeSafe AI, with a refreshingly different paradigm. The dust is still settling, but I'm already excited for a few reasons.

• The model is decidedly not a reasoning model and as such runs much faster and cheaper

• It's not even trained in the classical LLM style, of trying to predict the next output token

• It's specifically meant to quickly answer structured output questions, where a prompt can specify that the answer should be strictly "yes" or "no", or an item from a list of choices.

Why that matters

The way LLMs are trained ("Here is some text with a word missing. Guess the missing word!") makes them incredibly good at human-style language. But also frustrating and inefficient to embed into automated workflows. Inside a program, you'd like to have some predictability on what output format you're dealing with. For complex queries that need to be processed further, the standard format is called JSON, and would look like this:

{
  "name": "John",
  "age": 30,
  "is_student": false,
  "hobbies": ["reading", "coding"],
  "address": {
    "city": "New York",
    "zipcode": "10001"
  }
}

Getting a standard LLM to generate such structured output is possible, but it doesn't come naturally. Each bracket, colon, and comma needs to be generated as its own token, and that requires a full inference pass through the entire neural network. That gets expensive and slow and still doesn't guarantee that you won't get a hallucination with wrong grammar.

Where TypeSafe's model, called Jev, differs is that it doesn't given open-ended chat answers, and doesn't respond to completely unstructured queries. Instead, questions to the model come in the form of structured, typed, queries.

At the lowest layer, a single question can have one of three types:

• Choice: Asks the model to pick an answer to the query from a list of predetermined options. Good use case: Routing a support request to the correct department

• Score: Asks the model to answer with a score based on descriptions of levels. Good use case: Judging a bug's severity.

• Noul: Seems to be a made-up word, but TypeSafe uses it to refer to a yes/no question. Asks the model to make a yes/no decision on a question, such as, "Is this request urgent?"

More details are found in their documentation, https://docs.typesafe.ai/introduction but what's immediately exciting here is that it should allow for fast, genuinely useful customer support AI without having to worry about prompt injection: If the only thing the support bot does is decide which department to route the request to, a user can't ask the bot to do something else. No complex guardrails required!

If you were interested in AI use cases where quick judgment calls are required but were worried about the inconsistent and insecure nature of LLMs, exploring what Jev (and similar models like it, which I'm sure we'll now see an explosion of) can do for you might be worth while!