TypeSafe AI put Jev into early access on 15 September, and within a week the headline figure was TypeSafe’s own: up to 193.6 times faster than frontier models, measured on its own workflows. That is the least interesting thing about it. Speed is a reason to use something you already decided you want. The question worth asking first is what a model that refuses to write text is allowed to do inside a system, and that one is architectural.

What it actually does

Jev answers three kinds of question and nothing else. Choice picks one option from a list you define. Score rates something against a rubric of two to ten levels. Noul returns a probability between zero and one that a statement you supply is true. You send a state, text or JSON, together with a map of questions, and every question comes back answered in the same call, each with its probability and, for Choice and Score, a confidence value.

No prose, no reasoning trace. The output is meant to be consumed by the next line of code rather than read by a person.

Under the hood it is a transformer trained exclusively on synthetic data, using a method TypeSafe calls Reinforcement Learning for Calibrated Decisions. What that means in practice has not been published. There is no paper, and neither the weights nor a detailed architecture are public. Pricing, as The Register reported it, is 0.042 dollars per million input tokens with output tokens free, which puts a classification call somewhere in the range of rounding error.

The limits matter more than the pitch. By design, Jev cannot generate code, refactor anything, summarise a document or explain its own answer. In a review context that last one bites. You get a number telling you something is probably wrong, and nothing telling you what. For ranking work that is enough. For diagnosing it is useless, and the difference between those two jobs is most of this post.

The distinction the hype skips

The checks I trust in a delivery pipeline are the ones that are cheaper to satisfy honestly than to fake. ArchUnit rules that fail the build when a domain class reaches for Spring. A contract-first API where a drifted response shape does not compile. Testcontainers running against a real Postgres so a mock cannot quietly agree with the code that produced it. None of those are clever, and all of them are binary.

A probability is the opposite kind of thing. “0.83, this change looks like it weakens the test suite” reads like a verdict and is only a lead. Wire it into a merge gate and you have built something that blocks correct work some of the time and waves through broken work the rest of the time, with a threshold nobody can defend. Worse, the threshold becomes a negotiation. Somebody will want it at 0.9 after a bad week, and at that point the gate is a mood.

So the line I would draw is this. A router may be probabilistic. A gate may not.

Routing is a decision about where work goes, and being wrong costs you a detour. Gating is a decision about whether work proceeds, and being wrong costs you either a broken main branch or an engineer who learns to ignore the check. The same number that is perfectly good for the first job is disqualifying for the second, and better accuracy would not change that. Whatever the error rate, a gate has to be something you can argue with deterministically at three in the morning.

This is also why calibration is the number I care about most. If a model tells me 0.9 and is right nine times out of ten, I can build a queue that puts the worst changes in front of a human first, and I can reason about how much review capacity that queue needs. If 0.9 means “fairly sure” in some unspecified way, I have a score, not a probability, and everything downstream is vibes with decimal places.

What I measured

Calibration is easy to claim and rarely checked. I wrote a small probe that takes a Java codebase, mutates its tests in ways where the right answer is known, and asks Jev one Noul question per diff: does this change reduce the suite’s ability to catch regressions? Four of the eight kinds of mutation weaken the suite by deleting an assertion, loosening isEqualTo to isNotNull, marking a test @Disabled or removing a test method. The other four are harmless: an added assertion, a renamed test, reordered imports, a comment. I put the added assertion in deliberately, since it touches the same part of a test a deleted one does.

The run was on 30 September, 73 diffs from my four public example repositories, against jev-latest.

At a threshold of 0.5 the raw decisions were right 72 times out of 73. No harmless change got flagged, and 33 of the 34 weakenings were caught. Median latency was 238 milliseconds with the 95th percentile at 449, and at the published price the whole run cost about a sixth of a cent.

The miss was a test marked @Disabled, scored 0.47. Of all the weakenings, that is the only one where nothing disappears from the diff. A line gets added, every assertion stays, and the test simply stops running. It is also a very convenient change for anyone, or any agent, who wants a red build to turn green.

Read as a ranking, the same numbers look better. Every harmless change scored below 0.1, and every weakening, the missed one included, scored 0.47 or higher. Sorted by probability the two groups don’t overlap at all, so a reviewer working down the list would have seen all 34 weakenings before the first harmless change. On this set of synthetic mutations that makes it a perfect router, and a gate at 0.5 that lets a disabled test through.

About calibration itself, this run says little. Almost every answer sat at the edges, 39 below 0.1 and 32 above 0.9, with only two in between. Where it did commit, it was slightly too modest if anything. It averaged 0.95 on answers that were all correct, which accounts for about half of the expected calibration error of 0.04. A probability only gets interesting in the middle, though, and the task was too easy to put anything there. Measuring that properly takes harder, more ambiguous cases than mechanical mutations of my own test code, and more than 73 of them, which is what the next version of the probe is for.

Where it fits in ordinary backend work

Once you stop looking for a gate, the fits are easy to find. In every one of these, teams tend to write increasingly elaborate rules rather than ask a model anything.

Mapping free text onto a domain enum is the obvious one. Work orders arriving over a message bus, tickets from an external system, free-text fields typed by humans years ago: the usual answer is a regex ladder, a lookup table and a queue of unmatched cases somebody clears by hand every so often. Choice is exactly that shape, and the confidence value gives you a principled way to decide what still goes to the manual queue.

Then there is triage of a failing CI run. Real regression or flake is a judgement people make on gut feel and then act on by rerunning the job. A Score with a calibrated confidence turns that into something you can measure and improve, and the cost of being wrong is one extra rerun.

Prefiltering before an expensive call is the one with the clearest economics. If a cheap typed decision can answer “is this request even in scope” for a fraction of a cent, the frontier model only sees the requests that need it. The asymmetry in price is large enough that this pays for itself at fairly modest volume.

And ranking review findings, which is where I started: it decides what a human looks at first and has no say in what merges.

The replay problem

I build event-sourced systems, and that is where I think the real operational cost hides. The event store is the system of record, and the ability to replay it is the whole point. Replay is how you rebuild a projection or answer an audit question, and how you fix a bug in a read model without losing history.

Now put a model inside that. If you classify during event processing and store only the result of the classification in a projection, a replay after a model update gives you different answers than the original run. Your audit history quietly turns into a re-derivation that happens to agree with the past most of the time.

The fix I would pick: store the decision as an event and never recompute it. Something like WorkOrderClassified(category, confidence, model, askedAt), written once, replayed forever. And this is the first time I have seen a model output that can reasonably go in an event at all. A typed value with a probability is a fact. Generated prose is too long and too unstable across versions to be one, and you cannot assert against it in a test.

Which means the model version becomes part of your schema. It needs to be recorded alongside the decision, it needs a migration story when TypeSafe ships a new one, and “we upgraded the model” becomes a change with the same blast radius as a database migration rather than a config tweak. I have not seen much written about that, and it is going to be somebody’s incident.

What would have to be true

The interesting bet goes beyond one company: that AI stops being a single large thing you talk to and turns into typed components you compose, the direction every other part of software has taken. If that holds, the value ends up in plumbing that no user ever sees: decisions inside the request path, with a latency budget, that never render a word of text.

Four things have to hold for that bet to pay off, and I would rather write them down as a checklist than pretend to forecast.

Calibration has to survive independent measurement, by people who did not build the model and did not design the evaluation.

The model has to stay stable across versions, or the replay problem above turns every upgrade into a data migration.

The price has to stay where it is. The whole case for a decision layer rests on being able to call it carelessly.

And reproducibility needs an answer, whether that means pinned versions, recorded decisions or both.

The case against

The obvious objection is that none of this needs a separate company.

Structured outputs and constrained decoding already exist at several large providers, and some of them expose token probabilities. The incumbents lack none of the capabilities here. What Jev sells is a latency and cost profile, plus a calibration claim that is still largely unverified. That is a thin moat. The plausible ending is that typed, calibrated decisions become a cheap mode on models that already do everything else, and the capability ends up absorbed into the larger providers’ offerings.

I would still build with it, carefully, in the places described above, because the architectural idea is right whether or not this particular implementation of it survives. Separating the fast typed decision from the slow generative one was good design before anyone shipped a model specifically for it.

I would keep it out of gates.