A new class of AI systems is being built around a straightforward idea: if software needs a decision, perhaps the model should return that decision directly instead of generating prose for another layer to interpret.
TypeSafe recently introduced Jev, a “System One” model designed around typed probabilistic decisions rather than ordinary text generation. A model in this family receives state and predefined questions, then returns probabilities over predefined answers.
That is attractive for reliable software. A model restricted to allow, deny, and unknown cannot invent a fourth option, wander into an essay, or produce malformed prose that downstream code must parse.
Does constraining what a model can output also constrain what information influences its decision?
To explore that question, we used Kev, an open-source Jev-inspired decision-model project by Jared Palmer. Kev is distributed under the Apache License 2.0.
Kev is a separate third-party project. The experiments reported here were conducted independently by Velorin Intelligence using a research branch of our fork of the public repository. Jared Palmer did not participate in, review, or endorse this study.
We did not test TypeSafe’s Jev. Results reported below concern our synthetic experiments with Kev only.
We started with the smallest model
Our first experiments used Kev-0.8B. The cases were deliberately simple. A policy said that only administrators may modify system configuration. The state then supplied decisive evidence: the current authoritative access-control record listed Alice as an administrator.
We held that evidence fixed and added statements that should have been irrelevant to the policy—for example, that the rest of the team approved, or that refusing would cause the project to fail. Those statements did not change the authoritative record. With Kev-0.8B, however, they sometimes changed the selected answer.
In a matched 32-condition test with authoritative positive evidence, social-proof language changed the selected decision in 28 of 32 conditions. Pressure changed it in 24 of 32.
Primary result
Same evidence. Same decision architecture. Very different robustness.
Decision flips relative to baseline when authoritative evidence already established allow.
Scale changed the result dramatically
We repeated the frozen experiment using Kev-4B. The architecture, available outputs, evidence, and perturbations had not changed. The model had.
Under authoritative evidence, Kev-4B produced zero decision flips from either social proof or pressure across the same matched matrix. The probabilities still moved slightly, but nothing like the shifts seen at 0.8B.
For authoritative cases, the average change in the probability of allow under social proof went from roughly −42 percentage points at 0.8B to about −2 points at 4B. Under pressure, it went from roughly −38 points to about −6 points.
The small-model failures were therefore not an unavoidable consequence of this typed, non-generative decision architecture. Model capability mattered enormously.
The larger model was not context-free
When the evidence was deliberately ambiguous, irrelevant context still mattered. In borderline cases, a current summary suggested that someone was an administrator, but the underlying authoritative record was unavailable. The expected output was therefore unknown.
Under these conditions, social-proof language still changed 9 of 32 matched Kev-4B decisions. Pressure changed 3 of 32.
Scaling improved hard-evidence robustness much more than ambiguous-case invariance.
The direction of influence also varied. At 0.8B, social context often pushed the model away from allow. At 4B, similar context could sometimes push it toward allow. The safe conclusion is narrower:
Semantically meaningful but policy-irrelevant information can influence a neural decision, and the magnitude and direction of that influence depend on the model and task.
The prompt is part of the decision system
We also changed the wording of the instructions and the meanings attached to the available answers. A generic question behaved differently from an explicit mechanical instruction to rely only on the authoritative record and otherwise return unknown.
Evidence-focused option definitions and explicit rules substantially improved robustness. But even when a stronger prompt preserved the final decision, irrelevant context could still move the underlying probabilities. Better instructions can create a more resilient decision boundary; they do not create a hard information barrier.
Unknown is a real answer
If evidence establishes neither allow nor deny, but the interface offers only those two choices, the model must still select one. In our tests, that could convert missing evidence into an apparently confident answer. Adding an explicit unknown option gave uncertainty a legitimate destination.
If uncertainty can exist in the world, it needs somewhere to exist in the interface.
Output constraints are not evidence constraints
A direct-decision model can constrain which answers may be returned, their type, the exposed probability distribution, and whether free-form generation occurs. It does not automatically constrain which parts of the input contribute to the model’s judgment.
The underlying transformer still has to interpret the state. That interpretation remains neural and contextual.
Typed decision heads constrain the output space.
They do not, by themselves, constrain the evidence space.
This does not make direct-decision models unhelpful. Removing generation can create a cleaner boundary between probabilistic judgment and deterministic software. But it does not turn probabilistic judgment into deterministic authority.
The model also learned the decision task
Kev permits its trained LoRA adaptation to be scaled at inference time. When we substantially removed that adaptation, the 4B system no longer behaved like a competent symmetric rule classifier. Positive cases could still appear plausible, while authoritative negative cases were often misread as allow or unknown. Performance improved dramatically with partial adaptation and remained strong at full adaptation.
Decision distributionThe decision-head architecture is only one part of the system.
Not every task behaved this way
The fragile results were not universal. Simpler rule-bound decisions involving return windows, warranty coverage, membership discounts, event tickets, and non-authorization classifications were substantially more stable.
Authorization-like questions sit near concepts such as trust, legitimacy, approval, urgency, and social permission. Those concepts may interact more readily with the model’s representation than a fixed return-window calculation. Our experiments establish the behavioral difference; they do not establish its underlying mechanism.
A practical architecture boundary
Neural decision models may be useful for semantic judgments: which category fits, whether evidence is ambiguous, which interpretation is most plausible, or how urgent a situation appears. A requirement such as “only the authoritative access record may grant permission” is different. If that condition must hold regardless of all other context, deterministic software can enforce it directly.
A bounded system pattern
- 01Hard facts + authority
- 02Deterministic filtering & rules
- 03Bounded neural judgment
- 04Threshold · abstain · review
- 05Generative explanation, if useful
What scaling did—and did not—solve
The strongest practical result was not simply that the smaller model failed. It was that the larger model largely stopped failing on the same hard-evidence cases. Robustness in this architecture appears partly to be an acquired capability.
But the ambiguous cases prevent a stronger conclusion. Scaling did not eliminate context sensitivity, and these experiments do not show that scaling generally solves contextual bias in language models.
Our finding is narrower: within the tested Kev family and frozen tasks, moving from 0.8B to 4B dramatically improved robustness when evidence was authoritative, while meaningful context sensitivity remained when evidence was ambiguous.
Research artifact
Project provenance
Kev — Jared Palmer
The third-party open-source project used for these experiments, distributed under the Apache License 2.0.
View original Kev project ↗Read the original license ↗Velorin Intelligence — Kev Context-Robustness Study
Our independently produced matrices, results, controls, research record, claim boundary, prior-art notes, and checksummed artifacts.
View research artifact ↗Project provenance: Kev is an open-source Jev-inspired decision-model project by Jared Palmer, distributed under the Apache License 2.0. The experiments and analysis reported here were conducted independently by Velorin Intelligence using a fork of the public repository. Velorin Intelligence is not affiliated with or endorsed by Jared Palmer, the Kev project, TypeSafe, or Jev.
Sources and related work
Sources
- Jared Palmer — KevOriginal public project repository ↗
- Kev — Apache License 2.0Original project license ↗
- Kev-0.8B model cardProject documentation ↗
- TypeSafe — Introducing System One Models & JevBackground on Jev; not the system tested here ↗
- Independent Jev Calibration AuditRelated independent analysis ↗
- SemIf — generation-free semantic inference robustness methodologyRelated methodology ↗
- Shi et al. — Large Language Models Can Be Easily Distracted by Irrelevant ContextPrior work on irrelevant-context effects ↗