Jev in 25 Lines of Python
by bashbjorn on 9/23/2026, 7:26:23 AM
https://www.nobodywho.ai/posts/jev-in-25-lines/
Comments
by: sigmoid10
Going directly for the logprobs is always icky when you use a chat model as base, because they are trained to write prose as output. So your "choice" tokens and thus their probabilities might get diluted in whatever else it wanted to say. If you have to do it in the same way as this post, at least add clear system instructions and a carefully worded beginning to the assistant output section of the prompt to lower the chances of it wandering off immediately.<p>I've found that using structured outputs solves this problem much better. Instead of letting a model generate only "A", "B" or "C" and looking at the probs, have it directly generate "Legitimate", "Spam" or "Phishing" or any other pre-defined option from a set of multi-token sequences. Behind the scenes it boils down to something quite similar, but you're not running into the risk that the model actually wanted to say "A phishing attempt seems likely, so answer (C) is correct.", which would lead "A" to have the highest probability in the first token. You can even use a reasoning budget this way either via inherent reasoning or a free-form part preceding the remaining output structure. You can also have it assign probabilities (either in words or numbers) using more complex output structures, but I would not rely on them much more than the token logprobs (they can still be quite good though).
9/23/2026, 8:13:54 AM
by: antirez
Because of masked attention in LLMs, if you put the options <i>before</i> the body (the email to analyze), the transformer already knows what it needs to look for, and can use more tokens to create state to address that specific task (BERT has no mask in the attention, so tokens attend also to next tokens). You could also do a few examples in the system prompt to improve calibration.<p>Another trick that works is to repeat the question two times: "I'm repeating the task and labels for clarity: ..."
9/23/2026, 9:09:40 AM
by: iamflimflam1
The number of - “I did/invented Jev last year”, or, “here’s a version of Jev I vibed up last night” is getting a bit ridiculous.<p>Especially ridiculous is how the hacker news crowd seems to be taking these at face value…<p>There was one the other day with a compelling demo. But when you looked closely at it, it was feeding in the options with the word “best” on the option to pick and a fine tuned model designed to recognise that word…
9/23/2026, 9:16:21 PM
by: no-name-here
Beyond the missing latency and compute comparisons that Heaney commenter mentioned, also nothing about its error rate compared to Jev (nor if it even always outputs in a format the app can parse, not sure how solved that is).<p>But then at the end it says it’s parody. Maybe HN title should say it’s a joke.
9/23/2026, 8:10:16 AM
by: philipbk
> "25 lines of python" > "import Solution" ok
9/23/2026, 1:04:16 PM
by: visarga
I also built one, but mine uses embeddings. It classifies concepts defined by a collection of positive and negative examples. The classifier model is trained in <1 second using ridge regression. The model itself is exactly the same shape as the embedding, so it works as a concept embedding. Since I already have a dataset, I can use it to do conformal prediction in order to calibrate confidence scores. Jev, on the other hand, has a generic model, not trained on in-domain examples, so its confidence scores are uncalibrated for any non-generic task.<p>So you might ask: how do I obtain the training examples? Just collect samples and use a coding agent to classify them as match or no match. From time to time you can add more examples to the dataset to have your concept adapt to changes in input distribution. It's all automated, but it only uses LLMs to train concept vectors, after that it works like a regular embedding model with a calibrated classifier on top. It's also 20-30x faster than Jev, free, and runs on CPU.<p>An illustration of how it defines a concept as opposed to simple cosine similarity: <a href="https://github.com/horiacristescu/semlabel/raw/main/images/cosine-vs-semlabel.png" rel="nofollow">https://github.com/horiacristescu/semlabel/raw/main/images/c...</a>
9/23/2026, 6:59:04 PM
by: zeroq
How to write Jev in 25 lines of Python:<p><pre><code> 1. draw a circle 2. import the rest of the owl</code></pre>
9/23/2026, 12:12:54 PM
by: jorisw
Highly suspect of content marketing.<p>Ends with referring to a product, and saying "this is a parody post", after pretending to make a serious point.
9/23/2026, 10:03:52 AM
by: 0123456789ABCDE
this is such a trivial thing to do in DSPY, no one bothered to give it a name…<p>here's 7 lines<p><pre><code> import os import dspy lm = dspy.LM("openrouter/z-ai/glm-5.3-flash", api_key=os.environ["OPENROUTER_API_KEY"]) jev = dspy.Predict('email:str -> choice:Literal["Legitimate", "Spam", "Phishing"]') email = "Payroll asks for your password on a non-company sign-in page." pred = jev(email=email, lm=lm) print(pred.choice) </code></pre> there are other options, obviously. you can choose to give it some tools, maybe some reasoning stage before picking a choice, and that's on top of the "reasoning" the llm model already does api side
9/23/2026, 4:25:23 PM
by: onion2k
<i>It's fast.</i><p>If you're comparing with something, you need to state 'fast' in relative terms. Jev is definitely fast, and if this Python takes the same time to get a decision then it's also fast. If it's 100* slower than Jev though, you shouldn't be calling it 'fast', because relatively speaking it's really, really slow.
9/23/2026, 8:23:16 AM
by: alun
The one thing I can't wrap my head around with Jev is why they're trying to create that "System One" narrative.<p>In real life, a human doesn't do classification tasks with the System One part of their brain, they use System Two. So by definition what Jev does isn't System One thinking.<p>If anything, regular programming that automatically executes based on logic, without requiring "thinking" would be "System One".
9/23/2026, 11:49:26 AM
by: yipinwong
Any analogy works at certain abstraction level, and this works with a premise that it's a classifier that makes decisions.<p>Still good. In practice for Jev the devils in the details. As you all know by now, it's easy to write PoC and understand with AIs (or even manually, which is now a prestious practice).<p>That demo will get you 80% there<p>Getting to that 100% or even 99% to JEV level will be hard with all the edge cases, infra, API, communications, etc.<p>Still a good article.
9/23/2026, 7:33:05 PM
by: SylonZero
Haha - I did enjoy this read! And there is a point to the whole marketing-dresses-up-stuff that is certainly true. I think it's worth pointing out the other HN story earlier <a href="https://news.ycombinator.com/item?id=49765348">https://news.ycombinator.com/item?id=49765348</a> about an open-weight model called Laya.<p>P.S. I am evaluating that model for a production use case where I would have used Jev
9/23/2026, 9:11:07 PM
by: rcarmo
Very nice as a conceptual thing, but there's a bit more to it. I've bolted Gemma 4 onto a custom pipeline for that (<a href="https://rcarmo.github.io/projects/go-system-one/" rel="nofollow">https://rcarmo.github.io/projects/go-system-one/</a>) and it's OK-ish (a bit slow on my puny 3060, but I can use it to prototype a bunch of things locally until the Jev mania settles and we have better models).
9/23/2026, 8:50:12 PM
by: bruhhhhhh
I am hearing about Jev for the first time here so no idea about the hype. So their(Jev) is that the thing is faster at classification than a frontier model? Because the whole type safe aspect is already fully solvable with structured output. But their example is classification but that would also be possible and faster with a classic BERT model. So their pitch is a task specific smaller model or am I completely misunderstanding the whole thing?
9/23/2026, 12:21:21 PM
by: dhsysusbsjsi
Whilst I do like reading these things for technical know how, I can sympathise with the creator of jev who now presumably has to apply an order of magnitude effort to explain why the 100 smaller things done better than this add up to a much better product.
9/23/2026, 8:05:44 AM
by: vonStackelberg
Hey, I enjoyed the article! I think it’s helpful to break stuff down as much as possible to nail down what is happening. They did that. The important part is not importing a model or something, it’s what’s happening after. I learned something
9/23/2026, 7:38:05 PM
by: xigoi
This is like saying that cars are useless because you can achieve the same thing by removing the cannon from a tank.
9/23/2026, 9:05:31 PM
by: tducret
The website is currently returning "Site not available"<p>Here is an archive: <a href="https://web.archive.org/web/20260923122959/https://www.nobodywho.ai/posts/jev-in-25-lines/" rel="nofollow">https://web.archive.org/web/20260923122959/https://www.nobod...</a>
9/23/2026, 5:56:58 PM
by: beamy
> We didn't train a model with Reinforcement Learning for Calibrated Decisions (RLCD) to calibrate the decisions and probabilities<p>But aren’t calibrated predictions one of the defining features?
9/23/2026, 9:19:15 PM
by: armcat
Looking at the logprobs on tokens works for the local models, but not on the frontier ones. It's been more or less broken since GPT-4o for example. I wrote about it two years ago: <a href="https://medium.com/data-science/9-11-or-9-9-which-one-is-higher-6efbdbd6a025" rel="nofollow">https://medium.com/data-science/9-11-or-9-9-which-one-is-hig...</a>. Also, I've done some work in estimating confidence and on rubric evals using the same method, and you actually get better correlation to "real confidence" by just getting the LLM to say it.
9/23/2026, 11:34:30 AM
by: Oscalemor
First release people are getting more balanced at least. Less 'it's gonna change everything' to 'might be viable use cases'. If I have to look at one more video high lighting google flights, I might loose it though.
9/23/2026, 8:20:31 PM
by: cupofjoakim
I wonder if this could be a good stepping stone to write a local prompt router to optimise what model get what prompt. I.e. if the prompt is just a lookup, send it to haiku, if it's reasoning, send it to opus and if it's implementation send it to sonnet.
9/23/2026, 8:42:34 AM
by: pjankiewicz
What I'm missing here is also type guarantees. I don't think you can do it without token level logic which forces the model to output the tokens from a predefined pool of tokens. A logic like this given some JSON schema is not that difficult to implement. If the LLM must output JSON schema compatible value then you can also add that it doesn't "hallucinate". Which is funny too because just guaranteeing the type does not mean the model does not hallucinate but this is another story.
9/23/2026, 9:34:28 AM
by: chpatrick
Could someone explain how Jev is different from using any old model and constraining the output to "My choice is a/b/c..."?
9/23/2026, 4:46:50 PM
by: davidfekke
This is a system two model, and not a system one. To get the performance of Jev, and you want to run locally, use Laya. It is up on Hugging Face.
9/23/2026, 3:25:52 PM
by: kjshsh123
Maybe someone can explain why RL is even needed for post training with Jev? We have supervised labels.<p>I guess it's due to the calibrated decision part (and that's what LLMs tell me).<p>But I figure some supervised classification post training would still improve the model.
9/23/2026, 5:13:19 PM
by: fzysingularity
Am I missing something here:<p>p(y = next thinking+decision token | x = question) != p(y = next decision token | x = question)<p>The former is what LLMs are trained for, the latter is what Jev was likely trained on (likely used thinking alignment as an auxiliary loss, but not explicitly included in the probability calibration).
9/23/2026, 4:02:00 PM
by: boros2me
We have Jev at home
9/23/2026, 12:09:25 PM
by: rgbrgb
i love that people are trying to make OS jevs but what is the point of doing all this work and not ask your coding agent to do a little benchmarking. selfishly want an open weight model to beat jev here<p>just found this one <a href="https://huggingface.co/spaces/multimodalart/jev-decision-index" rel="nofollow">https://huggingface.co/spaces/multimodalart/jev-decision-ind...</a>
9/23/2026, 5:52:20 PM
by: xg15
I missed the hypewave so can't say a lot about Jev, but the double standards are entertaining:<p>About Jev:<p>> <i>We didn't train a model with Reinforcement Learning for Calibrated Decisions (RLCD) to calibrate the decisions and probabilities (even though they are not always correct).</i><p>Only 99% correctness! Borderline unusable!<p>About their model:<p>> <i>It classifies: it gets a prompt with choices and outputs probabilities.</i><p>You want numbers, it gives you numbers! What more could you want?
9/23/2026, 11:42:10 AM
by: K0IN
a hile ago (when big providers still provided logprobs) i created a VS Code highlighter that visualizes unsure tokens.<p>Since most chat models want to answer with a human-readable message i think their logprobs are not as meaningful. It would be interesting to see if one choice is like "correct" and if the model wants to choose it more often, cause it might not answer the question but to prose to the user.
9/23/2026, 11:00:32 AM
by: petercooper
You can also go beyond Jev. Qwen 3.5 0.8B is fantastic at basic <i>image</i> classification/question answering (including OCR elements) also. Though rather than looking at logits, I get it to output a structured JSON object and it does simple object classification tasks on a Mac at under 500ms a pop (I forget how far, but I think it's like ~250ms) with good accuracy (depending on task).
9/23/2026, 9:32:48 AM
by: leecarraher
kinda feel like the "this is a parody blog post, see these links for better/more complete open implementations of Jev..."<p>is a legal cya a la "Nathan For You" 's Dumb Starbucks
9/23/2026, 7:14:10 PM
by: faangguyindia
I've built something similar, i am hosting it.<p>You can test Jev like model at 26B parameter count here (built few weeks ago): <a href="https://gambler-relay-us-west1.leo-fish.ts.net/demo" rel="nofollow">https://gambler-relay-us-west1.leo-fish.ts.net/demo</a> (might not stay up for long)<p>Typesafe compatible API<p>This is just running on old hardware.
9/23/2026, 9:24:54 AM
by: brap
What I don’t understand is, why would you <i>not</i> want “reasoning” in a classifier?<p>Speed and cost are obvious reasons, but isn’t this a tradeoff?
9/23/2026, 8:37:07 AM
by: teravor
to be fair a Jev architecture would be better optimized for this particular workflow than an LLM.<p><pre><code> <think>\n\n</think> </code></pre> but letting an LLM think would trade latency and performance for significant reliability above that of Jev.
9/23/2026, 4:45:04 PM
by: qurren
> Email: {email}\n\n{options}<|im_end|><p>Why do I have to feed my e-mail into the model?
9/23/2026, 5:07:54 PM
by: param_gupta
Pretty interesting how a simple example like this makes the idea so easy to understand.
9/23/2026, 9:40:51 AM
by: philipbk
> "25 lines of python" > "import Solution"<p>ok
9/23/2026, 1:04:35 PM
by: tracyhenry
This, like the hype of Jev on Twitter, totally ignores accuracy and generality across domains.<p>In my experience even structured LLM output performs poorly on classifier tasks. LLMs are trained to talk and think longer. If you don't give LLM enough space to reason it would become very dumb.<p>I'm not saying that Jev is way better, but that people way overindexed cost and speed.
9/23/2026, 5:27:51 PM
by: imranq
This guy just seems a bit salty
9/23/2026, 3:08:41 PM
by: max979
This makes me wonder about using Jevko for configuration instead of YAML, especially with such a compact parser.
9/23/2026, 12:49:45 PM
by:
9/23/2026, 8:30:15 AM
by: nlpnerd
This is basically Temu Jev
9/23/2026, 7:37:26 PM
by: marcy_74
Reminds me of my own tiny Lisp interpreter attempts; that moment when it first evaluates a simple expression is pure magic.
9/23/2026, 11:32:43 AM
by: shawabawa3
strong "You can build dropbox quite trivially by getting an FTP account, mounting it locally with curlftpfs, and then using SVN or CVS on the mounted filesystem" vibes<p>You have built something like jev but not jev (for starters, the output of what you've built will be absolutely worthless, the whole reason Jev is getting so much hype is because the output is good enough)
9/23/2026, 8:45:37 AM
by: heaney-555
Latency and compute comparison needed.
9/23/2026, 8:04:26 AM
by: qainsights
THis is not Jev.
9/23/2026, 2:44:05 PM
by: revexos
Startup coming out of 2 years of stealth to be reproduced this easily
9/23/2026, 9:42:30 AM
by: pietz
I'm surprised something like Jev came out "so late", but the hype has been ridiculous. Yes, it's a good idea. No, it only helps when fast and cheap are important and I guarantee existing labs will have this figured out in a matter of days.<p>Add visual understanding, add reasoning and bring down the size to run on my computer. That's when it will be interesting.<p>So many people that don't understand the tech jumped on the hype train because "it cannot hallucinate" and else. It's crazy.
9/23/2026, 12:18:43 PM
by: ricardobeat
Now, can you do it in <200ms for 45 questions at once, have 0% malformed output, and any kind of meaningful benchmark? We’ll wait!
9/23/2026, 8:23:02 AM
by: teaonly
The principle is this.
9/23/2026, 8:15:02 AM
by: esafak
Latency-calibration charts or it didn't happen. (LLMs are not optimized for calibration.)
9/23/2026, 1:41:20 PM
by: zteppenwolf
I get dishonest vibes from this post? Jev claims to be cheaper/more efficient, and the post claims just to achieve the same functionality.
9/23/2026, 11:29:52 AM
by: iLoveOncall
Nothing I hate more than bullshit articles claiming X in Y lines of code, only to use libraries abstracting hundreds of thousands of lines of code.
9/23/2026, 8:44:57 AM
by: flashnik86
[flagged]
9/23/2026, 7:53:10 PM
by: itsmeduncan
[dead]
9/23/2026, 8:00:40 PM
by: dingody
[dead]
9/23/2026, 9:52:59 AM
by: mentalgear
[dead]
9/23/2026, 9:58:32 AM
by: 9deee6b320d
[dead]
9/23/2026, 9:24:01 PM
by: CROON_tv
[dead]
9/23/2026, 12:01:16 PM
by: portaldorateio
[dead]
9/23/2026, 9:00:48 PM
by: kanari
[dead]
9/23/2026, 8:30:35 AM
by: baobabKoodaa
I'm so sick of seeing these people who "made Jev in 25 lines of Python" or whatever the flavor of the day is. Do you people seriously think that Qwen3-0.6B-Q8_0.gguf is frontier intelligence? If you want to argue that Jev is NOT frontier intelligence, then go make that argument. Don't try to pretend that Qwen3-0.6B-Q8_0.gguf is frontier intelligence. That's retarded.
9/23/2026, 10:38:20 AM