AI · LLM evaluation
Jev Lab
Experiments with an LLM decision model, including a feed sieve that reads 500 feeds so you don’t have to.
Two small experiments with Jev, TypeSafe’s hosted decision model: a model that answers typed questions (pick one option, place something on a scale, yes or no) with calibrated probabilities instead of prose.
The main one is a feed sieve. It reads a week of posts from over 500 public RSS, Atom and YouTube feeds and asks four questions about each one in a single call: how worth the reader’s time it is, given their angle on that topic, and three yes-or-no checks for beginner material, marketing and anything with a deadline.
On a real run it read 8,345 items, judged the 1,063 from the last seven days in about a minute for four cents, and picked thirty for the week, with no more than three per topic and two per source and duplicates from different outlets collapsed.
The design work was in the questions and the profile. A topic list isn’t enough: an article on three-ball juggling basics is on topic and useless to someone who already juggles five, so the profile gives each topic the reader’s level and what they want and skip. The second experiment asks the same model to file bookmarks with the answers hidden, and measures how far its confidence can be trusted.
IN PICTURES
