Laya vs Jev: Two Ways to Classify a Support Ticket

Run Laya on a Modal GPU and compare its structured decisions with TypeSafe Jev through OpenRouter, using the same support ticket and questions.
AI
Classification
Machine Learning
Python
Author

David Gwyer

Published

September 23, 2026

Laya and Jev: Classifiers That Don’t Generate

People use LLMs not just for text generation, but to make decisions too. Things like ‘Which queue does this support ticket go in?’, ‘How urgent is it?’, ‘Does it need a human?’. These are all classification problems, and generating a paragraph of prose to answer them is slower and less direct than it needs to be.

Classification-first models are built for exactly this. Rather than generating text token by token, they return typed decisions such as labels, scores and probabilities directly. Two recent examples of this type of model are Jev and Laya.

This post compares them on the same inputs, focusing on what’s involved in actually running them rather than on benchmark scores. Everything below was run in SolveIt, a notebook-style environment where code, output and commentary live in a single dialog.

Meet the Two Models

Laya answers typed choice, score and Noul (yes/no) questions in one non-autoregressive pass, with no token-by-token decoding. It ships as three checkpoints:

checkpoint scope params context
laya English 421M 512
laya-multilingual 100+ languages 322M 1,024
laya-typed-decisions decision workflows 421M 1,024

Small enough to run yourself, which is exactly what we’ll do.

Jev takes the same idea further. It’s TypeSafe’s first System One model, currently in early access, and the trick is that the output schema is fixed up front, so it structurally can’t return a type error or invent an option you didn’t offer. A parallel sampler answers every question in one query, and each answer comes with a calibrated probability from a training method TypeSafe calls Reinforcement Learning for Calibrated Decisions (RLCD). At the time of this run, Jev cost $0.042 per million input tokens, with output free, and supported up to 255 options per choice.

What We’ll Run

Laya’s typed-decisions checkpoint on a Modal T4 GPU, driven from SolveIt, so you’ll need a Modal account configured. The same ticket and the same questions then go to hosted Jev 1.13 via OpenRouter, which needs an API key stored as a SolveIt secret named OPENROUTER_KEY. Swap in your own ticket text and questions to try a case of your own.

Model Setup

Laya can run locally, but for this example we put it on a Modal T4 GPU. We define a container image with laya installed, plus one function that loads the checkpoint and answers the questions remotely.

import modal

image = modal.Image.debian_slim(python_version="3.12").pip_install("laya==0.3.7")
app = modal.App("laya-solveit-mvp")

@app.function(image=image, gpu="T4", timeout=600)
def decide(ticket, questions):
    import laya
    import torch

    if not torch.cuda.is_available():
        raise RuntimeError("Modal GPU is unavailable")

    agent = laya.load(
        "convaiinnovations/laya",
        subfolder="typed-decisions",
        device="cuda",
    )
    result = agent.predict(ticket, questions)
    return {
        "gpu": torch.cuda.get_device_name(0),
        "answers": result["answers"],
    }

modal.Image describes the container: Debian, Python 3.12, and a pinned laya release. The @app.function decorator attaches a T4 GPU and a 10-minute ceiling to decide, which runs remotely rather than in this notebook. Inside it, laya.load pulls the typed-decisions checkpoint onto the GPU, and agent.predict answers every question in a single forward pass. The explicit cuda.is_available() check fails loudly if the GPU never attached, instead of silently falling back to CPU.

Jev is a hosted API, so its setup is much lighter. There’s no image to build and no GPU to reserve, just a key to read.

import json
import os
import urllib.error
import urllib.request

api_key = os.environ.get("OPENROUTER_KEY")
if not api_key:
    raise RuntimeError("Set the OPENROUTER_KEY environment variable.")

The Jev call itself needs only standard-library modules: json to build and parse the request body, and urllib to send it. Reading the key here rather than at call time means a missing or misnamed secret fails immediately, with a message naming the secret to create.

Both models see exactly the same input: one ticket and three questions, defined once here so the comparison is like for like.

ticket = {
    "subject": "Duplicate invoice #4411",
    "body": (
        "We were billed twice for March. Please refund the extra charge "
        "by Friday or we may cancel."
    ),
}

questions = {
    "team": {
        "type": "choice",
        "instructions": "Which team should handle this?",
        "criteria": {
            "billing": "invoices, charges, refunds",
            "technical": "bugs and outages",
            "sales": "plans and pricing",
        },
    },
    "urgency": {
        "type": "score",
        "instructions": "How urgent is this?",
        "criteria": [
            "routine",
            "needs attention soon",
            "blocking issue or deadline",
        ],
    },
    "refund_requested": {
        "type": "noul",
        "instructions": "Does the customer explicitly ask for a refund?",
    },
}

The ticket data is an ordinary Python dictionary of text. Each question names its answer type: choice picks one labelled option and so carries a criteria dict describing each one; score returns a rating, with criteria as an ordered list from lowest to highest; noul returns a probability from 0 to 1 that the answer is yes, and needs no criteria. The dict keys (team, urgency, refund_requested) become the keys of the returned answers.

Laya Inference on a Modal T4

We start with Laya, running the typed-decisions checkpoint on a Modal T4 GPU.

with app.run():
    laya_result = decide.remote(ticket, questions)

print("GPU:", laya_result["gpu"])
for name, answer in laya_result["answers"].items():
    print(f"{name}: {json.dumps(answer, sort_keys=True)}")
GPU: Tesla T4
team: {"action": {"act_probability": 1.0}, "choice": "billing", "confidence": 0.5301, "probabilities": {"billing": 0.854, "sales": 0.0797, "technical": 0.0662}, "type": "choice"}
urgency: {"action": {"act_probability": 1.0}, "confidence": 0.3723, "legend": {"0": "routine", "1": "needs attention soon", "2": "blocking issue or deadline"}, "probabilities": {"0": 0.023, "1": 0.2744, "2": 0.7026}, "score": 1.6796, "type": "score"}
refund_requested: {"action": {"act_probability": 1.0}, "confidence": 0.6579, "noul": 0.6579, "type": "noul"}

The output loop prints each full answer object, in the same form as the Jev cell below, so the two sets of results can be read side by side. In this run Laya returned billing for the team, an urgency score of 1.68, and a yes probability of 0.66 for the refund question. The urgency scale runs from 0 to 2 because the question supplied three ordered criteria. The action.act_probability field appears in each Laya answer, but Laya’s documentation says it currently carries no usable signal; the 1.0 values here should not be interpreted as confidence.

Jev Inference via OpenRouter

Now the same ticket and the same questions go to TypeSafe’s Jev 1.13, reached over HTTP through OpenRouter’s Decisions API. Nothing is installed locally and no GPU is reserved. The modules and the API key were prepared back in Setup, so all that remains here is to describe the request, send it, and read the answers back.

payload = {
    "model": "typesafe/jev-1.13",
    "state": ticket,
    "questions": questions,
}
request = urllib.request.Request(
    "https://openrouter.ai/api/alpha/decisions",
    data=json.dumps(payload).encode("utf-8"),
    headers={
        "Authorization": f"Bearer {api_key}",
        "Content-Type": "application/json",
    },
    method="POST",
)
try:
    with urllib.request.urlopen(request, timeout=60) as response:
        jev_result = json.load(response)
except urllib.error.HTTPError as error:
    detail = error.read().decode("utf-8", errors="replace")
    raise RuntimeError(f"OpenRouter HTTP {error.code}: {detail}") from error

print("Model:", jev_result.get("model"))
for name, answer in jev_result["answers"].items():
    print(f"{name}: {json.dumps(answer, sort_keys=True)}")
if "usage" in jev_result:
    print("Usage:", json.dumps(jev_result["usage"], sort_keys=True))
Model: typesafe/jev-1.13-20260917
team: {"choice": "billing", "confidence": 1, "probabilities": {"billing": 1, "sales": 0, "technical": 0}, "type": "choice"}
urgency: {"confidence": 0.71, "legend": {"0": "routine", "1": "needs attention soon", "2": "blocking issue or deadline"}, "probabilities": {"0": 0, "1": 0.19, "2": 0.81}, "score": 1.81, "type": "score"}
refund_requested: {"noul": 0.99, "type": "noul"}
Usage: {"cost": 1.785e-05, "input_tokens": 425, "output_tokens": 70}

In this run, OpenRouter returned the Jev snapshot typesafe/jev-1.13-20260917. The request used 425 input and 70 output tokens and cost $0.00001785; the output tokens were reported but not billed. Jev returned per-option probabilities and confidence for the team choice, and probabilities, confidence, and a legend for urgency. Its Noul refund answer is a yes probability of 0.99. The models agree on billing and the refund request, and return similar elevated urgency scores (Laya 1.68; Jev 1.81).

Results Compared

Question Laya · Modal T4 Jev 1.13 · OpenRouter
Team billing (85% probability) billing (100% probability)
Urgency 1.68 / 2 (70% blocking, 27% needs attention soon) 1.81 / 2 (81% blocking, 19% needs attention soon)
Explicit refund request? 0.66 (leans yes) 0.99 (yes)

The two models agree on all three questions and differ mainly in how strongly they commit. The clearest difference is the refund question, where 0.66 against 0.99 is a real gap in strength of belief.

Summary

On this one duplicate-invoice ticket the two models agree on every question. Both routed it to billing, both scored urgency high, and both read the message as an explicit refund request. That makes this a successful end-to-end test of two quite different inference paths, not an accuracy benchmark. However, a single ticket cannot tell us which model classifies better. It’s more of a smoke test.

The one answer worth a second look is Laya’s 0.66 on the refund question. The customer writes “please refund the extra charge”, which is about as explicit as a refund request gets, so a score nearer Jev’s 0.99 is what you would expect. Whether Laya is generally underconfident on this kind of question would take a set of labelled refund tickets to establish, rather than this single example.

Laya is a downloadable 421M-parameter model that we ran on a single T4 GPU. Jev follows a different deployment model. It can be accessed as a hosted service through OpenRouter.

Try SolveIt!

Every part of this post was written and run inside a single SolveIt dialog: the Modal app definition, the API call, the live outputs, and the commentary around them. Code and prose sit side by side. Plus you have full access to a range of powerful LLM models to assist your work.

Find out more about SolveIt at solve.it.com, and you can try it right away without needing a subscription. Signup here.

Community

Explore together.

Share what you’re learning, ask questions, and swap ideas.
Join the ExploringML Discord.

Join the community