Scoring existing text

Normally the probe scores tokens as the model generates them. The -analyze suffix scores text that already exists — output from another model, an archived transcript, a draft a human wrote. Nothing is generated.

Append -analyze to any model id and put the text to score in an assistant message:

curl https://api.telluvian.ai/v1/chat/completions \
  -H "Authorization: Bearer $TELLUVIAN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "google/gemma-4-31B-it-analyze",
    "messages": [
      {"role": "user", "content": "When did the Eiffel Tower open?"},
      {"role": "assistant", "content": "The Eiffel Tower opened in 1889."}
    ]
  }'

The response is an ordinary completion object. content is the text you sent back verbatim, and tokens / scores line up with it exactly as they do on a generated response:

{
  "model": "google/gemma-4-31B-it",
  "choices": [{ "message": { "content": "The Eiffel Tower opened in 1889." } }],
  "usage": { "prompt_tokens": 47, "completion_tokens": 11, "total_tokens": 58 },
  "tokens": ["The", " Eiffel", " Tower", " opened", " in", " ", "1", "8", "8", "9", "."],
  "scores": { "hallucination": [0.0, 0.0, 0.0, 0.0, 0.001, ...] }
}

How the messages are read

The rule is positional, not structural:

• The last assistant message is the text scored. Nothing else is.
• The last user message before it becomes the context the text is scored against. It is a framing prompt, not something the model answers.

Everything earlier in messages is ignored. To score a multi-turn transcript, send one request per assistant turn rather than the whole conversation.

The user turn is genuinely worth setting, and its effect is larger than it looks. The probe judges the text against the context it is given, so the same assistant text scores differently under different user turns. Scoring one identical false claim under two phrasings:

Same text, same probe, and a 0.5 threshold flags the first and misses the second. Send the real prompt that produced the text rather than a summary of it, and keep it stable across a batch you intend to compare — a prompt that varies run to run moves your scores independently of the content. If you omit the user turn entirely, a generic one is substituted.

The tokenizer is always the probe's

This is the part that surprises people, and it is the reason this page exists.

The -analyze suffix changes what is scored, never what does the scoring. The probe reads hidden states, and hidden states only exist inside the model that runs the forward pass — so analyze always prefills your text through the local Gemma model, whatever id you named.

// request:  "model": "anthropic/claude-opus-5-analyze"
// response:
{
  "model": "anthropic/claude-opus-5",
  "tokens": ["The", " Eiffel", " Tower", " opened", " in", " ", "1", "8", "8", "9", "."]
}

Those are Gemma's tokens, not Anthropic's, even though the response echoes the Anthropic model id. The suffix on a routed model affects billing and labelling — it does not route the scoring pass anywhere.

This is a statement about the tokenizer, not about which id you should send. You should still name the model that generated the text — see Name the model that wrote the text below. The point here is only that whatever you name, the token boundaries coming back are the probe model's, so that is what you must align against.

{"error": {"message": "The model 'anthropic/claude-sonnet-4-5-analyze' does not exist ...",
           "code": "model_not_found"}}

Syncing scores with your own tokens

tokens is the alignment key. It is guaranteed that:

"".join(tokens) == choices[0].message.content

That equality is what lets you map scores onto any other segmentation — your own tokenizer, words, sentences, or UI spans — without re-tokenising anything. Walk the tokens, keep a running character offset, and you have a character range for every score:

def spans(tokens, scores):
    """Yield (start, end, score) character ranges over the original text."""
    pos = 0
    for token, score in zip(tokens, scores):
        yield pos, pos + len(token), score
        pos += len(token)

From character ranges, everything else is a regrouping. To fold the probe's tokens up into your own — the usual case when you generated the text elsewhere and want scores against that provider's tokens — assign each of your tokens the scores whose ranges overlap it:

def align(their_tokens, probe_tokens, probe_scores, reduce=max):
    """Re-express probe scores over a different tokenization of the same text."""
    probe_spans = list(spans(probe_tokens, probe_scores))
    out, pos = [], 0
    for token in their_tokens:
        start, end = pos, pos + len(token)
        overlapping = [
            s for (a, b, s) in probe_spans
            if a < end and b > start and s is not None
        ]
        out.append(reduce(overlapping) if overlapping else None)
        pos = end
    return out

The same overlap logic covers sentence and paragraph alignment — replace their_tokens with the spans of your own segmentation. Because the join equality above holds by construction, this needs no tokenizer on your side at all.

Scores can be null

Exactly as on the generated path, an individual entry in a score array can be null — the array is always the same length as tokens, so index i always describes tokens[i], but a particular token's probe reading may be unavailable. A null means "no score for this token", not a score of zero. The align helper above drops them before reducing, and returns None for one of your tokens only when every overlapping probe token was null.

You don't need to route generation through us

The common reason to reach for analyze is wanting hallucination scores on output your own stack produced — a coding agent's diffs and reasoning, a pipeline you already run against your own provider account — without making this API the gateway that generation flows through.

That is what analyze is for. There is no "bring your own frontier-model token" option, and you do not need one: you never send us a provider key, because we never call your provider. You generate wherever you already do, then send the finished text here to be scored. Two independent calls, two separate bills, and no credential of yours ever leaves your own stack.

curl

Step 1 — generate against your provider, with your key. Telluvian is not involved and never sees $OPENAI_API_KEY:

ANSWER=$(curl -s https://api.openai.com/v1/chat/completions \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-5.6",
    "messages": [{"role": "user", "content": "Explain the fix in this diff."}]
  }' | jq -r '.choices[0].message.content')

Step 2 — send that finished text to Telluvian for scoring. The only key here is your Telluvian key:

# The model id names what GENERATED the text — the same model as step 1,
# with -analyze appended. Not the probe model that does the scoring.
jq -n --arg prompt "Explain the fix in this diff." --arg answer "$ANSWER" '{
    model: "openai/gpt-5.6-terra-analyze",
    messages: [
      {role: "user",      content: $prompt},
      {role: "assistant", content: $answer}
    ]
  }' | curl -s https://api.telluvian.ai/v1/chat/completions \
    -H "Authorization: Bearer $TELLUVIAN_API_KEY" \
    -H "Content-Type: application/json" \
    -d @- | jq '[.tokens, .scores.hallucination] | transpose
                | map(select(.[1] != null and .[1] > 0.5))'

That last jq prints just the flagged tokens and their scores:

[["indexed", 0.626]]

Python

# 1. Generate wherever you already generate. Your account, your key,
#    your provider — this API is not involved and never sees the key.
completion = your_existing_client.chat.completions.create(...)
answer = completion.choices[0].message.content

# 2. Score the finished text. The model id names whatever generated it
#    in step 1, with -analyze appended.
scored = telluvian.chat.completions.create(
    model="openai/gpt-5.6-terra-analyze",
    messages=[
        {"role": "user", "content": original_prompt},
        {"role": "assistant", "content": answer},
    ],
)

for token, score in zip(scored.tokens, scored.scores["hallucination"]):
    if score is not None and score > 0.5:
        print(f"flagged: {token!r}")

You are billed for the analyze request only — the scored text as completion tokens, plus the probe surcharge. Whatever the original generation cost, you paid your own provider for it directly, and it does not appear on your Telluvian bill.

Scoring agent output

For coding-agent output specifically, the useful signal is where the score moves within an explanation, not the average over it. Given a bad diff justified by a false claim, the probe localises the claim:

 0.001  'I'
 0.001  ' changed'
 0.180  'range'
 0.229  '('
 ...
 0.023  ' safe'
 0.014  ' because'
 0.057  ' Python'
 0.123  ' lists'
 0.110  '1'
 0.626  'indexed'      <-- "Python lists are 1-indexed"
 0.023  ' so'

Correct the claim to "0-indexed" and leave the rest of the message identical, and that spike disappears — the peak over the whole response drops to 0.25, which is the level the code fragments sit at anyway.

Streaming

Analyze mode streams. Chunks arrive in the same shape as a streamed completion: one token per chunk, its score on chunk.scores. The deltas replay the text you sent rather than producing anything new, so streaming here buys you progressive scores on a long document, not faster time-to-first-word.

Billing

Analyze is billed as an ordinary request against the base model — the registry strips -analyze before pricing, so google/gemma-4-31B-it-analyze bills at google/gemma-4-31B-it's rate. The scored text counts as completion tokens (it is what the probe did work on), and the probe surcharge applies on those, as set out on the pricing page.