> ## Documentation Index
> Fetch the complete documentation index at: https://telluvian.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Scoring existing text

> The -analyze suffix scores text you already have, and how to align its tokens with your own.

Source: https://telluvian.ai/docs/analyze

Normally the probe scores tokens as the model generates them. The `-analyze`
suffix scores text that **already exists**: output from another model, an
archived transcript, a draft a human wrote. Nothing is generated.

> **Want to use your own provider key?**
>
> For analyze you don't need one. **This endpoint never calls your model
> provider, so there is no key to bring.** You generate on your own account,
> with your own key, and send us only the finished text to score. Telluvian does
> not have to be your gateway. Jump to
> [the recipe](#you-dont-need-to-generate-through-us). To generate through
> Telluvian on your own provider account instead, add the key as a
> [provider key](https://telluvian.ai/docs/byok.md).

Append `-analyze` to a catalogued model id, set `include_scores: true`, and
put the text to score in an `assistant` message:

```bash
curl https://api.telluvian.ai/v1/chat/completions \
  -H "Authorization: Bearer $TELLUVIAN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "google/gemini-3.8-flash-analyze",
  "messages": [
    {
      "role": "user",
      "content": "When did the Eiffel Tower open?"
    },
    {
      "role": "assistant",
      "content": "The Eiffel Tower opened in 1889."
    }
  ],
  "include_scores": true
}'
```

The response is an ordinary completion object. `content` is the text you sent
back verbatim, and `tokens` / `scores` line up with it exactly as they do on
[a generated response](https://telluvian.ai/docs/scores.md):

```json
{
  "model": "google/gemini-3.8-flash",
  "choices": [{ "message": { "content": "The Eiffel Tower opened in 1889." } }],
  "usage": { "prompt_tokens": 47, "completion_tokens": 11, "total_tokens": 58 },
  "tokens": ["The", " Eiffel", " Tower", " opened", " in", " ", "1", "8", "8", "9", "."],
  "scores": { "hallucination": [0.0, 0.0, 0.0, 0.0, 0.001, ...] }
}
```

## How the messages are read

The rule is positional, not structural:

- **The last `assistant` message is the text scored.** Nothing else is.
- **The last `user` message before it** becomes the context the text is scored
  against. It is a framing prompt, not something the model answers.

Everything earlier in `messages` is ignored. To score a multi-turn transcript,
send one request per assistant turn rather than the whole conversation.

> **No assistant message is a 422**
>
> A request whose `messages` contain no `assistant` role fails with
> **422, No assistant message found to analyze**. This is the one analyze-mode
> error that is not shared with the normal chat path, and it is easy to hit by
> sending the conversation exactly as you would for a completion.

The user turn is genuinely worth setting, and its effect is larger than it
looks. The probe judges the text against the context it is given, so the same
assistant text scores differently under different user turns. Scoring one
identical false claim under two phrasings can produce different scores.
The following values illustrate the effect; they are not an operating benchmark:

| User turn | Score on `indexed` |
| --- | --- |
| `Fix the off-by-one in this loop and explain the change.` | 0.626 |
| `Explain the fix in this diff.` | 0.466 |

Same text, same probe, and a 0.5 threshold flags the first and misses the
second. Send the **real** prompt that produced the text rather than a summary
of it, and keep it stable across a batch you intend to compare. A prompt that
varies run to run moves your scores independently of the content. If you omit
the user turn entirely, a generic one is substituted.

## The tokenizer is always the probe's

This is the part that surprises people, and it is the reason this page exists.

The `-analyze` suffix changes **what is scored**, never **what does the
scoring**. The probe reads hidden states, and hidden states only exist inside
the model that runs the forward pass, so analyze always prefills your text
through the local Gemma model, whatever id you named.

```json
// request:  "model": "anthropic/claude-opus-5-analyze"
// response:
{
  "model": "anthropic/claude-opus-5",
  "tokens": ["The", " Eiffel", " Tower", " opened", " in", " ", "1", "8", "8", "9", "."]
}
```

Those are Gemma's tokens, not Anthropic's, even though the response echoes the
Anthropic model id. The suffix on a replayed model affects billing and
labelling. It does **not** send the scoring pass anywhere else.

This is a statement about the *tokenizer*, not about which id you should send.
You should still name the model that generated the text. See [Name the model
that wrote the text](#you-dont-need-to-generate-through-us) below. The
point here is only that whatever you name, the token boundaries coming back are
the probe model's, so that is what you must align against.

> **Never assume the token boundaries are your model's**
>
> If you send Claude's or GPT's output to `-analyze` and then zip the returned
> `scores` against **your** provider's token list, the two will silently
> disagree: same text, different splits, and no error anywhere. `1889` above
> is one token to most tokenizers and four here. Always align on the returned
> `tokens` array, never on a token list from somewhere else.

> **model_not_found on an -analyze id**
>
> A 404 like:
>
  ```json
  {"error": {"message": "The model 'anthropic/claude-sonnet-4-5-analyze' does not exist ...",
             "code": "model_not_found"}}
  ```
>
> is almost never about the suffix. The base id has to be a model that actually
> exists in the catalogue. The suffix is stripped and the remainder looked up,
> so `anthropic/claude-sonnet-4-5-analyze` and bare `anthropic/claude-sonnet-4-5`
> fail identically. Drop the suffix and retry to confirm, then check
> `GET /v1/models` for the exact id. Version numbers are the usual culprit:
> use the exact currently listed base ID instead of assuming an older spelling
> is supported.

## Syncing scores with your own tokens

`tokens` is the alignment key. It is guaranteed that:

```
"".join(tokens) == choices[0].message.content
```

That equality is what lets you map scores onto any other segmentation: your own
tokenizer, words, sentences, or UI spans, without re-tokenising anything.
Walk the tokens, keep a running character offset, and you have a character
range for every score:

```python
def spans(tokens, scores):
    """Yield (start, end, score) character ranges over the original text."""
    pos = 0
    for token, score in zip(tokens, scores):
        yield pos, pos + len(token), score
        pos += len(token)
```

From character ranges, everything else is a regrouping. To fold the probe's
tokens up into your own, the usual case when you generated the text elsewhere
and want scores against that provider's tokens, assign each of your tokens the
scores whose ranges overlap it:

```python
def align(their_tokens, probe_tokens, probe_scores, reduce=max):
    """Re-express probe scores over a different tokenization of the same text."""
    probe_spans = list(spans(probe_tokens, probe_scores))
    out, pos = [], 0
    for token in their_tokens:
        start, end = pos, pos + len(token)
        overlapping = [
            s for (a, b, s) in probe_spans
            if a < end and b > start and s is not None
        ]
        out.append(reduce(overlapping) if overlapping else None)
        pos = end
    return out
```

> **Pick the reduction deliberately**
>
> Where several probe tokens fall inside one of yours, `max` is the right
> default for flagging: it is the "was any part of this suspect?" question, and
> it keeps a single high-scoring fragment (`9` in a hallucinated year) from
> being averaged away by the neutral fragments around it. Use `mean` only when
> you are ranking spans rather than flagging them.

The same overlap logic covers sentence and paragraph alignment. Replace
`their_tokens` with the spans of your own segmentation. Because the join
equality above holds by construction, this needs no tokenizer on your side at
all.

> **Whitespace lives in the tokens**
>
> Token text includes its leading whitespace (`" Eiffel"`, and a bare `" "`
> before `1889`). The offsets are therefore exact against the original string,
> but if you `strip()` tokens before measuring lengths, every offset after the
> first space will be wrong. Measure first, strip only for display.

## Scores can be null

Exactly as on the generated path, an individual entry in a score array can be
`null`. The array is always the same length as `tokens`, so index `i` always
describes `tokens[i]`, but a particular token's probe reading may be
unavailable. A `null` means "no score for this token", not a score of zero. The
`align` helper above drops them before reducing, and returns `None` for one of
your tokens only when every overlapping probe token was null.

## You don't need to generate through us

The common reason to reach for analyze is wanting per-token scores on
output your own stack produced, such as a coding agent's diffs and reasoning or
a pipeline you already run against your own provider account, without making
this API the gateway that generation flows through.

That is what analyze is for. It needs no provider key: **you never send us
one, because we never call your provider.** You generate wherever you already
do, then send the finished text here to be scored. Two independent calls, two
separate bills, and no credential of yours ever leaves your own stack. (To
generate through Telluvian on your own provider account instead, see
[provider keys](https://telluvian.ai/docs/byok.md).)

### curl

Step 1. Generate against **your** provider, with **your** key. Telluvian is
not involved and never sees `$OPENAI_API_KEY`:

```bash
ANSWER=$(curl -s https://api.openai.com/v1/chat/completions \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-6.1-terra",
    "messages": [{"role": "user", "content": "Explain the fix in this diff."}]
  }' | jq -r '.choices[0].message.content')
```

Step 2. Send that finished text to Telluvian for scoring. The only key here is
your Telluvian key:

```bash
# The model id names what GENERATED the text: the same model as step 1,
# with -analyze appended. Not the probe model that does the scoring.
jq -n --arg prompt "Explain the fix in this diff." --arg answer "$ANSWER" '{
    model: "openai/gpt-6.1-terra-analyze",
    include_scores: true,
    messages: [
      {role: "user",      content: $prompt},
      {role: "assistant", content: $answer}
    ]
  }' | curl -s https://api.telluvian.ai/v1/chat/completions \
    -H "Authorization: Bearer $TELLUVIAN_API_KEY" \
    -H "Content-Type: application/json" \
    -d @- | jq '[.tokens, .scores.hallucination] | transpose
                | map(select(.[1] != null and .[1] > 0.5))'
```

That last `jq` prints just the flagged tokens and their scores:

```json
[["indexed", 0.626]]
```

> **Build the JSON with jq, not string interpolation**
>
> `jq -n --arg` is doing real work here: model output routinely contains quotes,
> backslashes and newlines, and for coding agents that is near-guaranteed, all of
> which break a hand-interpolated `-d "{\"content\": \"$ANSWER\"}"`. Passing
> the text through `--arg` escapes it correctly, and `-d @-` avoids a
> too-long-argument error on a large diff.

### Python

```python
# 1. Generate wherever you already generate. Your account, your key,
#    your provider. This API is not involved and never sees the key.
completion = your_existing_client.chat.completions.create(...)
answer = completion.choices[0].message.content

# 2. Score the finished text. The model id names whatever generated it
#    in step 1, with -analyze appended.
scored = telluvian.chat.completions.create(
    model="openai/gpt-6.1-terra-analyze",
    extra_body={"include_scores": True},
    messages=[
        {"role": "user", "content": original_prompt},
        {"role": "assistant", "content": answer},
    ],
)

for token, score in zip(scored.tokens, scored.scores["hallucination"]):
    if score is not None and score > 0.5:
        print(f"flagged: {token!r}")
```

You are billed for the analyze request only: the scored text as completion
tokens, plus any metered prompt tokens and the verification fee. Whatever the original generation cost, you
paid your own provider for it directly, and it does not appear on your
Telluvian bill.

> **Name the model that wrote the text**
>
> The base model in an analyze request identifies **what generated the text you
> are sending**. Scoring GPT-6.1 output means `openai/gpt-6.1-terra-analyze`;
> scoring Claude output means `anthropic/claude-sonnet-5-analyze`. Only use
> `google/gemma-4-31b-it-analyze` when Gemma actually wrote it.
>
> Nothing is sent to that provider. Analyze never calls out, it only prefills
> your text locally. But the id is how the request records which model's output
> was scored, so naming a different one both misattributes the result and bills
> you against the wrong rate card.

> **Today the scoring pass is identical whatever you name**
>
> Being straight about the current implementation: every analyze request is
> prefilled through the same local Gemma container. The same text scored under
> `openai/gpt-6.1-terra-analyze`, `anthropic/claude-sonnet-5-analyze` and
> `google/gemma-4-31b-it-analyze` returns **the same numbers**. Naming the true
> generating model is attribution and billing today, not fidelity.
>
> No per-provider adapter behavior is promised by this API. Validate scores
> on your own workload rather than assuming that naming a provider changes
> the scoring model.

### Scoring agent output

For coding-agent output specifically, the useful signal is where the score
moves *within* an explanation, not the average over it. Given a bad diff
justified by a false claim, the probe localises the claim:

```
 0.001  'I'
 0.001  ' changed'
 0.180  'range'
 0.229  '('
 ...
 0.023  ' safe'
 0.014  ' because'
 0.057  ' Python'
 0.123  ' lists'
 0.110  '1'
 0.626  'indexed'      <-- "Python lists are 1-indexed"
 0.023  ' so'
```

These numbers are illustrative, not guaranteed outputs. Compare corrected and
incorrect statements on your own workload before choosing a threshold.

> **Code fragments score higher than prose at rest**
>
> Note the baseline in that example: `range`, `(`, `n`, `+`, `1` all sit around
> 0.1 to 0.25 with nothing wrong at all. Identifier and punctuation tokens simply
> carry more probe noise than ordinary prose, so a single global threshold
> tuned on English will over-flag every diff you send it.
>
> Calibrate on your own traffic before trusting a cutoff: score a batch of
> known-good agent messages, look at where their peaks actually land, and set
> the threshold above that. Comparing a message against that baseline is far
> more reliable than comparing it against an absolute number.

> **Scope: claims, not compilation**
>
> The probe reads the model's internal state for signs the text is unsupported.
> That catches confident false *statements*, such as wrong invariants,
> invented API behaviour and misremembered semantics, which is a large share
> of what makes a bad diff convincing. It is not a substitute for
> type-checking or tests, and it will not tell you a diff fails to compile.
> Treat a spike as "this claim deserves a second look", and keep the tests.

## Streaming

Analyze mode streams. Chunks arrive in the same shape as [a streamed
completion](https://telluvian.ai/docs/scores.md#streaming): one token per chunk, its score on
`chunk.scores`. The deltas replay the text you sent rather than producing
anything new, so streaming here buys you progressive scores on a long document,
not faster time-to-first-word.

## Billing

Analyze is billed as an ordinary request against the **base** model. The
registry strips `-analyze` before pricing, so `google/gemini-3.8-flash-analyze`
bills at `google/gemini-3.8-flash`'s rate. The scored text counts as completion
tokens (it is what the probe did work on), and the verification fee applies on
those, as set out on the [pricing page](https://telluvian.ai/docs/pricing.md).

> **Analyze requires explicit opt-in**
>
> Omitted or false `include_scores` is rejected with `400` before the scoring
> request reaches the GPU. Set `include_scores: true` to request analysis.
>
