Reading the scores
What the probe returns, and how to interpret it.
Verification is optional and off by default. Send
include_scores: true to request a scores object
alongside the usual OpenAI fields, mapping each probe name to its output.
Choose which probes run
Discover available IDs and descriptions with authenticated GET /v1/probes.
Send include_scores: true and probes: ["hallucination"] to request only
hallucination detection. Omitting probes while opting in requests all loaded
response probes; omitting include_scores runs none. This also applies to
telluvian/gallery-1. See the selection table.
Keys in scores identify the returned probes. A missing key or null reading
means no score was returned, not that the text passed verification.
Non-streaming
scores holds one array per probe, the same length as the tokens array:
{
"choices": [{ "message": { "content": "The Eiffel Tower opened in 1889." } }],
"tokens": ["The", " Eiffel", " Tower", " opened", " in", " 1889", "."],
"scores": { "hallucination": [0.01, 0.02, 0.02, 0.05, 0.03, 0.61, 0.01] }
}tokens comes back alongside it: the decoded text of each generated token, in
order, such that joining them reproduces content exactly. This is what makes
the scores usable. Without it you would hold N scores against one opaque
string and have to re-tokenise with the identical tokeniser to line them up.
for token, score in zip(response.tokens, response.scores["hallucination"]):
if score is not None and score > 0.5:
print(f"flagged: {token!r}") # flagged: ' 1889'Scoring text you did not generate here
The same tokens/scores pairing is available for text that already
exists, such as another model's output or an archived transcript, via the
-analyze model
suffix. The token boundaries there are always the probe model's, which needs
care if you are holding a token list from another provider: see Scoring
existing text.
An individual score may be null
The score array is always the same length as tokens, so
scores[probe][i] always describes tokens[i]. An individual entry can
still be null when the probe's reading for that token was unavailable:
most often the last token of a response cut short by max_tokens.
A null means "no score for this token", not "a score of zero". Check for it
before comparing against a threshold, as above.
Streaming
With include_scores: true, content chunks can carry one number per probe.
Role, finish, and unavailable-score chunks may omit scores:
stream = client.chat.completions.create(
model="google/gemma-4-31b-it",
extra_body={"include_scores": True, "probes": ["hallucination"]},
messages=[{"role": "user", "content": "Tell me about Ada Lovelace."}],
stream=True,
)
for chunk in stream:
delta = chunk.choices[0].delta.content or ""
score = (getattr(chunk, "scores", None) or {}).get("hallucination")
print(delta, score)Turning verification off
Verification is off by default. Omit include_scores, or explicitly pass
include_scores: false, for a plain completion:
{
"model": "google/gemma-4-31b-it",
"messages": [{"role": "user", "content": "Hello"}],
"include_scores": false
}Both scores and tokens come back null, and the verification fee is not
charged. In the Python SDK this must go through extra_body; see
include_scores for the details and
Pricing for what it saves.
Interpreting a score
For the hallucination probe, a higher score means the probe considers the
token more likely to be unsupported. Other probes measure different properties;
do not interpret every probe's output as a hallucination score.
The score is not a probability of error
The scale is not calibrated to "chance this is wrong", and the useful cutoff is not 0.5. Pick a threshold from your own tolerance for false positives, and treat a flag as a prompt to verify a claim rather than a verdict. Email hello@telluvian.ai for the current operating point and its measured precision and recall.
How the score is produced
Externally generated completions are replayed through our probe model on our GPUs. Locally served models can instead be scored from their own forward pass. Each token is forced during the replay, so the probe reads hidden states for text it did not itself generate. That is a reconstruction rather than a direct reading, and it adds latency proportional to how much the model wrote.
Routed models use this replay mechanism, including models picked for you by Gallery. Scores still need validation on your workload; the replay does not expose the original provider model's hidden states.