> ## Documentation Index
> Fetch the complete documentation index at: https://telluvian.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Choosing probes

> Each probe answers one monitoring question. Find yours, run one or several, and read each the way it is meant to be read.

Source: https://telluvian.ai/docs/probes

Verification runs one or more **probes** over every generated token. Each probe
answers one question about the text, such as "is this claim likely false?" or
"does the model sound unsure here?". Start from the question you want
answered, then run the probes that answer it.

## Find your probe

| You want to | Probe | Read it | Starting threshold |
| --- | --- | --- | --- |
| [Catch hallucinations](https://telluvian.ai/docs/probes.md#probe-hallucination): claims that are likely false or unsupported | `hallucination` | token by token | 0.2 |
| [Know how uncertain the model is](https://telluvian.ai/docs/probes.md#probe-uncertainty): text where the model sounds unsure | `uncertainty` | over a window of 6 tokens | 0.1 |

Name the probes you want in `probes`, alongside `include_scores: true`:

```json
{
  "include_scores": true,
  "probes": ["hallucination", "uncertainty"]
}
```

Each probe comes back as its own key in `scores`, with one value per token,
aligned to `tokens`:

```json
{
  "tokens": ["It", " was", " probably", " built", " in", " 1887", "."],
  "scores": {
    "hallucination": [0.01, 0.01, 0.02, 0.02, 0.03, 0.58, 0.01],
    "uncertainty":   [0.02, 0.03, 0.41, 0.22, 0.12, 0.09, 0.04]
  }
}
```

> **Name every probe you want**
>
> With `include_scores: true` and no `probes` field, only `hallucination`
> runs. Any other probe runs only when you name it, so a probe added on our
> side never changes what your requests run or cost.

## Reading a probe

Every probe scores higher for more of what it measures, but probes measure
different things, so never read one probe's score as another's. Two things
differ between probes, and the table above gives both:

- **Token by token, or over a window.** A probe that flags a specific claim is
  read per token. A probe that measures a stretch of text, such as how unsure
  the writing sounds, is read as a rolling mean over the last few tokens, so a
  single word does not trip it. A backwards window works on a stream too.
- **The threshold.** Each probe has its own scale. The starting thresholds
  above are a place to begin, not an operating point: tune each one on your
  own data.

One function covers both kinds:

```python
def flagged(scores, window=1, threshold=0.2):
    """Indexes of tokens at or above the threshold, over a backwards window."""
    out = []
    for i in range(len(scores)):
        recent = [s for s in scores[max(0, i - window + 1) : i + 1] if s is not None]
        if scores[i] is not None and recent and sum(recent) / len(recent) >= threshold:
            out.append(i)
    return out

# (window, threshold) for each probe you run, from the table above
SETTINGS = {"hallucination": (1, 0.2), "uncertainty": (6, 0.1)}

for probe, (window, threshold) in SETTINGS.items():
    for i in flagged(response.scores[probe], window, threshold):
        print(f"{probe}: {response.tokens[i]!r}")
```

For one answer-level signal, such as "send this answer for review", use the
share of tokens a probe flags, or its highest windowed score.

> **No score is a probability**
>
> Scores are not calibrated to "chance this is wrong", and the useful cutoff
> is rarely 0.5. Treat a flag as a prompt to look again rather than a verdict.
> Email [hello@telluvian.ai](mailto:hello@telluvian.ai) for a probe's current
> operating point and its measured precision and recall.

## Running several probes

Probes are independent, and each covers something the others miss, so
combine them by need. Each probe below says which others it pairs with. For
example, `uncertainty` does not judge correctness, so a confidently wrong
answer scores low on it and high on `hallucination`: run both if you want to
catch that case.

Surface each probe's flags separately rather than adding scores together. When
streaming, each content chunk's `scores` carries one number per probe you
selected, for example `{"hallucination": 0.02, "uncertainty": 0.31}`. See
[Streaming](https://telluvian.ai/docs/scores.md#streaming).

The verification fee is charged **per probe**: $0.20 per 1M completion tokens for
each probe that runs, so two probes cost twice one. Run the probes your need
calls for, not all of them. See [Pricing](https://telluvian.ai/docs/pricing.md#verification).

## The probes

### Catch hallucinations: `hallucination`

Flags claims that are likely false or unsupported. It rises on:

- a specific claim, such as a date, name or number, that is likely false or unsupported

**Use it for** anything a reader will take as a factual claim: answers shown to users, summaries, research, and legal, finance and compliance output.

**It does not judge** how confident or unsure the answer sounds. Use the uncertainty probe for that.

**Read it** token by token, starting from a threshold of 0.2.

### Know how uncertain the model is: `uncertainty`

Flags text where the model sounds unsure. It rises on:

- hedging and guessing: "I think", "probably", "around"
- confusion: contradicting itself, losing the thread
- garbled text: broken grammar, switching language, repetition loops, cut-off sentences, made-up words
- refusals and partial refusals

**Use it for** deciding when an answer needs a second look: sending unsure answers for human review, asking a clarifying question, retrying with a stronger model, or catching a model that has started looping in a long agent run.

**It does not judge** correctness. A confidently wrong answer scores low.

**Read it** over a window of 6 tokens, with a backwards rolling mean, starting from a threshold of 0.1.

**Pair it with** `hallucination` to catch answers that are confidently wrong.

## Which probes your account can run

`GET /v1/probes` lists the probes you can select:

```bash
curl -sS https://api.telluvian.ai/v1/probes \
  -H "Authorization: Bearer $TELLUVIAN_API_KEY"
```

```json
{
  "object": "list",
  "data": [
    {
      "id": "hallucination",
      "object": "probe",
      "description": "...",
      "layers": [ ... ],
      "threshold": 0.2
    }
  ]
}
```

Each entry's `id` is the name you pass in `probes` and the key you read back
from `scores`. `threshold`, when present, is the probe's tuned per-token
operating point.

Naming a probe that is not available returns a `400` listing the ones that
are, before any generation runs, so nothing is charged. Accounts billed
through Google Cloud Marketplace can currently run `hallucination` only.
