Latency

Measured 2026-10-26 in the shipped configuration (inference threads pinned to 1, session fan-out up to 4), strictly serial calls. Payloads are real text from a 30 MB agent/user-conversation corpus: three different documents per size where the corpus had them, each truncated to the size. The median is the middle document's own median; min and max bound the observed calls.

Measured, per mode

Each row is one mode measured on its own. The two model modes keep different amounts of text from the same input, so a comparison across modes in this table is not like-for-like - see below for the paired measurement.

Sizes are quoted in characters, caps are in tokens

The rows above are character counts because that is how the payloads were cut. The service caps and advertises its model modes in TOKENS (max_input_tokens in GET /v1/limits), because model work tracks the tokenizer's token count and not bytes: the same 40,000 characters measured 9,775, 10,977 and 12,020 tokens on three real documents - a 23% swing in model work for an identical character budget. One token is roughly 3.3-4.1 characters on this corpus; code and non-English text tokenize differently, so convert with the named tokenizer rather than with a ratio.

The extractive cap is the ceiling

Extractive is linear in input size and then runs into the request budget. At the 40,000-character cap, three real documents measured 25.51 s, 26.53 s and 33.38 s against a 35 s budget: all three completed, and the slowest used 95% of the budget. Under the previous 30 s budget one of those three returned the request-timeout status (408).

Practical reading: 40,000 characters is a ceiling, not a comfortable size. If you care about latency rather than the maximum, stay near 20,000, which measured 10.8-15.6 s.

The two modes do not do the same amount of work

Verified runs the deterministic and extractive stages and then an NLI verification pass over the result, so it is tempting to read it as "extractive plus change". The measurements do not support a ranking in either direction, because the two modes ship different defaults and keep different amounts of text.

Paired measurement, same document truncated to the same 20,000 characters, same request body, two repeats each, alternating, in the shipped configuration:

On document 0 both modes returned the same bytes and verified cost 1.5x. On documents 1 and 2 verified kept more text and finished sooner. We do not have a stage-level decomposition for extractive mode (only the verified pipeline exports per-stage timings), so we do not claim an ordering between the modes and we do not explain the gap here. Treat each mode's number as its own budget.

What the same data does show: content, not size, dominates. Two documents of the same length differ by up to 3x within a mode, and the NLI window count does not predict the verified cost - at 20,000 characters a document that built 20 windows took 19.8 s while one that built 640 took 10.7 s.

What held

Every cell that ran more than one repeat per document was repeat-call deterministic: identical output bytes across the repeats, in both model modes and in deterministic mode.

How to read this

• These are the numbers behind the flat per-call prices, not a promise per call. Cost is per call, not per second.
• Each cell is a small sample of real documents, not a corpus average. A denser document of the same length can cost more; the min and max columns are part of the answer, not noise.
• Deterministic mode does structural work only and does not run a model.
• These are serial-call numbers. Concurrency and mixed-mode load are not characterised here.
• Latency is not fidelity. Whether the compressed output kept what you need is answered by the receipt, not by these timings (receipt fields).

Cost basis

Billing is per call and cost follows the executed mode (billing). Prices are flat per call: deterministic free, extractive $0.0008, verified $0.0015 (live at GET /v1/limits). The measured processing times above are the compute behind those prices.