Share of non-interactive tokens on batch tiers

How much background and evaluation traffic moves to 24-hour batch lanes for the 50% discount.

All trends

+28 % of background tokens over window

Jan

Feb

Mar

Apr

May

Jun

Measured in % of background tokens.

What this trend means.

Almost half of clearly non-interactive tokens now run on batch tiers. The blocker is rarely capability - it is that teams have not separated interactive from background traffic.

Tie-in

Trends describe the population; a diagnostic measures you. The two together tell you whether you are ahead of or behind the curve on reuse.

Other trend series.

Workloads with a >10k-token stable prefix

% of workloads

Median realized reuse ratio

% of input tokens

Routing mix by alias family

% of requests

Measure your reuse against the trend.

Run a diagnostic and see where your workload sits relative to these aggregate patterns.

Run a diagnostic The benchmark