How it works
What happens when you send Zumik a request
Zumik sits between your software and the AI providers. You send it a request the way you would send one to OpenAI; it works out which model should answer, avoids paying twice for anything the provider has already seen, and sends the answer back unchanged. Here is that, step by step.
One key instead of five. One bill instead of five. And a choice of model made per request rather than once, at the start of the project, by whoever set it up.
Your request arrives
It looks exactly like an OpenAI request, because that is what it is. Zumik reads it to work out what kind of job it is: a quick edit, a hard refactor, a long piece of reasoning.
DetailsStep 02A model is chosen
If you asked for a specific model, you get it. If you asked for auto, Zumik picks the cheapest one that will handle this job well - and records which one it picked and why.
DetailsStep 03The repeats are spotted
Most of an agent request is content the provider has already seen. Zumik arranges the request so the provider recognises it and charges the cache rate instead of full price.
DetailsStep 04It goes to the provider
Through our account or yours. If that provider is down or slow, Zumik moves to another model that can do the same job rather than handing you an error.
DetailsStep 05You get the answer back
In the same shape OpenAI would have sent it, streamed the same way, plus a header telling you what the routing saved you on that request.
DetailsStep 06And you can delete it
Remove stored context whenever you want. Zumik returns a receipt saying what was removed and when the last copy goes.
DetailsFor engineers · skip if you like
Logical state is not physical KV state.
Everything above is all most people need. The rest of this page is the architecture underneath, for anyone evaluating Zumik rather than using it. The most important decision in it is splitting identity into three layers, which keeps your handles stable while stopping cache implementation details from leaking into the API.
Logical identity
Artifacts, bundles, sessions, branches, snapshots. Customer-visible, opaque, independent of provider, model, or tokenizer.
snapshot_idMaterialization identity
The exact model-visible byte representation: tokenizer, prompt-compiler version, ordered block manifest. Two requests can share logical state but materialize differently.
materialization_keyKV realization compatibility
Whether an existing physical KV cache can be reused safely: model revision, quantization, engine, GPU topology, isolation namespace. The implementation detail that never leaks into product semantics.
kv_compatibility_keyTwo requests can share the same logical artifact yet need different KV realizations, different tokenizer, different quantization, managed versus BYOC. Collapsing these layers is how gateways end up leaking cache details into their API. Zumik refuses to.
A repeated prefix is not a cache hit.
Zumik reports what could be reused (opportunity) separately from what was reused (capture), and attaches an evidence level to every number so a prediction is never mistaken for a measurement.
Read the reasoningReproducibility
Every decision leaves a record you can replay.
A request pins one snapshot and one alias release. An alias release is immutable, changing a provider-model revision creates a new release rather than mutating the old one. Customer logs expose the release id, so any past routing decision can be explained, and a replay run can reproduce it on the same workload shape.
How aliases resolve{
"requested_model": "code.fast",
"alias_release": "alr_2026_06_09_003",
"resolved_model": "anthropic/claude-haiku-4-5",
"resolution_reason": "lowest_expected_latency_under_policy",
"trace_id": "trc_9f12…"
}That is the whole idea
Setting it up takes about five minutes and one changed setting. Or measure your current traffic first - that check is free and needs no payment method.