HIP-1211: AI — The Model API. Status Draft. Hanzo architectural specification.
ai is the model API: the OpenAI- and Anthropic-compatible wire and the routing, memory, retrieval and fine-tuning surfaces behind it. The implementation is the hanzoai/ai module, mounted into a cloud binary by hanzoai/cloud apps/ai with the money, ingest and telemetry callbacks cloud builds but cannot install (apps/ai/ai.go:1-17). This HIP states the target address surface: the wire family at the root, everything else under /v1/ai.
An org can also route a model id of its own to an engine on its own overlay. That is proposed by the Amendment below and is not served yet.
Proposed with HIP-1172's amendment, which starts the engines these routes reach. Nothing in this section is served today; the org route below is still to be built in hanzoai/ai. It retires one way of reaching an org's engine that exists today: a provider row added by a SuperAdmin at POST /v1/ai/providers (the add-provider endpoint), which is global rather than the org's and whose URL nothing checks, and the Go control binary's --register-provider, which calls it.
/v1/ai/routes is the org's own model routes: a model id the org owns that resolves to an engine on the org's overlay (HIP-1173).
GET /v1/ai/routes — any member of the org.POST /v1/ai/routes — an admin of the caller's own org (HIP-0118); noSuperAdmin needed. HIP-1172's compute registers a workload's route through the same operation, acting for the activity's org.
DELETE /v1/ai/routes/local/{label}/{name} — an admin of the org, or compute.{
"model": "local/evo/zen6-flash",
"upstream": "zen6-flash",
"endpoint": "http://evo-1.acme.zt/v1",
"token": "<bearer the endpoint requires>",
"price": {"input": 0, "output": 0},
"fallback": "",
"bodies": false
}(org, model), and a request resolves a route only within its own validated org. Two orgs MAY hold the same model; neither ever reaches the other's.
model is local/<label>/<name>, each part lowercase [a-z0-9._-], at most 128 bytes in all. local/ is reserved for routes: no hosted id begins with it, and neither auto nor the Enso router (HIP-0510) ever resolves to a route, so only a request that names a route's id exactly reaches it. A second route with the same key is 409. compute names a workload's route local/<node>/<name>, where <name> is the last segment of the repository path folded to lowercase, so every name a repository may carry is a valid route name.
upstream is the id the endpoint's /models lists.token is write-only and sealed in the platform's KMS (HIP-1134), not theorg's, so no org admin can read it back; no read answers it.
price is USD per thousand input and output tokens, zero unless the adminsets it, never negative, and at most the deployment's ceiling from the price list (HIP-1222).
fallback is a hosted model id, or empty, which is the default.bodies records prompt and completion bodies; it is off by default.node and workload are set by compute alone; a route carrying them isdeleted with its workload. A route without them is probed every 60 s and deleted after three failures in a row.
http://<label>.<org>.zt[:<port>]/<path>: <label> one DNS label, <org> the caller's org name as IAM holds it, compared byte for byte with nothing folded, and no userinfo, query or fragment. Anything else is 422 and nothing is dialled.
never through system DNS and never to an address, so it cannot reach the cluster, a metadata service or the internet. That identity can dial every org's names, so the suffix check is the tenant boundary.
GET <endpoint>/models with the token MUST answer 200 within 10 s, as JSON, in at most 1 MiB, without a redirect, listing upstream in data[].id; otherwise 422 with the reason.
the connection.
A request naming a route's model is sent to the endpoint with upstream as its model.
Content-Type, Accept and Authorization: Bearer <token>, and no other header of the caller's: no key, cookie or identity header reaches the machine.
application/json or text/event-stream; any other type is 502. It relays the status, Content-Type and the body and no other header of the machine's, and it overwrites model in every JSON body and every event with the route's model, so a machine cannot claim to be another model.
closes the overlay connection. A request that names media by URL is refused with 400: send media inline.
5xx status), the request goes to fallback when the route names one, and otherwise answers 503 route_unavailable. Once a response has begun there is no fallback; a stream ends with an error event, and a non-streamed request answers 502.
balance gates a request naming the hosted model directly would, and when a gate refuses, the caller gets that refusal. A machine that always fails buys its org nothing it could not already buy.
its prompts go, and a silent hosted fallback would send them elsewhere.
to the validated org: provider node, model the route's, tokens as the engine reports them, cost price times tokens.
one above the bytes the gateway relayed for that side of the exchange is that byte count, since no token is shorter than a byte. The cost is computed with overflow checked; a cost that overflows or comes out negative is refused, the request is recorded at zero, and nothing is debited.
price is debited like any priced call.
fallback is recorded as the hosted model's, at the hosted price, with fallbackFrom naming the route. The route records nothing for it. One request, one record, one debit.
gateway relayed, marked partial, and billed no more than that.
gen_ai span a hosted call emits, with gen_ai.hanzo.node_id and gen_ai.hanzo.workload_id beside the org the server stamps. The generation read (HIP-1240) shows them as node and workload. No body is recorded unless the route's bodies is on.
local/evo/zen6-flash to http://evo-1.acme.zt/v1. The same call with .other.zt, an IP, localhost, a public https URL or a URL with userinfo answers 422 and dials nothing. A member who is not an admin gets 403.
local/evo/zen6-flash gets its own route; a request from orgA naming that id reaches org A's route, and one from an org with no such route gets the unknown-model answer.
token, and no org KMS path holds it.fallback set and the endpoint unreachable, a request is served by the hosted model and leaves exactly one usage record, the hosted one, with fallbackFrom. Without fallback it answers 503 and debits nothing. An org whose plan or balance refuses the hosted model gets that refusal, not an answer.
and workload attributes, and no body. At a nonzero price, an endpoint reporting 2^62 completion tokens, or −1, is billed for at most the bytes relayed and never credited.
x-api-key, cookies and identity headers never reach the endpoint, and a Set-Cookie from the endpoint never reaches the caller.
text/html yields 502; one answering "model": "zen6" reaches the caller as "model": "local/evo/zen6-flash".
auto, an Enso id or a hosted id never resolves to a route.The module registers one greedy /v1/* endpoint and the manifest hands it the /v1 remainder (manifest/apps.go:444), which is how a dozen satellite roots — router, memory, rag and their compatibility twins — came to answer beside the wire under names nobody would call a product. The document now comes from the module's own router (apps/ai/ai.go:75-90), so the surface is honest about what it is; this HIP settles which of those roots are the capability's and which are gone.
The key words MUST, MUST NOT, SHOULD, SHOULD NOT and MAY are to be interpreted as in RFC 2119.
The vendor-compatible wire — /v1/chat/completions, /v1/completions, /v1/embeddings, /v1/messages, /v1/models, /v1/images/, /v1/audio/, /v1/videos/*, /v1/responses, /v1/rerank — is fixed by the protocol every vendor SDK hard-codes, is exempt by HIP-0139 §3.2, and belongs to ai and to no other capability. It MUST NOT move.
Which model a request names is HIP-0511's map: Zen ids generate, auto and the enso* ids are resolved by the Enso router (HIP-0510) before provider and billing resolution, and decisions go to Kai at /v1/decisions (HIP-1332), not to this wire.
Six satellite roots fold under the capability's own name, each an HTTP binding of the module's handlers with no store of its own in cloud: the routing policy/stats/rewards surface (/v1/ai/router), the org's routing settings (/v1/ai/org/settings), fine-tuning jobs — the module's TrainJob broker — (/v1/ai/finetune), per-user memories (/v1/ai/memory), the RAG surface (/v1/ai/rag: embed, query, delete, context, and ingest — /v1/docs/ingest writes the same index the query reads, so it folds to /v1/ai/rag/ingest rather than minting a second root), the per-request routing reward (/v1/ai/feedback), and the public geo aggregate of request traffic (/v1/ai/traffic).
Six roots are deleted, not moved: /v1/documents and /v1/query and /v1/query_multiple are compatibility spellings of routes that already exist under rag; /v1/dev-bridge is desktop tooling that spawns a local child process; /v1/install-patch is a device-management remnant; /v1/wecom-bot is a bot channel that belongs to bot if it is ever productized. There is no alias for any of them. Today's router still serves each satellite at its old root out of the /v1 remainder; every such pair is a line in hanzoai/cloud openapi/misfiled.txt until the fold lands. The route moves land in the hanzoai/ai module, where the router lives (routers/wired_gen.go).
The document is the module's own, read out of the pinned module at describe time (apps/ai/ai.go:39-71); operations answer the module's envelope, whose data stays untyped here because per-operation bodies are typing work in the module that owns the handlers.
The capability owns no store in cloud; all state lives behind the hanzoai/ai module. A request's tenant is the validated principal's org, and every debit and count is keyed by that namespace and never by a body field. Metering is per served call, self-reported by the module's one usage hook: a priced call debits its dollar cost over the internal plane to the commerce ledger, with the idempotency ref minted server-side per debit so a caller-supplied id can never dedupe a second answer into a first (apps/ai/ai.go:195-215); a free call counts once against its subject's plan allowance, and both halves always run (apps/ai/ai.go:229-256). The plugin declares Price: cloud.Metered (plugin/ai/main.go:43).
It publishes no events on the tenant bus, so a customer's webhooks receive nothing from this capability. It emits the gen_ai span family for every completion, through the host's one tracer provider — adopted at mount so the module cannot fork a second, dark provider (apps/ai/ai.go:271-287).
ga, and the row is marked Vital (manifest/apps.go:444).
The capability embeds github.com/hanzoai/ai v1.833.106 (Apache-2.0), which is the whole implementation — router, wire, memory, rag, finetune. The OpenAI and Anthropic wire shapes are implemented formats, not embedded code. Nothing else here derives from an OSS upstream.
The alternative for the satellites was to keep them where SDK compatibility put them and describe the exceptions. But the exemption in HIP-0139 §3.2 is exactly as wide as the protocol: the wire paths are hard-coded in vendor SDKs and the satellites are not — LangChain-era twins and device-management remnants have no SDK pinning them, so they take the ordinary rule. Folding rather than splitting follows from the store: there is none in cloud, so §7.1's default applies to every pair.
The wrong implementation here is free inference and mis-billed inference. The debit path is the guard on both: the charge is keyed to the validated namespace, the ref is server-minted (a client-controlled ref let one pinned owner/name pair dedupe every later completion into the first one's entry — apps/ai/ai.go:184-193), and the free-call count runs even when the debit fails, so neither leg's failure opens the other. Deleting the compatibility roots is also a security act: each was an unowned endpoint into the same index the canonical routes guard.
Released under CC0 1.0 Universal Public Domain Dedication.