TYPED DECISION LAYER · SELF-HOSTED · ~15ms ON CPU
A router that knows
when to say no.
mtlm-router turns natural-language requests into typed, calibrated routing decisions — on-prem, on CPU, in ~15 milliseconds. When a request doesn't fit a known route, it says escalate instead of guessing.
One call. Three typed answers.
A single request returns a validated route, a calibrated confidence, and a tool-fill proposal — or a clean escalate if it doesn't know.
$ curl -s localhost:8080/assess -d '{"req":"hi team, this is the third time my VPN drops today"}'
{
"route": "create_ticket",
"confidence": 0.87,
"noul": 0.91 // "novel / unhandled / loss" — abstention dial,
"candidate": "open_ticket",
"score": 0.83,
"action": "tool_fill",
"fill": { "subject": "VPN drops repeatedly", "priority": "high" }
}It doesn't chat
It's a router, not a chatbot. It emits machine-readable decisions your agent, helpdesk, or dispatcher can act on directly.
It doesn't guess
Out-of-scope requests hit the calibrated escalate route. Small models shouldn't pretend to know things — this one doesn't.
It doesn't argue
No temperature, no sampling, no prompt drift. Deterministic softmax over your routes — the same request gets the same answer, every time.
Mixture of experts — one trunk, many brains.
The frozen 7M trunk is the shared brain. Domain knowledge lives in ~7KB heads you can ship, swap, and audit like config files. A gate head picks the expert; the expert picks the route.
┌─────────────────────────────────────┐
request ──▶ │ mtlm-router trunk — frozen 7M model │
tokens └──────────────┬──────────────────────┘
prefill hidden state (1 × d_model)
┌────────────┐ ┌─────────────┐
│ gate.head │ │ expert.head │ ← per-domain, ~7KB
│ pick a domain │ │ pick a route │
└────┬───────┘ └─────┬───────┘
it_helpdesk ───┘ reset_password ───┘ + calibrated confShip brains, not models
A customer deployment = the frozen trunk + a handful of kilobyte head files. Onboard a new domain by training a head on ~1k labeled rows — hours, not weeks.
95.4% gate accuracy
The domain gate (tools / it-helpdesk / fleet / rpg / chat) held out at 95.35% accuracy, ECE 0.035. Wrong gate → generic expert → escalate. Never misroute.
Tenants = swap files
ANVIL_TENANTS="gate:moe_gate.head,helpdesk:helpdesk.head" — per-tenant route vocabularies on one serving instance.
Numbers — held-out, calibrated, honest.
| Metric | Head | Result |
|---|---|---|
| Route accuracy | router head, 706 disjoint requests | 97.6% |
| NOUL calibration | abstention dial, temperature-fitted | 98.9% |
| Score accuracy | tool-fill candidate scoring | 97.9% |
| Action agreement | vs composite policy | 100% |
| Gate accuracy | moe_gate head, 172 disjoint requests, 5 domains | 95.4% · ECE 0.035 |
Honest limits.
It's a router, not a reasoner
It can't summarize, explain, or compose. If you need text generation, route to a bigger model — that's exactly what escalate is for.
Thin lanes exist
The rpg domain gate trained on 64 rows. The manifest ships per-domain row counts and eval — you see the weak lanes before production does.
Calibration ages
Temperature was fit on today's request mix. Traffic drifts; refit from labeled production rows before trusting the confidences downstream.
Verify before authority
Run it as a shadow router first. Promote to authoritative only after real traffic proves the gate + head on your request mix.
One tarball. Runs anywhere.
The whole stack — engine, tokenizer, int8 model, heads, admin — ships as a ~7.5MB appliance. No pip install, no model registry, no GPU driver.
$ tar xzf mtlm-router-appliance.tar.gz && cd appliance
$ ./anvil-serve model.bin 8080 &
$ curl localhost:8080/assess -d '{"req":"reset my password"}'
{"route":"reset_password","confidence":0.99,"action":"tool_fill",...}Small enough to trust. Typed enough to use.
AGPL/MIT-clean pure-Machin source, full training and serving pipeline, and the honest-numbers discipline baked in. Read the product doc, audit the evals, ship your first head.