AI gateway
An open-source LLM gateway that speaks OpenAI to every provider.
Point your existing OpenAI client at the gateway and keep your code. AcruxCore resolves the model to a provider — OpenAI, Anthropic, Gemini, or any OpenAI-compatible endpoint you register — calls it with your key, prices the result and records the request. Streaming included.
What you get
Compatibility, your own keys, guardrails and cost.
Drop-in compatible
POST /gateway/chat/completions speaks the OpenAI wire format, streaming included. Swap the base URL and your current client, retries, and code paths all stay put.
Bring your own keys
Store an OpenAI, Anthropic, Gemini or OpenAI-compatible credential — base URL included — encrypted at rest. We sit in front of your provider account and never take it over.
Virtual keys with guardrails
Issue scoped keys that never expose a provider credential. Restrict one to named models or providers, cap its requests and tokens per minute, and cap spend by day, week, month or total.
Cost on every response
x-gateway-* headers return the request id, the provider that served it, the priced cost, and whether it was a cache hit. Accounting is not a batch job you run later.
Reliability
Automatic model fallbacks.
Each model you register carries an ordered list of backups. When a call cannot be served, the gateway walks that chain and answers from the first model that can — the request shape your app sends never changes.
| What went wrong | What the gateway does |
|---|---|
| Rate limit, 5xx or network timeout | Retries the same model first, then works down your chain until one answers. |
| Bad or revoked credential (401, 403) | Skips the retry — a wrong key will stay wrong — and moves straight to the next model in the chain. |
| Malformed request (400) | Comes back to you immediately. A request the provider rejected as invalid is not worth sending anywhere else. |
A fallback answer is priced, traced and reported like any other, with the provider that actually served it on the response headers. Sending a fallback list replaces the whole ordered chain, so send it as one set.
Cost control
Cache repeated requests.
Caching is opt-in per virtual key: give a key a TTL and identical requests inside that window are answered from your own team cache instead of the provider.
What counts as identical
A hash of the request parameters — same model, same messages, same settings. Only non-streaming calls that ask for temperature 0 are eligible, because anything else is a request to vary.
What you see
x-gateway-cache: hit or miss on every response, and $0 for a hit. The call is still traced and still written to the usage ledger, so a cached answer is never an invisible one.
What it measured
On one published run — 15 prompts, four repeats each, gpt-4o-mini, a five-minute window — the repeats came back from cache. Your own hit rate depends on how repetitive your traffic really is.
Caching is per team and per key, never shared between teams. It is a setting you turn on deliberately, not a default that quietly answers for a model.
In the dashboard
Everything without writing a line of code.
The API and SDKs cover the automated path. For the day-to-day, the dashboard does the same work in the browser.
- Register a public model name that maps to a credential and an upstream model.
- Test a registered model in one click before your app depends on it.
- Try any model in the Playground — ad-hoc messages, or a stored prompt by name.
- Read request logs with the exact prompt version that produced each call.
- Watch spend against a budget, per key or across the whole team.
- Rotate a provider credential without touching a line of application code.
Go deeper
Documentation for this piece.
The rest of the platform
It composes with the other five.
Each piece works on its own, and they get better together — a trace links back to a prompt version, a dataset is built from feedback.