Is a self-hosted LLM relay enough for an enterprise?
OneAPI-style self-hosted relays are a sound choice for individuals and small teams aggregating multiple model providers. Enterprises break on governance — permissions, audit, SLA, and compliance responsibility. A relay is a personal tool; what an enterprise needs is an AI gateway with a governance layer.
What do self-hosted relays get right?
Credit first: OneAPI-style open-source projects put many providers behind one OpenAI-compatible endpoint, normalize the protocol, and kill repetitive per-provider integration work. For individual developers and small teams they are simple to deploy, have a mature ecosystem, and are genuinely sufficient. At that scale, introducing enterprise infrastructure would be overkill.
Where do enterprises break? Five gaps
- No permission governance: authorization stops at the key level; no team / application / model-scope grants, revocation, or rotation.
- Audit that cannot reconcile: plain access logs cannot answer who used which model for what and when — no evidence chain for compliance review or cost allocation.
- Operations on you: no SLA, no incident commitment; failover at 3 a.m. is a human.
- Unclear compliance boundary: data flows through a self-maintained node to many providers; boundary and responsible party are hard to state, and due-diligence reviews stall.
- Weak key custody: provider keys typically live in the relay config, without central custody, rotation, or least-privilege scoping.
Dimension by dimension
| Dimension | Self-hosted relay (OneAPI-style) | Enterprise gateway (smaapi) |
|---|---|---|
| Positioning | Personal / small-team aggregator | Enterprise infrastructure |
| Permissions | Key-level, coarse | Team / app / key grants, revocable and rotatable |
| Audit | Access logs | Full-chain audit, reconcilable |
| Routing & failures | Static config, manual handling | Policy-driven smart routing, automatic failover |
| Cost governance | Bills after the fact | Budgets and quotas enforced up front, one metering pipeline |
| Operations & SLA | You own it | Platform commitment and incident handling |
| Compliance boundary | Unclear | Explicit data boundary, self-hosted option |
The relay column reflects the public docs of OneAPI-style open-source projects (verified 2026-06), not any specific deployment. smaapi Gateway capabilities are described on this site and governed by contract terms.
The conclusion is not "relays are bad" — it is a scope boundary. Relays optimize integration efficiency; gateways optimize organizational governance. Once usage, headcount, or compliance crosses a threshold, governance stops being optional.
Frequently asked questions
We already run OneAPI — how expensive is migrating to an enterprise gateway?
Both expose an OpenAI-compatible protocol, so the application-side change is usually just base_url and credentials; business code stays untouched. The real work is governance setup: consolidating scattered provider keys, assigning budgets and permissions per team and application. That work is a debt the organization owes anyway — the gateway just makes it explicit.
Can a self-hosted relay and an enterprise gateway coexist?
Yes, and it is common. Personal experiments and throwaway scripts keep using the relay; production traffic that touches enterprise data or budgets goes through the gateway, where metering, audit, and permissions live. As governance requirements grow, production naturally converges on the gateway.
What signals say it is time to upgrade?
Any of four: multiple teams or applications start sharing a model budget; security or compliance asks for call-level audit; provider keys appear in more than one codebase; handling model failures requires code changes. Two or more, and governance debt is already compounding.