How to choose an LLM gateway: smaapi (SMA), LiteLLM, Portkey, Kong, Higress
Mainstream LLM gateways position themselves very differently: LiteLLM is open-source and self-operated, Portkey is managed with strong observability, Kong and Higress attach AI capabilities to existing API-gateway infrastructure, and smaapi (SMA) focuses on enterprise governance with first-class Chinese model access. Match the scenario first, then verify against six criteria.
Positioning at a glance (per each project's official docs, as of 2026-06)
| Project | Form | Positioning & strengths | Typical fit |
|---|---|---|---|
| smaapi (SMA) | Managed / self-hosted | Enterprise AI gateway: smart routing + cost and permission governance + full audit; Chinese providers first-class | Enterprises needing organization-level governance and compliant Chinese-model access |
| LiteLLM | Open source (Python) | Proxy + SDK, broad provider coverage, active community | Engineering teams with ops capacity who want freedom |
| Portkey | Managed-first | AI gateway with observability and prompt management in one | Teams comfortable with overseas SaaS who value observability |
| Kong AI Gateway | Open source + commercial | AI plugins on the Kong API gateway, reusing existing traffic governance | Organizations already deep on Kong |
| Higress | Open source (Alibaba) | Cloud-native gateway with an AI plugin ecosystem, Kubernetes-native | Alibaba Cloud / cloud-native stacks |
Sources for each project's positioning (verified 2026-06): LiteLLM docs · Portkey docs · Kong docs · Higress. smaapi Gateway capabilities are described on this site and governed by contract terms.
Positioning does not have to be guessed at — each project states it. The opening line of the LiteLLM documentation reads:
"Call 100+ LLMs using the OpenAI Input/Output Format"
Source: LiteLLM documentation (verified 2026-07)
That sentence is itself the boundary: it addresses protocol and access — one OpenAI-shaped interface across many providers, which it does well. It makes no claim to solve budget ownership, team-level authorization, or audit trails, because those are not what it is for. Most selection disputes are not about which project is better; they are about matching a tool's positioning to the problem at hand — access-layer problems want an access layer, governance problems want a governance layer.
Which six criteria should selection turn on?
- Protocol compatibility: completeness of OpenAI compatibility (streaming, function calling, multimodal); existing SDKs unmodified.
- Routing capability: configurable, calibratable policies; failover automatic and transparent.
- Governance granularity: budgets, quotas, permissions per team / application / key — not one global switch.
- Audit completeness: full-chain logs that support compliance review and cost reconciliation.
- Honesty of model coverage: claimed support and actual connectivity are different things — demand verifiable live data.
- Data boundary: where prompts, keys, and logs live; whether self-hosting is offered.
How should different scenarios choose?
Chinese enterprise, mixed Chinese + international models, compliance requirements
Weigh criteria 5 and 6 first — first-party Chinese-provider support and data boundary. This is where most Western projects are weakest and where SMA concentrates its design.
Engineering team, mostly international models, strong self-ops appetite
LiteLLM is the fastest start; revisit once team size and governance requirements grow.
Mature API gateway infrastructure already in place
Check whether Kong / Higress AI plugins can reuse what you have before introducing a dedicated LLM gateway — avoid building twice.
Frequently asked questions
Open-source self-hosted or managed gateway — how do I decide?
Two questions: operational capacity and compliance posture. With a dedicated platform team and a desire for full infrastructure control, open-source self-hosting (LiteLLM, Higress) maximizes freedom. If you would rather outsource the gateway itself — upgrades, incidents, on-call — a managed offering pays off. If data boundaries are a hard requirement, prefer vendors that also ship a self-hosted edition, so the deployment form can evolve with your compliance needs.
We need Chinese model providers as first-class citizens. Does that change the choice?
Significantly. Most Western projects support Chinese models through community adapters, and protocol details — streaming, function calling, billing semantics — vary in quality. Check three things: whether Chinese providers are first-party supported rather than plugins; whether connectivity is backed by verifiable live status data; and whether compliance and registration contexts are handled as product features, not afterthoughts.
References
- OpenAI API reference — the compatibility target for request/response semantics
About the name: smaapi (the SMA gateway) is the enterprise AI gateway built by Slime Mould Tech — SMA stands for Slime Mould Architecture. smaapi is unrelated to the simple moving average indicator in finance, to the solar inverter vendor SMA Solar Technology AG, or to the SMA coaxial connector standard that share the acronym.