How to Use an OpenAI-Compatible API to Call Multiple Models

Almost every model provider now exposes an OpenAI-compatible API. That single fact is quietly the most useful thing in the current AI tooling ecosystem, because it means you can write your integration once and swap models, providers, and price points without rewriting your application.

Here's how it works, why it matters, and the specific gotchas that bite people.

Why the OpenAI-Compatible Standard Matters

The OpenAI API shape — a POST with a list of messages, returning a stream of tokens — became the de facto standard almost by accident. Providers realized that if they spoke the same language, they could inherit every existing SDK, library, and code sample in the world for free. So they did.

The consequence for you: the code that talks to one provider talks to all of them. Change the base URL and the API key, and the same function calls a different model.

This is what "no vendor lock-in" actually looks like in practice. You're not locked into a provider if switching is a two-line change.

The Core Pattern

The minimal integration looks like this in most languages:

base_url = "https://your-provider.example/v1"   # changes per provider
api_key  = "..."                                 # changes per provider
model    = "provider/model-name"                 # changes per model

response = chat_completion(
    model=model,
    messages=[{"role": "user", "content": "hello"}],
    base_url=base_url,
    api_key=api_key,
)

Three things change: the endpoint, the key, and the model name. Everything else — your prompt construction, your message history, your parsing, your streaming — stays identical.

The practical upshot is that you can route different requests to different models based on cost, speed, or capability, without maintaining separate code paths for each provider.

Setting It Up

Step 1: Use an SDK that respects the standard

Most popular SDKs let you point them at any OpenAI-compatible base URL. This is the key thing to verify when choosing a library: does it let me set a custom base URL? If it hardcodes a provider, it's less useful than it looks.

Step 2: Store provider config, not hardcode it

Keep a small config mapping friendly names to (base_url, key, model):

models:
  cheap:
    url: https://provider-a.example/v1
    key: ...
    model: provider-a/cheap-model
  strong:
    url: https://provider-b.example/v1
    key: ...
    model: provider-b/strong-model

Now routing is a matter of choosing which config entry to use — no code change per provider.

Step 3: Add a fallback layer

Once you have multiple providers, the obvious next step is resilience: if one provider errors or is slow, retry on another. A simple try/fallback on the config list gives you reliability that would otherwise require paying for it.

The Gotchas That Actually Bite

The /v1 suffix. Some providers want it, some don't, and some break in confusing ways if it's wrong. If you get a mysterious 404, check the base URL first.

Model names differ per provider. There's no consistent naming scheme. gpt-4o on one provider isn't gpt-4o on another. Keep model names in your config, never hardcoded in logic.

Not everything is actually compatible. The standard is a convention, not a contract. Providers support the chat completion endpoint well and vary everywhere else — embeddings, function calling, vision, and JSON mode are where divergence shows up. Test the specific features you need, don't assume.

Streaming format differences. Most support streaming, but the exact chunk shape can differ. If your UI depends on streaming, verify the token stream format for each provider you adopt.

Rate limits and billing differ wildly. The same call can be rate-limited, priced, and metered completely differently per provider. Build fallbacks and logging in from day one so you can see which provider is actually serving your traffic.

The Table

What Didn't Work

Assuming full compatibility. I assumed "OpenAI-compatible" meant "everything works." It doesn't — it means "chat completion works." Function calling and vision needed per-provider testing and, in one case, a workaround.

Hardcoding provider logic. My first version had if provider == "a" checks sprinkled through the code. Moving to a config-driven approach removed a whole class of bugs and made adding a provider a one-line change.

Ignoring the fallback layer until I needed it. The first provider outage forced a panic fix. A simple fallback would have made it a non-event. Build it before you need it.

Choosing providers by headline price. The cheapest provider had the worst latency and availability, and the "savings" evaporated in retries and timeouts. Test real behavior, not just the price sheet.

Verdict

Adopt the OpenAI-compatible abstraction as your default. Write one integration, config-driven with per-provider settings, add a fallback layer, and verify the specific features you use per provider.

It's the cheapest insurance against vendor lock-in available, and it turns "which model should I use" from an architectural decision into a config value.

FAQ

What does "OpenAI-compatible" actually guarantee? In practice, that the chat completion endpoint speaks the same request/response shape. Beyond that, support varies — test embeddings, function calling, vision, and streaming per provider.

Is this only useful for text models? Mostly text/chat, yes. Image, audio, and embedding APIs are less standardized, though several also follow OpenAI-style patterns. Verify per feature.

Do I need to use the OpenAI SDK? No. The point is the format, not the SDK. Any SDK that lets you set a custom base URL works; many providers also ship their own thin SDKs that speak the same shape.

How do I switch models at runtime? With a config-driven setup, changing models is changing which config entry you reference — base URL, key, and model name all come from config, not code.