Migrating From OpenAI to a Gateway: Change Two Lines

By Darus S Alfatah · 2025-08-06 · Tutorial · 8 menit

Migrating From OpenAI to a Gateway: Change Two Lines

Author note: Migration is usually just changing base_url and api_key. Test one endpoint before moving everything.

Why Migration Is Small

If your application uses the official OpenAI SDK, migrating to a compatible gateway usually means changing the base URL and the API key. Streaming, function calling, and JSON mode keep working because the request and response shapes are the same.

The real work is not code; it is verification.

Dual-Run Before Switching

Do not shut down the old provider until you are confident. Run both paths side by side: send the same request to each, store results, and compare quality and latency for several days. For critical apps, keep a switch to revert within seconds.

const provider = process.env.AI_PROVIDER || 'gateway';
const client = provider === 'gateway'
  ? new OpenAI({ baseURL: 'https://bankai.web.id/api', apiKey: process.env.BANKAI_API_KEY })
  : new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

Metrics to Compare

  • Time to first token (p50 and p95).
  • Failure rate and error types.
  • Output quality on ten fixed examples.
  • Cost per request, including context tokens.

Often-Missed Details

Tokenizer differences can change effective prompt length. If you are close to the context limit, re-test after migration. Check parameter support as well: some providers cap temperature differently or handle response formats slightly differently.

Rollback Plan

Keep the old configuration in a separate environment variable, add a provider flag, and test the rollback once in staging so the team knows the steps. For a large change, migrate gradually: ten percent of traffic first, watch errors and cost, then increase.

Checklist

  • Base URL and key moved to environment variables.
  • Test suite run against both providers.
  • Cost per feature recorded before and after.
  • Rollback exercised at least once.
  • Monitoring alerts updated for the new endpoint.

Beyond Cost

Migration also unlocks model choice: route simple tasks to cheaper models and keep expensive ones for hard cases. That flexibility usually matters more than the per-token difference.

Conclusion

Compatibility makes migration cheap, but verification makes it safe. Compare on your own data, keep a rollback path, and switch traffic gradually.

What Usually Breaks in Practice

Three things cause most migration pain. First, silently different defaults: a parameter that was optional upstream may be required here, or a value that was clamped is now rejected. Second, error handling: code that treated every non-200 response as fatal now crashes on a rate limit that should simply be retried. Third, token accounting: if your budget dashboards assumed one price, reports will look wrong until you update the rates.

Handle all three by writing a thin adapter with explicit normalisation. Validate inputs before sending, normalise errors into a small set of categories, and compute cost from the usage object returned by the response rather than from your own estimates. That adapter becomes the single place to change if you switch again.

Finally, tell users nothing changed. Run a canary period where five percent of traffic goes to the new provider and compare complaint rates. Users notice quality shifts long before dashboards do.

One More Check

Before switching the last ten percent of traffic, verify that your monitoring still fires. Alert rules often reference the old endpoint name, so failures arrive silently. Update the dashboards and run a deliberate failure test to confirm someone is notified.

Example: test one endpoint: First move one small feature to the gateway, compare results and latency for a day, then move the rest if it is equal or better.

Frequently Asked Questions

Can beginners follow this tutorial?

Yes. The steps are explained in order with code examples, suitable for beginners and experienced developers alike.

Is BankAI’s endpoint compatible with the library I already use?

Yes, BankAI is 100% OpenAI-compatible, so libraries such as the OpenAI SDK and LangChain work without code changes beyond the Base URL and API key.

Where do I get an API key to follow this tutorial?

You get the API key (and Secret Token) after registering and completing payment in the BankAI dashboard.

Sources & References

About the author

Darus S Alfatah — Founder & Writer

Darus S Alfatah is the founder of BankAI. He builds and operates the LLM API gateway that serves Indonesian developers, and writes about LLM integration, gateway architecture, and using AI practically and cost-effectively. Every guide is tested in a real environment — including code, configuration, and token calculations — before it is published, and updated when provider documentation or pricing changes.

Editorial policy

Related articles

🚀 Ready to Use the All-in-One API Key?

Sign up, top up with QRIS, and start building with ChatGPT, Claude, DeepSeek and every other model through one key.

🔑 SIGN UP NOW · Pricing