Migrating From OpenAI to a Gateway: Change Two Lines
By Darus S Alfatah · 2025-08-06 · Tutorial · 8 menit
Author note: Migration is usually just changing base_url and api_key. Test one endpoint before moving everything.
Why Migration Is Small
If your application uses the official OpenAI SDK, migrating to a compatible gateway usually means changing the base URL and the API key. Streaming, function calling, and JSON mode keep working because the request and response shapes are the same.
The real work is not code; it is verification.
Dual-Run Before Switching
Do not shut down the old provider until you are confident. Run both paths side by side: send the same request to each, store results, and compare quality and latency for several days. For critical apps, keep a switch to revert within seconds.
const provider = process.env.AI_PROVIDER || 'gateway';
const client = provider === 'gateway'
? new OpenAI({ baseURL: 'https://bankai.web.id/api', apiKey: process.env.BANKAI_API_KEY })
: new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
Metrics to Compare
- Time to first token (p50 and p95).
- Failure rate and error types.
- Output quality on ten fixed examples.
- Cost per request, including context tokens.
Often-Missed Details
Tokenizer differences can change effective prompt length. If you are close to the context limit, re-test after migration. Check parameter support as well: some providers cap temperature differently or handle response formats slightly differently.
Rollback Plan
Keep the old configuration in a separate environment variable, add a provider flag, and test the rollback once in staging so the team knows the steps. For a large change, migrate gradually: ten percent of traffic first, watch errors and cost, then increase.
Checklist
- Base URL and key moved to environment variables.
- Test suite run against both providers.
- Cost per feature recorded before and after.
- Rollback exercised at least once.
- Monitoring alerts updated for the new endpoint.
Beyond Cost
Migration also unlocks model choice: route simple tasks to cheaper models and keep expensive ones for hard cases. That flexibility usually matters more than the per-token difference.
Conclusion
Compatibility makes migration cheap, but verification makes it safe. Compare on your own data, keep a rollback path, and switch traffic gradually.
What Usually Breaks in Practice
Three things cause most migration pain. First, silently different defaults: a parameter that was optional upstream may be required here, or a value that was clamped is now rejected. Second, error handling: code that treated every non-200 response as fatal now crashes on a rate limit that should simply be retried. Third, token accounting: if your budget dashboards assumed one price, reports will look wrong until you update the rates.
Handle all three by writing a thin adapter with explicit normalisation. Validate inputs before sending, normalise errors into a small set of categories, and compute cost from the usage object returned by the response rather than from your own estimates. That adapter becomes the single place to change if you switch again.
Finally, tell users nothing changed. Run a canary period where five percent of traffic goes to the new provider and compare complaint rates. Users notice quality shifts long before dashboards do.
One More Check
Before switching the last ten percent of traffic, verify that your monitoring still fires. Alert rules often reference the old endpoint name, so failures arrive silently. Update the dashboards and run a deliberate failure test to confirm someone is notified.
Example: test one endpoint: First move one small feature to the gateway, compare results and latency for a day, then move the rest if it is equal or better.
Frequently Asked Questions
Can beginners follow this tutorial?
Yes. The steps are explained in order with code examples, suitable for beginners and experienced developers alike.
Is BankAI’s endpoint compatible with the library I already use?
Yes, BankAI is 100% OpenAI-compatible, so libraries such as the OpenAI SDK and LangChain work without code changes beyond the Base URL and API key.
Where do I get an API key to follow this tutorial?
You get the API key (and Secret Token) after registering and completing payment in the BankAI dashboard.