Prompt Tokens vs Completion Tokens: Why the Split Matters

By Darus S Alfatah · 2025-09-24 · Edukasi · 6 menit

Prompt Tokens vs Completion Tokens: Why the Split Matters

Author note: We always leave room for output. A prompt that fills the whole context truncates the answer.

Focus of this article

This article focuses on the difference between prompt tokens (input) and completion tokens (output), and why they are billed differently.

Read also: What Is a Token? The Unit Behind Every AI Bill · Setting max_tokens: A Practical Guide

Two Buckets, Two Prices

Every request has two parts: the prompt (system instructions, context, question) and the completion (the answer). Providers price them separately, and completion tokens usually cost two to four times more. Ignoring this split leads to wrong optimisation decisions.

Teams often try to shorten answers when the real cost sits in the prompt.

Why Prompts Dominate

A system prompt of 1,500 tokens plus a conversation history of a few turns can easily exceed 3,000 tokens before the user even asks a question. If the answer is 300 tokens, prompt tokens are more than ninety percent of the bill.

That ratio is typical for support bots and internal assistants, which is why prompt hygiene matters more than output caps.

Caching Changes the Maths

When a provider supports prompt caching, the unchanged prefix is billed at a lower rate. Structure prompts so static rules come first and dynamic content comes last, and the savings apply automatically on repeated requests.

Cache-friendly prompts are a design decision, not a tuning trick.

Balancing Quality

Cutting prompts too aggressively removes constraints and examples, which raises errors and retries. Review what each prompt section actually changes in behaviour; sections that never affect output are candidates for removal.

Cutting output too aggressively truncates answers or breaks JSON. Set limits close to the length you actually publish.

A Practical Audit

Take ten typical requests and print the token split for each. Identify the largest prompt contributors: static instructions, retrieved context, or history. Then measure again after trimming one of them.

Most teams find history is the easiest large win, because rolling summaries preserve intent at a fraction of the size.

Frequently Asked Questions

Are completion tokens always more expensive? Usually, but check your provider's table.

Does streaming change the split? No, but stopping early reduces completion tokens.

Do cached tokens count as prompt tokens? They are reported separately at a lower price.

How do I reduce history cost? Summarise older turns and drop irrelevant messages.

Is a bigger prompt always better? Only when it adds relevant, necessary constraints.

Conclusion

Understand the split before optimising. Trim prompts with caching in mind, keep enough constraints for accuracy, and cap output to the length you actually need.

Worked Example

Suppose a feature sends 1,600 prompt tokens and receives 250 completion tokens. If prompt costs 200 per million and completion costs 600 per million, the cost is dominated by the prompt despite the higher completion rate. Halving the prompt saves more than halving the answer.

Run this calculation on your own traffic before deciding what to optimise. Intuition about which half matters is usually wrong.

Design Implications

Move static content to cached prefixes, retrieve only relevant context, and keep examples minimal but sufficient. Each of these reduces the expensive side of the equation without touching answer quality.

Finally, recheck the split after every prompt change. A single added instruction block can shift the balance and quietly double the bill.

Track the ratio monthly so drift is visible early.

An example of asymmetric cost: A hundred-token prompt and a hundred-token answer do not cost the same. The output side is usually far pricier.

Frequently Asked Questions

What is the core topic of this article?

This article explains the fundamentals of AI APIs and gateways in plain language for developers, with practical examples.

Do I need a technical background?

No. The material builds up from the basics, so beginners can follow while experienced developers can jump to the relevant section.

How do I get started with an AI API after reading this?

Create a BankAI account, get an API key, then change your Base URL to BankAI’s OpenAI-compatible endpoint.

Sources & References

About the author

Darus S Alfatah — Founder & Writer

Darus S Alfatah is the founder of BankAI. He builds and operates the LLM API gateway that serves Indonesian developers, and writes about LLM integration, gateway architecture, and using AI practically and cost-effectively. Every guide is tested in a real environment — including code, configuration, and token calculations — before it is published, and updated when provider documentation or pricing changes.

Editorial policy

Related articles

🚀 Ready to Use the All-in-One API Key?

Sign up, top up with QRIS, and start building with ChatGPT, Claude, DeepSeek and every other model through one key.

🔑 SIGN UP NOW · Pricing