Every team I've worked with hits the same argument about three months after shipping something on top of an LLM: "Should we still be paying per token, or should we just self-host?"
And it's always argued on vibes. Someone read a blog post. Someone else got burned by a GPU bill once. Nobody's actually done the math. So here's the math — or at least the shape of it.
The API wins right up until it doesn't
Paying an API has one killer advantage people forget to credit it for: you pay for exactly what you use, and nothing when you're idle. No GPU sitting warm at 3am doing nothing. No serving stack to babysit. If your volume is low or spiky, this is basically unbeatable, and honestly you should stop reading and just use the API.
Self-hosting flips that. You take on a mostly fixed cost — the GPU (rented or amortised) plus the very real human cost of running it — and in exchange, each extra token is almost free once you've paid for the box.
So the whole decision collapses to one question: at your volume, has that fixed cost dropped below what the meter would've charged you? That's the break-even. Everything else is just inputs to it — which our LLM Cost Calculator charts over 12 months if you'd rather not do the arithmetic by hand.
The six things that move the break-even
- Monthly token volume. The big one. API cost scales straight up with it; self-hosting barely moves. Low → API. High and steady → self-hosting.
- Input:output ratio. Output usually costs more on APIs, and it's what eats GPU time when you host. A summariser (huge input, tiny output) and a chatbot look nothing alike on the curve.
- Quality tier. Here's where people quietly cheat: "self-hosting is cheaper" usually assumes a 30B open model is good enough. If you actually need frontier quality, compare frontier-to-frontier — and you probably can't run frontier yourself.
- GPU cost + amortisation. Your fixed cost, per hour or spread over the hardware's life.
- Utilisation. A GPU you use 8% of the day is a space heater with a fan. This one number wrecks or saves the whole thing.
- Ops overhead. The line nobody puts in the spreadsheet: someone runs it, patches it, and gets paged at 2am. At small scale that alone can be bigger than the API bill — think 10–20 senior-engineer hours a month.
The costs both sides conveniently forget
Self-hosting quietly adds: fine-tuning experiments, storage and egress, eval infrastructure, and your engineers' time — which is not free, no matter how much they enjoy it. The API quietly adds: price changes and rate limits you don't control, and the awkward bit — your data (and your customers' data) leaving your walls.
The break-even nobody charts: compliance
There's a second crossover that has nothing to do with dollars. If you handle regulated or client-confidential data, self-hosting wins the moment that data would otherwise be shipped off to a third-party API — full stop, whatever the token math says. When "our data can't leave our infrastructure" is a hard rule, you've already made the decision; you're just backfilling the spreadsheet. (If that's you — say, a professional-services firm — here's the specific version of that problem: is it safe to use ChatGPT with client data.)
Just run your numbers
The model is simple. The arithmetic — 12 months, amortisation, utilisation — is annoying. So we built the boring part: the LLM Cost Calculator charts API vs self-hosting over a year, no signup. Trying to work out whether a specific AI feature even earns its keep? Start with the AI Feature ROI Calculator first.