Running open models instead of paying per token
For narrow, high-volume tasks, a self-hosted open model is often cheaper and more predictable than an API call.
Per-token API pricing is the right default for most workloads — low volume, varied tasks, and no operational burden. But for a narrow, high-volume, well-defined task — classification, extraction, a fixed-format summary — the economics flip once volume passes a threshold, and a smaller open-weight model running on infrastructure you control gets both cheaper and more predictable.
The predictability matters as much as the cost. A self-hosted model doesn't change behaviour when a provider updates it. Latency is bounded by your own infrastructure, not a shared API's load. And there's no per-request cost that scales awkwardly with a spike in usage.
This isn't an argument against API-based models generally — for anything requiring broad reasoning or infrequent, varied requests, the API is still the right call, and building your own serving infrastructure for a low-volume task is wasted engineering effort in the other direction. The decision is a straightforward volume-and-scope calculation, not an ideological one, and it's worth actually running the numbers before defaulting to whichever approach is more familiar.
This is squarely MLOps territory: model serving, versioning, and monitoring for a self-hosted model is real infrastructure work, not a config change.