Operate Your AI Capacity

As coding agents move from occasional assistants to persistent infrastructure, the way you pay for inference has to change with them.

AI coding tools look inexpensive at first because usage starts small: a few developers, a few requests, a bill that rounds to nothing. Then the operating model changes. Agents begin running for hours instead of seconds. They read entire repositories, build dependency maps, generate and revise large amounts of code, run the test suite, diagnose the failures, review their own changes, and do it across several projects at once. One human request can trigger hundreds of model actions.

At that point AI is no longer a productivity feature bolted onto an editor. It is an infrastructure workload. Yet most organizations still buy it the way they bought the first small experiment, through the equivalent of an uncapped, usage-based utility bill.

The first generation of enterprise AI was funded with token budgets. The next is funded with capacity plans.

Why the consumption model breaks

None of this makes token pricing a mistake. Hosted APIs are the right tool for a huge range of work. They are available instantly, they require no infrastructure, they give you frontier models and elastic scale, and they let you commit to nothing while you learn what works. For spiky, exploratory, low-volume usage, that flexibility is worth paying for.

What changes is the unit of consumption. A traditional assistant is invoked by a human, occasionally. A coding agent runs on its own, continuously, and a single seat can drive an agent loop that consumes far more than the person sitting behind it. Once that happens, token spend stops correlating with head count, and "how many seats do we have" stops predicting the bill.

The more useful question is not how many tokens your developers will consume. It is how much inference capacity you want available.

Location is not the point. Ownership is.

The alternative to renting tokens is to run a capable open model as a shared internal service, behind a stable, OpenAI-compatible endpoint that your existing coding tools already know how to call. People hear "local model" and picture a server humming under someone's desk. That is the wrong image.

The distinction is not physical location. It is operational ownership.

A model you operate can live in your own cloud, in a dedicated virtual private cloud, on rented GPUs, or through a managed dedicated-inference provider. What matters is that you decide which model runs, where the data flows, when capacity is available, how much of it exists, how requests are routed, and whether any external provider is allowed to see the work. The coding tools on the other end do not need to know or care where the model runs. They talk to an endpoint. You operate what is behind it.

From an uncapped meter to a capacity envelope

The economics are not complicated once you make them explicit. The problem is that they are almost never made explicit.

Take a representative request of 8,000 input tokens and 1,500 output tokens. On a frontier coding model at roughly $5 per million input tokens and $30 per million output tokens, which is about where today's premium tier sits, that request costs about $0.085. A dedicated H200, rented at roughly $4.39 per hour, costs about $105.36 per day whether you use it or not. Divide one into the other and the break-even is around 1,240 of those requests per day. If the hosted path qualifies for batch-style pricing, the crossover moves to roughly 2,480 per day.

The crossover tracks the tier you actually need, not a single magic number. Run a cheaper mid-tier model instead, nearer $3 and $15 per million, and the per-request cost roughly halves, which pushes break-even up to around 2,300 requests per day. These are illustrative public rates, not a quote, but the shape of the decision holds regardless of the exact figures.

The number that actually governs the decision is utilization. A rented GPU is only cheaper when you keep it busy, and that single fact sorts every workload cleanly. Spiky, interactive, low-volume traffic belongs on hosted APIs, because you cannot keep a GPU busy with it and you would be paying for idle time. Batch and steady-state, high-volume traffic belongs on rented hardware, spun up on demand and torn down when the job is done. That is where operating your own capacity wins, often by a wide margin.

Reframing spend as capacity also hands you a set of levers a token meter never could. You get to set availability: run continuously, run only during business hours, or run in scheduled windows so that off-hours and weekend spend fall to zero on a plan you chose. You get to set spend directly: a hard daily or monthly ceiling, for one deployment or for the whole account, that either warns you, stops the work, or refuses new deployments when it is reached. And you get to set concurrency: a bounded cap on in-flight requests with a short wait queue, so a burst is shaped and throttled instead of turning into a surprise on the invoice.

The operating conversation shifts from "how many tokens might our developers consume this month" to "how much capacity do we want available, and what is the ceiling."

Hybrid by design, not by ideology

The strongest version of this is almost never all-or-nothing. It is a division of labor.

Use a capable open model for the majority of routine engineering work: exploring a repository, implementing well-specified changes, generating tests, writing documentation, refactoring, assisting with migrations, and the continuous background analysis that would be ruinous to meter one token at a time. Reserve a frontier commercial model for the work where its additional capability genuinely changes the outcome: hard architecture, ambiguous requirements, difficult debugging, high-risk code review, and the escalations where a routine attempt has already failed.

In that arrangement the frontier model is an escalation target and a reviewer, not the default path for every keystroke. You keep access to frontier intelligence for the problems that need it, without making every repository search and test-generation pass a frontier-priced action.

Where the economics work, and where they do not

I want to be blunt about the limits, because the failure mode of this argument is a team that rents a GPU, runs one job on it, and then pays for twenty idle hours a day.

Operating your own capacity is attractive when demand is steady, utilization is reasonably high, the workload fits a capable open model, you can tolerate a defined capacity envelope, data control matters, and predictable spend matters. It is not attractive when usage is sporadic or extremely bursty, when the workload needs frontier performance on every request, when the GPUs would sit mostly idle, when cold-start latency is unacceptable, or when the open model simply cannot do the work reliably.

A rented GPU is not automatically cheaper than an API. It becomes compelling when you keep it meaningfully utilized. So the honest promise is not universal savings. It is a knowable cost ceiling, a measurable capacity envelope, more operating control, better vendor optionality, and real savings at sustained utilization.

What this looks like with OpenLease

We built OpenLease as the control plane that turns operated capacity into something you can actually run. You describe the service you want; it provisions the GPU, deploys the model, and runs it under a plan.

It will provision rented GPUs and deploy any vLLM-servable open model, including large models that need to be sharded across several GPUs, with no catalog entry required. It exposes an OpenAI-compatible endpoint and routes to it by model name, so your existing coding tools connect without changes. It meters cost and token usage, and it reports the crossover metric, the dollar cost per million tokens delivered, so the self-host-versus-API decision is a number you can read rather than a guess you argue about. It fans a batch of thousands of prompts out over a deployment, which is the cheap way to run a large document job on hardware you keep busy.

And it runs that capacity under exactly the levers described above. Operating schedules bring a deployment up during business hours and shut it down overnight, on a plan. Spend ceilings cap cost over a daily or monthly window, per deployment or account-wide, and can warn, stop the work, or block new deployments when the limit is reached. Concurrency limits cap in-flight requests with a bounded wait queue. And it runs a model across several load-balanced replicas, autoscaling that pool to the request rate between a floor and a ceiling you set, so capacity follows demand. OpenLease is Apache-2.0, and it runs in your environment.

That control plane is deliberately deterministic software, not a model making operational decisions, and I think the reason that line matters is worth its own argument: Use Models to Reason. Use Software to Operate.

Some of the picture is still ahead of us, and it is worth naming plainly rather than implying it ships: an optional, tightly scoped burst to a frontier API when the local pool is saturated, and per-team and per-project quotas tied to identity. What is in your hands today is enough to operate a real capacity plan.

The one-line version: OpenLease turns open models and rented GPUs into governed, operated AI capacity.

The shift

The gains from operating your own capacity come down to four things. Predictability, because a capacity plan forecasts far better than an unconstrained token curve. Control, because you decide availability, spend, and concurrency instead of discovering them on the invoice. Optionality, because open weights and provider-neutral deployment reduce your dependence on any single vendor's roadmap and pricing. And leverage, because one operated deployment can serve many tools, agents, and teams at once.

This is not a proposal to replace every frontier model. Frontier models will stay essential, and the best architecture keeps them one escalation away. It is a proposal to stop routing every repository search, every test generation, every refactor, and every autonomous loop through the most expensive and least predictable way to buy inference.

The question is quietly changing. It used to be "which model should our developers use." It is becoming "how much AI capacity should our organization operate."