For Llama 3.3 70B requests containing 1,000 input and 1,000 output tokens, start investigating self-hosting around 5.03 million requests per 30 days at the current catalog prices. That is 5.03 billion output tokens, with the same input volume. At that traffic level, DeepInfra's standard FP8 API bill equals the modeled $2,112.00 cost of 1 continuously allocated replica, each using 2 × H100 SXM 80 GB, including $0.00 of assumed additional costs.
This threshold gives you a financial reason to benchmark self-hosting. A deployment must also handle your arrival pattern, latency and quality requirements. Adding an illustrative $1,000 of operating costs raises the traffic threshold to 7.41 million requests under the same workload assumptions.
At one million such requests, the API bill is $420.00, compared with the allocation's $2,112.00 modeled budget. Use your own request lengths and sustained traffic to judge where you sit: the three Llama 3.3 workloads below have different thresholds, and the required output rate shows what your deployment would need to deliver. The calculation identifies when testing becomes financially interesting; a rental test establishes whether the candidate works for your service.
What to take away
- At 1,000 input and 1,000 output tokens per request, the current 30-day threshold is 5.03 million requests; adding $1,000 of operating costs raises it to 7.41 million.
- Match your request lengths to the Llama 3.3 workload: the 128-input/2,048-output and 2,048-input/128-output references have different cost and throughput implications.
- Treat the two-H100 FP8 reference as a candidate to benchmark with your traffic. Original BF16 weights and long contexts need a separate memory check.
Calculate the traffic volume that makes a deployment test worthwhile
The selected quote is Vast.ai, configuration Marketplace offer #33017576, in France, FR: 2 × H100 SXM 80 GB at a whole-instance $2.933333/hour. It is a listed hourly on-demand quote with a verified GPU count and no minimum monthly commitment. Stock and final checkout terms still need confirmation.
We use the two-H100 configuration because it has the three sourced Llama 3.3 FP8 throughput references examined below. This calculation evaluates that candidate. Finding the most economical deployment for your application requires evaluating other configurations as well.
The API comparison is DeepInfra, endpoint meta-llama/Llama-3.3-70B-Instruct-Turbo, documented as FP8. Its standard uncached rates are $0.1 per million input tokens and $0.32 per million output tokens.
For 1,000 input and 1,000 output tokens, input/output ratio R is 1:
Rental period budget:
$2.933333/hour × 720 hours × 1 replica(s) + $0.00
= $2,112.00
API cost per million output tokens, including associated input:
$0.32 + 1 × $0.1 = $0.42
Output-token break-even:
$2,112.00 ÷ $0.42 × 1,000,000
≈ 5,028,571,429 output tokens
≈ 5.03 million requests at 1,000 output tokens eachThe associated API cost per request is $0.00042. Here is what that means at concrete traffic volumes:
| Monthly requests | Output tokens, billions | API bill, USD | Rental period budget, USD |
|---|---|---|---|
| 100,000 | 0.10 | 42.00 | 2,112.00 |
| 1,000,000 | 1.00 | 420.00 | 2,112.00 |
| 5,000,000 | 5.00 | 2,100.00 | 2,112.00 |
| 10,000,000 | 10.00 | 4,200.00 | 2,112.00 |
At one million such requests, the API is $1,692.00 cheaper than the allocation. At five million, the API is still $12.00 cheaper. Include the operating costs of the service you need before treating a difference as a saving.
Source checks: API price 2026-09-29 08:17 EDT; rental quote 2026-09-29 06:01 EDT. Computed 2026-09-29 08:47 EDT. Figures are rounded for display; arithmetic uses the stored precision. The base allocation price covers listed compute. Storage, traffic and operations are additional unless included in the stated extra-cost assumption.
Adjust the threshold to your Llama 3.3 request lengths
NVIDIA's pinned reference supplies three useful comparisons on the same two-H100 SXM FP8 configuration. The request lengths belong to each result: the output rate for a short prompt with a long answer cannot be assigned to a longer prompt with a short answer.
| Input → output tokens/request | API provider | API USD/million output including input | Output billions at break-even | Requests, millions | Published output tokens/s | Required average / published reference |
|---|---|---|---|---|---|---|
| 128 → 2048 | DeepInfra | 0.32625 | 6.47 | 3.16 | 5,892.94 | 42.38% |
| 1000 → 1000 | DeepInfra | 0.42 | 5.03 | 5.03 | 4,181.06 | 46.40% |
| 2048 → 128 | DeepInfra | 1.92 | 1.10 | 8.59 | 723.40 | 58.67% |
The published output rate differs by 8.15× between the 128-input/2,048-output row and the 2,048-input/128-output row. That difference is part of the Llama 3.3 evidence, not a provider price comparison.
For the prompt-heavy row, R is 2,048 ÷ 128 = 16. At DeepInfra's current rates, each million output tokens carries sixteen million input tokens: $0.32 + 16 × $0.1 = $1.92. It reaches price equality after 1.10 billion output tokens and 8.59 million requests. Here, the prompt-heavy workload needs more requests to reach equality than the balanced workload, despite its lower output-token crossover.
The table uses 720 allocated hours and the same replica/extra-cost assumptions as the balanced calculation. Each row requires its own sourced benchmark and fresh matching quote. Percentages compare required calendar-average output rates with published aggregate references; they are not GPU-utilization percentages. NVIDIA's pinned Llama 3.3 FP8 reference.
Check whether the candidate could serve that traffic
The balanced crossover requires 1,940.04 output tokens/second averaged over the entire period. Its published reference is 4,181.06 output tokens/second per two-H100 replica, making the ratio 46.40% under the stated replica assumption.
The required rate is below this published reference. That establishes a candidate to test, not a latency guarantee or measured capacity on the quoted rental.
At ten million balanced requests, the average rises to 3,858.02 output tokens/second. The API bill is $4,200.00, compared with a modeled allocation budget of $2,112.00. Before treating that difference as a saving, test the actual rental with your arrival pattern and response-time requirement. A calendar average spreads work over quiet hours; an interactive service has to handle the arrivals when they occur.
The reference was measured on NVIDIA's DGX H100, using its quantized checkpoint and TensorRT-LLM workflow. It is not a measurement of the listed rental, a per-user response speed or a peak-performance ceiling. Matching the GPU count and FP8 label also does not establish identical weights or API quality. For this comparison, the useful rental test is the same request shape, on the chosen checkpoint, with the latency and quality requirements your application actually has.
Include operating costs before deciding to test
Two sensitivity calculations show how fragile a compute-only saving can be. The $1,000 addition is an illustrative operating budget, not a provider charge. Doubling the allocation is a cost scenario, not a tested availability design.
| Balanced-workload scenario | Period budget, USD | Break-even requests, millions |
|---|---|---|
| Current assumptions | 2,112.00 | 5.03 |
| Add $1,000 of operating costs | 3,112.00 | 7.41 |
| Keep a second copy of the allocation | 4,224.00 | 10.06 |
Your comparison should use the row that represents the service you need. If another replica is required for traffic or availability, the single-allocation crossover does not price that service.
BF16 and long contexts need a separate memory check
The pinned original Meta checkpoint has a calculated tensor weight payload of 141,107,412,992 bytes: 141.11 GB, or 131.42 GiB. A single nominal 80 GB GPU cannot hold that BF16 payload entirely in its GPU memory. The throughput reference used later is for NVIDIA's separate FP8 checkpoint, running across two H100 SXM GPUs in a DGX H100 system.
Llama 3.3 70B's stored architecture has 80 layers, 8 key/value heads and a 128-element head dimension. With a BF16 cache, the whole-model cache calculation is:
2 (key and value) × 80 layers × 8 KV heads
× 128 elements × 2 bytes = 320 KiB per resident token
4,096 tokens × 4 simultaneous sequences → 5.00 GiB cache
131.42 GiB weights + 5.00 GiB cache = 136.42 GiBNow use the catalog's full 131,072-token context instead. One resident sequence needs 40.00 GiB of BF16 cache; four need 160.00 GiB. With the original weights, those four full-context sequences total 291.42 GiB before runtime overhead.
This is why the context-window number is a poor shortcut for choosing a rental. The short-cache case and the full-context case are materially different allocations for this particular checkpoint. These totals exclude temporary buffers and other serving overhead; they also do not prove how memory is distributed across devices. An FP8 serving checkpoint or quantized cache needs its own calculation. Pinned Meta weights.
For a cost-driven decision, begin with your observed input and output lengths, sustained monthly billable traffic and the full cost of the required service. Below the corresponding crossover, the API has the lower modeled bill. Near or above it, investigate self-hosting by benchmarking the candidate with your traffic, latency target and quality requirements. The result of that test determines whether the financial opportunity is practical. Control or privacy requirements can justify a deployment test at lower volumes.
API and rental observations refresh independently. All dollar amounts, traffic examples, substituted equations and price-dependent conclusions in this article recalculate from those observations. Expired checks suppress the affected calculations. Benchmark and checkpoint evidence remain pinned and labeled; replacing them requires review rather than assuming a new configuration behaves like the old one.