Meta Llama 3.3
Llama 3.3 70B Instruct
Meta's 70B instruction-tuned Llama 3.3 model. The published throughput rows use NVIDIA's FP8 checkpoint across two H100 SXM 80 GB GPUs. Memory sizing uses the pinned original BF16 checkpoint, separately from the FP8 benchmark.
Profile checked · Official model page ↗
Find rentals for this model →- Parameters
- 70B
- Context
- 131072
- BF16 weights + cacheOverhead additional
- 133.92 GiB
Workload memory estimate
Parallel active users means one simultaneous request per user. Include input and expected output in the token allowance; idle users do not count. This sizes memory, with response speed still unverified.
- Checkpoint weights
- 131.42 GiB
- KV cache
- 2.50 GiB
- Weights + cache
- 133.92 GiB
BF16 weights / BF16 cache · 8192 cached tokens per sequence · 1 resident sequence.
Pinned checkpoint ↗ · 6f6073b423013f6a7d4d9f39144961bfbfbc386b · huggingface.co: 6f6073b423013f6a7d4d9f39144961bfbfbc386b ↗ · raw.githubusercontent.com: sku list.py ↗
This is a single-GPU model-and-cache scenario. Distributed model sizing is not evaluated here. Runtime and allocator overhead are additional. This subtotal does not guarantee GPU compatibility. Input and output both contribute to cached tokens. No offloading, shared prefixes or cache quantization is assumed.
Find rentals for this model →Supported deployment evidence
The finder evaluates exact, freshly documented rental configurations. Original BF16 is the default; quantized alternatives require opt-in. A profile describes sourced compatibility and memory allocation, with response speed and live stock assessed separately.
Llama 3.3 70B Instruct · original BF16 · BF16
Pinned checkpoint ↗ · 141107412992 resident tensor bytes · BF16 cache.
BF16 original checkpoint; BF16 cache. All resident weights are counted, including every MoE expert. Matrix allocation is sourced from the pinned engine; norms are replicated. Runtime reserve remains an assumption.
9 curated deployment profiles. Profiles alone do not establish that an observed rental is eligible; the finder checks exact GPU count, instance topology, freshness and per-GPU memory.
Llama 3.3 70B Instruct · NVIDIA FP8 alternative · FP8
Pinned checkpoint ↗ · 72656376832 resident tensor bytes · BF16 cache.
NVIDIA ModelOpt FP8 linear weights and activations; BF16 embedding/head and BF16 cache. Exact checkpoint tensor-byte total includes scales; all scales and norms are replicated as an upper bound. Use --quantization modelopt and --kv-cache-dtype bfloat16. Quantization can change quality; published model-card evaluations are not a guarantee for your task. Review license and validate your prompts.
4 curated deployment profiles. Profiles alone do not establish that an observed rental is eligible; the finder checks exact GPU count, instance topology, freshness and per-GPU memory.
Hardware evidence
Benchmarked GPU profiles
| Benchmark GPU / count | Architecture | VRAM per GPU | Matching on-demand instance |
|---|---|---|---|
| NVIDIA H100 SXM 80 GB2 GPUs per instance | Hopper | 80 GB | No recent matching instance price |
Measured performance
Inference benchmarks
| Model | GPU setup | Engine | Quant. | Workload | Reported tok/s | Evidence | Max-load capacity scenarioUSD / million output tokens |
|---|---|---|---|---|---|---|---|
| Llama 3.3 70B Instruct | H100 SXM 80 GB2 GPUs | TensorRT-LLM PyTorchv0.21; methodology pinned at 0c9430e5a530ba958fc9dca561a3ad865ad9f492 | FP8nvidia/Llama-3.3-70B-Instruct-FP8 | 128 → 2048input → output | 5892.9 | Source ↗Published Sep 15, 2025 | —No recent non-reference on-demand USD offer matches this GPU variant and count.NVIDIA DGX H100 (H100 SXM 80 GB) |
| Llama 3.3 70B Instruct | H100 SXM 80 GB2 GPUs | TensorRT-LLM PyTorchv0.21; methodology pinned at 0c9430e5a530ba958fc9dca561a3ad865ad9f492 | FP8nvidia/Llama-3.3-70B-Instruct-FP8 | 1000 → 1000input → output | 4181.1 | Source ↗Published Sep 15, 2025 | —No recent non-reference on-demand USD offer matches this GPU variant and count.NVIDIA DGX H100 (H100 SXM 80 GB) |
| Llama 3.3 70B Instruct | H100 SXM 80 GB2 GPUs | TensorRT-LLM PyTorchv0.21; methodology pinned at 0c9430e5a530ba958fc9dca561a3ad865ad9f492 | FP8nvidia/Llama-3.3-70B-Instruct-FP8 | 2048 → 128input → output | 723.4 | Source ↗Published Sep 15, 2025 | —No recent non-reference on-demand USD offer matches this GPU variant and count.NVIDIA DGX H100 (H100 SXM 80 GB) |
Output throughput is aggregate maximum-load performance, not single-user generation speed. Workloads show input → output tokens. Compare rows only when the model, GPU count, quantization, engine, and workload match.
Capacity scenarios combine an exact GPU variant/count rental with published output throughput from a different system. Throughput was not measured on the selected rental. Taxes, extras, idle time and minimum billing are excluded; these are not measured operating costs. Calculation guide.