Qwen3
Qwen3-32B
Qwen3 checkpoint with sourced BF16 weights and full-attention cache configuration. Memory estimates exclude unverified runtime overhead.
Profile checked · Official model page ↗
Find rentals for this model →- Parameters
- 32.8B
- Context
- 40960
- BF16 weights + cacheOverhead additional
- 63.02 GiB
Workload memory estimate
Parallel active users means one simultaneous request per user. Include input and expected output in the token allowance; idle users do not count. This sizes memory, with response speed still unverified.
- Checkpoint weights
- 61.02 GiB
- KV cache
- 2.00 GiB
- Weights + cache
- 63.02 GiB
BF16 weights / BF16 cache · 8192 cached tokens per sequence · 1 resident sequence.
Pinned checkpoint ↗ · 9216db5781bf21249d130ec9da846c4624c16137 · huggingface.co: config.json ↗ · huggingface.co: model.safetensors.index.json ↗
This is a single-GPU model-and-cache scenario. Distributed model sizing is not evaluated here. Runtime and allocator overhead are additional. This subtotal does not guarantee GPU compatibility. Input and output both contribute to cached tokens. No offloading, shared prefixes or cache quantization is assumed.
Find rentals for this model →Supported deployment evidence
The finder evaluates exact, freshly documented rental configurations. Original BF16 is the default; quantized alternatives require opt-in. A profile describes sourced compatibility and memory allocation, with response speed and live stock assessed separately.
Qwen3-32B · original BF16 · BF16
Pinned checkpoint ↗ · 65524246528 resident tensor bytes · BF16 cache.
BF16 original checkpoint; BF16 cache. All resident weights are counted, including every MoE expert. Matrix allocation is sourced from the pinned engine; norms are replicated. Runtime reserve remains an assumption.
9 curated deployment profiles. Profiles alone do not establish that an observed rental is eligible; the finder checks exact GPU count, instance topology, freshness and per-GPU memory.
Hardware evidence
Benchmarked GPU profiles
| Benchmark GPU / count | Architecture | VRAM per GPU | Matching on-demand instance |
|---|---|---|---|
| NVIDIA H200 SXM 141 GB8 GPUs per instance | Hopper | 141 GB | No recent matching instance price |
Measured performance
Inference benchmarks
| Model | GPU setup | Engine | Quant. | Workload | Reported tok/s | Evidence | Max-load capacity scenarioUSD / million output tokens |
|---|---|---|---|---|---|---|---|
| Qwen3-32B | H200 SXM 141 GB8 GPUs | vLLM with NVIDIA DynamoMeasured build undisclosed; pinned guide example uses Dynamo vllm-runtime:1.5.0 and optimizer target vLLM 0.12.0 | FP8Qwen/Qwen3-32B-FP8 | 4000 → 500input → output | 3462.0 | Source ↗Published Sep 21, 2026 | —No recent non-reference on-demand USD offer matches this GPU variant and count.8 H200 SXM GPUs (walkthrough system h200_sxm), four TP2 workers behind one frontend |
Output throughput is aggregate maximum-load performance, not single-user generation speed. Workloads show input → output tokens. Compare rows only when the model, GPU count, quantization, engine, and workload match.
Capacity scenarios combine an exact GPU variant/count rental with published output throughput from a different system. Throughput was not measured on the selected rental. Taxes, extras, idle time and minimum billing are excluded; these are not measured operating costs. Calculation guide.