Qwen3-VL-30B-A3B: When to Test Self-Hosting on an H100 NVL

For Qwen3-VL-30B-A3B-Instruct, an API bill around 1,344.00 USD per 30 days can fund the modeled candidate compute budget. At 4,096 billed input and 1,024 billed output tokens per request, that is about 1.1 million requests, or 1.1 billion output tokens plus their input. Near this spending line, investigate the candidate below. Moving traffic requires measured capacity, acceptable task quality and a complete operating budget.

Your Qwen3-VL-30B-A3B-Instruct usage justifies investigating self-hosting when its API spending approaches the live candidate budget below. The proposed experiment is the pinned original BF16 checkpoint on one H100 NVL 94 GB, compared with DeepInfra's exact FP8 endpoint. For this vision model, matching the prompt text is insufficient: resizing a screenshot or changing video frame timing can change the task. Use the financial screen to decide when to investigate, then test the same media and completed tasks before deciding whether to move traffic. GPU fit, serving capacity and checkpoint equivalence remain unverified.

Capacity unverified

This article provides a financial investigation screen. It has no qualified aggregate output-throughput measurement for the candidate. The traffic figure does not establish a reachable saving, supported concurrency, response time or equivalence to the API.

Preserve the 16-pixel vision path before comparing screenshots

This checkpoint has a 16-pixel vision patch and a spatial merge size of two. Its DeepStack configuration injects visual features from layers 8, 16 and 24. The image processor is part of the experiment: its shipped size settings differ from the video processor's settings. A short typed prompt does not describe the work involved in reading a dense screenshot.

There is a concrete migration trap in the shared Qwen utility. Its function default is image_patch_size=14, while the Qwen3-VL example explicitly passes 16. That example also passes do_resize=False to the processor after the utility has resized the media. Copying an older caller without these choices can alter preprocessing. Check the actual library version and processed image dimensions rather than assuming the API and local defaults agree.

For a screenshot or document replay, retain the original files, page order, resizing policy and accepted answers. Include small text, closely spaced controls and multiple pages. Record processed tokens and task success alongside latency. Reducing resolution to fit the candidate changes the experiment; verify that the user can still complete the same task.

Evidence: architecture source; architecture source; architecture source; serving source; serving source.

For video, keep frame timing as well as frame content

Qwen3-VL's publisher describes timestamp-grounded video understanding. The utility's Qwen3-VL example requests return_video_metadata=True and forwards that metadata to the processor. Frame content alone is not the complete documented input path.

The captured utility has another practical distinction: file readers retain original frame indices and frame rate, but a caller supplying an already extracted list of frames gets synthesized metadata. That path defaults sample_fps to 2.0, uses sequential indices and pads an odd frame count by repeating the final frame. Its process_vision_info result also sets do_sample_frames=False. Do not assume an arbitrary list of extracted screenshots preserves the source clip's timing.

Replay a clip with a known event time, a sparsely sampled clip and a pre-extracted frame list. Inspect the timestamps actually sent to the model and check its event answers against the source video. Record decoder, sampling rate, frame count and processed size. These controls are especially relevant when moving an event-search application: a faster response with an incorrect event time is a failed task. Treat changes to frame selection as quality changes that need evaluation, not as an established GPU saving.

Evidence: model source; serving source; serving source.

The 72 GB guidance comes from CPU validation

The official vLLM YAML names this exact Instruct model, sets its default precision to BF16 and lists 72 GB minimum-VRAM guidance. Its verified hardware is Intel Xeon 6 CPU, and the shown launch uses tensor parallel size one. It provides no measured H100 NVL memory allocation, concurrent-request capacity or tested publisher weight revision. Our one-H100 proposal is therefore a candidate to test; GPU fit remains unverified.

The 3B-active label describes sparse language-model computation, not a 3B-sized checkpoint to load. The pinned configuration has 128 experts with eight selected per token, as well as a vision encoder. Keep the full BF16 checkpoint in the test. The API's FP8 declaration is a separate precision; it does not identify the provider's downloadable weight revision or establish equivalent answers.

The guide offers --language-model-only, which skips the vision encoder. That is useful for an explicitly text workload, but it cannot validate screenshot or video service. Test the full visual path at your actual input size and concurrency. The guide leaves scheduler limits and memory-utilization tuning to the deployment; neither its minimum nor a startup cache allocation supplies an output-throughput reference.

Evidence: architecture source; serving source.

Count a complete visual-agent task, including follow-up turns

The pinned Instruct template inserts image and video placeholders into structured user messages. It also renders assistant tool calls and groups consecutive tool results inside a user turn. The publisher describes visual-agent and visual-coding uses, but those claims do not show that a particular application's tool adapter or action loop works unchanged on a self-hosted server.

Replay the full exchange: the initial screenshot, the proposed tool call, its result, a fresh screenshot and the follow-up answer. Inspect ordering and media attachment, parse tool arguments, and verify the action against the resulting screen. The pinned generation policy uses sampling with temperature 0.7, top_p 0.8 and top_k 20. Record explicit settings instead of allowing different client defaults to decide the comparison. These settings are a reproducibility reference, not a guarantee of quality.

For the live financial screen, use provider-reported billed input and output. Keep retries and follow-up turns in the totals; prompt text alone cannot count visual input. The displayed text-shaped example is a spending measurement target, not an image-to-token conversion or a visual throughput claim. Before moving traffic, require the candidate to finish the same tasks within your latency target and a budget that includes media processing and operations.

Evidence: model source; architecture source; architecture source.

The candidate to test

Test 1 × H100 NVL (94 GB per GPU) with the pinned BF16 checkpoint. Serving-engine guidance: the captured official vLLM recipe lists a 72 GB minimum for this model and precision, with vLLM 0.11.0 or newer. Serving recipe, captured October 14, 2026 at 9:42 am EDT; the reviewed snapshot is fixed by its capture hash, even if the linked document changes. Reported validation scope: The captured recipe marks Intel Xeon 6 CPU hardware verified with BF16 serving. Its CPU launch uses tensor parallel size one and encoder data-parallel mode; it supplies no fixed context, request-concurrency or aggregate-output benchmark. The 72 GB default minimum is guidance, not measured one-H100 NVL GPU memory. No tested publisher weight revision is identified. GPU fit for the selected H100 NVL 94 GB remains unverified. GPU fit unverified: this guidance identifies a candidate to investigate; it does not verify the chosen GPU, checkpoint revision, context length, simultaneous requests or runtime memory. The checkpoint is pinned for your test; the engine recipe names the model without identifying its tested checkpoint revision. The API is separately tracked as fp8. Verify fit, formatting, task quality and checkpoint/precision equivalence before moving traffic. The calculation assumes the same billed token workload, without proving that the candidate generates the same tokens.

Spending investigation threshold

The current API observation is DeepInfra's Qwen/Qwen3-VL-30B-A3B-Instruct, fp8, at 0.15 USD/million input and 0.6 USD/million output tokens. The lowest qualifying quote in our catalog for this exact candidate is Vast.ai, Marketplace offer #33610858, , US, 1.87 USD/hour for the whole instance. It is listed on-demand hourly pricing with no minimum commitment; confirm stock and final charges. Stock status: listed. This is catalog coverage, not a whole-market comparison.

# Display values are rounded; totals use full stored rates.
Period budget ≈ 1.87 USD/hour × 720 hours × 1 replica(s) + 0.00 USD
              ≈ 1,344.00 USD
API USD/million output, including input = 0.6 + (4096 / 1024) × 0.15
                                       = 1.2
Spending investigation threshold = budget / API USD per million output × 1,000,000
                                ≈ 1.1 billion billed output tokens
Requests ≈ 1.1 million per 30 days

At the spending threshold the workload demands 432 billed output tokens/second averaged over the calendar period, or 432 per replica if traffic is evenly shared. These are measurement targets, not measured GPU speeds. Replay the actual request lengths, bursts, reasoning settings and tool turns, then check aggregate output throughput, tail latency, failures and task success.

With the same 4,096 input tokens but 8,192 billed output tokens per request, the same budget corresponds to 240 thousand requests and 768 average output tokens/second. This longer-output sensitivity illustrates the effect of billed generation; it does not predict a reasoning setting's output length or the GPU's performance.

The period assumes 720 always-on hours and 1 replica(s). The explicit additional operating allowance is 0.00 USD. Include storage, network, taxes, monitoring, operational work and redundancy before comparing total service cost. When this allowance is zero the screen covers compute only. Extra replicas increase the bill; they are not a tested availability design.

API checked October 14, 2026 at 9:47 am EDT, collector source; rental checked October 14, 2026 at 2:00 pm EDT, quote source. Recorded terms: On-demand compute billed for actual active runtime; no monthly commitment. Storage persists and remains billable until deleted; transfer and other charges are separate. Quote notes: API discovery sample, capped at 12/GPU model. Rentable at check; capacity can change. Compute only; storage and traffic extra. Prices and calculations refresh automatically. Figures are rounded for readability; calculations use the full stored precision. Stale, retired or changed-identity observations suppress this calculation.

Self-hosting terms and evidence

The pinned Qwen publisher model card declares Apache 2.0. The captured Apache 2.0 text in the official Qwen3-VL project repository has been reviewed for this self-hosting comparison; it is not relabelled as a license file in the pinned weights repository. When redistributing, provide the license, preserve applicable notices and identify modified files, including any applicable NOTICE requirements. These terms do not establish suitability or compliance for a particular application.

Publisher terms. Analysis reviewed October 14, 2026. Noach Ark did not rent this allocation or run inference, capacity or quality tests for this article. Prices and financial calculations refresh independently of this editorial review.