Inference model index

01 Gemma 4 26B A4B ITGemma 4 · BF16 Memory estimate unavailablePinned configuration needed → 02 Gemma 4 31B ITGemma 4 · BF16 Memory estimate unavailablePinned configuration needed → 03 Llama 3.1 405B InstructMeta Llama 3.1 · FP8 benchmark configuration 759.90 GiBBF16 weights + cache · 8192 tokens · 1 sequence · overhead additional → 04 Llama 3.1 8B InstructMeta Llama 3.1 · FP8 benchmark configuration 15.96 GiBBF16 weights + cache · 8192 tokens · 1 sequence · overhead additional → 05 Llama 3.3 70B InstructMeta Llama 3.3 · FP8 benchmark configuration 133.92 GiBBF16 weights + cache · 8192 tokens · 1 sequence · overhead additional → 06 Qwen3-14BQwen3 · BF16 34 GB+Published recommendation → 07 Qwen3-30B-A3BQwen3 · BF16 checkpoint 57.62 GiBBF16 weights + cache · 8192 tokens · 1 sequence · overhead additional → 08 Qwen3-32BQwen3 · BF16 checkpoint 63.02 GiBBF16 weights + cache · 8192 tokens · 1 sequence · overhead additional → 09 Qwen3-8BQwen3 · BF16 checkpoint 16.38 GiBBF16 weights + cache · 8192 tokens · 1 sequence · overhead additional → 10 Qwen3-VL-30B-A3B-InstructQwen3-VL · BF16 Memory estimate unavailablePinned configuration needed → 11 Qwen3.5-27BQwen3.5 · BF16 Memory estimate unavailablePinned configuration needed → 12 Qwen3.5-35B-A3BQwen3.5 · BF16 Memory estimate unavailablePinned configuration needed → 13 Qwen3.5-9BQwen3.5 · BF16 checkpoint Memory estimate unavailablePinned configuration needed → 14 Qwen3.8-27BQwen3.8 · BF16 Memory estimate unavailablePinned configuration needed → 15 gpt-oss-120bOpenAI gpt-oss · MXFP4 80 GB+Published recommendation → 16 gpt-oss-20bOpenAI gpt-oss · MXFP4 16 GB+Published recommendation →