vllm.model_executor.warmup.deep_gemm_warmup
¶
Warmup deep_gemm kernels. DeepGEMM JIT's the kernels. The warmup aims to JIT all the kernels that would be used during model execution beforehand.
_deep_gemm_linear_data(module)
¶
Return (weight, weight_scale) for a layer whose kernel stamped itself as
deep_gemm_warmup_provider in process_weights_after_loading, else None.
Source code in vllm/model_executor/warmup/deep_gemm_warmup.py
_extract_data_from_fused_moe_module(m_)
¶
Extract weights, weight scales and num_topk from MoERunner module.
Source code in vllm/model_executor/warmup/deep_gemm_warmup.py
_fp8_einsum_warmup_data(module)
¶
Return (wo_a weight, wo_a scale, einsum recipe, tma_aligned_scales,
quantize_output) for a DeepSeek-V4 or V4.1 attention layer whose o_proj
runs fp8_einsum (see deep_gemm_fp8_o_proj and V4.1 mega attention),
else None.
wo_a is an is_bmm linear: its DeepGEMM kernel skips the
fp8_gemm_nt warmup provider because the layer runs through
fp8_einsum instead, so it has to be warmed here.
Source code in vllm/model_executor/warmup/deep_gemm_warmup.py
_generate_optimal_warmup_m_values(max_tokens, n, device)
¶
Generate M values that cover all possible DeepGEMM kernel configurations.
The block sizes must mirror the layout candidates of the vendored DeepGEMM heuristic (csrc/jit_kernels/heuristics/sm100.hpp); a stale set here lets serving-time M values select kernel configs the warmup never compiled.
Parameters:
-
(max_tokens¶int) –Maximum number of tokens to warmup for
-
(n¶int) –The actual N dimension from the weight tensor
-
(device¶device) –The torch device to get properties from.
Source code in vllm/model_executor/warmup/deep_gemm_warmup.py
_get_fp8_einsum_m_values(weight, max_tokens)
¶
M values (attention tokens per step) to warm fp8_einsum with.
DeepGEMM picks the einsum tile config from M at runtime, so every config a
prefill chunk can land on has to be compiled up front. relax reuses the
fp8_gemm_nt boundary heuristic with N = the einsum output width, plus
every M a tile boundary can fall on.
Source code in vllm/model_executor/warmup/deep_gemm_warmup.py
_get_fp8_fp4_mega_moe_m_values(module, max_tokens)
¶
Get the M values that reach every MegaMoE kernel and, where used, bf16_mega_gate config.
Source code in vllm/model_executor/warmup/deep_gemm_warmup.py
_get_fp8_gemm_nt_m_values(w, max_tokens)
¶
Get the M values to warmup for a given weight tensor.