vllm.models.minimax_m3.nvidia.specialized_decode.kernels.moe.weights
¶
MoE engine operands from vLLM's loaded MiniMax-M3 MoE layer, and the mode-Q FC2 weight prep.
Source of every routed-expert operand: vLLM's NVFP4 MoE with the FLASHINFER_TRTLLM
backend after
process_weights_after_loading (prepare_nvfp4_moe_layer_for_fi_or_cutlass ->
prepare_static_weights_for_trtllm_fp4_moe). The engine reads these buffers in place,
never copies them:
================== ========================================== ==============================================
engine operand vLLM tensor layout
================== ========================================== ==============================================
w3_w1_weight routed_experts.w13_weight uint8 [E, 2 I, H / 2] packed e2m1 (low nibble =
even column); rows =
logical [w3 (up); w1
(gate)]
(reorder_w1w3_to_w3w1),
then the gated-act
row interleave and the
epilogue-tile-128
32-row
shuffle
(_maybe_get_cached_w3_w1_permute_indices)
w3_w1_weight_scale routed_experts.w13_weight_scale e4m3 [E, 2 I, H / 16], one per 16 elements, rows
permuted like the
weight, then
nvfp4_block_scale_interleave
(128x4 swizzle)
w2_weight routed_experts.w2_weight uint8 [E, H, I / 2], 32-row shuffle (epilogue 128)
w2_weight_scale routed_experts.w2_weight_scale e4m3 [E, H, I / 16], permuted + 128x4 swizzle
fc31_alpha quant_config.g1_alphas f32 [E] = a13_scale * w13_weight_scale_2
fc31_scale_c fused_experts.g1_scale_c (re-derived) f32 [E] = g1_alphas * fc2_enc
act_alpha fused_experts.gemm1_alpha f32 [E] (SwiGLU-OAI alpha, raw)
act_beta fused_experts.gemm1_beta f32 [E] = beta / g1_alphas (vLLM pre-folds it)
act_clamp fused_experts.gemm1_clamp_limit f32 [E] = limit / g1_alphas (vLLM pre-folds it)
fc2_alpha quant_config.g2_alphas (re-derived) f32 [E] = w2_weight_scale_2 / fc2_enc
fc31_input_scale quant_config.a1_gscale f32 [1], the checkpoint's 1.0 placeholder
router_weight block_sparse_moe.gate.weight f32 [E, H]
bias block_sparse_moe.e_score_correction_bias f32 [E]
shared gate_up/down shared_experts.{gate_up,down}_proj e4m3 weight [N, K] + UE8M0 scales, 128x4
swizzled (one per 32)
================== ========================================== ==============================================
Mode Q (the M3.1 QAT recipe): FC1 input per token (the post's s_t, the checkpoint's FC1
input scale is a 1.0
placeholder) and FC2 input under the fixed global encode scale 16. The FC2 encode scale
lives in two per-expert
scalars (exact powers of two): fc31_scale_c = g1_alphas * fc2_enc and fc2_alpha =
w2_scale_2 / fc2_enc.
vLLM folds whatever FC2 input scale the checkpoint carries (a2_gscale) into both;
prep_mode_q removes that
fold (it must be one power of two) and applies 16.
Classes:
Functions:
-
from_vllm_moe–MoeWeights from a loaded
MiniMaxM3MoE(decoder_layer.block_sparse_moe) on -
prep_mode_q–Mode Q FC2 prep as new tensors (idempotent): fc31_scale_c = g1_alphas x 16,
MoeWeights
dataclass
¶
Methods:
-
expert_args–The 11 positional operands of
M3MoeState.layer.
Source code in vllm/models/minimax_m3/nvidia/specialized_decode/kernels/moe/weights.py
77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 | |
expert_args()
¶
The 11 positional operands of M3MoeState.layer.
Source code in vllm/models/minimax_m3/nvidia/specialized_decode/kernels/moe/weights.py
_linear_mxfp8(lin, what, shape)
¶
(e4m3 weight [N, K] contiguous, flat UE8M0 128x4-swizzled scales) of a vLLM MXFP8
linear after weight
processing (undoes the CuTe-DSL kernel's [K, N] view; see _mxfp8).
Source code in vllm/models/minimax_m3/nvidia/specialized_decode/kernels/moe/weights.py
from_vllm_moe(moe)
¶
MoeWeights from a loaded MiniMaxM3MoE (decoder_layer.block_sparse_moe) on
the FLASHINFER_TRTLLM NVFP4
backend. Raises MoeWeightError with the reason when the layer is in any other
form.
Source code in vllm/models/minimax_m3/nvidia/specialized_decode/kernels/moe/weights.py
prep_mode_q(w)
¶
Mode Q FC2 prep as new tensors (idempotent): fc31_scale_c = g1_alphas x 16, fc2_alpha = w2_scale_2 / 16. Refuses non-placeholder routed FC1 input scales (mode Q's FC1 scale is the post's per-token s_t).