vllm.models.minimax_m3.nvidia.specialized_decode.kernels.o_proj.weights
¶
Router-hoist weights of the TP4 o_proj router CTAs.
The MoE router runs on the post-attention normed row, logits = W_r n with
n = h * rstd * (1 + g) and h = residual + sum_k o_k (o_k = rank k's
o_proj partial W_o,k a_k). rstd is a per-row scalar, so
logits = rstd * (W' h), W' = W_r diag(1 + g)
W' h = sum_k [ W'' _k a_k + W'[:, H_k] residual[H_k] ], W''_k = W' deq(W_o,k)
Each rank's router CTAs compute their bracket (a from the decode's MXFP8
output, r from this rank's quarter of the residual columns) while the
GEMV runs, push it to every rank's rx slot, and the post multiplies the rank
sum by rstd. W'' is folded in fp64 and rounded once to fp32.
Blob layouts (per rank, fp32), as the kernel's router CTAs index them:
blob_a [r][e][m][tid][4] = W''_k[8 r + e, 8 tid + 4 m + j] (2 MB)
blob_r [r][e][m][tid][4] = W'[8 r + e, c0 + 1024 m + 4 tid + j] (2 MB,
columns >= 1536 zero-padded; c0 = rank * 1536)
Functions:
-
blobs–(blob_a, blob_r, W', W''_k). o_sf_u8: the unswizzled UE8M0 [H, KO / 32] of this
-
emulate–CPU model of one rank's router CTAs with the kernel's flat indexing (fp64: layout
-
fold_oproj–W'' = W' deq(W_o_k), fp64 accumulate, one rounding to fp32. o_deq: [H, Ko].
-
fold_router–W' = W_r diag(1 + g): (1 + g) in fp32, then one fp32 product.
-
hoisted_logits_ref–parts: per source rank [T, E] fp32 partials (a + r); rstd [T]. Rank-order fp32
-
mxfp8_dequant–[N, K] e4m3 x UE8M0 per 32 columns -> fp64 (exact).
-
mxfp8_quant_rows–Row-wise MXFP8 (code = e8m0 roundup of amax/448, x / 2^code -> e4m3 RN sat), for
-
route_minimax2–The kernels' MiniMax2 routing, op for op: key = (0.5 tanh(0.5 x) + 0.5) + bias;
-
swizzle_128x4–[N, KB] uint8 -> flat swizzled uint8 (rows padded to 128, KB to a multiple of
blobs(router, post_norm, o_w_e4m3, o_sf_u8, rank, world=4)
¶
(blob_a, blob_r, W', W''_k). o_sf_u8: the unswizzled UE8M0 [H, KO / 32] of this rank's o_proj slice.
Source code in vllm/models/minimax_m3/nvidia/specialized_decode/kernels/o_proj/weights.py
emulate(blob_a, blob_r, xq_f, sc, res, rank, t)
¶
CPU model of one rank's router CTAs with the kernel's flat indexing (fp64: layout
check, not rounding).
xq_f [t][KO] e4m3 values, sc [t][KO / 32] scale factors, res [t][H].
Returns (z_a, z_r) [t][E].
Source code in vllm/models/minimax_m3/nvidia/specialized_decode/kernels/o_proj/weights.py
fold_oproj(wp, o_deq)
¶
W'' = W' deq(W_o_k), fp64 accumulate, one rounding to fp32. o_deq: [H, Ko].
fold_router(router, post_norm)
¶
W' = W_r diag(1 + g): (1 + g) in fp32, then one fp32 product.
Source code in vllm/models/minimax_m3/nvidia/specialized_decode/kernels/o_proj/weights.py
hoisted_logits_ref(parts, rstd)
¶
parts: per source rank [T, E] fp32 partials (a + r); rstd [T]. Rank-order fp32 sum times rstd.
Source code in vllm/models/minimax_m3/nvidia/specialized_decode/kernels/o_proj/weights.py
mxfp8_dequant(w_e4m3, sf_u8)
¶
[N, K] e4m3 x UE8M0 per 32 columns -> fp64 (exact).
Source code in vllm/models/minimax_m3/nvidia/specialized_decode/kernels/o_proj/weights.py
mxfp8_quant_rows(x)
¶
Row-wise MXFP8 (code = e8m0 roundup of amax/448, x / 2^code -> e4m3 RN sat), for tests.
Source code in vllm/models/minimax_m3/nvidia/specialized_decode/kernels/o_proj/weights.py
route_minimax2(logits, bias)
¶
The kernels' MiniMax2 routing, op for op: key = (0.5 tanh(0.5 x) + 0.5) + bias; top-4 by key, ties to the lower id; sk = key - bias; w = bf16(2 sk / ((sk0 + sk2) + (sk1 + sk3) + 1e-20)). Returns ids [T, 4] int64 (rank order), w [T, 4] fp32 (bf16 values), keys [T, E].
Source code in vllm/models/minimax_m3/nvidia/specialized_decode/kernels/o_proj/weights.py
swizzle_128x4(sf)
¶
[N, KB] uint8 -> flat swizzled uint8 (rows padded to 128, KB to a multiple of 4).