vllm.model_executor.kernels.linear.scaled_mm.ScaledMMLinearKernel
¶
Classes:
ScaledMMLinearKernel
¶
Bases: Generic[_ConfigT, _ParamsT], ABC
Methods:
-
executed_activation_key–The activation quantization the kernel runs for a layer requesting
-
input_quant_key–The activation quant key this kernel can consume pre-quantized.
Source code in vllm/model_executor/kernels/linear/scaled_mm/ScaledMMLinearKernel.py
executed_activation_key(activation_quant_key)
classmethod
¶
The activation quantization the kernel runs for a layer requesting
activation_quant_key. Weight-only kernels return None; kernel
selection then warns about, or under VLLM_STRICT_QUANT_SCHEME forbids,
the fallback.
Source code in vllm/model_executor/kernels/linear/scaled_mm/ScaledMMLinearKernel.py
input_quant_key()
¶
The activation quant key this kernel can consume pre-quantized.
Manual fusion uses this to decide whether to hoist activation quantization out of apply_weights into an upstream fused kernel. Return None when the kernel needs in-kernel quantization (custom padding or swizzling, dynamic scales, etc.). Kernels that return a key must consume the activation via as_quantized_activation.