vllm.model_executor.layers.quantization.online.loading
¶
Functions:
-
checkpoint_method–The pre-quantized checkpoint method that
specstands for, if any. -
quantize_on_load–Make quantized
layer.<name>load high-precision tensors viaquantize.
_load_quantized(layer, name, *scale_names)
¶
Load (values, *scales) as a checkpoint of that layout is loaded.
Source code in vllm/model_executor/layers/quantization/online/loading.py
_nvfp4_on_load(layer, method)
¶
w1 and w3 of an expert share one global scale: the first waits.
Source code in vllm/model_executor/layers/quantization/online/loading.py
checkpoint_method(layer, spec, default)
¶
The pre-quantized checkpoint method that spec stands for, if any.
Source code in vllm/model_executor/layers/quantization/online/loading.py
quantize_on_load(layer, name, quantize, *scale_names)
¶
Make quantized layer.<name> load high-precision tensors via quantize.