Skip to content

vllm.model_executor.layers.quantization.online

Modules:

  • base –
  • fp8 –
  • int4 –

    Online int4 weight-only quantization.

  • int8 –
  • legacy_env –

    Deprecated environment variables redirected to --quantization-config.

  • lm_head –

    Online lm_head quantization: weight-only methods for the model's own

  • loading –
  • lut_b –

    Online LUT-B quantization of routed MoE experts (MXFP8 activations).

  • moe_base –
  • mxfp4 –

    Online MXFP4 (microscaling FP4, block-32) quantization methods.

  • mxfp8 –

    Online MXFP8 (microscaling FP8, block-32) quantization methods.

  • nvfp4 –
  • requantization –
  • shared_expert –