Skip to content

vllm.model_executor.models.qwen3

Inference-only Qwen3 model compatible with HuggingFace weights.

Functions:

  • use_qwen2_attention –

    Make Qwen3 layers build Qwen2 attention: QKV bias and opt-in q/k-norm.

use_qwen2_attention(config)

Make Qwen3 layers build Qwen2 attention: QKV bias and opt-in q/k-norm.

Source code in vllm/model_executor/models/qwen3.py
def use_qwen2_attention(config: PreTrainedConfig) -> None:
    """Make Qwen3 layers build Qwen2 attention: QKV bias and opt-in q/k-norm."""
    config.attention_bias = True
    config.qk_norm = getattr(config, "qk_norm", False)
    # Every layer gets the global sliding window, so it can't be per layer
    if len(set(getattr(config, "layer_types", []))) > 1:
        assert config.max_window_layers == config.num_hidden_layers, (
            "Sliding window for some but all layers is not supported. "
            "This model uses sliding window but `max_window_layers` = "
            f"{config.max_window_layers} is less than `num_hidden_layers` = "
            f"{config.num_hidden_layers}. Please open an issue "
            "to discuss this feature."
        )