vllm.models.deepseek_v41.nvidia.specialized_decode.hopper.attention
¶
The DeepSeek-V4.1 attention layer with SM90 decode attention.
It is the FlashMLA layer with one change: decode tokens attend through the
CuTe DSL sparse MLA (ampere/sparse_mla.py) instead of FlashMLA's decode
kernel. Each decode token attends to its window keys (the SWA cache) and to
the compressed keys its indexer selected (the compressed cache), both stored
as 584-byte V4 records, which the kernel reads in place
(v4_records=True). The kernel leaves the output's RoPE dims rotated, as
FlashMLA does, because the stock FP8 output projection applies the inverse
RoPE.
FlashMLA's decode kernel was the only reason to pad the queries to 64 heads:
its sparse prefill takes 8, 16 or 32 heads on SM90 as they are. For those
head counts the queries are no longer padded; otherwise the decode kernel
reads the padded queries' first n_local_heads heads. The SWA metadata
builder skips FlashMLA's decode tile scheduler, which nothing here reads.
The layer's indexer is indexer.py's, whose candidate consumers score only
their candidate keys on decode steps. Prefill context parallelism keeps the
stock layer (the model selects it).
Classes:
-
DeepseekV41HopperAttention–DeepSeek-V4.1 attention on SM90 with CuTe DSL decode attention.
-
HopperSWABackend–The SWA cache backend of the SM90 layer.
-
HopperSWAMetadataBuilder–SWA metadata without FlashMLA's decode tile-scheduler plans.
DeepseekV41HopperAttention
¶
Bases: DeepseekV4FlashMLAAttention
DeepSeek-V4.1 attention on SM90 with CuTe DSL decode attention.
Methods:
-
get_padded_num_q_heads–No padding where FlashMLA's sparse prefill takes the heads as they
Source code in vllm/models/deepseek_v41/nvidia/specialized_decode/hopper/attention.py
_forward_decode(q, kv_cache, swa_metadata, attn_metadata, swa_only, output)
¶
Sparse MLA of the decode tokens over their window and selected keys, the output's RoPE dims left rotated.
Source code in vllm/models/deepseek_v41/nvidia/specialized_decode/hopper/attention.py
_indexer_cache_cls()
¶
_indexer_cls(k_cache)
¶
The indexer with compact candidate consumers on decode steps.
get_padded_num_q_heads(num_heads)
classmethod
¶
No padding where FlashMLA's sparse prefill takes the heads as they are; otherwise FlashMLA's padding, which the prefill still needs.
Source code in vllm/models/deepseek_v41/nvidia/specialized_decode/hopper/attention.py
HopperSWABackend
¶
Bases: DeepseekSparseSWAFlashMLABackend
The SWA cache backend of the SM90 layer.
Source code in vllm/models/deepseek_v41/nvidia/specialized_decode/hopper/attention.py
HopperSWAMetadataBuilder
¶
Bases: DeepseekSparseSWAFlashMLAMetadataBuilder
SWA metadata without FlashMLA's decode tile-scheduler plans.
Methods:
-
build_tile_scheduler–No plans: the base builds none for a step without decode tokens.
Source code in vllm/models/deepseek_v41/nvidia/specialized_decode/hopper/attention.py
build_tile_scheduler(num_decode_tokens)
¶
No plans: the base builds none for a step without decode tokens.