vllm.models.deepseek_v41.nvidia.specialized_decode.ampere
¶
DeepSeek-V4.1 on Ampere datacenter GPUs (A100, SM80).
The stock DeepSeek-V4.1 path needs kernels that A100 lacks: FlashMLA and DeepGEMM build for SM90 and SM100 only, and Triton cannot convert to or from FP8 E4M3 below SM89. This package replaces each of them with a CuTe DSL kernel that runs on SM80, keeping the stock model, scheduler and cache layout around them:
- the KV caches keep DeepSeek's V4.1 records -- the 528-byte MXFP8 sliding-window record and the 288-byte NVFP4 compressed record -- and the 68-byte MXFP4 index keys, as on SM100;
- every FP8 and FP4 value is decoded with integer operations into BF16, where
it is exact, and multiplied on BF16 tensor cores (
mma.sync) with FP32 accumulation; quantizers round to nearest even exactly as the hardware conversions of SM100 would.
The modules:
| Module | Contents |
|---|---|
attention.py |
The attention layer, its compressor and its metadata backends |
indexer.py |
The indexer: MXFP4 queries and keys, its op (logits, top-k) |
indexer_metadata.py |
The indexer metadata builder (no DeepGEMM schedule) |
sparse_mla.py |
The sparse MLA kernels |
mqa_logits.py |
The indexer logits kernel |
topk.py |
The indexer's decode top-k and candidate-block selection |
quantize.py |
The FP4 quantizers (queries, index keys, compressed records) |
mhc.py |
The mHC layer boundary: post-mix, collapse + RMSNorm, coefficients |
all_reduce.py |
The boundaries' two-shot Lamport all-reduce |
all_gather.py |
The Lamport all-gather (indexer rows; logits and Engram columns) |
columns.py |
The logits and Engram gathers along the last dimension |
codecs.py |
Device helpers: FP8/FP4 decoding and encoding, raw loads/stores |
Each kernel is specified by a PyTorch function of
tests/models/deepseek_v41/nvidia/specialized_decode/ampere/reference.py,
which the docstrings call reference.<function>.
This module holds only the platform predicate, the index K cache choice and a table helper, so that shared DeepSeek-V4.1 code can ask for the Ampere path without importing CuTe DSL.
Modules:
-
all_gather–The TP all-gathers of the SM80 decode step, in CuTe DSL.
-
all_reduce–The TP all-reduce of DeepSeek-V4.1's decode boundaries on SM80, in CuTe DSL.
-
attention–The DeepSeek-V4.1 attention layer on SM80.
-
codecs–CuTe DSL device helpers of the SM80 kernels: global loads by address and
-
columns–The SM80 and SM90 TP all-gathers along the last dimension: logits and
-
gemv–MXFP8 linear layers at decode batch sizes on SM80, in CuTe DSL.
-
indexer–The DeepSeek-V4.1 sparse-attention indexer on SM80.
-
indexer_metadata–The DeepSeek-V4.1 indexer metadata on SM80.
-
mhc–The mHC layer boundary of DeepSeek-V4.1 on SM80, in CuTe DSL.
-
mhc_projection–The mHC projection of DeepSeek-V4.1 prefill on SM80, in CuTe DSL.
-
mqa_logits–The DeepSeek-V4.1 indexer logits on SM80, in CuTe DSL.
-
quantize–The FP4 quantizers of DeepSeek-V4.1 on SM80, in CuTe DSL.
-
row_shard–The DeepSeek-V4.1 indexer's prefill rows split over the TP ranks, on SM80
-
sparse_mla–DeepSeek-V4.1 sparse MLA on SM80 in CuTe DSL.
-
topk–The DeepSeek-V4.1 indexer's decode top-k on SM80, in CuTe DSL.
Functions:
-
indexer_uses_fp4–The index K cache on SM80: always MXFP4, the only one its kernels
-
is_ampere–Whether the current device runs DeepSeek-V4.1 through this package.
-
rope_table_f32–The RoPE (cos, sin) table
[positions, 64]in FP32, which the
indexer_uses_fp4(vllm_config)
¶
The index K cache on SM80: always MXFP4, the only one its kernels write and score. An explicit FP8 request is rejected.
Source code in vllm/models/deepseek_v41/nvidia/specialized_decode/ampere/__init__.py
is_ampere()
¶
Whether the current device runs DeepSeek-V4.1 through this package.
Source code in vllm/models/deepseek_v41/nvidia/specialized_decode/ampere/__init__.py
rope_table_f32(rotary_emb)
¶
The RoPE (cos, sin) table [positions, 64] in FP32, which the
kernels read; a table of another dtype is converted once.