vllm.models.minimax_m3.common.sparse_attention
¶
Main block-sparse GQA attention for MiniMax M3 sparse layers.
The lightning indexer (indexer.py) selects the top-k KV blocks (written into
the shared layer.topk_indices_buffer); this module holds the main attention
that attends only to those blocks: the paged K/V cache backend, its metadata +
builder, and the impl that reads the indexer's top-k from that buffer. The generic
Triton attend kernel lives in common/ops; Hopper uses the query-tiled prefill
kernel in nvidia/ops. The SM100 (MSA)
build_k2q_csr + sparse_atten_func attend lives in
nvidia/sparse_attention_msa.py.
MiniMaxM3SparseBackend and MiniMaxM3SparseMetadata are referenced by the
attention-backend registry (by dotted path) and by spec-decode, so they must
keep these names and stay in this module.
Classes:
-
MiniMaxM3SparseBackend–Block-sparse GQA backend for MiniMax M3 sparse attention layers.
-
MiniMaxM3SparseDecodeMetadata–Per-decode state (cudagraph-safe).
decode_query_lenis the uniform -
MiniMaxM3SparseImpl–Abstract base for block-sparse GQA over the indexer-selected blocks.
-
MiniMaxM3SparseMetadata–Sparse-attention metadata, split into prefill and decode sub-metadata.
-
MiniMaxM3SparsePrefillMetadata–Per-prefill state;
cu_seqlens_k/total_kv_blocksfeed the MSA CSR. -
MiniMaxM3SparseTritonImpl–Triton block-sparse attend (
minimax_m3_sparse_attn) + Triton decode.
Functions:
-
minimax_m3_query_token_positions–Map each prefill query token to its request id and absolute KV position.
-
minimax_m3_rebase_slots_to_page16–Rebase a token slot mapping onto AITER's page-16 page numbering.
-
minimax_m3_use_aiter_sparse_pa–Whether to use the ROCm AITER page-16 sparse PA prototype.
-
minimax_m3_use_flashinfer_msa–Whether to use the flashinfer.msa_ops impls on SM90 or SM120/SM121.
-
select_main_backend_and_impl_cls–Pick the main attention backend and implementation.
-
select_main_impl_cls–Backward-compatible implementation-only selector.
MiniMaxM3SparseBackend
¶
Bases: AttentionBackend
Block-sparse GQA backend for MiniMax M3 sparse attention layers.
Source code in vllm/models/minimax_m3/common/sparse_attention.py
MiniMaxM3SparseDecodeMetadata
dataclass
¶
Per-decode state (cudagraph-safe). decode_query_len is the uniform
per-request query length (1, or 1 + num_speculative_tokens); it is 0 for
a ragged batch, whose query lengths come from query_start_loc.
Source code in vllm/models/minimax_m3/common/sparse_attention.py
MiniMaxM3SparseImpl
¶
Bases: AttentionImplBase[MiniMaxM3SparseMetadata]
Abstract base for block-sparse GQA over the indexer-selected blocks.
Inherits AttentionImplBase for a custom forward signature (the layer
pre-inserts K/V and runs the indexer, which writes the selected blocks into
the shared layer.topk_indices_buffer; the attend reads them back from
there). The Triton and MSA subclasses each own a full forward -- no
shared forward code.
Methods:
-
forward–Attend the queries to the indexer-selected blocks. Per kernel.
Source code in vllm/models/minimax_m3/common/sparse_attention.py
forward(layer, query, kv_cache, output)
¶
Attend the queries to the indexer-selected blocks. Per kernel.
query is E4M3 for impls with use_cutlass_decode, else the model
dtype.
The indexer has already written the top-k block ids into
layer.topk_indices_buffer (decode at [:, :nd], prefill at
[:, nd:num_tokens]); the attend reads them from there.
Source code in vllm/models/minimax_m3/common/sparse_attention.py
MiniMaxM3SparseMetadata
dataclass
¶
Bases: AttentionMetadata
Sparse-attention metadata, split into prefill and decode sub-metadata.
Source code in vllm/models/minimax_m3/common/sparse_attention.py
MiniMaxM3SparsePrefillMetadata
dataclass
¶
Per-prefill state; cu_seqlens_k/total_kv_blocks feed the MSA CSR.
Source code in vllm/models/minimax_m3/common/sparse_attention.py
MiniMaxM3SparseTritonImpl
¶
Bases: MiniMaxM3SparseImpl
Triton block-sparse attend (minimax_m3_sparse_attn) + Triton decode.
Source code in vllm/models/minimax_m3/common/sparse_attention.py
490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 | |
minimax_m3_query_token_positions(cu_seqlens_q, prefix_lens, total_q)
¶
Map each prefill query token to its request id and absolute KV position.
Both tensors are the same for every sparse layer in a step, so the AITER sparse PA block-table builder reads them from the metadata instead of rebuilding them once per layer.
Source code in vllm/models/minimax_m3/common/sparse_attention.py
minimax_m3_rebase_slots_to_page16(slot_mapping, block_size, out=None)
¶
Rebase a token slot mapping onto AITER's page-16 page numbering.
AITER's KV writer derives the destination page from slot // 16 and
assumes pages are numbered consecutively. When the resolved layout stores
both K/V sides inside a block, a block spans twice as many pages, so only
the block component of the slot doubles -- the offset within the page must
not move. Clamping before the division leaves padding slots at their
negative sentinel, which is what tells the writer to skip them.
Source code in vllm/models/minimax_m3/common/sparse_attention.py
minimax_m3_use_aiter_sparse_pa(num_kv_heads, *, emits_sparse_block_table=False)
¶
Whether to use the ROCm AITER page-16 sparse PA prototype.
More than one KV head per rank is only served when the layer's indexer emits the attend's page table itself, since the Triton builders this path otherwise falls back to address one head's cache. Whether an indexer does that is a ROCm-side fact, so its caller passes it in.
Source code in vllm/models/minimax_m3/common/sparse_attention.py
minimax_m3_use_flashinfer_msa(*, topk_blocks, kv_cache_dtype, num_heads, num_kv_heads, indexer_kv_dtype, model_dtype)
¶
Whether to use the flashinfer.msa_ops impls on SM90 or SM120/SM121.
One predicate feeds both selectors so the FlashInfer indexer and attend engage together or not at all. Mixed FlashInfer/Triton pairs are unsupported.
Source code in vllm/models/minimax_m3/common/sparse_attention.py
select_main_backend_and_impl_cls(*, topk_blocks, kv_cache_dtype, num_kv_heads, emits_sparse_block_table=False, use_flashinfer_msa=False)
¶
Pick the main attention backend and implementation.
Blackwell (SM100) uses the MSA attend for supported top-k block counts
when the KV cache is BF16, FP8 E4M3 or NVFP4; SM90 and SM120/SM121 use the
FlashInfer MSA attend when the caller passes the
minimax_m3_use_flashinfer_msa verdict; MI355 uses AITER sparse PA with
shuffle KV cache layout; Other platforms and FP8 E5M2 fall back to Triton.
NVFP4 is MSA-only. The MSA/FlashInfer modules are imported lazily to avoid
import errors on unsupported platforms.
Source code in vllm/models/minimax_m3/common/sparse_attention.py
613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 | |
select_main_impl_cls(*, topk_blocks, kv_cache_dtype, num_kv_heads, emits_sparse_block_table=False, use_flashinfer_msa=False)
¶
Backward-compatible implementation-only selector.