vllm.model_executor.layers.mamba.abstract
¶
Classes:
-
MambaBase–Base class for Mamba-like layers which support the v1 engine.
Functions:
-
get_kv_cache_checkpoint_alignment–The checkpoint alignment the Mamba layers in
layer_namesshare. -
get_mamba_num_speculative_blocks–State blocks each request reserves for draft tokens.
-
get_mamba_prefill_checkpoint_alignment–LCM of the Mamba layers' prefill checkpoint alignments.
-
get_mamba_prefill_checkpoint_grid–Token grid every Mamba layer keeps its prefill checkpoint offsets on.
MambaBase
¶
Bases: AttentionLayerBase
Base class for Mamba-like layers which support the v1 engine. Inherit from this class if you implement a custom layer.
Methods:
-
bind_kv_cache–Unpack a raw
[B, 1, 1, C]int8 page view into per-state views. -
get_attn_backend–Get the attention backend class for this Mamba layer.
-
get_kv_cache_checkpoint_alignment–Token alignment of the prefill checkpoints this layer's KV cache
-
get_prefill_checkpoint_alignment–Token alignment of the checkpoint offsets this layer can export from
-
get_state_shape–Defines the shape of the state.
Source code in vllm/model_executor/layers/mamba/abstract.py
bind_kv_cache(kv_cache)
¶
Unpack a raw [B, 1, 1, C] int8 page view into per-state views.
Each block's C bytes hold the layer's states (e.g. conv, ssm)
packed by dtype/shape with the layer's state alignment.
Source code in vllm/model_executor/layers/mamba/abstract.py
get_attn_backend()
¶
get_kv_cache_checkpoint_alignment(vllm_config)
¶
Token alignment of the prefill checkpoints this layer's KV cache reserves a block for, or None to reserve none. Read by both the KV cache manager config and the layer's metadata builder.
Source code in vllm/model_executor/layers/mamba/abstract.py
get_prefill_checkpoint_alignment()
¶
Token alignment of the checkpoint offsets this layer can export from inside a prefill, if any; in align mode, the platform makes Mamba blocks multiples of it.
Source code in vllm/model_executor/layers/mamba/abstract.py
get_state_shape()
abstractmethod
¶
Defines the shape of the state. For mamba layers this is usually a (conv_state, ssm_state) tuple. In this case, returns (conv_state_shape, ssm_state_shape).
Source code in vllm/model_executor/layers/mamba/abstract.py
get_kv_cache_checkpoint_alignment(vllm_config, layer_names)
¶
The checkpoint alignment the Mamba layers in layer_names share.
Source code in vllm/model_executor/layers/mamba/abstract.py
get_mamba_num_speculative_blocks(vllm_config)
¶
State blocks each request reserves for draft tokens.
Source code in vllm/model_executor/layers/mamba/abstract.py
get_mamba_prefill_checkpoint_alignment(vllm_config)
¶
LCM of the Mamba layers' prefill checkpoint alignments.
None unless every Mamba layer declares one: the scheduler takes internal checkpoints only when every Mamba group can export them.
Source code in vllm/model_executor/layers/mamba/abstract.py
get_mamba_prefill_checkpoint_grid(vllm_config)
¶
Token grid every Mamba layer keeps its prefill checkpoint offsets on.
Prefix-cache boundaries, made multiples of the layers' checkpoint alignment (the platform already keeps blocks on it). The scheduler checks offsets against one alignment, so every Mamba spec must use this one.