vllm.models.deepseek_v41.nvidia.specialized_decode.runtime.model_state
¶
The model states of the specialized decode and of SM80 and SM90.
DeepseekV41MetadataGraphModelState is DeepseekV41ModelState plus:
- CUDA graphs of the attention metadata builds of FULL decode steps
(
metadata_graphs), recorded with the inner CUDA graphs and replayed in place of the eager build. A step that replays one gathers its engram lookback windows on a side stream beside it, so the metadata graph hands off to the forward's graph with no eager kernel between them; - no replay-start upload for decode steps whose starts are known to be zero.
SM80 and SM90 use it as is. DeepseekV41SpecializedDecodeModelState adds
the decode executors of the target and of the DSpark draft model, built and
warmed up once the weights are loaded (the model runner creates the model
state after loading the target and draft models, and before memory
profiling).
Classes:
-
DeepseekV41MetadataGraphModelState–DeepseekV41ModelStatewith recorded metadata builds of FULL decode
Functions:
-
padded_decode_batch–A dummy uniform decode batch padded like a runtime step of
key. Like
DeepseekV41MetadataGraphModelState
¶
Bases: DeepseekV41ModelState
DeepseekV41ModelState with recorded metadata builds of FULL decode
steps. Its users run without micro-batching.
Source code in vllm/models/deepseek_v41/nvidia/specialized_decode/runtime/model_state.py
71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 | |
_join_staged_lookback()
¶
Make the current stream wait for a staged lookback gather, if any; returns the batch it gathered.
Source code in vllm/models/deepseek_v41/nvidia/specialized_decode/runtime/model_state.py
_stage_lookback(input_batch)
¶
Issue the batch's lookback gather on a side stream, beside the metadata graph about to replay. It reads the batch's request indices and the token history, which the runner has written by now.
Source code in vllm/models/deepseek_v41/nvidia/specialized_decode/runtime/model_state.py
padded_decode_batch(key, input_buffers, max_model_len)
¶
A dummy uniform decode batch padded like a runtime step of key. Like
a capture, it takes worst-case sequence lengths, which bound host values
only, so its recorded build serves every step of the key.