vllm.models.deepseek_v41.amd.mono.layer
¶
The mono decode layer's two launches (README): K1 = the attention seam
(stages.seam: slice, gate, the norm into vLLM's MXFP8) + the attention front
(attention.front); K2 = the attention back (attention.back, wo_b pushed
to every rank) + the FFN seam (the TP sum folded in) + the MoE (stages.moe,
its all-reduce in-kernel too). A layer whose attention vLLM runs takes the FFN
launch instead: K2 with the attention back replaced by a push of vLLM's
unreduced wo_b output (seam.stage_push).
Tags: K1's hand-offs carry 2 e + 1, K2's 2 e + 2 (e: the launch pair's epoch, moved on by K2's CTA 0 at its end), so K2's seam may reuse K1's seam regions. Peer regions alternate by e's parity; every rank runs the same launches, so the epochs agree across ranks.
Classes:
Functions:
-
build_mono_ffn–The FFN launch of a layer whose attention vLLM ran (
part: this rank's -
peer_half_bytes–One parity's peer regions: the attention's partials, then the MoE's.
-
publish_x8–Token t's two XRDY flags (one a wqkv K half) once its X8 / X8S landed.
-
run_ffn–K2's FFN half on this CTA: the FFN seam (the attention's TP partials --
-
scratch_layout–Every region of step width s's launches -> (byte offset, bytes), disjoint:
-
store_normed–The FFN seam's norm output -> the MoE input row (bf16, device scope:
-
store_x8–The attention seam's norm output (
seam.stage_norm's 8 columns at
MonoBuild
dataclass
¶
Attributes:
-
tuned(bool) –A target layer of compress ratio 2 at TP2 on gfx950: its tuned paths
-
tuned_seam(bool) –K2's and the FFN launch's FFN seam: tuned wherever K1 is (on the
Source code in vllm/models/deepseek_v41/amd/mono/layer.py
tuned
property
¶
A target layer of compress ratio 2 at TP2 on gfx950: its tuned paths
are on (bit-exact against the generic ones). Its scratch layout is its
own (scratch_layout).
tuned_seam
property
¶
K2's and the FFN launch's FFN seam: tuned wherever K1 is (on the generic scratch layout at TP4 / TP8).
_ffn_symbol_params(key)
¶
The FFN launch's symbol parameters. The timeline flag tells a timeline build apart, as in K1's and K2's symbols. The draft's build adds its expert count, so that a profile tells the two FFN launches apart.
Source code in vllm/models/deepseek_v41/amd/mono/layer.py
_symbol_params(key)
¶
The kernel symbol's draft-only parameters (none for a target layer).
Source code in vllm/models/deepseek_v41/amd/mono/layer.py
build_mono_ffn(key)
¶
The FFN launch of a layer whose attention vLLM ran (part: this rank's
unreduced wo_b output, bf16 [S, HIDDEN]): K2's FFN half, the partial pushed
to every rank first. It takes a launch pair's epoch, as K2 does.
Source code in vllm/models/deepseek_v41/amd/mono/layer.py
1284 1285 1286 1287 1288 1289 1290 1291 1292 1293 1294 1295 1296 1297 1298 1299 1300 1301 1302 1303 1304 1305 1306 1307 1308 1309 1310 1311 1312 1313 1314 1315 1316 1317 1318 1319 1320 1321 1322 1323 1324 1325 1326 1327 1328 1329 1330 1331 1332 1333 1334 1335 1336 1337 1338 1339 1340 1341 1342 1343 1344 1345 1346 1347 1348 1349 1350 1351 1352 1353 1354 1355 1356 1357 1358 1359 1360 1361 1362 1363 1364 1365 1366 1367 1368 1369 1370 1371 1372 1373 1374 1375 1376 1377 1378 1379 1380 1381 1382 1383 1384 1385 1386 1387 1388 1389 1390 1391 1392 1393 1394 1395 1396 1397 1398 1399 1400 1401 1402 1403 1404 1405 1406 1407 1408 1409 1410 1411 1412 1413 1414 1415 1416 1417 1418 1419 1420 1421 1422 1423 1424 1425 1426 1427 1428 1429 1430 1431 1432 1433 1434 1435 1436 1437 1438 1439 1440 1441 1442 1443 1444 1445 | |
peer_half_bytes(tp)
¶
One parity's peer regions: the attention's partials, then the MoE's.
publish_x8(c, t)
¶
Token t's two XRDY flags (one a wqkv K half) once its X8 / X8S landed.
Source code in vllm/models/deepseek_v41/amd/mono/layer.py
run_ffn(key, lds, a, amb, bases, own, rank, ep, part=None, tl=None, fuse=False)
¶
K2's FFN half on this CTA: the FFN seam (the attention's TP partials --
part pushed to every rank's ATTN region first -- summed and folded into
the residual), the MoE and its all-reduce into a["out"], then the
launch pair's epoch moved on. tl: K2's timeline record base (a
timeline build), stamped from point K2_SEAM_POINT on.
Source code in vllm/models/deepseek_v41/amd/mono/layer.py
696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 | |
scratch_layout(s, tp, experts=EXPERTS, topk=TOPK, ug_groups=2, tuned=False, front=False)
¶
Every region of step width s's launches -> (byte offset, bytes), disjoint:
K1's front and seam, K2's back, the MoE's regions, the normed rows and their
flags. Each width has a scratch of its own (DSV41MonoLayer.scratch):
under another width's layout a mailbox pair's tag word could be plain data,
which a poll may take for a current tag. The MoE's counters, whose slots
outlive a step width, are in the epoch buffer. The MoE regions follow the
runner's routing and FP4 groups of an ug task: with one group a task (the
DSpark draft's runner) the ug flags take twice the room of two groups.
tuned: a tuned build's layout (MonoBuild.tuned), in a scratch of its
own.
Source code in vllm/models/deepseek_v41/amd/mono/layer.py
store_normed(normed, t, col, ys, live, wide=False)
¶
The FFN seam's norm output -> the MoE input row (bf16, device scope:
this launch reads it). wide: the 8 values as one 16-byte store (a store
each kept the publish's drain behind 24 of them).
Source code in vllm/models/deepseek_v41/amd/mono/layer.py
store_x8(c, t, col, ys, live)
¶
The attention seam's norm output (seam.stage_norm's 8 columns at
col) -> vLLM's MXFP8 of each 32 (the attention's input quant) -> the
front's X8 / X8S, plain at device scope.