Skip to content

vllm.models.mistral.mistral_large4.amd.specialized_decode.mono.common

Mono decode: a model's decode step as one persistent kernel per layer, with the tensor-parallel reductions inside the kernel.

This package holds what every mono model shares -- the mechanisms: runtime (host: TP consensus, peer memory, kernel argument tables, compile-only builds), plan (plain Python: the execution model, build keys, the traced hand-off contract) and device (traced inside kernels: the tagged mailbox, wave / math primitives). The kernels, their layouts and schedules stay in the model's own mono package. It imports no model and holds no model constant.

Modules:

  • device –

    Device-side mechanisms every mono kernel shares, one module each: ops

  • plan –

    Build-time contracts of a mono kernel, plain Python (no torch, no FlyDSL):

  • runtime –

    Host-side mechanisms of a mono runner, one module each (imported directly, so

  • weights –

    Host-side weight preparation every mono kernel needs: the storage