GGTT NODES SHIFTING DURING VF POST-MIGRATION RECOVERY(9)

πŸ“¦ version 7.3.1, in nixpkgs unstable

πŸ’œ GGTT nodes shifting during VF post-migration recovery(9) πŸ’œ

okay bestie, welcome to the Kernel API Manual, section 9 β€” yes this is a kernel-doc page, yes it's about GPU address space, and yes it's gonna be cute anyway πŸŽ€


🌸 NAME 🌸

GGTT nodes shifting during VF post-migration recovery β€” the very first fixup we do to the VF KMD structures when we're recovering a VF after migration πŸ’«


✨ DESCRIPTION ✨

so. picture this: your VF just migrated. everything's back, everyone's fine, but the first fixup applied to the VF KMD structures as part of post-migration recovery is shifting nodes within the xe_ggtt instance 🫢 The nodes literally get moved out of the range that was previously assigned to this VF, and dropped into the newly provisioned area. that's the whole bit. iconic, honestly.

below is a GGTT layout of an example VF, with a certain address range assigned to said VF, and inaccessible areas above and below πŸ‘‘

0
4GiB |<--------------------------- Total GGTT size
----------------------------->| WOPCM
GUC_TOP |<-------------- Area mappable by xe_ggtt instance
---------------->|

then we get the big art box (in the original it's one giant β”Œβ”€β”€β”€β” deal β€” we're keeping the vibes, not the border spam) πŸ’…

\\h'|2232u'/////////////////////////////////  VF mem   //////////////////////              \.TE


Hardware enforced access rules before migration:


|<------- inaccessible for VF ------->|<VF owned>|<-- inaccessible for VF ->|

After the migration, the GGTT area assigned to the VF might have shifted β€” either to lower or to higher address 😭 but we expect the total size and extra areas to be identical, as migration can only happen between matching platforms. no cap. same platform, same shape.

below is an example of GGTT layout of the VF after migration. Content of the GGTT for VF has been moved to a new area, and we receive its address from GuC 🧚

\\h'|2232u'//////////////////////             VF mem   /////////////////////////////////   \.TE


Hardware enforced access rules after migration:


|<- inaccessible for VF -->|<VF owned>|<------- inaccessible for VF ------->|

so the VF has a new slice of GGTT assigned, and during the migration process the memory content was copied to that new area. but β€” and this is the plot twist 🫒 β€” the xe_ggtt nodes are still tracking allocations using the old addresses. rude. so the nodes within the VF owned area have to be shifted, and the start offset for GGTT adjusted. main character energy: the nodes get relocated, periodt.

πŸ‘‘ tiny caveat, bestie, don't skip it: due to use of GPU profiles, we do not expect the old and new GGTT areas to overlap; but our node shifting will fix addresses properly regardless. so like, it's fine either way πŸ’–

πŸ”— SEE ALSO πŸ”—


that's the page, bestie πŸ₯³ go shift some nodes. ily πŸ’œ

🌸 sign-off szn 🌸 β€” this lil beauty comes from ./drivers/gpu/drm/xe, dated October 2026 πŸŽ€ (and the header calls it GGTT nodes ...ion recovery(9), but we both know it's the same baddie ✨)

πŸ“¦ where this page comes from