Welcome back to our series on writing a Rust GPU kernel driver! In the previous part we looked at Mali's CSF architecture and explained how panvk creates scheduling groups: the KMD binds them to the finite CSG and CS slots exposed by the firmware, and the MCU consumes command streams from ring buffers after the KMD rings a doorbell.

Two things were deliberately deferred in that article, and both were really promises about memory. First, every group carries a vm_id, because command streams dereference GPU virtual addresses, and those addresses must resolve in the right per-client address space. Second, booting the MCU requires mapping the firmware sections at the exact virtual addresses the firmware expects. It is now time to make good on those promises: in this part, we will see how GPU memory is mapped, how it is allocated in the first place, and why the driver cannot even boot its firmware without it — in fact, allocating GPU memory is one of the most important jobs of a GPU kernel driver.

It's buffers all the way down

Before we look at any of the mechanics, it is worth pausing to consider how central GPU memory is to everything the UMD does. Let's briefly return to our VkCube example from the second part of this series: the geometry describing the cube, the textures applied to its faces, the shaders that place and rotate it, and the matrix data driving the animation all have to live somewhere the GPU can reach. That somewhere is a buffer. In fact, practically everything a GPU consumes or produces is stored in buffers, and it is up to the KMD to hand them out.

This is true even for the commands themselves. As we discussed in the previous part, a GPU is best understood as a command processor: it executes streams of hardware-specific instructions telling it which state to set and which draws to carry out, and it reads these streams from GPU memory just like it reads any texture. This means that when panvk records a VkCommandBuffer, the instructions it emits are ultimately written into a buffer as well. The same applies to the ring buffers consumed by the firmware and — as we will see by the end of this post— to the firmware itself.

This hierarchy is also visible in the Vulkan API. A VkDeviceMemory allocation performed through vkAllocateMemory is backed by a buffer object — or BO — allocated by the KMD. VkBuffers and VkImages are then bound to ranges of that memory, and it is these objects that hold most of the state needed to render: vertex data, textures, uniforms and descriptors, while the command buffers that reference them live in buffers of their own, allocated internally by the UMD. Without a way to be given GPU memory, a userspace driver like panvk simply cannot function: there would be nowhere to place the state needed to draw a scene or carry out compute jobs.

Binding memory to virtual addresses

We have established that the right sections of the firmware must be mapped at precise virtual addresses — which we will simply refer to as VA from now on — in order for the firmware to boot.

Mapping a particular region of physical memory at a given GPU virtual address is known as a VM_BIND operation, a name inherited from the first driver that introduced this capability in the Linux kernel. These can either be done synchronously — in which case the code has to block until the operation completes — or asynchronously, in which case the operation is treated like a job submission like any other, which carries the same signaling mechanisms used for jobs.

In other words, VM_BIND lets us bind a given section of a buffer object to a range of virtual addresses, and it is this collection of mappings that makes up a VM. The most immediate use case for the VM_BIND operation during driver boot-up is to properly map the firmware sections to the right VA ranges so the firmware can find them.

To do so, the KMD must modify the device's Input-Output Memory Management Unit (or IOMMU) — a hardware unit that keeps track of mappings between virtual addresses and their corresponding physical addresses. In this sense, an IOMMU is similar to a regular MMU, which is the hardware unit that performs the same translation for all CPU memory accesses. The difference is that the IOMMU performs translations for device virtual addresses, so the resulting mappings can be used by other cores in the system to access memory, whereas regular VAs can only be used by the CPU itself. This means that the IOMMU can be used to create mappings that can be used by the GPU hardware to refer to physical memory locations.

On this hardware, the translation tables themselves are ARM64 LPAE stage-1 page tables — the same format the CPU uses on arm64. This means Tyr does not hand-roll its page-table code: it builds and updates the tables through the kernel's io-pgtable framework, using 4 KiB pages and, where alignment and size allow, 2 MiB blocks.

The KMD can use the IOMMU facilities to build the required memory mappings, performing any relevant cache flushes along the way. The mappings built this way are only valid for a given address space, which represents a hardware-enforced isolation mechanism. At a later point, the KMD can create a particular set of mappings by telling the hardware which of the previously set up address spaces to load.

Mali hardware only allows for a limited number of such address spaces (which we will simply refer to as AS from now on) to be active at a time — up to sixteen, with the exact count depending on the hardware — where the first one (AS0) is permanently occupied by the firmware itself. Therefore, if more contexts are executing simultaneously than available AS slots, the KMD must mediate access to the underlying hardware slots by evicting a previously resident context and flushing its caches.

For a group's work to execute, its VM must therefore be resident in one of these slots, and the firmware is told which slot the group's VM occupies when the group is bound to a CSG slot. This is also where memory faults surface: if the GPU dereferences an address with no valid mapping, the MMU reports the guilty address space, and the kernel driver disables that address space and marks the VM as unusable — a state userspace can observe through DRM_PANTHOR_VM_GET_STATE. A VM that has faulted is dead, as the kernel provides no way to revive it. In fact, this is exactly how panvk implements Vulkan's device-loss semantics: its device status check queries the VM state and, upon finding the VM unusable, marks the VkDevice as lost and reports VK_ERROR_DEVICE_LOST to the application, whose only way forward is to recreate the device — and, with it, a fresh VM.

Additionally, the VM_BIND operation is also available to be used by the UMD. In Panthor's userspace API, this is encoded by means of the following argument:

/**
 * struct drm_panthor_vm_bind_op - VM bind operation
 */
struct drm_panthor_vm_bind_op {
	/** @flags: Combination of drm_panthor_vm_bind_op_flags flags. */
	__u32 flags;

	/**
	 * @bo_handle: Handle of the buffer object to map.
	 * MBZ for unmap or sync-only operations.
	 */
	__u32 bo_handle;

	/**
	 * @bo_offset: Buffer object offset.
	 * MBZ for unmap or sync-only operations.
	 */
	__u64 bo_offset;
	
	/**
	 * @va: Virtual address to map/unmap.
	 * MBZ for sync-only operations.
	 */
	__u64 va;

	/**
	 * @size: Size to map/unmap.
	 * MBZ for sync-only operations.
	 */
	__u64 size;

	/**
	 * @syncs: Array of struct drm_panthor_sync_op synchronization
	 * operations.
	 *
	 * This array must be empty if %DRM_PANTHOR_VM_BIND_ASYNC is not set on
	 * the drm_panthor_vm_bind object containing this VM bind operation.
	 *
	 * This array shall not be empty for sync-only operations.
	 */
	struct drm_panthor_obj_array syncs;

};

/**
 * enum drm_panthor_vm_bind_flags - VM bind flags
 */
enum drm_panthor_vm_bind_flags {
	/**
	 * @DRM_PANTHOR_VM_BIND_ASYNC: VM bind operations are queued to the VM
	 * queue instead of being executed synchronously.
	 */
	DRM_PANTHOR_VM_BIND_ASYNC = 1 << 0,
};

/**
 * struct drm_panthor_vm_bind - Arguments passed to DRM_IOCTL_PANTHOR_VM_BIND
 */
struct drm_panthor_vm_bind {
	/** @vm_id: VM targeted by the bind request. */
	__u32 vm_id;

	/** @flags: Combination of drm_panthor_vm_bind_flags flags. */
	__u32 flags;

	/** @ops: Array of struct drm_panthor_vm_bind_op bind operations. */
	struct drm_panthor_obj_array ops;
};

Creating a context in the GPU

Notice what has been achieved here: each client is given an isolated context with its own view of GPU memory, much like each process on the CPU is given its own view of system memory through its page tables. A client can neither observe nor corrupt memory that belongs to somebody else, simply because foreign addresses do not resolve to anything in its own VM.

This is what panvk uses to back a VkDevice. In Vulkan, a VkDevice is essentially a hardware context: a private connection to the GPU that owns the application's queues, its memory and its state, in isolation from any other application talking to the same hardware. When panvk creates a VkDevice, it creates a VM to go with it, and the scheduling groups from the previous part — created with that VM's vm_id — tie the two together: whatever the device's queues execute, they execute inside that VM.

In practice, this is a two-step process. When a VkDevice is created, panvk first asks the KMD for a fresh VM through DRM_PANTHOR_VM_CREATE, receiving a vm_id in return. It then passes that vm_id in struct drm_panthor_group_create when creating the device's scheduling group — in fact, we can see this field being set in the create_group() listing from the previous part. This association is permanent: from that point on, every job submitted through the group's queues executes in the context of that VM, and every VA in its command streams is resolved through that VM's page tables.

Also note that when creating a VM, userspace sizes the region of the VA space it controls; the remainder is reserved for objects the kernel itself must place in the client's address space — the ring buffers and suspend buffers we met in the previous part all need GPU addresses too.

Note that BOs themselves are not inherently tied to a single VM. Unless a BO is created with an exclusive_vm_id — which forbids sharing at creation time — the same BO can be mapped into several VMs at once, and possibly at different VAs in each. This is what makes buffer sharing work, be it between two Vulkan applications or between the GPU and another device entirely. From the point of view of a given VM, such a BO is considered external: it is not owned by that VM, and the KMD must track it separately, as its lifetime and synchronization have to be coordinated with every other user of the buffer.

GPU Virtual Address Manager (GPUVM)

The ability to map memory to a given VA range is particularly useful to implement Vulkan Sparse Resources and thus, it is usually offered by many GPU drivers apart from Panthor and Tyr. This led to the creation of a kernel component known as GPUVM to house the shared logic needed to properly manage a GPU's virtual address space.

By using GPUVM, the KMD only needs to express its desire to map or unmap a range of the VA space, while GPUVM computes the required sequence of map, unmap, or remap operations to make it happen. The KMD needs to implement the hooks to properly effect these operations, of course. Again, it can make use of the aforementioned IOMMU map and unmap APIs to carry out the operations in hardware.

Remap operations are where the shared logic earns its place: unmapping a small range from the middle of a larger mapping is logically one operation, but physically a hole punch that leaves two mappings behind, one on each side. GPUVM decomposes this correctly, and the driver only has to apply the resulting steps to its page tables.

In this sense, GPUVM is a major component in the memory management code for a lot of KMDs in the Linux kernel, including Tyr. Being able to interface with it from Rust is therefore a requirement for writing a modern Rust GPU driver. When this series started, no such interface existed; today Tyr's VM layer is built on the Rust GPUVM abstraction, and the driver's map, unmap, and remap hooks are safe Rust code. In any case, it should be clear by now that using GPUVM is needed if we want to get the firmware to boot.

Tyr's virtual memory module summarizes this whole layer nicely:

//! GPU virtual memory management using the DRM GPUVM framework.
//!
//! This module manages GPU virtual address spaces, providing memory
//! isolation and the illusion of owning the entire virtual address (VA)
//! range, similar to CPU virtual memory. Each virtual memory (VM) area is
//! backed by ARM64 LPAE Stage 1 page tables and can be mapped into
//! hardware address space (AS) slots for GPU execution.

The operations offered by the Panthor userspace API to interface with VMs are DRM_PANTHOR_VM_CREATE, DRM_PANTHOR_VM_DESTROY, DRM_PANTHOR_VM_BIND, and DRM_PANTHOR_VM_GET_STATE.

Allocating GPU memory with GEM

So far, we have discussed the issue of mapping a particular region of physical memory at a given GPU virtual address, noting that it will be needed to get the firmware to boot. We haven't yet described how GPU memory is allocated in the first place.

As allocating memory is one of the most fundamental operations offered by a KMD, owing to its extensive use throughout the whole stack (i.e., both by the UMD and the KMD itself), we now devote our attention to the topic of allocating memory using the GEM memory manager in the kernel.

Unlike drivers that support hardware with its own VRAM chips, as is usually the case with discrete graphics cards, for example, both Panthor and Tyr are drivers that use system memory as GPU memory. In this configuration, GEM uses shmfs to allocate anonymous pageable memory as its backing storage in a refcounted object that can be mapped both in the kernel (see drm_gem_shmem_vmap) and in userspace (see drm_gem_create_mmap_offset).

It is worth spelling out this chain in full: GEM allocates its backing storage from shmfs, and shmfs, in turn, gets its pages from the kernel's page allocator, like any other user of anonymous memory. This is what makes GPU buffers swappable: from the memory-management subsystem's point of view, the pages backing a BO look just like the pages of a regular process, and can thus be written out to swap when the system is under memory pressure. Naturally, the GPU cannot tolerate its memory disappearing from under a running job, so the pages belonging to a context are pinned while that context is executing on the hardware.

As a side note, drivers that do manage dedicated VRAM usually reach for a different DRM component instead: TTM, which specializes in migrating buffers between system memory and VRAM, and in evicting them when VRAM runs out. We will not be discussing it any further, as Mali GPUs have no VRAM to manage and shmem-backed GEM objects are all we need.

Each GEM object can be assigned a unique (per-fd) handle through drm_gem_handle_create, which can then be subsequently used in VM_BIND operations by setting the following field in struct drm_panthor_vm_bind_op, which has already been presented above:

	 * @bo_handle: Handle of the buffer object to map.
	 * MBZ for unmap or sync-only operations.
	 */
	__u32 bo_handle;

By specifying the handle, the GEM object will be bound to the GPUVM instance represented by the drm_panthor_vm_bind_op::vm_id field through struct drm_gpuvm_bo. This operation is what provides the backing memory for the VA ranges specified in the VM_BIND call.

The ioctl offered by the Panthor userspace API to create GEM objects is DRM_PANTHOR_BO_CREATE, while DRM_PANTHOR_BO_MMAP_OFFSET is used as a proxy for drm_gem_create_mmap_offset.

Note that being able to interface with the GEM API from Rust GPU drivers is absolutely required. Not only to allocate GPU memory to the UMD, but also to carry out the other operations presented in this section. This interface exists today: Tyr's buffer objects are built on the kernel's drm::gem::shmem Rust abstraction, and the ioctl handlers above reduce to a handful of calls into it.

Booting the firmware is a memory-management problem

We can now retell the MCU boot story from the previous part with the full vocabulary. The firmware binary that Tyr loads is divided into sections, and each section declares the virtual address range it must occupy, along with its protection: code is read-only and executable, data is writable, and some regions are shared with the host and must be mapped uncached.

Boot, then, proceeds as follows:

1. Tyr creates a VM for the MCU. Being the first VM activated, it takes AS slot 0.
2. The firmware binary is parsed, and each loadable section becomes a kernel-owned buffer object — the same shmem-backed objects userspace allocates, only created by the driver itself.
3. Each of these objects is mapped into the MCU's VM at the exact VA the section header demands, with page-table attributes derived from the section's flags. This flows through the very same GPUVM and io-pgtable machinery as a userspace VM_BIND.
4. The section contents are copied in, and only then is the MCU released from reset. By the time the firmware executes its first instruction, every address it was linked against is already backed by a live page-table entry.

The firmware's VM keeps serving after boot: the shared interface region we explored in the previous part lives in it, and so do the ring buffers and suspend buffers the driver allocates as groups and queues come and go.

The dependency chain from the previous part should now feel concrete. VM and MMU support is not an optional feature next to the scheduling machinery; the driver cannot even boot its firmware without it.

Tying this back to upstream

Most of what this article describes is no longer just downstream code. At the time of writing, Tyr in the DRM Rust tree (drm-rust-next) has grown, in order: a generic slot manager, MMU support on top of the kernel's io-pgtable framework, a VM layer built on the Rust GPUVM abstraction, kernel-owned buffer objects, the firmware parser, and MCU boot. That is precisely the dependency chain laid out above, landing piece by piece through the upstream review process.

The Rust abstractions this work sits on — GPUVM and io-pgtable bindings among them — did not exist when this series started. They were developed and reviewed with Tyr as a concrete user, which is the strategy we described in the first post: it is much easier to motivate infrastructure work when there is an actual driver that depends on it.

What upstream Tyr does not yet expose is the userspace-facing side of this layer: the BO and VM ioctls, and everything that builds on them. Those follow the same path, with the downstream tree serving as the proving ground.

What's next

With memory management in place, one big topic remains: synchronization. The next part will cover DMA fences, DRM sync objects, and how Tyr schedules jobs. This is the last big facility offered by the kernel driver to let userspace carry out work on the GPU.