Starting with Mesa 27.0, all Mali GPUs with Valhall (v9) or newer will use the new, faster compiler we’ve been building for the Panfrost driver stack, Kraid. After a little under six months of development, Kraid is now the default in Mesa’s main branch. Mesa 26.3 will be the last release with the old compiler.

Why did we write a new compiler? The very short answer is that there were some serious design flaws in the old compiler that made it very hard to properly optimize for Mali GPUs. You can read a little more about this in my previous blog post or you can check out our talk at the X.org Developer's Conference.

The biggest change with Kraid is that it's able to do a much better job of modeling the way register access works on Mali GPUs. The old compiler was emitting a lot of unnecessary instructions that just move data around because of its inability to model swizzles accurately. Kraid is also much better at handling 64-bit data and has full support for 64-bit integer arithmetic. This is especially important for OpenCL where 64-bit pointer arithmetic is ubiquitous. The result of this is that Kraid is shaving nearly 20% of instructions off our corpus of captured OpenGL shaders:

total instrs in shared programs: 714433 -> 582468 (-18.47%)
instrs in affected programs: 710788 -> 578823 (-18.57%)
helped: 7038

Kraid also emits much fewer floating point conversion instructions. Mali GPUs have a feature where most 32-bit floating-point arithmetic instructions can read 16-bit float sources and do the conversion to 32-bit for free. They can also usually do a conversion down to 16-bit as part of the same instruction if needed. Kraid's new swizzle modeling is able to take full advantage of this and elide most floating-point conversion instructions by folding them into the source or destination of some arithmetic instruction. For some workloads, especially those coming from OpenGL ES, the number of floating point conversions can really add up. But probably the best demonstration of this is the benchmark results from clpeak (v10, RK3588):

| benchmark                                           | Kraid/Old |
|-----------------------------------------------------|-----------|
| Single-precision compute (float)                    | 99.77%    |
| Single-precision compute (float2)                   | 95.95%    |
| Single-precision compute (float4)                   | 89.96%    |
| Single-precision compute (float8)                   | 90.19%    |
| Single-precision compute (float16)                  | 90.26%    |
| Half-precision compute (half)                       | 99.83%    |
| Half-precision compute (half2)                      | 99.97%    |
| Half-precision compute (half4)                      | 94.58%    |
| Half-precision compute (half8)                      | 87.26%    |
| Half-precision compute (half16)                     | 87.62%    |
| Mixed-precision compute fp16xfp16+fp32 (mp)         | 149.37%   |
| Mixed-precision compute fp16xfp16+fp32 (mp2)        | 164.54%   |
| Mixed-precision compute fp16xfp16+fp32 (mp4)        | 147.07%   |
| Mixed-precision compute fp16xfp16+fp32 (mp8)        | 152.75%   |
| Mixed-precision compute fp16xfp16+fp32 (mp16)       | 152.45%   |

With Kraid, we've also used a different register allocation and spilling strategy. We maintain static single assignment (SSA) form throughout the compile process, all the way to register allocation. One of the advantages of SSA-based register allocation is that it allows exact register pressure estimates before register allocation. This, in turn, allows us to do spilling before register allocation, while still in SSA form. Spilling in SSA form gives the spiller a lot more information about how values are used and lets us make better spill choices. The result is around an 80% reduction in spilling across our Vulkan shader database. This is important because every spill is a trip to memory and, on a mobile GPU, memory access is incredibly expensive.

Totals from 7098 (95.45% of 7436) affected shaders:
Instrs: 1424643 -> 1144579 (-19.66%); split: -19.91%, +0.25%
Spill count: 66682 -> 10454 (-84.32%); split: -84.47%, +0.15%
Fill count: 111898 -> 23988 (-78.56%); split: -78.57%, +0.01%
Estimated spill cost: 5585480 -> 887580 (-84.11%); split: -84.11%, +0.00%

SSA-based spilling and register allocation are also both more or less linear-time algorithms, much faster than the quadratic or cubic run time of conventional graph coloring algorithms. The result is that our Vulkan pipeline database compiles about 7x faster with Kraid and uses 35% less memory.

387.32 user 5.86 system 0:23.68 elapsed 1659%CPU (1794216 max resident)
2729.18 user 6.35 system 8:14.90 elapsed 552%CPU (2783724 max resident)

We've also taken the opportunity with Kraid to implement some optimizations for fragment shaders that we've had on the to-do list for a long time. One of these is taking advantage of the varying interpolation hidden register. This allows the hardware to avoid duplicate calculations when interpolating multiple varyings at the top of a fragment shader, saving power and making interpolation faster. For most things, this is a minor saving but it improved the performance of certain Candy Crush traces by 40%! Another is that we're now doing texture instruction fusion whenever we can. Mali hardware allows you to fuse a varying interpolation with a texture instruction in certain circumstances, avoiding a round-trip to the message unit, and possibly enabling other hardware optimizations. There's a similar optimization allowing us to fuse two texture instructions which are identical except they target different textures. This is fairly common in older apps which often use one texture for color and a second for a normal map.

All told, actual app performance is about the same or maybe a few percent better. But this is no surprise since apps are rarely shader execution bound on mobile GPUs. Things like memory bandwidth typically have a much larger impact than deleting a little shader math. However, in the world of GPU driver optimization, even a couple percent is a huge win.

# Vulkan:
| benchmark             | Kraid/Old FPS |
|-----------------------|---------------|
| Basemark BATS         | 100.42%       |
| 3DMark Wildlife       | 106.51%       |
| Super Fantasy Kingdom | 104.77%       |
| Sky Children of Light | 96.57%        |
| GFXBench Aztec Ruins  | 104.38%       |
| Roblox                | 99.73%        |
| Unity Viking Village  | 99.84%        |
| Android Contacts      | 101.22%       |
| Android Gallery       | 103.85%       |
| Android Maps          | 102.16%       |

# GLES:
| benchmark             | Kraid/Old FPS |
|-----------------------|---------------|
| HalfLife              | 100.49%       |
| CandyCrush Soda-Saga  | 139.36%       |
| Firefox               | 99.13%        |
| Genshin Scene 1       | 104.43%       |
| eBay Product Browsing | 106.38%       |
| GFXBench Manhattan    | 104.32%       |
| Honkai: Star Rail     | 102.88%       |
| CallOfDutyMobile      | 117.98%       |
| GFXBench Aztec        | 110.58%       |
| PUBG                  | 102.56%       |
| Genshin Scene 2       | 106.18%       |
| DeusEx                | 99.55%        |
| GFXBench TRex         | 103.94%       |
| Netflix UI            | 106.96%       |
| ApexLegends           | 105.34%       |

All in all, we're hopeful that Kraid will give us a much more stable base for compiler development going forward. We still have work to do (compiler work never ends) but we're fairly sure that most of the foundational problems with the old compiler stack are now solved. Not shown in the benchmarks is the fact that the whole compiler stack is now far more robust. While Kraid isn't bug-free, it has a lot fewer hidden dragons. And the performance numbers speak for themselves.

Related Posts

Kraid is a now a real compiler

Kraid: A new compiler for Panfrost

Re-thinking framebuffers in PanVK

Related Posts

Kraid is a now a real compiler

Kraid: A new compiler for Panfrost

Re-thinking framebuffers in PanVK