Skipping pixels and tearing down a 4K frame

We play and test Wreckwing on a 4K display. In the largest city at 4K, on the High preset, in Low Power Mode, the graphics card takes about 30 ms to draw a frame in our benchmark, which is about 33 frames a second. The biggest single step of that frame is the one that works out what every pixel looks like: what surface it shows, and how that surface is lit. It took 13.1 ms of the 30.

Taking the frame apart

That step runs in two stages. First every pixel records its surface (color, roughness, which way it faces, how far away), which graphics programmers call the G-buffer. Then a second pass lights each pixel from those records: the sun and its shadows, the sky, the street’s light bounced up onto walls, light from rooms behind broken windows, and the small moving lights of missiles, engines and fires.

To find where the 13.1 ms went, the benchmark can now leave one part out at a time. The picture comes out wrong, but the timing shows what that part costs. Each run flies the camera down the same avenue for 20 seconds, twice, in alternating order.

More than half of the lighting’s cost was the small moving lights. There are up to twelve of them, and every one of the eight million pixels on the screen checked each of them three times: for the light itself, for its glint on shiny surfaces, and for its glow in smoky air.

Most pixels are nowhere near a light

Each of these lights reaches from a few meters out to 16 m, a missile’s light the farthest. Seen from the street, almost every pixel is far outside every one of those reaches, so each pixel checked twelve lights and found that none of them touched it.

A light’s reach is a ball around it. Before the frame is drawn, the game now works out the rectangle on the screen that the ball could cover, padded a little so that it never misses a pixel. A pixel checks only the lights whose rectangle it falls inside.

A light added nothing to the pixels outside its rectangle before this change either, so the picture is unchanged. A test draws a wall lit by three lights (one in front of it, one 30 m off to the side, one behind the camera) both ways, and the two frames are identical to the byte. The step went from 13.8 to 11.8 ms on average over two runs each, about 2 ms of the 30 ms frame.

The recording also shows the limit. When the camera is inside a light’s reach, which happens often with the player’s own engines just ahead of the chase camera, that light’s rectangle is the whole screen and every pixel checks it. The other lights are still skipped.

What is left

With the lights done, we split the surface records the same way:

Part of the 2.9 ms is the noise that gives concrete its stains and mottling, worked out fresh at every pixel from several layers of random patterns. A throwaway build with the noise held constant ran about 1.3 ms faster. Reading the noise from a baked, tiling texture would save most of that, with a different pattern of stains.

Another 3.5 ms goes to drawing the city’s three million triangles into the G-buffer before any pixel is shaded. A second throwaway build skipped the buildings off the screen for that stage and saved about 0.4 ms, so most of the 3.5 ms is spent on what is in view. An earlier try at the same skip shaded 75 pixels differently, and 0.4 ms is not worth that, so it stays out.

The baked noise and fewer samples of how much sky each spot sees both change the picture, so each will be compared with before-and-after stills before it goes in.