Obviously games arenât magic. But they can look like it when you donât know whatâs going on behind the scenes. That was something recently highlighted when this GIF of Horizon: Zero Dawn appeared, showing how the game engine only renders (makes, effectively) what youâre looking at.
For a lot of people, this dynamic world creation was mind-blowing to see in action. Oddly, however, rather than being happy anyone was taking an interest, quite a lot of developers seemed sniffy that your average teenage gamer was ignorant of âfrustrum culling.â So letâs fix that and take a look at a few interesting technical things that most games do to make themselves look pretty. Or even just work at all, starting with Horizonâs magic camera.
Culling is all about reducing, and in games it means to reduce the amount of work an engine has to do by only rendering (or making) what the player can see. It makes a lot of sense. There isnât a console or PC alive that could process the entire world of Horizon at once, so this cuts down the workload by ignoring anything that doesnât fall into the cameraâs view.
There are two main types â frustum culling, which renders everything in the cameraâs field of view, and occlusion culling that only renders what the camera has a direct line of sight to (further reducing the work needed).
Hereâs a GIF that shows the two in action. As you can see, the player view in the bottom right shows the same view each time.
Another thing a game can do to save precious resources is to only draw the details on things close up. If you ever see anything âpopâ in while playing youâre seeing a transition from one level of detail to the next. Take a look at these bunnies:
Obviously the first one looks the best while the last looks like some kind of horrible Origami accident. However, now letâs look at the same models over a distance:
That last bunny doesnât look too bad when itâs that far away, and as a considerably simpler shape isnât as hard to make. Whatever youâre playing on only has so much power to draw the world, animate stuff, make mist look pretty and so on â so steps like this help conserve that power for where itâs best used. Even so, large worlds or poorly optimised games donât always blend between different levels of detail, which is when things âpopâ.
Rays, or âray castingâ is a fundamental way that games find out where things are in its world. If you ever heard mention of The Last of Usâ âpunch lasersâ, those were rays. The name is almost an explanation in of itself â an object casts out rays (draws a number of lines in certain directions) and, if they hit anything, they report back.
Hereâs Jacob Minkoff, the then lead designer on The Last of Us, explaining how those âpunch laserâ rays work practically when Joel is fighting: âWhen you push the punch button, it fires a bunch of rays around the game world, and says âWhatâs nearby?â. Oh, thereâs a wall nearby. Or thereâs a desk nearby. So if you press the punch button and make contact, it will smash this guy against the wall.â
Another good example of rays from The Last of Us is how the game decides where Ellie should stand in relation to Joel.
This image shows lines fired out from Joel to decide potential places for Ellie to stand. Other lines then answer various questions like if a spot is blocked by an object, can Ellie see ahead, or look at Joel and so on. Locations that tick all the boxes show up as green lines, with the red lines showing places Ellie canât go. When characters end up in walls this is the bit that has gone wrong.
Iâve not gone into huge amounts of depth on anything really, but Iâm definitely going to skim this because there are so many variations and options. The key thing is that what you see on screen doesnât appear in one go. A game isnât filming a virtual world and relaying it directly to your eyes, itâs creating a flat representation of it in layers, with various passes for different characteristics and details.
Hereâs a GIF showing how Killzone 2 renders a scene in layers, each one calculating or handling a different element. A good analogy would be to think of each pass working out light, shadow, colour, texture, and so on, creating a flat 2D image of what the gameâs simulating.
This is where all the post processing stuff follows. As the name suggests thatâs stuff that comes after all the rendering grunt work of basic shapes of bodies and buildings â stuff like motion blur, and particle effects. Much of these passes are where various shaders and lighting stuff happens, which brings us neatly on toâŚ
There are so many types of shaders but weâre going to focus just on vertex and pixel shaders to keep things simple, and show the two main effects these things can manage â changing the light effect or position of pixels in a game.
Pixel mapping usually affects colours and lighting, but can also affect the apparent position of pixels depending on that light. Take a look at this:
It shows a plain ball, a bump map (which is basically a black and white texture), and then the two combined showing how that black and white speckle has been turned into an orange skin like finish.
Thatâs the simple version. Hereâs what it looks like in a game like The Order: 1886:
Each one of those little balls shows a texture map that adds to the finish of the water pump. Itâs way of adding fine detail to a simple model without using loads of resources. The pump is a complex geometric shape where every point has to be calculated, the shaders are just a skin stretched over it.
Hereâs another example showing normal mapping.
The fine detail added by the normal map on the right doesnât use any more polygons â itâs still the same model on the left â itâs just a visual effect, creating something thatâs far less resource intensive but still looks complex. For the most part, if you ever see anything in a game that looks flat, muddy and like it hasnât loaded, itâs because the shaders havenât worked.
Another important type of shader is a vertex shader. These are more focused on physically moving pixels, and if youâve ever looked at water in a game youâve probably seen one in action. Itâs far easier to render water as a flat surface and apply a shader afterwards to move it than it would be to actually animate it. But itâs not just water, hereâs a fantastic example of how Naughty Dog used a shader to animate dense vegetation in Uncharted 4:
While it was technically possible to individually animate each plant (and an idea that was considered) it would have required a huge amount of processing to detect collisions, calculate reactions, direction of movement and so on. Instead a shader applies deformation to all the pixels of the plants, creating the effect same without all the effort.
And thatâs it for now. This is obviously only a tiny fraction of what happens inside a game like Prey or Overwatch, and a gentle introduction at that, but hopefully it should help you get a grasp on whatâs going on behind the scenes. A single beam of light reflecting off an object, from a source and into your eyes requires more maths then youâd ever likely see in an exam. And games have to do that for every object, every light beam, and do physics, culling, rays, LODs and so much more â all while youâre yanking the controls left, right and centre without warning. When framerates drop, or people glitch through walls inside out without a face itâs likely because so much is happening occasionally things misplace the decimal. And, when you start to look at how all this pulls together, you have to marvel how any of it works at all.