June 2022
Texture Atlasing in Red Dust
Modern games often have thousands of different objects on screen at any given time and this creates a big performance problem if it isn't handled properly.

Modern games often have thousands of different objects on screen at any given time and this creates a big performance problem if it isn't handled properly. In order to render a single object, the video card requires that all drawing details need to be set. These are things such as the shader, render buffers, textures and geometry. Unfortunately setting these details costs a significant amount of time, often more than the rendering actually takes and one of the worse of these details are the textures.

Texture atlases are large textures that contain textures used by multiple individual objects. Each object's texture only uses a small portion of the atlas texture. Some games use hand crafted atlases where the artists carefully place the individual textures in atlases themselves and reassign the mesh uvs but this approach is considered very inflexible and time consuming since if only a model needs to be changed it might require changes in other objects as well.

This article describes the benefits of using texture atlases, the theory behind it and then it goes over the successful implementation of a automated atlas generation system in the game Red Dust, an upcoming, colony management game on Mars made using the Unity game engine.

Benefits and Theory

Having many objects sharing the same atlas texture removes the cost of switching the textures on the video card by reducing them at a minimum often from thousands, down to 10-20 or in more extreme cases even to 3-5. Depending on the implementation, this can also allow for geometry batching, where instead of sending the meshes one by one and issuing multiple drawcalls, they are combined into a single mesh that gets rendered at once. This again can result in a huge increase in performance if the game is GPU bound.

The first step that needs to be done is to pack the individual textures into one. There are multiple algorithms for this some of them even being able to rotate the input in order to reduce the amount of wasted space. The algorithm used in this article is a simple one which doesn't rotate the textures but still yields good results. At the end of it, each texture is mapped to a specific rectangle inside the atlas texture.

Video cards also really like it when textures sizes are powers of 2 but not all objects need the whole space and texture atlases can help with this as well since the individual packed textures don't have this requirement.

The second step is to map objects from the rectangle of the original texture (which is all of it) to the rectangle in the atlas texture. This can be achieved by adding 2d transformation matrix in the material and keeping the original mesh intact but doing this prevents geometry batching. Another idea is to simply preprocess all meshes mapping the vertex UVs from the original texture to the atlas one. This way all objects can share one single material.

When using atlases, we deal with bigger texture sizes, up to 4096 or possibly even bigger on newer hardware. Given floating pointing rounding issues often there can be bleeding artefacts on the edges of uvs. A simple fix for this is to add padding. These are thin edges around the textures on the atlas that are filled with the clamped pixel values.

Two atlased textures with padding on the edges
Texture packing implementation

In order to figure out how to pack the textures we only need to know their sizes. Their content is only needed later, when the data gets copied. The steps of the algorithms go as following:

  1. We first add the rectangle belonging to the whole atlas to a list of free rectangles
  2. We then begin adding the target rectangles belonging to the individual textures one by one
  3. For each target rectangle we go through the whole list of free rectangles
  4. For each free rectangle if the target rectangle fits, we calculate a score based on the area it would occupy on the current rectangle and how slim the remaining free rectangle would be
  5. Using the best score, we determine the position of the rectangle, starting from the point closest to the origin
  6. Using this rectangle we then go over all the free rectangles and split the ones that overlap the current one.
  7. As a final step, after the insertion of each rectangle, we prune the list of free rectangles by checking all pairs one fully contains the other.

We begin by placing the first rectangle on our grid:

  • Empty rectangle the size of the atlas
  • List of input rectangles to be packed
  • Placing the first rectangle

For every rectangle placed we need to split the existing free rectangles

  • Splitting the free rectangle by the first placed rect
  • One of the resulting free rectangles
  • The other resulting free rectangle

In order to pack the second rectangle, we have to go over the 2 free rectangles we have, to see where it would fit better:

  • Area fit: 42 Short side fit: 3
  • Area fit: 35 Short side fit: 3
  • Final position based on the score

After adding the second rectangle we need to split the free rectangles by the newly added one. For this we go over the 2 free rectangles we had and apply the cuts:

  • Splitting the first free rectangle
  • Resulting free rectangle 1
  • Resulting free rectangle 2
  • Resulting free rectangle 3
  • Splitting the second rectangle
  • Resulting free rectangle 4
  • Resulting free rectangle 5

After adding the new rectangles, we can see that rectangle 1 (light grey) is fully contained in rectangle 5 (dark grey). The pruning step takes care of this and removes rectangle 1 from our free rectangles list.

Packing the last rectangle:

  • Checking the fit in rectangle 2
  • Checking the fit in rectangle 3
  • Checking the fit in rectangle 4
  • Checking the fit in rectangle 5
  • Final position based on the best score

And that's it. Our 3 rectangles are packed as efficient as they could be. This is not always the case but given the simplicity of the algorithm it's reasonably effective.

The following piece of code is an implementation of this algorithm. The initial size is passed in the constructor and then when trying to insert a list of rectangles (or just their sizes), it will automatically resize to the minimum that is necessary to fit all rects.

Creating the atlas

In Red Dust atlasing is handled as part of a scriptable object called Content Pipeline. This script also handles copying some animations and scans the asset database for issues but for the purpose of this article we'll only look at the atlasing aspect and ignore the rest. Also, Red Dust makes heavy use of the Odin Inspector since it makes the editor code so much better and easier so, if some editors look unfamiliar that's likely the reason.

We begin by using a list of atlas groups to keep track of names, output locations and padding settings. Using this list we can then assign different atlases to different game objects.

For each game object that needs to be atlased we keep track of the atlas group, where the output should be, if the resulting fbx needs to have read/write enabled and optionally, a list of variations. These are used in case we want to use the same mesh with multiple textures. For each of them we need to generate new meshes since the atlased uvs will be different. In the case of variations we also keep the file name for each variation. Lastly, these game objects are further split in some extra named lists just for ease of use since we're dealing with large number of entries.

This is how an entry with a few variations look like.

In order to account for different materials, shaders, and combinations of textures that we might encounter we need to take a series of steps:

First, we iterate through all atlas groups and skip the ones without a proper name

We then collect a Dictionary with all shaders used as keys and the list of all textures used by each shader.

We also collect all the texture names used by all the shaders. Due to some unexpected behaviour in Unity, we have to ignore property names that start with "unity_" or else we'll end up with some textures that are not actually used by the materials

Using the collected Dictionary from 1, we collect a list of all unique textures.

Using the unique textures we can compute the necessary packing and while at this step, we also check that all textures that need to be packed together have the same import settings like usage of mipmaps and filtering mode

Once we have the packing ready, we can go ahead and generate textures for each of the texture names collected from the shaders. We also make sure that the textures are cleared initially

For each shader used by the input objects, we can now generate the material and assign the textures as needed. This code also checks if the materials already exist end in that case it fetches those ones

In order to actually create the atlases, we go through all rectangles that have been packed and then copy the textures for each texture name

At this step we also apply padding.

Now that the textures are ready, we go ahead and save the to the disk taking into account the mipmap and filter settings and after that the materials too. After the textures are flushed to the disk, we refresh the database and get their new handles in order to assign them to their respective materials. Finally we also create assets for any new material since those also need to be saved to the disk

Now that those are taken care of we can preprocess the game objects

We instantiate each object, and then replace the materials with the new ones.

Using each mesh we create new ones by mapping the vs from the rectangle in 0,0,1,1 space to the one assigned to it in the atlas.

After exporting each object as an fbx file, we can use the Editor specific code to change the import settings, like remapping the new materials, animation types, and readable.

At this point we're done and all objects that were set to be atlased are now ready to be used.

Final words

Red Dust uses URP and only albedo/diffuse textures so the above code might have some issues with materials using multiple channels. Also, when performing atlasing, the previously created fbx files are overwritten with the new ones even when they don't change at all. Due to the fact that game objects are created every time, the resulting fbx files have different unique identifiers. This might pose some issues when using git.

Or maybe you need advice for your game?