Where did my pixel go? When Unity's frame debugger is not enough

Since version 5, Unity has shipped a new tool for visually debugging your frames: the Frame Debugger. This allows you to find out the reason for many graphic issues you’ll face: Z-fighting, strange GPU states, wrong render queues, incorrect blending operations, high number of draw calls, low performance, etc.. It offers a much more detailed information in comparison to the stats layout from the game’s view. By interacting with it and checking the render events/steps you’ll also massively learn about the GPU pipeline. Really, every developer should get to know it!

In this entry I’ll briefly describe how to use the frame debugger and afterwards I’ll point out the drawbacks that can be addressed through other tools. It is not my goal to give a thorough description about every point, but rather, to present an overview of the aforementioned technologies.

1. Frame Debugger: Usage

Open our main window through Window->Frame Debugger. I recommend seeing both the frame debugger and the game view at the same time, so you might need to adjust your layout a bit.

Get your game in play mode and get to the area that is causing you headaches. In the frame debugger window, click on enable so you get the details about the frame that was lastly rendered.

There are two main areas: the events and, to its right, its detailed information. The events are basically the commands that the CPU are sending to the GPU. They are surely way simplified, since you can not really see the API functions that are being called. But it is simple enough to keep the whole process timeboxed. Most of them are about clearing and drawing (opaque/transparent) geometry, but you might see more information like GUI rendering, shadows, image effects, etc. along the section’s number of callbacks.

Whenever you select an event you’ll instantly get more detailed information about it on the details panel. Among that information: the bound shader and its flags, the event rendered output and the shader properties. At the same time, the game view will display what has been rendered until that event (inclusive). An example is shown below:

2. Frame Debugger: What For?

There are a few useful hints that the frame debugger is giving us for free. The order of draw events (i.e. the rendering queues) are crucial for a correct rendering. If you have elements that are not being correctly displayed, you might discover that it is being rendered in the wrong render queue and therefore too early/late, especially if you are playing with custom shaders and parameters like Z writes and tests. I had many issues fixed by using this tool; for instance, I discovered that the skybox was overwriting the output of my shaders because I forgot to set the right flags and render queues.

Secondly, you relatively see how many draw calls are there and indirectly how “expensive” they are by looking at vertices/index counts. The amount of shader passes does also help, but their complexity is not shown here so you’ll have to use some common sense. Having this information will help you improving performance in your scenes; e.g. you might discover that some meshes are not statically/dynamically batching for some reason. In my example you might have noticed two draw calls for rendering two sprites that can be easily merged into one just by using texture atlas.

Thirdly and by interacting with the frame debugger, you quickly get to know the GPU architecture and discover how Unity handles the rendering process. You may use the keypad to navigate through the different events and see how the scene is being rendered step by step. In the aforementioned example you see that the scene starts with clearing three buffers (color, z, stencil), followed by rendering the opaque geometry (front-to-back), skybox and transparent geometry (back-to-front).

Lastly, you may access the shader properties for more information about the material and shaders. You also have a reference to the data that is being used by that object, like textures. Not many people do know about the advanced views in Unity; you can access them in the inspector by clicking on the paragraph icon in the top-right area and selecting “Debug”.

3. Caveats of Frame Debugger

While working for one of our clients, I experienced some drawbacks caused by the extreme simplicity of the frame debugger. I talked to a few Unity developers at Unite 2016 Europe and they seem to know about them, although they didn’t reveal if they had further plans to extend it. From my experience, some of them are:

• You don’t get low-level information about the API calls behind those events. Unity is, in this context, a black box that you can not (easily) improve in specific situations.
• It is hard to get an overview of the state of the GPU pipeline in every event. You just get information about the geometry and the textures but no data about the different stages like vertex/geometry/fragment shaders and rasterizer.
• You can’t really debug shaders. Either you output colors for visually debugging your code (even more complicated than prints) or you use external software like GLSL-Debugger.
• It is hard to associate resulting pixels with the responsible events that wrote them. You have to go step by step and check which event is writing the wrong information in your pixel; that can take a long time if you have a huge number of draw calls.
• No accurate indicators for measuring performance per draw-call. The number of vertices/indices is not necessarily the best one. Ok, sending more vertices require more memory bandwidth, but a complex pixel shader executed on that geometry or expensive state changes might be even worse.

If you are (still) reading this, you might be as well looking for answers. So, what about experimenting with complementary other tools?

4. RenderDoc

After testing several software, I can definitely recommend a few ones. The main one you should have in your toolchain is RenderDoc, a free tool originally developed at Crytek for solving lower-level issues. It is a tool that uses a classic approach used by many cheaters: it creates a virtual graphic driver and acts as a Man-in-the-Middle (MITM) to catch the application DX/GL/Vulkan calls so as to offer detailed debugging information. The good news are: there’s already a Unity native integration. You just need to install RenderDoc in your windows machine and restart the Unity editor.

My objective in this section is to give you an overview about the features of this tool so you feel intrigued enough to start playing with it and making your players happier both by playing smoothly and paying less electricity bills.

Let’s get down to business. If the installation went successfully, you should be able to right-click on Game and select Load RenderDoc:

After a few seconds, get into play mode and whenever you are happy with the frame you lastly rendered click on the new icon that looks like a bald person to the left of Maximize on Play:

Then switch to the newly opened RenderDoc window and double click the automatically captured log to analyze it:

Wow WTF! So much information. Calm down, everything’s gonna be ok. By now you will have all the frame’s gathered data in thousand windows. No worries, we’ll be highlighting a few points that I considered really interesting in the next paragraphs.

4.1. Event browser + API calls.

One advantage of this tool is being able to get lower-level information about what is going on in each event. The event browser is pretty much like an extended Unity’s frame debugger. It includes timings (in microseconds) and other hints that pretty much help measuring performance. RenderDoc is context-sensitive, meaning that, any time you select a certain event, most of the other windows and areas will only display the relevant information to that event. The API calls window under it lists the functions called for processing that event.

What a good thing to get these timings! Surely, they might not be absolutely precise, but they are relatively accurate, meaning, that by comparing the figures you will figure out which draw calls are the most expensive.

4.2. Pipeline state.

I always wondered how a pipeline looks like. How little cute geometries enter it and gently get transformed to pixels in your 50″ screen. Well, you’re not going to get such a user-friendly animation out of this tool, sorry.

There’re anyway great free resources for that. But anyway, this tool will give you access to a wide range of facts about the pipeline states in each event: input assembler, vertex/hull/domain/geometry/fragment/compute shaders, rasterizer and output merger. And trust me there, that is good stuff.

4.3. Mesh output

Another feature is getting the mesh data that was rendered in that event through vertex shaders: in/out positions, normals, texture coordinates, etc.. You may, in fact, capture and save meshes being rendered from other 3d programs/games, but I’ll not be dealing with ethical matters in this post 🙂

4.4. Texture viewer

One of my favorites. Visualize all input/output textures being used in the event, including render to textures and other intermediate buffers. It is really interesting if you are using intermediate buffers when dealing with image effects. Again, you may save textures.

4.5. Pixel debugging

It is not uncommon to see evil pixels overwriting good ones in our frame buffer. Many of us have had nightmares about it as they leave a strange feeling like if something is wrong in our scene but we are not really sure about its origin. It just doesn’t feel right. If only we could confirm our suspicions and determine what happened there…

Well, we can. Select a pixel and then click on either history (to get a list of events that wrote to that pixel in the frame buffer) or debug (for debugging its pixel shaders). Debugging them is for tough people: you will need assembler knowledge to make sense of the disassembled version of the pixel shader.

5. Example

There might be people wondering whether profiling is useful in real-life. Let me tell you: this is real-life and it is useful. Whenever you’re not sure about the performance impact of a feature, make your research. Use the profiler, frame debugger and if needed RenderDoc or similar tools.

Let us consider an example. It is not by all known that there is a difference between accessing the material and accessing the shaderMaterial of a renderer in your scripts. But not many people know the concrete performance impact of doing so. And that is still fine for most cases. But since we have the tools let us find that out!

Our test scenario consists of a single sprite being instantiated 10000 times with the same transform. It occupies nearly the whole screen, so a lot of overdraw is to be expected. The code is as follows:

void Start () {
  for (int i = 0; i < NumberSprites; ++i)
  {
    var sprite = Instantiate(Sprite);
    sprite.transform.parent = transform;
    var sharedMaterial = sprite.sharedMaterial;

    // Uncomment me for creating a material copy per sprite.
    // var mat = sprite.material;
  }
}

Accessing sharedMaterial is fine; you will access the unique material that is shared across all sprite instances. If you access material instead, you will be creating a unique material per sprite and therefore Unity won’t be able to batch the draw calls. Analyzing the results with RenderDoc for Render.TransparentGeometry the results are:

I don’t really trust RenderDoc timings; its readings do not seem accurate at all and I am not aware of reliable hardware counters for this matter. I would rather check Unity’s profiler. But in any case, it does make a difference sharing materials or instanciating new ones. The later impedes drawing everything in just one call and therefore you have more driver overhead and memory bandwith penalties.

What really expensive is is the state changes between drawcalls, not the amount of drawcalls itself. After setting a different random color to every sprite on Start through Material.SetColor, I noticed an extra 2 FPS drop. That is something that is being solved with the recently launched Vulkan API.

6. Conclusions

RenderDoc is neither a perfect tool nor a replacement for Unity’s frame debugger, but rather a complementary tool you can easily use if you need that kind of low-level information. It is yet interesting to see how far one can go; but the farther you go, the more complicated it gets, since you will need a better understanding of the GPU hardware architecture.

I confess that using RenderDoc can be quite time-intensive, so remember: profile before you optimize. It might be just fine to use Unity’s frame debugger 95% of the times, but you should be ready for the remaining 5% that is going to make your life hard like a mug.

Use these tools along the profiler. It is not always trivial to know where a bottleneck comes from, especially when targeting other platforms with reduced debugging capabilities.

As always, I’ll appreciate your feedback and questions.