Transparency Sorting

There is often a lot of confusion with how pixels get processed in relation to the z-buffer and why sorting is important. Most importantly it can be mind-warping trying to wrap one’s head around how to 'fix' the cases where proper sorting isn’t possible.

Hopefully these diagrams help show the futility of it all… I mean the trade offs one has to make to get transparency looking its best in the general case.

The first image I’ll show is the 'best case' where all objects are drawn back to front. I’ll then discuss briefly the steps JME goes through to try to make that happen.

transparency_sorting1.png

In JME, the opaque layer would generally be placed in the opaque bucket. All opaque layers are drawn first and within the ability to sort them, they are sorted front to back to prevent overdraw.

After the opaque bucket is drawn, the transparent bucket is drawn and it is sorted back to front. So in the ideal case it would appear exactly as in this picture.

Sorting is done at the object level and it can never be perfect. It is impossible to properly sort objects by distance in the general case even if they were just triangles. Simply imagine intersecting triangles and it’s clear there is no proper order. JME tries its best.

Next I’ll show an image of the worst case scenario to show why sorting is important and how your 'best friend' the z-buffer is really transparency’s worst enemy.

transparency_sorting2.png

This is what will happen if you put all of your objects in the opaque buffer. JME will sort them front to back and you will get these strange 'windows' into your background.

Because sorting is done at the object level, you may have triangle to triangle overlap even within the same mesh if it is non-convex. Think of a glass donut where the near surface triangles are drawn before the hole’s triangles. There will be this same issue where they occlude the farther triangles. A different angle on the donut might produce the correct results depending on the order of the triangles in the mesh.

Finally, I’ll augment the worst case sorting with something like alphaDiscardThreshold (or the old alphaTest/alphaFalloff values). In this example, let’s pretend we only discard pixels with alpha = 0.

transparency_sorting3.png

It’s better but not perfect. Any partially transparent pixels will still show the issue. Partial transparency will drive you crazy if you let it.

Bottom line:

  • back to front sorting would fix all issues

  • accurate back to front sorting in the general case is impossible

  • for purely on/off a = 1 or a = 0 transparency then a discard threshold is the best bet to mitigate sorting problems.

  • Where 0 < alpha < 1, improper sorting of triangles/pixels will always cause artifacts.