Fast global lighting with precomputed radiance and SH-based BRDF

Scene showing various low-frequency BRDFs in 2 lighting conditons with normal mapping. Simulated BRDFs are laminated wood,  frosted gold, brushed bronze, satin/velvet, cardboard and aluminium. Rendering times are approximately 2ms for the entire scene plus directional lighting with cascade shadows on Core i5 with ATI 5850 GPU.

Precomputed radiance is often used to simulate realistic global lighting for static scenes. Rather than just storing simple lightmaps, several convenient basis functions also exist that can approximate lighting around a hemisphere, and these can be used to reproduce global lighting with normal mapping. Still, most implementations just assume a standard diffuse model for global lighting, while local lighting might use a different BRDF model. This means that metallic or otherwise non-diffuse surfaces lit with global and local light sources can look inconsistent.

ARF addresses this issue by approximating arbitrary BRDF for surfaces lit by global lighting, represented by easily rotatable zonal harmonic coefficients.

First we generate lighting information at each surface element or vertex and then store these as spherical harmonic floats. This can be done by rendering the scene normally on the GPU from the position of each surface element oriented about the normal to capture lighting about the hemisphere and then processing the frame buffer on the CPU. ARF currently uses a paraboloid projection for the hemisphere instead of a cubemap projection when rendering the scene because it's significantly faster, but at the cost of interpolation artifacts around less tesselated geometry. ARF will probably also support cubemap projection for higher quality results in the future.

Secondly we also generate a BRDF response table for every light orientation in the hemisphere about a surface and compress that into spherical harmonics and repeat this also for every viewpoint. We compress these spherical harmonics further into approximate zonal harmonic representations for faster rotations at the expense of some accuracy and store the results in one 64x64 4-byte texture and another 64x64 2-byte texture. Each zonal harmonic coefficient is range-compressed to fit into 8-bits rather than a 32-bit float. Every entry in the texture(s) now contain the zonal harmonic representation of the BRDF for every viewpoint about the hemisphere.

At runtime, we index the two lookup textures in the GPU with the view direction and reconstruct the zonal harmonic coefficients representing the BRDF for that particular view and rotate that to the tangent frame for the surface. We then simply evaluate the lighting equation with the BRDF and incident lighting coefficients.

Some shortcomings with this implementation is that only lower-frequency lighting and BRDF can be practically approximated with spherical harmonics, and the additional compression into zonal harmonics for the BRDF also means that certain complex BRDF cannot be fully represented.

However, the simplicity of it's implementation, acceptable visual results in the general case and very fast performance (especially if lighting is stored and compressed per vertex) means that overall this lighting technique can still be quite practical even on lower end systems. On an ATI 5850 GPU and Core i5 CPU the example scene above renders in ~2 ms with cascade shadow directional lighting in a deferred pass. Without the cascade shadows, it takes only less than 1 ms on the same setup.

Fast deferred-pipeline friendly BRDF for local lights

Plastic, satin/velvet, polished metal,  burnished metal and brushed metal BRDFs with shadowed directional light and 8 point lights

The original LaFortune lobe can be useful for modelling various BRDFs, but when multiple lobes are used for more complex BRDFs, the performance cost can be high when each lobe is evaluated at runtime. Moreover, passing all the parameters for each lobe per-pixel to be used in deferred lighting is very bandwidth intensive. Ideally, we want to be able to (1) minimize bandwidth usage and (2) speed up lobe evaluation.

While we can store a material index in the alpha channel of the albedo buffer to be used later to lookup the appropriate BRDF response instead of storing all lobe parameters, the problem is really trying to store the BRDF response of the original LaFortune lobes into a 2D texture lookup table. Specifically, the issue is that the original LaFortune lobe has 3 inputs, not 2.

ARF uses modified LaFortune lobes to work around this issue. We sacrifice single-direction anisotropic effects to drop the number of inputs to 2, while retaining bi-directional anisotropy (so we can still simulate materials like velvet and brushed metal). By dropping single-direction anisotropy we also do not have to care about generating tangent space axes for normals sampled from the normal buffer when lighting the scene. The BRDF is then evaluated for the range of inputs and stored in a 128x128 1-byte texture.

So not only can you simulate different BRDF effects (including bi-anisotropy, off-specular or back-specular reflections) using 1 or more of these modified LaFortune lobes stored into a texture, but also only 1 texture fetch is required to evaluate a BRDF result. The fetch itself should be quite fast too since the BRDF texture is small-ish and the sampling pattern relatively cache friendly.

Multi-threaded task-based execution

Splitting work carefully into granular tasks  (while ensuring that each task has enough "meat" to offset synchronization overhead) and then running them in parallel is a performance win most of the time, especially when CPUs seem to have an increasing number of cores these days. Still, one must be careful to avoid false sharing or otherwise thrashing the local caches in each core as this is usually the biggest performance killer barring any unnecessary synchronization.

Most implementations I've seen define the order of execution of tasks implicitly by their dependencies. Something like:

Task taskA, taskB, taskC, taskD, taskE;
taskC.DependsOn(taskA, taskB);
taskD.DependsOn(taskC);
taskE.DependsOn(taskA, taskC);

AddAllTasks(pThreadPool);
pThreadPool->Run(); // Block until all tasks complete

Now figuring out which task goes first from just the dependencies is quite trivial in the above example, but it gets considerably more difficult in a real world situation:

Frostbite 2 task graph

Without a generated task graph like the one above, figuring out the order of execution of each task just from their dependencies alone in code can be tedious, which happens often enough especially when debugging.

While nothing stops ARF applications from using a task-execution framework like the one above (after all ARF is just a bunch of related libraries and is not really an "engine"), by and large I still prefer explicitly ordering tasks in code simply because it's much clearer:

pThreadPool->AddTask(taskA);

pThreadPool->AddTask(taskB);

pThreadPool->Run(); // Block until all tasks complete

pThreadPool->AddTask(taskC);
pThreadPool->Run();

pThreadPool->AddTask(taskD);
pThreadPool->AddTask(taskE);
pThreadPool->Run();

This is functionally equivalent to the one above but isn't the ordering immediately apparent?

In any case, in addition to the usual synchronization primitives and atomic instructions, one of the ARF modules also provides a robust and lightweight thread-pool implementation that supports multiple task consumers and producers, with the total amount of code within a contention block only about 8 instructions long, so squeezing maximum performance out of multi-threaded applications becomes less of a chore.

A modular and parallel renderer framework

ARF project in Visual Studio

It was determined from day 1 that significant effort should be spent at ensuring that ARF libraries are modular and only have clean 1-way dependencies between them and generally accessible only through opaque interfaces. There are many benefits to organizing an evolving codebase this way.

Having feature sets as pluggable modules mean that modules can be included and excluded as is appropriate for a particular project, or when old features are obsolete and replaced by new features. No code bloat. Development also becomes straightforward because new features become easier to integrate into the rest of the framework when the dependencies are few and only happen between modules. Writing scripting wrappers or self-contained unit tests for example for the various modules without polluting the rest of the codebase also becomes an easy thing compared to a monolithic framework. Even compilation can generally be faster simply because header dependencies and library linkages occur per module only.

ARF also exposes opaque interfaces in a module so changing the implementation without affecting application code becomes easy. Opaque interfaces also clearly differentiates to users between code that is relevant to the application and internal code. This might seem obvious but it's surprising how many professional 3D engines implement multi-platform code with a haphazard bunch of #ifdefs. It's really quite hard on the eyes.

For example in ARF, the renderer module currently has a DX9 implementation, but the DX11 implementation can simply be swapped in since it also derives from the same interface as the DX9 implementation. Of course, it's not possible to define an abstract interface that covers every functionality so DX11-only functionalities will be exposed in a separate DX11-ish interface should the application or other modules require it.

Using interfaces often imply some sort of overhead but the organizational benefits often outweigh it.

Another thing that is important when writing modules for ARF is that modules should support scalable multi-threading. This means that if the performance of a certain feature can theoretically benefit from multi-threading, then it must be implemented in such a way that performance should scale with the number of threads. Each feature must also allow the application the flexibility to decide how many and which threads to use. With new CPUs having an ever-increasing number of cores, the old paradigm of having 1 thread on rendering, another on physics and another on AI is fast becoming irrelevant.