Adding OpenVDB Support to Unreal

Expanding Unreal volumetric tooling and rendering features 

Introduction

Volumetric rendering is getting increasingly more popular in realtime rendering (e.g., volumetrics clouds, using volume textures, Embergen, FluidNinja etc.) while still being an active area of research (e.g. this or that). OpenVDB is the undisputed and Academy Award-winning standard for storing, manipulating, and rendering sparse and dynamic volumes, however it is best suited for offline rendering. With the introduction of NanoVDB, we now have access to a lightweight GPU-friendly data structure to store and render sparse volumes, opening a wide new range of possibilities in the context of realtime rendering.

We are excited to share an Unreal plugin that can import any OpenVDB (or NanoVDB) file into Unreal, automatically converts it to NanoVDB, and allows basic manipulations of VDB volumes in the editor. The plugin, made for Unreal 5, is available on our GitHub.

We believe and hope it will be a great sandbox for users to play and experiment with, whether it is for programmers or artists in the context of video games, virtual productions or even offline & research rendering using Unreal’s integrated pathtracer. All the core and integration components have already been implemented in the plugin, allowing users to just plug-and-play with VDB files. We also added an extra high-level layer of rendering and grid manipulation to quickly set up some pretty visuals while staying as customizable as possible and leaving a lot of room for user modifications.

Rendering cloud dataset provided by Walt Disney Animation Studios under the CCA-SA 3.0 Licence, using the Unreal pathtracer.

 

Importing an OpenVDB file in Unreal

The goal of this plugin is to give users a straightforward way to import OpenVDB (and NanoVDB) files into Unreal. Users only need to copy this plugin in their project plugin folder, compile Unreal or use the provided precompiled binaries. Once activated in the plugin list, OpenVDB files will be recognized by Unreal and importing them is as simple as dragging and dropping files in the content browser, like any other file format.

During import, all VDB files are converted to NanoVDB file format, making them read-only but GPU-compatible for faster rendering and sampling. Because VDB files can contain several Grids, we decided to import each grid as an Unreal asset to avoid any confusion. We only support LevelSets (also sometimes referred to as Signed Distance Fields, or OpenVDB surface) and FogVolumes for now. 

 

Using VDB files in Unreal

The interesting part now begins, and we can start experimenting with NanoVDB buffers/grids. We provide several ways to play with NanoVDB buffers in Unreal and try to make them as customizable as possible so that artists and users can explore different options.

VdbActor

The most common way to use a VDB file is to simply display it in the viewport. Dragging and dropping a VDB asset (called VdbVolume) in the viewport will create a VdbActor with a custom VdbComponent. VdbActors will only display if using Unreal Volume materials.

This rendering mode is compatible with some (but not all) Unreal features like selection, materials, directional lighting etc. Unreal’s renderer is not easily customizable from a plugin, so adding the missing features might be tedious or even impossible in some cases without modifying the engine source code.

  • LevelSets 

LevelSets are opaque surfaces and look like regular polygonal meshes although they are quite different. They are simple and unambiguous to display.

Narrow-band level set of the dragon model, courtesy of Stanford Computer Graphics Laboratory, with a simple position material.
  • FogVolumes 

FogVolumes are much more complex entities. Rendering them brute force is prohibitively expensive, so we must find cheaper and approximate ways to render them. One common approximation is to raymarch inside the volume. We provide a simple and approximate implementation, but it is not optimal nor gives the best results. That is also why we allow users to override the raymarching process using Unreal materials or simply by rewriting new shaders; please refer to the documentation for more details.

Simple rendering of the Stanford bunny, from the OpenVDB repository.

 

Niagara 

We also offer a custom VDB Niagara Data Interface, allowing VDB volume sampling in custom Niagara modules. For more complex operations on volumes, please use CustomHLSL nodes, as explained in the documentation.

Sampling the dragon level set volume and spawning particles on the surface.

 

Research

As previously mentioned, keeping Unreal pipeline compatibility can be tedious and require significant code writing and understanding. For programmers who do not care about these pipeline integrations and want to quickly experiment on VDB volumes, we propose another rendering mode called “Research” mode.

Coding, experimenting, writing custom rendering code and shaders is significantly easier using this mode. Iterations are much faster (less shader recompilations) and there is even room for a subsequent denoising pass to experiment with state-of-the-art volumetric rendering.

Rendering the Stanford bunny with the experimental “research” mode, allowing easier and faster iterations to tweak the rendering algorithm.

To push things even further, we add limited support to Unreal’s pathtracer to render VDB volumes interactively as an overlay. It is useful for testing offline rendering algorithms and progressively watch renders appear for beautiful screenshots. This is my personal favorite mode, as it allows for nicer and more physically correct results, testing algorithms that are already standard in other industries, all that in interactive time. It can be used as a photo mode, to bake a cinematic or a trailer, test the latest scientific paper etc.

Example of parameters tweaking on the Disney cloud, under the CCA-SA 3.0 Licence.

Features – Summary

  • Import OpenVDB files 
  • Import NanoVDB files 
  • Convert every VDB file to NanoVDB, once imported 
  • Viewport visualization 
  • Unreal volumetric materials support 
  • Ambient light and first directional light support 
  • Niagara modules can sample VDB Grids 
  • Option to convert Grids to Texture3Ds 
  • Pseudo pathtracer integration, for offline experiments 
  • More traditional graphics integration (no materials) for faster iterations 

 

Conclusion

We hope that this plugin can prove useful for artists and programmers alike. The new potential of having faster and easier volumetric rendering in Unreal is quite exciting. Even if rendering any volumetric effect in realtime is still very expensive, we expect the global quality and resolution of volumetric effects in the context of realtime rendering to really improve in the coming years, with faster algorithms & approximations, better denoising algorithms and more powerful hardware. We are looking forward to seeing what the community produces with this plugin and will be happy to follow up if it is raising a lot of interest.

 

Author

Thibault Lambert is a Senior R&D Graphics Engineer and joined Eidos-Montréal in 2018. He has worked in both the visual effects and video games industries in the last 13 years and has been lucky enough to work on several well-received and renowned movies and AAA games. He is passionate about real-time graphics and is dedicated to bridging the gap between offline and real-time rendering as much as possible, bringing real-time technologies to the film industry and reproducing complex offline techniques in real-time for video games. 

Llama