Pete Mc Nally's Blog

If you’re interested in 3D scanning technology or game art you will likely have heard about the experimental Unity De-Lighting tool recently released, you can find it here. 

“De-lighting” is the process of removing light and shadows from the diffuse texture map created during photogrammetry. This hasn’t been a huge issue in my previous scan work, mainly due to the weather in Ireland where it’s usually cloudy if not raining, which is ideal for outdoor scanning. A few weeks back I was lucky enough to be at the beach on a day that had 15 hours of sunshine, which was awesome for the farmer’s tan, but problematic for some of the scanning I’d planned on doing due to the harshness of the sunlight and heavy shadowing. I’d heard the Unity tool had been released so I figured I’d press on anyway and try it out. I captured about 120 photos of this sea stack, handheld with my DSLR.

Some of the potential problems are evident above – darkness in the shadowed areas, lens flare from the sun and the contrast between the brightly lit areas on the reverse side and the darker shadowing. Reality Capture breezed through the solve, even on my laptop and created a really nice mesh with just a few problem areas at the top where visibility was reduced. Here are the results, with the mesh decimated to 16 million tris and the texture baked at 16k.

You can see the shadow baked into the stack and falling across the beach rocks and pebble surrounding it, a good test case for the Unity De-Lighter. I downloaded the latest version of Unity and the De-Lighter package and read up on the requirements. The De-Lighter requires at least the base albedo texture, an object space normals map, a bent normals map and an ambient occlusion map, with optional an optional position map and mask. I wouldn’t usually bake an OS normals map (or bent normals) so I wanted a solution where I could bake as many of the required maps out in one pass as possible, so I settled on XNormal. I decimated the stack model from 16 million tris to 10k and baked the textures at 4k. Here is the output from the bake (the red areas on the albedo are missed hits).

Not much difference in this case between the OS normals and bent normals map, it’s quite possible I did something wrong here but it didn’t seem to hurt the end result much. It was easy to add the textures to the Unity De-Lighter and very quick to compute the light cancellation.

You can scrub back and forth between the original and the light cancelled version.

If there are areas where artifacts occur, you can paint up a mask for those areas and compute again. I’m impressed that it worked so well out of the box with textures that weren’t perfect. I did paint a mask for some of the areas that looked off or where there were obvious repeating detail issues, but this was super quick to do. Later, I fixed up the areas with the missed hits in Substance Painter and adjusted the albedo for PBR and painting a roughness map for wetness on the lower stack. Next I took it into Marmoset Toolbag 3 for testing. Here is the low poly model without textures, I deliberately pushed the decimation further on the higher, out of reach areas:

Here are the results in Toolbag 3, I added some beach grass billboards and a detail normal map, click through for 4k stills:

Here is the scene in real-time in Sketchfab, be sure to go full screen and let the HD textures load 🙂

Posted in Gallery, Personal Work
Tags: 3d, 3dsmax, 4k, albedo, beach, cancellation, capture, cliff, coast, de-light, de-lighted, de-lighting, de-lit, digital, erosion, game, gamedev, lighting, lookdev, madewithunity, Marmoset, material, nature, painter, pbr, photogrammetry, punk3dscan, real, reality, remove, rock, roughness, scan, sea, shader, stack, substance, time, toolbag, unity