Afternoon De-Lighting…

•July 26, 2017 • 5 Comments

If you’re interested in 3D scanning technology or game art you will likely have heard about the experimental Unity De-Lighting tool recently released, you can find it here. 

“De-lighting” is the process of removing light and shadows from the diffuse texture map created during photogrammetry. This hasn’t been a huge issue in my previous scan work, mainly due to the weather in Ireland where it’s usually cloudy if not raining, which is ideal for outdoor scanning. A few weeks back I was lucky enough to be at the beach on a day that had 15 hours of sunshine, which was awesome for the farmer’s tan, but problematic for some of the scanning I’d planned on doing due to the harshness of the sunlight and heavy shadowing. I’d heard the Unity tool had been released so I figured I’d press on anyway and try it out. I captured about 120 photos of this sea stack, handheld with my DSLR.

seastack

Some of the potential problems are evident above – darkness in the shadowed areas, lens flare from the sun and the contrast between the brightly lit areas on the reverse side and the darker shadowing. Reality Capture breezed through the solve, even on my laptop and created a really nice mesh with just a few problem areas at the top where visibility was reduced. Here are the results, with the mesh decimated to 16 million tris and the texture baked at 16k.

petemcnally_highpoly_stack_3dscan

You can see the shadow baked into the stack and falling across the beach rocks and pebble surrounding it, a good test case for the Unity De-Lighter. I downloaded the latest version of Unity and the De-Lighter package and read up on the requirements. The De-Lighter requires at least the base albedo texture, an object space normals map, a bent normals map and an ambient occlusion map, with optional an optional position map and mask. I wouldn’t usually bake an OS normals map (or bent normals) so I wanted a solution where I could bake as many of the required maps out in one pass as possible, so I settled on XNormal. I decimated the stack model from 16 million tris to 10k and baked the textures at 4k. Here is the output from the bake (the red areas on the albedo are missed hits).

bentnormals]#

Not much difference in this case between the OS normals and bent normals map, it’s quite possible I did something wrong here but it didn’t seem to hurt the end result much. It was easy to add the textures to the Unity De-Lighter and very quick to compute the light cancellation.

delighter

You can scrub back and forth between the original and the light cancelled version.

If there are areas where artifacts occur, you can paint up a mask for those areas and compute again. I’m impressed that it worked so well out of the box with textures that weren’t perfect. I did paint a mask for some of the areas that looked off or where there were obvious repeating detail issues, but this was super quick to do. Later, I fixed up the areas with the missed hits in Substance Painter and adjusted the albedo for PBR and painting a roughness map for wetness on the lower stack. Next I took it into Marmoset Toolbag 3 for testing. Here is the low poly model without textures, I deliberately pushed the decimation further on the higher, out of reach areas:

petemcnally_stack_06

Here are the results in Toolbag 3, I added some beach grass billboards and a detail normal map, click through for 4k stills:

petemcnally_stack_02

petemcnally_stack_03

petemcnally_stack_04

Here is the scene in real-time in Sketchfab, be sure to go full screen and let the HD textures load 🙂

Advertisements

Doagh Island terrain…

•June 23, 2017 • Leave a Comment

Quick update, I found some old photos that I thought were lost and ran them through Reality Capture. Quite a bit of cleanup needed, this is a bigger hunk of landscape than I’d typically try to scan in one session,  but overall I’m happy with how the sharp edges of the rocks have been captured. Lots of scope to mix in some Megascans too. More to come as I break off areas and conform for real-time purposes (currently the textures are at 16k resolution and the tri count is upwards of 12m).

petemcnally_RC_HIGH_full_realtime03petemcnally_RC_HIGH_full_flatpetemcnally_RC_HIGH_full_realtime02petemcnally_RC_HIGH_full2

Metal underfoot…

•May 18, 2017 • 2 Comments

I made this manhole cover material from a photograph taken on my phone on the way to work one day, with the intention of extracting enough info from it to populate a full PBR material. I was considering setting something similar as an assignment for my students, so I needed to be sure it could be done, and done fairly quickly. I modelled the L-shaped studs to try and get rid of the noise created by the usual image to normal map issues, but otherwise the material was created by manipulating the original photograph in Photoshop. Check out the real-time Sketchfab version in the last image.

Also couldn’t let this day go by without a shout out to Chris Cornell, who provided the soundtrack to the creation so much of my art, RIP.

petemcnally_blog_manholecover_WIP_01

petemcnally_blog_manholecover_WIP_02

Chancing it…

•April 24, 2017 • Leave a Comment

Driving around Malin Head last week we passed some beautiful natural rugged landscapes and I couldn’t help but wonder if it was worth sticking the phone out the window to record some video to try with photogrammetry. I was hoping that a 3D model could be derived from frames of less than optimal footage, so bumpy roads, motion blur, 1080p frames (low resolution for photogrammetry) and h264 compression were all working against an acceptable result. Surprisingly, Reality Capture used most of the extracted frames and produced a half-decent model, shown here in Toolbag 3.

It wouldn’t hold up to close scrutiny but would certainly work as background geometry somewhere or as the basis for a terrain heightmap.

petemcnally_Malin_landscape02

petemcnally_Malin_landscape03

Volume 11…

•April 14, 2017 • 6 Comments

This is a quick test of the Open VDB support in 3dsmax 2018, rendered in Arnold which replaces Mental Ray in this release. Two Arnold lights and samples set around 4 took  34 minutes for the 1k image on a quad core i7 🙂

Explosion orbit test:

xUA7bgManQfSpu8XLO

One more rock…

•March 30, 2017 • Leave a Comment

In the same lo-fi scanning session as the last one (ie. a 20 minute window between heavy rain showers on the beach, no tripod, colour checker etc.) I shot this on my phone. Processed in Reality Capture, 3DSMax and Photoshop. I’m fairly pleased with how this turned out, again I was lucky with the lighting and all of the jpegs captured on my phone were aligned. There are some small areas at the base of the rock that are missing some detail due to automatic hole filling but nothing that couldn’t be cloned back in Photoshop. I baked out ambient occlusion and cavity maps to use as layer masks in de-lighting a little for PBR and then painted a roughness map based on some known values and colour selections from the albedo map. Once again, all of these shots are rendered in Toolbag 3 at 1440p and you can check out a real-time version in your browser below, be sure to go full screen and if you hold alt and drag the mouse around you can move the lights!

Footprints…

•March 21, 2017 • 4 Comments

footprints

This year for St. Patrick’s Day I spent the weekend in Inishowen, Donegal, where the weather was unpleasant to say the least. For about 2 hours on Saturday, it was dry but seemed to always be a minute away from raining, so I headed for the beach without any of the proper equipment for outdoors photogrammetry. I did have my phone though, which can be hit and miss for 3D scanning as there are few manual controls on the Note 4, so if the sky is present in any part of a photograph, exposure goes crazy, making shots unusable. Also, it only shoots in jpeg so there is very little wiggle room on the texture side.

Three areas in particular caught my eye for some hasty shots as it began to rain- a sandy boardwalk descending from the car park to the beach, a rock about 3 feet in diameter that was exposed due to the tide, and some footprints perfectly preserved in the wet sand, which can be seen below. The beach was just damp enough to hold the shape of the impressions and the roughness of the displaced sand around each of them. All of the subjects were ground based too, which was a blessing while shooting with the phone as it kept exposure consistent between photos. I took about 45 photos in an orbit around the footprints, being careful not to compress the sand and disturb the scene. In the short space of time it took to take these shots it began to rain heavily, so I headed back to the house to process the images. I used the unprocessed jpegs straight off the camera, they seemed pretty clean and the cloudy conditions meant that there was no strong light or shadows to deal with. This time around I used Reality Capture for the photogrammetry, and it performed well, aligning every photo that I had taken which provided me with an 8 million poly high res model! Some decimation, texture cleanup and an export into Toolbag 3 yielded the results below. I picked an evening sky pano and stuck in a very low direct light to add rake shadows across the sandy surface, and a little fog. Inverting the albedo map gave just enough roughness to sell some dampness on the surface.

petemcnally_3d_scan_sand03

petemcnally_3d_scan_sand02

petemcnally_3d_scan_sand04

petemcnally_3d_scan_sand01

Under various PBR lighting conditions

If you’d like to check it out in real-time in your browser, check out the Marmoset Viewer scene here:

https://www.artstation.com/artwork/EX0xq