Photogrammetry and Torque 3D: Week 3 -First Results!

Photogrammetry and Torque 3D: Week 3 -First Results!

As many of you already know, I have been experimenting with a different way of producing models for game engines. I started off working with laser scanning and have since moved over to Photogrammetry. What is photogrammetry you might ask. Well, according to wikipedia photogrammetry is the science of taking measurements from photos and extracting exact positions for surface points. Sounds complicated and well, you would be correct in that assumption.

I was first introduced to this method of modeling when I purchased and played 'The Vanishing of Ethan Carter' created by 'The Astronauts'. They are a group of Polish game developers who split from Epic Games Poland aka 'People Can Fly' whom you may know from titles like 'Bulletstorm' and 'Painkiller' as well as ports for Gears of War. I enjoyed many of those games but, I can not ever recall being 'floored' by the visuals.

After playing the game, I started doing what I always do. Researching methods for how I could 'incorporate' more of the real world into my virtual worlds. Like I said, I already own the David Laser scanning system and it does really well for certain things. However, giant boulders and crazy dead trees and large areas of 'nature' are just not suited for scanning (at my price range anyway.)

ENTER PHOTOGRAMMETRY

The system that The Astronauts used to create most of the game assets in TVoEC was Agisoft PhotoScan. This is a neat little piece of software from a developer in Russia. I initially checked it out but, I was a tad 'sticker shocked' by the 'Pro' edition price. So I left and went back to my laser scans. I got involved in a conversation on one of my blogs here and found out I was mistaken. I did not need the 'pro' edition for what I wanted to do. I am not planning on measuring archeology sites or documenting the flow of rivers or any of that other crazy stuff. So I returned and found the Standard Edition for around $180 US Dollars. I tried it before I purchased it using a 30 day trial. I did 2 sets of 'scans' and I was SOLD.

So, I am currently learning Agisoft PhotoScan Standard Edition. You can see more about this nifty piece of software by following this link; www.agisoft.com. The concept is pretty simple, you take a series of photos of an object. Put those photos in the software and magically it produces a 3D model. Sounds simple, alas there is much more to it.

I started out with just my trusty 14 megapixel GE400X camera, a crap load of batteries and a dream. Needless to say, the equipment has grown a tad. I am still using the same camera but, now I drag along a bag with 'markers', tape, a solid tripod, a crap load+ of batteries, my trusty laptop and a bunch of SD cards, tape measure and laser line level and pointer (don't ask).

I had to re-learn photography. The closest I can compare photo-scan photography to would be texture photography. No harsh shadows (I live for cloudy days now), low ISO, no distance blur, no digital zoom, cut back on the Monsters (gotta have steady hands) etc. Then you have to walk around in circles taking photos (I mean that literally.... circles.) Some of the images below took anywhere between 50 and 120 individual shots, and I should have done more on many of them. At 14 megapixels per picture that is a TON of visual data. All those images eat up 16 gigs of ram on my development machine FAST. It's no joke, the software chews up ram like crazy, also I HIGHLY recommend a Geforce video card. They incorporated CUDA into the point cloud analyzing segment of the process and the GPU helps speed up the processing time GREATLY.

So, my first couple of simple experiments went ok. You can see some of the results below. These are all rendered in Blender using the Cycles Renderer.

.
.
.
Not bad but, how the heck do I get these high poly models into Torque? (Some of these are over 1/2 million polys ALONE.) Not only that but, I uninstalled 3D Studio Max and now I am ONLY using Blender. Well, BACK to the research. I have been experimenting and working out a solid workflow in order to make this happen. Alas, I have some results I am happy to show. Keep in mind these are experimental images and I just tossed them into my current build of T3D. The only major things I did to my build was incorporate Verve, the DirectX 9 Refactor, and a bunch of other little 'fixes and tweeks'. For the most part, it's stock T3D.

.
.
.
.
So, using NOTHING but Blender, Meshlab, and a TON of photos I am now able to produce these simple shots. (basically just to test the lighting and performance.) Heck, I even used the default tree just for leaf shadows. The maximum poly count is 2K for one model (that would be that little model with the dead tree growing out of the rock.) The lowest poly count is 250 (the path pieces) All of the models are also HEAVILY LODed. I am pleased so far with the results. It sort of reminds me of Crysis (the original) on Max setting for PC. The textures are still a bit muddy but, that is my workflow, NOT the system. Heck the original textures on the high poly models are actually true 4K images. You can zoom in and count grains of sand on those. So the texture problem is in my workflow. You will note some spots where they look almost 'smeared'. I think that is a bad "Object Cage' when baking the diffuse in Blender.

SO, WHAT IS NEXT?

Like I said, I am not thrilled with the texture quality on the low poly versions of these models. They could be better and once I get my brain more wrapped around Blender's 'cage' system for baking textures I am sure I can improve them even more. I am able to make really nice normal maps from the high poly models and project them onto the low poly model properly so, it has to be something with sample sizes or the cage is not properly aligned.

I started small, little rocks, twigs, ect. I moved up to objects (most are in the screen shots) that are slightly bigger than cars. Therefore, my next step is buildings. I think I have a system planned out that will allow me photo-scan the exterior and the interior of a building and merge them together. I will start with exteriors and interiors alone first, then move on from there. I also intend on mocking up a small interior project for testing with the Rift.

People/Humans, yep I am going to try photo-scanning a person. I am in the process of building a neat little rig that will allow for overlapping photos that can take photos from a set point on a set of outer rings that encircle the subject. This way I can use one camera to take the photo set. (Many large studios use multiple cameras and a geo-dome looking rig but, I am not that wealthy so, gotta make do.)

Basically, I need to build a HUGE library of this stuff. Thankfully, between my laptops, and my desktops I can have machines grinding on this stuff pretty much 24/7 for awhile.


IN CONCLUSION

Overall, I think this system of working will become my new 'norm'. Since the higher quality meshes are faster to produce, look better in the engine (in my opinion) and add the level of realism I have been after, I can't see going back to my old methods. Once I get a solid flexible work flow in Blender outlined, I will post it for everyone to read and review. (I may do a series of video tutorials on this whole process as well...I don't know yet. I am still pretty 'clunky' in Blender but, I should get more efficient as I go).

The last stage will be building a demo level in T3D/UnReal and Unity and video to show off what can be done. More to follow on this grand adventure. Thanks for reading and if you have any questions feel free to ask. I am by no means an expert but, I will try and help.

Ron