Workflow - Photoscan to game asset

[Andrew]

If you don't mind my asking, from your PS scans, do you essentially process until you end up with an optimized mesh + diffuse map and a normal map? Do you use it to extract any other data? Would you suggest sticking with Zbrush for everything after PS, or would you suggest other tools?

For a lot of scans we find that noise is not much of an issue. Especially for rocks and such, we just use raw scans as base for further work. From there, we either decimate automatically and optimize further manually, or build retopo from scratch if we feel we can get polycount down considerably, or whenever that allows for easier and more efficient layout of UVs. Then we bake down normalmaps from highres, and occasionally we derive some specular maps as well, but that is tricky and far from ideal. As for specific tools we use - everyone has their favourite tools in the office, here's the list:
Zbrush (I actually hate the UI), Mudbox, Max, Topogun, Roadkill, Unfold3d, Balancer.

-Andrew

[EMULAT3D]

Good thing our actors don't read internet forums, they would hate to learn they supposedly do a really bad job of looking like proper human beings

LOL, this definitely made me smile. It's amazing how game artists see things wrong with reality (faces and nature itself).

Gamegoof, I really think that your old company gave up too quickly on photogrammetry. Yes, its very process heavy and requires good artists to master, but a lot of that learning curve can be greatly reduced with good workflows. I use photogrammetry at my day job in simulation, and in our last project we shipped with 35+ faces...all captured with a simple rig and cleaned up by artists...not technical artists. But this was only possible because of proper planning and pre-production. My company gave me the necessary time to create the tools, processes and templates to make the cleanup process faster and more consistent. They saw the potential early on, that intermediate artists (not traditionally character) would be able to produce end results far faster and more realistic than sculpting by hand.

Thanks!

[Andrew]

Autodesk wont be spitting out game ready assets. Yet. I believe that is their intention

Game assets are, and will always be, about efficiency as well as looks. For that reason I don't count on, I don't even want my photogrammetry software to build game ready assets for me. I will always be able to build assets that look the same but take, say, 15% less triangles and 15% better UV fill ratio and 15% crisper textures. And I'll always prefer to go the extra mile to gain few additional frames per second so the game runs smoother.

Having said that, I can certainly appreciate good tools that get me as far as possible with as little hassle as possible, but since functionality like smoothing out noisy mesh, building retopo or adaptive decimation already exists in other programs, I encourage Agisoft to keep improving speed and quality of photogrammetry-related algorithms first, and only when that is as good as it gets, to focus on further steps of asset pipeline. Actually increased precision, lower noise of reconstructions and better texture blending (that would pick sharpest photos for texture generation) would go a long way towards speeding up further steps in the pipeline.

-Andrew

[gamegoof]

Having said that, I can certainly appreciate good tools that get me as far as possible with as little hassle as possible, but since functionality like smoothing out noisy mesh, building retopo or adaptive decimation already exists in other programs, I encourage Agisoft to keep improving speed and quality of photogrammetry-related algorithms first, and only when that is as good as it gets, to focus on further steps of asset pipeline. Actually increased precision, lower noise of reconstructions and better texture blending (that would pick sharpest photos for texture generation) would go a long way towards speeding up further steps in the pipeline.

Agree on all those counts, this just feels like Im back in 1996 trying to edit quake levels with no documentation or tools. Does any recommend a tutorial for the process? is there any 1 stop tools? 3D coat seems interesting..

And yes Emulated, I believe they gave up too quickly as well. In terms of face scanning its a given and PS almost loved this kind of work, because its already an expensive and important process, money has been spent on this. My old work paid to have this kind of scan done externally and weren't interested in buying 25k worth of gear to do it in-house. But there is no doubt facial scanning is being used worldwide but I think a few of us are approaching scanning from an environment perspective, which is slower to gain traction...

[EMULAT3D]

My old work paid to have this kind of scan done externally and weren't interested in buying 25k worth of gear to do it in-house.

Man, what I wouldn't give to have $25k for a highend rig, to be honest our little rig at my work (10 camera) cost us around $1500 to build. I think that is one of the misconceptions of photogrammetry as well, you only need the fidelity that will support your end goal. If it is film/tv/fullbody then yes, you'd want a rig like big guys on the forum, but for the quality we were aiming for our little inexpensive system worked just fine.

[gamegoof]

My old work paid to have this kind of scan done externally and weren't interested in buying 25k worth of gear to do it in-house.

Man, what I wouldn't give to have $25k for a highend rig, to be honest our little rig at my work (10 camera) cost us around $1500 to build. I think that is one of the misconceptions of photogrammetry as well, you only need the fidelity that will support your end goal. If it is film/tv/fullbody then yes, you'd want a rig like big guys on the forum, but for the quality we were aiming for our little inexpensive system worked just fine.

Well this was a massive game company that had already spent 20million and full production hadn't even begun yet so... What should have been a drop in the bucket turned out to be too much of a commitment but really, it was probably equal to their yearly sandwich budget for their fancy leads meetings...

[admir]

Have to agree with Andrew here, Photoscan, like Mocap/Facial rig isnt ever going to be magic bullet, solve all problems and spit out game ready assets.
However, it can produce at least in case of characters, good base with proportions, colors and fairly detail geometry (pending on your setup). If you are working on realistic (believable) game where you need real world architecture, characters, vehicles... Photoscan is the way to go and blows 123d big time.
Making game ready assets is something that needs time and work no matter are you using scans/sculpting or what ever approach.
From my experience in gamedev, finding artists to do retopo/bake/mesh lods.... is much easier than finding artists to do sculpts in quality you would get from 3d scanning and time needed for person to do sculpt/texture is much higher than to do actual scanning.

Admir

Autodesk wont be spitting out game ready assets. Yet. I believe that is their intention

Game assets are, and will always be, about efficiency as well as looks. For that reason I don't count on, I don't even want my photogrammetry software to build game ready assets for me. I will always be able to build assets that look the same but take, say, 15% less triangles and 15% better UV fill ratio and 15% crisper textures. And I'll always prefer to go the extra mile to gain few additional frames per second so the game runs smoother.

Having said that, I can certainly appreciate good tools that get me as far as possible with as little hassle as possible, but since functionality like smoothing out noisy mesh, building retopo or adaptive decimation already exists in other programs, I encourage Agisoft to keep improving speed and quality of photogrammetry-related algorithms first, and only when that is as good as it gets, to focus on further steps of asset pipeline. Actually increased precision, lower noise of reconstructions and better texture blending (that would pick sharpest photos for texture generation) would go a long way towards speeding up further steps in the pipeline.

-Andrew

[Mr Whippy]

Have to agree with Andrew here, Photoscan, like Mocap/Facial rig isnt ever going to be magic bullet, solve all problems and spit out game ready assets.
However, it can produce at least in case of characters, good base with proportions, colors and fairly detail geometry (pending on your setup). If you are working on realistic (believable) game where you need real world architecture, characters, vehicles... Photoscan is the way to go and blows 123d big time.
Making game ready assets is something that needs time and work no matter are you using scans/sculpting or what ever approach.
From my experience in gamedev, finding artists to do retopo/bake/mesh lods.... is much easier than finding artists to do sculpts in quality you would get from 3d scanning and time needed for person to do sculpt/texture is much higher than to do actual scanning.

Admir

I completely agree. You buy Photoscan for the SFM, and it's outputs.

Going too far down the routes for making assets you'd use straight out of Photoscan would seem like a waste of the devs time.

For rapid prototype work the basic tools are already good enough, and for re-working to high quality assets the meshers and tools are already pretty good too.

Not to say there isn't room for improvement, but I hope the developers don't spend too much time trying to make magic bullets, and instead focus on really useful usable SFM features that allow you to get more from your input images to begin with!



In the end nothing is fire and forget, and get good results out at the end. Go laser scanning with ?30,000 scanners, or GPS based helical scanning, and you can suffer from problems still.

Every approach has its pros and cons, and all have problems no matter how good the equipment or conditions are.

SFM and the Agisoft software are excellent in my view! So much so that I've just bought the software. Now for a UAV haha. Maybe a large mono-pod will do for now!


Thanks

Dave

[thesystemera]

Hi, I've been looking into a whole host of technology in regards to generating ultra-realistic game engine assets. I have an Oculus DKII coming in a month or so and am not too worried about poly-count as we are creating very small (but dense) environments. We have been working with 3D scanning, using technology such as a multiple Kinect Rig and I have always been a fan of Photosynth (remembering the wow factor when seeing their first TEDTalk). But I am absolutely blown away by how far Photogrammetry had came, I thought this was the stuff of science fiction.

Andrew, your blog was an absolute mind opener, and the big studios should be kicking themselves especially as Oculus comes into the scene, we will be striving for Ultra-realism and you can't get any more real that reality itself. Your work is inspiring and I would love to have the opportunity to discuss the process in detail when and if you have the chance. We are only just starting our studio and I'm sure your knowledge could help us reduce chances of making some costly mistakes. 

Okay I think that's enough praise LOL No, really, good stuff

[Andrew]

Thanks for the kind words, and I couldn't agree more that for applications where immersion is key (hello VR!), photogrammetry is just amazing.

We are just starting a bit of a crunch to get our game finally out the door So I strongly encourage you to search the forum as it has a ton of useful information scattered. This might turn out more efficient than asking me - I might be slow and brief to reply in the coming weeks. Having said that, if you can't find what you are looking for, pm me and I'll see what I can do to help