Turntable troubles

[BT106]

Hi Agisoft Community,

I'm hoping you may be able to assist me. I've been able to get some rough models generated when I walked around the subject but because the project I'm developing demands higher quality models to be imported into VR headsets I decided to invest in a photography turntable, however the results have been less than stellar. At first a lot of my photos looked as if they were all coming from a single point and not as if the camera rotated around the subject (like a turntable is supposed to mimic). I've been working off of this tutorial:

https://www.youtube.com/watch?v=9_F-b2hxP_o

So I used the "From Background" method of masking and Agisoft got most of the background but I had to manual go in and add the missing parts of the selection till I got something a little tighter (see images). This got my a better result (it actually looks more like the little robot I'm shooting) but it's still pretty garbled. I'm using a light tent and trying to keep the lighting as flat as possible and taking multiple shots but I'm still not getting the model to look the way I need it to look.

Am I missing something? Or is there some setting I need to put Agisoft on to process turntable photos? Is it because I'm taking too many photos (last scan of this robot was 59 photos)

I using a 16 mp Sony a55 DSLR Camera with 18-55mm lens
Light tent
Arqsping Turntable
Remote cable shutter trigger set at 1 photo/sec
Shooting all images in JPG & RAw

Once I have these models I was wondering  what the best way is to get a HD model without it killing my computer. Low poly model with HD texture? I need it to be able to operate withing a VR headset which that in and of itself is a big challenge.

Any help you can provide is greatly appreciated!

[ekbmuts]

From the work I've done with turntables and from looking at the screen grabs you included and your description of what you did, it might help if you could tell us:

1.  Are you shooting RAW?

2.  Are all your shots in-focus?  Are you looking through the viewfinder and confirming that your shots are in-focus before you shoot?

3.  What's your ISO set at?

4.  What's your aperture/f-stop?

5.  Is the model in the centre of your turntable or off to one side?  I'm wondering because from your camera positions it looks like Agisoft is not getting it or something? 

6.  I don't think you're taking too many photos.  I'm certain of that.

7.  Did you filter your photos (right-click on one of your images in the "Photos" pane and select "Estimate Image Quality...") and disable those that Photoscan returned a below-0.5 report for?  I find that sometimes helps.

8.  You could try making your own masks in Photoscan.  They don't have to be exact.  Just draw a rough marquee around the robot in each of your photos and see how that does.  I have found that just a rough mask works fine.

Hope this helps.

Jon

[BT106]

Hey Jon,  really appreciate this, thanks.

1.  Are you shooting RAW?
I'm shooting JPEG & Raw, but right now I'm using the JPEG's in Agisoft

2.  Are all your shots in-focus?  Are you looking through the viewfinder and confirming that your shots are in-focus before you shoot?
There were a few that were a touch blurry, what would you recommend in order for the images to remain sharp while using a turn table?

3.  What's your ISO set at?

100

4.  What's your aperture/f-stop?
f5.6

5.  Is the model in the center of your turntable or off to one side?  I'm wondering because from your camera positions it looks like Agisoft is not getting it or something?
It's in the center of my turntable, I've placed a little dot right in the middle of it after measuring the diameter out.

6.  I don't think you're taking too many photos.  I'm certain of that.
Great, one I get a feel for it, I want to then start doing shots at different angles with 30-45 shots/angle. Is that reasonable?

7.  Did you filter your photos (right-click on one of your images in the "Photos" pane and select "Estimate Image Quality...") and disable those that Photoscan returned a below-0.5 report for?  I find that sometimes helps.
I've right clicked on the images and have done estimate and it looks like it runs through it but I don't see where the results are.

8.  You could try making your own masks in Photoscan.  They don't have to be exact.  Just draw a rough marquee around the robot in each of your photos and see how that does.  I have found that just a rough mask works fine.
Can I apply a mask in bulk to the rest of the photos in Agisoft or do I have to do that externally?

My next round of photos will be food focused, if that should make a differences in what I need to do coming up.

[ekbmuts]

BT106,

Sorry for the delay.

So:

1.  I'd recommend shooting RAW and converting those RAW images to 16-bit TIFF files.  JPEG, in my opinion, are too compressed for Photoscan.  You CAN use them.  But the likelihood of there being compression artifacts is high and Photoscan won't appreciate that.  If you have Photoshop, you can convert your RAW images to TIFF automatically.  If you don't, there's probably another way to do it.

Alternately, I understand that Photoscan now handles .dng files.  .dng is a raw format.  It's an open-source Adobe product.  There must be converters on the internet.  I'm assuming your raw files are either .NEF (Nikon) or .CR2 (Canon).

Either way, you want to use your RAW files either as .dng or converted to 16-bit TIFF.

Let me know if you have any trouble with this and I'll help you out.

2.  On the focus, got it.  In Photoscan, in the "Photos" pane, change the viewer to "details".  Then run that "Image Quality" button again.  In the "details" view there will now be a column with numerical values that reflect the quality of each image.  Photoscan recommends that you disable any shots below 0.5.  Hopefully you don't have that many.  Simply select the images below 0.5, right-click and select "disable."

3.  ISO:  Perfect.  100 is great.

4.  I think your f-stop is too low.  I would say put it up to f11 or even f16.  I'm not sure if you're a photographer but this higher setting makes the opening that the light passes through in your lens smaller and increases depth of field.  Meaning, more of your model will be in-focus from front to back.  Having shallow depth of field in Photoscan is useless.  We want as much of that model in-focus, front-to-back, as possible.

WARNING:  You may find that ISO 100, f11 or f16 will give you dark photos.  You might find you need a LOT of light.  But, seeing as your camera is probably on a tripod, you can use a long shutter speed to make up for this.  Use a cable release or set it to delayed shutter so there's no camera shake and you should be fine.  It's all about super crisp, super low noise, super evenly lit shots. 

5.  Got it on your model being in the center of your turntable.  That's good.  Here's another tip, print a scale chart and put your model ON it.  A student named Samantha Porter made some for her work that are downloadable and she also put together some terrific tutorials.  Check these links:

https://www.youtube.com/watch?v=qKWjPNlRK_I

I couldn't find the link to those scales so I've included them.  You could always write to her if you want.

These scales will also help you scale your eventual model.

6.  On the quantity of photos,  yes:  shoot at those angles.  I generally shoot head-on, from top looking down and (if possible) from down looking up.  That way you get every nook and cranny.  Personally, I don't think you can take too many photos.  You can always disable in Photoscan if you find you have too many. 

7.  As covered above (image quality setting).

8.  Well, this masking business has many different techniques.  Try just pulling your shots into Photoscan and manually masking each one with Photoscan's masking tools.  Skip the external masks for now.  This sounds tedious and it will be.  But you don't have to make exact masks.  Just draw a marquee around the general shape of your model and move on to the next one.  I learned this trick at a seminar.  I'd recommend watching it.  It's fairly long but this guy knows what he's talking about.  Here's a link:

http://livestream.com/gnomon

You will want to watch "An Evening with Alex Alvarez."  It looks like it's archived but that doesn't mean it's not available.  It's an A to Z on Agisoft Photoscan as well as being really interesting stuff.  And Alex Alvarez is quite inspired so it's not boring or anything like that.

Okay. I think I answered your questions. This photogrammetry can be tough going at first.  So many pitfalls.  But this forum is a great place to learn and if you've got the time, there are some terrific videos and papers on the internet.

Hopefully this is helpful.  I'm no expert yet but I've been at this for a little while and I can get some very nice looking models with my rudimentary training...

Let me know if you need anything else.

Jon

[ekbmuts]

BT106,  One last thing:

You are shooting your shots with the turntable motionless, right?

In other words, you have a still turntable and you take a shot.  Then you rotate the turntable (for example) 10 degrees, stop it and then take another shot, rotate it 10 more degrees, stop it and take another shot and so on.

If you have that turntable in perpetual motion and your camera in some sort of machine-gun mode where it's just taking shot after shot after shot as the turntable rotates, then this is probably causing you trouble.

I don't think you are shooting like the second example above but wanted to check.

Jon

[ekbmuts]

Found the link for those scale sheets:

http://www.stporter.com/resources/

Check her whole site out.  Pretty cool.

Jon

[Kiesel]

@BT106,

your bad results came from your very difficult to scan, smooth object (robot) with a nearly textureless surface, where PhotoScan can't find enough keypoints and tiepoints. (Try to scan for example a rough stone or a potatoe and you will get a good scan and see what a difference it make).

One solution for a better camera alignment is to put something with a good texture under your object like a newspaper, a cork plate or something similar, the scale sheet with the painting recommended by ekbmuts is also something like that.

Then you should get a better alignment but the surface of your object hasn't changed and is smooth as before.  So what you need in addition is a better texture on your object. One way to do it, is to project a random pattern on your object. You can see how to work with pattern projection in PhotoScan here:
http://www.pi3dscan.com/index.php/instructions/item/agisoft-how-to-process-a-scan-with-projection

found in http://www.agisoft.com/community/articles/

Karsten

[BT106]

@ekbmuts
Hey Jon

Brilliant! Thank you so much!

I'm actually using a Sony A-55 which when set to JPEG&Raw gives me .ARW files. I'll search around for a converter to swtich them to .TIFF as I've seen a lot of good mode apparently use that format.

I'm going to do a reshoot with the new settings (ISO 100, F11, RAW) and work on getting white balancing right with this camera and see what results I can come up with. It looks like maybe one the of the reasons the initial model came out like crap was because my images weren't crisp enough, I followed your recommendation to check the 'image quality' and most (if not all) were running between .24-.56 so that needs some work to get it on point. I think part of it had to do with me shooting on the turntable, you are right in your guess that I was photographing while it was in motion, since then I've marked of 10 degree increments on the turntables face with a protractor to measure things out.

Also really appreciate you pointing me in the direction of that tutorial series, between the masking and the merging/editing chunks a lot of fog was lifted on next steps to take and the functions of the software that I have yet to get into and I've cued up An Evening with Alex Alvarez, I'm looking forward to checking it out.

@Kiesel
And thank you as well Karsten!

You bring up a great point which completely slipped my mind, I remembered reading up on the manual and recall how the tutorials said that Photoscan doesn't do textureless surfaces that well. I was thinking that somehow it would be able to recognize that it is a simple geometric object and use the marking as reference points and apply the paintings as a texture but I guess not. Welp you live and you learn! Also thanks a bunch for posting that link to the large scale sheet as well, between yours and @ekbmuts I think I'm in a good position to do larger scans...later down the line when I have some experience under my belt of course.


I'll be sure to report back in at the end of this week with a status update on the new scans. Thanks again for all your help everyone 

[ekbmuts]

BT106,  Great!  I think you're all set for now.

I've worked with .ARW files.  You can convert them to TIFF in Photoshop CS6, with Adobe Camera Raw (ACR) plugin version 9.1.1.  That's what I use.  I'm sure you can do it with other programs and certainly any of the Photoshop cloud versions but I'm on CS6.

Good luck!

Jon

[BT106]

Well I couldn't wait until the weekend because I'm really eager to get this up and running with my VR project so Ileft work early and came home to re-shoot with the suggestions made and WOW did it make a huge difference!

I marked off on my turntable 10 degree increments and show 2 full rotations of my little statue at 2 different heights in RAW (and then converted them to TIFF) and at first I ran into the same problem as before (the point cloud looked more like a shotgun blast than a rough model Pic #1) but I realized I didn't mask my object so I went in masked each of my photos by hand (I'll still need to work out a better batch process to expedite the process but this is ok for now) and that massively improved the point cloud. I did the estimated image quality and this time around I got mostly .65-.80's so I cleared off the .55's and below, after that it was all pretty straight forward I did batch process to align the photos and build the mesh and then put on the texture. (Picture 2)

I'm going to have to test these models in my Oculus Rift to see how many face it can support smoothly, at first my object has 200,000 but then I decimated it to 100,000 (Picture 3) I'm aiming to have something down in the 15,000 range for performance but to still look appealing. Is it possible to have a low poly count but with a very crisp texture or does it all end up looking wonky?

Also Picture 4 is what my shots mainly looked like, I feel like it was little on the bright side even though it gave a pretty good result. Would lowering the brightness help?

And what would be the  best way to fill in holes, I tried using the tool but it didn't seem to catch.

Thanks again for everything! It finally feels like I'm starting to get the hang of this 

[BT106]

Pictures 3 & 4

[Kiesel]

Congratulation for your first success! 

Much better object btw., PhotoScan like such objects!

Some little tips:

Fill your photos with as much of your object as possible!

...so I went in masked each of my photos by hand (I'll still need to work out a better batch process to expedite the process but this is ok for now)

Took a background photo without your object before or after your photo session and let PhotScan do the masking by "Tools/ Import/ Import masks/ by background".

Is it possible to have a low poly count but with a very crisp texture

Yes! You can even edit your mesh in an external application (You can also do the hole filling there) then reimport your modified mesh and let PhotoScan do the texturing in full resolution. 

Karsten

[ekbmuts]

What an amazing difference!

And if I knew what it was, I'd say, "That's a fine looking ......."  But I'm embarrassed to say that my best guess is that it's an under-water fossil.   

So, what is it?

I see what you mean about the masking.  Without it, everything just falls apart, huh?  And I agree: masking is pretty tedious but why don't you try doing rough masks in Photoscan, on each photo.  Just draw a marquee around your model on each shot - it doesn't even have to be close to exact. 

Alternately, for each of your camera angles, shoot a "blank plate", that is, take the model off the lazy susan and shoot the blank background.  Do this for every angle that you shoot.  Not every shot, but every angle.  You shot 2 angles, so you'd have 2 blank plates.  You can tell Photoscan to create masks based on your shots and the blank plates.  It does an internal difference matte.  There's a video on this somewhere that I'll see if I can dig up for you.  It's also covered in the Alex Alvarez video.

As far as Oculus Rift goes, unfortunately, I can't help you with that.  I'm a CGI artist myself and I use these models in 3D programs.  But I can tell you that ZBrush is an excellent tool for modifying and reducing poly counts as well as filling holes.  Alex Alvarez goes into this in DETAIL in his seminar.  And if you don't have it, ZBrush is quite inexpensive.  Some of this is a little over my head, I must admit, but he gets some really, really nice models at the other end of his pipeline.

I'd say you should carve out a couple of hours and watch that Alvarez video.  Have a note pad to-hand.

On your shots being too bright - I don't think so.  But these are peewee little pictures so I can't really tell.  I can tell you that over-exposure is your enemy and to see if your shots are over-exposed (particularly on the model itself) just pull the RAW files into Camera Raw, select one of them and in the controls on the right move the "Exposure" slider while holding the ALT key.  Anything in the image that is over-exposed will show up as red, yellow or green (I think, I might have those colors wrong) while the rest of the image will go black.  This is a very handy, little-known-about tool in Camera Raw.  If they are over-exposed, just slide the exposure down a little.  Then, click on "Select All" in the top left-hand corner and then "Synchronize", which will apply that exposure correction to all of your images.

Of course, if your shots are way over-exposed and "clipped", you won't get any detail back.  By "clipped" we mean that the highlights have gone entirely white and there is no hope to pull any detail out by lowering the exposure in Camera Raw.

And while we're on the subject, it's a good idea to remove chromatic aberration from all your shots as well.  That button is on one of the later tabs in the controls area of Camera Raw.

I just looked at that shot #4.  I don't think over-exposure is a problem.  The background is a little bright, but I don't think you should worry about that too much.  The model itself is fine.

Finally, to answer your question about whether or not you can have a low-poly model with a nice texture, yes: you can.  At a certain point however, the low-poly starts to show. I shot a ballerina when I was starting and knocked the poly count way down but put the full-res texture on.  She looked fine head-on but as soon as I started to move around her, you could see her flat face and that sort of thing.  But do some tests and you'll see what works and what doesn't.

I get what you're saying about just wanting to get back to this and not wait until the weekend.  Sounds like you've got the bug...

Jon

[BT106]

What an amazing difference!

And if I knew what it was, I'd say, "That's a fine looking ......."  But I'm embarrassed to say that my best guess is that it's an under-water fossil.   


So, what is it?

Lol, your guess is as good as mine, but yeah it's some sort of fossil, I'm happy with the way it turned out.

I see what you mean about the masking.  Without it, everything just falls apart, huh?  And I agree: masking is pretty tedious but why don't you try doing rough masks in Photoscan, on each photo.  Just draw a marquee around your model on each shot - it doesn't even have to be close to exact. 

Alternately, for each of your camera angles, shoot a "blank plate", that is, take the model off the lazy susan and shoot the blank background.  Do this for every angle that you shoot.  Not every shot, but every angle.  You shot 2 angles, so you'd have 2 blank plates.  You can tell Photoscan to create masks based on your shots and the blank plates.  It does an internal difference matte.

I tried shooting the black backing of my photo tent by for some reason my camera won't focus and take the shot, guess there's nothing to focus in on and it won't snap the shot even in manual mode. Any suggestions?

Just pull the RAW files into Camera Raw, select one of them and in the controls on the right move the "Exposure" slider while holding the ALT key.  Anything in the image that is over-exposed will show up as red, yellow or green (I think, I might have those colors wrong) while the rest of the image will go black.  This is a very handy, little-known-about tool in Camera Raw.  If they are over-exposed, just slide the exposure down a little.  Then, click on "Select All" in the top left-hand corner and then "Synchronize", which will apply that exposure correction to all of your images.

Is Camera Raw an external program or something within Photoscan?

The background is a little bright, but I don't think you should worry about that too much. 

I'm going to work on getting better lighting so I can make it as flat as possible so Photoscan can pick up as much detail as possible.

The model itself is fine.

I get what you're saying about just wanting to get back to this and not wait until the weekend.  Sounds like you've got the bug...

Haha I totally do have the bug. I went around my town last night and snapped a few shots with my cellphone and practiced getting the workflow down and even with some rough pictures I got some ok looking models. With more practice in a controlled environments (instead of me looking over my shoulder at Macy's to keep an eye out for any security guards looking for someone circling a mannequin like a crazy person) I'm sure I can get some really good looking models. 

[ekbmuts]

BT106, Wow!  Yeah, you're cooking now.  And LOL about Macy's... I can picture what that must have been like...

To answer your questions (not in the same order as you asked them):

- Adobe Camera Raw is a plugin for Photoshop.  It's a great tool for not only converting RAW files to TIFF or jpeg or png or whatever you want but also for "processing" the RAW files before you convert them.  You can do all manner of adjustments from exposure, white balance, chromatic aberration, noise removal, etc., etc.  And the beauty of working with RAW images is you can often get back details that you might have lost in both too dark and too bright areas.

- On that mask that you're shooting without your model, use the same focus that you had on your model.  It sounds like you might be in auto-focus mode, in which case, you might want to switch to manual focus.  (Your camera cannot auto-focus on black because it requires some contrast to focus on and black has no contrast).  With a high f-stop (I.e. f11), you can set focus on your model once and it will be fine throughout all the rotations.  That's as long as the model is in the centre of your scale sheet, which it usually is.  With focus set on the model manually, just remove the model and snap a shot of the background.  Don't worry about that shot of the background being in focus.   You want it to be the same focus as it is when the model is in the frame so that Photoscan can recognize it and then key it out. 

Don't change lighting or anything.  Just take the model out and click the shutter again.  Photoscan will do some sort of "difference matte", which is to say, it looks at the shots of the model and then the background "plate" that you shot and it works out what the difference is between all the model shots and the blank background shot (one at  a time) and from that it generates a mask for each shot.

If this doesn't make sense, Google "difference matte" and you'll get the idea.

- Lighting.  Yeah - as flat as possible.  Preferably no shadows at all.

And finally, for shots taken on the fly with a cell phone, those models that you made look pretty okay to me!

Jon