Forum

Author Topic: Smart Shooter, Nikon D3200 and noiseprojection.  (Read 36115 times)

Magnus

  • Jr. Member
  • **
  • Posts: 83
    • View Profile
Smart Shooter, Nikon D3200 and noiseprojection.
« on: June 12, 2013, 02:08:00 AM »
Hello all!

My brother and I have been doing lots of testing the last couple of months and things are really progressing (and changing drastically along the way). I thought I'd just share some initial points of interest.

Currently we have 16xCanon 1100D, 2xCanon 600D and 4xNikon D3200 which actually works without issues on just one laptop running Smart Shooter (no liveview on the D3200 though). We will be adding more Nikons soon and see if any issues pop up.
Might seem like an odd mix but there is a reason behind it, hehe. 
What we've been experimenting with is full-body capture using noise projected by DLP projectors and texture captured by cameras synced with flash.
The Nikons (as Lee and others has reported elsewhere) are damn good! We've managed to use our 4 at 1/200 shutter with flash (half-press seems to increase reliability, it also gives the option of having autofocus even with flash).
Our plan for full-body capture is (which could always change as we've experienced during our previous testing ;D) to use the Canons at low f-stop and around 1/50 or so shutter together with projectors to get geometry and the Nikons at high shutterspeed and f-stop at almost the same time synced with flash to get texture (it will not see the projected pattern due to the high settings). This will not be able to capture movement but should provide a decent starting point. It also seems as if, with some care, that even somewhat useable results can be gotten from shiny fabrics like satin for instance (varying the size of the projected pattern etc).

We will also use the same cameras for portrait, something we are quite close to getting (we will be trying with different types of hair to see what's possible).

I will share some pictures and some more info as it progresses.

Best, Magnus.

andy_s

  • Full Member
  • ***
  • Posts: 147
    • View Profile
Re: Smart Shooter, Nikon D3200 and noiseprojection.
« Reply #1 on: June 13, 2013, 11:47:17 AM »
Interesting stuff [as always] Magnus. Looking forward to this one hitting the streets.

If the rotating light tent [that never was] had been fitted with wings it might have "taken off"  ;) - super research though !
« Last Edit: June 13, 2013, 02:08:31 PM by andy_s »

Infinite

  • Sr. Member
  • ****
  • Posts: 365
    • View Profile
Re: Smart Shooter, Nikon D3200 and noiseprojection.
« Reply #2 on: June 14, 2013, 01:08:42 PM »
I'm interested to see your results.

I can tell you from experience you need SUPER bright projectors. I feel the current range of projectors even in 2013 are lacking in functionality compared to price. Much like HMD's were in the 90's. Companies are milking the franchise/industry.

There just aren't any suitable affordable projectors on the market. I know because I have a large collection of varying types. 20+

Also if you try and use more than 4 projectors at HD you will suffer problems, you need allot of signal boosters and powered splitters. Sending an HD signal across multiple projectors is very hard, without producing drop outs. It's possible but tricky. Something to bare in mind. This is for 360 capture though. 1 or 2 projectors is easy, especially at 1280x800.

So back to my main point. I wouldn't recommend anything lower than 4000 lumens.

LED and laser are the future. Laser not producing any DOF what so ever and LED with very long life spans.

Current DLP's deteriorate very quickly  :-\ Manufacturers also strongly recommend NOT to use them in portrait as they burn out, fast!
« Last Edit: June 14, 2013, 01:12:46 PM by Infinite »
_______________________________________________
I N F I N I T E
www.ir-ltd.net

Magnus

  • Jr. Member
  • **
  • Posts: 83
    • View Profile
Re: Smart Shooter, Nikon D3200 and noiseprojection.
« Reply #3 on: June 15, 2013, 11:50:45 PM »
Hello!

We started testing this due to the troubles we found with dof when we got to shoulder-to-shoulder, had to get the 1100Ds in pretty close to get them to see the clothtexture and it would mean having to go the route of more cameras, hehe. We found that the lack of brightness wasn't such a problem due to being so zoomed out.

Here are some of our earlier results. As can maybe be discerned from the screencaps this was with only 2 projectors (Acer H5360 and Optoma HD65), both of them are 720P.
This wasn't masked or anything and is pretty crude but should give a hint of what's possible.
The cameras were set at f5.6, 1/20th shutter and iso100. It was shot at 18mm with the kit lenses.
Zoomed out like this, with a distance of about 1 m to the subject, f5.6 should give about 76 cm near limit and a total dof of around 73 cm.

https://www.dropbox.com/s/gd6g2kkpgs5xa6e/130520_Brustest_01.jpg

https://www.dropbox.com/s/buvpcqajolxrvss/130520_Brustest_02.jpg

I hope those works, hehe.

On this test we first took shots with projectors on, with a noisepattern, then changed to black (layers in Gimp... hence the scrollbars and menus being present :P just testing after all... ) and took another bunch of shots with flash (this was without proper diffusion, just a test). Then we did the model from the noiseshots and then switched them out and did texture with the flashshots.

https://www.dropbox.com/s/e67tc3ug217r3yu/130523_Brustest_01.jpg

I will have more stuff to share soon.

Andy, the light tent might actually be resurrected... more to come ;)

Best, Magnus.

Infinite

  • Sr. Member
  • ****
  • Posts: 365
    • View Profile
Re: Smart Shooter, Nikon D3200 and noiseprojection.
« Reply #4 on: June 16, 2013, 12:16:00 AM »
Some very interesting test results. It will good to see future tests and how the faces, hands and full 360 would come out. For 1100D the results are very acceptable! f5.6 is certainly OK for standing arms down type shots, but anything more dynamic with greater distances involved DOF will kick in.
_______________________________________________
I N F I N I T E
www.ir-ltd.net

mala

  • Full Member
  • ***
  • Posts: 109
    • View Profile
Re: Smart Shooter, Nikon D3200 and noiseprojection.
« Reply #5 on: June 16, 2013, 01:14:14 AM »
Nice work Magnus
The results are looking good!

jeancampos

  • Newbie
  • *
  • Posts: 7
    • View Profile
Re: Smart Shooter, Nikon D3200 and noiseprojection.
« Reply #6 on: June 16, 2013, 03:57:41 AM »
Magnus, thank you for sharing your work process.

But I did not understand exactly how noiseprojection contributed to improving the scan. Could you explain?

The results are really exciting.  ;D

Magnus

  • Jr. Member
  • **
  • Posts: 83
    • View Profile
Re: Smart Shooter, Nikon D3200 and noiseprojection.
« Reply #7 on: June 16, 2013, 10:35:39 PM »
Hello!

Lee, most likely we will use the head captured this way more for reference (or add some zoomed in cameras). I will try some more with different distances and such later (we are getting short on space though, hehe).

Thanks, mala.

Jean, since I use few cameras and their view will cover a larger area I cannot get enough fine detail in the images that PhotoScan needs for reconstruction. This is also a problem if the subject you are shooting is too uniform. So using projected noise PhotoScan can find a lot more points it can use. 
I hope these following images will make it a bit clearer.

This first one was shot just with flash and as you can see it got 15877 points.
https://www.dropbox.com/s/3f7v9iwnhjz8afj/130523_Brustest_04.jpg
Here is the resulting model.
https://www.dropbox.com/s/c1t1z8xjc7plr5i/130523_Brustest_05.jpg

This next one we used the projectors too and this is what we got, 45733 points.
https://www.dropbox.com/s/p5kyc8rm6zgmbn0/130523_Brustest_02.jpg
Here is the resulting model from this one.
https://www.dropbox.com/s/hmbp5l825znd4td/130523_Brustest_03.jpg

So as you can see it does make quite a difference and I think this can be quite useful. Of note should also be that this was with two projectors and not too much care taken to coverage so keep that in mind.

Best, Magnus.

Magnus

  • Jr. Member
  • **
  • Posts: 83
    • View Profile
Re: Smart Shooter, Nikon D3200 and noiseprojection.
« Reply #8 on: June 16, 2013, 10:54:44 PM »
Hello again!

Just a small addition.

This subject is one of those plastic mannequin which has basically no texture, without noise projected it would not yield any usable points. The red part of the clothing is a shiny satin material which also gives huge problems. The results are noisy (especially the satin since it creates reflections) but better than nothing, hehe.

https://www.dropbox.com/s/6s1kj0cz9lbzfpx/130607_Brustest_01.jpg

This was with only one projector from the bottom front and the texture is from the D3200s which was synced with flash (the projector was on the whole time but due to the high settings on the D3200s the noise doesn't show up). We used 4 of them for texture and they were up high so coverage is not great, hehe. What's really neat is that PhotoScan manages to properly align the D3200s with the other cameras even though the lighting is very different. :D

Best, Magnus.

Infinite

  • Sr. Member
  • ****
  • Posts: 365
    • View Profile
Re: Smart Shooter, Nikon D3200 and noiseprojection.
« Reply #9 on: June 16, 2013, 11:26:38 PM »
Very interesting and inspiring test results Magnus. Projection surely has many benefits with less cameras and certain clothing. I would be interested to know the settings you used for noise cameras and flash cameras? I've always had some form of noise visible if all cameras are synced together, using mixed settings.

It's possible of course to fire twice but even the slightest delay between fires you will get subject movement. So that method isn't desirable.
_______________________________________________
I N F I N I T E
www.ir-ltd.net

Magnus

  • Jr. Member
  • **
  • Posts: 83
    • View Profile
Re: Smart Shooter, Nikon D3200 and noiseprojection.
« Reply #10 on: June 17, 2013, 01:17:15 AM »
Hello Lee!

For this test we had the Nikons at f14 and 1/200th shutter as highest and the Canons at f4.5 and 1/20th shutter, iso100 for both. There is actually a tiny bit of noise still present on the lower part of the satin where it shines the most and a bit on the legs (for this we used the Benq W710ST and it was pretty close to the subject). Turning down the brightness should solve that but I was incorrect in my previous statement. Sorry about that.

Here's an example of the one I posted before but this time with only flash.

https://www.dropbox.com/s/hkiyi08ieux7jpz/130607_Brustest_02.jpg

There will be some slight delay due to the D3200 going first with flash and then the Canons so it is not a perfect system but atleast a good steppingstone, hehe.

Best, Magnus.

Magnus

  • Jr. Member
  • **
  • Posts: 83
    • View Profile
Re: Smart Shooter, Nikon D3200 and noiseprojection.
« Reply #11 on: June 17, 2013, 01:43:11 AM »
Hello again!

Here's a crop of two D3200 images at f14 1/200 and f4.5 and 1/20 which shows the noise still present.

https://www.dropbox.com/s/z0aoflqpxvlidkq/130617_Brustest_01.jpg

Best, Magnus.

jeancampos

  • Newbie
  • *
  • Posts: 7
    • View Profile
Re: Smart Shooter, Nikon D3200 and noiseprojection.
« Reply #12 on: June 17, 2013, 04:38:01 AM »
Hello Magnus,

The results are really amazing, even using few cameras, you have done a great job. Comparing the two images you sent is the sharp jump in the quality of the calculation points Photoscan.

The choice of D3200 unlike the canons, the investment was for a better result in the capture of color?

I Think about how to apply it to results of facial scan, capture small pores and skin surfaces. Have you tried using projection and continuous light at the same time? It's a question I have, as spoken by Lee would have to switch the time between catching texture and capture points, correct?

Best Jean Campos

Infinite

  • Sr. Member
  • ****
  • Posts: 365
    • View Profile
Re: Smart Shooter, Nikon D3200 and noiseprojection.
« Reply #13 on: June 17, 2013, 11:46:38 AM »
Hello again!

Here's a crop of two D3200 images at f14 1/200 and f4.5 and 1/20 which shows the noise still present.

https://www.dropbox.com/s/z0aoflqpxvlidkq/130617_Brustest_01.jpg

Best, Magnus.

Very interesting results Magnus. It's the same method I tried back in 2009-10 with scanner killer and DI's software. Noise being present in the colour is always an issue sadly. But noise projection greatly improves results.

Doing split double capture of noise and colour is possible but even if we are able to sync at 1/200 you have to have a significant delay between each shot 0.24 seconds min, which leads to allot of subject movement  :-[ which means a big mismatch on using the colour texture on the noise scan. For static object it's fine.

An ideal scenario is infra red noise projection but this is very hard to do. As there isn't a decent IR projection system on the market and would also be costly converting DSLR to IR.
_______________________________________________
I N F I N I T E
www.ir-ltd.net

Magnus

  • Jr. Member
  • **
  • Posts: 83
    • View Profile
Re: Smart Shooter, Nikon D3200 and noiseprojection.
« Reply #14 on: June 19, 2013, 02:46:10 AM »
Hello!

Jean, actually a big reason that we got the D3200s was that they got supported by Smart Shooter and that their price was cheaper than the Canon 600D, hehe. With the 1100D I cannot quite get the level of detail that I want (for portrait) in the way that I want.
We will be going the Nikon route since they are great and we have some more D3200s on the way!
We intend to use the same rig for capturing faces but this will be done without any noise, it is not actually needed for this and would probably create more problems due to reflections.

Here you can see some results that we've gotten.

In this first one we processed at High quality. We had the front cameras closer to the subject and the cameras on the back a bit further out.
https://www.dropbox.com/s/9ej3fs4i2fs02fi/130610_Brustest_01.jpg
Much better results than this can be had by having the cameras closer together (closer stereo-pairs) but we are trying to find some good middle ground with coverage in mind too.

This is from the same as before but cropped and processed at Ultra High.
https://www.dropbox.com/s/2lnfom78rfbly12/130612_Brustest_01.jpg

We will process at High setting since the Ultra High takes too long and we retrieve the small details from the texture map as can be seen in this one.
https://www.dropbox.com/s/uj872lln5lth30f/130615_ZBtest_01.jpg
For this I decimated the model down to 200000 in PS and then I just did smoothing on it and no other cleanup. I am a beginner when it comes to post-processing so keep that in mind.

I have done tests with continuous light but the problem is that I currently don't have strong enough light to have them usable when diffused. Also for the portrait they aren't enough since one has to have pretty high f stop to get enough dof.

Lee, I seem to recall that SK had some pretty good guides about noise projection, hehe. I have some things in mind that I will be trying and I will share the results here later.
Funny you mention IR, I was actually thinking about having some cams converted (seems to cost about as much as a camera though :P) and using a bunch of kinects with the glasses mod (so as to minimize the patterns) to do this but I simply cannot due to budget reasons, hehe.
Another thought in that vein was making DIY projectors with some such lightsource.
I guess with the upcoming Kinect 2 and others that those might be a waste though, hehe.

Best, Magnus.