Agisoft Metashape
Agisoft Metashape => Face and Body Scanning => Topic started by: Mfranquelo on December 11, 2013, 01:43:28 PM
-
Dear all,
After many questions, doubts solved and problems addressed - Im very happy to share with you all my very first prototype of Stereophotogrammetry face scanner: :)
- https://www.dropbox.com/sc/suy6fwejacnbn50/aSMR2CZQ4E -
--------------- Key features: -------------
* I really wanted it to be "cheap" although at the cost of not being totally instant (it takes 2 seconds to go from 40 degrees to 140 degrees (there are angle limitations for the body, security of the cameras etc)
* It is important that the subject doesnt move - for alignment purposes.
* The scanning is done with total darkness as of yet, every camera opens their shutter for 1/50, in between that time, the flashes are fired at 1/16intensity.
* The prototype has a inclinometer (acelerometer) sensor on the rotation axis (similar to the ones our mobile phones have to detect inclination. (its the blue box)
* When the inclinometer detects the programmed ? angle it sends the shutter signal to each of the cameras and flashes
---------- Details -----------
Pre-mirror up signals are sent so the camera's total shutter lag is decreased.
- Time between fully shutter mechanical operations is around 0,4 seconds. That is : The camera is not ready to shoot again until 0,4 seconds.
- Cameras are Canon 700d (i bought those ones because they had better FPS)
- Flashes are Yongnuo 560 iii
- Lenses are canon 1.8f 50mm
- The radius of the semi circumference is 80cm
- Distance between cameras is around 26cm
* For 6 programmed angles, the total shooting time is 2 seconds of scanning.
(shoots every 20 degrees) 6 times. That is : 36 photos. (Maybe i?ll add two more cameras), but no more.
------------------------------------- ISSUES ------------------------------------------------
In my opinion, the fact that the flashes are moving relative to the subject is not very good for feature detection
Subject movement is a problem of course, but if i do the movement within 2 seconds and the subject tries to stand still, alignment is done correctly.
-----------
I Would really love to read opinions, thoughts or critics of course! :)
* I?ll keep this post up to date with any new tests, problems or results so that we can all share our experiences!
https://www.dropbox.com/sc/suy6fwejacnbn50/aSMR2CZQ4E
-
Nice idea!
i would love to see how well this setup performs.
really creative approach.
will be following this thread
-
Very innovative concept - love it!
Keep up the great work.
-
Emage, Mr_Curious Thank you!
I hope ill be able to post some tests tomorrow as well as some issues for sure! I just bought my photoscan license finally :)
Alexey, what do you think of this approach ? could you tell me any issues that the software could have with this system ?
-
Hello Manuel,
I think that the possible problems of such system you have already mentioned in your first post:
- slight movements of the person during scanning,
- change of the source of light position.
-
I promised some examples of my system, so here it goes :
(Those are very early tests, everything will be improved with noise projection)
Tell me what you think!
-
Mesh is still a little bit noisy, i guess i need more corresponding points to be found, at the moment i have around 39k.
-
Good work, can u pls upload textured one as well.
-
Okay, Ill post a few results (spraying paint on the subjects face) as well as texture rendering screenshot.
-
Results are still too noisy... i need to sort that out.
I dont like Zbrush smoothing brush, a filter as someone had mentioned here would be of course better, (Noise should of course be sorted out by increasing the scans quality...)
---
Im getting like 45k points, is that too few ?
Maybe the quality between correspondances is not good enough ? what numbers should i aim for ?
Thank you in advance for your help!
-
Did you sort out the resulation issue? :)
-
Not yet :(, ill try to fix the illumination source so it doesnt move relative to the subject, that may increase geometric reconstruction quality.
Also ill try Meshlab's noise filtering options.
-
right !... the lights should never move...
-
- Flashes are Yongnuo 560 iii
Hi Mfranquelo, thanks for sharing this - very interesting.
I hadn't really considered this type of flash before - but my memory is that light production period is around 40 µsec ?
Might you describe how you synchronise them ?
What sort of calculation did you do to decide on quantity ?
Does anybody know if the Yongnuo fit the spiffy gear projector [previously posted somewhere on the forum as http://www.the-digital-picture.com/Reviews/Spiffy-Gear-Light-Blaster.aspx (http://www.the-digital-picture.com/Reviews/Spiffy-Gear-Light-Blaster.aspx)] ?
Does anybody have an idea [to share] how many spiffy's might be needed for whole body capture?
Apologies for vagueness [lazy Sunday post]
-
FoodMan thats right, i didn?t know it was that important even with the light changing in a very diffuse way :(
andy_s Yongnuo's are amazing flashes that cost like 80 euro (where i live at least), they have more or less the same features as the so-expensive Canon 580 ex ii (This yongnuo flash is manual, however i dont care, for my needs)
I?ve sincronized them creating a custom cable and - a microcontroller was programmed to fire the flashes by closing the circuit exactly after the median shutter lag of my 6 cameras which is around 79ms + security time. This is done on total darkness, so the photos are mathematically syncronized.
I was about to test x2 light blaster + noise semi transparent slide, however i dont have the budget do this yet, so im faking the noise projection by painting little dots on my subjects face for the tests.
I can tell you the results when i have them. I guess that 3 light projects would be needed , or maybe two at 45 degrees to the face.
- For sure it fits, the yongnuo has the same shape as canon speedlites.
Quantity was just a guess, i kept adding flashes until i got a decent ratio between diffuse and intensity of light. Flash speed at 1/16 power was just more than enough for my needs. You could calculate light intesity fall off and reflectivity based on the flesh, but thats just a waste of time, its faster to test it.
Manuel.
-
Thanks Manuel - looking forward to more results.
-
Its improving ;)
-
Fantastic!
-
Its improving ;)
Wonderful...!
What have you done ? What did you change?
-
2 more cameras on the way for the ears.
- Ive made the light more diffuse, and i smoothed the mesh in Meshlab, and finally ive added a displacement map on zbrush with a wavelength decompose filter from GIMP.
- Mesoscopic high pass filter from disney research lab is so amazing (i hope one day i can get it on my hands)
-
Heres one with texture map and no displacement map.
-
yes, exactly improving...
Did you do Photoshop - Enhancing Photos with High Pass Filter ? if not, you may give a go.
http://abduzeedo.com/photoshop-quick-tips-3-enhancing-photos-high-pass-filter
What smoothing have you applied on Meshlab from filters section? Why not Zbrush smoothing?
Thx
-
Hello exhale
Im using this equivalent to photoshops high pass filter http://registry.gimp.org/node/11742 it works a little bit better from my tests.
I?ve applied "two step" smoothing filter, however it doesnt work as i?d like to, details arent preserved in the way i wanted. I might have a custom made bilateral denoise filter on the upcoming months... im not counting on it though. - Im very supprissed to not be able to find any software that have this kind of filter for 3d mesh... not even polyworks.
- The problem now with high pass filter is that my lights are moving relative to the subject so dark / light information is nowhere near accurate. I will change this on the following weeks.
-
http://faculty.cs.tamu.edu/schaefer/research/L0Smoothing.pdf I want this Denoise :'(
-
have a look at this one, pay attention also boosting.. that's cool...
http://www.youtube.com/watch?v=jliea54nNFM
there is no program lunched for this research :(
-
Wow,
I know :(, i?ve been researching code libraries and noone of the most used ones have implemented this smoothing algorithm.. most of them have old smoothing algorithms like laplacian, not even the bilateral denoise. Agisoft, you need to implement this algorithm! ;)
-
Have you improved your technic ? Is there less noise ?
Would you interested in uploading some photos :)
By the way, what have you done about the zbrush work flow?
Can you upload some compression photos please.
Good luck,
Thx
-
I?ve been away some time for vacations, im back now!
- Im adding two more cameras for my system so i?ll upload results this week or the next one. Noise should be less and it will allow me to record the ears as well.
- Im about to watch and study the videos that wishgranter shared so i will post results as well.
-
- Im routing a section of a face scan with 100 microns resolution on X , Y , Z axis.
I will post images of the physical object. :)
-
Nice rig !
On your installation mosaic picture it seems that the scanned person is laying on the floor which gives not very flatering faces with the fleshy parts moving backwards
couldn't you install it on the wall instead ?
-
FLuca - Thank you!, Yes, the next prototype will be fixed to the wall. This very first one was built on the floor for simplying sake
-
Hello,
I just wanted to share with you a routed physical test, geometry from agisoft as you know, and retouched a little bit with Zbrush.
Its not life size, the next text will be 1:1 :)
-
Awesome!
Cheers,
MC
-
Just curious, anyone know how ShapeShot by Direct Dimensions is pulling of such great ear to ear models with 4 cameras. I had a chance to have a picture taken at the makerbot store in NY when they had it, all 4 cams take one shot and its perfect. I was shocked. I attached my ugly mug :)
-
Just curious, anyone know how ShapeShot by Direct Dimensions is pulling of such great ear to ear models with 4 cameras. I had a chance to have a picture taken at the makerbot store in NY when they had it, all 4 cams take one shot and its perfect. I was shocked. I attached my ugly mug :)
LOL! For the record, Direct Dimensions isn't doing anything special whatsoever... nor is there anything special about their "system". DD's shapeshot system is based on the system built by Dimensional Imaging http://www.di3d.com/index.php (http://www.di3d.com/index.php). All DD has done is bolt a fancy cover on their system, put their logo on it and attach a flash. The reason the DD system can do it with 4 cameras and get great results is because in reality it is actually a Dimensional Imaging system and that requires you to pre-calibrate the system before using it. The system calibration is the crucial factor...along with the fact that the Dimensional Imaging system uses active photogrammetry rather than passive.
Cheers,
MC
-
Any chances to improve PS and make a system with calibration panel?
-
Any chances to improve PS and make a system with calibration panel?
Well obviously that's a question for the Agisoft developers but as with all things there are pro's and con's to both methods. From my point of view, one of the best aspects of Agisoft is the very fact that a lengthy calibration process is NOT necessary in order to scan... this is a HUGE time saver, especially when you are working with talent on location somewhere. Also, remember that a system where pre-calibration is required is often far less flexible... you can only work within a very specific volume/range and you are stuck with that. Yes, it would be interesting if Agisoft also had the ability to use a specific calibration...but then again, it might be somewhat limiting in other ways. I've used active/passive photogrammetry systems by Dimensional Imaging, 3DMD and others... in fact both INFINITE and Ten24 also have had direct experience with Dimensional Imaging... and notice that both of them are also using Agisoft, so I think that speaks pretty clearly. I also have an active photogrammetry machine that uses pre-calibration and I prefer the no calibration needed, passive Agisoft way of working myself. It really all depends on what you need to achieve... again, pro's and con's to everything and every system... there is no one one size fits all solution....yet.
Cheers,
MC
-
Interesting, I'm just starting out, so I only ask because as I was reading this thread I saw he has six cameras and wasn't getting the ears and he was getting two more cameras for the ears. As I am trying to set up a system to get an output like theres was curious how many cameras I would actually need, and what the best cameras are to use. Any help would be great
-
Interesting, I'm just starting out, so I only ask because as I was reading this thread I saw he has six cameras and wasn't getting the ears and he was getting two more cameras for the ears. As I am trying to set up a system to get an output like theres was curious how many cameras I would actually need, and what the best cameras are to use. Any help would be great
Ok, I'm not trying to be a pain, but honestly, the answer to this question is the more cameras you can have the better. If you are scanning humans then the problem will always be depth-of-field and occlusion and the only way to currently combat those issues are with more cameras... you simply can't have too many. So start off with a few and just keep building up your rig, little by little.
BTW - there are lots of threads on this forum with the topic of how many cameras has been discussed many, many times so you may want to search for more information. 20 - 30 cameras is a good range for heads... for bodies anything over 50 cameras generally and even better if you can double that number. Of course, you can always shoot your subjects on a turntable, which can reduce the number of cameras needed... but then you have to deal with all the problems associated with movement... not to mention it makes it very hard if not impossible to shoot dynamic poses which sort of defeats the whole purpose of a photogrammetry based rig anyway.
Once again, my advice is to just build up your rig/equipment overtime... little by little. Before you know it, you will have reached your goals.
Hope this helps.
Cheers,
MC
-
Brian33433 as an example, I have just started to set up a rig with 6 fixed cam with flash synchronisation
I get nice result for the front face, but the ears and back of the jaws are clearly missing details
I have tried assembling some chunks with photos taken with my head turned side to side but the alignement between chunk was not perfect and I will have to manually remove the gap between chunks as you can see in the attachement
12 cams seems the minimum for the face including ears (only front side of the ears)
good luck with your rig
-
Dear all,
I just wanted to share some updates on my lasts scans.. im working really hard to improve them
- Im implementing noise projection using 2 strobes on the next few weeks.
- I have to improve lighting
- Highpass is working nicely in some areas, but not on others. I heard infinite tried to contact disney research lab, are they willing to sell their mesoscopic algorithm ?
(raw scan from Agisoft, just applied texture bump)
- Anyways im quite happy!
Cheers!
-
- Anyways im quite happy!
Cheers!
You should be -- really nice result!
Keep up the great work.
Cheers,
MC
-
I see your getting the ears now, how many cameras are you up to?
-
8 cameras, just added two more on the sides.
-
- Mfranquelo, like the simplicity of your rig, any new photos of it? have you mounted it on the wall yet?
- General question, why does everyone use caps for the head, does it not render hair well?
B.
-
Dear all,
I just wanted to share some updates on my lasts scans.. im working really hard to improve them
- Im implementing noise projection using 2 strobes on the next few weeks.
- I have to improve lighting
- Highpass is working nicely in some areas, but not on others. I heard infinite tried to contact disney research lab, are they willing to sell their mesoscopic algorithm ?
(raw scan from Agisoft, just applied texture bump)
- Anyways im quite happy!
Cheers!
Great scan!!
One question, sorry if it's been covered before, but does using noise projection mean you won't get accurate color uv image of the scan and it will require hand painting for vleanup? how does all that work?
-
wowo great scan, and almost noisefree.. ! 8)
-
Hello exhale
Im using this equivalent to photoshops high pass filter http://registry.gimp.org/node/11742 it works a little bit better from my tests.
I?ve applied "two step" smoothing filter, however it doesnt work as i?d like to, details arent preserved in the way i wanted. I might have a custom made bilateral denoise filter on the upcoming months... im not counting on it though. - Im very supprissed to not be able to find any software that have this kind of filter for 3d mesh... not even polyworks.
- The problem now with high pass filter is that my lights are moving relative to the subject so dark / light information is nowhere near accurate. I will change this on the following weeks.
Do you mean that you take each photo individually and apply a high pass filter with the same parameters for each pictures?
-
Using noise projections means that you separate the process into two. One for geometry processing and the other one for texture (without the pattern) this could, and usually leads to mismatches. I haven?t tried it yet because im trying to print in 35mm slide at the highest resolution i can.
Im aiming to reproduce 38 microns dots projected onto my subjects face. I'll use stochastic photolith because it tops the resolution of a lambda or inkjet printer for transparent slides.
Thank you FoodMan!
- I apply a highpass filter onto the image agisoft generates, when calculating texture projection onto the models geometry.
-
My new Toy..., in development. :)
(the structure will be simplified..) Actually on the photo, the scanners structure is attached to other structure which doesnt have anything to do with it, just for stability and testing purposes.
-
Wow, that looks amazing!
Where did you get that epic sphere? It's purpose is to give a more even lighting?
-
Wow, that's really cool !!! Great concept, love it.
Looking forward to seeing how you progress with this, awesome!
Cheers,
MC
-
I casted it in resin from a routed polystyrene sphere ^^
Its has various purposes, one is to "psychologically" protect the scanned people from the fast movement of the semi circle, another one is to serve as an object to attach the custom made flashes, and the other one is to serve as a light bouncer.
-
great device.. can't wait to see some scans 8) 8)
f/
btw how fast to do a full shot..?
-
Thank you for your kind responses people!
- It takes 2.2 seconds for 48 images. (its limited to camera internal mechanics, thats why its 2.2 seconds)
-
I just noticed the giant stepper motor on the side XD
2.2 second for the whole scan? That is seriously fast! You don't need to add a small delay before taking the photos (to avoid camera shake)? I made a fast panorama robot a while back and I had to add 0.3 seconds delay before each shot to get rid of small wobble.
-
Yeah its a big servomotor, lifting 15kg from the axis takes a serious amount of resources hehe. (there are smaller and lighter ways however, like a linear actuator... but i decided to do it simple for now.
Cameras are synced by flash so shake is not really a problem. :)
-
When you are done, please make a small video of it in motion? I would love to see that!
Are you into building automation machines professionally?
Here is an older version of my panorama robot:
https://www.youtube.com/watch?v=SPgCeUcDdcs
The later version was much faster (it used timing belts instead of geared stepper motors, the geared versions had too much backlash)
What do you use to control the steppers? I would like to build an XY motion rig for a camera in the future. I have some stepper motors from Nanotec, but I was wondering if there was something easier.
-
Yeah its a big servomotor, lifting 15kg from the axis takes a serious amount of resources hehe. (there are smaller and lighter ways however, like a linear actuator... but i decided to do it simple for now.
Cameras are synced by flash so shake is not really a problem. :)
Is it a time machine ! ? :o
Would love to see this in motion. Very interesting concept :)
-
That's one crazy looking machine!
I would love to see a video of it working scanning someone too.
-
OMG...!
Since I lost track on this topic, you made great improvement Mfranquelo...! Congrat.
Can't wait to see some scans.
-
Out of curiosity: why do you rotate the cameras up/down like this? I would expect that it would be mechanically much easier to rotate them around your subject (from left to right)? (with the pivot of the arm located above your head).
-
H?mm, yes... Earlier, I was thinking the same. I think it was for only demonstration. ?
-
This looks amazing! Pretty nice rig.
Whenever you get a little more money saved, maybe you can think about constructing most of the stuff in carbon fiber since it is getting cheaper and it will help reduce the weight of the complete system and therefore the requirements for the stepper motor, bearings, etc.
It will also make it easier to rotate fast and stop fast without much vibration.
Leo
-
Lee - It does indeed, someone from my work mentioned this today :D dont miss me if someday i disappear...
I will prepare a home made video of the working system, tomorrow im going to test the optic sensors for triggering...
Wow thats an amazing panoramic robot marcel! i have a CLAUSS system on my home, but its WAY too expensive for what it does...
- Personally i dont know much about about automation systems, just the basics:
Im using a programmable inverter to control de servomotor, with an angular encoder attached. Its way too complicated for this task i know ^^. Im not the best person you can ask about the stepper motors. (I know that Maxon is a good servomotor manufacturer although).
- indeed Lambo, im just aiming for functionality right now, getting the most out of the cameras speed has been complicated for me as well as programming the logic of the micro controller, (im still a beginner in this field :-[ )
-
Out of curiosity: why do you rotate the cameras up/down like this? I would expect that it would be mechanically much easier to rotate them around your subject (from left to right)? (with the pivot of the arm located above your head).
- Well thats a good question, at the first stage of the prototype i thought that the gravity force could help the system... so i would just need some kind of hydraulic brake... But things got more complicated so i decided to use a servomotor, i might change it on the near future... (I have another concept drawing made on solidworks ready)
* Yeah its just for a demonstration for a surgery convention in Prague.
* we've developed a laser scanner as well, if you're curious have a look at www.factum-arte.com all the work we're doing is mainly for restoration and conservation purposes.
-
https://www.dropbox.com/sh/3mj3f4k1z5wx2fv/1ssE0x86tM
:) - Movement takes 3 seconds right now - for testing purposes, it will be 2.2 seconds soon.
Flashes are not in position, i was building my own micro flashes, (with just the bulb and the capacitor) but i got electroshocked, never manipulate a flash if you dont know perfectly what youre doing! :o ... so im using comercial ones at the moment ..
-
https://www.dropbox.com/sh/3mj3f4k1z5wx2fv/1ssE0x86tM
:) - Movement takes 3 seconds right now - for testing purposes, it will be 2.2 seconds soon.
Flashes are not in position, i was building my own micro flashes, (with just the bulb and the capacitor) but i got electroshocked, never manipulate a flash if you dont know perfectly what youre doing! :o ... so im using comercial ones at the moment ..
It looks very smart but 2.2 seconds is a long time in the world of face scanning. You can see the face moving even from this distance. Do you find this an issue during build? How do you solve that?
-
Building is perfect, no misalignments. :) (for neutral faces) i dont know for expressions that are hard to hold... I can post results when i attach the flashes next week.
In theory : If the pixel resolution used by Photoscan for building is less than "the resolution of the movement" then no building error should occur. Since when using "High" the image is down-sampled to 50% (i think?) the movement is not noticeable in pixels. (Alexey correct me if im wrong) ???
Anyway building looks good.
The only critical thing in this system is DOF and not blinding the subject with the so-close flashes. I might change to super-luminescent LEDS used like a flash, but i dont have enough budget for that yet.
- Of course its never better than instant system, but just cheaper... :)
-
I was experiencing this object movements & alignment with Agisoft®. I believe, PhotoScan® can correct some movements at the alignment, unless there is no face movement such as lips, eye-browns..etc. Mfranquelo , also you use something to make stabilization for the head, don't you?
What do you think Lee ?
I don't know perhaps you made the experiment what I'm going to write.. Let's imagine you have 20 cameras, 10 for lower face, another 10 for upper face. Make 2 different bunches with separate trigger. Trigger the first bunch, and a bit later (2-3 sec) trigger the second bunch. Have you ever tried sth like that? In this case, Does small movements effect so much?
-
Not yet... i was planning to build a head stabilizer like on the old photographic times. (more ergonomic ofc...) but just seeing my tests i dont need it, it is useful for placing the head "on the recording volume" however.