Camera matching and points not seeming to align?

[Mr Whippy]

Hi all,

I quite like modelling cars in 3D.

I've taken photos of my car on a Ricoh GR and let Photoscan do it's work to get camera alignments really nice.


I then undistort and export images too for use in the background.

I export the model (patchy but usable for quick reference) to 3DS Max via FBX or 3DS format.



But for some reason points I place don't seem to line up so great.

Things towards the centre of the images usually line up ok over several images, but elements towards the edges of the shot seem to drift out by a few centimetres. Ie, a part near the centre of the car looks ok in several shots but as the camera distance increases from that point it can drift.

Is this normal?

Surely for Photoscan to work correctly these points should be perfectly aligned?

Is there any way to check inside Photoscan how accurate a camera position is? I usually use gradual selection and make sure reprojection error tie points over 0.25 or 0.5 or so are deleted (if this helps?)

Am I better shooting the checkerboard to get a lens profile to use from Lens, since currently I've just used the automatic Photoscan camera calibration detection.


Any help would be much appreciated since this workflow 'should' work, but I can't figure out why it's not.


Many thanks

Dave

[James]

Are you running optimize after gradual selection/sparse points deleting? That should help.

[Mr Whippy]

I'll have another bash at it.

I've never been too sure about the whole optimisation process in PS.

I wish there were a bit more automation in this regard since PS must know internally if things are better or worse than before... there is no way to really tell as a user if what you've just done has made things better or worse, or just not changed them... or even if certain cameras will now skip when doing dense reconstruction due to the Tie Point being lost during optimisation!


I'll post up pics and samples of my issues when I see them again. I'm certain they shouldn't be a problem so I want to get to the bottom of it!


Thanks

Dave

[stihl]

You can right click on the chunk in the workspace pane and select "Show Info" which shows you all sorts of information. I solely use this to go by when filtering the tie points.
Values to aim for at least is 1 pix RMS error and 3 pix Max error for the tie points. With this information you can decide whether the thing you just did was good or bad. The filtering values are relative to the project, so giving you straight up values doesn't help.

And if certain cameras are skipped because their tie points are too weak then they weren't good enough to begin with .

[Mr Whippy]

Here are my chunk details (I'm using non-pro version btw)

Please see attached document.

But basically.

50 cameras around the car.

6,857 of 16,284 points
2.859 overlap
0.474 error (0.996 max)


From what I see here this match should be really great.

I'll re-do everything and post up more details/images later since I'm guessing this workflow should give me perfect alignment from every camera on every point on the car!


Thanks

Dave

[Mr Whippy]

OK I ran it again and have much better figures now.

16,632 of 59,365
2.4856 overlap
0.129738 error, max 0.395091



But lots of points I generate in 3D at certain features on the car still don't line up very elegantly in the images. Some images seem solid, others don't.
Ie, I place a point in cam1 on the aerial tip, swap to cam 2 and move the point in cam1 orientation to the aerial tip and so on. Some other camera angles the point is on the aerial tip, others it's several centimetres away.


In the dense point cloud, which is obviously unreliable due to the reflective car, I can see ghosted/duplicated features that are offset, quite a lot in some areas (several centimetres).
Clearly some cameras are *not* in the right place but think they are. But there is no clear way to figure that out.


Is there any way to check each cameras alignment in PS?

Ie, 'look through' the camera, but also show the image in the background and the Tie Point cloud overlaid so I can see if there is an obvious error?


Thanks

Dave

[stihl]

If you right click on an image you can use the option "Look Through" which shows the orientation from that particular camera facing the tie-points. If you then double click on that image, it will show it and if you then click on "Show points" it will show the tie points found and used in that particular image.

[James]

Is there any way to check each cameras alignment in PS?

Ie, 'look through' the camera, but also show the image in the background and the Tie Point cloud overlaid so I can see if there is an obvious error?

It would be lovely - Feature request here: http://www.agisoft.com/forum/index.php?topic=4773

If you right click on an image you can use the option "Look Through"...

The only time i use the existing 'look through' feature is to get the model the right way up when i get confused or lost. It would be greatly improved if it could just show the undistorted image in the background, if only until you rotate the viewport.

[Mr Whippy]

Thanks for the thoughts. I knew there was a way to view the points somehow, but it's really hard to see the form of the car still.

Ah feature requested. Fingers crossed. It'd help understand problems like this much faster than going to FBX > Max and undistorting images and so on again and again.



Well I've been having a play and learn a bit about tie points and cropping them out and impacts on dense cloud creation etc.

It still seems quite 'magic' and not so intuitive what is going to happen at each step. It'd be nice if there were more interactivity with what has happened at each step to know you've got a 'solid' result before moving to the next steps.

But hey, it works. It's just these little things you kinda have to learn as you go along.


I've instead continued to focus on just scanning the car to a high accuracy to the point it's clear that the image and model deviate. I'm not sure how else to show the problem but the undistorted images still seem to not align with the model/point cloud.


So far I'm getting this kinda result (check attachment)... but I've already tried some other approaches to improve it in several ways, so once I combine them the results should start to get sharper again.

I'm waiting for the ideal weather but should be able to capture the whole car again in the coming week and really see how good I can get things.
Still not laser scan good, but without proper matching photos to build a spline cage from (till I find out how!), I can just remodel over this kinda data in point cloud form nice and quickly!

Cheers

Dave

[Mr Whippy]

Just more thoughts on this whole alignment thing.

I'd been using a wide area auto focus setting on the Ricoh GR which seemed sensible to make sure most of the elements of the car in view were in sharp focus. I obviously used f-7 or so for a fairly deep DOF.

But since I got the camera on a tripod today to do lots of testing today I really noticed how much the focus alters the resultant image recorded. OK it's subtle once the focus gets out beyond 1m, but it's probably enough between some shots which got a tighter focus vs others, to cause issues of misalignment in some photos.

I'm assuming some disconnect between the distortion correction outputs vs the internal corrections for dense cloud generation.


I've now halved my calibration error again by using a fixed focus for this task which is good.

Point clouds now seem more solid and have less surface noise.


I'll do more testing tomorrow to see if this fixes the problem I initally had.

[Mr Whippy]

Well for anyone interested my car results seem pretty robust.

Here is quick few screenshots to show the profile of the car at this point.
The profile here is way off what I've been able to fathom via the usual ortho blueprints, since this part of the car is hidden in blueprints... and the quality of the outline is easily as good as drawing splines over photo matched background images.
So I'll likely work over the model in this way to rebuild the final surface.

I've not worked on a laser scanned car before but these results so far seem quite good for photogrammetry taken in relative haste in non-ideal conditions!

You can even see where the wings sit slightly higher than the bonnet at each side of the car which is reassuring.

Next week I'll hopefully get chance to re-scan the whole car with all the best techniques so far and have a solid low noise full car scan.
I just need to try on some other colours now. I bet a black car or white car are tough!


Thanks

Dave

[FrankWolf]

Wow, your car scan looks great.
I want to scan a motorcycle, equally shiney, and I want paint job to be visible.

Did you use a polarising lines to avoid reflections?
Thanks!
Frank.
www.formato3d.com

[Mr Whippy]

Hey there,

I didn't use polarising for that specific reconstruction, but it can help depending on the surfaces orientation vs the camera.

At some point I'll write up some best/worst practice approaches for this job, but for now I'm testing different cars/colours and techniques still to find the best ones.

Also I'm remodelling this car which is driving me nuts. The scan is so accurate I'm having to consider the panel offsets and the way each panel edge is formed against the next one.

More detailed reference material just means more things to get right!