Point cloud / Ortho are skewed/tilted (no GCPs)

[ATMZF4]

Hi folks,

I gonna be straight up: Its my first try with Metashape, but i spend a couple hours already trying to fix this bug and searching for solutions. So the answer to this might be obvious... I have photos of a  building site shot with an UAV to generate an orthohoto. But after aligning the photos and every othger step after, the photos are tilted  and not "nadir" as they were shot. As a result the projected point cloud is also tilted but worse, also skewed.... which leads to othophotos where i can see the wall of a house etc... I think the problem is in the calibration of the camera, but i couldnt find the solution...

I the exact same problem already with pix4D with this project, but there is an option where you can prioritise internal camera parameters, which fixes this problem. I shot the photos with the DJI Mavic 2 Pro. If anyone could help me out, this would be great! been struggeling with this for days...

Cheers

Jan

[James]

Do the images have GPS coordinates, and are you using Standard or Pro edition?

If you're using standard, or the images don't have GPS coordinates, then I think you will have to manually align the chunk using the 'rotate model' tool.

I'm not sure if standard version works with gps coordinates or not.

[James]

Actually if you're generating an orthomosaic then you must be using Pro i think, so what does the reference pane look like?

This should contain camera GPS coordinates if any were found, and show any errors between the source coordinates and estimated positions after alignment.

[ATMZF4]

Hi James,

Thanks for your answer! Yes, I'm using the pro version. but dont have any GCPs. I attached a screenshot of the reference pane.

Thanks in advance for your effort!

[Paulo]

Hi ATMZF4,

I am wondering what type of flight plan you used^?  By any chance, is it a single line ?

The reason is your orientation error is quite high (30 degrees) for this type of gimballed flight....

I would try to do alignment with your orientation angles checked to see if you get a more nadir result....

Remember also that the Mavic 2 Pro  has a rolling shutter camera and this should be checked in your camera calibration window..

By the way, If you can share a small dataset of Mavic 2 Pro images, we could look at aligning it on our side...

[ATMZF4]

Hi Paulo,

Do you mean single line like shown in the image below? (screenshot of one mission plan, I fly the same mission on two altitudes with adjusted overlap). Yes, its really high. I have no idea why tho, as the pictures were taken straight to the ground. Okay i'll try that! Here is a link to the pictures taken at 95m above ground.

https://drive.google.com/open?id=1zN6dMgbDwZssitnZ1NOrj9bdi3hmjRPP

Thanks so much for your help!

Cheers

[James]

Just another thought while i download the images, if you flew 2 sets of images, at different altitudes it could be that although the GPS XY are good, the altitudes are not so good relative to each other (i don't understand how altitude is calculated but believe it can be a source of problems!), which may cause it to tilt up in an effort to reduce the overall error.

It would be easy to test that by unchecking the coordinates for either the upper or lower flight, and hitting the update button to see if it levels out.

[James]

Based on the images in that set, I'd say that Paulo is right about the rolling shutter correction.

It made a huge difference to the alignment and camera calibration, which was fairly nonsensical without it applied.

I found that I had to go into Tools->Camera Calibration and enable it before aligning cameras, as I couldn't get back to a sensible solution by enabling it retrospectively and then optimising if I'd already done the alignment.

Out of habit, after alignment, i also used Model->Gradual Selection to remove points with reprojection error greater than 0.3, reconstruction uncertainty greater than 100, and projection accuracy greater than 10 before doing an optimise, before building the mesh and orthomosaic, but the big difference was in enabling rolling shutter correction.

I haven't worked with that kind of image before, but it could be that 60 images over that sort of area at that altitude with that sort of camera, in just 2.5 minutes, is moving too fast.

You can have a look at my project here in case it helps https://we.tl/t-qFY25pcF8m

[ATMZF4]

Just another thought while i download the images, if you flew 2 sets of images, at different altitudes it could be that although the GPS XY are good, the altitudes are not so good relative to each other (i don't understand how altitude is calculated but believe it can be a source of problems!), which may cause it to tilt up in an effort to reduce the overall error.

It would be easy to test that by unchecking the coordinates for either the upper or lower flight, and hitting the update button to see if it levels out.

Would I do that by uncheking those photos in the Reference window in the camera column?

[ATMZF4]

Based on the images in that set, I'd say that Paulo is right about the rolling shutter correction.

It made a huge difference to the alignment and camera calibration, which was fairly nonsensical without it applied.

I found that I had to go into Tools->Camera Calibration and enable it before aligning cameras, as I couldn't get back to a sensible solution by enabling it retrospectively and then optimising if I'd already done the alignment.

Out of habit, after alignment, i also used Model->Gradual Selection to remove points with reprojection error greater than 0.3, reconstruction uncertainty greater than 100, and projection accuracy greater than 10 before doing an optimise, before building the mesh and orthomosaic, but the big difference was in enabling rolling shutter correction.

I haven't worked with that kind of image before, but it could be that 60 images over that sort of area at that altitude with that sort of camera, in just 2.5 minutes, is moving too fast.

You can have a look at my project here in case it helps https://we.tl/t-qFY25pcF8m

Thanks a lot! I had a look at the project you send and the error rates are much lower than the ones i have using the same phjtots and checking "Enabling rolling shutter compensation"... Im trying again to align them right now. Could it have an important impact wether i use "source", estimated or sequential? (left it on source till now)

[ATMZF4]

Okay, I don't know what i do wrong but I have an error of  14m (total error) instead of the 1,9m you have. Even tho I checked the " rolling shutter compensation" before aligning the pictures... Is there anything im missing?

Thank you so much!

BR

[Paulo]

 ::)James,

thanks for the update....

I get also a wild calibration with f varying greatly from initial value and huge principal point offset (almost  1/2 sensor size) using High Accuracy alignment 50 000 kpt, 10 000 tpt limit and both generic and reference pre selection options with rolling shutter compensation set in camera Calibration....

Now I am doing a new alignment without generic pre selection set to see if I get a better result as you did....

So what would be the reason for failed alignment when using generic pre selection? Of course without generic, matching takes much more time....

[Paulo]

Hi all,

I finally got a good result with following alignment parameters:

Alignment parameters   
Accuracy   High
Generic preselection   Yes
Reference preselection   Source
Key point limit   50,000
Tie point limit   5,000
Guided image matching   No
Adaptive camera model fitting   Yes

The trick, for me, was using adaptive camera fitting so that in initial alignment, f, cx and cy was not touched and then as James, I did gradual selection before final optimization with f, cx, cy, k1 to k3 and p1, p2....Also I reduced TiePt limit to 5 000 from 10 000.

But it is quite tricky to get a good result and I think with slow exposure time 1/640s. rolling shutter effect is quite great....

See attachement for result... Note that I also get quite a big error on GPS camera centers of 16 m.....

[Paulo]

Sorry,

my error was checking the image orientation values (yaw, pitch, roll) in adjustment. After unchecking the orientation angles, I now get results very similar to James for camera center position coordinates (errors less than 2 m)... It seems that the Gimbal Yaw values are quite erroneous in this drone, while pitch and roll are good...

So everything looks good now!

[Paulo]

Attached is report from this latest alignment....