Agisoft Metashape

Agisoft Metashape => General => Topic started by: JyunPingJhan on May 11, 2022, 02:10:30 PM

Title: Rolling shutter compensation in 1.8 does not correct all parameters
Post by: JyunPingJhan on May 11, 2022, 02:10:30 PM
Hi

I am using a rolling shutter camera and check the compensation to increase the alignment accuracy

However, I find that the compensation in ver 1.8 does not corret all rolling shutter parameters and results in image measurement errors of markers, while 1.6 does.

Are there bugs in 1.8 ?

Please see attchement for details.
Title: Re: Rolling shutter compensation in 1.8 does not correct all parameters
Post by: Paulo on May 11, 2022, 04:19:19 PM
Hello Jyun,

Since version 1.7.4, rolling shutter compensation has been restricted to using only 2 parameters Tx and Ty. Quote from support:
Quote
The rolling shutter model introduces a lot of additional degrees of freedom for cameras optimization. This sometimes leads to rolling shutter parameters divergence as camera model becomes too general.
To increase alignment stability, in version 1.7.4 we constrained rolling shutter model to translation in xy-plane.

Now if you want to use full compensation model (6 parameters) then you can use following tweaks (since version 1.8 ) :
Quote
You need to use 2 tweaks for rolling shutter algorithm:
"AlignCameras/full_shutter_model = True"
"OptimizeCameras/full_shutter_model = True"

Try them and see if it improves results after redoing alignment and optimization
Title: Re: Rolling shutter compensation in 1.8 does not correct all parameters
Post by: JyunPingJhan on May 11, 2022, 09:54:36 PM
Thanks, it works now.
Title: Re: Rolling shutter compensation in 1.8 does not correct all parameters
Post by: mdasilva on May 28, 2022, 05:48:02 AM


Now if you want to use full compensation model (6 parameters) then you can use following tweaks (since version 1.8 ) :
Quote
You need to use 2 tweaks for rolling shutter algorithm:
"AlignCameras/full_shutter_model = True"
"OptimizeCameras/full_shutter_model = True"



Could you clarify how you apply these? I don't see full_shutter_model within the Python API nor the Metashape guide, so not sure how to do this. My z error estimatesare horrendous after reprocessing with the Rolling Shutter Compensation.

thanks
Title: Re: Rolling shutter compensation in 1.8 does not correct all parameters
Post by: Paulo on May 28, 2022, 06:05:43 AM
hi mdasilva,

yes the tweaks are set in Tools Menu/Preferences. In Advanced tab you will see a  Tweaks button, click and there you set the tweaks...as in capture screen below

Title: Re: Rolling shutter compensation in 1.8 does not correct all parameters
Post by: mdasilva on May 28, 2022, 06:28:45 AM
hi mdasilva,

yes the tweaks are set in Tools Menu/Preferences. In Advanced tab you will see a  Tweaks button, click and there you set the tweaks...as in capture screen below

Thank you! The learning curve of this software is driving me crazy. I've been processing/re-processing for months now trying to optimise all the settings and understand what they do.

Any chance you've got a link to where one finds other available Tweaks?
Title: Re: Rolling shutter compensation in 1.8 does not correct all parameters
Post by: Paulo on May 28, 2022, 11:39:53 AM
Unfortunately not!

Tweaks are a little mystery that would really need a better documentation....
Title: Re: Rolling shutter compensation in 1.8 does not correct all parameters
Post by: mdasilva on May 28, 2022, 12:24:49 PM
Unfortunately not!

Tweaks are a little mystery that would really need a better documentation....

Thanks Paulo, I really don't understand how Agisoft hasn't addressed this in its documentation. From what I can tell, Rolling Shutter Compensations (RSC) are NOT being applied to z coordinates (unless with additional Tweaks that are not listed or documented anywhere). Across the internet I'm seeing people saying to not use RSC if their data looks better without it, which essentially means people are producing models/3d data without RSC because the software is such a black box (even though their camera specification is a rolling shutter camera).

I'm probably going to make a fresh post addressing this soon, no idea how to be confident I'm using the optimised settings for the software otherwise.
Title: Re: Rolling shutter compensation in 1.8 does not correct all parameters
Post by: Paulo on May 28, 2022, 01:02:43 PM
mdsilva,

what I can say is that if you use tweak (full_shutter_model True), then the model will use 6 parameters for camera center correction during camera image readout: 3 translation (Tx,Ty,Tz) and 3 rotations (Omega, Phi, Kappa). Whereas if you do not use tweak, it will use, for rolling shutter correction, only 2 parameters (Tx, Ty) i.e. translation in camera XY plane..

But IMHO, the limited shutter model (default) should at least have 3 parameters (Tx, Ty, Tz) and not just 2 as presently....As camera movement during readout is not mainly limited to camera XY plane, especially in case of obliques...
Title: Re: Rolling shutter compensation in 1.8 does not correct all parameters
Post by: Alexey Pasumansky on May 30, 2022, 04:17:34 PM
In the version 1.8.4 (pre-release of build 14465 is already available via direct link) it would be possible to select the rolling shutter compensation mode in Camera Calibration dialog from the following options:
- disabled,
- XY regularized (default, as in 1.8.3 without tweaks),
- Full.
Title: Re: Rolling shutter compensation in 1.8 does not correct all parameters
Post by: RHenriques on May 30, 2022, 11:58:45 PM
Hi Alexey

Taking advantage of some notes suggested in some messages above, it would be interesting to create a detailed manual about tweaks, their function and respective parameters. Some are extremely interesting and useful and the syntax is unknown to most users. I apologize if this information already exists and I am the one who does not know where to access it.
Best Regards
Title: Re: Rolling shutter compensation in 1.8 does not correct all parameters
Post by: mdasilva on June 01, 2022, 08:19:26 AM
In the version 1.8.4 (pre-release of build 14465 is already available via direct link) it would be possible to select the rolling shutter compensation mode in Camera Calibration dialog from the following options:
- disabled,
- XY regularized (default, as in 1.8.3 without tweaks),
- Full.

Hi Alexey,
I'm not able to follow what you are saying. Could you please help me understand?

Is 1.8.4 released? Does it do something different to 1.8.3 in regards to Camera Calibration? Or have you just added an option to apply it within Optimize Camera Alignment?

thanks
Title: Re: Rolling shutter compensation in 1.8 does not correct all parameters
Post by: DayGeckoArt on June 03, 2022, 06:28:35 AM
If you check all the boxes when you align or optimize cameras, will that take care of the skew from rolling shutter? And what if all your photos don't have rolling shutter? Will they all get the same calibration and get messed up if you have a few rolling shutter pictures in your set?

I shoot manually in raw, not with an automated flight program. Sometimes I end up with a few photos where I didn't let the drone stop fully
Title: Re: Rolling shutter compensation in 1.8 does not correct all parameters
Post by: mdasilva on June 03, 2022, 07:58:14 AM
If you check all the boxes when you align or optimize cameras, will that take care of the skew from rolling shutter? And what if all your photos don't have rolling shutter? Will they all get the same calibration and get messed up if you have a few rolling shutter pictures in your set?

I shoot manually in raw, not with an automated flight program. Sometimes I end up with a few photos where I didn't let the drone stop fully

Hi,
I've been experimenting for weeks with different camera optimisation parameters. It seems like the RS correction is causing the other optimisation parameters to go absolutely haywire.

The drone that I used, Mavic 2 Pro, has an electronic shutter (rolling shutter) so it is recommended to use the correction.
Title: Re: Rolling shutter compensation in 1.8 does not correct all parameters
Post by: mdasilva on June 09, 2022, 04:08:12 AM
In the version 1.8.4 (pre-release of build 14465 is already available via direct link) it would be possible to select the rolling shutter compensation mode in Camera Calibration dialog from the following options:
- disabled,
- XY regularized (default, as in 1.8.3 without tweaks),
- Full.

Hi Alexey,
I'm not able to follow what you are saying. Could you please help me understand?

Is 1.8.4 released? Does it do something different to 1.8.3 in regards to Camera Calibration? Or have you just added an option to apply it within Optimize Camera Alignment?

thanks

Hi,
Bumping this to try to get some clarity on this.
Title: Re: Rolling shutter compensation in 1.8 does not correct all parameters
Post by: DayGeckoArt on June 10, 2022, 11:01:36 AM
I recently tried loading camera calibration data exported from another project where I know there was no rolling shutter effect. Then I checked the rolling shutter compensation box. I figured this way the program would have the full calibration data, but the adjust the rolling shutter parameters only. The results seemed good
Title: Re: Rolling shutter compensation in 1.8 does not correct all parameters
Post by: mdasilva on June 25, 2022, 06:36:45 AM
In the version 1.8.4 (pre-release of build 14465 is already available via direct link) it would be possible to select the rolling shutter compensation mode in Camera Calibration dialog from the following options:
- disabled,
- XY regularized (default, as in 1.8.3 without tweaks),
- Full.

Hi Alexey,
I'm not able to follow what you are saying. Could you please help me understand?

Is 1.8.4 released? Does it do something different to 1.8.3 in regards to Camera Calibration? Or have you just added an option to apply it within Optimize Camera Alignment?

thanks

Hi,
Bumping this to try to get some clarity on this.