Forum

Author Topic: Rolling shutter compensation  (Read 2351 times)

ManyPixels

  • Newbie
  • *
  • Posts: 40
    • View Profile
Rolling shutter compensation
« on: June 29, 2023, 04:20:36 PM »
Hi,

Using a precalibrated system, I realised that the rolling shutter has a huge impact on final camera positions. Knowing accurate speed, heading, orientation and position for each frame and the sensor readout time, is there a way to help the software to get more accurate? Position already helps a lot, but sometimes alignment still goes wrong.

Thanks!

lunar

  • Newbie
  • *
  • Posts: 36
    • View Profile
Re: Rolling shutter compensation
« Reply #1 on: July 24, 2023, 03:40:34 PM »
Hi
 try this:   MetaShape-> tools -> camera calibration -> for each group of photos select "rolling shutter compensation" = full

ManyPixels

  • Newbie
  • *
  • Posts: 40
    • View Profile
Re: Rolling shutter compensation
« Reply #2 on: July 24, 2023, 04:11:31 PM »
Thanks, but I know that. The question is, if it is possible to fix some of the parameters estimated in this compensation (which is precalibration).

lunar

  • Newbie
  • *
  • Posts: 36
    • View Profile
Re: Rolling shutter compensation
« Reply #3 on: July 24, 2023, 06:52:55 PM »
You can calibrate each camera using camera-> calibrate camera - chessboard procedure
Then in calibrate camera panel you can then choose witch parameters are FIXED / image variant
« Last Edit: July 24, 2023, 06:55:49 PM by lunar »

ManyPixels

  • Newbie
  • *
  • Posts: 40
    • View Profile
Re: Rolling shutter compensation
« Reply #4 on: July 24, 2023, 09:06:39 PM »
Sure, but it does not include parameters estimated in the rolling shutter calibration. If you take for example the trajectory and speed, and could be given by the user but it can't be saved from a previous project. These informations would help when you're activating the rolling shutter compensation because the distorsions added by the rolling shutter can make your alignment go totally wrong, That's what I'm looking for...

lunar

  • Newbie
  • *
  • Posts: 36
    • View Profile
Re: Rolling shutter compensation
« Reply #5 on: July 25, 2023, 12:47:20 AM »
Hi
I did a small test - photos with rolling shutter  (dji mini2 cam). divided into a group with compensation in the full option, and the other with off.
align with ADAPTIVE option (medium)
results:
« Last Edit: July 25, 2023, 12:52:31 AM by lunar »

lunar

  • Newbie
  • *
  • Posts: 36
    • View Profile
Re: Rolling shutter compensation
« Reply #6 on: July 25, 2023, 12:48:34 AM »
rolling compensation off

lunar

  • Newbie
  • *
  • Posts: 36
    • View Profile
Re: Rolling shutter compensation
« Reply #7 on: July 25, 2023, 12:51:21 AM »
I am having trouble interpreting this data, only k3 & k4 are equal...
« Last Edit: July 25, 2023, 12:55:17 AM by lunar »

ManyPixels

  • Newbie
  • *
  • Posts: 40
    • View Profile
Re: Rolling shutter compensation
« Reply #8 on: July 25, 2023, 10:10:15 AM »
Okay, I'll explain the problem clearly.

Rolling shutter is a method of image capture in which a picture or frame is captured not by taking a snapshot of the entire scene at once, but rather by scanning across the scene rapidly, either vertically or horizontally. This can introduce distortions to the captured image, especially when capturing fast-moving objects or when the camera itself is moving rapidly.

During camera calibration, distortion parameters including tangential distortions (P1, P2) are typically estimated to correct for lens-induced distortions. However, these corrections are calculated under the assumption that the image is captured all at once, i.e., under a global shutter mechanism.

When a rolling shutter mechanism is used, this assumption no longer holds true. The different parts of the image are captured at slightly different times, and thus the distortions in different parts of the image can be slightly different.

As such, when rolling shutter compensation is applied, it might impact the precalibration done on tangential distortions. It effectively compensates for the time delay between the capturing of different parts of the image. Therefore, it might modify the effects of the precalibration, leading to further adjustments being required for an accurate representation of the scene.

In essence, even though precalibration helps in correcting tangential distortions, the presence of a rolling shutter effect may require additional corrections to achieve an accurate and undistorted image representation.

Given these considerations, it becomes beneficial to provide known parameters that can further assist with the rolling shutter compensation process. Notably, if the sensor readout time is known, it can be provided to help reduce the uncertainty in the correction. The sensor readout time, or the time taken by the rolling shutter to scan from one side of the image to the other, is a crucial piece of information in determining the nature and degree of distortion present.

Additionally, information on the trajectory or speed of acquisition can also be invaluable. This information can help model the relative motion of the camera and the scene during the time it takes for the rolling shutter to capture an image. This, in turn, can provide insights into the spatial variance of the distortions across the image.

By providing these parameters, we can minimize the unknowns in the rolling shutter compensation process, thereby reducing its potential impact on the precalibration. This ensures that the benefits of precalibration, in terms of correcting for tangential distortions, are not unduly compromised by the necessary adjustments made to compensate for the rolling shutter effect.