Forum

Author Topic: Rolling Shuter - How do you check the accuracy and reliability of your models?  (Read 7732 times)

anet

  • Newbie
  • *
  • Posts: 5
    • View Profile
Hi, I have a question about rolling shutter. We work with Phantom 3pro and you advise to use a fit rolling shutter but the results do not seem right to me. I have an object - a landfill in the embankment, scanned with ScanStationC10 and P3p.
The results are attached in the picture1.
Gray is scanned.
Blue is a version on PS1.26 and camera adaptive camera model fitting and all parameters f cx cy k1 k2 k3 k4 b1 b2 p1 p2 p3 p4 - so we have run it.
Green is processing with version PS1.31 like: Align Photos – no camera model fitting/ /50 000/0; optimize - rolling shutter-OFF; F cx cy k1 k2 k3 b1 b2 p1 p2
Purple is processing with PS1.31 like: Align Photos – no camera model fitting/50 000/0; optimize - rolling shutter-ON; F cx cy k1 k2 k3 b1 b2 p1 p2
As can be seen, the difference between ONrSh and OFFrollSh is 10 cm. But OFFrSh is closer to the scanned, which I think is true. The difference between the scanned and the OFFrollSh  is 1-3 cm.
Interestingly, the ONrollSh  is lower. I have another depot that is in the excavation and there a rolling shooter ON is 10 cm higher from rolling shutter OFF.
Help me to optimize my projects.  When object is in excavation or ditches - the results is higher, and when embankments or convex elements the model is lower.
How do you check the accuracy and reliability of your models?
The height of flying both objects is 45 m. I have an each project of 10 GCP.
Thanks

Packy_GIS

  • Full Member
  • ***
  • Posts: 107
    • View Profile
Try to check our paper ... and there are several other ways using geostatistics to check the accuracy.
I have focused on this topic in the past year - but my other papers are not published yet (will be in about a month).
Jan

BobvdMeij

  • Full Member
  • ***
  • Posts: 157
    • View Profile
I have found somewhat similar results. Whether I turn on or off Rolling Shutter Optimization, the quality of the eventual model hardly differs. Sometimes, the software even suggests that the model without RS optimization is of a higher quality than the model including such compensation.

Strangely enough, when testing Pix4D a while back, I found that their RS optimization mattered a substantial amount! But then again, the model produced therein WITH RS optmization turned on was just about as good as a model I produced in Agisoft, whether RS optimization therein was activated or not.

JMR

  • Hero Member
  • *****
  • Posts: 504
    • View Profile
Can you please descrive the flight procedure? fixed heading, flight speed, stot & go, agl altitude?

cadm8

  • Jr. Member
  • **
  • Posts: 74
    • View Profile
I haven't yet read Jan's paper (but I will), what I would ask is what is your flight speed? I've ran some tests in a sort of "test field" and things get a little difficult for the P3P at speeds above 8-9m/s up until 10-15 m/s (as the https://www.google.gr/url?sa=t&rct=j&q=&esrc=s&source=web&cd=2&cad=rja&uact=8&ved=0ahUKEwjG2uyJh-HTAhUPEVAKHa89BG4QFggtMAE&url=https%3A%2F%2Fpix4d.com%2Fwp-content%2Fuploads%2F2016%2F05%2Fpix4d-isprs-paper-rolling-shutter-final-edited.pdf&usg=AFQjCNEF6qH4VszC1bLCSSCYwJiy4EyCeg&sig2=uZA_6AbOrKzKWG_GoW9Y3g Pix4D paper suggets too)

Regarding my experience with the correction model, I'm getting algorithm wise, a 3-4 times (easily) increased accuracy, especially in a recent project where we surveyed buildings with the total station too, we can reproduce previous accuracies (4-5cm) and maybe half that. Though hadn't had the chance (nor will, we don't own a laser scanner!) to run a similar test as yours

PS: is there some reason why your tie point limit is 0?
« Last Edit: May 08, 2017, 11:08:58 PM by cadm8 »

anet

  • Newbie
  • *
  • Posts: 5
    • View Profile
We use Dronedeploy. We work - 1 object - 1 battery if possible
For depot 1 we have a flying speed of about 4.5 m/s; 1 battery; 412 photos (2sec); Overlap 80% and sitelap 75%;  Height: 48 m (according to the report). Ortho Height GCP 723 m Baltic system
For depot 2 we have a flying speed of about 4.25 m/s; 1 battery; 452 photos; Overlap 80% and sitelap 60%; Height: 45m (report)
The next time we will fly slower, but this will be the next time  :-\. The situation is dynamic, so I have no chance to fly again.

We have made comparisons with PIX4D. The results are Photoscan without roller shutter is close to pix4d with the roller shutter on?!?!

anet

  • Newbie
  • *
  • Posts: 5
    • View Profile
Cadm8,
I will not be able to check the laser scan every time, because the meaning of the drones is lost :). In this case, I had the object scanned and just to check the dron model we decided to fly over the scanned depot.
I know that I need 6 GCP to define an object, what is visible, for depot2 we have put 11 - the idea is to use 6 and 5 to be as controlling. But it was so that the model with 6 and 11 points differed by another 5 centimeters, so we decided to release it with all the points.
I want to be sure, that the next time we fly again with the drone, we will give the right results. I also want to know, if I can not put control points inside the landfill (as in this case) how I will give the client a fair result?

PS: My tie points limit is 0 because we want have no limit and use all tie points. Then, Gradual - reprojection error 0.5; So we save time to make a dence cloud. There is no significant difference from 40000/4000( and then time for dence cloud) and 50000/0. When we see the results and decide the correct method and model we build a dence cloud
« Last Edit: May 09, 2017, 11:51:32 AM by anet »

cadm8

  • Jr. Member
  • **
  • Posts: 74
    • View Profile
Regarding your speed, I would say it's OK, maybe a little slow? I'm not sure though about the flight height, I think it is way too low for such type of work. Do you need the over-detail you're getting?

Really, I cannot comment something more about the 0 point limit, maybe someone other might have an idea. We currently use 40k/10k and it achieves alignment even with the majority of oblique imagery

BobvdMeij

  • Full Member
  • ***
  • Posts: 157
    • View Profile
Regarding your speed, I would say it's OK, maybe a little slow? I'm not sure though about the flight height, I think it is way too low for such type of work. Do you need the over-detail you're getting?

Really, I cannot comment something more about the 0 point limit, maybe someone other might have an idea. We currently use 40k/10k and it achieves alignment even with the majority of oblique imagery

To me the speed seems reasonable, albeit marginally slow. Then again, if you want to keep motion blur under 1xGSD you need to significantly lower your speed, particularly at 48m AGL.

We generally operate at 50m at a speed of 4m/s using an Inspire and Zenmuse X5 camera. Obviously the X5 produces a finer GSD so, in theory, you should be able to slightly increase your speed using a P3P as the GSD is larger and therefore allows a faster speed to keep blur within 1xGSD. Then again, the P3P may have more problems when lightning conditions are poorer, hence you might want to compensate your speed.

anet

  • Newbie
  • *
  • Posts: 5
    • View Profile
Now I've finished my last try with an object that has no displacement.
Drone of flat terrain with 10 marks on the ground. It does not matter whether it is optimized with or without roller shutter. The differences are negligible.
I have a question. How do you shoot such types of objects that have large earthwork
 and large embankments, not just a mass of material concentrated in one place?
« Last Edit: May 09, 2017, 03:23:00 PM by anet »

Yoann Courtois

  • Sr. Member
  • ****
  • Posts: 316
  • Engineer in Geodesy, Cartography and Surveying
    • View Profile
Hi everyone !

I'm a bit lost about rolling shutter subjet....:

- According to what I understood, the P4P camera is the first Phantom camera which has a global shutter camera when taking pictures (still rolling shutter in video).
Can somebody corroborate that ?

- What about GoPro camera ? I'm using Hero 4 Black and even sometimes I've to deal with blurred pictures, I've never noticed jello effect...

Hope somebody can secure those ideas in my mind !
Regards

Note: As I thing I've never faced to rolling shutter, I've never checked "fit rolling shutter" during optimization. Some test have shown me it destroy a lot our accurate alignments.
« Last Edit: May 17, 2017, 12:07:36 PM by Yoann Courtois »
--
Yoann COURTOIS
R&D Engineer in photogrammetric process and mobile application
Lyon, FRANCE
--

Alexey Pasumansky

  • Agisoft Technical Support
  • Hero Member
  • *****
  • Posts: 14965
    • View Profile
Hello Yoann,

I think, you can check if the camera has rolling shutter quite easily: just take a picture of fast moving object, like car rotating fan wings, or even easier - take a photo of still object while camera is rotating.
Best regards,
Alexey Pasumansky,
Agisoft LLC

Yoann Courtois

  • Sr. Member
  • ****
  • Posts: 316
  • Engineer in Geodesy, Cartography and Surveying
    • View Profile
According to my test handled today, it looks that:

DJI Phantom 4 Pro:
- Video: Rolling shutter
- Photo: Global shutter

GoPro Hero4:
- Video and Photo: Rolling shutter

Somebody to corroborate ?
--
Yoann COURTOIS
R&D Engineer in photogrammetric process and mobile application
Lyon, FRANCE
--

gto234

  • Newbie
  • *
  • Posts: 42
    • View Profile
Hi,
I noticed the same problem described in the first post with my Phantom 4 dataset.
I have +10cm on measured objects e.g. rooftop of a building when I turn on the rolling shutter correction.
When i turn it off the differences are 2-4cm which is more realistic.
I also compared the exported contours of the two DEMs and they show the difference.
I like seeing the RMS of XYZ go down to 1cm with rolling shutter correction but the real accuracy matters.
I will test some more datasets but I think the rolling shutter correction is not correcting anything!