Forum

Author Topic: Taking fisheye to the extreme  (Read 25851 times)

bigben

  • Sr. Member
  • ****
  • Posts: 406
    • View Profile
Re: Taking fisheye to the extreme
« Reply #15 on: February 02, 2015, 10:50:20 PM »
Reviewing the processing while I wait for some parts to finish my painter's pole rig.  Found an error in the masking of the gatehouse scene (my error) and am halfway through repeating the masks, although it shouldn't have that great an effect on the outcome. The one thing I had failed to consider though was the that there was still a lot of noise. I went back to the first test and ran though a few repeats of cycles of optimising and re-aligning and it made a pretty big difference.  https://skfb.ly/CLRQ

I pulled out the images of the alcove on the gatehouse as a quick calibration scene and repeated the process to get some better lens parameters and then re-ran the alignment on the entire project. Much better result with less noise and more points generated from the periphery of the images. I'll re-run it fully after I finish fixing the masks and I'll shoot my normal calibration scene today to refine the lens parameters...  Much happier with the prospects of using this lens now.

bigben

  • Sr. Member
  • ****
  • Posts: 406
    • View Profile
Re: Taking fisheye to the extreme
« Reply #16 on: February 06, 2015, 12:36:04 AM »
After doing a rough calibration of my lens from some images in the previous post, I applied the settings to another project to test them out. The result was a pretty good improvement. Here's the pointcloud http://files.digitisation.unimelb.edu.au/potree/pointclouds/msd-atrium-optimised.html

The lower stairs and the floor below the atrium is a single camera path, 2 shots every 2-3 steps with camera pointing forward and back as I left the building. It's not a complete scan of the interior, just a test to see the results of different camera orientations in different situations.
« Last Edit: February 06, 2015, 01:39:53 AM by bigben »

MeHoo

  • Full Member
  • ***
  • Posts: 136
    • View Profile
Re: Taking fisheye to the extreme
« Reply #17 on: February 08, 2015, 02:58:52 AM »
Your fisheye results are really making me want to invest in a wider lens than my 16-35mm.

bigben

  • Sr. Member
  • ****
  • Posts: 406
    • View Profile
Re: Taking fisheye to the extreme
« Reply #18 on: February 08, 2015, 08:39:39 AM »
Your fisheye results are really making me want to invest in a wider lens than my 16-35mm.

The tests with circular fisheye images have certainly been very interesting but from a practical sense I think a full frame fisheye is a more practical lens in most situations. 

Longer term, I'm considering putting together to proposal for a local research grant to come up with a pole mounted rig that researchers could use for things like archaeological dig sites. I finally got my painter's pole set up with my DSLR and CamRanger which I'll use to figure out the camera/lens combinations and then hopefully get an internal grant to build it.

bigben

  • Sr. Member
  • ****
  • Posts: 406
    • View Profile
Re: Taking fisheye to the extreme
« Reply #19 on: February 08, 2015, 03:22:11 PM »
Reprocessed my second test after doing a pseudo calibration of my lens and refining the masks. This made a HUGE difference. Alignment and dense cloud run at medium to save time. I even allowed myself a "wow" when I saw this.  ;)

https://skfb.ly/CPpn

Updated the online pointcloud. Camera path did not go onto the road.
http://go.unimelb.edu.au/zj7n

Without using a specific reference scene it's very difficult to get good settings for the lens distortion of a circular fisheye, particularly the outer 20°.  The alcove of this building has so far proved to be the best one I've found... essentially a brick-lined box 2-3m wide would be good.

Canon 5D2, Sigma 8mm, 513 images

igor73

  • Full Member
  • ***
  • Posts: 228
    • View Profile
Re: Taking fisheye to the extreme
« Reply #20 on: February 09, 2015, 01:40:05 AM »
That is awesome!  I can see some uses for circular fish eye for sure! 


igor73

  • Full Member
  • ***
  • Posts: 228
    • View Profile
Re: Taking fisheye to the extreme
« Reply #21 on: February 09, 2015, 01:52:12 AM »


Could you please explain a bit more about the calibration process you mention using for this lens?  Do you mean you adjusted the mask so the black edges of the circular fish eye are removed?  Or do you mean you used Agisoft camera calibration tool?   

bigben

  • Sr. Member
  • ****
  • Posts: 406
    • View Profile
Re: Taking fisheye to the extreme
« Reply #22 on: February 09, 2015, 04:00:05 AM »

Could you please explain a bit more about the calibration process you mention using for this lens?  Do you mean you adjusted the mask so the black edges of the circular fish eye are removed?  Or do you mean you used Agisoft camera calibration tool?

Yes, this is definitely something I have to write up.... but it goes something like this.

First, the lens calibration tool is not so practical in this case because of the extreme fov.

Secondly, you need to understand what is actually being calculated. The position of each point in an image is being "fixed" by recalculating it's distance from the optical centre of the image using an equation with the lens parameters as constants for the equation.  In order to do the reverse, you need to provide images where points in the scene appear in multiple images, in different positions spread from the edge to the centre of the image.  With a circular fisheye, this actually requires turning the camera, not just panning.

I prefer brick walls in reference scenes as you can visually assess whether the settings are good or not. Remember that this is a mathematical algorithm... Photoscan doesn't "see" bricks and straight edges.  The walls in the point cloud should be flat, with little noise, and the bricks should be rectangular and of consistent size.   If you look at a screengrab of the project I used to generate the lens parameters you can see the camera positions I chose to capture the confined space.  1. Walking back and forth through the space, and then a vertical pan with the camera looking across the space from opposite sides.  I probably would have done a grid of paths if I were shooting a calibration scene, but this seems to have worked in this case.

So then it's just a case of cycling through the alignment steps to refine the point cloud.
1. Align images
2. Gradual selection and removal of points using Reprojection error = 0.5 and Reconstruction uncertainty = 35 (it's important to do both in this case)
3. Optimise cameras (Do not optimise skew)
4. In the Camera calibration, save the Adjusted settings and load them into the Initial settings (this should chnage it to Auto to Precalibrated)

Repeat this a couple of times.  You know you're getting close when the maximum Reprojection error < 2.5 and Reconstruction uncertainty < 150

Once you've got your final settings I tend to Fix the lens parameters for the first alignment of a project to avoid the lens parameters getting re-calculated to values that will throw things out.  For testing, go out and shoot a big brick wall from a distance of 2m or less. If the bricks at the outer edges of the model are distorted then you may have some more work to do as this will have an adverse affect on calculating camera positions and create a lot more noise throughout the point cloud.

Full frame fisheyes are a bit easier to work with, and  probably wouldn't go below 18mp cameras for circular fisheyes.

igor73

  • Full Member
  • ***
  • Posts: 228
    • View Profile
Re: Taking fisheye to the extreme
« Reply #23 on: February 09, 2015, 01:50:15 PM »
I have a FX 16mm Nikon fisheye that i will try this with and a 36mpix Nikon d810.

Your calibration procedure is very interesting! This is new to me.  Thanks for sharing.   Would you calibrate your lens this way if you used a more normal non fish eye wide angle lens to? 

bigben

  • Sr. Member
  • ****
  • Posts: 406
    • View Profile
Re: Taking fisheye to the extreme
« Reply #24 on: February 09, 2015, 03:40:33 PM »
I use a similar approach for all of my lenses, but the requirements are a bit more relaxed for narrower fovs.  A brick wall and cobblestone laneway worked well for standard lenses, and there's  a nice little corner of brick next to my window at work.  I've been trying other reference scenes for my circular fisheye on my camera (previous one was my work camera) and in the end I went back to same spot as my previous one.   I'll post some images later to demonstrate what incorrect settings look like. Much easier to explain with pictures.

igor73

  • Full Member
  • ***
  • Posts: 228
    • View Profile
Re: Taking fisheye to the extreme
« Reply #25 on: February 09, 2015, 06:07:30 PM »
And  you find that you get significant better alignment accuracy when using this method even on a normal wide angle lens?  I have to try this so please do keep the info coming.  Much appreciated! 

I have a very special application taking pictures underwater and the lens is behind a doom port that cancels out the water magnification effect but introduces its own distortion especially at the edge of the doom.  .  Do you think this could work using the tiles of a swimming pool as calibration target? 

Using Agisoft lens calibration is not practical as i can't take pictures of the monitor screen under water.  Also i was under the impression calibration was not really needed if you had EXIF data at least not for on fish eye lenses.   


« Last Edit: February 09, 2015, 06:16:27 PM by igor73 »

bigben

  • Sr. Member
  • ****
  • Posts: 406
    • View Profile
Re: Taking fisheye to the extreme
« Reply #26 on: February 09, 2015, 10:46:31 PM »
I wouldn't say it's a better method... just an alternative one.  I started using this for panoramic stitching software which uses similar parameters. (an old page on an unmaintained archive: http://archive.bigben.id.au/tutorials/360/calibrate/calibrate1.html)

A swimming pool was the first thing that popped into my mind for your underwater application. ;)

EXIF data doesn't contain any info on lens distortion, so the distortion parameters always have to be calculated. If you don't use a calibration process to determine these beforehand, then Photoscan calculates them during the image alignment step. In this case, the accuracy of these values depends in part on the type of subject matter in the images and the choice of camera positions and orientations.   Other software like Adobe Camera RAW just use the lens make/model to look up a table of distortion parameters. For Photoscan though, you really need to do this for every camera/lens combination ... although you can use the values for k1,k2,k3 as starting points between cameras.

A statistical approach can also be useful... list all of the lens parameters for "successful" projects and compare them, looking for clusters of values that have the same pattern of being positive/negative e.g. for my fisheye lens k1,k2,k3 are -,+,-. In this case, though I'd just use these as starting values for k1,k2,k3 and let Photoscan optimise them.

Once you do have calibrated values, you can load them into projects and hit the "Fixed" check box in the Lens calibration window.  This will preventing incorrect values being calculated in some situations (especially important with circular fisheye lenses, less relevant to normal lenses)

bigben

  • Sr. Member
  • ****
  • Posts: 406
    • View Profile
Re: Taking fisheye to the extreme
« Reply #27 on: March 07, 2015, 10:26:50 AM »
Finally learned to take my own advice and not optimise calibrated values... especially for fisheye lenses.  "Calibrated" my circular fisheye using grid of 6 camera stations, with each camera station using 3 rows at 0, 30 and 60° tilt, and 15° rotation increments.

Shot a complex object on my way home (building within a building).
Optimised lens parameters: https://skfb.ly/Dq7Q
Fixed, calibrated lens parameters: https://skfb.ly/Dr6P

bigben

  • Sr. Member
  • ****
  • Posts: 406
    • View Profile
Re: Taking fisheye to the extreme
« Reply #28 on: March 10, 2015, 12:30:18 PM »
The last few tests I did had more than enough overlap.  It was time to stretch things out a bit.  Another hour's shooting time, 310 circular fisheye images, camera on a 3.6m pole, slightly random camera path.  Camera was tilted downwards a bit too much, but a level camera and a more methodical camera path it would make a great set of images to complement a UAV survey, capturing the ground and lower building details that UAVs have difficulty with.
Medium quality alignment and dense cloud:  http://files.digitisation.unimelb.edu.au/potree/pointclouds/uom-poledancing.html

thesystemera

  • Newbie
  • *
  • Posts: 19
    • View Profile
    • realityvirtual
Re: Taking fisheye to the extreme
« Reply #29 on: March 14, 2015, 09:47:09 AM »
Fisheye and video, wow! Really impressive results guys. This is what I'm doing for my lazy sunday!
Simon Che de Boer @ http://www.realityvirtual.co/