Calibration when using images from apochromatic microscope

[ISeeBees]

Hi all, 

I’ve started a project using an SMZ18 microscope to take images of insect heads as I rotate them on a goniometer, and then reconstruct models of them using PhotoScan. Things are going surprisingly smoothly, but as I need to take accurate measurements of the dimensions and positions of the insect eyes from the models, I wanted to find out whether there sensible calibration method to use for a microscope.

When I perform the image alignment step, things generally go fairly smoothly, however, PhotoScan comes up with some pretty widely variable values for all the parameters it adjusts. I looked at this a bit further using the Lens program, and again found the parameters it calculated were very variable. Looking at the distortion maps from the chequerboards indicates they don’t really match the standard distortion modes for a pinhole camera, as small patches of points are locally distorted inwards (see the attachment). Given the complexity of the microscope optics though, and its specific design to minimize distortion, perhaps this isn’t surprising! So calculating parameters doesn’t seem likely to work in this case.

As a workaround, can I manually enter parameters for the calibration into PhotoScan that would make sense? Given the quality of the optics, I think everything relating to distortion (and maybe the image center) can be set to zero. The microscope is calibrated, so I know the pixel size with micron precision, and obviously the magnification. But I don’t know the focal length of the microscope, and don’t think I can manually calculate this without knowing more about the construction of the microscope. Is there a way to let PhotoScan calculate this parameter alone?

Any help is much appreciated!

[bigben]

Do you have a calibration target for the microscope that you could use. You should be able to get more sensible numbers if you can supply a grid of GCPs.

[ISeeBees]

Well, I was using a printed checkerboard with 0.5mm side length squares for calibration, which is shown in the original attached image. This should provide a grid of ground control points shouldn't it? Or were there any other calibration targets you were referring to specifically?

I know our vendor calibrates the microscopes in terms of microns/pixel using reference objects of known lengths, but this isn't designed to fit a distortion model. Presumably the microscope manufacturer Nikon has some calibration process to they use for design and testing, but I have no knowledge of what targets would be used for this.

With respect to the current target, 0.5mm squares are about the smallest my laser printer can make without obviously distorting the squares. This seems to work OK at x1 and x2 magnification, although higher magnifications easily show the printing imperfections and the Lens program then has problems to identify the corners of the squares. Unfortunately displaying any sized pattern on an LCD screen doesn't work well as the microscope easily resolves the individual colour elements of each pixel, even at low magnification. Possibly I could make a higher resolution checkerboard for higher magnifications, although I am not sure this would be helpful for Lens if the distortion model is fundamentally different to a pinhole camera.

[bigben]

The checkerboard should be fine. The principle being that you want a spread of known positions across the field of view of the lens. The advantage of the checkerboard is that it also gives you a visual confirmation whether the correction is good or not.  For higher magnifications, you need a more accurate scale... maybe a low magnification SEM calibration grid? (e.g. http://www.tedpella.com/calibration_html/SEM_Magnification_Calibration_Standards.htm

While these may not be suitable for the lens calibration tool, You can still use PS Pro to calculate the values using GCPs to aid the alignment.

[ISeeBees]

Thanks for the further suggestions. We have an eletronmicroscopy group in my department, so I'll look into obtaining some SEM standards or perhaps a TEM grid as a fine resolution target to check the camera calibration at higher magnifications. Still, as I've already tried to use lens and failed to get a good calibration, I'm not sure these will provide much additional help.

I guess ground control points could help, but PhotoScan doesn't seem to have any problems aligning my images. Would they be able to provide any other assistance besides alignment? The other issue would be that my imaging setup dosn't easily allow for GCPs, but that is another matter.

Anyway, to resolve my worries about calibration difficulties, I will measure the distances between some  features on a models when I am happy with it. These can then be compared to distances directly measured on the microscope between the same features, which I think will provide the most accurate reference measurement for the model. Lets see how it goes!

[bigben]

The reason I suggest using GCPs as well is partly related to the calibration for spherical panoramas which is similar in some respects.  The calculation of distortion parameters for the lens as part of the alignment phase is in many respects a mathematical guessing process. The combination of parameters provides the best mathematical approximation but that doesn't necessarily mean that its geometrically accurate. Depending on the number and orientation of images it's possible that the results may be slightly distorted to provide a best fit, so it looks well aligned but the distortion parameters may vary between image sets.

The addition of GCPs at specified local coordinates provides an extra reference that should improve the geometric accuracy.  I'd be happy to test a few things if you can provide me with a set of images.

[ISeeBees]

Hmm.. ok well in that case I suspect PS is playing with the distortion parameters to get a good looking alignment then, as they do vary a lot between image sets.

Thanks for offering to take a look. I'm taking high resolution data set today and tomorrow that will hopefully provide a best case comparison of what can be achieved with my imaging method. I'll put it up to drop box and send you a link to it when its there then.

Out of interest, what can function as a ground control point in an image? I don't have the pro version so I haven't played with them at all. If I can simply take measurements between distinctive points on the original images and assign them onto the model later that could be a good work around.

[bigben]

In this case you'd be manually selecting GCPs, so it's whatever you can identify and know the position of.  The junctions of the grids are probably the best as there easy to pick and you know the distances between them.  We still haven't purchased the Pro version yet either but the demo is fully functional so you can still use it to determine the lens settings.

[ISeeBees]

Ok, well after a few other things I got around to getting a good data set of the bee eyes that I am interested in.

Photoscan reconstructs what I think is a pretty plausible model of the bee eye, and seems to get the camera's well positioned if I specify the focal length at 38793, based on a calibration I did with the Lens program. The calibration I did here was a bit dodgy as I used Lens on a checkerboard at x1 magnification and divided the focal length by four, as the images were taken at x4 magnification. Perhaps its not ideal, and I would like to improve this with a proper calibration. The only issues is with the reconstruction is that the eye mesh is inside out compared to its actual orientation relative to the cameras, but maybe this is just relative..

However, if I let PS estimate the camera parameters it picks some pretty wild values (fx: 657771, fy: 657771, cx: 2036:, cy: 1550, k1: -125, k2: -6.6x10^7, k3: 2.5x10^11) and does a hopeless job of positioning cameras and the reconstruction. Ideally I would like PS to only estimate the focal length and keep the distortion parameters at zero, as I don't believe it does a good job of estimating distortion for the microscope at all.

If any one can offer any suggestions, I've got both the reconstruction project, as well as some checkerboards on dropbox. The attached figure is the eye after I loaded it into matlab, which doesn't have very good resolution, but gives an idea of the reconstruction I'm aiming for.
Dropbox of eye images and project: https://www.dropbox.com/sh/kuhfd12wnuxtcfn/AAB4zY5DhUbttrOIMjtuGVKAa?dl=0
Dropbox of checkerboard images: https://www.dropbox.com/sh/0stmg13z68oa8bk/AACkTRvSDxjjycjraQVb94JXa?dl=0