Sawtooth pattern being generated in error for very flat surfaces.

[Eichhorn18]

Hello,

I am processing image data from a very flat surface that has been textured visually to allow good tie point recognition. After processing the point cloud in Metashape, I am exporting the pointcloud to a data analysis program to generate contour plots of the flat surface. I have noticed some sort of noise, or rounding error being generated in Metashape. The parameters I am using to generate the sparse and dense point cloud area below:

number of photos: 35
Align photo accuracy: Highest
Generic preselection: ON
Key point limit: 40000
Tie point linit: 0

The model was then filtered by reprojection error to remove points with error above 0.2. After each gradual filtering and removal, the camera calibration was optimized for f, cx, cy, k1, k2, k3, p1 and p2. Several rounds of the gradual selection were necessary. The sparse point cloud had 12,572 points after filtering.

To generate the dense cloud:
Quality: Medium
Depth filtering: Disabled
Calculate point colours: ON

After this step, I create some markers in a local coordinate system. The markers are placed on the checkerboard grid below the object and z = 0 is at the level of the checkerboard. The textured surface physically sits about 15 mm above the checkerboard. The markers are added and turned on and then the refresh button is pressed in the references tab. My understanding is that this updates all the coordinates of the point cloud. The cloud is then exported as a .PTS file.

I then do some analysis of the data to grid the information at regular X Y spacing and return a Z value of the point cloud. This is essentially fitting surface by triangulation through the point cloud and then extracting the height value at regular X Y intervals. The data is then contoured, and the noise I am seeing is appearing in this contoured data as well (contoured_data.jpg).

The process of gridding the data and contouring it is not producing the sawtooth pattern in my opinion as the patterning is visible in the raw point cloud when you view this externally from Metashape, with no manipulation of the data. Simply viewing the 3D point cloud with no point colours (processing_01.jpg) shows this pattern or a very similar pattern.

Has anyone encountered this kind of noise / sawtooth pattern when generating point clouds for very flat surfaces?

Thanks,

[Alexey Pasumansky]

Hello Eichhorn18,

Can you please send the source image set and the processed project in PSZ format to support@agisoft.com, so that we could reproduce the result and investigate the case?

[Eichhorn18]

Hello Eichhorn18,

Can you please send the source image set and the processed project in PSZ format to support@agisoft.com, so that we could reproduce the result and investigate the case?

I'm happy to email these files over. The photos are all 6MB each and when compressed as a ZIP file, it's 198 MB. Can you recommend how I should go about transferring these?

Thanks,

[Alexey Pasumansky]

Hello Eichhorn18,

You can use WeTransfer, DropBox or any similar service to transfer the data archive. Alternatively, we can provide an FTP account data for you, if it is more convenient.

[Alexey Pasumansky]

Hello Eichhorn18,

Thank you for sending the project data.

I think that there could be several reasons of the effect that you are observing:
- low quality images - they seem noisy and the surface of interest looks to be out of focus (could be related to aperture settings or unstable camera orientation), so I cannot fully agree that the features on the surface of interest are sharp,
- inaccurate marker placement - projections do not seem to be adjusted on the source images, so there's quite a large placement error compared to accuracy settings used (also the image quality doesn't allow to get precise location of the marker),
- accuracy of 0.001 mm doesn't seem to reasonable value selected, as the nodes of the printed checkerboard pattern cannot be measured with such precision (I would suggest to use 0.1 mm max),
- disabled filtering for the dense cloud would result in considerably higher noise compared to standard settings (mild, moderate, aggressive),
- the surface of interest may have some inclination compared to the base level defined by markers, in this case the contour lines would represent the certain levels of the "slope".

[Eichhorn18]

Thanks for your comments Alexey. Please see some questions I have below.

I think that there could be several reasons of the effect that you are observing:
- low quality images - they seem noisy and the surface of interest looks to be out of focus (could be related to aperture settings or unstable camera
orientation), so I cannot fully agree that the features on the surface of interest are sharp,

I agree that the sharpness of the surface of interest in these imaging is not perfect. These images were taken with a Raspberry Pi camera module, and as such the pixel size is not retained in the EXIF data. I've had to manually enter this as 0.00112 mm / px based on what is posted here https://www.raspberrypi.org/documentation/hardware/camera/
Focusing this camera is done manually by moving the lens towards or away from the CCD. This makes focusing the camera subjective.

- inaccurate marker placement - projections do not seem to be adjusted on the source images, so there's quite a large placement error compared to accuracy settings used (also the image quality doesn't allow to get precise location of the marker),

Would using coded markers and auto locating these in images be much more precise than what I've done?

- accuracy of 0.001 mm doesn't seem to reasonable value selected, as the nodes of the printed checkerboard pattern cannot be measured with such precision (I would suggest to use 0.1 mm max),

Can you please explain what changing this value in metashape actually has an impact on in the generation of sparse / dense cloud process? Does this affect the SIFT algorithm somehow or the bundle adjustment? How does having a grosely wrong accuracy value come into play in the reconstrution?

- disabled filtering for the dense cloud would result in considerably higher noise compared to standard settings (mild, moderate, aggressive),

My understanding is that if small surface details are important to retain that depth filtering should be disabled. This was the advice in one of the tutorials or white papers listed on your site, for the example of drone captured aerial imagery. It's obviously several orders of magnitude of scale difference between drone footage and what I'm doing but I assume the mathematics of the depth filtering is equally applicable for both scales. If I'm trying to retain micron level detail, then should this not be disabled?

- the surface of interest may have some inclination compared to the base level defined by markers, in this case the contour lines would represent the certain levels of the "slope".

I agree, there is likely a differential slope between the checkerboard surface and the surface of interest. This accounts for the gradual elevation change across the surface of interest but not the sawtooth patten.

[chrisd]

Hi Eichhorn,

Are you using raw images?

[Eichhorn18]

Are you using raw images?

The image format is .jpeg caputred with the minimal amount of compression. There is no image manipulation / processing prior to loading into chunks in Metashape. ISO = 100 and the aperature and shutter speed are locked between photographs.

[Eichhorn18]

Looking a bit more in depth at the dense point cloud within Metashape, if I look at the underside of the dense point cloud, a latticework of gaps in the pointcloud becomes apparent.

What are the gaps in the generation of the lattice work that is being shown in the screencapture?

The spacing between these in the referenced model scale is 2 mm and this matches closely with the sawtooth like surface structure that I'm seeing when the elevation data is contoured.

[chrisd]

Are you using raw images?

The image format is .jpeg caputred with the minimal amount of compression. There is no image manipulation / processing prior to loading into chunks in Metashape. ISO = 100 and the aperature and shutter speed are locked between photographs.

It appears raw is an option, at least in the hardware spec. I don't know that getting maybe 11 or 12 bits / channel instead of 8 will make any difference here though.

[Eichhorn18]

A bit of an update on this post.

I've investiaged the problem of the sawtooth pattern in the very flat surface. I experimented with lighting sources with no effect. The orientation of my cameras relative to the flat surface of my model was parallel (the plane of the image sensor was paralell to the object of interest). It has been suggested that this orientation can result in systematic errors and poor depth construction in the direction perpendicular to the plane of the object.

I re-captured the photos in a ring around the flat surface with the camera orientied approximately 45 degrees to the surface. The result had much less of that sawtooth pattern.

I wonder if the systematic error is caused by the SIFT algorithm that metashape is using to identify tie-points between images. It would be nice if someone from Agisoft could comment on whether the SIFT algorithm for image matching is using sub-pixel interpolation--or pixel locking to the nearest pixel. If sub-pixel interpolation is not being used in the program, it could contribute to the systemtic error. Any comments from Agisoft?

Thanks

[stihl]

- accuracy of 0.001 mm doesn't seem to reasonable value selected, as the nodes of the printed checkerboard pattern cannot be measured with such precision (I would suggest to use 0.1 mm max),

Can you please explain what changing this value in metashape actually has an impact on in the generation of sparse / dense cloud process? Does this affect the SIFT algorithm somehow or the bundle adjustment? How does having a grosely wrong accuracy value come into play in the reconstrution?

The accuracy of the markers directly impacts the 'weight' of the manually placed markers. If a manual marker is slightly off from the actual position then this introduces a misalignment of the images due to the weight of the markers being higher than the weight of the tie-points found by Photoscan.
The alignment determines the location and orientation of each aligned image. When building a dense cloud, an algorithm is used to create depth maps from stereo paired images. If the images are slightly misaligned then this will introduce noise in the final point cloud.

[Alexey Pasumansky]

Hello Eichhorn18,

I think we need the project where the referencing is properly performed via markers (valid projections specified and coordinate information input). Otherwise, if the coordinate system is not properly assigned, the model itself would be incorrectly oriented in space.
The accuracy of the marker's coordinates should be set to the real measurement accuracy. Even if the checkerboard pattern is printed with 600 dpi, the coordinates cannot be resolved with accuracy higher than 0.05 mm.

[Eichhorn18]

I think we need the project where the referencing is properly performed via markers (valid projections specified and coordinate information input). Otherwise, if the coordinate system is not properly assigned, the model itself would be incorrectly oriented in space.
The accuracy of the marker's coordinates should be set to the real measurement accuracy. Even if the checkerboard pattern is printed with 600 dpi, the coordinates cannot be resolved with accuracy higher than 0.05 mm.

I understand better how the marker accuracy affects the dense cloud reconstruction now. Thank you.

Can you comment on whether the photo alignment for the sparse cloud uses a SIFT algorithm, and if so, does is do sub-pixel alignment?