Forum

Author Topic: Cannot align more than 2 images at once  (Read 1058 times)

Jakbac

  • Newbie
  • *
  • Posts: 7
    • View Profile
Cannot align more than 2 images at once
« on: October 08, 2024, 06:29:57 PM »
Hi all together,

I have the following problem:
I want to align (just for a test) a very small dataset of 4 images. I have detected the tie-points between the images already, so when I click on "view matches" I can see them.
However, when I select all images and do right click --> Align selected cameras, only two of the images are aligned, but not the rest. Also my log output does seem strange, as the values for the default pair score look strange (everything is 0).

Following things are also important / I noticed:
- I try to match these images with custom tie-points I found myself using lightglue (see https://www.agisoft.com/forum/index.php?topic=16650.0). I'm recreating the p0.ply to pX.ply and the tracks.ply files and add them again to the .files folder. On the first look, the files are identical to the ones Agisoft is creating. Furthermore I can see my own matches in "view matches"
- I know the images have enough overlap, because when I do the complete process within Agisoft Metashape (e.g. using Tools -> "Build tie-points" and then "Align selected cameras") everything is working fine
- The custom tie-points I am using are evenly distributed and also have correct coordinates
- When I just select only 2 images, independent which ones (e.g pair [1,2] or pair [2,3] or pair [3,4] or even pair [1,3] [2,4] I get an alignment.

Does anyone have an idea what is wrong? Or how I can fix this problem? I would love to use my custom tie-points and I found a lot more and for some images I use in my project I cannot find any tie-points with Agisoft Metashape but with my workflow. I've also tried to add markers, but this would only be a Plan B.

Thanks a lot for your help!

Please see here the log output when trying to align all 4 images at the same time:
Code: [Select]
2024-10-08 17:18:40 AlignCameras: cameras = [0, 1, 2, 3], adaptive_fitting = on, use_markers = on
2024-10-08 17:18:40 Estimating camera locations...
2024-10-08 17:18:40 processing matches... done in 0.009425 sec
2024-10-08 17:18:40 selecting camera groups...
2024-10-08 17:18:40 groups: 4
2024-10-08 17:18:40 n groups: 1, total: 4, minmax: [4, 4]
2024-10-08 17:18:40 done in 0.000109 sec
2024-10-08 17:18:40 scheduled 1 alignment groups
2024-10-08 17:18:40 saved camera partition in 0.003687 sec
2024-10-08 17:18:40 loaded camera partition in 1e-05 sec
2024-10-08 17:18:40 processing block: 4 photos
2024-10-08 17:18:40 pair 0 and 2: 812 robust from 815
2024-10-08 17:18:40 pair 1 and 3: 2625 robust from 2625
2024-10-08 17:18:40 pair 0 and 1: 4837 robust from 4838
2024-10-08 17:18:40 pair 2 and 3: 5000 robust from 5000
2024-10-08 17:18:40 pair 1 and 2: 5000 robust from 5000
2024-10-08 17:18:40 evaluating initial pair...
2024-10-08 17:18:40 initial pair evaluated in 0.116213 sec.
2024-10-08 17:18:40 initial pair unstable, considering additional pairs...
2024-10-08 17:18:40 additional pairs considered in 6e-06 sec.
2024-10-08 17:18:40 optimizing initial pair...
2024-10-08 17:18:40 ****************************************************************************************************
2024-10-08 17:18:41 default pair score: n_aligned: 2, n_points_tier: 0, accuracy_tier: 0, reprojection_error: 0, accuracy: 0, n_points: 0, ids: [2, 3]
2024-10-08 17:18:41 optimal pair not found
2024-10-08 17:18:41 initial pair optimized in 0.534593 sec.
2024-10-08 17:18:41 pair 2 and 3: 5000 robust
2024-10-08 17:18:41 adding 5000 points, 0 far, 0 inaccurate, 0 invisible, 0 weak
2024-10-08 17:18:41 adjusting: xxxxxxxxxx 1.19572 -> 0.921882
2024-10-08 17:18:41 adding 0 points, 0 far, 0 inaccurate, 0 invisible, 0 weak
2024-10-08 17:18:41 optimized in 0.030927 seconds
2024-10-08 17:18:41 (3 px, 2 3d) sigma filtering...
2024-10-08 17:18:41 adjusting: xxxx 0.921882 -> 0.921862
2024-10-08 17:18:41 point variance: 0.651512 px, threshold: 1.95454 px
2024-10-08 17:18:41 adding 0 points, 65 far (1.95454 px threshold), 0 inaccurate, 0 invisible, 0 weak
2024-10-08 17:18:41 adjusting: xxxx 0.859036 -> 0.858881
2024-10-08 17:18:41 point variance: 0.606997 px, threshold: 1.82099 px
2024-10-08 17:18:41 adding 0 points, 32 far (1.82099 px threshold), 0 inaccurate, 0 invisible, 0 weak
2024-10-08 17:18:41 adjusting: xxxxx 0.840306 -> 0.840208
2024-10-08 17:18:41 point variance: 0.593796 px, threshold: 1.78139 px
2024-10-08 17:18:41 adding 0 points, 12 far (1.78139 px threshold), 0 inaccurate, 0 invisible, 0 weak
2024-10-08 17:18:41 adjusting: xxxx 0.833297 -> 0.833254
2024-10-08 17:18:41 point variance: 0.58888 px, threshold: 1.76664 px
2024-10-08 17:18:41 adding 0 points, 2 far (1.76664 px threshold), 0 inaccurate, 0 invisible, 0 weak
2024-10-08 17:18:41 adjusting: xxxxx 0.832024 -> 0.832
2024-10-08 17:18:41 point variance: 0.587994 px, threshold: 1.76398 px
2024-10-08 17:18:41 adding 0 points, 1 far (1.76398 px threshold), 0 inaccurate, 0 invisible, 0 weak
2024-10-08 17:18:41 optimized in 0.250334 seconds
2024-10-08 17:18:41 f 6177.2, cx 0, cy 0, k1 -0.0234523, k2 0, k3 0
2024-10-08 17:18:41 f 6177.2, cx 0, cy 0, k1 -0.0236788, k2 0, k3 0
2024-10-08 17:18:41 finished sfm in 1.17139 seconds
2024-10-08 17:18:41 loaded projections in 8.2e-05 sec
2024-10-08 17:18:41 tracks initialized in 0.001866 sec
2024-10-08 17:18:41 adding 5000 points, 0 far, 0 inaccurate, 0 invisible, 0 weak
2024-10-08 17:18:41 loaded camera partition in 3.4e-05 sec
2024-10-08 17:18:41 1 blocks: 4
2024-10-08 17:18:41 calculating match counts... done in 0.001604 sec
2024-10-08 17:18:41 overlapping groups selected in 1e-06 sec
2024-10-08 17:18:41 loaded camera partition in 2.8e-05 sec
2024-10-08 17:18:41 1 blocks: 4
2024-10-08 17:18:41 block: 2 sensors, 4 cameras, 18278 points, 36556 projections
2024-10-08 17:18:41 block_sensors: 0.00167847 MB (0.00167847 MB allocated)
2024-10-08 17:18:41 block_cameras: 0.0015564 MB (0.0015564 MB allocated)
2024-10-08 17:18:41 block_points: 0.8367 MB (0.915527 MB allocated)
2024-10-08 17:18:41 block_tracks: 0.069725 MB (0.069725 MB allocated)
2024-10-08 17:18:41 block_obs: 1.6734 MB (1.6734 MB allocated)
2024-10-08 17:18:41 block_ofs: 0.139458 MB (0.139458 MB allocated)
2024-10-08 17:18:41 block_fre: 0 MB (0 MB allocated)
2024-10-08 17:18:41 final block size: 4
2024-10-08 17:18:41 adding 5000 points, 0 far, 0 inaccurate, 0 invisible, 0 weak
2024-10-08 17:18:41 (3 px, 2 3d) sigma filtering...
2024-10-08 17:18:41 adjusting: xxxxxx 0.974849 -> 0.921862
2024-10-08 17:18:41 point variance: 0.651512 px, threshold: 1.95454 px
2024-10-08 17:18:41 adding 0 points, 65 far (1.95454 px threshold), 0 inaccurate, 0 invisible, 0 weak
2024-10-08 17:18:41 adjusting: xxxx 0.859036 -> 0.858881
2024-10-08 17:18:41 point variance: 0.606997 px, threshold: 1.82099 px
2024-10-08 17:18:41 adding 0 points, 32 far (1.82099 px threshold), 0 inaccurate, 0 invisible, 0 weak
2024-10-08 17:18:41 adjusting: xxxxx 0.840306 -> 0.840208
2024-10-08 17:18:41 point variance: 0.593796 px, threshold: 1.78139 px
2024-10-08 17:18:41 adding 0 points, 12 far (1.78139 px threshold), 0 inaccurate, 0 invisible, 0 weak
2024-10-08 17:18:41 adjusting: xxxx 0.833297 -> 0.833254
2024-10-08 17:18:41 point variance: 0.58888 px, threshold: 1.76664 px
2024-10-08 17:18:41 adding 0 points, 2 far (1.76664 px threshold), 0 inaccurate, 0 invisible, 0 weak
2024-10-08 17:18:41 adjusting: xxxxx 0.832024 -> 0.832
2024-10-08 17:18:42 point variance: 0.587994 px, threshold: 1.76398 px
2024-10-08 17:18:42 adding 0 points, 1 far (1.76398 px threshold), 0 inaccurate, 0 invisible, 0 weak
2024-10-08 17:18:42 optimized in 0.445276 seconds
2024-10-08 17:18:42 coordinates applied in 0 sec
2024-10-08 17:18:42 Finished processing in 1.73741 sec (exit code 1)

Alexey Pasumansky

  • Agisoft Technical Support
  • Hero Member
  • *****
  • Posts: 15118
    • View Profile
Re: Cannot align more than 2 images at once
« Reply #1 on: October 08, 2024, 09:10:14 PM »
Hello Jakbac,

If you want to use external tie points, then I would suggest to use bundler.out format for camera import feature. You can use some arbitrary information for camera positions and orientation - which could be reset later for re-alignment based on imported tie points.

Editing project files is not recommended and are not intended to be modified or created without use of Metashape itself.
Best regards,
Alexey Pasumansky,
Agisoft LLC

Jakbac

  • Newbie
  • *
  • Posts: 7
    • View Profile
Re: Cannot align more than 2 images at once
« Reply #2 on: October 09, 2024, 04:27:24 PM »
Hi Alexey,

thanks your your answer! I'll look into it, that sounds promising!

While working on it I noticed some things:
- While checking the specifications of the bundler format, I see that in the bundler.out format also the 3D positions of the point is already specified. Do I need to calculate that before or can I also substitute it with arbitrary information?
- Is the color field important as well? If yes, how is the color calculated? An average of the color of all tie-points for that point?
- Currently when importing the bundler file, all points are removed, what could be the reason for that?
Code: [Select]
2024-10-09 15:18:09 Loading cameras...
2024-10-09 15:18:09 removed points for 1 components
2024-10-09 15:18:09 removed 3629 tracks
2024-10-09 15:18:09 Updating tie points...
2024-10-09 15:18:09 adding 0 points, 0 far, 0 inaccurate, 0 invisible, 0 weak
2024-10-09 15:18:09 coordinates applied in 0 sec

I'v also attached an example bundler file that I'm currently recreating. I hope that file looks correct.

Thank's a lot for your help!

Jakbac

  • Newbie
  • *
  • Posts: 7
    • View Profile
Re: Cannot align more than 2 images at once
« Reply #3 on: October 10, 2024, 11:09:53 PM »
Hi Alexey,

I've fixed the problem with the removed tracks, it was just the problem that the rotation matrix of the cameras was not set up correctly as a 3x3 matrix.
Now the tie-points are correctly added to the Program, thanks for that!

However, I still have the problem that only 2 out of the 4 images are aligned (If I select all images for alignment at the same time). If I select only 2 images for alignment, I can align different pairs together. So the problems will probably the tie-points themselves.

I was just thinking, I only ever calculate matches between 2 images directly and never calculate matches that can be found on 3 or more images. It that a problem for the alignment?

Thanks a lot in advance!