Photo Alignment produces very strang output result

[DenisIaq]

Hello,

I’m having significant difficulty aligning the photos in Metashape. The results are quite strange, and I’m having trouble understanding what’s going wrong.

To get some help, I’ll be uploading the PSZ file to WeTransfer. The file is about 680 MB. I would greatly appreciate a prompt response.

I’m currently using the latest demo version of Metashape Pro, which will expire in 7 days. I hope to hear from you soon.

Thank you in advance for your assistance.

Best regards!

https://we.tl/t-a8KGbO7YU2

[Bzuco]

For this you will need camera with a lot higher resolution of sensor than just 1200x1200.
With better camera 18/24Mpix Metashape will be able to recognize also feature points from white plaster walls, if they are not completely flat.
I would suggest you to change the path/positions from where you were taking the photos, because you took a lot of pictures which are useless.
To shoot the white walls you should be ~ 2-3m far away from them(in the middle of the benches) and point the camera perpendicular to wall and also take photos from oblique angles. You took the photos from too close and too far distance only.
You can try put some markers on the walls or at least to have some bench part on each photo to keep the continuity of shots.
It is interesting you were able to get almost correct result in Copy of Chunk 2.

So try to focus more on the surface you really need, choose the correct distance from the objects/surfaces and that will set you on the right path(positions from which you will be taking photos).
With better camera choose only High alignment quality, which is 1:1 to original resolution and set some tie point limit(2000-4000) to speed up the process.

[DenisIaq]

Thanks for your reply.

I have used a insta360 camera and converted the video to cube sized images. thats why i have a 1200 x 1200 resolution. this workflow has worked for me several times and also at the same spot (church). thats why i wonder so much that the alignment completely failed. no useable pointcloud.

for example when i through the same images into Reality Capture i get a usable point cloud. Thats weired, because for my taste, metashape is more professional than Reality Capture and should be able to produce a point cloud in this case too.

nevertheless i hear your suggestions. thanks for that too :-)

[James]

Hi DenisIaq,

I downloaded your project and started processing with what seems to be a promising workflow, but there are so many images that I'm not sure it's going to finish before tomorrow!

So anyway I can tell you what I did so you can try it if you like.

As Bzuco noted, a lot of the images in Copy of Chunk 2 did align reasonably well, though it's hard to tell because they are dwarfed by a smaller number of very poorly aligned images which throw the scale way off.

I didn't look in any of the other chunks.

I figured that you had converted 360 images to cube faces, so I got an AI to write a python script to set each set of cube faces to a station camera group. see camgroup.py attached. This tells metashape that these image share a 'nodal point' (this might be the wrong term but i know what i mean) and the images are forced to be located at the same spot.

I selected all images and right clicked -> reset camera alignment. Then I tried just aligning the first 50 or so images (select ~50 sequential images in the photos pane and right click -> align selected cameras), and they aligned pretty nicely.

I then selected a load more and repeated, and they aligned nicely to the previous ones.

I did a bit of gradual selection to remove bad points and ran optimise and wondered if it was valid for all images to share a single calibration group as they would have been cropped from different parts of the 360 image and so it might be more appropriate to calibrate them according to which part of the 360 they were cropped from.

The AI struggled a bit with that script, and it's not great, and i didn't spend any time improving it, but it works for now - calibgroup.py attached.

I don't know if that actually makes much difference, but it assumes that all images suffixed with _0 are from the same part of the 360 and same for _1, _2 etc and creates calibration groups for each. you can see the calibration groups in tools -> camera calibration.

I then continued aligning additional batches of images, occasionally doing a gradual selection and optimise, until i got up to ~600 aligned images, with still most of 5000 left.

At that point I got it to align all the remaining images, and that's what i'm waiting for now.

As Bzuco points out it's not an ideal camera and the white surfaces won't help much with reconstruction, but the alignment might come out ok.

I would also suggest massively reducing the number of photos, at least initially just to check that you can get a good alignment. I think you could just keep every 5th set of cube faces so you have more like 1000 images and you'd still have enough overlap for alignment.

I'll let you know tomorrow if the alignment completed this end, and share the result if it's not awful!

[James]

Hi DenisIaq,

Here's the completed alignment. Well, 5266/5337 images aligned. The remaining 71 unaligned images are all very featureless. https://we.tl/t-TAv4Oc50p0

The sparse cloud was still very noisy, so I did some heavy model->gradual selection and deletion and used tools->tie points->thin point cloud with a tie point limit of 5,000. I think the noise is predominantly from the white surfaces, and is what causes the 'fuzz' all around it, but the data on the detail areas seems much better.

I don't know if the calibration groups helped particularly, but I am confident that the camera station grouping did help a lot.

Another thought, did you know that you can use equirectangular images directly in metashape? You just have to set the camera type to spherical in tools->camera calibration.

Another thought, you could add the equirectangular images in with the cube face images, and group them in the same station camera groups so they were locked together, and that may help align the 71 unaligned images, because they would only have to align with their equirectangular counterpart to be properly aligned. Then you could discard the equirectangular images if they were not useful for your next stages of processing.

James

[DenisIaq]

Dear James.

I completely missed this dialogue here with you. I appologize for that ver much!

I remebered today because i am running into an issue thats very close to what we were talking about before.

I have a dataset of 1700 images. again i converted 360 pano images here. when i use estimated and high for alignment it only alignes 808 images. when i choose sequential and high it does only 560 images.

Whats strange to me is, that the estimated image quality shows me, that even images with an estimated quality of 1 or above were not aligned. Thats so strange to me.

(When i do the hole alignment process in RealityCapture its a complet mess though!)

Now, i can not use the raw 360 pano because i use the alignment and sparse point cloud for gaussian splatting training.

Do you have any idea why this is happening again?

Images of poorer quality like 0.5 were aligned but better and sharper ones with alot more reference points in my eyes were not aligned. Strange.

Would appreciate your help here again.