Metashape Pro Crashing During Image Alignment

[colbyrand]

Hello,

I am having an issue with Agisoft Metashape Pro (Version 2.1.0) consistently crashing during the image alignment step. Metashape is able to get through the matching points phase, but usually fails during the estimating camera locations phase. Occasionally, Metashape will successfully be able to align photos, but other times the application crashes, despite the fact that it is processing the same set of photos with identical parameters. Thus, I can't make sense of why it will work some of the time, but usually results in a crash.

The specs of my newly built PC are as follows:

OS: Windows 11 Enterprise (Version 22H2)
CPU: Intel Core i9-13900K
GPU: NVIDIA GeForce RTX 4070 Ti (MSI model)
Memory: 128GB DDR5 RAM (4 sticks of Corsair Vengeance CMK64GX5M2B6400C32)
Storage: 4TB NVMe SSD
Motherboard: Gigabyte Z790 AORUS ELITE AX

I have all the latest drivers, and I am using the most recent version of Agisoft Metashape (I encountered the same errors on the previous version as well). After submitting my crash logs to Agisoft customer support, they inspected them and let me know that they do not appear to be connected to any known issues. They suggested there was a high chance that these problems were caused by faulty RAM modules, and recommended I test the RAM with Memtest86. I did this, but the test passed with zero errors (see the attached report). I am running my system without XMP or any other overclocking features.

Please let me know if anyone has any suggestions or ideas about what is causing these application crashes!

Thanks,
Colby

[JW]

Thanks for posting this problem,

I am looking for a new 128GB computer and I hesitate between DDR4 and DDR5 for this precise issue.
Would the alignement instability persist with only 2 sticks (64GB) or if you use the ram at a lower frequency?

[PolarNick]

The more RAM sticks you use - the bigger chance of RAM instability and the lower maximum frequency at which memory is stable. Also, if you will check some kind of motherboard/CPU memory-compatibility list - you will note that supported 4xSticks have lower maximum frequency than 2xSticks.

memtest is a good stress test but it has different code, so it is possible that only Metashape-like code provokes RAM instability. For example - may be it is important to increase overal temperature inside PC, and in case of Metashape processing this is more probable due to GPU usage, which is ignored in memtest. Or may be it is important to use some AVX-instructions and so on.

> Would the alignement instability persist with only 2 sticks (64GB) or if you use the ram at a lower frequency?

It depends on the nature of the problem:

1) If the instability is because of too high frequency - setting lower frequency will help. F.e. you bought two sets of 2xSticks (4xSticks in total), they are declared to run at 6400Mhz, but this is guaranteed only for 2xSticks and only for compatible motherboards/CPUs. So if you face crashes due to RAM instability - you can try to lower memory frequency - for example to 3200Mhz or lower (just to be sure) and/or use only 2xSticks.

2) If the reason is a defective RAM stick - nothing will help - the best you can do is to return RAM to the shop. Also you can try to locate the defective RAM Stick:

2.1) Set lower memory frequency (f.e. 3200Mhz instead of 6400Mhz)
2.2) Remove the first stick out of four - check processing for instability (i.e. using 2+3+4 sticks)
2.3) Put the first stick back and remove the second stick out of four - check processing for instability (i.e. using 1+3+4 sticks)
2.4-2.5) Repeate the same processing attempt with 1+2+4 sticks and with 1+2+3 sticks

And so - if, for example, processing is stable only if you use 1+3+4 sticks - then the second stick is defective and shouldn't be used. Note that you shouldn't use these 1+3+4 sticks AND some other random single stick bought after that. Because memory sticks are guaranteed to be stable only if they are all from the same factory batch.

[colbyrand]

Hi JW and PolarNick, thank you for the suggestions regarding RAM troubleshooting! I am starting to believe this issue doesn't have anything to do with my RAM, and instead there must be an issue somewhere else. I am having this issue with Metashape crashing on my PC at work. I also have my own personal PC at home, which I tested the same Metashape project on, and it was able to align photos without crashing!

My theory is this: If Metashape is able to function properly on my home PC, then those sticks of RAM (two sticks of 32GB DDR5 RAM) must be good. I also tested them with Memtest86, and they passed with zero errors. If I then brought those functional sticks of RAM to work and slotted them into my work computer, and Metashape still crashed, then I could conclude that the issue wasn't with the RAM, since I had already verified those worked on a different computer. However, if Metashape was able to align photos without crashing, then I could conclude that it was indeed an issue with faulty RAM.

When I ran this test at work with my personal RAM sticks, Metashape crashed again when aligning photos. This leaves me to conclude that there is some other issue not related to the RAM. Perhaps an issue with the CPU or GPU? Or some Metashape settings? I'm not sure what else to troubleshoot, besides maybe swapping my personal GPU with the one at work to see if the GPU is causing issues.

Any other thoughts would be much appreciated!

Specs of my home computer:
OS: Windows 11 Pro
CPU: Intel Core i5-12600K
GPU: NVIDIA GeForce RTX 3060 (MSI model)
Memory: 64GB DDR5 RAM (4 sticks of Corsair Vengeance)
Storage: 2TB NVMe SSD
Motherboard: Gigabyte Z790 AORUS ELITE AX

Specs of my work computer:
OS: Windows 11 Enterprise (Version 22H2)
CPU: Intel Core i9-13900K
GPU: NVIDIA GeForce RTX 4070 Ti (MSI model)
Memory: 128GB DDR5 RAM (4 sticks of Corsair Vengeance CMK64GX5M2B6400C32)
Storage: 4TB NVMe SSD
Motherboard: Gigabyte Z790 AORUS ELITE AX

[colbyrand]

See attached for the Windows Event Viewer error log.

[PolarNick]

When you changed memory sticks - was their frequency in BIOS automatically updated? Asking just in case if the problem is because of memory overclocking (I don't know does it reset automatically on memory change).

To troubleshoot CPU vs GPU:

1) You can just enable writing log to disk (Tools->Preferences->tick Write log to file) and after reproducing the crash - check what last 20 lines in log was. Please post them here - they will tell is it a GPU-bound processing stage or is it a CPU-only stage.
2) You can try to align photos without GPU (Tools->Preferences->GPU->untick GPU).

In case if the problem seems to be with CPU:

11) Ensure there are no overclocking in BIOS.
12) What are your CPU temperatures while processing? If temperatures are high - it can be due to poorly applied stock thermal paste or something like that.

[geodamkru]

Hi, has anyone managed to solve this problem. I have a similar problem when aligin photo - crash or agisoft turns off.

OS: Windows 11
CPU: Intel Core i9-14900K
GPU: NVIDIA GeForce RTX 4070
Memory: 64GB DDR5 RAM 2x32 5600Mhz kingstone fury beast black
Storage: 1TB samsung 980 pro
Motherboard: Gigabyte Z790 AORUS ELITE AX

[Alexey Pasumansky]

Hello all,

In most cases when such crashes are observed the problem is related to hardware: RAM failures, CPU/RAM overclocking, system overheating and similar.

Quite a lot of cases with i9-13900K and i9-14900K (and similar) CPU system crashes were related to the mentioned hardware issues, as confirmed by users.
So if such random crashes occur during CPU/RAM-intensive steps, it is recommended to:
- perform multithreaded Memtest86 checks,
- confirm if crashes happen regardless the number of RAM modules used (crashes on any single module installed),
- confirm that system and CPU temperature under heavy load is reasonable (like less than 80-90 C degrees),
- disable any overclocking,
- lower RAM frequency considerably and check if it helps (like 2000-2500 MHz).

[geodamkru]

Good morning, thank you for your answer, I followed all the points on your list, unfortunately without result.

[colbyrand]

I have also tried all of those troubleshooting tips but have not been able to fix this issue. This seems to be a Metashape issue only, as I have not had crashing issues with any other applications. I am not overclocking any of my components, the temperatures are fine, and my RAM does not have any issues.

[Bzuco]

@colbyrand  Check also your NVMe SSDs at least with some SMART monitoring tools if there are no errors reported or warnings. SSDs are also just memory modules. There could be problem when Metashape is saving temporary files during estimating camera locations phase....you never know.

[Tumno]

Hi there,

I also built a new PC a few weeks ago with nearly identical specs:

OS: first Windows 11 Professional, later on downgraded to Windows 10 Professional while testing and kept it for now
CPU: Intel Core i9-13900K
GPU: NVIDIA GeForce RTX 4070 Ti (ASUS TUF model)
Memory: 64GB DDR5 RAM (2 sticks of Corsair Vengeance)
Storage: different Samsung NVMe SSDs
Motherboard: Gigabyte Z790 AORUS ELITE AX

I've also experienced a quite similar problem, especially with big datasets with 1500+ high res pictures, where Metashape completed the matching points phase, but simply closed all the time while estimating camera locations (at a progress of about 70% - 85%) most of the time without an error message, or anything unusual in logs..

First I tested as suggested with Memtest86 without negative results, every stresstest on GPU, CPU, SSDs and RAM finished successfull, reinstall of every driver and so on, but still metashape was crashing..

I even tried RAM modules of different brands Corsair, Kingston, G.Skill, because the most suggested solution was about faulty RAMs, but it kept crashing with every single brand..

I figured out that metashape had no problem with datasets around 300-500 pictures, so I split up my latest project (5050 pictures) in pieces and merged it back together later on, what caused lots of additional work and bad results in some parts..

I kept following the thread, but had to wait until I got approved on the forums to contribute.

Yesterday evening I read Bzuco's post about SMART tests on NVMe SSDs. I did SMART tests again, like I already did when the problem first occurred, with good results for every parameter.

What made the difference this time and finally seems to led to the solution, at least on my side, was the fact that this time I didn't use any 3rd party software for the SMART tests, but the Samsung Magician software, because there are only Samsung SSDs and NVMes in my current setup (990Pro, 970Evo, 870Evo)..

When inspecting Drive Details in Samsung Magician I realized that all NVMes had firmware updates pending..

I updated the firmware and started the alignment process, which failed every time before (around 20 to 25 tries with different hardware setups and metashape settings), with all of the 5050 pictures..

A few hours later, today in the morning the process finished successfully for the first time on my new PC, all pictures aligned, no crash, full satisfaction

I did a reset of the alignment and ran the process successfully again right before I wrote this post, to confirm the problem of crashing while camera alignment as solved, as stated before, at least for my side..


Long story short..

@Bzuco: Thanks for pushing me towards the solution!

@colbyrand: You didn't mention what NVMe brand you are using, if it's Samsung as well take a look if there are pending firmware updates.

@geodaamkru: Since you mentioned that you are using a Samsung NVMe check for firmware updates in Samsung Magician "Drive Details", if you didn't already do that.


I'll keep you informed if there'll be a crash again, or if it's been the final solution for me

(Alignments successful since fix 2/2)

[geodamkru]

Good morning,

I tried to replace every part of the system, frames, graphics card, disk, but with no results. Only replacing the motherboard with an Asus Prome z790-P allowed it to work without errors. Thank you for all the suggestions.

[3dMB Ltd]

I’m having the same issues with the following:
I9-14900k
RTX 3090Ti
128GB 5600mhz DDR5 Corsair Vengeance (4 x 32GB)
Gigabyte Aorus Master X

I have tried swapping and changing pretty much every piece of hardware, done memtest86, removed & swapped RAM around, different motherboard. The only way I’ve found to rectify if my turning off the Efficiency cores in the BIOS.

[Alexey Pasumansky]

Getting more reports from i9-13900 and i9-14900 CPU users, we also found system stability issues reports (unrelated to Metashape), for example:
https://community.intel.com/t5/Processors/i9-13900K-very-frequent-crashes-Windows-11-with-apps-games-and/td-p/1527297

Discussions related to those problems suggest to run stress tests to detect problems, such as:
-  Intel Processor Diagnostic Tool (https://www.intel.com/content/www/us/en/download/15951/intel-processor-diagnostic-tool.html), may require to customize the default test time and set longer time.
-  Prime95 (https://www.mersenne.org/download/#stresstest)

If anyone who has mentioned CPU models and issues during processing in Metashape could try at least one (or even both) tests, please share the results.