can't write pyramid data

[doug_peterson@fws.gov]

Dear Forum,

I'm experiencing repeated "can't write pyramid data" errors when attempting to create an orthomosaic from a DEM.  The general conditions are:

Running the process locally on laptop with 32 GB RAM and 1 TB solid state hard drive with 293 GB of free space

2775 images, 10 ground control points, 1,073,203 tie points,  DEM 36802x44405, 1.57 cm/pix, images are RGB (.jpg, .dng) from a Ricoh GR2 camera

Agisoft Metashape Professional Version 1.6.2 build 7838 (64 bit) with a shared license within US Department of Interior (i.e., computer looks to DOI server occasionally to confirm the license, but all processing is done locally)


First time ran with 113 GB free drive space and error was cannot write pyramid data, not enough disk space
Second time ran with 293 GB free drive space and error was cannot write pyramid data, not enough disk space
Third and fourth time ran with 293 GB free drive space and error was cannot write pyramid data.
The job always fails in about an hour, after working to orthorectify sets of 28 photos at a time

The job is a recurrent analysis based on monthly flights over a study site.  The last .TIFF orthmosaic file that was created was only 9.3 GB.

Any insight would be appreciated.

[Alexey Pasumansky]

Hello doug_peterson,

The orthomosaic generation procedure begins with the orthorectification of the source images by projecting them on the selected surface. The orthorectified images are stored in the project.files subfolders in imagesN.zip archives. You can monitor the size of these archives during the processing and make an estimation, how much space they would occupy. After the orthoretification the images will be blended to the orthomosaic, which would require additional disk space.

The size of the orthomosaic data (including the orthorectified depends on the selected orthomosaic resolution and bit depth of the source images.

Additionally I suggest to check, if you are observing the same problem in the latest 1.5.5 release version.

[doug_peterson@fws.gov]

Hi Alexey,

Thanks for the reply.

In the project folder I monitored a folder labeled "0" that increased in size during the process of orthorectifying (I couldn't find the  imagesN.zip that you referenced).  In any case, the job failed and the "can't write pyramid data" error returned when the "0" folder exceeded the remaining space of the hard drive, so it appears to be a space issue.

The processing job crashed at 12% completion when the "0" folder was approximately 345 GB.  Assuming a linear relationship between the processing time & required drive space, this would imply that 2.785 TB of drive space would be needed to complete the processing.  This seems very odd.

We have processed a similar job before on the same computer (1 TB drive) and we successfully produced the orthmomosaic.  Here is the project file information for the runs:

Successful run:
2197 cameras
12 markers
756,647 tie points
Depth Maps (2184, High quality, moderate filtering)
Dense Cloud (729,670,601 points, High quality)
DEM (40827x49075, 1.45 cm/pix)
Orthomosaic (53759x79873, 7.26 mm/pix)  - resulting .TIFF file was 9.8 GB



Failed run (this time):
2775 cameras
10 markers
1,073,203 tie points
Depth Maps (2771, High quality, Aggressive filtering)
Dense Cloud (664,752,233 points, High quality)
DEM (36802x44405, 1.57cm/pix)


Do you see any differences here that would explain why we were able to complete the job on this computer before, but not now?

[Alexey Pasumansky]

Hello doug_peterson,

You should see new files being generated in project.files/0/0/orthomosaic/ directory (providing that you have only one chunk and one orthomosaic in the project.

Are you processing DNG images or JPGs this time?

[doug_peterson@fws.gov]

Hello Alexey,

Yes, I believe that is the folder I was monitoring where the temporary files were being generated.  I wanted the folder size increase and the processing job failed when it approached the hard drive capacity, so this seems to be the issue.

The project has only one chunk, and we are processing both the DNG and JPG in the same job.  I am following the same protocol that has worked before when there was the same or even less space on the hard drive, so am confused to why it no longer works.  Perhaps the temporary files are not being zipped and compressed as the job proceeds?

Any insight is appreciated as I am unable to complete this project.

Thank you,

Doug