Generate Ortho - Big Tiff Error

[aloerch]

This problem is occurring during the "Generate Ortho" step, not the export ortho step.

I have a WorldView 3 satellite image that I am trying to generate an orthoimage for using an imported DSM. The image has 8 bands, and a pixel size of 0.3 m.

If I choose "Generate Ortho" and try to generate the ortho with the 0.3 m pixel size, I get this error (from the logs):

Code: [Select]

2020-04-06 12:16:27 Generating orthomosaic...
2020-04-06 12:16:27 initializing...
2020-04-06 12:16:27 Analyzing DEM...
2020-04-06 12:16:27 estimating tile boundaries... done in 7.39302 sec
2020-04-06 12:16:35 generating 24192x21504 orthomosaic (9 levels, 0 resolution)
2020-04-06 12:16:35 selected 1 cameras
2020-04-06 12:16:35 saved orthomosaic data in 0.006465 sec
2020-04-06 12:16:35 saved camera partition in 0.000693 sec
2020-04-06 12:16:35 scheduled 1 orthophoto groups
2020-04-06 12:16:35 loaded camera partition in 0.000307 sec
2020-04-06 12:16:35 loaded orthomosaic data in 0.000518 sec
2020-04-06 12:16:35 Orthorectifying images...
2020-04-06 12:16:35 Orthorectifying 1 images
2020-04-06 12:18:24 19jul24184818-p2as-012267371010_01_p001: 24192x21504 -> 23831x20773
2020-04-06 12:19:45 libtiff error: Maximum TIFF file size exceeded
2020-04-06 12:19:45 libtiff error: Maximum TIFF file size exceeded
2020-04-06 12:19:46 Finished processing in 198.518 sec (exit code 0)
2020-04-06 12:19:46 Error: TIFFWriteTile: unexpected error: memory stream
However, if I change the desired pixel size to 1.6 m, the orthomosaic generates properly, just at a reduced resolution.

I know I can select "BigTiff" during the export orthomosaic process, but this does not seem to be possible with the Generate Orthomosaic process. Is there a "Tweak" parameter I can add, or can this be fixed? I need the output orthomosaic to be the same pixel size as the original image (0.3 m).

[Alexey Pasumansky]

Hello aloerch,

Can you please specify the dimensions of the input images and the version of Metashape Pro used, including the build number?

[aloerch]

Hi Alexey,

Here are the details you asked for, followed by the general workflow I'm following:

Agisoft MetaShape Professional 1.6.2 Build 10247 (64 Bit)
Image dimensions: 23996 x 20888 (9 bands)
Image file size: 9,022,144,080 bytes
Linux Mint 19.3 64-bit
128 GB Ram

Goal:
Take a single Worldview 3 (WV3) satellite image and generate an orthorectified image using an imported DEM.

General Workflow:
1. Load the pan-sharpened WV3 9-band image
2. Load the same image with only 1 band, the panchromatic band
3. Check Camera Calibration settings to ensure both are listed as RPC type
4. Add ground control points and optimize cameras
5. Import a LiDAR-based DEM
6. Disable the panchromatic-only image
7. Generate Orthomosaic using the imported DEM

The failure occurs on step #7 with the error I posted previously. That error appears to show that Generate Orthomosaic's LibTIFF implementation is not able to generate a tiff that is larger than 4GB.

If I use the exact same input image but instead of 9-bands I reduce it to 4-bands, thus reducing the input image's file size, then Generate Orthomosaic works without any errors, and I can then export the orthomosaic without any errors.

Also, if I use the exact same input image but during "Generate Orthomosaic" I set the output pixel size to 1.6 m instead of the native 0.3 m, that also works without errors and i can then export the orthomosaic without any errors.

Ideally, I would be able to generate the orthomosaic at the native resolutions (0.3m) with the given number of bands (9), but the error seems to indicate that LibTIFF can't write the generated file(s) because they are larger than 4GB.

Now that Agisoft Metashape supports satellite imagery, it would be helpful to be able to generate the orthomosaic using BigTiff when necessary... I am guessing that is the problem.

[aloerch]

Modified previous post with more details.

[Alexey Pasumansky]

Hello aloerch,

Thank you for providing the detailed information.

Is it possible to share any sample source image to support@agisoft.com or provide the link to the public page, where from such image samples can be downloaded?

[aloerch]

Hi Alexey,

Thank you for looking into this. Unfortunately, we are prohibited from sharing this particular dataset by the satellite vendor (Digital Globe). But, we can work around the problem by generating and outputting several 4 band images instead of a single 9-band image. It would be helpful though if you might look into the libtiff usage for large files in the generate Ortho workflow.

[jslyder]

Was there a resolution to this issue?  I'm getting a similar error, and am wondering if it may be related.  When building the orthomosaic for a pair of WorldView 3 scenes, I get the error below.  This happens for both the panchromatic images as well as for the multispectral ones.  I'm on 1.6.2 build 10247.Thanks. 

2020-04-14 09:25:10 Orthorectifying images...
2020-04-14 09:25:10 Orthorectifying 2 images
2020-04-14 09:34:18 16APR21181234-P1BS-500658906010_01_P001.TIF: 42523x51601 -> 42523x51601
2020-04-14 09:34:31 16APR21181146-P1BS-500658906010_01_P001.TIF: 42523x51601 -> 42523x51601
2020-04-14 09:35:57 libtiff error: Write error at scanline 3712
2020-04-14 09:35:57 libtiff error: Write error at scanline 3712
2020-04-14 09:36:13 libtiff error: Write error at scanline 48896
2020-04-14 09:36:13 libtiff error: Write error at scanline 48896
2020-04-14 09:36:22 Error: TIFFWriteTile: unexpected error: memory stream

[Alexey Pasumansky]

Hello,

We are currently studying the issue.

[Alexey Pasumansky]

The problem will be fixed in the 1.6.3 version update.

[aloerch]

Hi Alexey,

Thank you again for working on this. Can you please confirm that the recently released 1.6.3 version fixed this issue?

Thanks!

[Alexey Pasumansky]

It should be fixed now. If you are still observing any problems, please report.

[vbarty]

Hi,

It is a quite old thread, but I just had similar error when generating orthomosaic on version 1.7.1 build 11797 (64 bit).
Can it be related to the quite big size of the orthomosaic and maby PC not powerful enough?
The processing already lasted for a few hours, and I don't know if I should restart it. 
Please take a look at the part of log from console

Code: [Select]

2021-02-26 14:44:47 boundaries extracted in 0.055 sec
2021-02-26 14:44:50 13 images blended in 3.127 sec
2021-02-26 14:44:51 loaded partition in 0.306 sec
2021-02-26 14:44:51 boundaries extracted in 0.056 sec
2021-02-26 14:44:52 image key: 12
2021-02-26 14:44:52 region overflow: reading [-1, 76] - [62, 80] from 186x145 image
2021-02-26 14:44:52 libtiff error: 4294967168: Col out of range, max 185
2021-02-26 14:44:58 Finished processing in 9911.1 sec (exit code 0)
2021-02-26 14:44:58 Error: TIFFReadTile: unexpected error: memory stream

[Alexey Pasumansky]

Hello vbarty,

Which orthomosaic generation parameters you are using and what is the source images resolution?

Are you observing same issues, if you leave only a few images (like 20) enabled for this project and used for orthomosaic generation?

[vbarty]

Hi Alexey,

Orthomosaic was build on DEM surface, blending mode Mosaic, hole filling enabled, default pixel size (~0,03cm) and I didn't specify boundaries.
Source images are 6000x4000.

I've tried with 100 pictures and orthomosaic was generated without problems.
I've also tested the cloud processing with bigger dataset and it also went fine. 

[Alexey Pasumansky]

Hello vbarty,

Is the problem reproducible, if you leave only first 20 enabled images in the original project prior to the orthomosaic generation?