Forum

Author Topic: Export DEM (.TIF) for importing in Civil 3D  (Read 9304 times)

BobvdMeij

  • Full Member
  • ***
  • Posts: 157
    • View Profile
Export DEM (.TIF) for importing in Civil 3D
« on: August 01, 2018, 03:20:18 PM »
Dear all,

I have been trying to import several DEM files, created in Agisoft and exportedas .TIF files, into our Civil 3D environment. Unfortunately I haven't been very succesful at it; Civil 3D does not seem to recognize the file, at all.

Importing in Civil 3D using the MAPIINSERT command produces an error message stating that <filename>.tif was not found or not valid. Considering the DEM file is well over 3GB, and knowing Civil3D dislikes such large file sizes, I exported the DEM at a coarser resolution to produce a considerably smaller file of only 300MB. Unfortunately importing the smaller TIF file produces an identical error message.

Searching on the internet demonstrated that importing .TIF files produced in Agisoft (or P4D for that matter) is often problematic. The issues described above are seemingly well known to many others who have tried this before. Following suggestions mentioned by some I imported the .TIF file into our QuantumGIS (QGIS) software and exported the DEM to a new .TIF file at the same resolution. Again, however, this does not resolve the problem and importing into Civil3D remains impossible (the same error message appears).

It may be worth noting that importing the 300MB .TIF file into QGIS, then saving/ exporting it again to a new .TIF file (at the very same resolution), suddenly produces a 2.4GB .TIF file.

Any help is much appreciated, thanks in advance!

stihl

  • Sr. Member
  • ****
  • Posts: 408
    • View Profile
Re: Export DEM (.TIF) for importing in Civil 3D
« Reply #1 on: August 01, 2018, 03:27:33 PM »
Hey Bob,

How did you import the TIFF into Civil 3D? Did you use the XREF function? See: https://forums.autodesk.com/t5/civil-3d-forum/insert-raster-image/td-p/4878368

As for the file size difference between QGIS and Photoscan;  Photoscan uses LZW lossless compression on the TIFF file which greatly reduces the file size without data loss. I'm uncertain that QGIS also does this by default.

When using GlobalMapper to export TIFFs you'll notice a huge difference in file size if you enable or disable LZW compression. I'd advice to use the compression as this usually means less loading times when importing into a GIS.

BobvdMeij

  • Full Member
  • ***
  • Posts: 157
    • View Profile
Re: Export DEM (.TIF) for importing in Civil 3D
« Reply #2 on: August 01, 2018, 03:37:41 PM »
Hey Bob,

How did you import the TIFF into Civil 3D? Did you use the XREF function? See: https://forums.autodesk.com/t5/civil-3d-forum/insert-raster-image/td-p/4878368

As for the file size difference between QGIS and Photoscan;  Photoscan uses LZW lossless compression on the TIFF file which greatly reduces the file size without data loss. I'm uncertain that QGIS also does this by default.

When using GlobalMapper to export TIFFs you'll notice a huge difference in file size if you enable or disable LZW compression. I'd advice to use the compression as this usually means less loading times when importing into a GIS.

Thanks for your quick response Stihl,

I haven't used XREF before so I gave it a try straight away using the dropdown option 'Attach Image'. Unfortunately this produces the following error message:

Code: [Select]
This file cannot be opened.

The problem can be caused by several things:
- The data in the file is corrupted
- The file is an earlier format that is no longer supported
- The file format is not supported

Try to obtain a better copy of the file, recreate the file using different settings, or convert the file to a supported file format.

For what it's worth, the same prompt appears when I try to import either the smaller .TIF file (straight from Agisoft) or the larger (and plausibly uncompressed) .TIF file exported from QGIS.

Any other ideas ;)?

BobvdMeij

  • Full Member
  • ***
  • Posts: 157
    • View Profile
Re: Export DEM (.TIF) for importing in Civil 3D
« Reply #3 on: August 01, 2018, 04:38:11 PM »
Just ran another test but this time trying to import an Orthomosaic .TIF file. The original 15GB file wouldn't work, for obviously reasons, but exporting a much coarser resolution 98MB version of the same orthomosaic worked straight away! Also when highlighting the DEM .TIF file in Civil3D's MAPIINSERT dialog all preview and file metadata fields are empty. On the contrary, when highlighting the Orthofoto .TIF it succesfully gives me a preview image of the file, in addition all file metadata fields contain data (Please see the screendump attached.). Apparently critical information required by Civil3D to correctly read a file IS exported for Orthomosaics but missing or corrupted for exported DEMS.

The orthofoto also being a .TIF file (similar to the DEM file) the cause must be related to how Agisoft exports/compresses/alters the .TIF format of DEMs prior to exporting. When exporting the Orthomosaic I ensured to keep all settings default, meaning TIFF compression is set to LZW and JPEG quality to 90. These options, however, are not available when exporting a DEM.




BobvdMeij

  • Full Member
  • ***
  • Posts: 157
    • View Profile
Re: Export DEM (.TIF) for importing in Civil 3D
« Reply #4 on: August 06, 2018, 12:47:32 PM »
No one else?

gto234

  • Newbie
  • *
  • Posts: 40
    • View Profile
Re: Export DEM (.TIF) for importing in Civil 3D
« Reply #5 on: August 07, 2018, 12:09:45 PM »
Hi friend,
I think that MAPIINSERT works only for georeferenced raster images. DEM files are also tiffs but have elevation information in them. I think you can import DEM files in Civil 3D but with different command. Check Civil 3D's manual or autodesk forums.
EDIT: Check https://knowledge.autodesk.com/support/civil-3d/learn-explore/caas/CloudHelp/cloudhelp/2016/ENU/Civil3D-UserGuide/files/GUID-5E670EBA-4898-4CEB-A619-EE0F89E416E6-htm.html
« Last Edit: August 07, 2018, 12:11:34 PM by gto234 »

GPC

  • Jr. Member
  • **
  • Posts: 83
    • View Profile
    • Geopro
Re: Export DEM (.TIF) for importing in Civil 3D
« Reply #6 on: August 09, 2018, 05:59:35 AM »
Right click surfaces from the toolspace and create surface from DEM.  Alternatively just export a point cloud in. LAS format and read it in as a point cloud.
When things get weird, the weird turn pro.

BobvdMeij

  • Full Member
  • ***
  • Posts: 157
    • View Profile
Re: Export DEM (.TIF) for importing in Civil 3D
« Reply #7 on: August 10, 2018, 09:32:34 AM »
Hi friend,
I think that MAPIINSERT works only for georeferenced raster images. DEM files are also tiffs but have elevation information in them. I think you can import DEM files in Civil 3D but with different command. Check Civil 3D's manual or autodesk forums.
EDIT: Check https://knowledge.autodesk.com/support/civil-3d/learn-explore/caas/CloudHelp/cloudhelp/2016/ENU/Civil3D-UserGuide/files/GUID-5E670EBA-4898-4CEB-A619-EE0F89E416E6-htm.html

I have your suggestion a try but, unfortunately, the DEM file is still considered invalid and cannot be imported.

In the meantime we have succesfully imported a DEM file by importing it into QGIS and then exporting the same DEM to a new .TIF file. Unfortunately, however, because this re-exporting through QGIS turns a reasonably sized 300mb file into a mega 2GB+ file, the DEM is barely usable in C3D.

Alexey Pasumansky

  • Agisoft Technical Support
  • Hero Member
  • *****
  • Posts: 13211
    • View Profile
Re: Export DEM (.TIF) for importing in Civil 3D
« Reply #8 on: August 10, 2018, 02:38:12 PM »
Hello BobvdMeij,

Is there any difference in raster header between PhotoScan-exported and QGIS re-saved DEM?
Best regards,
Alexey Pasumansky,
Agisoft LLC

BobvdMeij

  • Full Member
  • ***
  • Posts: 157
    • View Profile
Re: Export DEM (.TIF) for importing in Civil 3D
« Reply #9 on: August 10, 2018, 03:42:37 PM »
Hello BobvdMeij,

Is there any difference in raster header between PhotoScan-exported and QGIS re-saved DEM?

Any suggestions to extract the raster header?

I tried to import the QGIS-DEM into 'GeoTiffExamine' but the file is too big as the program runs out of memory.

Its also worth mentioning that the Photoscan-DEM (exported without the WorldFile) is 'not understood' by the program. The Photoscan-DEM (exported with the WorldFile) is imported succesfully into the GeoTiffExamine software.

Alexey Pasumansky

  • Agisoft Technical Support
  • Hero Member
  • *****
  • Posts: 13211
    • View Profile
Re: Export DEM (.TIF) for importing in Civil 3D
« Reply #10 on: August 10, 2018, 04:02:24 PM »
Hello BobvdMeij,

Maybe you can export DEM in lower resolution and re-save it in QGIS? Then send both files to us to support@agisoft.com.
Best regards,
Alexey Pasumansky,
Agisoft LLC

BobvdMeij

  • Full Member
  • ***
  • Posts: 157
    • View Profile
Re: Export DEM (.TIF) for importing in Civil 3D
« Reply #11 on: August 10, 2018, 04:26:59 PM »
Hello BobvdMeij,

Maybe you can export DEM in lower resolution and re-save it in QGIS? Then send both files to us to support@agisoft.com.

You've got mail!

Alexey Pasumansky

  • Agisoft Technical Support
  • Hero Member
  • *****
  • Posts: 13211
    • View Profile
Re: Export DEM (.TIF) for importing in Civil 3D
« Reply #12 on: August 10, 2018, 10:14:44 PM »
Hello BobvdMeij,

Thanks, we have received the files. I will inform you about our investigation results.
Best regards,
Alexey Pasumansky,
Agisoft LLC

gto234

  • Newbie
  • *
  • Posts: 40
    • View Profile
Re: Export DEM (.TIF) for importing in Civil 3D
« Reply #13 on: August 10, 2018, 11:20:40 PM »
Check also here https://forums.autodesk.com/t5/civil-3d-forum/creating-surface-from-geotiff-dem/td-p/7826894
What version of Civil 3D do you have? If you have old version it might not support DEM files.

Alexey Pasumansky

  • Agisoft Technical Support
  • Hero Member
  • *****
  • Posts: 13211
    • View Profile
Re: Export DEM (.TIF) for importing in Civil 3D
« Reply #14 on: August 20, 2018, 02:33:05 PM »
After more detailed investigation, we assume that the problem is related to the internal TIFF representation: in the current release version of PhotoScan all TIFFs are saved as "tiled TIFF".  It appears that Civil3D for the DEM may accept only non-tiled TIFF and we'll try to implement this option in the version 1.5.0 pre-release. 
Best regards,
Alexey Pasumansky,
Agisoft LLC