Forum

Author Topic: How to export depth map as matlab file without saving the image ?  (Read 2785 times)

lyhour

  • Newbie
  • *
  • Posts: 41
    • View Profile
Excuse me! I want to extract the depth data of selected camera. I have the python code which I found in the forum and also in the way to export depth in the GUI. However, The mentions method are save the depth data to the image file. I want to know is there any method to obtain the depth data and save to other file such as Matlab. Thank you very much.

Alexey Pasumansky

  • Agisoft Technical Support
  • Hero Member
  • *****
  • Posts: 14813
    • View Profile
Re: How to export depth map as matlab file without saving the image ?
« Reply #1 on: March 21, 2022, 05:27:31 PM »
Hello lyhour,

Please see the following script that allows to save the scaled depth maps in F32 format. The script uses numpy module for faster processing.
I believe you can adjust the output, based on numpy array, to fit your needs and output format expectations.

Code: [Select]
import Metashape
from PySide2 import QtGui, QtCore, QtWidgets

global NUMPY
NUMPY = True

try:
import numpy
except ImportError:
NUMPY = False
print("numpy module not installed!")

class ExportDepthDlg(QtWidgets.QDialog):

def __init__ (self, parent):
QtWidgets.QDialog.__init__(self, parent)
self.setWindowTitle("Export depth maps")

self.btnQuit = QtWidgets.QPushButton("&Close")
self.btnP1 = QtWidgets.QPushButton("&Export")
self.pBar = QtWidgets.QProgressBar()
self.pBar.setTextVisible(False)

#self.selTxt =QtWidgets.QLabel()
#self.selTxt.setText("Apply to:")
self.radioBtn_all = QtWidgets.QRadioButton("Apply to all cameras")
self.radioBtn_sel = QtWidgets.QRadioButton("Apply to selected")
self.radioBtn_all.setChecked(True)
self.radioBtn_sel.setChecked(False)

self.formTxt = QtWidgets.QLabel()
self.formTxt.setText("Export format:")
self.formCmb = QtWidgets.QComboBox()
self.formCmb.addItem("1-band F32")
self.formCmb.addItem("Grayscale 8-bit")
self.formCmb.addItem("Grayscale 16-bit")

layout = QtWidgets.QGridLayout()   #creating layout
#layout.setSpacing(10)
layout.addWidget(self.radioBtn_all, 0, 1)
layout.addWidget(self.radioBtn_sel, 1, 1)
layout.addWidget(self.formTxt, 0, 2)
layout.addWidget(self.formCmb, 1, 2)
layout.addWidget(self.pBar, 2, 0)
layout.addWidget(self.btnP1, 2, 1)
layout.addWidget(self.btnQuit, 2, 2)
self.setLayout(layout) 

proc_depth = lambda : self.export_depth()
QtCore.QObject.connect(self.btnP1, QtCore.SIGNAL("clicked()"), proc_depth)
QtCore.QObject.connect(self.btnQuit, QtCore.SIGNAL("clicked()"), self, QtCore.SLOT("reject()"))

self.exec()


def export_depth(self):

global NUMPY
if not NUMPY:
Metashape.app.messageBox("Script aborted: numpy module is not installed.")
return 0

app = QtWidgets.QApplication.instance()
global doc
doc = Metashape.app.document
chunk = doc.chunk #active chunk

if self.formCmb.currentText() == "1-band F32":
F32 = True
elif self.formCmb.currentText() == "Grayscale 8-bit":
F32 = False
elif self.formCmb.currentText() == "Grayscale 16-bit":
F32 = False
else:
print("Script aborted: unexpected error.")
return 0

selected = False
camera_list = list()
if self.radioBtn_sel.isChecked():
selected = True
for camera in chunk.cameras:
if camera.selected and camera.transform and (camera.type == Metashape.Camera.Type.Regular):
camera_list.append(camera)
elif self.radioBtn_all.isChecked():
selected = False
camera_list = [camera for camera in chunk.cameras if (camera.transform and camera.type == Metashape.Camera.Type.Regular)]


if not len(camera_list):
print("Script aborted: nothing to export.")
return 0

output_folder = Metashape.app.getExistingDirectory("Specify the export folder:")
if not output_folder:
print("Script aborted: invalid output folder.")
return 0

print("Script started...")
app.processEvents()
if chunk.transform.scale:
scale = chunk.transform.scale
else:
scale = 1
count = 0

for camera in camera_list:
if camera in chunk.depth_maps.keys():
depth = chunk.depth_maps[camera].image()
if not F32:
img = numpy.frombuffer(depth.tostring(), dtype=numpy.float32)
depth_range = img.max() - img.min()
img = depth - img.min()
img = img * (1. / depth_range)
if self.formCmb.currentText() == "Grayscale 8-bit":
img = img.convert("RGB", "U8")
img = 255 - img
img = img - 255 * (img * (1 / 255))#normalized
img = img.convert("RGB", "U8")
elif self.formCmb.currentText() == "Grayscale 16-bit":
img = img.convert("RGB", "U16")
img = 65535 - img
img = img - 65535 * (img * (1 / 65535))#normalized
img = img.convert("RGB", "U16")
else:
img = depth * scale
img.save(output_folder + "/" + camera.label + ".tif")
print("Processed depth for " + camera.label)
count += 1
self.pBar.setValue(int(count / len(camera_list) * 100))
app.processEvents()
self.pBar.setValue(100)
print("Script finished. Total cameras processed: " + str(count))
print("Depth maps exported to:\n " + output_folder)
return 1


def export_depth_maps():

app = QtWidgets.QApplication.instance()
parent = app.activeWindow()
dlg = ExportDepthDlg(parent)

Metashape.app.addMenuItem("Custom menu/Export Depth Maps", export_depth_maps)

If you have any problems with the script, please provide the example of the expected output and format description.
Best regards,
Alexey Pasumansky,
Agisoft LLC

lyhour

  • Newbie
  • *
  • Posts: 41
    • View Profile
Re: How to export depth map as matlab file without saving the image ?
« Reply #2 on: March 21, 2022, 05:44:20 PM »
Dear Alexey,
Thank you very much for your kindly response and your script. I have been tried to modify the script already by creating the converting metashape image class to numpy array but it cannot achieve it. It would be better if you could guide me to adding the script.
Code: [Select]
for camera in camera_list:
    if camera in chunk.depth_maps.keys():
depth = chunk.depth_maps[camera].image()
if not F32:
img = numpy.frombuffer(depth.tostring(), dtype=numpy.float32)
depth_range = img.max() - img.min()
img = depth - img.min()
img = img * (1. / depth_range)
if self.formCmb.currentText() == "Grayscale 8-bit":
img = img.convert("RGB", "U8")
img = 255 - img
img = img - 255 * (img * (1 / 255))#normalized
img = img.convert("RGB", "U8")
elif self.formCmb.currentText() == "Grayscale 16-bit":
img = img.convert("RGB", "U16")
img = 65535 - img
img = img - 65535 * (img * (1 / 65535))#normalized
img = img.convert("RGB", "U16")
else:
img = depth * scale
#img.save(output_folder + "/" + camera.label + ".tif")
               # what should I put here to convert img to np array ? this is just my concept. It cannot work.
                  tmp = np.empty(shape=(img.width,img.height))
          for y in range(img.height):
      for x in range(img.width):
tmp[x,y] = img[x,y]
                 

Best Regards,

LYHOUR

Alexey Pasumansky

  • Agisoft Technical Support
  • Hero Member
  • *****
  • Posts: 14813
    • View Profile
Re: How to export depth map as matlab file without saving the image ?
« Reply #3 on: March 21, 2022, 05:49:37 PM »
Hello lyhour,

numpy.frombuffer(depth.tostring(), dtype=numpy.float32) line already returns a numpy array based on the depth maps values.
Best regards,
Alexey Pasumansky,
Agisoft LLC

lyhour

  • Newbie
  • *
  • Posts: 41
    • View Profile
Re: How to export depth map as matlab file without saving the image ?
« Reply #4 on: March 21, 2022, 05:55:24 PM »
I use this function already. When I use it, it return the string very strange and I save to other file, it cannot work. Do I miss or something wrong ? Also, I might confuse the function numpy.frombuffer return only one dimensional array not two dimensional as the image height and width. I want to confirm that this function generate two dimensional array and save the depth value to the other file. I am sorry about your inconveniences and disturb your valuable time.

Best Regards,
LYHOUR

Alexey Pasumansky

  • Agisoft Technical Support
  • Hero Member
  • *****
  • Posts: 14813
    • View Profile
Re: How to export depth map as matlab file without saving the image ?
« Reply #5 on: March 21, 2022, 06:24:52 PM »
Hello lyhour,

tostring() method is similar to tobytes() method available in the older numpy versions. The result is represented as a linear array (in the given case all the depth image lines go one by one).

If you would like to convert it into 2D array you should use reshape method:
numpy.frombuffer(depth.tostring(), dtype=numpy.float32).reshape(height, width)
where height and width are related to the depth image dimensions.
Best regards,
Alexey Pasumansky,
Agisoft LLC

lyhour

  • Newbie
  • *
  • Posts: 41
    • View Profile
Re: How to export depth map as matlab file without saving the image ?
« Reply #6 on: March 21, 2022, 07:11:26 PM »
Dear Alexey,
I really appreciate your kindness. I can solve it right now. Thank you very much. However, I face other problem when generate the depth map, the pixel of depth map and RGB image is not well corresponding. I will show you the figure in the attachment file. Noted that the figure I show you is cropped from the original for comparing the pixel location. However, when I evaluate the depth map from the the model.renderdepth, it generate well corresponding RGB pixel location and depth pixel location. But it take to long for computation time.

Best Regards,
LYHOUR

lyhour

  • Newbie
  • *
  • Posts: 41
    • View Profile
Re: How to export depth map as matlab file without saving the image ?
« Reply #7 on: March 22, 2022, 02:41:04 PM »
Dear Alexey,

the statement : numpy.frombuffer(depth.tostring(), dtype=numpy.float32).reshape(height, width) is working only for F32. If I want to work with 8bit or 16 bit, it doesn't work due to three channel of img.convert to RGB. So what should I do ? Thank you very much.

Best Regards,
LYHOUR

Alexey Pasumansky

  • Agisoft Technical Support
  • Hero Member
  • *****
  • Posts: 14813
    • View Profile
Re: How to export depth map as matlab file without saving the image ?
« Reply #8 on: March 22, 2022, 03:33:55 PM »
Hello lyhour,

You can use dtype=numpy.uint8 or dtype=numpy.uint16 instead of numpy.float32 in your code.

In case you need to form a three-channel output, modify the reshape command to:
Code: [Select]
.reshape(height, width, 3)
Best regards,
Alexey Pasumansky,
Agisoft LLC

lyhour

  • Newbie
  • *
  • Posts: 41
    • View Profile
Re: How to export depth map as matlab file without saving the image ?
« Reply #9 on: March 22, 2022, 05:02:25 PM »
Dear Alexey,

Thank you very much for solving this problem I really appreciate your kindness for helping. Anyway do you check my above comment related to the pixel RGB corresponding to depth pixel ? I want to fix this problem. It would be better if you could help me on this.

Best Regards,

LYHOUR

Paulo

  • Hero Member
  • *****
  • Posts: 1301
    • View Profile
Re: How to export depth map as matlab file without saving the image ?
« Reply #10 on: March 22, 2022, 09:14:01 PM »
Hello Lyhour,

for your question comparing pixel position of depth map with original RGB image, there are 2 things to consider:
  • the depth map will be downscaled according to quality. For medium quality depth map, the depth image will be downscaled by a factor of 4. So if original image has 4000 by 3000 pixel, then depth image will be 1000 by 750 pixel. So to compare with original image you would need to scale depth image by 4;
  • depth map image is created with no distorsion. So to compare correctly with RGB, you would need to undistort the RGB image first;
See example where I compare undistorted image with depth map after scaling it by 2 (high quality depth)

Hope this helps,
« Last Edit: March 23, 2022, 02:25:12 AM by Paulo »
Best Regards,
Paul Pelletier,
Surveyor

lyhour

  • Newbie
  • *
  • Posts: 41
    • View Profile
Re: How to export depth map as matlab file without saving the image ?
« Reply #11 on: March 23, 2022, 07:30:52 AM »
Dear Paulo,

Thank you very much for your response and your explanation. How can I get the undistort RGB image ?

Best Regards,

LYHOUR

Paulo

  • Hero Member
  • *****
  • Posts: 1301
    • View Profile
Re: How to export depth map as matlab file without saving the image ?
« Reply #12 on: March 23, 2022, 08:35:09 AM »
Helo LyHour,

you must use API Python. Given a camera in your chunk, then following code would create an undistorted image (und_image) and save it to export_path on disk:
Code: [Select]
export_path = r'C:\Users\paul.pelletier\Downloads\BenWAus\100_0006\undistorted'
chunk = Metashape.app.document.chunk
camera = chunk.cameras[0]
image = camera.photo.image()
und_image = image.undistort(camera.sensor.calibration,center_principal_point=False, square_pixels=True)
und_image.save(path=export_path+'/'+camera.label)

Hope you can do this,
Best Regards,
Paul Pelletier,
Surveyor

lyhour

  • Newbie
  • *
  • Posts: 41
    • View Profile
Re: How to export depth map as matlab file without saving the image ?
« Reply #13 on: March 23, 2022, 09:24:38 AM »
Dear Paulo,

I really appreciate your response. Thank you very much. However, I notice that from this line of this code, the center_principal_point should be True. I investigate that when the value is False, it is the same pixel position as the distort image. However, when I change it to true, the pixel of RGB and depth are corresponding nearly exactly (considering as same location) so It should be put the value to "True".
Code: [Select]
und_image = image.undistort(camera.sensor.calibration,center_principal_point=False, square_pixels=True)
« Last Edit: March 23, 2022, 09:56:14 AM by lyhour »

Paulo

  • Hero Member
  • *****
  • Posts: 1301
    • View Profile
Re: How to export depth map as matlab file without saving the image ?
« Reply #14 on: March 23, 2022, 09:48:14 AM »
Dear Lyhour,

great to hear this. In fact if center_principal_point is set to true then camera   undistorted image will have principal point set to 0 (cx = cy = 0) and this should correspond to chunk.depth_maps[camera].calibration.cx = chunk.depth_maps[camera].calibration.cx = 0 i.e. corresponding depth map will also have principal point at 0....

Have a good one,
Best Regards,
Paul Pelletier,
Surveyor