Agisoft Metashape

Agisoft Metashape => General => Topic started by: andysfd on January 21, 2021, 11:13:31 AM

Title: Import Laserscan data -registration lost
Post by: andysfd on January 21, 2021, 11:13:31 AM
Hello,

i tried the new import laserscan function by importing a E57 file containing 54 Laserscans, exportet from Leica Register 360 (RTC360 data)

I have a problem when finishing the import:

Warning: disabled 54/54 reference records with zero GPS coordinates

The registration is lost.
Title: Re: Import Laserscan data -registration lost
Post by: Alexey Pasumansky on January 21, 2021, 01:51:18 PM
Hello andysfd,

Do you see the imported XYZ coordinates for the cameras in the Reference pane? If they correspond to the actual measured coordinates, check them on in the pane and press Update button (make sure that the coordinate system is properly selected in the Reference pane settings dialog).
Title: Re: Import Laserscan data -registration lost
Post by: andysfd on January 21, 2021, 02:41:37 PM
Thanks, one step further...

i see the camera positions and corresponding coordinates + orientation for every scan

I toggle / activate both, the position and orientation check box, when i update i get an error:

"Not enough reference data" ...
Checked the correct coordinate system
Title: Re: Import Laserscan data -registration lost
Post by: Dieter on January 21, 2021, 08:41:48 PM
Same problem here.

I first have to allign the 360°-Pictures, then it works.

But should it work without allignment? The coordinates and orientations are known...
Title: Re: Import Laserscan data -registration lost
Post by: andysfd on January 22, 2021, 09:59:45 AM
Thanks for jumping on board Dieter, i am thinking the same!

The whole Import Laserscan function is useless when it can't utilize the known pre registration. The imported Laserscans are registerted with survey accuracy (sub <1cm) and i don't want to mess this registration up.

Any idea Alexey?

Title: Re: Import Laserscan data -registration lost
Post by: flyzk on January 22, 2021, 06:12:28 PM
Following !
I need the same feature: using the accuracy of laser scan to be use as reference for my picture alignement.
Title: Re: Import Laserscan data -registration lost
Post by: Alexey Pasumansky on January 22, 2021, 08:36:42 PM
Hello all,

It would be helpful, if anyone can send 3-5 overlapping laser scans in E57 format (with coordinate information) to support@agisoft.com, so that we can reproduce the same problem and see, how the laser scans support can be improved, when the coordinate data is available.
Title: Re: Import Laserscan data -registration lost
Post by: Dieter on January 23, 2021, 02:15:06 PM
Dear Metashape team, I am happy to provide my data.

I uploaded 5 point clouds, once as an aligned Metashape project and once as original E57 files.

As you can see, the point clouds are not scanned in color. In this case it would be nice if the intensity images were not only displayed in the image processing window, but also as an image preview. At the moment, the image shown as a preview can hardly be seen.

Scan 5 could not be aligned over the 360 ​​° images because it has too little overlap. However, it has the correct coordinates and the correct rotation from our scan calculation program.

It would be very desirable if Scan 5 could now be included in the mesh processing due to its known orientation.

It would be a great advantage for us if the scans were also included in further processing without alignment, as these are normally already available with their orientation and are condensed with photogrammetry or a texture is assigned to them.

Here is the link to download: https://www.dropbox.com/sh/hsdh7dto1x3u48u/AADPS0lCShNQxg2i0nvElWVYa?dl=0
Title: Re: Import Laserscan data -registration lost
Post by: jedfrechette on January 23, 2021, 07:07:00 PM
Thanks for putting together the example data set Dieter and I agree with the other posts that the laser scan support as currently implemented is not very useful.

I think the current approach of attempting to find and match key points in a panorama representation of a laser scan is also fundamentally limited. Compared to a photo, even high resolution scans are fairly low resolution making the task difficult. Furthermore, many scans are done without color so key points that are identified in IR reflectance may not correspond to ones in color images. If scans are captured with color, that means an additional calibration between the scanner's internal camera and it's measured 3D points, which may introduce an additional source of error.

I think it would be much better to follow an approach similar to what is described in the paper:

A Fast and Robust Framework for Semiautomatic and Automatic Registration of Photographs to 3D Geometry (https://dl.acm.org/doi/10.1145/2629514)

which describes an algorithm for using known 3D geometry to refine the estimated external and internal camera parameters. This would be a much more general solution and would eliminate many of the limitations of the current implementation.
Title: Re: Import Laserscan data -registration lost
Post by: Alexey Pasumansky on January 23, 2021, 09:53:17 PM
Dear Metashape team, I am happy to provide my data.

I uploaded 5 point clouds, once as an aligned Metashape project and once as original E57 files.

As you can see, the point clouds are not scanned in color. In this case it would be nice if the intensity images were not only displayed in the image processing window, but also as an image preview. At the moment, the image shown as a preview can hardly be seen.

Scan 5 could not be aligned over the 360 ​​° images because it has too little overlap. However, it has the correct coordinates and the correct rotation from our scan calculation program.

It would be very desirable if Scan 5 could now be included in the mesh processing due to its known orientation.

It would be a great advantage for us if the scans were also included in further processing without alignment, as these are normally already available with their orientation and are condensed with photogrammetry or a texture is assigned to them.

Here is the link to download: https://www.dropbox.com/sh/hsdh7dto1x3u48u/AADPS0lCShNQxg2i0nvElWVYa?dl=0

Hello Dieter,

Thank you for sharing the sample data.

Here is the script that applies exterior orientation to the laser scans (based on the information in the Reference pane), performs quite matching (in order to estimate the bounding box), preserving the original EO information.

Code: [Select]
import Metashape
chunk = Metashape.app.document.chunk
crs = chunk.crs
origin = chunk.cameras[0].reference.location
chunk.transform.matrix = Metashape.Matrix.Translation(origin)

for camera in chunk.cameras:
pos = camera.reference.location
m = crs.localframe(pos)
if camera.sensor.antenna.rotation_ref:
antenna_rot = camera.sensor.antenna.rotation_ref
else:
antenna_rot = Metashape.Vector([0,0,0])
rot = Metashape.utils.ypr2mat(camera.reference.rotation) * Metashape.utils.ypr2mat(antenna_rot).t() * Metashape.Matrix().Diag([1, -1, -1])
R = Metashape.Matrix().Translation(pos) * Metashape.Matrix().Rotation(m.rotation().t() * rot)
camera.transform = chunk.transform.matrix.inv() * R
camera.reference.location_enabled = True
camera.reference.rotation_enabled = True


chunk.matchPhotos(downscale=4)
chunk.triangulatePoints()
chunk.resetRegion()

After the script execution you should be able to run dense cloud or mesh generation operation based on the depth maps.

Before running the script, create a new chunk and add the laser scans (as TIFFs, if you already have them pre-saved). Then just start this script.
Title: Re: Import Laserscan data -registration lost
Post by: Dieter on January 24, 2021, 09:50:37 PM
I've done a few tests now, what can I say, it's going perfectly for me so far.

Many thanks for the very quick implementation of the function.

 ;D
Title: Re: Import Laserscan data -registration lost
Post by: Dieter on January 25, 2021, 10:00:14 AM
Okay, I spoke too soon.....

When combining a laser scan project chunk with a photo project chunk the error message "incompatible keypoints" appears, but the chunks are merged.

Is this a problem?
Title: Re: Import Laserscan data -registration lost
Post by: Dieter on January 25, 2021, 10:42:58 AM
I'm afraid that after merging the scans with the photos, the reference will be destroyed. It looks as if the scan chunk is too large by an incorrect scaling factor, and as a result the coordinates of the chunks no longer match.

Should I make my model available or can you understand this with your own data?
Title: Re: Import Laserscan data -registration lost
Post by: Alexey Pasumansky on January 25, 2021, 12:15:00 PM
Hello Dieter,

Yes, it would be helpful to share the project (in the state prior to the merging operation) and brief description that we should follow to reproduce the problem on our side.
Title: Re: Import Laserscan data -registration lost
Post by: Dieter on January 25, 2021, 03:40:51 PM
There are two chunks: scan and photo.

In the first chunk, the scans were read in and referenced with your script. The scans were then set as a reference with an accuracy of 1 mm. Then control points for the photo chunk were determined in the scans and output as a file.

In the second step, the photos were read in and aligned. Then the photos were referenced and balanced using the control points that were determined in the scans. The control points were set as reference with an accuracy of 5mm.

In the third step, I wanted to merge the two chunks, as they are created differently (photo and scan), but are based on the same coordinates. After that I would have the dense point cloud, mesh, etc. generated.

But that doesn't work that way. After merging the chunks, the references no longer fit and I get differences of up to a few meters.


Here is the link: https://www.dropbox.com/sh/braxt55qajlcipp/AAA1LWPdZfJgC70O21tS3oEAa?dl=0
Title: Re: Import Laserscan data -registration lost
Post by: andysfd on January 26, 2021, 05:57:36 PM
Thanks for the script,

Luckily i tested this prior the need for it in a project. ATM i can't test a mixed capture with photos but i will soon.
Thanks Dieter for taking the time cause i think we are trying to achieve the same.

In the end this has to work to be a serious competitor to RC regarding Meshing combined lidar / photogrammetry datasets. :)
Title: Re: Import Laserscan data -registration lost
Post by: Dieter on February 01, 2021, 12:05:32 PM
Hello Dieter,

Yes, it would be helpful to share the project (in the state prior to the merging operation) and brief description that we should follow to reproduce the problem on our side.

Hi Alexey,

were the problems understandable?
Title: Re: Import Laserscan data -registration lost
Post by: Alexey Pasumansky on February 01, 2021, 10:21:31 PM
Hello Dieter,

I can suggest the following workflow with the help of the updated script (attached):
- process the images from the digital camera only,
- reference the processing results in the same local coordinate system used for the laser scanner (chunk 1 from your project seems to be in this state),
- add the laser scans to the same chunk (you can use Add Photos and add pre-processed TIFFs),
- run the attached script - it should apply the EO information for the laser scans, so you do not need to match photos with the laser scans),
- run Build Dense Cloud or Build Mesh operation based on the depth maps.
Title: Re: Import Laserscan data -registration lost
Post by: Paulo on February 02, 2021, 04:08:22 PM
Hi Alexey,

the quick layout script seems to set the rotation of last laser scan in chunk to previous scan and thus false the estimeted EO of such laser scan.

In main module align_cameras, there is a call to estimate_rotation_matrices, that modifies rotation for cameras having None rotation, except last camera or laser scan as :

Quote
      for idx, c in enumerate(group_cameras[0:-1]):
         next_camera = group_cameras[idx + 1]

         if c.reference.rotation is None:
            if c.reference.location is None or next_camera.reference.location is None:
               continue

            prev_location = chunk.crs.unproject(c.reference.location)
            next_location = chunk.crs.unproject(next_camera.reference.location)

            direction = chunk.crs.localframe(prev_location).mulv(next_location - prev_location)

            yaw = math.degrees(math.atan2(direction.y, direction.x)) + 90
            if yaw < 0:
               yaw = yaw + 360

            c.reference.rotation = ps.Vector([yaw, 0, 0])

      group_cameras[-1].reference.rotation = group_cameras[-2].reference.rotation

Thus last laser scan will have estimated rotation set to previous laser scan rotation, thus falsing any subsequent data generation.

In attached screen, I ran quick layout on chunk, and you see that last laser scan has estimated yaw set to previous and thus falses its orientation as seen when a marker placed at corner of carpet is  not reprojected correctly on last laser scan.

Actually, I really do not see the use of estimate_rotation_matrices module in case of Laser Scans with precise EO.

Hope this makes sense,
Title: Re: Import Laserscan data -registration lost
Post by: Alexey Pasumansky on February 03, 2021, 09:31:34 PM
Hello Paul,

Thank you for pointing on the script issue.

Originally it allows to approximate the orientation for the NA cameras that do not have orientation angles input to the Reference pane. So there should be additional checks (whether reference.rotation is available) and apply this assignment only if reference.rotation is missing.
Title: Re: Import Laserscan data -registration lost
Post by: Paulo on February 04, 2021, 02:22:13 AM
Thanks Alexey for the update,

I see that PolarNick239 updated the script to reflect this https://github.com/agisoft-llc/metashape-scripts/commit/8955104c7a72582dbf8358019ece16d96cc2d74e
Title: Re: Import Laserscan data -registration lost
Post by: Dieter on March 21, 2021, 12:42:34 PM
Something must go wrong with the coordinate systems, I can't get the laser scans and cameras to cover.

My project consists of 5000 drone photos with PPK coordinates. In addition, 3000 photos were taken by hand, without coordinates. I imported the drone photos and the others, had them aligned and converted to UTM32. Then I created my own coordinate system, which only contains the last 4 coordinates of the UTM coordinates. Then 25 laser scans with the same coordinates (only the last 4 digits) were imported in the same coordinate system and the coordinates were set as calculated coordinates with your script.

Everything works perfectly, control points, which were determined in the photos (not reference Points), appear in the right place in the laser scans with gray symbols. But if I now determine a point in the laser scan, the coordinates deviate by 25 cm from the target coordinates. There are some differences in the coordinate systems that I cannot explain to myself.

If I unmark all photos as a reference and then press update, the coordinates are the same, when i Import a scan, run your script, den press update, the coordinates shift by the same amount and nothing matches any longer.

Even a convert in "Local Coordinate System" make no change.

UPDATE:
I did a test with a smaller project.
1) Import photos with PPK coordinates
2) Calculation with the coordinates as control points
3) The result is correct, the error is 1 cm.
4) Conversion of the GPS coordinates to utm32
5) Reading in the scans with coordinates UTM32
6) Start your script
7) Everything seems ok, all coordinates are correct, points that were determined in the photos correspond graphically to the laser scans on the screen.
8) When I determine a point coordinate from the laser scans, the coordinates are NOT correct. When I click on update, all coordinates shift and nothing is right anymore.

How can I solve the problem?
Title: Re: Import Laserscan data -registration lost
Post by: Alexey Pasumansky on March 21, 2021, 04:44:34 PM
Hello Dieter,

Can you share the project with the scene that you've got on Step 4 or 5 of the procedure that you have mentioned in PSZ format? If the project with multiple scans is too large, keep only a couple of scans, if the problem can still be reproduced.
Title: Re: Import Laserscan data -registration lost
Post by: Dieter on March 22, 2021, 10:02:42 AM
Thank you for the quick reply, here you can find the data:
https://www.dropbox.com/sh/ni0k5vmgb7f3a8v/AAAEsToauRq5LtK4tfcy4_Rha?dl=0

-Metashape Project
-Fotos as JPG-File
-Scan as Tif-File
-coordinates Fotos as TXT-File

The coordinates of the Scanner are 9014.849 8336.246 491.459 and they are correct calculated.

The Project is in status after Point 4 of my list. The coordinates has been converted to my own coordinate-System ETRS89 / UTM zone 32N Meistersehl and match to the scanner-coordinate-System.

Please import the tif-file of the Scanner and run your script, then push the update Button, all coordinates are wrong now.


Point 1 you can find in the fotos and in the scan-file.

Thanks
Dieter
Title: Re: Import Laserscan data -registration lost
Post by: Paulo on March 22, 2021, 03:10:19 PM
Hi Dieter,

Thanks for sharing your data set. I was able to load your laser scan tif into chunk 1 and set the EO using quick layout script. Note that estimated yaw is a little different from reference because of grid convergence.Now projection of  Point 1 in Laser Scan  falls close to target but not at center. And when I generated a dense cloud,  points from drone agree quite well with those from Scan (dark brown)... See first attachment.

I then created another chunk (Chunk 2) loading both drone images and laser scan and setting correct CRS ETRS89 / UTM32 Meistersehl + EGM2008 and did a new alignment for these 8 images. I got an estimated EO for laser scan very close to one defined  in Chunk 1. However now, there is no offset in Point 1 projection on scan and dense cloud fits perfectley. see 2nd attachment.

Please note that I shifted all Z coordinates by -15 m to better reflect actual MSL (egm2008) heights as original values (in the 490 m range ) neither corresponds to elispsoidal (520 m) or MSL (470 m).

The corresponding psz file with both chunks can be downloaded from https://drive.google.com/file/d/1bdB6amNmOf2lfJ6TmXiWQtWOGHn0tyIf/view?usp=sharing

Maybe this lead to some more insight....
Title: Re: Import Laserscan data -registration lost
Post by: Dieter on March 22, 2021, 05:28:38 PM
Paulo wrote:

"I was able to load your laser scan tif into chunk 1 and set the EO using quick layout script. Note that estimated yaw is a little different from reference because of grid convergence.Now projection of  Point 1 in Laser Scan  falls close to target but not at center."

Please move reference point 1 in the scan to the mark and watch the deviation, I have 27 pixels. This is too much.
You can also set a new point on the target mark in the scan, and then compare the coordinates with those of the target mark from the photos. The deviation is approx. 55 cm in the high value, 42 cm in the legal value and 20 cm in height.
In contrast to the photos, the scan is shifted, although this should fit.

What i've done:
-read the Original Project
-input laserscan
-run your script
-move Marker 1 to Target in the scan, this should fit with the Foto orientation!
-watch the derivation-> 27 Pixel

or:
-read the Original Project
-input laserscan
-run your script
-Set a new Marker Point 1 to the target in the Laserscan
-watch the derivation between Marker 1 (Fotos) and Point 1 (scan)
-you will see the difference

Title: Re: Import Laserscan data -registration lost
Post by: Paulo on March 22, 2021, 06:20:40 PM
Dieter,

I looked at distance between projected Point 1 and center of target, I get something like 10 pixels or 3.54 cm in world distance.
After, you run the script what are the estimated  position and orientation coordinates for the laser scan?

Mine are: W11_050_0.tif    X=9012.398129685549   Y=8334.558593555354   Z=475.528094960229   Yaw=10.4121035277475   Pitch=0   Roll=-8.746525374024672e-15 or 0   

A small error in rotation components can easily bring an error of a few cm, especially for points farther fron Scan perspective center.
Title: Re: Import Laserscan data -registration lost
Post by: Dieter on March 22, 2021, 07:03:57 PM
Take a look at my coordinates from the ruler, i don't understand this.

Why do you changed my coordinate System?
Title: Re: Import Laserscan data -registration lost
Post by: Paulo on March 22, 2021, 07:43:00 PM
Dieter,

your estimated coordinates and orientation for Scan are same as mine except for Z value which I just changed by substracting 15 m to all camera centers so to have the model correspond better to real world elevation as seen in attached screen copy. With original values model was floating some 15 m in the air. So this is just a shift of -15 m in all heights with definition of CRS inluding egm2008 heights.

But this does not explain why you get a different error on Point 1 relative to target center .....?? :-\ Though are much closer to mine wrt original report of 27 pixels and 55 cm max....
Title: Re: Import Laserscan data -registration lost
Post by: Dieter on March 22, 2021, 08:22:54 PM
I have completely different ruler coordinates than you, that's why my big deviations.

Have you tried that in my original project? So without changing the coordinate system or recalculating.
How are your deviations there?

Thank you for your help....

Title: Re: Import Laserscan data -registration lost
Post by: Paulo on March 22, 2021, 09:07:57 PM
Yes Dieter,

I redid the procedure without changeing any Z or other:

1. Add Laser scan into Chunk 1 with Add Photos from Workspace pane,
2. Run Quick Layout script to set estimated EO to Reference EO,
3. Measure distance between projected point 1 and center of target on scan and it is practically the same as previously 3.4 cm. Note that first coordinates of distance should correspond to Point 1 which they do. In your case, they are different.

The reason I changed your Z coordinates is that in ellipsoidal height, your Z values are about 30 m lower than actual elipsoidal height for hilltop  where Castle ruin is. See attached...Are you certain that the PPK Z coordinates of your cameras are correct?
Title: Re: Import Laserscan data -registration lost
Post by: Dieter on March 23, 2021, 12:48:38 AM
What Version of Metashape you are using? Tomorrow i will try a reinstall of the Software.

My heights are German DHDN2016 heights, not ellipsod heights, they are 48 meters lower then the elipsoid heights.  But thats not the problem, really i dont understand what the problem is ?! Why is it running correct on your Side?

Thanks
Dieter
Title: Re: Import Laserscan data -registration lost
Post by: Paulo on March 23, 2021, 02:50:14 AM
Dieter,

I think I found the reason we get different results on distance .... I am using a prior verson of   quicklayout script which uses a boolean variable has_laser_scans to test if we are dealing with laser scans. If we do then at the end it does a triangulatePoints which seems to be the trick as latest version of quicklayout does not do this  and I tried the latest script on same chunk and then measured the distance and got 1.21 m:
Code: [Select]
2021-03-22 17:33:36 Ruler coordinates:
2021-03-22 17:33:36 9014.835938 X  8336.277344 Y  491.450 m
2021-03-22 17:33:41 9015.798828 X  8336.902344 Y  491.819 m
2021-03-22 17:39:55 Measured length: 1.21 m

and then I did a triangulatePoints on the chunk. This updates the point cloud and actually adds a few points ( 22 327 Tie Points instead of original 21 610) and then same distance is measured as 3.4 cm as I previously obtained
Code: [Select]
chunk.triangulatePoints()
2021-03-22 17:34:12 TriangulatePoints
2021-03-22 17:34:12 Updating point cloud...
2021-03-22 17:34:12 adding 23018 points, 0 far (10 threshold), 680 inaccurate, 11 invisible, 0 weak
2021-03-22 17:34:12 coordinates applied in 0 sec
2021-03-22 17:34:12 Finished processing in 0.108 sec (exit code 1)

2021-03-22 17:40:46 Ruler coordinates:
2021-03-22 17:40:46 9014.834961 X  8336.277344 Y  491.450 m
2021-03-22 17:40:49 9014.852539 X  8336.250000 Y  491.459 m
2021-03-22 17:42:10 Measured length: 3.40 cm

So I hope Alexey can explain which version should be used. I attach my version which includes the triangulatePoints  part so that you can try on your side....

PS am using  Agisoft Metashape Professional Version: 1.7.2 build 12070 (64 bit)
Title: Re: Import Laserscan data -registration lost
Post by: Dieter on March 23, 2021, 10:07:34 AM
Yes, that is the solution. I did NOT use this script because I also scanned inside the castle, so I don't have any tie-points with the photos.

I have now found the following solution for the moment: I take a scan that has enough tie points with photos and align it with the photos. Then I calculate an update of the project with fixed GPS coordinates AND fixed scan coordinates of this individual scan.

Then I can read in the remaining scans and all the coordinate transformations match up to the cm.

It would be nice if the developers offered a simpler solution here in the future. It is also desirable if these external scripts were built into the fixed code of Metashape, then all users would have the same software version and would prevent any irritation.

Thank you for your help, Paulo


Dieter
Title: Re: Import Laserscan data -registration lost
Post by: Alexey Pasumansky on March 23, 2021, 12:48:55 PM
Hello Dieter,

The projections of the marker placed on the laser scan also considers depth information, so the raise of the error that you are observing for the marker in the project after adding the projection on the laser scan is related to the depth difference between the laser scan data and 3D marker location based on its projections on photos.
Title: Re: Import Laserscan data -registration lost
Post by: Paulo on March 23, 2021, 01:23:48 PM
Hi Alexey,

thank you for the explanation, but it does not explain why after triangulatePoints, same distance measured in scan from the projection of Marker 1 to center of target goes from 1.21 m to 3.4 cm!

The start point of measurement in both cases corresponds to coordinates of Marker 1. But it is the end point of measuremen (center of target on Scan) which varies greatly. So why does this happen?

So it is not very clear what does triangulatePoints do to fix this distance measurement.....

Title: Re: Import Laserscan data -registration lost
Post by: Paulo on March 23, 2021, 02:18:50 PM
Hi all,

I think I found why distance is so different between Point 1 and center of target on Scan. I ran the procedure again using the latest quick layout (without triangulatePoints). And I again measure distance and get wrong value of 1.21 m. The reason is that Scan at target center has no Depth Value. If I measure a distance a little farther from target center where there is depth, I get correct distance of 3.57 cm. See attachment...

The question is now: Why after running triangulatePoints, the absense of depth does not affect the distance measurement?

More to follow, I hope...
Title: Re: Import Laserscan data -registration lost
Post by: Dieter on March 23, 2021, 02:52:19 PM
I don't think that's the reason.

When I import laser scans, I have to align at least one scan with markings to the photos, otherwise the systems are not correct and I get incorrect coordinates. I've tried hundreds of ways now and that's the only way. I can't say why that is either, but it is.

That is why the old script also brings the right results, because the scans are aligned with the photos there. The new script does not create a connection between the photos and the scans. And then it just doesn't work.

Although the coordinates match, there is in some way no connection between the photos and the scans.
Title: Re: Import Laserscan data -registration lost
Post by: Alexey Pasumansky on March 23, 2021, 04:08:18 PM
If you match photos with laser scans and align cameras or triangulate points, then the tie points will appear which connect photo alignment and TLS.

If you align photos first and then load TLS and use the script to apply the exterior orientation information - these source will be isolated one from another. You can actually reset alignment for all your cameras, add TLS and use the script to define the camera EO based on the information from the Reference pane - you'll see that even in this case the 3D location of the marker based on the projections on photos is about 70 cm away from the 3D point defined on the spherical panorama (based on the TLS depth data).

The proposed procedure, described in the related tutorial assumes that image matching is performed and scans are aligned with each other and with the common digital photos:
https://agisoft.freshdesk.com/support/solutions/articles/31000159101-terrestrial-laser-scanning-data-processing

The script (quick_layout) can be used, if you would like to skip the image matching and alignment procedure and apply EO to the cameras or TLS data based on the coordinates and rotation angles from the Reference pane. The latest version of the script on github is not running image matching or alignment (triangulation), as it is a multipurpose script that also allows to apply the camera transformation for the not aligned images taken above the water bodies, for example, for the orthomosaic generation without any holes.
Title: Re: Import Laserscan data -registration lost
Post by: Dieter on March 23, 2021, 04:36:22 PM
Please follow the steps from this manual (freshdesk) with my Original Project.

"In case the laser scanning locations are considered as precise and you would like to avoid image matching procedure between the TLS data and digital images, it is possible to apply the exterior orientation data from the Reference pane to the loaded laser scanning data using the following script:

https://github.com/agisoft-llc/metashape-scripts/blob/master/src/quick_layout.py

If the project contains also the common digital images to be aligned, it is recommended to align such image set first and then load the TLS data to the same chunk and run the mentioned script. Note that the digital images alignment should be referenced in the same coordinate space as laser scanning data in order to be properly processed together."


You will see, that it is NOT working. I've tried this several times, there is no chance to get this working. After you import the scandata (same coordiate System as fotos), run the script and then build mesh, you will see holes and fracments in the mesh. The coordinates are NOT matching. Paulo tried this too, there is no exact matching between the fotos and the scans.

Title: Re: Import Laserscan data -registration lost
Post by: jedfrechette on March 23, 2021, 05:35:11 PM
It would be nice if the developers offered a simpler solution here in the future.

I can only second this. After experimenting with the proposed workflow and following threads like this the entire process seems a little half baked, and definitely not something I would consider production ready. I hope the developers reconsider their approach to this problem and can consult with users who regularly combine photogrammetry with laser scan data so that they can better understand our use cases.
Title: Re: Import Laserscan data -registration lost
Post by: Paulo on March 23, 2021, 05:55:11 PM
Hello Dieter,

i did a test generating a dense cloud with Laser scan added to chunk and referenced with quick layout (without triangulating points). Then I triangulated points and redid a dense cloud. After exporting the 2 clouds and comparing  them in Global Mapper, I can say that there is no visible difference between the 2. And comparing same profile along vertical wall of ruin, there is same offset (few cm) between points generated from Laser depth (dark brown) and drone photos (greyish). See attachment...

AS far as I`m concerned there is no sensible difference between 2 clouds generated from 2 methods (with or without trianguatePoints). The dense cloud and hence mesh can be much tighter or better doing a combined alignment of Scan with photos (more time consuming)...

There remains still the mystery of distance measurement in scan from point 1 to center of target (nodepth)....

Hope this can contribute to better understabding,
Title: Re: Import Laserscan data -registration lost
Post by: Dieter on March 23, 2021, 06:17:07 PM
I am at a loss.

I downloaded the original project again and referenced the photos. Then I loaded the scan and then ran the script.

Look for the coordinates of the photos and scan, there are no deviations or almost zero.

After that I calculated the mesh and the result is shown below. There are two completely separate walls to the hole in the castle.

I'm afraid we won't follow the same steps.
Title: Re: Import Laserscan data -registration lost
Post by: Alexey Pasumansky on March 23, 2021, 06:40:26 PM
Upon further digging into the project the source of the problem seems to be discovered:

to be properly applied, the laser scanner depth data requires the chunk.transform.scale be 1.0. It happens automatically, if the TLS data is included to the alignment procedure. In this project, since the TLS data is added to pre-aligned project, chunk.transform.scale is not 1 and the depth data from the scanner is not properly treated.

So the workaround for your project could be the following:
- perform Align Photos operation to all available data: digital photos + TLS data,
- reset camera alignment for TLS scans,
- run quick_layout script,
then build mesh from the depth maps.

If the digital photos are already aligned, but you have key points in the project, you can add TLS data and run Align Photos procedure with "reset current alignment" option unchecked. Then reset alignment for laser scans and use the script.

I'll check, if we could update the script, to modify the chunk.transform.martix, if there are any laser scans in the project, but it requires not only to change the matrix itself, but also to re-calculate the coordinates of all the cameras, tie points, mesh vertices and etc.
Title: Re: Import Laserscan data -registration lost
Post by: Dieter on March 23, 2021, 06:58:45 PM
Well, that was just easy again ;)

 ::)
Title: Re: Import Laserscan data -registration lost
Post by: Paulo on March 23, 2021, 07:05:18 PM
Dieter,

I am really confused here! I did the same as you, import scan, run quick layout (without triangulating) and generated a mesh with medium quality. I do not get gap in wall going down in hole. Scratch my head  :o..

Post Script: reading Alexey`s post, now I understand why I get good  results compared to you, Dieter. My edited quick_layout script just commented out the chunk.triangulatePoints() line but still has a section where it checks if sensor has depth. If True then it calls normalize_chunk_scale which sets transform scale to 1 as pointed out by Alexey.

Eureka! with determination we have moved the learning clock along!

Thanks all for your patience.
Title: Re: Import Laserscan data -registration lost
Post by: Paulo on March 24, 2021, 01:28:30 PM
Hi Alexey,

me again, trying to get a better understanding of different parts of this issue and enjoying it! So I ran again  my edited quick layout on project (just with triangulatePoints line commented out). And as you said the normalize chunk scale fixes the scale for Laser Scan Depth but shifts the whole point_cloud (see left side of attachment with Marker 1 shifted from wall). But after running triangulatePoints(), it does reset the point_cloud to its correct position (see right side).

So a quick layout including normalize_chunk_scale and chunk.triangulatePoints()  does not affect the point_cloud and thus no need to recalculate the coordinates of tie points...

Am I right in this ?
Title: Re: Import Laserscan data -registration lost
Post by: Alexey Pasumansky on March 25, 2021, 06:45:05 PM
Hello Paulo,

Please check, if the attached version of quick-layout script works in any case.

It includes the normalization of tie point coordinates and bounding box location.
Title: Re: Import Laserscan data -registration lost
Post by: Paulo on March 25, 2021, 11:25:49 PM
Hi Alexey,

Thanks for this update. I see that this script does now scale the point cloud within normalize_chunk_scale without having to use triangulatePoints as in my modified script. And it does normaize the Bounding Box but the modified script did a chunk resetRegion after triangulation which is equivalent. So result is the same for both scripts. Any advantage, in using  one or the other?

Also, please note that this script in estimate_rotation_matrices sets the reference orientation of last camera to previous. So this falses the orientation of the loaded Laser Scan. See screen copy.

This problem was fixed for quick layout in github https://github.com/agisoft-llc/metashape-scripts/commit/8955104c7a72582dbf8358019ece16d96cc2d74e by changing:
Code: [Select]
group_cameras[-1].reference.rotation = group_cameras[-2].reference.rotation
with:
Code: [Select]
if group_cameras[-1].reference.rotation is None and group_cameras[-1].reference.location is not None:
group_cameras[-1].reference.rotation = group_cameras[-2].reference.rotation
Title: Re: Import Laserscan data -registration lost
Post by: Dieter on March 28, 2021, 05:20:31 PM
I've lost track of the scripts a little now.

Is there now a final script to download in which the error Paulo mentioned in the last post does not appear?

Dieter
Title: Re: Import Laserscan data -registration lost
Post by: Paulo on March 28, 2021, 05:30:55 PM
I've lost track of the scripts a little now.

Is there now a final script to download in which the error Paulo mentioned in the last post does not appear?

Dieter
Dieter,
as far as i am concerned the following script quick_layout_normalize_tls modified to correct the last camera rotation error does the same as quick_layout script I already posted and reposting here.
One uses scaling of point cloud to correct for original chunk scale not being 1 and the other triangulatePoints to fix same issue.

So you could use any of the 2 with same result, I think. Maybe, Alexey can confirm.

Title: Re: Import Laserscan data -registration lost
Post by: Dieter on April 06, 2021, 06:41:05 PM

Unfortunately I have to find out that none of the scripts work properly. I have now tested "quick_layout_normalize_tls.py" from Alexey and the two scripts from Paulo. In doing so, I have already read coordinate-oriented scans into an existing project with photos. Then I tried all the scripts with the following result:

The rotation of the scans is incorrect. If I tap existing points from the photos in the scans, I have deviations of several meters.

The problem must be manageable somehow.

Unfortunately, I don't have enough matches between scans and photos to really align the scans with them.

I urgently ask for help here, otherwise my project will be invalid and I will have to go to the field service again to complete the scans.

Of course I want to prevent that.

Dieter
Title: Re: Import Laserscan data -registration lost
Post by: Paulo on April 06, 2021, 08:07:44 PM
Hello Dieter,

this is confusing. For the latest TLS you have tested could you confirm that for imported TLS in MS then that its camera.sensor.mata has 'has_depth' attribute equal to True and camera.sensor.antenna.rotation is ((90,0,-90)) as in following screen capture.

Or better share the tif of latest TLS....
Title: Re: Import Laserscan data -registration lost
Post by: Dieter on April 07, 2021, 09:47:53 AM
Hello Paul,

how can I see your displayed values ​​on the screen? What do I have to enter / set?


I have now found out the following:

Project 1: Georeferencing was done via GCP. The scans match the coordinates.

Project 2:
Georeferencing was carried out using drone flights and GPS. The scans match the coordinates.

Merge project 1 and 2. The images and coordinates match, all control points from GCP and image coordinates match. The scans do NOT match the coordinates and are twisted.

Dieter
Title: Re: Import Laserscan data -registration lost
Post by: Paulo on April 07, 2021, 10:55:43 AM
Hi Dieter,

you could run the attached script in each project to print out for each TLS:

- its name;
- X Y Z Yaw Pitch Roll from E57 pose;
- sensor metadata (has depth and has intensity attributes);
- sensor antenna rotation(should be 90.000 0.000 -90.000)

See example capture screen

I would recommend that for each project, once you import the TLSs, only run one of the scripts I sent u, not the one Alexey provided as this has a bug that falses the orientation of last TLS...
Title: Re: Import Laserscan data -registration lost
Post by: Dieter on April 07, 2021, 10:57:47 AM
I found my mistake  >:(

In one of the projects, the reference settings were set instead of Yaw, Nick, Roll -> Alpha, Nu, Kappa before merging.

I have no idea how that got there.

What the heck, thank you very much for your help, now it seems to be working.

Dieter
Title: Re: Import Laserscan data -registration lost
Post by: Stef on October 17, 2021, 02:22:04 PM
Just want to say thank you for all these testing, I am actually trying to fit a registered laser scan cloud inside Metashape, but all this seems to go a little too far. I've been reading through all the posts, I've been lost a couple times, and my conclusion is: Metashape is treating laser scans as photogrammetry and that's not good at all, since it's a whole different thing based on direct measurements and not triangulations.

For the times to come, until Metashape is ready (because I'm sure it will be), I'll keep doing the way I do it right now: E57 file from Drone and E57 file from Scanner, both in CloudCompare for the alignment.

Thank you to Dieter and Paulo for the in-depth tests and to Alexey for the support.
Title: Re: Import Laserscan data -registration lost
Post by: bassistas on January 04, 2022, 08:40:32 AM
Hi guys, I have 39 registered scans exported from Faro scene in e57 format , 2300 photos from a phantom 4 pro and about 5600 photos from a handheld camera. I want to combine all these together and use the laser scans as a reference for the scale. The final outcome will be a textured 3D mesh and some orthophotos for architectural drawings in Autocad. So what is the best approach for this task? Do I import first the laser scans in one chunk and then I apply one of the scripts uploaded here? Then I create a second chunk with all the photos, align them and then merge two chunks for the creation of the final model?
Can anyone write the steps in summary? I'm a bit lost and the deadline is approaching  :-[
thanks
Title: Re: Import Laserscan data -registration lost
Post by: Dieter on January 04, 2022, 02:17:25 PM
I assume that you have no orientation for the photos?

Then I would import the laser scans, then run the script for orientation over them in order to define the laser scans as fixed points. Then import the photos and allign them with the laser scans. With a little (a lot of) luck you will have identical points between laser scans and photos. You then have to decide whether that is accurate enough and sufficient. If that doesn't work, you only have the option of looking for identical points in the laser scans and in the photos and using them as GCP for photo matching.

Good luck with it.
Dieter
Title: Re: Import Laserscan data -registration lost
Post by: bassistas on January 04, 2022, 11:05:51 PM
I assume that you have no orientation for the photos?

Then I would import the laser scans, then run the script for orientation over them in order to define the laser scans as fixed points. Then import the photos and allign them with the laser scans. With a little (a lot of) luck you will have identical points between laser scans and photos. You then have to decide whether that is accurate enough and sufficient. If that doesn't work, you only have the option of looking for identical points in the laser scans and in the photos and using them as GCP for photo matching.

Good luck with it.
Dieter

I just tried that, Imported the laser scans, I run the script and it shows the scans as aligned. Then I import the photos and I try to run the alignment again with the reset current alignment option unticked but I get this error in console: 

Matching photos...
Warning: Can't resume matching without keypoints
Title: Re: Import Laserscan data -registration lost
Post by: Paulo on January 05, 2022, 09:37:03 AM
Hello Bassistas,

running the quick layout scrit will assign Estimated EO from the source values but not generate tie points. So to generate the tie points you should use Tools > Tie Points > Build Poind Cloud command (assuring that you set Keep Key Points option in Preferences settings). Then you can import photos and do alignment in incremental fashion. However, if you wnt to keep the Laser scan registration, give them a very high accuracy in m and degrees....

The other alternative is to measure a few well identifiables features in the laser scans, and use these feature points as GCPs to align the images in separate chunk as specified by Dieter...then merge chunks...

Hope this helpful,
Title: Re: Import Laserscan data -registration lost
Post by: bassistas on January 05, 2022, 10:44:09 AM
Hello Bassistas,

running the quick layout scrit will assign Estimated EO from the source values but not generate tie points. So to generate the tie points you should use Tools > Tie Points > Build Poind Cloud command (assuring that you set Keep Key Points option in Preferences settings). Then you can import photos and do alignment in incremental fashion. However, if you wnt to keep the Laser scan registration, give them a very high accuracy in m and degrees....

The other alternative is to measure a few well identifiables features in the laser scans, and use these feature points as GCPs to align the images in separate chunk as specified by Dieter...then merge chunks...

Hope this helpful,

Thank you, I'll give it a try, but what do you mean by "give them a very high accuracy in m and degrees"? Is there any setting for that?
I also tried to follow this tutorial I found here https://agisoft.freshdesk.com/support/solutions/articles/31000159101-terrestrial-laser-scanning-data-processing (https://agisoft.freshdesk.com/support/solutions/articles/31000159101-terrestrial-laser-scanning-data-processing) which at the bottom it says:
"If the project contains also the common digital images to be aligned, it is recommended to align such image set first and then load the TLS data to the same chunk and run the mentioned script. Note that the digital images alignment should be referenced in the same coordinate space as laser scanning data in order to be properly processed together."
But how this can work if you try to import registered scans on top of already aligned photos that are probably out of scale already? In my case a drone like the standard phantom 4 pro without RTK



Title: Re: Import Laserscan data -registration lost
Post by: Dieter on January 05, 2022, 11:40:06 AM
Wait a minute, if I do set the laser scans as fixed points and then do the allignment via laser scans and photos, the tie points must be generated in the scans and in the photos, or am I completely wrong? The same function also works with photos if I have RTK coordinates, what's the difference?

As I have already noticed several times here, I find it totally unfortunate to do an essential function of the program (setting the laser scans as fixed points) with an external script. I never know exactly whether the script is up-to-date or whether it still fits the new version. This function must be integrated into the main Metashape program.
Title: Re: Import Laserscan data -registration lost
Post by: Paulo on January 05, 2022, 12:08:20 PM
Yes Dieter,

the quick layout will only set the aligned  TLSs position and orientation to values from source. To get tie Points you must run Tools > Tie Points > Build Point Coud... (having previuosly set Keep Key Points option). And then you can add orther images and do an incremental alignment.... However the matching between the intensity based TLSs and optical images may not generate many points ...

So other options would be used GCPs measured from the TLS chunk in the added optical image chunk to aligne and then merge with TLS..

Eventually, I guess we could add the matchPhotos() + triangulatePoints() prcess inside a quick layout script to generate the tie points ...

Title: Re: Import Laserscan data -registration lost
Post by: Dieter on January 05, 2022, 04:18:29 PM
@Paulo

I've tested that again now. For me, the tie points are also searched for in the laser scans and compared with those of the photos. Of course you have to select all elements (photos and scans) for allignment. To be honest, I haven't even known the menu item you mentioned until today.

Dieter
Title: Re: Import Laserscan data -registration lost
Post by: Paulo on January 06, 2022, 11:11:20 AM
Hi Dieter,

what exactly were u testing? The quick layout script or trying to align all scans with images? Because it it is just a common alignment then of course you can just do a normal alignment with both TLSs and images.To keep scans original  reference fixed, set a very high accuracy on scans position and orientation...
Title: Re: Import Laserscan data -registration lost
Post by: Dieter on January 06, 2022, 11:30:50 AM
My approach:

1) Import scans
2) Quick layout script
3) Import photos
4) mark only the scans as reference with an accuracy of 0.001 meters
5) allign scans and photos

Tie points are determined in the scans and the photos and compared with each other. The coordinates of the scans are held as a reference system.

Dieter
Title: Re: Import Laserscan data -registration lost
Post by: Paulo on January 06, 2022, 12:20:13 PM
OK, I see,

but IMHO if you use the scans with reference marked with high accuracy then I do not see the utility of quick layout script....
Title: Re: Import Laserscan data -registration lost
Post by: James on January 06, 2022, 12:45:17 PM
but IMHO if you use the scans with reference marked with high accuracy then I do not see the utility of quick layout script....

Some TLS scans may not get aligned at all if they do not get sufficient tie points to other scans and images.

I guess the script ensures that these 'orphan' scans still end up in the result and in the right place.

Unfortunately this is not often enough part of my workflow for me to know it inside out yet, but it does seem that given that the discussion is ongoing it would benefit from some clarity from above. Part of the reason I don't do it more often is that I'm not confident of what the outcome will be.

Making the selection of markers for manually tying the photos to the scans part of the official workflow with proper tools would be huge. I use Cyclone for TLS and take a lot of time looking at the data in 3D to choose my reference points as accurately as possible. I'm almost always working with buildings, so I'm looking at windows and cornices etc, to get my points correct to within a couple of mm, which is not possible currently in Metashape just by putting a marker on a depth map in '2D'.
Title: Re: Import Laserscan data -registration lost
Post by: jedfrechette on January 06, 2022, 08:02:49 PM
I use Cyclone for TLS and take a lot of time looking at the data in 3D to choose my reference points as accurately as possible. I'm almost always working with buildings, so I'm looking at windows and cornices etc, to get my points correct to within a couple of mm, which is not possible currently in Metashape just by putting a marker on a depth map in '2D'.

I think this is the key observation that the Agisoft developers need to make if they want to make laser scan support actually useful in their product. Those of us coming from the TLS world have very good tools to make sure that our laser scans are very well aligned relative to each other as well as independent reference points.  In most cases those registrations are also much better constrained, and therefore more reliable, than pure photogrammetry solutions. As a result, just treating laser scans as low resolution panoramas and throwing them in to your bundle adjustment largely ignores their benefits and is never going to be a satisfactory solution.

Currently, the only practical method I've found for combining laser scans and photogrammetry with Metashape is the traditional approach of manually picking ground control points. However, that method is also time consuming, error prone, and requires jumping back and forth between multiple pieces of software. We've worked on a few Python tools to make it slightly less painful but the process is by no means as efficient as it should be.
Title: Re: Import Laserscan data -registration lost
Post by: Dieter on January 06, 2022, 08:18:31 PM
but IMHO if you use the scans with reference marked with high accuracy then I do not see the utility of quick layout script....
Some TLS scans may not get aligned at all if they do not get sufficient tie points to other scans and images.

That is exact the point!

I use Cyclone for TLS and take a lot of time looking at the data in 3D to choose my reference points as accurately as possible. I'm almost always working with buildings, so I'm looking at windows and cornices etc, to get my points correct to within a couple of mm, which is not possible currently in Metashape just by putting a marker on a depth map in '2D'.
Currently, the only practical method I've found for combining laser scans and photogrammetry with Metashape is the traditional approach of manually picking ground control points. However, that method is also time consuming, error prone, and requires jumping back and forth between multiple pieces of software. We've worked on a few Python tools to make it slightly less painful but the process is by no means as efficient as it should be.

If you take the scans and photos of the same objects (e.g. walls) and do the scan in color and choose the right exposure for the color scans, and also about the same resolution between photos and color scans, the photos are linked to the scans. It takes some practice and experience, but it works.

We often work like this, the base point cloud comes from the scans and inaccessible areas are condensed by photos.
Title: Re: Import Laserscan data -registration lost
Post by: bassistas on February 22, 2022, 09:50:19 AM
My approach:

1) Import scans
2) Quick layout script
3) Import photos
4) mark only the scans as reference with an accuracy of 0.001 meters
5) allign scans and photos

Tie points are determined in the scans and the photos and compared with each other. The coordinates of the scans are held as a reference system.

Dieter

I had some success with this workflow but in the case I already have a project with imported laser scans and photos with markers attached, if I disable all the photos and leave just the laser scans enabled after the execution of the quick layout script , I go to Tools-> Tie points -> build point cloud and it messes up the position of the scans for some reason. It brings all the scans to the same center like a ball.
If I create a new project and add just the scans first then this is working as it should but how can I make it work on an existing project in which I have more than 150 markers already placed on photos and laser scans and is a pain to place all of them from the begining?