Datum Transformation / Coordinate Shift

[BenW22]

Hi All,
I have searched this forum for an answer but could find a solution to my problem and hope someone could help. I think I'm just not experience enough in Metashape and am missing something very obvious.

I just want to do a co-ordinate shift on my images to align with my GCP's. But I was hoping to do this via the Datum Transformation found in - Reference Settings dialogue box / Camera Reference  / Settings Icon / Custom Shift  - X Y Z - Please see the explanation below

I'm using Metashape Pro 1.8.1 build 13915.

I have carried out a DJI P4 RTK flight all on nadar with the final deliverable being a ortho image. The DJI was connected to VRSNOW (similar to CORSNET)  and recorded standard EXIF data during the flight (including WGS 84 lat long and  RTK precisions). I also carried out a RTK GPS survey (using Trimble R10) to survey about 7 GCP's. The GCP survey was calibrated in on a State Survey Mark with published E N RL values (Australian GDA2020 Zone 56 co-ordinate system / Aus Geoid 2020).  The E N RL (XYZ) units are in metres.
.
I have -
1. imported the images  (with "load cameras location accuracy from xmp meta data" ticked in preferences/advanced tab). So rtk accuracies have been imported
2. Run the "Convert Reference" from the Reference Pane - converting images from WGS84 to a compound co-ordinate system  - that being GDA 2020 / MGA zone56 + Aus Geoid 2020.  (The geiod was downloaded from the Agisoft Geoid webpage) . And changed Camera and Marker reference co-ordinate systems to be GDA 2020 / MGA zone56 + Aus Geoid 2020
3. Run the "estimate image quality"  - there are no images under 0.5 so none deleted
4. Aligned Images -Settings  -  Medium / Source /  40,000 and 4000. Generic preselection ticked , reference preselection ticked
5. Imported GCP and moved markers to match where they can be seen in the images.
6. Optimize Cameras was run with "fit additional corrections" ticked. "Update Transform" was also run
6. Measurement Marker Accuracy set to  0.025m and Image Coordinate Accuracy for Marker is set to 1pix.

When having all 296 images ticked the reference pane "view error" page shows a Total Image error of  E 0.004m  N 0.004m    ALT 0.014m    - which I'm very happy with.
The Marker average check point error  values are E 0.547m   N 1.545m    ALT 0.023m  - which I am also happy with because each individual GCP error value are reasonably consistent with a std dev of only about 0.03m
I know the reason the images and the GCP estimated positions differ by so much is because the GCP survey is calibrated to a known survey mark near site and the drone is not. The drone VRSNOW reference station is 10km away.
What I have spent hours trying to do in Metashape is to use the Datum Transformation found in - Reference Settings dialogue box / Camera Reference  / Settings Icon / Custom Shift  - X Y Z to enter  a shift of X -0.547   Y -1.545   Z  -0.023. With my logic being I just want to shift all the image data by that amount to align with the GCP co-ordinates. But datum transformation doesn't seem to work like that. When I untick all the images and enter those E N RL corrections into the XYZ of the Camera Coordinate Shift  it gives crazy errors . And nothing I try will work.
The reason I haven't given up and just used the GCP's to align the images is
1. I have good image and GCP RTK data to an accuracy that is fit for purpose and should just being able to do a co-ordinate shift to get it to align well enough. Also I know I could do this shift on exporting the ortho but I want the data correct in Metashape not just correct in an export.
2. I want to get a work flow that I can use on all future projects and having the images on correct coordinatesbso if some else opens the job in metashape they can see the data is correct.
3. Also I have this great equipment and software and want to use best practice.

I'm hoping there is a very easy solution to what seems a simple problem. I have attached a pdf of my data. Thanks for reading this and if anyone has a solution thanks in advance.

Cheers
Ben

[dpitman]

Hi Ben,

Hopefully someone will respond than knows for certain whether or not that "shift" can be done in MS.  I know others are doing the "shift" in other software.

[Paulo]

Hello Ben,

First I would be very suspicious if your camera coordinates from VRSNow RTK have a shift of some 1 to 2 m in E,N compared to your GCP RTK surveyed points. Are you sure they are in same datum? I think maybe your GCP reference Survey Mark coordinates were in GDA94 datum....see https://www.agisoft.com/forum/index.php?topic=12063.msg54003#msg54003 for difference between GDA94 and GDA2020...
Anyhow the X, Y, Z datum shift parameters that you are specifying are related to the geocentric CS (red axii in attachment) and not projected compound CS (E, N, H green axii). See folowing screen copy for representation of the different CSs. So you would need to do following:

1. find E, N coordinate values of some point in center of project P1 (Eavg, Navg, Havg)
2. transform to geocentric P1gc = chunk.crs.unproject(Metashape.Vector((Eavg, Navg, Havg)))
3. define second point geocentric coordinates from first point with offset applied P2gc = chunk.crs.unproject(Metashape.Vector((Eavg + 0.547, Navg + 1.545, Havg + 0.023)))
4. P2gc - P1gc should represent the off set in X, Y, Z for datum transformation (check sign for  correct TOWGS84 transformation).

But again this is really not a rigorous procedure as it only takes into account 3 parameters instead of 7. And then you would need to define 2 datums: one for for markers (GDA94 ?) with these 3 calculated  parameters and one for cameras with all 0 towgs84 parameters. Best would be to use the complete 7 parameters of GDA94 to GDA2020 or WGS94 transformation defined in above link...supposing that your GCPs are actually referenced in GDA94.

Hope this can help,

[BenW22]

Hello Ben,

First I would be very suspicious if your camera coordinates from VRSNow RTK have a shift of some 1 to 2 m in E,N compared to your GCP RTK surveyed points. Are you sure they are in same datum? I think maybe your GCP reference Survey Mark coordinates were in GDA94 datum....see https://www.agisoft.com/forum/index.php?topic=12063.msg54003#msg54003 for difference between GDA94 and GDA2020...
Anyhow the X, Y, Z datum shift parameters that you are specifying are related to the geocentric CS (red axii in attachment) and not projected compound CS (E, N, H green axii). See folowing screen copy for representation of the different CSs. So you would need to do following:

1. find E, N coordinate values of some point in center of project P1 (Eavg, Navg, Havg)
2. transform to geocentric P1gc = chunk.crs.unproject(Metashape.Vector((Eavg, Navg, Havg)))
3. define second point geocentric coordinates from first point with offset applied P2gc = chunk.crs.unproject(Metashape.Vector((Eavg + 0.547, Navg + 1.545, Havg + 0.023)))
4. P2gc - P1gc should represent the off set in X, Y, Z for datum transformation (check sign for  correct TOWGS84 transformation).

But again this is really not a rigorous procedure as it only takes into account 3 parameters instead of 7. And then you would need to define 2 datums: one for for markers (GDA94 ?) with these 3 calculated  parameters and one for cameras with all 0 towgs84 parameters. Best would be to use the complete 7 parameters of GDA94 to GDA2020 or WGS94 transformation defined in above link...supposing that your GCPs are actually referenced in GDA94.

Hope this can help,

Hi Paulo (and everyone else),
Thanks for your great response, and you are correct, there are two different datums.  The GCP's are on GDA 2020 Zone 56 but after reading your post, I did some digging, and it looks like the drone's VRSnow mount point (TVN_Nearest_RTCM_32) was transmitting on GDA94 Zone 56 datum.
I need all my data on GDA2020 + AusGeiod2020
One of the problems I’m having now is converting the image exif data (lat and longs) to GDA94.
I ran the "Convert Reference" from the Reference Pane - converting images from their exif WGS84 data to a compound co-ordinate system - that being GDA 94 / MGA zone56 + Aus Geoid 09, but the reference co-ordinates don’t change to Easting Northing. They stay at Lat Long? Has anyone been able to convert WGS 84 to GDA94 using the CS supplied with Metashape?
Previously when I ran convert using (the incorrect) WGS84 to GDA2020 it converted all image data to Easting Northing coords.
Am I correct in saying that if I could get the converter to work for MGA 94 then I could run “convert reference” again to convert from GDA94 to GDA2020? Then my images and GPS would be on the same datum.
Paulo did a great job of explaining my XYZ datum transformation parameters didn’t work (geocentric CS vs compound CS) but am I very new to this (this is the first P4 RTK using VRSNOW I have reduced) so could someone explain if Metashape has the functionality to do small Compound CS shifts by directly entering the shift needed? If the E, N, distance needed to shift the data is small (say under 10metres) shouldn’t that be doable with a command within Metashape?  In my case it is E 0.547 N 1.545 H 0.023. Or is my only option to calculate the geocentric offset as Paulo detailed and enter it into the “datum transformation parameters” XYZ (m) shift?
Would it be a lot easier if in “datum transformation Parameter” there was the option to select either XYZ shift in Geocentric CS or XYZ shift E N RL shift in Compound CS?
Any help very much appreciated

[Paulo]

Hello Ben,

things are getting a little bit clearer. But some things need more explanaton:
 - if your VRSNow was transmitting in GDA94/MGA56, then it supposes that the camera RTK corrected coordinates are already in this datum (either exif or external text file). So why would you need to transform them from WGS84 to GDA94? You would rather need to transform from GDA94 to GDA2020 using transformation (7 parameter) referred to in previous link...
- if VRSNow is an official VRS service it is surprising that it transmits corrections in an obsolete datum and not the official Aussi datum (GDA2020).

In example, I show coordinates of a point first in GDA2020/MGA56 Ausgeoid2020 and then in GDA94/MGA56 AusGeoid09 after CS convert...

If you need more explanations, fell free to PM me.

Anyway we are getting closer to solution,

[BenW22]

Hi Paulo,

Thanks very much for your response, I appreciate your time and knowledge.
Because I'm on my training wheels I don't quite understand yet why the Exif data is in Lats Long.
I will take you up on the offer to PM you. 

Thanks,
Ben

[Paulo]

Hello Ben,

looking at following fact sheet https://www.icsm.gov.au/sites/default/files/Technical%20Fact%20Sheet%202%20-%20Using%20GDA2020%20with%20CORS.pdf it states:

Connection Details
After the changeover, all streams from Geoscience
Australia via the AUSCORS caster that lie within the
extents of GDA2020, will have GDA2020 mountpoints.
CORSnet-NSW and GPSnet will provide both GDA2020
and GDA94 mountpoints for single basestations and/or
network Real-Time Kinematic (RTK/NRTK) services.
SmartNet, AllDayRTK and VRS Now Australia will
provide GDA2020 and GDA94 mountpoints.

It is clear that VRS Now offers mountpoints or corrections referred to either GDA2020 or GDA94. So if tou are sure that your RTK camera positions were corrected from a GDA94 mount point then all your camera positions are in GDA94 geograhic coordinates So you need to convert them to GDA2020 / MGA 56 AusGeoid2020 before importing your GDA2020 RTK GCP coordinates ....and everything shoud fit fine....

Hope to hear from you,

Cheers,

[aerial_survey]

Hi Guys,
I need something like this but to process the job GRID to GROUND, all GPS/RTK data is on GRID but i need to scale the job from a known point. Do you know if there is a way?
Thanks
Allan

[jetdog6]

I'm also in need of an answer to the shift in datum transformation parameters.  Coordinate system is EPSG:3431 ellipsoidal for the photos, EPSG:3431 with geoid 18 applied for the markers.  Ebee X processed PPK.  Systemic shift vertically is 7.0' plus or minus only a tenth so it is a systemic shift, model is good.  I'd love to just shift this using the datum transformation parameters, but when I put in -7.0 into the shift field and click ok, nothing happens.  All markers are unselected.  The error on those unselected markers (no markers are selected) should have changed by the shift amount.  But nothing happens.  If this isn't the intended functionality of the datum transformation parameters, what is their intended use?