Agisoft Metashape
Agisoft Metashape => Python and Java API => Topic started by: mks_gis on August 21, 2018, 05:17:53 PM
-
***Please see later posts for updated script
Hi,
I started a script a while ago to automate some steps, which turned into a full blown automated script, including gradual selection. I use it for UAV and close range work.
I'd like some opinions from you good people as to the underlying processing logic. I often have to run multiple models with different settings, so I create one PS doc and have multiple chunks, although it does work with just one chunk. I set up the images, coord sys for each chunk, add GCP where applicable and then, so I don't need to supervise, I just run the script, leave it for the weekend and have all data ready come Monday. Seems to work...
The script creates menu options to cater to my specific needs, but can be adapted.
Some points I could use some clarification on:
I use adaptive, although it is now set to False by default (http://www.agisoft.com/forum/index.php?topic=9401.0). I'm not sure why that was changed and when it should be used.
I use optimizeCameras without parameters, trusting PS to select the right ones. Should I set parameters?
My automated gradual selection process is:
Reconstruction uncertainty to 10 - optimize
Projection Accuracy to 10 (models) or 3 (UAV) - optimize
Reprojection error - self-adjusting to find the right value to leave me with 10% of the original points - optimize
Suggestions very welcome
Cheers
M
-
***Please see later posts for updated script
I've tweaked the optional grad sel process:
1. rec uncert of 10, unless > 50% are selected, then self adjusting - optimise
2. proj acc self adjusting to select < 50% of ties left after 1 - optimise
optional break - allow for adding GCP
3. repro error self adjusting in 10% steps/optimise to select up to 80% of the initial ties.
Loosely based on USGS workflow discussed here: http://www.agisoft.com/forum/index.php?topic=9485.msg44134#msg44134
Seems to work with P4 images, Sony a6000 UVA images and DSLR close range models to reduce pixel errors. I know it's not a substitute for trying to tweak individual models for max accuracy, but good for quickly processing to a good standard, I believe. Happy to take comments.
Also added logging of values used in each step for reference if needed later.
-
Spotted a mistake, gradual selection currently only works for single chunks...
-
fixed
needs to run as one session though, i.e if you've run the script keep the session open for grad sel process
-
Thank you Mks_gis for sharing this ! Have to try this.
Quick stop after "log file to check progress :" followed by a mixed / & \ file name.
-
thanks for your really useful posts. also may i have your workflows as you put your custom menu for each command
-
Currently needs to be run as one session, valid ties not accessible after
gradual selection. To be fixed in next version using tracks.
len(chunk.point_cloud.points) - valid ties
len(chunk.point_cloud.tracks) - all ties
With gradual selection:
After alignment the model is optimised and then a gradual selection is run.
After the align process all tie points have errors attached
to them. The values used are a matter of debate.
Rec Uncert is normally set to 10 and run only once. If
the value of 10 selects more than 50% of the tie points the value
self-adjusts to less than 50%.
Proj acc is set to start at 2 and self adjusts so less
than 50% of the tie points left after Rec Uncert are selected.
For Reprojection Error the aim is to select up to a set percentage
(default 80) of the initial tie points. The value used is self
selecting. The selection process is in steps of 10% and optimisation
is run after each step.
Values used are saved in the log file.
Without gradual selection:
After alignment the model is optimised once.
3D Model/Geo: Geo basically is anything where the photos have coordinates or you are using Ground Control Points(GCP). If you have photos of objects without that info use 3D Model
Set-up workflow:
1. Set up your chunk(s)
import photos
name chunks (optional - helps in sorting the output)
save a copy of the PS Doc
Custom workflow:
2. (Geo) Set reference system for each chunk
3. Load script
chose output folder
chose file prefix (this gets added to each output file - helps if trying different settings)
4. Under "Change Values" select "Get Current Parameter info" (check console window for output)
check image quality threshold
check filtering (options: Mild (default), Moderate, Aggressive, none)
(Geo) check export CRS (in case you want the output to be different from the CRS of the camera/GCP)
No GCP or checking alignment - runs whole process and exports:
5a. (Geo) Choose "Custom":"Run - Geo" (with or without gradual selection)
or
5b. (3D Model) Choose "Custom":"Run - 3D Model" (with or without gradual selection)
With GCP or scales or if wanting to check alignment:
5. Choose "Custom":"Align only" (with or without gradual selection)
6. Set up GCP for each chunk or set scale
7. Check alignment
8a. (Geo) Choose "Custom":"Run all after alignment - Geo" (with or without gradual selection)
or
8b. (3D Model) Choose "Custom":"Run all after alignment - 3D Model" (with or without gradual selection)
Files generated depend on method used (3D or Geo) and input photos.
You can run methods individually as well after opening the script, in console type "ps_doc." and then hit "TAB" to see the options.
-
Hey mks_gis nice coding.
len(chunk.point_cloud.tracks) - counts valid and invalid tiepoints right?
len(chunk.point_cloud.points) - and this counts the initial number of valid tiepoints before gradual selection
Have you managed to count tiepoints with Python after gradual selection? Every time I use len(chunk.point_cloud.points) I get the same initial value
Currently needs to be run as one session, valid ties not accessible after
gradual selection. To be fixed in next version using tracks.
len(chunk.point_cloud.points) - valid ties
len(chunk.point_cloud.tracks) - all ties
-
Just found out. Python tricks...
points = point_cloud.points
len(points)Gives the total valid points.
Then Gradual selection is done.
And then:
chunk = doc.chunk
len(point_cloud.points)Gives the valid tiepoints number after gradual selection and for that you can assign to a new variable. You could reassign the variable points but you will loose the total points value before gradual selection.
-
Hi Luiz,
thanks for commenting. PhotoScan doesn't seem to update the points until the model is optimised.
The approach in your second post works, but only until the session is shut down. Tracks would allow you to re-open the document later and carry on. But tracks give all points, what I'm really after is the valid points after alignment. I might not change to tracks because of this. I've made a few changes to the code and am testing, will possibly upload a newer version soon-ish.
Cheers
-
Hey mks_gis,
What do you think of the following code structure, for instance, to range from a value to another of filter levels and try to remove points if a condition is True? It also can restart searching through the same range values if one is found. I'm working on still, just thought of sharing this idea to implement filtering for the USGS workflow (thinking of reconstruction uncertainty running more than once till no points are selected based on the chosen range). This of course would rapidly decrease points in point cloud.
threshold = range(0, 11, 1) # here would be the filter levels range
listx = [] # throwing a list just for the exemple
for i in threshold:
listx.append(i)
restart = 0
restartLoop = True
while restartLoop:
restartLoop = False
for idx, i in enumerate(listx):
print("do something as printing i:", i)
if i > 5: # if this condition: remove points and restart loop to search again in range
print("found value for condition: ", i)
del listx[idx] # simulates removing points for a certain i value
# optimization would happen here
restartLoop = True
print("RESTARTING LOOP\n")
restart += 1
break # break inner while and restart for loop
else:
# continue if the inner loop wasn't broken
continue
else:
continue
print("restart - outer while", restart)
Looking forward to see your new version of the code!
-
Hi Luiz,
Thanks for your code. Not been able to do much on this lately, and to go through your code properly, but have updated to accommodate Metashape and some minor tweaks. Thought I'd share until I have more time.
Cheers
Martin
-
Hi Luiz,
Thanks for your code. Not been able to do much on this lately, and to go through your code properly, but have updated to accommodate Metashape and some minor tweaks. Thought I'd share until I have more time.
Cheers
Martin
Hi the script is updated? i have problems with the variable Hightaccurazy
-
Hi fjgarciam,
i updated the script to v8 to be compatible with MS v1.6 upwards.... Basically matchPhotos uses downscale parameter instead of accuracy and buildDepthMaps uses also parameter downscale instead of quaity...
Aso introduced 2 boolean parameters (gen_pre and ref_pre) to use to define preselection values (T or F) in matchPhotos process...
Also changed filter parameter of buildDepthMaps process to filter_mode
There maybe other changes necessary for builModels, dem and ortho but look if it works for you....
-
Thanks Paul, I'll look at your version and perhaps merge them.
The maintained version is on Github:
https://github.com/gisportsmouth/PhotoScan-Automation-Script
-
Thanks mks_gis,
actually in your Github, I see that most parameter changes I made (downscale for matchPhotos and buildDepthMaps, filter_mode for buildDepthMaps) are already introduced in your v7.4.
However, i do not really understand why generic_preselection is always set to False. Should it not be selectable or changeable by user?
-
TBH, I can't remember why it is False. This script develops as and when I need it, SfM is only a small part of what I do. I'll look into it when I next edit it, but you seem to have the skill to develop it further for your needs.
-
Hi Everyone
Thanks to mks_gis for creating the "PScan Chunk Scripts v8 Metashape" script. It looks very comprehensive. Unfortunately for me, I have no scripting knowledge, but when I follow the instructions, I get the follownig error - 2023-03-18 11:17:58 Error: 'Metashape.Chunk' object has no attribute 'analyzePhotos'
Could anyone suggest how I could fix this? I am using metashape verision 2.0.0
I am testing the script on a small sample of 200 photos. I have successfully used the same photos in another job (without using the script) so I know what results should be.
Log extract -
2023-03-18 11:17:23 Log file to check progress: F:/testing of scripts\test for v8 py script_log_2023-03-18 11-17.txt
2023-03-18 11:17:30 The current path is: F:/testing of scripts
2023-03-18 11:17:30 The current file name prefix is: v8
2023-03-18 11:17:30 The current minimum acceptable image quality is: 0.7
2023-03-18 11:17:30 The current filtering method is: Metashape.FilterMode.MildFiltering
2023-03-18 11:17:30 The current projection accuracy is: 2
2023-03-18 11:17:30 The current reconstruction accuracy is: 10
2023-03-18 11:17:30 The current generic preselection is: True
2023-03-18 11:17:30 The current reference preselection is: True
2023-03-18 11:17:30 The current export CRS (EPSG Number) is [0 = project CRS]: 0
2023-03-18 11:17:58 Error: 'Metashape.Chunk' object has no attribute 'analyzePhotos'
Any help appreciated
>>>
-
Hi
The script was last updated for version 1.8. There must have been changes to the Python API in the update to v2 which break the script, hence the error.
I will look at this and update the script when I have time, hopefully next week
I'll also need to check why the major version check didn't flag this as an issue.
Cheers
-
Yes for version 2 and higher use chunk.analyzeImages instead of chunk.analyzePhotos
-
Thanks to you both - mks_gis and Paulo. I will substituate chunk.analyzeImages and see if I can get it to work.
-
Hi BenW,
you will need to replace all instances of PointCloud with TiePoints
and point_cloud with tie_points
-
Thank you Paulo,
Maybe that is why I'm getting these errors below. I will change as you suggest and try again.
2023-03-19 13:04:27 Error: 'NoneType' object has no attribute 'points'
2023-03-19 13:41:08 Error: type object 'Metashape.PointCloud' has no attribute 'Filter'
-
Exactly and also change buildDenseCloud to buildPointCloud
-
I've updated the code for 2.0, script v8.0 on github:
https://github.com/gisportsmouth/PhotoScan-Automation-Script