Messages

16

Python and Java API / Re: Automated PS script to process single or multiple chunks - opinions please

« on: November 06, 2018, 12:30:38 PM »

Currently needs to be run as one session, valid ties not accessible after
    gradual selection. To be fixed in next version using tracks.
   
    len(chunk.point_cloud.points) - valid ties
    len(chunk.point_cloud.tracks) - all ties

With gradual selection:
    After alignment the model is optimised and then a gradual selection is run.
    After the align process all tie points have errors attached
            to them. The values used are a matter of debate.
            Rec Uncert is normally set to 10 and run only once. If
            the value of 10 selects more than 50% of the tie points the value
            self-adjusts to less than 50%.
            Proj acc is set to start at 2 and self adjusts so less
            than 50% of the tie points left after Rec Uncert are selected.
            For Reprojection Error the aim is to select up to a set percentage
            (default 80) of the initial tie points. The value used is self
            selecting. The selection process is in steps of 10% and optimisation
            is run after each step.
            Values used are saved in the log file.
           
Without gradual selection:
    After alignment the model is optimised once.

3D Model/Geo: Geo basically is anything where the photos have coordinates or you are using Ground Control Points(GCP). If you have photos of objects without that info use 3D Model

Set-up workflow:
1. Set up your chunk(s)
    import photos
    name chunks (optional - helps in sorting the output)
    save a copy of the PS Doc

Custom workflow:
2. (Geo) Set reference system for each chunk
3. Load script
    chose output folder
    chose file prefix (this gets added to each output file - helps if trying different settings)
4. Under "Change Values" select "Get Current Parameter info" (check console window for output)
    check image quality threshold
    check filtering (options: Mild (default), Moderate, Aggressive, none)
    (Geo) check export CRS (in case you want the output to be different from the CRS of the camera/GCP)

No GCP or checking alignment - runs whole process and exports:
5a. (Geo) Choose "Custom":"Run - Geo" (with or without gradual selection)
or
5b. (3D Model) Choose "Custom":"Run - 3D Model" (with or without gradual selection)

With GCP or scales or if wanting to check alignment:
5. Choose "Custom":"Align only" (with or without gradual selection)
6. Set up GCP for each chunk or set scale
7. Check alignment
8a. (Geo) Choose "Custom":"Run all after alignment - Geo" (with or without gradual selection)
or
8b. (3D Model) Choose "Custom":"Run all after alignment - 3D Model" (with or without gradual selection)

Files generated depend on method used (3D or Geo) and input photos.

You can run methods individually as well after opening the script, in console type "ps_doc." and then hit "TAB" to see the options.

17

Python and Java API / Re: How do you access total tie point number in Python

« on: September 01, 2018, 04:06:06 PM »

Thanks Alexey!

18

Python and Java API / Re: Automated PS script to process single or multiple chunks - opinions please

« on: August 31, 2018, 07:46:21 PM »

fixed

needs to run as one session though, i.e if you've run the script keep the session open for grad sel process

19

General / Re: Your opinion on USGS Agisoft Processing Workflow

« on: August 31, 2018, 05:52:12 PM »

I understand commercial implications, it's a shame though as sharing our settings and experiences helps the community overall. We'll slog on, cheers.

20

Python and Java API / Re: Automated PS script to process single or multiple chunks - opinions please

« on: August 31, 2018, 04:14:51 PM »

Spotted a mistake, gradual selection currently only works for single chunks...

21

Python and Java API / How do you access total tie point number in Python

« on: August 31, 2018, 03:42:35 PM »

Hi,

 In the tie point information PS gives you y of x tie points. You can get the active number of tie points (y) by using len([i for i in chunk.point_cloud.points]). How do I access x, i.e the original number of tie points in Python, I can't find it in the reference doc?

Cheers
M.

22

General / Re: Your opinion on USGS Agisoft Processing Workflow

« on: August 31, 2018, 11:58:48 AM »

Funny, seeing how I followed this workflow and got worse results than following our own developed workflow.

I suppose it all depends on your data set.

Could you outline your workflow and describe your type of data, out of interest?

23

General / Re: Your opinion on USGS Agisoft Processing Workflow

« on: August 31, 2018, 11:08:11 AM »

Oh Geeze, the results after using USGS methodology are 100x better than not.
This should be part of the agisoft standard workflow IMO.

I wonder if the gradual selection steps could be automated. I am a .net guy mainly but can adapt to other languages.
I sent my "re-write" of the USGS workflow to agisoft sales. Hope they do something with it.
In the end, the PS results were better than Dronedeploy and pix4d.

I've automated grad sel here using Python, loosely based on this, but not fully (only do RecUncert/ProcAcc once each):

http://www.agisoft.com/forum/index.php?topic=9578.0

workflow with GCP:
run script
set export folder
set any file prefix for export files
in Custom Menu Align with grad
Do your GCP work
in Custom menu Run all after alignment - Geo with grad

Opinions welcome.

24

Python and Java API / tweaked automatic gradual selection process

« on: August 29, 2018, 01:16:34 PM »

***Please see later posts for updated script

I've tweaked the optional grad sel process:
1. rec uncert of 10, unless > 50% are selected, then self adjusting - optimise
2. proj acc self adjusting to select < 50% of ties left after 1 - optimise
optional break - allow for adding GCP
3. repro error self adjusting in 10% steps/optimise to select up to 80% of the initial ties.

Loosely based on USGS workflow discussed here: http://www.agisoft.com/forum/index.php?topic=9485.msg44134#msg44134

Seems to work with P4 images, Sony a6000 UVA images and DSLR close range models to reduce pixel errors. I know it's not a substitute for trying to tweak individual models for max accuracy, but good for quickly processing to a good standard, I believe. Happy to take comments.

Also added logging of values used in each step for reference if needed later.

25

General / Re: Your opinion on USGS Agisoft Processing Workflow

« on: August 24, 2018, 01:42:42 PM »

I'm unclear of one thing, amongst others. Perhaps someone can help.

In the gradual selection process they mention the percentage of points selected in each step. Is that percentage of total tie points you started with, or percentage of tie points left after each step. The latter obviously reduces the tie points considerably, when I tried it with some P4 data by about 90% of tie points we started with.

26

General / Re: Your opinion on USGS Agisoft Processing Workflow

« on: August 23, 2018, 05:37:16 PM »

This is really interesting! How would you update the workflow with the adaptive fitting option that PS now has? In the expanded document they say to use the default values or use the checked parameters for optimisation, which is best?

Cheers
Martin

27

Python and Java API / Automated PS script to process single or multiple chunks - opinions please

« on: August 21, 2018, 05:17:53 PM »

***Please see later posts for updated script

Hi,

 I started a script a while ago to automate some steps, which turned into a full blown automated script, including gradual selection. I use it for UAV and close range work.

 I'd like some opinions from you good people as to the underlying processing logic. I often have to run multiple models with different settings, so I create one PS doc and have multiple chunks, although it does work with just one chunk. I set up the images, coord sys for each chunk, add GCP where applicable and then, so I don't need to supervise, I just run the script, leave it for the weekend and have all data ready come Monday. Seems to work...

The script creates menu options to cater to my specific needs, but can be adapted.

Some points I could use some clarification on:
I use adaptive, although it is now set to False by default (http://www.agisoft.com/forum/index.php?topic=9401.0). I'm not sure why that was changed and when it should be used.
 
I use optimizeCameras without parameters, trusting PS to select the right ones. Should I set parameters?

My automated gradual selection process is:
Reconstruction uncertainty to 10 - optimize
Projection Accuracy to 10 (models) or 3 (UAV) - optimize
Reprojection error - self-adjusting to find the right value to leave me with 10% of the original points - optimize

Suggestions very welcome
Cheers
M

28

Python and Java API / Why is Adaptive to false now?

« on: July 26, 2018, 04:09:43 PM »

Hi,

Given the complexity of PS I'm trying to work through settings and why we set them, and when to chose on over the other. In laymans terms, why was Adaptive set to False by default in the latest update, and when should or shouldn't we use it. I know the latter has been answered before, but perhaps you can provide some specific reasons to do it or not to do it. What are the decision criteria I should go through for a project?

Cheers
Martin

29

Python and Java API / Re: Dense cloud not in project after processing

« on: February 15, 2018, 12:28:54 PM »

Hi Alexey,

 Thanks for getting back to me. The alignment completed properly and the log file is for the chunk with the missing dense cloud.

 I've tried to reproduce the issue with a subset of images, with the same code and I can't reproduce it now. It was most likely related to having only a single chunk; I had written this code to work with multiple chunks originally. I've since rewritten the code, making the doc the basis for the class and it's working fine. It was a glitch at the time, but I'm not sure it's reproducible. I would not want to waste your time for something I most likely did wrong. But I'm happy to learn if you can point out my errors.

Cheers
Martin


import PhotoScan
import os

# Checking compatibility
compatible_major_version = "1.4"
found_major_version = ".".join(PhotoScan.app.version.split('.')[:2])
if found_major_version != compatible_major_version:
    raise Exception("Incompatible PhotoScan version: {} != {}".format(found_major_version, compatible_major_version))

class PS_Proc(object):
   
    def __init__(self, doc):
        self.path = PhotoScan.app.getExistingDirectory('Specify TIF DSM/Ortho export folder:')
        self.prefix = PhotoScan.app.getString(label='Enter file prefix: ', value='')
        self.doc = doc
        if not len(self.doc.chunks):
                raise Exception("No chunks!")
        self.chunks = self.doc.chunks

    def remove_align(self):
        for _ in self.chunks:
            for c in _.cameras:
                c.transform = None
               
    def align(self):
        for _ in self.chunks:
            _.matchPhotos(accuracy=PhotoScan.HighAccuracy,
                            generic_preselection=False,
                            reference_preselection=True,
                            filter_mask=True,
                            keypoint_limit=40000,
                            tiepoint_limit=0)
            _.alignCameras()
            self.doc.save()

    def dense_c(self):
        for _ in self.chunks:
            _.buildDepthMaps(quality=PhotoScan.HighQuality,
                                filter=PhotoScan.MildFiltering)           
            _.buildDenseCloud(point_colors=True)
            self.doc.save()
           
    def dem(self):
        for _ in self.chunks:
            _.buildDem(source=PhotoScan.DenseCloudData,
                        interpolation=PhotoScan.EnabledInterpolation)
        self.doc.save()               

    def ortho(self):
        for _ in self.chunks:
            _.buildOrthomosaic(surface=PhotoScan.ElevationData,
                                blending=PhotoScan.MosaicBlending,
                                fill_holes=False)
        self.doc.save()
                               
    def export(self):
        for _ in self.chunks:
            num = str(_)[-3]
            file = '{}_DSM_{}.tif'.format(self.prefix, num)
            _.exportDem(path = os.path.join(self.path, file),
                        write_world = True)
            file = '{}_Ortho_{}.tif'.format(self.prefix, num)
            _.exportOrthomosaic(path = os.path.join(self.path, file),
                                write_world = True)

    def exp_jpg(self):
        jpgpath = os.path.join(self.path, 'JPG Orthos')
        if not os.path.exists(jpgpath):
            jpgpath = PhotoScan.app.getExistingDirectory('Specify JPG export folder:')
        for _ in self.chunks:
            num = str(_)[-3]
            file = '{}_Ortho_{}.jpg'.format(self.prefix, num)
            _.exportOrthomosaic(path = os.path.join(jpgpth, file),
                                write_world = True)   
    def run_all(self):
        self.align()
        self.dense_c()
        self.dem()
        self.ortho()

    def run_post_align(self):
        self.dense_c()
        self.dem()
        self.ortho()   

def menu(label, function):
    PhotoScan.app.addMenuItem(label, function)
    print('To execute this script press {}'.format(label))

#initiate object   
ps_doc = PS_Proc(PhotoScan.app.document)
#create menu options
menu('Custom/Remove alignment(optional)', ps_doc.remove_align)
menu('Custom/Align', ps_doc.align)
menu('Custom/Build dense cloud', ps_doc.dense_c)
menu('Custom/Build DSM', ps_doc.dem)
menu('Custom/Build Ortho', ps_doc.ortho)
menu('Custom/Run all', ps_doc.run_all)
menu('Custom/Run all after alignment', ps_doc.run_post_align)
menu('Custom/Export TIF', ps_doc.export)
menu('Custom/Export JPG', ps_doc.exp_jpg)

           

30

Python and Java API / Re: Dense cloud not in project after processing

« on: February 14, 2018, 12:20:21 PM »

Hi,

I've rewritten my code and it now works, although I don't know what I did to fix it. It seems to have been related to only having one chunk in this particular project:

import PhotoScan
import os

# Checking compatibility
compatible_major_version = "1.4"
found_major_version = ".".join(PhotoScan.app.version.split('.')[:2])
if found_major_version != compatible_major_version:
    raise Exception("Incompatible PhotoScan version: {} != {}".format(found_major_version, compatible_major_version))

class PS_Proc(object):
   
    def __init__(self, doc):
        self.doc = doc
        if not len(self.doc.chunks):
                raise Exception("No chunks!")
        self.chunks = self.doc.chunks

    def remove_align(self):
        for _ in self.chunks:
            for c in _.cameras:
                c.transform = None
               
    def align(self):
        for _ in self.chunks:
            _.matchPhotos(accuracy=PhotoScan.HighAccuracy,
                            generic_preselection=False,
                            reference_preselection=True,
                            filter_mask=True,
                            keypoint_limit=40000,
                            tiepoint_limit=0)
            _.alignCameras()
            self.doc.save()

    def dense_c(self):
        for _ in self.chunks:
            _.buildDepthMaps(quality=PhotoScan.HighQuality,
                                filter=PhotoScan.MildFiltering)           
            _.buildDenseCloud(point_colors=True)
            self.doc.save()
           
    def dem(self):
        for _ in self.chunks:
            _.buildDem(source=PhotoScan.DenseCloudData,
                        interpolation=PhotoScan.EnabledInterpolation)
        self.doc.save()               

    def ortho(self):
        for _ in self.chunks:
            _.buildOrthomosaic(surface=PhotoScan.ElevationData,
                                blending=PhotoScan.MosaicBlending,
                                fill_holes=False)
        self.doc.save()
                               
    def export(self, pth, prefix):
        for _ in self.chunks:
            num = str(_)[-3]
            file = '{}_DSM_{}.tif'.format(prefix, num)
            _.exportDem(path = os.path.join(pth, file),
                        write_world = True)
            file = '{}_Ortho_{}.tif'.format(prefix, num)
            _.exportOrthomosaic(path = os.path.join(pth, file),
                                write_world = True)

    def exp_jpg(self, pth, prefix, w = None, h = None):
        doc.save()
        for _ in self.chunks:
            num = str(_)[-3]
            file = '{}_Ortho_{}.jpg'.format(prefix, num)
            _.exportOrthomosaic(path = os.path.join(pth, file),
                                write_world = True)   

ps_doc = PS_Proc(PhotoScan.app.document)
pth = r'E:\Dominica\Data Output Folder'
jpgpth = r'E:\Dominica\Data Output Folder\JPG Orthos'
prefix = input('Enter file prefix: ')
ps_doc.align()
ps_doc.dense_c()
ps_doc.dem()
ps_doc.ortho()
ps_doc.export(pth, prefix)
ps_doc.exp_jpg(jpgpth, prefix)
doc.save()