Messages

16

General / Re: Your opinion on USGS Agisoft Processing Workflow

« on: August 31, 2018, 11:08:11 AM »

Oh Geeze, the results after using USGS methodology are 100x better than not.
This should be part of the agisoft standard workflow IMO.

I wonder if the gradual selection steps could be automated. I am a .net guy mainly but can adapt to other languages.
I sent my "re-write" of the USGS workflow to agisoft sales. Hope they do something with it.
In the end, the PS results were better than Dronedeploy and pix4d.

I've automated grad sel here using Python, loosely based on this, but not fully (only do RecUncert/ProcAcc once each):

http://www.agisoft.com/forum/index.php?topic=9578.0

workflow with GCP:
run script
set export folder
set any file prefix for export files
in Custom Menu Align with grad
Do your GCP work
in Custom menu Run all after alignment - Geo with grad

Opinions welcome.

17

Python and Java API / tweaked automatic gradual selection process

« on: August 29, 2018, 01:16:34 PM »

***Please see later posts for updated script

I've tweaked the optional grad sel process:
1. rec uncert of 10, unless > 50% are selected, then self adjusting - optimise
2. proj acc self adjusting to select < 50% of ties left after 1 - optimise
optional break - allow for adding GCP
3. repro error self adjusting in 10% steps/optimise to select up to 80% of the initial ties.

Loosely based on USGS workflow discussed here: http://www.agisoft.com/forum/index.php?topic=9485.msg44134#msg44134

Seems to work with P4 images, Sony a6000 UVA images and DSLR close range models to reduce pixel errors. I know it's not a substitute for trying to tweak individual models for max accuracy, but good for quickly processing to a good standard, I believe. Happy to take comments.

Also added logging of values used in each step for reference if needed later.

18

General / Re: Your opinion on USGS Agisoft Processing Workflow

« on: August 24, 2018, 01:42:42 PM »

I'm unclear of one thing, amongst others. Perhaps someone can help.

In the gradual selection process they mention the percentage of points selected in each step. Is that percentage of total tie points you started with, or percentage of tie points left after each step. The latter obviously reduces the tie points considerably, when I tried it with some P4 data by about 90% of tie points we started with.

19

General / Re: Your opinion on USGS Agisoft Processing Workflow

« on: August 23, 2018, 05:37:16 PM »

This is really interesting! How would you update the workflow with the adaptive fitting option that PS now has? In the expanded document they say to use the default values or use the checked parameters for optimisation, which is best?

Cheers
Martin

20

Python and Java API / Automated PS script to process single or multiple chunks - opinions please

« on: August 21, 2018, 05:17:53 PM »

***Please see later posts for updated script

Hi,

 I started a script a while ago to automate some steps, which turned into a full blown automated script, including gradual selection. I use it for UAV and close range work.

 I'd like some opinions from you good people as to the underlying processing logic. I often have to run multiple models with different settings, so I create one PS doc and have multiple chunks, although it does work with just one chunk. I set up the images, coord sys for each chunk, add GCP where applicable and then, so I don't need to supervise, I just run the script, leave it for the weekend and have all data ready come Monday. Seems to work...

The script creates menu options to cater to my specific needs, but can be adapted.

Some points I could use some clarification on:
I use adaptive, although it is now set to False by default (http://www.agisoft.com/forum/index.php?topic=9401.0). I'm not sure why that was changed and when it should be used.
 
I use optimizeCameras without parameters, trusting PS to select the right ones. Should I set parameters?

My automated gradual selection process is:
Reconstruction uncertainty to 10 - optimize
Projection Accuracy to 10 (models) or 3 (UAV) - optimize
Reprojection error - self-adjusting to find the right value to leave me with 10% of the original points - optimize

Suggestions very welcome
Cheers
M

21

Python and Java API / Why is Adaptive to false now?

« on: July 26, 2018, 04:09:43 PM »

Hi,

Given the complexity of PS I'm trying to work through settings and why we set them, and when to chose on over the other. In laymans terms, why was Adaptive set to False by default in the latest update, and when should or shouldn't we use it. I know the latter has been answered before, but perhaps you can provide some specific reasons to do it or not to do it. What are the decision criteria I should go through for a project?

Cheers
Martin

22

Python and Java API / Re: Dense cloud not in project after processing

« on: February 15, 2018, 12:28:54 PM »

Hi Alexey,

 Thanks for getting back to me. The alignment completed properly and the log file is for the chunk with the missing dense cloud.

 I've tried to reproduce the issue with a subset of images, with the same code and I can't reproduce it now. It was most likely related to having only a single chunk; I had written this code to work with multiple chunks originally. I've since rewritten the code, making the doc the basis for the class and it's working fine. It was a glitch at the time, but I'm not sure it's reproducible. I would not want to waste your time for something I most likely did wrong. But I'm happy to learn if you can point out my errors.

Cheers
Martin


import PhotoScan
import os

# Checking compatibility
compatible_major_version = "1.4"
found_major_version = ".".join(PhotoScan.app.version.split('.')[:2])
if found_major_version != compatible_major_version:
    raise Exception("Incompatible PhotoScan version: {} != {}".format(found_major_version, compatible_major_version))

class PS_Proc(object):
   
    def __init__(self, doc):
        self.path = PhotoScan.app.getExistingDirectory('Specify TIF DSM/Ortho export folder:')
        self.prefix = PhotoScan.app.getString(label='Enter file prefix: ', value='')
        self.doc = doc
        if not len(self.doc.chunks):
                raise Exception("No chunks!")
        self.chunks = self.doc.chunks

    def remove_align(self):
        for _ in self.chunks:
            for c in _.cameras:
                c.transform = None
               
    def align(self):
        for _ in self.chunks:
            _.matchPhotos(accuracy=PhotoScan.HighAccuracy,
                            generic_preselection=False,
                            reference_preselection=True,
                            filter_mask=True,
                            keypoint_limit=40000,
                            tiepoint_limit=0)
            _.alignCameras()
            self.doc.save()

    def dense_c(self):
        for _ in self.chunks:
            _.buildDepthMaps(quality=PhotoScan.HighQuality,
                                filter=PhotoScan.MildFiltering)           
            _.buildDenseCloud(point_colors=True)
            self.doc.save()
           
    def dem(self):
        for _ in self.chunks:
            _.buildDem(source=PhotoScan.DenseCloudData,
                        interpolation=PhotoScan.EnabledInterpolation)
        self.doc.save()               

    def ortho(self):
        for _ in self.chunks:
            _.buildOrthomosaic(surface=PhotoScan.ElevationData,
                                blending=PhotoScan.MosaicBlending,
                                fill_holes=False)
        self.doc.save()
                               
    def export(self):
        for _ in self.chunks:
            num = str(_)[-3]
            file = '{}_DSM_{}.tif'.format(self.prefix, num)
            _.exportDem(path = os.path.join(self.path, file),
                        write_world = True)
            file = '{}_Ortho_{}.tif'.format(self.prefix, num)
            _.exportOrthomosaic(path = os.path.join(self.path, file),
                                write_world = True)

    def exp_jpg(self):
        jpgpath = os.path.join(self.path, 'JPG Orthos')
        if not os.path.exists(jpgpath):
            jpgpath = PhotoScan.app.getExistingDirectory('Specify JPG export folder:')
        for _ in self.chunks:
            num = str(_)[-3]
            file = '{}_Ortho_{}.jpg'.format(self.prefix, num)
            _.exportOrthomosaic(path = os.path.join(jpgpth, file),
                                write_world = True)   
    def run_all(self):
        self.align()
        self.dense_c()
        self.dem()
        self.ortho()

    def run_post_align(self):
        self.dense_c()
        self.dem()
        self.ortho()   

def menu(label, function):
    PhotoScan.app.addMenuItem(label, function)
    print('To execute this script press {}'.format(label))

#initiate object   
ps_doc = PS_Proc(PhotoScan.app.document)
#create menu options
menu('Custom/Remove alignment(optional)', ps_doc.remove_align)
menu('Custom/Align', ps_doc.align)
menu('Custom/Build dense cloud', ps_doc.dense_c)
menu('Custom/Build DSM', ps_doc.dem)
menu('Custom/Build Ortho', ps_doc.ortho)
menu('Custom/Run all', ps_doc.run_all)
menu('Custom/Run all after alignment', ps_doc.run_post_align)
menu('Custom/Export TIF', ps_doc.export)
menu('Custom/Export JPG', ps_doc.exp_jpg)

           

23

Python and Java API / Re: Dense cloud not in project after processing

« on: February 14, 2018, 12:20:21 PM »

Hi,

I've rewritten my code and it now works, although I don't know what I did to fix it. It seems to have been related to only having one chunk in this particular project:

import PhotoScan
import os

# Checking compatibility
compatible_major_version = "1.4"
found_major_version = ".".join(PhotoScan.app.version.split('.')[:2])
if found_major_version != compatible_major_version:
    raise Exception("Incompatible PhotoScan version: {} != {}".format(found_major_version, compatible_major_version))

class PS_Proc(object):
   
    def __init__(self, doc):
        self.doc = doc
        if not len(self.doc.chunks):
                raise Exception("No chunks!")
        self.chunks = self.doc.chunks

    def remove_align(self):
        for _ in self.chunks:
            for c in _.cameras:
                c.transform = None
               
    def align(self):
        for _ in self.chunks:
            _.matchPhotos(accuracy=PhotoScan.HighAccuracy,
                            generic_preselection=False,
                            reference_preselection=True,
                            filter_mask=True,
                            keypoint_limit=40000,
                            tiepoint_limit=0)
            _.alignCameras()
            self.doc.save()

    def dense_c(self):
        for _ in self.chunks:
            _.buildDepthMaps(quality=PhotoScan.HighQuality,
                                filter=PhotoScan.MildFiltering)           
            _.buildDenseCloud(point_colors=True)
            self.doc.save()
           
    def dem(self):
        for _ in self.chunks:
            _.buildDem(source=PhotoScan.DenseCloudData,
                        interpolation=PhotoScan.EnabledInterpolation)
        self.doc.save()               

    def ortho(self):
        for _ in self.chunks:
            _.buildOrthomosaic(surface=PhotoScan.ElevationData,
                                blending=PhotoScan.MosaicBlending,
                                fill_holes=False)
        self.doc.save()
                               
    def export(self, pth, prefix):
        for _ in self.chunks:
            num = str(_)[-3]
            file = '{}_DSM_{}.tif'.format(prefix, num)
            _.exportDem(path = os.path.join(pth, file),
                        write_world = True)
            file = '{}_Ortho_{}.tif'.format(prefix, num)
            _.exportOrthomosaic(path = os.path.join(pth, file),
                                write_world = True)

    def exp_jpg(self, pth, prefix, w = None, h = None):
        doc.save()
        for _ in self.chunks:
            num = str(_)[-3]
            file = '{}_Ortho_{}.jpg'.format(prefix, num)
            _.exportOrthomosaic(path = os.path.join(pth, file),
                                write_world = True)   

ps_doc = PS_Proc(PhotoScan.app.document)
pth = r'E:\Dominica\Data Output Folder'
jpgpth = r'E:\Dominica\Data Output Folder\JPG Orthos'
prefix = input('Enter file prefix: ')
ps_doc.align()
ps_doc.dense_c()
ps_doc.dem()
ps_doc.ortho()
ps_doc.export(pth, prefix)
ps_doc.exp_jpg(jpgpth, prefix)
doc.save()

24

Python and Java API / Dense cloud not in project after processing

« on: February 14, 2018, 01:28:28 AM »

Hi,

 I'm processing images from UAV to generate DEM and Orthos. I wrote a Python script to process multiple chunks at the same time (and to learn using Python for PS).

 I'm now using the script and in the build dense cloud step after processing for several hours the dense cloud is not there, even though the log file says it ran successfully. Depth maps don't seem to be there either.  I'm on 1.4.

This has worked in a previous project with multiple chunks, although one one occasion I had the same problem, but on a smaller image set and I just ran it again, this time successfully. I run each step (align, dense, dem, ortho, export) separately, as I check the results and clean up before continuing.

Why is the Dense Cloud missing, what did I miss?

Cheers
Martin


code snippet that runs dense cloud:
def dense_c(self):
        for _ in self.cks:
            _.buildDepthMaps(quality=PhotoScan.HighQuality,
                                filter=PhotoScan.MildFiltering)           
            _.buildDenseCloud(point_colors=True)

Log (last few lines):
2018-02-13 16:45:27
2018-02-13 16:45:27 Depth reconstruction devices performance:
2018-02-13 16:45:27  - 5%    done by CPU
2018-02-13 16:45:27  - 95%    done by GeForce GTX 980
2018-02-13 16:45:27 Total time: 6356.12 seconds
2018-02-13 16:45:27
2018-02-13 16:45:28 Finished processing in 6375.36 sec (exit code 1)
2018-02-13 16:45:28 BuildDenseCloud
2018-02-13 16:45:28 Generating dense cloud...
2018-02-13 16:45:28 Generating dense point cloud...
2018-02-13 16:45:32 selected 284 cameras in 4.853 sec
2018-02-13 16:45:32 working volume: 8580x3829x2775
2018-02-13 16:45:32 tiles: 2x1x1
2018-02-13 16:45:32 selected 135 cameras in 0.001 sec
2018-02-13 16:45:32 preloading data... done in 3.479 sec
2018-02-13 16:45:36 filtering depth maps... done in 881.324 sec
2018-02-13 17:00:18 preloading data... done in 7.266 sec
2018-02-13 17:00:25 accumulating data... done in 12.639 sec
2018-02-13 17:00:39 building point cloud... done in 2.298 sec
2018-02-13 17:00:42 selected 232 cameras in 0.001 sec
2018-02-13 17:00:42 preloading data... done in 5.479 sec
2018-02-13 17:00:47 filtering depth maps... done in 10398 sec
2018-02-13 19:54:06 preloading data... done in 13.742 sec
2018-02-13 19:54:20 accumulating data... done in 29.615 sec
2018-02-13 19:54:51 building point cloud... done in 2.034 sec
2018-02-13 19:54:54 52859925 points extracted
2018-02-13 19:54:55 Finished processing in 11367.2 sec (exit code 1)

Full code:
import PhotoScan
import os

# Checking compatibility
compatible_major_version = "1.4"
found_major_version = ".".join(PhotoScan.app.version.split('.')[:2])
if found_major_version != compatible_major_version:
    raise Exception("Incompatible PhotoScan version: {} != {}".format(found_major_version, compatible_major_version))

class ChunksProc(object):
   
    def __init__(self, cks):
        self.cks = cks

    def remove_align(self):
        for _ in self.cks:
            for c in _.cameras:
                c.transform = None
               
    def align(self):
        for _ in self.cks:
            _.matchPhotos(accuracy=PhotoScan.HighAccuracy,
                            generic_preselection=False,
                            reference_preselection=True,
                            filter_mask=True,
                            keypoint_limit=40000,
                            tiepoint_limit=0)
            _.alignCameras()

    def dense_c(self):
        for _ in self.cks:
            _.buildDepthMaps(quality=PhotoScan.HighQuality,
                                filter=PhotoScan.MildFiltering)           
            _.buildDenseCloud(point_colors=True)

    def dem(self):
        for _ in self.cks:
            _.buildDem(source=PhotoScan.DenseCloudData,
                        interpolation=PhotoScan.EnabledInterpolation)

    def ortho(self):
        for _ in self.cks:
            _.buildOrthomosaic(surface=PhotoScan.ElevationData,
                                blending=PhotoScan.MosaicBlending,
                                fill_holes=False)

    def export(self, pth, prefix):
        for _ in self.cks:
            num = str(_)[-3]
            file = '{}_DSM_{}.tif'.format(prefix, num)
            _.exportDem(path = os.path.join(pth, file),
                        write_world = True)
            file = '{}_Ortho_{}.tif'.format(prefix, num)
            _.exportOrthomosaic(path = os.path.join(pth, file),
                                write_world = True)

    def exp_jpg(self, pth, prefix, w = None, h = None):
        doc.save()
        for _ in self.cks:
            num = str(_)[-3]
            file = '{}_Ortho_{}.jpg'.format(prefix, num)
            _.exportOrthomosaic(path = os.path.join(pth, file),
                                write_world = True)   

doc = PhotoScan.app.document
if not len(doc.chunks):
        raise Exception("No chunks!")
cks = ChunksProc(doc.chunks)
pth = r'E:\Dominica\Data Output Folder'
jpgpth = r'E:\Dominica\Data Output Folder\JPG Orthos'
prefix = input('Enter file prefix: ')
cks.align()
doc.save()
cks.dense_c()
doc.save()
cks.dem()
doc.save()
cks.ortho()
doc.save()
cks.export(pth, prefix)
cks.exp_jpg(jpgpth, prefix)
doc.save()

25

General / Documentation on smoothing mesh

« on: June 27, 2017, 06:25:11 PM »

Hi,

 I'm smoothing a mesh in Tools and am wondering if there's any documentation on the tool. What do the numbers mean?

Cheers
Martin

26

General / Compact Dense Point cloud does not remove deleted points

« on: June 19, 2017, 12:45:34 PM »

Hi,

 I've got a rather large point cloud and I'd like to mesh just a section of it, as PS will not mesh the whole thing.

 I've cropped to a selection and ran compact dense cloud, which, as I understand it, should remove deleted points. I also ran update dense cloud, but the point number remains stubbornly high.

How do I delete deleted points completely?

Thanks
Martin

27

General / Re: Aerial photography overlap overspecified? What works?

« on: October 03, 2015, 12:31:21 PM »

Restrictions in time, batteries, tide, access to site might require a more complex approach.

28

General / Re: Sony Arw files not recognised

« on: October 02, 2015, 06:30:12 PM »

Hi,

 A massive advantage of reading the Canon RAW files directly is that RAW files are smaller than 16-bit or even 8-bit tif, by quite a margin. Converting the Sony ARW to tif is annoyingly time consuming. As I can't hang my DSLR from our octocopter I think I might only set the Sony to RAW  for special projects, as I'm getting some good results with JPG.

M.

29

General / Aerial photography overlap overspecified? What works?

« on: October 02, 2015, 06:17:33 PM »

Hi,

 The PhotoScan manual specifies 60% of side overlap + 80% of forward overlap for aerial photography. I'm struggling to plan a mission for this in Mission Planner, as the interval between pictures drops below 2 second if you do this. My A6000 camera cannot keep up with this pace and I get fewer images than CAM triggers; I need to keep above 2s for continuous shooting without dropping images. If I reduce the flying speed my mission time increases beyond the capacity of my battery. A balance needs to be struck.

 I have tested PS derived DEM vs differential GPS and with a 60% overlap and 70% side overlap I have achieved mean 4.3cm elevation error on a DEM. That's not bad at all.

 What are the experiences of other users? What overlaps do you specify and what works for you? Do you increase altitude or drop speed for instance? Change focal length (I shoot 35mm)?

Cheers

 

30

General / Re: Sony Arw files not recognised

« on: October 01, 2015, 06:09:12 PM »

Thanks James, that's useful.