Forum

Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.


Messages - james_AU

Pages: [1] 2
1
General / Re: Agisoft Metashape 1.5.0 pre-release
« on: January 18, 2019, 10:58:10 AM »


Hello James,

There might be a problem with the new meshing method started in the cloud solution, we'll try to fix it as soon as possible. Currently looks like the arguments of the task are not properly interpreted by the cloud and it still tries to build the mesh from the dense cloud, while there's no cloud ("Null dense cloud" error is raised).

Hi Alexey,

Indeed, there does seem to be a problem with the arguments.

I changed the metadata, using the line of code provided. However, I think it exceeded the storage quota (I received a mail to that effect). What is a little strange is that it didn't save the file.  I have exceeded the quota before by 3-7GB before, but in those cases the file was still saved in its entirety.  I think it is a good feature that one is able to exceed the storage capacity temporarily, and if necessary pay for that excess. However, in this case, that did not happen. I am running mesh generation from the depth maps again to see if the same thing happens.

Regards,

James
[/quote]

Hello James,

As for "exceeded the quota" I suppose it was a problem with our email engine, recently it resent some messages that had already been sent.
[/quote]

Hi Zakhar & Alexey,

Interesting about the email engine Zakhar. The process has just failed again with the 'null dense cloud' error when trying to run reusing the depth maps. Therefore, Alexey's conclusion holds.

Regards,

James

2
General / Re: Agisoft Metashape 1.5.0 pre-release
« on: January 18, 2019, 10:35:47 AM »

[/quote]

Hello James,

There might be a problem with the new meshing method started in the cloud solution, we'll try to fix it as soon as possible. Currently looks like the arguments of the task are not properly interpreted by the cloud and it still tries to build the mesh from the dense cloud, while there's no cloud ("Null dense cloud" error is raised).
[/quote]

Hi Alexey,

Indeed, there does seem to be a problem with the arguments.

I changed the metadata, using the line of code provided. However, I think it exceeded the storage quota (I received a mail to that effect). What is a little strange is that it didn't save the file.  I have exceeded the quota before by 3-7GB before, but in those cases the file was still saved in its entirety.  I think it is a good feature that one is able to exceed the storage capacity temporarily, and if necessary pay for that excess. However, in this case, that did not happen. I am running mesh generation from the depth maps again to see if the same thing happens.

Regards,

James

3
General / Re: Agisoft Metashape 1.5.0 pre-release
« on: January 17, 2019, 02:34:33 PM »
Hello James,

The workaround with Python would work for any depth maps filtering option, it just changes the meta information for the available depth maps to "Mild".

As for the "null dense cloud" problem, what were the processing parameters of the task that you have sent for processing in the cloud?

Hi Alexey,

The processing parameters for the log previously supplied were according the attached screenshot. The process was then started and left to its own devices. When I synced the project, a set of depth maps with aggressive filtering were delivered back. There was not explanation why. I then tried to run the process again, just as you see, but with the reuse depth maps option checked. It was then that the "null dense cloud" error message was returned.

As you have pointed out, the program will have of course become confused, given the default settings in operation as you have previously outlined.

Regards,

James

P.S. Update. I have run the project with the default settings for visibility consistent mesh generation. The process still failed. Depth maps were generated in 1hr33min, with aggressive filtering, despite this not being specified. The project has then synced to the local machine, with no error message. I will now try specifying the set of  depth maps using python.

4
General / Re: Agisoft Metashape 1.5.0 pre-release
« on: January 17, 2019, 01:20:16 PM »
Hello James,

Currently depth maps based mesh generation option is creating the depth maps with the Mild Filtering by default and requires it on input, so depth maps calculated with the different filtering option cannot be re-used for this method (at least in the version 1.5.0). There's, however, a workaround how to make it possible to re-use aggressive and moderate depth maps - it's necessary to modify the meta information of the depth maps instance in the chunk, for example, using Python console:
Code: [Select]
Metashape.app.document.chunk.depth_maps.meta["depth/depth_filter_mode"] = "0"After that the available depth maps could be re-used. However, the Mild option tends to produce better results in most projects.

In the version 1.5.0 the depth maps generation procedure has been revised and optimized, so you shouldn't expect noise for mild filtering depth maps. Moreover, new meshing method should be good for internal noise suppression.

I can try to compare the processing time for some rock-art / stone carving projects that we have for dense cloud and depth maps based meshing methods (both for mild and aggressive filtering) and provide the comparison chart. the new meshing is exploiting GPU for reconstruction, so it can be accelerated with the good discrete GPU cards, whereas the dense cloud reconstruction (especially dense cloud filtering part) time is quite dependent on the image overlap, while the old meshing method could be very RAM demanding for large datasets (hundreds of photos) in High / Ultra quality.

As for the issues that you have been observing with the cloud, could you specify, when does it happen?

Hi Alexey,

Thanks for the swift response. Perhaps it wasn't so clear in my message, but what I would like to do is to disable filtering. Just to reiterate, with well exposed, focused images, taken in strong light, I am able to process without any filtering because there is no noise evident in the dense surface reconstruction (though I note what you say and have observed myself that version 1.5 has revisions and optimisations in this regard).

Therefore, if I have generated the depth maps via the usual procedure without depth filtering, should the workaround in python work for depth maps generated without filtering too?

 In the meantime, to get an idea of the output I have set EC2 to work again with mild filtering, but not as a batch process.

As to when the error with the cloud happens... see the attached log. I think I forgot to attach it last time.

Regards,

James

5
General / Re: Small scale DEM
« on: January 17, 2019, 11:25:03 AM »
I have a rock outcrop about 2 square meters that I'd to use to generate like to generate a DEM with contours. I've gone through the steps of creating the 3D model, but the DEM option isn't available I assume because I don't have any GPS data. I did use markers and they have been detected on my model. Is there a way to make a DEM using the position of the markers as reference points?

Hi,

Yes. Providing you know the distance between some, if not all of the markers, and if at least 3 are at the same elevation, you can give them false coordinates, set an appropriate coordinate system and update the model.  Remember to correct ray projections and set rotate the bounding box before generating the DEM. I do it all the time now and it works a treat.

Regards,

James

6
General / Re: Agisoft Metashape 1.5.0 pre-release
« on: January 17, 2019, 11:20:57 AM »
Hi Alexey et al.,

I would like to make a point regarding visibility consistent mesh generation option. I have started running a couple of experiments using Agisoft Cloud.

There does not seem to be an option visible in my build (7492, Mac OS X) to set the depth filtering. When running as a non-batch process, interpolation and depth filtering options appear greyed out. Are they locked? I am not sure, as when I attempted to run mesh generation as a batch process from depth maps, the process failed (355 cameras, planar surface, highest quality). Depth maps were saved (see log: 'downloading 3.xxxGB'), but they came out with aggressive filtering. The reason for failure was not given, but when attempting to run the process again, the error 'null dense cloud' was returned. Obviously something has gone awry somewhere along the road....

Given I work with rock art, and the best images that I capture have little or no noise, it is important that small details are not discarded as outliers, which I have observed can occur when using filtering (smoothing options). There is also the additional benefit of reduced computational cost. Therefore, it is for these reasons I ask about filtering options that many do not completely disable as a matter of course. All this you may of course know already.

I can see the benefits of visibility mesh generation, and Diana Ovod  has previously recommended that I try out the method. Unfortunately, due to reasons beyond my control, I have not had the chance to test it on HPC until the launch of Agisoft Cloud. I am particularly interested if the method reduces RAM demands (although we unfortunately can't see performance readouts from EC2), which can be a problem for me, and hence Diana's recommendation.  There is also the time saving element through full exploitation of parallelism. I know there are some of our colleagues saying things are now taking longer, but lets see...

Regards,

James

7
General / Re: Agisoft Metashape 1.5.0 pre-release
« on: January 04, 2019, 01:37:23 PM »
HI DriesS
Seems they are there... just in a very grey font (if you have  eye saver mode activated on your screen, then you will see the data labels clearly.
Something to do with display settings I imagine
Regards,
James

8
General / Exporting large orthomosaics
« on: November 30, 2018, 12:25:18 PM »
Dear all,

After having searched the forum for a similar query and having asked a few colleagues, I would be interested to know how one works around the following problem.

For the case of large models of around 600+ cameras, when trying to view an exported orthomosaic in JPEG or TIFF (source cameras are in JPEG) in an image viewing programme, including Preview  and Photoshop (on Mac), I find that whilst I can view the exported orthomosaic from the global viewpoint (ie the whole surface), I can not zoom in to inspect details. A blank screen is displayed instead.

There are no problems viewing the orthomosaic in the  Metashape / Photoscan GUI itself.

As I am increasingly processing larger and larger models, this is becoming more of an issue than has been previously the case.

Is there a reason for this? Is there a solution? I must surely not be the only one who has encountered this...?

Kind regards,


James

9
General / Re: Agisoft Metashape 1.5.0 pre-release
« on: November 25, 2018, 11:38:04 PM »
I had some time to test a few datasets, mainly captures by UAV's and Metashape really has a problem in dense clouds. Usually they are a lot poor than the ones produced in version 1.44. Even with the filtering disabled, lots of areas disappear from the cloud leaving huge holes. This problem is common mainly in areas with vegetation or more complex terrain. The last version usually solves these areas quite well. Metashape tends to eliminate them, apparently with excessive filtering. It's not a case of a single dataset or special cases. Its consistent whenever there's terrain and vegetation. The tied model improves things a bit because these areas get solved. However there are a few things that still need a dense cloud to be generated (for instance point classification). Meshes from dense clouds are also more detailed, at the present stage, than meshes produced using tiled model. Probably things will improve if some bugs in high quality tiled model generation are solved and a DEM can be generated from it.
Cheers

Hi RHenriques & Alexey

Interesting. I have been rereading the post on this issue as well as the replies. I have also noticed more holes in the final result. However, for me these emerge at a scale that is less problematic for my purposes. However, I think Alexey's observation is very pertinent: there was more noise in those areas. This lowers the statistical confidence of the matches, resulting in rejection during the alignment stage. The lack of sufficient key and tie points is then carried through into the dense cloud

Agisoft have stated in another post (http://www.agisoft.com/forum/index.php?topic=89.0), the workflow is similar to SIFT, although  there have been changes since 2011. However, SIFT only considers matches to be true if they pass a very high level of confidence. I suspect that, as the software has developed, the threshold has been successively tightened (and why not) and further improvements have been implemented to improve precision. Personally, working with depth differences in rock art regularly as little as 1mm, I value that focus on precision.  If we are not as sure as we thought we before: ok. Then we need to reevaluate the quality of our pictures and/or look critically on why this might be the case, particularly surfaces with a very uniform texture.

Regards

James

10
General / Re: Agisoft Metashape 1.5.0 pre-release
« on: November 22, 2018, 02:58:19 PM »
Hi Alexey / Zakhar,


I have tried to set in motion a batch job consisting of a dense cloud on ultra high quality and a DEM. It will take perhaps 20 hours to run. Fine.  However, after having disconnected from the service to leave it to get on with it, when I try check back on the progress of the result, the following error is returned. I left the same process to run overnight and the same error message was returned this morning. The account status page said the job was still in progress. Perhaps I should leave it until the job has finished / a long period of time has elapsed (at least a day and a half). Or perhaps it is another bug?

Update: the processes successfully completed and the model is now downloading... so whatever the error message refers to, the system is robust enough to survive. However, for a model at dense cloud + DEM of 46 GB, I am going to need some more memory to go to the next level. Files tend to become 3 times as large after a mesh is created. The model is of one of Denmark's most significant rock art sites that was recently discovered on Bornholm.

Kind regards,

James

11
General / Re: Agisoft Metashape 1.5.0 pre-release
« on: November 20, 2018, 11:01:01 AM »
Hi Alexey / Zakhar,

Thanks for all the fixes in the new release! All seems to be working!

Kind regards,

James

12
General / Re: Agisoft Metashape 1.5.0 pre-release
« on: November 19, 2018, 01:50:22 PM »
Tried opening up to download a completed project in the cloud... and encountered the error message attached. Is it correct that Metashape needs to use OpenSSL for cloud processing?
Hello James,

Than you for reporting.

We will publish the pre-release update in a few days that would properly work with default Mac OS X installation without a need of OpenSSL. Sorry for the inconvenience.

Hi Alexey,

Whilst I do not wish to sound impatient... I was wondering if you could give an update on the progress of the forthcoming update to fix this particular issue?

Kind regards,

James

13
General / Re: Agisoft Metashape 1.5.0 pre-release
« on: November 13, 2018, 11:07:26 PM »
Hi Alexey & Zakhar,

Just downloaded the new installer for Metashape... including the bug fixes  :) :).

Tried opening up to download a completed project in the cloud... and encountered the error message attached. Is it correct that Metashape needs to use OpenSSL for cloud processing?

I see that Apple have stopped shipping OpenSSL around 2015. Therefore, I have tried installing it manually. The correct results are returned in Terminal, but the same error message is returned in Metashape. I tried deactivating Metashape, deleting the old copy and emptying the trash, followed by restarting, downloading Metashape and reactivating it. I was not successful. So before I try again installing Open SSL, I thought I should check back with you guys.

Like an idiot, I didn't take a copy of the previous installer, in order to carry on working. But I guessed any problems arising will be solved, given how admirably quick your response times are.

Regards,


James

Update: According to terminal, I have managed to install and can run OpenSSL 1.0.2. Could I hazard a  guess that Metashape is not searching the "usual" places when it checks for version?

14
General / Re: Agisoft Metashape 1.5.0 pre-release
« on: November 13, 2018, 12:08:59 PM »
Hi Zakhar,

Another small problem. See attached screenshot of error message. The model has finished processing according to the agisoft cloud account page, but when trying to download the completed product, all I get so far is the attached error message.

How does one go about retrieving the project in such a circumstance? (though I note your previous comment that Agisoft are working on the interface).

Update: after a couple of hours, it has finally worked. Is there a manual way of forcing the action, or is one simply required to wait patiently.?

I note that Agisoft have transferred me over to more powerful server instances. Great!  :) Thank you. The difference of additional RAM and another GPU is already noticeable.

Kind regards,

James

15
General / Re: Agisoft Metashape 1.5.0 pre-release
« on: November 08, 2018, 02:18:00 PM »
Hi Zakhar

I wonder if there might also be an issue with connecting to the project when switching between wireless and the wired ethernet when these mean there is a switch of IP address?

If I: start a process using the ethernet, then disconnect and close Metashape and then, at a later point in time, try to connect over the wireless network, I find that the connection times out. The project then reverts to whatever is stored locally.  One is then  unable to connect to the cloud process and sync to the file located there once it has finished. The only option left is then to delete the project and start again, and then check back online on my account page to see if the process has completed. When it is completed, one can connect the ethernet and everything syncs as it should.

Much of the traffic I generate originates from the Danish e-Infrastructure Collaboration 'researcher network'. The wireless connection is over eduroam. Could the problem be something to do with a change in the IP address of the instance? (Amazon specify that regions must be set for instances, but I am not sure if the system would discriminate between eduroam and ethernet traffic from the same institution. I have not systematically tested the issue enough to see if the same fault occurs when changing from working from home and / or another business premises.

And another thing....the issue has already been mentioned here of 'status bad_alloc'. I appreciate that the models with a large number of cameras and with strong geometry pose the greatest demands on the memory in the model creation stage. In the forthcoming payment plans, will be be able to run on the higher capacity G3 instances.

An alternative to this issue would be to able to move the light source in the dense cloud classes mode. Agisoft oc3 format can only be opened in a limited number of applications, but the speed of Metashape's ability to handle very large point clouds is admirable. It is a pity that we cannot harness that potential a little more. With the ability to create bigger and bigger models, this will become a more important issue over time.

Regards,

James

Pages: [1] 2