The HPC has high-speed file system access from every node. Waaay faster than my workstation (like between 15GB/s and 150GB/s depending on cache, with very large cache). Would it help troubleshoot if I respawn the processing node with --verbose?
Also how far back doesit fail to if I pause then kill the worker node? I had one die (expire) and a newer one kicked in about a day ago, but it looked like I lost some progress. It looks like I'll find out in about four hours when this node expires :-/ - I'll turn verbose logging on for the next node I spawn.