Hello bgreenstone,
Calculating the point confidence doesn't improve or affect the processing results, unless manual editing is applied.
The processing time also wouldn't be extended, however, the size of the dense cloud data in the project directory would be bigger, if the confidence values are calculated. So you can skip confidence calculation, if you do not consider it in your pipeline.