Masking is used to tell PhotoScan which parts of the photographs to ignore. This is most important when the background of your object changes (e.g., if it is on a rotating platform) or if you have things moving in the photographs that would mess up your 3D processing (e.g., the shoes or shadows of the photographer who walks around an object on the ground). Often, it makes sense to mask the background even if it is static - in that case, the background would not disturb the 3D processing but simply become part of the generated model, but it may be easier to mask the background in the photographs than to delete superfluous parts of the final model.