I figured the mask format out by exporting the existing ones.
And as for the script, I think below may do the work.
new_chunk = PhotoScan.Chunk()
p = PhotoScan.Camera()
p.open("/01_1.jpg")
m = PhotoScan.Image()
m.load("/01_1_mask.png")
p.setMask(m)
new_chunk.cameras.add(p)
new_chunk.matchPhotos(filter_mask = True)
Although I am not sure whether this is a standard way to do that.
Wish there were some samples about this.