I figured it out in the hard way, stuff like this should be in the manual, but it is scattered all over the places, and that makes it really hard to work with the software.
Here are the 2 transformations combined: (2 of them, 1 is from left to right handed, 1 for image's coordinate)
Left handed (Unity): x to right, y to up, z to in, image 's x follow x, image's y follow y (z follow z)
Xx Yx Zx Tx
Xy Yy Zy Ty
Xz Yz Zz Tz
0 0 0 1
To Right Handed (PhotoScan): x to right, y to up, z to out, image 's x follow x, image's y follow -y (z follow z)
Xx -Yx Zx Tx
Xy -Yy Zy Ty
-Xz Yz -Zz -Tz
0 0 0 1
(Minor correction -Yx, -Yy, Yz)
Hope this help others.