question about projection of 3d point generated from TSDF

[Kyungdon]

Hi @andyzeng
I'm trying to use your code (tsdf2mesh.m function) to generate the 3d point cloud.
From MSR 7-scene dataset, when I project the 3d point cloud generated by your function into image domain, it shows some misalignment. It could be my mistake (I considered extrinsic pose between depth and color camera).

Have you tried your function for MSR 7-scene dataset? or any suggestion for this kind of procedure?

[andyzeng]

Hello! Yes, the function should work for the MSR 7-scene dataset. In fact, the RGB-D frames used in the demo code is actually from the redkitchen sequence from the 7-scene dataset :)

Personally, I have not tried back-projecting the 3D point cloud back onto the image domain. However, upon revisiting tsdf2mesh.m, I noticed a bug where the mesh being saved was still in voxel coordinates rather than camera coordinates. I've pushed a fix for this in the latest commit. I wonder if that will fix your problem.

[Kyungdon]

Thanks for quick reply, @andyzeng
Still, there is a few issues.

1. binary file of MSR 7-scene dataset does not have "tsdfHeader". Thus I just got the value from *.mhd, but still, it doesn't have truncMargin, etc. and your code does not use truncMargin.

2. With some modification, I could align them roughly but it still has misalignment (please see attached image file, red point cloud from tsdf, other color points from depth. you can see some misalignment at the pumpkin).
   

Do you have any advice to figure this issue out?

[andyzeng]

I see, I had misunderstood your original question. You were asking about whether the tsdf2mesh function works for the binary voxel volume files of the MSR 7-scene dataset. I haven't tried that myself, so I don't know all of the details behind setting it up.

Although, from looking at your misaligned point cloud, I'm guessing that the problem is in your camera pose, or voxel volume offset. From the description on the dataset's website, they quote "The spacings and offsets in millimeters are provided in the header file." Does your current implementation use those offsets?

[Kyungdon]

Yes, I'm using the offsets in millimeters.