Questions about quantitative results for upsampling

[morsingher]

Hi, thanks for open-sourcing your work. I have a question about quantitative results in table 2.

As reported in section 4.3, I assume you have re-trained each network to obtain those results. However, the metrics are significantly different than the original ones reported in their respective papers. In particular, CD is 1-2 orders of magnitude smaller (you report a scale of 10^-5), with HD and P2F being even worse than the original ones.

It is also true that HD and P2F computation is known to be wrong in most previous works, so it might be possible that you finally got the correct numbers for all those networks. Would you mind sharing the evaluation code you used to compute those metrics?

Thank you in advance.

[WanquanF]

There shouldn't be such large difference, can you provide your evaluation script (send to my email: [email protected])?

[morsingher]

Hi, thanks for the quick answer. The difference is with the values reported in papers such as PU-GCN and PU-GAN. The evaluation script I'm using is from the PU-GCN official repo: https://github.com/guochengqian/PU-GCN/blob/master/evaluate.py

Apparently, that script computes a "wrong" version of HD and P2F (see the issue here: guochengqian/PU-GCN#10), but still the CD values should be ok and you report 1-2 orders of magnitude smaller. Could you please upload to this repo your evaluation script?