output of HMC and VI,

[aazz7777]

Hello, Will,

When I run code using VI, and HMC, it show totally opposite casual impact VI - for most of sample, and HMC is +, Would anyone can help me to understand why?

[WillianFuks]

Hi @aazz7777 ,

This is expected indeed. The VI algorithm is mainly used for quick experimentation. Once you have confirmed that the model is working then it's recommended to use the HMC model which is slower but precise (and stable).

[aazz7777]

Thank you for the answer. Also, when I use the HMC , the impact turns out to be totally opposite compare to the case when I run the same data in R casual impact package. For example, if R said (-) impact, HMC returns (+) for many ample cases. I was expecting similar output since HMC is a special variant of MCMC. I'm wondering how I interpreted or handled it. because we can't say one is more accurate. and also, the probability is really difference. Python said not significant, and R said significant... I'm running many test cases.

[WillianFuks]

This is unexpected indeed. Is it possible to share a transformed dataset for experimenting on my side? Both packages should converge to same conclusions.

[aazz7777]

I don't have authority to the data so I'm not sure I can share the dataset... but I can show the output of probability and impact of with R, VI, and HMC.

[aazz7777]

This is sample output in R casual impact and your VI and HMC result. It can lead pretty different results when it comes to business.. some test results are similar.and many are different
when I test I set both 0.05 I saw your package can't change alpha. since it only affect CI.

[WillianFuks]

Hi @aazz7777 ,

Results from VI are indeed less stable. The ones that should be considered are the ones from HMC

[aazz7777]

Thank you for the commend, another my question is that case 1: R -8 (negative impact) 4.72 is positive impact Case2: R : negative impact -9, HMC positive impact 2.26 but one said significant, another said not significant. Case 3, both negative impact, R said significant, another said HMC said not significant. if this is the case, which one need to be chosen as recommendation ?? for example, both said negative impact, but one said significant, another said not significant?