Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add methods like logLik() etc. for *multinom_weightit* and *ordinal_weightit*? #70

Open
strengejacke opened this issue Sep 2, 2024 · 1 comment

Comments

@strengejacke
Copy link

We started implementing support the WeightIt package in our easystats-eco-system, mainly in insight (https://cran.r-project.org/package=insight) and parameters (https://cran.r-project.org/package=parameters, update to be submitted).

I'd also like to provide support in the performance package (https://cran.r-project.org/package=performance). While this works well for glm_weightit(), the multinom and ordinal regressions don't provide methods like logLik() or AIC() etc. Do you plan to add those methods to the models, too?

Another question would be how the weighting affects R2 measures - if the log-lik is available for, say, multinom, can you use the same pseudo R2 measures (Nagelkerke, Cox & Snell) for multinom_weightit or is it more complicated?

@strengejacke strengejacke changed the title Add methods like logLik() etc. for _multinom_weightit_ and _ordinal_weightit_? Add methods like logLik() etc. for *multinom_weightit* and *ordinal_weightit*? Sep 2, 2024
@ngreifer
Copy link
Owner

ngreifer commented Sep 2, 2024

Hi Daniel,

Thanks for adding support for WeightIt's modeling functions! I'll look into adding logLik() and related methods. The reason I didn't implement them already is that I don't know exactly how likelihoods and related statistics are computed when the model is weighted. I wanted to discourage users from doing LR tests or comparing models based on AIC/BIC because I'm not sure that they work with weighted models. Of course, these functions support unweighted models, too, and there is no reason to omit these functions for such cases. Also, the models are fit using weighted MLE, so it is definitely possible to compute a weighted likelihood. I also wanted these models to be primarily used for outcome models in the context of causal effect estimation rather than general modeling functions, so I wasn't too concerned with providing statistics that summarize the whole model beyond the treatment effect. See #59 for my reasoning.

I'll work on this in the next few weeks. If you find any interesting resources that explain these issues, I would appreciate that. I think functions in the survey package have similar challenges, so I'll read up more on how those are addressed. I don't think the same amount of research has gone into PS-weighted models, though.

Noah

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants