Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

about loss_in and loss_out in paper #149

Open
hweejuni opened this issue Apr 30, 2024 · 1 comment
Open

about loss_in and loss_out in paper #149

hweejuni opened this issue Apr 30, 2024 · 1 comment

Comments

@hweejuni
Copy link

Hello. Thank you for your hard-working!
In your paper page 5, you stated that
'The two loss formulations are not, however, equivalent. Because log is a concave function, Jensen’s Inequality [23] implies that L_in ≤ L_out. One would thus expect L_out to be in the superior supervised loss function'
this might be a silly question, but I am wondering that why L_in ≤ L_out indicates loss_out is superior loss function ? In your paper page 6, you showed loss_out is more stable for traning and I understood it. But I can't connect this idea with being the superior loss function.

@HobbitLong
Copy link
Owner

Hi, thanks for the question. Actually the correct loss is actually a simple cross-entropy. Hope this makes it easier to understand

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants