Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rotate fast #43

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open

rotate fast #43

wants to merge 7 commits into from

Conversation

ljleb
Copy link
Owner

@ljleb ljleb commented Nov 28, 2024

faster rotate, ~20m with alignment=1

closes #20

@ljleb
Copy link
Owner Author

ljleb commented Dec 3, 2024

I need to test 2-3 variants of this. I don't know if rotating the fourrier of conv layers is empirically better than simply rotating like the previous version was doing it.

@ljleb
Copy link
Owner Author

ljleb commented Dec 6, 2024

For the record: the fourrier approach does not work at all. Other alternatives work better.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

SDXL rotation mixes CLIP G attention layers to_q to_k to_v
1 participant