Future directions regarding optimization

[x02Sylvie]

I'm kinda curious how much more we can squeeze out of llama on cpu, what other directions can we take.

Just casual topic for brainstorming

[MillionthOdin16]

They're always coming up with new approaches :) it's cool to see how some small changes can have a huge impact.

[PriNova]

What I really like to operating on is the evolved RNN from this https://github.com/BlinkDL/ChatRWKV
It looks very promising in regards to "unlimited" context window.