Cash grant?

[TikkunCreation]

I'd be interested in doing a $500 or $1000 cash grant to exllama

Was inspired by the a16z grants thing :)

p.s. i also suggested it to a16z here

edit: oh and my email is [email protected]

[niceblue88]

I saw this was provided by turboderp: https://ko-fi.com/turboderp and gave one-off myself.
I would also be interested in giving a monthly, although ideally with some kind of "supporters forum" where we could get some more specific interaction on exllama dev. I might also be able to help dev (but usually too busy).

[turboderp]

@TikkunCreation I'm sort of in two minds about it. I don't mind taking donations, but I'm a little wary about what expectations might come attached to a grant like that. I'm already spending as much time as I can on ExLlama, and it's tough keeping up with the number of requests, issues, suggestions etc. while I'm also trying to focus my attention on ExLlamaV2. It's been a lot of fun so far, and I've learned a bunch about machine learning, language models and compute, and I kind of want to keep that going without turning it all into too much of an obligation.

As for V2, incidentally, a very early pre-release should be right around the corner now. I'm running 70B on a single 24 GB GPU now, and while it can go a little crazy sometimes, it is mostly quite coherent:

So yeah, a lot happening and hundreds of hours in it is very encouraging and I guess motivating when I get the occasional alert from kofi. But up to a point, I guess is what I'm saying. Make of that what you will. :)

@niceblue88 I don't know about a supporters' forum, mostly because it'd be work setting that up and managing it, all distractions from the real goal, which is, at the moment, kernel optimization. But I am sometimes active over on TheBloke's Discord server.

[niceblue88]

completely understand! as you know, you have many many thankful and grateful people hanging onto everything you have been doing, and are doing. I love your work too. I will endeavour to read your python and CUDA as much as possible so that I can be helpful to you and the exllama community at some point. I didnt realise you were working on a V2 until now ... will see if you have written about this elsewhere. if not, keen to know what was the step change in implementation that meant you decided to break into a new version?

[1aienthusiast]

@turboderp these are really awesome projects you're working on, make sure you pick a worthwhile license for your code, I'd personally recommend AGPL/GPL/LGPL depending on your preferences

[turboderp]

Yeah, I'm struggling with this. I want the license to be quite permissive, and I'm not against commercial usage either. But I guess I'd be sad to see someone turn it into a commercial product without involving me, if the license is too permissive. Then again I don't really care that much. But idk. (shrug emoji)

[1aienthusiast]

AFAIK all GPL licenses allow commercial use, but any code changes must be published if its being distributed (AGPL is the best in this regard since it also considers network distribution as distribution), you could also take a look at the EUPL (from what i know, its similar to LGPL).