TheBloke

/

alpaca-lora-65B-GGML

like 89

https://www.reddit.com/r/LocalLLaMA/comments/12sqo3r/fatty_alpaca_alpacalora65b_ggml_quantised_to_4bit/

I've added the RAM requirements to the README here as well now.

As for perplexity, I'm probably not going to calculate that in full. I started the process running on the q4_0 and...
287.18 seconds per pass - ETA 52.25 hours

So yeah I don't think I'm going to be running that in full any time soon :)

But I will leave it going for a few hours and get the first few numbers for q4 and q2, so we have at least some idea.