More qunt models

#1
by mtasic85 - opened

Can you upload q4_k_m and q8_0 models?

Pints.ai org

@mtasic85 at that level of quantisation, they will practically be not very useful though.

Maybe you can let me know why would you want to consider those?

I quantized both versions locally and did not have much trouble.

Sign up or log in to comment