Q2_K_L vs IQ3_M shows favorable results for Q2_K_L
#1
by
anxcat
- opened
Hey man just sharing this from /lmg/ since you said you're looking for user reports about whether L quants are actually worth it
(This is my post and I was talking about this model)
Basically I'm finding Q2_K_L is as good as IQ3_M while being 50% faster, so it seems like a free lunch, at least for this model
Oo that's quite an interesting one. It does make sense that the lower you go the more having a high fidelity embed/output could affect the quality, thanks for the report!