Big Tiger Gemma 27B v1 - EXL2 5.6bpw

This quant was made using exllamav2-0.1.8 with default dataset.

I tested this quant shortly in some random RPs and a few assistant-type tasks and it seems to work fine.

This quant fits nicely in 24GB VRAM on Windows with no KV cache quantization (using standard 8k context length)

Prompt Templates

Seems to use Gemma 2 format.

Decensored Gemma 27B. No refusals so far (other than some rare instances from 9B). No apparent brain damage.

In memory of Tiger (the happy street cat on the right)