Create README.md

README.md

@@ -0,0 +1,37 @@
+---
+license: cc-by-nc-4.0
+tags:
+- not-for-all-audiences
+- nsfw
+---
+## Description
+
+Exllama 2 quant of [NeverSleep/Echidna-13b-v0.3](https://huggingface.co/NeverSleep/Echidna-13b-v0.3)
+
+8 BPW, Head bit set to 8
+
+## VRAM
+My VRAM usage with 13B models are:
+| Bits per weight  | Context | VRAM  |
+|--|--|--|
+| 8bpw | 8k | 22gb | 
+| 8bpw | 4k | 19gb | 
+| 6bpw | 8k | 19gb | 
+| 6bpw | 4k | 16gb | 
+| 4bpw | 8k | 16gb | 
+| 4bpw | 4k | 13gb | 
+| 3bpw | 8k | 15gb | 
+| 3bpw | 4k | 12gb | 
+I have rounded up, these arent exact numbers, this is also on a windows machine, they should be slightly lower on linux.
+
+## Prompt template: Alpaca
+
+```
+Below is an instruction that describes a task. Write a response that appropriately completes the request.
+
+### Instruction:
+{prompt}
+
+### Response:
+
+```