lgraz
/

mixtral-offloading

Text Generation

Transformers

Safetensors

5 languages

mixtral

text-generation-inference

conversational

Inference Endpoints

Model card Files Files and versions Community

Edit model card

Attention quantization: HQQ 4-bit, groupsize 64, compress zero, compress scale with groupsize 256
Experts quantization: HQQ 2-bit, groupsize 16, compress zero, compress scale with groupsize 128

Downloads last month: 5

Inference Examples

Text Generation

Inference API (serverless) is not available, repository is disabled.