pipeline_tag: text-generation | |
tags: | |
- Neuron | |
- Optimum-Neuron | |
- AWS | |
Compiled lmsys/vicuna-7b-v1.5 using optimum-neuron | |
```shell | |
ptimum-cli export neuron --model lmsys/vicuna-7b-v1.5 --batch_size 1 --sequence_length 1024 --num_cores 2 --auto_cast_type fp16 ./models/lmsys/vicuna-7b-v1.5 | |