ollama

#2
by goodasdgood - opened

The model is very slow in ollama
not slow with lama cpp

@goodasdgood I don't use ollama, but ensure you have the right prompt format:

FROM your_model_name.gguf
TEMPLATE """
<|start_header_id|>system<|end_header_id|>
{{ .System }}<|eot_id|><|start_header_id|>user<|end_header_id|>
{{ .Prompt }}<|eot_id|><|start_header_id|>assistant<|end_header_id|>
{{ .Response }}<|eot_id|>
"""
SYSTEM ""

add in rest of parameters as necessary.

Orenguteng changed discussion status to closed
Orenguteng changed discussion status to open

sloooooooow on ollama

it ius run good
thank you

Orenguteng changed discussion status to closed

Sign up or log in to comment