Instruct mode responses seems ok, but Chat-instruct responses are not relevant.
#1
by
robert1968
- opened
Hi,
Loading in Text generation web UI - solarc-moe-10.7bx4.Q5_K_M.gguf response a non relevant answer to a simple test question in "chat-instruct" mode window.
In "instruct" mode window it seems give valid answers.
robert1968
changed discussion title from
Chat-instruct response are not ok. but Instruct one seems ok...
to Instruct mode responses seems ok, but Chat-instruct responses are not relevant.
@robert1968 the reason is probably that chat instruct uses a different prompt template i believe which probably massively lowers quality of the model. With the correct prompt template it should be much better.
what prompt template are you using and what is quantized level? (for both, one that works and one that does not work).