Issues with ROPE
It seems 3.1 has issues with llama.cpp and ROPE that has been fixed just recently today. I saw that the new changes does not work in LM studio among others, I will add the old files again to the repo until these changes have been updated to the corresponding softwares.
EDIT:
This repo now contains
1: New GGUF's with rope fix with the latest llama.cpp (that has no support for backward compatibility, meaning if you run LM Studio or other software, they must update ot the latest llama.cpp)
2: Previous GGUF with older llama.cpp for backwards compatibility until whatever software you use is updated to the latest llama.cpp
That's weird, is llama.cpp usually not backwards compatible or just this version?
Thanks again for updating the model to use 3.1!!!
I'm running Ollama in a Docker container using the latest update from today. I attempted to use the new GGUF with the ROPE fix but encountered issues. Switching back to the old GGUF resolved the problem, and it's working well.
it's not uncesored... i'm having the same stupid restrictions as the normal llama 3.1 8b...
@seshodev That's incorrect. You are not providing any information about your prompt, what inference method you are using or what prompt format you are using whatsoever that can help me see where the issue lies.
1: Make sure you are including the prompt format as instructed on the model page, including the system tokens that must always be present.
2: Since it is retaining the original 3.1 instruct intelligence without lobotimizing it and making it dumb like other "uncensored" models, there might be some slight refusals sometimes, you can simply bypass these with a simple system message, or even just follow up the prompt with something like "Thanks for the tips. Now answer the question." it will continue.
An even better version will be released soon.