The tiger models tend to go crazy on LM Studio ?
Here a screen capture after a long answer.
Use Kobold 1.7
cc @concedo
It begins fine, but after some time...
Use Kobold 1.7
cc @concedo
I will try. Thank you. 🙏👍💥
LM-Studio supports Gemma-2, but it is using an old commit. which mean it is lacking the new fixes for Gemma-2. I was running into similar issues where Gemma-2-9B would spam "\n". Kobold 1.70 fixed it 😸
And KoboldCPP has many UIs to pick from. Including the new "corpo" theme
Use Kobold 1.7
cc @concedo
I will try. Thank you. 🙏👍💥
LM-Studio supports Gemma-2, but it is using an old commit. which mean it is lacking the new fixes for Gemma-2. I was running into similar issues where Gemma-2-9B would spam "\n". Kobold 1.70 fixed it 😸
And KoboldCPP has many UIs to pick from. Including the new "corpo" theme
Thank you for the explanation. I struggle deeply when installing new AI tools because I'm just a common user, I don't know Python, and most things are difficult, that's why I use LM Studio, which is incredibly easy, you install without having to know almost anything. I'd like I could use some advanced tools like Open WebUI, Lamma cpp, and others. I will have to study them. A ton of things indeed. 🙏👍💥
Thank you for the explanation. I struggle deeply when installing new AI tools because I'm just a common user, I don't know Python, and most things are difficult, that's why I use LM Studio, which is incredibly easy, you install without having to know almost anything. I'd like I could use some advanced tools like Open WebUI, Lamma cpp, and others. I will have to study them. A ton of things indeed. 🙏👍💥
I personally dont run llama.cpp, KoboldCPP comes with quite a few benefits like ContextShift which utilizes KV cache shifting to automatically remove old tokens from context and add new ones without requiring any reprocessing. Which is exceptionally useful if you are using large context. (it's like magic)
KoboldCPP is a nice thing to learn, because most things learnt in KoboldCPP transfer over to llama.cpp just in command line instead.
An easy example is context size selection
This slider
becomes
[--contextsize [256,512,1024,2048,3072,4096,6144,8192,12288,16384,24576,32768,49152,65536,98304,131072]]
[ Command [Size Selection]]
And KoboldCPP has an explation for almost anything in the ui just by hovering over the text. If not in the UI its probably in the wiki
It seems daunting at first, but it becomes second nature really quick.
Lastly, LM-Studio should recieve an update pretty soon which will most likely fix the Gemma-2-9B issues
KOBO WON
@Dihelson
if its overwhelming don't worry, KoboldCpp presents a lot of options but if you want to just try the model all you have to do is select the model and click launch.
If you know the optimal layers for your GPU you can input them before launching otherwise it will try to guess it (We are working on a better algorithm for the auto guessing so it will guess a bit conservatively in KoboldCpp 1.70). Once the UI is loaded you can immediately begin chatting like you can in LM Studio, but we have many more options available in the UI.