Cannot get it to stop.
Unfortunately it goes completely off the rails. I put the "Custom" ChatML instruct template into ST, as well as the custom stopping strings, the model will write and write and write and eventually only nonsense comes out. It starts using weird syntax, random ]] and ] and begins summarizing or narrating.
Turning off instruct mode makes it a little better, but responses need to be token limited and incomplete sentences trimmed. With instruct mode off it defaults to asterisk talk and refuses to put dialogue "in quotation marks." It also ignores almost all other instructions. But it's much more coherent in general.
What little coherent text I managed to tease out was amazing, both sfw and nsfw, incredible prose and variety, good pacing. But I had to artificially cut it off after ~400 tokens and have to edit out incomplete parts out constantly.
I absolutely believe I set something up wrong, it would be really helpful to get any tips from someone who got this running perfectly. Maybe a Chat Template and Instruct Template for sillytavern.
A very strange model, where the genius of the answers borders on the delirium of a schizophrenic with a touch of a tipsy stand-up comedian. After the answers, it produces a bunch of artifacts from a mixture of symbols and words in different languages, even throws out Komodoji and once wrote to me "Caution, hentai!" and gave Komodoji boobs! I have never seen anything like this in 1.5 years of working with LLM.
It feels like in every LLM answer, among all this nonsense, you find a diamond... It is very ambiguous, if the model is cured of delirium and artifacts, then it will be truly brilliant.
I am having the exact same problem with SillyTavern and have been dealing with it in the same way. It does feel kinda like something is set up wrong on my end, almost like it's generating multiple responses sequentially, but from what I can tell everything is set as it should be according to the model card's instructions. If anyone has some tips on getting it to stop properly on its own or some ST templates that work right that would be fantastic.
Some screenshots of my ST templates if anyone can tell me if I've done something wrong.
