Feedback
Tried both the L3.1 (Q8) and this version (Q5KM). L3 is about what I'd expect from a L3 fine-tune. It's fine, better than 0.1 for sure.
This one, however... It's kinda weird. Vanilla Nemo is pretty much unaligned/uncensored as it is. While it's very bad at using asterisks, it's not the end of the world. This version, using the exact same scene(s) for comparison's sake, does feel like a downgrade. I can recognize the usual sentences I'd expect from the datasets, and the model itself is a lot worse at keeping track of locations, moods, and such. I'm sure it has more varied ERP vocab, but it's at the cost of a lot of coherence and context understanding. On the good side, it's less prone to repetitions without having to toy with the inference settings, and a tad better at using asterisks.
Also, it's no longer following (OOC: something) prompts like the base model used to do.
It really feels like it was over-fitted (and the fact it needs an even lower temp than the base model to be coherent seems to point in this direction).
Hello, thanks for your feedback.
It really feels like it was over-fitted (and the fact it needs an even lower temp than the base model to be coherent seems to point in this direction).
We released the epoch 2.5 from the model, here is the epoch 2 : https://huggingface.co/NeverSleepHistorical/lumi-nemo-e2.0
I made it public for the people who have somewhat the same issue than you.
Maybe a little less could be enough?
I will check that anyway, thank you!
You're welcome, and thanks for the link, I'll give it a try later, for sure.
Yeah I can echo everything said above. It feels more eloquent than plain Nemo, but noticeably less coherent.
Was epoch 2 better ?