Any one who use the script in the Model Card for inference purpose?
#64
by
disper84
- opened
I've tried the following script in the Model Card screenshot down below:
However, the model spits out nonsense words (down below), any thought on that?
I've tried deployed gemma-2b (not the gemma-2b-it) in Ollama, it works perfect, all responses are solid.
Don't know why this discrepancies happen.
the tokenizer you're using is not fixed correctly, try checking for PRs in the community tab here to see if someone has provided it
