SJ-Donald

/

SJ-SOLAR-10.7b-DPO

like 0

I tried convert it to gguf and got this error :
RuntimeError: Internal: src/sentencepiece_processor.cc(1101) [model_proto->ParseFromArray(serialized.data(), serialized.size())]

I`m sorry too late response. But I've never converted it to GGUF file before, so it's hard to give you an accurate answer now.
If you tell me which source you use to convert GGUF file, I can try it and answer it later.

@SJ-Donald Thanks, no problem - Maybe @TheBloke could help ?
To convert to gguf is using https://github.com/ggerganov/llama.cpp, but it way too much to ask if you're not using it (and I can't convert it anyway)

I check source, and i got same error.
Maybe i think, convert.py is not compatible some models, and this issue is opend. Here

@notzero GGUF Version is deployed!!! You can use SJ-Donald/SJ-SOLAR-10.7b-DPO-GGUF

@SJ-Donald Cool, but unfortunately I can only use q4 version, is it now the problem is solved (converting to gguf using llamacpp ?) ?
Thanks for your kind effort

@notzero q4_0 model is uploaded now.

@SJ-Donald Thanks I'll check it out

@SJ-Donald It seems that gguf result still have problem, it adding '<0x0A>' word in generation, maybe there are still bugs in the llama cpp convert/quantize (I had this problem also before trying yours), but thanks so much for your help

@notzero Yes, you`re right. Now llama.cpp has bug when convert to gguf file from hfft vocab type model. I reupload gguf file, check new gguf file. The bug is solved.
If you want to convert gguf file without bug like "<0x0A> bytes bug", you can use my folk repo.

@SJ-Donald Thanks, I'll check it out
Edit : Yeah its work now thanks

@SJ-Donald Is the bug in llama cpp when converting or when quantizing ?

@notzero The bug occured when convert hfft type vocab -> gguf type vocab.
You can see issue`s detail here

@SJ-Donald Thanks