Reflection-Llama-3.1-70B was sonnet 3.5.
Why is no one on local-llama actually runnign the f**king thing locally and posting their results?
Also feel free to apply my chat template fix before converting to gguf.
As for the threads on openrouter, that looks to my like it was the openrouter model-of-the-week api answering. My local tests on q8_0 showed this:
Can you read titles of these posts? They're talking about thees official "reflection 70b APIs". Do you test on these APIs?
I see you're posting the results related to this twitter.
Do you really understand what he is trying to prove? He's trying to prove that the LLM behind "reflection 70b API" is using the same tokenizer as claude 3, chatgpt4o or whatever. Images he posted stands by his point.
What are you trying to prove here by posting this image? I think you're proving that what they uploaded here and what they host after API are totally different. You should explain what you want to prove in detail.
Also, I see you're using local models, so you're testing different models from all these posts claims. A natural question is that can you reproduce the evaluation results @mattshumer provided? Why not post your independent evaluation results here so you can help everyone decide whether they're genuine or overclaiming?
try it yourself. Don't be a fat sheep in the herd effect
Talk is cheap. Show me your independent evaluation results here.