Failed to submit models, request logs to check the reason for failure.

#684
by TwT-6 - opened

I have diligently filled out all the necessary information for model submission as per the five requirements. Although my submission successfully entered the queue, it disappeared without undergoing proper validation. I have thoroughly rechecked each requirement and could not identify any issues.
Therefore, I humbly request access to the failure logs for further investigation. The name of my models are TwT-6/open_llm_leaderboard_demo & TwT-6/open_llm_leaderboard_demo2.
Your kind assistance in this matter is greatly appreciated.

Open LLM Leaderboard org

Hi @TwT-6 !

Could you please provide me with the request files for these two models?

I'm sorry! I'm not sure what you mean by 'request files'. Could you please provide more information? Thanks!

Open LLM Leaderboard org

Hi @TwT-6 ,

According to the FAQ section, you can find community queries and running status in the requests Hugging Face dataset.

Here is the folder with your requests โ€“ link.

I see that your "TwT-6/open_llm_leaderboard_demo2" model was submitted with two different types of precision (bfloat16 and float16) and was successfully evaluated, you can find them on the leaderboard (see my screenshot)
Screenshot 2024-04-18 at 18.59.17.png

Considering the "TwT-6/open_llm_leaderboard_demo" model, the bfloat16 version has been successfully evaluated and is available on the leaderboard, but there is a problem with this model in float16 precision.

I have resubmitted this model, hopefully the evaluation will now go well. I will close this issue, please reopen it / write here in case of any problems with these models

alozowski changed discussion status to closed

Sign up or log in to comment