Remove ` (default)` from MTEB scores caused by an MTEB bug
#3
by
tomaarsen
HF staff
- opened
Hello!
Pull Request overview
- Remove
(default)from MTEB scores
Details
Recently, a bug on MTEB caused the dataset names to be updated from e.g. "ArguAna" to include the config (usually "default", but sometimes a language) in the dataset name. This resulted in a discrepancy between the expected dataset names and the true dataset names in e.g. this model card. This PR should fix it. I've verified that it should work by copying this model card to a private repository and checking that MTEB correctly parses the updated one.
Apologies for the inconvenience. Once we've successfully put this model on MTEB, I'd be glad to help announce the model release on LI/X.
- Tom Aarsen
spacemanidol
changed pull request status to
merged