rttl commited on
Commit
e67babf
1 Parent(s): 1efde63

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -7
README.md CHANGED
@@ -12,16 +12,15 @@ tags:
12
  # rttl-ai/BIOptimus v.0.4
13
 
14
  ## Model Details
15
- **Model Description:** BIOptimus v.0.4 model pretrained on [PubMed](https://pubmed.ncbi.nlm.nih.gov/) abstracts.
16
- It is introduced in the paper BIOptimus: Pre-training an Optimal Biomedical Language Model with Curriculum Learning for Named Entity Recognition (BioNLP workshop @ ACL 2023). More information is available in [this repository](https://github.com/rttl-ai/BIOptimus).
 
17
 
18
- This model achieves state-of-the-art performance on several biomedical NER datasets from [BLURB](https://microsoft.github.io/BLURB/).
19
 
20
  - **Developed by:** rttl-ai
21
- - **Model Type:** Text Classification
22
  - **Language(s):** English
23
  - **License:** Apache-2.0
24
  - **Resources for more information:**
25
- - The model was pre-trained with task-adaptive pre-training [TAPT](https://arxiv.org/pdf/2004.10964.pdf) with an increased masking rate, no corruption strategy, and using WWM, following [this paper](https://aclanthology.org/2023.eacl-main.217.pdf)
26
- - fine-tuned on sst with subtrees
27
- - fine-tuned on sst2
 
12
  # rttl-ai/BIOptimus v.0.4
13
 
14
  ## Model Details
15
+ **Model Description:** BIOptimus v.0.4 model is a BERT-like model pre-trained on [PubMed](https://pubmed.ncbi.nlm.nih.gov/) abstracts.
16
+ It is a biomedical language model pre-trained using contextualized weight distillation and Curriculum Learning.
17
+ This model achieves state-of-the-art performance on several biomedical NER datasets from [BLURB benchmark](https://microsoft.github.io/BLURB/).
18
 
 
19
 
20
  - **Developed by:** rttl-ai
21
+ - **Model Type:** Language model
22
  - **Language(s):** English
23
  - **License:** Apache-2.0
24
  - **Resources for more information:**
25
+ - It is introduced in the paper BIOptimus: Pre-training an Optimal Biomedical Language Model with Curriculum Learning for Named Entity Recognition (BioNLP workshop @ ACL 2023).
26
+ - More information is available in [this repository](https://github.com/rttl-ai/BIOptimus).