julien-c HF staff commited on
Commit
aa14b80
1 Parent(s): 0fef132

Migrate model card from transformers-repo

Browse files

Read announcement at https://discuss.huggingface.co/t/announcement-all-model-cards-will-be-migrated-to-hf-co-model-repos/2755
Original file history: https://github.com/huggingface/transformers/commits/master/model_cards/microsoft/codebert-base/README.md

Files changed (1) hide show
  1. README.md +27 -0
README.md ADDED
@@ -0,0 +1,27 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ## CodeBERT-base
2
+ Pretrained weights for [CodeBERT: A Pre-Trained Model for Programming and Natural Languages](https://arxiv.org/abs/2002.08155).
3
+
4
+ ### Training Data
5
+ The model is trained on bi-modal data (documents & code) of [CodeSearchNet](https://github.com/github/CodeSearchNet)
6
+
7
+ ### Training Objective
8
+ This model is initialized with Roberta-base and trained with MLM+RTD objective (cf. the paper).
9
+
10
+ ### Usage
11
+ Please see [the official repository](https://github.com/microsoft/CodeBERT) for scripts that support "code search" and "code-to-document generation".
12
+
13
+ ### Reference
14
+ 1. [CodeBERT trained with Masked LM objective](https://huggingface.co/microsoft/codebert-base-mlm) (suitable for code completion)
15
+ 2. 🤗 [Hugging Face's CodeBERTa](https://huggingface.co/huggingface/CodeBERTa-small-v1) (small size, 6 layers)
16
+
17
+ ### Citation
18
+ ```bibtex
19
+ @misc{feng2020codebert,
20
+ title={CodeBERT: A Pre-Trained Model for Programming and Natural Languages},
21
+ author={Zhangyin Feng and Daya Guo and Duyu Tang and Nan Duan and Xiaocheng Feng and Ming Gong and Linjun Shou and Bing Qin and Ting Liu and Daxin Jiang and Ming Zhou},
22
+ year={2020},
23
+ eprint={2002.08155},
24
+ archivePrefix={arXiv},
25
+ primaryClass={cs.CL}
26
+ }
27
+ ```