sparsh35 commited on
Commit
ada3848
1 Parent(s): 5619f8d

update config.json vocab_size to tokenizer length

Browse files

.i.e., 32000 as for high throughput in vllm there can be sampling of padded tokens which will result in error in vllm , it is an open issue here . https://github.com/vllm-project/vllm/issues/340

Files changed (1) hide show
  1. config.json +1 -1
config.json CHANGED
@@ -30,5 +30,5 @@
30
  "torch_dtype": "float16",
31
  "transformers_version": "4.38.2",
32
  "use_cache": false,
33
- "vocab_size": 32032
34
  }
 
30
  "torch_dtype": "float16",
31
  "transformers_version": "4.38.2",
32
  "use_cache": false,
33
+ "vocab_size": 32000
34
  }