LeroyDyer
/

_Spydaz_Web_AI_MistralStar_001_Project

LeroyDyer commited on Jul 20

Commit

41451ce

•

1 Parent(s): 264575b

Update configuration_mistral.py

Files changed (1) hide show

configuration_mistral.py CHANGED Viewed

@@ -117,6 +117,7 @@ class MistralConfig(PretrainedConfig):
         sliding_window=4096,
         attention_dropout=0.0,
         max_thoughts=16,
         merged_talk_heads=True,
         merged_lm_and_talk_heads=False,
         merged_lm_and_think_heads=True,

         sliding_window=4096,
         attention_dropout=0.0,
         max_thoughts=16,
+        thought_length = 10,
         merged_talk_heads=True,
         merged_lm_and_talk_heads=False,
         merged_lm_and_think_heads=True,