Update README.md

README.md

@@ -23,10 +23,13 @@ This is the sixth in a series of models designed to replicate the prose quality
 Model has been Instruct tuned with the Mistral formatting. A typical input would look like this:
 
 ```py
-"""[INST] Hi there! [/INST]Nice to meet you!</s>[INST] Can I ask a question? [/INST]
-"""
+<s>[INST] SYSTEM MESSAGE\nUSER MESSAGE[/INST] ASSISTANT MESSAGE</s>[INST] USER MESSAGE[/INST]
 ```
 
+We also provide the appropriate SillyTavern presets for [Context](https://huggingface.co/anthracite-org/Magnum-123b-v1/resolve/main/Magnum-Mistral-Context.json) and [Instruct](https://huggingface.co/anthracite-org/Magnum-123b-v1/raw/main/Magnum-Mistral-Instruct.json) respectively.
+
+The default Mistral preset included in SillyTavern seems to be misconfigured by default, so please keep these presets in mind.
+
 ## Credits
 - Stheno dataset (filtered)
 - [anthracite-org/kalo-opus-instruct-22k-no-refusal](anthracite-org/kalo-opus-instruct-22k-no-refusal)
@@ -42,9 +45,8 @@ In addition to this, we noticed that Mistral Large models seemed much more sensi
 ![image/png](https://cdn-uploads.huggingface.co/production/uploads/6491e00e057b0928b3e07b75/xCK3ISKF6pWcMyO7MEzTA.png)
 
 We hypothesize this is primarily due to the particularly narrow and low variance weight distributions typical of Mistral derived models regardless of their scale.
-In the end, we settled on 2e-6 with an effective batch size of 64 (and a packed tokens batch size of 8192; effectively ~500,000 tokens per batch).
 
-We also trained with a weight decay of 0.01 to help further stabilize the loss trajectory and mitigate overfitting.
+In the end, due to the costs that would be involved in training another full 2 epochs run ($600), we settled on our third attempt: 2e-6 with an effective batch size of 64, stopped earlier than the target 2 epochs.
 
 [<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)