Awesome- Could you help with pointers on doing same for Other languages(Swedish)?
Great job, very impressed!
Im trying to figure out how to go about creating something useful for ’general interested public’ with a reasonable machine spec.
Would you feel comfortable sharing the steps you would take if doing this for another language, and the tools to use.
Especially interested in how you improved the grammar from regular english->german translation...although the other steps feel alittle like black-magic as well at this stage ;-)
In my case it’s swedish, with the goal of 10-100x:ing speed of translation/ localization of Khan Academy content to swedish.
I’m sure other languages have similar interests.
again, great work!
Hello Olofp,
thank you very much for your feedback, we appreciate it.
So generally you can say that data augmentation is one of the most important key points. You can use a strong transformer model not just to translate from a to b, but also to use augmentation to get similar sentences that thematically correspond to the original of the reference dataset. This means that many errors that occur during direct translation can be avoided in the language to be translated.
However, our model is still far from perfect. There is always a fine line between a loss of performance and good German, which is why the model can sometimes answer with poor grammar.
The whole thing is currently simply too limited by “pure” fine-tuning.
But we are making a big effort to improve everything further. We have now at least reached a point where we have been able to significantly improve the German of the original model and still retain the skills of the original language (in Solar's case, English).
You can find more information in the model card and our MT-Bench True German data set.
I hope I could help you with this.
Thank you for sharing your work.
I'm wondering if you'd unfreeze some of the layers such as the embedding and the language head layers and actually update those layers during finetuning if it would improve performance on German. Is that something you are considering trying?
Hey Samuel! Interesting idea. We will look into that.