metadata

language:
  - en
license: apache-2.0

Zion_Alpha

Model Details

Zion_Alpha is the first REAL Hebrew model in the world. It wasn't finetuned for any tasks yet, but it actually understands and comprehends Hebrew. It can even do some decent translation. I've tested GPT4 vs Zion_Alpha, and out of the box, Zion_Alpha did a better job translating. I did the finetune using SOTA techniques and using my insights from years of underwater basket weaving. If you wanna offer me a job, just add me on Facebook.

Model architecture

Based on Mistral 7B. I didn't even bother to alter the tokenizer.

The recommended prompt setting is Debug-deterministic:

temperature: 1
top_p: 1
top_k: 1
typical_p: 1
min_p: 1
repetition_penalty: 1

The recommended instruction template is Mistral:

{%- for message in messages %}
    {%- if message['role'] == 'system' -%}
        {{- message['content'] -}}
    {%- else -%}
        {%- if message['role'] == 'user' -%}
            {{-'[INST] ' + message['content'].rstrip() + ' [/INST]'-}}
        {%- else -%}
            {{-'' + message['content'] + '</s>' -}}
        {%- endif -%}
    {%- endif -%}
{%- endfor -%}
{%- if add_generation_prompt -%}
    {{-''-}}
{%- endif -%}

English to hebrew example:

Zion_Alpha English to Hebrew example

English to hebrew example:

Zion_Alpha Hebrew to English example

Support

My Ko-fi page ALL donations will go for research resources and compute, every bit counts 🙏🏻
My Patreon ALL donations will go for research resources and compute, every bit counts 🙏🏻