Hugging Face's logo
Michael Goin's picture
TFLOPS icon 82.6 TFLOPS
59 13 19

Michael Goin

mgoin
platypusai's profile picture charlesdedampierre's profile picture rgreenberg1's profile picture
·

AI & ML interests

LLM inference optimization, compression, quantization, pruning, distillation

Organizations

Neural Magic's profile picture garage-bAInd's profile picture Blog-explorers's profile picture Revel Labs's profile picture ZeroGPU Explorers's profile picture NM Testing's profile picture MLX Community's profile picture Social Post Explorers's profile picture

mgoin's activity

New activity in neuralmagic/Meta-Llama-3.1-8B-Instruct-quantized.w4a16 14 days ago

Issue with loading model

1
#1 opened 15 days ago by xSumukhax
New activity in neuralmagic/DeepSeek-Coder-V2-Instruct-FP8 22 days ago

Can it run on A100/A800 with VLLM?

3
#1 opened about 2 months ago by Parkerlambert123
New activity in neuralmagic/Meta-Llama-3.1-405B-Instruct-quantized.w4a16 about 1 month ago

weights does not exist when trying to deploy in sagemaker endpoint

1
#1 opened about 1 month ago by LorenzoCevolaniAXA
New activity in meta-llama/Meta-Llama-3.1-405B-Instruct about 1 month ago

8-kv-heads

4
#17 opened about 2 months ago by ArthurZ
New activity in meta-llama/Meta-Llama-3.1-405B about 1 month ago

8-kv-heads

3
#21 opened about 2 months ago by ArthurZ
New activity in neuralmagic/Meta-Llama-3.1-70B-Instruct-quantized.w8a16 about 1 month ago

run with vllm

8
#4 opened about 2 months ago by kuliev-vitaly
New activity in neuralmagic/Meta-Llama-3.1-8B-Instruct-FP8 about 1 month ago

Not able to run Model using VLLM

1
#3 opened about 1 month ago by Pchaudhary
New activity in neuralmagic/gemma-2-9b-it-FP8 about 1 month ago

getting issue while loading in llm

1
#1 opened about 1 month ago by Abhinav6310
New activity in neuralmagic/Meta-Llama-3.1-8B-Instruct-FP8 about 2 months ago

How to fast inference with FP8

1
#2 opened about 2 months ago by CCRss
New activity in neuralmagic/Meta-Llama-3.1-8B-Instruct-quantized.w8a16 about 2 months ago

Unable to load model onto multiple GPUs

2
#2 opened about 2 months ago by bprice9
New activity in neuralmagic/Meta-Llama-3.1-405B-Instruct-FP8 about 2 months ago

What are the differences between yours and meta's offical one?

2
#2 opened about 2 months ago by c6sneaky
New activity in neuralmagic/Meta-Llama-3.1-8B-Instruct-FP8 about 2 months ago

OSError, is the config correct?

2
#1 opened about 2 months ago by jackinthebox52
New activity in mgoin/Nemotron-4-340B-Instruct-hf-FP8 about 2 months ago

Thanks your great work!

2
#1 opened about 2 months ago by bay-llm
New activity in neuralmagic/Mistral-7B-Instruct-v0.3-FP8 about 2 months ago

Compression script limits context length to 4098?

1
#1 opened about 2 months ago by Kayvane
New activity in nvidia/Minitron-4B-Base about 2 months ago

Where is Minitron-4B-Instruct?

1
#2 opened about 2 months ago by mgoin
New activity in neuralmagic/Meta-Llama-3.1-405B-Instruct-FP8 about 2 months ago

Is this compatible with the KV_Cache_dtype being FP8?

2
#1 opened about 2 months ago by nickandbro
New activity in neuralmagic/Meta-Llama-3.1-70B-Instruct-FP8 about 2 months ago

Are these models limited to H100s?

7
#2 opened about 2 months ago by RonanMcGovern
New activity in nvidia/Minitron-8B-Base about 2 months ago

Replace kv_channels with head_dim
#1 opened about 2 months ago by mgoin
New activity in neuralmagic/Mistral-Nemo-Instruct-2407-FP8 2 months ago

Error serving model

3
#2 opened 2 months ago by EvGUT
New activity in neuralmagic/Phi-3-mini-128k-instruct-quantized.w8a8 2 months ago

Model doesnt load in vllm version 0.5.0 : ValueError: `rope_scaling`'s type field must be one of ['su', 'yarn'], got longrope

1
#1 opened 2 months ago by sujjosep
New activity in neuralmagic/Meta-Llama-3-8B-Instruct-FP8-KV 3 months ago

How to load this model?

1
#1 opened 3 months ago by Frz614
New activity in neuralmagic/Meta-Llama-3-70B-Instruct-FP8 3 months ago

How to run Meta-Llama-3-70B-Instruct-FP8 using several devices?

5
#3 opened 3 months ago by Fertel

Update model.safetensors.index.json
#2 opened 3 months ago by mgoin
New activity in neuralmagic/Meta-Llama-3-8B-Instruct-FP8 3 months ago

Update model.safetensors.index.json
#4 opened 3 months ago by mgoin

`model.safetensors.index.json` still has the legacy name`act_scale` for activation scales

1
#3 opened 3 months ago by Alchan
New activity in nm-testing/SparseLlama-3-8B-pruned_50.2of4-FP8 3 months ago

Update README.md
#1 opened 3 months ago by alexmarques
New activity in neuralmagic/SparseLlama-3-8B-pruned_50.2of4 3 months ago

Update README.md
#1 opened 3 months ago by alexmarques
New activity in neuralmagic/Qwen2-72B-Instruct-FP8 3 months ago

Update README.md
#1 opened 3 months ago by abhinavnmagic
New activity in neuralmagic/Mixtral-8x7B-Instruct-v0.1-FP8 3 months ago

Update README.md
#1 opened 3 months ago by abhinavnmagic
New activity in neuralmagic/Meta-Llama-3-8B-Instruct-FP8 3 months ago

Update README.md
#2 opened 3 months ago by abhinavnmagic
New activity in neuralmagic/Meta-Llama-3-70B-Instruct-FP8 3 months ago

Create README.md
#1 opened 3 months ago by abhinavnmagic
New activity in neuralmagic/Meta-Llama-3-8B-Instruct-FP8 3 months ago

Fails to run with nm-vllm

1
#1 opened 4 months ago by clintonruairi
New activity in mgoin/ultrachat_2k 4 months ago

Librarian Bot: Add language metadata for dataset
#2 opened 4 months ago by librarian-bot
New activity in neuralmagic/Mistral-7B-Instruct-v0.3-GPTQ-4bit 4 months ago

Inference GPU Ram requirement >60GB

1
#1 opened 4 months ago by Ksgk-fy
New activity in mgoin/Meta-Llama-3-70B-Instruct-Marlin 4 months ago

What conversion process are you using?

2
#2 opened 4 months ago by matt-psaltis-devbricks
New activity in mgoin/Meta-Llama-3-70B-Instruct-Marlin 5 months ago

What is Marlin?

2
#1 opened 5 months ago by Samvanity
New activity in mgoin/Meta-Llama-3-8B-Instruct-Marlin 5 months ago

Inference Issues

7
#1 opened 5 months ago by qeternity
New activity in neuralmagic/Llama-2-7b-dolphin-open_platypus-pruned_50 6 months ago

Update README.md
#2 opened 6 months ago by shubhrapandit
New activity in neuralmagic/Llama-2-7b-dolphin-open_platypus-pruned_70-quantized-deepsparse 6 months ago

Update README.md
#1 opened 6 months ago by shubhrapandit
New activity in neuralmagic/Llama-2-7b-dolphin-open_platypus-pruned_50-quantized-deepsparse 6 months ago

Update README.md
#1 opened 6 months ago by shubhrapandit
New activity in neuralmagic/Llama-2-7b-dolphin-open_platypus-pruned_70 6 months ago

Update README.md
#1 opened 6 months ago by shubhrapandit
New activity in neuralmagic/Llama-2-7b-dolphin-open_platypus-pruned_50 6 months ago

Update README.md
#1 opened 6 months ago by shubhrapandit
New activity in neuralmagic/Llama-2-7b-evolcodealpaca 6 months ago

Update README.md
#1 opened 6 months ago by abhinavnmagic
New activity in neuralmagic/Llama-2-7b-evol-code-alpaca-pruned_70-quantized-deepsparse 6 months ago

Update README.md
#1 opened 6 months ago by abhinavnmagic
New activity in neuralmagic/Llama-2-7b-evol-code-alpaca-pruned_50-quantized-deepsparse 6 months ago

Update README.md
#1 opened 6 months ago by abhinavnmagic
New activity in neuralmagic/Llama-2-7b-evol-code-alpaca-pruned_70 6 months ago

Update README.md
#1 opened 6 months ago by abhinavnmagic
New activity in neuralmagic/Llama-2-7b-evol-code-alpaca-pruned_50 6 months ago

Update README.md
#1 opened 6 months ago by abhinavnmagic
New activity in neuralmagic/Llama-2-7b-ultrachat200k-pruned_70-quantized-deepsparse 6 months ago

Update README.md
#1 opened 6 months ago by alexmarques
New activity in neuralmagic/Llama-2-7b-ultrachat200k-pruned_70 6 months ago

Update README.md
#1 opened 6 months ago by alexmarques
New activity in neuralmagic/Llama-2-7b-ultrachat200k-pruned_50 6 months ago

Update README.md
#1 opened 6 months ago by alexmarques
New activity in neuralmagic/Llama-2-7b-pruned70-retrained 6 months ago

Update README.md
#1 opened 6 months ago by alexmarques
New activity in neuralmagic/Llama-2-7b-pruned50-retrained 6 months ago

Update README.md
#1 opened 6 months ago by alexmarques
New activity in neuralmagic/CLIP-ViT-B-32-256x256-DataComp-s34B-b86K-quant-ds 8 months ago

Update README.md
#4 opened 8 months ago by chrisxx

Update README with model author names and speedup numbers.
#3 opened 8 months ago by jen
New activity in neuralmagic/CLIP-ViT-B-32-256x256-DataComp-s34B-b86K-quant-ds 9 months ago

Update README.md
#1 opened 9 months ago by wendlerc
New activity in neuralmagic/mpt-7b-gsm8k-pruned60-pt 10 months ago

Adding `safetensors` variant of this model
#1 opened 10 months ago by mgoin
New activity in reciprocate/llama2-7b-gsm8k 12 months ago

Create README.md
#2 opened 12 months ago by mgoin