2 20 72

neuralink

AI & ML interests

Don't bet against deep learning

Articles

A failed experiment: Infini-Attention, and why we should keep trying?

Aug 14

• 40

Organizations

neuralink's activity

upvoted a paper 9 days ago

Small-scale proxies for large-scale Transformer training instabilities

Paper • 2309.14322 • Published Sep 25, 2023 • 19

upvoted an article 13 days ago

Article

How NuminaMath Won the 1st AIMO Progress Prize

Jul 11

• 92

upvoted a paper 22 days ago

Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets

Paper • 2201.02177 • Published Jan 6, 2022 • 2

upvoted an article about 1 month ago

Article

A failed experiment: Infini-Attention, and why we should keep trying?

Aug 14

• 40

upvoted 2 papers about 1 month ago

Grokfast: Accelerated Grokking by Amplifying Slow Gradients

Paper • 2405.20233 • Published May 30 • 5

Transformer Explainer: Interactive Learning of Text-Generative Models

Paper • 2408.04619 • Published Aug 8 • 152

upvoted 2 papers 3 months ago

What matters when building vision-language models?

Paper • 2405.02246 • Published May 3 • 98

DiPaCo: Distributed Path Composition

Paper • 2403.10616 • Published Mar 15 • 12

upvoted an article 3 months ago

Article

Putting RL back in RLHF

Jun 12

• 58

upvoted an article 4 months ago

Article

Let's talk about LLM evaluation

•

May 23

• 104

upvoted 3 papers 6 months ago

upvoted 2 papers 7 months ago

MegaScale: Scaling Large Language Model Training to More Than 10,000 GPUs

Paper • 2402.15627 • Published Feb 23 • 33

FP8-LM: Training FP8 Large Language Models

Paper • 2310.18313 • Published Oct 27, 2023 • 31

upvoted a paper 8 months ago

Zero Bubble Pipeline Parallelism

Paper • 2401.10241 • Published Nov 30, 2023 • 22

upvoted a paper 9 months ago

Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Action

Paper • 2312.17172 • Published Dec 28, 2023 • 26

upvoted 3 papers 10 months ago

OBELICS: An Open Web-Scale Filtered Dataset of Interleaved Image-Text Documents

Paper • 2306.16527 • Published Jun 21, 2023 • 47

RWKV: Reinventing RNNs for the Transformer Era

Paper • 2305.13048 • Published May 22, 2023 • 12

Llemma: An Open Language Model For Mathematics

Paper • 2310.10631 • Published Oct 16, 2023 • 49