It activates only 37B per query, but you don’t know which ones ahead of time, so... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

coolspot on Dec 31, 2024 | parent | context | favorite | on: Deepseek: The quiet giant leading China’s AI race

It activates only 37B per query, but you don’t know which ones ahead of time, so you gotta store all 671B in (V)RAM.

cma on Jan 1, 2025 [–]

But you don't need cluster networking or nvlink so much like with splitting out llama 405B. You could even split them out with friends over internet levels of bandwidth.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact