Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Power isn't proportional to parameters. It may be vaguely proportional to tokens/s although batching screws that up.

Claude Sonnet is probably running on a 8 GPU box that consumes 10 kW while Opus might use more like 50 kW but that's shared by a bunch of users thanks to batching.



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: