In reality it’s never as simple as a single soundbite. If you are a startup with $1k/mo AWS bills, throwing more hardware at the problem can be orders of magnitude cheaper. If you are running resource-intensive workloads then at some point efficiency work becomes ROI-positive.
The reason the rule of thumb is to throw more hardware at the problem is that most (good) engineers bias towards wanting to make things performant, in my experience often beyond the point where it’s ROI positive. But of course you should not take that rule of thumb as a universal law, rather it’s another reminder of a cognitive bias to keep an eye on.
Yes - context matters. This article is about Pixar where it pays to have someone think about performance. My data points are only companies of 50 people and higher - in those cases cloud consumption was the number 2 cost line item behind people. It matters there. In most cases the people who know cost performance tend to be good at fighting latency - you need the same skills.
This may not apply on smaller side projects or places where technology is secondary.
The reason the rule of thumb is to throw more hardware at the problem is that most (good) engineers bias towards wanting to make things performant, in my experience often beyond the point where it’s ROI positive. But of course you should not take that rule of thumb as a universal law, rather it’s another reminder of a cognitive bias to keep an eye on.