116% would be 2x which will break the laws of physics. 16% is possible if you're not allocating heap memory, which I believe is the main selling point here.
Not if he can also deliver equally good or better results, which I suspect he will. Not everyone needs it to do a good job, and for others, it could be a net negative.
Using computers created an unambiguous productivity boost. Same cannot be said for all types of software engineering with AI; that's why we keep having this debate. The results are mixed.
Sad side of this is that decision makers are seeing these and think that AI can replace yet another department in their office.
I just saw a nice presentation from SecFest from a guy who ran a red team, but now they pivoted to basically anything and will be more than happy to fix your car if that would pay their bills.
Tired seeing them is one thing the real problem is tired updating your software stack with a frequency the team & tooling was never designed to sustain. Worse: You are not tired enough to forget the real security issues that would need some calm period to fix.
Kagi is amazing. Worth the subscription, 10 times over. I don't have to worry about sponsored search results, or slop from the "AI overview". High quality results, like Google used to be.
I extensively report obvious slop websites. Most of them get rejected/marked as “not AI”. That initiative is cute on paper, but they apparently do not trust the community.
This is nonsense. Tokio was built for I/O, not crunching numbers. Most programmers know to use spawn_blocking to crunch 10MB JSON, if needed.
Additionally, the proposed workload per thread model will be orders of magnitude slower than Tokio for I/O-bound workloads, which most applications are.
Funnily enough, spawn_blocking is not the right tool here. It is meant for blocking I/O, such as DNS lookups, where your platform might not give you anything better.
For genuine CPU-bound work, submitting to a Rayon worker pool is the way to go. It solved a runtime starvation issue for us at work, spawn_blocking did not work.
The reason for all this is spawn_blocking having a very large underlying thread pool, in the hundreds. That is okay if you assume work will yield those threads and mostly sleep/wait. It is not okay if the work never yields, like pure data crunching. (Go solves this by forcefully preempting loops, no such thing in Rust without a language runtime)
Our solution shape was: multi-threaded Tokio (2 threads), then give the rest of available_concurrency to a Rayon thread pool. If you grant 6 vCPU you should see a thread pool of 4, and a maximum CPU consumption of about 400%, as the Tokio threads sit mostly idle (under low load single-threaded runtime should also suffice).
You inject the thread pool using an Arc.
Then, when work comes in, just spawn Tokio tasks liberally (cheap) and submit to the thread pool. Rayon will internally queue and limit concurrency and parallelism to 4 (this is the important bit compared to spawn_blocking: no way your system can hog all 6 threads with non-yielding work and starve Tokio runtime threads).
We use one-shot channels to submit results back, they are designed for exactly this. The tx aka sender end is sync, as there is never a wait (cannot block). The rx aka receiver side is async and can be awaited normally on the async side. This is a cheap operation, similar to Go.
Optionally you can reach for semaphores to also limit I/O concurrency. You probably want to do this for more control and avoiding resource exhaustion loudly (that is, not silently accidentally peg thousands of FDs, database connections, …).
It ended up working beautifully for our purposes and relatively simply. No lifetime woes, Arc solves those. Oneshot channels just transfer ownership etc.
Perhaps this is what TFA talks about, I have not read it.
One caveat: to reach all the above conclusions and designs, we had help from some genuine Rust experts. As much as I dislike Go, it "just works" there even if one writes naive code.
Exactly. I do not know the specifics, but for example if libraries you call into liberally spawn_blocking under the expectation that it is okay, you will be in trouble.
Says it right there actually:
> It’s recommended to not set this limit too low in order to avoid hanging on operations requiring spawn_blocking.
So a total like 6 - reasonable for a web backend - would be way too low.
reply