Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

If you wanted to go crazy you can use transferables to get around the memory copy cost. It would be a fun experiment to figure out at what point does it make sense to use multiple workers considering the initial cost of worker distribution.


Having had a quick read (I'd somehow missed nontransferables and thought less efficient message passing was the only way, thanks for the pointer) that could help but because the objects are moved to the new context and lost in the original you do need to create and populate fresh ones for each worker (so extra copy operations that wouldn't be needed if true shared memory were possible).

Still, that might work well for some algorithms over certain data types. For example a divide and conquer based sort like merge or quick might benefit as you can send the first few partitions to different workers and let the first worker to the final merging. It wouldn't work for Shell's sort though as splitting the data into transferable chunks for that algorithm is not going to be efficient as it touches from the beginning of the array to the end in each iteration. And you'd need to be analyse if the gaining time splitting the actual sort between threads is enough to not be negated (or worse, dwarfed) by the time needed for all the extra data copying.

Outside sorting, number crunching tasks where the operations have a high locality of reference and so are easily partitioned would work well without hitting too much sharing trouble. Fractals for instance: computing one pixel of a Mandelbrot display does not rely on knowing anything about the neighbouring ones (or any others) so partitioning the work is going to be trivial. Compression too, if you don't mind losing a small amount of compression efficiency (due to dictionary resets at the partition boundaries) in exchange for the bump in potential parallelism.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: