start a bunch of threads and e.g. invoke subprocess.run() from them
Done that many, many times and honestly combining python with parallel is in many cases the best way to go. Write your python script to be as fast as possible on one core and then use parallel to run it on all your cores. This has the added advantage that you can go from running on all the cores on your machine to running on all the cores on a 100 machine cluster by just changing a couple of lines of code.
Done that many, many times and honestly combining python with parallel is in many cases the best way to go. Write your python script to be as fast as possible on one core and then use parallel to run it on all your cores. This has the added advantage that you can go from running on all the cores on your machine to running on all the cores on a 100 machine cluster by just changing a couple of lines of code.