From the course: Python Parallel and Concurrent Programming Part 2
Join today to access over 22,600 courses taught by industry experts or purchase this course individually.
Process pool: Python demo - Python Tutorial
From the course: Python Parallel and Concurrent Programming Part 2
Process pool: Python demo
- [Instructor] Python's ThreadPoolExecutor is great when you have a bunch of IO-dependent tasks to perform. However, if the tasks are primarily CPU intensive, then there isn't much benefit to running them all in concurrent threads due to the global interpreter lock. The workaround in Python is to use multiple processes. So, in addition to the ThreadPoolExecutor, the Python concurrent.futures module also has a ProcessPoolExecutor. To modify our previous example to use the ProcessPoolExecutor instead, I'll change line five from ThreadPoolExecutor to ProcessPoolExecutor. And then, I'll make a similar change on line 12. And that's it. Now, the program will use a pool with multiple processes instead of multiple threads to execute the vegetable chopper tasks I submit to it. Now, one more improvement I can make to this program, both the ThreadPoolExecutor and the ProcessPoolExecutor are designed to be used with a context manager. So, I'll modify line 12 and turn it into a with statement. So,…
Practice while you learn with exercise files
Download the files the instructor uses to teach the course. Follow along and learn by watching, listening and practicing.
Download courses and learn on the go
Watch courses on your mobile device without an internet connection. Download courses using your iOS or Android LinkedIn Learning app.