We always hear about Concurrency, Parallel Programming, and Asynchronous Programming, etc. in Python. It can be confusing when you first learn about these concept (so am I!), so in the next few blogs, we are going to dig deep into the Abyss of Concurrency and have a swim around!
concurrent vs. parallel vs. asynchronous programming
First things first, concurrency is not parallelism.
Concurrency is when multiple tasks which start, run, and complete in overlapping time periods, without defined order (tasks take turns to run). It is the composition of independently executing processes.The CPU picks the task to run partly and then assign it to waiting state. While it’s in waiting, the CPU picks another task to run part of its task. The process continues until all tasks finish running all their task and then return. Even to the end user all the tasks finish at the same time, but in reality they are not.
Parallel programming is when multiple tasks/or subtasks of one parent tasks run at the same time, e.g. on a multi-core processor. It is the simultaneous execution of computations where a multi-core CPU will assign each task one of its core.
Note that Parallel programming is deterministic as such that a central control distributes the work among several processors. In Concurrency programming each task running is independent of each other and no order of execution should not matter.
Asynchronous programming is not directly related to Concurrency or Parallelism, but rather a mechanism that allows the program to achieve concurrency. It involves to do some computation-heavy/time-intensive tasks by deploying a thread in the background away from the main application so the current thread can remain unblocked while waiting for response. Typical pattern can be master/worker or map/reduce.
Speaking of this, there’s another pair of concepts we need to demystify.
What is the difference between I/O bound vs CPU bound task?
CPU bound is when the CPU has to do a lot of work, computes data, without the need to wait for new data to arrive. Parallel programming we mentioned above is really good for CPU bound task, such as string operations, search algorithms, graphic processing, etc.
I/O bound is the opposite when that a given task often waits for I/O, for example networking and filesystem. We don’t normally need multiple threads to run I/O in parallel: asynchronous I/O and event-based systems good enough. Typical tasks can be database reads and writes, web service calls, downloading and uploading data, etc.
For JavaScript, Node is slow for CPU-bound tasks by default, its asynchronous nature makes it really suitable for running I/O bound tasks though.
For Python, there’s Threading (1.5+), Multiprocessing(2.6+), Concurrent.Futures(3.2+), and the latest Asyncio module(3.4+).
We will talk more about Threading module in next blog.
Happy Reading!
