asyncio - Asynchronous I/O, Event Loops, Coroutines, and Tasks

CPU execution is sequential; threads are a mechanism provided by the operating system that let us achieve “parallelism” at the OS level. Coroutines, on the other hand, can be thought of as a mechanism provided by the application itself (implemented by the user or a library) that let us achieve “parallelism” at the application level.

Since a program is inherently executed sequentially, to create the illusion of this “parallelism” we need a mechanism to “pause” the current execution flow and “resume” a previous one later. In operating systems and multi-threading/multi-processing, this is called a “context switch”. The “context” records the execution state of a thread, including the variables it uses and its call stack. “Switching” means saving a thread’s current running state and later restoring it from that saved state. The only difference is that thread-related work is done by the operating system, while coroutine-related work is done by the application itself.

Unlike threads, coroutines typically accomplish smaller pieces of work, so there’s often a need to chain different coroutines together — for now we’ll call this a coroutine chain. [1]

This module provides the infrastructure for writing single-threaded concurrent code using coroutines, multiplexing I/O over sockets and other resources, and running network clients and servers. Here is a more detailed list of what the package includes:

A pluggable event loop with various system-specific implementations
Transport and protocol abstractions
Support for TCP, UDP, SSL, subprocess pipes, delayed calls, and other concrete facilities (some of which may be platform-specific)
A Future class that mimics the concurrent.futures module but is designed for use with the event loop
Coroutines and tasks based on yield from, helping you write concurrent code in a sequential style
Cancellation support for Futures and coroutines
Synchronization primitives for use between coroutines in a single thread, mirroring those threading modules
An interface for a thread pool to hand off work to, for use when you’re stuck with a library that only offers a blocking I/O call

Before doing asynchronous programming with the asyncio module, you need to understand the following concepts [2]:

Event loop — the event loop multiplexes I/O, working through selectors, and serializes event handling. The program starts an infinite loop and registers functions with the event loop. When the conditions for an event are met, the corresponding coroutine function is called.
coroutine — a coroutine object refers to a function defined with the async def keyword; calling it does not execute the function immediately, but instead returns a coroutine object. A coroutine cannot run directly — the coroutine object needs to be registered with the event loop, which then invokes it.
Futures — this is the abstraction for those deferred producers. The asyncio.Future class is similar to the Future class introduced in Python 3.2, i.e. concurrent.futures.Future. However, in this case Future is meant for use with coroutines. The asyncio module does not reuse the existing concurrent.futures.Future class, because that one was designed for threaded work. This module encourages using await within a coroutine to suspend the current task while waiting for a result, so as to avoid blocking your application. Your coroutine’s code block — that is, your coroutine — is suspended until a result is produced, but the event loop itself is not blocked. If the same event loop has other task sequences, they may run in the meantime. When the coroutine produces a result, the suspended coroutine resumes, and you can write code as if it executed sequentially. You can read the code without needing to think about the presence of await. When you use await within a function to get back an awaited object, you can forget about the specific details of how Future executes and its particular API. If an exception occurs — for instance you called a function that doesn’t return a Future but instead runs synchronously — that exception is raised. So, writing asynchronous code looks just like writing synchronous code, except for adding await.
Tasks — each Task is a coroutine wrapped by a Future, and it runs as the event loop runs. The asyncio.Task class is a subclass of asyncio.Future. Tasks also work together with await.

Syntax

async def — keyword used to define a coroutine
await — suspends a blocking asynchronous call interface
asyncio.get_event_loop — creates a default event loop
asyncio.gather — accepts a bunch of coroutines
asyncio.wait — accepts a list made up of coroutines

Here is an example:

import asyncio


async def hello_world():  # 创建一个协程（async def），它是 Future 对象
    print("Hello World!")
    await asyncio.sleep(2)
    print("Bye World!")
    return "Hello World"


async def hello_lfzyx():  # 创建一个协程（async def），它是 Future 对象
    print("Hello lfzyx!")
    await asyncio.sleep(3)
    print("Bye lfzyx!")
    return "lfzyx"


def callback(re):
    print('Callback: ', re.result())


if __name__ == "__main__":
    loop = asyncio.get_event_loop()  # 创建一个默认的事件循环
    task = asyncio.gather(hello_world(), hello_lfzyx())
    task.add_done_callback(callback)
    loop.run_until_complete(task)  # 将协程注册到事件循环，并启动事件循环
    loop.close()  # 关闭事件循环

References