Python - Generators

A generator in Python is a special type of function that returns an iterator object. It appears similar to a normal Python function in that its definition also starts with def keyword. However, instead of return statement at the end, generator uses the yield keyword.

Syntax

def generator():
 . . .
 . . .
 yield obj
it = generator()
next(it)
. . .

The return statement at the end of function indicates that the execution of the function body is over, all the local variables in the function go out of the scope. If the function is called again, the local variables are re-initialized.

Generator function behaves differently. It is invoked for the first time like a normal function, but when its yield statement comes, its execution is temporarily paused, transferring the control back. The yielded result is consumed by the caller. The call to next() built-in function restarts the execution of generator from the point it was paused, and generates the next object for the iterator. The cycle repeats as subsequent yield provides next item in the iterator it is exhausted.

Example 1

The function in the code below is a generator that successively yield integers from 1 to 5. When called, it returns an iterator. Every call to next() transfers the control back to the generator and fetches next integer.

def generator(num):
   for x in range(1, num+1):
      yield x
   return
   
it = generator(5)
while True:
   try:
      print (next(it))
   except StopIteration:
      break

It will produce the following output −

The generator function returns a dynamic iterator. Hence, it is more memory efficient than a normal iterator that you obtain from a Python sequence object. For example, if you want to get first n numbers in Fibonacci series. You can write a normal function and build a list of Fibonacci numbers, and then iterate the list using a loop.

Example 2

Given below is the normal function to get a list of Fibonacci numbers −

def fibonacci(n):
   fibo = []
   a, b = 0, 1
   while True:
      c=a+b
      if c>=n:
         break
      fibo.append(c)
      a, b = b, c
   return fibo
f = fibonacci(10)
for i in f:
   print (i)

It will produce the following output −

The above code collects all Fibonacci series numbers in a list and then the list is traversed using a loop. Imagine that we want Fibonacci series going upto a large number. In which case, all the numbers must be collected in a list requiring huge memory. This is where generator is useful, as it generates a single number in the list and gives it for consumption.

Example 3

Following code is the generator-based solution for list of Fibonacci numbers −

def fibonacci(n):
   a, b = 0, 1
   while True:
      c=a+b
      if c>=n:
         break
      yield c
      a, b = b, c
   return
   
f = fibonacci(10)
while True:
   try:
      print (next(f))
   except StopIteration:
      break

Asynchronous Generator

An asynchronous generator is a coroutine that returns an asynchronous iterator. A coroutine is a Python function defined with async keyword, and it can schedule and await other coroutines and tasks. Just like a normal generator, the asynchronous generator yields incremental item in the iterator for every call to anext() function, instead of next() function.

Syntax

async def generator():
. . .
. . .
yield obj
it = generator()
anext(it)
. . .

Example 4

Following code demonstrates a coroutine generator that yields incrementing integers on every iteration of an async for loop.

import asyncio

async def async_generator(x):
   for i in range(1, x+1):
      await asyncio.sleep(1)
      yield i
      
async def main():
   async for item in async_generator(5):
      print(item)
      
asyncio.run(main())

It will produce the following output −

Example 5

Let us now write an asynchronous generator for Fibonacci numbers. To simulate some asynchronous task inside the coroutine, the program calls sleep() method for a duration of 1 second before yielding the next number. As a result, you will get the numbers printed on the screen after a delay of one second.

import asyncio

async def fibonacci(n):
   a, b = 0, 1
   while True:
      c=a+b
      if c>=n:
         break
      await asyncio.sleep(1)
      yield c
      a, b = b, c
   return
   
async def main():
   f = fibonacci(10)
   async for num in f:
      print (num)
      
asyncio.run(main())

It will produce the following output −