Multiprocessing In Python

Multiprocessing in Python allows you to execute multiple processes simultaneously, utilizing multiple CPU cores for improved performance. The multiprocessing module provides tools to create and manage separate processes that can run concurrently.

Basic Process Creation

To create a process, import the Process class and define a target function ?

from multiprocessing import Process

def display():
    print('Hi !! I am Python')

if __name__ == '__main__':
    p = Process(target=display)
    p.start()
    p.join()

Hi !! I am Python

The start() method begins the process execution, while join() waits for the process to complete before continuing.

Passing Arguments to Processes

Use the args parameter to pass arguments to the target function ?

from multiprocessing import Process

def display(my_name):
    print('Hi !!! ' + my_name)

if __name__ == '__main__':
    p = Process(target=display, args=('Python',))
    p.start()
    p.join()

Hi !!! Python

Multiple Processes Example

Create multiple processes to perform different tasks simultaneously ?

from multiprocessing import Process

def cube(numbers):
    for x in numbers:
        print('%s cube is %s' % (x, x**3))

def evenno(numbers):
    for x in numbers:
        if x % 2 == 0:
            print('%s is an even number' % (x))

if __name__ == '__main__':
    my_numbers = [3, 4, 5, 6, 7, 8]
    process1 = Process(target=cube, args=(my_numbers,))
    process2 = Process(target=evenno, args=(my_numbers,))
    
    process1.start()
    process2.start()
    
    process1.join()
    process2.join()
    print("Done")

3 cube is 27
4 cube is 64
5 cube is 125
6 cube is 216
7 cube is 343
8 cube is 512
4 is an even number
6 is an even number
8 is an even number
Done

Communication Between Processes

Using Pipes

Pipes provide a way for processes to communicate by sending and receiving data ?

from multiprocessing import Process, Pipe

def myfunction(conn):
    conn.send(['hi!! I am Python'])
    conn.close()

if __name__ == '__main__':
    parent_conn, child_conn = Pipe()
    p = Process(target=myfunction, args=(child_conn,))
    p.start()
    print(parent_conn.recv())
    p.join()

['hi!! I am Python']

Using Queues

Queues allow multiple processes to share data safely using FIFO (First In, First Out) principle ?

import multiprocessing

def evenno(numbers, q):
    for n in numbers:
        if n % 2 == 0:
            q.put(n)

if __name__ == "__main__":
    q = multiprocessing.Queue()
    p = multiprocessing.Process(target=evenno, args=(range(10), q))
    p.start()
    p.join()
    
    while not q.empty():
        print(q.get())

0
2
4
6
8

Process Synchronization with Locks

Locks ensure that only one process accesses shared resources at a time ?

from multiprocessing import Process, Lock

def display_name(l, name):
    l.acquire()
    print('Hi', name)
    l.release()

if __name__ == '__main__':
    my_lock = Lock()
    names = ['Aadrika', 'Adwaita', 'Sakya', 'Sanj']
    
    processes = []
    for name in names:
        p = Process(target=display_name, args=(my_lock, name))
        processes.append(p)
        p.start()
    
    for p in processes:
        p.join()

Hi Aadrika
Hi Adwaita
Hi Sakya
Hi Sanj

Comparison of Communication Methods

Conclusion

Multiprocessing in Python enables true parallelism by utilizing multiple CPU cores. Use Process for basic parallel execution, Pipe and Queue for inter-process communication, and Lock for synchronization.