- Trending Categories
- Data Structure
- Operating System
- C Programming
- Selected Reading
- UPSC IAS Exams Notes
- Developer's Best Practices
- Questions and Answers
- Effective Resume Writing
- HR Interview Questions
- Computer Glossary
- Who is Who
Multiprocessing In Python
The multiprocessing package supports spawning processes. It refers to a function that loads and executes a new child processes. For the child to terminate or to continue executing concurrent computing,then the current process hasto wait using an API, which is similar to threading module.
When we work with Multiprocessing,at first we create process object. Then it calls a start() method.
from multiprocessing import Process def display(): print ('Hi !! I am Python') if __name__ == '__main__': p = Process(target=display) p.start() p.join()
In this example, at first we import the Process class then initiate Process object with the display() function.
Then process is started with start() method and then complete the process with the join() method.
We can also pass arguments to the function using args keyword.
from multiprocessing import Process def display(my_name): print ('Hi !!!' + " " + my_name) if __name__ == '__main__': p = Process(target=display, args=('Python',)) p.start() p.join()
In this example, we create a process that calculates the cube of numbers and prints all results to the console.
from multiprocessing import Process def cube(x): for x in my_numbers: print('%s cube is %s' % (x, x**3)) if __name__ == '__main__': my_numbers = [3, 4, 5, 6, 7, 8] p = Process(target=cube, args=('x',)) p.start() p.join print ("Done")
Done 3 cube is 27 4 cube is 64 5 cube is 125 6 cube is 216 7 cube is 343 8 cube is 512
We can also create more than one process at atime.
In this example, at first we create one process which is process1, this process just calculates the cube of a number and at the same time second process process2 is checking that this number is even or odd.
from multiprocessing import Process def cube(x): for x in my_numbers: print('%s cube is %s' % (x, x**3)) def evenno(x): for x in my_numbers: if x % 2 == 0: print('%s is an even number ' % (x)) if __name__ == '__main__': my_numbers = [3, 4, 5, 6, 7, 8] my_process1 = Process(target=cube, args=('x',)) my_process2 = Process(target=evenno, args=('x',)) my_process1.start() my_process2.start() my_process1.join() my_process2.join() print ("Done")
3 cube is 27 4 cube is 64 5 cube is 125 6 cube is 216 7 cube is 343 8 cube is 512 4 is an even number 6 is an even number 8 is an even number Done
Communication between Processes
Multiprocessing supports Pipes and Queues, which are two types of communication channels between processes.
In multiprocessing, when we want to communicate between processes, in that situation Pipes areused.
from multiprocessing import Process, Pipe def myfunction(conn): conn.send(['hi!! I am Python']) conn.close() if __name__ == '__main__': parent_conn, child_conn = Pipe() p = Process(target=myfunction, args=(child_conn,)) p.start() print (parent_conn.recv() ) p.join()
['hi !!! I am Python']
Pipes return two connection objects and these are representing the two ends of the pipe. Each connection object has two methods one is send() and another one is recv() method.
In this example, at first we create a process and this process prints the message "hi!! I am Python" and then shares the data across.
When we pass data between processes then at that time we can use Queue object.
import multiprocessing def evenno(numbers, q): for n in numbers: if n % 2 == 0: q.put(n) if __name__ == "__main__": q = multiprocessing.Queue() p = multiprocessing.Process(target=evenno, args=(range(10), q)) p.start() p.join() while q: print(q.get())
0 2 4 6 8
In this example, at first create a function that checks weather a number is even or not. If the number is even, then insert it at the end of the queue. Then we create a queue object and a process object then we start the process.
And finally check whether the queue is empty or not.
When we print the numbers, at first we print the value which is in front of the queue then next one and so on.
When we want that only one process is executed at a time in that situation Locks is use. That means that time blocks other process from executing similar code. Lock will be released after the process gets completed.
Example using Locks method
from multiprocessing import Process, Lock def dispmay_name(l, i): l.acquire() print ('Hi', i) l.release() if __name__ == '__main__': my_lock = Lock() my_name = ['Aadrika', 'Adwaita', 'Sakya', 'Sanj'] for name in my_name: Process(target=dispmay_name, args=(my_lock,name)).start()
Hi Aadrika Hi Adwaita Hi Sakya Hi Sanj
The multiprocessing module also provides logging module to ensure that, if the logging package doesn't use locks function, the messages between processes mixed up during execution.
import multiprocessing, logging logger = multiprocessing.log_to_stderr() logger.setLevel(logging.INFO) logger.warning('Error has occurred')
In this example at first we import the logging and multiprocessing module then we use multiprocessing.log_to_stderr() method. And it call get_logger() as well as adding to sys.stderr and finally we set the level of logger and convey the message.
- Symmetric Multiprocessing
- Asymmetric Multiprocessing
- Symmetric Multiprocessing Architecture
- Difference between Multiprocessing and Multithreading
- Difference between Asymmetric and Symmetric Multiprocessing
- Difference Between Symmetric and Asymmetric Multiprocessing
- What is a Multiprocessing Operating System?
- Differentiate between multiprogramming, multitasking and multiprocessing.
- ChainMap in Python
- Namedtuple in Python
- Deque in Python
- OrderedDict in Python
- Quine in Python
- Underscore(_) in Python
- Keywords in Python