Data Structure
Networking
RDBMS
Operating System
Java
MS Excel
iOS
HTML
CSS
Android
Python
C Programming
C++
C#
MongoDB
MySQL
Javascript
PHP
Physics
Chemistry
Biology
Mathematics
English
Economics
Psychology
Social Studies
Fashion Studies
Legal Studies
- Selected Reading
- UPSC IAS Exams Notes
- Developer's Best Practices
- Questions and Answers
- Effective Resume Writing
- HR Interview Questions
- Computer Glossary
- Who is Who
Multiprocessing In Python
The multiprocessing package supports spawning processes. It refers to a function that loads and executes a new child processes. For the child to terminate or to continue executing concurrent computing,then the current process hasto wait using an API, which is similar to threading module.
Introduction
When we work with Multiprocessing,at first we create process object. Then it calls a start() method.
Example code
from multiprocessing import Process
def display():
print ('Hi !! I am Python')
if __name__ == '__main__':
p = Process(target=display)
p.start()
p.join()In this example, at first we import the Process class then initiate Process object with the display() function.
Then process is started with start() method and then complete the process with the join() method.
We can also pass arguments to the function using args keyword.
Example
from multiprocessing import Process
def display(my_name):
print ('Hi !!!' + " " + my_name)
if __name__ == '__main__':
p = Process(target=display, args=('Python',))
p.start()
p.join()In this example, we create a process that calculates the cube of numbers and prints all results to the console.
Example code
from multiprocessing import Process
def cube(x):
for x in my_numbers:
print('%s cube is %s' % (x, x**3))
if __name__ == '__main__':
my_numbers = [3, 4, 5, 6, 7, 8]
p = Process(target=cube, args=('x',))
p.start()
p.join
print ("Done")Output
Done 3 cube is 27 4 cube is 64 5 cube is 125 6 cube is 216 7 cube is 343 8 cube is 512
We can also create more than one process at atime.
In this example, at first we create one process which is process1, this process just calculates the cube of a number and at the same time second process process2 is checking that this number is even or odd.
Example
from multiprocessing import Process
def cube(x):
for x in my_numbers:
print('%s cube is %s' % (x, x**3))
def evenno(x):
for x in my_numbers:
if x % 2 == 0:
print('%s is an even number ' % (x))
if __name__ == '__main__':
my_numbers = [3, 4, 5, 6, 7, 8]
my_process1 = Process(target=cube, args=('x',))
my_process2 = Process(target=evenno, args=('x',))
my_process1.start()
my_process2.start()
my_process1.join()
my_process2.join()
print ("Done")Output
3 cube is 27 4 cube is 64 5 cube is 125 6 cube is 216 7 cube is 343 8 cube is 512 4 is an even number 6 is an even number 8 is an even number Done
Communication between Processes
Multiprocessing supports Pipes and Queues, which are two types of communication channels between processes.
Pipes
In multiprocessing, when we want to communicate between processes, in that situation Pipes areused.
Example
from multiprocessing import Process, Pipe def myfunction(conn): conn.send(['hi!! I am Python']) conn.close() if __name__ == '__main__': parent_conn, child_conn = Pipe() p = Process(target=myfunction, args=(child_conn,)) p.start() print (parent_conn.recv() ) p.join()
Output
['hi !!! I am Python']
Pipes return two connection objects and these are representing the two ends of the pipe. Each connection object has two methods one is send() and another one is recv() method.
In this example, at first we create a process and this process prints the message "hi!! I am Python" and then shares the data across.
Queues
When we pass data between processes then at that time we can use Queue object.
Example
import multiprocessing def evenno(numbers, q): for n in numbers: if n % 2 == 0: q.put(n) if __name__ == "__main__": q = multiprocessing.Queue() p = multiprocessing.Process(target=evenno, args=(range(10), q)) p.start() p.join() while q: print(q.get())
Output
0 2 4 6 8
In this example, at first create a function that checks weather a number is even or not. If the number is even, then insert it at the end of the queue. Then we create a queue object and a process object then we start the process.
And finally check whether the queue is empty or not.
When we print the numbers, at first we print the value which is in front of the queue then next one and so on.
Locks
When we want that only one process is executed at a time in that situation Locks is use. That means that time blocks other process from executing similar code. Lock will be released after the process gets completed.
Example using Locks method
Example
from multiprocessing import Process, Lock
def dispmay_name(l, i):
l.acquire()
print ('Hi', i)
l.release()
if __name__ == '__main__':
my_lock = Lock()
my_name = ['Aadrika', 'Adwaita', 'Sakya', 'Sanj']
for name in my_name:
Process(target=dispmay_name, args=(my_lock,name)).start()Output
Hi Aadrika Hi Adwaita Hi Sakya Hi Sanj
Logging
The multiprocessing module also provides logging module to ensure that, if the logging package doesn't use locks function, the messages between processes mixed up during execution.
Example
import multiprocessing, logging
logger = multiprocessing.log_to_stderr()
logger.setLevel(logging.INFO)
logger.warning('Error has occurred')In this example at first we import the logging and multiprocessing module then we use multiprocessing.log_to_stderr() method. And it call get_logger() as well as adding to sys.stderr and finally we set the level of logger and convey the message.
4K+ Views
