Data Structure
Networking
RDBMS
Operating System
Java
MS Excel
iOS
HTML
CSS
Android
Python
C Programming
C++
C#
MongoDB
MySQL
Javascript
PHP
Physics
Chemistry
Biology
Mathematics
English
Economics
Psychology
Social Studies
Fashion Studies
Legal Studies
- Selected Reading
- UPSC IAS Exams Notes
- Developer's Best Practices
- Questions and Answers
- Effective Resume Writing
- HR Interview Questions
- Computer Glossary
- Who is Who
How to implement Concurrency with Threads in Python?
Introduction
Python has different approaches like using threads, subprocesses, generators and other tricks for concurrent programming.Before we go on and implement threads, let us understand what exactly is concurrency.
Concurrency is a piece of logic Within a single program that allows to open up many distinct paths of execution, including separate streams of I/O, running SQL queries, so on in a way that the execution seems to be both simultaneous and independent of each other.
How to do it..
First we create a single thread to go through the site urls and later look at how to use threading concepts to speed up the program.
# Step 1 - Make a list of website url you want to visit today import requests tutorialpoints_url = ['https://www.tutorialspoint.com/python/index.htm', 'https://www.tutorialspoint.com/cplusplus/index.htm', 'https://www.tutorialspoint.com/java/index.htm', 'https://www.tutorialspoint.com/html/index.htm', 'https://www.tutorialspoint.com/cprogramming/index.htm']
# function to request the url passed and return the status code
def visit_site(site_url):
"""
Makes a GET request to a website URL and prints response information
"""
response = requests.get(site_url)
print(f' *** {site_url} returned {response.status_code} after {response.elapsed} seconds')# Let us create a single thread to fetch the response if __name__ == '__main__': for site_url in tutorialpoints_url: visit_site(site_url) print(f" *** end of the program ***")
*** https://www.tutorialspoint.com/python/index.htm returned 200 after 0:00:00.091103 seconds *** https://www.tutorialspoint.com/cplusplus/index.htm returned 200 after 0:00:00.069889 seconds *** https://www.tutorialspoint.com/java/index.htm returned 200 after 0:00:00.075864 seconds *** https://www.tutorialspoint.com/html/index.htm returned 200 after 0:00:00.075270 seconds *** https://www.tutorialspoint.com/cprogramming/index.htm returned 200 after 0:00:00.077984 seconds *** end of the program ***
What did you observe from the output ? The site url are processed sequentially, just imagine if there are 100's of url from various geographical location you want to visit then your program might be spending a lot of time waiting for a response from a server.
Let us now write a threaded program to submit the requests in parallel and proceed to next step without waiting.
from threading import Thread
# function to request the url passed and return the status code
def visit_site(site_url):
"""
Makes a GET request to a website URL and prints response information
"""
response = requests.get(site_url)
print(f' *** {site_url} returned {response.status_code} after {response.elapsed} seconds')
# Loop through the website url and create threads for each url
if __name__ == '__main__':
for site_url in tutorialpoints_url:
t = Thread(target=visit_site, args=(site_url,))
t.start()*** https://www.tutorialspoint.com/python/index.htm returned 200 after 0:00:00.082176 seconds *** https://www.tutorialspoint.com/html/index.htm returned 200 after 0:00:00.086269 seconds *** https://www.tutorialspoint.com/java/index.htm returned 200 after 0:00:00.100746 seconds *** https://www.tutorialspoint.com/cplusplus/index.htm returned 200 after 0:00:00.120744 seconds *** https://www.tutorialspoint.com/cprogramming/index.htm returned 200 after 0:00:00.111489 seconds
Discussion..
The threading library can be used to execute any Python callable in its own thread.
start() method invokes the visit_site function with the site_url arguments.
Threads once started are executed in their own thread which is fully manged by the Operating system.
Now if you want to see if your threads are alive or dead (completed) we can use is_alive function.
if t.is_alive():
print(f' *** {t} is Still executing')
else:
print(f' *** {t} is Completed')*** <Thread(Thread-10, stopped 4820)> is Completed
187 Views
