How to Implement Priority Queue in Python?

PythonServer Side ProgrammingProgramming

Introduction...

The queue module provides a first-in, first-out (FIFO), Last-in, First out (LIFO) data structure suitable for multi-threaded programming. Queues can be used to pass data or any wide range of information e.g. session details, paths, variables,.. between creator and consumer threads safely. Locking is generally handled for the caller.

Note : This discussion assumes you already understand the general nature of a queue. If you do not, you may want to read some of the references before continuing.

1. Let us implement a basic FIFO Queue.

import queue
fifo = queue.Queue()

# put numbers into queue
for i in range(5):
fifo.put(i)

# if not empty get the numbers from queue
print(f"Ouput \n")
while not fifo.empty():
print(f" {fifo.get()} ")

Output

0
1
2
3
4

2. The above example uses a single thread to show how the elements are removed from the queue in the same order in which they are inserted.

3. Let us implement a basic LIFO Queue.

import queue
lifo = queue.LifoQueue()

# put numbers into queue
for i in range(5):
lifo.put(i)

print(f"Ouput \n")
# if not empty get the numbers from queue
while not lifo.empty():
print(f" {lifo.get()} ")

Output

4
3
2
1
0

4. The above example shows, that the most recently put into the queue is removed by get.

5. Finally, we will see how to implement priority queue.

Sometimes the processing order of the items in a queue needs to be based on priority of those items, rather than just the order they are created or added to the queue. For instance, a business critical job running in producton requires highest CPU and precedence over a print something job that a developer wants to print. PriorityQueue uses the sort order of the contents of the queue to decide which item to retrieve.

import queue
import threading

# Class to get the priority and description and validate the priority
class Job:
def __init__(self, priority, description):
self.priority = priority
self.description = description
print('New job:', description)
return

def __eq__(self, other):
try:
return self.priority == other.priority
except AttributeError:
return NotImplemented

def __lt__(self, other):
try:
return self.priority < other.priority
except AttributeError:
return NotImplemented

# create a priority queue and define the priority
q = queue.PriorityQueue()
q.put(Job(90, 'Developer-Print job'))
q.put(Job(2, 'Business-Report job'))
q.put(Job(1, 'Business-Critical Job'))

# process the job
def process_job(q):
while True:
next_job = q.get()
print(f" *** Now, Processing the job - {next_job.description}")
q.task_done()

# define the threads
workers = [
threading.Thread(target=process_job, args=(q,)),
threading.Thread(target=process_job, args=(q,)), ]

# call the threads and join them.
for w in workers:
w.setDaemon(True)
w.start()

q.join()

Output

job: Developer-Print job
New job: Business-Report job
New job: Business-Critical Job

Output

*** Now, Processing the job - Business-Critical Job
*** Now, Processing the job - Business-Report job
*** Now, Processing the job - Developer-Print job

6. This example has multiple threads consuming the jobs, which are processed based on the priority of items in the queue at the time get() was called. The order of processing is based on the business criticallity irrespective of what order they have been added.

raja
Published on 10-Nov-2020 05:19:39
Advertisements