async.queue() Method in Node.js

The async module provides different functionalities to work with asynchronous JavaScript in a Node.js application. The async.queue() method creates a queue object for concurrent processing of tasks, allowing multiple items to be processed simultaneously based on a specified concurrency limit.

Installation

First, initialize your Node.js project and install the async module:

npm init
npm install --save async

Then import the async module in your program:

const async = require('async');

Syntax

async.queue(worker, concurrency)

Parameters

• worker - A function that processes each task added to the queue. It receives the task data and a callback function.

• concurrency - The number of tasks to process simultaneously (default is 1).

Queue Methods and Properties

The queue object returned by async.queue() provides several useful methods:

• push(task, callback) - Adds a task to the end of the queue

• unshift(task, callback) - Adds a task to the front of the queue (priority)

• length() - Returns the number of tasks waiting to be processed

• started - Boolean property indicating if the queue has started processing

• drain - Callback executed when all tasks are completed

• pause() - Pauses queue processing

• resume() - Resumes queue processing

• kill() - Removes all remaining tasks and stops the queue

• idle - Boolean property indicating if the queue is idle

Example

Here's a practical example demonstrating queue operations with priority handling:

// Including the async module
const async = require('async');

// Creating an array for all elements execution
const tasks = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10];

// Initializing the queue
const queue = async.queue((task, executed) => {
    console.log("Currently Busy Processing Task " + task);
    
    setTimeout(() => {
        // Number of tasks remaining and to be processed
        const tasksRemaining = queue.length();
        executed(null, {task, tasksRemaining});
    }, 1000);
    
}, 1); // concurrency value = 1

// Queue is idle initially as no elements are there...
console.log(`Queue Started ? ${queue.started}`);

// Adding each task from the tasks list
tasks.forEach((task) => {
    // Adding the task 5 to the head for priority execution
    if(task == 5) {
        queue.unshift(task, (error, {task, tasksRemaining}) => {
            if(error) {
                console.log(`An error occurred while processing task ${task}`);
            } else {
                console.log(`Finished processing task ${task}. ${tasksRemaining} tasks remaining`);
            }
        });
        // Adding all the tasks at tail to be executed except task 5
    } else {
        queue.push(task, (error, {task, tasksRemaining}) => {
            if(error) {
                console.log(`An error occurred while processing task ${task}`);
            } else {
                console.log(`Finished processing task ${task}. ${tasksRemaining} tasks remaining`);
            }
        });
    }
});

// Executes the callback when the queue is done processing all the tasks
queue.drain(() => {
    console.log('All items are successfully processed !');
});

// Checking if the queue is started after adding tasks
console.log(`Queue Started ? ${queue.started}`);

Queue Started ? false
Queue Started ? true
Currently Busy Processing Task 5
Finished processing task 5. 9 tasks remaining
Currently Busy Processing Task 1
Finished processing task 1. 8 tasks remaining
Currently Busy Processing Task 2
Finished processing task 2. 7 tasks remaining
Currently Busy Processing Task 3
Finished processing task 3. 6 tasks remaining
Currently Busy Processing Task 4
Finished processing task 4. 5 tasks remaining
Currently Busy Processing Task 6
Finished processing task 6. 4 tasks remaining
Currently Busy Processing Task 7
Finished processing task 7. 3 tasks remaining
Currently Busy Processing Task 8
Finished processing task 8. 2 tasks remaining
Currently Busy Processing Task 9
Finished processing task 9. 1 tasks remaining
Currently Busy Processing Task 10
Finished processing task 10. 0 tasks remaining
All items are successfully processed !

Key Features

Notice how task 5 was processed first due to using unshift() instead of push(). This demonstrates priority handling in the queue. The concurrency value of 1 ensures tasks are processed sequentially, but you can increase this value to process multiple tasks simultaneously.

Conclusion

The async.queue() method provides powerful task management with features like priority queuing, concurrency control, and lifecycle callbacks. It's ideal for managing workloads that need controlled parallel processing in Node.js applications.