Run Python script from Node.js using child process spawn() method?

Node.js and Python are two popular languages among developers. While Node.js excels at web development, Python provides extensive libraries for scientific computing, AI, and machine learning. Fortunately, we can combine both by running Python scripts from Node.js using the child_process module.

The spawn() method creates a child process to run Python scripts in the background and stream results back to Node.js in real-time.

Creating the Python Script

First, let's create a Python script that accepts command-line arguments and outputs messages over time ?

# myscript.py
import sys, getopt, time

def main(argv):
    argument = ''
    usage = 'usage: myscript.py -f <sometext>'
    
    # parse incoming arguments
    try:
        opts, args = getopt.getopt(argv, "hf:", ["foo="])
    except getopt.GetoptError:
        print(usage)
        sys.exit(2)
        
    for opt, arg in opts:
        if opt == '-h':
            print(usage)
            sys.exit()
        elif opt in ("-f", "--foo"):
            argument = arg
            
    # print output with delays
    print("Start : %s" % time.ctime())
    time.sleep(2)
    print('Foo is')
    time.sleep(2)
    print(argument)
    print("End : %s" % time.ctime())

if __name__ == "__main__":
    main(sys.argv[1:])

Testing the Python Script

Run the script from command line to verify it works ?

> python myscript.py -f "Hello, Python"
Start : Wed Feb 20 07:52:45 2019
Foo is
Hello, Python
End : Wed Feb 20 07:52:49 2019

You can also use the long form flag or help option ?

> python myscript.py --foo "Hello, Python"
> python myscript.py -h
usage: myscript.py -f <sometext>

Node.js Integration with spawn()

Now let's create a Node.js script that spawns the Python process and captures its output in real-time ?

// server.js
const path = require('path')
const {spawn} = require('child_process')

/**
 * Run python myscript, pass in `-u` to prevent output buffering
 * @return {ChildProcess}
 */
function runScript(){
    return spawn('python', [
        "-u",
        path.join(__dirname, 'myscript.py'),
        "--foo", "some value for foo",
    ]);
}

const subprocess = runScript()

// Handle stdout data
subprocess.stdout.on('data', (data) => {
    console.log(`data: ${data}`);
});

// Handle stderr data
subprocess.stderr.on('data', (data) => {
    console.log(`error: ${data}`);
});

// Handle process completion
subprocess.on('close', (code) => {
    console.log(`Process finished with exit code ${code}`);
});

Output

When you run the Node.js script, you'll see the Python output streamed in real-time ?

> node server.js
data: Start : Wed Feb 20 10:56:11 2019
data: Foo is
data: some value for foo
data: End : Wed Feb 20 10:56:15 2019
Process finished with exit code 0

Key Points

Conclusion

Using spawn() allows seamless integration between Node.js and Python, enabling real-time output streaming. Remember to use the -u flag to prevent output buffering and handle both stdout and stderr events for robust error handling.