How to use the Subprocess Module in Python?


Understanding Process -

When you code and execute a program on Windows, MAC or Linux, your Operating System creates a process(single).It uses system resources like CPU, RAM, Disk space and also data structures in the operating system’s kernel. A process is isolated from other processes—it can’t see what other processes are doing or interfere with them.

Note: This code has to be run on Linux like sytems. When executed on windows might throw exceptions.

Goals of Operating System -

The main twin goals of OS are to spread the work of the process fairly and be responsive to the user. These are acheived by keeping a track of all the running processes, giving each a little time to run and then switching to another. You can see the state of your processes with graphical interfaces such as the Task Manager on Windows-based computers, the Mac’s Activity Monitor (macOS),or the top command in Linux.

Being a programmer, we can access the process data from our own program. But How? Simply by using standard library OS module. I will show you few examples.

# This script works only on linux/unix
import os
print(f" *** Process ID - {os.getpid()}")
print(f" *** My User ID - {os.getuid()} and My Group ID - {os.getgid()} ")
print(f" *** Current Working Directory is - {os.getcwd()}")

Running and spinning up a new system process can be quite useful to developers and system administrators who want to automate specific operating system tasks.

Python has a subprocess module, which can spin a new processes, send and receive information from the processes, and also handle error and return codes.

The official Python documentation recommends the subprocess module for accessing system commands.

The subprocess call() function waits for the called command to finish reading the output. We will see couple of examples below to extract the systems disk space information.

Below code will execute df -h command and captures the information. The output is then captured to a pandas dataframe for any further processing.

Example

# python code to create a subprocess for extracting disk space on linux using df -h

from io import StringIO
import pandas as pd
import subprocess
import ast
diskspace = "df"
diskspace_arg = "-h"

sp = subprocess.Popen([diskspace,diskspace_arg], stdout=subprocess.PIPE)
b = StringIO(sp.communicate()[0].decode('utf-8'))
df = pd.read_csv(b, sep=",")
print(df)

Output

<_io.StringIO object at 0x7ff67ef52798>
Filesystem Size Used Avail Use% Mounted on
0 devtmpfs 7.8G 0 7.8G 0% /dev
1 tmpfs 7.8G 0 7.8G 0% /dev/shm
2 tmpfs 7.8G 33M 7.8G 1% /run
3 tmpfs 7.8G 0 7.8G 0% /sys/fs/...
4 /dev/xvda2 20G 16G 4.3G 79% /
5 /dev/xvdb 246G 16G 218G 7% /tdm
6 tmpfs 1.6G 0 1.6G 0% /run/use...

To get a more detailed output with subprocess see below code.

Example

from io import StringIO
import pandas as pd
import subprocess
def uname_func():
uname = "uname"
uname_arg = "-a"
user_info = subprocess.call([uname, uname_arg])
return user_info

def disk_func():
diskspace = "pydf"
diskspace_arg = "-a"
discinfo_df = diskspace
stdout = subprocess.check_output([diskspace, diskspace_arg])
return stdout

def main():
userinfo = uname_func()
discinfo = disk_func()
print("Displaying values now... ")
# print(stdout.decode('utf-8'))
print(discinfo.decode('utf-8'))
print(type(discinfo.decode('utf-8')))
content = discinfo.decode('utf-8').split("\n")
print(content)

main()

Output

Linux ip-00-000-00-000.xxxx.xxxx.xx.xx 0.00.0-000.el7.x86_64 #1 SMP Tue Aug 18 14:50:17 EDT 2020 x86_64 x86_64 x86_64 GNU/Linux
Displaying values now...
Filesystem Size Used Avail Use% Mounted on
/dev/xvda2 20G 16G 4318M 78.9 [#####.] /
devtmpfs 7918M 0 7918M 0.0 [......] /dev
hugetlbfs 0 0 0 - [......] /dev/hugepages
mqueue 0 0 0 - [......] /dev/mqueue
devpts 0 0 0 - [......] /dev/pts
tmpfs 7942M 0 7942M 0.0 [......] /dev/shm
proc 0 0 0 - [......] /proc
binfmt_misc 0 0 0 - [......] /proc/sys/fs/binfmt_misc
tmpfs 7942M 32M 7909M 0.4 [......] /run
tmpfs 1588M 0 1588M 0.0 [......] /run/user/1000
sysfs 0 0 0 - [......] /sys
tmpfs 7942M 0 7942M 0.0 [......] /sys/fs/cgroup
cgroup 0 0 0 - [......] /sys/fs/cgroup/blkio
cgroup 0 0 0 - [......] /sys/fs/cgroup/cpu,cpuacct
cgroup 0 0 0 - [......] /sys/fs/cgroup/cpuset
cgroup 0 0 0 - [......] /sys/fs/cgroup/devices
cgroup 0 0 0 - [......] /sys/fs/cgroup/freezer
cgroup 0 0 0 - [......] /sys/fs/cgroup/hugetlb
cgroup 0 0 0 - [......] /sys/fs/cgroup/memory
cgroup 0 0 0 - [......] /sys/fs/cgroup/net_cls,net_prio
cgroup 0 0 0 - [......] /sys/fs/cgroup/perf_event
cgroup 0 0 0 - [......] /sys/fs/cgroup/pids
cgroup 0 0 0 - [......] /sys/fs/cgroup/systemd
pstore 0 0 0 - [......] /sys/fs/pstore
configfs 0 0 0 - [......] /sys/kernel/config
debugfs 0 0 0 - [......] /sys/kernel/debug
securityfs 0 0 0 - [......] /sys/kernel/security
/dev/xvdb 246G 16G 218G 6.4 [......] /tdm

Updated on: 09-Nov-2020

865 Views

Kickstart Your Career

Get certified by completing the course

Get Started
Advertisements