How to run multiple Python files in a folder one after another?

The subprocess module can be used to run multiple Python files in a folder one after another. Running multiple files one after another is required in various situations like when processing large data sets or performing complex analysis. In this article, we will discuss the steps involved in running multiple Python files in a folder sequentially with examples.

Method 1: Using the subprocess() method

Steps for creating and running multiple Python files sequentially

Step 1: Create multiple Python files to run

We need to have three Python files in a folder to run them sequentially. So the first step is to create three Python files named file1.py,file2.py and file3.py. We will write a simple print statement in each Python file to just check if the files are running sequentially.

The content of the files is shown below −

File1.py

print("This is file 1")

File2.py

print("This is file 2")

File3.py

print("This is file 3")

Step 2: Create a Python file to run the other files

Now, we will create a Python file that will run the script to run the files - file1.py,file2.py, and file3.py sequentially. We will create a file named run_script.py which will contain the program to run the other three files using the subprocess module.

In this file, we will use the subprocess.run() function to run each of the other Python files. The shell=True argument is used to run the command in a shell environment, which is necessary for running multiple files.

The content of the run_script.py is shown below −

import subprocess

subprocess.run("python file1.py", shell=True)
subprocess.run("python file2.py", shell=True)
subprocess.run("python file3.py", shell=True)

Step 3: Run the run_script file

In the final step, we will run the run_script.py file which will run the other three files sequentially and print the output. To run the file simply type python run_script.py in your terminal or command prompt.

The output is shown below −

This is file 1
This is file 2
This is file 3

Method 2: Importing and running the files using the run() method

The run() function is another way to execute multiple Python files in a specific order. In this method, you create a Python file that calls the run() function from each of the other Python files

The content of the individual files is shown below −

File1.py

def run():
   print("This is file 1")

File2.py

def run():
   print("This is file 2")

File3.py

def run():
   print("This is file 3")

Now we can import these files in our run_script.py file one after another and run each of the files using the run function.

The content of the run_script.py is shown below −

import file1
import file2
import file3

file1.run()
file2.run()
file3.run()

Output

This is file 1
This is file 2
This is file 3

Method 3: Using the Command Line

We can also use the command line to execute multiple Python files in a specific order. In this method, you use the && operator to chain the execution of multiple Python files.

python file1.py && python file2.py && python file3.py

Output

This is file 1
This is file 2
This is file 3

Method 4: Using Bash Script

We can also create a Bash script to execute multiple Python files in a specific order. In this method, you create a script that uses the Python command to execute each Python file.

Example contents of the run.sh Bash file −

#!/bin/bash

python file1.py
python file2.py
python file3.py

Output

This is file 1
This is file 2
This is file 3

Applications of Running multiple files in a folder sequentially

Running multiple Python files in a folder one after another can be useful in several scenarios. Here are a few examples &inus;

• Data processing − When processing large data sets, it may be necessary to split the processing into several Python files. For example, one file may clean the data, while another file analyzes the data, and a third file generates reports. By running these files sequentially, you can process the data efficiently and effectively.

• Machine learning − In machine learning, it is common to split the training and testing of models into several Python files. For example, one file may preprocess the data, while another file trains the model, and a third file evaluates the performance of the model. By running these files sequentially, you can efficiently train and test machine learning models.

• Web development − In web development, you may have several Python files that make up a web application. For example, one file may contain the backend logic, while another file contains the frontend templates. By running these files sequentially, you can test the web application and ensure that all components are working correctly.

• Automation − In automation tasks, it may be necessary to run several Python scripts in a specific order. For example, one file may download data from a website, while another file processes the data, and a third file generates reports. By running these files sequentially, you can automate complex tasks and ensure that they are executed in the correct order.

In all of these scenarios, running multiple Python files in a folder one after another can help you to efficiently execute complex tasks and ensure that all components are working correctly.

Conclusion

In this article, we discussed how we can run multiple Python files in a folder using the subprocess module in Python. We can also use import the individual file in a single file and run them sequentially using the run() method. We can use the command line to run multiple files one after another using & operator. We can also use a bash script to write a script that runs all the files one after another. By following the steps explained in this article you will easily be able to run multiple Python files sequentially.