How to merge multiple folders into one folder using Python?

Data in today’s time is generated in large volumes, and organizing that data can be a challenging task. One of the most common issues people face is merging multiple folders into one. It can be quite frustrating to search for a specific file when you have multiple folders to navigate through. Fortunately, Python provides a simple solution to this problem.

In this article, we will learn how to merge multiple folders into one folder using Python. To merge multiple folders into one, we will be using the os module that provides a portable way of using operating system−dependent functionality like reading or writing to the file system.

Steps to merge multiple folders into one folder using Python

Below are the complete steps to merge multiple folders into one folder using Python:

Step 1: Import the OS and shutil modules

The first step in merging multiple folders into one is to import the required modules like os and shutil modules in Python. Here, the os module is used in providing the functions to interact with the operating system, and the shutil module provides functions to operate on files and directories. Below is how we can import these modules:

import os
import shutil

Step 2: Define the folder's path

The next step in the process is to define the source and destination folders. The source folder is the folder containing the folders that we want to merge, and the destination folder is the folder in which we want to merge all the folders. Below is how we can define the path to source and destination folders:

mysource_folder = "yourpath/to/source/folder"
mydestination_folder = "yourpath/to/destination/folder"

Step 3: Merge the folders

After defining the paths, we will use the os.walk() function to recursively iterate over all the folders and files in the source folder. We will then use the shutil.copy2() function to copy all the files to the destination folder. Below is how we merge the different folders:

for root, dirs, files in os.walk(mysource_folder):
    for file in files:
        mysrc_file = os.path.join(root, file)
        shutil.copy2(mysrc_file, mydestination_folder)

In the above syntax, we first use os.walk() to iterate over all the folders and files in the source folder. The os.walk() function returns a tuple consisting of the root directory, a list of directories, and a list of files. We then use a for loop to iterate over all the files in the files list.

After looping over all files in the list, we now use the os.path.join() function to create the source file path by joining the root directory and the file name. Finally, we use the shutil.copy2() function to copy the file to the destination folder.

That’s all! Now we have seen the complete steps to merge the folders into one folder using Python. Let’s see some of examples merging different or multiple folders into one.

Example 1: Using os.listdir() and shutil.copy2()

In the below example, our approach to merge multiple folders into one folder is to use os.listdir() to loop through all files and subdirectories in the source folder. For each file or directory, we check if it is a file or a directory using os.path.isfile() and os.path.isdir() respectively. If it is a file, we simply copy it to the destination folder using shutil.copy2(). If it is a directory, we loop through all files and subdirectories in the directory and copy each subitem to the destination folder using shutil.copy2().

import os
import shutil

# Define your source and destination folders
mysource_folder = "path/to/source/folder"
mydestination_folder = "path/to/destination/folder"

# Loop through all files and subdirectories in source folder
for item in os.listdir(mysource_folder):
    
    # Create full path for the item
    myitem_path = os.path.join(mysource_folder, item)
    
    # Check if item is a file
    if os.path.isfile(myitem_path):
        
        # Copy the file to the destination folder
        shutil.copy2(myitem_path, mydestination_folder)
        
    # Check if item is a directory
    elif os.path.isdir(myitem_path):
        
        # Loop through all files and subdirectories in the directory
        for subitem in os.listdir(myitem_path):
            
            # Create full path for the subitem
            mysubitem_path = os.path.join(myitem_path, subitem)
            
            # Copy the subitem to the destination folder
            shutil.copy2(mysubitem_path, mydestination_folder)

Output

Before:

After:

Example 2: Using os.walk() and os.path.join()

In the above example, our approach to merge multiple folders into one folder is to use os.walk() to recursively loop through all files and subdirectories in the source folder. For each file, we create the full path using os.path.join() and then rename the file to the destination folder using os.rename(). For each subdirectory, we loop through all files and subdirectories in the subdirectory using os.walk() and then rename each subfile to the destination folder using os.rename().

import os
import shutil

# Define your source and destination folders
mysource_folder = "path/to/source/folder"
mydestination_folder = "path/to/destination/folder"

# Loop through all files and subdirectories in source folder
for root, dirs, files in os.walk(mysource_folder):
    
    # Loop through all files in current directory
    for file in files:
        
        # Create full path for the file
        file_path = os.path.join(root, file)
        
        # Copy the file to the destination folder
        os.rename(file_path, os.path.join(mydestination_folder, file))
        
    # Loop through all subdirectories in current directory
    for dir in dirs:
        
        # Create full path for the subdirectory
        dir_path = os.path.join(root, dir)
        
        # Copy all files and subdirectories in the subdirectory to the destination folder
        for subroot, subdirs, subfiles in os.walk(dir_path):
            for subfile in subfiles:
                subfile_path = os.path.join(subroot, subfile)
os.rename(subfile_path, os.path.join(mydestination_folder, subfile))

Output

Before:

After:

Example 3: Using distutils.dir_util.copy_tree()

In the above example, we define the source_folder and destination_folder variables with the appropriate paths. After which we use the copy_tree() function, passing the source_folder and destination_folder as arguments. This function recursively copies the contents of the source_folder to the destination_folder, effectively merging all the folders into one.

import os
from distutils.dir_util import copy_tree

# Define your source and destination folders
mysource_folder = "path/to/source/folder"
mydestination_folder = "path/to/destination/folder"

# Merge folders using copy_tree()
copy_tree(mysource_folder, mydestination_folder)

# Optional: Used in removing the source folders after merging
for root, dirs, files in os.walk(mysource_folder, topdown=False):
    for dir in dirs:
        dir_path = os.path.join(root, dir)
        os.rmdir(dir_path)
    os.rmdir(root)

Output

Before:

After:

Conclusion

Organizing data generated in large volumes can be a challenging task. Merging multiple folders into one can be a daunting task, especially when you have multiple folders to navigate through. Python provides a simple solution to this problem by using modules like os and shutil. The steps to merge multiple folders into one include importing the required modules, defining the source and destination folders, and merging the folders using os.walk() and shutil.copy2() functions. Alternatively, os.listdir() and os.path.join() can also be used to merge folders. These methods allow users to quickly and efficiently merge multiple folders into one folder.