How to merge multiple folders into one folder using Python?

Data in today's time is generated in large volumes, and organizing that data can be a challenging task. One of the most common issues people face is merging multiple folders into one. It can be quite frustrating to search for a specific file when you have multiple folders to navigate through. Fortunately, Python provides a simple solution to this problem.

In this article, we will learn how to merge multiple folders into one folder using Python. To merge multiple folders into one, we will be using the os and shutil modules that provide a portable way of using operating systemdependent functionality like reading or writing to the file system.

Steps to Merge Multiple Folders

Below are the complete steps to merge multiple folders into one folder using Python ?

Step 1: Import Required Modules

The first step is to import the required modules os and shutil. The os module provides functions to interact with the operating system, and the shutil module provides functions to operate on files and directories ?

import os
import shutil

Step 2: Define Folder Paths

Next, define the source and destination folders. The source folder contains the folders you want to merge, and the destination folder is where all files will be merged ?

import os
import shutil

# Create sample directory structure for demonstration
os.makedirs("sample_source/folder1", exist_ok=True)
os.makedirs("sample_source/folder2", exist_ok=True)
os.makedirs("sample_destination", exist_ok=True)

# Create sample files
with open("sample_source/folder1/file1.txt", "w") as f:
    f.write("Content from folder1")

with open("sample_source/folder2/file2.txt", "w") as f:
    f.write("Content from folder2")

source_folder = "sample_source"
destination_folder = "sample_destination"
print("Sample structure created successfully!")

Sample structure created successfully!

Step 3: Merge the Folders

Use os.walk() to recursively iterate over all folders and files in the source folder, then copy files using shutil.copy2() ?

import os
import shutil

# Previous setup code...
os.makedirs("demo_source/folder1", exist_ok=True)
os.makedirs("demo_source/folder2", exist_ok=True)
os.makedirs("demo_destination", exist_ok=True)

with open("demo_source/folder1/file1.txt", "w") as f:
    f.write("Content from folder1")
with open("demo_source/folder2/file2.txt", "w") as f:
    f.write("Content from folder2")

source_folder = "demo_source"
destination_folder = "demo_destination"

# Merge folders
for root, dirs, files in os.walk(source_folder):
    for file in files:
        src_file = os.path.join(root, file)
        shutil.copy2(src_file, destination_folder)
        print(f"Copied: {file}")

print("Merge completed!")

Copied: file1.txt
Copied: file2.txt
Merge completed!

Method 1: Using os.listdir() and shutil.copy2()

This approach uses os.listdir() to loop through files and subdirectories, checking each item with os.path.isfile() and os.path.isdir() ?

import os
import shutil

# Define source and destination folders
source_folder = "path/to/source/folder"
destination_folder = "path/to/destination/folder"

# Loop through all items in source folder
for item in os.listdir(source_folder):
    item_path = os.path.join(source_folder, item)
    
    # Check if item is a file
    if os.path.isfile(item_path):
        shutil.copy2(item_path, destination_folder)
        
    # Check if item is a directory
    elif os.path.isdir(item_path):
        # Loop through all files in the subdirectory
        for subitem in os.listdir(item_path):
            subitem_path = os.path.join(item_path, subitem)
            if os.path.isfile(subitem_path):
                shutil.copy2(subitem_path, destination_folder)

Method 2: Using os.walk() with File Moving

This method uses os.walk() to recursively traverse directories and moves files using os.rename() instead of copying ?

import os

# Define source and destination folders
source_folder = "path/to/source/folder"
destination_folder = "path/to/destination/folder"

# Walk through all files and subdirectories
for root, dirs, files in os.walk(source_folder):
    for file in files:
        file_path = os.path.join(root, file)
        # Move file to destination folder
        os.rename(file_path, os.path.join(destination_folder, file))

Method 3: Using distutils.dir_util.copy_tree()

The copy_tree() function recursively copies entire directory structures, making it ideal for merging folder contents ?

import os
from distutils.dir_util import copy_tree

# Define source and destination folders
source_folder = "path/to/source/folder"
destination_folder = "path/to/destination/folder"

# Merge folders using copy_tree()
copy_tree(source_folder, destination_folder)

# Optional: Remove source folders after merging
for root, dirs, files in os.walk(source_folder, topdown=False):
    for dir in dirs:
        dir_path = os.path.join(root, dir)
        os.rmdir(dir_path)
    os.rmdir(root)

Comparison

Conclusion

Python provides multiple approaches to merge folders using os and shutil modules. Use os.walk() with shutil.copy2() for most scenarios, or copy_tree() for preserving directory structures. Choose the method that best fits your specific requirements.