List all files of certain type in a directory using Python

Python's flexible features and strong libraries make manipulating files and directories a breeze. One common requirement is listing all files of a specific type in a directory. This tutorial will walk you through various approaches using real-world examples to demonstrate Python's proficiency with filesystem operations.

Introduction to Python's Os and Glob Libraries

The standard Python library includes several modules for filesystem operations. The os and glob modules are two powerful options:

• os module Provides tools for communicating with the operating system, including operations for creating, deleting, and browsing directories.

• glob module Uses Unix shell-style wildcards to find pathnames matching a pattern. It supports wildcard characters like * and ?, making it ideal for file filtering.

Using os and fnmatch Modules

Here's how to list all .txt files in a directory using the os and fnmatch modules:

import os
import fnmatch

def list_files(directory, filetype):
    for file in os.listdir(directory):
        if fnmatch.fnmatch(file, filetype):
            print(file)

# Create some sample files for demonstration
os.makedirs('sample_dir', exist_ok=True)
with open('sample_dir/file1.txt', 'w') as f:
    f.write('Sample content')
with open('sample_dir/file2.txt', 'w') as f:
    f.write('Sample content')
with open('sample_dir/image.jpg', 'w') as f:
    f.write('Sample content')

list_files('sample_dir', '*.txt')

file1.txt
file2.txt

The os.listdir() function returns a list of all files and directories in the specified path. The fnmatch() function checks if each file matches the specified pattern.

Using glob Module

The glob module provides a more direct approach for pattern matching:

import glob
import os

def list_files(directory, filetype):
    pattern = os.path.join(directory, filetype)
    for file in glob.glob(pattern):
        print(os.path.basename(file))

# Using the same sample directory
list_files('sample_dir', '*.txt')

file1.txt
file2.txt

The glob.glob() function returns a list of paths matching the pathname pattern. We use os.path.join() for proper path construction and os.path.basename() to display just the filename.

Recursive Search Using glob Module

To search for files in subdirectories recursively, use the ** pattern:

import glob
import os

def list_files_recursive(directory, filetype):
    pattern = os.path.join(directory, '**', filetype)
    for file in glob.glob(pattern, recursive=True):
        print(file)

# Create nested directory structure
os.makedirs('sample_dir/subdir', exist_ok=True)
with open('sample_dir/subdir/nested.txt', 'w') as f:
    f.write('Nested file content')

list_files_recursive('sample_dir', '*.txt')

sample_dir/file1.txt
sample_dir/file2.txt
sample_dir/subdir/nested.txt

The ** pattern tells glob to search recursively through all subdirectories. The recursive=True parameter enables this functionality.

Using os.walk() for Recursive Search

The os.walk() function provides another approach for recursive directory traversal:

import os
import fnmatch

def list_files_walk(directory, filetype):
    for dirpath, dirnames, files in os.walk(directory):
        for file in files:
            if fnmatch.fnmatch(file, filetype):
                print(os.path.join(dirpath, file))

list_files_walk('sample_dir', '*.txt')

sample_dir/file1.txt
sample_dir/file2.txt
sample_dir/subdir/nested.txt

The os.walk() function traverses the directory tree, yielding the directory path, subdirectory names, and filenames for each directory visited.

Comparison of Methods

Conclusion

Python offers multiple approaches for listing files by type. Use glob for simple pattern matching, os.walk() for complex recursive operations, and os.listdir() with fnmatch for basic filtering needs. These tools are essential for file organization, log analysis, and system automation tasks.