How to list non-hidden files and directories in windows using Python?

Listing files and directories using Python is a common task in many applications, from file management tools to automated scripts. However, when working on the Windows Operating System, hidden files and folders are marked using specific file attributes, which can clutter the output if not handled properly.

Unlike Unix-based systems, where hidden files typically start with a dot (such as .hidden), Windows uses metadata to mark files as hidden. In this article, we'll explore several methods to list only non-hidden files and directories on Windows systems.

Basic Filtering by Name Prefix

This basic filtering method checks if the filename starts with a dot. This approach works on Unix-like systems but is insufficient on Windows because hidden files do not use this naming convention.

Example

Following is an example of basic name-based filtering ?

import os

files = [f for f in os.listdir('.') if not f.startswith('.')]
print(files)

Following is the output of the above program ?

['config.py', 'empty_file.txt', 'file1.txt', 'file2.txt', 'helper.py']

Note: This will not exclude hidden files on Windows that do not start with a dot.

Using os.scandir() with Windows API Check

The most accurate way to detect hidden files on Windows is by checking their system attributes. Python's built-in ctypes module can interface with the Windows API to retrieve file attributes, while os.scandir() provides an efficient way to iterate through directory contents.

Each file or directory on Windows has a set of attributes, including the FILE_ATTRIBUTE_HIDDEN flag. By calling the Windows API function GetFileAttributesW, we can determine whether the hidden attribute is set for a given path.

Example

Following is an example using os.scandir() to get non-hidden files ?

import os
import ctypes

FILE_ATTRIBUTE_HIDDEN = 0x2

def is_hidden(filepath):
    attrs = ctypes.windll.kernel32.GetFileAttributesW(str(filepath))
    return attrs != -1 and (attrs & FILE_ATTRIBUTE_HIDDEN)

def list_non_hidden(path):
    with os.scandir(path) as entries:
        for entry in entries:
            if not is_hidden(entry.path):
                print(entry.name)

# Example usage
list_non_hidden(".")

This method accurately filters non-hidden files using Windows file attributes.

config.py
empty_file.txt
file1.txt
file2.txt
helper.py

Using os.walk() with Attribute Filtering

For recursive directory traversal, we can combine os.walk() with the is_hidden() function. This approach walks through all subdirectories and filters hidden items at each level.

Example

Following example shows recursive listing of non-hidden files ?

import os
import ctypes

FILE_ATTRIBUTE_HIDDEN = 0x2

def is_hidden(filepath):
    """Check if a file or folder is hidden using Windows API."""
    attrs = ctypes.windll.kernel32.GetFileAttributesW(str(filepath))
    return attrs != -1 and (attrs & FILE_ATTRIBUTE_HIDDEN)

def walk_non_hidden(path):
    """Recursively walk through directory tree and list non-hidden files."""
    for root, dirs, files in os.walk(path):
        # Filter hidden directories and files
        dirs[:] = [d for d in dirs if not is_hidden(os.path.join(root, d))]
        files = [f for f in files if not is_hidden(os.path.join(root, f))]
        
        # Print non-hidden files
        for name in files:
            print(os.path.join(root, name))

# Example usage
walk_non_hidden(".")

Below is the output of the above program ?

.\config.py
.\empty_file.txt
.\file1.txt
.\file2.txt
.\helper.py

Using pathlib with Windows API

Python's pathlib module provides a modern object-oriented interface for path operations. However, it requires combining with ctypes to properly detect hidden files on Windows.

Example

Following example uses pathlib for listing non-hidden files ?

from pathlib import Path
import ctypes

def is_hidden(path):
    return ctypes.windll.kernel32.GetFileAttributesW(str(path)) & 0x2

def list_non_hidden_pathlib(path):
    for item in Path(path).iterdir():
        if not is_hidden(item):
            print(item.name)

list_non_hidden_pathlib(".")

Here is the output of the above program ?

config.py
empty_file.txt
file1.txt
file2.txt
helper.py

Comparison of Methods

Conclusion

For Windows systems, always use the Windows API through ctypes to accurately detect hidden files. The os.scandir() method is most efficient for single directories, while os.walk() is ideal for recursive operations.