How to monitor Python files for changes?

Monitoring Python files for changes is essential in many development and automation scenarios such as triggering automatic reloads, running tests, or updating services when the source code is modified. This is particularly useful in web development, machine learning pipelines or custom tooling where real-time responsiveness to file updates is beneficial.

One effective way to achieve this in Python is by using the watchdog library which is a simple and efficient tool that listens for file system events like creation, modification, and deletion.

With watchdog library, developers can set up observers that watch specific directories and respond to the changes through event handlers. For instance, when a .py file is edited or saved, the system can automatically log the change, restart a server, or initiate any custom action that we define. This eliminates the need for manual restarts or checks by improving productivity and reliability.

Installing Watchdog

Before using the Watchdog library we need to install it in our system using the below command −

pip install watchdog

Create a Python Script to Watch Files

Below is an example that monitors a directory for changes to .py files and prints a message when they are modified, created or deleted −

from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler

class FileChangeHandler(FileSystemEventHandler):
    def on_modified(self, event):
        if not event.is_directory:
            print(f"File {event.src_path} has been modified!")
    
    def on_created(self, event):
        if not event.is_directory:
            print(f"File {event.src_path} has been created!")
    
    def on_deleted(self, event):
        if not event.is_directory:
            print(f"File {event.src_path} has been deleted!")

def monitor_file_changes(directory):
    event_handler = FileChangeHandler()
    observer = Observer()
    observer.schedule(event_handler, path=directory, recursive=True)
    observer.start()
    
    try:
        print(f"Monitoring {directory} for changes. Press Ctrl+C to stop.")
        while True:
            pass
    except KeyboardInterrupt:
        observer.stop()
        print("Monitoring stopped.")
    
    observer.join()

# Example usage
monitor_file_changes('/path/to/your/directory')

Monitoring /path/to/your/directory for changes. Press Ctrl+C to stop.
File /path/to/your/directory/example.py has been modified!
File /path/to/your/directory/new_file.py has been created!

Fine-tuned Control with the watchdog.events Module

The watchdog.events module is a core part of the Python watchdog library, offering event handler classes to help us define custom responses to file system events such as file creation, modification, and deletion.

Example

Here is an example that deals with file monitoring by employing the watchdog.events module directly, allowing for granular control over file events −

from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler

class PythonFileHandler(FileSystemEventHandler):
    def on_modified(self, event):
        if not event.is_directory and event.src_path.endswith('.py'):
            print(f"Python file {event.src_path} has been modified!")
    
    def on_created(self, event):
        if not event.is_directory and event.src_path.endswith('.py'):
            print(f"Python file {event.src_path} has been created!")

def monitor_python_files(directory):
    event_handler = PythonFileHandler()
    observer = Observer()
    observer.schedule(event_handler, path=directory, recursive=True)
    observer.start()
    
    try:
        print(f"Monitoring Python files in {directory}. Press Ctrl+C to stop.")
        while True:
            pass
    except KeyboardInterrupt:
        observer.stop()
        print("Monitoring stopped.")
    
    observer.join()

# Example usage
monitor_python_files('/path/to/your/python/project')

Monitoring Python files in /path/to/your/python/project. Press Ctrl+C to stop.
Python file /path/to/your/python/project/main.py has been modified!

Using the Polling Approach

The polling approach means the watchdog regularly checks the status of files by comparing their modification timestamps at intervals. This method doesn't rely on system notifications but proactively checks for changes.

Example

Following is an example which deals with file monitoring using the polling approach, where we engage in periodic checks for file changes −

import time
import os
from pathlib import Path

def monitor_files_polling(directory, file_extension='.py', interval=1):
    directory = Path(directory)
    monitored_files = {}
    
    # Initialize file timestamps
    for file_path in directory.rglob(f'*{file_extension}'):
        monitored_files[file_path] = file_path.stat().st_mtime
    
    print(f"Monitoring {len(monitored_files)} files. Press Ctrl+C to stop.")
    
    try:
        while True:
            for file_path in directory.rglob(f'*{file_extension}'):
                if file_path.exists():
                    current_timestamp = file_path.stat().st_mtime
                    
                    if file_path in monitored_files:
                        if monitored_files[file_path] != current_timestamp:
                            print(f"File {file_path} has been modified!")
                            monitored_files[file_path] = current_timestamp
                    else:
                        print(f"New file {file_path} detected!")
                        monitored_files[file_path] = current_timestamp
            
            time.sleep(interval)
    
    except KeyboardInterrupt:
        print("Monitoring stopped.")

# Example usage
monitor_files_polling('/path/to/your/directory', '.py', 2)

Monitoring 5 files. Press Ctrl+C to stop.
File /path/to/your/directory/script.py has been modified!
New file /path/to/your/directory/new_script.py detected!

Comparison of Approaches

Conclusion

The watchdog library provides the most versatile solution for monitoring Python file changes across platforms. Use polling for simple cases or when watchdog isn't available. Choose the approach that best fits your platform requirements and performance needs.