Iterate over lines from multiple input streams in Python

Python's built-in open() function handles single file operations, but when you need to process multiple files sequentially, the fileinput module provides a powerful solution. This module allows you to iterate over lines from multiple input streams as if they were a single continuous stream.

Basic Usage with fileinput.input()

The primary interface is the fileinput.input() function, which returns a FileInput object that can iterate over multiple files ?

import fileinput

# Reading from a single file
for line in fileinput.input('data.txt'):
    print(line, end='')

Processing Multiple Files

You can pass multiple filenames as a tuple to process them sequentially ?

import fileinput

# Reading from multiple files
for line in fileinput.input(('file1.txt', 'file2.txt')):
    print(line, end='')

Using as Context Manager

The FileInput class can be used as a context manager for automatic resource cleanup ?

import fileinput

with fileinput.input(('file1.txt', 'file2.txt')) as files:
    for line in files:
        print(line, end='')

Useful Helper Functions

The fileinput module provides several helper functions to track file processing ?

Example with Line Numbers

Here's how to display each line with its line number within each file ?

import fileinput

# Create sample files for demonstration
with open('sample1.txt', 'w') as f:
    f.write("First file line 1\nFirst file line 2\n")

with open('sample2.txt', 'w') as f:
    f.write("Second file line 1\nSecond file line 2\n")

# Process files with line numbers
for line in fileinput.input(('sample1.txt', 'sample2.txt')):
    print('{}->{}'.format(fileinput.filelineno(), line), end='')

1->First file line 1
2->First file line 2
1->Second file line 1
2->Second file line 2

Processing Files with Glob Patterns

You can use glob patterns to process multiple files matching a pattern ?

import fileinput
import glob

# Create sample Python files
with open('script1.py', 'w') as f:
    f.write("print('Hello from script1')\nx = 10\n")

with open('script2.py', 'w') as f:
    f.write("print('Hello from script2')\ny = 20\n")

# Process all Python files
for line in fileinput.input(glob.glob("script*.py")):
    if fileinput.isfirstline():
        print(f"{fileinput.filename()} >")
    print(f"{fileinput.filelineno()}.{line}", end='')

script1.py >
1.print('Hello from script1')
2.x = 10
script2.py >
1.print('Hello from script2')
2.y = 20

In-Place File Modification

Setting inplace=True allows you to modify files directly while reading them ?

import fileinput
import sys

# Create a sample file
with open('message.txt', 'w') as f:
    f.write("Hello Python. Good morning\n")

# Modify the file in-place
for line in fileinput.input('message.txt', inplace=True):
    line = line.replace('morning', 'evening')
    sys.stdout.write(line)

# Read the modified file to verify changes
with open('message.txt', 'r') as f:
    print("Modified content:", f.read())

Modified content: Hello Python. Good evening

Conclusion

The fileinput module provides an elegant way to process multiple files as a single stream. Use it for batch file processing, log analysis, or any scenario where you need to iterate over multiple input sources sequentially.