Different Ways to Read File in Bash Script Using While Loop

Reading files is an essential aspect of shell scripting in the Bash environment. In Bash, reading files can be achieved using different techniques, and one of the most popular and versatile methods is using the while loop. The while loop in Bash is a powerful construct that allows us to iterate through a file line by line, making it easier to manipulate and process data.

Understanding the While Loop in Bash

The while loop in Bash follows a specific syntax

while [condition]
do
   # code to execute
done

The loop runs repeatedly until the condition returns false. In file reading operations, we typically use the read command as the condition, which returns false when it reaches the end of file (EOF).

Method 1: Using the read Command with I/O Redirection

The most common and efficient method uses the read command with input redirection

#!/bin/bash

filename="sample.txt"

while IFS= read -r line
do
   echo "$line"
done < "$filename"

Here, IFS= preserves leading/trailing whitespace, and -r prevents backslash interpretation. This method is preferred because it handles special characters correctly and doesn't create a subshell.

Method 2: Using cat Command with Pipe

The cat command can display file contents and pipe them to a while loop

#!/bin/bash

filename="sample.txt"

cat "$filename" | while read -r line
do
   echo "$line"
done

Note: This method creates a subshell, so variable modifications inside the loop won't persist outside it.

Method 3: Using File Descriptor

You can open a file descriptor for more control over file reading

#!/bin/bash

filename="sample.txt"

exec 3< "$filename"
while IFS= read -r line <&3
do
   echo "$line"
done
exec 3<&-

This method opens file descriptor 3, reads from it, then closes it. It's useful when you need to read from multiple files simultaneously.

Method 4: Reading with Field Separation

When processing structured data, you can read multiple fields per line

#!/bin/bash

filename="data.csv"

while IFS=',' read -r field1 field2 field3
do
   echo "Field 1: $field1, Field 2: $field2, Field 3: $field3"
done < "$filename"

This example uses comma as the delimiter to parse CSV files.

Method 5: Using Process Substitution

Process substitution allows reading command output as if it were a file

#!/bin/bash

while IFS= read -r line
do
   echo "Processed: $line"
done < <(grep "pattern" "sample.txt")

This reads only lines matching a specific pattern from the file.

Comparison of Methods

Best Practices

• Always use IFS= read -r to handle special characters and whitespace correctly

• Quote variables: "$line" instead of $line

• Use input redirection (<) instead of piping when possible to avoid subshells

• Check if the file exists before reading: [[ -f "$filename" ]]

Conclusion

Reading files with while loops in Bash offers multiple approaches, each suited for different scenarios. The read command with input redirection is generally the most efficient and reliable method. Understanding these techniques enables you to choose the right approach based on your specific file processing requirements.