Read Random Line From a File in Linux

In Linux, reading a random line from a file can be a useful task in various scenarios. For example, when you want to select a random word from a dictionary or randomly select a line from a log file for analysis purposes. There are several ways to read a random line from a file in Linux. In this article, we will explore different methods to achieve this task along with their pros and cons.

Method 1 Using shuf Command

The shuf command is a simple and efficient way to read a random line from a file in Linux. shuf command is included in most Linux distributions and is part of GNU coreutils package. The basic syntax for using shuf command is as follows

shuf -n 1 filename

In this command, -n 1 specifies that we want to select one random line from the file, and filename is the name of the file from which we want to select a random line. Here is an example of how to use shuf command to read a random line from a file named sample.txt

$ shuf -n 1 sample.txt

Hello World

This command will output one random line from the sample.txt file. The shuf command also has additional options like -r to allow lines to be repeated, or -e to specify a list of items to choose from.

Pros

• Simple and efficient syntax

• Included in most Linux distributions by default

• Fast execution even with large files

Cons

• May not be available on all Unix-like systems

• Part of GNU coreutils, which may not be installed on minimal systems

Method 2 Using sort Command

The sort command with the -R option can randomize file lines. The basic syntax for using sort command is as follows

sort -R filename | head -n 1

In this command, -R specifies that we want to randomize lines in the file, and filename is the name of the file from which we want to select a random line. The output of sort command is then piped to head command, which selects the first line of output (which is a random line from the file).

$ sort -R sample.txt | head -n 1

Random line from file

Pros

• Available on most Unix-like systems

• Can be used to randomize entire file contents

Cons

• Slower than other methods for large files

• Reads entire file into memory before randomizing

Method 3 Using awk Command

The awk command can generate random numbers for each line and then select one. The basic syntax is as follows

awk 'BEGIN {srand();} {print rand() " " $0;}' filename | sort -n | cut -d ' ' -f2- | head -n 1

In this command, awk generates a random number for each line using the rand() function. The output is piped through sort, cut, and head to extract one random line.

$ awk 'BEGIN {srand();} {print rand() " " $0;}' sample.txt | sort -n | cut -d ' ' -f2- | head -n 1

Another random line

Pros

• Highly flexible and customizable

• Available on virtually all Unix-like systems

Cons

• Complex syntax requiring multiple commands

• Slower execution due to multiple pipe operations

Method 4 Using sed Command

The sed command can select a specific line number generated randomly. The basic syntax is as follows

sed -n $((RANDOM%$(wc -l < filename)+1))p filename

In this command, the expression $((RANDOM%$(wc -l < filename)+1)) generates a random number between 1 and the number of lines in the file. The sed -n option suppresses default output, and the p command prints the selected line.

$ sed -n $((RANDOM%$(wc -l < sample.txt)+1))p sample.txt

Yet another random line

Pros

• Memory efficient for large files

• Fast execution once line count is determined

Cons

• Requires counting lines first, which can be slow for large files

• Complex syntax that may be difficult to remember

Method 5 Using Python Script

Python provides a flexible approach to read a random line from a file. Here is an example Python script

#!/usr/bin/env python3

import random

filename = "sample.txt"
with open(filename, "r") as f:
   lines = f.readlines()
print(random.choice(lines).strip())

In this script, the open() function opens the file for reading, readlines() reads all lines into a list, random.choice() selects a random line, and strip() removes trailing whitespace.

Pros

• Highly customizable and readable code

• Can be extended for complex text processing tasks

Cons

• Requires Python interpreter to be installed

• Loads entire file into memory

Comparison

Conclusion

There are multiple methods available in Linux to read a random line from a file, each with distinct advantages. The shuf command is the most efficient for simple tasks, while sed provides good performance with universal availability. For complex text processing requirements, Python offers the greatest flexibility and readability.