Guide to the sed Stream Editor on Linux

Introduction

Sed, also known as the "stream editor", is a powerful command-line tool on Linux that allows you to perform basic text transformations on an input stream (either a file or input from a pipe). Sed is especially useful for making bulk changes to large numbers of files, or for editing files that are difficult to open with a text editor. This guide will serve as a detailed introduction to sed, including its syntax, basic commands, and examples of how to use sed in different scenarios. We'll also explore advanced features like grouping and in-place editing to help you master using this versatile tool.

Syntax

The basic syntax of the sed command is as follows −

$ sed 'command' file

The single quotes around the command are important, as they allow sed to interpret the command correctly. The file is the input file that sed will operate on. Sed can also accept input from a pipeline, in which case the file argument is not required.

Commands

Substitution

One of the most common uses of sed is to do replacements. The surrogate command is indicated by the ‘s’ command. The basic syntax is as follows −

$ sed 's/old-text/new-text/' file

For example, to replace all occurrences of the word "old" with "new" in a file named "file.txt", use the following command −

$ sed 's/old/new/' file.txt

Deleting Lines

Another useful command is the d command, which is used to remove rows that match a certain pattern. For example, to remove all lines containing the word "remove" in a file named "file.txt", use the following command −

$ sed '/delete/d' file.txt

Inserting and Aappending

The ‘a’ command is used to insert text after a specific line. For example, to insert the word "new" after the fifth line of a file named "file.txt", use the following command:

$ sed '5a new' file.txt

The ‘i’ command is used to insert text before a specific line. For example, to insert the word "new" before the fifth line of a file named "file.txt", use the following command:

$ sed '5i new' file.txt

Single sed commands

Let's put the knowledge we've gained so far into practice by learning different ways to run sed commands. First, let's see how we can run a single sed command from within the Unix shell:

$ sed [-Ealn] [-i extension] command [file ...]

In addition to editing the contents of the file, we should also be able to view them. Let's see how we can use the print function and execute the corresponding sed command −

$ sed 'p' input.txt
line-1
line-1
line-2
line-2

We can see that when the print function is used explicitly, each of the lines is printed twice. This is because sed always performs a default action of printing each line after all commands have finished executing. Also, to suppress this default behavior, we can use the -n flag in the same command and take things under our control −

$ % sed -n 'p' input.txt
line-1
line-2

Multiple sed commands

Since search is crucial to working with text, wouldn't it be great if we could achieve this power with our own sed commands? Yes, it is very possible!

So, let's go ahead and use sed to get the first line of our input.txt file that contains the word "line" −

$ sed -n -e '/line/ p' -e '/line/ q' input.txt
line-1

We should note the use of the “-e” flag to separate sed commands. The first command finds the pattern and uses the print function to print the row. While the second command takes advantage of the q function, quit which interrupts the execution of sed commands on the input stream. Without the q function, sed will print all lines that match the regular expression.

Alternatively, we can also separate multiple commands with a semicolon −

$ sed -n '/line/ p; /line/ q' input.txt
line-1

Find and replace with sed

sed offers a variety of functions that we can use to find and replace text. Let's explore a few important ones.

Basic Substitution

Let's say our team follows a convention to use spaces for code indentation. However, we often end up using tabs. As a result, our colleagues often ask us to change tabs to spaces during the code review cycle. To solve this use case, we can use the replace function −

[address[,address]]s/regular expression/replacement/flags

Thus, the substitution function searches for the regular expression on each line selected from the range of addresses. Then it replaces only the first matching substring with the replacement string, unless we override this default behavior using a flag. Since we intend to reuse our solution, let's add our sed command in a script called “indentation_fix.sed” −

/^[ ]+/ s/ / /g

Replacement with Pooling

Sometimes we often need to make syntax fixes in our code. Let's take a case of accessing characters in a Java string where we wrongly treated it as an array, as is the case in many other programming languages ​​such as C −

String name = "string";
if (name.length() > 0) {
   System.out.println("Name starts with " + name[0]);
}

Instead, we should use the charAt() method to access the character at position 0. Since we may have this error in multiple places in our file, we can save time by using sed to make this change for us.

$ sed -i 's/(name)[ ]*[[ ]*]/\1.charAt(0)/g' input.txt

This command uses pooling to match the entire pattern "name[0]" and replace it with "name.charAt(0)".

Conclusion

In conclusion, sed is a powerful tool for editing text streams. It has a wide range of features for matching and replacing text, making it suitable for various use cases. With this guide, we learned the basics of using sed and how to run commands in different scenarios, including replace, delete, insert, and add. We also explored advanced features like grouping and in-place editing to help you master using this versatile tool.