Using sed to Replace a Multi-Line String

Sed (Stream Editor) is a powerful command-line tool used for parsing and transforming text in files or streams. While sed operates line-by-line by default, it can be configured to handle multi-line string replacements through special commands and pattern space manipulation.

Understanding Sed's Basic Syntax

The fundamental structure of sed's substitute command is:

s/pattern/replacement/flags

Where pattern is a regular expression matching the text to replace, replacement is the new text, and flags modify the command's behavior (such as g for global replacement).

Multi-Line String Replacement Technique

To replace multi-line strings, sed must load the entire file into its pattern space rather than processing line-by-line. This is achieved using the pattern :a;N;$!ba;.

Basic Multi-Line Replacement Example

Consider a file example.txt containing:

This is a multi-line
string that we want to replace
with a new string.

To replace this with "New content", use:

sed -i ':a;N;$!ba;s/This is a multi-line\nstring that we want to replace\nwith a new string\./New content/g' example.txt

Command breakdown:

• -i Edit file in-place

• :a;N;$!ba; Load entire file into pattern space

• 
Represents newline characters in the pattern

Using Variables for Readability

Long sed commands can be made more manageable using shell variables:

OLD_STRING="Lorem ipsum dolor sit amet,\nconsectetur adipiscing elit,\nsed do eiusmod tempor incididunt"
NEW_STRING="Replacement text"
sed -i ":a;N;$!ba;s/${OLD_STRING}/${NEW_STRING}/g" example.txt

Pattern Matching with Regular Expressions

For dynamic replacements, regular expressions can match multiple variations. To replace all email addresses:

sed -i 's/[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}/newemail@example.com/g' example.txt

This regex matches standard email format and replaces all occurrences globally.

Advanced Techniques

Using Hold Buffer for Data Preservation

The hold buffer allows preserving parts of matched text. To replace phone numbers while keeping names:

sed -i -e '/^.*:/{h;s/[^:]*: //;s/([0-9]{3}) [0-9]{3}-[0-9]{4}/(555) 555-1234/;G;s/<br>/: /}' example.txt

Multiple Replacements in Single Command

Combine multiple substitute commands with semicolons:

sed -i 's/line/sentence/g;s/first/initial/g' example.txt

Removing Blank Lines

Delete empty lines using the delete command:

sed -i '/^$/d' example.txt

Common Use Cases

Key Points

• Escape special characters Use backslashes for literal dots, brackets, etc.

• Test first Remove -i flag to preview changes before modifying files

• Backup files Use -i.bak to create backup before editing

• Regular expressions Powerful for pattern matching but require careful construction

Conclusion

Sed provides robust capabilities for multi-line string replacement through pattern space manipulation and regular expressions. While the syntax can be complex, mastering these techniques enables efficient text processing and file transformation tasks from the command line.