sed command in Linux

Introduction

There are several versions of sed that are available on different platforms, but we will focus on GNU sed version 4.x.

Many of you have heard of sed, or have already used it, most likely as a replacement tool. But this is just one of sed’s destinations, and we will try to show you all aspects of using this utility. Its name stands for “Stream EDitor” and the word “stream” (stream) in this case can mean a file, channel, or just stdin.

We hope that you already have basic knowledge of Linux, and if you have already worked with regular expressions, or at least know what it is, then everything will be much easier for you. The length of the article does not allow us to include a complete guide to regular expressions in it; instead, we will introduce the basic concepts and give a large number of examples of using sed.

Installation

There is no need to tell much. Most likely, all sed you have already installed, as it is used by various system scripts, as well as Linux users who want to increase the efficiency of their work. You can find out which version of sed you have installed using the command:

$ sed --version

On my system, this command shows that I have installed GNU sed 4.2.2 plus gives a link to the program’s home page and other useful information. The package is called “sed” regardless of the distribution, except for Gentoo, where it is present implicitly.

Conceptions

Before moving on, we consider it important to focus on what sed does, as the phrase “stream editor” says little about its purpose. sed accepts text as input, performs the specified operations on each line (unless otherwise specified), and displays the modified text. The indicated operations may be adding, inserting, deleting, or replacing. It’s not as simple as it looks: I warn you that there are a large number of options and their combinations that can make the sed command very difficult to understand. Therefore, we recommend that you learn the basics of regular expressions to understand how this works. Before starting the tutorial, we would like to thank Eric Pement and others for their inspiration and for what he has done for anyone who wants to learn and use sed.

Regular expressions

Since sed commands (scripts) remain a mystery to many, we feel that our readers should understand the basic concepts and not blindly copy and paste commands whose values ​​they do not understand. When a person wants to understand what regular expressions are, the keyword is “match,” or more precisely, “match pattern.” For example, in the report for your department, you wrote the name, Nick, referring to the network architect. But Nick left, and John replaced him, so now you have to replace Nick with John. If the report file is called report.txt, you must run the following command:

$ cat report.txt | sed 's/Nick/John/g' > report_new.txt

By default sed uses stdout, you can use the output redirection operator, as shown in the example above. This is a very simple example, but we illustrated a few points: we look for all the matching pattern “Nick” and replace it in all cases with “John”. Note that sed performs case-sensitive searches, so be careful and check the output file to make sure all replacements have been completed. The above example could be written like this:

$ sed 's/Nick/John/g' report.txt > report_new.txt

Well, you say, but where are the regular expressions? Yes, we wanted to set an example first, and now the most interesting part begins.

Examples of sed commands

SyntaxDescription
sed ‘s/Nick/John/g’ report.txtReplaces each occurrence of Nick with John in the file report.txt
sed ‘s/Nick\|nick/John/g’ report.txtReplaces every occurrence of Nick or nick with John.
sed ‘s/^/ /’ file.txt > file_new.txtAdds 8 spaces to the left of the text to improve print quality.
sed -n ‘/Of course/,/attention you pay/p’ myfileDisplays all paragraphs starting with “Of course” and ending with “attention you pay”.
sed -n 12,18p file.txtDisplays only lines 12-18 of file.txt
sed 12,18d file.txtDisplays the entire file.txt file except lines 12 through 18
sed G file.txtInserts an empty line after each line in file.txt
sed -f script.sed file.txtWrites all commands to script.sed and executes them.
sed ‘5!s/ham/cheese/’ file.txtReplaces ham with cheese in file.txt except for the 5th line
sed ‘$d’ file.txtremove last line
sed ‘/boom/s/aaa/bb/’ file.txtif find “boom”, replace aaa to bb
sed ’17,/disk/d’ file.txtDeletes all lines from the 17th to “disk”. If there are several lines with “disk”, delete before the first of them.
echo ONE TWO | sed “s/one/unos/I”Replaces one with unos, case insensitive, so “unos TWO” will be printed
sed ‘G;G’ file.txtInserts two empty lines after each line in file.txt
sed ‘s/^[ \t]*//’ file.txtRemoves all spaces / tabs before each line in file.txt
sed ‘s/[ \t]*$//’ file.txtRemoves all spaces / tabs at the end of each line in file.txt
sed ‘s/foo/bar/’ file.txtReplaces foo with bar only at the first occurrence in a string.
sed ‘s/foo/bar/4’ file.txtReplaces foo with bar only at the fourth occurrence in a string.
sed ‘s/foo/bar/g’ file.txtReplaces foo with bar on all occurrence in a string.
sed ‘/baz/s/foo/bar/g’ file.txtReplace foo with bar only if the string contains baz.
sed ‘/./,$!d’ file.txtDelete all leading blank lines
sed -e :a -e ‘/^\n*$/{$d;N;};/\n$/ba’ file.txtDelete all trailing blank lines
sed -n ‘/regex/,+5p’ file.txtDisplays 5 lines after a line containing regex
sed ‘1~3d’ file.txtDelete every third row, starting with the first.
sed -n ‘2~5p’ file.txtPrint every fifth line, starting with the second.
sed -n ‘/RE/{p;q;}’ file.txtPrints a string with the first matching RE (regular expression)
sed ‘0,/RE/{//d;}’ file.txtDeletes the first match string
sed ‘0,/RE/s//to_that/’ file.txtChanges only the first match
sed ‘s/^[^,]*,/9999,/’ file.csvReplaces with 9999 all the values in the first column of the CSV file
sed ‘:a;s/\(^\|[^0-9.]\)\([0-9]\+\)\([0-9]\{3\}\)/\1\2,\3/g;ta’ file.txtChanges the number format in file.txt from 1234.56 to 1.234.56
sed -r “s/\<(reg|exp)[a-z]+/\U&/g”Converts any word starting with reg or exp to uppercase.
sed ‘1,20 s/Johnson/White/g’ file.txtReplaces Johnson with White only in lines 1 – 20.
sed ‘1,20 !s/Johnson/White/g’ file.txtThe previous example is the opposite (replaces everywhere except lines 1-20)
sed ‘/from/,/until/ { s/\<red\>/magenta/g; s/<blue\>/cyan/g; }’ file.txtReplaces only between “from” and “until”. If there are several areas “from” – “until”, it replaces in each of them.
sed ‘/ENDNOTES:/,$ { s/Schaff/Herzog/g; s/Kraft/Ebbing/g; }’ file.txtReplaces only with the word “ENDNOTES:” and before EOF
sed ‘/./{H;$!d;};x;/regex/!d’ file.txtPrints a paragraph only if it contains regex
sed -e ‘/./{H;$!d;}’ -e ‘x;/RE1/!d;/RE2/!d;/RE3/!d’ file.txtPrints paragraphs only if they contain RE1, RE2, and RE3. The order of RE1, RE2, and RE3 does not matter.
sed ‘s/14″/fourteen inches/g’ file.txtSo you can use double quotes
sed ‘s/\/some\/UNIX\/path/\/a\/new\/path/g’ file.txtWorking with Unix Paths
sed ‘s/[a-g]//g’ file.txtDeletes all characters, starting with a and ending with g from file.txt
sed ‘s/\(.*\)foo/\1bar/’ file.txtReplaces only the last matching string foo with bar
sed ‘1!G;h;$!d’tac command replacement
sed ‘/\n/!G;s/\(.\)\(.*\n\)/&\2\1/;//D;s/.//’rev command replacement
sed 10q file.txthead command replacement
sed -e :a -e ‘$q;N;11,$D;ba’ file.txttail command replacement
sed ‘$!N; /^\(.*\)\n\1$/!P; D’ file.txtuniq command replacement
sed ‘/regexp/!d’ file.txtequivalent of grep
sed ‘/pattern/d’ file.txtDeletes lines matching pattern
sed ‘/./!d’ file.txtRemoves all empty lines from a file.
sed ‘/^$/N;/\n$/N;//D’ file.txtCompresses all consecutive blank lines to two blank lines. Single empty lines are not changed.
sed -n ‘/^$/{p;h;};/./{x;/./p;}’ file.txtDeletes the last line of each paragraph
sed ‘1,/^$/d’Gets the body of the letter. In other words, deletes everything up to the first empty line.
sed ‘s/^/> /’Quote a message by inserting a “>” before each line.
sed ‘s/^> //’Reverse command (remove quoting from message)
sed -e :a -e ‘s/<[^>]*>//g;/</N;//ba’remove HTML-tags
sed ‘/./{H;$!d;};x;s/\n/={NL}=/g’ file.txt\ | sort | sed ‘1s/={NL}=//;s/={NL}=/\n/g’Sorts paragraphs in file.txt in alphabetical order
sed ‘s@/usr/bin@&/local@g’ path.txtreplace /usr/bin to /usr/bin/local in path.txt
sed ‘s/\([^:]*\).*/\1/’ /etc/passwdAwk replacement – shows only users from passwd file
sed -e ‘/^#/d’ /etc/services | moreShows the services file without comment lines
sed -n ‘/^.\{65\}/p’ file.txtPrint lines of 65 characters or more
sed -n ‘/^.\{65\}/!p’ file.txtPrint lines of 65 characters or less
sed ‘/regex/G’ file.txtInserts an empty line below each line containing regex
sed = file.txt | sed ‘N;s/\n/\t/’Numbers lines in file.txt

Conclusion

This is only part of what sed could be said about, but this series of articles is more of a practical guide that we hope will help you appreciate the power of Unix utilities and make your work more efficient.

You can read more Linux guides by link You can read more Linux guides by link https://linuxnotes.org/category/linux-how-to/

Viva La Linux!

Leave A Comment