Unix Sed Command Utility

09/12/2010

RETAIL & CPG Anto Thomas anto.thomas@tcs.com

HU UH

TCS Public

Unix Sed Command Utility

Contents
Contents .........................................................................................................................2 Sed - An Introduction.....................................................................................................3 Sed Utility ......................................................................................................................4 syntax for basic sed commands: ................................................................................4 Forward slash .............................................................................................................4 s -- Search and replace ...............................................................................................4 Backslash ...................................................................................................................4 g flag ..........................................................................................................................5 Ampersand .................................................................................................................5 Applications of sed command........................................................................................5 Deleting lines in a file using sed command ...............................................................6 Deleting the last line in the input file.........................................................................6 To delete the lines 2 through 4 in the above file, using sed command ......................6 Replace or substitute all occurrences of a pattern using sed command.....................6 Delete all occurrences of a pattern in a file using sed command...............................7 To remove all occurrences of the character space using sed command ....................7 Deleting lines which matches a pattern using sed command.....................................8 Deleting the first line which matches a pattern using sed command.........................8 Deleting the last line which matches a pattern using sed command..........................8

Internal Use

Unix Sed Command Utility

Sed - An Introduction
The most powerful text filtering tools in the UNIX environment are sed and awk. They let shell programmers easily edit text files and filter the output of other commands using regular expressions.

sed (which stands for stream editor) was created as an editor exclusively for executing scripts. As its name implies, sed is stream oriented, thus all the input you feed into it passes through and goes to STDOUT and it does not change the input file.

Internal Use

Unix Sed Command Utility

Sed Utility
sed is short for stream editor.The traditional, modern-day definition of a text editor is an interactive application that can be used to create and edit text files. sed is also a text editor, but it is a command-line utility instead of an interactive utility, which makes it an extremely powerful tool for batch editing. sed is commonly used in UNIX shell scripts to filter large sets of text files.

syntax for basic sed commands: sed s/REGULAREXPRESSION/REPLACEMENTSTRING/flags INPUT_FILE

Forward slash All of the special characters explained earlier for use with grep also work in sed. To use sed, however, you must learn some additional syntax. A basic expression in sed is composed of four parts, separated by forward slashes (/). s -- Search and replace The s indicates you want to execute a search and replace. The forward slashes are used to bind regular expressions in sed. For instance, if you simply want to replace the term logfile with logfile.txt, you would run the following command:

sed s/logfile/logfile.txt/ sed.txt For future reference, the output can be captured or sent into a new file. For example, to send the output to sed_new.txt, run this command:

sed s/logfile/logfile.txt/ sed.txt > sed_new.txt

Backslash The backslash (\) is called the escape character, because it escapes the next character from the regular expression interpretation. More simply, putting a backslash before a special character makes the character a plain item instead of a command item. This is important because many files, especially when you are writing code, make extensive use of the same characters that are used to execute a regular expression. In your sed.txt file, you'll notice

Internal Use

Unix Sed Command Utility

that the dollar sign is used. If you want to replace $project but not project, you need to use the escape character in your search and replace: sed s/\$project/\$project_name/ sed.txt

g flag Execute the same sed command, but this time tack a g on the end.Introducing filters and regular expressions sed s/project/project_name/g sed.txt This time, both instances of project were changed to project_name on the first

Ampersand The ampersand (&) represents the string that was matched by your regular expression. In other words, if [a-z]*: turned out to be project: on a particular line, the ampersand would hold that value. sed s/[a-z]*:/new_\&/g sed.txt

Applications of sed command

Sed command can be used to search and manipulate input text in many ways, though mainly used for substitution, it can also do numerous other pattern matching tasks like searching and printing lines which matches a pattern, removing lines which matches a pattern, deleting specific lines in the input file.

The text file used to illustrate the below sed commands is shown below. bash-3.00# This is line 1 This is line 2 This is line 3 This is line 4 This is line 5 cat textfile.txt

Internal Use

Unix Sed Command Utility

Deleting lines in a file using sed command Deleting the first line in the input file bash-3.00# sed '1d' textfile.txt This is line 2 This is line 3 This is line 4 This is line 5

Deleting the last line in the input file bash-3.00# sed '$d' textfile.txt This is line 1 This is line 2 This is line 3 This is line 4

To delete the lines 2 through 4 in the above file, using sed command bash-3.00# sed '2,4d' textfile.txt This is line 1 This is line 5 the 'd' flag is used to delete lines.

Replace or substitute all occurrences of a pattern using sed command In the above text file, if you need to replace all occurrences of the pattern "line" with the text "line number", the following sed command can be used (the 's' flag in the sed command is used to substitute text which matches a pattern).

bash-3.00# sed 's/line/line number/g' textfile.txt This is line number 1 This is line number 2 This is line number 3 This is line number 4

Internal Use

Unix Sed Command Utility

This is line number 5 Note: The /g flag in the above command is used to instruct sed to substitute globally i.e. all occurrences of the input pattern in the line will be replaced and as a result all words matching the pattern in the input file will be substituted by the other, without /g only the first word matching the pattern in a line will be replaced.

Delete all occurrences of a pattern in a file using sed command This is a simple application of the /s flag where the target string is none. For example, to remove all occurrences of word "line" in the above file, the sed command would be bash-3.00# sed 's/line//' textfile.txt This is This is This is This is This is 1 2 3 4 5

Note that the word to be replaced upon matching the pattern is none in the above command ('/s/line//')

To remove all occurrences of the character space using sed command bash-3.00# sed 's/ //g' textfile.txt Thisisline1 Thisisline2 Thisisline3 Thisisline4 Thisisline5 Also note that if you don't use single quotes in the above sed command, then the space pattern should be enclosed in quotes for the command to work.

bash-3.00# sed s/' '//g textfile.txt Thisisline1 Thisisline2 Thisisline3 Thisisline4 Thisisline5

Internal Use

Unix Sed Command Utility

Deleting lines which matches a pattern using sed command To delete lines which matches a pattern, use the /d flag along with the pattern to be matched, for example, to delete lines matching the pattern "line 1" in the above file, the command would be

bash-3.00# sed '/line 1/d' textfile.txt This is line 2 This is line 3 This is line 4 This is line 5 Now using the above commands, one can also delete the first and last line in a file which matches a pattern.

Deleting the first line which matches a pattern using sed command bash-3.00# sed '/line/{1d;}' textfile.txt This is line 2 This is line 3 This is line 4 This is line 5 In the above command, '1d' is used in braces to indicate that its a command (to delete the specific line, which matches a pattern), this is the way to separate sed flags in command/scripts.

Deleting the last line which matches a pattern using sed command bash-3.00# sed '/line/{$d;}' textfile.txt This is line 1 This is line 2 This is line 3 This is line 4

Internal Use