SED

Sed

sed is a program used for editing data. It stands for stream editor. Unlike ed, sed cannot be used interactively. However, its commands are similar. The general form of the sed command is

sed command file 

[root@jaguar ~/Desktop] variable=ashi
[root@jaguar ~/Desktop] echo $variable
ashi
[root@jaguar ~/Desktop] echo $variable | sed 's/$variable/ashish/'
ashi
[root@jaguar ~/Desktop] echo $variable | sed "s/$variable/ashish/"

ashish 

[root@server1 ~]# sed 's/Unix/UNIX/' file9

The UNIX operating system was pioneered by Ken

Thompson and Dennis Ritchie at Bell Laboratories

in the late 1960s.  One of the primary goals in

the design of the UNIX system was to create an

environment that promoted efficient program

development.

"or"

$ sed 's/Unix/UNIX/' file9 > temp                           Make the changes 

 $ mv temp intro                                                    And now make them permanent 

[root@server1 ~]# who |sed 's/ .*$//'

root
root
ashiashi
ashi

The sed command says to substitute a blank space followed by any characters up to the end of the line ( .*$) with nothing (//); that is, delete the characters from the first blank to the end of the line from each line of the input.

The -n Option

We pointed out that sed always writes each line of input to standard output, whether or not it gets changed. Sometimes, however, you'll want to use sed just to extract some lines from a file. For such purposes, use the -n option. This option tells sed that you don't want it to print any lines unless explicitly told to do so. This is done with the p command. By specifying a line number or range of line numbers, you can use sed to selectively print lines of text. So, for example, to print just the first two lines from a file, the following could be used:

$ sed -n '1,2p' intro       Just print the first 2 lines

The UNIX operating system was pioneered by Ken

Thompson and Dennis Ritchie at Bell Laboratories

If, instead of line numbers, you precede the p command with a string of characters enclosed in slashes, sed prints just those lines from standard input that contain those characters. The following example shows how sed can be used to display just the lines that contain a particular string:

$ sed -n '/UNIX/p' intro    Just print lines containing UNIX

The UNIX operating system was pioneered by Ken

the design of the UNIX system was to create an

Deleting Lines

To delete entire lines of text, use the d command. By specifying a line number or range of numbers, you can delete specific lines from the input. In the following example, sed is used to delete the first two lines of text from intro:

$ sed '1,2d' intro      Delete lines 1 and 2

in the late 1960s. One of the primary goals in

the design of the UNIX system was to create an

environment that promoted efficient program

development.

Remembering that by default sed writes all lines of the input to standard output, the remaining lines in text—that is, lines 3 through the end—simply get written to standard output.

By preceding the d command with a string of text, you can use sed to delete all lines that contain that text. In the following example, sed is used to delete all lines of text containing the word UNIX:

$ sed '/UNIX/d' intro   Delete all lines containing UNIX

Thompson and Dennis Ritchie at Bell Laboratories

in the late 1960s. One of the primary goals in

environment that promoted efficient program

development.

Sed Commands & Usage

sed '5d'
Delete line 5

sed '/[Tt]est/d'
Delete all lines containing Test or test

sed -n '20,25p' text
Print only lines 20 through 25 from text

sed '1,10s/unix/UNIX/g' intro
Change unix to UNIX wherever it appears in 
the first 10 lines of intro

sed '/jan/s/-1/-5/'
Change the first -1 to -5 on all lines containing jan


sed 's/...//' data
Delete the first three characters from each line of data


sed 's/...$//' data
Delete the last 3 characters from each line of data


sed -n 'l' text
Print all lines from text, showing nonprinting characters as \nn (where nn is the octal value of the character), and tab characters as \t

Ways to Run sed

sed can be run on the command line as follows:

cat sample.txt | sed -e '1,15d'

You can cat the file sample.txt and use the pipe to redirect its output (the lines of text) into the sed command. The -e option to sed tells it to use the next item as the sed command. The d command tells sed to delete lines 1–15 of the input stream, which in this case is the lines read from sample.txt. The rest of the file (if any) appears on standard output, your terminal window, unless redirected elsewhere.
Also, you simply can specify the input file as a command-line argument, so the above sed command also can be written as:

sed -e '1,15d' sample.txt

You also can tell sed to read commands from a script file by using the [-f script-file] option.

Sed Command Format

A sed command has this format:

[pattern1][,pattern2][!] command [args]

The pattern1 and pattern2 are optional line ranges. Some commands don't use the patterns, some commands use only one and some can use both to specify a range of lines that the sed command can operate on, as we did in our simple example above:


pattern1 and pattern2 can be numbers, in which case they are treated like line numbers. They can also be a regular expression delimited by slashes (/pattern/). When using regular expression patterns, all lines that match the expression are filtered through the sed command.
If no pattern is specified, the sed command operates on every line of input.
The ! causes sed to operate on every line not included in the pattern range. You can change our example above to be:

cat sample.txt | sed -e '1,15!d'

This command deletes all lines except lines 1–15.

A Few sed Commands

Here are a few basic sed examples. These can all be run right from the command line. Testing and debugging your sed commands individually on the command line before integrating them into a larger script will save you a lot of time that otherwise would be spent debugging the commands from within a running script.


Let's say that you have a file that lists customers called customer.txt. For the following examples, it contains simple lines of text, like this:

Sam Jones
Brenda Jones
Carl Simon
Liz Smith

Let's use some sed commands to manipulate this file. For example, if you want to remove lines containing Carl Simon and update your customer file, you can do the following:

cat customer.txt | \sed -e '/Carl Simon/d' > customer.txt

The pattern /Carl Simon/ is used by sed as a regular expression and matches every line that has that pattern somewhere on the line. The d command deletes every line that matches the pattern. So, any lines containing Carl Simon are removed from the file.


If you want to perform some type of text substitution on a text file, the s command is probably what you are looking for. It substitutes one text string for another. We tend to use this a lot in our scripts. For example, if Sam Jones calls up and tells you that you should have him listed as Samuel Jones, you can use this command to make the change:

cat customer.txt | \ sed -e 's/Sam Jones/Samuel Jones/' > customer.txt

The s command in sed has three slashes that follow the s. The text between the first and second slash is the pattern you want to match. The text between the second and third slash contains the pattern that you want to substitute for the first pattern. If you wanted all instances of Sam to be Samuel (not just Sam Jones), you could rewrite this example as follows:

cat customer.txt | \ sed -e 's/Sam/Samuel/' > customer.txt

The commands for append (a), replace (c) and insert (i) typically need to have the sed commands specified in a separate script file. For example, say you want to append the line After Brenda right after the line that contains the text Brenda. You can use the a sed command to append the text there. However, you need to put the sed commands in a separate script file, so fire up your favorite editor and create the following sed command file:

#
# sed command file (# are comment lines)
#
# append the line 'After Brenda'
# in this customer file
#
/Brenda/a\
After Brenda

Save this script file as sed1.cmd. Then, to run sed using this script file, use this syntax:

sed -f sed1.cmd customer.txt

You should see the contents of your customer file with the additional line added after the line Brenda Jones. The pattern /Brenda/ (in the sed command file) determine where in the output our appended line appears.
The difference between the append command and the insert command is where the text is added. For the append command, the text is added after the line containing the match. For the insert command, the text is added before the line that contains the match.

Filter out empty lines from a file:

sed -e '/^$/d' your_file.txt

Add the computer named mycomputer to the end of every line in /etc/exports:

cat /etc/exports |  \
sed -e 's/$/ mycomputer/' > /etc/exports

Add the computer named comp2 only to the directories beginning with /data/ in /etc/exports:

cat /etc/exports | \
sed -e '/^\/data\//s/$/ comp2/' > /etc/exports

See how the forward slashes used in the directory name have to be escaped using back slashes? Without the back slashes, sed interprets the forward slashes in the directory specifier as the delimiters in the sed command itself. However, the back slashes can make the sed command difficult to read and follow.
Remove the first word on each line (including any leading spaces and the trailing space):

cat test3.txt | sed -e 's/^ *[^ ]* //'

More regular expression matching is used in this example. Here's what it is doing.
The initial ^ * is used to match any number of spaces at the beginning of the line. The [^ ]* then matches any number of characters that are not spaces (the ^ inside the brace reverses the match on the space), so it matches a single word. The trailing space at the end matches the space found at the end of the first word. The empty replace pattern removes the text.
Remove the last word on each line:

cat test3.txt | sed -e 's/^\(.*\) .*/\1/'

This command introduces the concept of hold buffers. Hold buffers are used to keep parts of the matched text and to insert that text into the result. The pattern that matches the text between the parentheses is recalled in the substitution pattern by the \1. If an additional set of parentheses were in the match pattern, they would be addressed in the substitution pattern as \2, and so on, for more sets of parentheses. Up to nine hold buffers can be specified. In this example, the pattern contained within the parentheses matches from the start of the line up to the last space (the space after the parentheses).
To remove leading { and trailing }, or a } from each line:

sed -e 's/^.*{\(.*\)},*/\1/' table.txt

I'll leave it to the reader to dig in to this regular expression to see how it operates. Keep this in mind—the more comfortable you are with regular expressions and hold buffers, the more powerful the sed command becomes.

Conclusion

sed recognizes many other commands. However, even with these basic commands, you can successfully manipulate text files from within your own shell scripts or right from the command line.

sed G myfile.txt > newfile.txt

In the above example using the sed command with G would double space the file myfile.txt and output the results to the newfile.txt.

sed = myfile.txt | sed 'N;s/\n/\. /'

The above example will use the sed command to output each of the lines in myfile.txt with the line number followed by a period and a space before each line. As done with the first example the output could be redirected to another file using > and the file name.

sed 's/test/example/g' myfile.txt > newfile.txt

Opens the file myfile.txt and searches for the word "test" and replaces every occurrence with the word "example".

sed -n '$=' myfile.txt

Above this command count the number of lines in the myfile.txt and output the results.