Unix Sed Command to Delete Lines in File - Examples

Sed Command to Delete Lines: Sed command can be used to delete or remove specific lines which matches a given pattern or in a particular position in a file. Here we will see how to delete lines using sed command with various examples.



The following file contains a sample data which is used as input file in all the examples:
> cat file
linux
unix
fedora
debian
ubuntu

Sed Command to Delete Lines - Based on Position in File

In the following examples, the sed command removes the lines in file that are in a particular position in a file.

1. Delete first line or header line

The d option in sed command is used to delete a line. The syntax for deleting a line is:
> sed 'Nd' file

Here N indicates Nth line in a file. In the following example, the sed command removes the first line in a file.
> sed '1d' file
unix
fedora
debian
ubuntu

2. Delete last line or footer line or trailer line

The following sed command is used to remove the footer line in a file. The $ indicates the last line of a file.
> sed '$d' file
linux
unix
fedora
debian

3. Delete particular line

This is similar to the first example. The below sed command removes the second line in a file.
> sed '2d' file
linux
fedora
debian
ubuntu

4. Delete range of lines

The sed command can be used to delete a range of lines. The syntax is shown below:
> sed 'm,nd' file

Here m and n are min and max line numbers. The sed command removes the lines from m to n in the file. The following sed command deletes the lines ranging from 2 to 4:
> sed '2,4d' file
linux
ubuntu

5. Delete lines other than the first line or header line

Use the negation (!) operator with d option in sed command. The following sed command removes all the lines except the header line.
> sed '1!d' file
linux

6. Delete lines other than last line or footer line
> sed '$!d' file
ubuntu

7. Delete lines other than the specified range
> sed '2,4!d' file
unix
fedora
debian

Here the sed command removes lines other than 2nd, 3rd and 4th.

8. Delete first and last line

You can specify the list of lines you want to remove in sed command with semicolon as a delimiter.
> sed '1d;$d' file
unix
fedora
debian

9. Delete empty lines or blank lines
> sed '/^$/d' file

The ^$ indicates sed command to delete empty lines. However, this sed do not remove the lines that contain spaces.

Sed Command to Delete Lines - Based on Pattern Match

In the following examples, the sed command deletes the lines in file which match the given pattern.

10. Delete lines that begin with specified character
> sed '/^u/d' file
linux
fedora
debian

^ is to specify the starting of the line. Above sed command removes all the lines that start with character 'u'.

11. Delete lines that end with specified character
> sed '/x$/d' file
fedora
debian
ubuntu

$ is to indicate the end of the line. The above command deletes all the lines that end with character 'x'.

12. Delete lines which are in upper case or capital letters
> sed '/^[A-Z]*$/d' file

13. Delete lines that contain a pattern
> sed '/debian/d' file
linux
unix
fedora
ubuntu

14. Delete lines starting from a pattern till the last line
> sed '/fedora/,$d' file
linux
unix

Here the sed command removes the line that matches the pattern fedora and also deletes all the lines to the end of the file which appear next to this matching line.

15. Delete last line only if it contains the pattern
> sed '${/ubuntu/d;}' file
linux
unix
fedora
debian

Here $ indicates the last line. If you want to delete Nth line only if it contains a pattern, then in place of $ place the line number.

Note: In all the above examples, the sed command prints the contents of the file on the unix or linux terminal by removing the lines. However the sed command does not remove the lines from the source file. To Remove the lines from the source file itself, use the -i option with sed command.
> sed -i '1d' file

If you dont wish to delete the lines from the original source file you can redirect the output of the sed command to another file.
sed '1d' file > newfile

To print a specific line from a file

sed -n 5p <file>
You can get one specific line during any procedure. Very interesting to be used when you know what line you want.

Sample Output
> sed -n 5p test line 5 

> cat test line 1 line 2 line 3 line 4 line 5 line 6 line 7 line 8 line 9 line 10

Print all lines between two line numbers

This command uses awk(1) to print all lines between two known line numbers in a file. Useful for seeing output in a log file, where the line numbers are known. The above command will print all lines between, and including, lines 3 and 6.

awk 'NR >= 3 && NR <= 6' /path/to/file


Print all lines between two line numbers
Print all lines between two line numbers This command uses sed(1) to print all lines between two known line numbers in a file. Useful for seeing output in a log file, where the line numbers are known. The above command will print all lines between, and including, lines 3 and 6. Show Sample

Output
sed -n '3,6p' /path/to/file

Sed: simple pattern address usage

Most people I know use sed for simple and fast translation of some keyword in files. For instance, changing ports and tags inside configuration files during deployment to production servers. This results in sometimes clumsy scripts to make sure that sed changes a keword on line 4, but not on line 40. Most people I know have no idea that the way you can actually limit the range in which sed has to operate. Let’s explore…

Sed’s general form is something like this:

[address[,address]][!]command [args]

As you can see, it LOOKS  simple enough, but it really isn’t. Let’s take a look at a simple sed-action:

echo "Hello world" | sed -e 's/world/people/'

This will output “Hello people” since it changes the text “world” into “people”. If instead of 1 line, we streamed a file into sed, it will replace the first “world” string for every line in the file. If you have 2 “world” strings on one line, you need to add the “g” option (g meaning global here).

cat largefile.txt | sed -e 's/world/people/g'

Meet sed’s addressing

This is all fine and dandy, until you don’t want to replace all world-tags in a file, but just the tags in the first, say, 10 lines. It’s very easy to accomplish this in sed:

cat largefile.txt | sed -e '1,10 s/world/people/g'

This looks the same, only we have added an address-range to sed. In this case, it will change world into people on line 1 to 10, but sometimes we want to do the opposite: change on all lines EXCEPT 1 to 10. Still easy enough:

cat largefile.txt | sed -e '1,10 ! s/world/people/g'

Now for the fun part

It is not only possible to add line numbers for addressing, but you can also add a regex if you like (or any mix).

cat largefile.txt | sed -e '/^start/,/^end/ s/world/people/g'

this will change world to people for every line after finding a line that starts with “START”. It will stop after finding a line that starts with “END”. So let’s see, which lines will be changed in the following files:

hello world
    
this is a world readable test file
let's start shall we?

start
the world is not enough. But the world should be.
end

start with a better world
ending with a world of hate is not an option

Did you catch them all? Here’s the answer:

hello world

this is a world readable test file
let's start shall we?

start
the people is not enough. But the people should be.
end

start with a better people
ending with a people of hate is not an option

An advanced example

Let’s see how some more advanced usages work. These examples use application.ini, which is a zend framework configuration file, but basically any INI file will do. Let’s “remove” the [testing] section by commenting out the data. Note that we don’t know (nor care) where (or if) this [testing] section is present or how long it is.

cat application.ini |
sed -e '/^\[testing]/,/^\[/ { /^\[testing]/b ; /^\[/b; s/\(.*\)/**DELETED** \1/ }'</pre>

So, what does this actually do then? Let’s take a look at the address range:

The address-range:

/^\[testing]/,/^\[/ 

this sed script matches (every) line that starts with [testing] and ends with a line that starts with [.  When dealing with ini-files, this basically means you match a complete [testing] section, INCLUDING the [testing] and next [..] lines. It looks a bit messy but the same /RE1,RE2/ syntax can be found.

The { } brackets makes it possible to group multiple sed commands just like php. In this case, we have 3 commands, all separated by a ;(sounds very familiar does it?)

/^\[testing]/b

This command tells sed to “branch” to the end of the script (and it will continue with a new line). In effect, it’s a check that no other commands will be run when we match the initial [testing block. It would be easier if sed had a way to exclude the address-ranges but alas. So we are forced to check the start and end matches ourselves.

/^\[/b

As said, we checked the start, so we have to check the end as well, so these lines don’t get changed by sed.

s/\(.*\)/; **DELETED** \1/

When we arrived here, we can safely change our lines. In this case, it will do a substitution (the ‘s’) where it will match .*, which is regex-speak for the whole line. The () are needed to capture this whole line into the \1 parameter we use inside the substition.

We substitute the line by changing the line to  “; **DELETED** “, and adding the \1 to it. This effectively appends the “; ** deleted **” string in front of the line.

That’s it.. The whole INI-section called [testing] is commented out in one go and you know a bit more about sed’s address usage.

Sed command in Linux

Substitution command stream editor for filtering and transforming text

sed 's/as/sd/' <file-name>

sd replace as in file-name output only in first occurrence of line

cat file-name | sed ‘s/as/sd/’

same as above use cat and pipe command in between

Here in this command, we used

s        --> used as substitute command
/        --> used as Delimiter
as        --> search pattern string
sd        --> replacement string

Sometime when we try sed on Unix path, we need to use backward slash(Esc character). Like

sed 's//etc/resolv.conf//etc//dns-settings/’ 

but this create confusion for some users. we can avoid this confusion and replace delimiter with our own choice

sed 's_/etc/resolv.conf_/etc/dns-settings_'

We can also use these settings with global occurrence instead of only first in one line

sed 's/as/sd/g' <file-name>

We can also use these settings for only second occurrence instead of only first in one line

sed 's/as/sd/2' <file-name>

sed 's/as/sd/2g' <file-name>

Second and after second occurrence

sed '/root/d' sudoers

Delete the lines that contain root

sed '3d' sudoers

Delete 3rd line

sed -n '/root/p' sudoers

Print only lines containing root

sed  ‘/^$/d’ suders

Delete blank lines from file

Sed command in Linux

sed is a stream editor that accepts a list of commands and input file names. It applies the commands one by one on each line of input and writes the resulting lines on the standard output. The commands are separated by newlines.

2.0 sed Command syntax

sed [OPTIONS] 'list-of-sed-commands' filenames ...

sed is a filter. It prints each line of input on its output. More often than not, there are one or two commands that transform the input file in some way. Since it reads standard input and writes on standard output, sed is often used in command pipelines. sed commands are basically ed text editor commands.

By default, the sed command is executed for each line of input. However, it is possible to specify an address range before a command and the command is executed for the corresponding line numbers. If only one number is specified, the command is executed for that line only. For line numbers, $ matches the last line of file.

3.0 Examples

3.1 Substitution

The most common use of sed is substituting strings. It is of the form,

[m,n]s/old-string/new-string/x
if x is not given, only the first occurrence of old-string in a line is changed to new-string
if x = g, all occurrences of old-string in a line are changed to new-string
if x = p, the modified line is printed (in addition to the default printing of the line)
if x = w file, the modified line is written to file

For example, if printf is misspelled as print in a C program, we can correct it with the command,

$ sed 's/print *(/printf (/g' hello.c

The above code reads as, for each line of input, match print *(, and change all its occurrences to printf (. The * after the space takes care of zero or more spaces between print and the left parenthesis. This, of course, only takes care of the case where print and the left parenthesis are on the same line.

3.2 Append, Insert and Change lines

a \ Append text between backslash and the end of line as a line. 
i \ Insert text between backslash and the end of line as a line. 
c \ Change selected lines with text between backslash and the end of line as a line. 
For example,

$ # add Copyright notice at the top of file $ sed '1i \# Copyright (c) Tom, Dick and Harry > ' hello.c $ # add newline after each line (double space) $ sed 'a \ > ' hello.c $ # if a line contains main, change it $ sed '/main/c\int main (int argc, char **argv) > ' hello.c

3.3 Delete lines

The d command deletes a line. By default, the current line is deleted. To delete more lines, a range is selected. For example,

$ # if a line contains main, delete it $ sed '/main/d' hello.c $ # delete line numbered 1 through 4 $ sed '1,4d' hello.c

3.4 Quit

The q command quits the sed program. For example, the following scripts prints 10 lines of the file, hello.c, and quits.

$ sed '10q' hello.c

Which is just like the head command.

3.5 Read File

The r file command reads a file and appends its contents to the line. For example,

$ # read justaline and append its contents after each line of hello.c $ sed 'r justaline' hello.c $ # read justaline and append its contents after line number 10 of hello.c $ sed '10r justaline' hello.c

3.6 Write line to file

The w file command writes the current line to file. For example,

$ sed 'w hello.bak' hello.c # makes a copy of hello.c in hello.bak, writing each line as sed goes through hello.c $ # write first 10 lines to new $ sed '1,10w new' hello.c

3.7 sed -n

The default printing may not be required in some cases and the -n option accomplishes that. When the -n option is used, printing of lines can be done with the p command. For example, in the following command, sed prints a line if it contains the string int.

$ sed -n '/int/p' hello.c

Which is almost the grep command.

3.8 !cmd

The syntax !cmd means, execute the command, only if a line is not selected. So, in the example of para 3.7, if we wanted a list of lines not matching a pattern, we can execute,

$ # print lines in hello.c not containing the string int $ sed -n '/int/!p' hello.c

Which the like the grep -v command. Also, we can do the same thing by letting sed print all lines except those containing the pattern.

$ # print lines in hello.c not containing the string int $ sed '/int/d' hello.c

3.9 Replace characters

The command is of the form, y/string1/string2/. It replaces each character in input which is in string1 to the corresponding character in string2. The two strings must have the same length. For example,

$ # replace lowercase characters by uppercase $ sed 'y/abcdefghijklmnopqrstuvwxyz/ABCDEFGHIJKLMNOPQRSTUVWXYZ/' hello.c $ # replace tab by space $ sed 'y/\t/ /' hello.c

3.10 Read commands from file

Instead of passing commands on the command line, we can put commands in a file and invoke sed with the option, -ffile-name. For example, we can make two files, upper-to-lower and lower-to-upper with contents,

$ cat upper-to-lower # convert uppercase to lowercase y/ABCDEFGHIJKLMNOPQRSTUVWXYZ/abcdefghijklmnopqrstuvwxyz/ $ cat lower-to-upper # convert lowercase to uppercase y/abcdefghijklmnopqrstuvwxyz/ABCDEFGHIJKLMNOPQRSTUVWXYZ/ $ # convert file, lowercase to uppercase $ sed -f lower-to-upper hello.c $ # convert lowercase to uppercase and back to lowercase $ sed -f lower-to-upper hello.c | sed -f upper-to-lower $ sed -f lower-to-upper -f upper-to-lower hello.c # same effect as the previous command

We can pass multiple scripts with -f option and the subsequent script is appended to the previous one. So lower-to-upper converts lowercase to uppercase and upper-to-lower converts back the uppercase to lowercase.

3.11 Multiple scripts on command line

We can pass multiple scripts with the -e option. For example,

$ # find /etc/passwd entries for users alice, bob and carol $ sed -n -e '/alice/p' -e '/bob/p' -e '/carol/p' /etc/passwd alice:x:1000:1000:Alice B,,,:/home/alice:/bin/bash bob:x:1001:1001:Bob C,,,:/home/bob:/bin/bash carol:x:1002:1002:Carol D,,,:/home/carol:/bin/bash

3.12 Extended Regular Expressions

sed -r gives extended regular expressions, just like the grep E command. Suppose we have a file named expenditurecontaining a list of expenditure for three years,

Expenditure 2012 2013 2014 Advertising 200015 233912 189928 Bank charges 23029 26667 34990 Boarding and Lodging 237899 453326 356625 Communication 34556 28928 34222 Conveyance 23444 27889 43882 Insurance 43344 41992 45667 Repairs and maintenance 26609 19887 29008 Servers 34556 35662 32118 Stationery 38828 28779 39887 Sub-contracting 29988 30992 21334 Travel 891112 788277 655489 END-OF-FILE

This has a table, where each row gives an expenditure for three years. Suppose we want to convert it into a Comma Separated Values (CSV) file, where we replace spaces by a comma. The last line in the file has no spaces and is not required in the new CSV file. We can run the sed command,

$ sed -nr 's/[[:blank:]]+/,/gp' expenditure Expenditure,2012,2013,2014 Advertising,200015,233912,189928 Bank,charges,23029,26667,34990 Boarding,and,Lodging,237899,453326,356625 Communication,34556,28928,34222 Conveyance,23444,27889,43882 Insurance,43344,41992,45667 Repairs,and,maintenance,26609,19887,29008 Servers,34556,35662,32118 Stationery,38828,28779,39887 Sub-contracting,29988,30992,21334 Travel,891112,788277,655489

This is mostly OK but has a small problem. Some of the expenditure descriptions in the first column have spaces and these are also substituted by commas. So, we ask sed to substitute only two or more spaces by a comma.

$ sed -nr 's/[[:blank:]][[:blank:]]+/,/gp' expenditure Expenditure,2012,2013,2014 Advertising,200015,233912,189928 Bank charges,23029,26667,34990 Boarding and Lodging,237899,453326,356625 Communication,34556,28928,34222 Conveyance,23444,27889,43882 Insurance,43344,41992,45667 Repairs and maintenance,26609,19887,29008 Servers,34556,35662,32118 Stationery,38828,28779,39887 Sub-contracting,29988,30992,21334 Travel,891112,788277,655489

Linux: sed Command ( Stream Editor )

sed command is a Stream Editor which is used for modifying the files in Linux. Sed helps to make changes to the file.

Example
This file can be taken as an example file for using this command.

[root@linuxhelp  user1]# cat example.txt 
unix is great os.  unix is opensource.  unix is free os.
learn operating system.
unixlinux which one you choose.

 

To replace the word 'Unix' with 'Linux'

[root@linuxhelp  user1]# sed 's/unix/linux/' example.txt 
linux is great os.  unix is opensource.  unix is free os.
learn operating system.
linuxlinux which one you choose.

Options
s – substitution operation
/ - delimiters
Default: It will replace the first occurrence in the line only.

Replacing the specific occurrence in a line

To replace the second occurrence of 'Unix' with 'Linux' in a line.

[root@linuxhelp user1]# sed 's/unix/linux/2' example.txt 
unix is great os.  linux is opensource.  unix is free os.
learn operating system.
unixlinux which one you choose.

Options
/2 – to notify the second occurrence

Replacing the entire occurrence in a line

To replace 'Unix' with 'Linux' for all occurrence in a line.

[root@ linuxhelp ~]# sed 's/unix/linux/g' example.txt 
linux is great os.  linux is opensource.  linux is free os.
learn operating system.
linuxlinux which one you choose.

Options
/g – global replacement

Replacing from the specific occurrence to all

To replace all 'Unix' with ' Linux' from the given occurrence to the entire occurrence following back.

[root@ linuxhelp ~]# sed 's/unix/linux/3g' example.txt 
unix is great os.  unix is opensource.  linux is free os.
learn operating system.
unixlinux which one you choose.

Options
/3g – to notify the change to be made from third occurrence.

Using '&' flag to mention string

There may be need to replace the pattern by adding some characters to it. To find the specific word that need to make change, this '&' is used.

[root@ linuxhelp ~]# sed 's/unix/{&}/' example.txt 
{unix} is great os.  unix is opensource.  unix is free os.
learn operating system.
{unix}linux which one you choose.

Duplicates the replaced line

If the change is made then the line will be displayed twice in terminal. If not, it remains the same.

[root@ linuxhelp ~]# sed 's/unix/linux/p' example.txt 
unix is great os.  unix is opensource.  unix is free os.
unix is great os.  unix is opensource.  unix is free os.
learn operating system.
linuxlinux which one you choose.
linuxlinux which one you choose.

Options
/p – print flag to display twice.

Printing only replaced line

If the pattern is altered then the specific line will be displayed alone.

[root@ linuxhelp ~]# sed -n 's/unix/linux/p' example.txt 
linux is great os.  unix is opensource.  unix is free os.
linuxlinux which one you choose.

Options
-n with /p – to display twice the altered line alone.

Nested sed

Running multiple sed conditions in a single command, i.e., taking the output of first instance as the input for second instance and so on.

[root@ linuxhelp ~]# sed 's/unix/linux/' example.txt | sed 's/os/system/' 
linux is great system.  unix is opensource.  unix is free os.
learn operating system.
linuxlinux which one you chosysteme.

Another method

[root@ linuxhelp ~]# sed -e 's/unix/linux/' -e 's/is/are/' example.txt 
linux is great system.  unix is opensource.  unix is free os.
learn operating system.
linuxlinux which one you chosysteme.

Options
e – To run multiple sed commands.

Replacing a word in specified line number

The command can be made to execute only in the specific line number.

[root@ linuxhelp ~]# sed  '3 s/unix/linux/' example.txt 
unix is great os.  unix is opensource.  unix is free os.
learn operating system.
linuxlinux which one you choose.

Options
3 – it mentions the line number.

Replacing words in the range of lines

You can specify a range of line numbers to execute the command.

[root@ linuxhelp ~]# sed  '1,3 s/unix/linux/' example.txt 
linux is great os.  unix is opensource.  unix is free os.
learn operating system.
linuxlinux which one you choose.

Options
1, 3 – it mentions the range of line numbers.

Another method

[root@ linuxhelp ~]# sed  '2,$ s/unix/linux/' example.txt 
linux is great os.  unix is opensource.  unix is free os.
learn operating system.
linuxlinux which one you choose.
Here, ‘$’ indicates the last line, so the changes is made from 2nd line to last line.

Replace if pattern matches

Specify a pattern in sed command. If pattern match occurs then sed command looks for the string to be replaced.

[root@ linuxhelp ~]# sed  '/linux/ s/unix/centos/' example.txt 
unix is great os.  unix is opensource.  unix is free os.
learn operating system.
centoslinux which one you choose.

Here, it looks for the pattern Linux and then replace the word Unix with centos.

 

Deleting lines

You can delete the lines of a file by specifying line number or a range.

[root@ linuxhelp ~]# sed '2 d' example.txt 
unix is great os.  unix is opensource.  unix is free os.
unixlinux which one you choose.

To delete the range of lines

[root@ linuxhelp ~]# sed '2,$ d' example.txt 
unix is great os.  unix is opensource.  unix is free os.

Options:
d - Delete
$ - to indicate last line

Adding a new line after a match

sed command can add a new line after a pattern match is found.

[root@ linuxhelp ~]# sed '/unix/ a "new line"' example.txt 
unix is great os.  unix is opensource.  unix is free os.
“new line”
learn operating system.
unixlinux which one you choose.
    “new line”

Options
a – to add a new line after a match is found.

 

Adding a new line before a match

It can add a new line before a pattern is found.

[root@ linuxhelp ~]# sed '/unix/ i "new line"' example.txt 
“new line”
unix is great os.  unix is opensource.  unix is free os.
learn operating system.
“new line”
Unixlinux which one you choose.

Options
i – To add a new line before a match is found.

Replacing entire line

sed command can be used to replace the entire line with a new line.

[root@ linuxhelp ~]# sed '/linux/ c "change of line"' example.txt 
unix is great os.  unix is opensource.  unix is free os.
learn operating system.
“change of line”

Options
c – To notify change of line.

Case-sensitive option

sed command can be used to convert lower case to upper case letters.

[root@ linuxhelp ~]# sed 'y/ul/UL/' example.txt 
Unix is great os.  Unix is opensoUrce.  Unix is free os.
Learn operating system.
UnixLinux which one yoU choose.

Options
y – to transform the case of letters.
rh – letters to change from lower case to upper case in a file.
RH – to notify the upper-case option.