Skip to content

Sed & Awk: Some Examples of These Wonders

Introduction

Sed & Awk, but what are they? These are two binaries for content filtering. For example, if you have a file and you want to output only specific lines, or even replace things in a file. Of course, what I'm telling you isn't extraordinary, but combined with regex, it becomes very powerful.

Sed

Display certain lines of a large file:

1
sed -n "$LINE1,${LINE2}p;${LINEA2}q;" "$FILE"

Replace all occurrences of the word "stain" with the word "whiteness" in the file my_laundry, redirecting everything to the file my_clean_laundry:

1
sed s/tache/blancheur/g mon_linge > mon_linge_propre

Replace all line beginnings with a "." in the file my_laundry, redirecting everything to my_well_arranged_laundry:

1
sed s/^/./ > mon_linge_bien_range

Replace all line endings with a "." in the file my_laundry, redirecting everything to my_well_arranged_laundry:

1
sed s/$/./ > mon_linge_bien_range

Delete all lines containing the word "stain", redirecting everything to the file my_clean_laundry:

1
sed s/.*tache.*/\\0/ > mon_linge_propre

To delete lines matching a pattern (here we delete empty lines):

1
sed '/^$/d' monfichier

Delete all lines containing the word "stain" and replace all line endings with a ".", redirecting to my_good_smelling_laundry:

1
sed -e 's/$/./' -e  's/.*tache.*/\\0/' > mon_linge_qui_sent_bon

To make modifications directly without going through another file (on the fly):

1
sed -i 's/sysconfig/defaults/' cgconfig

For purists, we can also do a substitution in perl:

1
perl -pe 's/toto/tata/g' < monfichier

Replace the content of a file on the fly:

1
perl -pi -e "s/HOSTNAME=.*/HOSTNAME=node3.deimos.fr/" /etc/sysconfig/network

Or even with the replace command (which replaces in all *.ini of the current folder):

1
replace -v from to from to "persistence=FILE" "persistence=BRIDGEPERSISTENCE" -- *.ini

or also:

1
find . -name "*.c" -exec sed -i "s/oldWord/newWord/g" '{}' \;

Delete the 10th line of a file:

1
sed -i '10d'

Delete x lines from a line number:

1
sed '1,55d' fichier.old > fichier.new

This will delete lines 1 to 55

Awk

Display the 2nd column (PID) of the result of a ps aux:

1
ps aux|awk '{print $2}'

Display the 2nd column (PID) of the result of a ps aux, preceded by a label:

1
ps aux|awk '{printf("PID: %s\n"), $2}'

Display the 2nd column (PID) of the result of a ps aux, preceded by a label and showing the user:

1
ps aux|awk '{printf("PID: %s used by %s\n"), $2, $1}'

Display /etc/passwd well arranged:

1
awk -F: '{printf("User: %s\nUID: %s\nGID: %s\nName: %s\nHome: %s\nShell: %s\n", $1, $3, $4, $5, $6, $7)}' /etc/passwd

Use mathematical functions:

1
echo "3 4" | awk '{print int($1+$2)}'

(normally displays 7, if all goes well) It is possible to have the equivalent of a grep and an awk in the same awk:

1
ip a | awk '/inet/{print $2}'

This allows me to get all IPs present on my machine.

For more examples: AWK - the reference scripting language for file processing

Grep

Grep was not planned for the program, but here it is, I'll add it anyway :-)

To display only the root line in /etc/passwd:

1
grep ^root < /etc/passwd

or

1
cat /etc/passwd | grep ^root

But this last line calls another instruction (cat), which is not necessary.

To display all lines except those containing root:

1
grep -v root < /etc/passwd

Combine greps:

1
cat /etc/passwd | grep -e toto -e tata -e titi

It's possible to see the lines before or after the grep result. To see 2 lines before:

1
grep -A2 string filename

To see 2 lines after:

1
grep -B2 string filename

To see 2 lines before and after:

1
grep -C2 string filename

FAQ

Binary database matches

If you encounter this error during a grep, your file may be too large, so grep considers it binary. But if this is not the case, you can force it to read anyway:

1
grep -a monfichier

The -a bypasses the restriction

Resources