Introduction

Sed & Awk, but what are they? These are two binaries for content filtering. For example, if you have a file and you want to output only specific lines, or even replace things in a file. Of course, what I’m telling you isn’t extraordinary, but combined with regex, it becomes very powerful.

Sed

Display certain lines of a large file:

1

sed -n "$LINE1,${LINE2}p;${LINEA2}q;" "$FILE"

Replace all occurrences of the word “stain” with the word “whiteness” in the file my_laundry, redirecting everything to the file my_clean_laundry:

1

sed s/tache/blancheur/g mon_linge > mon_linge_propre

Replace all line beginnings with a “.” in the file my_laundry, redirecting everything to my_well_arranged_laundry:

1

sed s/^/./ > mon_linge_bien_range

Replace all line endings with a “.” in the file my_laundry, redirecting everything to my_well_arranged_laundry:

1

sed s/$/./ > mon_linge_bien_range

Delete all lines containing the word “stain”, redirecting everything to the file my_clean_laundry:

1

sed s/.*tache.*/\\0/ > mon_linge_propre

To delete lines matching a pattern (here we delete empty lines):

1

sed '/^$/d' monfichier

Delete all lines containing the word “stain” and replace all line endings with a “.”, redirecting to my_good_smelling_laundry:

1

sed -e 's/$/./' -e  's/.*tache.*/\\0/' > mon_linge_qui_sent_bon

To make modifications directly without going through another file (on the fly):

1

sed -i 's/sysconfig/defaults/' cgconfig

For purists, we can also do a substitution in perl:

1

perl -pe 's/toto/tata/g' < monfichier

Replace the content of a file on the fly:

1

perl -pi -e "s/HOSTNAME=.*/HOSTNAME=node3.deimos.fr/" /etc/sysconfig/network

Or even with the replace command (which replaces in all *.ini of the current folder):

1

replace -v from to from to "persistence=FILE" "persistence=BRIDGEPERSISTENCE" -- *.ini

or also:

1

find . -name "*.c" -exec sed -i "s/oldWord/newWord/g" '{}' \;

Delete the 10th line of a file:

1

sed -i '10d'

Delete x lines from a line number:

1

sed '1,55d' fichier.old > fichier.new

This will delete lines 1 to 55

Awk

Display the 2nd column (PID) of the result of a ps aux:

1

ps aux|awk '{print $2}'

Display the 2nd column (PID) of the result of a ps aux, preceded by a label:

1

ps aux|awk '{printf("PID: %s\n"), $2}'

Display the 2nd column (PID) of the result of a ps aux, preceded by a label and showing the user:

1

ps aux|awk '{printf("PID: %s used by %s\n"), $2, $1}'

Display /etc/passwd well arranged:

1

awk -F: '{printf("User: %s\nUID: %s\nGID: %s\nName: %s\nHome: %s\nShell: %s\n", $1, $3, $4, $5, $6, $7)}' /etc/passwd

Use mathematical functions:

1

echo "3 4" | awk '{print int($1+$2)}'

(normally displays 7, if all goes well) It is possible to have the equivalent of a grep and an awk in the same awk:

1

ip a | awk '/inet/{print $2}'

This allows me to get all IPs present on my machine.

For more examples: AWK - the reference scripting language for file processing

Grep

Grep was not planned for the program, but here it is, I’ll add it anyway :-)

To display only the root line in /etc/passwd:

1

grep ^root < /etc/passwd

or

1

cat /etc/passwd | grep ^root

But this last line calls another instruction (cat), which is not necessary.

To display all lines except those containing root:

1

grep -v root < /etc/passwd

Combine greps:

1

cat /etc/passwd | grep -e toto -e tata -e titi

It’s possible to see the lines before or after the grep result. To see 2 lines before:

1

grep -A2 string filename

To see 2 lines after:

1

grep -B2 string filename

To see 2 lines before and after:

1

grep -C2 string filename

FAQ

Binary database matches

If you encounter this error during a grep, your file may be too large, so grep considers it binary. But if this is not the case, you can force it to read anyway:

1

grep -a monfichier

The -a bypasses the restriction

Resources

Edit this page

Last updated 01 Jan 2014, 15:45 +0200. history