sed/awk/grep Cheatsheet - Linux Text Processing Reference

Q: How should grep, sed and awk be combined when debugging logs?

The usual chain is filter, then edit, then aggregate: use grep -i "error" app.log to narrow to the lines you care about, use sed rules such as '/^#/d' to strip noise, then use awk to sum by column. To find the top errors: grep "error" app.log | awk "{print $1}" | sort | uniq -c | sort -rn | head. grep locates, sed rewrites and filters by line, awk extracts columns and computes — piping them together beats writing a script.

Q: sed -i edits in place and feels risky — how do I limit the damage?

Use sed -i.bak 's/old/new/g' file.txt so an automatic .bak copy is kept, and delete it once you are satisfied. Or run sed 's/old/new/g' file.txt without -i first and read the output before writing anything to disk. These two habits prevent nearly every irreversible mistake.

Q: What do $0, $1, NR and NF mean in awk?

$0 is the whole line; $1, $2 are the columns split by the field separator ($NF is the last one); NR is the current record number; NF is the number of fields in the current line. All are built-in and need no declaration. So awk '{print NR, $1}' file prints line number plus first column, and awk 'NR>=10 && NR<=20' file processes only lines 10 through 20.

Q: Why do sort -u and uniq need to be combined with sort?

uniq only compares adjacent lines — deduplication assumes equal values already sit next to each other. sort brings equal lines together, which is why the idiom is sort file.txt | uniq -c, or the one-step sort -u file.txt. Without sorting first, duplicates scattered through the file can never be recognized.

Q: Both cut and awk extract columns — when should I use which?

cut -d: -f1 /etc/passwd is fastest for "one single-character delimiter, a few fixed fields" and prints clean output, but it does not understand multi-character delimiters and does no computation. awk -F: '{print $1, $3}' /etc/passwd is more flexible: multi-character separators, conditional filtering, and aggregation. Rule of thumb — plain column extraction with cut, anything involving calculation or complex delimiters with awk.

Log triage and report statistics are almost always three stages: grep to keep the lines that matter, sed to rewrite the values that need changing, awk to aggregate by column. The sheet runs along those three paths, then widens into pipeline recipes, sort/uniq/comm for ordering and set comparison, and cut/tr/paste for column surgery. The choice between them is granularity rather than preference: grep filters whole lines, sed rewrites each line in place, and awk is the only one that can accumulate, group-count and emit sorted results per column. Pick the wrong one and the cost is obvious — counting occurrences per IP is one awk line, while grep plus wc -l means running the pipeline once per key. The pipeline section supplies the joins between stages, including filter-then-aggregate and dedupe-with-count. Two operational details carry weight: add --line-buffered to grep on large inputs, and remember $1 is safe inside single quotes in awk but must be escaped inside double quotes.

SysOps·52 commands·Last updated 2026-07-21

Typical Use Case

For developers doing log triage and report aggregation: grep to locate, sed to rewrite, awk to pull columns and total, and sort/uniq to count and deduplicate. Piped together these four cover most day-to-day text processing. The key is keeping their roles distinct — grep filters lines, sed rewrites or deletes lines, awk extracts columns and computes. Mixing them up (using grep to pull a column, say) is the most common source of wasted time for newcomers. This sheet is organised as five ready-made chains, usable directly as a troubleshooting reference.

grep Match & Search 10

grep "error" app.log

Find lines containing 'error'

grep -i "error" app.log

Case-insensitive match

grep -rn "TODO" src/

Recursively search a dir, show file and line number

grep -v "debug" app.log

Invert match, exclude lines with 'debug'

grep -c "error" app.log

Count matching lines

grep -E "error|warning" app.log

Extended regex, equivalent to egrep

grep -oE "[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+" access.log

Print only the matched IP address

grep -A 3 "error" app.log

Show 3 lines of context after a match

grep -B 2 "error" app.log

Show 2 lines of context before a match

grep -f pattern.txt app.log

Read multiple patterns from a file

sed Replace & Edit 9

sed 's/old/new/g' file.txt

Replace old with new globally

sed -i 's/old/new/g' file.txt

Edit the file in place

sed -i.bak 's/old/new/g' file.txt

Edit in place and back up the original as .bak

sed '5d' file.txt

Delete line 5

sed '/^#/d' file.conf

Delete all comment lines starting with #

sed '/^$/d' file.txt

Delete blank lines

sed -n '10,20p' file.txt

Print only lines 10-20

sed 's/[[:space:]]*$//' file.txt

Strip trailing whitespace

sed -E 's/([0-9]+)/\1/' file.txt

Extended-regex capture group, -E same as -r

awk Extract & Summarize 9

awk '{print $1}' access.log

Print column 1 (whitespace-delimited by default)

awk -F: '{print $1, $3}' /etc/passwd

Colon-delimited, print username and UID

awk 'NR==10' file.txt

Print line 10 (NR = line number)

awk 'NR>=10 && NR<=20' file.txt

Print lines 10-20

awk '{sum += $1} END {print sum}' nums.txt

Sum column 1

awk '{count[$1]++} END {for(k in count) print k, count[k]}' access.log

Group by column 1 and count occurrences

awk '$3 > 100' data.txt

Print rows where column 3 > 100

awk 'length > 80' file.txt

Print rows longer than 80 chars

awk -F: '$3 >= 1000 {print $1}' /etc/passwd

List normal users with UID >= 1000

Pipelines 4

grep "error" app.log | awk "{print \$1}" | sort | uniq -c | sort -rn | head

Top timestamps by error-log count

cat access.log | awk "{print \$1}" | sort | uniq -c | sort -rn | head -20

Top 20 IPs by request volume

grep -oE "[0-9]+ms" app.log | sort -rn | head -10

Top 10 response times

sed '/^#/d; /^$/d' nginx.conf

Delete comment and blank lines in one pass (separate commands with ;)

sort / uniq / comm 10

sort file.txt

Sort file contents (lexical by default)

sort -n file.txt

Sort numerically

sort -r file.txt

Reverse sort

sort -t: -k3 -n /etc/passwd

Sort by column 3 (colon-delimited) numerically

sort -u file.txt

Sort and deduplicate

sort file.txt | uniq -c | sort -rn | head

Classic combo: top row frequencies

uniq -c file.txt

Count consecutive duplicates (sort first)

uniq -d file.txt

Show only duplicated lines

comm file1 file2

Compare two sorted files: three columns (only file1 / only file2 / both)

diff -u file1 file2

Compare files in unified format

cut / tr / paste 10

cut -d: -f1 /etc/passwd

Extract column 1 by colon delimiter

cut -c1-10 file.txt

Extract characters 1-10 by position

cut -f1,3 file.txt

Extract columns 1 and 3 (Tab-delimited by default)

tr 'a-z' 'A-Z' < file.txt

Lowercase to uppercase

tr -d ' \t' < file.txt

Delete all spaces and tabs

tr -s '\n' < file.txt

Squeeze consecutive blank lines into one

paste file1.txt file2.txt

Merge files side by side (Tab-delimited)

column -t file.txt

Align content into a table

expand file.txt

Convert tabs to spaces

command | tee output.txt

Output to both screen and file (common in pipelines)

Parameter matrix

Parameter	Effect	Example
`grep -r / -R`	Search directories recursively; the capital -R additionally follows symbolic links.	`grep -rn "TODO" src/`
`grep -i / -v`	Ignore case when matching, or invert the test to keep only non-matching lines.	`grep -iv "debug" app.log`
`grep -C <n>`	Print n lines of context on both sides of every match.	`grep -C 3 "error" app.log`
`grep -o`	Print only the matching part of the line instead of the whole line.	`grep -oE '[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+' access.log`
`sed -i.bak`	Edit the file in place while keeping a .bak copy of the original.	`sed -i.bak 's/old/new/g' f.txt`
`sed 's/a/b/g'`	Substitute globally. Without the g flag only the first occurrence per line is replaced.	`sed 's/localhost/0.0.0.0/g' c.yaml`
`sed '/pat/d'`	Delete every line matching the pattern.	`sed '/^#/d' nginx.conf`
`awk -F<sep>`	Set the field separator, since awk splits on runs of whitespace by default.	`awk -F: '{print $1, $3}' /etc/passwd`
`awk $n / $NF`	Reference the nth field, or $NF for the last field of the record.	`awk '{print $1, $NF}' f.txt`
`awk NR / NF`	NR is the current line number and NF the number of fields on that line.	`awk 'NR>=10 && NR<=20' f.txt`
`sort \| uniq -c \| sort -rn`	Count duplicates and order the result numerically, highest first.	`awk '{print $1}' access.log \| sort \| uniq -c \| sort -rn`
`cut -d -f`	Pull fixed columns out of each line by delimiter.	`cut -d: -f1 /etc/passwd`

Common pitfalls

SymptomAfter uniq the duplicates are still there and the deduplication looks broken.

Causeuniq only compares adjacent lines, and nothing sorted the input so identical lines are scattered.

FixAlways write sort file | uniq -c, or use sort -u which does both in one step.

SymptomYou wanted to undo a sed edit, but the file on disk is already changed.

Causesed -i rewrote the file in place and no backup was kept.

FixUse sed -i.bak so a .bak file survives, or run without -i first and inspect the output before writing.

Symptomgrep fails to find a string you can plainly see in the file.

CauseThe case differs, or a metacharacter in your pattern was interpreted as a regex.

FixAdd -i, and use -F for literal search (a plain pattern would also match axb when looking for a.b).

SymptomEvery column awk prints comes back empty.

CauseWrong separator: the file is colon-delimited but -F: was omitted, so awk split on whitespace and the whole line became one field.

FixPass the real separator with awk -F: and inspect a few lines with head before writing the expression.

SymptomThe IP address counts are wrong because every line is counted as one entry.

CauseThe extraction step piped into sort and uniq was missing, or grep -o was omitted so uniq counted whole lines.

FixRun grep -oE "pattern" file | sort | uniq -c | sort -rn; the -o flag is what makes this correct.

SymptomA sed substitution emptied the file.

CauseThe / delimiter collided with a path inside the pattern, or the d command was used where s was intended.

FixSwitch to # as the delimiter when the text contains /, and preview without -i before applying.

Troubleshooting

1Look at the raw data before writing any expression
head -3 access.log
Confirming the separator and column positions up front prevents most failed awk one-liners.
2Get the top 20 entries by access volume
awk '{print $1}' access.log | sort | uniq -c | sort -rn | head -20
Extract the column, sort so identical values become adjacent, count, then order numerically: all four steps are required.
3Locate an error with surrounding context
grep -n -C 3 "FATAL" app.log
-n prints line numbers and -C 3 adds three lines either side, which is usually all the evidence you need.
4Dry-run an edit before touching the file
sed 's/old/new/g' config.yaml
Without -i it only prints; once the output looks right, apply it with sed -i.bak.
5Strip comments and blank lines in a single pass
sed '/^#/d; /^$/d' nginx.conf
Separate the commands with semicolons and one scan does the whole job.

Command Examples

Top 20 IPs by request count

awk '{print $1}' access.log | sort | uniq -c | sort -rn | head -20

awk 取第一列，sort 让相同值相邻，uniq -c 计数，sort -rn 按数值降序——四步缺一不可，直接 uniq 会因为行不相邻而失效。

Output

   1842 203.0.113.7
    931 198.51.100.22
    407 192.0.2.180

Extract URLs that returned 5xx

awk '$9 ~ /^5/ {print $7}' access.log | sort | uniq -c | sort -rn

标准 combined 日志格式里第 9 列是状态码、第 7 列是请求路径，所以条件直接落在第 9 列上。比 grep 整行后再切分快得多。

Output

     87 /api/orders
     34 /api/checkout
     12 /api/login

Strip comment and blank lines from a config

sed '/^#/d; /^$/d' nginx.conf

sed 的多条命令用分号分隔，可以在一次扫描里完成。配合 -i.bak 就能原地改文件且留有备份。

Output

worker_processes auto;
events {
    worker_connections 1024;
}

Dry-run a substitution before writing to disk

sed 's#http://#https://#g' config.yaml

先把 / 换成 # 作分隔符，避免 URL 里的斜杠与 sed 的分隔符冲突。不加 -i 只输出不改文件，确认无误后再加 -i.bak 落盘。

Common Pitfalls

sed -i edits in place with no backup and no undo; verify output without -i first, then use -i.bak.
sed defaults to BRE, where \+ \| and \( are not special; add -r for extended regular expressions.
grepping a URL path containing / collides with the delimiter — switch to another delimiter such as #.
uniq only compares adjacent lines; without a preceding sort it silently misses duplicates scattered through the file.
awk fields start at 1, unlike array indices: $0 is the whole line and $NF is the last field.
cut does not understand multi-character delimiters and computes nothing; use awk for complex separators or aggregation.

Tips

Back up before sed -i edits a file (-i.bak), or preview without -i first.
For large files, add --line-buffered so grep flushes output live; otherwise the pipe waits for the buffer to fill.
In awk, $1 inside single quotes gets the column; inside double quotes escape it as \$1 or the shell will expand it.

FAQ

How should grep, sed and awk be combined when debugging logs?

The usual chain is filter, then edit, then aggregate: use grep -i "error" app.log to narrow to the lines you care about, use sed rules such as '/^#/d' to strip noise, then use awk to sum by column. To find the top errors: grep "error" app.log | awk "{print $1}" | sort | uniq -c | sort -rn | head. grep locates, sed rewrites and filters by line, awk extracts columns and computes — piping them together beats writing a script.

sed -i edits in place and feels risky — how do I limit the damage?

Use sed -i.bak 's/old/new/g' file.txt so an automatic .bak copy is kept, and delete it once you are satisfied. Or run sed 's/old/new/g' file.txt without -i first and read the output before writing anything to disk. These two habits prevent nearly every irreversible mistake.

What do $0, $1, NR and NF mean in awk?

$0 is the whole line; $1, $2 are the columns split by the field separator ($NF is the last one); NR is the current record number; NF is the number of fields in the current line. All are built-in and need no declaration. So awk '{print NR, $1}' file prints line number plus first column, and awk 'NR>=10 && NR<=20' file processes only lines 10 through 20.

Why do sort -u and uniq need to be combined with sort?

uniq only compares adjacent lines — deduplication assumes equal values already sit next to each other. sort brings equal lines together, which is why the idiom is sort file.txt | uniq -c, or the one-step sort -u file.txt. Without sorting first, duplicates scattered through the file can never be recognized.

Both cut and awk extract columns — when should I use which?

cut -d: -f1 /etc/passwd is fastest for "one single-character delimiter, a few fixed fields" and prints clean output, but it does not understand multi-character delimiters and does no computation. awk -F: '{print $1, $3}' /etc/passwd is more flexible: multi-character separators, conditional filtering, and aggregation. Rule of thumb — plain column extraction with cut, anything involving calculation or complex delimiters with awk.

Official References

Each command links to its official documentation below, so you can verify the latest usage and read deeper.

Linux man-pages

Maintained by LaoHand

Publicly updated on Jul 21, 2026, continuously proofread against official docs.

Contact Us

Wrong command or description? Send us corrections, business inquiries or product feedback by email.

Typical Use Case

grep Match & Search 10

sed Replace & Edit 9

awk Extract & Summarize 9

Pipelines 4

sort / uniq / comm 10

cut / tr / paste 10

Parameter matrix

Common pitfalls

Troubleshooting

Command Examples

Top 20 IPs by request count

Extract URLs that returned 5xx

Strip comment and blank lines from a config

Dry-run a substitution before writing to disk

Common Pitfalls

Tips

FAQ

Related Cheatsheets

Share

Official References

Maintained by LaoHand

Contact Us