Linux Pipes — Cheatsheet

ls -l | sort -n -k 5 | tail -n 1 | awk '{print $NF}'

Prints the name of the largest file in the current directory.

1. ls -lLong listing, one file per line. Field 5 is the size in bytes.
2. sort -n -k 5-k 5 sorts on field 5, -n compares those fields as numbers, not text (so 9 sorts before 100).
3. tail -n 1Keeps only the last line — the biggest, since the sort was ascending.
4. awk '{print $NF}'NF is the number of fields on the line, so $NF is the last one: the filename.

ls -1 | wc -l

How many entries are in this directory?

1. ls -1One entry per line (digit one, not letter L). Hidden dotfiles are skipped; add -A to include them.
2. wc -lCounts lines on stdin.

grep -c "error" logfile

How many lines contain "error"? No pipe needed.

1. grep -cCounts matching lines directly. grep x file | wc -l does the same thing with an extra process.

./myprogram | head -n 20

Look at the first 20 lines of a noisy program.

1. ./myprogramWrites to stdout, which is now the pipe.
2. head -n 20Prints 20 lines, then exits and closes its read end.

sed -n '81,100p' file.txt

Show lines 81–100.

1. sed -nSuppresses the default "print every line".
2. '81,100p'For lines 81 through 100, do p (print). The pipeline version, head -n 100 | tail -n 20, also works and is easier to remember.

dmesg | less

Page through long output instead of losing it off the top of the screen.

1. dmesgKernel ring buffer — thousands of lines.
2. lessInteractive pager. /word searches, q quits, arrows scroll.

sort file.txt | uniq -c | sort -nr | head -n 10

The ten most common lines, with counts. The single most useful pipeline in Unix.

1. sortGroups identical lines next to each other. Required — see the note.
2. uniq -cCollapses each run of identical lines into one, prefixed with the count.
3. sort -nrSorts by that count, -n numerically, -r descending.
4. head -n 10Top ten.

cut -d: -f1 /etc/passwd | sort

Every username on the machine, alphabetically.

1. cut -d: -f1Splits each line on : and keeps field 1. /etc/passwd is colon-separated.
2. sortAlphabetical order.

awk '{print $1}' access.log | sort | uniq -c | sort -rn | head

Which clients hit this server most?

1. awk '{print $1}'Prints field 1 of every line (the IP). awk splits on whitespace by default, so runs of spaces do not confuse it the way cut -d' ' would.
2. sort | uniq -cThe count-the-duplicates idiom from above.
3. sort -rn | headBiggest first, top ten (head defaults to 10).

tr -s '[:space:]' '\n' < essay.txt | sort | uniq -c | sort -nr | head

Word frequency.

1. < essay.txtFeeds the file in as stdin. No cat required.
2. tr -s '[:space:]' '\n'Translates every whitespace character to a newline; -s squeezes repeats so blank lines do not pile up. Result: one word per line.
3. sort | uniq -c | sort -nr | headCount and rank, as before.

grep -v '^#' config.conf | grep -v '^$'

A config file with the comments and blank lines stripped out.

1. grep -v '^#'-v inverts the match; ^# is "hash at start of line". So: drop comments.
2. grep -v '^$'^$ is a line with nothing between start and end. So: drop blank lines.

sed 's/localhost/127.0.0.1/g' hosts.txt | tee fixed.txt | wc -l

Substitute, save the result, and count it in one pass.

1. sed 's/old/new/g'Substitute; g means every occurrence on the line, not just the first.
2. tee fixed.txtWrites stdin to the file and passes it along. A T-junction in the pipe.
3. wc -lCounts what came through.

ps aux | grep myprogram | grep -v grep

Is my program running, and what is its PID?

1. ps auxEvery process on the machine, with user, PID, %CPU, %MEM and command.
2. grep myprogramKeeps the lines mentioning it.
3. grep -v grepRemoves the grep process itself, which is running with "myprogram" in its own command line. pgrep -a myprogram avoids the whole problem.

ps aux | sort -rn -k 3 | head -n 5

The five processes burning the most CPU.

1. sort -rn -k 3Field 3 of ps aux is %CPU. Numeric, reversed.
2. head -n 5Worst five. Swap to -k 4 for memory.

ls /proc/$(pgrep -n myprogram)/fd | wc -l

How many file descriptors does my program have open? The fastest way to catch a missing close().

1. pgrep -n myprogramThe PID of the newest matching process.
2. $( ... )Command substitution: run it and paste its output into the command line.
3. /proc/PID/fdA directory with one entry per open descriptor. On Linux the kernel invents it on the fly; symlinks point at files, sockets or pipe:[12345].
4. wc -lThe count. Watch it climb in a leaky loop.

ls -l /proc/$(pgrep -n myprogram)/fd | grep pipe

Which of those descriptors are pipes, and which pipe is which.

1. ls -lShows the symlink targets.
2. grep pipeKeeps the pipe:[N] ones. Two descriptors showing the same N are the two ends of one pipe — exactly what you are counting in Lab 2.

history | awk '{$1=""; print}' | sort | uniq -c | sort -nr | head

Which commands do I actually type?

1. historyYour shell history, each line numbered.
2. awk '{$1=""; print}'Blanks out field 1 (the number) and prints the rest.
3. sort | uniq -c | sort -nr | headRank them.

ss -tlnp | grep LISTEN

What is listening on a port?

1. ssSocket statistics; the modern replacement for netstat.
2. -tlnpt TCP, l listening only, n numeric ports (no DNS lookups), p show the owning process.

make 2>&1 | head -n 30

See the first compiler errors, not the last screenful.

1. 2>&1Redirects stderr (fd 2) into whatever stdout (fd 1) currently is — the pipe. Compiler errors go to stderr, and stderr is not piped by default. Without this you pipe an empty stream.
2. head -n 30The first error is the one that matters; the rest are usually fallout.

make 2>&1 | grep -E 'error|warning'

Just the problems, out of a wall of build output.

1. grep -EExtended regular expressions, so | inside the quotes means "or". The quotes are what stop the shell from reading that | as another pipe.

./myprogram | diff - expected.txt

Does my output match the expected output? No temp file.

1. diff - expected.txtA lone - means "read stdin here". So the left side of the diff is your program's live output. Silence means identical; echo $? gives 0.

./myprogram | tee out.txt | tail -n 5

Keep the full output for later while watching the end of it now.

1. tee out.txtFull stream to the file, full stream onward.
2. tail -n 5Last five lines on screen.

strace -f -e trace=pipe,fork,clone,read,write,close ./myprogram 2>&1 | less

Watch the actual system calls. When a pipe program hangs, this shows you exactly which read() is blocked.

1. strace -fTrace system calls, -f follows children through fork() — essential, since your bug is usually in the child.
2. -e trace=...Only these calls, otherwise the noise is unreadable.
3. 2>&1 | lessstrace writes to stderr, so redirect before paging.

valgrind --leak-check=full ./myprogram 2>&1 | tail -n 20

The leak summary without scrolling.

1. valgrindReports leaks and invalid memory use, on stderr.
2. tail -n 20The summary block lives at the end.

grep -rn "close(" src/ | wc -l

Count your closes — the first thing to check when a pipeline deadlocks.

1. grep -rn-r recurses into the directory, -n prints line numbers so you can jump straight there.
2. wc -lFour closes per pipe per process pair is the target.

find . -name '*.c' -print0 | xargs -0 wc -l | sort -n

Line counts for every C file, smallest to largest.

1. find . -name '*.c'Recursive search by name. Quote the pattern so the shell does not expand it first.
2. -print0 / xargs -0Separate names with a NUL byte instead of a newline. NUL is the one character a filename cannot contain, so this survives spaces in names. Plain | xargs breaks on my file.c.
3. xargsTurns lines of stdin into arguments for a command. Necessary because wc takes filenames as arguments, not on stdin.

du -sh * | sort -h | tail -n 5

What is eating my disk quota?

1. du -sh *Disk usage, -s one summary line per entry, -h human-readable (4.0K, 12M, 1.3G).
2. sort -hSorts those human-readable suffixes correctly — -n alone would rank 9K above 1G.
3. tail -n 5The five biggest.

df -h | grep -v tmpfs

Real filesystems and their free space, without the noise.

1. df -hFree space per mounted filesystem, human-readable.
2. grep -v tmpfsDrops the in-memory filesystems you do not care about.

find . -name '*.o' -print0 | xargs -0 rm -f

Delete every object file in the tree.

1. find ... -print0Find them, NUL-separated.
2. xargs -0 rm -fHand them to rm in batches.

tar -czf - project/ | ssh user@vm 'cat > backup.tgz'

Archive a directory straight onto another machine, no temp file.

1. tar -czf -Create, gzip, and write the archive to -, meaning stdout.
2. ssh user@vm 'cat > backup.tgz'ssh forwards its stdin to the remote command, which writes it to a file there. The pipe crossed a network.