Skip to content
Laviyx Learn

Try

Linux and Bash course menu

Linux Text Processing: sort, uniq, cut, tr and paste

Process text from the command line: sort lines, remove and count duplicates with uniq, extract columns with cut, translate characters with tr, and build a top 10 report.

Beginner5 min readUpdated
On this page

These small tools turn the command line into a quick data tool. We use this file:

Terminal
cat staff.csv
Output
name,city,department,salary
Asha,Hyderabad,Engineering,90000
Ravi,Pune,Sales,55000
Meena,Hyderabad,Engineering,105000
Karan,Chennai,Support,42000
Divya,Pune,Engineering,88000
Arjun,Hyderabad,Sales,61000

sort#

sort orders lines alphabetically. -n sorts numbers, -r reverses, -t sets the separator, and -k chooses the column:

Terminal
tail -n +2 staff.csv | sort -t, -k4 -nr | head -n 3
Output
Meena,Hyderabad,Engineering,105000
Asha,Hyderabad,Engineering,90000
Divya,Pune,Engineering,88000

Without -n, 9 would sort after 10, because text is compared one character at a time.

cut#

cut extracts columns. -d is the separator and -f lists the fields:

Terminal
cut -d, -f1,4 staff.csv | head -n 3
Output
name,salary
Asha,90000
Ravi,55000

cut handles single-character separators only. For columns separated by runs of spaces, use awk.

uniq#

uniq removes adjacent duplicates, so always sort first. -c counts:

Terminal
tail -n +2 staff.csv | cut -d, -f2 | sort | uniq
echo ---
tail -n +2 staff.csv | cut -d, -f2 | sort | uniq -c
Output
Chennai
Hyderabad
Pune
---
      1 Chennai
      3 Hyderabad
      2 Pune

sort -u is a shortcut for sort | uniq.

The top N pattern#

This pipeline answers "what are the most common values?". Learn it by heart:

Terminal
tail -n +2 staff.csv | cut -d, -f3 | sort | uniq -c | sort -nr | head -n 2
Output
      3 Engineering
      2 Sales

You will use it for top IP addresses, top error messages, and much more. The log report project relies on it.

tr#

tr translates or deletes characters:

Terminal
echo "Laviyx Learn" | tr 'a-z' 'A-Z'
echo "a,b,c" | tr ',' '\n'
echo "Order   1042" | tr -s ' '
echo "Phone: 98-76-54" | tr -d '-'
Output
LAVIYX LEARN
a
b
c
Order 1042
Phone: 987654

-s squeezes repeats, and -d deletes characters.

paste#

paste joins lines side by side:

Terminal
printf 'a\nb\n' > left.txt
printf '1\n2\n' > right.txt
paste -d: left.txt right.txt
Output
a:1
b:2

Count things#

wc -l counts lines, so add it to the end of a pipe:

Terminal
tail -n +2 staff.csv | grep -c Hyderabad
tail -n +2 staff.csv | cut -d, -f2 | sort -u | wc -l
Output
3
3

Common mistakes#

Practice#

Practice exercise

Print the name of the highest-paid person in Pune.

Show solution
Terminal
grep Pune staff.csv | sort -t, -k4 -nr | head -n 1 | cut -d, -f1
Output
Divya

Key takeaway#

Next: Editing Text with sed.

Finished this lesson?

Related lessons

Linux and BashLinux Pipes and Redirection Explained: |, >, >> and 2>&1Join commands with pipes, send output to files with > and >>, understand stdin, stdout and stderr, hide errors with /dev/null, use tee, command substitution and xargs.Beginner6 min readLinux and BashLinux grep Command with ExamplesSearch text with grep: ignore case, count matches, show line numbers and context, search folders recursively, and use regular expressions with grep -E.Beginner5 min readLinux and BashLinux awk Tutorial: Columns, Filters and TotalsLearn awk with examples: print columns, set field separators, filter rows, calculate totals and averages, group with associative arrays, and format reports with printf.Intermediate5 min read