Ad space (banner)
🖥️Linux Lessons
Lesson 24 / 61

Merging Duplicate Lines with uniq

uniq is for when "I want to collapse duplicate content in a file where the same lines repeat in a row." uniq collapses adjacent identical lines down into a single line.

Use it in the form uniq filename. Add the -c option to also show how many times each line repeated. Since uniq only compares neighboring lines, running sort first is standard practice.

Running uniq sorted-fruits.txt against an already-sorted file collapses each run of consecutive duplicate lines into a single line.

A common early mistake is using uniq alone without sorting first, so identical lines that aren't adjacent don't get collapsed. Remember the combination sort file | uniq.

In real work, this is used constantly as part of aggregation — for example, tallying duplicate IP addresses from an access log.

Linux
Try it (type this into the pseudo terminal)
uniq sorted-fruits.txt

🧑‍💻 You can type this command into the Linux pseudo terminal to try it yourself (this page itself has no interactive terminal).

Ad space (banner)
Ad space (in-article)