{"id":304,"date":"2013-04-11T12:02:52","date_gmt":"2013-04-11T16:02:52","guid":{"rendered":"\/blog\/?p=304"},"modified":"2018-07-13T10:17:08","modified_gmt":"2018-07-13T14:17:08","slug":"cool-shell-one-liner-of-the-day","status":"publish","type":"post","link":"https:\/\/www.devolve.local\/cool-shell-one-liner-of-the-day\/","title":{"rendered":"Cool Shell One-Liner of the Day"},"content":{"rendered":"<p><code>awk -F, '{print $1}' CSV | sort | uniq -c | grep -vw 1 | tee \/dev\/tty | wc -l<\/code><\/p>\n<p>UPDATE: I went back and saw this post and thought to myself, &#8220;Self, why didn&#8217;t you annotate this garbage, you cheeky bastard?&#8221; OK, so the first part is pretty clear: get the first (or whichever) column you want from a simple (unquoted) csv file, and then count dupes. The grep is where we remove non-dupes and should probably be <code>grep -Ev '^ *1 '<\/code> to avoid matching any of the csv data. Now here&#8217;s the magic. The pipe to tee \/dev\/tty echoes everything to <code>stdout<\/code>, but one copy of the output can go through more pipes before being displayed. So the <code>wc -l<\/code> is actually counting the number of entries which have duplicates (not the total number of all duplicates!), and displays that number at the bottom. <\/p>\n<p>Here&#8217;s the tail end of what I get from this on a sample csv:<\/p>\n<pre>   6 WOOD\r\n   4 WRIGHT\r\n   3 YOUNG\r\n   2 ZIMMERMAN\r\n     360<\/pre>\n","protected":false},"excerpt":{"rendered":"<p>awk -F, &#8216;{print $1}&#8217; CSV | sort | uniq -c | grep -vw 1 | tee \/dev\/tty | wc -l UPDATE: I went back and saw this post and thought to myself, &#8220;Self, why didn&#8217;t you annotate this garbage, you cheeky bastard?&#8221; OK, so the first part is pretty clear: get the first (or whichever) [&hellip;]<\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[6],"tags":[34,41],"_links":{"self":[{"href":"https:\/\/www.devolve.local\/wp-json\/wp\/v2\/posts\/304"}],"collection":[{"href":"https:\/\/www.devolve.local\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.devolve.local\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.devolve.local\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/www.devolve.local\/wp-json\/wp\/v2\/comments?post=304"}],"version-history":[{"count":6,"href":"https:\/\/www.devolve.local\/wp-json\/wp\/v2\/posts\/304\/revisions"}],"predecessor-version":[{"id":646,"href":"https:\/\/www.devolve.local\/wp-json\/wp\/v2\/posts\/304\/revisions\/646"}],"wp:attachment":[{"href":"https:\/\/www.devolve.local\/wp-json\/wp\/v2\/media?parent=304"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.devolve.local\/wp-json\/wp\/v2\/categories?post=304"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.devolve.local\/wp-json\/wp\/v2\/tags?post=304"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}