miller: CSV and JSON Without Opening a Spreadsheet

The file is a CSV. You do not need a spreadsheet to sum a column. mlr reads the header and does the math.

miller is a tool for CSV, TSV, and JSON. The package is named miller. The command is mlr. It uses the header row as field names, so you ask for n instead of column 2. Checked on 6.22.0: stats1 summed a column to 5, cut kept one field, and --json cat pretty-printed an object. A put that multiplied n by 10 also ran.

install

macOS
# macOS. The formula is miller. The command is mlr
brew install miller
Debian/Ubuntu
# package name is miller. The command is still mlr
sudo apt install miller
Fedora
# package name is miller
sudo dnf install miller
Arch
# package name is miller
sudo pacman -S miller

examples

# Sum the n column
mlr --csv stats1 -a sum -f n data.csv
# Keep one column
mlr --csv cut -f name data.csv
# Pretty-print JSON from stdin
echo '{"n":1}' | mlr --json cat
# Compute a field
echo '{"n":2}{"n":3}' | mlr --json put '$n = $n * 10'

pro tip

Verbs chain with then: mlr --csv stats1 -a sum -f n then sort -f n_sum data.csv. Format flags like --c2j convert. If mlr says the filename is not a verb, you forgot the verb: mlr --c2j cat data.csv, or send the file on stdin.

honest limits

mlr is not a database. A multi-gigabyte CSV will work until it does not fit the way you chained the verbs. It will not fix a CSV whose header is a lie. The command is mlr. Typing miller will not find it.