Convert CSV to JSON Lines using Miller
Transforms tabular CSV data into newline-delimited JSON (JSONL) for easy ingestion into document databases or log analyzers.
Setup
- → Install Miller (mlr): brew install miller OR apt-get install miller
- → Have a CSV file named data.csv with a header row
Cost per run
Free
The one-liner
$ mlr --icsv --ojsonl cat data.csvWhat each stage does
- [01] mlr
mlrInvokes Miller, a command-line tool for processing structured data formats. - [02] mlr
--icsvSets the input format to CSV, assuming the first row contains column headers. - [03] mlr
--ojsonlSets the output format to JSON Lines (NDJSON), where each record is a separate JSON object on a new line. - [04] mlr
catThe Miller verb that passes input records directly to the output without modification. - [05] mlr
data.csvThe target input file to be processed.
Expected output (sample)
{"id": 1, "name": "Alice", "role": "Admin"}
{"id": 2, "name": "Bob", "role": "User"}
{"id": 3, "name": "Charlie", "role": "User"}
{"id": 4, "name": "Diana", "role": "Guest"}Caveats & tips
- Footgun: If your CSV lacks a header row, Miller will treat the first data row as headers unless you pass the --implicit-csv-header flag.
- Cost/Permission: Runs entirely locally, requiring read permissions on the input file. Processing extremely large files is memory-efficient as Miller streams the data.