← All one-liners·#059·Data Processing·duckdb·beginner

Run SQL GROUP BY on a CSV using DuckDB

Instantly query and aggregate large CSV files in-memory using DuckDB without importing them into a database.

Setup
  • → Install DuckDB (e.g., brew install duckdb)
  • → Have a valid CSV file named large_dataset.csv in your working directory
Cost per run
Free
The one-liner
$ duckdb -c "SELECT status, COUNT(*) as count FROM 'large_dataset.csv' GROUP BY status ORDER BY count DESC;"
What each stage does
  1. [01] duckdbduckdb
    Invokes the DuckDB command-line interface.
  2. [02] duckdb-c
    Tells DuckDB to execute the following SQL query string and exit immediately.
  3. [03] duckdbFROM 'large_dataset.csv'
    DuckDB automatically infers the schema and reads the CSV file directly from disk without a prior import step.
  4. [04] duckdbGROUP BY status ORDER BY count DESC
    Standard SQL aggregation to group rows by the status column and sort the results by frequency.
Expected output (sample)
┌────────────┬───────┐
│   status   │ count │
│  varchar   │ int64 │
├────────────┼───────┤
│ SUCCESS    │ 45210 │
│ PENDING    │  8432 │
│ FAILED     │  1205 │
└────────────┴───────┘
Caveats & tips
  • Memory usage scales with the size of the aggregation, though DuckDB is highly optimized for out-of-core processing if memory runs low.
  • Ensure the CSV file path is correct and the file has read permissions for the executing user.