The 250 one-liners that built
data engineering in 2026.

Hand-curated shell pipelines combining apify · jq · claude · gemini · duckdb · groq · miller. Every entry runs as-is on a fresh laptop. Copy. Run. Learn.

apifyjqclaudegeminiduckdbgroqmillerparallel

Browse the list ↓🎲 Pick one at random

81 of 250 · 169 to go · finish line August 31, 2026 🏁

last updated2026-09-03

Filter by tier

all self-contained llm-cli duckdb apify aws

Filter by tool

apify ast-grep awk aws claude croc curl duckdb envfix ffprobe firecrawl gemini gron head htmlq hyperfine jc jo jq kubectl llm mlr oc parallel rga sort sponge tail uniq witr xargs yq

showing 81 of 81

Today's HN top 10, ranked by score

curl -s "https://hacker-news.firebaseio.com/v0/topstories.json" \ | jq -r '.[0:30][]' \ | …

Full-text search HN by keyword

curl -s "https://hn.algolia.com/api/v1/search?query=claude+code&tags=story&hitsPerPage=20"…

Any GitHub user's stars → markdown list

curl -s "https://api.github.com/users/torvalds/starred?per_page=50" \ | jq -r '.[] | "- [\…

Wikipedia article summary in 1 line

curl -s "https://en.wikipedia.org/api/rest_v1/page/summary/Functional_programming" \ | jq …

HN front page → Gemini 3.1 Pro analysis

curl -s "https://hacker-news.firebaseio.com/v0/topstories.json" \ | jq -r '.[0:30][]' \ | …

curljqxargsgemini

Wikipedia article → 5-year-old explanation

curl -s "https://en.wikipedia.org/api/rest_v1/page/summary/Homotopy_type_theory" \ | jq -r…

Apify hello-world → jq field extract

apify call apify/hello-world -s -o | jq -r '.[0].url'

CSV → instant stats with Miller

mlr --csv stats1 -a mean,p90,max -f price -g category sales.csv | column -t

DuckDB: SQL over a CSV in one shot

duckdb -c " SELECT category, AVG(price) AS avg_price, QUANTILE_CONT(price, 0.9…

JSON from curl → DuckDB analysis via /dev/stdin

curl -s "https://hn.algolia.com/api/v1/search?query=duckdb&hitsPerPage=100" \ | jq '.hits'…

DuckDB: query a remote Parquet over HTTPS

duckdb -c " INSTALL httpfs; LOAD httpfs; SELECT l_returnflag, l_linestatus, SUM(l_quantity…

What's my IP and where is it?

curl -fsSL -A 'Mozilla/5.0 oneliner101' https://ifconfig.co/json \ | jq -r '"\(.ip)\t\(.ci…

Live currency rates, no API key

curl -s "https://open.er-api.com/v6/latest/USD" \ | jq '{base: .base_code, time: .time_las…

Most-starred repos in any language

curl -s "https://api.github.com/search/repositories?q=language:rust&sort=stars&order=desc&…

DuckDB on NYC taxi public Parquet

duckdb -c " INSTALL httpfs; LOAD httpfs; SELECT passenger_count, ROUND(AVG(fare_amount), 2…

DuckDB: peek at any CSV's schema

duckdb -c "DESCRIBE FROM 'data.csv'"

Should I read this HN thread? Claude TL;DR

ID=42583107 # any HN story id curl -s "https://hn.algolia.com/api/v1/items/$ID" \ | jq -…

PyPI download stats for any package

curl -s "https://pypistats.org/api/packages/pandas/recent" \ | jq '.data'

Wikipedia page views over time

curl -s "https://wikimedia.org/api/rest_v1/metrics/pageviews/per-article/en.wikipedia/all-…

Schema-validated LLM output with claude --json-schema

curl -s \ "https://hn.algolia.com/api/v1/search?query=local+llm&tags=story&hitsPerPage=20"…

Gemini structured JSON envelope with -o json

curl -s "https://en.wikipedia.org/api/rest_v1/page/summary/Transformer_(deep_learning_arch…

Production-grade reasoning that doesn't stall — --fallback-model

curl -s "https://hacker-news.firebaseio.com/v0/topstories.json" \ | jq -r '.[0:30][]' \ | …

curljqxargsclaude

Pure-LLM lockdown mode with --tools ""

curl -s "https://hn.algolia.com/api/v1/search?tags=front_page&hitsPerPage=20" \ | jq -r '.…

LLM persona via --append-system-prompt

curl -s \ "https://hn.algolia.com/api/v1/search?query=AI&tags=front_page&hitsPerPage=15" \…

Capped fan-out across N inputs with --max-budget-usd

TOPICS=(machine-learning local-llm agi claude openai) mkdir -p /tmp/topic # Stage 1: scra…

parallelcurljqclaude

Groq speed lane via llm -m groq-llama-3.3-70b-versatile

curl -s "https://hn.algolia.com/api/v1/search?query=duckdb&hitsPerPage=20" \ | jq -r '.hit…

SQS: drain a queue to local JSONL

Q="https://sqs.eu-west-1.amazonaws.com/123456789012/my-queue" while MSGS=$(aws sqs receiv…

SQS: replay DLQ messages back to source

DLQ="https://sqs.eu-west-1.amazonaws.com/123/my-dlq" SRC="https://sqs.eu-west-1.amazonaws.…

Lambda: tail recent errors → claude clusters them

aws logs tail /aws/lambda/my-function \ --since 1h --filter-pattern '?ERROR ?Exception ?Tr…

CloudWatch Logs Insights from the CLI

QID=$(aws logs start-query \ --log-group-name /aws/lambda/my-function \ --start-time $(($(…

EC2: every running instance on one line each

aws ec2 describe-instances \ --filters 'Name=instance-state-name,Values=running' \ | jq -r…

S3: top 20 biggest objects in a bucket

aws s3api list-objects-v2 --bucket my-bucket --max-items 5000 \ | jq -r '.Contents | sort_…

AWS Cost Explorer: top spend → gemini analysis

START=$(date -u -v-30d +%Y-%m-%d 2>/dev/null || date -u -d '-30 days' +%Y-%m-%d) END=$(dat…

Lambda: every function ranked by memory + timeout

aws lambda list-functions --max-items 500 \ | jq -r '.Functions | sort_by(-.MemorySize) | …

SQS DLQ: peek poison pills → claude triage table

aws sqs receive-message \ --queue-url https://sqs.eu-central-1.amazonaws.com/123456789012/…

CloudWatch alarm → 30-min context dump → claude RCA memo

ALARM=prod-orders-api-5xx-rate; LOGS=/aws/lambda/prod-orders-api; \ A=$(aws cloudwatch des…

Unattached EBS volumes + snapshots → claude cleanup plan

REGION=${AWS_REGION:-eu-central-1}; NOW=$(date -u +%s); SNAPS=$(aws ec2 describe-snapshots…

RDS audit (CPU + connections, 7d) → claude downsize plan

REGION=${AWS_REGION:-eu-central-1}; END=$(date -u +%Y-%m-%dT%H:%M:%SZ); START=$(date -u -v…

SQS fleet health snapshot → claude wedge-vs-starve diagnosis

END=$(date -u +%Y-%m-%dT%H:%M:%SZ); START=$(date -u -v-1H +%Y-%m-%dT%H:%M:%SZ 2>/dev/null …

ECS stopped-task postmortem → claude root-cause classifier

aws ecs list-clusters --query 'clusterArns[]' --output text | tr '\t' '\n' | while read -r…

awsxargsjqclaude

Athena: audit last 50 queries for bytes-scanned hogs via claude

IDS=$(aws athena list-query-executions --work-group analytics-prod --max-items 50 --query …

Glue: triage last 100 job runs for failure patterns with claude

aws glue get-job-runs --job-name reprocess-events-daily --max-items 100 \ | jq '{ job: "re…

Analyze yesterday's S&P 500 performance with Claude

curl -fsSL -A 'Mozilla/5.0' "https://query1.finance.yahoo.com/v8/finance/chart/SPY?range=5…

Aggregate K8s Pod Counts by Namespace and Phase

kubectl get pods -A -o json | \ jq -c '.items[]' | \ duckdb -c "SELECT metadata.namespace,…

kubectljqduckdb

Analyze CrashLoopBackOff Pod Logs with Gemini

kubectl get pods -A -o json | \ jq -r '.items[] | select(any(.status.containerStatuses[]?;…

kubectljqxargsgemini

Analyze Kubernetes Events for Anomalies

kubectl get events -A --sort-by='.lastTimestamp' | tail -n 100 | claude -p "Review these K…

kubectltailclaude

Audit Kubernetes RBAC Roles with Claude

kubectl get clusterroles,roles -A -o yaml | claude -p "Audit this Kubernetes RBAC configur…

Find Pods Missing Resource Limits

kubectl get pods -A -o json | jq -r '.items[] | select(any(.spec.containers[]; .resources.…

List and Sort Running Container Images by Frequency

kubectl get pods -A -o json | jq -r '.items[].spec.containers[].image' | sort | uniq -c | …

kubectljqsortuniq

Analyze Top Kubernetes Warning Events with DuckDB

kubectl get events -A --field-selector type=Warning -o json | jq -c '.items[]' | duckdb -c…

kubectljqduckdb

Find Top 10 Restarting Kubernetes Containers

kubectl get pods -A -o json | jq -r '.items[] | select(.status.containerStatuses != null) …

kubectljqsorthead

Evaluate Kubernetes Node Scaling with LLM

kubectl top nodes | llm -m gpt-4o "Analyze this node utilization. Are we over-provisioned …

Find all OOMKilled pods across a Kubernetes cluster

kubectl get pods -A -o json | jq -r '.items[] | select(.status.containerStatuses[]?.lastSt…

Scan App Configs for Exposed Secrets

curl -sL https://api.example.com/config.json | jq -r '. | to_entries | map("\(.key): \(.va…

Convert CSV to JSON Lines using Miller

mlr --icsv --ojsonl cat data.csv

Run SQL GROUP BY on a CSV using DuckDB

duckdb -c "SELECT status, COUNT(*) as count FROM 'large_dataset.csv' GROUP BY status ORDER…

Scan webpages for exposed secrets using LLM

curl -sL https://targeting.ai | llm -s "Scan this text for exposed API keys, passwords, or…

Fetch and Display City Weather in Terminal

curl -s -H "Accept-Language: en" "wttr.in/London?0&m"

Analyze news with game theory via curl and OpenAI

curl -s "https://newsapi.org/v2/everything?q=OPEC&pageSize=3&apiKey=$NEWS_API_KEY" \ | jq …

Extract and Group Outbound Links via Firecrawl

curl -sX POST https://api.firecrawl.dev/v1/scrape \ -H "Authorization: Bearer $FIRECRAWL_A…

firecrawljqcurlsort

Extract and Group Page Links by Domain using Firecrawl

curl -sX POST https://api.firecrawl.dev/v1/scrape \ -H "Authorization: Bearer $FIRECRAWL_A…

curlfirecrawljq

Benchmark JSON Parsing Speed: jq vs gron

hyperfine --warmup 3 'jq . large.json > /dev/null' 'gron large.json > /dev/null'

hyperfinejqgron

Compare sentiment across Twitter, Reddit, and HN

TOPIC="nuclear energy" && \ (apify call -i "{\"search\":\"$TOPIC\",\"maxItems\":20}" quack…

Extract Website Branding with Firecrawl

curl -sX POST https://api.firecrawl.dev/v1/scrape \ -H "Authorization: Bearer $FIRECRAWL_A…

curljqfirecrawl

Extract and Count Link Domains using Firecrawl

firecrawl scrape "https://news.ycombinator.com" --json | jq -r '.data.linksOnPage[]? | sel…

firecrawljqsortuniq

Query Firecrawl search results directly with DuckDB

curl -sX POST https://api.firecrawl.dev/v1/search \ -H "Authorization: Bearer $FIRECRAWL_A…

curlfirecrawlduckdb

Scrape Google News and cluster articles by narrative angle

apify call nstojiljkovic/google-news-scraper -i '{"query":"AI regulation","maxItems":30}' …

Scrape Google Maps and query top-rated places with DuckDB

curl -s -X POST "https://api.apify.com/v2/acts/apify~google-maps-scraper/run-sync-get-data…

curlapifyduckdb

Summarize breaking news from the last hour

firecrawl search "breaking news" --tbs "qdr:h" --json \ | jq -r '.data.web[] | .title + ":…

firecrawljqclaude

Send a file securely via croc relay

croc send --code "super-secret-phrase" ./payload.tar.gz

Find leftover console.log calls by AST, not text grep

ast-grep run -p 'console.log($$$)' -l ts --json=compact . | jq -r '.[].file' | sort | uniq…

Catch .env drift from .env.example before CI does

npx --yes envfix doctor --json 2>/dev/null | jq -r '.pairs[] | .env as $e | (.missing[] | …

One-Line Health Check for Any Video or Audio File

ffprobe -v error -print_format json -show_format -show_streams input.mp4 \ | jq -r ' {dura…

Scrape a live page with CSS selectors — no LLM, no API key

curl -sS "https://pkg.go.dev/search?q=cli" | htmlq -a data-clicked-package '.SearchSnippet…

Audit a mount point for missing noexec/nosuid

mount | jc --mount \ | jq '.[] | select(.mount_point=="/dev") | {mount_point, type, noexec…

Turn a shell loop into a JSON array with jo, jq's inverse

for f in *; do [ -f "$f" ] && jo name="$f" bytes=$(stat -c%s "$f"); done | jq -s '.' | jq …

Check a Go Package's Health Before You Import It

oc go pkg golang.org/x/sync/errgroup --json \ | jq -r ' .title, (.blocks[] | select(.text?…

grep inside zips, PDFs and sqlite files, not just plain text

rga --json "aws_secret_access_key" . | jq -c 'select(.type=="match") | .data.path.text' | …

Dedupe a JSON file in place, safely

jq 'sort_by(.host) | unique_by(.host)' hosts.json | sponge hosts.json

Answer 'why is this running?' as structured JSON

set -o pipefail && ( sleep 300 & pid=$!; trap 'kill $pid 2>/dev/null' EXIT; sleep 0.1; wit…

Catch Kubernetes Containers Missing Resource Limits — Before You Apply

curl -s https://raw.githubusercontent.com/kubernetes/website/main/content/en/examples/appl…