Scrape a live page with CSS selectors — no LLM, no API key
htmlq applies jq-style querying to HTML: give it a CSS selector, get back matched text or attributes. No Firecrawl call, no headless browser, no rendering — just curl piped into a real HTML parser, the same selector syntax you'd type into browser devtools.
Setup
- → cargo install htmlq # Rust; no apt or npm package as of Sep 2026
Cost per run
free
The one-liner
$ curl -sS "https://pkg.go.dev/search?q=cli" | htmlq -a data-clicked-package '.SearchSnippet h2 a' | sort -uWhat each stage does
- [01] curl
curl -sS "https://pkg.go.dev/search?q=cli"Fetch the rendered search-results HTML for the query 'cli' from Go's official package index — a real server-rendered page, not a JSON API. - [02] htmlq
htmlq -a data-clicked-package '.SearchSnippet h2 a'CSS-select every `<a>` inside an `<h2>` inside a `.SearchSnippet` block, then print its `data-clicked-package` attribute instead of the visible link text — `-a` switches htmlq from text extraction to attribute extraction. - [03] sort
sort -uDedupe and alphabetize the extracted Go import paths.
Expected output (sample)
code.cloudfoundry.org/cli/util/configv3 github.com/apex/log/handlers/cli github.com/cli/go-gh/v2 github.com/cloudflare/cfssl/cli github.com/cometbft/cometbft/libs/cli github.com/cosmos/cosmos-sdk/x/auth/client/cli github.com/desertbit/grumble github.com/docker/cli/cli/command github.com/jawher/mow.cli github.com/micro/cli/v2 github.com/minio/cli github.com/mitchellh/cli ... (25 unique packages total)
Caveats & tips
- Verified on htmlq 0.4.0 (built via `cargo install htmlq`) against the live pkg.go.dev search page on 2026-09-02.
- htmlq panics with a BrokenPipe backtrace if a downstream command closes the stream early (e.g. piping straight into `head`) — harmless to the extracted data already printed, but pipe into something that consumes the full stream (sort, wc, grep) instead.
- CSS class names are markup-version-dependent: any site, pkg.go.dev included, can rename them on a redesign — re-inspect with devtools if a selector stops matching.
- Only works on server-rendered HTML; a client-side-rendered (JS-hydrated) page needs something that executes JS first — this pattern is the no-JS-needed half of the spectrum.