← All one-liners·#066·Data Analysis·apify·expert

Compare sentiment across Twitter, Reddit, and HN

Scrapes recent posts about a topic from three major social platforms using Apify and uses Claude to analyze and compare their sentiments.

Setup
  • → npm install -g apify-cli @anthropic-ai/claude-cli
  • → apify login
  • → export ANTHROPIC_API_KEY=your_api_key
Cost per run
~$0.05 per run (Apify compute + Claude API)
The one-liner
$ TOPIC="nuclear energy" && \
(apify call -i "{\"search\":\"$TOPIC\",\"maxItems\":20}" quacker/twitter-scraper | jq '[.[] | {site:"Twitter", text:.full_text}]'; \
 apify call -i "{\"search\":\"$TOPIC\",\"maxItems\":20}" trudax/reddit-scraper | jq '[.[] | {site:"Reddit", text:.title}]'; \
 apify call -i "{\"query\":\"$TOPIC\",\"maxItems\":20}" lukaskrivka/hacker-news-scraper | jq '[.[] | {site:"HN", text:.title}]') \
| jq -s 'flatten' \
| claude -p "Analyze sentiment for '$TOPIC' across platforms. Output a markdown table comparing Twitter, Reddit, and HN."
What each stage does
  1. [01] bashTOPIC="nuclear energy"
    Sets the search topic as an environment variable to ensure consistency across the three scraper calls.
  2. [02] apifyapify call -i "{...}" <actor>
    Invokes Apify actors for Twitter, Reddit, and Hacker News to scrape recent posts matching the topic.
  3. [03] jqjq '[.[] | {site:"...", text:...}]'
    Normalizes the disparate JSON schemas from the three scrapers into a standard format.
  4. [04] jqjq -s 'flatten'
    Slurps the three separate JSON arrays into a single, unified flat array of posts.
  5. [05] claudeclaude -p "Analyze sentiment..."
    Prompts the Claude LLM to read the combined JSON and perform a cross-platform sentiment analysis.
Expected output (sample)
| Platform | Sentiment | Key Themes |
|---|---|---|
| Twitter | Highly Polarized | Quick reactions, political framing, high emotion |
| Reddit | Cautiously Optimistic | Technical discussions, regulatory concerns, long-form debate |
| Hacker News | Pragmatic / Positive | Engineering challenges, startup opportunities, cost analysis |
Caveats & tips
  • Apify actors are maintained by third parties; their input schemas (like `search` vs `query`) or output formats may change over time.
  • Running multiple scrapers concurrently consumes Apify compute units (CUs) and Claude tokens; monitor your billing limits.