Compare sentiment across Twitter, Reddit, and HN
Scrapes recent posts about a topic from three major social platforms using Apify and uses Claude to analyze and compare their sentiments.
Setup
- → npm install -g apify-cli @anthropic-ai/claude-cli
- → apify login
- → export ANTHROPIC_API_KEY=your_api_key
Cost per run
~$0.05 per run (Apify compute + Claude API)
The one-liner
$ TOPIC="nuclear energy" && \
(apify call -i "{\"search\":\"$TOPIC\",\"maxItems\":20}" quacker/twitter-scraper | jq '[.[] | {site:"Twitter", text:.full_text}]'; \
apify call -i "{\"search\":\"$TOPIC\",\"maxItems\":20}" trudax/reddit-scraper | jq '[.[] | {site:"Reddit", text:.title}]'; \
apify call -i "{\"query\":\"$TOPIC\",\"maxItems\":20}" lukaskrivka/hacker-news-scraper | jq '[.[] | {site:"HN", text:.title}]') \
| jq -s 'flatten' \
| claude -p "Analyze sentiment for '$TOPIC' across platforms. Output a markdown table comparing Twitter, Reddit, and HN."What each stage does
- [01] bash
TOPIC="nuclear energy"Sets the search topic as an environment variable to ensure consistency across the three scraper calls. - [02] apify
apify call -i "{...}" <actor>Invokes Apify actors for Twitter, Reddit, and Hacker News to scrape recent posts matching the topic. - [03] jq
jq '[.[] | {site:"...", text:...}]'Normalizes the disparate JSON schemas from the three scrapers into a standard format. - [04] jq
jq -s 'flatten'Slurps the three separate JSON arrays into a single, unified flat array of posts. - [05] claude
claude -p "Analyze sentiment..."Prompts the Claude LLM to read the combined JSON and perform a cross-platform sentiment analysis.
Expected output (sample)
| Platform | Sentiment | Key Themes | |---|---|---| | Twitter | Highly Polarized | Quick reactions, political framing, high emotion | | Reddit | Cautiously Optimistic | Technical discussions, regulatory concerns, long-form debate | | Hacker News | Pragmatic / Positive | Engineering challenges, startup opportunities, cost analysis |
Caveats & tips
- Apify actors are maintained by third parties; their input schemas (like `search` vs `query`) or output formats may change over time.
- Running multiple scrapers concurrently consumes Apify compute units (CUs) and Claude tokens; monitor your billing limits.