Scrape Google News and cluster articles by narrative angle
Extracts recent Google News articles for a topic using Apify and uses Claude to analyze and cluster the headlines by their underlying narrative angles.
Setup
- → Install Apify CLI (npm install -g apify-cli) and authenticate
- → Install jq
- → Install claude CLI and set ANTHROPIC_API_KEY
Cost per run
~$0.02 per run (Apify compute + Claude API)
The one-liner
$ apify call nstojiljkovic/google-news-scraper -i '{"query":"AI regulation","maxItems":30}' --output-json | jq -r '.[] | "\(.title) - \(.source)"' | claude -p "Cluster these news headlines by narrative angle. Provide a short summary for each cluster:"What each stage does
- [01] apify
apify call nstojiljkovic/google-news-scraper -i '{"query":"AI regulation","maxIt…Executes the Google News scraper actor on Apify's cloud and outputs the dataset as JSON. - [02] jq
jq -r '.[] | "\(.title) - \(.source)"'Extracts the title and publication source from each article, formatting them as raw text. - [03] claude
claude -p "Cluster these news headlines by narrative angle..."Prompts Claude to analyze the list of headlines and group them by thematic narrative or bias.
Expected output (sample)
Cluster 1: Pro-Innovation & Economic Growth - "AI regulation could stifle startup ecosystem" - TechCrunch - "Why we need to let AI build the future" - Wired Cluster 2: Safety & Existential Risk - "Lawmakers push for strict AI guardrails" - NYT - "The hidden dangers of unregulated AI models" - The Guardian
Caveats & tips
- Apify actors may change their input/output schema over time, which could break the jq parsing.
- Requires Apify platform credits and Anthropic API credits to run.