How can I collect news headlines from different sources?

You have a few good options, depending on how many sources you want and how automated you need it to be:

1. Use news APIs

Best if you want structured data and reliable access.

Examples:

• NewsAPI – aggregates headlines from many outlets
• GDELT – global news monitoring
• Bing News Search API
• The Guardian API, NYT API, etc. for specific publishers

Typical workflow:

1. Sign up for an API key
2. Send a request for headlines by topic, country, language, or source
3. Parse the JSON response and store title, source, URL, and publish time

Example in Python:

import requests

url = "https://newsapi.org/v2/top-headlines"
params = {
    "country": "us",
    "apiKey": "YOUR_API_KEY"
}

response = requests.get(url, params=params)
data = response.json()

for article in data["articles"]:
    print(article["source"]["name"], "-", article["title"])

---

2. Scrape RSS feeds

Best if the news sites provide RSS. This is often the easiest free method.

Steps:

1. Find RSS feed URLs from each source
2. Use a parser like feedparser
3. Extract headline, link, and published date

Example:

import feedparser

feeds = [
    "https://rss.nytimes.com/services/xml/rss/nyt/HomePage.xml",
    "https://feeds.bbci.co.uk/news/rss.xml"
]

for feed_url in feeds:
    feed = feedparser.parse(feed_url)
    for entry in feed.entries[:5]:
        print(feed_url, "-", entry.title)

---

3. Scrape the website directly

Useful when there’s no API or RSS feed, but this is less stable.

Tools:

• requests + BeautifulSoup
• Scrapy
• playwright / selenium for JavaScript-heavy sites

Be careful:

• Check the site’s terms of service
• Respect robots.txt
• Avoid aggressive crawling

---

4. Use a news aggregator platform

If you only need collected headlines, these may already do the aggregation:

• Google News
• Apple News
• Flipboard
• Feedly

Some offer export or integration options.

---

5. Build a multi-source pipeline

A common approach:

• Collect from RSS and APIs
• Normalize fields into one format:
  • source
  • headline
  • url
  • published_at
  • topic
• Deduplicate similar headlines
• Store in a database or spreadsheet

Example schema:

{
  "source": "BBC",
  "headline": "Markets rally after inflation data",
  "url": "https://...",
  "published_at": "2026-07-15T08:30:00Z"
}

---

Recommended approach

• Few sources, low effort: RSS feeds
• Many sources, cleaner structure: News APIs
• Custom or niche sites: Web scraping

If you want, I can help you with:

1. a Python script that collects headlines from multiple RSS feeds,
2. a script using a news API, or
3. a scraper for specific websites.