How do I extract table data from websites that only show it in HTML?

To extract table data from websites that present it in HTML, you usually have a few options depending on how the page is built.

1) Check if the table is directly in the HTML

If the table appears in the page source, you can scrape it with tools like:

• Python + pandas
• BeautifulSoup
• requests + lxml
• Node.js + cheerio

Python example with pandas

import pandas as pd

url = "https://example.com/page-with-table"
tables = pd.read_html(url)

print(len(tables))      # number of tables found
print(tables[0].head()) # first table

This is often the easiest method if the table is a normal <table> element.

---

2) Use BeautifulSoup for more control

If pandas.read_html() doesn’t work well, parse the HTML yourself.

import requests
from bs4 import BeautifulSoup

url = "https://example.com/page-with-table"
html = requests.get(url).text
soup = BeautifulSoup(html, "html.parser")

table = soup.find("table")
rows = table.find_all("tr")

data = []
for row in rows:
    cells = row.find_all(["th", "td"])
    data.append([cell.get_text(strip=True) for cell in cells])

for row in data:
    print(row)

---

3) If the table is loaded dynamically with JavaScript

Sometimes the HTML source doesn’t contain the table data yet; it’s inserted later by JavaScript. In that case:

• Inspect the page network requests in browser dev tools
• Look for an API endpoint returning JSON
• Use a browser automation tool if needed:
  • Selenium
  • Playwright
  • Puppeteer

Selenium example

from selenium import webdriver
from bs4 import BeautifulSoup
import time

driver = webdriver.Chrome()
driver.get("https://example.com/page-with-table")
time.sleep(3)  # wait for JS to load

html = driver.page_source
soup = BeautifulSoup(html, "html.parser")
table = soup.find("table")

driver.quit()

---

4) If data is embedded in scripts or hidden JSON

Some websites put the data in <script> tags or JavaScript variables. In that case:

• Inspect the page source
• Search for JSON-like data
• Parse it directly if possible

---

5) Export the extracted data

Once extracted, you can save it as CSV or Excel.

CSV with pandas

df = tables[0]
df.to_csv("table.csv", index=False)

Excel

df.to_excel("table.xlsx", index=False)

---

6) Important considerations

• Check the website’s terms of service
• Respect robots.txt
• Don’t overload the site with too many requests
• Use headers and rate limiting if scraping responsibly

---

Quick recommendation

If the table is in plain HTML, start with:

import pandas as pd
df = pd.read_html("https://example.com")[0]

If that fails, inspect whether the data is loaded by JavaScript and switch to Selenium/Playwright or find the backend API.

If you want, I can show you:

1. a Python script for a specific website, or
2. how to tell whether a table is static HTML or JavaScript-loaded.