Reading HTML tables in Data Analysis Python

Introduction

We use reading HTML tables to get data from web pages easily. It helps us collect information without typing it all manually.

You want to get sports scores from a website table.

You need financial data shown in tables on a web page.

You want to collect weather data presented in HTML tables.

You are scraping product prices listed in a table on an online store.

You want to analyze election results published as tables on news sites.

Syntax

Data Analysis Python

import pandas as pd

# Read all tables from a webpage
tables = pd.read_html('URL')

# Access the first table
first_table = tables[0]

pd.read_html() returns a list of tables found on the page.

You can select the table you want by its position in the list, starting at 0.

Examples

This code reads all tables from the URL and prints how many tables were found.

Data Analysis Python

import pandas as pd

url = 'https://example.com/data'
tables = pd.read_html(url)
print(len(tables))

This reads the first table from the webpage and shows the first 5 rows.

Data Analysis Python

import pandas as pd

url = 'https://example.com/data'
table = pd.read_html(url)[0]
print(table.head())

This reads tables that contain the word 'Population' in their content.

Data Analysis Python

import pandas as pd

url = 'https://example.com/data'
tables = pd.read_html(url, match='Population')
print(tables[0])

Sample Program

This program fetches the Wikipedia page listing countries by GDP. It reads all tables, selects the first one, and prints the first 5 rows to see the data.

Data Analysis Python

import pandas as pd

# URL with sample HTML tables
url = 'https://en.wikipedia.org/wiki/List_of_countries_by_GDP_(nominal)'

# Read all tables from the page
all_tables = pd.read_html(url)

# Select the first table which lists countries and GDP
gdp_table = all_tables[0]

# Show the first 5 rows
print(gdp_table.head())

Important Notes

Sometimes web pages have multiple tables; you may need to try different indexes to find the right one.

Make sure you have internet connection when reading tables from live URLs.

Some tables may require cleaning after reading to fix headers or data types.

Summary

Use pd.read_html() to get tables from web pages easily.

The function returns a list of tables; pick the one you want by index.

This method helps automate data collection from websites without manual copying.

Practice

1. What does the pd.read_html() function in pandas do?

easy

A. It converts a DataFrame into an HTML table string.

B. It reads all HTML tables from a webpage and returns them as a list of DataFrames.

C. It downloads an HTML file from the internet.

D. It parses JSON data from a webpage.

Solution

1. Step 1: Understand the purpose of pd.read_html()

   This function is designed to extract tables from HTML content on web pages.

2. Step 2: Recognize the output type

   The function returns a list where each element is a DataFrame representing one HTML table found.

3. Final Answer:

   It reads all HTML tables from a webpage and returns them as a list of DataFrames. -> Option B

4. Quick Check:

   pd.read_html() = list of DataFrames [OK]

Hint: Remember: read_html returns a list of tables, not a single table [OK]

Common Mistakes:

• Thinking it returns a single DataFrame instead of a list
• Confusing it with functions that write HTML
• Assuming it downloads files instead of parsing HTML

2. Which of the following is the correct way to read the first HTML table from a webpage URL stored in url using pandas?

easy

A. df = pd.read_html(url)[1]

B. df = pd.read_html(url, index=0)

C. df = pd.read_html(url).first()

D. tables = pd.read_html(url); df = tables[0]

Solution

1. Step 1: Recall the return type of pd.read_html()

   The function returns a list of DataFrames representing all tables found.

2. Step 2: Access the first table correctly

   Use list indexing [0] to get the first table from the list.

3. Final Answer:

   tables = pd.read_html(url); df = tables[0] -> Option D

4. Quick Check:

   Use list index 0 to get first table [OK]

Hint: Use index 0 on the list returned by read_html to get first table [OK]

Common Mistakes:

• Using .first() method which does not exist
• Passing index=0 as a parameter to read_html
• Using index 1 instead of 0 for first table

3. Given the code below, what will be the output of print(len(tables)) if the webpage contains 3 HTML tables?

import pandas as pd
url = 'https://example.com'
tables = pd.read_html(url)
print(len(tables))

medium

A. 0

B. 1

C. 3

D. Error: pd.read_html() requires a file path

Solution

1. Step 1: Understand what pd.read_html() returns

   It returns a list of DataFrames, one for each HTML table found on the page.

2. Step 2: Count the number of tables

   Since the webpage has 3 tables, the list length will be 3.

3. Final Answer:

   3 -> Option C

4. Quick Check:

   len(tables) = number of tables found [OK]

Hint: Count tables by checking length of list returned by read_html [OK]

Common Mistakes:

• Assuming it returns a single DataFrame
• Thinking it returns 0 if tables exist
• Believing it needs a local file path only

4. What is wrong with this code snippet to read the first HTML table from a URL?

import pandas as pd
url = 'https://example.com'
df = pd.read_html(url)[1]

medium

A. It tries to access the second table but may cause IndexError if only one table exists.

B. The syntax for pd.read_html is incorrect; it needs a file path, not a URL.

C. pd.read_html returns a DataFrame, so indexing with [1] is invalid.

D. The variable name df is reserved and cannot be used.

Solution

1. Step 1: Check the indexing used

   The code uses index [1], which means it tries to get the second table.

2. Step 2: Consider possible errors

   If the webpage has only one table, index 1 is out of range and causes IndexError.

3. Final Answer:

   It tries to access the second table but may cause IndexError if only one table exists. -> Option A

4. Quick Check:

   Indexing beyond list length causes error [OK]

Hint: Always check list length before accessing index [OK]

Common Mistakes:

• Assuming pd.read_html returns a single DataFrame
• Ignoring possible IndexError from wrong index
• Thinking URLs are invalid input for read_html

5. You want to extract the second table from a webpage but only if it contains a column named 'Price'. Which code snippet correctly reads the tables and selects the second one only if it has the 'Price' column?

hard

A. tables = pd.read_html(url)
if 'Price' in tables[1].columns:
df = tables[1]
else:
df = None

B. df = pd.read_html(url)[1]
if 'Price' in df.index:
pass
else:
df = None

C. tables = pd.read_html(url)
df = tables[1].filter('Price')

D. df = pd.read_html(url, match='Price')[1]

Solution

1. Step 1: Read all tables from the webpage

   Use pd.read_html(url) to get a list of tables.

2. Step 2: Check if the second table has the 'Price' column

   Access tables[1] and check if 'Price' is in its columns.

3. Step 3: Assign the table or None accordingly

   If the column exists, assign the table to df, else assign None.

4. Final Answer:

   tables = pd.read_html(url) if 'Price' in tables[1].columns: df = tables[1] else: df = None -> Option A

5. Quick Check:

   Check columns before selecting table [OK]

Hint: Check columns attribute before using a table [OK]

Common Mistakes:

• Using filter() incorrectly on DataFrame
• Assuming match parameter filters tables by column
• Not checking if index 1 exists before access