Reading HTML tables in Data Analysis Python - Mini Project: Build & Apply

Reading HTML tables

📖 Scenario: Imagine you found a webpage that shows a table of daily temperatures for a week. You want to get this data into your program to analyze it easily.

🎯 Goal: You will write a program that reads the temperature table from an HTML string and shows it as a table you can work with.

📋 What You'll Learn

Use the pandas library to read HTML tables

Extract the first table from the HTML content

Store the table in a variable called temperature_table

Print the temperature_table to see the data

💡 Why This Matters

🌍 Real World

Websites often show data in tables. Being able to read these tables lets you collect and analyze data easily without manual copying.

💼 Career

Data analysts and scientists frequently extract data from web pages. Knowing how to read HTML tables with pandas is a useful skill for data gathering and cleaning.

1

Create the HTML string with the temperature table

Create a variable called html_data and set it to this exact HTML string containing a table with days and temperatures:

<table>
  <tr><th>Day</th><th>Temperature</th></tr>
  <tr><td>Monday</td><td>20</td></tr>
  <tr><td>Tuesday</td><td>22</td></tr>
  <tr><td>Wednesday</td><td>19</td></tr>
  <tr><td>Thursday</td><td>23</td></tr>
  <tr><td>Friday</td><td>21</td></tr>
</table>

Data Analysis Python

# Create the variable html_data with the exact HTML table string
# Your code here

Hint

Use triple quotes ''' to create a multi-line string for the HTML table.

2

Import pandas library

Write a line to import the pandas library using the alias pd.

Data Analysis Python

html_data = '''
  DayTemperature
  Monday20
  Tuesday22
  Wednesday19
  Thursday23
  Friday21
'''

# Import pandas as pd
# Your code here

Hint

Use the syntax import pandas as pd to import pandas with the short name pd.

3

Read the HTML table into a DataFrame

Use pd.read_html with the variable html_data to read the tables. Store the first table in a variable called temperature_table.

Data Analysis Python

import pandas as pd

html_data = '''
  DayTemperature
  Monday20
  Tuesday22
  Wednesday19
  Thursday23
  Friday21
'''

# Read the HTML tables and save the first one to temperature_table
# Your code here

Hint

Use pd.read_html(html_data) to get a list of tables, then select the first with [0].

4

Print the temperature table

Write a line to print the variable temperature_table to see the table data.

Data Analysis Python

import pandas as pd

html_data = '''
  DayTemperature
  Monday20
  Tuesday22
  Wednesday19
  Thursday23
  Friday21
'''

temperature_table = pd.read_html(html_data)[0]

# Print the temperature_table
# Your code here

Hint

Use print(temperature_table) to show the table.

Practice

1. What does the pd.read_html() function in pandas do?

easy

A. It converts a DataFrame into an HTML table string.

B. It reads all HTML tables from a webpage and returns them as a list of DataFrames.

C. It downloads an HTML file from the internet.

D. It parses JSON data from a webpage.

Solution

1. Step 1: Understand the purpose of pd.read_html()

   This function is designed to extract tables from HTML content on web pages.

2. Step 2: Recognize the output type

   The function returns a list where each element is a DataFrame representing one HTML table found.

3. Final Answer:

   It reads all HTML tables from a webpage and returns them as a list of DataFrames. -> Option B

4. Quick Check:

   pd.read_html() = list of DataFrames [OK]

Hint: Remember: read_html returns a list of tables, not a single table [OK]

Common Mistakes:

• Thinking it returns a single DataFrame instead of a list
• Confusing it with functions that write HTML
• Assuming it downloads files instead of parsing HTML

2. Which of the following is the correct way to read the first HTML table from a webpage URL stored in url using pandas?

easy

A. df = pd.read_html(url)[1]

B. df = pd.read_html(url, index=0)

C. df = pd.read_html(url).first()

D. tables = pd.read_html(url); df = tables[0]

Solution

1. Step 1: Recall the return type of pd.read_html()

   The function returns a list of DataFrames representing all tables found.

2. Step 2: Access the first table correctly

   Use list indexing [0] to get the first table from the list.

3. Final Answer:

   tables = pd.read_html(url); df = tables[0] -> Option D

4. Quick Check:

   Use list index 0 to get first table [OK]

Hint: Use index 0 on the list returned by read_html to get first table [OK]

Common Mistakes:

• Using .first() method which does not exist
• Passing index=0 as a parameter to read_html
• Using index 1 instead of 0 for first table

3. Given the code below, what will be the output of print(len(tables)) if the webpage contains 3 HTML tables?

import pandas as pd
url = 'https://example.com'
tables = pd.read_html(url)
print(len(tables))

medium

A. 0

B. 1

C. 3

D. Error: pd.read_html() requires a file path

Solution

1. Step 1: Understand what pd.read_html() returns

   It returns a list of DataFrames, one for each HTML table found on the page.

2. Step 2: Count the number of tables

   Since the webpage has 3 tables, the list length will be 3.

3. Final Answer:

   3 -> Option C

4. Quick Check:

   len(tables) = number of tables found [OK]

Hint: Count tables by checking length of list returned by read_html [OK]

Common Mistakes:

• Assuming it returns a single DataFrame
• Thinking it returns 0 if tables exist
• Believing it needs a local file path only

4. What is wrong with this code snippet to read the first HTML table from a URL?

import pandas as pd
url = 'https://example.com'
df = pd.read_html(url)[1]

medium

A. It tries to access the second table but may cause IndexError if only one table exists.

B. The syntax for pd.read_html is incorrect; it needs a file path, not a URL.

C. pd.read_html returns a DataFrame, so indexing with [1] is invalid.

D. The variable name df is reserved and cannot be used.

Solution

1. Step 1: Check the indexing used

   The code uses index [1], which means it tries to get the second table.

2. Step 2: Consider possible errors

   If the webpage has only one table, index 1 is out of range and causes IndexError.

3. Final Answer:

   It tries to access the second table but may cause IndexError if only one table exists. -> Option A

4. Quick Check:

   Indexing beyond list length causes error [OK]

Hint: Always check list length before accessing index [OK]

Common Mistakes:

• Assuming pd.read_html returns a single DataFrame
• Ignoring possible IndexError from wrong index
• Thinking URLs are invalid input for read_html

5. You want to extract the second table from a webpage but only if it contains a column named 'Price'. Which code snippet correctly reads the tables and selects the second one only if it has the 'Price' column?

hard

A. tables = pd.read_html(url)
if 'Price' in tables[1].columns:
df = tables[1]
else:
df = None

B. df = pd.read_html(url)[1]
if 'Price' in df.index:
pass
else:
df = None

C. tables = pd.read_html(url)
df = tables[1].filter('Price')

D. df = pd.read_html(url, match='Price')[1]

Solution

1. Step 1: Read all tables from the webpage

   Use pd.read_html(url) to get a list of tables.

2. Step 2: Check if the second table has the 'Price' column

   Access tables[1] and check if 'Price' is in its columns.

3. Step 3: Assign the table or None accordingly

   If the column exists, assign the table to df, else assign None.

4. Final Answer:

   tables = pd.read_html(url) if 'Price' in tables[1].columns: df = tables[1] else: df = None -> Option A

5. Quick Check:

   Check columns before selecting table [OK]

Hint: Check columns attribute before using a table [OK]

Common Mistakes:

• Using filter() incorrectly on DataFrame
• Assuming match parameter filters tables by column
• Not checking if index 1 exists before access