Creating DataFrame from list of dictionaries in Pandas

Introduction

We use a DataFrame to organize data in rows and columns, like a table. Creating a DataFrame from a list of dictionaries helps turn messy data into a neat table quickly.

You have data collected as multiple records, each with named fields.

You want to analyze or visualize data stored as dictionaries.

You receive JSON-like data and want to convert it into a table.

You want to combine several small data entries into one table.

Syntax

Pandas

import pandas as pd

list_of_dicts = [
    {'column1': 'value1', 'column2': 'value2'},
    {'column1': 'value3', 'column2': 'value4'}
]

dataframe = pd.DataFrame(list_of_dicts)

Each dictionary in the list represents one row in the DataFrame.

Keys in dictionaries become column names in the DataFrame.

Examples

This creates a DataFrame with two rows and columns 'name' and 'age'.

Pandas

import pandas as pd

# Example 1: Normal case with two dictionaries
list_of_dicts = [
    {'name': 'Alice', 'age': 30},
    {'name': 'Bob', 'age': 25}
]
df = pd.DataFrame(list_of_dicts)
print(df)

When the list is empty, the DataFrame is empty with no rows or columns.

Pandas

import pandas as pd

# Example 2: Empty list
list_of_dicts = []
df = pd.DataFrame(list_of_dicts)
print(df)

DataFrame with one row is created.

Pandas

import pandas as pd

# Example 3: One dictionary only
list_of_dicts = [{'name': 'Charlie', 'age': 40}]
df = pd.DataFrame(list_of_dicts)
print(df)

Missing values appear as NaN in the DataFrame.

Pandas

import pandas as pd

# Example 4: Dictionaries with missing keys
list_of_dicts = [
    {'name': 'Dana', 'age': 22},
    {'name': 'Eli'}
]
df = pd.DataFrame(list_of_dicts)
print(df)

Sample Program

This program shows how to create a DataFrame from a list of dictionaries, print it, then add another dictionary and create a new DataFrame to see the update.

Pandas

import pandas as pd

# Create a list of dictionaries representing people
people_data = [
    {'name': 'Alice', 'age': 30, 'city': 'New York'},
    {'name': 'Bob', 'age': 25, 'city': 'Los Angeles'},
    {'name': 'Charlie', 'age': 35, 'city': 'Chicago'}
]

# Print original list
print('Original list of dictionaries:')
print(people_data)

# Create DataFrame from list of dictionaries
people_df = pd.DataFrame(people_data)

# Print DataFrame before any changes
print('\nDataFrame created from list of dictionaries:')
print(people_df)

# Add a new person to the list
people_data.append({'name': 'Dana', 'age': 28, 'city': 'Houston'})

# Create new DataFrame after adding
updated_people_df = pd.DataFrame(people_data)

# Print updated DataFrame
print('\nUpdated DataFrame after adding one more person:')
print(updated_people_df)

Important Notes

Time complexity is O(n) where n is the number of dictionaries, because each dictionary is processed once.

Space complexity is O(n*m) where m is average number of keys per dictionary, as data is stored in memory.

Common mistake: forgetting that missing keys in some dictionaries become NaN in the DataFrame.

Use this method when you have data naturally organized as records (dictionaries). For large datasets, consider reading from files or databases directly.

Summary

Creating a DataFrame from a list of dictionaries turns each dictionary into a row.

Keys in dictionaries become column names; missing keys show as NaN.

This is a quick way to organize and analyze structured data in Python.

Practice

1. What happens when you create a pandas DataFrame from a list of dictionaries?

easy

A. Each dictionary becomes a row, and keys become column names.

B. Each dictionary becomes a column, and keys become row labels.

C. The list is converted into a single column with dictionary objects.

D. The DataFrame will be empty because dictionaries cannot be converted.

Solution

1. Step 1: Understand DataFrame creation from list of dicts

   Each dictionary in the list is treated as one row in the DataFrame.

2. Step 2: Identify how keys are used

   Keys from dictionaries become the column names in the DataFrame.

3. Final Answer:

   Each dictionary becomes a row, and keys become column names. -> Option A

4. Quick Check:

   List of dicts -> rows, keys -> columns [OK]

Hint: Remember: dict keys become columns, dicts become rows [OK]

Common Mistakes:

• Thinking dictionaries become columns instead of rows
• Assuming keys become row labels
• Believing the DataFrame will be empty

2. Which of the following is the correct syntax to create a DataFrame from a list of dictionaries named data?

easy

A. df = pd.DataFrame(data)

B. df = pd.DataFrame.from_list(data)

C. df = pd.DataFrame.create(data)

D. df = pd.DataFrame.list(data)

Solution

1. Step 1: Recall pandas DataFrame constructor

   The correct way to create a DataFrame from data is using pd.DataFrame().

2. Step 2: Check method names

   There is no from_list, create, or list method in pandas DataFrame.

3. Final Answer:

   df = pd.DataFrame(data) -> Option A

4. Quick Check:

   Use pd.DataFrame() constructor [OK]

Hint: Use pd.DataFrame(your_list) to create DataFrame [OK]

Common Mistakes:

• Using non-existent methods like from_list or create
• Confusing DataFrame constructor syntax
• Missing pd prefix or parentheses

3. Given the code:

import pandas as pd
data = [{'name': 'Alice', 'age': 25}, {'name': 'Bob', 'city': 'NY'}]
df = pd.DataFrame(data)
print(df)

What will be the output?

medium

A. A DataFrame with columns 'name' and 'city' only.

B. A DataFrame with only columns 'name' and 'age', ignoring 'city'.

C. An error because dictionaries have different keys.

D. A DataFrame with columns 'name', 'age', 'city' and NaN where data is missing.

Solution

1. Step 1: Check keys across dictionaries

   Keys are 'name', 'age' in first dict and 'name', 'city' in second dict.

2. Step 2: Understand pandas behavior with missing keys

   Pandas creates columns for all keys found and fills missing values with NaN.

3. Final Answer:

   DataFrame with columns 'name', 'age', 'city' and NaN where data is missing. -> Option D

4. Quick Check:

   Missing keys -> NaN in DataFrame [OK]

Hint: Missing keys become NaN in DataFrame columns [OK]

Common Mistakes:

• Assuming missing keys cause errors
• Thinking columns are limited to first dict keys
• Ignoring NaN values for missing data

4. What is wrong with this code snippet?

import pandas as pd
data = [{'a': 1, 'b': 2}, {'a': 3, 'c': 4}]
df = pd.DataFrame(data, columns=['a', 'b', 'c', 'd'])
print(df)

medium

A. DataFrame will only include columns 'a' and 'b'.

B. No error; columns not in data will be filled with NaN.

C. Error because columns list length does not match dictionaries.

D. Error because column 'd' does not exist in any dictionary.

Solution

1. Step 1: Understand columns parameter effect

   Specifying columns includes those columns in DataFrame, even if missing in data.

2. Step 2: Check missing columns behavior

   Columns not found in dictionaries are filled with NaN, no error occurs.

3. Final Answer:

   No error; columns not in data will be filled with NaN. -> Option B

4. Quick Check:

   Extra columns -> NaN, no error [OK]

Hint: Extra columns in list become NaN, no error [OK]

Common Mistakes:

• Expecting error for missing columns
• Thinking columns list must match keys exactly
• Assuming DataFrame excludes columns not in data

5. You have a list of dictionaries representing sales data:

sales = [
  {'product': 'A', 'units': 10, 'price': 2.5},
  {'product': 'B', 'units': 5},
  {'product': 'A', 'price': 2.5},
  {'product': 'C', 'units': 7, 'price': 3.0}
]

You want to create a DataFrame and add a new column 'total' which is units * price, treating missing values as zero. Which code correctly does this?

hard

A. df = pd.DataFrame(sales)
df['total'] = df['units'].fillna(0) + df['price'].fillna(0)
print(df)

B. df = pd.DataFrame(sales)
df['total'] = df['units'] * df['price']
print(df)

C. df = pd.DataFrame(sales)
df['units'] = df['units'].fillna(0)
df['price'] = df['price'].fillna(0)
df['total'] = df['units'] * df['price']
print(df)

D. df = pd.DataFrame(sales)
df['total'] = df['units'] * df['price']
df.fillna(0)
print(df)

Solution

1. Step 1: Create DataFrame from list of dicts

   Use pd.DataFrame(sales) to convert list to DataFrame with NaNs for missing keys.

2. Step 2: Replace NaN with 0 before multiplication

   Fill missing 'units' and 'price' with 0 to avoid NaN in multiplication.

3. Step 3: Calculate 'total' as product of 'units' and 'price'

   Multiply filled columns to get correct totals.

4. Final Answer:

   Fill NaNs first, then multiply -> Option C

5. Quick Check:

   Fill NaN before math to avoid NaN results [OK]

Hint: Fill missing values with 0 before multiplying columns [OK]

Common Mistakes:

• Multiplying columns with NaN without filling
• Adding instead of multiplying for total
• Filling NaN after multiplication