Structured arrays vs DataFrames in NumPy - Hands-On Comparison

Structured arrays vs DataFrames

📖 Scenario: You work in a small shop that tracks sales data. You want to organize the sales information to analyze it easily. You will use two common ways to store data in Python: NumPy structured arrays and Pandas DataFrames.This project will help you see how to create and use both data structures with the same data.

🎯 Goal: Create a NumPy structured array and a Pandas DataFrame with the same sales data. Then, select and print the sales records where the amount is greater than 50.

📋 What You'll Learn

Create a NumPy structured array with fields: 'product' (string), 'quantity' (integer), and 'price' (float).

Create a Pandas DataFrame with the same data.

Filter and select sales where 'quantity' is greater than 50 in both data structures.

Print the filtered results.

💡 Why This Matters

🌍 Real World

Organizing and filtering sales data helps businesses understand which products sell more and manage inventory better.

💼 Career

Data scientists and analysts often use NumPy and Pandas to clean and analyze data efficiently.

1

Create a NumPy structured array

Create a NumPy structured array called sales_array with these exact entries: ('apple', 30, 0.5), ('banana', 60, 0.3), ('orange', 80, 0.7). Use the data type with fields: 'product' as a string of length 10, 'quantity' as integer, and 'price' as float.

NumPy

# Create the NumPy structured array called sales_array
# Your code here

Hint

Use np.array with a list of tuples and specify dtype as a list of tuples with field names and types.

2

Create a Pandas DataFrame

Import pandas as pd. Create a DataFrame called sales_df with columns 'product', 'quantity', and 'price' using the same data as in sales_array.

NumPy

import numpy as np

sales_array = np.array([
    ('apple', 30, 0.5),
    ('banana', 60, 0.3),
    ('orange', 80, 0.7)
], dtype=[('product', 'U10'), ('quantity', 'i4'), ('price', 'f4')])

# Import pandas as pd
# Create the DataFrame sales_df with the same data
# Your code here

Hint

Use pd.DataFrame and pass a dictionary with keys as column names and values as the fields from sales_array.

3

Filter sales with quantity greater than 50

Create a variable called filtered_array that selects rows from sales_array where quantity is greater than 50. Also create a variable called filtered_df that selects rows from sales_df where quantity is greater than 50.

NumPy

import numpy as np
import pandas as pd

sales_array = np.array([
    ('apple', 30, 0.5),
    ('banana', 60, 0.3),
    ('orange', 80, 0.7)
], dtype=[('product', 'U10'), ('quantity', 'i4'), ('price', 'f4')])

sales_df = pd.DataFrame({
    'product': sales_array['product'],
    'quantity': sales_array['quantity'],
    'price': sales_array['price']
})

# Filter sales_array where quantity > 50 into filtered_array
# Filter sales_df where quantity > 50 into filtered_df
# Your code here

Hint

Use boolean indexing with sales_array['quantity'] > 50 and sales_df['quantity'] > 50 to filter.

4

Print the filtered results

Print the variables filtered_array and filtered_df to show the sales records where quantity is greater than 50.

NumPy

import numpy as np
import pandas as pd

sales_array = np.array([
    ('apple', 30, 0.5),
    ('banana', 60, 0.3),
    ('orange', 80, 0.7)
], dtype=[('product', 'U10'), ('quantity', 'i4'), ('price', 'f4')])

sales_df = pd.DataFrame({
    'product': sales_array['product'],
    'quantity': sales_array['quantity'],
    'price': sales_array['price']
})

filtered_array = sales_array[sales_array['quantity'] > 50]
filtered_df = sales_df[sales_df['quantity'] > 50]

# Print filtered_array and filtered_df
# Your code here

Hint

Use print(filtered_array) and print(filtered_df) to show the filtered data.

Practice

1. What is a key difference between a numpy structured array and a pandas DataFrame?

easy

A. Structured arrays automatically handle missing data, DataFrames do not.

B. Structured arrays can only store numbers, DataFrames can only store text.

C. DataFrames do not support named columns, structured arrays do.

D. Structured arrays have fixed data types per column, while DataFrames allow mixed types and more flexible operations.

Solution

1. Step 1: Understand data type handling in structured arrays

   Structured arrays in numpy require fixed data types for each named column, meaning each column's type is set and consistent.

2. Step 2: Compare with DataFrame flexibility

   DataFrames from pandas allow columns to have different data types and provide many flexible operations like handling missing data and complex indexing.

3. Final Answer:

   Structured arrays have fixed data types per column, while DataFrames allow mixed types and more flexible operations. -> Option D

4. Quick Check:

   Data type flexibility = D [OK]

Hint: Remember: structured arrays fix types, DataFrames are more flexible [OK]

Common Mistakes:

• Thinking structured arrays can handle missing data like DataFrames
• Assuming DataFrames cannot have mixed data types
• Believing structured arrays only store numbers

2. Which of the following is the correct way to create a numpy structured array with fields 'name' (string) and 'age' (integer)?

easy

A. np.array([('Alice', 25), ('Bob', 30)], dtype=[('name', 'int'), ('age', 'str')])

B. np.array([{'Name': 'Alice', 'age': 25}, {'Name': 'Bob', 'age': 30}])

C. np.array([('Alice', 25), ('Bob', 30)], dtype=[('name', 'U10'), ('age', 'i4')])

D. np.array([['Alice'], ['Bob']], dtype=[('name', 'U10'), ('age', 'i4')])

Solution

1. Step 1: Check dtype specification for structured arrays

   The dtype must be a list of tuples with field names and valid numpy data types, e.g., 'U10' for string and 'i4' for 4-byte integer.

2. Step 2: Verify the data matches the dtype

   np.array([('Alice', 25), ('Bob', 30)], dtype=[('name', 'U10'), ('age', 'i4')]) uses tuples matching the dtype fields correctly. np.array([{'name': 'Alice', 'age': 25}, {'name': 'Bob', 'age': 30}]) uses dicts which numpy does not accept directly for structured arrays. np.array([('Alice', 25), ('Bob', 30)], dtype=[('name', 'int'), ('age', 'str')]) swaps types incorrectly. np.array([['Alice', 25], ['Bob', 30]], dtype=[('name', 'U10'), ('age', 'i4')]) uses lists instead of tuples, which is invalid here.

3. Final Answer:

   np.array([('Alice', 25), ('Bob', 30)], dtype=[('name', 'U10'), ('age', 'i4')]) -> Option C

4. Quick Check:

   Correct dtype and tuple data = A [OK]

Hint: Use tuples and correct dtype list for structured arrays [OK]

Common Mistakes:

• Using dicts instead of tuples for structured array data
• Mixing up data types in dtype list
• Using lists instead of tuples for records

3. Given the code below, what will be the output?

import numpy as np
import pandas as pd

arr = np.array([(1, 'A'), (2, 'B')], dtype=[('id', 'i4'), ('label', 'U1')])
df = pd.DataFrame(arr)
print(df['label'][1])

medium

A. B

B. A

C. 1

D. Error: KeyError

Solution

1. Step 1: Understand conversion from structured array to DataFrame

   Creating a DataFrame from a structured array converts named fields into columns with the same names.

2. Step 2: Access the 'label' column and index 1

   df['label'] is a Series with values ['A', 'B']. Index 1 corresponds to 'B'.

3. Final Answer:

   B -> Option A

4. Quick Check:

   DataFrame column access = B [OK]

Hint: Structured array fields become DataFrame columns [OK]

Common Mistakes:

• Confusing index 0 and 1 values
• Expecting error due to structured array
• Mixing up field names and indices

4. What is wrong with this code snippet that tries to convert a pandas DataFrame to a numpy structured array?

import pandas as pd
import numpy as np

df = pd.DataFrame({'name': ['Tom', 'Jerry'], 'age': [5, 7]})
arr = np.array(df, dtype=[('name', 'U10'), ('age', 'i4')])
print(arr)

medium

A. The dtype should use 'S10' instead of 'U10' for strings.

B. The dtype argument is ignored; conversion does not create a structured array as expected.

C. The DataFrame must be converted to a list of tuples before creating the structured array.

D. There is no error; the code works correctly.

Solution

1. Step 1: Check how numpy.array handles DataFrame input with dtype

   Passing a DataFrame directly to np.array with dtype does not convert it into a structured array; dtype is ignored and a 2D array of objects is created.

2. Step 2: Identify correct conversion method

   To get a structured array, convert DataFrame to records (e.g., df.to_records()) before calling np.array.

3. Final Answer:

   The dtype argument is ignored; conversion does not create a structured array as expected. -> Option B

4. Quick Check:

   Direct np.array(df, dtype=...) ignores dtype [OK]

Hint: Convert DataFrame to records before numpy structured array [OK]

Common Mistakes:

• Assuming dtype works directly on DataFrame in np.array
• Not converting DataFrame to records first
• Confusing string dtype codes

5. You have a numpy structured array with fields 'city' (string) and 'temperature' (float). You want to convert it to a pandas DataFrame, filter rows where temperature > 20, then convert back to a structured array with the same fields. Which code snippet correctly does this?

hard

A. df = pd.DataFrame(arr); filtered = df.query('temperature > 20'); result = np.array(filtered.to_records(index=False), dtype=arr.dtype)

B. df = pd.DataFrame(arr); filtered = df[df.temperature > 20]; result = np.array(filtered, dtype=arr.dtype)

C. df = pd.DataFrame(arr); filtered = df[df['temperature'] > 20]; result = np.array(filtered.to_records())

D. df = pd.DataFrame(arr); filtered = df[df['temperature'] > 20]; result = np.array(filtered.to_dict())

Solution

1. Step 1: Convert structured array to DataFrame

   Creating a DataFrame from the structured array is straightforward: df = pd.DataFrame(arr).

2. Step 2: Filter rows where temperature > 20

   Using df.query('temperature > 20') or df[df['temperature'] > 20] both work, but query is concise and clear.

3. Step 3: Convert filtered DataFrame back to structured array with original dtype

   Use filtered.to_records(index=False) to get a structured array-like record array, then convert to numpy array with original dtype to keep field types consistent.

4. Final Answer:

   df = pd.DataFrame(arr); filtered = df.query('temperature > 20'); result = np.array(filtered.to_records(index=False), dtype=arr.dtype) -> Option A

5. Quick Check:

   Filter with query + to_records + dtype = A [OK]

Hint: Use to_records() and specify dtype when converting back [OK]

Common Mistakes:

• Not using to_records() before np.array conversion
• Forgetting to specify dtype on conversion back
• Using to_dict() which is incorrect here