Numeric types (int64, float64) in Pandas - Time & Space Complexity

Time Complexity: Numeric types (int64, float64)

O(n)

Understanding Time Complexity

We want to understand how fast operations on numeric data types like int64 and float64 run in pandas.

How does the time to process numbers grow when we have more data?

Scenario Under Consideration

Analyze the time complexity of the following code snippet.

import pandas as pd

# Create a DataFrame with numeric columns
n = 1000
df = pd.DataFrame({
    'A': pd.Series(range(n), dtype='int64'),
    'B': pd.Series([float(x) for x in range(n)], dtype='float64')
})

# Calculate the sum of each column
result = df.sum()

This code creates a DataFrame with two numeric columns and sums each column.

Identify Repeating Operations

Identify the loops, recursion, array traversals that repeat.

• Primary operation: Summing values in each column by visiting every number once.
• How many times: Each number in the column is visited exactly once during the sum.

How Execution Grows With Input

When the number of rows grows, the time to sum grows roughly the same way.

Input Size (n)	Approx. Operations
10	About 10 additions per column
100	About 100 additions per column
1000	About 1000 additions per column

Pattern observation: The work grows directly with the number of rows.

Final Time Complexity

Time Complexity: O(n)

This means the time to sum numeric columns grows linearly with the number of rows.

Common Mistake

[X] Wrong: "Summing numeric columns is instant no matter how big the data is."

[OK] Correct: Even though pandas is fast, it still needs to visit each number once, so more data means more time.

Interview Connect

Knowing how numeric operations scale helps you explain performance and choose the right methods when working with data.

Self-Check

"What if we summed only a subset of rows using a filter? How would the time complexity change?"

Practice

1. Which pandas numeric type is best for storing whole numbers without decimals?

easy

A. bool

B. float64

C. int64

D. object

Solution

1. Step 1: Understand numeric types in pandas

   int64 is used for integers (whole numbers), float64 for decimals.

2. Step 2: Identify the type for whole numbers

   Whole numbers require int64 because float64 stores decimals.

3. Final Answer:

   int64 -> Option C

4. Quick Check:

   Whole numbers = int64 [OK]

Hint: Whole numbers use int64, decimals use float64 [OK]

Common Mistakes:

• Confusing float64 as default for all numbers
• Choosing object type for numeric data
• Using bool for numeric storage

2. Which of the following is the correct way to convert a pandas DataFrame column df['col'] to float64 type?

easy

A. df.astype('float64')['col']

B. df['col'].convert('float64')

C. astype(df['col'], 'float64')

D. df['col'].astype('float64')

Solution

1. Step 1: Recall pandas method to change column type

   The method to change a column's type is astype() called on the column.

2. Step 2: Apply correct syntax

   Use df['col'].astype('float64') to convert the column to float64.

3. Final Answer:

   df['col'].astype('float64') -> Option D

4. Quick Check:

   astype() on column = correct syntax [OK]

Hint: Use astype() on the column to change type [OK]

Common Mistakes:

• Calling astype() on the whole DataFrame incorrectly
• Using non-existent convert() method
• Wrong function syntax like astype(df['col'], 'float64')

3. Given the code:

import pandas as pd
df = pd.DataFrame({'A': [1, 2, 3], 'B': [1.0, 2.5, 3.5]})
df['A'] = df['A'].astype('float64')
print(df.dtypes)

What will be the output?

medium

A. A float64
B float64
dtype: object

B. A int64
B float64
dtype: object

C. A object
B float64
dtype: object

D. A float64
B int64
dtype: object

Solution

1. Step 1: Check initial dtypes

   Column 'A' starts as int64, 'B' as float64.

2. Step 2: Convert 'A' to float64

   Using astype('float64') changes 'A' from int64 to float64.

3. Final Answer:

   A float64 B float64 dtype: object -> Option A

4. Quick Check:

   astype changes int64 to float64 [OK]

Hint: astype changes column type; check dtypes after conversion [OK]

Common Mistakes:

• Assuming original types remain unchanged
• Confusing int64 and float64 outputs
• Misreading print output format

4. What is wrong with this code snippet?

import pandas as pd
df = pd.DataFrame({'num': [1, 2, 3]})
df['num'] = df['num'].astype(int64)

medium

A. int64 is not defined without quotes

B. astype() cannot convert to int64

C. DataFrame column cannot be reassigned

D. Missing import for astype

Solution

1. Step 1: Identify the error in astype argument

   int64 must be passed as a string 'int64', not as a bare name.

2. Step 2: Understand Python name error

   Without quotes, Python looks for a variable named int64 which is undefined.

3. Final Answer:

   int64 is not defined without quotes -> Option A

4. Quick Check:

   Use quotes around type names in astype() [OK]

Hint: Always quote type names in astype() [OK]

Common Mistakes:

• Passing type names without quotes
• Thinking astype() needs import
• Believing columns cannot be reassigned

5. You have a DataFrame df with a column 'values' containing numbers as strings, e.g., ['1', '2.5', '3']. You want to convert this column to numeric types efficiently, storing integers as int64 and decimals as float64 where needed. Which approach is best?

hard

A. Use pd.to_numeric(df['values'], downcast='integer')

B. Use pd.to_numeric(df['values'], errors='coerce') and then convert integers to int64

C. Use df['values'] = df['values'].astype('float64') for all values

D. Manually check each value and convert to int64 or float64

Solution

1. Step 1: Understand mixed numeric strings

   Strings include integers and decimals, so direct astype('float64') converts all to float.

2. Step 2: Use pd.to_numeric with errors='coerce'

   This converts strings to numbers, coercing errors to NaN, allowing safe conversion.

3. Step 3: Convert integers to int64 after identifying them

   After conversion, integers can be cast to int64 to save memory.

4. Final Answer:

   Use pd.to_numeric(df['values'], errors='coerce') and then convert integers to int64 -> Option B

5. Quick Check:

   pd.to_numeric + careful conversion handles mixed types best [OK]

Hint: Use pd.to_numeric with errors='coerce' for mixed numeric strings [OK]

Common Mistakes:

• Converting all to float64 losing integer type
• Trying manual conversion for each value
• Ignoring errors in conversion causing crashes