Log transformation for skewed data in Data Analysis Python - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Output of log transformation on skewed data

What is the output of the following Python code that applies a log transformation to a skewed data list?

Data Analysis Python

import numpy as np
import pandas as pd

skewed_data = pd.Series([1, 10, 100, 1000, 10000])
log_data = np.log(skewed_data)
print(log_data.round(2))

A [0.69, 2.30, 4.61, 6.91, 9.21]

B [0.00, 2.30, 4.61, 6.91, 9.21]

C [1, 10, 100, 1000, 10000]

D [0.00, 1.00, 2.00, 3.00, 4.00]

💡 Hint

Remember that np.log() computes the natural logarithm (base e).

The natural log of 1 is 0.00, and the logs of 10, 100, 1000, and 10000 are approximately 2.30, 4.61, 6.91, and 9.21 respectively.

❓ data_output

intermediate

Effect of log transformation on skewness

Given a skewed dataset, which option shows the correct skewness values before and after applying a log transformation?

Data Analysis Python

import numpy as np
import pandas as pd
from scipy.stats import skew

skewed_data = pd.Series([1, 2, 2, 3, 100])
skew_before = skew(skewed_data)
skew_after = skew(np.log(skewed_data))
print(round(skew_before, 2), round(skew_after, 2))

A 2.24 2.24

B 1.50 1.50

C 2.24 0.68

D 0.00 0.00

💡 Hint

Log transformation reduces skewness by compressing large values.

The original data is highly skewed (2.24), and after log transformation, skewness reduces significantly (0.68).

❓ visualization

advanced

Identify the correct plot after log transformation

Which plot correctly shows the distribution of data before and after log transformation?

Data Analysis Python

import matplotlib.pyplot as plt
import numpy as np
import pandas as pd

skewed_data = pd.Series([1, 2, 2, 3, 100])
log_data = np.log(skewed_data)

plt.figure(figsize=(8,4))
plt.subplot(1,2,1)
plt.hist(skewed_data, bins=5, color='blue')
plt.title('Original Data')
plt.subplot(1,2,2)
plt.hist(log_data, bins=5, color='green')
plt.title('Log Transformed Data')
plt.tight_layout()
plt.show()

A Left plot is right-skewed; right plot is more symmetric

B Left plot is symmetric; right plot is right-skewed

C Both plots are symmetric

D Both plots are left-skewed

💡 Hint

Log transformation reduces right skewness.

The original data histogram is right-skewed due to the large value 100. After log transform, the distribution looks more balanced.

🧠 Conceptual

advanced

Why use log transformation on skewed data?

Which option best explains why log transformation is applied to skewed data in data science?

A To reduce skewness and make data distribution more normal-like for better modeling

B To increase the range of data values for better visualization

C To convert categorical data into numerical data

D To remove missing values from the dataset

💡 Hint

Think about how skewness affects statistical models.

Log transformation compresses large values, reducing skewness and helping models that assume normality.

🔧 Debug

expert

Identify the error in log transformation code

What error will the following code raise when applying log transformation to data containing zero or negative values?

Data Analysis Python

import numpy as np
import pandas as pd

data = pd.Series([0, 1, 2, -1, 3])
log_data = np.log(data)
print(log_data)

A No error, prints log values with NaN for invalid entries

B ValueError: math domain error

C TypeError: unsupported operand type(s) for log

D RuntimeWarning: divide by zero encountered in log

💡 Hint

Logarithm of zero or negative numbers is undefined in real numbers.

np.log(0) causes a divide by zero warning; np.log of negative values results in NaN but no ValueError.

Practice

1. What is the main purpose of applying a log transformation to skewed data?

easy

A. To reduce skewness and make data more normal

B. To increase the range of data values

C. To convert categorical data into numbers

D. To remove missing values from the dataset

Solution

1. Step 1: Understand skewness in data

   Skewed data means the distribution is not symmetric and can affect analysis.

2. Step 2: Purpose of log transformation

   Log transformation reduces skewness by compressing large values and spreading small values, making data more normal.

3. Final Answer:

   To reduce skewness and make data more normal -> Option A

4. Quick Check:

   Log transformation reduces skewness = D [OK]

Hint: Log transformation makes skewed data more normal [OK]

Common Mistakes:

• Thinking log increases data range
• Confusing log transform with encoding
• Assuming it removes missing data

2. Which of the following is the correct Python syntax to apply a natural log transformation to a pandas DataFrame column named 'income'?

easy

A. df['income_log'] = log(df['income'])

B. df['income_log'] = np.log10(df['income'])

C. df['income_log'] = df.log('income')

D. df['income_log'] = np.log(df['income'])

Solution

1. Step 1: Identify the correct log function

   Natural log in Python is computed using np.log() from the numpy library.

2. Step 2: Apply log to DataFrame column

   Use df['income_log'] = np.log(df['income']) to create a new column with log values.

3. Final Answer:

   df['income_log'] = np.log(df['income']) -> Option D

4. Quick Check:

   Use np.log for natural log in pandas = B [OK]

Hint: Use np.log() for natural log on DataFrame columns [OK]

Common Mistakes:

• Using log() without np prefix
• Trying df.log() method which doesn't exist
• Using log10 instead of natural log

3. Given the following code, what will be the output of print(df['log_value'])?

import pandas as pd
import numpy as np
data = {'value': [1, 10, 100, 1000]}
df = pd.DataFrame(data)
df['log_value'] = np.log(df['value'])
print(df['log_value'])

medium

A. [1, 10, 100, 1000]

B. [0.0, 2.302585, 4.605170, 6.907755]

C. [0, 1, 2, 3]

D. Error because log cannot be applied to integers

Solution

1. Step 1: Understand np.log on positive numbers

   Natural log of 1 is 0, log(10) ~ 2.302585, log(100) ~ 4.605170, log(1000) ~ 6.907755.

2. Step 2: Check output of print statement

   Printing the 'log_value' column shows these calculated log values as floats.

3. Final Answer:

   [0.0, 2.302585, 4.605170, 6.907755] -> Option B

4. Quick Check:

   np.log applied to values = A [OK]

Hint: Log of 1 is 0; log grows slowly for larger values [OK]

Common Mistakes:

• Expecting original values instead of logs
• Confusing log base 10 with natural log
• Thinking log can't apply to integers

4. The following code throws an error. What is the most likely cause?

import numpy as np
values = [-5, 0, 10]
log_values = np.log(values)
print(log_values)

medium

A. Logarithm cannot be applied to zero or negative numbers

B. np.log() requires integer inputs only

C. Missing import statement for pandas

D. Syntax error in the code

Solution

1. Step 1: Identify input values for log

   Values include -5 and 0, which are invalid for logarithm since log is undefined for zero or negative numbers.

2. Step 2: Understand error cause

   Applying np.log to these values causes a runtime warning or error due to invalid input domain.

3. Final Answer:

   Logarithm cannot be applied to zero or negative numbers -> Option A

4. Quick Check:

   Log undefined for ≤ 0 values = C [OK]

Hint: Log only works on positive numbers, not zero or negatives [OK]

Common Mistakes:

• Thinking np.log needs integers only
• Confusing missing imports as cause
• Assuming syntax error without checking inputs

5. You have a dataset with a column 'sales' that contains positive values but also some zeros. You want to apply a log transformation to reduce skewness. Which approach is best to avoid errors and keep data meaningful?

hard

A. Remove all zero values and then apply log: df = df[df['sales'] > 0]; df['log_sales'] = np.log(df['sales'])

B. Replace zeros with -1 before log: df['log_sales'] = np.log(df['sales'].replace(0, -1))

C. Add 1 to all values before log: df['log_sales'] = np.log(df['sales'] + 1)

D. Apply log directly without changes: df['log_sales'] = np.log(df['sales'])

Solution

1. Step 1: Understand log domain restrictions

   Log cannot be applied to zero or negative values, so zeros cause errors.

2. Step 2: Choose a method to handle zeros

   Adding 1 shifts all values positive, allowing log transform without error and preserving data.

3. Step 3: Evaluate other options

   Replacing zeros with -1 causes errors; removing zeros loses data; applying log directly errors on zeros.

4. Final Answer:

   Add 1 to all values before log: df['log_sales'] = np.log(df['sales'] + 1) -> Option C

5. Quick Check:

   Add 1 to avoid log(0) errors = A [OK]

Hint: Add 1 to data before log to handle zeros safely [OK]

Common Mistakes:

• Replacing zeros with negative numbers
• Dropping zeros losing important data
• Applying log directly on zeros