Profiling NumPy operations - Mini Project: Build & Apply

Profiling NumPy operations

📖 Scenario: You are working with numerical data using NumPy. You want to understand how long certain NumPy operations take to run. This helps you find which parts of your code are slow and need improvement.

🎯 Goal: Build a simple program that creates a NumPy array, sets a threshold value, measures the time taken to perform an operation on the array, and prints the elapsed time.

📋 What You'll Learn

Create a NumPy array with specific values

Set a threshold variable

Use time.perf_counter() to measure elapsed time of a NumPy operation

Print the elapsed time in seconds

💡 Why This Matters

🌍 Real World

Profiling helps data scientists find slow parts of their code so they can make it faster and more efficient.

💼 Career

Knowing how to measure and improve code performance is important for data scientists working with large datasets and complex calculations.

1

Create a NumPy array

Import NumPy as np and create a NumPy array called data with these exact values: [10, 20, 30, 40, 50].

NumPy

import numpy as np
# Create the NumPy array called data with values [10, 20, 30, 40, 50]
# Your code here

Hint

Use np.array() to create the array with the given list of numbers.

2

Set a threshold value

Create a variable called threshold and set it to the integer 25.

NumPy

import numpy as np
data = np.array([10, 20, 30, 40, 50])
# Set the variable threshold to 25
# Your code here

Hint

Just assign the number 25 to the variable named threshold.

3

Measure time for a NumPy operation

Import time. Use time.perf_counter() to record the start time in a variable called start_time. Then create a new NumPy array called filtered that contains only the values from data greater than threshold. After that, record the end time in a variable called end_time using time.perf_counter().

NumPy

import numpy as np
data = np.array([10, 20, 30, 40, 50])
threshold = 25
# Import time and record start_time using time.perf_counter()
# Create filtered array with values from data greater than threshold
# Record end_time using time.perf_counter()
# Your code here

Hint

Use boolean indexing data[data > threshold] to filter the array.

4

Print the elapsed time

Calculate the elapsed time by subtracting start_time from end_time and store it in a variable called elapsed. Then print the text Elapsed time: followed by the elapsed time rounded to 6 decimal places and the string seconds.

NumPy

import numpy as np
import time
data = np.array([10, 20, 30, 40, 50])
threshold = 25
start_time = time.perf_counter()
filtered = data[data > threshold]
end_time = time.perf_counter()
# Calculate elapsed time and print it with the text 'Elapsed time:' and 'seconds'
# Your code here

Hint

Use an f-string to format the elapsed time with 6 decimal places.

Practice

1. What is the main purpose of profiling NumPy operations?

easy

A. To make the code look cleaner

B. To find which parts of the code run slowly

C. To check if the code has syntax errors

D. To add comments to the code

Solution

1. Step 1: Understand profiling meaning

   Profiling means measuring how long parts of code take to run.

2. Step 2: Identify profiling goal in NumPy

   In NumPy, profiling helps find slow operations to improve speed.

3. Final Answer:

   To find which parts of the code run slowly -> Option B

4. Quick Check:

   Profiling = Find slow parts [OK]

Hint: Profiling = measuring speed to find slow code [OK]

Common Mistakes:

• Confusing profiling with debugging syntax errors
• Thinking profiling cleans code
• Assuming profiling adds comments

2. Which Python function is commonly used to measure time for profiling NumPy operations?

easy

A. time.time()

B. print()

C. len()

D. np.array()

Solution

1. Step 1: Recall timing functions in Python

   Python's time.time() returns current time in seconds.

2. Step 2: Use time.time() for profiling

   Subtracting start and end time.time() gives elapsed time.

3. Final Answer:

   time.time() -> Option A

4. Quick Check:

   Timing = time.time() [OK]

Hint: Use time.time() to get current time in seconds [OK]

Common Mistakes:

• Using print() to measure time
• Confusing len() with timing
• Using np.array() instead of timing

3. What will be the output of this code snippet?

import numpy as np
import time
arr = np.arange(1000000)
start = time.time()
sum_val = np.sum(arr)
end = time.time()
print(round(end - start, 4))

medium

A. A very small number close to 0, like 0.001

B. The sum of numbers from 0 to 999999

C. An error because np.sum is wrong

D. The time in seconds as a large number like 1000

Solution

1. Step 1: Understand code timing

   The code measures time to sum 1 million numbers using np.sum.

2. Step 2: Expect timing output

   np.sum is fast; timing will be a small float near zero seconds.

3. Final Answer:

   A very small number close to 0, like 0.001 -> Option A

4. Quick Check:

   Timing output = small float [OK]

Hint: np.sum is fast; timing shows small seconds [OK]

Common Mistakes:

• Thinking output is the sum value
• Expecting an error from np.sum
• Assuming timing is very large

4. Identify the error in this profiling code:

import numpy as np
import time
arr = np.arange(1000)
start = time.time()
sum_val = np.sum(arr)
print(round(end - start, 5))
end = time.time()

medium

A. round() cannot take two arguments

B. np.sum does not work on arrays

C. time.time() cannot be used twice

D. Variable 'end' is used before assignment

Solution

1. Step 1: Check variable usage order

   The code prints end - start before end is assigned.

2. Step 2: Identify error cause

   This causes a NameError because end is undefined at print time.

3. Final Answer:

   Variable 'end' is used before assignment -> Option D

4. Quick Check:

   Use end after assignment [OK]

Hint: Assign end before using it in print [OK]

Common Mistakes:

• Thinking np.sum is invalid
• Believing time.time() can't be called twice
• Confusing round() arguments

5. You want to compare the speed of two NumPy operations on large arrays. Which approach is best for profiling?

hard

A. Run each operation once and compare times

B. Only check the code visually without timing

C. Run each operation many times and average the times

D. Use print statements inside the operations

Solution

1. Step 1: Understand timing variability

   Running once can give misleading times due to random delays.

2. Step 2: Use repeated runs for accuracy

   Running many times and averaging smooths out random noise.

3. Final Answer:

   Run each operation many times and average the times -> Option C

4. Quick Check:

   Average multiple runs for reliable timing [OK]

Hint: Average many runs to get stable timing [OK]

Common Mistakes:

• Trusting single run timing
• Ignoring timing and just reading code
• Using print instead of timing