Overlapping Subproblems and Optimal Substructure in DSA Python - Build from Scratch

Understanding Overlapping Subproblems and Optimal Substructure with Fibonacci

📖 Scenario: Imagine you want to find the number of ways to climb stairs where you can take 1 or 2 steps at a time. This problem can be solved using the Fibonacci sequence, which shows overlapping subproblems and optimal substructure.

🎯 Goal: Build a simple program to calculate Fibonacci numbers using a dictionary to store results and avoid repeated calculations, demonstrating overlapping subproblems and optimal substructure.

📋 What You'll Learn

Create a dictionary to store Fibonacci numbers

Set a base case for Fibonacci numbers

Use a loop to fill the dictionary using previous results

Print the Fibonacci number for a given position

💡 Why This Matters

🌍 Real World

Calculating Fibonacci numbers is a classic example to understand how to optimize problems by storing intermediate results, which is useful in many real-world tasks like route planning, resource allocation, and dynamic programming.

💼 Career

Understanding overlapping subproblems and optimal substructure is key for software engineers working on optimization problems, algorithms, and efficient code design.

1

Create the Fibonacci dictionary with base cases

Create a dictionary called fib with these exact entries: 0: 0 and 1: 1 to represent the base Fibonacci numbers.

DSA Python

# Create the fib dictionary with base cases 0 and 1
# Your code here

Hint

The Fibonacci sequence starts with 0 and 1. Store these in a dictionary with keys 0 and 1.

2

Set the position to find Fibonacci number

Create a variable called n and set it to 7 to find the 7th Fibonacci number.

DSA Python

fib = {0: 0, 1: 1}
# Set n to 7
# Your code here

Hint

We want to find the Fibonacci number at position 7, so set n = 7.

3

Fill the dictionary using previous Fibonacci numbers

Use a for loop with variable i from 2 to n (inclusive) to fill the fib dictionary. Calculate each Fibonacci number as the sum of the two previous numbers: fib[i] = fib[i-1] + fib[i-2].

DSA Python

fib = {0: 0, 1: 1}
n = 7
# Use a for loop from 2 to n to fill fib
# Your code here

Hint

Start from 2 because 0 and 1 are base cases. Each new Fibonacci number is the sum of the two before it.

4

Print the Fibonacci number at position n

Write print(fib[n]) to display the Fibonacci number stored in fib at position n.

DSA Python

fib = {0: 0, 1: 1}
n = 7
for i in range(2, n + 1):
    fib[i] = fib[i-1] + fib[i-2]
# Print the Fibonacci number at position n
# Your code here

Hint

The 7th Fibonacci number is 13. Print it using print(fib[n]).

Practice

1. Which of the following best describes the overlapping subproblems property in dynamic programming?

easy

A. The problem can be broken down into smaller subproblems that are reused multiple times.

B. The problem has a unique solution without repeating any subproblems.

C. The problem requires sorting before solving.

D. The problem can only be solved using recursion without memoization.

Solution

1. Step 1: Understand overlapping subproblems

   Overlapping subproblems means the same smaller problems appear multiple times during solving.

2. Step 2: Match definition to options

   The problem can be broken down into smaller subproblems that are reused multiple times. correctly states that subproblems are reused multiple times, which is the key idea.

3. Final Answer:

   The problem can be broken down into smaller subproblems that are reused multiple times. -> Option A

4. Quick Check:

   Overlapping subproblems = reuse of subproblems [OK]

Hint: Look for repeated smaller problems in the main problem [OK]

Common Mistakes:

• Confusing overlapping subproblems with unique subproblems
• Thinking sorting is required for overlapping subproblems
• Assuming recursion alone solves overlapping subproblems

2. Which of the following Python function definitions correctly uses memoization to solve overlapping subproblems?

easy

A. def fib(n): if n <= 1: return n else: return fib(n-1) + fib(n-2)

B. def fib(n): return n * fib(n-1)

C. def fib(n): for i in range(n): print(i)

D. def fib(n, memo={}): if n in memo: return memo[n] if n <= 1: memo[n] = n else: memo[n] = fib(n-1, memo) + fib(n-2, memo) return memo[n]

Solution

1. Step 1: Identify memoization usage

   Memoization stores results of subproblems to avoid repeated work, usually with a dictionary.

2. Step 2: Check each option

   def fib(n, memo={}): if n in memo: return memo[n] if n <= 1: memo[n] = n else: memo[n] = fib(n-1, memo) + fib(n-2, memo) return memo[n] uses a dictionary 'memo' to save and reuse results, correctly implementing memoization.

3. Final Answer:

   def fib(n, memo={}): if n in memo: return memo[n] if n <= 1: memo[n] = n else: memo[n] = fib(n-1, memo) + fib(n-2, memo) return memo[n] -> Option D

4. Quick Check:

   Memoization = caching results [OK]

Hint: Look for a cache (dictionary) storing computed results [OK]

Common Mistakes:

• Missing memo dictionary parameter
• Not returning cached results
• Using loops without caching results

3. What is the output of the following code?

def fib(n, memo={}):
    if n in memo:
        return memo[n]
    if n <= 1:
        memo[n] = n
    else:
        memo[n] = fib(n-1, memo) + fib(n-2, memo)
    return memo[n]

print(fib(5))

medium

A. 13

B. 8

C. 5

D. 3

Solution

1. Step 1: Trace fib(5) calls with memoization

   fib(5) calls fib(4) and fib(3), which further call smaller fib values, caching results to avoid repeats.

2. Step 2: Calculate fib(5)

   fib(0)=0, fib(1)=1, fib(2)=1, fib(3)=2, fib(4)=3, fib(5)=5

3. Final Answer:

   5 -> Option C

4. Quick Check:

   Fibonacci(5) = 5 [OK]

Hint: Recall Fibonacci sequence: 0,1,1,2,3,5,8... [OK]

Common Mistakes:

• Confusing fib(5) with fib(6)
• Ignoring memoization effect
• Miscounting base cases

4. The following code attempts to use memoization for Fibonacci numbers but has a bug. What is the error?

def fib(n, memo={}):
    if n <= 1:
        return n
    if n not in memo:
        memo[n] = fib(n-1) + fib(n-2)
    return memo[n]

print(fib(4))

medium

A. Missing memo argument in recursive calls causes infinite recursion.

B. Base case is incorrect for n <= 1.

C. Memo dictionary is not initialized.

D. Return statement is missing.

Solution

1. Step 1: Check recursive calls

   Recursive calls fib(n-1) and fib(n-2) do not pass 'memo', so memo is empty each time.

2. Step 2: Understand effect of missing memo argument

   Without passing memo, caching fails and recursion repeats infinitely, causing a stack overflow.

3. Final Answer:

   Missing memo argument in recursive calls causes infinite recursion. -> Option A

4. Quick Check:

   Pass memo in recursive calls to fix [OK]

Hint: Always pass memo/cache in recursive calls [OK]

Common Mistakes:

• Forgetting to pass memo in recursive calls
• Assuming base case alone stops recursion
• Not initializing memo outside function

5. You want to find the minimum cost to reach the top of stairs where each step has a cost. You can climb 1 or 2 steps at a time. Which approach best uses optimal substructure and overlapping subproblems to solve this?

hard

A. Use recursion without caching to try all paths.

B. Use dynamic programming to store minimum cost for each step and build up solution.

C. Sort the costs and pick the smallest steps first.

D. Use a greedy approach to always pick the next step with minimum cost.

Solution

1. Step 1: Understand problem structure

   Minimum cost to reach step i depends on minimum cost to reach steps i-1 and i-2 plus cost at i.

2. Step 2: Apply dynamic programming

   Store minimum costs for each step to avoid recomputing (overlapping subproblems) and use previous results (optimal substructure).

3. Final Answer:

   Use dynamic programming to store minimum cost for each step and build up solution. -> Option B

4. Quick Check:

   DP uses optimal substructure + overlapping subproblems [OK]

Hint: Build solution step-by-step storing results [OK]

Common Mistakes:

• Using recursion without memoization causing slow solution
• Trying greedy approach which may fail
• Sorting costs which is irrelevant