Overlapping Subproblems and Optimal Substructure in DSA Python - Time & Space Complexity

Time Complexity: Overlapping Subproblems and Optimal Substructure

O(2^n)

Understanding Time Complexity

We want to understand how solving problems with repeated parts affects the time it takes.

How does breaking a problem into smaller parts and reusing answers change the work done?

Scenario Under Consideration

Analyze the time complexity of this Fibonacci function using simple recursion.

def fib(n):
    if n <= 1:
        return n
    return fib(n-1) + fib(n-2)

This code calculates the nth Fibonacci number by calling itself twice for smaller values.

Identify Repeating Operations

Look at what repeats when the function calls itself.

• Primary operation: Recursive calls to fib(n-1) and fib(n-2)
• How many times: Many calls repeat the same fib values multiple times, especially for smaller n.

How Execution Grows With Input

Each call splits into two more calls, growing very fast.

Input Size (n)	Approx. Operations
10	About 177 calls
20	Over 21,000 calls
30	More than 2 million calls

Pattern observation: The number of calls grows roughly like the Fibonacci sequence, causing very fast growth.

Final Time Complexity

Time Complexity: O(2^n)

This means the work doubles with each increase in n, making it very slow for large n.

Common Mistake

[X] Wrong: "Since fib(n) calls fib(n-1) and fib(n-2), it should be about 2n calls."

[OK] Correct: Each call creates two more calls, so the total calls grow exponentially, not linearly.

Interview Connect

Understanding this helps you see why some problems need smarter methods to avoid repeating work.

Self-Check

"What if we store results of fib(n) after computing them once? How would the time complexity change?"

Practice

1. Which of the following best describes the overlapping subproblems property in dynamic programming?

easy

A. The problem can be broken down into smaller subproblems that are reused multiple times.

B. The problem has a unique solution without repeating any subproblems.

C. The problem requires sorting before solving.

D. The problem can only be solved using recursion without memoization.

Solution

1. Step 1: Understand overlapping subproblems

   Overlapping subproblems means the same smaller problems appear multiple times during solving.

2. Step 2: Match definition to options

   The problem can be broken down into smaller subproblems that are reused multiple times. correctly states that subproblems are reused multiple times, which is the key idea.

3. Final Answer:

   The problem can be broken down into smaller subproblems that are reused multiple times. -> Option A

4. Quick Check:

   Overlapping subproblems = reuse of subproblems [OK]

Hint: Look for repeated smaller problems in the main problem [OK]

Common Mistakes:

• Confusing overlapping subproblems with unique subproblems
• Thinking sorting is required for overlapping subproblems
• Assuming recursion alone solves overlapping subproblems

2. Which of the following Python function definitions correctly uses memoization to solve overlapping subproblems?

easy

A. def fib(n): if n <= 1: return n else: return fib(n-1) + fib(n-2)

B. def fib(n): return n * fib(n-1)

C. def fib(n): for i in range(n): print(i)

D. def fib(n, memo={}): if n in memo: return memo[n] if n <= 1: memo[n] = n else: memo[n] = fib(n-1, memo) + fib(n-2, memo) return memo[n]

Solution

1. Step 1: Identify memoization usage

   Memoization stores results of subproblems to avoid repeated work, usually with a dictionary.

2. Step 2: Check each option

   def fib(n, memo={}): if n in memo: return memo[n] if n <= 1: memo[n] = n else: memo[n] = fib(n-1, memo) + fib(n-2, memo) return memo[n] uses a dictionary 'memo' to save and reuse results, correctly implementing memoization.

3. Final Answer:

   def fib(n, memo={}): if n in memo: return memo[n] if n <= 1: memo[n] = n else: memo[n] = fib(n-1, memo) + fib(n-2, memo) return memo[n] -> Option D

4. Quick Check:

   Memoization = caching results [OK]

Hint: Look for a cache (dictionary) storing computed results [OK]

Common Mistakes:

• Missing memo dictionary parameter
• Not returning cached results
• Using loops without caching results

3. What is the output of the following code?

def fib(n, memo={}):
    if n in memo:
        return memo[n]
    if n <= 1:
        memo[n] = n
    else:
        memo[n] = fib(n-1, memo) + fib(n-2, memo)
    return memo[n]

print(fib(5))

medium

A. 13

B. 8

C. 5

D. 3

Solution

1. Step 1: Trace fib(5) calls with memoization

   fib(5) calls fib(4) and fib(3), which further call smaller fib values, caching results to avoid repeats.

2. Step 2: Calculate fib(5)

   fib(0)=0, fib(1)=1, fib(2)=1, fib(3)=2, fib(4)=3, fib(5)=5

3. Final Answer:

   5 -> Option C

4. Quick Check:

   Fibonacci(5) = 5 [OK]

Hint: Recall Fibonacci sequence: 0,1,1,2,3,5,8... [OK]

Common Mistakes:

• Confusing fib(5) with fib(6)
• Ignoring memoization effect
• Miscounting base cases

4. The following code attempts to use memoization for Fibonacci numbers but has a bug. What is the error?

def fib(n, memo={}):
    if n <= 1:
        return n
    if n not in memo:
        memo[n] = fib(n-1) + fib(n-2)
    return memo[n]

print(fib(4))

medium

A. Missing memo argument in recursive calls causes infinite recursion.

B. Base case is incorrect for n <= 1.

C. Memo dictionary is not initialized.

D. Return statement is missing.

Solution

1. Step 1: Check recursive calls

   Recursive calls fib(n-1) and fib(n-2) do not pass 'memo', so memo is empty each time.

2. Step 2: Understand effect of missing memo argument

   Without passing memo, caching fails and recursion repeats infinitely, causing a stack overflow.

3. Final Answer:

   Missing memo argument in recursive calls causes infinite recursion. -> Option A

4. Quick Check:

   Pass memo in recursive calls to fix [OK]

Hint: Always pass memo/cache in recursive calls [OK]

Common Mistakes:

• Forgetting to pass memo in recursive calls
• Assuming base case alone stops recursion
• Not initializing memo outside function

5. You want to find the minimum cost to reach the top of stairs where each step has a cost. You can climb 1 or 2 steps at a time. Which approach best uses optimal substructure and overlapping subproblems to solve this?

hard

A. Use recursion without caching to try all paths.

B. Use dynamic programming to store minimum cost for each step and build up solution.

C. Sort the costs and pick the smallest steps first.

D. Use a greedy approach to always pick the next step with minimum cost.

Solution

1. Step 1: Understand problem structure

   Minimum cost to reach step i depends on minimum cost to reach steps i-1 and i-2 plus cost at i.

2. Step 2: Apply dynamic programming

   Store minimum costs for each step to avoid recomputing (overlapping subproblems) and use previous results (optimal substructure).

3. Final Answer:

   Use dynamic programming to store minimum cost for each step and build up solution. -> Option B

4. Quick Check:

   DP uses optimal substructure + overlapping subproblems [OK]

Hint: Build solution step-by-step storing results [OK]

Common Mistakes:

• Using recursion without memoization causing slow solution
• Trying greedy approach which may fail
• Sorting costs which is irrelevant