Huffman Encoding in DSA C++ - Time & Space Complexity

Time Complexity: Huffman Encoding

O(n log n)

Understanding Time Complexity

We want to understand how the time needed to build a Huffman tree changes as the number of characters grows.

How does the work increase when we have more characters to encode?

Scenario Under Consideration

Analyze the time complexity of the following code snippet.

#include <queue>
#include <vector>
#include <iostream>

struct Node {
  char ch;
  int freq;
  Node* left;
  Node* right;
};

struct Compare {
  bool operator()(const Node* a, const Node* b) const {
    return a->freq > b->freq;
  }
};

void buildHuffmanTree(std::vector<Node*> &nodes) {
  std::priority_queue<Node*, std::vector<Node*>, Compare> pq(nodes.begin(), nodes.end());
  while (pq.size() > 1) {
    Node* left = pq.top(); pq.pop();
    Node* right = pq.top(); pq.pop();
    Node* merged = new Node{'\0', left->freq + right->freq, left, right};
    pq.push(merged);
  }
}
    

This code builds a Huffman tree by repeatedly merging the two nodes with the smallest frequencies.

Identify Repeating Operations

Identify the loops, recursion, array traversals that repeat.

• Primary operation: Removing two smallest nodes and inserting their merged node back into the priority queue.
• How many times: This happens once for each merge, which is one less than the number of initial nodes.

How Execution Grows With Input

Each merge involves removing two nodes and adding one node back, and the priority queue adjusts itself each time.

Input Size (n)	Approx. Operations
10	About 10 log 10 steps
100	About 100 log 100 steps
1000	About 1000 log 1000 steps

Pattern observation: The work grows a bit faster than the number of characters because of the sorting inside the priority queue.

Final Time Complexity

Time Complexity: O(n log n)

This means the time needed grows a little more than linearly as the number of characters increases.

Common Mistake

[X] Wrong: "Huffman encoding runs in simple linear time O(n) because it just merges nodes."

[OK] Correct: Each merge requires adjusting the priority queue, which takes extra time proportional to the log of the number of nodes.

Interview Connect

Understanding how Huffman encoding scales helps you explain efficient data compression and priority queue usage clearly in interviews.

Self-Check

"What if we used a simple sorted list instead of a priority queue? How would the time complexity change?"

Practice

1. What is the main purpose of Huffman Encoding in data compression?

easy

A. To sort characters alphabetically

B. To assign equal length codes to all characters

C. To encrypt data for security

D. To assign shorter codes to more frequent characters

Solution

1. Step 1: Understand Huffman Encoding goal

   Huffman Encoding aims to reduce the total number of bits by giving shorter codes to characters that appear more often.

2. Step 2: Compare options with this goal

   Only To assign shorter codes to more frequent characters matches this goal. Options A, B, and C do not describe Huffman Encoding correctly.

3. Final Answer:

   To assign shorter codes to more frequent characters -> Option D

4. Quick Check:

   Huffman Encoding = shorter codes for frequent chars [OK]

Hint: Remember: frequent chars get shorter codes [OK]

Common Mistakes:

• Thinking Huffman assigns equal length codes
• Confusing encoding with encryption
• Assuming sorting is the main goal

2. Which of the following C++ code snippets correctly declares a priority queue to build a Huffman tree based on character frequencies?

easy

A. std::priority_queue, std::vector>, std::greater<>> pq;

B. std::priority_queue> pq;

C. std::priority_queue> pq;

D. std::priority_queue, std::vector>, std::less<>> pq;

Solution

1. Step 1: Identify priority queue type for Huffman

   Huffman requires a min-heap priority queue to get the smallest frequency first. std::greater<> creates a min-heap.

2. Step 2: Check each option

   std::priority_queue, std::vector>, std::greater<>> pq; uses std::greater<> with pair, which is correct. Options B and C default to max-heap. std::priority_queue, std::vector>, std::less<>> pq; uses std::less<> which is max-heap.

3. Final Answer:

   std::priority_queue, std::vector>, std::greater<>> pq; -> Option A

4. Quick Check:

   Min-heap priority queue = std::priority_queue, std::vector>, std::greater<>> pq; [OK]

Hint: Use std::greater<> for min-heap in priority_queue [OK]

Common Mistakes:

• Using default max-heap instead of min-heap
• Swapping pair elements order incorrectly
• Forgetting to specify comparator

3. Given the frequencies: {'a': 5, 'b': 9, 'c': 12, 'd': 13}, what is the Huffman code for character 'a' after building the Huffman tree?

medium

A. 111

B. 00

C. 110

D. 10

Solution

1. Step 1: Build Huffman tree from frequencies

   Combine lowest frequencies: 5('a') + 9('b') = 14, then combine 12('c') + 13('d') = 25, then combine 14 + 25 = 39.

2. Step 2: Assign codes from tree

   'a' is in the left subtree of the first combined node, so it gets code '110' (left-right-left). Other codes differ.

3. Final Answer:

   110 -> Option C

4. Quick Check:

   'a' gets code = 110 [OK]

Hint: Lowest freq chars get codes with more leading zeros [OK]

Common Mistakes:

• Assigning wrong code by mixing tree branches
• Confusing 'a' code with 'b' or 'c'
• Not building tree stepwise

4. Identify the error in this C++ snippet for building a Huffman tree:

std::priority_queue, Compare> pq;
while(pq.size() > 1) {
  Node* left = pq.top(); pq.pop();
  Node* right = pq.top(); pq.pop();
  Node* parent = new Node(left->freq + right->freq, left, right);
  pq.push(parent);
}

medium

A. The loop condition should be pq.size() >= 1

B. The priority queue comparator Compare is missing or incorrect

C. Nodes should be popped after pushing the parent

D. The parent node frequency calculation is wrong

Solution

1. Step 1: Check priority queue usage

   Priority queue needs a comparator to order nodes by frequency ascending for Huffman. If Compare is missing or incorrect, ordering fails.

2. Step 2: Validate other parts

   Loop condition pq.size() > 1 is correct. Popping nodes before pushing parent is correct. Frequency sum is correct.

3. Final Answer:

   The priority queue comparator Compare is missing or incorrect -> Option B

4. Quick Check:

   Missing comparator breaks min-heap order [OK]

Hint: Always define comparator for priority_queue of pointers [OK]

Common Mistakes:

• Using wrong loop condition
• Popping nodes in wrong order
• Incorrect frequency sum

5. You have the string "abbcccdddd". Using Huffman Encoding, what is the total number of bits required to encode this string if the codes are assigned optimally?

hard

A. 19 bits

B. 22 bits

C. 24 bits

D. 20 bits

Solution

1. Step 1: Calculate character frequencies

   'a':1, 'b':2, 'c':3, 'd':4.

2. Step 2: Build Huffman tree and assign codes

   Combine lowest frequencies: 1+2=3, then 3+3=6, then 4+6=10. Codes assigned roughly: 'd' = 1 bit, 'c' = 2 bits, 'b' = 3 bits, 'a' = 3 bits.

3. Step 3: Calculate total bits

   Total bits = (1*3) + (2*3) + (3*2) + (4*1) = 3 + 6 + 6 + 4 = 19 bits.

4. Final Answer:

   19 bits -> Option A

5. Quick Check:

   Total bits = sum(freq * code length) = 19 [OK]

Hint: Multiply frequency by code length, then sum all [OK]

Common Mistakes:

• Mixing frequencies and code lengths
• Assigning wrong code lengths
• Adding frequencies instead of bits