Row key design strategies in Hadoop - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Why is row key design important in Hadoop HBase?

In HBase, the design of the row key affects performance and data retrieval. Why is it important to choose a good row key?

A Because row keys control the replication factor of data in HDFS.

B Because row keys determine the file format used to store data on disk.

C Because a well-designed row key ensures even data distribution and fast lookups.

D Because row keys decide the compression algorithm applied to data.

💡 Hint

Think about how data is stored and accessed in HBase tables.

Row keys in HBase determine how data is sorted and distributed across regions. A good row key design avoids hotspots and improves query speed.

❓ Predict Output

intermediate

What is the output of this row key generation code?

Consider this Python code generating row keys for time-series data:

keys = [f"user_{i}_{20240101 + i}" for i in range(3)]
print(keys)

What is the output?

Hadoop

keys = [f"user_{i}_{20240101 + i}" for i in range(3)]
print(keys)

A ["user_0_20240101", "user_1_20240102", "user_2_20240103"]

B ["user_0_20240101", "user_1_20240101", "user_2_20240101"]

C ["user_0_20240101", "user_1_20240103", "user_2_20240105"]

D ["user_0_20240100", "user_1_20240101", "user_2_20240102"]

💡 Hint

Look at how the date part increments with i.

The date part adds i to 20240101, so it increases by 1 each time.

❓ data_output

advanced

How many regions will be created with this row key pattern?

In HBase, row keys are prefixed with a hash of the user ID modulo 4 to distribute load:

def generate_key(user_id, timestamp):
    prefix = user_id % 4
    return f"{prefix}_{user_id}_{timestamp}"

keys = [generate_key(i, 20240101) for i in range(8)]
regions = set(k.split('_')[0] for k in keys)
print(len(regions))

How many unique region prefixes are there?

Hadoop

def generate_key(user_id, timestamp):
    prefix = user_id % 4
    return f"{prefix}_{user_id}_{timestamp}"

keys = [generate_key(i, 20240101) for i in range(8)]
regions = set(k.split('_')[0] for k in keys)
print(len(regions))

A 1

B 8

C 2

D 4

💡 Hint

Consider the modulo operation and how many distinct values it produces for 0 to 7.

Modulo 4 of numbers 0 to 7 produces 4 distinct prefixes: 0,1,2,3.

🔧 Debug

advanced

Identify the error in this row key design code snippet

This code tries to create a row key by concatenating user ID and timestamp, but it raises an error:

user_id = 123
timestamp = 20240101
row_key = user_id + "_" + timestamp
print(row_key)

What error does this code raise?

Hadoop

user_id = 123
timestamp = 20240101
row_key = user_id + "_" + timestamp
print(row_key)

A NameError: name 'timestamp' is not defined

B TypeError: unsupported operand type(s) for +: 'int' and 'str'

C SyntaxError: invalid syntax

D ValueError: invalid literal for int() with base 10

💡 Hint

Check the data types of variables used with the plus operator.

Adding an integer and a string directly causes a TypeError in Python.

🚀 Application

expert

Which row key design best avoids hotspotting for sequential timestamps?

You have a dataset with many records per user, each with a sequential timestamp. You want to design row keys in HBase to avoid hotspotting (too much load on one region). Which row key design is best?

A Use row keys as <code>hash(userID)_timestamp</code> where hash distributes users evenly.

B Use row keys as <code>reverse_timestamp_userID</code> where timestamp digits are reversed.

C Use row keys as <code>userID_timestamp</code> with timestamps increasing sequentially.

D Use row keys as <code>timestamp_userID</code> with timestamps increasing sequentially.

💡 Hint

Think about how to spread writes evenly across regions.

Hashing userID prefixes the row key with a value that distributes data across regions, avoiding hotspotting caused by sequential timestamps.

Practice

1. What is the main purpose of designing a good row key in Hadoop tables?

easy

A. To speed up data retrieval by organizing data efficiently

B. To increase the size of the data stored

C. To make data encryption easier

D. To reduce the number of columns in the table

Solution

1. Step 1: Understand the role of row keys

   Row keys determine how data is stored and accessed in Hadoop tables, affecting retrieval speed.

2. Step 2: Identify the main goal of row key design

   The goal is to organize data so that searches and reads are fast and efficient.

3. Final Answer:

   To speed up data retrieval by organizing data efficiently -> Option A

4. Quick Check:

   Row key design = speed up retrieval [OK]

Hint: Good row keys help find data faster [OK]

Common Mistakes:

• Thinking row keys reduce data size
• Confusing row keys with encryption keys
• Assuming row keys reduce columns

2. Which of the following is a correct way to create a composite row key in Hadoop?

easy

A. row_key = user_id + '_' + timestamp

B. row_key = user_id * timestamp

C. row_key = timestamp - user_id

D. row_key = user_id / timestamp

Solution

1. Step 1: Understand composite row keys

   Composite keys combine meaningful parts as strings to create unique keys.

2. Step 2: Check the syntax for combining parts

   Concatenating with an underscore is a common and correct way to combine parts as strings.

3. Final Answer:

   row_key = user_id + '_' + timestamp -> Option A

4. Quick Check:

   Composite key = string concat with separator [OK]

Hint: Use string concat with separator for composite keys [OK]

Common Mistakes:

• Using arithmetic operations instead of string concat
• Not adding a separator between parts
• Mixing data types without conversion

3. Given the following Python code to create row keys, what will be the output list?

user_ids = ['u1', 'u2']
timestamps = [100, 200]
row_keys = [f'{uid}_{ts}' for uid in user_ids for ts in timestamps]
print(row_keys)

medium

A. ['u1_100', 'u2_200', 'u1_200', 'u2_100']

B. ['u1_100', 'u2_100', 'u1_200', 'u2_200']

C. ['u2_100', 'u2_200', 'u1_100', 'u1_200']

D. ['u1_100', 'u1_200', 'u2_100', 'u2_200']

Solution

1. Step 1: Understand the nested loop order in list comprehension

   The outer loop is over user_ids, inner loop over timestamps, so for each user_id, all timestamps are combined.

2. Step 2: Generate the list step-by-step

   For 'u1': 'u1_100', 'u1_200'; for 'u2': 'u2_100', 'u2_200'.

3. Final Answer:

   ['u1_100', 'u1_200', 'u2_100', 'u2_200'] -> Option D

4. Quick Check:

   Outer loop user_ids, inner timestamps = ['u1_100', 'u1_200', 'u2_100', 'u2_200'] [OK]

Hint: Outer loop first, inner loop second in list comprehensions [OK]

Common Mistakes:

• Mixing order of loops
• Assuming timestamps loop first
• Confusing list comprehension syntax

4. Identify the error in this row key design code snippet:

def create_row_key(user_id, timestamp):
    return user_id + timestamp

key = create_row_key('user1', 1234567890)
print(key)

medium

A. SyntaxError due to missing colon

B. TypeError because user_id is string and timestamp is int

C. NameError because function is not defined

D. No error, code runs fine

Solution

1. Step 1: Analyze the function parameters and operation

   user_id is a string, timestamp is an integer; adding them directly causes a type error.

2. Step 2: Understand Python type rules for + operator

   Python cannot add string and int directly without conversion.

3. Final Answer:

   TypeError because user_id is string and timestamp is int -> Option B

4. Quick Check:

   String + int without conversion = TypeError [OK]

Hint: Cannot add string and int directly in Python [OK]

Common Mistakes:

• Ignoring data types in addition
• Assuming implicit conversion
• Thinking syntax or name error

5. You want to design a row key for a large Hadoop table storing user activity logs. The goal is to avoid hot-spotting and allow fast range scans by time. Which strategy is best?

hard

A. Use only user ID as the row key

B. Use timestamp as the row key without modification

C. Use a reversed timestamp prefix followed by user ID, like row_key = reversed_timestamp + '_' + user_id

D. Use a random UUID as the row key

Solution

1. Step 1: Understand hot-spotting and range scan needs

   Hot-spotting happens when many writes go to the same key range; reversed timestamp spreads writes.

2. Step 2: Evaluate each option for balancing and scan speed

   Reversed timestamp prefix balances data and allows scanning by time; user ID alone or random UUIDs do not support range scans well.

3. Final Answer:

   Use a reversed timestamp prefix followed by user ID, like row_key = reversed_timestamp + '_' + user_id -> Option C

4. Quick Check:

   Reverse timestamp + user ID = balanced, fast scans [OK]

Hint: Reverse timestamp to avoid hot-spotting [OK]

Common Mistakes:

• Using only user ID causes hot-spotting
• Using raw timestamp causes write concentration
• Random UUIDs prevent range scans