Cluster planning and sizing in Hadoop - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Key factor in determining Hadoop cluster size

Which of the following is the most important factor when deciding the size of a Hadoop cluster?

A The total volume of data to be processed

B The number of users accessing the cluster

C The brand of hardware used

D The color of the server racks

💡 Hint

Think about what directly impacts storage and processing needs.

The total volume of data determines how much storage and processing power is needed, which directly affects cluster size.

❓ data_output

intermediate

Calculate total storage needed for a Hadoop cluster

You have 10 TB of raw data. Hadoop uses replication factor 3 by default. What is the total storage needed in the cluster?

A 10 TB

B 20 TB

C 3 TB

D 30 TB

💡 Hint

Remember Hadoop stores multiple copies of data for fault tolerance.

With replication factor 3, each data block is stored 3 times, so total storage is 10 TB * 3 = 30 TB.

❓ Predict Output

advanced

Estimate number of nodes needed based on CPU cores

Given a Hadoop job requires 200 CPU cores and each node has 16 cores, how many nodes are needed?

Consider only whole nodes.

A 13 nodes

B 12 nodes

C 14 nodes

D 15 nodes

💡 Hint

Divide total cores needed by cores per node and round up.

200 cores / 16 cores per node = 12.5 nodes, so 13 nodes are needed to meet the requirement.

❓ visualization

advanced

Visualize cluster storage distribution

You have a Hadoop cluster with 5 nodes. Each node has 4 TB storage. The replication factor is 3. How much usable storage is available in the cluster?

A 20 TB usable storage

B 60 TB usable storage

C 6.67 TB usable storage

D 15 TB usable storage

💡 Hint

Calculate total raw storage then divide by replication factor.

Total raw storage is 5 nodes * 4 TB = 20 TB. Usable storage = 20 TB / 3 = 6.67 TB.

🚀 Application

expert

Optimize cluster size for mixed workload

You manage a Hadoop cluster running both batch and real-time jobs. Batch jobs need high storage, real-time jobs need low latency and high CPU. Which cluster sizing strategy best balances these needs?

A Use small nodes with minimal storage and CPU, run all workloads on same nodes

B Use large nodes with high storage and many CPU cores, and separate batch and real-time workloads on different nodes

C Use nodes with only high storage, ignore CPU needs

D Use nodes with only high CPU, ignore storage needs

💡 Hint

Think about workload isolation and resource specialization.

Separating workloads on nodes optimized for their needs improves performance and resource use.