Compression codecs (Snappy, LZO, Gzip) in Hadoop - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Which compression codec is best suited for fast decompression in Hadoop?

In Hadoop, you want to quickly read compressed data with minimal CPU overhead. Which compression codec is generally best for fast decompression?

A Snappy, because it is optimized for speed over compression ratio.

B Gzip, because it has the highest compression ratio.

C LZO, because it uses the least memory during decompression.

D No compression, to avoid decompression overhead.

💡 Hint

Think about which codec is designed for speed rather than maximum compression.

Snappy is designed to provide fast compression and decompression speeds, making it ideal for scenarios where speed is more important than maximum compression ratio.

❓ data_output

intermediate

What is the output size difference when compressing a 100MB text file with Gzip vs Snappy?

You compress the same 100MB text file using Gzip and Snappy codecs in Hadoop. Which statement correctly describes the expected compressed file sizes?

A Both files will be about the same size because they use similar algorithms.

B Gzip file will be smaller than Snappy file due to better compression ratio.

C Snappy file will be smaller than Gzip file because it compresses better.

D Snappy file will be larger than the original 100MB file.

💡 Hint

Consider which codec compresses better but is slower.

Gzip typically achieves a better compression ratio than Snappy, so the compressed file size will be smaller with Gzip.

🔧 Debug

advanced

Why does Hadoop fail to read LZO compressed files by default?

You try to read LZO compressed files in Hadoop but get errors. What is the most likely cause?

A LZO codec is not installed or native libraries are missing on the Hadoop nodes.

B LZO files are corrupted and cannot be decompressed.

C Hadoop does not support LZO compression at all.

D The input files are not compressed with LZO but with Gzip.

💡 Hint

Think about Hadoop's support for LZO and what is required to use it.

Hadoop requires native LZO libraries to be installed on all nodes to read LZO compressed files. Without these, decompression fails.

🚀 Application

advanced

Choosing the best codec for a Hadoop job with large binary files needing moderate compression and fast read times

You have large binary files to process in Hadoop. You want moderate compression to save space but also fast read times for analysis. Which codec should you choose?

A Gzip, for maximum compression despite slower reads.

B No compression, to avoid any decompression overhead.

C LZO, for a balance of compression and speed but requires setup.

D Snappy, for fast reads with moderate compression.

💡 Hint

Consider codecs that balance speed and compression but may need extra setup.

LZO offers a good balance between compression ratio and speed, making it suitable for large binary files where moderate compression and fast reads are needed. However, it requires native libraries installed.

❓ Predict Output

expert

What is the output of this Hadoop compression codec configuration snippet?

Given the following Hadoop configuration snippet, what compression codec will be used for output files?

Hadoop

conf.set("mapreduce.output.fileoutputformat.compress", "true")
conf.set("mapreduce.output.fileoutputformat.compress.codec", "org.apache.hadoop.io.compress.SnappyCodec")
conf.set("mapreduce.output.fileoutputformat.compress.type", "BLOCK")

A Output files will be compressed using LZO codec with record compression.

B Output files will be compressed using Gzip codec with block compression.

C Output files will not be compressed because compression type is invalid.

D Output files will be compressed using Snappy codec with block compression.

💡 Hint

Look carefully at the codec class name and compression type.

The configuration explicitly sets compression to true, uses SnappyCodec class, and block compression type, so output files will be compressed with Snappy in block mode.