Hive query optimization in Hadoop - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is the main goal of Hive query optimization?

The main goal is to make Hive queries run faster and use fewer resources by improving how data is processed and accessed.

beginner

Explain what partitioning means in Hive.

Partitioning divides a large table into smaller parts based on a column, like splitting sales data by year. This helps Hive read only the needed parts, speeding up queries.

intermediate

What is bucketing in Hive and how does it help optimization?

Bucketing splits data into a fixed number of files (buckets) based on a hash of a column. It helps by making joins and sampling faster because data is organized predictably.

intermediate

How does predicate pushdown improve Hive query performance?

Predicate pushdown means filtering data early, close to where it is stored, so less data moves around. This reduces the amount of data Hive processes and speeds up queries.

beginner

Why is using ORC or Parquet file formats recommended for Hive optimization?

ORC and Parquet store data in a columnar way, which means Hive reads only the columns needed. They also support compression and indexing, making queries faster and saving storage.

What does partitioning in Hive help with?

A Reading only relevant parts of data

B Compressing data files

C Encrypting data for security

D Changing data types automatically

Partitioning splits data so Hive reads only the needed parts, improving query speed.

Which file format is best for columnar storage in Hive?

A CSV

B JSON

C TXT

D ORC

ORC is a columnar format that helps Hive read only needed columns, speeding up queries.

What is bucketing used for in Hive?

A Splitting data into fixed files for faster joins

B Encrypting data buckets

C Backing up data automatically

D Changing data schema

Bucketing organizes data into fixed files to optimize joins and sampling.

Predicate pushdown helps Hive by:

A Changing data format

B Increasing data size

C Filtering data early to reduce processing

D Sorting data alphabetically

Filtering data close to storage reduces the amount Hive processes, speeding queries.

Which of these is NOT a Hive optimization technique?

A Partitioning

B Adding random delays

C Bucketing

D Using ORC format

Adding random delays does not optimize Hive queries.

Describe three ways Hive query optimization can improve query speed and resource use.

Think about how data is organized and filtered.

Explain how using ORC or Parquet file formats helps Hive process data more efficiently.

Focus on how data is stored and accessed.