LOAD, FILTER, and STORE operations in Hadoop - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

What is the output of this Pig Latin script after filtering?

Given the following Pig Latin script, what will be the content of filtered_data?

data = LOAD 'input.txt' USING PigStorage(',') AS (name:chararray, age:int, city:chararray);
filtered_data = FILTER data BY age > 30;
DUMP filtered_data;

Assume input.txt contains:
John,25,New York
Mary,35,Chicago
Bob,40,Seattle

Hadoop

data = LOAD 'input.txt' USING PigStorage(',') AS (name:chararray, age:int, city:chararray);
filtered_data = FILTER data BY age > 30;
DUMP filtered_data;

A (John,25,New York)

B

(John,25,New York)
(Mary,35,Chicago)
(Bob,40,Seattle)

C

(Mary,35,Chicago)
(Bob,40,Seattle)

D Empty output

💡 Hint

Think about the filter condition age > 30 and which rows satisfy it.

The filter keeps only rows where age is greater than 30. John is 25, so excluded. Mary and Bob are included.

❓ data_output

intermediate

How many records remain after filtering?

Consider this Pig Latin script:

records = LOAD 'data.csv' USING PigStorage(',') AS (id:int, score:int);
passed = FILTER records BY score >= 50;
STORE passed INTO 'passed_output';

If data.csv has 5 records with scores: 45, 50, 60, 30, 55, how many records will be stored in passed_output?

Hadoop

records = LOAD 'data.csv' USING PigStorage(',') AS (id:int, score:int);
passed = FILTER records BY score >= 50;
STORE passed INTO 'passed_output';

A 3

B 2

C 5

D 0

💡 Hint

Count how many scores are 50 or more.

Scores 50, 60, and 55 meet the condition, so 3 records remain.

🔧 Debug

advanced

Identify the error in this Pig Latin script

What error will this Pig Latin script produce?

data = LOAD 'file.txt' USING PigStorage(',') AS (name:chararray, age:int);
filtered = FILTER data BY age > '30';
DUMP filtered;

Hadoop

data = LOAD 'file.txt' USING PigStorage(',') AS (name:chararray, age:int);
filtered = FILTER data BY age > '30';
DUMP filtered;

A SyntaxError: Invalid comparison operator

B RuntimeError: File not found

C No error, runs successfully

D TypeError: Cannot compare int with chararray

💡 Hint

Check the data types used in the filter condition.

The filter compares an integer field age with a string '30', causing a type mismatch error.

🚀 Application

advanced

Which Pig Latin command stores filtered data correctly?

You want to load a CSV file, filter rows where status is 'active', and save the result. Which command correctly stores the filtered data?

data = LOAD 'users.csv' USING PigStorage(',') AS (user_id:int, status:chararray);
active_users = FILTER data BY status == 'active';

Hadoop

data = LOAD 'users.csv' USING PigStorage(',') AS (user_id:int, status:chararray);
active_users = FILTER data BY status == 'active';

A STORE active_users INTO 'active_output' USING PigStorage();

B STORE data INTO 'active_output' USING PigStorage();

C STORE active_users INTO 'active_output';

D STORE active_users INTO 'active_output' USING JsonStorage();

💡 Hint

Remember to specify the storage format when using STORE.

Option A correctly stores the filtered relation active_users using PigStorage.

🧠 Conceptual

expert

What happens if you STORE a relation without filtering after LOAD?

Consider this Pig Latin script:

data = LOAD 'records.txt' USING PigStorage(',') AS (id:int, value:int);
STORE data INTO 'output_dir' USING PigStorage();

What will be the content of output_dir?

Hadoop

data = LOAD 'records.txt' USING PigStorage(',') AS (id:int, value:int);
STORE data INTO 'output_dir' USING PigStorage();

A Only records with non-null values will be stored.

B All records from 'records.txt' will be stored in 'output_dir'.

C No records will be stored because no FILTER was applied.

D An error will occur because STORE requires a FILTER operation first.

💡 Hint

Think about what STORE does with a loaded relation.

STORE saves all data in the relation. Filtering is optional. Without filtering, all loaded records are stored.