Pig Latin basics in Hadoop - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Output of a simple LOAD and FILTER operation

Given the following Pig Latin script, what is the output relation after the FILTER operation?

data = LOAD 'input.txt' USING PigStorage(',') AS (name:chararray, age:int);
adults = FILTER data BY age >= 18;

Assume input.txt contains:

John,17
Mary,22
Bob,18

Hadoop

data = LOAD 'input.txt' USING PigStorage(',') AS (name:chararray, age:int);
adults = FILTER data BY age >= 18;
DUMP adults;

A (Mary,22)

B

(John,17)
(Mary,22)
(Bob,18)

C (John,17)

D

(Mary,22)
(Bob,18)

💡 Hint

FILTER keeps only rows where the condition is true.

The FILTER keeps only rows where age is 18 or more, so John (17) is excluded.

❓ data_output

intermediate

Result of a GROUP and COUNT operation

What is the output of the following Pig Latin script?

data = LOAD 'input.txt' USING PigStorage(',') AS (category:chararray, value:int);
grp = GROUP data BY category;
counted = FOREACH grp GENERATE group, COUNT(data);
DUMP counted;

Assume input.txt contains:

A,10
B,20
A,30
B,40
A,50

Hadoop

data = LOAD 'input.txt' USING PigStorage(',') AS (category:chararray, value:int);
grp = GROUP data BY category;
counted = FOREACH grp GENERATE group, COUNT(data);
DUMP counted;

A

(A,1)
(B,1)

B

(A,3)
(B,2)

C

(A,5)
(B,5)

D

(A,2)
(B,3)

💡 Hint

GROUP collects rows by category, COUNT counts rows per group.

Category A appears 3 times, B appears 2 times, so counts are (A,3) and (B,2).

🔧 Debug

advanced

Identify the error in a JOIN operation

What error will this Pig Latin script produce?

users = LOAD 'users.txt' USING PigStorage(',') AS (user_id:int, name:chararray);
orders = LOAD 'orders.txt' USING PigStorage(',') AS (order_id:int, user:int, amount:float);
joined = JOIN users BY user_id, orders BY user_id;
DUMP joined;

Hadoop

users = LOAD 'users.txt' USING PigStorage(',') AS (user_id:int, name:chararray);
orders = LOAD 'orders.txt' USING PigStorage(',') AS (order_id:int, user:int, amount:float);
joined = JOIN users BY user_id, orders BY user_id;
DUMP joined;

A Error: Field 'user_id' not found in relation 'orders'

B No error, outputs joined data

C Error: Syntax error near JOIN statement

D Error: Duplicate field names in output

💡 Hint

Check field names used in JOIN keys.

The 'orders' relation has a field named 'user', not 'user_id', so JOIN fails with field not found error.

❓ visualization

advanced

Visualize the result of a FOREACH GENERATE with arithmetic

Given this Pig Latin script, what is the output after the FOREACH GENERATE?

data = LOAD 'input.txt' USING PigStorage(',') AS (item:chararray, price:float, quantity:int);
total = FOREACH data GENERATE item, price * quantity AS total_cost;
DUMP total;

Assume input.txt contains:

Pen,1.5,10
Notebook,2.0,5
Eraser,0.5,20

Hadoop

data = LOAD 'input.txt' USING PigStorage(',') AS (item:chararray, price:float, quantity:int);
total = FOREACH data GENERATE item, price * quantity AS total_cost;
DUMP total;

A

(Pen,15.0)
(Notebook,10.0)
(Eraser,10.0)

B

(Pen,1.5)
(Notebook,2.0)
(Eraser,0.5)

C

(Pen,10)
(Notebook,5)
(Eraser,20)

D

(Pen,150)
(Notebook,200)
(Eraser,100)

💡 Hint

Multiply price by quantity for each item.

Pen: 1.5*10=15.0, Notebook: 2.0*5=10.0, Eraser: 0.5*20=10.0

🚀 Application

expert

Determine the number of records after a COGROUP and FILTER

Consider these Pig Latin commands:

students = LOAD 'students.txt' USING PigStorage(',') AS (student_id:int, name:chararray);
grades = LOAD 'grades.txt' USING PigStorage(',') AS (student_id:int, grade:int);
grouped = COGROUP students BY student_id, grades BY student_id;
passed = FILTER grouped BY COUNT(grades) > 0 AND AVG(grades.grade) >= 60;
DUMP passed;

Assuming:

students.txt:
1,Alice
2,Bob
3,Charlie
grades.txt:
1,70
1,80
2,50
3,90
3,55

How many records will passed contain?

Hadoop

students = LOAD 'students.txt' USING PigStorage(',') AS (student_id:int, name:chararray);
grades = LOAD 'grades.txt' USING PigStorage(',') AS (student_id:int, grade:int);
grouped = COGROUP students BY student_id, grades BY student_id;
passed = FILTER grouped BY COUNT(grades) > 0 AND AVG(grades.grade) >= 60;
DUMP passed;

A 3

B 1

C 2

D 0

💡 Hint

Check which students have grades and average grade >= 60.

Student 1: grades 70,80 avg=75 (passes); Student 2: grade 50 avg=50 (fails); Student 3: grades 90,55 avg=72.5 (passes). So 2 records pass.