Input splits and data locality in Hadoop - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to create input splits for a file in Hadoop.

Hadoop

FileInputFormat.addInputPath(job, new Path([1]));

Drag options to blanks, or click blank then click option'

Ajob

B"inputfile.txt"

C"/user/data/input"

Dnew Path()

Attempts:

3 left

💡 Hint

Common Mistakes

Passing the job object instead of a path string.

Not using quotes around the path string.

The addInputPath method requires a Path object pointing to the input directory or file. Here, "/user/data/input" is the correct path string.

2fill in blank

medium

Complete the code to get the number of input splits from the job context.

Hadoop

List<InputSplit> splits = inputFormat.getSplits([1]);

Drag options to blanks, or click blank then click option'

Ajob

Bcontext

Cconfiguration

DinputFormat

Attempts:

3 left

💡 Hint

Common Mistakes

Passing the inputFormat object itself.

Passing an unrelated variable like context.

The getSplits method requires the JobContext or Job object to generate splits. Here, job is the correct argument.

3fill in blank

hard

Fix the error in the code to ensure data locality is considered when processing splits.

Hadoop

for (InputSplit split : splits) {
    String[] locations = split.getLocations();
    if (locations.length > [1]) {
        System.out.println("Data is local to node: " + locations[0]);
    }
}

Drag options to blanks, or click blank then click option'

A1

B0

C-1

Dlocations.length

Attempts:

3 left

💡 Hint

Common Mistakes

Checking if length > 1 which skips cases with exactly one location.

Using negative numbers in the condition.

The condition should check if there is at least one location, so the length must be greater than 0.

4fill in blank

hard

Fill both blanks to create a map of split locations and their corresponding split sizes.

Hadoop

Map<String, Long> splitSizes = new HashMap<>();
for (InputSplit split : splits) {
    String[] locations = split.getLocations();
    long size = split.[1]();
    for (String loc : locations) {
        splitSizes.put(loc, splitSizes.getOrDefault(loc, [2]) + size);
    }
}

Drag options to blanks, or click blank then click option'

AgetLength

B0L

CgetSize

Dnull

Attempts:

3 left

💡 Hint

Common Mistakes

Using getSize() which does not exist.

Using null as default value causing errors.

The method to get the size of an InputSplit is getLength(). The default value for the map when a location is not present should be 0L (zero as a long).

5fill in blank

hard

Fill all three blanks to filter input splits larger than 128MB and print their first location.

Hadoop

for (InputSplit split : splits) {
    if (split.[1]() > [2]) {
        String[] locs = split.[3]();
        if (locs.length > 0) {
            System.out.println("Large split at: " + locs[0]);
        }
    }
}

Drag options to blanks, or click blank then click option'

AgetLength

B134217728

CgetLocations

DgetSize

Attempts:

3 left

💡 Hint

Common Mistakes

Using getSize() which is not a valid method.

Using incorrect byte value for 128MB.

Confusing getLocations() with other methods.

getLength() returns the size of the split in bytes. 128MB equals 134,217,728 bytes. getLocations() returns the array of nodes where the split data is located.