YARN scheduling policies in Hadoop - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to set the YARN scheduler to Capacity Scheduler.

Hadoop

yarn.scheduler.class=[1]

Drag options to blanks, or click blank then click option'

Aorg.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler

Borg.apache.hadoop.yarn.server.resourcemanager.scheduler.RoundRobinScheduler

Corg.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler

Dorg.apache.hadoop.yarn.server.resourcemanager.scheduler.PriorityScheduler

Attempts:

3 left

💡 Hint

Common Mistakes

Choosing FairScheduler instead of CapacityScheduler

Using a non-existent scheduler class

Misspelling the class name

The Capacity Scheduler class is org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler. This sets YARN to use Capacity Scheduler.

2fill in blank

medium

Complete the code to configure the Fair Scheduler policy in YARN.

Hadoop

yarn.scheduler.class=[1]

Drag options to blanks, or click blank then click option'

Aorg.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler

Borg.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler

Corg.apache.hadoop.yarn.server.resourcemanager.scheduler.PriorityScheduler

Dorg.apache.hadoop.yarn.server.resourcemanager.scheduler.RoundRobinScheduler

Attempts:

3 left

💡 Hint

Common Mistakes

Confusing CapacityScheduler with FairScheduler

Using incorrect class names

Misspelling the scheduler class

The Fair Scheduler class is org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler. This sets YARN to use Fair Scheduler.

3fill in blank

hard

Fix the error in the code to set the YARN scheduler to Priority Scheduler.

Hadoop

yarn.scheduler.class=[1]

Drag options to blanks, or click blank then click option'

Aorg.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.PriorityScheduler

Borg.apache.hadoop.yarn.server.resourcemanager.scheduler.priority.PriorityScheduler

Corg.apache.hadoop.yarn.server.resourcemanager.scheduler.PriorityScheduler

Dorg.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.PriorityScheduler

Attempts:

3 left

💡 Hint

Common Mistakes

Using wrong package paths for PriorityScheduler

Misspelling the class name

Confusing PriorityScheduler with other schedulers

The correct class for Priority Scheduler is org.apache.hadoop.yarn.server.resourcemanager.scheduler.PriorityScheduler. Other options are incorrect package paths.

4fill in blank

hard

Fill both blanks to create a dictionary mapping scheduler names to their class paths.

Hadoop

scheduler_classes = {'capacity': '[1]', 'fair': '[2]'}

Drag options to blanks, or click blank then click option'

Aorg.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler

Borg.apache.hadoop.yarn.server.resourcemanager.scheduler.PriorityScheduler

Corg.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler

Dorg.apache.hadoop.yarn.server.resourcemanager.scheduler.RoundRobinScheduler

Attempts:

3 left

💡 Hint

Common Mistakes

Mixing up scheduler names and class paths

Using incorrect class paths

Swapping the values for 'capacity' and 'fair'

The dictionary maps 'capacity' to CapacityScheduler class and 'fair' to FairScheduler class.

5fill in blank

hard

Fill all three blanks to create a dictionary with scheduler names as keys, their class paths as values, and a filter for schedulers starting with 'org.apache.hadoop.yarn'.

Hadoop

schedulers = {k: v for k, v in {'capacity': '[1]', 'fair': '[2]', 'priority': '[3]'}.items() if v.startswith('org.apache.hadoop.yarn')}

Drag options to blanks, or click blank then click option'

Aorg.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler

Borg.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler

Corg.apache.hadoop.yarn.server.resourcemanager.scheduler.PriorityScheduler

Dcom.example.custom.scheduler.CustomScheduler

Attempts:

3 left

💡 Hint

Common Mistakes

Including custom or invalid scheduler class paths

Not matching the exact class paths

Incorrectly filtering the dictionary

The dictionary includes CapacityScheduler, FairScheduler, and PriorityScheduler class paths. The filter keeps only those starting with 'org.apache.hadoop.yarn'.