Job priorities and queues in Ruby on Rails - Step-by-Step Execution

Concept Flow - Job priorities and queues

Job Created

↓

Assign Priority

↓

Place in Queue Based on Priority

↓

Worker Picks Job from Highest Priority Queue

↓

Job Executed

↓

Job Completed or Failed

↓

Repeat for Next Job

Jobs are created with priorities, placed in queues accordingly, and workers pick jobs from the highest priority queue first.

Execution Sample

Ruby on Rails

class MyJob < ApplicationJob
  queue_as :default
  def perform(*args)
    # job code here
  end
end

MyJob.set(queue: :high_priority).perform_later(*args)

This code creates a job assigned to a high priority queue and enqueues it for later execution.

Execution Table

Step	Action	Job Priority	Queue Placed	Worker Picks Job	Job Status
1	Job Created	default	default	No	Pending
2	Job Created with High Priority	high_priority	high_priority	No	Pending
3	Worker Checks Queues	-	-	Picks from high_priority	-
4	Worker Executes Job	high_priority	high_priority	Yes	Running
5	Job Completes Successfully	high_priority	high_priority	Yes	Completed
6	Worker Checks Queues Again	-	-	Picks from default if any	-
7	No More Jobs	-	-	No	Idle

💡 No more jobs in any queue, workers become idle

Variable Tracker

Variable	Start	After Step 1	After Step 2	After Step 4	After Step 5	Final
Job Priority	none	default	high_priority	high_priority	high_priority	none
Queue State	empty	default:1 job	default:1 job, high_priority:1 job	default:1 job, high_priority:0 jobs	default:1 job, high_priority:0 jobs	empty
Worker Status	idle	idle	idle	running job	idle	idle

Key Moments - 3 Insights

Why does the worker pick from the high_priority queue before the default queue?

Because the execution_table at Step 3 shows the worker checks queues and picks from the highest priority queue first, ensuring urgent jobs run before others.

What happens if no jobs are in the high_priority queue but jobs exist in the default queue?

At Step 6, the worker checks queues again and picks jobs from the default queue if the high_priority queue is empty, as shown in the execution_table.

How does assigning a queue affect job execution order?

Assigning a queue with a priority (like high_priority) places the job in that queue, so workers pick jobs from higher priority queues first, as seen in Steps 2 and 3.

Visual Quiz - 3 Questions

Look at the execution_table, what is the job status at Step 4?

A Pending

B Running

C Completed

D Idle

💡 Hint

Check the 'Job Status' column at Step 4 in the execution_table

At which step does the worker pick a job from the default queue?

A Step 6

B Step 5

C Step 3

D Step 7

💡 Hint

Look at the 'Worker Picks Job' column in the execution_table for when default queue is chosen

If a job is assigned to a low_priority queue, how would the worker pick order change?

A Worker picks low_priority jobs before high_priority

B Worker picks low_priority jobs before default but after high_priority

C Worker picks low_priority jobs after default and high_priority

D Worker ignores low_priority jobs

💡 Hint

Recall that workers pick jobs from highest priority queues first as shown in the concept_flow and execution_table

Concept Snapshot

In Rails, jobs can be assigned to different queues representing priorities.
Workers pick jobs from higher priority queues first.
Use queue_as or set(queue:) to assign jobs.
Jobs run asynchronously in order of queue priority.
This helps manage workload and urgent tasks efficiently.

Full Transcript

In Rails, background jobs can be assigned to different queues to manage priorities. When a job is created, it is assigned a priority by placing it in a specific queue, such as 'default' or 'high_priority'. Workers look at these queues and always pick jobs from the highest priority queue available. The execution table shows that when a job is created with high priority, it goes into the high_priority queue. Workers pick jobs from this queue before the default queue. After the job runs and completes, the worker checks for more jobs. If no jobs remain, the worker becomes idle. This system ensures urgent jobs run first and helps organize background work efficiently.

Practice

1. What is the main purpose of setting different job priorities in Rails Active Job?

easy

A. To run more important tasks before less important ones

B. To make jobs run slower

C. To delete jobs automatically

D. To run all jobs at the same time

Solution

1. Step 1: Understand job priorities concept

   Job priorities help decide which jobs run first based on importance.

2. Step 2: Identify the purpose in Rails

   Rails uses priorities to run important jobs before less important ones.

3. Final Answer:

   To run more important tasks before less important ones -> Option A

4. Quick Check:

   Job priorities = run important jobs first [OK]

Hint: Priorities mean important jobs run before others [OK]

Common Mistakes:

• Thinking priorities slow down jobs
• Assuming priorities delete jobs
• Believing all jobs run simultaneously

2. Which syntax correctly sets a queue name for a job class in Rails?

easy

A. queue_as :default

B. queue = :default

C. set_queue :default

D. queue_name = 'default'

Solution

1. Step 1: Recall Rails Active Job syntax

   Rails uses queue_as to set the queue name inside job classes.

2. Step 2: Match correct syntax

   queue_as :default is the correct way to assign the queue.

3. Final Answer:

   queue_as :default -> Option A

4. Quick Check:

   Use queue_as to set queue [OK]

Hint: Use queue_as :name inside job class [OK]

Common Mistakes:

• Using assignment (=) instead of queue_as
• Trying to set queue with unknown methods
• Using string instead of symbol incorrectly

3. Given this job class:

class EmailJob < ApplicationJob
  queue_as :mailers
  def perform(user_id)
    # send email
  end
end

Which queue will this job be added to when enqueued?

medium

A. jobs

B. default

C. emails

D. mailers

Solution

1. Step 1: Check the queue_as setting in the job class

   The job class sets queue_as :mailers, so it uses the 'mailers' queue.

2. Step 2: Understand enqueue behavior

   When enqueued, the job goes to the queue named 'mailers'.

3. Final Answer:

   mailers -> Option D

4. Quick Check:

   queue_as :mailers means queue is 'mailers' [OK]

Hint: queue_as sets the queue name exactly [OK]

Common Mistakes:

• Assuming default queue if queue_as is set
• Confusing queue name with job class name
• Picking similar sounding queue names

4. Identify the error in this job class:

class CleanupJob < ApplicationJob
  queue_as = :low_priority
  def perform
    # cleanup code
  end
end

medium

A. Job class must inherit from ActiveJob::Base

B. perform method must have an argument

C. queue_as should be called as a method, not assigned

D. queue_as must be a string, not a symbol

Solution

1. Step 1: Check how queue_as is set

   queue_as is a method call, not an assignment. Using '=' causes syntax error.

2. Step 2: Correct syntax for queue_as

   It should be queue_as :low_priority without '='.

3. Final Answer:

   queue_as should be called as a method, not assigned -> Option C

4. Quick Check:

   queue_as is a method call, not variable assignment [OK]

Hint: Use queue_as :name, not queue_as = :name [OK]

Common Mistakes:

• Using '=' with queue_as
• Forgetting to inherit from ApplicationJob
• Thinking perform must have arguments always

5. You want to run urgent jobs before normal jobs in Rails. You have two job classes:

class UrgentJob < ApplicationJob
  queue_as :urgent
  def perform
    # urgent task
  end
end

class NormalJob < ApplicationJob
  queue_as :normal
  def perform
    # normal task
  end
end

How can you ensure urgent jobs run before normal jobs when using Sidekiq?

hard

A. Rename UrgentJob to NormalJob

B. Configure Sidekiq to process the 'urgent' queue before the 'normal' queue

C. Set both jobs to the same queue

D. Run urgent jobs manually without Sidekiq

Solution

1. Step 1: Understand queue priority in Sidekiq

   Sidekiq processes queues in the order they are listed in its configuration.

2. Step 2: Set Sidekiq to prioritize 'urgent' queue

   Configure Sidekiq to check the 'urgent' queue before 'normal' to run urgent jobs first.

3. Final Answer:

   Configure Sidekiq to process the 'urgent' queue before the 'normal' queue -> Option B

4. Quick Check:

   Sidekiq queue order controls job priority [OK]

Hint: Set queue order in Sidekiq config for priority [OK]

Common Mistakes:

• Assuming queue_as alone controls priority
• Renaming job classes does not affect queue order
• Running urgent jobs manually is inefficient