Pipeline scheduling and triggers in MLOps - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Understanding pipeline trigger types

Which pipeline trigger type will start a pipeline automatically when new code is pushed to the main branch?

A Webhook trigger

B Scheduled trigger

C Manual trigger

D Pull request trigger

💡 Hint

Think about what happens immediately after code changes are pushed.

A webhook trigger listens for events like code pushes and starts the pipeline automatically. Scheduled triggers run pipelines at set times. Manual triggers require user action. Pull request triggers start pipelines on PR events.

💻 Command Output

intermediate

Output of a cron schedule in pipeline config

Given this cron schedule in a pipeline YAML config, what is the next run time if today is Monday 10:00 AM?

schedule: '0 12 * * 1'

A Today at 12:00 PM

B Next Monday at 12:00 PM

C Tomorrow at 12:00 PM

D Today at 10:00 AM

💡 Hint

Check what the cron expression means: minute hour day month weekday.

The cron '0 12 * * 1' means every Monday at 12:00 PM. If today is Monday 10:00 AM, the next run is today at 12:00 PM.

🔀 Workflow

advanced

Pipeline trigger order with multiple triggers

A pipeline has both a scheduled trigger set for daily 2 AM and a webhook trigger on code push. If code is pushed at 1:50 AM, which triggers will start the pipeline and in what order?

A Only webhook trigger starts pipeline immediately at 1:50 AM

B Scheduled trigger runs at 2 AM, webhook ignored

C Webhook trigger starts pipeline at 1:50 AM, then scheduled trigger runs again at 2 AM

D Pipeline runs only once at 2 AM combining both triggers

💡 Hint

Consider if triggers are independent or if one cancels the other.

Webhook triggers start the pipeline immediately on code push. Scheduled triggers run at their set time regardless. Both triggers cause separate runs.

❓ Troubleshoot

advanced

Troubleshooting pipeline not triggering on code push

A pipeline with a webhook trigger does not start when code is pushed. Which is the most likely cause?

A The code push was to a branch not monitored by the webhook

B The pipeline is scheduled to run only at midnight

C The pipeline YAML file has syntax errors

D The webhook URL is incorrect or missing in the code repository settings

💡 Hint

Check the connection between the code repository and the pipeline system.

If the webhook URL is wrong or missing, the code repository cannot notify the pipeline system to start. Scheduled triggers and YAML errors do not affect webhook triggers directly. Branch filters can also cause issues but are less common than webhook misconfiguration.

✅ Best Practice

expert

Best practice for combining scheduled and event triggers

What is the best practice when configuring a pipeline with both scheduled and event-based triggers to avoid redundant runs?

A Disable one trigger type to avoid any overlap

B Use a flag file or state check in the pipeline to skip runs if recent event-triggered run occurred

C Set both triggers to run at the exact same time

D Ignore redundancy; pipelines can run multiple times without issues

💡 Hint

Think about how to prevent running the same work twice unnecessarily.

Using a flag or state check allows the pipeline to detect if a recent run already processed the latest changes, avoiding duplicate work. Disabling triggers or running simultaneously is not efficient. Ignoring redundancy can waste resources.

Practice

1. What is the main purpose of pipeline scheduling in MLOps?

easy

A. To store pipeline logs for debugging

B. To manually start pipelines whenever needed

C. To run tasks automatically at specific times without manual intervention

D. To create new machine learning models from scratch

Solution

1. Step 1: Understand pipeline scheduling

   Pipeline scheduling is designed to run tasks automatically at set times, like daily or hourly, without needing a person to start them.

2. Step 2: Compare options

   Only To run tasks automatically at specific times without manual intervention describes automatic running at specific times. Other options describe manual actions or unrelated tasks.

3. Final Answer:

   To run tasks automatically at specific times without manual intervention -> Option C

4. Quick Check:

   Pipeline scheduling = automatic timed runs [OK]

Hint: Scheduling means automatic runs at set times [OK]

Common Mistakes:

• Confusing scheduling with manual triggering
• Thinking scheduling stores logs
• Assuming scheduling creates models directly

2. Which of the following is a correct cron expression to schedule a pipeline to run every day at 3 AM?

easy

A. 3 0 * * *

B. 0 3 * * *

C. * 3 * * *

D. 0 0 3 * * *

Solution

1. Step 1: Understand cron format

   Cron syntax is: minute hour day month weekday. To run at 3 AM daily, minute=0, hour=3, day/month/weekday=any (*).

2. Step 2: Match expression

   0 3 * * * "0 3 * * *" means minute 0, hour 3, every day. Others have wrong order or extra fields.

3. Final Answer:

   0 3 * * * -> Option B

4. Quick Check:

   Minute=0, Hour=3 daily = 0 3 * * * [OK]

Hint: Cron: minute hour day month weekday; 3 AM is '0 3 * * *' [OK]

Common Mistakes:

• Swapping hour and minute fields
• Adding extra fields in cron
• Using '*' in wrong positions

3. Given this pipeline trigger configuration snippet:

{
  "trigger": {
    "event": "data_arrival",
    "filter": {
      "file_type": "csv"
    }
  }
}

What happens when a new JSON file arrives in the data folder?

medium

A. The pipeline does not run because the file type is not CSV

B. The pipeline runs because any new file triggers it

C. The pipeline runs only if the JSON file is large

D. The pipeline runs but ignores the file type

Solution

1. Step 1: Analyze trigger filter

   The trigger listens for 'data_arrival' events but only runs if the file type is 'csv'.

2. Step 2: Apply to JSON file

   A JSON file does not match the 'csv' filter, so the pipeline will not run.

3. Final Answer:

   The pipeline does not run because the file type is not CSV -> Option A

4. Quick Check:

   Filter file_type=csv blocks JSON files [OK]

Hint: Triggers with filters run only on matching events [OK]

Common Mistakes:

• Ignoring filter conditions
• Assuming any file triggers pipeline
• Confusing event type with file type

4. You wrote this cron expression to schedule a pipeline every hour:

60 * * * *

Why does the pipeline never run?

medium

A. Because the hour field is missing

B. Because cron requires seconds field

C. Because the asterisks are misplaced

D. Because 60 is not a valid minute value in cron syntax

Solution

1. Step 1: Check minute field validity

   Cron minute values must be 0-59. '60' is invalid and causes no runs.

2. Step 2: Confirm other fields

   The hour and other fields are correct as '*', meaning every hour/day. The error is only the minute value.

3. Final Answer:

   Because 60 is not a valid minute value in cron syntax -> Option D

4. Quick Check:

   Minute must be 0-59; 60 is invalid [OK]

Hint: Minutes in cron go 0-59, never 60 [OK]

Common Mistakes:

• Using 60 as minute value
• Thinking cron needs seconds field
• Misplacing asterisks

5. You want a pipeline to run automatically when new data arrives and also every Sunday at midnight. Which setup correctly combines scheduling and event triggers?

hard

A. Use a cron schedule '0 0 * * 0' and an event trigger for 'data_arrival' together

B. Use only a cron schedule '0 0 * * 0' because event triggers conflict with schedules

C. Use only an event trigger for 'data_arrival' and manually run on Sundays

D. Use a cron schedule '0 0 * * 7' and ignore event triggers

Solution

1. Step 1: Understand combined triggers

   Pipelines can have both cron schedules and event triggers to run on different conditions.

2. Step 2: Verify cron expression for Sunday midnight

   '0 0 * * 0' runs at midnight on Sundays (0 or 7 can represent Sunday, but 0 is standard).

3. Step 3: Confirm event trigger for data arrival

   Adding an event trigger for 'data_arrival' ensures pipeline runs when new data arrives.

4. Final Answer:

   Use a cron schedule '0 0 * * 0' and an event trigger for 'data_arrival' together -> Option A

5. Quick Check:

   Combine cron and event triggers for full automation [OK]

Hint: Combine cron and event triggers for multiple run conditions [OK]

Common Mistakes:

• Thinking schedules and triggers cannot coexist
• Using wrong cron day for Sunday
• Ignoring event triggers for data arrival