Kubeflow Pipelines overview in MLOps - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

What is the primary purpose of Kubeflow Pipelines?

Choose the best description of what Kubeflow Pipelines is mainly used for.

A To monitor server health and resource usage in Kubernetes clusters.

B To automate and manage machine learning workflows on Kubernetes.

C To deploy web applications using container orchestration.

D To store large datasets for machine learning training.

💡 Hint

Think about what Kubeflow Pipelines helps automate in ML projects.

Kubeflow Pipelines is designed to create, deploy, and manage end-to-end machine learning workflows on Kubernetes.

💻 Command Output

intermediate

Output of listing Kubeflow Pipelines runs

What is the expected output of the command kubectl get runs -n kubeflow if there are two pipeline runs named 'train-model' and 'data-prep'?

MLOps

kubectl get runs -n kubeflow

A

NAME          STATUS    AGE
train-model   Succeeded  2h
data-prep    Running    30m

B

NAME          STATUS    AGE
train-model   Running    2h
data-prep    Succeeded  30m

C

NAME          STATUS    AGE
train-model   Failed     2h
data-prep    Pending    30m

D

NAME          STATUS    AGE
train-model   Succeeded  2h
 data-prep    Running    30m

💡 Hint

Look carefully at the status and spacing in the output.

The output lists pipeline runs with their names, statuses, and ages. 'train-model' succeeded 2 hours ago, 'data-prep' is currently running for 30 minutes.

❓ Configuration

advanced

Correct YAML snippet for a Kubeflow Pipeline component

Which YAML snippet correctly defines a Kubeflow Pipeline component that runs a Python script with an input parameter?

A

apiVersion: v1
kind: Component
metadata:
  name: example-component
spec:
  implementation:
    container:
      image: python:3.8
      command: ["python", "script.py", "--input", "{{inputs.parameters.input_param}}"]
  inputs:
    - name: input_param

B

apiVersion: v1
kind: Component
metadata:
  name: example-component
spec:
  implementation:
    container:
      image: python:3.8
      command: ["python", "script.py", "--input", {input_param}]
  inputs:
    - name: input_param

C

apiVersion: v1
kind: Component
metadata:
  name: example-component
spec:
  implementation:
    container:
      image: python:3.8
      command: ["python", "script.py", "--input", input_param]
  inputs:
    - name: input_param

D

apiVersion: v1
kind: Component
metadata:
  name: example-component
spec:
  implementation:
    container:
      image: python:3.8
      command: ["python", "script.py", "--input", "$input_param"]
  inputs:
    - name: input_param

💡 Hint

Look for the correct syntax to reference input parameters in Kubeflow component YAML.

Kubeflow Pipelines uses the syntax {{inputs.parameters.}} to reference input parameters inside container commands.

🔀 Workflow

advanced

Order of steps in a Kubeflow Pipeline workflow

Arrange the following steps in the correct order for a typical Kubeflow Pipeline workflow:

A 3,1,2,4

B 2,1,3,4

C 1,3,2,4

D 1,2,3,4

💡 Hint

Think about how you build and run a pipeline step by step.

First, you define components, then compile the pipeline, upload it, and finally run and monitor it.

❓ Troubleshoot

expert

Identifying cause of pipeline run failure

A Kubeflow Pipeline run fails immediately with the error message: "Failed to pull image 'myregistry/myimage:latest': unauthorized: authentication required". What is the most likely cause?

A The container image does not exist in the registry.

B The pipeline YAML file has syntax errors causing the failure.

C The Kubernetes cluster nodes lack permission to access the private container registry.

D The pipeline components are missing required input parameters.

💡 Hint

Think about what "unauthorized: authentication required" means when pulling images.

This error usually means the cluster cannot authenticate to the private container registry to pull the image.

Practice

1. What is the main purpose of Kubeflow Pipelines in machine learning workflows?

easy

A. To store large datasets for training

B. To create user interfaces for ML models

C. To automate and manage ML workflows with clear, reusable steps

D. To replace Kubernetes as a container platform

Solution

1. Step 1: Understand Kubeflow Pipelines' role

   Kubeflow Pipelines are designed to automate ML workflows by defining clear steps that can be reused and tracked.

2. Step 2: Compare options with this role

   Options describing UI creation, data storage, and replacing Kubernetes do not match this role.

3. Final Answer:

   To automate and manage ML workflows with clear, reusable steps -> Option C

4. Quick Check:

   Automation of ML workflows [OK]

Hint: Kubeflow Pipelines automate ML steps, not UI or storage [OK]

Common Mistakes:

• Confusing Kubeflow Pipelines with data storage tools
• Thinking Kubeflow replaces Kubernetes
• Assuming it builds user interfaces

2. Which of the following is the correct way to define a step in a Kubeflow Pipeline using Python?

easy

A. def step():
return dsl.ContainerOp(name='step', image='python:3.8')

B. def step():
return dsl.ContainerOp(image='python:3.8')

C. def step():
return dsl.ContainerOp(name='step')

D. def step():
return dsl.ContainerOp(name='step', image='python:3.8', command=['python', 'script.py'])

Solution

1. Step 1: Understand ContainerOp usage

   ContainerOp requires at least a name, image, and usually a command to run the step's container.

2. Step 2: Check each option

   def step(): return dsl.ContainerOp(name='step', image='python:3.8', command=['python', 'script.py']) correctly includes name, image, and command. The version with name and image misses command. The version with only image misses name. The version with only name misses image and command.

3. Final Answer:

   def step():\n return dsl.ContainerOp(name='step', image='python:3.8', command=['python', 'script.py']) -> Option D

4. Quick Check:

   ContainerOp needs name, image, and command [OK]

Hint: ContainerOp needs name, image, and command to run [OK]

Common Mistakes:

• Omitting the command argument
• Not specifying the container image
• Missing the step name

3. Given this Kubeflow Pipeline snippet, what will be the output when the pipeline runs?

from kfp import dsl

@dsl.pipeline(name='Sample Pipeline')
def sample_pipeline():
    step1 = dsl.ContainerOp(
        name='echo-step',
        image='alpine',
        command=['echo', 'Hello Kubeflow']
    )

medium

A. The pipeline prints 'Hello Kubeflow' in the step logs

B. The pipeline fails because 'echo' is not a valid command

C. The pipeline prints 'Hello Kubeflow' on the console where pipeline is defined

D. The pipeline does nothing because no output is defined

Solution

1. Step 1: Understand ContainerOp execution

   The ContainerOp runs a container with the alpine image and executes the command 'echo Hello Kubeflow'. This prints to the container's standard output logs.

2. Step 2: Identify where output appears

   The output appears in the step logs inside Kubeflow Pipelines UI, not on the local console or nowhere.

3. Final Answer:

   The pipeline prints 'Hello Kubeflow' in the step logs -> Option A

4. Quick Check:

   ContainerOp command output = step logs [OK]

Hint: Container output appears in step logs, not local console [OK]

Common Mistakes:

• Thinking output appears on local console
• Assuming 'echo' command is invalid in alpine
• Believing pipeline does nothing without explicit output

4. You wrote this Kubeflow Pipeline step but it fails to run:

def step():
    return dsl.ContainerOp(name='step', image='python:3.8')

What is the most likely cause of the failure?

medium

A. Missing the command argument to specify what to run inside the container

B. The image 'python:3.8' does not exist

C. The step name cannot be 'step'

D. ContainerOp requires a volume to run

Solution

1. Step 1: Check ContainerOp requirements

   ContainerOp needs a command to run inside the container; without it, the container starts and exits immediately.

2. Step 2: Validate other options

   Image 'python:3.8' exists on Docker Hub, step name can be any string, and volume is optional.

3. Final Answer:

   Missing the command argument to specify what to run inside the container -> Option A

4. Quick Check:

   ContainerOp needs command to run [OK]

Hint: Always specify command in ContainerOp to avoid immediate exit [OK]

Common Mistakes:

• Assuming image is missing or invalid
• Thinking step name is restricted
• Believing volume is mandatory

5. You want to create a Kubeflow Pipeline that runs two steps sequentially: first preprocess data, then train a model. Which code snippet correctly defines this dependency?

hard

A. step1 = dsl.ContainerOp(name='preprocess', image='python:3.8', command=['python', 'preprocess.py'])
step2 = dsl.ContainerOp(name='train', image='python:3.8', command=['python', 'train.py'])
step1.after(step2)

B. step1 = dsl.ContainerOp(name='preprocess', image='python:3.8', command=['python', 'preprocess.py'])
step2 = dsl.ContainerOp(name='train', image='python:3.8', command=['python', 'train.py'])
step2.after(step1)

C. step1 = dsl.ContainerOp(name='preprocess', image='python:3.8', command=['python', 'preprocess.py'])
step2 = dsl.ContainerOp(name='train', image='python:3.8', command=['python', 'train.py'])
step1.before(step2)

D. step1 = dsl.ContainerOp(name='preprocess', image='python:3.8', command=['python', 'preprocess.py'])
step2 = dsl.ContainerOp(name='train', image='python:3.8', command=['python', 'train.py'])

Solution

1. Step 1: Understand step dependencies in Kubeflow Pipelines

   To run step2 after step1, use step2.after(step1) to set the order.

2. Step 2: Analyze each option

   step1 = dsl.ContainerOp(name='preprocess', image='python:3.8', command=['python', 'preprocess.py']) step2 = dsl.ContainerOp(name='train', image='python:3.8', command=['python', 'train.py']) step2.after(step1) correctly sets step2 to run after step1. Using step1.after(step2) reverses the order. Using step1.before(step2) calls a non-existent method. No dependency causes parallel execution.

3. Final Answer:

   step2.after(step1) -> Option B

4. Quick Check:

   Use step2.after(step1) for sequential steps [OK]

Hint: Use step2.after(step1) to run steps sequentially [OK]

Common Mistakes:

• Reversing the order with after()
• Using before() which does not exist
• Not setting any dependency causing parallel runs