Parameterized pipeline runs in MLOps - Step-by-Step Execution

Process Flow - Parameterized pipeline runs

Define pipeline with parameters

↓

Trigger pipeline run

↓

Pass parameter values

↓

Pipeline reads parameters

↓

Execute pipeline steps using parameters

↓

Pipeline completes with outputs based on parameters

This flow shows how a pipeline is defined with parameters, triggered with specific values, and runs using those values.

Execution Sample

MLOps

pipeline:
  parameters:
    - name: learning_rate
      type: float
  steps:
    - train_model:
        run: train.py
        args: --lr ${{ parameters.learning_rate }}

A pipeline definition with a learning_rate parameter passed to a training step.

Process Table

Step	Action	Parameter	Value Passed	Effect
1	Pipeline defined	learning_rate	float type, no value yet	Pipeline ready to accept parameter
2	Pipeline triggered	learning_rate	0.01	Parameter value set for this run
3	Step train_model starts	learning_rate	0.01	Training script receives --lr 0.01
4	Training executes	learning_rate	0.01	Model trains with learning rate 0.01
5	Pipeline completes	learning_rate	0.01	Outputs reflect training with 0.01
6	Pipeline triggered again	learning_rate	0.1	New run with different parameter
7	Step train_model starts	learning_rate	0.1	Training script receives --lr 0.1
8	Training executes	learning_rate	0.1	Model trains with learning rate 0.1
9	Pipeline completes	learning_rate	0.1	Outputs reflect training with 0.1

💡 Pipeline runs stop after all steps complete using passed parameters.

Status Tracker

Variable	Start	After Run 1	After Run 2	Final
learning_rate	undefined	0.01	0.1	0.1

Key Moments - 3 Insights

Why does the pipeline need parameters instead of hardcoded values?

Parameters let you run the same pipeline with different inputs without changing the code, as shown in rows 2 and 6 where different values are passed.

What happens if you trigger the pipeline without passing a parameter value?

The pipeline may use a default or fail; in this example, the parameter must be passed as shown in the execution table rows where values are explicitly set.

How does the training step know which parameter value to use?

The pipeline passes the parameter value to the step as an argument, seen in rows 3 and 7 where the training script receives the --lr argument.

Visual Quiz - 3 Questions

Look at the execution table, what is the learning_rate value passed at step 7?

A 0.1

B 0.01

C undefined

D 0.001

💡 Hint

Check the 'Value Passed' column at step 7 in the execution table.

At which step does the pipeline first complete using the parameter value?

A Step 6

B Step 4

C Step 5

D Step 9

💡 Hint

Look for the 'Pipeline completes' action in the execution table.

If you change the parameter learning_rate to 0.05 in the second run, which step would change?

A Step 2 and 3

B Step 7 and 8

C Step 1 only

D Step 9 only

💡 Hint

Parameter values are passed and used in the training step as shown in steps 7 and 8.

Concept Snapshot

Parameterized Pipeline Runs:
- Define parameters in pipeline YAML
- Trigger runs passing parameter values
- Pipeline steps use parameters as inputs
- Enables flexible runs without code changes
- Outputs depend on parameter values

Full Transcript

This visual execution shows how parameterized pipeline runs work in MLOps. First, a pipeline is defined with parameters like learning_rate. When triggered, the pipeline run receives specific values for these parameters. Each step in the pipeline reads the parameter values and uses them, for example passing --lr 0.01 to a training script. The pipeline completes and outputs results based on the parameter. Running the pipeline again with a different parameter value (like 0.1) repeats the process with new inputs. This allows flexible experimentation without changing pipeline code.