Point-in-time correctness in MLOps - Commands & Configuration

Introduction

Point-in-time correctness means making sure your machine learning model and data match exactly at the same moment. This helps avoid mistakes when you check or use your model later.

When you want to compare model results with the exact data used to train it.

When you need to reproduce a model's prediction exactly as it was made before.

When you want to audit or debug a model's behavior at a specific time.

When you deploy a model and want to ensure it uses the same data snapshot as during training.

When you track experiments and want to keep data and model versions aligned.

Commands

This command runs an MLflow project specifying the exact data version and model version to ensure point-in-time correctness.

Terminal

mlflow run . -P data_version=2024-06-01 -P model_version=1.0

Expected Output

2024/06/01 12:00:00 INFO mlflow.projects: === Run (ID='123abc') succeeded ===

→

-P - Passes parameters to specify exact data and model versions

Downloads the exact model artifacts from the run to verify or use the model matching the data snapshot.

Terminal

mlflow artifacts download -r 123abc -d ./downloaded_model

Expected Output

Successfully downloaded artifacts to: ./downloaded_model

Starts a local server to serve the downloaded model for testing or deployment, ensuring the model matches the point-in-time data.

Terminal

mlflow models serve -m ./downloaded_model --no-conda

Expected Output

2024/06/01 12:01:00 INFO mlflow.models: Serving model at http://127.0.0.1:5000

→

--no-conda - Avoids creating a new environment, using current setup

Key Concept

If you remember nothing else from this pattern, remember: always link your model version with the exact data snapshot to avoid mismatches.

Code Example

MLOps

import mlflow

# Log data version and model version as tags
with mlflow.start_run() as run:
    mlflow.set_tag("data_version", "2024-06-01")
    mlflow.set_tag("model_version", "1.0")
    # Log a simple metric
    mlflow.log_metric("accuracy", 0.95)
    print(f"Run ID: {run.info.run_id} logged with point-in-time correctness tags")

Common Mistakes

Using the latest model without specifying the data version

This causes the model to be tested or deployed with data it was not trained on, leading to wrong results.

Always specify both model and data versions together when running or deploying.

Not downloading the exact model artifacts before serving

Serving a different or outdated model can cause inconsistent predictions.

Download and serve the model artifacts from the exact run that used the matching data.

Summary

Run MLflow projects specifying exact data and model versions to keep them aligned.

Download model artifacts from the specific run to verify or deploy the correct model.

Serve the downloaded model to ensure predictions match the data snapshot used during training.