Model documentation and model cards in MLOps - Commands & Configuration

Introduction

When you create a machine learning model, you need to explain what it does and how to use it safely. Model documentation and model cards help share this information clearly so others can understand and trust your model.

When you want to share your model with teammates or other teams so they know its purpose and limits

When you need to record how your model was trained and tested for future reference

When you want to be transparent about your model’s strengths and weaknesses to avoid misuse

When you prepare your model for deployment and want to include usage instructions

When you want to comply with company or legal requirements for AI transparency

Commands

This command starts a local server to serve the ML model saved in MLflow. It lets you test the model and document its behavior by sending requests.

Terminal

mlflow models serve -m runs:/1234567890abcdef/model --no-conda -p 1234

Expected Output

2024/06/01 12:00:00 INFO mlflow.models.cli: Starting MLflow model server...

2024/06/01 12:00:00 INFO mlflow.models.cli: Listening on port 1234

→

-m - Specifies the model path to serve

→

--no-conda - Avoids creating a new conda environment for faster startup

→

-p - Sets the port number for the server

This command sends a test input to the running model server to get a prediction. It helps verify the model’s output matches expectations for documentation.

Terminal

curl -X POST -H 'Content-Type:application/json' --data '{"data": [[5.1, 3.5, 1.4, 0.2]]}' http://127.0.0.1:1234/invocations

Expected Output

{"predictions": [0]}

→

-X POST - Sends data to the server

→

-H 'Content-Type:application/json' - Sets the data format

→

--data - Provides the input features for prediction

This command launches the MLflow tracking UI in your browser. You can view model runs, parameters, metrics, and add notes to document your model clearly.

Terminal

mlflow ui

Expected Output

2024/06/01 12:05:00 INFO mlflow.server: Starting MLflow UI at http://127.0.0.1:5000

This command builds a Docker image for your ML model. It packages the model and its environment so others can run it easily with documentation included.

Terminal

mlflow models build-docker -m runs:/1234567890abcdef/model -n my-model-image

Expected Output

Successfully built image my-model-image

→

-m - Specifies the model path to package

→

-n - Names the Docker image

Key Concept

If you remember nothing else, remember: clear model documentation and cards make your model easy to understand, trust, and reuse.

Code Example

MLOps

import mlflow
from mlflow.models import Model

# Log a simple metric and parameter
mlflow.start_run()
mlflow.log_param("model_type", "decision_tree")
mlflow.log_metric("accuracy", 0.87)
mlflow.end_run()

print("Model run logged with parameters and metrics.")

Common Mistakes

Skipping model documentation and only sharing the model file

Others won’t know how to use the model correctly or understand its limits, causing errors or misuse

Always create a model card or documentation that explains the model’s purpose, data, performance, and usage

Not testing the model server with real inputs before sharing

You might share a broken or misconfigured model that fails in production

Run the model server locally and send test requests to verify outputs before deployment

Ignoring the MLflow UI and tracking features for documentation

You lose valuable history and context about model training and evaluation

Use MLflow UI to log and view model runs, parameters, and notes for clear documentation

Summary

Use mlflow models serve to start a local server for your model to test and document it.

Send test inputs with curl to verify the model’s predictions before sharing.

Launch mlflow ui to view and document model runs, parameters, and metrics.

Build a Docker image with mlflow models build-docker to package your model for easy sharing.

Practice

1. What is the main purpose of a model card in MLOps?

easy

A. To store the model's training data

B. To explain what a model does and how to use it safely

C. To deploy the model to production

D. To monitor the model's runtime performance

Solution

1. Step 1: Understand the role of model cards

   Model cards provide clear information about a model's purpose, performance, and safe use.

2. Step 2: Differentiate from other MLOps tasks

   Storing data, deployment, and monitoring are separate tasks not covered by model cards.

3. Final Answer:

   To explain what a model does and how to use it safely -> Option B

4. Quick Check:

   Model card purpose = Explain model use safely [OK]

Hint: Model cards describe model use and safety, not deployment [OK]

Common Mistakes:

• Confusing model cards with deployment tools
• Thinking model cards store training data
• Assuming model cards monitor runtime

2. Which section is NOT typically included in a model card?

easy

A. Model performance metrics

B. Ethical considerations and limitations

C. Source code for model training

D. Intended use and users

Solution

1. Step 1: Identify typical model card contents

   Model cards usually include performance, ethics, limitations, and intended use.

2. Step 2: Recognize what is excluded

   Source code is not part of the model card; it belongs in code repositories.

3. Final Answer:

   Source code for model training -> Option C

4. Quick Check:

   Model card excludes source code [OK]

Hint: Model cards describe, not contain source code [OK]

Common Mistakes:

• Including source code in model cards
• Confusing documentation with code repositories
• Ignoring ethical sections

3. Given this snippet from a model card:

"performance": {"accuracy": 0.92, "f1_score": 0.89},
"limitations": "Not tested on non-English data",
"ethical_considerations": "May reflect training data bias"

What does this information tell you about the model?

medium

A. The model is only for English data and has perfect fairness

B. The model has no ethical concerns and works on all languages

C. The model's accuracy is below 50%, so it is unreliable

D. The model is highly accurate but may not work well on non-English data

Solution

1. Step 1: Analyze performance metrics

   Accuracy 0.92 and F1 score 0.89 indicate good performance.

2. Step 2: Review limitations and ethics

   Limitations mention lack of testing on non-English data; ethics warn about bias.

3. Final Answer:

   The model is highly accurate but may not work well on non-English data -> Option D

4. Quick Check:

   Performance + limits = Accurate but language-limited [OK]

Hint: Check performance numbers and limitations for model scope [OK]

Common Mistakes:

• Ignoring limitations about language
• Assuming no ethical issues from bias note
• Misreading accuracy as low

4. You find a model card missing the "ethical considerations" section. What is the best way to fix this?

medium

A. Add a section describing potential biases and fairness issues

B. Remove the model card entirely since it is incomplete

C. Ignore it because ethics are not important for model cards

D. Replace it with just performance metrics

Solution

1. Step 1: Identify missing ethical info

   Ethical considerations help users understand risks and biases.

2. Step 2: Add relevant ethical details

   Include potential biases, fairness, and impact to complete the card.

3. Final Answer:

   Add a section describing potential biases and fairness issues -> Option A

4. Quick Check:

   Ethics section needed = Add bias/fairness info [OK]

Hint: Always include ethics to build trust and transparency [OK]

Common Mistakes:

• Deleting incomplete model cards
• Ignoring ethical importance
• Replacing ethics with only metrics

5. You want to create a model card for a new image classification model. Which combination of information should you include to ensure clear communication and trust?

hard

A. Purpose, performance metrics, limitations, ethical considerations, and intended users

B. Only the model's accuracy and training dataset size

C. The source code and deployment scripts

D. The hardware specifications used for training

Solution

1. Step 1: Identify key model card components

   Include purpose, performance, limits, ethics, and intended users for clarity.

2. Step 2: Exclude unrelated details

   Source code, deployment scripts, and hardware specs belong elsewhere.

3. Final Answer:

   Purpose, performance metrics, limitations, ethical considerations, and intended users -> Option A

4. Quick Check:

   Complete model card info = Purpose + performance + ethics + limits [OK]

Hint: Include purpose, performance, limits, ethics, users for trust [OK]

Common Mistakes:

• Including code or hardware in model cards
• Providing only metrics without context
• Ignoring ethical and user info