Model documentation and model cards in MLOps - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to create a basic model card title.

MLOps

model_card = {"title": "[1]"}

Drag options to blanks, or click blank then click option'

AModel Info

BModel Documentation

CMy Model Card

DCard Title

Attempts:

3 left

💡 Hint

Common Mistakes

Using a generic term like 'Card Title' instead of a descriptive title.

The title field in a model card should clearly state the model name or purpose. "My Model Card" is a clear and simple title.

2fill in blank

medium

Complete the code to add a description field to the model card.

MLOps

model_card["description"] = "[1]"

Drag options to blanks, or click blank then click option'

AThis model predicts house prices.

BModel accuracy is high.

CVersion 1.0

DCreated by team

Attempts:

3 left

💡 Hint

Common Mistakes

Using metadata like version or creator instead of description.

The description should explain what the model does. "This model predicts house prices." clearly describes the model's purpose.

3fill in blank

hard

Fix the error in the model card dictionary to include the 'metrics' field correctly.

MLOps

model_card = {"title": "House Price Model", "metrics": [1]

Drag options to blanks, or click blank then click option'

A{"accuracy": 0.95, "rmse": 3.2}

B["accuracy", "precision"]

C"accuracy: 0.95, rmse: 3.2"

Daccuracy=0.95, rmse=3.2

Attempts:

3 left

💡 Hint

Common Mistakes

Using a list or string instead of a dictionary for metrics.

The metrics field should be a dictionary with metric names as keys and their values. Option A correctly uses a dictionary.

4fill in blank

hard

Fill both blanks to create a model card dictionary with 'version' and 'author' fields.

MLOps

model_card = {"version": [1], "author": "[2]"}

Drag options to blanks, or click blank then click option'

A1.0

B0.1

CAlice

DBob

Attempts:

3 left

💡 Hint

Common Mistakes

Putting author as a number or version as a name.

The version should be a string or number like '1.0'. The author is a string with the creator's name. Here, '1.0' and 'Alice' are correct.

5fill in blank

hard

Fill all three blanks to add 'license', 'date', and 'tags' fields to the model card.

MLOps

model_card.update({"license": "[1]", "date": "[2]", "tags": [3])

Drag options to blanks, or click blank then click option'

AMIT

B2024-05-01

C["regression", "housing"]

DGPL

Attempts:

3 left

💡 Hint

Common Mistakes

Using license as a list or date as a number.

The license is a string like 'MIT'. The date is a string in YYYY-MM-DD format. Tags is a list of strings describing the model. The correct combination is 'MIT', '2024-05-01', and the list of tags.

Practice

1. What is the main purpose of a model card in MLOps?

easy

A. To store the model's training data

B. To explain what a model does and how to use it safely

C. To deploy the model to production

D. To monitor the model's runtime performance

Solution

1. Step 1: Understand the role of model cards

   Model cards provide clear information about a model's purpose, performance, and safe use.

2. Step 2: Differentiate from other MLOps tasks

   Storing data, deployment, and monitoring are separate tasks not covered by model cards.

3. Final Answer:

   To explain what a model does and how to use it safely -> Option B

4. Quick Check:

   Model card purpose = Explain model use safely [OK]

Hint: Model cards describe model use and safety, not deployment [OK]

Common Mistakes:

• Confusing model cards with deployment tools
• Thinking model cards store training data
• Assuming model cards monitor runtime

2. Which section is NOT typically included in a model card?

easy

A. Model performance metrics

B. Ethical considerations and limitations

C. Source code for model training

D. Intended use and users

Solution

1. Step 1: Identify typical model card contents

   Model cards usually include performance, ethics, limitations, and intended use.

2. Step 2: Recognize what is excluded

   Source code is not part of the model card; it belongs in code repositories.

3. Final Answer:

   Source code for model training -> Option C

4. Quick Check:

   Model card excludes source code [OK]

Hint: Model cards describe, not contain source code [OK]

Common Mistakes:

• Including source code in model cards
• Confusing documentation with code repositories
• Ignoring ethical sections

3. Given this snippet from a model card:

"performance": {"accuracy": 0.92, "f1_score": 0.89},
"limitations": "Not tested on non-English data",
"ethical_considerations": "May reflect training data bias"

What does this information tell you about the model?

medium

A. The model is only for English data and has perfect fairness

B. The model has no ethical concerns and works on all languages

C. The model's accuracy is below 50%, so it is unreliable

D. The model is highly accurate but may not work well on non-English data

Solution

1. Step 1: Analyze performance metrics

   Accuracy 0.92 and F1 score 0.89 indicate good performance.

2. Step 2: Review limitations and ethics

   Limitations mention lack of testing on non-English data; ethics warn about bias.

3. Final Answer:

   The model is highly accurate but may not work well on non-English data -> Option D

4. Quick Check:

   Performance + limits = Accurate but language-limited [OK]

Hint: Check performance numbers and limitations for model scope [OK]

Common Mistakes:

• Ignoring limitations about language
• Assuming no ethical issues from bias note
• Misreading accuracy as low

4. You find a model card missing the "ethical considerations" section. What is the best way to fix this?

medium

A. Add a section describing potential biases and fairness issues

B. Remove the model card entirely since it is incomplete

C. Ignore it because ethics are not important for model cards

D. Replace it with just performance metrics

Solution

1. Step 1: Identify missing ethical info

   Ethical considerations help users understand risks and biases.

2. Step 2: Add relevant ethical details

   Include potential biases, fairness, and impact to complete the card.

3. Final Answer:

   Add a section describing potential biases and fairness issues -> Option A

4. Quick Check:

   Ethics section needed = Add bias/fairness info [OK]

Hint: Always include ethics to build trust and transparency [OK]

Common Mistakes:

• Deleting incomplete model cards
• Ignoring ethical importance
• Replacing ethics with only metrics

5. You want to create a model card for a new image classification model. Which combination of information should you include to ensure clear communication and trust?

hard

A. Purpose, performance metrics, limitations, ethical considerations, and intended users

B. Only the model's accuracy and training dataset size

C. The source code and deployment scripts

D. The hardware specifications used for training

Solution

1. Step 1: Identify key model card components

   Include purpose, performance, limits, ethics, and intended users for clarity.

2. Step 2: Exclude unrelated details

   Source code, deployment scripts, and hardware specs belong elsewhere.

3. Final Answer:

   Purpose, performance metrics, limitations, ethical considerations, and intended users -> Option A

4. Quick Check:

   Complete model card info = Purpose + performance + ethics + limits [OK]

Hint: Include purpose, performance, limits, ethics, users for trust [OK]

Common Mistakes:

• Including code or hardware in model cards
• Providing only metrics without context
• Ignoring ethical and user info