Bird
Raised Fist0

Model Hub and Model Registry - Versioning, Lineage, Governance

SYSTEM-DESIGNhardInterview freq: high
Choose your preparation mode6 modes available
LearnTraceDebugTrade-offChallenge
▶
Execution Trace
Step 1/5
Data Ingest

Raw data is ingested and tagged with version ID 'v1.0'.

🎯 Understanding data versioning is crucial for reproducibility.
data = ingest_data(source)
data_version = tag_version(data, 'v1.0')
Feat Eng

Features are extracted and stored with version ID 'v1.0-feat'.

🎯 Feature versioning allows tracking of feature changes over time.
features = extract_features(data)
feat_version = tag_version(features, 'v1.0-feat')
Model Train

Model is trained and saved with version ID 'v1.0-model'.

🎯 Model versioning is essential for tracking model performance.
model = train_model(features)
model_version = save_model(model, 'v1.0-model')
Model Hub

Model version 'v1.0-model' is stored in the model hub.

🎯 Centralized model storage facilitates easy access and deployment.
model_hub.store(model_version)
Model Serve

Model 'v1.0-model' is deployed for serving.

🎯 Serving the correct model version is critical for consistent results.
serve_model(model_hub.retrieve('v1.0-model'))
📊
Diagram
Diagram
┌──────────────┐     ┌──────────────┐     ┌──────────────┐
│ Data Ingest  │ ──▶│ Feat Eng     │ ──▶│ Model Train  │
└──────────────┘     └──────────────┘     └──────────────┘
       │                    │                    │
       ▼                    ▼                    ▼
┌──────────────┐     ┌──────────────┐     ┌──────────────┐
│ Data Version │     │ Feat Version │     │ Model Version│
└──────────────┘     └──────────────┘     └──────────────┘
       │                    │                    │
       ▼                    ▼                    ▼
┌──────────────┐     ┌──────────────┐     ┌──────────────┐
│ Model Hub    │ ◀── │ Registry     │ ◀── │ Governance   │
└──────────────┘     └──────────────┘     └──────────────┘
       │
       ▼
┌──────────────┐
│ Model Serve  │
└──────────────┘
Labels
  • Data Ingest: Entry point for raw data
  • Feat Eng: Transformation of raw data into features
  • Model Train: Training models with version control
  • Model Hub: Centralized storage for model versions
  • Governance: Ensures compliance and traceability
🔑
Key Moments
▲
Q: Why is data versioning important in this pipeline?
A: Data versioning ensures that the exact dataset used for training can be retrieved, enabling reproducibility and debugging.
Q: How does the model hub facilitate governance?
A: The model hub centralizes model storage, allowing for easy tracking of model versions and ensuring compliance with governance policies.
Q: What role does feature versioning play in model training?
A: Feature versioning allows tracking of changes in feature engineering, which can impact model performance and comparability.
Trace Summary

This diagram illustrates a comprehensive ML pipeline from data ingestion to model serving, emphasizing the importance of versioning, lineage, and governance. Data is ingested and tagged with a version ID, ensuring reproducibility. Features are extracted and versioned, allowing for tracking changes over time. Models are trained and stored in a centralized model hub, facilitating easy access and deployment. Governance ensures compliance and traceability throughout the process. Understanding these components is crucial for building robust and maintainable ML systems.

Primary Focus

✓ ML pipeline from data ingestion to serving with versioning and governance

Practice

(1/5)
1. For an LLM serving system focused on minimizing response time, which metric should be prioritized?
easy
A. Latency
B. Memory Usage
C. Accuracy
D. Throughput

Solution

  1. Step 1: Identify the goal.

    Minimizing response time directly relates to reducing latency.

  2. Step 2: Eliminate wrong options.

    Throughput measures volume, not speed. Accuracy is about correctness, not speed. Memory Usage affects capacity, not response time.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Minimize latency to cut response time. [OK]
Hint: Minimize latency to cut response time. [OK]
Common Mistakes:
2. In the context of recommendation systems, which of the following best describes collaborative filtering?
easy
A. Using user-item interaction data to recommend items.
B. Using user demographic data to recommend items.
C. Using item metadata to recommend similar items.
D. Using content-based features to recommend items.

Solution

  1. Step 1: Understand collaborative filtering.

    Collaborative filtering relies on user-item interaction data to make recommendations.

  2. Step 2: Eliminate wrong options.

    Options A, B, and D describe content-based or demographic-based approaches, not collaborative filtering.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Collaborative = user-item interactions [OK]
Hint: Collaborative = user-item interactions [OK]
Common Mistakes:
3. As the user base scales 10x, which component of a DLRM-based CTR prediction system is most likely to become a bottleneck?
hard
A. Data preprocessing pipeline
B. Training data storage capacity
C. Model inference latency
D. Embedding table size

Solution

  1. Step 1: Consider scaling effects on components.

    Embedding tables grow with user and feature diversity.

  2. Step 2: Eliminate wrong options.

    Preprocessing (A) and inference (B) scale with compute resources. Storage (D) is less critical than real-time constraints.

  3. Final Answer:

    Option D -> Option D

  4. Quick Check:

    Embedding tables grow with scale. [OK]
Hint: Embedding tables grow with scale. [OK]
Common Mistakes:
4. As the number of users and items increases in a session-based recommendation system, which component is likely to become a bottleneck first?
hard
A. Model training time
B. Data storage capacity
C. Real-time inference latency
D. Feature extraction pipeline

Solution

  1. Step 1: Consider scaling impacts

    Real-time inference latency often becomes a bottleneck as the system scales, due to increased computational demands.

  2. Step 2: Eliminate less likely bottlenecks

    Model training time and data storage can be managed with batch processing and scalable storage solutions; feature extraction is typically pre-computed.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Inference latency scales poorly. [OK]
Hint: Inference latency scales poorly. [OK]
Common Mistakes:
5. As the volume of emails increases 10x, which component of a spam detection system is most likely to fail first?
hard
A. Feature extraction pipeline
B. Model training process
C. Real-time inference engine
D. Data storage system

Solution

  1. Step 1: Consider scalability of components

    Real-time inference engines often struggle with increased load due to latency constraints.

  2. Step 2: Eliminate components less affected by scale

    Feature extraction (A) and data storage (D) can be scaled horizontally. Model training (B) is not real-time.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Inference engines struggle with scale. [OK]
Hint: Inference engines struggle with scale. [OK]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard