Multi-tenancy and isolation in MLOps - Commands & Configuration

Introduction

When many users or teams share the same machine learning platform, their work must stay separate and safe. Multi-tenancy and isolation help keep each user's data and models private and prevent interference.

When multiple data science teams use the same ML platform but need their projects separated.

When running different ML experiments on shared hardware without affecting each other.

When deploying models for different clients on the same server but ensuring data privacy.

When you want to limit resource use per user to avoid one user slowing down others.

When managing access control so users only see their own models and data.

Commands

Start the MLflow tracking server with a local SQLite database and artifact storage. This server will handle multiple users' experiments in isolated runs.

Terminal

mlflow server --backend-store-uri sqlite:///mlflow.db --default-artifact-root ./mlruns --host 0.0.0.0 --port 5000

Expected Output

2024/06/01 12:00:00 INFO mlflow.server: Starting MLflow tracking server

2024/06/01 12:00:00 INFO mlflow.store.db.utils: Creating initial MLflow database tables...

2024/06/01 12:00:00 INFO mlflow.server: Listening at: http://0.0.0.0:5000

→

--backend-store-uri - Sets the database for experiment metadata

→

--default-artifact-root - Sets where model files and artifacts are stored

→

--host - Makes the server accessible on all network interfaces

Create a new experiment named 'team_alpha_project' to isolate this team's runs and data from others.

Terminal

mlflow experiments create --experiment-name team_alpha_project

Expected Output

Created experiment with ID 1

Run an MLflow project under the 'team_alpha_project' experiment to keep its runs separate from other teams.

Terminal

mlflow run . --experiment-name team_alpha_project

Expected Output

2024/06/01 12:05:00 INFO mlflow.projects: Running run with ID '1234567890abcdef'

2024/06/01 12:05:10 INFO mlflow.projects: Run succeeded

→

--experiment-name - Specifies which experiment to log this run under

List all experiments to verify that each team has its own isolated experiment space.

Terminal

mlflow experiments list

Expected Output

experiment_id  name               artifact_location

1              team_alpha_project  ./mlruns/1

2              team_beta_project   ./mlruns/2

Key Concept

If you remember nothing else, remember: separate experiments and artifact storage keep each user's ML work private and isolated on a shared platform.

Common Mistakes

Running all experiments under the default experiment without naming separate ones

This mixes all users' runs together, making it hard to separate or secure data per user.

Always create and use named experiments for each team or user to isolate their runs.

Using the same artifact storage path for all users without subfolders

Artifacts can overwrite each other or be accessible to unauthorized users.

Use separate artifact root paths or subfolders per experiment or user.

Summary

Start the MLflow server with a shared backend and artifact storage.

Create named experiments to isolate each team's or user's runs.

Run ML projects specifying the experiment to keep data separated.

List experiments to verify isolation and organization.

Practice

1. What is the main purpose of multi-tenancy in MLOps platforms?

easy

A. To speed up model training by using multiple GPUs

B. To store all data in a single shared database without restrictions

C. To allow multiple users to share the same system safely

D. To run only one user's workload at a time

Solution

1. Step 1: Understand multi-tenancy concept

   Multi-tenancy means many users share one system but remain separate and safe.

2. Step 2: Identify the correct purpose

   The goal is to let users share resources without interfering with each other.

3. Final Answer:

   To allow multiple users to share the same system safely -> Option C

4. Quick Check:

   Multi-tenancy = safe shared use [OK]

Hint: Multi-tenancy means safe sharing, not exclusive use [OK]

Common Mistakes:

• Confusing multi-tenancy with faster hardware use
• Thinking all data is mixed without separation
• Believing only one user runs at a time

2. Which configuration snippet correctly isolates tenant data in a Kubernetes namespace?

easy

A. apiVersion: v1
kind: ConfigMap
metadata:
name: tenant-a-config

B. apiVersion: v1
kind: Namespace
metadata:
name: tenant-a

C. apiVersion: v1
kind: Service
metadata:
name: tenant-a-service

D. apiVersion: v1
kind: Pod
metadata:
name: tenant-a-pod

Solution

1. Step 1: Identify resource for tenant isolation

   Kubernetes namespaces isolate resources per tenant.

2. Step 2: Match correct YAML kind

   Namespace kind with tenant name isolates tenant data correctly.

3. Final Answer:

   apiVersion: v1 kind: Namespace metadata: name: tenant-a -> Option B

4. Quick Check:

   Namespace = tenant isolation [OK]

Hint: Namespaces isolate tenants, not pods or services alone [OK]

Common Mistakes:

• Choosing Pod or Service which do not isolate tenants
• Confusing ConfigMap with isolation resource
• Missing correct YAML syntax for namespaces

3. Given this code snippet for tenant isolation in a shared ML platform, what will be the output?

tenants = {"tenant1": {"models": ["modelA"]}, "tenant2": {"models": ["modelB"]}}

for tenant, data in tenants.items():
    print(f"{tenant} has models: {', '.join(data['models'])}")

medium

A. tenant1 has models: modelA
tenant2 has models: modelB

B. tenant1 has models: modelB
tenant2 has models: modelA

C. tenant1 has models:
tenant2 has models:

D. Error: KeyError

Solution

1. Step 1: Understand dictionary structure

   Each tenant key maps to a dict with a 'models' list.

2. Step 2: Loop and print models per tenant

   Loop prints tenant name and joins model names correctly.

3. Final Answer:

   tenant1 has models: modelA tenant2 has models: modelB -> Option A

4. Quick Check:

   Correct tenant-model mapping printed [OK]

Hint: Check keys and values carefully in dict loops [OK]

Common Mistakes:

• Swapping models between tenants
• Printing empty model lists
• Mistyping keys causing KeyError

4. You have this Kubernetes YAML snippet meant to isolate tenant workloads:

apiVersion: v1
kind: Namespace
metadata:
  name: tenant1
---
apiVersion: v1
kind: Pod
metadata:
  name: tenant1-pod
  namespace: tenant2
spec:
  containers:
  - name: app
    image: ml-app:latest

What is the main issue here?

medium

A. Pod is assigned to a different namespace than tenant's namespace

B. Namespace name is invalid

C. Container image name is incorrect

D. Pod spec is missing container ports

Solution

1. Step 1: Check namespace assignment

   Pod metadata namespace is 'tenant2' but tenant namespace defined is 'tenant1'.

2. Step 2: Identify isolation problem

   Pod runs in wrong namespace, breaking tenant isolation.

3. Final Answer:

   Pod is assigned to a different namespace than tenant's namespace -> Option A

4. Quick Check:

   Namespace mismatch breaks isolation [OK]

Hint: Pod namespace must match tenant namespace exactly [OK]

Common Mistakes:

• Ignoring namespace mismatch
• Assuming image name causes error
• Thinking missing ports cause isolation failure

5. In a multi-tenant MLOps platform, you want to ensure that tenant A's models cannot access tenant B's data. Which combination of strategies best achieves this?

hard

A. Run all tenant workloads on the same node without resource limits

B. Store all tenant data in one database and rely on application code to separate access

C. Allow tenants to share the same service account for simplicity

D. Use separate namespaces for each tenant and enforce RBAC policies limiting access

Solution

1. Step 1: Understand isolation requirements

   Tenant data must be separated and access controlled to prevent leaks.

2. Step 2: Identify best isolation methods

   Namespaces isolate resources; RBAC controls who can access what.

3. Step 3: Evaluate options

   Only Use separate namespaces for each tenant and enforce RBAC policies limiting access uses both namespaces and RBAC for strong isolation.

4. Final Answer:

   Use separate namespaces for each tenant and enforce RBAC policies limiting access -> Option D

5. Quick Check:

   Namespaces + RBAC = strong tenant isolation [OK]

Hint: Combine namespaces and RBAC for secure tenant isolation [OK]

Common Mistakes:

• Relying only on application code for data separation
• Sharing service accounts across tenants
• Ignoring resource limits and node sharing risks