Bird
Raised Fist0

Design a Multi-Tenant SaaS Platform - Isolation, Billing, Feature Flags

HLDhardInterview freq: high
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
🏗️
Design a Multi-Tenant SaaS Platform - Isolation, Billing, Feature Flags
Design a scalable multi-tenant SaaS platform ensuring tenant isolation, accurate billing, and dynamic feature flag management at 2B DAU scale.
hardModern Infrastructure DesignFreq: high
Prerequisites:System Design FundamentalsCAP TheoremMulti-Tenancy PatternsDistributed Billing Systems
⚙️
Functional Requirements
  • Support strict tenant isolation for data, compute, and configuration to prevent cross-tenant data leakage.
  • Implement usage-based billing with real-time metering and batch reconciliation for accuracy.
  • Provide dynamic feature flag management per tenant to enable/disable features without redeployments.
  • Allow tenant onboarding, configuration, and lifecycle management via APIs and UI.
  • Support multi-region deployment with data locality per tenant for compliance.
  • Expose audit logs and usage reports for tenants and internal monitoring.
📊
Non-Functional Requirements
  • Support 2 billion daily active users (DAU) distributed across 100,000 tenants.
  • Handle peak write throughput of 925K events/sec (derived from usage assumptions).
  • Ensure 99.99% platform availability with <100ms P99 latency for API calls.
  • Data retention for billing and audit logs for at least 1 year with GDPR compliance.
  • Feature flag evaluation latency under 10ms P99 to avoid user impact.
❓
Clarifying Questions to Ask
  • What level of tenant isolation is required: shared DB with row-level isolation, schema-level, or separate DB instances?
  • What is the expected average usage per tenant and peak usage distribution?
  • Are feature flags global or can they be overridden per tenant or user?
  • What billing granularity is required: per API call, per feature usage, or per resource consumption?
  • Is multi-region deployment mandatory or optional for compliance?
  • What are the SLAs for billing accuracy and dispute resolution?
Out of Scope
✗ Designing the SaaS application business logic beyond platform-level features.
✗ Tenant-specific data schema design or application-level data modeling.
✗ Payment gateway integration and external financial reconciliation.
✗ Detailed UI/UX design for tenant management portals.
📌
Assumptions
  • 2 billion DAU distributed over 100,000 tenants (average 20K users/tenant).
  • Each user generates 40 usage events per day (e.g., API calls, feature toggles).
  • Average event size is 100 bytes (includes metadata for billing and audit).
  • Billing data retained for 365 days with 3x replication for durability.
⚡
QPS Calculation
write qps2,000,000,000 users × 40 events/day ÷ 86,400 sec/day ≈ 925,926 events/sec
peak write qpsAssuming 3x peak factor: 925,926 × 3 ≈ 2,777,778 events/sec
read qpsAssuming 5 reads per write event for dashboards and feature flag checks: 925,926 × 5 ≈ 4,629,630 reads/sec
💾
Storage Estimate
daily storage2,000,000,000 × 40 × 100 bytes = 8,000,000,000,000 bytes = 8 TB/day
retention storage8 TB/day × 365 days = 2,920 TB ≈ 2.9 PB
with replication2.9 PB × 3 replicas = 8.7 PB total storage
🖥️
Server Estimate
api serversAssuming each API server handles 5,000 QPS, need 2,777,778 ÷ 5,000 ≈ 556 API servers for peak writes
feature flag evaluationAssuming Redis cluster nodes handle 50,000 reads/sec each, need 4,629,630 ÷ 50,000 ≈ 93 Redis nodes
🏗️
Architecture Diagram
Architecture Diagram
          +----------------+
          | Tenant Onboard |
          +-------+--------+
                  |
                  v
          +-------+--------+
          | API Gateway    |
          +-------+--------+
                  |
    +-------------+-------------+
    |                           |
    v                           v
 +--+--+                   +----+----+
 | Auth |                   | Feature |
 | Svc  |                   | Flag    |
 +--+---+                   | Svc     |
    |                       +----+----+
    |                            |
    |                            v
    |                     +------+-------+
    |                     | Redis Cluster|
    |                     +------+-------+
    |                            |
    |                            v
    |                     +------+-------+
    |                     | Billing Svc  |
    |                     +------+-------+
    |                            |
    |                            v
    |                     +------+-------+
    |                     | Kafka Topic  |
    |                     +------+-------+
    |                            |
    |                            v
    |                     +------+-------+
    |                     | Data Warehouse|
    |                     +--------------+
    |
    v
+---+---+
| Tenant |
| DB     |
+-------+
🧩
Components
API Gateway
Entry point for all tenant API requests, routing to appropriate services with tenant context.
Why thisEnvoy proxy supports multi-tenant routing with JWT validation and rate limiting at scale.
Why notNginx lacks native multi-tenant routing and dynamic configuration at 925K QPS.
Auth Service
Authenticate tenants and users, issue scoped tokens with tenant claims.
Why thisKeycloak supports multi-tenant realm isolation and OAuth2 flows with extensibility.
Why notCustom auth lacks maturity and scalability for 2B DAU and complex tenant isolation.
Feature Flag Service
Manage and evaluate feature flags per tenant with low latency.
Why thisLaunchDarkly or open-source Unleash with Redis backend provides sub-10ms flag evaluation and dynamic updates.
Why notPolling-based flag systems cause high latency and stale flags at scale.
Redis Cluster
Store feature flag states and tenant session caches with TTL for fast access.
Why thisRedis Cluster supports horizontal scaling and TTL expiration natively, reducing stale data.
Why notMemcached lacks persistence and TTL granularity, causing stale cache issues.
Billing Service
Aggregate usage events, calculate billing metrics, and emit billing records asynchronously.
Why thisJava microservice with Kafka consumer scales horizontally and supports exactly-once processing with Kafka Streams.
Why notBatch ETL jobs cause billing delays and inaccuracies at 925K events/sec.
Kafka Topic
Durable event streaming backbone for usage events and billing data pipeline.
Why thisKafka supports partitioning, replication, and high throughput for real-time event processing.
Why notTraditional message queues lack scalability and durability at multi-million QPS scale.
🗄️
TenantUsageEvents
PK: tenant_id
Stores usage events per tenant ordered by event time for billing and audit.
🗄️
FeatureFlags
PK: tenant_id
Stores feature flag configurations per tenant with versioning.
🔌
POST /tenants/onboard
Request
"{ \"tenantName\": \"string\", \"adminEmail\": \"string\" }"
Response
"{ \"tenantId\": \"string\", \"status\": \"success\" }"
Onboards a new tenant with initial configuration.
🔌
GET /billing/usage
Request
"{ \"tenantId\": \"string\", \"startDate\": \"date\", \"endDate\": \"date\" }"
Response
"{ \"usageSummary\": { ... }, \"billingAmount\": \"decimal\" }"
Fetches usage and billing summary for a tenant over a date range.
🔌
POST /featureflags/evaluate
Request
"{ \"tenantId\": \"string\", \"userId\": \"string\", \"flagKey\": \"string\" }"
Response
"{ \"enabled\": true|false, \"variant\": \"string\" }"
Evaluates a feature flag for a user within a tenant context.

Practice

(1/5)
1. In a bulkhead-isolated microservice architecture, at step 4 in the request trace, the Bulkhead Manager checks the thread pool status before dispatching a request. What happens if the thread pool is fully occupied?
easy
A. The request is queued indefinitely until a thread becomes available
B. The request is forwarded to a fallback service without delay
C. The Bulkhead Manager increases the thread pool size dynamically to handle the request
D. The Bulkhead Manager rejects the request immediately to prevent resource exhaustion

Solution

  1. Step 1: Trace Bulkhead Manager behavior at Step 4

    When the thread pool is full, queuing indefinitely (A) risks resource exhaustion. Dynamic resizing (D) breaks isolation guarantees. Forwarding to fallback (C) is a separate mechanism.

  2. Step 2: Understand bulkhead failure containment

    Rejecting requests immediately (B) prevents overload and contains failure blast radius.

  3. Final Answer:

    Option D -> Option D

  4. Quick Check:

    Bulkhead rejects requests when resource limits reached [check]
Hint: Reject requests when bulkhead resources are exhausted [check]
Common Mistakes:
2. In a Cache Aside pattern, at step 3 the application checks the cache and finds a miss. What happens next at step 4?
easy
A. The application queries the database, then updates the cache with the retrieved data
B. The cache automatically loads data from the database without application intervention
C. The application returns a null response to the user and retries later
D. The cache triggers a background refresh asynchronously and returns stale data

Solution

  1. Step 1: Understand Cache Aside flow

    On cache miss, application queries DB and updates cache explicitly.

  2. Step 2: Eliminate other options

    Cache Aside does not auto-load cache (A); it does not return null or stale data (C, D).

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Application loads DB and updates cache on miss [check]
Hint: Cache Aside: app loads DB and updates cache on miss [check]
Common Mistakes:
3. In the payment system's request flow, at Step 5 the Settlement Service receives a batch of transactions marked as 'authorized'. What does the Settlement Service do when it encounters a transaction flagged as 'already settled' during this step?
easy
A. It marks the transaction as failed and triggers a manual review
B. It retries settlement immediately to ensure idempotency
C. It skips the transaction and logs it as a duplicate to avoid double settlement
D. It forwards the transaction back to the Authorization Service for re-validation

Solution

  1. Step 1: Trace Settlement Service behavior at Step 5

    The Settlement Service must avoid double settlement by skipping transactions already marked as settled.

  2. Step 2: Evaluate other options

    Retrying settlement risks duplicate charges; marking as failed triggers unnecessary manual work; forwarding back to Authorization Service is redundant.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Settlement skips duplicates -> prevents double charges [check]
Hint: Settlement skips already settled transactions [check]
Common Mistakes:
4. Without the Kafka message queue in the architecture, what exactly breaks during message delivery from Alice to Bob?
medium
A. Alice's send blocks until Bob's server responds; if Bob's server crashes, the message is lost with no retry
B. Messages are encrypted incorrectly causing data leaks
C. Messages are duplicated due to lack of idempotency enforcement
D. Bob receives messages out of order but eventually all messages arrive

Solution

  1. Step 1: Understand Kafka's role in decoupling sender and receiver

    Kafka buffers messages asynchronously, preventing sender blocking and enabling retries.

  2. Step 2: Trace failure without Kafka

    Without Kafka, Alice's server must synchronously wait for Bob's server response; if Bob's server crashes, no retry occurs and message is lost. Encryption (A) and duplication (C) are unrelated; ordering (D) is guaranteed by Kafka partitions.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Without Kafka, synchronous send blocks and message loss on Bob's crash [check]
Hint: Kafka decouples sender/receiver; without it, sender blocks and loses messages [check]
Common Mistakes:
5. At 10x scale, which component in a consistent hashing distributed cache system is most likely to fail first, and why?
hard
A. Consistent hashing ring metadata service, because it becomes a bottleneck updating and distributing ring changes rapidly
B. Cache nodes, because their memory capacity limits are exceeded by increased key volume
C. Client-side hashing logic, because it cannot handle the increased number of nodes efficiently
D. Network switches, because increased traffic saturates bandwidth causing packet loss

Solution

  1. Step 1: Identify scaling bottlenecks

    Cache nodes scale horizontally; memory limits can be increased. Client hashing logic is local and scales well. Network switches scale with infrastructure upgrades.

  2. Step 2: Consistent hashing ring metadata bottleneck

    The ring metadata service must update and distribute node membership changes; at 10x scale, frequent updates cause high coordination overhead and latency, becoming the first failure point.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Metadata service update overhead limits scaling first [check]
Hint: Metadata service update overhead limits scaling first [check]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard