Bird
Raised Fist0

Service Discovery - Client-Side vs Server-Side Patterns

HLDmediumInterview freq: common
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
🏗️
Service Discovery - Client-Side vs Server-Side Patterns
Designing scalable service discovery using client-side and server-side patterns with trade-offs
mediumMicroservices InfrastructureFreq: common
⚙️
Functional Requirements
  • Register service instances dynamically with metadata (IP, port, version).
  • Allow clients to discover healthy service instances efficiently.
  • Support health checking and automatic removal of stale instances.
  • Provide load balancing hints or direct routing info to clients.
  • Support service versioning and canary deployments.
  • Enable fast propagation of instance state changes (register/deregister).
📊
Non-Functional Requirements
  • Handle 2 billion daily active users generating service calls.
  • Achieve 99.99% availability for discovery queries.
  • Support 100ms P99 latency for service discovery requests.
  • Scale to 925K writes/sec for instance registrations and updates.
  • Support eventual consistency with max 5 seconds staleness.
❓
Clarifying Questions to Ask
  • What is the expected number of microservices and instances per service?
  • What is the read-to-write ratio for service discovery queries?
  • Are clients trusted to perform load balancing or should server-side handle it?
  • What is the expected churn rate of service instances (register/deregister frequency)?
  • Is strong consistency required or is eventual consistency acceptable?
  • Are there multi-datacenter or geo-distributed requirements?
Out of Scope
✗ Designing the microservices themselves or their business logic.
✗ Authentication and authorization of service calls.
✗ Network-level routing or DNS infrastructure design.
✗ Detailed client-side load balancing algorithms.
📌
Assumptions
  • 2 billion DAU generating 40 service calls per user per day.
  • Average service discovery request size is 100 bytes.
  • Each service instance registers/deregisters 10 times per day on average.
  • Retention period for logs and metrics is 7 days.
⚡
QPS Calculation
write qps925926
peak write qps4629630
read qps9259260
💾
Storage Estimate
daily storage8 TB
retention storage56 TB
with replication168 TB total
🖥️
Server Estimate
service registry servers463
cache servers93
🏗️
Architecture Diagram
Architecture Diagram
         +----------------------+
         |  Service Instances   |
         | (Register/Deregister)|
         +----------+-----------+
                    |
                    v
         +----------------------+
         |  Service Registry    |
         |  (e.g. Consul, etcd) |
         +----------+-----------+
                    |
                    v
         +----------------------+
         |  Cache Layer (Redis) |
         +----------+-----------+
                    |
                    v
         +----------------------+
         |  Client Service      |
         |  (Client-Side Disc.) |
         +----------------------+

Alternative Server-Side Pattern:

         +----------------------+
         |  Client Service      |
         +----------+-----------+
                    |
                    v
         +----------------------+
         |  Server-Side Proxy   |
         |  (Load Balancer)     |
         +----------+-----------+
                    |
                    v
         +----------------------+
         |  Service Registry    |
         +----------------------+
🧩
Components
Service Registry (Consul)
Stores service instance metadata and health status with strong consistency
Why thisConsul provides built-in health checks, leader election, and supports high write QPS with Raft consensus
Why notZookeeper requires complex manual sharding and has higher write latency at 4M writes/sec scale
Cache Layer (Redis Cluster)
Caches service registry data for low-latency reads by clients
Why thisRedis TTL natively expires stale entries without cron jobs and supports sub-millisecond lookups at 100K QPS per node
Why notMemcached lacks persistence and TTL-based eviction, risking stale data during failures
Client Service (Client-Side Discovery)
Queries cache or registry directly to discover service instances and load balances locally
Why thisReduces server-side load and latency by pushing discovery logic to clients, suitable for high read QPS
Why notServer-side discovery centralizes load but adds network hop and single point of failure
Server-Side Proxy (Envoy)
Acts as a centralized load balancer and service discovery proxy for clients
Why thisEnvoy supports dynamic service discovery via xDS APIs and can perform retries and circuit breaking
Why notNginx lacks native dynamic service discovery and requires manual config reloads
Health Check Worker
Periodically verifies service instance health and updates registry status
Why thisDecouples health checking from registry to improve scalability and isolate failures
Why notEmbedding health checks in registry increases load and risks cascading failures
🔄
Data Flow
1
Service Instance → Service Registry
Register or deregister instance with metadata and heartbeat
Keeps registry updated with live instances
2
Health Check Worker → Service Registry
Update health status of instances based on probes
Ensures stale or unhealthy instances are marked
3
Service Registry → Cache Layer
Push or sync updated instance info with TTL
Caches data for fast client reads
4
Client Service → Cache Layer
Query for healthy service instances
Client-side discovery reduces server load
5
Client Service → Service Instance
Direct request routed to selected instance
Client load balances using local info
6
Client Service → Server-Side Proxy
Optional: forward request for server-side discovery
Used if client-side discovery is disabled
7
Server-Side Proxy → Service Registry
Query registry for instance list
Proxy performs discovery and load balancing
🗄️
ServiceInstances
PK: serviceName
instanceId UUIDserviceName StringipAddress Stringport Intversion Stringstatus StringlastHeartbeat Timestamp
Stores metadata and status of each service instance
🗄️
ServiceHealthChecks
PK: serviceName
serviceName StringinstanceId UUIDcheckTimestamp TimestamphealthStatus StringresponseTime Int
Stores health check results for service instances
🔌
RegisterInstance
Request
{
  "serviceName": "String",
  "instanceId": "UUID",
  "ipAddress": "String",
  "port": "Int",
  "version": "String"
}
Response
{
  "success": "Boolean",
  "message": "String"
}
Registers a new service instance or updates existing
🔌
DiscoverInstances
Request
{
  "serviceName": "String",
  "version": "String (optional)"
}
Response
{
  "instances": [
    {
      "instanceId": "UUID",
      "ipAddress": "String",
      "port": "Int",
      "version": "String",
      "status": "String"
    }
  ]
}
Returns list of healthy instances for a service
🔌
DeregisterInstance
Request
{
  "serviceName": "String",
  "instanceId": "UUID"
}
Response
{
  "success": "Boolean",
  "message": "String"
}
Removes a service instance from registry

Practice

(1/5)
1. A real-time leaderboard system must handle millions of score updates per second with low latency and provide instant rank queries. Which component is best suited to store and update scores efficiently while maintaining sorted order?
easy
A. A NoSQL document store like MongoDB for flexible schema
B. A relational database with indexed score columns
C. A distributed log like Kafka for storing score events
D. Redis Sorted Sets to maintain scores and ranks in-memory

Solution

  1. Step 1: Identify data structure for sorted order and fast updates

    Redis Sorted Sets provide O(log N) insertion and rank queries with in-memory speed, ideal for real-time leaderboards.

  2. Step 2: Eliminate other options based on latency and ordering

    Relational DBs have higher write latency and complex locking; Kafka stores events but doesn't maintain sorted order; MongoDB lacks efficient sorted queries at scale.

  3. Final Answer:

    Option D -> Option D

  4. Quick Check:

    Redis Sorted Sets maintain sorted scores with low latency [check]
Hint: Sorted Sets keep scores sorted with fast rank queries [check]
Common Mistakes:
2. In the AI Code Assistant request flow, at step 4 the Retriever component queries the vector index with the user's code context embedding. What happens if the vector index is temporarily unavailable at this step?
easy
A. Retriever switches to a relational DB fallback with exact keyword matching
B. Retriever falls back to querying the raw code files directly, increasing latency
C. Retriever retries indefinitely, blocking the entire request pipeline
D. Retriever returns an empty result set, causing the generator to produce generic completions

Solution

  1. Step 1: Understand Retriever fallback logic

    Retriever cannot query vector index if unavailable; raw code files (A) are not indexed for fast retrieval; indefinite retries (C) would block latency budget; empty results (B) degrade quality but not typical fallback.

  2. Step 2: Recognize fallback to relational DB

    System uses relational DB fallback (D) with keyword matching to maintain some retrieval capability under vector index failure.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Fallback to keyword DB preserves retrieval under vector index failure [check]
Hint: Retriever falls back to keyword DB if vector index fails [check]
Common Mistakes:
3. In the request flow for generating a user's feed, at step 5 the Feed Aggregator receives multiple candidate posts from different microservices. What does the Feed Aggregator do at this step before sending the feed to the user?
easy
A. It merges, deduplicates, and ranks posts based on relevance scores before returning
B. It immediately returns the raw posts without filtering or ranking
C. It stores all posts in a persistent database before responding to the user
D. It sends posts to the notification service to alert the user before aggregation

Solution

  1. Step 1: Understand Feed Aggregator's role at Step 5

    The Feed Aggregator must combine posts from multiple sources, remove duplicates, and rank them to personalize the feed.

  2. Step 2: Eliminate incorrect options

    It immediately returns the raw posts without filtering or ranking skips ranking, leading to poor UX. It stores all posts in a persistent database before responding to the user delays response unnecessarily. It sends posts to the notification service to alert the user before aggregation misplaces notification logic before feed aggregation.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Feed Aggregator merges and ranks posts -> personalized feed [check]
Hint: Aggregate, dedupe, rank before sending feed [OK]
Common Mistakes:
4. For a globally distributed collaborative editor with many offline users, which approach better supports eventual consistency and offline edits, and why?
medium
A. Use Operational Transformation (OT) because it requires a central server to order operations
B. Use a relational database with strict locking to serialize all edits globally
C. Use CRDTs because they allow local operations to be merged without coordination and support offline edits
D. Use a message queue with timestamp ordering to ensure global operation order

Solution

  1. Step 1: Compare OT and CRDT for offline and distributed scenarios

    OT typically requires a central server to order and transform operations, making offline support complex.

  2. Step 2: Understand CRDT advantages

    CRDTs are designed for decentralized environments, allowing local operations to be merged eventually without coordination, ideal for offline edits and global distribution.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    CRDTs enable offline edits and eventual consistency without central coordination [check]
Hint: CRDTs merge offline edits without central coordination [OK]
Common Mistakes:
5. You must design a system that processes high-volume event streams with strict ordering per customer but can tolerate some message duplication. Which approach is best?
medium
A. Partition the topic by customer ID and use multiple consumers in a group
B. Use Kafka with exactly-once semantics and idempotent producers
C. Use multiple topics per customer to isolate streams
D. Use a single Kafka topic with one partition to guarantee ordering

Solution

  1. Step 1: Evaluate ordering and throughput needs

    Single partition guarantees ordering but limits throughput severely.

  2. Step 2: Assess partitioning by customer ID

    Partitioning by customer ID preserves ordering per customer and allows parallel consumption.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Partition key + consumer group balances order and scale [check]
Hint: Partition by customer ID for order and parallelism [OK]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard