0
0
HLDsystem_design~12 mins

Shard key selection in HLD - Architecture Diagram

Choose your learning style9 modes available
LearnWhyDeepArchPracticeChallengeDesignRecallScale
System Overview - Shard key selection

This system demonstrates how to select an effective shard key for a distributed database. The goal is to distribute data evenly across multiple database shards to improve scalability and performance while minimizing hotspots and cross-shard queries.

Architecture Diagram
User
  |
  v
Load Balancer
  |
  v
API Gateway
  |
  v
Shard Key Selector Service
  |
  v
+-----------------------+
|  Distributed Database  |
|  +-------+ +-------+  |
|  |Shard 1| |Shard 2|  |
|  +-------+ +-------+  |
+-----------------------+
  |
  v
Cache Layer
Components
User
client
Sends requests to the system
Load Balancer
load_balancer
Distributes incoming requests evenly to API Gateway instances
API Gateway
api_gateway
Handles client requests and routes them to the Shard Key Selector Service
Shard Key Selector Service
service
Determines the shard key based on request data to route queries to the correct shard
Distributed Database
database_cluster
Stores data partitioned across multiple shards based on the shard key
Shard 1
database_shard
Stores a subset of data determined by shard key range or hash
Shard 2
database_shard
Stores another subset of data determined by shard key range or hash
Cache Layer
cache
Caches frequently accessed data to reduce database load
Request Flow - 8 Hops
User → Load Balancer
Load Balancer → API Gateway
API Gateway → Shard Key Selector Service
Shard Key Selector Service → Cache Layer
Cache Layer → Distributed Database
Distributed Database → Cache Layer
Cache Layer → API Gateway
API Gateway → User
Failure Scenario
Component Fails:Shard Key Selector Service
Impact:Requests cannot be routed to the correct shard, causing failures or delays in data access
Mitigation:Deploy multiple instances with health checks and failover; cache shard key mappings to reduce dependency
Architecture Quiz - 3 Questions
Test your understanding
Which component decides how to split data across shards?
ACache Layer
BShard Key Selector Service
CLoad Balancer
DAPI Gateway
Design Principle
This architecture shows the importance of selecting a good shard key to evenly distribute data and reduce cross-shard queries. Using a dedicated service for shard key selection helps centralize routing logic. Caching improves performance by reducing database load. Load balancing ensures availability and scalability.