Bird
Raised Fist0

Design a Job Portal - Search, Candidate Matching, Application Tracking

HLDhardInterview freq: high
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
🏗️
Design a Job Portal - Search, Candidate Matching, Application Tracking
Design a scalable job portal supporting search, candidate-job matching, and application tracking at internet scale
hardSearch and DiscoveryFreq: high
Prerequisites:Inverted IndexElasticsearch
⚙️
Functional Requirements
  • Job Search: Candidates can search jobs by keywords, location, skills, company, and filters.
  • Candidate Matching: System recommends relevant jobs to candidates based on profile and behavior.
  • Application Tracking: Candidates can apply to jobs and track application status.
  • Job Posting Management: Employers can post, update, and manage job listings.
  • Notifications: Notify candidates about new matching jobs and application status changes.
  • User Profiles: Store candidate resumes, skills, experience, and preferences.
📊
Non-Functional Requirements
  • Support 2 billion daily active users (DAU) globally.
  • Handle peak write throughput of 925K job postings/applications per second.
  • Search query latency P99 under 100ms for a good user experience.
  • System availability SLA of 99.99% uptime.
  • Data consistency: Eventual consistency acceptable for search indexing; strong consistency for application status.
❓
Clarifying Questions to Ask
  • What is the expected scale of job postings and candidate profiles?
  • Are job search queries mostly read-heavy or write-heavy?
  • What is the acceptable latency for search and matching results?
  • Should candidate-job matching be real-time or batch processed?
  • What consistency guarantees are required for application tracking?
  • Are there any geographic or compliance constraints?
Out of Scope
✗ Resume parsing and enrichment.
✗ Payment and billing systems for employers.
✗ Social networking features like endorsements or messaging.
✗ Detailed analytics dashboards for employers.
📌
Assumptions
  • 2 billion DAU globally.
  • Each user generates on average 40 events (searches, applications, profile updates) per day.
  • Average event size is 100 bytes.
  • Retention period for logs and data is 30 days.
⚡
QPS Calculation
write qps2,000,000,000 × 40 / 86400 ≈ 925,926 writes/sec
peak write qpsAssuming 3× peak factor: 925,926 × 3 ≈ 2,777,778 writes/sec
read qpsAssuming 5 reads per write: 925,926 × 5 ≈ 4,629,630 reads/sec
💾
Storage Estimate
daily storage925,926 writes/sec × 100 bytes × 86400 sec ≈ 8 TB/day
retention storage8 TB/day × 30 days = 240 TB
with replication240 TB × 3 (replication factor) = 720 TB total storage
🖥️
Server Estimate
search nodesTo handle 4.6M QPS with 100ms P99 latency, ~200 Elasticsearch nodes (each ~25K QPS)
application trackingFor 2.7M writes/sec with strong consistency, ~300 Cassandra nodes (each ~10K writes/sec)
🏗️
Architecture Diagram
Architecture Diagram
          +----------------+
          |  User Clients  |
          +-------+--------+
                  |
          +-------v--------+
          |  API Gateway   |
          +-------+--------+
                  |
    +-------------+--------------+
    |                            |
 +--v---+                   +----v----+
 | ES   |                   | Cassandra|
 |Cluster|                   | Cluster  |
 +--+---+                   +----+-----+
    |                            |
 +--v---+                   +----v-----+
 | Kafka |                   | Redis    |
 |Cluster|                   | Cluster  |
 +-------+                   +----------+
    |                            |
 +--v----------------------------v----+
 |       Application Processing        |
 |       & Notification Service        |
 +-------------------------------------+
🧩
Components
API Gateway
Entry point for all client requests, routes to appropriate backend services.
Why thisNGINX + Envoy proxy for high throughput, low latency routing with TLS termination and rate limiting.
Why notAWS ALB lacks fine-grained routing and custom logic needed for multi-tenant job portal.
Elasticsearch Cluster
Full-text search engine for job listings and candidate profiles with inverted index.
Why thisElasticsearch supports distributed inverted index, near real-time search, and complex queries with filters and scoring.
Why notSolr requires more manual sharding and slower cluster state updates at 4.6M QPS.
Apache Cassandra Cluster
Stores application tracking data and user profiles with high write throughput and availability.
Why thisCassandra provides linear write scalability, tunable consistency, and multi-datacenter replication.
Why notPostgreSQL cannot handle 2.7M writes/sec without complex sharding and suffers from write bottlenecks.
Kafka Cluster
Event streaming platform for decoupling write ingestion, asynchronous indexing, and notification processing.
Why thisKafka supports high throughput, partitioned logs, and consumer groups for scalable asynchronous processing.
Why notRabbitMQ lacks partitioning and throughput at this scale; Kinesis is vendor-locked and costlier.
Redis Cluster
Caching layer for hot data like user sessions, recent searches, and application status for low latency.
Why thisRedis supports TTL, sub-millisecond lookups, and data structures like sorted sets for leaderboard-like queries.
Why notMemcached lacks persistence and advanced data structures needed for application state caching.
Application Processing & Notification Service
Consumes Kafka events to update search index, send notifications, and update application status asynchronously.
Why thisKubernetes-based microservices allow independent scaling and fault isolation for processing pipelines.
Why notMonolithic services would cause scaling bottlenecks and increase blast radius on failures.
🔄
Data Flow
1
User Clients → API Gateway
Send job search queries, applications, and profile updates.
API Gateway authenticates and routes requests.
2
API Gateway → Kafka Cluster
Publish write events (job posts, applications) asynchronously.
Decouples ingestion from processing for write scalability.
3
Kafka Cluster → Application Processing Service
Consume events to update Cassandra and Elasticsearch.
Ensures eventual consistency between storage and search index.
4
Application Processing Service → Elasticsearch Cluster
Index new/updated job postings and candidate profiles asynchronously.
Provides near real-time search updates with eventual consistency.

Practice

(1/5)
1. In a CDN request flow, at step 4 the edge node checks its cache for the requested content. What happens if the content is not found in the cache at this step?
easy
A. The edge node forwards the request to the origin server to fetch content
B. The edge node immediately returns a 404 error to the client
C. The edge node redirects the client to another edge node
D. The edge node serves stale content from a previous cache entry

Solution

  1. Step 1: Trace edge node cache miss behavior at Step 4

    If content is missing, the edge node must fetch it from the origin server.

  2. Step 2: Eliminate other options

    Returning 404 is incorrect because content may exist; redirecting to another edge node is inefficient; serving stale content violates freshness guarantees.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Cache miss -> edge node fetches from origin server [check]
Hint: Cache miss -> fetch from origin server [check]
Common Mistakes:
2. During a flash sale, the system must handle sudden bursts of 10x traffic within 60 seconds, ensuring order preservation of purchase requests per user session. Which component best handles this requirement?
easy
A. Use a relational database with row-level locking to serialize requests
B. Use Redis as a cache to store purchase requests temporarily
C. Use Kafka with partitioning keyed by user session ID to preserve order
D. Use a simple HTTP load balancer to distribute requests evenly

Solution

  1. Step 1: Identify ordering requirement per user session

    Kafka partitions keyed by user session ID guarantee ordering within that partition, ensuring requests are processed sequentially per user.

  2. Step 2: Evaluate alternatives

    Relational DB row-level locking (A) causes high contention and poor scalability under 10x burst. Redis (C) lacks strong ordering guarantees and durability. Load balancer (D) only distributes load but does not preserve order.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Kafka partition key -> per-session ordering guaranteed [check]
Hint: Kafka partitions preserve order per key [check]
Common Mistakes:
3. For a distributed file system requiring strong consistency on metadata updates but high availability, which architecture is most appropriate?
medium
A. Use a consensus-based Metadata Server cluster (e.g., Paxos or Raft) to coordinate updates
B. Use a single Metadata Server with synchronous replication to backups
C. Use multiple independent Metadata Servers sharding namespace without coordination
D. Use eventual consistency with asynchronous metadata replication across servers

Solution

  1. Step 1: Analyze consistency and availability needs

    Strong consistency requires coordination; high availability requires fault tolerance.

  2. Step 2: Evaluate architectures

    Consensus-based clusters provide strong consistency and tolerate failures; single server risks downtime; independent shards lack coordination; eventual consistency violates strong consistency.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Consensus clusters balance strong consistency and availability [check]
Hint: Consensus clusters ensure strong consistency with availability [check]
Common Mistakes:
4. At 10x scale in user requests and live traffic updates, which component is most likely to fail first and why?
hard
A. The live traffic ingestion pipeline, as message queues saturate and delay update propagation
B. The Road Graph database, due to inability to handle increased write volume for live updates
C. The Routing Engine servers, because shortest path computations become CPU-bound and latency spikes
D. The client-side map rendering, due to increased data volume overwhelming browser memory

Solution

  1. Step 1: Analyze scaling bottlenecks

    Road Graph DB (A) can be sharded; ingestion pipeline (C) can scale horizontally; client rendering (D) is local and less critical.

  2. Step 2: Identify Routing Engine CPU bottleneck

    Routing Engine performs complex shortest path computations per request; 10x load causes CPU saturation and latency spikes, failing first.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Routing Engine CPU-bound under 10x load causes first failure [check]
Hint: Routing Engine CPU saturates first under 10x load [check]
Common Mistakes:
5. At 10x scale, which component in a sliding window rate limiter is most likely to fail first and why?
hard
A. The timestamp storage, because memory usage grows linearly with request volume and causes exhaustion
B. The fixed window counter, because it cannot handle bursty traffic at scale
C. The token bucket refill scheduler, because it cannot keep up with token replenishment at high rates
D. The leaky bucket drain mechanism, because it becomes a bottleneck under bursty load

Solution

  1. Step 1: Analyze sliding window memory scaling

    Sliding window counters store timestamps for each request, so memory grows linearly with request volume.

  2. Step 2: At 10x scale, timestamp storage exhausts memory first, causing failures before other components become bottlenecks

    Fixed window counters use constant memory, token/leaky buckets have fixed state sizes.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Sliding window timestamp storage grows with traffic -> memory exhaustion [check]
Hint: Sliding window memory grows with requests, fixed window constant [check]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard