Bird
Raised Fist0

Design LinkedIn - Professional Graph, Feed, Job Recommendations

HLDhardInterview freq: high
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
🎯
Design LinkedIn - Professional Graph, Feed, Job Recommendations
45 min · senior
senior

Design a scalable system for a professional social network with 50 million daily active users (DAU). The system must handle a professional graph of hundreds of millions of users and connections, generate personalized feeds with posts and job recommendations, and support real-time updates with P99 latency under 200ms. The system must ensure data encryption at rest and in transit, support offline feed delivery, and maintain strong consistency for connection updates.

Constraints
  • P99 latency for feed and job recommendation queries under 200ms
  • Data encryption at rest and in transit (TLS + AES-256)
  • Support offline feed delivery with eventual consistency
  • Strong consistency for professional graph updates (connections, endorsements)
  • Handle 50M DAU with peak write throughput of 1M connection updates/day and 10M feed updates/min
📋
Guided Steps
1/5
Step 1
Designing the Professional Graph Storage
How would you store and serve the professional graph (connections, endorsements) to support 50M DAU with strong consistency?
💡 Hint
Consider graph databases or wide-column stores optimized for relationship queries and strong consistency.
Key Decision
Use Apache Cassandra with a data model keyed by user_id and connection_id, leveraging lightweight transactions (LWT) for strong consistency on connection updates; reject Neo4j due to poor horizontal scalability at 50M DAU.
Interviewer Expectation
Mid-level: Names Cassandra for scalability. Senior: Explains LWT for strong consistency and why Neo4j's single-master architecture fails at this scale.
⚠️Common Wrong Answer
Using a relational database with joins - leads to high latency and poor horizontal scalability under 50M DAU.
Step 2
Feed Generation and Storage
How would you design the feed generation pipeline to deliver personalized posts and job recommendations with P99 < 200ms latency?
💡 Hint
Think about stream processing and caching layers to handle high write throughput and low read latency.
Key Decision
Use Apache Kafka for feed event ingestion partitioned by user_id, Apache Flink for real-time feed computation, and Redis as a per-user feed cache with TTL for low-latency reads; reject RabbitMQ due to lower throughput and lack of partitioning guarantees.
Interviewer Expectation
Mid-level: Mentions Kafka for event streaming. Senior: Details partitioning by user_id for ordering, Flink for stateful stream processing, and Redis TTL cache for offline delivery support.
⚠️Common Wrong Answer
Polling a database for feed updates - causes excessive load and fails to meet 200ms P99 latency.
Step 3
Job Recommendation Engine
How would you design the job recommendation system to provide personalized, relevant job suggestions integrated into the feed?
💡 Hint
Consider offline batch processing combined with online feature stores for real-time scoring.
Key Decision
Use Apache Spark for offline model training on user and job data, store features in Redis or Amazon DynamoDB for low-latency access, and deploy TensorFlow Serving for online inference; reject purely online ML due to latency and throughput constraints.
Interviewer Expectation
Mid-level: Names Spark for batch training. Senior: Explains hybrid offline-online architecture with feature store and TensorFlow Serving for sub-200ms inference latency.
⚠️Common Wrong Answer
Running ML inference synchronously on user request without caching - leads to high latency and poor scalability.
Step 4
Ensuring Data Security and Encryption
How would you ensure encryption at rest and in transit for sensitive user data and professional graph information?
💡 Hint
Think about TLS for network encryption and disk-level encryption for storage systems.
Key Decision
Enable TLS 1.3 for all client-server and inter-service communication, use AWS KMS-managed AES-256 encryption for Cassandra and Kafka data at rest; reject application-level encryption due to complexity and performance overhead.
Interviewer Expectation
Mid-level: Mentions TLS and disk encryption. Senior: Details TLS 1.3 benefits, KMS integration for key management, and trade-offs of application-level encryption.
⚠️Common Wrong Answer
Relying only on network encryption without encrypting data at rest - exposes data if storage is compromised.
Step 5
API Design for Client Interaction
Design the APIs for (1) managing connections, (2) fetching personalized feed, and (3) acknowledging feed interactions. What are the key considerations?
💡 Hint
Consider idempotency, pagination, and consistency guarantees in your API design.
Key Decision
RESTful APIs with idempotent POST for connection updates using UUIDs, paginated GET with cursor-based pagination for feed fetching, and POST for interaction acknowledgments with event sourcing to Kafka; reject simple offset pagination due to inconsistency under concurrent updates.
Interviewer Expectation
Mid-level: Defines basic REST endpoints. Senior: Explains idempotency with UUIDs, cursor-based pagination for consistent feed sync, and event sourcing for interaction tracking.
⚠️Common Wrong Answer
Using offset-based pagination - leads to missing or duplicate feed items during concurrent updates.
🔌
API Design Challenge
Sketch 3 core API endpoints for the system: (1) Add or remove a professional connection, (2) Fetch personalized feed with posts and job recommendations, (3) Acknowledge feed item consumption or interaction.
Hints
  • How does the client send connection updates and ensure consistency?
  • How does the client paginate and sync feed items efficiently?
  • How does the system track user interactions to improve recommendations?
📊
Scoring Rubric
▼ Show
Must Have - 60%
  • Use of Cassandra with LWT for strong consistency in professional graph
  • Kafka partitioning by user_id for feed event ordering and scalability
  • Real-time stream processing with Apache Flink for feed generation
  • Redis caching layer for low-latency feed reads and offline support
  • Encryption at rest and in transit with TLS 1.3 and AES-256
Good to Have - 30%
  • Hybrid offline-online ML architecture with Spark and TensorFlow Serving
  • Idempotent and cursor-based API design for consistency and scalability
  • Event sourcing of user interactions for improving recommendations
Bonus - 10%
  • Discussion of trade-offs rejecting graph databases like Neo4j due to scalability
  • Explanation of why RabbitMQ or offset pagination are insufficient at scale
Anti-patterns to Avoid
Using relational databases with joins for professional graph causes high latency and poor horizontal scalability under 50M DAU, making the system unable to scale.
Polling databases for feed updates at 50M DAU leads to excessive load and fails to meet 200ms P99 latency SLAs, causing system overload and poor user experience.
Synchronous ML inference on user requests without caching results in high latency and poor scalability, degrading recommendation quality and system throughput.
✅
Model Answer
▼ Reveal

This system design outlines a scalable architecture for LinkedIn's professional graph, personalized feed, and job recommendation system supporting 50M DAU with strict latency and security requirements. It leverages proven distributed technologies and design patterns to meet consistency, throughput, and security constraints.

Key Decisions
  • Choosing Apache Cassandra with LWT over graph databases like Neo4j to achieve horizontal scalability and strong consistency for professional graph updates.
  • Using Kafka partitioned by user_id to maintain event ordering and high throughput for feed event ingestion.
  • Employing Apache Flink for stateful stream processing to compute personalized feeds in real-time with P99 latency under 200ms.
  • Caching feeds in Redis with TTL to enable offline feed delivery and reduce read latency.
  • Implementing a hybrid offline-online ML architecture with Spark for batch training and TensorFlow Serving for low-latency online inference.
  • Enforcing TLS 1.3 for all client-server and inter-service communication and using AWS KMS-managed AES-256 encryption for data at rest to meet security requirements.
  • Designing RESTful APIs with idempotency via UUIDs, cursor-based pagination for consistent feed synchronization, and event sourcing for tracking user interactions to improve recommendations.
  • Apache Cassandra with LWT for strong consistency on professional graph updates
  • Apache Kafka partitioned by user_id for ordered feed event ingestion
  • Apache Flink for real-time feed computation and enrichment
  • Redis as a per-user feed cache with TTL to support offline delivery and low latency
  • Apache Spark for offline job recommendation model training
  • TensorFlow Serving for online job recommendation inference with sub-200ms latency
  • TLS 1.3 for all network communication and AWS KMS-managed AES-256 encryption at rest
  • RESTful APIs with idempotent connection updates, cursor-based feed pagination, and event sourcing for interaction tracking

Practice

(1/5)
1. A real-time location tracking system must efficiently query nearby devices within a geographic radius with low latency. Which component is best suited to handle these geo-spatial queries?
easy
A. A relational database with spatial extensions (e.g., PostGIS) for all queries
B. A geo-indexing service like a geospatially-enabled NoSQL store (e.g., Redis GEO or Elasticsearch)
C. A message queue to buffer location updates before querying
D. A CDN to cache location data for faster access

Solution

  1. Step 1: Identify geo-indexing role

    Relational DBs with spatial extensions provide accuracy but have higher latency and poor horizontal scaling for frequent geo queries.

  2. Step 2: Evaluate alternatives

    Message queues buffer data but do not support geo queries. CDNs cache static content, not dynamic geo data.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    Geo-indexing NoSQL stores optimize radius queries with low latency [check]
Hint: Geo-indexing NoSQL stores excel at fast radius queries [check]
Common Mistakes:
2. If the event ingestion pipeline fails to persist incoming metric data, what is the exact failure observed in the alerting system?
medium
A. Threshold and anomaly detectors receive incomplete data, causing missed or false alerts
B. Alerts are generated with delayed timestamps but eventually delivered
C. Composite alerts continue to function normally using cached inputs
D. The system switches to a backup data source automatically without alert impact

Solution

  1. Step 1: Role of event ingestion pipeline

    This pipeline persists raw metric data for downstream alerting components.

  2. Step 2: Consequence of pipeline failure

    Without persistence, threshold and anomaly detectors get incomplete or missing data, leading to missed alerts or false positives.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Missing data -> detectors miss or misfire alerts [check]
Hint: No data persistence -> detectors get incomplete data, miss alerts [check]
Common Mistakes:
3. For storing aggregated logs with fast query response and cost efficiency, which storage architecture is best when logs are mostly write-once and rarely updated?
medium
A. Use a distributed relational database with strong consistency
B. Use a NoSQL key-value store with frequent compactions
C. Use a columnar data warehouse optimized for append-only writes
D. Use a distributed file system with manual indexing

Solution

  1. Step 1: Analyze log storage requirements

    Logs are write-once, large volume, and require fast analytical queries.

  2. Step 2: Evaluate columnar data warehouse

    Columnar stores optimize for append-only writes and fast scans, reducing cost and improving query speed.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Columnar warehouse -> optimized for append-only and analytics [check]
Hint: Columnar storage excels at append-only analytics [check]
Common Mistakes:
4. If the reverse proxy responsible for SSL termination crashes, what exactly breaks in the request flow?
medium
A. Clients can still connect directly to backend servers without interruption
B. Clients' SSL handshakes fail, so connections never reach backend servers
C. Backend servers receive unencrypted traffic, exposing internal data
D. The reverse proxy forwards requests without decrypting, causing backend errors

Solution

  1. Step 1: Understand reverse proxy SSL termination failure

    If the reverse proxy crashes, SSL termination stops, so backend servers receive unencrypted traffic if clients connect directly or fallback occurs.

  2. Step 2: Eliminate other options

    Clients cannot connect directly if proxy is mandatory; SSL handshake fails only if proxy blocks connection; proxy can't forward encrypted requests without decrypting if crashed.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Reverse proxy crash exposes backend to unencrypted traffic [check]
Hint: Reverse proxy crash exposes backend to unencrypted traffic [check]
Common Mistakes:
5. At 10x current query volume, which DNS component is most likely to become the first bottleneck and why?
hard
A. The recursive DNS servers, because they maintain stateful caches and have limited CPU and memory resources per instance
B. The root DNS servers, because they must handle all queries directly and cannot be scaled horizontally
C. The authoritative DNS servers, because they handle write-heavy updates and cannot cache responses
D. The local stub resolvers, because they have limited cache size and cannot share state across clients

Solution

  1. Step 1: Analyze scaling characteristics of DNS components

    Root servers are highly distributed and rarely queried directly by clients. Authoritative servers mainly handle writes and updates, which are less frequent than queries. Stub resolvers are client-local and limited in scope.

  2. Step 2: Identify recursive server bottleneck

    Recursive DNS servers handle all client queries, maintain caches, and have finite CPU/memory. At 10x scale, their resource limits cause the first bottleneck due to stateful caching and query processing load.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Recursive servers stateful caches -> CPU/memory bottleneck first [check]
Hint: Recursive servers stateful caches limit scaling first [check]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard