Bird
Raised Fist0

Design a Code Repository System - Diff, Branch, Merge, Pull Request

HLDhardInterview freq: medium
Choose your preparation mode4 modes available
🎯
Design a Code Repository System - Diff, Branch, Merge, Pull Request
45 min · senior
senior

You are designing a code repository system used by a 50 million daily active users startup. The system must support efficient diff computation, branching, merging, and pull requests with P99 latency under 200ms for core operations. All data must be encrypted at rest and in transit. The system should handle concurrent edits and offline client operations gracefully.

  • P99 latency for core operations (diff, merge, pull request) under 200ms
  • End-to-end encryption of repository data at rest and in transit
  • Support offline client operations with eventual sync
  • Handle concurrent edits and merge conflicts efficiently
  • Scale to 50 million daily active users with millions of repositories
📋
Guided Steps
1/6
Step 1
Repository Data Model and Storage
How would you design the data model and storage backend to efficiently store code versions, branches, and commits at scale?
💡 Hint
Consider immutable commit objects and content-addressable storage to avoid duplication and enable efficient diffs.
Use Git-compatible content-addressable storage with Apache Cassandra for commit metadata and Amazon S3 for immutable blob storage of file contents, enabling scalable, distributed storage with efficient deduplication.
Mid-level: Names Git object model and Cassandra for metadata. Senior: Explains why Cassandra's LSM-tree and partition keys enable efficient commit history queries and why S3 is chosen for large immutable blobs over alternatives like HDFS.
⚠️Common Wrong Answer
Using a relational database like MySQL for storing blobs and commits leads to poor write scalability and inefficient diff computations at scale.
Step 2
Diff Computation and Caching
How would you implement efficient diff computation between branches or commits to meet P99 latency under 200ms?
💡 Hint
Leverage content-addressable storage and precomputed diffs or delta compression to avoid recomputing diffs on every request.
Implement diff computation using libgit2 on server-side with Redis caching of recent diffs keyed by commit pairs, reducing repeated expensive computations and meeting latency requirements.
Mid-level: Suggests server-side diff computation. Senior: Explains caching strategy with Redis TTL, cache invalidation on new commits, and why client-side diff is insufficient due to offline and scale constraints.
⚠️Common Wrong Answer
Computing diffs on-demand without caching causes high CPU usage and latency spikes, violating the 200ms P99 latency SLA.
Step 3
Branching and Merge Conflict Resolution
How would you design branching and merging to handle concurrent edits and merge conflicts efficiently?
💡 Hint
Consider optimistic concurrency control and CRDTs or three-way merge algorithms to resolve conflicts.
Use three-way merge algorithm with server-side conflict detection and Git-style merge conflict markers; store branch pointers in Cassandra with lightweight transactions for concurrency control.
Mid-level: Describes three-way merge basics. Senior: Details how Cassandra lightweight transactions prevent lost updates on branch pointers and how conflict markers enable client-side resolution with offline support.
⚠️Common Wrong Answer
Using last-write-wins without conflict detection causes silent data loss and inconsistent repository states.
Step 4
Pull Request Workflow and Notifications
How would you design the pull request system to support review, approval, and merge with real-time notifications?
💡 Hint
Think about event-driven architecture and pub/sub systems for notifying reviewers and triggering CI pipelines.
Use Apache Kafka for event streaming of pull request lifecycle events, triggering asynchronous CI/CD pipelines and WebSocket-based real-time notifications to clients.
Mid-level: Names Kafka for event streaming. Senior: Explains partitioning by repository ID to ensure ordering, idempotent consumers for retries, and integration with CI/CD systems.
⚠️Common Wrong Answer
Polling for pull request status causes excessive load and latency, failing to meet real-time notification requirements.
Step 5
Security and Encryption
How would you ensure end-to-end encryption of repository data at rest and in transit while maintaining performance?
💡 Hint
Consider envelope encryption and TLS for transport security, and key management systems for encryption keys.
Use AWS KMS for envelope encryption of blobs stored in S3, TLS 1.3 for all client-server communication, and client-side encryption for sensitive metadata to meet compliance and performance goals.
Mid-level: Mentions TLS and encryption at rest. Senior: Details envelope encryption benefits, key rotation policies, and client-side encryption trade-offs for metadata privacy.
⚠️Common Wrong Answer
Relying solely on TLS without encryption at rest exposes data to breaches if storage is compromised.
Step 6
Offline Client Support and Sync
How would you design the system to support offline client operations and eventual synchronization?
💡 Hint
Consider conflict resolution strategies and efficient sync protocols that minimize data transfer.
Implement client-side Git repositories with local commits and branch management; server sync uses Git fetch/push protocols with conflict detection and resolution on merge, minimizing data transfer via delta compression.
Mid-level: Recognizes offline commits possible with Git. Senior: Explains sync protocol optimizations, conflict detection during push, and how delta compression reduces bandwidth for large repos.
⚠️Common Wrong Answer
Forcing online-only operations prevents offline productivity and leads to poor user experience.
🔌
API Design Challenge
Sketch 3 core API endpoints for the code repository system that enable clients to create branches, submit pull requests, and fetch diffs.
  • How does the client specify the source and target branches in a pull request?
  • How can the client efficiently fetch diffs without downloading entire files?
  • How does the system acknowledge merge conflicts or success in pull requests?
📊
Scoring Rubric
▼ Show
Must Have - 60%
  • Use Git-compatible content-addressable storage with Apache Cassandra and Amazon S3
  • Implement server-side diff computation with Redis caching
  • Use three-way merge with concurrency control via Cassandra lightweight transactions
  • Employ Apache Kafka for event streaming and real-time notifications
  • Ensure end-to-end encryption with AWS KMS and TLS 1.3
  • Support offline client operations with Git fetch/push sync protocols
Good to Have - 30%
    Bonus - 10%
      ✅
      Model Answer
      ▼ Reveal

      This design leverages Git-compatible content-addressable storage with Apache Cassandra and Amazon S3 to store commits, branches, and blobs efficiently at scale. Diff computation is optimized using libgit2 on the server with Redis caching to meet strict latency requirements. Branching and merging use three-way merge algorithms with Cassandra lightweight transactions to handle concurrency and conflicts. Pull requests are managed via an event-driven architecture using Apache Kafka for lifecycle events and WebSocket notifications. Security is ensured with AWS KMS envelope encryption and TLS 1.3. Offline client support is enabled through local Git repositories and efficient sync protocols.

      • Use Apache Cassandra with partition keys based on repository ID to efficiently query commit history and branch pointers.
      • Store large immutable blobs in Amazon S3 to leverage scalable, durable object storage.
      • Implement server-side diff computation using libgit2 and cache results in Redis with TTL and invalidation on new commits.
      • Use Cassandra lightweight transactions to prevent lost updates on branch pointers during concurrent merges.
      • Adopt three-way merge with Git-style conflict markers to enable client-side conflict resolution and offline support.
      • Stream pull request lifecycle events via Apache Kafka partitioned by repository ID to guarantee ordering and enable idempotent consumers.
      • Use WebSocket connections to push real-time notifications to clients based on Kafka events.
      • Apply AWS KMS envelope encryption for data at rest and TLS 1.3 for all client-server communication.
      • Enable offline client operations with local Git repositories and sync using Git fetch/push protocols with delta compression to minimize bandwidth.
      • Git-compatible content-addressable storage with Apache Cassandra for metadata and Amazon S3 for blobs enables scalable, immutable storage.
      • Server-side diff computation using libgit2 combined with Redis caching meets strict P99 latency requirements.
      • Three-way merge with Cassandra lightweight transactions ensures concurrency control and conflict detection for branching.
      • Apache Kafka event streaming partitions by repository ID to guarantee ordered pull request event processing and real-time notifications.
      • AWS KMS envelope encryption for data at rest and TLS 1.3 for transport secure repository data end-to-end.
      • Client-side Git repositories enable offline commits; sync uses efficient Git fetch/push protocols with delta compression.
      • WebSocket-based real-time notifications integrated with Kafka events provide low-latency user updates.
      • Lightweight transactions in Cassandra prevent lost updates on branch pointers during concurrent merges.

      Practice

      (1/5)
      1. In a cache stampede prevention system, at Step 4 the Cache Rebuilder checks if a distributed lock is held before regenerating the cache. What happens if the lock is already held by another process?
      easy
      A. The Cache Rebuilder immediately regenerates the cache anyway, risking duplicate work
      B. The Cache Rebuilder waits or returns stale data until the lock is released
      C. The Cache Rebuilder deletes the cache key to force a fresh fetch next time
      D. The Cache Rebuilder escalates the request to the backend synchronously

      Solution

      1. Step 1: Trace Cache Rebuilder behavior at Step 4

        If the distributed lock is held, another process is regenerating the cache.
      2. Step 2: Understand correct handling

        The Cache Rebuilder waits or serves stale data to avoid duplicate regeneration and backend overload.
      3. Final Answer:

        Option B -> Option B
      4. Quick Check:

        Waiting or serving stale data prevents duplicate cache rebuilds [check]
      Hint: Lock held -> wait or serve stale, no duplicate rebuilds [OK]
      Common Mistakes:
      2. In the cache warming system, at step 3 the Cache Loader fetches data from the database. What happens if the database response is delayed beyond the configured timeout?
      easy
      A. Cache Loader retries indefinitely until data is fetched, blocking subsequent steps
      B. Cache Loader falls back to a secondary cache layer to fetch the data synchronously
      C. Cache Loader marks the key as stale and skips warming that key for this cycle
      D. Cache Loader writes a null value to cache to prevent cache miss storms

      Solution

      1. Step 1: Trace Cache Loader timeout behavior

        On timeout, Cache Loader skips warming that key to avoid blocking the entire warming process.
      2. Step 2: Eliminate retry and fallback options

        Indefinite retries block warming; fallback cache is not part of warming design; null writes risk stale data.
      3. Final Answer:

        Option C -> Option C
      4. Quick Check:

        Timeout causes skipping key warming to maintain overall progress [check]
      Hint: Timeout skips key warming to avoid blocking cache load [check]
      Common Mistakes:
      3. A search system must provide instant prefix matches for millions of users typing simultaneously, with minimal latency and no external dependencies. Which data structure is best suited to handle this requirement?
      easy
      A. Use Elasticsearch with inverted indexes to handle prefix queries efficiently
      B. Use a prefix hash map that stores all prefixes as keys for O(1) lookups
      C. Use a Trie data structure in-memory to provide fast prefix lookups without network overhead
      D. Use a relational database with LIKE queries on indexed columns for prefix matching

      Solution

      1. Step 1: Analyze latency and dependency constraints

        Elasticsearch involves network calls and disk I/O, increasing latency and external dependencies, so it's unsuitable for minimal latency prefix lookups.
      2. Step 2: Evaluate data structure suitability

        A Trie provides efficient in-memory prefix matching with O(k) lookup time (k = prefix length), ideal for millions of concurrent users with minimal latency.
      3. Final Answer:

        Option C -> Option C
      4. Quick Check:

        Trie offers fast prefix search with no external calls [check]
      Hint: Trie = fast prefix search in-memory, no network [check]
      Common Mistakes:
      4. If the Kafka cluster is unavailable, what exact failure occurs in the notification system's delivery path?
      medium
      A. Notifications are lost silently because the delivery workers cannot fetch messages
      B. Notification producers block synchronously, causing upstream request latency spikes and potential timeouts
      C. Delivery workers switch to a backup queue automatically with no message loss
      D. The system falls back to synchronous HTTP calls to devices, causing slow delivery

      Solution

      1. Step 1: Trace failure impact of Kafka unavailability

        Kafka unavailability blocks producers or causes retries; delivery workers cannot fetch messages from Kafka.
      2. Step 2: Evaluate fallback mechanisms

        In a robust design, a backup queue or dead-letter queue is used to avoid message loss; synchronous HTTP fallback (D) is not automatic; silent loss (A) is unacceptable.
      3. Final Answer:

        Option C -> Option C
      4. Quick Check:

        Backup queue prevents message loss during Kafka downtime [check]
      Hint: Backup queue prevents loss when Kafka is down [check]
      Common Mistakes:
      5. At 10x scale, which component fails first in a chat system with millions of concurrent WebSocket connections, and why?
      hard
      A. The database fails first due to increased query volume
      B. The message queue fails first due to partition overload
      C. The load balancer fails first due to inability to route increased traffic
      D. The chat servers fail first because WebSocket connections are stateful and cannot be resharded easily

      Solution

      1. Step 1: Identify stateful bottlenecks at 10x scale

        Databases (A) and message queues (C) can be scaled horizontally with sharding and partitioning. Load balancers (D) are stateless and scale well with more instances.
      2. Step 2: Understand WebSocket connection statefulness

        Chat servers maintain persistent WebSocket connections bound to server memory. These connections cannot be moved or resharded easily, causing chat servers to hit resource limits first.
      3. Final Answer:

        Option D -> Option D
      4. Quick Check:

        Stateful WebSocket connections -> chat servers bottleneck first at scale [check]
      Hint: Stateful WebSocket connections block chat server scaling first [OK]
      Common Mistakes:
      Recommended

      Ready to prove it?

      Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

      10 questions~5 mineasy hard