Bird
Raised Fist0

Design a Code Repository System - Diff, Branch, Merge, Pull Request

HLDhardInterview freq: medium
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
🏗️
Design a Code Repository System - Diff, Branch, Merge, Pull Request
Design a scalable, consistent, and collaborative code repository system supporting branching, merging, diffs, and pull requests at large scale.
hardStorage and CollaborationFreq: medium
Prerequisites:System Design FundamentalsCAP TheoremDistributed Version Control ConceptsConsistency ModelsStorage Systems
⚙️
Functional Requirements
  • Support creating, deleting, and listing branches in repositories.
  • Enable users to commit code changes with diffs stored efficiently.
  • Support merging branches with conflict detection and resolution.
  • Allow pull requests with code review, comments, and approval workflows.
  • Provide efficient diff generation between commits and branches.
  • Support user authentication and access control per repository.
📊
Non-Functional Requirements
  • System must support 50 million active repositories and 200 million users globally.
  • 99.99% availability SLA for repository read and write operations.
  • P99 latency for commit and merge operations under 200ms.
  • Data durability with replication across at least 3 data centers.
  • Support up to 1 million concurrent pull requests and merges daily.
❓
Clarifying Questions to Ask
  • What is the expected average and peak commit frequency per repository?
  • Should the system support large binary files or only source code?
  • What consistency guarantees are required for branch and merge operations?
  • Is the system expected to support private and public repositories with different access controls?
  • What is the expected size range of repositories and average commit size?
  • Are pull requests expected to support multi-branch or multi-repository merges?
Out of Scope
✗ Continuous Integration/Continuous Deployment (CI/CD) pipelines.
✗ Code search and indexing features.
✗ IDE integrations and local git client implementations.
✗ Advanced analytics and repository insights.
📌
Assumptions
  • 200 million active users globally.
  • Average 0.2 commits per user per day (40 million commits/day).
  • Average commit size 100 KB (diffs compressed).
  • Retention period for commits and metadata: 3 years.
  • Replication factor: 3 for durability.
⚡
QPS Calculation
write qps40M commits/day ÷ 86400 sec ≈ 463 writes/sec
peak write qpsAssuming 5x peak factor → 463 × 5 = 2315 writes/sec
read qpsAssuming 10 reads per write → 463 × 10 = 4630 reads/sec
peak read qpsAssuming 5x peak factor → 4630 × 5 = 23150 reads/sec
💾
Storage Estimate
daily storage40M commits × 100 KB = 4,000,000,000 KB = 3.73 TB/day
retention storage3.73 TB/day × 365 × 3 years = 4,086 TB ≈ 4 PB
with replication4 PB × 3 = 12 PB total storage
🖥️
Server Estimate
commit processingAssuming 1000 writes/sec per commit server → 3 commit servers for peak load
metadata storageUsing Cassandra with 1 TB/node → 12 PB / 1 TB = 12000 nodes; with compression and tiering, estimate 3000 nodes
🏗️
Architecture Diagram
Architecture Diagram
          +----------------------+
          |      User Clients     |
          +----------+-----------+
                     | HTTPS REST API
          +----------v-----------+
          |   API Gateway Layer  |
          +----------+-----------+
                     | Authenticated Requests
          +----------v-----------+
          |  Commit Service (Git) |
          +----------+-----------+
                     | Write commits, diffs
          +----------v-----------+
          | Metadata Store (Cassandra) |
          +----------+-----------+
                     | Store commit metadata, branches
          +----------v-----------+
          | Blob Storage (S3/MinIO) |
          +----------+-----------+
                     | Store raw file blobs and diffs
          +----------v-----------+
          | Merge & Conflict Service |
          +----------+-----------+
                     | Handles merges, conflict detection
          +----------v-----------+
          | Pull Request Service  |
          +----------+-----------+
                     | Manages PR lifecycle, reviews
          +----------v-----------+
          | Notification Service  |
          +----------------------+
🧩
Components
API Gateway Layer
Authenticate, authorize, and route user requests to backend services.
Why thisKong API Gateway supports high throughput, plugin-based auth, and rate limiting at scale.
Why notNginx lacks native API management features and complex auth plugins needed for multi-tenant repos.
Commit Service (Git Backend)
Process commit requests, generate diffs, and update branch pointers.
Why thisCustom Git-compatible service optimized for distributed commit processing and incremental diffs.
Why notUsing vanilla Git server (e.g., git-http-backend) does not scale to millions of repos and concurrent commits.
Metadata Store (Apache Cassandra)
Store commit metadata, branch info, pull request states with high availability and linear scalability.
Why thisCassandra provides tunable consistency, high write throughput, and easy horizontal scaling for large metadata volumes.
Why notPostgreSQL requires complex manual sharding at 2315 writes/sec; B-tree writes don't scale linearly.
Blob Storage (Amazon S3 / MinIO)
Store raw file blobs, compressed diffs, and large binary objects reliably and durably.
Why thisS3 offers virtually unlimited storage, 99.999999999% durability, and cost-effective tiering for large data.
Why notHDFS is optimized for batch workloads and has higher latency unsuitable for interactive commit reads.
Merge & Conflict Service
Perform branch merges, detect conflicts, and provide merge results asynchronously.
Why thisDedicated microservice using libgit2 for efficient merge algorithms and conflict detection at scale.
Why notDoing merges inline in Commit Service blocks write throughput and increases latency.
Pull Request Service
Manage pull request lifecycle, code reviews, comments, and approval workflows.
Why thisEvent-driven microservice architecture with Kafka for asynchronous PR state updates and notifications.
Why notMonolithic design leads to poor scalability and tight coupling of commit and review logic.
Notification Service
Send notifications for PR updates, merge results, and comments via email and push.
Why thisKafka and Redis Streams enable scalable, reliable, and low-latency notification delivery.
Why notPolling or synchronous notification methods increase latency and reduce system responsiveness.

Practice

(1/5)
1. In the recommendation system request flow, at Step 4 the Embedding Service receives a user rating event from Kafka. What does it do if the user embedding is not found in the cache?
easy
A. It returns an error to the user indicating embedding missing
B. It fetches the embedding from persistent storage and updates the cache
C. It ignores the event and waits for the next rating update
D. It recomputes the embedding synchronously before processing the event

Solution

  1. Step 1: Trace Embedding Service behavior at Step 4

    When cache miss occurs, the service must retrieve embedding from persistent storage to maintain consistency.

  2. Step 2: Evaluate options

    Returning error (A) breaks user experience; ignoring event (C) loses data; synchronous recomputation (D) is expensive and blocks processing. Fetching from storage and updating cache (B) ensures availability and eventual consistency.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    Cache miss triggers fetch from storage and cache update [check]
Hint: Cache miss -> fetch from storage and update cache [check]
Common Mistakes:
2. A webhook delivery system must guarantee ordered delivery of events per subscriber, even under retries and failures. Which component is best suited to enforce per-subscriber ordering?
easy
A. A partitioned message queue like Kafka keyed by subscriber ID
B. A distributed cache like Redis to store last delivered event per subscriber
C. A relational database with a global sequence number column
D. A load balancer that routes events to the same delivery worker

Solution

  1. Step 1: Identify ordering guarantee mechanism

    Kafka partitions keyed by subscriber ID ensure all events for that subscriber go to the same partition, preserving order.

  2. Step 2: Eliminate other options

    Redis cache lacks strong ordering guarantees under retries; relational DB global sequence causes contention and latency; load balancer routing alone doesn't guarantee ordering without stateful processing.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Kafka partition key -> per-subscriber ordering guaranteed [check]
Hint: Partition by subscriber ID for ordering [check]
Common Mistakes:
3. If the BFF layer crashes and is unavailable, what is the immediate failure impact on client requests?
medium
A. Microservices handle requests directly, bypassing BFF automatically
B. Clients' requests fail immediately as BFF is the only entry point for client-specific aggregation
C. API Gateway retries requests indefinitely until BFF recovers
D. Clients receive stale cached data from API Gateway without error

Solution

  1. Step 1: Understand BFF's role in request flow

    BFF is the dedicated layer for client-specific aggregation; if it crashes, requests cannot be processed as usual.

  2. Step 2: Analyze fallback behavior

    Microservices handle requests directly, bypassing BFF automatically is correct because some architectures allow API Gateway to fallback to direct microservice calls, though without client-specific aggregation. Clients receive stale cached data from API Gateway without error is wrong as API Gateway typically does not cache full responses. Clients' requests fail immediately as BFF is the only entry point for client-specific aggregation is incorrect because requests do not fail immediately if fallback exists. API Gateway retries requests indefinitely until BFF recovers is wrong since indefinite retries cause client timeouts.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    API Gateway fallback to microservices bypasses BFF on failure [check]
Hint: API Gateway fallback bypasses BFF if it crashes [check]
Common Mistakes:
4. If the distributed priority queue component fails and becomes unavailable, what is the immediate failure mode observed in the task scheduler?
medium
A. Worker nodes block waiting for tasks, causing execution delays and potential timeouts
B. Tasks are silently dropped without retries, causing data loss
C. Tasks continue executing but with incorrect priority ordering
D. The cron scheduler automatically reschedules tasks on all nodes

Solution

  1. Step 1: Understand role of distributed priority queue

    The queue is the source of truth for pending tasks; if unavailable, new tasks cannot be fetched.

  2. Step 2: Analyze failure impact

    Without the queue, tasks submitted may be lost if not persisted elsewhere; workers cannot fetch tasks, so blocking (B) is unlikely if queue is down; cron scheduler (D) does not reschedule tasks automatically.

  3. Step 3: Evaluate silent drops (C)

    Without queue durability, tasks may be lost silently if producers cannot enqueue or retries are not implemented.

  4. Final Answer:

    Option B -> Option B

  5. Quick Check:

    Queue failure causes silent task loss without retries [check]
Hint: Queue failure causes silent task loss without retries [check]
Common Mistakes:
5. For a system requiring cost optimization with infrequent data access but occasional retrieval within hours, which storage architecture is preferable?
medium
A. Implement warm storage with HDDs and cold storage with object storage, migrating data based on access patterns
B. Use only hot storage with SSDs for all data to ensure fastest access
C. Archive all data immediately to tape storage to minimize cost
D. Store all data in cold storage with no tiering to simplify design

Solution

  1. Step 1: Analyze access and cost requirements

    Data is infrequently accessed but needs retrieval within hours, so cold storage alone is too slow.

  2. Step 2: Evaluate tiering benefits

    Warm storage offers moderate cost and latency; cold storage reduces cost further. Tiering balances cost and retrieval time.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Warm + cold tiering balances cost and retrieval latency [check]
Hint: Warm + cold tiering balances cost and latency [check]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard