Distributed counters pattern in Firebase - Step-by-Step Execution

Process Flow - Distributed counters pattern

Start increment request

↓

Select a shard randomly

↓

Increment counter in selected shard

↓

Save updated shard count

↓

Repeat increments as needed

↓

To get total count

↓

Read all shards

↓

Sum all shard counts

↓

Return total count

Each increment updates one shard randomly to reduce contention. Total count is sum of all shards.

Execution Sample

Firebase

function incrementCounter() {
  const shardId = randomShard();
  const shardRef = db.collection('counters').doc(shardId);
  shardRef.update({count: firebase.firestore.FieldValue.increment(1)});
}

function getTotalCount() {
  return db.collection('counters').get().then(sumShards); 
}

Increment updates one shard; total count sums all shards.

Process Table

Step	Action	Shard Selected	Shard Count Before	Shard Count After	Total Count Calculation
1	Increment request received	-	-	-	-
2	Select shard randomly	shard_3	-	-	-
3	Read shard_3 count	shard_3	5	-	-
4	Increment shard_3 count	shard_3	5	6	-
5	Save updated shard_3 count	shard_3	-	6	-
6	Increment request received	-	-	-	-
7	Select shard randomly	shard_1	-	-	-
8	Read shard_1 count	shard_1	3	-	-
9	Increment shard_1 count	shard_1	3	4	-
10	Save updated shard_1 count	shard_1	-	4	-
11	Request total count	-	-	-	-
12	Read all shards counts	-	-	-	shard_1=4, shard_2=2, shard_3=6
13	Sum all shards	-	-	-	4 + 2 + 6 = 12
14	Return total count	-	-	-	12

💡 Total count returned after summing all shard counts.

Status Tracker

Variable	Start	After Step 4	After Step 9	After Step 13
shard_1_count	3	3	4	4
shard_2_count	2	2	2	2
shard_3_count	5	6	6	6
total_count	0	0	0	12

Key Moments - 3 Insights

Why do we update only one shard per increment instead of all shards?

Updating one shard reduces conflicts and improves performance, as shown in steps 2-5 and 7-10 where only one shard is incremented each time.

How is the total count calculated from shards?

The total count is the sum of all shard counts, as shown in steps 12-14 where all shards are read and summed.

What happens if two increments select the same shard at the same time?

Firebase handles concurrent updates safely with atomic increments, so counts are correctly updated without loss, as implied in steps 3-5 and 8-10.

Visual Quiz - 3 Questions

Look at the execution table, what is the count of shard_3 after step 4?

A 5

B 4

C 6

D 3

💡 Hint

Check the 'Shard Count After' column at step 4 in the execution_table.

At which step does the total count become available?

A Step 14

B Step 12

C Step 11

D Step 10

💡 Hint

Look at the 'Total Count Calculation' column in the execution_table for when the total is returned.

If increments always selected the same shard, what would change in the execution_table?

A Total count would not change

B Only one shard's count would increase repeatedly

C All shards would increase evenly

D Shards would be deleted

💡 Hint

Refer to the 'Shard Selected' column in the execution_table to see shard selection impact.

Concept Snapshot

Distributed counters split count into shards.
Each increment updates one shard randomly.
Total count is sum of all shards.
Reduces write conflicts and scales better.
Use atomic increments for safe updates.

Full Transcript

The distributed counters pattern splits a counter into multiple shards to reduce write conflicts. When an increment request happens, the system picks one shard randomly and increments its count atomically. This avoids many clients trying to update the same document at once. To get the total count, the system reads all shards and sums their counts. This pattern improves performance and scalability in Firebase by distributing load. The execution table shows increments updating individual shards and the total count calculated by summing all shards. Variables track shard counts changing step-by-step. Key moments clarify why only one shard is updated per increment and how totals are computed. The visual quiz tests understanding of shard counts and total calculation steps.

Practice

1. What is the main reason to use the distributed counters pattern in Firebase?

easy

A. To avoid write conflicts by splitting counts into multiple shards

B. To store large files efficiently

C. To speed up user authentication

D. To reduce network bandwidth usage

Solution

1. Step 1: Understand the problem with single counters

   Single counters cause write conflicts when many users update at once.

2. Step 2: How distributed counters solve this

   Splitting the count into shards lets many users write to different parts, avoiding conflicts.

3. Final Answer:

   To avoid write conflicts by splitting counts into multiple shards -> Option A

4. Quick Check:

   Distributed counters = avoid conflicts [OK]

Hint: Distributed counters split counts to avoid conflicts [OK]

Common Mistakes:

• Thinking it speeds up authentication
• Confusing with file storage
• Assuming it reduces bandwidth

2. Which Firebase data structure is typically used to implement shards in a distributed counter?

easy

A. A single document with a numeric field

B. A collection of user profiles

C. Multiple documents each representing a shard

D. A storage bucket

Solution

1. Step 1: Identify how shards are stored

   Each shard is stored as a separate document to allow independent updates.

2. Step 2: Why multiple documents

   Multiple documents reduce write conflicts by spreading writes across shards.

3. Final Answer:

   Multiple documents each representing a shard -> Option C

4. Quick Check:

   Shards = multiple documents [OK]

Hint: Shards are separate documents, not single fields [OK]

Common Mistakes:

• Using a single document for all shards
• Confusing shards with user profiles
• Thinking shards are stored in storage buckets

3. Given this code snippet to increment a distributed counter shard in Firebase Firestore, what is the expected behavior?

const shardId = Math.floor(Math.random() * 5);
const shardRef = firestore.collection('counters').doc('counter1').collection('shards').doc(shardId.toString());
await shardRef.update({ count: firebase.firestore.FieldValue.increment(1) });

medium

A. It increments the count in a random shard to reduce write conflicts

B. It increments the count in all shards simultaneously

C. It causes an error because shardId is a number

D. It resets the count to 1 in the selected shard

Solution

1. Step 1: Understand shard selection

   The code picks a random shard ID between 0 and 4 to update.

2. Step 2: What update does

   It increments the count field in that shard by 1, avoiding conflicts.

3. Final Answer:

   It increments the count in a random shard to reduce write conflicts -> Option A

4. Quick Check:

   Random shard increment = reduce conflicts [OK]

Hint: Random shard update avoids conflicts [OK]

Common Mistakes:

• Thinking it updates all shards at once
• Assuming shardId type causes error
• Believing it resets count instead of incrementing

4. You have a distributed counter with 3 shards but the total count is incorrect. Which of these is a likely cause?

medium

A. You incremented the same shard multiple times

B. You forgot to sum all shard counts when reading the total

C. You used a single document instead of shards

D. You used a random shard for increments

Solution

1. Step 1: Check how total count is calculated

   Total count must be the sum of all shard counts.

2. Step 2: Identify missing sum

   If you read only one shard or forget to sum, total will be wrong.

3. Final Answer:

   You forgot to sum all shard counts when reading the total -> Option B

4. Quick Check:

   Sum all shards for total [OK]

Hint: Always sum all shards to get total count [OK]

Common Mistakes:

• Assuming incrementing same shard causes wrong total
• Confusing single document with shards
• Thinking random shard increments cause errors

5. You want to design a distributed counter with 10 shards for a high-traffic app. Which approach best balances accuracy and performance?

hard

A. Increment shards sequentially in a fixed order

B. Increment all shards on each update and read any one shard for total

C. Use a single document counter with transaction retries

D. Increment a random shard on each update and sum all shards when reading total

Solution

1. Step 1: Understand update strategy

   Incrementing a random shard spreads writes and avoids conflicts.

2. Step 2: Understand read strategy

   Summing all shards gives accurate total, though slightly delayed.

3. Step 3: Why other options fail

   Incrementing all shards wastes writes; single document causes conflicts; sequential increments cause bottlenecks.

4. Final Answer:

   Increment a random shard on each update and sum all shards when reading total -> Option D

5. Quick Check:

   Random increment + sum shards = best balance [OK]

Hint: Random increments + sum shards = best practice [OK]

Common Mistakes:

• Trying to increment all shards each time
• Using single document counter under high load
• Incrementing shards in fixed order causing bottlenecks