Firestore collections and documents in GCP - Deep Dive

Overview - Firestore collections and documents

What is it?

Firestore is a cloud database that stores data in collections and documents. Collections are like folders that hold documents, and documents are like files that store data as key-value pairs. This structure helps organize data in a way that is easy to find and update. Firestore automatically scales and keeps data synced across devices.

Why it matters

Without collections and documents, storing and organizing data in the cloud would be chaotic and slow. Firestore's model makes it simple to group related data and access it quickly, even as your app grows. This means apps can respond faster and handle more users without extra work from developers.

Where it fits

Before learning Firestore collections and documents, you should understand basic database concepts like tables and records. After this, you can learn about Firestore queries, security rules, and data modeling for complex apps.

Mental Model

Core Idea

Firestore organizes data as collections of documents, where each document holds data fields and can link to subcollections, creating a flexible, nested structure.

Think of it like...

Imagine a filing cabinet where each drawer is a collection, and inside each drawer are folders (documents) holding sheets of paper (data fields). Some folders can have smaller folders inside them (subcollections) for more details.

Firestore Structure:

┌─────────────┐
│ Collection  │
│  (Drawer)   │
└─────┬───────┘
      │ contains
      ▼
┌─────────────┐
│ Document    │
│  (Folder)   │
│ ┌─────────┐ │
│ │ Fields  │ │
│ │ (Data)  │ │
│ └─────────┘ │
│     │       │
│     ▼       │
│ Subcollection│
│  (Smaller   │
│   Drawer)   │
└─────────────┘

Build-Up - 7 Steps

1

Foundation

Understanding Firestore Basics

Concept: Firestore stores data in a NoSQL format using collections and documents instead of tables and rows.

Firestore is a cloud database where data is stored in documents. Each document is a set of key-value pairs, like a small JSON object. Documents are grouped inside collections. Collections do not store data directly but hold documents. This structure is flexible and easy to scale.

Result

You can store and retrieve data organized in collections and documents instead of traditional tables.

Understanding that Firestore uses collections and documents instead of tables helps you think differently about data organization in cloud databases.

2

Foundation

Documents as Data Containers

Concept: Documents hold the actual data fields and can contain simple or complex data types.

A document in Firestore contains fields with values like strings, numbers, booleans, arrays, or even nested objects. Each document has a unique ID within its collection. Documents are independent and can be created, updated, or deleted without affecting others.

Result

You can create documents with data fields and uniquely identify them within collections.

Knowing that documents are self-contained data units helps you design your data for easy access and updates.

3

Intermediate

Collections as Groupings of Documents

Concept: Collections group related documents and provide a way to organize data hierarchically.

Collections are containers for documents. You can think of them as categories or groups. For example, a collection named 'users' might hold documents for each user. Collections do not store data themselves but organize documents. You can create multiple collections at the root or nested inside documents.

Result

You can organize your data logically by grouping documents into collections.

Understanding collections as logical groupings helps you plan your database structure for clarity and performance.

4

Intermediate

Subcollections for Nested Data

🤔Before reading on: do you think documents can contain other documents directly, or only collections can hold documents? Commit to your answer.

Concept: Documents can have subcollections, allowing nested data structures without embedding all data in one document.

Firestore allows documents to have subcollections, which are collections nested inside a document. This lets you model complex relationships, like a 'users' document having an 'orders' subcollection. Subcollections are independent collections and can have their own documents and subcollections.

Result

You can create nested data structures that are easy to query and update separately.

Knowing that subcollections exist prevents the mistake of putting too much data in one document and helps keep data organized and scalable.

5

Intermediate

Document IDs and Paths

🤔Before reading on: do you think document IDs are assigned automatically or must always be set by the user? Commit to your answer.

Concept: Each document has a unique ID within its collection, and documents are accessed by their full path including collections and document IDs.

Documents have IDs that can be set manually or generated automatically by Firestore. The full path to a document includes the collection name and document ID, like 'users/user123'. This path is used to read, write, or listen to the document. Paths help locate data precisely in the database.

Result

You can uniquely identify and access any document using its path and ID.

Understanding document paths and IDs is key to accessing and managing data correctly in Firestore.

6

Advanced

Data Modeling with Collections and Documents

🤔Before reading on: do you think embedding all related data in one document is better than using subcollections? Commit to your answer.

Concept: Choosing when to embed data in documents or use subcollections affects performance, scalability, and complexity.

Firestore limits document size to 1 MB, so large or growing data should be split into subcollections. Embedding small related data in a document is fast for reads but can cause issues if data grows. Using subcollections allows querying parts of data independently and keeps documents small. Good data modeling balances these tradeoffs.

Result

You design your database to be efficient, scalable, and easy to query.

Knowing how to model data with collections and documents prevents performance problems and supports app growth.

7

Expert

Consistency and Transactions in Firestore

🤔Before reading on: do you think Firestore guarantees strong consistency across all documents in different collections automatically? Commit to your answer.

Concept: Firestore provides strong consistency within documents and supports transactions to update multiple documents atomically.

Firestore ensures that reads and writes to a single document are strongly consistent. However, when updating multiple documents across collections or subcollections, you must use transactions or batch writes to keep data consistent. Transactions run atomically, so either all changes succeed or none do. This is crucial for maintaining data integrity in complex operations.

Result

You can safely update multiple related documents without risking partial updates.

Understanding Firestore's consistency model and transactions helps avoid subtle bugs and data corruption in production.

Under the Hood

Firestore stores data in a distributed cloud system where collections and documents are logical containers. Each document is stored as a separate entity with its own metadata and data fields. The system uses document paths to locate data quickly. Subcollections are stored as separate collections linked to parent documents. Firestore uses indexes to speed up queries and supports real-time synchronization by listening to document changes.

Why designed this way?

Firestore was designed to be flexible and scalable for mobile and web apps. Using collections and documents instead of tables allows dynamic schemas and nested data. This design supports offline use, real-time updates, and easy scaling without complex joins. Alternatives like relational databases were less suited for these needs, so Firestore chose a document model for simplicity and performance.

Firestore Internal Structure:

┌───────────────┐
│ Client App    │
└──────┬────────┘
       │ Reads/Writes
       ▼
┌───────────────┐
│ Firestore API │
└──────┬────────┘
       │ Maps paths
       ▼
┌───────────────┐
│ Distributed   │
│ Storage Nodes │
│ ┌───────────┐ │
│ │ Documents │ │
│ │ & Indexes │ │
│ └───────────┘ │
└───────────────┘

Myth Busters - 4 Common Misconceptions

Quick: Do you think Firestore collections store data directly, or only documents do? Commit to your answer.

Common Belief: Collections store data directly like tables in a database.

Reality: Collections only hold documents; they do not store data themselves.

Why it matters: Thinking collections store data can lead to confusion when queries return documents, not collections, causing errors in data handling.

Quick: Do you think documents can contain other documents directly? Commit to your answer.

Common Belief: Documents can contain other documents inside them.

Reality: Documents cannot contain documents directly; only collections can hold documents, including subcollections inside documents.

Why it matters: Misunderstanding this leads to trying to nest documents inside documents, which Firestore does not support, causing design and query problems.

Quick: Do you think Firestore automatically handles large nested data inside one document? Commit to your answer.

Common Belief: You can store unlimited nested data inside a single document.

Reality: Documents have a size limit of 1 MB; large or growing data should be split into subcollections.

Why it matters: Ignoring document size limits causes write failures and poor app performance.

Quick: Do you think Firestore guarantees strong consistency across multiple documents without transactions? Commit to your answer.

Common Belief: All document updates are strongly consistent automatically, even across collections.

Reality: Strong consistency is guaranteed per document; multi-document updates require transactions or batch writes.

Why it matters: Assuming automatic consistency can cause data corruption or partial updates in multi-document operations.

Expert Zone

1

Subcollections are independent collections and can have their own security rules and indexes, allowing fine-grained access control.

2

Firestore's automatic indexing can be customized or disabled per field to optimize query performance and cost.

3

Document IDs can be chosen to encode meaningful information or use random IDs to avoid hotspots in distributed storage.

When NOT to use

Firestore collections and documents are not ideal for complex relational data requiring multi-table joins or heavy transactions. In such cases, a relational database like Cloud SQL is better. Also, for very large datasets with complex analytics, BigQuery or other data warehouses are more suitable.

Production Patterns

In production, developers use shallow queries to avoid reading large subcollections unnecessarily. They design data models to minimize document size and use transactions for critical multi-document updates. Security rules are applied at collection and document levels to protect data. Offline support is enabled by syncing documents selectively.

Connections

Relational Databases

Firestore collections and documents are a NoSQL alternative to tables and rows in relational databases.

Understanding relational tables helps grasp Firestore's different approach to organizing data without fixed schemas or joins.

JSON Data Format

Firestore documents store data as key-value pairs similar to JSON objects.

Knowing JSON structure helps understand how Firestore documents hold nested and flexible data.

File Systems

Firestore's collections and documents resemble folders and files in a file system hierarchy.

Recognizing this similarity aids in visualizing data organization and navigation in Firestore.

Common Pitfalls

#1 Trying to store large nested data inside one document exceeding size limits.

Wrong approach:

db.collection('users').doc('user1').set({ profile: { bio: '...', posts: [/* thousands of posts */] } })

Correct approach:

db.collection('users').doc('user1').set({ profile: { bio: '...' } });
db.collection('users').doc('user1').collection('posts').add({ /* post data */ })

Root cause:

Misunderstanding document size limits and how to use subcollections for large or growing data.

#2 Assuming collections can be queried like documents with data fields.

Wrong approach:

const data = db.collection('users').data(); // incorrect, collections have no data() method

Correct approach:

const snapshot = await db.collection('users').get();
snapshot.forEach(doc => console.log(doc.data()));

Root cause:

Confusing collections as data holders instead of containers for documents.

#3 Updating multiple documents without transactions, causing partial updates.

Wrong approach:

db.collection('users').doc('user1').update({ balance: 100 });
db.collection('accounts').doc('acc1').update({ balance: 200 });

Correct approach:

const batch = db.batch();
batch.update(db.collection('users').doc('user1'), { balance: 100 });
batch.update(db.collection('accounts').doc('acc1'), { balance: 200 });
await batch.commit();

Root cause:

Not using transactions or batch writes for atomic multi-document updates.

Key Takeaways

Firestore organizes data in collections and documents, where collections hold documents and documents store data fields.

Documents have unique IDs and can contain subcollections, enabling flexible and nested data structures.

Understanding document size limits and when to use subcollections is crucial for scalable data modeling.

Firestore guarantees strong consistency per document, but multi-document updates require transactions or batch writes.

Proper use of collections, documents, and transactions ensures efficient, reliable, and secure cloud data storage.

Practice

1. What is the correct way to describe Firestore's data structure?

easy

A. Documents are the top-level containers holding collections.

B. Documents contain collections, and collections contain key-value pairs.

C. Collections contain key-value pairs directly without documents.

D. Collections contain documents, and documents contain key-value pairs.

Solution

1. Step 1: Understand Firestore data hierarchy

   Firestore organizes data with collections at the top level, which hold documents.

2. Step 2: Identify what documents contain

   Documents store data as key-value pairs, not collections directly.

3. Final Answer:

   Collections contain documents, and documents contain key-value pairs. -> Option D

4. Quick Check:

   Firestore data = collections > documents > key-value pairs [OK]

Hint: Remember: Collections hold documents, documents hold data [OK]

Common Mistakes:

• Thinking documents hold collections directly
• Assuming collections store key-value pairs without documents
• Confusing the order of collections and documents

2. Which of the following is the correct syntax to get a reference to a document named user123 inside a collection users in Firestore using Node.js?

easy

A. const docRef = firestore.doc('user123/users');

B. const docRef = firestore.collection('users').doc('user123');

C. const docRef = firestore.collection('user123').doc('users');

D. const docRef = firestore.doc('users').doc('user123');

Solution

1. Step 1: Identify correct method to access collection and document

   Use collection('users') to access the collection, then doc('user123') to get the document.

2. Step 2: Check path correctness

   Direct firestore.doc(full/path) works for existing paths, but incorrect paths, reversed order, or misusing doc() on document references fail. Chaining collection().doc() is recommended.

3. Final Answer:

   const docRef = firestore.collection('users').doc('user123'); -> Option B

4. Quick Check:

   Use collection().doc() chaining for document refs [OK]

Hint: Chain collection() then doc() for document reference [OK]

Common Mistakes:

• Swapping collection and document names
• Using firestore.doc() with incorrect path format
• Confusing collection() and doc() order

3. Given the following Firestore code snippet, what will be the output if the document user123 exists with fields {name: 'Alice', age: 30}?

const docRef = firestore.collection('users').doc('user123');
const docSnap = await docRef.get();
if (docSnap.exists) {
  console.log(docSnap.data());
} else {
  console.log('No such document!');
}

medium

A. 'No such document!'

B. undefined

C. { name: 'Alice', age: 30 }

D. Error: Document not found

Solution

1. Step 1: Check if document exists

   The code checks docSnap.exists, which is true if the document is present.

2. Step 2: Output document data

   If exists, docSnap.data() returns the document fields as an object.

3. Final Answer:

   { name: 'Alice', age: 30 } -> Option C

4. Quick Check:

   docSnap.exists true means data() returns fields [OK]

Hint: docSnap.exists true means data() returns document fields [OK]

Common Mistakes:

• Assuming data() returns undefined if document exists
• Confusing exists check with data presence
• Expecting error instead of 'No such document!' message

4. You wrote this Firestore code to get a document but docSnap.exists is false:

const docRef = firestore.collection('users').doc();
const docSnap = await docRef.get();

What is the problem?

medium

A. Missing document ID in doc() causes a reference to a non-existing document.

B. You cannot call get() on a document reference.

C. Collection name 'users' is invalid without quotes.

D. You must use firestore.doc() instead of collection().doc().

Solution

1. Step 1: Check doc() usage

   Calling doc() without an ID creates a reference to a new document with an auto-generated ID, but it does not exist yet.

2. Step 2: Understand get() on new doc reference

   Trying to get() returns a snapshot with exists=false because the document does not exist.

3. Final Answer:

   Missing document ID in doc() causes a reference to a non-existing document. -> Option A

4. Quick Check:

   doc() needs ID to get existing document [OK]

Hint: Always provide document ID in doc() to read existing document [OK]

Common Mistakes:

• Assuming doc() without ID points to existing document
• Thinking get() cannot be called on document references
• Confusing collection and document method usage

5. You want to store user profiles in Firestore where each user has multiple addresses. Which is the best Firestore structure to model this data?

hard

A. A collection 'users' with documents for each user, and each user document has a subcollection 'addresses' with address documents.

B. A single collection 'addresses' with documents containing user info and address fields combined.

C. A collection 'users' where each document stores all addresses as a large array field.

D. Multiple collections named after each user ID, each containing address documents.

Solution

1. Step 1: Understand Firestore best practice for one-to-many

   Use a main collection for users, and subcollections for related multiple items like addresses.

2. Step 2: Evaluate options

   A collection 'users' with documents for each user, and each user document has a subcollection 'addresses' with address documents. uses subcollections for addresses, which is scalable and organized. A single collection 'addresses' with documents containing user info and address fields combined. mixes user and address data in one collection, which is less clear. A collection 'users' where each document stores all addresses as a large array field. stores all addresses in one field, which can be inefficient. Multiple collections named after each user ID, each containing address documents. creates many collections dynamically, which is not recommended.

3. Final Answer:

   A collection 'users' with documents for each user, and each user document has a subcollection 'addresses' with address documents. -> Option A

4. Quick Check:

   Use subcollections for related multiple items [OK]

Hint: Use subcollections for multiple related documents per user [OK]

Common Mistakes:

• Storing multiple addresses in one array field causing size limits
• Creating dynamic collections per user causing management issues
• Mixing unrelated data in one collection