Atlas data federation concept in MongoDB - Mini Project: Build & Apply

Atlas Data Federation with MongoDB

📖 Scenario: You work for a company that stores customer data in multiple MongoDB collections across different clusters. You want to create a unified view of all customers using Atlas Data Federation, so you can query all data as if it were in one place.

🎯 Goal: Build a simple Atlas Data Federation query that combines customer data from two collections into one result set.

📋 What You'll Learn

Create two collections named customers_us and customers_eu with sample customer documents

Create a data federation virtual database that includes both collections

Write a query to find all customers from both collections

Filter customers with age greater than 30

💡 Why This Matters

🌍 Real World

Companies often store data in multiple databases or clusters. Atlas Data Federation lets you query all data together without moving it.

💼 Career

Understanding data federation helps in roles like data engineering and database administration where combining data from multiple sources is common.

1

Create two collections with sample customer data

Create two collections named customers_us and customers_eu with these exact documents: {"name": "Alice", "age": 28} and {"name": "Bob", "age": 35} in customers_us, and {"name": "Clara", "age": 40} and {"name": "David", "age": 25} in customers_eu.

MongoDB

// Insert documents into customers_us and customers_eu collections
// Your code here

Hint

Use insertMany to add multiple documents to each collection.

2

Configure the Atlas Data Federation virtual database

Create a data federation virtual database named customerFederation that includes the collections customers_us and customers_eu from your clusters.

MongoDB

db.customers_us.insertMany([{ "name": "Alice", "age": 28 }, { "name": "Bob", "age": 35 }])
db.customers_eu.insertMany([{ "name": "Clara", "age": 40 }, { "name": "David", "age": 25 }])
// Define the data federation virtual database configuration
// Your code here

Hint

Use the Atlas UI or API to create a data federation virtual database named customerFederation that includes both collections.

3

Write a query to find all customers from both collections

Write a MongoDB query on the customerFederation virtual database to find all documents from the combined collections customers_us and customers_eu.

MongoDB

db.customers_us.aggregate([
  // Your aggregation pipeline here
])

Hint

Use $unionWith to combine documents from customers_us and customers_eu.

4

Filter customers with age greater than 30

Extend the query on customerFederation to only return customers whose age is greater than 30.

MongoDB

db.customers_us.aggregate([
  { $unionWith: "customers_eu" },
  // Add a filter stage here
])

Hint

Add a $match stage after $unionWith to filter by age.

Practice

1. What is the main purpose of MongoDB Atlas Data Federation?

easy

A. To query data from multiple sources as if they were one database

B. To create backups of MongoDB databases automatically

C. To optimize single-node MongoDB performance

D. To convert SQL queries into MongoDB queries

Solution

1. Step 1: Understand the concept of Atlas Data Federation

   Atlas Data Federation allows querying data from different sources like cloud object stores and databases as a single database.

2. Step 2: Identify the main purpose

   The main goal is to simplify querying distributed data by treating it as one unified database.

3. Final Answer:

   To query data from multiple sources as if they were one database -> Option A

4. Quick Check:

   Atlas Data Federation = Unified querying [OK]

Hint: Think 'one query, many sources' for Atlas Federation [OK]

Common Mistakes:

• Confusing federation with backup features
• Thinking it only optimizes single-node performance
• Assuming it translates SQL queries automatically

2. Which of the following is the correct way to create a federated database in MongoDB Atlas Data Federation?

easy

A. Run a MongoDB shell command: createFederatedDB()

B. Use the Atlas UI to define data sources and create a federated database

C. Install a special MongoDB plugin on your local machine

D. Write a SQL script to link data sources

Solution

1. Step 1: Recall how federated databases are created

   Atlas Data Federation uses the Atlas web interface to configure data sources and create federated databases.

2. Step 2: Evaluate the options

   Only the Atlas UI method is correct; there is no shell command or plugin for this, nor SQL scripts.

3. Final Answer:

   Use the Atlas UI to define data sources and create a federated database -> Option B

4. Quick Check:

   Federated DB creation = Atlas UI [OK]

Hint: Atlas UI is your tool for federation setup [OK]

Common Mistakes:

• Trying to create federated DB via shell commands
• Looking for local plugins instead of cloud setup
• Using SQL scripts which are not supported

3. Given a federated database combining a MongoDB collection and an S3 bucket, what will the following query return?

db.federatedCollection.find({"type": "image"})

medium

A. Only files from S3 bucket with type 'image'

B. Only documents from MongoDB collection with type 'image'

C. Documents from MongoDB collection and files from S3 with type 'image'

D. An error because S3 data cannot be queried

Solution

1. Step 1: Understand federated query behavior

   Atlas Data Federation allows querying multiple sources like MongoDB and S3 as one collection.

2. Step 2: Analyze the query

   The query filters documents/files where the field "type" equals "image" across all federated sources.

3. Final Answer:

   Documents from MongoDB collection and files from S3 with type 'image' -> Option C

4. Quick Check:

   Federated query returns combined matching data [OK]

Hint: Federated queries combine all sources transparently [OK]

Common Mistakes:

• Assuming only MongoDB data is returned
• Thinking S3 data cannot be queried
• Expecting an error due to mixed sources

4. You wrote this query on a federated database:

db.federatedCollection.find({"status": "active"})

But it returns no results, even though you know some data has status 'active'. What could be the problem?

medium

A. You must use SQL syntax instead of MongoDB queries

B. The query syntax is invalid for federated databases

C. MongoDB Atlas does not support filtering on 'status' field

D. The federated database was not properly linked to all data sources

Solution

1. Step 1: Check query correctness

   The query syntax is valid MongoDB syntax and filtering on 'status' is supported.

2. Step 2: Consider federation setup

   If no results appear, likely the federated database is missing some data sources or not linked properly.

3. Final Answer:

   The federated database was not properly linked to all data sources -> Option D

4. Quick Check:

   Missing data sources cause empty results [OK]

Hint: Check federation links if queries return no data [OK]

Common Mistakes:

• Blaming query syntax instead of setup
• Thinking Atlas forbids filtering on fields
• Trying to use SQL syntax in MongoDB queries

5. You want to create a federated database that combines a MongoDB Atlas cluster, an S3 bucket, and a Google Cloud Storage bucket. Which of the following is true about querying this federated database?

hard

A. You can run a single MongoDB query that returns matching data from all three sources

B. You must query each source separately and merge results manually

C. Federation supports only two sources at a time, so you must create two federated databases

D. You can only query MongoDB Atlas data; cloud storage buckets are read-only

Solution

1. Step 1: Understand multi-source federation

   Atlas Data Federation supports combining multiple data sources including MongoDB Atlas and various cloud storage buckets.

2. Step 2: Query behavior on federated database

   A single MongoDB query can access and return matching data from all linked sources transparently.

3. Final Answer:

   You can run a single MongoDB query that returns matching data from all three sources -> Option A

4. Quick Check:

   Federation = unified multi-source queries [OK]

Hint: Federation lets one query cover many sources [OK]

Common Mistakes:

• Thinking you must query sources separately
• Believing federation limits to two sources
• Assuming cloud buckets are read-only and not queryable