Bigtable for time-series data in GCP - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Architecture

intermediate

Choosing the best row key design for time-series data in Bigtable

You want to store sensor readings collected every second in Bigtable. Which row key design will best prevent hotspotting and allow efficient range scans for recent data?

A Use a row key with the sensor ID followed by the reversed timestamp (e.g., sensor123#99999999999999 - timestamp).

B Use a row key with the timestamp in ascending order followed by the sensor ID (e.g., 20240601120000#sensor123).

C Use a row key with the sensor ID followed by the timestamp in ascending order (e.g., sensor123#20240601120000).

D Use a row key with a random UUID followed by the timestamp (e.g., uuid#20240601120000).

💡 Hint

Think about how Bigtable stores rows lexicographically and how to avoid all writes going to the same region.

Using the sensor ID followed by a reversed timestamp distributes writes across Bigtable nodes and allows efficient scans for recent data by scanning rows with decreasing timestamps.

❓ service_behavior

intermediate

Bigtable read latency for time-series queries

You query Bigtable for sensor data over the last hour using a row key range scan. What behavior should you expect regarding read latency?

A Read latency will be constant regardless of the time range size because Bigtable indexes all data.

B Read latency increases linearly with the number of rows scanned in the time range.

C Read latency decreases as the time range increases because Bigtable caches more data.

D Read latency is unpredictable and depends only on network speed.

💡 Hint

Consider how Bigtable stores data and how scanning more rows affects performance.

Bigtable stores data sorted by row key. Scanning a larger time range means reading more rows, so latency increases roughly linearly with the number of rows scanned.

❓ security

advanced

Securing Bigtable access for time-series ingestion

You have multiple IoT devices sending time-series data to Bigtable. Which approach best secures write access while minimizing operational overhead?

A Use a proxy service that authenticates devices and writes to Bigtable using a single service account with limited permissions.

B Use Cloud IAM to grant each device a unique service account with only write permissions to the specific Bigtable instance.

C Allow all devices to write anonymously to Bigtable by disabling authentication temporarily during ingestion.

D Create a single service account with full Bigtable admin permissions and share its key with all devices.

💡 Hint

Think about security best practices and operational simplicity when many devices are involved.

Using a proxy service centralizes authentication and authorization, reducing the risk of key leakage and simplifying permission management. Devices authenticate to the proxy, which writes to Bigtable with a controlled service account.

✅ Best Practice

advanced

Optimizing Bigtable schema for time-series data compression

Which schema design choice helps Bigtable compress time-series data most effectively?

A Store sensor readings in a single column family with timestamps as part of the row key.

B Store each sensor reading as a separate column with a unique timestamp as the column qualifier.

C Store sensor readings in a single column with timestamps encoded in the cell value.

D Store sensor readings in a single column family with timestamps as column qualifiers sorted in ascending order.

💡 Hint

Consider how Bigtable compresses data within column families and the effect of sorted column qualifiers.

Bigtable compresses data within column families and benefits from sorted column qualifiers. Using timestamps as sorted column qualifiers in a single column family allows better compression and efficient scans.

🧠 Conceptual

expert

Impact of hot-spotting on Bigtable write throughput for time-series data

You notice Bigtable write throughput drops significantly during peak sensor data ingestion. Which cause best explains this behavior?

A Write throughput drops because Bigtable limits writes per column family regardless of row key distribution.

B Bigtable automatically balances writes evenly, so throughput drops are caused by network congestion only.

C All writes are targeting a small range of row keys due to monotonically increasing timestamps causing hotspotting on a single tablet server.

D The Bigtable instance is running out of storage space, causing automatic throttling of all writes.

💡 Hint

Think about how Bigtable partitions data and how row key design affects write distribution.

Monotonically increasing row keys cause all writes to go to the same tablet server, creating a hotspot that limits write throughput. Proper row key design avoids this by distributing writes.

Practice

1. What is the main reason to use Google Cloud Bigtable for time-series data?

easy

A. It provides built-in machine learning models for time-series forecasting.

B. It automatically creates visual dashboards for time-series data.

C. It can handle large volumes of data with fast read and write speeds.

D. It stores data only in JSON format for easy access.

Solution

1. Step 1: Understand Bigtable's design for data volume

   Bigtable is designed to store very large amounts of data efficiently, especially for time-series data which grows quickly.

2. Step 2: Recognize Bigtable's performance benefits

   It offers fast read and write speeds, which is essential for time-series data that updates frequently.

3. Final Answer:

   It can handle large volumes of data with fast read and write speeds. -> Option C

4. Quick Check:

   Bigtable = fast, large data [OK]

Hint: Bigtable excels at fast, large-scale data storage [OK]

Common Mistakes:

• Confusing Bigtable with visualization tools
• Assuming Bigtable has built-in ML models
• Thinking Bigtable only stores JSON

2. Which of the following is the correct way to define a Bigtable instance using gcloud CLI for time-series data?

easy

A. gcloud bigtable instance create my-instance --cluster=my-cluster --zone=us-central1-b

B. gcloud bigtable create instance my-instance --zone=us-central1-b --type=DEVELOPMENT

C. gcloud bigtable instances new my-instance --cluster-zone=us-central1-b --instance-type=PRODUCTION

D. gcloud bigtable instances create my-instance --cluster=my-cluster --cluster-zone=us-central1-b --display-name="My Instance" --instance-type=PRODUCTION

Solution

1. Step 1: Review gcloud CLI syntax for Bigtable instance creation

   The correct command uses 'gcloud bigtable instances create' with required flags for cluster, zone, display name, and instance type.

2. Step 2: Compare options with correct syntax

   gcloud bigtable instances create my-instance --cluster=my-cluster --cluster-zone=us-central1-b --display-name="My Instance" --instance-type=PRODUCTION matches the official syntax exactly, including cluster and zone flags.

3. Final Answer:

   gcloud bigtable instances create my-instance --cluster=my-cluster --cluster-zone=us-central1-b --display-name="My Instance" --instance-type=PRODUCTION -> Option D

4. Quick Check:

   Correct gcloud syntax = gcloud bigtable instances create my-instance --cluster=my-cluster --cluster-zone=us-central1-b --display-name="My Instance" --instance-type=PRODUCTION [OK]

Hint: Use 'gcloud bigtable instances create' with cluster and zone [OK]

Common Mistakes:

• Using incorrect command verbs like 'create instance' or 'instances new'
• Missing required flags like --cluster-zone
• Confusing instance type flags

3. Given the following Bigtable row key design for time-series data: deviceID#timestamp, what will be the order of rows when scanned?

medium

A. Rows are ordered by deviceID first, then by timestamp ascending.

B. Rows are ordered by timestamp first, then by deviceID ascending.

C. Rows are unordered and returned randomly.

D. Rows are ordered by timestamp descending, then deviceID.

Solution

1. Step 1: Understand Bigtable row key sorting

   Bigtable sorts rows lexicographically by row key from left to right.

2. Step 2: Analyze the key format deviceID#timestamp

   Since deviceID is first, rows are grouped by deviceID, then sorted by timestamp ascending within each device.

3. Final Answer:

   Rows are ordered by deviceID first, then by timestamp ascending. -> Option A

4. Quick Check:

   Row key order = deviceID then timestamp [OK]

Hint: Row keys sort left to right lex order [OK]

Common Mistakes:

• Assuming timestamp sorts first
• Thinking rows are unordered
• Confusing ascending vs descending order

4. You notice your Bigtable time-series data query is slow. Which of the following is a likely cause and fix?

medium

A. Cause: Using multiple clusters. Fix: Use a single cluster only.

B. Cause: Row keys are timestamp first causing hotspotting. Fix: Reverse the key to deviceID#timestamp.

C. Cause: Too many column families. Fix: Add more column families.

D. Cause: Data stored in JSON format. Fix: Convert data to XML.

Solution

1. Step 1: Identify hotspotting from row key design

   If timestamp is first in the key, writes concentrate on a small range causing slow queries.

2. Step 2: Fix by changing row key order

   Placing deviceID first distributes writes and reads better, improving performance.

3. Final Answer:

   Cause: Row keys are timestamp first causing hotspotting. Fix: Reverse the key to deviceID#timestamp. -> Option B

4. Quick Check:

   Hotspotting fix = reorder row key [OK]

Hint: Avoid timestamp-first keys to prevent hotspots [OK]

Common Mistakes:

• Thinking multiple clusters slow queries
• Adding column families improves speed
• Confusing data format impact on speed

5. You want to design a Bigtable schema for IoT sensor data that arrives every second from thousands of devices. Which approach best supports efficient queries for recent data per device?

hard

A. Use row keys as deviceID#reversed-timestamp to keep recent data first and group by device.

B. Use row keys as timestamp#deviceID to sort by time globally.

C. Store all data in a single row per device with columns for each timestamp.

D. Use random UUIDs as row keys to distribute load evenly.

Solution

1. Step 1: Consider query pattern for recent data per device

   Queries often request recent data, so sorting recent timestamps first helps.

2. Step 2: Use reversed timestamp in row key

   Reversing timestamp (e.g., max - timestamp) sorts recent data at the top for fast access.

3. Step 3: Group by deviceID first

   This groups data by device, making queries per device efficient.

4. Final Answer:

   Use row keys as deviceID#reversed-timestamp to keep recent data first and group by device. -> Option A

5. Quick Check:

   Reversed timestamp + deviceID = fast recent queries [OK]

Hint: Reverse timestamp in key for recent-first order [OK]

Common Mistakes:

• Using timestamp first causes hotspots
• Storing all data in one row causes size limits
• Random UUID keys prevent efficient range scans