BigQuery SQL and pricing model in GCP - Time & Space Complexity

Time Complexity: BigQuery SQL and pricing model

O(n)

Understanding Time Complexity

We want to understand how the cost and time of running BigQuery SQL queries grow as the amount of data increases.

Specifically, how does scanning more data affect the number of operations and pricing?

Scenario Under Consideration

Analyze the time complexity of running a simple SELECT query scanning a large table.

SELECT user_id, COUNT(*) AS visits
FROM `project.dataset.web_logs`
WHERE event_date BETWEEN '2024-01-01' AND '2024-01-31'
GROUP BY user_id
ORDER BY visits DESC
LIMIT 10
    

This query scans web log data for one month, groups by user, and returns the top 10 visitors.

Identify Repeating Operations

Look at what happens repeatedly during query execution.

• Primary operation: Scanning data blocks from storage.
• How many times: Once for each data block covering the date range scanned.
• Secondary operations: Grouping and sorting happen after scanning but depend on scanned data size.

How Execution Grows With Input

As the amount of data scanned grows, the number of data blocks read grows roughly in proportion.

Input Size (GB scanned)	Approx. Data Blocks Read
10	10 blocks
100	100 blocks
1000	1000 blocks

Pattern observation: The scanning cost grows linearly with the amount of data scanned.

Final Time Complexity

Time Complexity: O(n)

This means the time and cost increase directly in proportion to the size of the data scanned.

Common Mistake

[X] Wrong: "Query cost stays the same no matter how much data is scanned."

[OK] Correct: BigQuery charges and time depend on how much data the query reads, so scanning more data costs more time and money.

Interview Connect

Understanding how query cost grows with data size helps you design efficient queries and manage cloud costs wisely.

Self-Check

What if we added a filter that reduces scanned data by half? How would the time complexity and cost change?

Practice

1. What does BigQuery charge you for when you run a SQL query?

easy

A. The amount of data scanned by the query

B. The number of SQL commands used

C. The time it takes to write the query

D. The number of tables in your project

Solution

1. Step 1: Understand BigQuery pricing basics

   BigQuery charges based on how much data your query scans, not on commands or tables.

2. Step 2: Identify the correct pricing factor

   The cost depends on the volume of data scanned during query execution.

3. Final Answer:

   The amount of data scanned by the query -> Option A

4. Quick Check:

   Data scanned = cost [OK]

Hint: Remember: BigQuery bills by data scanned, not query length [OK]

Common Mistakes:

• Thinking cost depends on query complexity
• Assuming storage size affects query cost directly
• Confusing number of tables with cost

2. Which of the following is the correct way to write a simple BigQuery SQL SELECT statement?

easy

A. SELECT * FROM dataset.table;

B. GET * FROM dataset.table;

C. FETCH ALL FROM dataset.table;

D. READ * dataset.table;

Solution

1. Step 1: Recall BigQuery SQL syntax

   BigQuery uses standard SQL syntax where SELECT is used to retrieve data.

2. Step 2: Identify the correct syntax

   Only SELECT * FROM dataset.table; uses the correct SELECT statement format with FROM clause.

3. Final Answer:

   SELECT * FROM dataset.table; -> Option A

4. Quick Check:

   SELECT + FROM = correct query [OK]

Hint: SELECT is the keyword to get data in SQL [OK]

Common Mistakes:

• Using GET or FETCH instead of SELECT
• Omitting FROM keyword
• Incorrect command verbs

3. Consider this query:
SELECT COUNT(*) FROM `project.dataset.sales` WHERE sale_date >= '2023-01-01';
What will this query return?

medium

A. A list of all sales records from 2023-01-01 onwards

B. The sum of all sales amounts after 2023-01-01

C. The total number of sales records from 2023-01-01 onwards

D. An error because of incorrect date format

Solution

1. Step 1: Analyze the SELECT clause

   COUNT(*) counts the number of rows matching the condition.

2. Step 2: Understand the WHERE clause

   It filters records with sale_date on or after 2023-01-01.

3. Final Answer:

   The total number of sales records from 2023-01-01 onwards -> Option C

4. Quick Check:

   COUNT(*) + WHERE = count filtered rows [OK]

Hint: COUNT(*) counts rows, not sums values [OK]

Common Mistakes:

• Confusing COUNT with SUM
• Thinking it returns all records instead of count
• Assuming date format causes error

4. You wrote this query but it fails:
SELECT * FROM dataset.table WHERE date = 2023-01-01;
What is the error and how to fix it?

medium

A. Missing semicolon; add ; at end

B. Date value missing quotes; fix with WHERE date = '2023-01-01'

C. Table name incorrect; use backticks around table

D. SELECT * not allowed; specify columns explicitly

Solution

1. Step 1: Identify the error in WHERE clause

   Date literals must be in quotes in SQL; missing quotes cause syntax error.

2. Step 2: Correct the date format

   Enclose date in single quotes: '2023-01-01' to fix the query.

3. Final Answer:

   Date value missing quotes; fix with WHERE date = '2023-01-01' -> Option B

4. Quick Check:

   Date literals need quotes [OK]

Hint: Always quote date strings in SQL WHERE clauses [OK]

Common Mistakes:

• Forgetting quotes around dates
• Assuming semicolon fixes syntax
• Thinking SELECT * is invalid

5. You want to reduce BigQuery costs by limiting data scanned. Which approach is best?

hard

A. Use SELECT * to get all columns for flexibility

B. Run queries without filters to scan all data quickly

C. Store data in multiple small tables instead of one large table

D. Use partitioned tables and filter on partition column in queries

Solution

1. Step 1: Understand cost drivers in BigQuery

   Costs depend on data scanned; scanning less data reduces cost.

2. Step 2: Identify cost-saving technique

   Partitioning tables lets queries scan only relevant partitions, saving cost.

3. Final Answer:

   Use partitioned tables and filter on partition column in queries -> Option D

4. Quick Check:

   Partition + filter = less data scanned [OK]

Hint: Partition tables and filter to scan less data [OK]

Common Mistakes:

• Thinking scanning all data is cheaper
• Splitting tables without partitioning helps cost
• Using SELECT * increases scanned data