FLATTEN for nested data in Snowflake - Time & Space Complexity

Time Complexity: FLATTEN for nested data

O(n)

Understanding Time Complexity

When working with nested data in Snowflake, we often use FLATTEN to turn arrays into rows. Understanding how the time to run FLATTEN grows as the nested data grows helps us plan and optimize queries.

We want to know: how does the work Snowflake does change as the nested array gets bigger?

Scenario Under Consideration

Analyze the time complexity of the following operation sequence.

SELECT
  t.id,
  f.value
FROM
  my_table t,
  LATERAL FLATTEN(input => t.nested_array) f;
    

This query takes each row from my_table and expands the nested array nested_array into multiple rows, one per element.

Identify Repeating Operations

Identify the API calls, resource provisioning, data transfers that repeat.

• Primary operation: The FLATTEN function iterates over each element in the nested array.
• How many times: Once for each element inside the nested array for every row in the table.

How Execution Grows With Input

As the number of elements in the nested array grows, the number of rows output by FLATTEN grows proportionally.

Input Size (n)	Approx. Api Calls/Operations
10	About 10 operations per row
100	About 100 operations per row
1000	About 1000 operations per row

Pattern observation: The work grows directly with the number of elements in the nested array.

Final Time Complexity

Time Complexity: O(n)

This means the time to run FLATTEN grows linearly with the size of the nested array.

Common Mistake

[X] Wrong: "FLATTEN runs in constant time no matter how big the array is."

[OK] Correct: FLATTEN must look at each element to output it, so more elements mean more work and more time.

Interview Connect

Understanding how FLATTEN scales helps you explain query performance and data processing costs clearly. This skill shows you can think about how data size affects work done, a key part of cloud data engineering.

Self-Check

"What if the nested array contains nested arrays itself? How would the time complexity of FLATTEN change if we flatten multiple levels?"

Practice

1. What does the FLATTEN function do in Snowflake when working with nested data?

easy

A. It encrypts nested data for security.

B. It compresses data to save storage space.

C. It converts nested arrays or objects into simple rows.

D. It creates a backup of nested data.

Solution

1. Step 1: Understand the purpose of FLATTEN

   FLATTEN is designed to take nested arrays or objects and turn them into individual rows so they are easier to query.

2. Step 2: Compare options to FLATTEN's function

   Options A, B, and D describe encryption, compression, and backup, which are unrelated to FLATTEN's role.

3. Final Answer:

   It converts nested arrays or objects into simple rows. -> Option C

4. Quick Check:

   FLATTEN = convert nested data to rows [OK]

Hint: FLATTEN breaks nested data into rows for easy reading [OK]

Common Mistakes:

• Thinking FLATTEN compresses or encrypts data
• Confusing FLATTEN with backup or storage functions
• Assuming FLATTEN changes data format instead of structure

2. Which of the following is the correct syntax to use FLATTEN on a JSON column named data in Snowflake?

easy

A. SELECT FLATTEN(data) FROM table;

B. SELECT * FROM FLATTEN(input => data);

C. SELECT FLATTEN(input = data) FROM table;

D. SELECT * FROM table, LATERAL FLATTEN(input => data);

Solution

1. Step 1: Recall FLATTEN usage in FROM clause

   FLATTEN is used as a table function in the FROM clause with LATERAL to expand nested data.

2. Step 2: Analyze each option's syntax

   SELECT * FROM table, LATERAL FLATTEN(input => data); correctly uses FROM table, LATERAL FLATTEN(input => data). Options A and C misuse FLATTEN as a scalar function. SELECT * FROM FLATTEN(input => data); misses the table reference.

3. Final Answer:

   SELECT * FROM table, LATERAL FLATTEN(input => data); -> Option D

4. Quick Check:

   FLATTEN used with LATERAL in FROM clause [OK]

Hint: Use FLATTEN with LATERAL in FROM clause for nested data [OK]

Common Mistakes:

• Using FLATTEN as a scalar function in SELECT
• Omitting LATERAL keyword
• Not specifying input parameter correctly

3. Given the JSON column data with value '{"items": ["apple", "banana", "cherry"]}', what will the query below return?

SELECT f.value FROM table, LATERAL FLATTEN(input => data:items) f;

medium

A. Rows with values: apple, banana, cherry

B. A single row with the entire array as a string

C. An error because data:items is invalid syntax

D. Rows with keys and values of the JSON object

Solution

1. Step 1: Understand FLATTEN on JSON array

   FLATTEN(input => data:items) expands the array under 'items' into multiple rows, each with one element.

2. Step 2: Analyze the query output

   The query selects f.value, which will be each element: 'apple', 'banana', 'cherry' as separate rows.

3. Final Answer:

   Rows with values: apple, banana, cherry -> Option A

4. Quick Check:

   FLATTEN on array returns each element as a row [OK]

Hint: FLATTEN on JSON array returns each element as a separate row [OK]

Common Mistakes:

• Expecting a single row with the whole array
• Confusing keys with values in FLATTEN output
• Misreading JSON path syntax

4. You wrote this query to flatten nested JSON data:

SELECT f.value FROM table, FLATTEN(input => data:items) f;

But it returns an error. What is the likely cause?

medium

A. Missing LATERAL keyword before FLATTEN

B. Incorrect JSON path syntax in input

C. FLATTEN cannot be used on JSON arrays

D. SELECT statement missing WHERE clause

Solution

1. Step 1: Identify FLATTEN usage requirements

   FLATTEN is a table function that requires LATERAL when used with another table to expand nested data.

2. Step 2: Check query syntax

   The query misses the LATERAL keyword before FLATTEN, causing a syntax error.

3. Final Answer:

   Missing LATERAL keyword before FLATTEN -> Option A

4. Quick Check:

   FLATTEN needs LATERAL in FROM clause [OK]

Hint: Always add LATERAL before FLATTEN in FROM clause [OK]

Common Mistakes:

• Forgetting LATERAL keyword
• Assuming FLATTEN works without LATERAL
• Blaming JSON path syntax instead of syntax structure

5. You have a table with a column orders storing nested JSON arrays of items per order. You want to list each item with its order ID. Which query correctly uses FLATTEN to achieve this?

hard

A. SELECT order_id, FLATTEN(orders) FROM orders_table;

B. SELECT order_id, f.value FROM orders_table, LATERAL FLATTEN(input => orders) f;

C. SELECT order_id, f.value FROM orders_table JOIN FLATTEN(input => orders) f ON TRUE;

D. SELECT order_id, f.value FROM orders_table, FLATTEN(orders) f;

Solution

1. Step 1: Understand how to join FLATTEN with table

   FLATTEN must be used with LATERAL in the FROM clause to expand nested arrays per row.

2. Step 2: Evaluate each option's correctness

   SELECT order_id, f.value FROM orders_table, LATERAL FLATTEN(input => orders) f; correctly uses FROM orders_table, LATERAL FLATTEN(input => orders) f, selecting order_id and each item value. Options A and D misuse FLATTEN syntax. SELECT order_id, f.value FROM orders_table JOIN FLATTEN(input => orders) f ON TRUE; uses JOIN incorrectly without LATERAL.

3. Final Answer:

   SELECT order_id, f.value FROM orders_table, LATERAL FLATTEN(input => orders) f; -> Option B

4. Quick Check:

   Use LATERAL FLATTEN with table to list nested items [OK]

Hint: Use LATERAL FLATTEN(input => column) to expand nested arrays per row [OK]

Common Mistakes:

• Using FLATTEN without LATERAL
• Trying to JOIN FLATTEN without LATERAL
• Incorrect FLATTEN syntax in SELECT clause