Query history and profiling in Snowflake - Mini Project: Build & Apply

Query History and Profiling in Snowflake

📖 Scenario: You are a data analyst working with Snowflake cloud data warehouse. Your manager wants you to track recent queries run on the system and profile their execution times to find slow queries.

🎯 Goal: Build a set of SQL queries to retrieve query history and profile query execution times in Snowflake.

📋 What You'll Learn

Use the SNOWFLAKE.ACCOUNT_USAGE.QUERY_HISTORY view to get recent queries

Filter queries executed in the last 7 days

Select query id, user name, query text, start time, end time, and execution time

Calculate execution time in seconds

Order results by execution time descending

💡 Why This Matters

🌍 Real World

Tracking query performance helps optimize data warehouse usage and reduce costs by identifying slow or expensive queries.

💼 Career

Data analysts and cloud engineers often profile query history to troubleshoot performance and improve data workflows.

1

Retrieve recent query history

Write a SQL query to select QUERY_ID, USER_NAME, QUERY_TEXT, START_TIME, and END_TIME from SNOWFLAKE.ACCOUNT_USAGE.QUERY_HISTORY where START_TIME is within the last 7 days.

Snowflake

-- Your SQL query here to select QUERY_ID, USER_NAME, QUERY_TEXT, START_TIME, END_TIME from QUERY_HISTORY for last 7 days

Hint

Use DATEADD(day, -7, CURRENT_TIMESTAMP()) to filter queries from the last 7 days.

2

Add execution time calculation

Add a new column EXECUTION_TIME_SECONDS that calculates the difference between END_TIME and START_TIME in seconds using DATEDIFF function.

Snowflake

SELECT QUERY_ID, USER_NAME, QUERY_TEXT, START_TIME, END_TIME,
-- Calculate execution time in seconds here
FROM SNOWFLAKE.ACCOUNT_USAGE.QUERY_HISTORY
WHERE START_TIME >= DATEADD(day, -7, CURRENT_TIMESTAMP());

Hint

Use Datediff(second, START_TIME, END_TIME) to get execution time in seconds.

3

Filter out queries with null end time

Add a condition to the WHERE clause to exclude queries where END_TIME is NULL.

Snowflake

SELECT QUERY_ID, USER_NAME, QUERY_TEXT, START_TIME, END_TIME,
       DATEDIFF(second, START_TIME, END_TIME) AS EXECUTION_TIME_SECONDS
FROM SNOWFLAKE.ACCOUNT_USAGE.QUERY_HISTORY
WHERE START_TIME >= DATEADD(day, -7, CURRENT_TIMESTAMP())
-- Add condition to exclude NULL END_TIME

Hint

Use AND END_TIME IS NOT NULL to exclude queries still running.

4

Order queries by execution time descending

Add an ORDER BY clause to sort the results by EXECUTION_TIME_SECONDS in descending order.

Snowflake

SELECT QUERY_ID, USER_NAME, QUERY_TEXT, START_TIME, END_TIME,
       DATEDIFF(second, START_TIME, END_TIME) AS EXECUTION_TIME_SECONDS
FROM SNOWFLAKE.ACCOUNT_USAGE.QUERY_HISTORY
WHERE START_TIME >= DATEADD(day, -7, CURRENT_TIMESTAMP())
  AND END_TIME IS NOT NULL
-- Add ORDER BY clause here

Hint

Use ORDER BY EXECUTION_TIME_SECONDS DESC to see slowest queries first.

Practice

1. What is the main purpose of the QUERY_HISTORY view in Snowflake?

easy

A. To see details of past queries executed in the system

B. To create new tables and schemas

C. To manage user permissions and roles

D. To monitor network traffic between Snowflake and clients

Solution

1. Step 1: Understand the role of QUERY_HISTORY

   The QUERY_HISTORY view stores information about queries that have already run, including their text, execution time, and status. Its main purpose is to see details of past queries executed in the system.

2. Final Answer:

   To see details of past queries executed in the system -> Option A

3. Quick Check:

   QUERY_HISTORY = past query details [OK]

Hint: QUERY_HISTORY shows past queries and their info [OK]

Common Mistakes:

• Confusing QUERY_HISTORY with user management
• Thinking it manages network or security settings
• Assuming it creates or modifies database objects

2. Which SQL clause correctly filters queries executed by a specific user in the QUERY_HISTORY view?

easy

A. FILTER BY USER = 'john_doe'

B. WHERE USER_NAME = 'john_doe'

C. SELECT USER_NAME FROM QUERY_HISTORY WHERE 'john_doe'

D. HAVING USER_NAME = 'john_doe'

Solution

1. Step 1: Recall SQL filtering syntax

   To filter rows in SQL, the WHERE clause is used with a condition like USER_NAME = 'value'. WHERE USER_NAME = 'john_doe' is valid and standard SQL syntax.

2. Final Answer:

   WHERE USER_NAME = 'john_doe' -> Option B

3. Quick Check:

   Filter with WHERE clause = WHERE USER_NAME = 'john_doe' [OK]

Hint: Use WHERE to filter rows by user name [OK]

Common Mistakes:

• Using FILTER BY instead of WHERE
• Misplacing HAVING without GROUP BY
• Incorrect SELECT syntax without WHERE

3. Given the query:

SELECT query_text, total_elapsed_time FROM TABLE(INFORMATION_SCHEMA.QUERY_HISTORY()) WHERE execution_status = 'FAILED' ORDER BY start_time DESC LIMIT 1;

What does this query return?

medium

A. The most recent failed query's text and its total elapsed time

B. All successful queries ordered by start time

C. The oldest failed query's text and elapsed time

D. An error because QUERY_HISTORY is not a table

Solution

1. Step 1: Analyze the query clauses

   The query uses TABLE(INFORMATION_SCHEMA.QUERY_HISTORY()) to access query history, filters for execution_status = 'FAILED', orders by start_time DESC (most recent first), and limits to 1 row, returning the most recent failed query's text and total elapsed time.

2. Final Answer:

   The most recent failed query's text and its total elapsed time -> Option A

3. Quick Check:

   Filter failed + order desc + limit 1 = most recent failed query [OK]

Hint: ORDER BY DESC + LIMIT 1 gets latest record [OK]

Common Mistakes:

• Thinking QUERY_HISTORY is a normal table
• Confusing oldest vs most recent due to ORDER BY
• Ignoring the WHERE filter on execution_status

4. You wrote this query to find slow queries but it returns no results:

SELECT query_text FROM TABLE(INFORMATION_SCHEMA.QUERY_HISTORY()) WHERE total_elapsed_time > 1000;

What is the likely issue?

medium

A. The TABLE() function cannot be used with QUERY_HISTORY

B. QUERY_HISTORY does not have total_elapsed_time column

C. The query_text column cannot be selected from QUERY_HISTORY

D. total_elapsed_time is in microseconds, so 1000 is too small a threshold

Solution

1. Step 1: Check the unit of total_elapsed_time

   In Snowflake, total_elapsed_time is measured in microseconds, so 1000 microseconds (1 millisecond) is too small a threshold, and few or no queries exceed it, resulting in no results.

2. Final Answer:

   total_elapsed_time is in microseconds, so 1000 is too small a threshold -> Option D

3. Quick Check:

   Elapsed time unit = microseconds, threshold too low [OK]

Hint: Check units: elapsed time is microseconds, not milliseconds [OK]

Common Mistakes:

• Assuming elapsed time is in seconds or milliseconds
• Thinking QUERY_HISTORY lacks columns
• Misusing TABLE() function syntax

5. You want to profile query performance by grouping queries by user and calculating average execution time. Which query correctly achieves this?

hard

A. SELECT user_name, SUM(total_elapsed_time) FROM TABLE(INFORMATION_SCHEMA.QUERY_HISTORY()) WHERE execution_status = 'SUCCESS';

B. SELECT user_name, total_elapsed_time FROM QUERY_HISTORY GROUP BY user_name;

C. SELECT user_name, AVG(total_elapsed_time) AS avg_time FROM TABLE(INFORMATION_SCHEMA.QUERY_HISTORY()) GROUP BY user_name ORDER BY avg_time DESC;

D. SELECT user_name, AVG(total_elapsed_time) FROM QUERY_HISTORY WHERE total_elapsed_time > 1000 ORDER BY user_name;

Solution

1. Step 1: Identify correct aggregation and grouping

   To get average execution time per user, use AVG(total_elapsed_time) with GROUP BY user_name from TABLE(INFORMATION_SCHEMA.QUERY_HISTORY()), as in SELECT user_name, AVG(total_elapsed_time) AS avg_time ... GROUP BY user_name ORDER BY avg_time DESC.

2. Final Answer:

   SELECT user_name, AVG(total_elapsed_time) AS avg_time FROM TABLE(INFORMATION_SCHEMA.QUERY_HISTORY()) GROUP BY user_name ORDER BY avg_time DESC; -> Option C

3. Quick Check:

   Group by user + AVG + ORDER BY avg_time = SELECT user_name, AVG(total_elapsed_time) AS avg_time FROM TABLE(INFORMATION_SCHEMA.QUERY_HISTORY()) GROUP BY user_name ORDER BY avg_time DESC; [OK]

Hint: Use GROUP BY user_name with AVG for profiling [OK]

Common Mistakes:

• Missing GROUP BY when using aggregation
• Selecting columns without aggregation
• Not using TABLE() function for QUERY_HISTORY