Analyzing index usage with pg_stat in PostgreSQL - Mini Project: Build & Apply

Analyzing index usage with pg_stat

📖 Scenario: You are a database administrator for an online bookstore. You want to understand how often the indexes on your books table are used to improve query performance and optimize your database.

🎯 Goal: Build a query that retrieves index usage statistics from the pg_stat_user_indexes system view for the books table, so you can analyze which indexes are being used and how often.

📋 What You'll Learn

Create a query that selects the index name, number of index scans, and table name from pg_stat_user_indexes.

Filter the results to only include indexes on the books table.

Order the results by the number of index scans in descending order.

💡 Why This Matters

🌍 Real World

Database administrators often need to monitor index usage to improve query speed and reduce unnecessary index overhead.

💼 Career

Understanding how to query PostgreSQL system views for index statistics is a valuable skill for roles like DBA, backend developer, and data engineer.

1

DATA SETUP: Identify the table name

Create a variable called table_name and set it to the string 'books' to specify the table you want to analyze.

PostgreSQL

table_name = 'books'
# Your code here

Hint

Use single quotes around the table name string.

2

CONFIGURATION: Prepare the base query string

Create a variable called base_query and set it to the SQL string that selects indexrelname, idx_scan, and relname from pg_stat_user_indexes.

PostgreSQL

table_name = 'books'
base_query = "SELECT indexrelname, idx_scan, relname FROM pg_stat_user_indexes"
# Your code here

Hint

Write the exact SELECT statement as a string.

3

CORE LOGIC: Add filtering for the table name

Create a variable called filtered_query that adds a WHERE clause to base_query to filter rows where relname equals the table_name variable.

PostgreSQL

table_name = 'books'
base_query = "SELECT indexrelname, idx_scan, relname FROM pg_stat_user_indexes"
filtered_query = base_query + f" WHERE relname = '{table_name}'"
# Your code here

Hint

Use an f-string to insert the table_name variable inside the WHERE clause.

4

COMPLETION: Add ordering by index scans

Create a variable called final_query that adds an ORDER BY clause to filtered_query to sort results by idx_scan in descending order.

PostgreSQL

table_name = 'books'
base_query = "SELECT indexrelname, idx_scan, relname FROM pg_stat_user_indexes"
filtered_query = base_query + f" WHERE relname = '{table_name}'"
final_query = filtered_query + " ORDER BY idx_scan DESC"
# Your code here

Hint

Append the ORDER BY clause as a string to the filtered query.

Practice

1. What does the idx_scan column in pg_stat_user_indexes represent?

easy

A. The number of times an index was used in a scan operation

B. The total size of the index in bytes

C. The number of rows in the indexed table

D. The creation date of the index

Solution

1. Step 1: Understand the purpose of pg_stat_user_indexes

   This system view tracks usage statistics for user-created indexes in PostgreSQL.

2. Step 2: Identify the meaning of idx_scan

   The idx_scan column counts how many times the index was used in scan operations, showing its usage frequency.

3. Final Answer:

   The number of times an index was used in a scan operation -> Option A

4. Quick Check:

   idx_scan = index usage count [OK]

Hint: Remember idx_scan counts index scans, not size or rows [OK]

Common Mistakes:

• Confusing idx_scan with index size
• Thinking idx_scan shows table row count
• Assuming idx_scan is creation date

2. Which of the following SQL queries correctly retrieves index usage statistics from pg_stat_user_indexes for a table named customers?

easy

A. SELECT * FROM pg_stat_user_indexes WHERE relname = 'customers';

B. SELECT * FROM pg_stat_user_indexes WHERE tablename = 'customers';

C. SELECT * FROM pg_stat_user_indexes WHERE indexname = 'customers';

D. SELECT * FROM pg_stat_user_indexes WHERE table_name = 'customers';

Solution

1. Step 1: Check the column names in pg_stat_user_indexes

   The correct column for the table name is relname, not tablename or table_name.

2. Step 2: Verify the query syntax

   SELECT * FROM pg_stat_user_indexes WHERE relname = 'customers'; uses relname = 'customers', which is correct to filter indexes for the table named 'customers'.

3. Final Answer:

   SELECT * FROM pg_stat_user_indexes WHERE relname = 'customers'; -> Option A

4. Quick Check:

   Use relname to filter by table name [OK]

Hint: Use relname column to filter by table in pg_stat_user_indexes [OK]

Common Mistakes:

• Using incorrect column names like tablename or table_name
• Filtering by indexname instead of table name
• Syntax errors in WHERE clause

3. Given the following query:

SELECT indexrelname, idx_scan FROM pg_stat_user_indexes WHERE relname = 'orders';

Which output correctly shows index usage if the table orders has two indexes orders_pkey with 150 scans and orders_date_idx with 0 scans?

medium

A. [{"indexrelname": "orders_pkey", "idx_scan": null}, {"indexrelname": "orders_date_idx", "idx_scan": null}]

B. [{"indexrelname": "orders_pkey", "idx_scan": 0}, {"indexrelname": "orders_date_idx", "idx_scan": 150}]

C. [{"indexrelname": "orders_pkey", "idx_scan": 150}]

D. [{"indexrelname": "orders_pkey", "idx_scan": 150}, {"indexrelname": "orders_date_idx", "idx_scan": 0}]

Solution

1. Step 1: Understand the query output

   The query selects index names and their scan counts for the 'orders' table, so both indexes should appear with their respective scan counts.

2. Step 2: Match the scan counts to indexes

   Given orders_pkey has 150 scans and orders_date_idx has 0, the output must show both with correct values.

3. Final Answer:

   [{"indexrelname": "orders_pkey", "idx_scan": 150}, {"indexrelname": "orders_date_idx", "idx_scan": 0}] -> Option D

4. Quick Check:

   Index names match scan counts correctly [OK]

Hint: Check idx_scan values match index names exactly [OK]

Common Mistakes:

• Swapping scan counts between indexes
• Showing null instead of zero for unused indexes
• Omitting indexes with zero scans

4. You run this query to find unused indexes:

SELECT indexrelname FROM pg_stat_user_indexes WHERE idx_scan = 0;

But it returns no rows, even though you know some indexes are unused. What is the likely cause?

medium

A. The idx_scan column does not track usage

B. The query syntax is incorrect and missing a semicolon

C. The statistics collector has not been reset or updated recently

D. The table has no indexes at all

Solution

1. Step 1: Verify query correctness

   The query syntax is correct and will return indexes with zero scans if any exist.

2. Step 2: Understand pg_stat_user_indexes behavior

   Index usage stats depend on the statistics collector. If it was recently reset or the server restarted, idx_scan may be zeroed or not updated yet.

3. Final Answer:

   The statistics collector has not been reset or updated recently -> Option C

4. Quick Check:

   Stats collector state affects idx_scan values [OK]

Hint: Check if stats collector was reset before trusting idx_scan [OK]

Common Mistakes:

• Assuming query syntax error causes no results
• Thinking idx_scan never tracks usage
• Believing no indexes exist without checking

5. You want to improve database performance by removing unused indexes. Which query helps you identify indexes that have never been scanned since the last stats reset, for table products?

hard

A. SELECT indexrelname FROM pg_stat_user_indexes WHERE idx_scan > 0 AND relname = 'products';

B. SELECT indexrelname FROM pg_stat_user_indexes WHERE relname = 'products' AND idx_scan = 0;

C. SELECT indexrelname FROM pg_stat_user_indexes WHERE relname = 'products' AND idx_tup_read = 0;

D. SELECT indexrelname FROM pg_stat_user_indexes WHERE relname = 'products' AND idx_scan IS NULL;

Solution

1. Step 1: Filter indexes by table name

   Use relname = 'products' to focus on indexes for the products table.

2. Step 2: Identify unused indexes by scan count

   Indexes with idx_scan = 0 have never been used since last stats reset, so they are candidates for removal.

3. Final Answer:

   SELECT indexrelname FROM pg_stat_user_indexes WHERE relname = 'products' AND idx_scan = 0; -> Option B

4. Quick Check:

   Unused indexes have idx_scan = 0 [OK]

Hint: Filter by relname and idx_scan = 0 to find unused indexes [OK]

Common Mistakes:

• Using idx_scan > 0 to find unused indexes
• Confusing idx_tup_read with idx_scan
• Checking for NULL instead of zero scans