Querying historical data in SCADA systems - Step-by-Step Execution

Process Flow - Querying historical data

Start Query

↓

Specify Time Range

↓

Select Data Points

↓

Execute Query

↓

Retrieve Data

↓

Display or Analyze Data

↓

End

The flow shows how a historical data query starts by specifying time and data points, then executes, retrieves, and displays results.

Execution Sample

SCADA systems

SELECT timestamp, temperature FROM sensor_data
WHERE timestamp BETWEEN '2024-04-01 00:00:00' AND '2024-04-01 01:00:00'
ORDER BY timestamp;

This query retrieves temperature readings from sensor_data for one hour on April 1, 2024, ordered by time.

Process Table

Step	Action	Evaluation	Result
1	Start Query	Begin query process	Ready to specify parameters
2	Specify Time Range	Time range set to 2024-04-01 00:00 to 2024-04-01 01:00	Time filter applied
3	Select Data Points	Selecting 'timestamp' and 'temperature' columns	Columns selected
4	Execute Query	Run SQL query on database	Query sent to database engine
5	Retrieve Data	Database returns matching rows	Data rows received
6	Display or Analyze Data	Show data in table or graph	Data ready for user
7	End	Query complete	Process finished

💡 Query ends after data retrieval and display steps complete successfully

Status Tracker

Variable	Start	After Step 2	After Step 3	After Step 5	Final
time_range	None	'2024-04-01 00:00' to '2024-04-01 01:00'	'2024-04-01 00:00' to '2024-04-01 01:00'	'2024-04-01 00:00' to '2024-04-01 01:00'	'2024-04-01 00:00' to '2024-04-01 01:00'
selected_columns	None	None	['timestamp', 'temperature']	['timestamp', 'temperature']	['timestamp', 'temperature']
query_result	None	None	None	Data rows with timestamps and temperatures	Data rows with timestamps and temperatures

Key Moments - 2 Insights

Why do we need to specify a time range before executing the query?

Specifying the time range filters the data to only the relevant period, reducing the amount of data retrieved and speeding up the query, as shown in step 2 of the execution table.

What happens if we don't select specific data points?

If no columns are selected, the query might return all columns, which can be slow and overwhelming. Step 3 shows selecting only needed columns improves efficiency.

Visual Quiz - 3 Questions

Look at the execution table, what is the result after step 4 (Execute Query)?

A Data rows received

B Ready to specify parameters

C Query sent to database engine

D Process finished

💡 Hint

Check the 'Result' column for step 4 in the execution table.

At which step does the system retrieve the actual data rows?

A Step 3

B Step 5

C Step 2

D Step 6

💡 Hint

Look for the step labeled 'Retrieve Data' in the execution table.

If the time range was not specified, how would the variable 'time_range' appear after step 2 in variable_tracker?

A None

B ['timestamp', 'temperature']

C '2024-04-01 00:00' to '2024-04-01 01:00'

D Data rows with timestamps and temperatures

💡 Hint

Refer to the 'time_range' row in variable_tracker after step 2.

Concept Snapshot

Querying historical data involves:
- Specifying a time range to filter data
- Selecting relevant data points (columns)
- Executing the query on the database
- Retrieving and displaying the results
Always filter by time to improve speed and relevance.

Full Transcript

This visual execution trace shows how querying historical data in SCADA systems works step-by-step. First, the query process starts. Then, a time range is specified to limit data to a certain period. Next, specific data points like timestamp and temperature are selected. The query is executed on the database, which returns matching data rows. Finally, the data is displayed or analyzed, and the query process ends. Variables like time_range and selected_columns change as the query progresses. Key moments include understanding why time filtering is important and why selecting specific columns matters. The quiz questions help reinforce these steps by referencing the execution table and variable states.

Practice

1. What is the main purpose of querying historical data in SCADA systems?

easy

A. To control real-time device operations

B. To review past system behavior and analyze trends

C. To update firmware on sensors

D. To configure network settings

Solution

1. Step 1: Understand the role of historical data

   Historical data stores past readings and events from the system.

2. Step 2: Identify the purpose of querying it

   Querying helps analyze past behavior and detect trends or issues.

3. Final Answer:

   To review past system behavior and analyze trends -> Option B

4. Quick Check:

   Historical data = review past behavior [OK]

Hint: Historical data is for past info, not real-time control [OK]

Common Mistakes:

• Confusing historical data with real-time control
• Thinking it updates devices
• Assuming it changes network settings

2. Which SQL query correctly selects temperature readings from sensor 'S1' recorded after '2024-01-01'?

easy

A. SELECT sensor_id, timestamp FROM readings WHERE type = 'temperature' OR sensor_id = 'S1' AND timestamp > '2024-01-01';

B. SELECT * FROM readings WHERE sensor_id = 'S1' AND timestamp < '2024-01-01' AND type = 'temperature';

C. SELECT * FROM readings WHERE sensor_id == 'S1' AND timestamp > '2024-01-01' AND type = 'temperature';

D. SELECT * FROM readings WHERE sensor_id = 'S1' AND timestamp > '2024-01-01' AND type = 'temperature';

Solution

1. Step 1: Check correct SQL syntax for conditions

   Use single equals (=) for comparison and AND to combine conditions.

2. Step 2: Verify logical conditions match requirements

   SELECT * FROM readings WHERE sensor_id = 'S1' AND timestamp > '2024-01-01' AND type = 'temperature'; correctly filters sensor_id = 'S1', timestamp > '2024-01-01', and type = 'temperature'.

3. Final Answer:

   SELECT * FROM readings WHERE sensor_id = 'S1' AND timestamp > '2024-01-01' AND type = 'temperature'; -> Option D

4. Quick Check:

   Correct syntax and filters = SELECT * FROM readings WHERE sensor_id = 'S1' AND timestamp > '2024-01-01' AND type = 'temperature'; [OK]

Hint: Use = for comparison and AND to combine filters [OK]

Common Mistakes:

• Using '==' instead of '=' in SQL
• Mixing AND and OR without parentheses
• Using wrong comparison operators
• Filtering with wrong timestamp direction

3. Given this query:
SELECT timestamp, value FROM readings WHERE sensor_id = 'S2' AND timestamp BETWEEN '2024-03-01' AND '2024-03-05' ORDER BY timestamp DESC LIMIT 3;
What will be the output?

medium

A. The 3 latest readings from sensor S2 between March 1 and 5, sorted descending

B. An error because LIMIT cannot be used with ORDER BY

C. All readings from sensor S2 between March 1 and 5, unsorted

D. The 3 earliest readings from sensor S2 between March 1 and 5, sorted ascending

Solution

1. Step 1: Understand the WHERE and BETWEEN clause

   Filters readings from sensor 'S2' between '2024-03-01' and '2024-03-05'.

2. Step 2: Analyze ORDER BY and LIMIT

   ORDER BY timestamp DESC sorts newest first; LIMIT 3 returns top 3 newest readings.

3. Final Answer:

   The 3 latest readings from sensor S2 between March 1 and 5, sorted descending -> Option A

4. Quick Check:

   ORDER BY DESC + LIMIT 3 = latest 3 readings [OK]

Hint: ORDER BY DESC + LIMIT gets newest records first [OK]

Common Mistakes:

• Confusing ascending vs descending order
• Thinking LIMIT returns earliest records
• Assuming LIMIT causes syntax error with ORDER BY

4. You wrote this query to get pressure data:
SELECT * FROM readings WHERE sensor_id = 'P1' AND timestamp > '2024-02-01' AND type = 'pressure'
But it returns no results, even though data exists. What is the likely problem?

medium

A. The query is missing a GROUP BY clause

B. The sensor_id should be numeric, not string

C. The timestamp format is incorrect and does not match stored data

D. The type column does not exist in the readings table

Solution

1. Step 1: Check timestamp format correctness

   Timestamp format must match stored data format exactly to filter correctly.

2. Step 2: Verify other query parts

   Sensor_id as string is valid; GROUP BY not needed; type column likely exists.

3. Final Answer:

   The timestamp format is incorrect and does not match stored data -> Option C

4. Quick Check:

   Timestamp format mismatch = no results [OK]

Hint: Match timestamp format exactly to stored data [OK]

Common Mistakes:

• Assuming sensor_id must be numeric
• Adding unnecessary GROUP BY
• Ignoring timestamp format differences

5. You want to find the average temperature for each of sensors 'T1' and 'T2' during January 2024, but only for readings above 20°C. Which SQL query achieves this?

hard

A. SELECT sensor_id, AVG(value) FROM readings WHERE (sensor_id = 'T1' OR sensor_id = 'T2') AND timestamp BETWEEN '2024-01-01' AND '2024-01-31' AND value > 20 AND type = 'temperature' GROUP BY sensor_id;

B. SELECT AVG(value) FROM readings WHERE sensor_id IN ('T1', 'T2') AND timestamp >= '2024-01-01' AND timestamp <= '2024-01-31' AND value > 20 AND type = 'temperature';

C. SELECT sensor_id, AVG(value) FROM readings WHERE sensor_id = 'T1' AND sensor_id = 'T2' AND timestamp BETWEEN '2024-01-01' AND '2024-01-31' AND value > 20 AND type = 'temperature' GROUP BY sensor_id;

D. SELECT sensor_id, AVG(value) FROM readings WHERE sensor_id = 'T1' OR sensor_id = 'T2' AND timestamp BETWEEN '2024-01-01' AND '2024-01-31' AND value > 20 AND type = 'temperature';

Solution

1. Step 1: Filter sensors correctly

   Use (sensor_id = 'T1' OR sensor_id = 'T2') or sensor_id IN ('T1', 'T2') to include both sensors.

2. Step 2: Apply date and value filters with grouping

   Filter timestamp between January 1 and 31, value > 20, and type = 'temperature'. Group by sensor_id to get averages per sensor.

3. Step 3: Check query correctness

   SELECT sensor_id, AVG(value) FROM readings WHERE (sensor_id = 'T1' OR sensor_id = 'T2') AND timestamp BETWEEN '2024-01-01' AND '2024-01-31' AND value > 20 AND type = 'temperature' GROUP BY sensor_id; uses correct syntax with parentheses and GROUP BY; SELECT AVG(value) FROM readings WHERE sensor_id IN ('T1', 'T2') AND timestamp >= '2024-01-01' AND timestamp <= '2024-01-31' AND value > 20 AND type = 'temperature'; misses GROUP BY; SELECT sensor_id, AVG(value) FROM readings WHERE sensor_id = 'T1' AND sensor_id = 'T2' AND timestamp BETWEEN '2024-01-01' AND '2024-01-31' AND value > 20 AND type = 'temperature' GROUP BY sensor_id; has impossible condition; SELECT sensor_id, AVG(value) FROM readings WHERE sensor_id = 'T1' OR sensor_id = 'T2' AND timestamp BETWEEN '2024-01-01' AND '2024-01-31' AND value > 20 AND type = 'temperature'; lacks parentheses causing wrong logic.

4. Final Answer:

   SELECT sensor_id, AVG(value) FROM readings WHERE (sensor_id = 'T1' OR sensor_id = 'T2') AND timestamp BETWEEN '2024-01-01' AND '2024-01-31' AND value > 20 AND type = 'temperature' GROUP BY sensor_id; -> Option A

5. Quick Check:

   Correct filters + grouping = SELECT sensor_id, AVG(value) FROM readings WHERE (sensor_id = 'T1' OR sensor_id = 'T2') AND timestamp BETWEEN '2024-01-01' AND '2024-01-31' AND value > 20 AND type = 'temperature' GROUP BY sensor_id; [OK]

Hint: Use parentheses for OR and GROUP BY for averages per sensor [OK]

Common Mistakes:

• Missing GROUP BY when aggregating by sensor
• Using AND instead of OR between sensor_ids
• Incorrect timestamp filtering logic