Querying historical data in SCADA systems - Time & Space Complexity

Time Complexity: Querying historical data

O(n)

Understanding Time Complexity

When we ask a SCADA system for past data, the time it takes depends on how much data we want. We want to understand how this time grows as we ask for more history.

The question is: How does the system's work increase when we query more historical records?

Scenario Under Consideration

Analyze the time complexity of the following code snippet.

// Query historical data for a sensor
function queryHistoricalData(sensorId, startTime, endTime) {
  let data = [];
  let records = database.getRecords(sensorId, startTime, endTime);
  for (let record of records) {
    data.push(process(record));
  }
  return data;
}
    

This code fetches all records for a sensor between two times and processes each record one by one.

Identify Repeating Operations

• Primary operation: Looping through each historical record returned from the database.
• How many times: Once for every record between startTime and endTime.

How Execution Grows With Input

As the time range grows, more records are returned and processed, so the work grows with the number of records.

Input Size (n)	Approx. Operations
10 records	About 10 processing steps
100 records	About 100 processing steps
1000 records	About 1000 processing steps

Pattern observation: The work grows directly with the number of records requested.

Final Time Complexity

Time Complexity: O(n)

This means the time to get and process data grows in a straight line with how many records we ask for.

Common Mistake

[X] Wrong: "Querying more time means the system takes the same time because it just asks once."

[OK] Correct: Actually, the system must handle each record returned, so more records mean more work and more time.

Interview Connect

Understanding how data queries scale helps you explain system behavior clearly. This skill shows you can think about real-world system limits and performance.

Self-Check

"What if the system cached recent records? How would that change the time complexity when querying recent data?"

Practice

1. What is the main purpose of querying historical data in SCADA systems?

easy

A. To control real-time device operations

B. To review past system behavior and analyze trends

C. To update firmware on sensors

D. To configure network settings

Solution

1. Step 1: Understand the role of historical data

   Historical data stores past readings and events from the system.

2. Step 2: Identify the purpose of querying it

   Querying helps analyze past behavior and detect trends or issues.

3. Final Answer:

   To review past system behavior and analyze trends -> Option B

4. Quick Check:

   Historical data = review past behavior [OK]

Hint: Historical data is for past info, not real-time control [OK]

Common Mistakes:

• Confusing historical data with real-time control
• Thinking it updates devices
• Assuming it changes network settings

2. Which SQL query correctly selects temperature readings from sensor 'S1' recorded after '2024-01-01'?

easy

A. SELECT sensor_id, timestamp FROM readings WHERE type = 'temperature' OR sensor_id = 'S1' AND timestamp > '2024-01-01';

B. SELECT * FROM readings WHERE sensor_id = 'S1' AND timestamp < '2024-01-01' AND type = 'temperature';

C. SELECT * FROM readings WHERE sensor_id == 'S1' AND timestamp > '2024-01-01' AND type = 'temperature';

D. SELECT * FROM readings WHERE sensor_id = 'S1' AND timestamp > '2024-01-01' AND type = 'temperature';

Solution

1. Step 1: Check correct SQL syntax for conditions

   Use single equals (=) for comparison and AND to combine conditions.

2. Step 2: Verify logical conditions match requirements

   SELECT * FROM readings WHERE sensor_id = 'S1' AND timestamp > '2024-01-01' AND type = 'temperature'; correctly filters sensor_id = 'S1', timestamp > '2024-01-01', and type = 'temperature'.

3. Final Answer:

   SELECT * FROM readings WHERE sensor_id = 'S1' AND timestamp > '2024-01-01' AND type = 'temperature'; -> Option D

4. Quick Check:

   Correct syntax and filters = SELECT * FROM readings WHERE sensor_id = 'S1' AND timestamp > '2024-01-01' AND type = 'temperature'; [OK]

Hint: Use = for comparison and AND to combine filters [OK]

Common Mistakes:

• Using '==' instead of '=' in SQL
• Mixing AND and OR without parentheses
• Using wrong comparison operators
• Filtering with wrong timestamp direction

3. Given this query:
SELECT timestamp, value FROM readings WHERE sensor_id = 'S2' AND timestamp BETWEEN '2024-03-01' AND '2024-03-05' ORDER BY timestamp DESC LIMIT 3;
What will be the output?

medium

A. The 3 latest readings from sensor S2 between March 1 and 5, sorted descending

B. An error because LIMIT cannot be used with ORDER BY

C. All readings from sensor S2 between March 1 and 5, unsorted

D. The 3 earliest readings from sensor S2 between March 1 and 5, sorted ascending

Solution

1. Step 1: Understand the WHERE and BETWEEN clause

   Filters readings from sensor 'S2' between '2024-03-01' and '2024-03-05'.

2. Step 2: Analyze ORDER BY and LIMIT

   ORDER BY timestamp DESC sorts newest first; LIMIT 3 returns top 3 newest readings.

3. Final Answer:

   The 3 latest readings from sensor S2 between March 1 and 5, sorted descending -> Option A

4. Quick Check:

   ORDER BY DESC + LIMIT 3 = latest 3 readings [OK]

Hint: ORDER BY DESC + LIMIT gets newest records first [OK]

Common Mistakes:

• Confusing ascending vs descending order
• Thinking LIMIT returns earliest records
• Assuming LIMIT causes syntax error with ORDER BY

4. You wrote this query to get pressure data:
SELECT * FROM readings WHERE sensor_id = 'P1' AND timestamp > '2024-02-01' AND type = 'pressure'
But it returns no results, even though data exists. What is the likely problem?

medium

A. The query is missing a GROUP BY clause

B. The sensor_id should be numeric, not string

C. The timestamp format is incorrect and does not match stored data

D. The type column does not exist in the readings table

Solution

1. Step 1: Check timestamp format correctness

   Timestamp format must match stored data format exactly to filter correctly.

2. Step 2: Verify other query parts

   Sensor_id as string is valid; GROUP BY not needed; type column likely exists.

3. Final Answer:

   The timestamp format is incorrect and does not match stored data -> Option C

4. Quick Check:

   Timestamp format mismatch = no results [OK]

Hint: Match timestamp format exactly to stored data [OK]

Common Mistakes:

• Assuming sensor_id must be numeric
• Adding unnecessary GROUP BY
• Ignoring timestamp format differences

5. You want to find the average temperature for each of sensors 'T1' and 'T2' during January 2024, but only for readings above 20°C. Which SQL query achieves this?

hard

A. SELECT sensor_id, AVG(value) FROM readings WHERE (sensor_id = 'T1' OR sensor_id = 'T2') AND timestamp BETWEEN '2024-01-01' AND '2024-01-31' AND value > 20 AND type = 'temperature' GROUP BY sensor_id;

B. SELECT AVG(value) FROM readings WHERE sensor_id IN ('T1', 'T2') AND timestamp >= '2024-01-01' AND timestamp <= '2024-01-31' AND value > 20 AND type = 'temperature';

C. SELECT sensor_id, AVG(value) FROM readings WHERE sensor_id = 'T1' AND sensor_id = 'T2' AND timestamp BETWEEN '2024-01-01' AND '2024-01-31' AND value > 20 AND type = 'temperature' GROUP BY sensor_id;

D. SELECT sensor_id, AVG(value) FROM readings WHERE sensor_id = 'T1' OR sensor_id = 'T2' AND timestamp BETWEEN '2024-01-01' AND '2024-01-31' AND value > 20 AND type = 'temperature';

Solution

1. Step 1: Filter sensors correctly

   Use (sensor_id = 'T1' OR sensor_id = 'T2') or sensor_id IN ('T1', 'T2') to include both sensors.

2. Step 2: Apply date and value filters with grouping

   Filter timestamp between January 1 and 31, value > 20, and type = 'temperature'. Group by sensor_id to get averages per sensor.

3. Step 3: Check query correctness

   SELECT sensor_id, AVG(value) FROM readings WHERE (sensor_id = 'T1' OR sensor_id = 'T2') AND timestamp BETWEEN '2024-01-01' AND '2024-01-31' AND value > 20 AND type = 'temperature' GROUP BY sensor_id; uses correct syntax with parentheses and GROUP BY; SELECT AVG(value) FROM readings WHERE sensor_id IN ('T1', 'T2') AND timestamp >= '2024-01-01' AND timestamp <= '2024-01-31' AND value > 20 AND type = 'temperature'; misses GROUP BY; SELECT sensor_id, AVG(value) FROM readings WHERE sensor_id = 'T1' AND sensor_id = 'T2' AND timestamp BETWEEN '2024-01-01' AND '2024-01-31' AND value > 20 AND type = 'temperature' GROUP BY sensor_id; has impossible condition; SELECT sensor_id, AVG(value) FROM readings WHERE sensor_id = 'T1' OR sensor_id = 'T2' AND timestamp BETWEEN '2024-01-01' AND '2024-01-31' AND value > 20 AND type = 'temperature'; lacks parentheses causing wrong logic.

4. Final Answer:

   SELECT sensor_id, AVG(value) FROM readings WHERE (sensor_id = 'T1' OR sensor_id = 'T2') AND timestamp BETWEEN '2024-01-01' AND '2024-01-31' AND value > 20 AND type = 'temperature' GROUP BY sensor_id; -> Option A

5. Quick Check:

   Correct filters + grouping = SELECT sensor_id, AVG(value) FROM readings WHERE (sensor_id = 'T1' OR sensor_id = 'T2') AND timestamp BETWEEN '2024-01-01' AND '2024-01-31' AND value > 20 AND type = 'temperature' GROUP BY sensor_id; [OK]

Hint: Use parentheses for OR and GROUP BY for averages per sensor [OK]

Common Mistakes:

• Missing GROUP BY when aggregating by sensor
• Using AND instead of OR between sensor_ids
• Incorrect timestamp filtering logic