Why caching improves performance in Azure - Performance Analysis

Time Complexity: Why caching improves performance

O(1)

Understanding Time Complexity

We want to see how caching changes the speed of getting data in cloud systems.

How does caching reduce the number of slow operations?

Scenario Under Consideration

Analyze the time complexity of fetching data with and without caching.

// Pseudo-code for data fetch with caching
var cache = new Dictionary<string, string>();

string GetData(string key) {
  if (cache.ContainsKey(key)) {
    return cache[key]; // fast return from cache
  } else {
    var data = FetchFromDatabase(key); // slow operation
    cache[key] = data;
    return data;
  }
}
    

This code tries to get data from a fast cache first, and only calls the slow database if needed.

Identify Repeating Operations

Look at what happens each time we ask for data:

• Primary operation: Checking cache and possibly fetching from database.
• How many times: Once per data request.
• Dominant operation: Database fetch when cache miss happens, which is slow.

How Execution Grows With Input

When many requests come in, cache hits avoid slow database calls.

Input Size (n)	Approx. Database Calls
10	Up to 10 without cache, fewer with cache
100	Up to 100 without cache, much fewer with cache
1000	Up to 1000 without cache, far fewer with cache

Pattern observation: Cache reduces repeated slow calls, so growth in slow operations is much slower than total requests.

Final Time Complexity

Time Complexity: O(1) for cache hits, O(n) for initial cache misses

This means once data is cached, each request is very fast and does not grow with number of requests.

Common Mistake

[X] Wrong: "Caching always makes every request faster from the start."

[OK] Correct: The first time data is requested, it must be fetched from the slow source before caching helps.

Interview Connect

Understanding caching shows you can improve cloud system speed by reducing repeated slow work, a key skill in real projects.

Self-Check

"What if the cache size is limited and old data is removed? How would that affect the time complexity?"