Open file cache in Nginx - Deep Dive

Overview - Open file cache

What is it?

Open file cache is a feature in nginx that stores information about files that nginx accesses frequently. Instead of opening and reading the file from disk every time, nginx keeps the file details in memory for faster access. This helps speed up serving static files and reduces the load on the server's file system.

Why it matters

Without open file cache, nginx would repeatedly open and read files from disk for every request, causing slower response times and higher disk usage. This can make websites feel sluggish and increase server resource consumption. Open file cache improves performance and efficiency, especially for busy websites serving many static files.

Where it fits

Before learning open file cache, you should understand basic nginx configuration and how nginx serves static files. After mastering open file cache, you can explore other nginx performance features like proxy caching, buffer tuning, and load balancing.

Mental Model

Core Idea

Open file cache keeps file information in memory so nginx can quickly reuse it without reopening files from disk each time.

Think of it like...

It's like keeping your frequently used tools on your desk instead of fetching them from a toolbox every time you need them.

┌─────────────────────────────┐
│ Client requests file        │
└─────────────┬───────────────┘
              │
      ┌───────▼────────┐
      │ Check open file │
      │ cache for file  │
      └───────┬────────┘
              │
    ┌─────────┴──────────┐
    │ File info in cache? │
    └───────┬───────┬────┘
            │       │
         Yes│       │No
            │       │
   ┌────────▼─┐  ┌──▼─────────┐
   │ Use cache│  │ Open file  │
   │ info     │  │ from disk  │
   └──────────┘  └────┬───────┘
                       │
               ┌───────▼────────┐
               │ Store file info │
               │ in cache       │
               └─────────────────┘

Build-Up - 7 Steps

1

Foundation

What is open file cache

Concept: Introduce the basic idea of caching file information in nginx.

Nginx serves files by opening them from disk each time a client requests them. This can be slow if many requests come in. Open file cache stores file details like file descriptor, size, and modification time in memory. This avoids reopening the file repeatedly.

Result

Nginx can serve files faster by reusing cached file info instead of opening files from disk every time.

Understanding that file operations are costly helps see why caching file info speeds up nginx.

2

Foundation

Basic open file cache configuration

Concept: Learn how to enable and configure open file cache in nginx.

You enable open file cache in nginx with the directive 'open_file_cache' inside the http, server, or location block. For example: open_file_cache max=1000 inactive=20s; open_file_cache_valid 30s; open_file_cache_min_uses 2; open_file_cache_errors on; - max=1000 limits cache size - inactive=20s removes unused entries - valid=30s sets how long cache is valid - min_uses=2 caches files used at least twice - errors on caches file errors too

Result

Nginx starts caching file info according to these settings, improving performance for repeated file requests.

Knowing these settings lets you control cache size, freshness, and which files get cached.

3

Intermediate

How cache expiration works

🤔Before reading on: do you think cached file info updates immediately when the file changes, or only after cache expiration? Commit to your answer.

Concept: Understand when nginx refreshes cached file info based on time and usage.

Nginx keeps cached file info valid for the time set by 'open_file_cache_valid'. After this time, nginx rechecks the file on disk to update the cache. Also, if a file is not used for the 'inactive' period, it is removed from cache. This balances freshness and performance.

Result

Cached file info is refreshed periodically, so nginx serves up-to-date files without checking disk every time.

Knowing cache expiration prevents stale file serving and helps tune performance vs freshness.

4

Intermediate

Caching file errors and misses

🤔Before reading on: do you think nginx caches file errors like 'file not found'? Commit to your answer.

Concept: Learn that nginx can cache file errors to avoid repeated disk checks for missing files.

With 'open_file_cache_errors on;', nginx caches errors like 'file not found' or permission denied. This prevents repeated disk checks for missing or inaccessible files, reducing load. Without this, nginx would check disk every time for missing files, slowing down responses.

Result

Nginx quickly returns errors for missing files without hitting disk repeatedly.

Caching errors improves performance for bad requests and reduces disk overhead.

5

Intermediate

Using open file cache with static file serving

Concept: See how open file cache improves static file delivery in nginx.

Static files like images, CSS, and JavaScript are served often. Open file cache stores their info so nginx can quickly send them without reopening files. This reduces latency and server load, especially under heavy traffic.

Result

Static file requests are faster and use fewer server resources.

Understanding this shows why open file cache is essential for high-performance static websites.

6

Advanced

Cache size and eviction policies

🤔Before reading on: do you think nginx evicts least recently used files first or random files when cache is full? Commit to your answer.

Concept: Learn how nginx manages cache size and removes old entries.

Nginx uses a least recently used (LRU) policy to evict files when cache reaches 'max' size. Files not accessed recently or inactive longer than 'inactive' time are removed first. This keeps cache fresh and within memory limits.

Result

Cache stays within size limits and keeps frequently used files for fast access.

Knowing eviction policies helps tune cache size and avoid performance drops.

7

Expert

Open file cache internals and edge cases

🤔Before reading on: do you think open file cache stores file content or only metadata? Commit to your answer.

Concept: Understand what exactly nginx caches and how it handles tricky cases like file changes or symbolic links.

Open file cache stores file metadata like file descriptor, size, modification time, and inode, but not file content. When a file changes, nginx detects it after cache expiration and updates info. Symbolic links are resolved and cached as well. However, caching can cause stale info if files change rapidly within cache validity period.

Result

Nginx serves files efficiently but may briefly serve stale files if they change too fast.

Knowing cache stores metadata, not content, clarifies its limits and helps avoid subtle bugs in dynamic file environments.

Under the Hood

Nginx maintains an in-memory hash table for open file cache entries. Each entry holds file metadata such as file descriptor, size, modification time, and inode number. When a request comes, nginx checks this cache first. If the entry exists and is valid, nginx reuses the open file descriptor and metadata, avoiding system calls to open or stat the file again. If not, nginx opens the file, reads metadata, and stores it in the cache. The cache uses LRU eviction and respects configured max size and inactive time. Cached errors are stored similarly to avoid repeated failed lookups.

Why designed this way?

Opening files and checking metadata on disk is expensive and slows down serving static content. Early nginx versions opened files on every request, causing high disk I/O. The open file cache was introduced to reduce system calls and improve throughput. It caches metadata, not content, to keep memory usage low and avoid complexity of content caching. The design balances performance, memory use, and freshness by configurable expiration and eviction policies.

┌───────────────────────────────┐
│ Client Request for File        │
└───────────────┬───────────────┘
                │
        ┌───────▼────────┐
        │ Check Cache    │
        │ (Hash Table)   │
        └───────┬────────┘
                │
      ┌─────────┴──────────┐
      │ Cache Hit?          │
      └───────┬───────┬────┘
              │       │
           Yes│       │No
              │       │
      ┌───────▼─┐  ┌──▼─────────┐
      │ Use FD  │  │ Open File  │
      │ & Meta  │  │ & Stat     │
      └─────────┘  └────┬───────┘
                         │
                 ┌───────▼────────┐
                 │ Store in Cache │
                 │ (Hash Table)   │
                 └────────────────┘

Myth Busters - 4 Common Misconceptions

Quick: Does open file cache store the actual file content in memory? Commit to yes or no.

Common Belief: Open file cache stores the full content of files in memory for faster serving.

Reality: Open file cache only stores file metadata like file descriptor and modification time, not the file content itself.

Why it matters: Believing it caches content can lead to expecting instant updates on file changes, but cached metadata can cause brief stale content serving.

Quick: Does open file cache immediately detect file changes? Commit to yes or no.

Common Belief: Open file cache instantly updates when a file changes on disk.

Reality: Open file cache updates file info only after the cache validity period expires or the file is accessed again after inactivity.

Why it matters: This delay can cause nginx to serve outdated files briefly, which matters for frequently changing content.

Quick: Does enabling open file cache always improve performance? Commit to yes or no.

Common Belief: Turning on open file cache always makes nginx faster.

Reality: If cache settings are too small or too large, or if files change very often, open file cache can cause overhead or stale data issues.

Why it matters: Misconfigured cache can degrade performance or cause incorrect file serving.

Quick: Does open file cache cache errors like 'file not found' by default? Commit to yes or no.

Common Belief: Nginx does not cache file errors; it always checks disk for missing files.

Reality: With 'open_file_cache_errors on;', nginx caches file errors to avoid repeated disk checks for missing files.

Why it matters: Not caching errors can cause unnecessary disk load and slower error responses.

Expert Zone

1

Open file cache entries are shared across worker processes using shared memory, enabling efficient multi-process caching.

2

The cache stores file descriptors which can hit system limits if max cache size is too large, requiring careful tuning.

3

Symbolic links are resolved and cached as the target file, which can cause confusion if symlinks change frequently.

When NOT to use

Avoid open file cache for highly dynamic files that change multiple times per second, as cache validity delays updates. Instead, use proxy caching or content delivery networks (CDNs) for dynamic content acceleration.

Production Patterns

In production, open file cache is commonly used with static file locations to reduce disk I/O. It is combined with gzip compression and HTTP caching headers for best performance. Operators tune max size and inactive time based on traffic patterns and file change frequency.

Connections

Operating System File Descriptor Cache

Builds-on

Understanding OS-level file descriptor caching helps grasp why nginx caches file descriptors too, reducing system calls and improving performance.

Web Browser Cache

Similar pattern

Both caches store information to avoid repeated work: browsers cache files to avoid re-downloading, nginx caches file info to avoid re-opening files.

Human Memory Recall

Analogous process

Just like humans remember frequently used information to avoid re-learning, nginx remembers file info to avoid re-accessing disk.

Common Pitfalls

#1 Setting open_file_cache max size too low causing frequent cache evictions.

Wrong approach:

open_file_cache max=10 inactive=20s;
open_file_cache_valid 30s;
open_file_cache_min_uses 2;
open_file_cache_errors on;

Correct approach:

open_file_cache max=1000 inactive=20s;
open_file_cache_valid 30s;
open_file_cache_min_uses 2;
open_file_cache_errors on;

Root cause:

Misunderstanding cache size impact leads to too small cache causing constant evictions and performance loss.

#2 Not enabling open_file_cache_errors causing repeated disk checks for missing files.

Wrong approach:

open_file_cache max=1000 inactive=20s;
open_file_cache_valid 30s;
open_file_cache_min_uses 2;

Correct approach:

open_file_cache max=1000 inactive=20s;
open_file_cache_valid 30s;
open_file_cache_min_uses 2;
open_file_cache_errors on;

Root cause:

Ignoring error caching causes unnecessary disk I/O and slower error responses.

#3 Expecting immediate file update detection with open file cache enabled.

Wrong approach:

open_file_cache max=1000 inactive=20s;
open_file_cache_valid 300s;
open_file_cache_min_uses 2;
open_file_cache_errors on;

Correct approach:

open_file_cache max=1000 inactive=20s;
open_file_cache_valid 5s;
open_file_cache_min_uses 2;
open_file_cache_errors on;

Root cause:

Setting too long cache validity causes stale file info; misunderstanding cache expiration timing.

Key Takeaways

Open file cache stores file metadata in memory to speed up nginx file serving by avoiding repeated disk operations.

It caches file descriptors, sizes, modification times, and can also cache file errors to reduce disk load.

Cache entries expire based on configured time and usage, balancing freshness and performance.

Misconfiguring cache size or expiration can cause performance issues or stale content serving.

Open file cache is essential for high-performance static file delivery in nginx but is not suitable for very dynamic files.