Storing files on disk vs memory in Express - Performance Comparison

Performance: Storing files on disk vs memory

MEDIUM IMPACT

This affects server response time and resource usage during file uploads and downloads.

Handling file uploads in an Express server

Express

const multer = require('multer');
const upload = multer({ dest: 'uploads/' });
app.post('/upload', upload.single('file'), (req, res) => {
  // File saved on disk automatically
  res.send('File saved on disk');
});

Streams file directly to disk, reducing memory use and avoiding blocking event loop.

📈 Performance GainLow memory usage, faster response for large files, stable server performance

Handling file uploads in an Express server

Express

app.post('/upload', (req, res) => {
  let fileData = [];
  req.on('data', chunk => fileData.push(chunk));
  req.on('end', () => {
    const fileBuffer = Buffer.concat(fileData);
    // Process file in memory
    res.send('File received in memory');
  });
});

Storing entire file in memory can exhaust RAM for large files, causing slowdowns or crashes.

📉 Performance CostBlocks event loop longer for large files, high memory usage, risk of server crash

Performance Comparison

Pattern	Memory Usage	CPU Load	Response Time Impact	Verdict
Store files in memory	High (depends on file size)	High (buffering large files)	Can block event loop causing slow response	[X] Bad
Store files on disk with streaming	Low (buffers small chunks)	Moderate (I/O overhead)	More stable response times for large files	[OK] Good

Rendering Pipeline

File storage choice affects server-side processing before response is sent; memory storage uses RAM and CPU, disk storage involves I/O operations.

→Server Processing

→I/O Operations

⚠️ BottleneckMemory storage can block event loop if large files are handled; disk storage bottleneck is slower I/O speed.

Optimization Tips

1Avoid storing large files fully in memory to prevent high RAM use and event loop blocking.

2Use streaming and disk storage for file uploads to keep server responsive and stable.

3Monitor server memory and CPU during file handling to detect performance bottlenecks early.

Performance Quiz - 3 Questions

What is a main risk of storing uploaded files entirely in memory on an Express server?

A Increased network latency

B Slower disk I/O operations

C High memory usage leading to server crashes

D Longer CSS rendering times

DevTools: Node.js Profiler or PM2 Monitoring

How to check: Use Node.js profiler or PM2 to monitor memory and CPU usage during file uploads; check event loop delay metrics.

What to look for: High memory spikes or event loop blocking indicate memory storage issues; stable memory and CPU usage indicate good disk streaming.

Practice

1. In Express, what is the main difference between storing uploaded files on disk versus in memory?

easy

A. Disk storage is faster than memory storage for file uploads.

B. Disk storage saves files physically on the server, memory storage keeps files temporarily in RAM.

C. Memory storage saves files permanently, disk storage deletes files after upload.

D. Disk storage only works with images, memory storage only works with text files.

Solution

1. Step 1: Understand disk storage in Express

   Disk storage saves uploaded files physically on the server's hard drive, making them persistent.

2. Step 2: Understand memory storage in Express

   Memory storage keeps files temporarily in RAM, which is faster but not persistent after server restarts.

3. Final Answer:

   Disk storage saves files physically on the server, memory storage keeps files temporarily in RAM. -> Option B

4. Quick Check:

   Disk = physical, Memory = temporary RAM [OK]

Hint: Disk = saved on server, Memory = temporary RAM [OK]

Common Mistakes:

• Thinking memory storage saves files permanently
• Assuming disk storage is always faster
• Confusing file types with storage methods

2. Which of the following is the correct way to configure multer for storing files in memory in Express?

easy

A. const upload = multer({ storage: multer.memoryStorage() });

B. const upload = multer({ storage: multer.diskStorage({}) });

C. const upload = multer({ dest: '/uploads' });

D. const upload = multer({ storage: 'memory' });

Solution

1. Step 1: Identify multer memory storage syntax

   Memory storage is set by calling multer.memoryStorage() and passing it to the storage option.

2. Step 2: Check options for correct syntax

   const upload = multer({ storage: multer.memoryStorage() }); correctly uses multer.memoryStorage() inside the storage property.

3. Final Answer:

   const upload = multer({ storage: multer.memoryStorage() }); -> Option A

4. Quick Check:

   Use multer.memoryStorage() for memory storage [OK]

Hint: Use multer.memoryStorage() to store files in RAM [OK]

Common Mistakes:

• Using string 'memory' instead of multer.memoryStorage()
• Confusing diskStorage with memoryStorage
• Setting dest property for memory storage

3. Given this Express code snippet using multer with memory storage:

const upload = multer({ storage: multer.memoryStorage() });
app.post('/upload', upload.single('file'), (req, res) => {
  console.log(req.file.buffer.length);
  res.send('File size: ' + req.file.size);
});

What will be logged and sent if a 5000-byte file is uploaded?

medium

A. Throws an error because buffer is not available

B. Logs undefined, sends 'File size: undefined'

C. Logs 0, sends 'File size: 0'

D. Logs 5000, sends 'File size: 5000'

Solution

1. Step 1: Understand multer memoryStorage behavior

   When using memoryStorage, the uploaded file is stored in req.file.buffer as a Buffer object containing the file data.

2. Step 2: Check properties used in code

   req.file.buffer.length gives the byte length of the file buffer, which will be 5000 for a 5000-byte file. req.file.size also holds the file size in bytes.

3. Final Answer:

   Logs 5000, sends 'File size: 5000' -> Option D

4. Quick Check:

   Memory storage buffer length = file size [OK]

Hint: Memory storage files have buffer and size properties [OK]

Common Mistakes:

• Assuming buffer is undefined in memoryStorage
• Confusing size with buffer length
• Expecting disk path properties in memory storage

4. You wrote this Express code to store files on disk:

const upload = multer({ storage: multer.diskStorage({
  destination: './uploads',
  filename: (req, file, cb) => cb(null, file.originalname)
}) });
app.post('/upload', upload.single('file'), (req, res) => {
  res.send('File saved');
});

But files are not saved and no error appears. What is the likely problem?

medium

A. The destination path './uploads' does not exist or lacks write permission.

B. filename callback must return a Promise instead of using cb.

C. upload.single should be upload.array for disk storage.

D. multer.diskStorage cannot be used with Express.

Solution

1. Step 1: Check diskStorage destination requirements

   The destination folder must exist and be writable by the server process; multer does not create folders automatically.

2. Step 2: Analyze why files are not saved

   If the './uploads' folder is missing or permission denied, multer silently fails to save files without throwing errors.

3. Final Answer:

   The destination path './uploads' does not exist or lacks write permission. -> Option A

4. Quick Check:

   Disk storage needs existing writable folder [OK]

Hint: Ensure upload folder exists and is writable [OK]

Common Mistakes:

• Expecting multer to create upload folders automatically
• Misusing filename callback with Promises
• Confusing single vs array upload methods

5. You want to upload large files in Express and process them quickly without saving to disk. Which approach is best and why?

hard

A. Use multer.diskStorage to save files on disk for persistence and later processing.

B. Use multer.memoryStorage to keep files in RAM for fast access but risk high memory use.

C. Use multer.memoryStorage but limit file size to avoid memory overflow.

D. Use multer.diskStorage with a temporary folder and delete files after processing.

Solution

1. Step 1: Consider large file upload challenges

   Large files can consume a lot of RAM if stored in memory, risking server crashes.

2. Step 2: Evaluate memoryStorage with file size limits

   Using memoryStorage with strict file size limits allows fast processing while preventing excessive memory use.

3. Step 3: Compare with diskStorage options

   Disk storage is persistent but slower; temporary disk storage with deletion adds complexity and latency.

4. Final Answer:

   Use multer.memoryStorage but limit file size to avoid memory overflow. -> Option C

5. Quick Check:

   MemoryStorage + size limit = fast and safe [OK]

Hint: Limit file size when using memory storage for large files [OK]

Common Mistakes:

• Ignoring memory limits causing crashes
• Assuming disk storage is always faster
• Not cleaning up temporary disk files