How to Store Large Files in MongoDB Using GridFS

To store large files in MongoDB, use GridFS, a specification for storing and retrieving files that exceed the 16MB document size limit. GridFS splits files into smaller chunks and stores them in two collections, allowing efficient storage and retrieval.

Syntax

GridFS stores files by splitting them into chunks and saving metadata separately. It uses two collections: fs.files for file metadata and fs.chunks for file data chunks.

The main commands are:

• openUploadStream(filename): Uploads a file stream to GridFS.
• openDownloadStreamByName(filename): Downloads a file stream by filename.

javascript

const { MongoClient, GridFSBucket } = require('mongodb');

async function storeFile(db, filename, fileStream) {
  const bucket = new GridFSBucket(db);
  const uploadStream = bucket.openUploadStream(filename);
  fileStream.pipe(uploadStream);
  return new Promise((resolve, reject) => {
    uploadStream.on('finish', () => resolve(uploadStream.id));
    uploadStream.on('error', reject);
  });
}

Example

This example shows how to upload a large file to MongoDB using GridFS and then download it back.

javascript

const { MongoClient, GridFSBucket } = require('mongodb');
const fs = require('fs');

async function run() {
  const client = new MongoClient('mongodb://localhost:27017');
  await client.connect();
  const db = client.db('myfilesdb');
  const bucket = new GridFSBucket(db);

  // Upload file
  const uploadStream = bucket.openUploadStream('largefile.txt');
  fs.createReadStream('largefile.txt').pipe(uploadStream);

  uploadStream.on('finish', async () => {
    console.log('File uploaded with id:', uploadStream.id);

    // Download file
    const downloadStream = bucket.openDownloadStreamByName('largefile.txt');
    const writeStream = fs.createWriteStream('downloaded_largefile.txt');
    downloadStream.pipe(writeStream);

    writeStream.on('finish', () => {
      console.log('File downloaded successfully');
      client.close();
    });
  });
}

run().catch(console.error);

Output

File uploaded with id: <ObjectId>
File downloaded successfully

Common Pitfalls

• Trying to store large files directly in a single document will fail due to MongoDB's 16MB document size limit.
• Not using streams can cause memory issues when handling large files.
• Forgetting to handle upload or download errors can cause silent failures.
• Using the same filename for multiple files without versioning can overwrite data.

javascript

/* Wrong way: storing large file as a single document (will fail if >16MB) */
// db.files.insertOne({ filename: 'bigfile', data: <large binary> });

/* Right way: use GridFS with streams */
const bucket = new GridFSBucket(db);
const uploadStream = bucket.openUploadStream('bigfile');
fileStream.pipe(uploadStream);

Quick Reference

Command	Description
openUploadStream(filename)	Starts uploading a file stream to GridFS.
openDownloadStreamByName(filename)	Starts downloading a file stream by filename.
fs.files collection	Stores file metadata like filename, length, upload date.
fs.chunks collection	Stores file data in chunks of 255KB by default.

Key Takeaways

Use GridFS to store files larger than 16MB in MongoDB.

GridFS splits files into chunks and stores metadata separately.

Always use streams to upload and download files to avoid memory issues.

Handle errors during upload and download to ensure reliability.

Avoid overwriting files by managing filenames or using unique IDs.