How to Read File Line by Line in Node.js Easily

In Node.js, you can read a file line by line using the readline module combined with fs.createReadStream. This approach streams the file content and processes each line efficiently without loading the entire file into memory.

Syntax

Use the readline.createInterface method with an input stream from fs.createReadStream. Then listen for the line event to process each line as it is read.

• fs.createReadStream(path): Opens the file as a readable stream.
• readline.createInterface({ input }): Creates an interface to read lines from the stream.
• line event: Fired for each line read from the file.
• close event: Fired when the file reading is complete.

javascript

import fs from 'fs';
import readline from 'readline';

const fileStream = fs.createReadStream('file.txt');

const rl = readline.createInterface({
  input: fileStream,
  crlfDelay: Infinity
});

rl.on('line', (line) => {
  // process each line here
});

rl.on('close', () => {
  // done reading file
});

Example

This example reads a file named example.txt line by line and prints each line to the console. It demonstrates how to set up the stream and handle events properly.

javascript

import fs from 'fs';
import readline from 'readline';

async function readFileLineByLine(filePath) {
  const fileStream = fs.createReadStream(filePath);

  const rl = readline.createInterface({
    input: fileStream,
    crlfDelay: Infinity
  });

  for await (const line of rl) {
    console.log(`Line from file: ${line}`);
  }

  console.log('Finished reading file.');
}

readFileLineByLine('example.txt');

Output

Line from file: First line of the file
Line from file: Second line of the file
Line from file: Third line of the file
Finished reading file.

Common Pitfalls

Common mistakes include trying to read the entire file at once, which can cause memory issues with large files, or not handling the asynchronous nature of streams properly. Also, forgetting to set crlfDelay: Infinity can cause issues with different line endings.

Another pitfall is not using for await...of or event listeners correctly, which can lead to missing lines or incomplete reads.

javascript

import fs from 'fs';
import readline from 'readline';

// Wrong: Reading entire file at once (not line by line)
// const data = fs.readFileSync('example.txt', 'utf8');
// console.log(data);

// Right: Using readline with async iteration
async function readLinesCorrectly(filePath) {
  const fileStream = fs.createReadStream(filePath);
  const rl = readline.createInterface({ input: fileStream, crlfDelay: Infinity });

  for await (const line of rl) {
    console.log(line);
  }
}

readLinesCorrectly('example.txt');

Quick Reference

Remember these key points when reading files line by line in Node.js:

• Use fs.createReadStream to avoid loading the whole file.
• Use readline.createInterface with the stream as input.
• Set crlfDelay: Infinity to handle all line endings.
• Use for await...of or line event to process lines.
• Handle the close event to know when reading is done.

Key Takeaways

Use the readline module with fs.createReadStream to read files line by line efficiently.

Always set crlfDelay: Infinity to correctly handle different line endings.

Use async iteration (for await...of) or the 'line' event to process each line.

Avoid reading the entire file into memory for large files to prevent performance issues.

Listen for the 'close' event to know when the file reading is complete.