Reading files line by line in Python

Introduction

Reading files line by line helps you handle large files without using too much memory. It lets you process each line one at a time, like reading a book page by page.

When you want to read a big text file without loading it all at once.

When you need to process or analyze each line separately, like counting words per line.

When you want to read logs or data streams that are organized line by line.

When you want to stop reading after a certain line or condition is met.

When you want to save memory and avoid crashes with large files.

Syntax

Python

with open('filename.txt', 'r') as file:
    for line in file:
        # process the line
        print(line, end='')

The with statement automatically closes the file when done.

Each line includes the newline character \n at the end.

Examples

This reads each line and removes extra spaces and the newline at the end before printing.

Python

with open('example.txt', 'r') as file:
    for line in file:
        print(line.strip())

This uses readline() to read one line at a time until the file ends.

Python

file = open('example.txt', 'r')
line = file.readline()
while line:
    print(line.strip())
    line = file.readline()
file.close()

This reads all lines into a list first, then prints each line. Not memory efficient for big files.

Python

with open('example.txt', 'r') as file:
    lines = file.readlines()
for line in lines:
    print(line.strip())

Sample Program

This program first creates a file with three lines. Then it reads the file line by line and prints each line with a message.

Python

filename = 'sample.txt'

# Create a sample file
with open(filename, 'w') as f:
    f.write('Hello\n')
    f.write('World\n')
    f.write('Python is fun!\n')

# Read the file line by line
with open(filename, 'r') as file:
    for line in file:
        print(f'Read line: {line.strip()}')

Important Notes

Remember to use strip() if you want to remove the newline \n from each line.

Using with is best practice because it closes the file automatically.

Reading line by line is good for big files to save memory.

Summary

Reading files line by line lets you handle big files easily.

Use with open(filename) as file and a for loop to read lines.

Use strip() to clean lines before using them.