Reading files line by line in Python

Introduction

Reading files line by line helps you handle large files without using too much memory. It lets you process each line one at a time, like reading a book page by page.

When you want to read a big text file without loading it all at once.

When you need to process or analyze each line separately, like counting words per line.

When you want to read logs or data streams that are organized line by line.

When you want to stop reading after a certain line or condition is met.

When you want to save memory and avoid crashes with large files.

Syntax

Python

with open('filename.txt', 'r') as file:
    for line in file:
        # process the line
        print(line, end='')

The with statement automatically closes the file when done.

Each line includes the newline character \n at the end.

Examples

This reads each line and removes extra spaces and the newline at the end before printing.

Python

with open('example.txt', 'r') as file:
    for line in file:
        print(line.strip())

This uses readline() to read one line at a time until the file ends.

Python

file = open('example.txt', 'r')
line = file.readline()
while line:
    print(line.strip())
    line = file.readline()
file.close()

This reads all lines into a list first, then prints each line. Not memory efficient for big files.

Python

with open('example.txt', 'r') as file:
    lines = file.readlines()
for line in lines:
    print(line.strip())

Sample Program

This program first creates a file with three lines. Then it reads the file line by line and prints each line with a message.

Python

filename = 'sample.txt'

# Create a sample file
with open(filename, 'w') as f:
    f.write('Hello\n')
    f.write('World\n')
    f.write('Python is fun!\n')

# Read the file line by line
with open(filename, 'r') as file:
    for line in file:
        print(f'Read line: {line.strip()}')

Important Notes

Remember to use strip() if you want to remove the newline \n from each line.

Using with is best practice because it closes the file automatically.

Reading line by line is good for big files to save memory.

Summary

Reading files line by line lets you handle big files easily.

Use with open(filename) as file and a for loop to read lines.

Use strip() to clean lines before using them.

Practice

1. What does the following code do?

with open('data.txt') as file:
    for line in file:
        print(line)

easy

A. Creates a new file named 'data.txt'

B. Reads the whole file at once and prints it

C. Writes lines to 'data.txt'

D. Reads and prints each line from 'data.txt' including newline characters

Solution

1. Step 1: Understand the with open statement

   The code opens 'data.txt' for reading and ensures it closes automatically after use.

2. Step 2: Analyze the for loop over the file object

   Looping over the file reads it line by line, printing each line including its newline character.

3. Final Answer:

   Reads and prints each line from 'data.txt' including newline characters -> Option D

4. Quick Check:

   Reading lines one by one = Reads and prints each line from 'data.txt' including newline characters [OK]

Hint: Looping file reads lines one by one including newlines [OK]

Common Mistakes:

• Thinking it reads whole file at once
• Confusing reading with writing
• Assuming it creates a file

2. Which of these is the correct syntax to read a file line by line in Python?

easy

A. with open('file.txt') as f:
for line in f:
print(line)

B. open('file.txt') for line in f:
print(line)

C. with open('file.txt') as f:
while line in f:
print(line)

D. with open('file.txt') as f:
for line in file:
print(line)

Solution

1. Step 1: Check the correct use of with open

   with open('file.txt') as f: for line in f: print(line) correctly uses 'with open(filename) as f:' to open the file safely.

2. Step 2: Verify the for loop syntax

   with open('file.txt') as f: for line in f: print(line) uses 'for line in f:' which is the proper way to iterate lines in the file object.

3. Final Answer:

   with open('file.txt') as f:\n for line in f:\n print(line) -> Option A

4. Quick Check:

   Correct with open and for loop syntax = with open('file.txt') as f: for line in f: print(line) [OK]

Hint: Use 'with open' and 'for line in file' to read lines [OK]

Common Mistakes:

• Missing 'with' keyword
• Using wrong loop syntax like 'while line in f'
• Using undefined variable 'file' instead of 'f'

3. What will be the output of this code if 'test.txt' contains:\nLine1\nLine2\n\nLine4\n?

with open('test.txt') as f:
    for line in f:
        print(line.strip())

medium

A. Line1\nLine2\n\nLine4

B. Line1\nLine2\n\n\nLine4

C. Line1\nLine2\nLine4

D. Line1\nLine2\nLine4\n

Solution

1. Step 1: Understand strip() effect on lines

   strip() removes whitespace including newlines, so empty lines become empty strings.

2. Step 2: Analyze printing each stripped line

   Empty lines print as blank lines, but print() adds a newline, so empty lines show as blank lines.

3. Final Answer:

   Line1\nLine2\n\nLine4 -> Option A

4. Quick Check:

   strip() on empty -> '', print('') = blank line = Line1\nLine2\n\nLine4 [OK]

Hint: strip() removes newlines, print adds one newline per line [OK]

Common Mistakes:

• Assuming strip() keeps newlines
• Confusing blank lines with extra newlines
• Ignoring print() adds newline automatically

4. Find the error in this code snippet:

with open('file.txt') as f:
    for line in file:
        print(line)

medium

A. Indentation error inside the loop

B. Missing colon after for loop

C. Variable 'file' is undefined; should use 'f' instead

D. File not opened in write mode

Solution

1. Step 1: Check variable names used in the loop

   The file is opened as 'f', but the loop uses 'file' which is undefined.

2. Step 2: Confirm correct variable usage

   Changing 'file' to 'f' fixes the error and allows reading lines properly.

3. Final Answer:

   Variable 'file' is undefined; should use 'f' instead -> Option C

4. Quick Check:

   Use same variable name as in with open = Variable 'file' is undefined; should use 'f' instead [OK]

Hint: Use same variable name from with open in for loop [OK]

Common Mistakes:

• Using different variable names
• Assuming file is a keyword
• Ignoring variable scope inside with block

5. You want to read a large log file line by line and count how many lines contain the word 'error'. Which code snippet correctly does this?

hard

A. count = 0
with open('log.txt') as f:
lines = f.readlines()
for line in lines:
if 'error' in line:
count += 1
print(count)

B. count = 0
with open('log.txt') as f:
for line in f:
if 'error' in line:
count += 1
print(count)

C. count = 0
f = open('log.txt')
lines = f.readlines()
for line in lines:
if 'error' in line:
count += 1
f.close()
print(count)

D. count = 0
with open('log.txt') as f:
for line in f:
if line == 'error':
count += 1
print(count)

Solution

1. Step 1: Choose efficient line-by-line reading

   count = 0 with open('log.txt') as f: for line in f: if 'error' in line: count += 1 print(count) uses 'with open' and iterates file line by line, which is memory efficient for large files.

2. Step 2: Check condition for counting 'error' in line

   count = 0 with open('log.txt') as f: for line in f: if 'error' in line: count += 1 print(count) correctly checks if 'error' is anywhere in the line and increments count.

3. Final Answer:

   count = 0\nwith open('log.txt') as f:\n for line in f:\n if 'error' in line:\n count += 1\nprint(count) -> Option B

4. Quick Check:

   Efficient reading + correct condition = count = 0 with open('log.txt') as f: for line in f: if 'error' in line: count += 1 print(count) [OK]

Hint: Use with open and for line in file for big files [OK]

Common Mistakes:

• Using readlines() for big files (memory issue)
• Forgetting to close file without with
• Checking line equality instead of substring