Reading files line by line in Python - Time & Space Complexity

Time Complexity: Reading files line by line

O(n)

Understanding Time Complexity

When reading a file line by line, we want to know how the time it takes grows as the file gets bigger.

We ask: How does the program's work increase when there are more lines to read?

Scenario Under Consideration

Analyze the time complexity of the following code snippet.

with open('file.txt', 'r') as file:
    for line in file:
        print(line.strip())

This code opens a file and reads it one line at a time, printing each line after removing extra spaces.

Identify Repeating Operations

Identify the loops, recursion, array traversals that repeat.

• Primary operation: Looping through each line in the file.
• How many times: Once for every line in the file.

How Execution Grows With Input

As the number of lines grows, the program reads and processes more lines one by one.

Input Size (n)	Approx. Operations
10	About 10 line reads and prints
100	About 100 line reads and prints
1000	About 1000 line reads and prints

Pattern observation: The work grows directly with the number of lines; double the lines, double the work.

Final Time Complexity

Time Complexity: O(n)

This means the time to read the file grows in a straight line with the number of lines.

Common Mistake

[X] Wrong: "Reading a file line by line is always very slow because it reads each character individually."

[OK] Correct: The code reads line by line, not character by character, so it processes one line at a time efficiently.

Interview Connect

Understanding how reading files scales helps you write programs that handle big data smoothly and shows you think about efficiency.

Self-Check

"What if we read the whole file at once into memory instead of line by line? How would the time complexity change?"

Practice

1. What does the following code do?

with open('data.txt') as file:
    for line in file:
        print(line)

easy

A. Creates a new file named 'data.txt'

B. Reads the whole file at once and prints it

C. Writes lines to 'data.txt'

D. Reads and prints each line from 'data.txt' including newline characters

Solution

1. Step 1: Understand the with open statement

   The code opens 'data.txt' for reading and ensures it closes automatically after use.

2. Step 2: Analyze the for loop over the file object

   Looping over the file reads it line by line, printing each line including its newline character.

3. Final Answer:

   Reads and prints each line from 'data.txt' including newline characters -> Option D

4. Quick Check:

   Reading lines one by one = Reads and prints each line from 'data.txt' including newline characters [OK]

Hint: Looping file reads lines one by one including newlines [OK]

Common Mistakes:

• Thinking it reads whole file at once
• Confusing reading with writing
• Assuming it creates a file

2. Which of these is the correct syntax to read a file line by line in Python?

easy

A. with open('file.txt') as f:
for line in f:
print(line)

B. open('file.txt') for line in f:
print(line)

C. with open('file.txt') as f:
while line in f:
print(line)

D. with open('file.txt') as f:
for line in file:
print(line)

Solution

1. Step 1: Check the correct use of with open

   with open('file.txt') as f: for line in f: print(line) correctly uses 'with open(filename) as f:' to open the file safely.

2. Step 2: Verify the for loop syntax

   with open('file.txt') as f: for line in f: print(line) uses 'for line in f:' which is the proper way to iterate lines in the file object.

3. Final Answer:

   with open('file.txt') as f:\n for line in f:\n print(line) -> Option A

4. Quick Check:

   Correct with open and for loop syntax = with open('file.txt') as f: for line in f: print(line) [OK]

Hint: Use 'with open' and 'for line in file' to read lines [OK]

Common Mistakes:

• Missing 'with' keyword
• Using wrong loop syntax like 'while line in f'
• Using undefined variable 'file' instead of 'f'

3. What will be the output of this code if 'test.txt' contains:\nLine1\nLine2\n\nLine4\n?

with open('test.txt') as f:
    for line in f:
        print(line.strip())

medium

A. Line1\nLine2\n\nLine4

B. Line1\nLine2\n\n\nLine4

C. Line1\nLine2\nLine4

D. Line1\nLine2\nLine4\n

Solution

1. Step 1: Understand strip() effect on lines

   strip() removes whitespace including newlines, so empty lines become empty strings.

2. Step 2: Analyze printing each stripped line

   Empty lines print as blank lines, but print() adds a newline, so empty lines show as blank lines.

3. Final Answer:

   Line1\nLine2\n\nLine4 -> Option A

4. Quick Check:

   strip() on empty -> '', print('') = blank line = Line1\nLine2\n\nLine4 [OK]

Hint: strip() removes newlines, print adds one newline per line [OK]

Common Mistakes:

• Assuming strip() keeps newlines
• Confusing blank lines with extra newlines
• Ignoring print() adds newline automatically

4. Find the error in this code snippet:

with open('file.txt') as f:
    for line in file:
        print(line)

medium

A. Indentation error inside the loop

B. Missing colon after for loop

C. Variable 'file' is undefined; should use 'f' instead

D. File not opened in write mode

Solution

1. Step 1: Check variable names used in the loop

   The file is opened as 'f', but the loop uses 'file' which is undefined.

2. Step 2: Confirm correct variable usage

   Changing 'file' to 'f' fixes the error and allows reading lines properly.

3. Final Answer:

   Variable 'file' is undefined; should use 'f' instead -> Option C

4. Quick Check:

   Use same variable name as in with open = Variable 'file' is undefined; should use 'f' instead [OK]

Hint: Use same variable name from with open in for loop [OK]

Common Mistakes:

• Using different variable names
• Assuming file is a keyword
• Ignoring variable scope inside with block

5. You want to read a large log file line by line and count how many lines contain the word 'error'. Which code snippet correctly does this?

hard

A. count = 0
with open('log.txt') as f:
lines = f.readlines()
for line in lines:
if 'error' in line:
count += 1
print(count)

B. count = 0
with open('log.txt') as f:
for line in f:
if 'error' in line:
count += 1
print(count)

C. count = 0
f = open('log.txt')
lines = f.readlines()
for line in lines:
if 'error' in line:
count += 1
f.close()
print(count)

D. count = 0
with open('log.txt') as f:
for line in f:
if line == 'error':
count += 1
print(count)

Solution

1. Step 1: Choose efficient line-by-line reading

   count = 0 with open('log.txt') as f: for line in f: if 'error' in line: count += 1 print(count) uses 'with open' and iterates file line by line, which is memory efficient for large files.

2. Step 2: Check condition for counting 'error' in line

   count = 0 with open('log.txt') as f: for line in f: if 'error' in line: count += 1 print(count) correctly checks if 'error' is anywhere in the line and increments count.

3. Final Answer:

   count = 0\nwith open('log.txt') as f:\n for line in f:\n if 'error' in line:\n count += 1\nprint(count) -> Option B

4. Quick Check:

   Efficient reading + correct condition = count = 0 with open('log.txt') as f: for line in f: if 'error' in line: count += 1 print(count) [OK]

Hint: Use with open and for line in file for big files [OK]

Common Mistakes:

• Using readlines() for big files (memory issue)
• Forgetting to close file without with
• Checking line equality instead of substring