Word frequency analysis in Data Analysis Python - Step-by-Step Execution

Concept Flow - Word frequency analysis

Start with text input

↓

Split text into words

↓

Create empty frequency dictionary

↓

For each word in list

↓

Check if word in dictionary?

No→Add word with count 1

Yes↓

Increase word count by 1

↓

After all words processed

↓

Output frequency dictionary

The process splits text into words, counts each word's occurrences, and stores counts in a dictionary.

Execution Sample

Data Analysis Python

text = "apple banana apple orange banana apple"
words = text.split()
freq = {}
for w in words:
    freq[w] = freq.get(w, 0) + 1
print(freq)

Counts how many times each word appears in the given text.

Execution Table

Step	Current Word (w)	freq Dictionary Before	Action	freq Dictionary After
1	apple	{}	Add 'apple' with count 1	{'apple': 1}
2	banana	{'apple': 1}	Add 'banana' with count 1	{'apple': 1, 'banana': 1}
3	apple	{'apple': 1, 'banana': 1}	Increment 'apple' count to 2	{'apple': 2, 'banana': 1}
4	orange	{'apple': 2, 'banana': 1}	Add 'orange' with count 1	{'apple': 2, 'banana': 1, 'orange': 1}
5	banana	{'apple': 2, 'banana': 1, 'orange': 1}	Increment 'banana' count to 2	{'apple': 2, 'banana': 2, 'orange': 1}
6	apple	{'apple': 2, 'banana': 2, 'orange': 1}	Increment 'apple' count to 3	{'apple': 3, 'banana': 2, 'orange': 1}
7	-	{'apple': 3, 'banana': 2, 'orange': 1}	All words processed	{'apple': 3, 'banana': 2, 'orange': 1}

💡 All words processed, loop ends.

Variable Tracker

Variable	Start	After 1	After 2	After 3	After 4	After 5	After 6	Final
freq	{}	{'apple': 1}	{'apple': 1, 'banana': 1}	{'apple': 2, 'banana': 1}	{'apple': 2, 'banana': 1, 'orange': 1}	{'apple': 2, 'banana': 2, 'orange': 1}	{'apple': 3, 'banana': 2, 'orange': 1}	{'apple': 3, 'banana': 2, 'orange': 1}
w	-	apple	banana	apple	orange	banana	apple	-

Key Moments - 2 Insights

Why do we use freq.get(w, 0) + 1 instead of just freq[w] + 1?

Because freq.get(w, 0) returns 0 if the word is not yet in the dictionary, so we can add it with count 1. This avoids errors when the word appears for the first time, as shown in steps 1 and 2 of the execution_table.

Why does the count for 'apple' increase multiple times?

Because 'apple' appears multiple times in the text. Each time the loop sees 'apple', it increases its count by 1, as seen in steps 1, 3, and 6 in the execution_table.

Visual Quiz - 3 Questions

Look at the execution_table at step 4. What is the freq dictionary after processing the word 'orange'?

A {'apple': 2, 'banana': 1, 'orange': 1}

B {'apple': 1, 'banana': 1, 'orange': 1}

C {'apple': 2, 'banana': 2, 'orange': 1}

D {'apple': 3, 'banana': 2, 'orange': 1}

💡 Hint

Check the 'freq Dictionary After' column for step 4 in the execution_table.

At which step does the count for 'banana' first increase?

A Step 1

B Step 5

C Step 2

D Step 6

💡 Hint

Look at the 'Current Word (w)' and 'Action' columns in the execution_table for when 'banana' is first added.

If the text had one more 'orange' at the end, how would the final freq dictionary change?

A {'apple': 3, 'banana': 2, 'orange': 1}

B {'apple': 3, 'banana': 2, 'orange': 2}

C {'apple': 3, 'banana': 3, 'orange': 1}

D {'apple': 4, 'banana': 2, 'orange': 1}

💡 Hint

Adding one more 'orange' increases its count by 1, check how counts update in the variable_tracker.

Concept Snapshot

Word frequency analysis:
- Split text into words using split()
- Use a dictionary to count words
- For each word: freq[word] = freq.get(word, 0) + 1
- Result is a dictionary with words as keys and counts as values
- Useful for text analysis and understanding word usage

Full Transcript

This example shows how to count how many times each word appears in a sentence. We start with a text string, split it into words, then create an empty dictionary to hold counts. For each word, we check if it is already in the dictionary. If not, we add it with count 1. If yes, we increase its count by 1. After processing all words, the dictionary shows the frequency of each word. This method helps us understand which words appear most often in text.