Natural language processing basics in Intro to Computing - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to tokenize the sentence into words using Python.

Intro to Computing

from nltk.tokenize import word_tokenize
sentence = "Hello world!"
tokens = [1](sentence)

Drag options to blanks, or click blank then click option'

Asplit

Btokenize

Ctokenize_words

Dword_tokenize

Attempts:

3 left

💡 Hint

Common Mistakes

Using the built-in split() method which doesn't handle punctuation well.

The word_tokenize function from NLTK splits a sentence into words correctly, handling punctuation.

2fill in blank

medium

Complete the code to convert all tokens to lowercase.

Intro to Computing

tokens = ['Hello', 'World']
lower_tokens = [[1] for token in tokens]

Drag options to blanks, or click blank then click option'

Atoken.capitalize()

Btoken.upper()

Ctoken.lower()

Dtoken.title()

Attempts:

3 left

💡 Hint

Common Mistakes

Using upper() which converts to uppercase instead.

Using token.lower() converts each word to lowercase, which is common in NLP preprocessing.

3fill in blank

hard

Fix the error in the code to remove stopwords from the token list.

Intro to Computing

from nltk.corpus import stopwords
stop_words = set(stopwords.words('english'))
tokens = ['this', 'is', 'a', 'test']
filtered = [word for word in tokens if [1] not in stop_words]

Drag options to blanks, or click blank then click option'

Astop_words

Bword

Ctokens

Dfiltered

Attempts:

3 left

💡 Hint

Common Mistakes

Using stop_words or tokens instead of the loop variable word.

The variable word is the element being checked in the list comprehension to filter out stopwords.

4fill in blank

hard

Fill both blanks to create a dictionary of word counts from a list of tokens.

Intro to Computing

tokens = ['apple', 'banana', 'apple', 'orange', 'banana', 'apple']
word_counts = { [1]: tokens.count([2]) for [1] in set(tokens) }

Drag options to blanks, or click blank then click option'

Aword

Btoken

Ditem

Attempts:

3 left

💡 Hint

Common Mistakes

Using different variable names for the key and count argument.

We use word as the variable name in the dictionary comprehension and count how many times each word appears in tokens.

5fill in blank

hard

Fill all three blanks to create a list of stemmed tokens using NLTK's PorterStemmer.

Intro to Computing

from nltk.stem import PorterStemmer
ps = PorterStemmer()
tokens = ['running', 'jumps', 'easily']
stemmed = [ps.[1](token) for [2] in tokens if len([3]) > 2]

Drag options to blanks, or click blank then click option'

Astem

Bword

Dtoken

Attempts:

3 left

💡 Hint

Common Mistakes

Using inconsistent variable names or wrong method names like 'stemmer'.

The method to stem words is stem. The loop variable is token, which is also used in the length check.

Practice

1. What is the main goal of Natural Language Processing (NLP)?

easy

A. To help computers understand and work with human language

B. To create images from text descriptions

C. To speed up computer hardware

D. To encrypt messages for security

Solution

1. Step 1: Understand NLP purpose

   NLP is about making computers understand human language, like English or Spanish.

2. Step 2: Compare options

   Only To help computers understand and work with human language talks about computers working with human language, which matches NLP's goal.

3. Final Answer:

   To help computers understand and work with human language -> Option A

4. Quick Check:

   NLP = computers understand human language [OK]

Hint: NLP = computers + human language understanding [OK]

Common Mistakes:

• Confusing NLP with image processing
• Thinking NLP speeds up hardware
• Mixing NLP with encryption tasks

2. Which of the following is the correct way to split a sentence into words in Python?

easy

A. sentence.add()

B. sentence.split()

C. sentence.remove()

D. sentence.join()

Solution

1. Step 1: Recall Python string methods

   To split a sentence into words, we use the split() method which breaks text at spaces by default.

2. Step 2: Check each option

   Only split() breaks text into parts; join() combines parts, remove() and add() are not string methods.

3. Final Answer:

   sentence.split() -> Option B

4. Quick Check:

   Split = break text into words [OK]

Hint: Use split() to break text into words [OK]

Common Mistakes:

• Using join() instead of split()
• Trying to use remove() on strings
• Confusing add() with string methods

3. What will be the output of this Python code?

text = "I love AI"
tokens = text.split()
print(len(tokens))

medium

A. 1

B. 0

C. 4

D. 3

Solution

1. Step 1: Tokenize the text

   Using split() on "I love AI" splits it into ['I', 'love', 'AI'], which has 3 words.

2. Step 2: Count tokens

   len(tokens) counts the number of items in the list, which is 3.

3. Final Answer:

   3 -> Option D

4. Quick Check:

   Words count = 3 [OK]

Hint: Count words after split() to get token count [OK]

Common Mistakes:

• Counting characters instead of words
• Assuming split() returns a string
• Forgetting split() splits by spaces

4. Find the error in this code snippet for tokenizing text:

text = "Hello world"
tokens = text.split('')
print(tokens)

medium

A. text should be a list, not a string

B. print() is missing parentheses

C. split('') is invalid; split() should have no argument or a space

D. tokens should be joined, not split

Solution

1. Step 1: Check split() argument

   split('') is invalid in Python because empty string is not a valid separator.

2. Step 2: Correct usage

   Use split() with no argument or split(' ') to split by spaces correctly.

3. Final Answer:

   split('') is invalid; split() should have no argument or a space -> Option C

4. Quick Check:

   split('') causes error [OK]

Hint: split('') causes error; use split() or split(' ') [OK]

Common Mistakes:

• Using empty string as split separator
• Thinking print() needs no parentheses in Python 3
• Confusing string and list types

5. You want to count how many times each word appears in a sentence using Python. Which code correctly does this?

hard

A. from collections import Counter
words = sentence.split()
counts = Counter(words)
print(counts)

B. words = sentence.join()
counts = len(words)
print(counts)

C. counts = sentence.count()
print(counts)

D. words = sentence.split('')
counts = dict(words)
print(counts)

Solution

1. Step 1: Tokenize sentence into words

   sentence.split() breaks the sentence into a list of words.

2. Step 2: Use Counter to count words

   Counter(words) counts how many times each word appears and stores results in a dictionary-like object.

3. Step 3: Check other options

   Options A, B, D use wrong methods or syntax for counting words.

4. Final Answer:

   from collections import Counter words = sentence.split() counts = Counter(words) print(counts) -> Option A

5. Quick Check:

   Use Counter + split() for word counts [OK]

Hint: Use Counter with split() to count words fast [OK]

Common Mistakes:

• Using join() instead of split()
• Calling count() without arguments
• Trying to split with empty string