Natural language processing basics in Intro to Computing - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to tokenize the sentence into words using Python.

Intro to Computing

from nltk.tokenize import word_tokenize
sentence = "Hello world!"
tokens = [1](sentence)

Drag options to blanks, or click blank then click option'

Asplit

Btokenize

Ctokenize_words

Dword_tokenize

Attempts:

3 left

💡 Hint

Common Mistakes

Using the built-in split() method which doesn't handle punctuation well.

The word_tokenize function from NLTK splits a sentence into words correctly, handling punctuation.

2fill in blank

medium

Complete the code to convert all tokens to lowercase.

Intro to Computing

tokens = ['Hello', 'World']
lower_tokens = [[1] for token in tokens]

Drag options to blanks, or click blank then click option'

Atoken.capitalize()

Btoken.upper()

Ctoken.lower()

Dtoken.title()

Attempts:

3 left

💡 Hint

Common Mistakes

Using upper() which converts to uppercase instead.

Using token.lower() converts each word to lowercase, which is common in NLP preprocessing.

3fill in blank

hard

Fix the error in the code to remove stopwords from the token list.

Intro to Computing

from nltk.corpus import stopwords
stop_words = set(stopwords.words('english'))
tokens = ['this', 'is', 'a', 'test']
filtered = [word for word in tokens if [1] not in stop_words]

Drag options to blanks, or click blank then click option'

Astop_words

Bword

Ctokens

Dfiltered

Attempts:

3 left

💡 Hint

Common Mistakes

Using stop_words or tokens instead of the loop variable word.

The variable word is the element being checked in the list comprehension to filter out stopwords.

4fill in blank

hard

Fill both blanks to create a dictionary of word counts from a list of tokens.

Intro to Computing

tokens = ['apple', 'banana', 'apple', 'orange', 'banana', 'apple']
word_counts = { [1]: tokens.count([2]) for [1] in set(tokens) }

Drag options to blanks, or click blank then click option'

Aword

Btoken

Ditem

Attempts:

3 left

💡 Hint

Common Mistakes

Using different variable names for the key and count argument.

We use word as the variable name in the dictionary comprehension and count how many times each word appears in tokens.

5fill in blank

hard

Fill all three blanks to create a list of stemmed tokens using NLTK's PorterStemmer.

Intro to Computing

from nltk.stem import PorterStemmer
ps = PorterStemmer()
tokens = ['running', 'jumps', 'easily']
stemmed = [ps.[1](token) for [2] in tokens if len([3]) > 2]

Drag options to blanks, or click blank then click option'

Astem

Bword

Dtoken

Attempts:

3 left

💡 Hint

Common Mistakes

Using inconsistent variable names or wrong method names like 'stemmer'.

The method to stem words is stem. The loop variable is token, which is also used in the length check.