Lexicon-based approaches (VADER) in NLP - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

VADER Sentiment Polarity Scores Output

What is the output of the following Python code using VADER sentiment analyzer?

NLP

from vaderSentiment.vaderSentiment import SentimentIntensityAnalyzer
analyzer = SentimentIntensityAnalyzer()
sentence = "I love sunny days but hate the rain."
scores = analyzer.polarity_scores(sentence)
print(scores)

A {"neg": 0.292, "neu": 0.458, "pos": 0.25, "compound": -0.0516}

B {"neg": 0.25, "neu": 0.5, "pos": 0.25, "compound": 0.0}

C {"neg": 0.0, "neu": 0.5, "pos": 0.5, "compound": 0.6249}

D {"neg": 0.292, "neu": 0.458, "pos": 0.25, "compound": 0.6249}

💡 Hint

Remember that VADER returns four scores: negative, neutral, positive, and compound.

VADER analyzes the sentence and assigns scores for negative, neutral, positive, and a compound score summarizing overall sentiment. The sentence has both positive and negative parts, so the compound score is close to zero but slightly negative.

🧠 Conceptual

intermediate

Understanding VADER's Compound Score Range

What is the range of the compound score produced by VADER sentiment analysis, and what does it represent?

A Range is from 0 to 100; it represents the percentage of positive words in the text.

B Range is from 0 to 1; it represents the probability of positive sentiment.

C Range is from -1 to 0; it represents the intensity of negative sentiment only.

D Range is from -1 to 1; it represents the overall sentiment from most negative to most positive.

💡 Hint

Think about how VADER summarizes sentiment in one number.

The compound score is a normalized value between -1 (most negative) and 1 (most positive) summarizing the overall sentiment of the text.

❓ Metrics

advanced

Evaluating VADER Sentiment Classification Accuracy

You have a dataset of 1000 sentences labeled as positive or negative. Using VADER's compound score with a threshold of 0.05 for positive and -0.05 for negative, which metric best measures how well VADER classifies sentiment?

A Accuracy, because it measures the proportion of correctly classified sentences.

B Silhouette Score, because it measures cluster separation.

C Mean Squared Error, because it measures the difference between predicted and true labels.

D Perplexity, because it measures language model uncertainty.

💡 Hint

Consider the task is classification, not regression or clustering.

Accuracy is the appropriate metric for classification tasks to measure the proportion of correct predictions.

🔧 Debug

advanced

Identifying Error in VADER Usage Code

What error will the following code raise when run? from vaderSentiment.vaderSentiment import SentimentIntensityAnalyzer analyzer = SentimentIntensityAnalyzer() scores = analyzer.polarity_scores(12345) print(scores)

A ValueError: invalid literal for int() with base 10

B TypeError: expected string or bytes-like object

C No error; outputs sentiment scores

D AttributeError: 'int' object has no attribute 'lower'

💡 Hint

VADER preprocesses input by converting it to a string first.

No error is raised. VADER's polarity_scores converts non-string inputs to strings using str() before processing. '12345' contains no sentiment words, yielding neutral scores.

❓ Model Choice

expert

Choosing the Best Sentiment Analysis Approach for Social Media Text

You want to analyze sentiment of short social media posts with slang, emojis, and informal language. Which approach is best suited?

A Traditional bag-of-words model with logistic regression, because it captures word frequency.

B Lexicon-based approach using VADER, because it is tuned for social media text and handles emojis and slang well.

C Topic modeling with LDA, because it finds themes in text.

D Rule-based sentiment analysis using fixed dictionaries without handling slang.

💡 Hint

Consider which method is designed for informal, short texts with emojis.

VADER is specifically designed for social media text and handles slang, emojis, and informal language better than traditional models.

Practice

1. What is the main purpose of the VADER lexicon-based approach in NLP?

easy

A. To generate new text based on input prompts

B. To translate text from one language to another

C. To detect named entities like people and places

D. To analyze the sentiment of text using a list of words with scores

Solution

1. Step 1: Understand VADER's function

   VADER uses a predefined list of words with sentiment scores to analyze feelings in text.

2. Step 2: Compare with other NLP tasks

   Translation, text generation, and entity detection are different tasks not done by VADER.

3. Final Answer:

   To analyze the sentiment of text using a list of words with scores -> Option D

4. Quick Check:

   VADER = sentiment analysis [OK]

Hint: VADER scores words to find text feelings fast [OK]

Common Mistakes:

• Confusing sentiment analysis with translation
• Thinking VADER generates text
• Mixing up sentiment with entity recognition

2. Which of the following is the correct way to import and initialize VADER's SentimentIntensityAnalyzer in Python?

easy

A. from vaderSentiment.vaderSentiment import SentimentIntensityAnalyzer
analyzer = SentimentIntensityAnalyzer()

B. import vader
analyzer = vader.SentimentIntensityAnalyzer()

C. from vaderSentiment import SentimentAnalyzer
analyzer = SentimentAnalyzer()

D. import SentimentIntensityAnalyzer from vaderSentiment
analyzer = SentimentIntensityAnalyzer()

Solution

1. Step 1: Recall correct import syntax

   The correct import is from vaderSentiment.vaderSentiment import SentimentIntensityAnalyzer.

2. Step 2: Check initialization

   Creating an instance is done by calling SentimentIntensityAnalyzer() with parentheses.

3. Final Answer:

   from vaderSentiment.vaderSentiment import SentimentIntensityAnalyzer\nanalyzer = SentimentIntensityAnalyzer() -> Option A

4. Quick Check:

   Correct import and init = from vaderSentiment.vaderSentiment import SentimentIntensityAnalyzer analyzer = SentimentIntensityAnalyzer() [OK]

Hint: Use full module path and parentheses to init [OK]

Common Mistakes:

• Using wrong module name or missing submodule
• Forgetting parentheses when creating analyzer
• Incorrect import syntax causing errors

3. Given the code below, what will be the output of print(scores)?

from vaderSentiment.vaderSentiment import SentimentIntensityAnalyzer
analyzer = SentimentIntensityAnalyzer()
scores = analyzer.polarity_scores('I love sunny days but hate the rain.')

medium

A. {'neg': 0.5, 'neu': 0.5, 'pos': 0.0, 'compound': -0.5}

B. {'neg': 0.25, 'neu': 0.5, 'pos': 0.25, 'compound': 0.34}

C. {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound': 0.0}

D. SyntaxError due to wrong method call

Solution

1. Step 1: Analyze the sentence sentiment

   The sentence has positive words ('love', 'sunny') and negative word ('hate'). VADER balances these.

2. Step 2: Understand VADER output format

   VADER returns a dict with 'neg', 'neu', 'pos', and 'compound' scores summing to 1 for neg, neu, pos.

3. Final Answer:

   {'neg': 0.25, 'neu': 0.5, 'pos': 0.25, 'compound': 0.34} -> Option B

4. Quick Check:

   Mixed sentiment sentence = balanced scores [OK]

Hint: Positive and negative words balance scores near 0.3-0.4 [OK]

Common Mistakes:

• Expecting all positive or all negative scores
• Confusing compound score with individual scores
• Thinking method call causes syntax error

4. Identify the error in the following code snippet using VADER and how to fix it:

from vaderSentiment.vaderSentiment import SentimentIntensityAnalyzer
analyzer = SentimentIntensityAnalyzer
scores = analyzer.polarity_scores('This is great!')

medium

A. String input must be a list; fix by wrapping text in []

B. Wrong import statement; fix by changing module name

C. Missing parentheses when creating analyzer instance; fix by adding ()

D. Method polarity_scores does not exist; fix by using analyze_scores

Solution

1. Step 1: Check how analyzer is created

   Analyzer is assigned the class itself, missing parentheses to create an instance.

2. Step 2: Fix by adding parentheses

   Change to SentimentIntensityAnalyzer() to create an object before calling polarity_scores.

3. Final Answer:

   Missing parentheses when creating analyzer instance; fix by adding () -> Option C

4. Quick Check:

   Instance creation needs () [OK]

Hint: Remember () to create object instances [OK]

Common Mistakes:

• Calling method on class, not instance
• Incorrect import causing attribute errors
• Passing wrong input types to polarity_scores

5. You want to analyze a batch of short tweets using VADER and classify each as positive if the compound score is above 0.05, negative if below -0.05, and neutral otherwise. Which code snippet correctly implements this?

hard

A. from vaderSentiment.vaderSentiment import SentimentIntensityAnalyzer
analyzer = SentimentIntensityAnalyzer()
tweets = ['Good job!', 'I hate this', 'It is okay.']
results = []
for tweet in tweets:
score = analyzer.polarity_scores(tweet)['compound']
if score > 0.05:
results.append('positive')
elif score < -0.05:
results.append('negative')
else:
results.append('neutral')
print(results)

B. from vaderSentiment.vaderSentiment import SentimentIntensityAnalyzer
analyzer = SentimentIntensityAnalyzer()
tweets = ['Good job!', 'I hate this', 'It is okay.']
results = []
for tweet in tweets:
score = analyzer.polarity_scores(tweet)['compound']
if score >= 0.05:
results.append('positive')
elif score <= -0.05:
results.append('negative')
else:
results.append('neutral')
print(results)

C. from vaderSentiment.vaderSentiment import SentimentIntensityAnalyzer
analyzer = SentimentIntensityAnalyzer()
tweets = ['Good job!', 'I hate this', 'It is okay.']
results = []
for tweet in tweets:
score = analyzer.polarity_scores(tweet)['compound']
if score > 0:
results.append('positive')
elif score < 0:
results.append('negative')
else:
results.append('neutral')
print(results)

D. from vaderSentiment.vaderSentiment import SentimentIntensityAnalyzer
analyzer = SentimentIntensityAnalyzer()
tweets = ['Good job!', 'I hate this', 'It is okay.']
results = []
for tweet in tweets:
score = analyzer.polarity_scores(tweet)['compound']
if score > 0.1:
results.append('positive')
elif score < -0.1:
results.append('negative')
else:
results.append('neutral')
print(results)

Solution

1. Step 1: Understand classification thresholds

   The problem states positive if compound > 0.05, negative if < -0.05, neutral otherwise.

2. Step 2: Check code conditions

   from vaderSentiment.vaderSentiment import SentimentIntensityAnalyzer analyzer = SentimentIntensityAnalyzer() tweets = ['Good job!', 'I hate this', 'It is okay.'] results = [] for tweet in tweets: score = analyzer.polarity_scores(tweet)['compound'] if score > 0.05: results.append('positive') elif score < -0.05: results.append('negative') else: results.append('neutral') print(results) uses > 0.05 and < -0.05 exactly, matching the problem statement.

3. Final Answer:

   Option A code correctly implements the classification thresholds -> Option A

4. Quick Check:

   Thresholds match problem = from vaderSentiment.vaderSentiment import SentimentIntensityAnalyzer analyzer = SentimentIntensityAnalyzer() tweets = ['Good job!', 'I hate this', 'It is okay.'] results = [] for tweet in tweets: score = analyzer.polarity_scores(tweet)['compound'] if score > 0.05: results.append('positive') elif score < -0.05: results.append('negative') else: results.append('neutral') print(results) [OK]

Hint: Match exact threshold signs for correct classification [OK]

Common Mistakes:

• Using >= or <= instead of > and <
• Changing threshold values incorrectly
• Misclassifying neutral scores