BLEU score evaluation in NLP - Cheat Sheet & Quick Revision

Recall & Review

beginner

What does BLEU stand for in NLP?

BLEU stands for Bilingual Evaluation Understudy. It is a metric to evaluate how close a machine-generated text is to a human reference text.

beginner

How does BLEU score measure similarity between texts?

BLEU measures similarity by counting matching n-grams (small word groups) between the machine output and reference text, then combines these counts into a score between 0 and 1.

intermediate

Why does BLEU use a brevity penalty?

BLEU uses a brevity penalty to avoid giving high scores to very short machine outputs that match parts of the reference but miss most content.

beginner

What is an n-gram in the context of BLEU score?

An n-gram is a sequence of n words. For example, a 2-gram (bigram) is two words in a row. BLEU compares n-grams from machine output and reference to check similarity.

beginner

What does a BLEU score of 1.0 mean?

A BLEU score of 1.0 means the machine-generated text exactly matches the reference text in terms of n-grams and length.

What does BLEU score primarily compare between machine output and reference?

A Sentence length only

B Grammar correctness

C Word frequency only

D Matching n-grams

BLEU score compares matching n-grams (word sequences) between machine output and reference to measure similarity.

Why is a brevity penalty used in BLEU score?

A To ignore punctuation

B To reward shorter sentences

C To penalize very short outputs that miss content

D To increase score for longer sentences

The brevity penalty prevents giving high scores to very short outputs that match only part of the reference.

Which of these is NOT a component of BLEU score calculation?

A N-gram precision

B Recall of words

C Brevity penalty

D Geometric mean of precisions

BLEU uses precision of n-grams, not recall, combined with brevity penalty and geometric mean.

If a machine translation has a BLEU score close to 0, what does it mean?

A It has almost no matching n-grams with reference

B It perfectly matches the reference

C It is too short

D It is grammatically correct

A BLEU score near 0 means very few or no matching n-grams with the reference text.

What is the typical range of BLEU scores?

A 0 to 1

B -1 to 1

C 1 to 10

D 0 to 100

BLEU scores range from 0 (no match) to 1 (perfect match).

Explain how BLEU score evaluates machine translation quality.

Think about how small word groups are compared and how length affects the score.

Describe why BLEU score might not fully capture translation quality.

Consider what BLEU measures and what it might miss about good translation.