• ROGUE = Recall-Oriented Understudy for Gisting Evaluation
  • As ROGUE compare with the ALL target sentences, it is often compared with Recall
  • ROGUE-N score is the sum of Recall of N-grams
  • Heavily used in Text Summarization, Also usually used in Machine Translation with BLEU Score

[!def] ROGUE Score
\text{ROGUE-N} = \sum_{i=1}^N Recall_i
\text{Recall}_n = \frac{\text{# of n-grams matched both on generation and on reference}}{\text{# of n-grams in reference}}

Problems with ROGUE Score

  1. Doesn't consider semantic meaning
  2. Hard to compare with different tokenizers
  3. Doesn't consider synonyms