Engineering: Evaluation and error analysis

Canonical error categories

• Error rate - 37.6% train, 38.3% test
• Deletions - 24.0% train, 24.5% test
• Insertions - 13.6% train, 13.8% test
• Shows difference between canonical segments and RVLMs

Modified error categories

• Count deletions in VV sequences separately
• Count insertions by vowels, semivowels, consonants
• Allow skewed detections in semivowels
• Observations -
  • Skewed detection in semivowels account for 15% of error rate
  • Deletions in VV sequences account for 5% of error rate
  • Most consonant insertions happen in stops (burst) and fricatives
  • Most deletions are schwas (57%)

Skewed detections happen in consonants too

• Vowel is usually destressed or unvoiced schwa
• Consonant is usually stop or fricative
• Segments are usually quite short, boundary placement arbitrary

Extra Modification - skewed detections in consonants

• More than half of consonant insertions are skewed detections
• Consonant almost always precedes vowel