Translation of Untranslatable Words -- Integration of Lexical Approximation and Phrase-Table Extension Techniques into Statistical Machine Translation

Michael PAUL  Karunesh ARORA  Eiichiro SUMITA  

Publication
IEICE TRANSACTIONS on Information and Systems   Vol.E92-D   No.12   pp.2378-2385
Publication Date: 2009/12/01
Online ISSN: 1745-1361
DOI: 10.1587/transinf.E92.D.2378
Print ISSN: 0916-8532
Type of Manuscript: Special Section PAPER (Special Section on Natural Language Processing and its Applications)
Category: Machine Translation
Keyword: 
statistical machine translation,  out-of-vocabulary words,  lexical approximation,  phrase-table extension,  

Full Text: PDF(286.2KB)>>
Buy this Article




Summary: 
This paper proposes a method for handling out-of-vocabulary (OOV) words that cannot be translated using conventional phrase-based statistical machine translation (SMT) systems. For a given OOV word, lexical approximation techniques are utilized to identify spelling and inflectional word variants that occur in the training data. All OOV words in the source sentence are then replaced with appropriate word variants found in the training corpus, thus reducing the number of OOV words in the input. Moreover, in order to increase the coverage of such word translations, the SMT translation model is extended by adding new phrase translations for all source language words that do not have a single-word entry in the original phrase-table but only appear in the context of larger phrases. The effectiveness of the proposed methods is investigated for the translation of Hindi to English, Chinese, and Japanese.