Phrase-Based Statistical Model for Korean Morpheme Segmentation and POS Tagging

Seung-Hoon NA  Young-Kil KIM  

IEICE TRANSACTIONS on Information and Systems   Vol.E101-D   No.2   pp.512-522
Publication Date: 2018/02/01
Online ISSN: 1745-1361
DOI: 10.1587/transinf.2017EDP7085
Type of Manuscript: PAPER
Category: Natural Language Processing
phrase-based model,  segmentation,  tagging,  morphological analysis,  Korean morphological analysis,  

Full Text: PDF(373.5KB)
>>Buy this Article

In this paper, we propose a novel phrase-based model for Korean morphological analysis by considering a phrase as the basic processing unit, which generalizes all the other existing processing units. The impetus for using phrases this way is largely motivated by the success of phrase-based statistical machine translation (SMT), which convincingly shows that the larger the processing unit, the better the performance. Experimental results using the SEJONG dataset show that the proposed phrase-based models outperform the morpheme-based models used as baselines. In particular, when combined with the conditional random field (CRF) model, our model leads to statistically significant improvements over the state-of-the-art CRF method.