For Full-Text PDF, please login, if you are a member of IEICE,|
or go to Pay Per View on menu list, if you are a nonmember of IEICE.
Joint Chinese Word Segmentation and POS Tagging Using an Error-Driven Word-Character Hybrid Model
Canasai KRUENGKRAI Kiyotaka UCHIMOTO Jun'ichi KAZAMA Yiou WANG Kentaro TORISAWA Hitoshi ISAHARA
IEICE TRANSACTIONS on Information and Systems
Publication Date: 2009/12/01
Online ISSN: 1745-1361
Print ISSN: 0916-8532
Type of Manuscript: Special Section PAPER (Special Section on Natural Language Processing and its Applications)
Category: Morphological/Syntactic Analysis
word segmentation, POS tagging, error-driven, word-character hybrid model,
Full Text: PDF(337.7KB)>>
In this paper, we present a discriminative word-character hybrid model for joint Chinese word segmentation and POS tagging. Our word-character hybrid model offers high performance since it can handle both known and unknown words. We describe our strategies that yield good balance for learning the characteristics of known and unknown words and propose an error-driven policy that delivers such balance by acquiring examples of unknown words from particular errors in a training corpus. We describe an efficient framework for training our model based on the Margin Infused Relaxed Algorithm (MIRA), evaluate our approach on the Penn Chinese Treebank, and show that it achieves superior performance compared to the state-of-the-art approaches reported in the literature.