For Full-Text PDF, please login, if you are a member of IEICE,|
or go to Pay Per View on menu list, if you are a nonmember of IEICE.
Contextualized Character Embedding with Multi-Sequence LSTM for Automatic Word Segmentation
Hyunyoung LEE Seungshik KANG
IEICE TRANSACTIONS on Information and Systems
Publication Date: 2020/11/01
Online ISSN: 1745-1361
Type of Manuscript: PAPER
Category: Natural Language Processing
contextualized character embedding, LSTM, linear-chain CRF, word segmentation,
Full Text: PDF(973.3KB)>>
Contextual information is a crucial factor in natural language processing tasks such as sequence labeling. Previous studies on contextualized embedding and word embedding have explored the context of word-level tokens in order to obtain useful features of languages. However, unlike it is the case in English, the fundamental task in East Asian languages is related to character-level tokens. In this paper, we propose a contextualized character embedding method using n-gram multi-sequences information with long short-term memory (LSTM). It is hypothesized that contextualized embeddings on multi-sequences in the task help each other deal with long-term contextual information such as the notion of spans and boundaries of segmentation. The analysis shows that the contextualized embedding of bigram character sequences encodes well the notion of spans and boundaries for word segmentation rather than that of unigram character sequences. We find out that the combination of contextualized embeddings from both unigram and bigram character sequences at output layer rather than the input layer of LSTMs improves the performance of word segmentation. The comparison showed that our proposed method outperforms the previous models.