An Attention-Based Hybrid Neural Network for Document Modeling

Dengchao HE  Hongjun ZHANG  Wenning HAO  Rui ZHANG  Huan HAO  

Publication
IEICE TRANSACTIONS on Information and Systems   Vol.E100-D   No.6   pp.1372-1375
Publication Date: 2017/06/01
Online ISSN: 1745-1361
DOI: 10.1587/transinf.2016EDL8231
Type of Manuscript: LETTER
Category: Artificial Intelligence, Data Mining
Keyword: 
document modeling,  Bi-LSTM,  attention-based model,  dynamic CNN,  

Full Text: PDF(422.6KB)
>>Buy this Article


Summary: 
The purpose of document modeling is to learn low-dimensional semantic representations of text accurately for Natural Language Processing tasks. In this paper, proposed is a novel attention-based hybrid neural network model, which would extract semantic features of text hierarchically. Concretely, our model adopts a bidirectional LSTM module with word-level attention to extract semantic information for each sentence in text and subsequently learns high level features via a dynamic convolution neural network module. Experimental results demonstrate that our proposed approach is effective and achieve better performance than conventional methods.