Relation Extraction with Deep Reinforcement Learning

Hongjun ZHANG  Yuntian FENG  Wenning HAO  Gang CHEN  Dawei JIN  

Publication
IEICE TRANSACTIONS on Information and Systems   Vol.E100-D   No.8   pp.1893-1902
Publication Date: 2017/08/01
Online ISSN: 1745-1361
Type of Manuscript: PAPER
Category: Natural Language Processing
Keyword: 
relation extraction,  deep reinforcement learning,  CNN,  Tree-LSTM,  

Full Text: PDF(1.3MB)
>>Buy this Article


Summary: 
In recent years, deep learning has been widely applied in relation extraction task. The method uses only word embeddings as network input, and can model relations between target named entity pairs. It equally deals with each relation mention, so it cannot effectively extract relations from the corpus with an enormous number of non-relations, which is the main reason why the performance of relation extraction is significantly lower than that of relation classification. This paper designs a deep reinforcement learning framework for relation extraction, which considers relation extraction task as a two-step decision-making game. The method models relation mentions with CNN and Tree-LSTM, which can calculate initial state and transition state for the game respectively. In addition, we can tackle the problem of unbalanced corpus by designing penalty function which can increase the penalties for first-step decision-making errors. Finally, we use Q-Learning algorithm with value function approximation to learn control policy π for the game. This paper sets up a series of experiments in ACE2005 corpus, which show that the deep reinforcement learning framework can achieve state-of-the-art performance in relation extraction task.