Relation Prediction in Multilingual Data Based on Multimodal Relational Topic Models

Yosuke SAKATA  Koji EGUCHI  

IEICE TRANSACTIONS on Information and Systems   Vol.E100-D   No.4   pp.741-749
Publication Date: 2017/04/01
Online ISSN: 1745-1361
Type of Manuscript: Special Section PAPER (Special Section on Data Engineering and Information Management)
latent topic models,  relational topic models,  multimodal data,  margin maximization,  

Full Text: PDF(751.3KB)
>>Buy this Article

There are increasing demands for improved analysis of multimodal data that consist of multiple representations, such as multilingual documents and text-annotated images. One promising approach for analyzing such multimodal data is latent topic models. In this paper, we propose conditionally independent generalized relational topic models (CI-gRTM) for predicting unknown relations across different multiple representations of multimodal data. We developed CI-gRTM as a multimodal extension of discriminative relational topic models called generalized relational topic models (gRTM). We demonstrated through experiments with multilingual documents that CI-gRTM can more effectively predict both multilingual representations and relations between two different language representations compared with several state-of-the-art baseline models that enable to predict either multilingual representations or unimodal relations.