Mining Approximate Primary Functional Dependency on Web Tables

Siyu CHEN  Ning WANG  Mengmeng ZHANG  

IEICE TRANSACTIONS on Information and Systems   Vol.E102-D   No.3   pp.650-654
Publication Date: 2019/03/01
Online ISSN: 1745-1361
DOI: 10.1587/transinf.2018EDL8130
Type of Manuscript: LETTER
Category: Artificial Intelligence, Data Mining
functional dependencies,  web table,  metrics,  pruning strategies,  

Full Text: PDF(575.6KB)>>
Buy this Article

We propose to discover approximate primary functional dependency (aPFD) for web tables, which focus on the determination relationship between primary attributes and non-primary attributes and are more helpful for entity column detection and topic discovery on web tables. Based on association rules and information theory, we propose metrics Conf and InfoGain to evaluate PFDs. By quantifying PFDs' strength and designing pruning strategies to eliminate false positives, our method could select minimal non-trivial approximate PFD effectively and are scalable to large tables. The comprehensive experimental results on real web datasets show that our method significantly outperforms previous work in both effectiveness and efficiency.