Highly Efficient Universal Coding with Classifying to Subdictionaries for Text Compression

Yasuhiko NAKANO  Hironori YAHAGI  Yoshiyuki OKADA  Shigeru YOSHIDA  

IEICE TRANSACTIONS on Fundamentals of Electronics, Communications and Computer Sciences   Vol.E77-A   No.9   pp.1520-1526
Publication Date: 1994/09/25
Online ISSN: 
Print ISSN: 0916-8508
Type of Manuscript: PAPER
Category: Algorithms, Data Structures and Computational Complexity
universal coding,  Lempel-Ziv algorithm,  dictionary coding,  CSD,  LZC,  

Full Text: PDF>>
Buy this Article

We developed a simple, practical, adaptive data compression algorithm of the LZ78 class. According to the Lempel-Ziv greedy parsing, a string boundary is not related to the statistical history modeled by finite-state sources. We have already reported an algorithm classifying data into subdictionaries (CSD), which uses multiple subdictionaries and conditions the current string by using the previous one to obtain a higher compression ratio. In this paper, we present a practical implementation of this method suitable for any kinds of data, and show that CSD is more efficient than the LZC which is the method used by the program compress available on UNIX systems. The CSD compression performance was about 10% better than that of LZC with the practical dictionary size, an 8k-entry dictionary when the test data was from the Calgary Compression Corpus. With hashing, the CSD processing speed became as fast as that of LZC, although the CSD algorithm was more complicated than LZC.