Sparsely Encoded Associative Memory Model with Forgetting Process

Tomoyuki KIMOTO  Masato OKADA  

Publication
IEICE TRANSACTIONS on Information and Systems   Vol.E85-D   No.12   pp.1938-1945
Publication Date: 2002/12/01
Online ISSN: 
DOI: 
Print ISSN: 0916-8532
Type of Manuscript: PAPER
Category: Biocybernetics, Neurocomputing
Keyword: 
associative memory model,  sparse coding,  forgetting process,  SCSNA,  short-term memory,  

Full Text: PDF(464.1KB)>>
Buy this Article




Summary: 
In this paper, an associative memory model with a forgetting process proposed by Mezard et al. is investigated as a means of storing sparsely encoded patterns by the SCSNA proposed by Shiino and Fukai. Similar to the case of storing non-sparse (non-biased) patterns as analyzed by Mezard et al., this sparsely encoded associative memory model is also free from a catastrophic deterioration of the memory caused by memory pattern overloading. We theoretically obtain a relationship between the storage capacity and the forgetting rate, and find that there is an optimal forgetting rate leading to the maximum storage capacity. We call this the optimal storage capacity rate. As the memory pattern firing rate decreases, the optimal storage capacity increases and the optimal forgetting rate decreases. Furthermore, we shown that the capacity rate (i.e. the ratio of the storage capacity for the conventional correlation learning rule to the optimal storage capacity) is almost constant with respect to the memory pattern firing rate.