« Swifty 0.03 | Main | KeyedMutex - a mutex for web services »

Swifty 0.05 and the Thundering Herd

This is the release announcement of swifty 0.05.

Changes from previous version (0.03) are:

  • added code to support PowerPC and SPARC (not tested at all)
  • added internal checksum routine to guaranty integrity of cached data
  • added refresh_before property

About the refresh_before property

Normally, a cache entry has a single lifetime. The problem is that if a cache entry is read frequently and if it takes time to update the entry, a situation known as thundering herd would occur on expiration; i.e. many cache consumers will detect expiration and send same update requests to the backend, causing a performance decline. There are two solutions to the problem:

  • use a proxy that combines identical update requests as a single request
  • initiate an update request prior to expiration, when lifetime being left gets below a certain threshold

In version 0.05, as I did in Cache::Adaptive, I implemented the latter approach*1 to let applications start updating cache entries prior to their expiration. When the refresh_before property is set to nonzero, swifty would set the do_refresh flag when a cache entry is read for the first time after its left lifetime becomes below refresh_before.

The application logic can check if the flag is being set, and start updating the cache entry. Here's the example code.

my $swifty = Cache::Swifty->new({
    dir            => ...,
    lifetime       => 300, # cache lifetime is 5 minutes
    refresh_before => 10,  # one of the consumer will be notified when left lifetime<10
});

(snip)

$entry = $swifty->get(...);
if ($swifty->do_refresh) {
    # start updating the cache entry
}
*1: The best way to solve the problem depends on each application, but there is nothing a cache system can help when the former approach is being taken.

Post a comment

(If you haven't left a comment here before, you may need to be approved by the site owner before your comment will appear. Until then, it won't appear on the entry. Thanks for waiting.)