To:  mjd-perl-memoize
cc: mjd, jhi@iki.fi
Bcc: mjd-ok
Subject:  Memoize to be standard in Perl 5.7.2
Organization: Plover Systems

Memoize 0.65 is now available on my web site, and will appear on CPAN
worldwide over the next 48 hours.

What's new since 0.62:

Memoize 0.65 will be included with Perl 5.8.  This is great news!

However:

                            N O T I C E !

    ****************************************************************
    **                                                            **
    **   The TIE option is now strongly deprecated.  It will be   **
    **   permanently removed in the NEXT release of Memoize.      **
    **   Please convert all extant software to use HASH instead.  **
    **                                                            **
    **   See the manual for details.                              **
    **                                                            **
    ****************************************************************

I'm sorry about this.  I hate making incompatible changes.  But as of
v0.65, Memoize is included in the Perl core.  It is about to become
much more difficult to make incompatible interface changes; if I don't
get rid of TIE now, I may not get another chance.

TIE presented serious problems.  First, it had a bizarre syntax.  It
also was difficult for the user to do adequate error detection when
using it.  But the big problem was that it was difficult and
complicated for expiration manager authors to support; every
expiration manager had to duplicate the logic for handling TIE.  

The HASH option is much simpler to use, more powerful, and is trivial
for expiration managers to support.

The bottom of this message contains a detailed explanation and
description of these features.

Other changes in 0.65 are mostly small, internal changes to facilitate
inclusion into the Perl core package:

        Some cleanups and bug fixes. 

        Memoize now works under threaded perl

        Slow tests speeded up.  More test file improvements.

        Long-standing LIST_CACHE bug cleared up---it turns out that there
        never was a bug.  I put in tests for it anyway.

        Manual increased.


Explanation of the TIE -> HASH change and reasons for it.


* Brief summary of the change
*****************************

I am getting rid of the TIE option completely, in favor of the HASH
option, which does the same thing in a better way.

Starting in 0.65, the current release, TIE is STRINGLY DEPRECATED.
What does this means?  It means that it will be GONE from the NEXT
RELEASE of Memoize.  If you are using it, STOP.  I'm sorry this is
necessary, but a few of us will have to bite the bullet now so that
everyone else doesn't have to suffer forever.


* Detailed explanation of the change
************************************

At present, you can get a persistent cache saved on the disk by doing
something like this:

        memoize 'foo', 
          SCALAR_CACHE => ['TIE', DB_File, $filename, O_RDWR|O_CREAT, 0666];

The $filename argument and following arguments are used in a 'tie'
call that ties the cache hash to the DB_File package.

This works well enough in this simple case, but it has a problem when
you try to combine more than one cache manager. 

Suppose you want to have an LRU expiration regime:

        memoize 'foo', 
          SCALAR_CACHE => ['TIE', 'Memoize::ExpireLRU', CACHESIZE => 500 ];

Still no problem.  Now suppose that you want *both* features:  LRU
expiration and persistence via DB_File. You say:

        memoize 'foo', 
          SCALAR_CACHE => ['TIE', 'Memoize::ExpireLRU', CACHESIZE => 500,
                               TIE => [ 'DB_File', $filename, O_RDWR|O_CREAT, 0666],
                          ];

There are several problems with this:

* The nested TIEs are hard to understand and hard for the user to get
   right.  Note that the syntaxes of the TIE option for the two
   modules are subtly different.

*  Error recovery is difficult.  Suppose $filename cannot be created?
   Then Memoize::ExpireLRU croaks.  How do you recover from that?
   You'd need to use 'eval' to trap it.  But there's no way to know
   what you need to do without inspecting the code of both
   Memoize::ExpireLRU *and* Memoize.  Conceivably, you might need to
   use several nested evals!

*  Every cache manager module has to implement a TIE option.  The code
   for the TIE option is complicated and may be hard to get right.
   The code that implements TIE in Memoize and in Memoize::ExpireLRU
   is similar but not identical, and both are rather complicated.  I
   could provide an option-parsing module that cache managers could
   share, but it's better if the code is simpler to begin with.

The HASH option is a lot simpler to use and to implement.  For the
Memoize user, usage is straightforward: Take a hash variable, tie it
any way you want, and pass the tied variable to Memoize.  The first
example becomes:

      tie my %persistent => 'DB_File', $filename, O_RDWR|O_CREAT, 0666;
      memoize 'foo', HASH => \%persistent;

So now, instead of having to learn a weird new TIE option with syntax
similar to (but not the same as) the Perl 'tie' operator, the feature
is implemented in terms of things you already know:  Just pass a tied
hash to the 'memoize' function.

Cache managers also need to support HASH, when appropriate, so that
one can do:

      tie my %persistent => 'DB_File', $filename, O_RDWR|O_CREAT, 0666;

      tie my %expires => 'Memozie::ExpireLRU', CACHESIZE => 500, 
                                            HASH => \%persistent;

      memoize 'foo', HASH => \%expires;
    }

There are a lot more opportunities here for the end-user to dictate
error recovery policy.  Memoize::ExpireLRU will need to implement a
HASH option, but this is much simpler than implementing the TIE
option---it is essentially a one-line change:

        my %args = @_;
        my $cache = $args{HASH} || {};
        # now use $cache as before.

Contact me if you need assistance with this, or with converting your
applications to use HASH instead of TIE.