How to use Zope 3 generations - Random notes from mg

When you're developing a Zope 3 based application, and you want to make old object databases continue to work as you change object attributes, you should consider Zope 3 database generations.

The DatabaseGenerations proposal in the Zope 3 wiki describes the problem and possible solutions. README.txt for the zope.app.generations package tells you about the internals. The zope.app.zopeappgenerations package is a real-world example of generations. This mini-article strives to be a HOWTO.

This is a DRAFT version. I plan to show it to Zope 3 gurus, get feedback, and fix all bugs. After that, I will remove this notice.

Short summary

Here's how it works (slightly simplified): you tell Zope 3 the name of your application, and the current generation number. Zope 3 stores this number in the ZODB. Later, when you change your application, you will change the current generation number and write a function that can upgrade existing data. On every startup Zope 3 will check the current generation number with the number stored in the ZODB for your application, and, if necessary, call your upgrade functions.

Zeroth version

So, you have a Zope 3 based application that is being used. You have worked long and hard on new shining features, but before you release a newer version, you want to make sure existing users can upgrade without losing any data. You decide to use generations. How do you start?

Well, you should have declared that the old version of your application used generation 0 long ago. However all is not lost, you can rectify that omission without too much trouble. Add the following bit of ZCML into all Zope instances that run the old version of your application:

<utility
    name="myapp"
    provides="zope.app.generations.interfaces.ISchemaManager"
    factory="zope.app.generations.generations.SchemaManager"
    />

Now restart that Zope 3 and go to http://localhost:8080/++etc++process/@@generations.html. Make sure your application is shown there. Congratulations -- you have just told your Zope that the old version of your application used database generation 0. This is necessary if you want to write an upgrade script to your new application version, which will be generation 1.

First version

Create a new package, e.g. myapp.generations. This package will contain the current version number and database upgrade scripts. Create a __init__.py with the following contents:

from zope.app.generations.generations import SchemaManager

schemaManager = SchemaManager(
    minimum_generation=1,
    generation=1,
    package_name='myapp.generations')

Create a evolve1.py with the following contents:

from zope.app.zopeappgenerations import getRootFolder

def evolve(context):
    """Upgrade myapp database from generation 0 to generation 1.

    (describe database schema changes here)
    """
    root_folder = getRootFolder(context)
    # write code to perform database upgrades here
    # e.g.
    from zope.app.generations.utility import findObjectsProviding
    for myobj in findObjectsProviding(root_folder, IMyAppObject):
        myobj.newattribute = 42
    from zope.app.generations.utility import findObjectsMatching
    for myobj in findObjectsMatching(root_folder,
                                     lambda ob: isinstance(ob, MyCppClass)):
        myobj.newattribute = 42

Create a configure.zcml file with the following content:

<configure xmlns="http://namespaces.zope.org/zope">

  <utility
      name="myapp"
      provides="zope.app.generations.interfaces.ISchemaManager"
      component=".schemaManager"
      />

</configure>

Test the script with an old Data.fs file.

Now when you upgrade your application, Zope 3 will automatically run your evolve function once.

Further versions

Increment generation and minimum_generation in __init__.py, and write a new evolveNUM.py module as soon as you make a change to the database schema. Update the evolve function as you continue evelopment, but freeze it when you make a new release.