[TransWarp] FYI: 'api' packages namespace standardization

[Phillip J. Eby]

The Problem: Exports in __init__ Considered Harmful for Frameworks
==================================================================

People often want to use part of a package without importing all of 
it.  This is especially relevant to large frameworks such as Zope and 
PEAK.  However, people also don't want to have to import pieces from a 
zillion individual modules in order to get something done.

Often, packages are written whose __init__ imports items from contained 
modules or packages in order to 'export' them as a unit to external 
code.  But this can create problems for people who need access to only a 
part of the code, often including the framework developers themselves!  If 
two packages in a framework have dependencies on each other, and any of the 
interdependent modules are imported from an __init__ module, it becomes 
almost impossible to make the imports robust and non-circular.

Currently, PEAK has or will develop some of these problems.  The current 
arrangement is an inconsistent mix of export policies: some packages have 
an 'api' module or other submodule which does the "exporting" in place of 
__init__, while other packages do their exports directly from the __init__ 
module.

Implementing a consistent policy for how API functions are "exported" from 
packages should help to simplify the task of documenting, explaining, and 
understanding/using PEAK.  (Not to mention some of the tasks of writing it!)


The Solution
------------

Individual PEAK packages, such as peak.naming, peak.binding, peak.model, 
etc., will each include an 'api' module, which will contain or import the 
"commonly used" functions and classes for users of that package.  The 
__init__ module for each package will not import/export anything, and will 
generally contain only a docstring.  It is acceptable for the __init__ 
module to contain actual code or data, in the rare case that it is justifiable.

The root package, 'peak', will also have an 'api' module, which will 
contain a few very commonly used general PEAK API functions (such as 
'setupModule()') and a set of lazyImport objects for each of the top-level 
packages' API modules.  In other words, the following two imports produce 
essentially the same results:

from peak.api import subpackage

import peak.subpackage.api as subpackage

The only difference being that the 'subpackage' object imported from 
'peak.api' will be a lazyImport proxy for the actual 'peak.subpackage.api' 
module.  This is only relevant if you need the actual module object for 
some reason.

The standard way to access any PEAK functionality will be through the 
appropriate 'api' module or its lazyImport proxy from 'peak.api'.  Some 
code within PEAK itself, however, will necessarily break this rule.  For 
example: imports within the same package, and any cross-package imports 
which would otherwise create a circular import dependency.

Note, by the way, that "housekeeping" packages like peak.util and 
peak.tests will not have 'api' modules, since they do not have any kind of 
coherent API which they export to their clients.


The Implementation
------------------

Here's a general idea of the moves that will take place:

peak.binding.__init__  -> peak.binding.api
peak.util.Import       -> peak.binding.imports
peak.model.basic       -> peak.model.api
peak.naming.__init__   -> peak.naming.api (merge)

For right now, the peak.metamodels package won't get an API, since its 
status is rather fuzzy at the moment.  (Primarily, it's there to drive unit 
tests.)

Over time, I'd like to begin moving modules out of peak.util and into other 
suitable subpackages.  At this point, the Import module is the only one 
with an obvious place to go.  There are also a few peak.util modules which 
will disappear as soon as PEAK requires Python 2.2.1 (which will be as soon 
as I upgrade my Windows PC's at home and work to 2.2.1!).

The changes will take place as soon as I get to them, possibly right after 
I mail this announcement.  Some will be by direct repository moves, which 
may break current checkouts (Ty, take note!), although in practice it seems 
that moving or renaming files in the repository doesn't actually break a 
checkout.  Anyway, the moves will also require changes to source code, and 
those changes will therefore be checked in normally.