[TransWarp] peak.web Design Notes

Phillip J. Eby pje at telecommunity.com
Sun Jun 15 09:43:50 EDT 2003


(Comments and questions welcomed.)


Principles
==========

* Get 80% of the flexibility and power of Zope 3, for 20% or less of the 
complexity, by keeping to a bare minimum of required components and 
interfaces.  That's *not* the same thing as having a bunch of interfaces 
but providing default implementations!  Eliminate the need for registries 
by using adaptation wherever possible.

* Leverage code.  peak.web is for Python developers, not end users.  It's 
okay to have XML or ZConfig mechanisms that can replace coding later, but 
we can and should build with the assumption that application logic and 
navigation is made of code, not data.

* Separate presentation from navigation/security/logic.  Presentation 
components should be in a separate namespace from application components, 
and presentation components should not contain application code.  Ideally, 
HTML templates shouldn't even contain presentation code, only markup to 
indicate what code or data is to be applied where.

* Leverage "application-ness".  The developer is building a web 
*application*, so components should allow refinement of interfaces to ones 
that are application-specific.  For example, the developer should be able 
to use a custom "user" object with application-specific methods, or add 
their own collaborator objects like a "cart" or a "clipboard".

* Leverage Apache.  If Apache can serve it directly, let it.  We don't need 
to innovate in the static-file space.  Apache can handle ETags, last 
modified, persistent connections and a whole bunch of other stuff that can 
speed things up for the end user, that we don't want to have to copy.

* Leverage zope.publisher.  A lot of knowledge is embodied in the existing 
Request and Response classes; we don't want to have to duplicate this either.


Architecture Overview
=====================

Introduction
------------

(New terms marked with *asterisks*.)  An application's *publisher* 
interface creates a *publication* component for each web hit.  A 
publication has bindings for the *request*, *response*, *user*, *skin*, and 
any other application-specific bindings (e.g. cart, clipboard, etc.), and 
the root *navigator*.  A navigator is responsible for translating a URL 
path component (i.e. a name) to another navigator or to a 
*renderer*.  (This includes validating the user's permission to access that 
name.)  A renderer is responsible for writing output to the response, and 
may use *resources* accessed via the publication's skin.  Navigators and 
renderers are accessed by adaptation, using contextual interfaces supplied 
by the publisher (thus allowing publisher-specific adaptations to be 
registered).


Publisher
---------

The publisher is accessed by adapting a specified application object to an 
appropriate publisher interface, e.g. IHTTPPublisher.  The publisher must 
provide contextual interfaces (see "Big Example 2" in the PyProtocols 
reference manual) for at least IHTTPNavigator and IHTTPRenderer.  This is 
so that the application-specific navigators and renderings can be 
registered as adaptations from application classes, while still having 
fallback to more generally registered navigators and renderers.

The publishing interface should provide a way to look up a "skin" by name, 
a way to create a Publication from a zope.publisher Request object, and a 
way to access the original application root.


Publication
-----------

The Publication interface is an extension of 
zope.publisher.IHTTPPublication.  Zope's "publication" objects can be 
reused from one web hit to another, but ours are one-per-hit.  Anything 
that can live for more than one web hit should be in the publisher.  Of 
course, an application's publication class can define bindings that 
retrieve things from the publisher.

The Publication object is central to the architecture, in that it provides 
all the "glue" to make things work.  It contains the request, response, 
user, current skin, and any application-specific context or user-preference 
information needed by the application's navigators and renderers.  It 
provides these things as named bindings, e.g. "user" for the user object, 
"skin" for the skin, etc.  The application's policy for obtaining these 
objects is encoded in the definitions of the bindings.  So, to create a new 
authentication policy, I subclass a Publication class and change the 
bindings.  Or, I can change the Publisher class to create the Publication 
with a specified user object or skin or whatever.  The mechanism and policy 
are up to the developer.

This doesn't mean we may not end up with an IAuthService or something like 
that, with the 'user' Once binding using the IAuthService to look up the 
user.  It's just that we don't *need* all that extra machinery up front, to 
do simple things.  Changing to complicated things can always be done later, 
if we really need it.

It should be really easy to break up the bindings into lower-level bindings 
like "skinName", "userName" and the like, in order to do this.  I can 
envision having bindings like 'userName = 
bindTo("request/cookies/__username")' in the Publication class to do this 
kind of thing.  Maybe a 'userDM' binding that refers to a DM that the user 
name is looked up in to get the user object.  Skins wouldn't be that different.

Publication responsibilities also include setting up and finishing each web 
hit, and traversing to an object and invoking it.  So an application can 
add app-specific code to these things as well, if it needs to.  The default 
implementation will be to wrap the hit in a transaction, to traverse 
objects by adapting them to Navigators (using the Publisher's Navigator 
protocol), and invoke objects by adapting them to Renderers (using the 
Publisher's Renderer protocol).


Navigator
---------

A navigator is responsible for mapping names to objects.  While doing this, 
it has access to the Publication object, so it can check the user's 
permission to access a given name, or look up resources from the skin, 
etc.  Since navigators can be application-specific, this can take whatever 
form is desired.  Ideally, navigators would also have the ability to get 
metadata like a location title or URL, and the ability to iterate over a 
"menu" of choices that the user has permission for.  But these are not 
required for the core, and individual applications could define their own 
app-specific interfaces for this if needed.


Renderer
--------

A renderer is responsible for sending an HTTP response, by returning a 
value or using response.setBody(), streaming output, etc.  The typical 
renderer is some type of page template, but it may also be a wrapper on 
some object's method (or methods).  (For example, one might define an 
"action" renderer that accepted form data and then set various features of 
a persistent object.)  Renderers also get access to the Publication object, 
so that they can get access to Resources via the Skin.  Renderers may 
themselves be Resources from the Skin, that were found via a Navigator or 
registered as an adaptation for a particular application class.


Resource
--------

A resource is a presentation component, like a page template, image, or 
i18n message catalog.  Resources are accessed via a skin, so that they can 
be changed for different presentation contexts.  (E.g. WAP vs HTML, 
high/low bandwidth, "themes", etc.)



Skin
----

A skin is a namespace of presentation components.  In the Zope 3 
architecture a skin is a collection of "layers" that contain the actual 
components.  We will probably want to do something similar, but it should 
also be possible to define a skin by just using a class and putting in 
bindings for the resources, thus making it easy to initially create an 
"unskinned" application.  (It can always be upgraded later in a 
straightforward fashion.)  Skins can and should cache access to their 
contained resources; this is especially important since there is no per-hit 
resource caching.  (Although, if a resource is expected to be used multiple 
times in a hit, it can always be referenced as a binding on the 
Publication, e.g. 'someResource = bindTo("skin/somewhere/aResource")', and 
then navigators/renderers can refer to publication.someResource.)


Page Templates and Views
------------------------

A page template is a resource that represents a web page or portion 
thereof.  I think I would like us to use a modified version of Twisted's 
"Woven Web Widgets" page template approach.  Like ZPT, Twisted uses only 
attributes -- not special tags -- to mark up a page.  Unlike ZPT, however, 
Woven allows a more declarative approach.  Instead of coding conditionals 
and variable references and such, Woven has "model", "view", and "pattern" 
markers that contain only names.  "model" and "view name the subcomponents 
to be selected for data and presentation, respectively, and "pattern" 
identifies portions of markup to be supplied as parameters to the view 
component when rendering.

The result is impressive: you can create *structural* markup on an existing 
web page to identify patterns that exist in it.  By contrast (if I 
understand ZPT correctly), if you identify a loop on an existing page using 
ZPT, you have to mark all of its current contents as dummy elements.  For 
Woven, you just pick an entry as an example and tag it with a 
'pattern="listItem"' (and the loop's container with a 'view="list"') to do 
the same thing.

Because views are Python objects, you can define your own views to do any 
kind of widgety things you want.  You could make your own calendar widget, 
tree widget, menu bars, tab bars, breadcrumb viewer, etc.  And these 
widgets could take all of their appearance *by example* from the input 
page.  That means I could rough something out in text, render it via the 
program, then save the page from the program and fiddle with it in a 
WYSIWYG editor, and roundtrip it back with a fancier look to it.  But even 
if we don't or can't roundtrip a page, the process of inserting "model", 
"view" and "pattern" attributes isn't nearly as tedious as reconstructing a 
lot of actual code embedded in a page.

There are some downsides to Woven.  For one, I don't see any way to pull 
out and use its rendering engine, so we'll have to write our own.  (On the 
bright side, remember that it only has to interpret three attributes!)  I'm 
not sure how fast it can be made to run, although I suspect it might be 
capable of going faster than ZPT.  In fact, I can see a way to perhaps 
compile Woven-style templates into Python code, that would be run as long 
as a given view didn't need to manipulate the HTML being output.  That 
would be faster than ZPT, or indeed any templating engine that doesn't 
compile to Python or use some kind of C speedup mechanism.  Views of course 
will already be "compiled to Python" since they will be written in it by 
the view developer. :)



Open Issues and Next Steps
==========================

There's a lot of stuff to decide, and a lot of stuff to write, not 
necessarily in that order (since we may have to try some things to learn 
what we like).  One risk here is that we don't know how well contextual 
adaptation really works yet, nor have we established standards and 
practices for its use.

But many of the things we don't know, don't need to be addressed in the 
early stages.  The core interfaces are the Publisher, Publication, 
Navigator, and Renderer, and by themselves they are sufficient to create a 
web application.  We can solidify the basic interfaces for these without 
yet nailing down skins or page templates, or defining a permissions scheme, 
or creating static resource classes, naming conventions for resources, etc.

There is some conceptual overlap between our Navigator and Woven's 
"models", and between our Renderer and Woven's "view".  It's not clear to 
me yet whether we can make them 100% overlaps.  It'd be nice, though, 
because then page templates would be "secure", in the sense that a page 
template couldn't access any data that the navigator didn't permit it 
to.  This could be leveraged so that parts of a page just don't appear if 
they're not accessible.  Also, it would mean that anything that could be 
"navigated" would be browsable-to and potentially renderable.  We could 
also possibly use lookupComponent() with the values in the "model" and 
"view" attributes, which means that we wouldn't have to design Yet Another 
Interface for looking up submodels and subviews.

It seems to me that we should in fact adapt objects to navigators to do 
normal lookupComponent() path traversal, and optionally supply a "lookup 
context" or "security context" to them as well.  I've often found it 
annoying that 'bindTo' paths can't go through mapping objects like 
dictionaries.  But if the lookup process adapted such objects to 
navigators, they could go through them as well.  With a little bit of work, 
we could integrate security into this as well.  And if page templates had 
an "owner" that was used for the "view" lookups (the logged-in user would 
be used for the "model" lookups), then we could even support "untrusted" 
page template authors, that would be locked down with even more 
restrictions than ZPT can muster.

So it sounds like I need to review the current lookupComponent() mechanism 
with an eye to making it use adaptation to a navigation protocol, where the 
protocol has a means of providing security or other context information for 
the traversal.  This data (protocol and context) would be able to be 
supplied to lookupComponent(), and if not supplied should have reasonable 
defaults that assume they are being used from "trusted" code.  Adapters 
that don't know how to do security checking should deny access to anything 
if they are run in an "untrusted" mode, except for simple mappings or 
sequences, which can leave the checking to their contents.  Finally, 
getParentComponent() and its brethren need to be able to supply security 
context as well, so that access above a certain point can be blocked as 
well.  (This also allows "pseudo-roots" to exist, which could be useful 
when referring to models or views in a page template; the root view should 
be the skin, and the root model would be a context with standard names like 
"request", "response", "here", etc.)

At that point, we will have our navigation facility!  Publication objects 
will only need to use normal 'lookupComponent()' operations (with extra 
parameters) to securely traverse application components.  It would also be 
possible to use the application components as their own navigation 
components, for simpler applications.  (More complex apps, especially those 
that are reusing components from other apps, may want to keep navigation 
and security separate from the actual app components.)

One other issue that would need to be addressed is the use of URLs in view 
or model attributes; presumably they should be disallowed, or else we'll 
need to security-enable the naming system as well.  (Although that might 
not be a bad idea!)  If we security-enabled URLs, we could perhaps provide 
access to formulas or other interesting models or views via URLs.  But that 
seems awfully close to puting code in the presentation again.

Finally, to make all this work we'll need to actually have page templates 
and other resource classes, although naturally we'll be able to do a lot of 
initial development and testing without them.

Apart from adding page templates and resources, though, this is nearly all 
refactoring of existing code and interfaces, and sounds pretty doable for 
0.5 alpha 3.




More information about the PEAK mailing list