[PEAK] Messages to the future (in the Trellis)

[Phillip J. Eby]

Now that the high-level API is complete and I've started work on 
doc/test refactoring, I've started thinking about supporting mutable 
data structures such as lists, sets, and dictionaries.

In the curent Trellis model, you can't detect when such things have 
changed; you have to use custom objects (similar to ZODB's 
PersistentList and PersistentDict) that can track their own changes.

However, implementing such structures correctly is rather tricky, 
especially the mutation parts.

Within a given Trellis pulse, values aren't allowed to actually 
change, because then different rules might see different values.  So, 
when you implement a mutable data structure, it has to defer making 
any actual changes until the next trellis pulse.

In the API examples I've written previously, I've used a number of 
different hacks to handle this, including my recent "hub/spoke" examples.

But after a lot of thought, it seems to me that what we need is a 
simple way to manage "messages to the future".  For example, here's a 
simple Set implementation:

     class Set(trellis.Component):

         added = trellis.todo(lambda self: set())
         removed = trellis.todo(lambda self: set())

         to_add = added.future
         to_remove = removed.future

         @trellis.rule
         def data(self):
             data = self.data
             if data is None:
                 data = set()
             if self.removed:
                 for item in self.removed:
                     data.remove(item)
             if self.added:
                 for item in self.added:
                     data.add(item)
             return data

         def __iter__(self):
             return iter(self.data)

         def __len__(self):
             return len(self.data)

         def __contains__(self, item):
             return item in self.data

         def add(self, item):
             if item in self.to_remove:
                 self.to_remove.remove(item)
             else:
                 self.to_add.add(item)

         def remove(self, item):
             if item in self.to_add:
                 self.to_add.remove(item)
             else:
                 self.to_remove.add(item)

To work, it requires a new feature: "todo".  The idea is that any 
attribute marked "todo" is a data structure containing changes to be 
made to the object, and you can also use it to create a "future" 
version of the same attribute.  This "future" version is a read-only 
property that lets you access the *future* version of the "todo" attribute.

If there is no future version available yet, a new value is 
calculated using the todo attribute's rule.

Effectively, a 'todo' creates a 'receiver' cell whose default value 
is created using the supplied rule.  Accessing its future 
automatically assigns a fresh value to it, if needed.  And presto, we 
can now operate on our future events directly, by simply referring to 
their future versions.  And, since the todo's are receivers, they 
automatically reset to their empty value after everyone has seen the changes.

This makes it fairly simple to create mutable data structures, since 
one can simply design whatever change records are appropriate (such 
as added/removed in the case of a set), and have the manipulation 
methods manipulate this future state.  Rules for the current state 
can then apply the changes, so that any attempt to read the current 
state, brings it up to date with the planned changes.

At this point, I'm not 100% certain this is the way to go, but it 
looks quite promising.  I'll have to see how useful the approach is 
for other data types, though.  For dictionaries, one could do 
something like updated/removed, or added/changed/removed.  Lists are 
harder to describe changes to, however.  Perhaps the best thing would 
just be a list of co-ordinates and replacement slices, since all list 
manipulation can be described in terms of setslice operations.

One other thing that's needed, that 'todo' doesn't provide, is any 
way to tell that the object itself has "changed", so that if you use 
one of these structures in another cell, any dependent rules can be 
updated.  Overall, however, in such a case you should probably just 
refer directly to the change events, rather than treating the data 
structure as if the whole thing changed.