[PEAK] Connecting the Trellis to non-trellis systems

[Peter Damoc]

Hi Phillip,

Are there any GUI examples? wxPython examples would be ideal. :)

Thank you.

Peter.


On Tue, Mar 11, 2008 at 3:05 AM, Phillip J. Eby <pje at telecommunity.com>
wrote:

> There are two types of connections to non-trellis systems I'd like to
> talk about for a bit: connections to callback-based systems, and
> connections to "pure imperative" systems.  For example, connecting to
> a socket using Twisted or receiving wxPython GUI events might be an
> example of the first kind of system, and connecting to a database via
> SQLAlchemy or controlling Ecco via DDE would be examples of the second.
>
> The catch with the first kind of system is that in the Trellis, we
> don't like callbacks.  :)  More specifically, we don't like having to
> explicitly register them.  What we really want is to just do
> something like 'is_readable(socket)' in a rule, and have the
> callbacks automatically get set up -- and automatically disconnected
> when they're not needed.
>
> One crude way of doing this is already implemented in Trellis "Time"
> rules.  There's a weak dictionary that discards the timed event cells
> if they are no longer being referenced.  This is fine for Trellis
> time events, as they don't have any reference cycles and the Trellis
> ignores discarded events.
>
> But, for more complex applications, there can be undesirable
> consequences for leaving subscriptions in effect when they are not
> being used.  For example, consider the consequences of say, capturing
> the mouse in a particular window, and not releasing it!  (i.e.,
> imagine you could have a cell like 'captured_mouse_position', and
> capture the mouse as a side effect of depending on its value).
>
> (Also, when interacting with an external system, unless special
> precautions are taken, the callback from that system is going to have
> a reference to the target cell...  meaning it won't be able to be
> garbage collected, and thus won't be able to rely on getting
> automatically cleaned up.)
>
> So, we really need an easy way to make or break callback arrangements
> when a cell gains -- or loses -- subscribers.  Perhaps a specialized
> cell type that's easily customized (either per-instance or
> per-subclass) to do the right sort of callback-making and
> callback-breaking.
>
> The second kind of interfacing is a bit different.  Instead of
> wanting to receive input via callbacks from the outside world, we
> want to simply *wrap* an externally-supplied data value, passing both
> reads and writes through to the underlying system (maybe with some
> caching), but notifying other cells when the value is changed.
>
> Generally, these external systems may be costly to retrieve data
> from, so we probably want them to be "optional" attributes, i.e., not
> initialized until/unless you read their values.  Costly retrieval
> also implies that we may want to cache the read value, rather than
> passing through every read.
>
> In fact, even if retrieval isn't costly, we *still* need to do
> caching, because within a given trellis recalculation, a cell's value
> is supposed to be stable.  It can't just go changing on its own.
>
> If we're connecting to a system where values *can* change on the fly,
> but does not offer notification callbacks, then we may need some way
> to use a "time to live" (TTL) before the value is automatically
> refreshed.  And we can provide an explicit refresh operation as a
> @modifier.
>
> Having the cached value is also handy for writes; we can compare the
> cached value to the written value in order to decide whether to pass
> on the write to the underlying system.
>
> It seems like it might be possible to create a single cell type that
> handles all of these use cases.  There could be methods like:
>
> _poll() -> return the current value from the outside system
> _send(value) -> send the value to the outside system
> _receive(value) -> receive a value from a callback
>
> _subscribe() -> arrange for _receive() to be called on changes
> _unsubscribe() -> stop calling _receive()
>
> refresh() -> self._receive(self._poll())
>
> With the exception of _receive(), and refresh(), these methods would
> be specific to the type of thing being interfaced with.  When the
> cell is uninitialized -- or if its TTL has expired -- reading its
> value would do a refresh() first, before reading the cached value.
>
> Whenever the cell is written with a changed value, it would pass the
> value through to _send(), as soon as the current recalculation was
> completed.  And when listeners are added or removed from the cell, it
> would make sure to arrange for the appropriate _subscribe() or
> _unsubscribe() operation to occur when the current recalculation commits.
>
> In practice, it will probably be best to have two cell types: one for
> a read-only connections, and one for read-write connections.  For
> SQLAlchemy integration, we'll take the read-write version and set it
> up to talk through a delegated descriptor.  (Which will be an
> interesting sub-project in itself, I suspect.)
>
> TTL is also an interesting subproject.  If reading the cell value
> checks the TTL using the standard Time service, then any rule that
> reads the cell will also implicitly depend on the TTL and be
> refreshed when the TTL expires.  In some respects this is reasonable
> and perhaps even desirable, except that it will cause rules to be
> re-run even when the value hasn't changed.  Perhaps it would be
> better to make the TTL system a part of the subscribe/unsubscribe
> mechanism, such that refresh() is called when the TTL expires, if and
> only if there are listeners still looking for the value.
>
> Yes, that seems to make more sense.  In fact, if this cell type is a
> variation of the standard rule+value cell type, then it's even
> easier.  The rule will simply amount to something like:
>
>     if ttl_has_expired:
>         reset the ttl
>         return _poll()
>
> So that could work pretty decently, I think.  What's rather
> interesting about this is that it would make it *really* easy to
> interface to systems that have to be polled, such as inter-thread
> queues, filesystem directory contents, and so forth.  We could even
> have an API function like "sense(interval, func, *args)" so you could
> do something like::
>
>     if trellis.sense(10, os.path.isfile, some_file):
>        ...
>
> in a rule, so as to automatically detect when some_file is created,
> checking every 10 seconds.  (The function would similar to the Time
> service, i.e., by referring to a service that holds cells in a weak
> value dictionary, keyed by the interval, function, and function
> arguments.  Thus, the same cell would be reused by any/every rule
> that's polling for the same thing.)
>
> So, I'm thinking that the two cell types we need could be called a
> Sensor (read-only) and an Effector (read-write).  The only difference
> between the two would be that an Effector would be writable, and
> would need a _send() method.  (Well, and Effector would probably mix
> in different classes to get write behavior, but that's an
> implementation detail.)
>
> [insert **long** delay while I sketch lots of code for hours on end...]
>
> So, I think I've got this figured out now.  Sensor and Effector will
> be generic cell types, and the standard Cell() constructor will be
> enhanced to automatically create them if needed.  There will be a
> 'writer' keyword you can use to supply a 'write(value)' function, and
> if specified it will make your cell an Effector.
>
> To implement subscription-based rules, you'll be able to subclass a
> base called AbstractConnector, implementing read(), subscribe(cell)
> and unsubscribe(cell, key) methods.  Using such a rule will make your
> cell a Sensor (unless you also specify a writer, in which case you'll
> get an Effector).  You'll also be able to use
> Connector(read,sub,unsub) to create a connector from three functions,
> without needing to make a subclass.
>
> The net result, once I do the implementation and testing, should be
> that connecting to read-only outside data sources will require only
> making appropriate connectors, or using a polling factory to wrap the
> rule.  Connecting a writer to a data source will require only that
> the write function be known.
>
> There may also need to be some changes to the high-level API to allow
> specifying this sort of thing for rules in a class body.  More
> likely, however, connectors will get managed via services or explicit
> cell creation and manipulation.
>
> For example, it's likely that testing a socket's readability or
> writability will occur through an API, rather than by having a cell
> attribute tied directly to this.  These APIs will simply create the
> appropriate cell and read its value, caching the cell according to
> its creation parameters (e.g., by using an add-on, or a weak
> dictionary).  For example, a socket management service would probably
> cache such cells by fileno(), while for a wx event listener, there
> would probably be a cache of cells by event ID attached as an add-on
> to the target window or other object.  Querying for these events is
> then reduced to a dictionary lookup followed by a .value access in
> the common case.
>
> Whew!  I think that's more than enough for today.  :)
>
> _______________________________________________
> PEAK mailing list
> PEAK at eby-sarna.com
> http://www.eby-sarna.com/mailman/listinfo/peak
>



-- 
There is NO FATE, we are the creators.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.eby-sarna.com/pipermail/peak/attachments/20080311/4956cd37/attachment.html