[PEAK] PEAK tasks and yield (Was: subscribable/conditional lists)

Phillip J. Eby pje at telecommunity.com
Wed Apr 21 10:36:59 EDT 2004


At 02:48 PM 4/21/04 +0100, Paul Moore wrote:
>"Phillip J. Eby" <pje at telecommunity.com> writes:
>
> > class Work(binding.Component):
> >
> >      queue = binding.Make(list)
> >      itemsQueued = binding.Make(events.Semaphore)
> >
> >      def worker(self):
> >          while True:
> >              yield self.itemsQueued; events.resume()
> >              item = self.queue.pop(0)
> >              self.itemsQueued.take()   # mine! nobody else can have it!
> >              doWork(item)
> >
> >      worker = binding.Make(events.taskFactory(worker),
> >      uponAssembly=True)
> >
> >      def addWork(self,data):
> >          self.queue.append(data)   # put it in the queue, then
> >          self.itemsQueued.put()    # let waiting tasks know it's there
> >
> > Which is just as simple as your version, but more explicit about where
> > task switching can or can't take place.
>
>This is something that has bugged me for a while about PEAK tasks. I
>can't make any sense of this - clearly the yield/resume is a
>conventional construct for doing co-operative multitasking, but it's
>completely obscure to me. The original code used the result of
>events.resume() where this code doesn't. I see nothing that uses the
>yielded value, to help me understand what I should yield.

The yielded value of a semaphore is its current count.  The code above 
doesn't need the count, since a semaphore yields only when it is nonzero, 
and the code takes a single item out of the queue.


>It probably doesn't help that I don't like co-operative multitasking
>as a model. I tend to go for event-driven asynchronous models,

Note that peak.events can be used in a pure callback-oriented mode: just 
don't use Tasks.  Of course, at that point you almost might as well use 
Twisted, except that peak.events has a more general and more uniform model 
for events, conditions, values, semaphores, etc.


>  or real
>threads, both of which feel more straightforward. But I don't see any
>support for these models in PEAK (or at least, not in a way that
>avoids the need for the yield/resume co-operative machinery).

As I mentioned above, just don't use tasks, use callbacks instead.  Any 
yield/resume pair can effectively be replaced by:

1. Split the function that contains them into a function for the activity 
before the yield/resume, and one to occur afterwards.

2. Replace 'yield x' with 'x.addCallback(f)' where f is the successor function.

3. Replace 'y=events.resume()' with 'def f(x,y):', and ensure that 'f' 
returns a true value (to indicate that it has "consumed" the event)

This is somewhat approximate, since really the 'def f(x,y):' block must 
occur *before* 'x.addCallback(f)', unless f is a method, in which case it's 
going to be 'x.addCallback(self.f)'.

Anyway, I prefer to look at events.Task objects as allowing me to replace 
this tedious assortment of callbacks with (relatively) linear code.  :)


>If there are any pointers to explanatory documentation, or better
>still, documentation on how to do "real" multithreading with PEAK,
>that would be a great help.

There is some mailing list chatter on that subject, but it may be from a 
few years ago.  Essentially, PEAK is intended for an application 
architecture where threads are independent and share no data at all, if 
it's at all possible.  This is one reason why the configuration 
architecture requires root components and has no true "global" 
configuration: it allows an entire self-contained application hierarchy to 
be passed to a thread for execution without any possibility of cross-thread 
interactions.

Currently, however, PEAK does not have any special support for threads, 
other than some limited locking on certain objects that *must* be global 
(such as interfaces and computed class attributes).  If you are developing 
a multi-threaded event-driven application, I currently recommend you use a 
Twisted reactor, as they are designed for multi-threaded use, and offer the 
'deferToThread()' API, for which there is no PEAK equivalent at 
present.  Someday there probably will be, and the default PEAK scheduler 
will then also support scheduling commands being issued from multiple 
threads.  But for now, Twisted has an actual implementation that works, AFAIK.




More information about the PEAK mailing list