[PEAK] Generator pickling for workflow (was Re: A taxonomy of event sources: designing the events API)

[Phillip J. Eby]

At 04:45 PM 1/7/04 -0500, Bob Ippolito wrote:
>On Jan 7, 2004, at 10:13 AM, Phillip J. Eby wrote:
>
>>At 10:02 AM 1/7/04 -0500, Bob Ippolito wrote:
>>>Stackless Python pickles generators! :)
>>
>>...and IIRC, it does so by also pickling the code object involved, which 
>>doesn't provide any opportunity for workflow process evolution.
>>That is, if you change the code, existing instances are unaffected.
>>For certain circumstances that's useful, for others it's not what you 
>>want.  At least, it's not what I want as the primary mechanism for saving 
>>state.
>
>Well uh.. of course it has to pickle the code object.  It wouldn't work 
>otherwise, changing the code object would make the live generator invalid 
>in a whole bunch of different ways.

Right; but my workflow requirements call for workflow process evolution, as 
I stated above.  That means pickling generators isn't a solution for what I 
require.  So, it doesn't matter how well Stackless pickles them, it's not 
any good for my purposes.

Although, now that I think about it, I suppose in principle there could be 
ways to upgrade the pickled state so as to do the right 
thing.  Hmmm....  interesting.

In fact, as I think of it, it seems that it could even be done in pure 
Python...  imagine that you had 'save()' and 'restore()' functions that 
could be called inside the generator...  they could pull out the frame 
locals, line number, etc.  (In Python 2.3 and up, f_lineno is settable from 
Python code.)

Hm.  That's potentially a lot simpler than the approach I'd originally 
planned.  There'd still need to be a way to match up yield points between 
two versions of the code, but that could possibly be done by scanning the 
bytecode for the right information.  Very cool.  Thanks for the idea.  :)