[PEAK] bug in peak.util.imports

[Phillip J. Eby]

At 11:50 AM 10/21/04 +0300, Niki Spahiev wrote:
>Phillip J. Eby wrote:
>
>>At 06:10 PM 10/20/04 -0400, Bob Ippolito wrote:
>>
>>>It's valid to replace sys.modules with a new list object,
>>                                               ^^^^
>>Er, I hope not.  ;)
>>
>>>  so peak.util.imports should not do "from sys import modules".
>>
>>You're probably right.  I can't wait till I can get around to 
>>implementing a multi-interpreter API, so that I can actually have some 
>>unit tests for the 'imports' module.
>
>I would like to help with multi-interpreter API, but don't know how to 
>start (despite all info that you gave me already) :(
>
>Can we arrange some good use of my time?

Well, you could attempt to wrap the existing multi-interpreter C API with 
Pyrex, according to the interface I previously described.  Maybe write some 
unit tests.  In my email of 8/3, I drafted this:


     def running_interpreter():
         """Return an IInterpreter object for the interpreter currently in 
use"""

     def all_interpreters():
         """Return a list of IInterpreter objects representing all running 
interpreters"""


     class IInterpreter(Interface):

         parent = Attribute("""Parent interpreter or 'None'.  May be 
changed.""")

         sys = Attribute("""This interpreter's 'sys' module""")
         modules = Attribute("""This interpreter's 'sys.modules' dictionary""")
         builtins = Attribute("""This interpreter's '__builtin__' module""")

         def eval(codestring):
             """Eval 'codestring' in this interpreter and return result"""

         def call(callable, *__args, *__kw):
             """Run 'callable(*__args, *__kw)' in this interpreter"""

         def import_object(name):
             """Import 'name' in this interpreter context and return the 
result"""


You could probably just implement an Interpreter object type, and not 
bother with the 'all_interpreters' and 'running_interpreter' functions.

I also drafted a general approach in this email, regarding the use of 
Py_InterpreterState/Py_ThreadState structures:

http://mail.python.org/pipermail/python-dev/2004-August/046593.html

The trick is that call, eval, and import_object would all need to swap the 
interpreter state pointer in the current thread state, with a pointer to 
the interpreter structure for the target Intepreter object.  The 'sys', 
'modules', and 'builtins' attributes can be simple read-only properties 
that retrieve those things from the interpreter state that's wrapped.

I wouldn't expect this basic wrapping to take more than a few hours, to get 
something that's moderately usable.  I expect it to take more time to 
implement the 'running_interpreter()' and 'all_interpreters()' functions, 
since they will have to manage a list of active interpreter objects, and 
automatically create a dummy interpreter object to represent the Python 
root interpreter.  Creating an adequate set of tests for the whole thing 
would take a while too, as would implementing many of the import hooks I'd 
really like to see.

Anyway, if you don't get tot it, I'll probably take a look at this in 
November, after the 'peak.web' stuff I'm working on now is finished.