[PEAK] Re: setuptools test command generates exception
after interpreter shutdown
pje at telecommunity.com
Tue May 4 14:46:34 EDT 2010
At 11:58 AM 5/4/2010 -0400, Nikolaus Rath wrote:
>On 05/04/2010 11:23 AM, P.J. Eby wrote:
> > Now, to narrow it down further, we'll have to remove stuff from the
> > setuptools code execution path to find out what part of setuptools is
> > relevant. I would start with removing code from the test command's
> > 'run()' method, especially the running of other commands such as
> > egg_info, and the saving and restoring of sys.path and sys.modules. My
> > guess at this point is that it may be the sys.modules restoration that
> > triggers it.
> > Actually, there's a quick way to test that: add 'import multiprocessing,
> > logging' to your setup.py (not the test code). If that makes it work,
> > then the sys.modules restoration is probably the culprit. If it still
> > doesn't work, then it's almost certainly a multiprocessing/logging bug.
>Good guess, adding the import to setup.py indeed fixes the problem. Are
>you able to do something about this? Or is that an intrinsic problem
>with the restoration?
Well, the original intention of the addition was to prevent problems
with importing older (installed) versions of a project, and only
secondarily to prevent the imported contents of the package from
lingering in sys.modules. I suppose it could be more selective in
clearing only the modules that were imported from the project
directory. However, even those could be doing atexit stuff or having
other methods run. Really, it looks like it's not safe to drop
references to module objects in the general case.
So, I think what's going to have to happen is that the sys.modules
save-and-restore will have to be skipped. I'm worried that it'll
introduce other problems, but I don't really see a way around it.
(FWIW, I think that the multiprocessing+logging combination causing
an atexit function to be registered is a hideous abomination, but
then I'm not fond of the logging module to begin with. Modifying
global state during module import is just plain evil.)
More information about the PEAK