代码之家  ›  专栏  ›  技术社区  ›  Muhd Marcin

Python 2.7:进程池不工作(创建了inifinite worker)

  •  0
  • Muhd Marcin  · 技术社区  · 11 年前

    我的多进程池无法工作,并且运行异常:

    from multiprocessing import Pool
    import multiprocessing, logging
    logger = multiprocessing.log_to_stderr()
    logger.setLevel(multiprocessing.SUBDEBUG)
    
    
    def f(x):
        return x*x
    
    if __name__ == '__main__':
        pool = Pool(processes=2)
        print pool.map(f,range(10))
    

    记录器会打印出一大堆东西,比如:

    [DEBUG/MainProcess] created semlock with handle 420
    [DEBUG/MainProcess] created semlock with handle 432
    [DEBUG/MainProcess] added worker
    [DEBUG/MainProcess] added worker
    [DEBUG/MainProcess] cleaning up worker 1
    [DEBUG/MainProcess] cleaning up worker 0
    [DEBUG/MainProcess] added worker
    [DEBUG/MainProcess] added worker
    [DEBUG/MainProcess] cleaning up worker 1
    [DEBUG/MainProcess] cleaning up worker 0
    [DEBUG/MainProcess] added worker
    [DEBUG/MainProcess] added worker
    [DEBUG/MainProcess] cleaning up worker 1
    [DEBUG/MainProcess] cleaning up worker 0
    [DEBUG/MainProcess] added worker
    [DEBUG/MainProcess] added worker
    [DEBUG/MainProcess] cleaning up worker 1
    [DEBUG/MainProcess] cleaning up worker 0
    [DEBUG/MainProcess] added worker
    [DEBUG/MainProcess] added worker
    [DEBUG/MainProcess] cleaning up worker 1
    [DEBUG/MainProcess] cleaning up worker 0
    [DEBUG/MainProcess] added worker
    [DEBUG/MainProcess] added worker
    [DEBUG/MainProcess] cleaning up worker 1
    [DEBUG/MainProcess] cleaning up worker 0
    [DEBUG/MainProcess] added worker
    [DEBUG/MainProcess] added worker
    [DEBUG/MainProcess] cleaning up worker 1
    [DEBUG/MainProcess] cleaning up worker 0
    [DEBUG/MainProcess] added worker
    [DEBUG/MainProcess] added worker
    [DEBUG/MainProcess] cleaning up worker 1
    [DEBUG/MainProcess] cleaning up worker 0
    [DEBUG/MainProcess] added worker
    [DEBUG/MainProcess] added worker
    [DEBUG/MainProcess] cleaning up worker 1
    [DEBUG/MainProcess] cleaning up worker 0
    [DEBUG/MainProcess] added worker
    [DEBUG/MainProcess] added worker
    [DEBUG/MainProcess] cleaning up worker 1
    [DEBUG/MainProcess] cleaning up worker 0
    [DEBUG/MainProcess] added worker
    [DEBUG/MainProcess] added worker
    

    …无限期,从不打印结果。

    为什么不起作用?

    2 回复  |  直到 11 年前
        1
  •  3
  •   Muhd Marcin    11 年前

    该问题是由在windows上运行PyScripter(v2.5.3.0)中的代码引起的。从命令行运行很好。

        2
  •  1
  •   AKX Bryan Oakley    11 年前

    FWIW,你的代码在我的盒子上运行良好。(Windows 7、x64、Python 2.7.0):

    C:\Users\X\Desktop>python mp.py
    [DEBUG/MainProcess] created semlock with handle 384
    [DEBUG/MainProcess] created semlock with handle 396
    [DEBUG/MainProcess] added worker
    [DEBUG/MainProcess] added worker
    [DEBUG/PoolWorker-2] recreated blocker with handle 12
    [DEBUG/PoolWorker-2] recreated blocker with handle 12
    [DEBUG/PoolWorker-2] recreated blocker with handle 24
    [[DEBUG/PoolWorker-1] recreated blocker with handle 32
    DEBUG/PoolWorker-2] recreated blocker with handle 24
    [DEBUG/PoolWorker-1] recreated blocker with handle 32
    [[DEBUG/PoolWorker-1] recreated blocker with handle 48
    INFO/PoolWorker-2] child process calling self.run()
    [DEBUG/PoolWorker-1] recreated blocker with handle 48
    [INFO/PoolWorker-1] child process calling self.run()
    [[INFO/PoolWorker-1] child process calling self.run()
    INFO/PoolWorker-2] child process calling self.run()
    [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
    [INFO/MainProcess] process shutting down
    [DEBUG/MainProcess] running all "atexit" finalizers with priority >= 0
    [SUBDEBUG/MainProcess] calling <Finalize object, callback=_terminate_pool, args=(<Queue.Queue instance at 0x01FCCBE8>, <multiprocessing.queues.SimpleQueue object at 0x01F57890>, <multiprocessing.queues.SimpleQueue object
     at 0x01FAFBB0>, [<Process(PoolWorker-1, started daemon)>, <Process(PoolWorker-2, started daemon)>], <Thread(Thread-1, started daemon 84304)>, <Thread(Thread-2, started daemon 84900)>, <Thread(Thread-3, started daemon 42
    352)>, {}), exitprority=15>
    [SUBDEBUG/MainProcess] finalizer calling <bound method type._terminate_pool of <class 'multiprocessing.pool.Pool'>> with args (<Queue.Queue instance at 0x01FCCBE8>, <multiprocessing.queues.SimpleQueue object at 0x01F5789
    0>, <multiprocessing.queues.SimpleQueue object at 0x01FAFBB0>, [<Process(PoolWorker-1, started daemon)>, <Process(PoolWorker-2, started daemon)>], <Thread(Thread-1, started daemon 84304)>, <Thread(Thread-2, started daemo
    n 84900)>, <Thread(Thread-3, started daemon 42352)>, {}) and kwargs {}
    [DEBUG/MainProcess] finalizing pool
    [DEBUG/MainProcess] helping task handler/workers to finish
    [DEBUG/MainProcess] task handler got sentinel
    [DEBUG/MainProcess] removing tasks from inqueue until task handler finished
    [DEBUG/MainProcess] task handler sending sentinel to result handler
    [DEBUG/MainProcess] result handler got sentinel
    [DEBUG/MainProcess] task handler sending sentinel to workers
    [[DEBUG/MainProcess] ensuring that outqueue is not full
    DEBUG/PoolWorker-1] worker got sentinel -- exiting
    [[[DEBUG/PoolWorker-2] worker got sentinel -- exiting
    DEBUG/MainProcess] task handler exiting
    DEBUG/PoolWorker-1] worker got sentinel -- exiting
    [[DEBUG/PoolWorker-1] worker exiting after 5 tasks
    [[DEBUG/PoolWorker-1] worker exiting after 5 tasks
    DEBUG/PoolWorker-2] worker got sentinel -- exiting
    [INFO/PoolWorker-1] process shutting down
    DEBUG/MainProcess] terminating workers
    [DEBUG/MainProcess] result handler exiting: len(cache)=0, thread._state=2
    [DEBUG/MainProcess] worker handler exiting
    [DEBUG/MainProcess] joining task handler
    [DEBUG/MainProcess] joining result handler
    [DEBUG/MainProcess] joining pool workers
    [DEBUG/MainProcess] running the remaining "atexit" finalizers
    C:\Users\X\Desktop>