发电机是螺纹安全的吗?

它不是线程安全的;同时调用可能会交错,并与局部变量混淆。

常用的方法是使用主从模式(在PC中现在称为农工模式)。创建第三个线程来生成数据,并在主线程和从线程之间添加一个队列,从线程将从该队列中读取数据,主线程将写入该队列。标准队列模块提供必要的线程安全性,并安排阻塞主队列,直到从队列准备好读取更多数据。

编辑以在下面添加基准。

你可以用锁把发电机包起来。例如,

import threading
class LockedIterator(object):
    def __init__(self, it):
        self.lock = threading.Lock()
        self.it = it.__iter__()

    def __iter__(self): return self

    def next(self):
        self.lock.acquire()
        try:
            return self.it.next()
        finally:
            self.lock.release()

gen = [x*2 for x in [1,2,3,4]]
g2 = LockedIterator(gen)
print list(g2)

锁定在我的系统上需要50毫秒,队列需要350毫秒。当您确实有队列时,队列很有用;例如,如果您有传入的HTTP请求,并且您希望将它们排队以供工作线程处理。(这不适用于python迭代器模型——一旦迭代器用完了项,它就完成了。)如果确实有迭代器,那么lockeditor是一种更快、更简单的方法来保证它的线程安全。

from datetime import datetime
import threading
num_worker_threads = 4

class LockedIterator(object):
    def __init__(self, it):
        self.lock = threading.Lock()
        self.it = it.__iter__()

    def __iter__(self): return self

    def next(self):
        self.lock.acquire()
        try:
            return self.it.next()
        finally:
            self.lock.release()

def test_locked(it):
    it = LockedIterator(it)
    def worker():
        try:
            for i in it:
                pass
        except Exception, e:
            print e
            raise

    threads = []
    for i in range(num_worker_threads):
        t = threading.Thread(target=worker)
        threads.append(t)
        t.start()

    for t in threads:
        t.join()

def test_queue(it):
    from Queue import Queue
    def worker():
        try:
            while True:
                item = q.get()
                q.task_done()
        except Exception, e:
            print e
            raise

    q = Queue()
    for i in range(num_worker_threads):
         t = threading.Thread(target=worker)
         t.setDaemon(True)
         t.start()

    t1 = datetime.now()

    for item in it:
        q.put(item)

    q.join()

start_time = datetime.now()
it = [x*2 for x in range(1,10000)]

test_locked(it)
#test_queue(it)
end_time = datetime.now()
took = end_time-start_time
print "took %.01f" % ((took.seconds + took.microseconds/1000000.0)*1000)

不,它们不是线程安全的。您可以在以下位置找到有关生成器和多线程的有趣信息:

http://www.dabeaz.com/generators/Generators.pdf

这取决于您使用的是哪种Python实现。在cpython中,gil对python对象执行所有操作都是线程安全的,因为在任何给定的时间只有一个线程可以执行代码。

http://en.wikipedia.org/wiki/Global_Interpreter_Lock