代码之家 › 专栏 › 技术社区 › jason m

使用期货。与类并发

concurrent.futures python

0

jason m · 技术社区 · 8 年前

我有一个main,如下所示:

import gather_filings_per_filer
import os
from concurrent import futures

def put_last_i(arg):
    with open(os.path.dirname(os.path.abspath(__file__)) + '/last_i.txt','w') as f:
            f.write(arg)

def get_last_i():
    with open(os.path.dirname(os.path.abspath(__file__)) + '/last_i.txt','r') as f:
            data = f.readlines()
        return data[0]

if __name__=='__main__':
    i = int(get_last_i())
    g_f_p_f = gather_filings_per_filer.gather_filings_per()
    jobs=[]


    with futures.ThreadPoolExecutor(max_workers = 3) as executor:
        my_d = dict((executor.submit(g_f_p_f.mt_main),j) for j in range(i,297085))

        for future in futures.as_completed(my_d):
            print future
            print future.exception()

和 g_f_p_f.mt_main 如下所示:

class gather_filings_per:
       def mt_main(self, i):
            print "Started:",i
            self.w_m = wrap_mysql.wrap_mysql()
            flag = True
            #do stuff...

给出以下结果:

<Future at 0x5dc2710 state=finished raised TypeError>
mt_main() takes exactly 2 arguments (1 given)
<Future at 0x5dc25b0 state=finished raised TypeError>
mt_main() takes exactly 2 arguments (1 given)

从我的角度来看 mt_main 仅接受1个参数(鉴于典型的 self 行为)。

我错过了什么?

1 回复 | 直到 8 年前

1

3

ShadowRanger 8 年前

你是对的,你只需要提供一个额外的参数 self 。但你一点都没给。所以你还差一个。拆分提交以使其视觉清晰:

my_d = dict(  # Making a dict
            # With key as result object from invoking `mt_main` with no additional arguments
            (executor.submit(g_f_p_f.mt_main),
            # And value j
             j)
            for j in range(i,297085))

也许你是想过去 j 作为论点?假设它也应该是该值,则为:

# Note second argument to submit, which becomes only non-self argument to mt_main
my_d = dict((executor.submit(g_f_p_f.mt_main, j),j) for j in range(i,297085))

或者简化一针一线,因为 concurrent.futures 应该意味着你可以使用 dict comprehensions (这也将 submit 从其价值调用 : 为了更好的视觉解析):

my_d = {executor.submit(g_f_p_f.mt_main, j): j for j in range(i, 297085)}