我有一个基本的脚本导入英国邮政编码到数据库与SQL炼金术。为了提高效率,我尝试使用AsyncIO来完成这个任务,下面是一些文档和一些“指南”博客文章。
下面的代码正在工作(没有抛出异常,正确地导入到数据库中),但是它看起来是同步的—文件和相应的行都是有序的,而我希望所有三个文件和行都是有序的。我不明白为什么。如何修复它,使给定CSV中每一行的导入不会阻止下一行的导入?
import csv
import os
import asyncio
from db.db import Session, engine, Base
from Models.PostCode import PostCode
from sqlalchemy.exc import IntegrityError
Base.metadata.create_all(engine)
session = Session()
csv_path = os.path.dirname(os.path.realpath(__file__)) + '/postcode_lists/'
def runImport(fname):
with open(csv_path + fname + '_postcodes.csv', newline='') as csvfile:
reader = csv.DictReader(csvfile)
tasks = [asyncio.ensure_future(saveRow(row)) for row in reader]
loop = asyncio.get_event_loop()
responses = loop.run_until_complete(asyncio.gather(*tasks, return_exceptions=True))
loop.close()
return responses
async def saveRow(row):
if ('In Use?' not in row) or (row['In Use?']=='Yes'):
await persist(row)
def persist(row):
EXISTS = len(session.query(PostCode).filter(PostCode.postcode == row['Postcode']).all())
if EXISTS == 0:
pc = PostCode(
row['Postcode'],
)
session.add(pc)
session.commit()
print(pc)
return pc
datasets = ['CM', 'CO']
for d in datasets:
runImport(d)
print(Done)
输出示例
<PostCode(postcode='CA7 5HU')>
<PostCode(postcode='CA7 5HW')>
<PostCode(postcode='CA7 5HX')>
<PostCode(postcode='CA7 5HY')>
<PostCode(postcode='CA7 5HZ')>
<PostCode(postcode='CA7 5JB')>
我期望有点混乱的输出,而不是按CSV排序的alpha。