用python原子写入文件

将数据写入临时文件,成功写入数据后,将文件重命名为正确的目标文件,例如

f = open(tmpFile, 'w')
f.write(text)
# make sure that all data is on disk
# see http://stackoverflow.com/questions/7433057/is-rename-without-fsync-safe
f.flush()
os.fsync(f.fileno()) 
f.close()

os.rename(tmpFile, myFile)

根据文件 http://docs.python.org/library/os.html#os.rename

如果成功,重命名将是一个原子操作(这是一个 POSIX要求)。在Windows上,如果DST 已经存在,将引发OSError 即使是一个文件;可能没有 实现原子重命名的方法 DST命名现有文件

也

如果SRC和DST位于不同的文件系统上,则在某些Unix风格上的操作可能会失败。

注:

• 如果src和dest位置不在同一个文件系统上,则它可能不是原子操作。

• os.fsync 如果在电源故障、系统崩溃等情况下,性能/响应比数据完整性更重要,则可以跳过步骤。

使用python实现原子写入的简单代码段 tempfile .

with open_atomic('test.txt', 'w') as f:
    f.write("huzza")

或者读写同一个文件:

with open('test.txt', 'r') as src:
    with open_atomic('test.txt', 'w') as dst:
        for line in src:
            dst.write(line)

使用两个简单的上下文管理器

import os
import tempfile as tmp
from contextlib import contextmanager

@contextmanager
def tempfile(suffix='', dir=None):
    """ Context for temporary file.

    Will find a free temporary filename upon entering
    and will try to delete the file on leaving, even in case of an exception.

    Parameters
    ----------
    suffix : string
        optional file suffix
    dir : string
        optional directory to save temporary file in
    """

    tf = tmp.NamedTemporaryFile(delete=False, suffix=suffix, dir=dir)
    tf.file.close()
    try:
        yield tf.name
    finally:
        try:
            os.remove(tf.name)
        except OSError as e:
            if e.errno == 2:
                pass
            else:
                raise

@contextmanager
def open_atomic(filepath, *args, **kwargs):
    """ Open temporary file object that atomically moves to destination upon
    exiting.

    Allows reading and writing to and from the same filename.

    The file will not be moved to destination in case of an exception.

    Parameters
    ----------
    filepath : string
        the file path to be opened
    fsync : bool
        whether to force write the file to disk
    *args : mixed
        Any valid arguments for :code:`open`
    **kwargs : mixed
        Any valid keyword arguments for :code:`open`
    """
    fsync = kwargs.get('fsync', False)

    with tempfile(dir=os.path.dirname(os.path.abspath(filepath))) as tmppath:
        with open(tmppath, *args, **kwargs) as file:
            try:
                yield file
            finally:
                if fsync:
                    file.flush()
                    os.fsync(file.fileno())
        os.rename(tmppath, filepath)

有一个简单的原子文件帮助程序: https://github.com/sashka/atomicfile

i_m使用此代码自动替换/写入文件:

import os
from contextlib import contextmanager

@contextmanager
def atomic_write(filepath, binary=False, fsync=False):
    """ Writeable file object that atomically updates a file (using a temporary file).

    :param filepath: the file path to be opened
    :param binary: whether to open the file in a binary mode instead of textual
    :param fsync: whether to force write the file to disk
    """

    tmppath = filepath + '~'
    while os.path.isfile(tmppath):
        tmppath += '~'
    try:
        with open(tmppath, 'wb' if binary else 'w') as file:
            yield file
            if fsync:
                file.flush()
                os.fsync(file.fileno())
        os.rename(tmppath, filepath)
    finally:
        try:
            os.remove(tmppath)
        except (IOError, OSError):
            pass

用途:

with atomic_write('path/to/file') as f:
    f.write("allons-y!\n")

它基于 this recipe .

因为很容易弄乱细节,所以我建议使用一个小的库。图书馆的优势在于它能处理所有这些细节,并且 reviewed and improved 一个社区。

一个这样的图书馆是 python-atomicwrites 通过 收货人 它甚至具有适当的Windows支持:

• https://github.com/untitaker/python-atomicwrites

从自述文件:

from atomicwrites import atomic_write

with atomic_write('foo.txt', overwrite=True) as f:
    f.write('Hello world.')
    # "foo.txt" doesn't exist yet.

# Now it does.

用于Windows循环文件夹和重命名文件的原子解决方案。测试,原子自动化,您可以增加概率,以最大限度地减少不具有相同文件名的事件的风险。字母符号组合的随机库对数字str(random.random.range(509999999999,2))使用random.choice方法。您可以根据需要更改数字范围。

import os import random

path = "C:\\Users\\ANTRAS\\Desktop\\NUOTRAUKA\\"

def renamefiles():
    files = os.listdir(path)
    i = 1
    for file in files:
        os.rename(os.path.join(path, file), os.path.join(path, 
                  random.choice('ABCDEFGHIJKL') + str(i) + str(random.randrange(31,9999999,2)) + '.jpg'))
        i = i+1

for x in range(30):
    renamefiles()