应用程序引擎上的故障保护数据存储更新

当然,应用引擎数据存储 downtime . 但是,我想要一个“故障保险箱” put 在数据存储错误面前,它更为强大(见下面的动机)。当数据存储不可用时,任务队列似乎是延迟写入的明显位置。不过,我不知道其他任何解决方案(除了通过urlfetch将数据发送给第三方之外)。

动机 :我有一个实体 真正地 需要放在数据存储中-只向用户显示一条错误消息是不行的。例如,可能发生了一些不容易消除的副作用(可能与第三方站点进行了一些交互)。

我想出了一个简单的包装器,它(我认为)提供了一个合理的“故障安全”放置(见下文)。您是否看到了这方面的任何问题,或者对更健壮的实现有了想法?(注:感谢尼克·约翰逊和萨克森·德鲁斯在回答中提出的建议,本文经过了编辑,对代码进行了一些改进。)

import logging
from google.appengine.api.labs.taskqueue import taskqueue
from google.appengine.datastore import entity_pb
from google.appengine.ext import db
from google.appengine.runtime.apiproxy_errors import CapabilityDisabledError

def put_failsafe(e, db_put_deadline=20, retry_countdown=60, queue_name='default'):
    """Tries to e.put().  On success, 1 is returned.  If this raises a db.Error
    or CapabilityDisabledError, then a task will be enqueued to try to put the
    entity (the task will execute after retry_countdown seconds) and 2 will be
    returned.  If the task cannot be enqueued, then 0 will be returned.  Thus a
    falsey value is only returned on complete failure.

    Note that since the taskqueue payloads are limited to 10kB, if the protobuf
    representing e is larger than 10kB then the put will be unable to be
    deferred to the taskqueue.

    If a put is deferred to the taskqueue, then it won't necessarily be
    completed as soon as the datastore is back up.  Thus it is possible that
    e.put() will occur *after* other, later puts when 1 is returned.

    Ensure e's model is imported in the code which defines the task which tries
    to re-put e (so that e can be deserialized).
    """
    try:
        e.put(rpc=db.create_rpc(deadline=db_put_deadline))
        return 1
    except (db.Error, CapabilityDisabledError), ex1:
        try:
            taskqueue.add(queue_name=queue_name,
                          countdown=retry_countdown,
                          url='/task/retry_put',
                          payload=db.model_to_protobuf(e).Encode())
            logging.info('failed to put to db now, but deferred put to the taskqueue e=%s ex=%s' % (e, ex1))
            return 2
        except (taskqueue.Error, CapabilityDisabledError), ex2:
            return 0

任务的请求处理程序:

from google.appengine.ext import db, webapp

# IMPORTANT: This task deserializes entity protobufs.  To ensure that this is
#            successful, you must import any db.Model that may need to be
#            deserialized here (otherwise this task may raise a KindError).

class RetryPut(webapp.RequestHandler):
    def post(self):
        e = db.model_from_protobuf(entity_pb.EntityProto(self.request.body))
        e.put() # failure will raise an exception => the task to be retried

我 不要 希望将其用于 每一个 输入-大多数情况下,显示错误消息是很好的。对每个Put都使用它是很有诱惑力的,但我认为有时如果我告诉用户更改将在稍后出现(并继续向他们显示旧数据,直到数据存储被备份为止),用户可能会更加困惑 和 延期执行)。