代码之家 › 专栏 › 技术社区 › user3407267

Scala:使用try/catch重试以处理异常

user-defined-functions try-catch apache-spark scala json

user3407267 · 技术社区 · 7 年前

我试图为JSON转换添加一个重试逻辑。将对象转换为json时,如果有任何异常,我将重试3次。我正在做:

   var mapper = new ObjectMapper() with ScalaObjectMapper
  intializeMapper( )

  def intializeMapper() = {
    // jackson library does not support seralization and deserialization of
    // of scala classes like List and Map, this is needed to support it
    mapper.registerModule( DefaultScalaModule )
    // enables parsing of NaN. Enabling it here as JsonUtil class currently in
    // use supports it.
    mapper.configure(JsonParser.Feature.ALLOW_NON_NUMERIC_NUMBERS, true )
    mapper.setSerializationInclusion(Include.NON_NULL)
    mapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)

  } 


 def getPersonRDD(result: DataFrame): RDD[(String, String)] = {

    val finalValue = result.rdd.map({
      r =>
        val customerId = r.getAs[String](CUSTOMER_ID)
        val itemId = r.getAs[Map[String, Int]](ITEM_ID)
        val itemName = r.getAs[Map[String, Int]](ITEM_NAME)


        val person = Person(itemId, itemName)

        val jsonString = toJson(person)

        (customerId, jsonString)

    })

    return finalValue
  }

  def fromJson(json: String, clazz: Class[_]) = {
    mapper.readValue(json, clazz)
  }

  def toJson(value: Any): String = {

    var jsonString: String = " "
    jsonString = mapper.writeValueAsString(value)

    try {
      fromJson(jsonString, clazz)
      return jsonString
    } catch {
      case Exception => {

        publishMetrics(PARSING_EXCEPTION, 1.0)
        val result = util.Try(retry() {
          jsonString = mapper.writeValueAsString(value)
          val features = fromJson(jsonString, clazz)
        })

        result match {
          case util.Success(value) => jsonString

          case util.Failure(error) => {
            log.error("Error while parsing JSON " + jsonString)
            return jsonString
          }

        }

      }
    }

  }

  // Returning T, throwing the exception on failure
  @annotation.tailrec
  def retry[T](n: Int = 3)(fn: => T): T = {
    util.Try {
      fn
    } match {
      case util.Success(x) => x
      case _ if n > 1 => retry(n - 1)(fn)
      case util.Failure(e) => throw e
    }
  }

case class Person(itemId: Map[String, Int], itemName: Map[String, Int]) extends Serializable

这是正确的吗?我是Scala的新手。有没有人能建议我有没有更好的方法来实现这一点?Scala中是否有预定义的重试逻辑?我尝试为JSON转换添加重试逻辑的原因是由于我使用的Jackson版本(目前无法更改),有时我的writeValueAsString会导致JSON不完整。

1 回复 | 直到 7 年前

ouchxp 7 年前

您的重试功能似乎正确。我能想到的唯一缺陷是,如果您希望某些东西会失败,那么最好只使用返回类型 Try[T] ,因此您可以在外部以scala方式处理它。

以下是我的一个实现:

def retry[T](n: Int)(block: => T): Try[T] = {
  val stream = Stream.fill(n)(Try(block))
  stream find (_.isSuccess) getOrElse stream.head
}