如果通过:paste粘贴代码,则Spark-captured变量为空

如果我执行这段代码

  val foo = "foo"

  def f(arg: Any): Unit = {
    Option(42).foreach(_ => java.util.Objects.requireNonNull(foo, "foo"))
  }

  sc.parallelize(Seq(1, 2), 2).foreach(f)

在里面 spark2-shell :paste 它抛出

scala> :paste
// Entering paste mode (ctrl-D to finish)

  val foo = "foo"

  def f(arg: Any): Unit = {
    Option(42).foreach(_ => java.util.Objects.requireNonNull(foo, "foo"))
  }

  sc.parallelize(Seq(1, 2), 2).foreach(f)

// Exiting paste mode, now interpreting.

19/03/11 15:02:06 WARN scheduler.TaskSetManager: Lost task 0.0 in stage 1.0 (TID 2, hadoop.company.com, executor 1): java.lang.NullPointerException: foo
        at java.util.Objects.requireNonNull(Objects.java:228)

但是,如果我执行它没有粘贴 :粘贴 或通过 spark2-shell -i 没有。

foo 里面 f 代码开始工作,即使通过粘贴 :粘贴

scala> :paste
// Entering paste mode (ctrl-D to finish)

val foo = "foo"

  def f(arg: Any): Unit = {
    println(foo)
    Option(42).foreach(_ => java.util.Objects.requireNonNull(foo, "foo"))
  }

  sc.parallelize(Seq(1, 2), 2).foreach(f)

// Exiting paste mode, now interpreting.

foo: String = foo
f: (arg: Any)Unit

scala>

发生什么事?