基于列值的Pyspark时间戳差异

clock_in , clock_out sign_in ,和 sign_out ). 我需要第三个纵队( duration

1. 如果 clock_in > clock_out duration = clock_in - clock_out
2. 否则,如果 sign_in > clock_out ,那么 duration = sign_out - clock_out
3. clock_in > sign_in ,那么
4. 否则, duration = 0

from pyspark.sql import *

def calc_dur(clock_in, clock_out, sign_in, sign_out):
    if clock_in > clock_out:
        return clock_in - clock_out
    elif sign_out > clock_out:
        return sign_out - clock_out
    elif sign_in < clock_in:
        return clock_in - sign_in
    else:
        return 0

dur_udf = udf(calc_dur)

df2 = df.withColumn("duration", dur_udf(df.clock_in, df.clock_out, df.sign_in, df.sign_out))

然而,这填补了我的空白 期间 带时间增量的列(例如。, Timedelta: 0 days, 1740 seconds, 0 microseconds (total: 1740 seconds) 期间 列,则返回零行。