我目前正在尝试基于Apache Beam SDK v2.1.0创建数据流模板,如
Google tutorial
这是我的主课
public static void main(String[] args) {
DispatcherOptions options = PipelineOptionsFactory.fromArgs(args).withValidation().as(DispatcherOptions.class);
Pipeline pipeline = Pipeline.create(options);
PCollection<PubsubMessage> messages = pipeline.apply("ReadMain", PubsubIO.readMessages().fromSubscription(options.getSubscription()));
}
如果我执行
mvn compile exec:java \
-Dexec.mainClass=com.example.myclass \
-Dexec.args="
命令如果我使用该方法,它会起作用
PubsubIO.readMessages().fromTopic(options.getTopic()));
PubsubIO.readMessages().fromSubscription(options.getSubscription()));
错误
[WARNING]
java.lang.RuntimeException: Not called from a runtime context.
at org.apache.beam.sdk.options.ValueProvider$RuntimeValueProvider.get(ValueProvider.java:223)
at org.apache.beam.sdk.options.ValueProvider$NestedValueProvider.get(ValueProvider.java:131)
at org.apache.beam.sdk.options.ValueProvider$NestedValueProvider.get(ValueProvider.java:131)
at org.apache.beam.sdk.io.gcp.pubsub.PubsubUnboundedSource.getSubscription(PubsubUnboundedSource.java:1374)
at org.apache.beam.sdk.io.gcp.pubsub.PubsubUnboundedSource$PubsubSource.<init>(PubsubUnboundedSource.java:1103)
at org.apache.beam.sdk.io.gcp.pubsub.PubsubUnboundedSource.expand(PubsubUnboundedSource.java:1407)
at org.apache.beam.sdk.io.gcp.pubsub.PubsubUnboundedSource.expand(PubsubUnboundedSource.java:110)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:514)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:454)
at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:44)
at org.apache.beam.sdk.io.gcp.pubsub.PubsubIO$Read.expand(PubsubIO.java:730)
at org.apache.beam.sdk.io.gcp.pubsub.PubsubIO$Read.expand(PubsubIO.java:536)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:514)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:473)
at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:56)
at org.apache.beam.sdk.Pipeline.apply(Pipeline.java:180)
at com.example.myclass.main(MyClass.java:43)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:282)
at java.lang.Thread.run(Thread.java:748)