代码之家 › 专栏 › 技术社区 › powder366

无法连接并描述Kafka群集。检查工作进程的代理连接和安全属性

apache-kafka-connect apache-kafka

powder366 · 技术社区 · 5 年前

安装:

kafka_2.11-2.1.0 and zookeeper-3.4.13

运行:

bin/zookeeper-server-start.sh config/zookeeper.properties
bin/kafka-server-start.sh config/server.properties

运行连接器:

bin/connect-distributed.sh config/connect-distributed.properties

执行:

echo '{"name":"load-kafka-config", "config":{"connector.class":"FileStream-
    Source","file":"config/server.properties","topic":"kafka-config-topic"}}' |
    curl -X POST -d @- http://localhost:8083/connectors --header "content-
    Type:application/json"

curl: (7) Failed to connect to localhost port 8083: Connection refused

输出连接器:

    [2019-03-08 13:04:43,839] INFO Kafka version : 2.1.0 (org.apache.kafka.common.utils.AppInfoParser:109)
[2019-03-08 13:04:43,839] INFO Kafka commitId : 809be928f1ae004e (org.apache.kafka.common.utils.AppInfoParser:110)
[2019-03-08 13:06:43,854] ERROR Stopping due to error (org.apache.kafka.connect.cli.ConnectDistributed:117)
org.apache.kafka.connect.errors.ConnectException: Failed to connect to and describe Kafka cluster. Check worker's broker connection and security properties.
    at org.apache.kafka.connect.util.ConnectUtils.lookupKafkaClusterId(ConnectUtils.java:64)
    at org.apache.kafka.connect.util.ConnectUtils.lookupKafkaClusterId(ConnectUtils.java:45)
    at org.apache.kafka.connect.cli.ConnectDistributed.main(ConnectDistributed.java:78)
Caused by: java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment.
    at org.apache.kafka.common.internals.KafkaFutureImpl.wrapAndThrow(KafkaFutureImpl.java:45)
    at org.apache.kafka.common.internals.KafkaFutureImpl.access$000(KafkaFutureImpl.java:32)
    at org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:89)
    at org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:262)
    at org.apache.kafka.connect.util.ConnectUtils.lookupKafkaClusterId(ConnectUtils.java:58)
    ... 2 more
Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment.

如何配置违约为了避免卡夫卡下载错误,在分布式模式下启动连接器,同时让rest接口接受命令?

我正在使用连接的默认配置:

bootstrap.servers=localhost:9092

group.id=connect-cluster

key.converter=org.apache.kafka.connect.json.JsonConverter
value.converter=org.apache.kafka.connect.json.JsonConverter
key.converter.schemas.enable=true
value.converter.schemas.enable=true

offset.storage.topic=connect-offsets
offset.storage.replication.factor=1

config.storage.topic=connect-configs
config.storage.replication.factor=1

status.storage.topic=connect-status
status.storage.replication.factor=1

offset.flush.interval.ms=10000

所有文件: https://bitbucket.org/powder366/kafka_problem/src/master/

0 回复 | 直到 5 年前

powder366 5 年前

问题是默认配置中不包括localhost:

listeners=PLAINTEXT://localhost:9093

还采用了文件来使其按照(独立、分布式、多代理)工作:

演示: https://kafka.apache.org/quickstart

新文件: https://bitbucket.org/powder366/kafka_working/src/master/

推荐文章

Vaibhav Sharma · 使用Servlet连接KafkaProducer

7 年前

Evaldas Buinauskas · Kafka Connect将JSON字符串转换为实际JSON

7 年前

Nandish Kotadia · 我可以编写自定义kafka connect转换来将JSON转换为AVRO吗?

7 年前

Cody Ferguson · PubSub-Kafka连接节点连接文件结束异常

7 年前

Jaime Caffarel Daniel Koverman · kafka SourceTask类的poll()方法是否线程安全?

7 年前

Warsame · confluent-3.2.0 S3连接器默认分区设置

7 年前

Prajakta Kulkarni · 如何从flink访问/读取卡夫卡主题数据?

8 年前

Renukaradhya · Kafka Connect:在分布式模式下创建新连接器就是创建新组

8 年前

LaserJesus · 为什么Kafka分布式连接器在我创建它的节点被杀死时死亡?

8 年前

Rushabh Shah · 如何在Windows 7上使用Kafka Connect,因为我看不到独立连接或分布式连接。kafka 2.10.0.1中/bin/Windows文件夹下的bat?

8 年前