代码之家 › 专栏 › 技术社区 › Nikolas Charalambidis

带有嵌入式Kafka的Spring Kafka测试找不到ZKClientConfig

spring-kafka apache-kafka spring-boot spring java

Nikolas Charalambidis · 技术社区 · 4 年前

我很难启动嵌入式Kafka进行集成测试。显示此错误:

原因:org.springframework.beans.factory。BeanCreationException:创建名为“embeddedKafka”的bean时出错:调用init方法失败;嵌套异常是java.lang.NoClassDefFoundError:org/apache/zookeeper/client/ZKClientConfig

这是我的最小样本。我需要使用 @ExtendWith(SpringExtension.class) 只要我需要自动连接更多的Spring beans。

@ExtendWith(SpringExtension.class)
@EmbeddedKafka(
        partitions = 1,
        bootstrapServersProperty = "spring.kafka.bootstrap-servers")
public class KafkaTest {

    @Test
    public void test(EmbeddedKafkaBroker broker) {
        var brokerList = broker.getBrokersAsString();
        log.debug("Embedded Kakfa topics {}", broker.getTopics());
    }
}

我使用:

Java 11
弹簧靴2.3
j单元5

我猜这里发生了依赖不一致的情况,然而,这很奇怪,因为我相信Spring Boot父级可以实现依赖兼容性。

<parent>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-parent</artifactId>
    <version>2.3.0.RELEASE</version>
    <relativePath/>
</parent>

当然,我在中列出了相关的依赖关系 pom.xml 在......下面

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.kafka</groupId>
    <artifactId>spring-kafka</artifactId>
</dependency>
<dependency>
    <groupId>io.confluent</groupId>
    <artifactId>kafka-avro-serializer</artifactId>
    <version>5.2.3</version>
</dependency>

<!-- TEST SCOPE -->
<dependency>
    <groupId>org.junit.jupiter</groupId>
    <artifactId>junit-jupiter-engine</artifactId>
    <version>5.6.2</version>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-test</artifactId>
    <scope>test</scope>
    <exclusions>
        <exclusion>
            <groupId>org.junit.vintage</groupId>
            <artifactId>junit-vintage-engine</artifactId>
        </exclusion>
    </exclusions>
</dependency>
<dependency>
    <groupId>org.springframework.kafka</groupId>
    <artifactId>spring-kafka-test</artifactId>
    <scope>test</scope>
</dependency>

我试图排除 org.apache.kafka:kafka-clients 两者都 org.springframework.kafka:spring-kafka-test 和 io.confluent:kafka-avro-serializer 运气不佳,同样的错误也发生了。

<exclusion>
    <groupId>org.apache.kafka</groupId>
    <artifactId>kafka-clients</artifactId>
</exclusion>

0 回复 | 直到 4 年前

Sooron 4 年前

我想在前面的回答中补充一些信息,因为我今天遇到了类似的问题。

主要问题是kafka客户端2.5.0依赖于Zookeeper 3.5,而不再是Zookeeper 3.4(参见 https://spring.io/projects/spring-kafka 相容性基质)

ZkClientConfig类已在Zookeeper 3.5.2中引入因此,任何低于此的Zookeeper版本都会引发此错误。出于测试目的,您需要删除3.5.2以下已经存在的任何zookeeper依赖关系,并将3.5.2后添加到您的测试依赖关系中。

Nikolas Charalambidis 4 年前

简短回答:

这些依赖关系存在冲突:

io.confluent:kafka-avro-serializer:5.2.3
org.springframework.kafka:spring-kafka-test:jar:2.5.0.RELEASE
…来自父母 org.springframework.boot:spring-boot-starter-parent:2.3.0.RELEASE

前者使用 org.apache.zookeeper:zookeeper:jar:3.4.13:compile 应排除以下情况:

<dependency>
    <groupId>io.confluent</groupId>
    <artifactId>kafka-avro-serializer</artifactId>
    <version>5.2.3</version>
    <exclusions>
        <exclusion>
            <groupId>org.apache.zookeeper</groupId>
            <artifactId>zookeeper</artifactId>
        </exclusion>
    </exclusions>
</dependency>

长答案:

检查 mvn dependency:tree 之前显示 kafka-avro-serializer 令人惊讶地包括 org.apache.zookeeper 依赖同时 spring-kafka-test 不:

提取用于 kafka-avro串行器

[INFO] +- io.confluent:kafka-avro-serializer:jar:5.2.3:compile  
[INFO] |  +- io.confluent:kafka-schema-registry-client:jar:5.2.3:compile    
[INFO] |  +- io.confluent:common-config:jar:5.2.3:compile   
[INFO] |  \- io.confluent:common-utils:jar:5.2.3:compile    
[INFO] |     +- org.apache.zookeeper:zookeeper:jar:3.4.13:compile   
[INFO] |     |  +- jline:jline:jar:0.9.94:compile   
[INFO] |     |  +- org.apache.yetus:audience-annotations:jar:0.5.0:compile  
[INFO] |     |  \- io.netty:netty:jar:3.10.6.Final:compile  
[INFO] |     \- com.101tec:zkclient:jar:0.10:compile

提取用于 春季卡夫卡试验

[INFO] \- org.springframework.kafka:spring-kafka-test:jar:2.5.0.RELEASE:test    
[INFO]    +- org.apache.kafka:kafka-clients:jar:test:2.5.0:test 
[INFO]    +- org.apache.kafka:kafka-streams:jar:2.5.0:test  
[INFO]    |  +- org.apache.kafka:connect-json:jar:2.5.0:test    
[INFO]    |  |  \- org.apache.kafka:connect-api:jar:2.5.0:test  
[INFO]    |  \- org.rocksdb:rocksdbjni:jar:5.18.3:test  
[INFO]    +- org.apache.kafka:kafka-streams-test-utils:jar:2.5.0:test   
[INFO]    +- org.apache.kafka:kafka_2.12:jar:2.5.0:test 
[INFO]    |  +- com.fasterxml.jackson.module:jackson-module-scala_2.12:jar:2.11.0:test  
[INFO]    |  |  \- com.fasterxml.jackson.module:jackson-module-paranamer:jar:2.11.0:test    
[INFO]    |  |     \- com.thoughtworks.paranamer:paranamer:jar:2.8:test 
[INFO]    |  +- com.fasterxml.jackson.dataformat:jackson-dataformat-csv:jar:2.11.0:test 
[INFO]    |  +- net.sf.jopt-simple:jopt-simple:jar:5.0.4:test   
[INFO]    |  +- com.yammer.metrics:metrics-core:jar:2.2.0:test  
[INFO]    |  +- org.scala-lang.modules:scala-collection-compat_2.12:jar:2.1.3:test  
[INFO]    |  +- org.scala-lang.modules:scala-java8-compat_2.12:jar:0.9.0:test   
[INFO]    |  +- org.scala-lang:scala-library:jar:2.12.10:test   
[INFO]    |  +- org.scala-lang:scala-reflect:jar:2.12.10:test   
[INFO]    |  +- com.typesafe.scala-logging:scala-logging_2.12:jar:3.9.2:test    
[INFO]    |  \- commons-cli:commons-cli:jar:1.4:test    
[INFO]    +- org.apache.kafka:kafka_2.12:jar:test:2.5.0:test    
[INFO]    \- org.junit.jupiter:junit-jupiter-api:jar:5.6.2:test 
[INFO]       +- org.apiguardian:apiguardian-api:jar:1.1.0:test  
[INFO]       +- org.opentest4j:opentest4j:jar:1.2.0:test    
[INFO]       \- org.junit.platform:junit-platform-commons:jar:1.6.2:test

这个 春季卡夫卡试验 不见了 org.apache.zookeeper:zookeeper 因此,测试无法连接到Zookeeper,并显示错误:

原因:org.springframework.beans.factory。BeanCreationException:创建名为“embeddedKafka”的bean时出错:调用init方法失败;嵌套异常是java.lang.NoClassDefFoundError:org/apache/zookeeper/client/ZKClientConfig

排除此类依赖 kafka-avro串行器 修复问题(请参阅 简短回答 ). The ouptut of mvn依赖关系:树 确认 org.apache.动物园管理员:动物园管理员 被适当地排除在外 kafka-avro串行器 并包含在 春季卡夫卡试验 .

提取用于 kafka-avro串行器 :

+- io.confluent:kafka-avro-serializer:jar:5.2.3:compile
[INFO] |  +- io.confluent:kafka-schema-registry-client:jar:5.2.3:compile
[INFO] |  +- io.confluent:common-config:jar:5.2.3:compile
[INFO] |  \- io.confluent:common-utils:jar:5.2.3:compile
[INFO] |     \- com.101tec:zkclient:jar:0.10:compile

提取用于 春季卡夫卡试验 :

\- org.springframework.kafka:spring-kafka-test:jar:2.5.0.RELEASE:test
[INFO]    +- org.apache.kafka:kafka-clients:jar:test:2.5.0:test
[INFO]    +- org.apache.kafka:kafka-streams:jar:2.5.0:test
[INFO]    |  +- org.apache.kafka:connect-json:jar:2.5.0:test
[INFO]    |  |  \- org.apache.kafka:connect-api:jar:2.5.0:test
[INFO]    |  \- org.rocksdb:rocksdbjni:jar:5.18.3:test
[INFO]    +- org.apache.kafka:kafka-streams-test-utils:jar:2.5.0:test
[INFO]    +- org.apache.kafka:kafka_2.12:jar:2.5.0:test
[INFO]    |  +- com.fasterxml.jackson.module:jackson-module-scala_2.12:jar:2.11.0:test
[INFO]    |  |  \- com.fasterxml.jackson.module:jackson-module-paranamer:jar:2.11.0:test
[INFO]    |  |     \- com.thoughtworks.paranamer:paranamer:jar:2.8:test
[INFO]    |  +- com.fasterxml.jackson.dataformat:jackson-dataformat-csv:jar:2.11.0:test
[INFO]    |  +- net.sf.jopt-simple:jopt-simple:jar:5.0.4:test
[INFO]    |  +- com.yammer.metrics:metrics-core:jar:2.2.0:test
[INFO]    |  +- org.scala-lang.modules:scala-collection-compat_2.12:jar:2.1.3:test
[INFO]    |  +- org.scala-lang.modules:scala-java8-compat_2.12:jar:0.9.0:test
[INFO]    |  +- org.scala-lang:scala-library:jar:2.12.10:test
[INFO]    |  +- org.scala-lang:scala-reflect:jar:2.12.10:test
[INFO]    |  +- com.typesafe.scala-logging:scala-logging_2.12:jar:3.9.2:test
[INFO]    |  +- org.apache.zookeeper:zookeeper:jar:3.5.7:test
[INFO]    |  |  +- org.apache.zookeeper:zookeeper-jute:jar:3.5.7:test
[INFO]    |  |  +- org.apache.yetus:audience-annotations:jar:0.5.0:test
[INFO]    |  |  +- io.netty:netty-handler:jar:4.1.49.Final:test
[INFO]    |  |  |  +- io.netty:netty-common:jar:4.1.49.Final:test
[INFO]    |  |  |  +- io.netty:netty-resolver:jar:4.1.49.Final:test
[INFO]    |  |  |  +- io.netty:netty-buffer:jar:4.1.49.Final:test
[INFO]    |  |  |  +- io.netty:netty-transport:jar:4.1.49.Final:test
[INFO]    |  |  |  \- io.netty:netty-codec:jar:4.1.49.Final:test
[INFO]    |  |  \- io.netty:netty-transport-native-epoll:jar:4.1.49.Final:test
[INFO]    |  |     \- io.netty:netty-transport-native-unix-common:jar:4.1.49.Final:test
[INFO]    |  \- commons-cli:commons-cli:jar:1.4:test
[INFO]    +- org.apache.kafka:kafka_2.12:jar:test:2.5.0:test
[INFO]    \- org.junit.jupiter:junit-jupiter-api:jar:5.6.2:test
[INFO]       +- org.apiguardian:apiguardian-api:jar:1.1.0:test
[INFO]       +- org.opentest4j:opentest4j:jar:1.2.0:test
[INFO]       \- org.junit.platform:junit-platform-commons:jar:1.6.2:test