本文介紹了我的Kafka流媒體應用程序退出,代碼為0,什么也不做的處理方法,對大家解決問題具有一定的參考價值,需要的朋友們下面隨著小編來一起學習吧!
問題描述
為了嘗試Kafka流,我這樣做了:
public static void main(String[] args) {
final StreamsBuilder builder = new StreamsBuilder();
final Properties streamsConfiguration = new Properties();
streamsConfiguration.put(StreamsConfig.APPLICATION_ID_CONFIG, "generic-avro-integration-test");
streamsConfiguration.put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG, Utils.BOOTSTRAP_SERVER);
streamsConfiguration.put(StreamsConfig.DEFAULT_KEY_SERDE_CLASS_CONFIG, StringDeserializer.class);
streamsConfiguration.put(StreamsConfig.DEFAULT_VALUE_SERDE_CLASS_CONFIG, KafkaAvroDeserializer.class);
streamsConfiguration.put(AbstractKafkaSchemaSerDeConfig.SCHEMA_REGISTRY_URL_CONFIG, Utils.SCHEMA_REGISTRY_URL);
builder.stream(Utils.ALL_FX_EVENTS_TOPIC).foreach((key, value) -> System.out.println(key));
KafkaStreams kafkaStreams = new KafkaStreams(builder.build(), streamsConfiguration);
kafkaStreams.start();
}
但當我在本地運行它時,我只得到以下結果:
基本上我從我的IDE運行它,1秒后它就停止了,而它應該正在等待推送到主題中的新事件。
我不明白。
Kafka主題在另一臺計算機上,但我也編寫了一個非常簡單的使用者的代碼,并且我能夠閱讀來自此遠程主題的消息。
出于某種原因,這個非常簡單的Kafka流應用程序退出,代碼為0。
我無能為力,你知道嗎?
由于該問題似乎與此處的slf4j依賴項有關,因此pom:
4.0.0
廣口瓶
<name>Ingestor :: Bigdata :: Ingestor</name>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>8</source>
<target>8</target>
</configuration>
</plugin>
</plugins>
</build>
<parent>
<groupId>com.parent-pom</groupId>
<artifactId>parent-pom</artifactId>
<version>4.0.2</version>
</parent>
<groupId>com</groupId>
<artifactId>ingestor</artifactId>
<version>1.0.1-SNAPSHOT</version>
<properties>
<sq.artifact.type>internal</sq.artifact.type>
<maven-compiler-plugin.version>3.1</maven-compiler-plugin.version>
<maven-assembly-plugin.version>3.3.0</maven-assembly-plugin.version>
<maven.compiler.source>7</maven.compiler.source>
<maven.compiler.target>7</maven.compiler.target>
<revision>1.0.0-SNAPSHOT</revision>
<sq.scs>fx-dan</sq.scs>
</properties>
<dependencies>
<!-- https://mvnrepository.com/artifact/org.apache.kafka/kafka-streams -->
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka-streams</artifactId>
<version>3.1.0</version>
</dependency>
<dependency>
<groupId>com</groupId>
<artifactId>libs-schemas</artifactId>
<version>1.0.6-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>io.confluent</groupId>
<artifactId>kafka-avro-serializer</artifactId>
<version>7.0.1</version>
</dependency>
<dependency>
<groupId>io.confluent</groupId>
<artifactId>kafka-schema-registry-client</artifactId>
<version>7.0.1</version>
</dependency>
<dependency>
<groupId>io.confluent</groupId>
<artifactId>common-config</artifactId>
<version>7.0.1</version>
</dependency>
<dependency>
<groupId>io.confluent</groupId>
<artifactId>common-utils</artifactId>
<version>7.0.1</version>
</dependency>
<dependency>
<groupId>io.confluent</groupId>
<artifactId>kafka-schema-serializer</artifactId>
<version>7.0.1</version>
</dependency>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>31.0.1-jre</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>1.7.30</version>
</dependency>
</dependencies>
更新:
除了缺少log4j屬性文件之外,錯誤還是由于缺少Serdes的配置。
更新后的代碼如下所示:
public static void main(String[] args) {
final StreamsBuilder builder = new StreamsBuilder();
final Properties streamsConfiguration = new Properties();
streamsConfiguration.put(StreamsConfig.APPLICATION_ID_CONFIG, "generic-avro-integration-test");
streamsConfiguration.put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG, Utils.BOOTSTRAP_SERVER);
streamsConfiguration.put(StreamsConfig.DEFAULT_KEY_SERDE_CLASS_CONFIG, Serdes.String().getClass().getName());
streamsConfiguration.put(StreamsConfig.DEFAULT_VALUE_SERDE_CLASS_CONFIG, SpecificAvroSerde.class);
streamsConfiguration.put(AbstractKafkaSchemaSerDeConfig.SCHEMA_REGISTRY_URL_CONFIG, Utils.SCHEMA_REGISTRY_URL);
final Serde<String> stringSerde = Serdes.String();
final Serde<AllTypesFxEvents> specificAvroSerde = new SpecificAvroSerde<>();
final boolean isKeySerde = false;
specificAvroSerde.configure(Collections.singletonMap(AbstractKafkaSchemaSerDeConfig.SCHEMA_REGISTRY_URL_CONFIG, Utils.SCHEMA_REGISTRY_URL),
isKeySerde);
builder.stream(Utils.ALL_FX_EVENTS_TOPIC).foreach((key, value) -> System.out.println(key));
KafkaStreams kafkaStreams = new KafkaStreams(builder.build(), streamsConfiguration);
kafkaStreams.cleanUp();
kafkaStreams.start();
我不再有異常堆棧,但此鏈接幫助我修復了它。
proper guide for java kafka stream with avro schema registry
推薦答案
我已修復問題,以下是摘要。
第一個問題是我沒有正確設置slf4j。Kafka希望slf4j可用,因為它使用slf4j來打印遇到的錯誤。沒有它,我的程序就會默默不及格。
若要解決此問題,您需要在Resources文件夾(在src文件夾中)中添加一個log4j.properties文件。
我的是這樣的(它不打印調試日志):
log4j.rootLogger=INFO, stdout
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%-4r [%t] %-5p %c %x - %m%n
一旦我有了日志,我就有了關于序列化錯誤的堆棧跟蹤。我不再有日志了,但另一篇讓我修復它的帖子是:
Use Kafka Streams with Avro Schema Registry
基本上,我對Serde配置使用了錯誤的對象。
更新后的代碼如下所示:
PUBLIC STATIC VID Main(字符串[]args){
final StreamsBuilder builder = new StreamsBuilder();
final Properties streamsConfiguration = new Properties();
streamsConfiguration.put(StreamsConfig.APPLICATION_ID_CONFIG, "generic-avro-integration-test");
streamsConfiguration.put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG, Utils.BOOTSTRAP_SERVER);
streamsConfiguration.put(StreamsConfig.DEFAULT_KEY_SERDE_CLASS_CONFIG, Serdes.String().getClass().getName());
streamsConfiguration.put(StreamsConfig.DEFAULT_VALUE_SERDE_CLASS_CONFIG, SpecificAvroSerde.class);
streamsConfiguration.put(AbstractKafkaSchemaSerDeConfig.SCHEMA_REGISTRY_URL_CONFIG, Utils.SCHEMA_REGISTRY_URL);
final Serde<String> stringSerde = Serdes.String();
final Serde<AllTypesFxEvents> specificAvroSerde = new SpecificAvroSerde<>();
final boolean isKeySerde = false;
specificAvroSerde.configure(Collections.singletonMap(AbstractKafkaSchemaSerDeConfig.SCHEMA_REGISTRY_URL_CONFIG, Utils.SCHEMA_REGISTRY_URL),
isKeySerde);
builder.stream(Utils.ALL_FX_EVENTS_TOPIC).foreach((key, value) -> System.out.println(key));
KafkaStreams kafkaStreams = new KafkaStreams(builder.build(), streamsConfiguration);
kafkaStreams.cleanUp();
kafkaStreams.start();
這篇關于我的Kafka流媒體應用程序退出,代碼為0,什么也不做的文章就介紹到這了,希望我們推薦的答案對大家有所幫助,