kafka源码编译

养马小卒
3,376 阅读2分钟

env prepare

java
gradle
scala

kafka source code download

build

tar -zxvf source-code.tar.tz -C <path-of-kafka>
cd <path-of-kafka>
gradle

遇到错误：

21:10:29.882 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]
21:10:29.882 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] FAILURE: Build failed with an exception.
21:10:29.882 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]
21:10:29.882 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] * Where:
21:10:29.882 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] Build file '/Users/sys/code/kafka/kafka-1.0.1/build.gradle' line: 540
21:10:29.882 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]
21:10:29.882 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] * What went wrong:
21:10:29.882 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] A problem occurred evaluating root project 'kafka-1.0.1'.
21:10:29.882 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] > Failed to apply plugin [id 'org.scoverage']
21:10:29.882 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]    > Could not create an instance of type org.scoverage.ScoverageExtension_Decorated.
21:10:29.882 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]       > You can't map a property that does not exist: propertyName=testClassesDir
21:10:29.882 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]
21:10:29.882 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] * Try:
21:10:29.882 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] Run with --stacktrace option to get the stack trace.  Run with --scan to get full insights.
21:10:29.882 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]
21:10:29.883 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] * Get more help at https://help.gradle.org
21:10:29.883 [WARN] [org.gradle.internal.featurelifecycle.LoggingDeprecatedFeatureHandler]
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.1.1/userguide/command_line_interface.html#sec:command_line_warnings
21:10:29.883 [ERROR] [org.gradle.internal.buildevents.BuildResultLogger]
21:10:29.883 [ERROR] [org.gradle.internal.buildevents.BuildResultLogger] BUILD FAILED in 1s

解决方法参考：KAFKA-7706

修改 build.gradle 文件

buildscript {
  repositories {
    mavenCentral()
    jcenter()
  }
  apply from: file('gradle/buildscript.gradle'), to: buildscript

  dependencies {
    // For Apache Rat plugin to ignore non-Git files
    classpath "org.ajoberstar:grgit:1.9.3"
    classpath 'com.github.ben-manes:gradle-versions-plugin:0.15.0'
    classpath 'org.scoverage:gradle-scoverage:2.5.0' ## 将2.1.0修改为2.5.0
    classpath 'com.github.jengelman.gradle.plugins:shadow:2.0.1'
  }
}

build a jar and run it

./gradlew jar      然后直接导入 idea / gradle项目即可
./gradlew idea/eclipse      可有可无

Add args

idea 中 RUN -> EDIT CONFIGURATION -> Program arguments :

config/server.properties

start zookeeper

start kafka in idea...

log error ：
/Library/Java/JavaVirtualMachines/jdk1.8.0_192.jdk/Contents/Home/bin/java "-javaagent:/Applications/IntelliJ IDEA.app/Contents/lib/idea_rt.jar=64234:/Applications/IntelliJ IDEA.app/Contents/bin" -Dfile.encoding=UTF-8 -classpath /Library/Java/JavaVirtualMachines/jdk1.8.0_192.jdk/Contents/Home/jre/lib/charsets.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_192.jdk/Contents/Home/jre/lib/deploy.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_192.jdk/Contents/Home/jre/lib/ext/cldrdata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_192.jdk/Contents/Home/jre/lib/ext/dnsns.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_192.jdk/Contents/Home/jre/lib/ext/jaccess.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_192.jdk/Contents/Home/jre/lib/ext/jfxrt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_192.jdk/Contents/Home/jre/lib/ext/localedata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_192.jdk/Contents/Home/jre/lib/ext/nashorn.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_192.jdk/Contents/Home/jre/lib/ext/sunec.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_192.jdk/Contents/Home/jre/lib/ext/sunjce_provider.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_192.jdk/Contents/Home/jre/lib/ext/sunpkcs11.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_192.jdk/Contents/Home/jre/lib/ext/zipfs.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_192.jdk/Contents/Home/jre/lib/javaws.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_192.jdk/Contents/Home/jre/lib/jce.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_192.jdk/Contents/Home/jre/lib/jfr.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_192.jdk/Contents/Home/jre/lib/jfxswt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_192.jdk/Contents/Home/jre/lib/jsse.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_192.jdk/Contents/Home/jre/lib/management-agent.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_192.jdk/Contents/Home/jre/lib/plugin.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_192.jdk/Contents/Home/jre/lib/resources.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_192.jdk/Contents/Home/jre/lib/rt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_192.jdk/Contents/Home/lib/ant-javafx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_192.jdk/Contents/Home/lib/dt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_192.jdk/Contents/Home/lib/javafx-mx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_192.jdk/Contents/Home/lib/jconsole.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_192.jdk/Contents/Home/lib/packager.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_192.jdk/Contents/Home/lib/sa-jdi.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_192.jdk/Contents/Home/lib/tools.jar:/Users/sys/code/kafka/kafka-1.0.1/core/out/production/classes:/Users/sys/code/kafka/kafka-1.0.1/core/out/production/resources:/Users/sys/code/kafka/kafka-1.0.1/clients/out/production/classes:/Users/sys/code/kafka/kafka-1.0.1/clients/out/production/resources:/Users/sys/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-databind/2.9.1/716da1830a2043f18882fc036ec26eb32cbe5aff/jackson-databind-2.9.1.jar:/Users/sys/.gradle/caches/modules-2/files-2.1/net.sf.jopt-simple/jopt-simple/5.0.4/4fdac2fbe92dfad86aa6e9301736f6b4342a3f5c/jopt-simple-5.0.4.jar:/Users/sys/.gradle/caches/modules-2/files-2.1/com.yammer.metrics/metrics-core/2.2.0/f82c035cfa786d3cbec362c38c22a5f5b1bc8724/metrics-core-2.2.0.jar:/Users/sys/.gradle/caches/modules-2/files-2.1/org.scala-lang/scala-library/2.11.12/bf5534e6fec3d665bd6419c952a929a8bdd4b591/scala-library-2.11.12.jar:/Users/sys/.gradle/caches/modules-2/files-2.1/com.101tec/zkclient/0.10/c54d4b5a5e89af75a80b6d5857400165ce5188d0/zkclient-0.10.jar:/Users/sys/.gradle/caches/modules-2/files-2.1/org.apache.zookeeper/zookeeper/3.4.10/8eebdbb7a9df83e02eaa42d0e5da0b57bf2e4da/zookeeper-3.4.10.jar:/Users/sys/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-log4j12/1.7.25/110cefe2df103412849d72ef7a67e4e91e4266b4/slf4j-log4j12-1.7.25.jar:/Users/sys/.gradle/caches/modules-2/files-2.1/org.lz4/lz4-java/1.4/9bedb74f461a87ff2161bdf0778ad8ca6bad3e1c/lz4-java-1.4.jar:/Users/sys/.gradle/caches/modules-2/files-2.1/org.xerial.snappy/snappy-java/1.1.4/d94ae6d7d27242eaa4b6c323f881edbb98e48da6/snappy-java-1.1.4.jar:/Users/sys/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-api/1.7.25/da76ca59f6a57ee3102f8f9bd9cee742973efa8a/slf4j-api-1.7.25.jar:/Users/sys/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-annotations/2.9.0/7c10d545325e3a6e72e06381afe469fd40eb701/jackson-annotations-2.9.0.jar:/Users/sys/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-core/2.9.1/60077fe98b11e4e7cf8af9b20609326a166d6ac4/jackson-core-2.9.1.jar:/Users/sys/.gradle/caches/modules-2/files-2.1/log4j/log4j/1.2.17/5af35056b4d257e4b64b9e8069c0746e8b08629f/log4j-1.2.17.jar kafka.Kafka config/server.properties
log4j:WARN No appenders could be found for logger (kafka.server.KafkaConfig).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

解决方法：添加 vm option

RUN -> EDIT CONFIGURATION -> Program arguments -> VM option :

-Dlog4j.configuration=file:config/log4j.properties.

可能还有权限问题：

log4j:ERROR setFile(null,true) call failed.
java.io.FileNotFoundException: /server.log (Permission denied)
	at java.io.FileOutputStream.open0(Native Method)
	at java.io.FileOutputStream.open(FileOutputStream.java:270)
	at java.io.FileOutputStream.<init>(FileOutputStream.java:213)
	at java.io.FileOutputStream.<init>(FileOutputStream.java:133)
	at org.apache.log4j.FileAppender.setFile(FileAppender.java:294)
	at org.apache.log4j.FileAppender.activateOptions(FileAppender.java:165)
	at org.apache.log4j.DailyRollingFileAppender.activateOptions(DailyRollingFileAppender.java:223)
	at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:307)
	at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:172)
	at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:104)
	at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:842)
	at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:768)
	at org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:648)
	at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:514)
	at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:580)
	at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:526)
	at org.apache.log4j.LogManager.<clinit>(LogManager.java:127)
	at org.slf4j.impl.Log4jLoggerFactory.<init>(Log4jLoggerFactory.java:66)
	at org.slf4j.impl.StaticLoggerBinder.<init>(StaticLoggerBinder.java:72)
	at org.slf4j.impl.StaticLoggerBinder.<clinit>(StaticLoggerBinder.java:45)
	at org.slf4j.LoggerFactory.bind(LoggerFactory.java:150)
	at org.slf4j.LoggerFactory.performInitialization(LoggerFactory.java:124)
	at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:412)
	at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:357)
	at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:383)
	at org.apache.kafka.common.utils.Utils.<clinit>(Utils.java:75)
	at kafka.Kafka$.getPropsFromArgs(Kafka.scala:44)
	at kafka.Kafka$.main(Kafka.scala:81)
	at kafka.Kafka.main(Kafka.scala)

解决方法：

在 log4j.properties 文件中添加文件路径（并赋权）。
kafka.logs.dir=/tmp/kafka_broker_logs

start producer && consumer.

producer：kafka-console-producer

consumer：kafka-console-consumer

avatar
文章
阅读
粉丝
相关推荐
聊聊 Kafka：编译 Kafka 源码并搭建源码环境
1.1k阅读
 · 
1点赞
Kafka 科普
38k阅读
 · 
340点赞
【kafka源码】kafka分区副本的分配规则
11k阅读
 · 
23点赞
Kafka生产者源码解析
3.6k阅读
 · 
6点赞
Kafka探秘
9.4k阅读
 · 
135点赞
Kafka源码见解
1.8k阅读
 · 
2点赞
go操作Kafka
14k阅读
 · 
47点赞
Kafka 常用命令总结
20k阅读
 · 
9点赞
KafkaProducer源码分析
2.2k阅读
 · 
5点赞
真的，Kafka 入门一篇文章就够了
86k阅读
 · 
683点赞

友情链接：