为提高效率,请提供以下信息,问题描述清晰能够更快得到解决:
【TiDB 版本】
4.0.4
【问题描述】
binlog 往kfk写数据,报错,发现kafka内存溢出 能否调节binlog往kafka写的数据包大小
tidb drainer报错
-
- fail to produce message to kafka, please check the state of kafka server"] [error="kafka: Failed to produce message to topic tidb_binlog_kfk: kafka server: Unexpected (unknown?) server error.
producer/broker/1 state change to [closing] because write tcp 10.10.24.71:44176->10.138.61.131:9092: i/o timeout\ "]
[“Failed to close syncer”] [error=“fail to push msg to kafka after 30s, check if kafka is up and working”] [errorVerbose="fail to push msg to kafka after 30s, check if kafka is up and working\
kafka报错
2021-02-03 17:18:44,587] ERROR [ReplicaManager broker=1] Error processing append operation on partition tidb_binlog_kfk-0 (kafka.server.ReplicaManager)
java.lang.OutOfMemoryError: Direct buffer memory
at java.nio.Bits.reserveMemory(Bits.java:695)
at java.nio.DirectByteBuffer.(DirectByteBuffer.java:123)
at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311)
at sun.nio.ch.Util.getTemporaryDirectBuffer(Util.java:241)
at sun.nio.ch.IOUtil.write(IOUtil.java:58)
at sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:211)
at org.apache.kafka.common.record.MemoryRecords.writeFullyTo(MemoryRecords.java:90)
at org.apache.kafka.common.record.FileRecords.append(FileRecords.java:164)
at kafka.log.LogSegment.append(LogSegment.scala:157) at kafka.log.Log$$anonfun$append$2.apply(Log.scala:973)
at kafka.log.Log$$anonfun$append$2.apply(Log.scala:850) at kafka.log.Log.maybeHandleIOException(Log.scala:2085) at kafka.log.Log.append(Log.scala:850)
at kafka.log.Log.appendAsLeader(Log.scala:819) at kafka.cluster.Partition$$anonfun$14.apply(Partition.scala:771)
at kafka.cluster.Partition$$anonfun$14.apply(Partition.scala:759) at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:253)
at kafka.utils.CoreUtils$.inReadLock(CoreUtils.scala:259) at kafka.cluster.Partition.appendRecordsToLeader(Partition.scala:758) at kafka.server.ReplicaManager$$anonfun$appendToLocalLog$2.apply(ReplicaManager.scala:763)
at kafka.server.ReplicaManager$$anonfun$appendToLocalLog$2.apply(ReplicaManager.scala:751) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234) at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:130)
at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:130) at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:236)
若提问为性能优化、故障排查类问题,请下载脚本运行。终端输出的打印结果,请务必全选并复制粘贴上传。