drainer 同步 kafka 失败

【背景】

tidb 版本:v5.1.1
kafka 版本: kafka_2.12-2.3.0

【drainer配置】

drainer_servers:
- host: 192.168.x.86
  ssh_port: 22
  port: 8248
  deploy_dir: /home/tidb_test/tidb-deploy/drainer-8248
  data_dir: /home/tidb_test/tidb-data/drainer-8248
  commit_ts: -1
  config:
    syncer.db-type: kafka
    syncer.to.kafka-addrs: "192.168.x.60:9092"
    syncer.to.kafka-version: "2.3.0"
    syncer.to.topic-name: "starrocks"
    syncer.to.zookeeper-addrs: "192.168.x.60:2181"
  arch: amd64
  os: linux  

【现象】

开启 drainer 之后报错并退出,已 telnet 192.168.x.60 9092 确认过 kafka 端口是可以连接的,为什么drainer 的日志却是显示连接本地的?

[2021/12/17 14:42:42.250 +08:00] [INFO] [broker.go:168] ["[sarama] Failed to connect to broker localhost:9092: dial tcp 127.0.0.1:9092: connect: connection refused\
"]
[2021/12/17 14:42:42.250 +08:00] [INFO] [async_producer.go:971] ["[sarama] producer/broker/0 state change to [closing] because dial tcp 127.0.0.1:9092: connect: connection refused\
"]
[2021/12/17 14:42:42.250 +08:00] [INFO] [async_producer.go:578] ["[sarama] producer/leader/starrocks/0 state change to [retrying-17]\
"]
[2021/12/17 14:42:42.251 +08:00] [INFO] [async_producer.go:588] ["[sarama] producer/leader/starrocks/0 abandoning broker 0\
"]
[2021/12/17 14:42:42.251 +08:00] [INFO] [async_producer.go:717] ["[sarama] producer/broker/0 input chan closed\
"]
[2021/12/17 14:42:42.251 +08:00] [INFO] [async_producer.go:801] ["[sarama] producer/broker/0 shut down\
"]
[2021/12/17 14:42:42.751 +08:00] [INFO] [client.go:772] ["[sarama] client/metadata fetching metadata for [starrocks] from broker 192.168.x.60:9092\
"]
[2021/12/17 14:42:42.753 +08:00] [INFO] [async_producer.go:711] ["[sarama] producer/broker/0 starting up\
"]
[2021/12/17 14:42:42.753 +08:00] [INFO] [async_producer.go:727] ["[sarama] producer/broker/0 state change to [open] on starrocks/0\
"]
[2021/12/17 14:42:42.753 +08:00] [INFO] [async_producer.go:570] ["[sarama] producer/leader/starrocks/0 selected broker 0\
"]
[2021/12/17 14:42:42.753 +08:00] [INFO] [async_producer.go:594] ["[sarama] producer/leader/starrocks/0 state change to [flushing-17]\
"]
[2021/12/17 14:42:42.753 +08:00] [INFO] [async_producer.go:616] ["[sarama] producer/leader/starrocks/0 state change to [normal]\
"]

报错日志:
[sarama] Failed to connect to broker localhost:9092: dial tcp 127.0.0.1:9092: connect: connection refused

建议先不要配置到集群的组件当中,先用单个drainer跑一下试试,下面是配置文件

drainer --config tokafka.toml

detect-interval = 10
log-level = "info"
log-file = "tokf.log"
### 指定kafka checkpoint 存放目录
data-dir="/data/test_drainer"
initial-commit-ts = 420647292646457851

[syncer]
db-type = "kafka"
disable-dispatch = false
[syncer.to]
kafka-addrs = "172.16.188.140:9092,172.16.188.143:9092,172.16.188.113:9092"
kafka-max-messages = 1024000
kafka-version = "2.4.1"
topic-name = "mem-kafka"

下游 kafka 状态有问题吗,看看下游的 kafka 日志

此话题已在最后回复的 1 分钟后被自动关闭。不再允许新回复。