TiDB版本：3.0.11
drainer 报错。但是有数据写入kafka中，不清楚这个报错影响了哪些。
[2020/08/04 21:01:50.214 +08:00] [WARN] [kafka.go:276] ["unknown mysql type"] [type=9]
辛苦上传先 drainer 的配置文件。
反馈下 tidb-binlog 版本，可以在 drainer 或者 pump log 中搜索 welcome 字样
[2020/08/04 17:23:05.545 +08:00] [INFO] [version.go:50] [“Welcome to Drainer”] [“Release Version”=v3.0.16] [“Git Commit Hash”=1d976dda375d86fc38549fc2ed841c63ab2b0973] [“Build TS”=“2020-07-03 11:15:12”] [“Go Version”=go1.13] [“Go OS/Arch”=linux/amd64]
[2020/08/04 17:23:05.546 +08:00] [INFO] [main.go:46] [“start drainer…”] [config=“{"log-level":"info","node-id":"","addr":"http://10.204.49.100:8249","advertise-addr":"http://10.204.49.100:8249","data-dir":"/data/deploy/DRAINER1/data.drainer","detect-interval":10,"pd-urls":"http://10.204.49.100:2379,http://10.204.49.101:2379,http://10.204.49.102:2379","log-file":"/data/deploy/DRAINER1/log/drainer.log","initial-commit-ts":418521297210310656,"sycner":{"sql-mode":null,"ignore-txn-commit-ts":,"ignore-schemas":"INFORMATION_SCHEMA,PERFORMANCE_SCHEMA,mysql","ignore-table":null,"txn-batch":20,"loopback-control":false,"sync-ddl":true,"channel-id":0,"worker-count":1,"to":{"host":"","user":"","password":"","security":{"ssl-ca":"","ssl-cert":"","ssl-key":"","cert-allowed-cn":null},"encrypted_password":"","sync-mode":0,"port":0,"checkpoint":{"type":"","schema":"","host":"","user":"","password":"","encrypted_password":"","port":0},"dir":"","zookeeper-addrs":"","kafka-addrs":"10.204.49.100:9092","kafka-version":"1.0.1","kafka-max-messages":1024,"kafka-client-id":"","topic-name":"uic_binlog"},"replicate-do-table":null,"replicate-do-db":["uic"],"db-type":"kafka","disable-dispatch":true,"relay":{"log-dir":"","max-file-size":10485760},"safe-mode":true,"disable-detect":false},"security":{"ssl-ca":"","ssl-cert":"","ssl-key":"","cert-allowed-cn":null},"synced-check-time":5,"compressor":"","EtcdTimeout":5000000000,"MetricsAddr":"","MetricsInterval":15}”]
大致是什么问题。目前发现kafka里有空的数据。
辛苦反馈下 draoner log 的完整内容 和 drainer.toml 的副本信息，
drainer.toml
# drainer Configuration.
# the interval time (in seconds) of detect pumps' status
detect-interval = 10
# Use the specified compressor algorithm to compress payload between pump and drainer
# compressor = "gzip"
# syncer Configuration.
[syncer]
# Assume the upstream sql-mode.
# If this is setted , drainer will use the sql-mode to parse DDL statment
# sql-mode = "STRICT_TRANS_TABLES,NO_ENGINE_SUBSTITUTION
# disable sync these schema
ignore-schemas = "INFORMATION_SCHEMA,PERFORMANCE_SCHEMA,mysql"
# number of binlog events in a transaction batch
txn-batch = 20
# work count to execute binlogs
# if the latency between drainer and downstream(mysql or tidb) are too high, you might want to increase this
# to get higher throughput by higher concurrent write to the downstream
worker-count = 16
# whether to disable the SQL feature of splitting a single binlog event.
# If it is set to "true", binlog events are restored to a single transaction for synchronization based on the order of binlogs.
# If the downstream service is MySQL, set it to "False".
disable-dispatch = false
# safe mode will split update to delete and insert
safe-mode = true
# downstream storage, equal to --dest-db-type
# valid values are "mysql", "file", "tidb", "flash", "kafka"
db-type = "kafka"
# ignore syncing the txn with specified commit ts to downstream
ignore-txn-commit-ts = []
# replicate-do-db priority over replicate-do-table if have same db name
# and we support regex expression , start with '~' declare use regex expression.
replicate-do-db = ["uic"]
# [[syncer.replicate-do-table]]
# db-name ="test"
# tbl-name = "log"
# [[syncer.replicate-do-table]]
# db-name ="test"
# tbl-name = "~^a.*"
# disable sync these table
# [[syncer.ignore-table]]
# db-name = "test"
# tbl-name = "log"
# the downstream mysql protocol database
#[syncer.to]
#host = "127.0.0.1"
#user = "root"
#password = ""
#port = 3306
# Uncomment this if you want to use file as db-type.
# [syncer.to]
# dir = "data.drainer"
# when db-type is kafka, you can uncomment this to config the down stream kafka, it will be the globle config kafka default
[syncer.to]
# only need config one of zookeeper-addrs and kafka-addrs, will get kafka address if zookeeper-addrs is configed.
# zookeeper-addrs = "127.0.0.1:2181"
kafka-addrs = "10.204.9.90:9092"
kafka-version = "1.0.1"
kafka-max-messages = 1024
# the topic name drainer will push msg, the default name is <cluster-id>_obinlog
# be careful don't use the same name if run multi drainer instances
topic-name = "uic_binlog"
log
[2020/08/05 10:58:39.737 +08:00] [INFO] [version.go:50] ["Welcome to Drainer"] ["Release Version"=v3.0.11] ["Git Commit Hash"=56ced1e197735eacae5179c7da3bd749bac55690] ["Build TS"="2020-03-04 12:04:10"] ["Go Version"=go1.13] ["Go OS/A
rch"=linux/amd64]
[2020/08/05 10:58:39.738 +08:00] [INFO] [main.go:46] ["start drainer..."] [config="{\"log-level\":\"info\",\"node-id\":\"\",\"addr\":\"http://10.204.9.90:8249\",\"advertise-addr\":\"http://10.204.9.90:8249\",\"data-dir\":\"/data33/dep
loy/DRAINER1/data.drainer\",\"detect-interval\":10,\"pd-urls\":\"http://10.204.9.111:2379,http://10.204.9.112:2379,http://10.204.9.113:2379\",\"log-file\":\"/data33/deploy/DRAINER1/log/drainer.log\",\"initial-commit-ts\":4185212972103
10656,\"sycner\":{\"sql-mode\":null,\"ignore-txn-commit-ts\":[],\"ignore-schemas\":\"INFORMATION_SCHEMA,PERFORMANCE_SCHEMA,mysql\",\"ignore-table\":null,\"txn-batch\":20,\"loopback-control\":false,\"sync-ddl\":true,\"channel-id\":0,\"
worker-count\":1,\"to\":{\"host\":\"\",\"user\":\"\",\"password\":\"\",\"encrypted_password\":\"\",\"sync-mode\":0,\"port\":0,\"checkpoint\":{\"type\":\"\",\"schema\":\"\",\"host\":\"\",\"user\":\"\",\"password\":\"\",\"encrypted_pass
word\":\"\",\"port\":0},\"dir\":\"\",\"zookeeper-addrs\":\"\",\"kafka-addrs\":\"10.204.9.90:9092\",\"kafka-version\":\"1.0.1\",\"kafka-max-messages\":1024,\"topic-name\":\"uic_binlog\"},\"replicate-do-table\":null,\"replicate-do-db\":
[\"uic\"],\"db-type\":\"kafka\",\"disable-dispatch\":true,\"relay\":{\"log-dir\":\"\",\"max-file-size\":10485760},\"safe-mode\":true,\"disable-detect\":false},\"security\":{\"ssl-ca\":\"\",\"ssl-cert\":\"\",\"ssl-key\":\"\"},\"synced-
check-time\":5,\"compressor\":\"\",\"EtcdTimeout\":5000000000,\"MetricsAddr\":\"\",\"MetricsInterval\":15}"]
[2020/08/05 10:58:39.739 +08:00] [INFO] [client.go:144] ["[pd] create pd client with endpoints"] [pd-address="[http://10.204.9.111:2379,http://10.204.9.112:2379,http://10.204.9.113:2379]"]
[2020/08/05 10:58:39.744 +08:00] [INFO] [client.go:252] ["[pd] switch leader"] [new-leader=http://10.204.9.112:2379] [old-leader=]
[2020/08/05 10:58:39.744 +08:00] [INFO] [client.go:163] ["[pd] init cluster id"] [cluster-id=6814443536625732093]
[2020/08/05 10:58:39.744 +08:00] [INFO] [server.go:115] ["get cluster id from pd"] [id=6814443536625732093]
[2020/08/05 10:58:39.747 +08:00] [INFO] [checkpoint.go:64] ["initialize checkpoint"] [type=file] [checkpoint=418522898324783241] [cfg="{\"CheckpointType\":\"file\",\"Db\":null,\"Schema\":\"\",\"Table\":\"\",\"ClusterID\":6814443536625
732093,\"InitialCommitTS\":418521297210310656,\"dir\":\"/data33/deploy/DRAINER1/data.drainer/savepoint\"}"]
[2020/08/05 10:58:39.747 +08:00] [INFO] [store.go:69] ["new store"] [path="tikv://10.204.9.111:2379,10.204.9.112:2379,10.204.9.113:2379?disableGC=true"]
[2020/08/05 10:58:39.747 +08:00] [INFO] [client.go:144] ["[pd] create pd client with endpoints"] [pd-address="[10.204.9.111:2379,10.204.9.112:2379,10.204.9.113:2379]"]
[2020/08/05 10:58:39.750 +08:00] [INFO] [client.go:252] ["[pd] switch leader"] [new-leader=http://10.204.9.112:2379] [old-leader=]
[2020/08/05 10:58:39.750 +08:00] [INFO] [client.go:163] ["[pd] init cluster id"] [cluster-id=6814443536625732093]
[2020/08/05 10:58:39.755 +08:00] [INFO] [store.go:75] ["new store with retry success"]
[2020/08/05 10:58:39.871 +08:00] [INFO] [client.go:127] ["[sarama] Initializing new client"]
[2020/08/05 10:58:39.871 +08:00] [INFO] [client.go:174] ["[sarama] Successfully initialized new client"]
[2020/08/05 10:58:39.872 +08:00] [INFO] [store.go:69] ["new store"] [path="tikv://10.204.9.111:2379,10.204.9.112:2379,10.204.9.113:2379?disableGC=true"]
[2020/08/05 10:58:39.872 +08:00] [INFO] [client.go:144] ["[pd] create pd client with endpoints"] [pd-address="[10.204.9.111:2379,10.204.9.112:2379,10.204.9.113:2379]"]
[2020/08/05 10:58:39.874 +08:00] [INFO] [client.go:252] ["[pd] switch leader"] [new-leader=http://10.204.9.112:2379] [old-leader=]
[2020/08/05 10:58:39.874 +08:00] [INFO] [client.go:163] ["[pd] init cluster id"] [cluster-id=6814443536625732093]
[2020/08/05 10:58:39.876 +08:00] [INFO] [store.go:75] ["new store with retry success"]
[2020/08/05 10:58:39.883 +08:00] [INFO] [server.go:259] ["register success"] ["drainer node id"=dcn-tidb-cluster1-p-l-07:8249]
[2020/08/05 10:58:39.884 +08:00] [INFO] [server.go:309] ["start to server request"] [addr=http://10.204.9.90:8249]
[2020/08/05 10:58:39.885 +08:00] [INFO] [merge.go:222] ["merger add source"] ["source id"=dcn-tidb-cluster1-p-l-01:8253]
[2020/08/05 10:58:39.885 +08:00] [INFO] [merge.go:222] ["merger add source"] ["source id"=dcn-tidb-cluster1-p-l-05:8253]
[2020/08/05 10:58:39.885 +08:00] [INFO] [merge.go:222] ["merger add source"] ["source id"=dcn-tidb-cluster1-p-l-06:8253]
[2020/08/05 10:58:39.886 +08:00] [INFO] [pump.go:134] ["pump create pull binlogs client"] [id=dcn-tidb-cluster1-p-l-06:8253]
[2020/08/05 10:58:39.886 +08:00] [INFO] [pump.go:134] ["pump create pull binlogs client"] [id=dcn-tidb-cluster1-p-l-01:8253]
[2020/08/05 10:58:39.886 +08:00] [INFO] [pump.go:134] ["pump create pull binlogs client"] [id=dcn-tidb-cluster1-p-l-05:8253]
[2020/08/05 10:58:40.880 +08:00] [WARN] [schema.go:243] ["ddl job schema version is less than current version, skip this ddl job"] [job="ID:491, Type:create table, State:cancelled, SchemaState:none, SchemaID:480, TableID:490, RowCount
:0, ArgLen:0, start time: 2020-07-05 02:16:35.441 +0800 CST, Err:[schema:1050]Table 'xue3_syncer_checkpoint' already exists, ErrCount:1, SnapshotVersion:0"] [currentVersion=0]
[2020/08/05 10:58:40.880 +08:00] [WARN] [schema.go:243] ["ddl job schema version is less than current version, skip this ddl job"] [job="ID:116, Type:create schema, State:cancelled, SchemaState:none, SchemaID:115, TableID:0, RowCount:
0, ArgLen:0, start time: 2020-06-22 23:50:03.974 +0800 CST, Err:[schema:1007]Can't create database 'dm_meta'; database exists, ErrCount:1, SnapshotVersion:0"] [currentVersion=0]
[2020/08/05 10:58:40.880 +08:00] [WARN] [schema.go:243] ["ddl job schema version is less than current version, skip this ddl job"] [job="ID:120, Type:create table, State:cancelled, SchemaState:none, SchemaID:113, TableID:119, RowCount
:0, ArgLen:0, start time: 2020-06-22 23:50:04.474 +0800 CST, Err:[schema:1050]Table 'xue_loader_checkpoint' already exists, ErrCount:1, SnapshotVersion:0"] [currentVersion=0]
[2020/08/05 10:58:40.880 +08:00] [WARN] [schema.go:243] ["ddl job schema version is less than current version, skip this ddl job"] [job="ID:124, Type:create table, State:cancelled, SchemaState:none, SchemaID:113, TableID:123, RowCount
:0, ArgLen:0, start time: 2020-06-22 23:50:05.524 +0800 CST, Err:[schema:1050]Table 'xue_syncer_sharding_meta' already exists, ErrCount:1, SnapshotVersion:0"] [currentVersion=0]
[2020/08/05 10:58:40.880 +08:00] [WARN] [schema.go:243] ["ddl job schema version is less than current version, skip this ddl job"] [job="ID:128, Type:create table, State:cancelled, SchemaState:none, SchemaID:113, TableID:127, RowCount
:0, ArgLen:0, start time: 2020-06-22 23:50:06.574 +0800 CST, Err:[schema:1050]Table 'xue_syncer_checkpoint' already exists, ErrCount:1, SnapshotVersion:0"] [currentVersion=0]
[2020/08/05 10:58:40.880 +08:00] [WARN] [schema.go:243] ["ddl job schema version is less than current version, skip this ddl job"] [job="ID:305, Type:create table, State:cancelled, SchemaState:none, SchemaID:294, TableID:304, RowCount
:0, ArgLen:0, start time: 2020-07-03 00:44:34.691 +0800 CST, Err:[schema:1050]Table 'xue1_syncer_sharding_meta' already exists, ErrCount:1, SnapshotVersion:0"] [currentVersion=0]
[2020/08/05 10:58:40.880 +08:00] [WARN] [schema.go:243] ["ddl job schema version is less than current version, skip this ddl job"] [job="ID:309, Type:create table, State:cancelled, SchemaState:none, SchemaID:294, TableID:308, RowCount
:0, ArgLen:0, start time: 2020-07-03 00:44:35.741 +0800 CST, Err:[schema:1050]Table 'xue1_syncer_checkpoint' already exists, ErrCount:1, SnapshotVersion:0"] [currentVersion=0]
[2020/08/05 10:58:40.880 +08:00] [WARN] [schema.go:243] ["ddl job schema version is less than current version, skip this ddl job"] [job="ID:411, Type:create table, State:cancelled, SchemaState:none, SchemaID:402, TableID:410, RowCount
:0, ArgLen:0, start time: 2020-07-03 06:21:58.741 +0800 CST, Err:[schema:1050]Table 'xue2_syncer_sharding_meta' already exists, ErrCount:1, SnapshotVersion:0"] [currentVersion=0]
[2020/08/05 10:58:40.880 +08:00] [WARN] [schema.go:243] ["ddl job schema version is less than current version, skip this ddl job"] [job="ID:415, Type:create table, State:cancelled, SchemaState:none, SchemaID:402, TableID:414, RowCount
:0, ArgLen:0, start time: 2020-07-03 06:21:59.791 +0800 CST, Err:[schema:1050]Table 'xue2_syncer_checkpoint' already exists, ErrCount:1, SnapshotVersion:0"] [currentVersion=0]
[2020/08/05 10:58:40.880 +08:00] [WARN] [schema.go:243] ["ddl job schema version is less than current version, skip this ddl job"] [job="ID:485, Type:create table, State:cancelled, SchemaState:none, SchemaID:480, TableID:484, RowCount
:0, ArgLen:0, start time: 2020-07-05 02:16:34.691 +0800 CST, Err:[schema:1050]Table 'xue3_loader_checkpoint' already exists, ErrCount:1, SnapshotVersion:0"] [currentVersion=0]
[2020/08/05 10:58:40.880 +08:00] [WARN] [schema.go:243] ["ddl job schema version is less than current version, skip this ddl job"] [job="ID:174, Type:create schema, State:cancelled, SchemaState:none, SchemaID:173, TableID:0, RowCount:
0, ArgLen:0, start time: 2020-06-23 00:47:36.024 +0800 CST, Err:[schema:1007]Can't create database 'dm_meta'; database exists, ErrCount:1, SnapshotVersion:0"] [currentVersion=0]
[2020/08/05 10:58:40.880 +08:00] [WARN] [schema.go:243] ["ddl job schema version is less than current version, skip this ddl job"] [job="ID:182, Type:create table, State:cancelled, SchemaState:none, SchemaID:171, TableID:181, RowCount
:0, ArgLen:0, start time: 2020-06-23 00:47:37.574 +0800 CST, Err:[schema:1050]Table 'xue_syncer_sharding_meta' already exists, ErrCount:1, SnapshotVersion:0"] [currentVersion=0]
[2020/08/05 10:58:40.880 +08:00] [WARN] [schema.go:243] ["ddl job schema version is less than current version, skip this ddl job"] [job="ID:186, Type:create table, State:cancelled, SchemaState:none, SchemaID:171, TableID:185, RowCount
:0, ArgLen:0, start time: 2020-06-23 00:47:38.624 +0800 CST, Err:[schema:1050]Table 'xue_syncer_checkpoint' already exists, ErrCount:1, SnapshotVersion:0"] [currentVersion=0]
[2020/08/05 10:58:40.880 +08:00] [WARN] [schema.go:243] ["ddl job schema version is less than current version, skip this ddl job"] [job="ID:218, Type:create schema, State:cancelled, SchemaState:none, SchemaID:217, TableID:0, RowCount:
0, ArgLen:0, start time: 2020-06-23 21:35:23.024 +0800 CST, Err:[schema:1007]Can't create database 'dm_meta'; database exists, ErrCount:1, SnapshotVersion:0"] [currentVersion=0]
[2020/08/05 10:58:40.880 +08:00] [WARN] [schema.go:243] ["ddl job schema version is less than current version, skip this ddl job"] [job="ID:222, Type:create table, State:cancelled, SchemaState:none, SchemaID:215, TableID:221, RowCount
:0, ArgLen:0, start time: 2020-06-23 21:35:23.524 +0800 CST, Err:[schema:1050]Table 'xue_loader_checkpoint' already exists, ErrCount:1, SnapshotVersion:0"] [currentVersion=0]
[2020/08/05 10:58:40.881 +08:00] [WARN] [schema.go:243] ["ddl job schema version is less than current version, skip this ddl job"] [job="ID:226, Type:create table, State:cancelled, SchemaState:none, SchemaID:215, TableID:225, RowCount
:0, ArgLen:0, start time: 2020-06-23 21:35:24.574 +0800 CST, Err:[schema:1050]Table 'xue_syncer_sharding_meta' already exists, ErrCount:1, SnapshotVersion:0"] [currentVersion=0]
[2020/08/05 10:58:40.881 +08:00] [WARN] [schema.go:243] ["ddl job schema version is less than current version, skip this ddl job"] [job="ID:297, Type:create schema, State:cancelled, SchemaState:none, SchemaID:296, TableID:0, RowCount:
0, ArgLen:0, start time: 2020-07-03 00:44:33.091 +0800 CST, Err:[schema:1007]Can't create database 'dm_meta'; database exists, ErrCount:1, SnapshotVersion:0"] [currentVersion=0]
[2020/08/05 10:58:40.881 +08:00] [WARN] [schema.go:243] ["ddl job schema version is less than current version, skip this ddl job"] [job="ID:230, Type:create table, State:cancelled, SchemaState:none, SchemaID:215, TableID:229, RowCount
:0, ArgLen:0, start time: 2020-06-23 21:35:25.624 +0800 CST, Err:[schema:1050]Table 'xue_syncer_checkpoint' already exists, ErrCount:1, SnapshotVersion:0"] [currentVersion=0]
[2020/08/05 10:58:40.881 +08:00] [WARN] [schema.go:243] ["ddl job schema version is less than current version, skip this ddl job"] [job="ID:301, Type:create table, State:cancelled, SchemaState:none, SchemaID:294, TableID:300, RowCount
:0, ArgLen:0, start time: 2020-07-03 00:44:33.691 +0800 CST, Err:[schema:1050]Table 'xue1_loader_checkpoint' already exists, ErrCount:1, SnapshotVersion:0"] [currentVersion=0]
[2020/08/05 10:58:40.881 +08:00] [WARN] [schema.go:243] ["ddl job schema version is less than current version, skip this ddl job"] [job="ID:407, Type:create table, State:cancelled, SchemaState:none, SchemaID:402, TableID:406, RowCount
:0, ArgLen:0, start time: 2020-07-03 06:21:57.691 +0800 CST, Err:[schema:1050]Table 'xue2_loader_checkpoint' already exists, ErrCount:1, SnapshotVersion:0"] [currentVersion=0]
[2020/08/05 10:58:40.881 +08:00] [WARN] [schema.go:243] ["ddl job schema version is less than current version, skip this ddl job"] [job="ID:577, Type:create table, State:cancelled, SchemaState:none, SchemaID:572, TableID:576, RowCount
:0, ArgLen:0, start time: 2020-07-05 03:09:37.691 +0800 CST, Err:[schema:1050]Table 'xue5_loader_checkpoint' already exists, ErrCount:1, SnapshotVersion:0"] [currentVersion=0]
[2020/08/05 10:58:40.881 +08:00] [WARN] [schema.go:243] ["ddl job schema version is less than current version, skip this ddl job"] [job="ID:178, Type:create table, State:cancelled, SchemaState:none, SchemaID:171, TableID:177, RowCount
:0, ArgLen:0, start time: 2020-06-23 00:47:36.524 +0800 CST, Err:[schema:1050]Table 'xue_loader_checkpoint' already exists, ErrCount:1, SnapshotVersion:0"] [currentVersion=0]
[2020/08/05 10:58:40.883 +08:00] [WARN] [kafka.go:276] ["unknown mysql type"] [type=9]
[2020/08/05 10:58:40.883 +08:00] [WARN] [kafka.go:276] ["unknown mysql type"] [type=9]
[2020/08/05 10:58:40.883 +08:00] [WARN] [kafka.go:276] ["unknown mysql type"] [type=9]
[2020/08/05 10:58:40.883 +08:00] [WARN] [kafka.go:276] ["unknown mysql type"] [type=9]
[2020/08/05 10:58:40.883 +08:00] [INFO] [client.go:772] ["[sarama] client/metadata fetching metadata for [uic_binlog] from broker 10.204.9.90:9092\
"]
[2020/08/05 10:58:40.884 +08:00] [WARN] [kafka.go:276] ["unknown mysql type"] [type=9]
[2020/08/05 10:58:40.884 +08:00] [WARN] [kafka.go:276] ["unknown mysql type"] [type=9]
[2020/08/05 10:58:40.884 +08:00] [INFO] [broker.go:214] ["[sarama] Connected to broker at 10.204.9.90:9092 (unregistered)\
"]
[2020/08/05 10:58:40.884 +08:00] [WARN] [kafka.go:276] ["unknown mysql type"] [type=9]
[2020/08/05 10:58:40.884 +08:00] [WARN] [kafka.go:276] ["unknown mysql type"] [type=9]
[2020/08/05 10:58:40.885 +08:00] [WARN] [kafka.go:276] ["unknown mysql type"] [type=9]
[2020/08/05 10:58:40.885 +08:00] [WARN] [kafka.go:276] ["unknown mysql type"] [type=9]
[2020/08/05 10:58:40.885 +08:00] [WARN] [kafka.go:276] ["unknown mysql type"] [type=9]
[2020/08/05 10:58:40.885 +08:00] [WARN] [kafka.go:276] ["unknown mysql type"] [type=9]
[2020/08/05 10:58:40.885 +08:00] [WARN] [kafka.go:276] ["unknown mysql type"] [type=9]
[2020/08/05 10:58:40.885 +08:00] [WARN] [kafka.go:276] ["unknown mysql type"] [type=9]
[2020/08/05 10:58:40.886 +08:00] [WARN] [kafka.go:276] ["unknown mysql type"] [type=9]
[2020/08/05 10:58:40.886 +08:00] [WARN] [kafka.go:276] ["unknown mysql type"] [type=9]
[2020/08/05 10:58:40.886 +08:00] [WARN] [kafka.go:276] ["unknown mysql type"] [type=9]
[2020/08/05 10:58:40.886 +08:00] [WARN] [kafka.go:276] ["unknown mysql type"] [type=9]
[2020/08/05 10:58:40.886 +08:00] [INFO] [client.go:534] ["[sarama] client/brokers registered new broker #0 at 10.204.9.90:9092"]
[2020/08/05 10:58:40.887 +08:00] [WARN] [kafka.go:276] ["unknown mysql type"] [type=9]
[2020/08/05 10:58:40.887 +08:00] [WARN] [kafka.go:276] ["unknown mysql type"] [type=9]
[2020/08/05 10:58:40.887 +08:00] [INFO] [async_producer.go:711] ["[sarama] producer/broker/0 starting up\
"]
[2020/08/05 10:58:40.887 +08:00] [INFO] [async_producer.go:727] ["[sarama] producer/broker/0 state change to [open] on uic_binlog/0\
"]
[2020/08/05 10:58:40.887 +08:00] [WARN] [kafka.go:276] ["unknown mysql type"] [type=9]
[2020/08/05 10:58:40.887 +08:00] [WARN] [kafka.go:276] ["unknown mysql type"] [type=9]
[2020/08/05 10:58:40.887 +08:00] [INFO] [broker.go:212] ["[sarama] Connected to broker at 10.204.9.90:9092 (registered as #0)\
"]
[2020/08/05 10:58:40.887 +08:00] [WARN] [kafka.go:276] ["unknown mysql type"] [type=9]
[2020/08/05 10:58:40.887 +08:00] [WARN] [kafka.go:276] ["unknown mysql type"] [type=9]
[2020/08/05 10:58:40.888 +08:00] [WARN] [kafka.go:276] ["unknown mysql type"] [type=9]
[2020/08/05 10:58:40.888 +08:00] [WARN] [kafka.go:276] ["unknown mysql type"] [type=9]
[2020/08/05 10:58:40.888 +08:00] [WARN] [kafka.go:276] ["unknown mysql type"] [type=9]
[2020/08/05 10:58:40.888 +08:00] [WARN] [kafka.go:276] ["unknown mysql type"] [type=9]
[2020/08/05 10:58:40.889 +08:00] [WARN] [kafka.go:276] ["unknown mysql type"] [type=9]
[2020/08/05 10:58:40.889 +08:00] [WARN] [kafka.go:276] ["unknown mysql type"] [type=9]
[2020/08/05 10:58:40.889 +08:00] [WARN] [kafka.go:276] ["unknown mysql type"] [type=9]
[2020/08/05 10:58:40.889 +08:00] [WARN] [kafka.go:276] ["unknown mysql type"] [type=9]
[2020/08/05 10:58:40.890 +08:00] [WARN] [kafka.go:276] ["unknown mysql type"] [type=9]
[2020/08/05 10:58:40.890 +08:00] [WARN] [kafka.go:276] ["unknown mysql type"] [type=9]
[2020/08/05 10:58:40.891 +08:00] [WARN] [kafka.go:276] ["unknown mysql type"] [type=9]
[2020/08/05 10:58:40.891 +08:00] [WARN] [kafka.go:276] ["unknown mysql type"] [type=9]
hi，以为已知问题，我们在 v3.0.17 已经修复，
问题原因是：tidb binlog 在此版本之前对上游 mediumint 传给 kafka 的处理方式上有问题，会以 string 传给 kafka
对于数据准确性，需要知道kafka 的下游是如何消费这个消息的
相关 issue：
https://github.com/pingcap/tidb-binlog/pull/962
感谢，晚点升级一下
ok~
此话题已在最后回复的 1 分钟后被自动关闭。不再允许新回复。