drainer同步部分表到kafka,savepoint中commitTS 的值一直为0

🪐 TiDB 技术问题 备份&数据迁移
1

为提高效率，请提供以下信息，问题描述清晰能够更快得到解决：
【 TiDB 使用环境】
tidb 4.0.5

【概述】 使用drainer同步到下游kafka中进行下一步处理，日志始终打印[INFO] [schema.go:451] [“Finished dropping column”] [job=“ID:6145, Type:drop column, State:synced, SchemaState:none, SchemaID:5038, TableID:5623, RowCount:0, ArgLen:0, start time: 2020-08-04 17:12:49.413 +0800 CST, Err:, ErrCount:0, SnapshotVersion:0”]，savepoint文件中commitTS=0

【业务影响】
无法启动同步

【TiDB 版本】
tidb4.0.5

【附件】

  • 相关日志、配置文件、Grafana 监控（https://metricstool.pingcap.com/)
    [2022/02/14 15:51:13.047 +08:00] [INFO] [version.go:50] [“Welcome to Drainer”] [“Release Version”=v4.0.3-1-g09a8cfd] [“Git Commit Hash”=09a8cfda8a305416f11d56bfcc939210eeea804e] [“Build TS”=“2020-08-10 02:22:56”] [“Go Version”=go1.13] [“Go OS/Arch”=linux/amd64]
    [2022/02/14 15:51:13.048 +08:00] [INFO] [main.go:46] [“start drainer…”] [config="{“log-level”:“info”,“node-id”:“10.74.131.103:8253”,“addr”:“http://10.74.131.103:8253”,“advertise-addr”:“http://10.74.131.103:8253”,“data-dir”:"/data/deploy/data/drainer-8253",“detect-interval”:5,“pd-urls”:“http://10.74.131.81:2379,http://10.74.131.82:2379,http://10.74.131.83:2379”,“log-file”:"/data/deploy/log/drainer-8253.log",“initial-commit-ts”:0,“sycner”:{“sql-mode”:null,“ignore-txn-commit-ts”:null,“ignore-schemas”:“INFORMATION_SCHEMA,PERFORMANCE_SCHEMA,mysql”,“ignore-table”:null,“txn-batch”:20,“loopback-control”:false,“sync-ddl”:true,“channel-id”:0,“worker-count”:1,“to”:{“host”:"",“user”:"",“password”:"",“security”:{“ssl-ca”:"",“ssl-cert”:"",“ssl-key”:"",“cert-allowed-cn”:null},“encrypted_password”:"",“sync-mode”:0,“port”:0,“checkpoint”:{“type”:"",“schema”:"",“host”:"",“user”:"",“password”:"",“encrypted_password”:"",“port”:0,“security”:{“ssl-ca”:"",“ssl-cert”:"",“ssl-key”:"",“cert-allowed-cn”:null}},“dir”:"",“retention-time”:0,“merge”:false,“zookeeper-addrs”:"",“kafka-addrs”:“host-kf001.ymt.io:9092,host-kf002.ymt.io:9092,host-kf003.ymt.io:9092”,“kafka-version”:“2.6.0”,“kafka-max-messages”:4096,“kafka-client-id”:"",“topic-name”:“drainer_kafka_dts_adb_ymtdw_special”},“replicate-do-table”:[{“db-name”:“ymtdw”,“tbl-name”:“dwd_growth_page_achieve_rank_m”},{“db-name”:“ymtdw”,“tbl-name”:“dwd_growth_page_achievement_m”},{“db-name”:“ymtdw”,“tbl-name”:“service_page_achieve_rank”}],“replicate-do-db”:null,“db-type”:“kafka”,“relay”:{“log-dir”:"",“max-file-size”:10485760},“disable-dispatch-flag”:null,“enable-dispatch-flag”:null,“disable-dispatch”:null,“enable-dispatch”:null,“safe-mode”:false,“disable-detect-flag”:null,“enable-detect-flag”:null,“disable-detect”:null,“enable-detect”:null},“security”:{“ssl-ca”:"",“ssl-cert”:"",“ssl-key”:"",“cert-allowed-cn”:null},“synced-check-time”:5,“compressor”:"",“EtcdTimeout”:5000000000,“MetricsAddr”:"",“MetricsInterval”:15}"]
    [2022/02/14 15:51:13.048 +08:00] [INFO] [client.go:135] ["[pd] create pd client with endpoints"] [pd-address="[http://10.74.131.81:2379,http://10.74.131.82:2379,http://10.74.131.83:2379]"]
    [2022/02/14 15:51:13.052 +08:00] [INFO] [base_client.go:242] ["[pd] switch leader"] [new-leader=http://10.74.131.81:2379] [old-leader=]
    [2022/02/14 15:51:13.052 +08:00] [INFO] [base_client.go:92] ["[pd] init cluster id"] [cluster-id=6634740172973472258]
    [2022/02/14 15:51:13.052 +08:00] [INFO] [server.go:120] [“get cluster id from pd”] [id=6634740172973472258]
    [2022/02/14 15:51:13.052 +08:00] [INFO] [checkpoint.go:64] [“initialize checkpoint”] [type=file] [checkpoint=0] [cfg="{“CheckpointType”:“file”,“Db”:null,“Schema”:"",“Table”:"",“ClusterID”:6634740172973472258,“InitialCommitTS”:0,“dir”:"/data/deploy/data/drainer-8253/savepoint"}"]
    [2022/02/14 15:51:13.052 +08:00] [INFO] [store.go:68] [“new store”] [path=“tikv://10.74.131.81:2379,10.74.131.82:2379,10.74.131.83:2379?disableGC=true”]
    [2022/02/14 15:51:13.053 +08:00] [INFO] [client.go:135] ["[pd] create pd client with endpoints"] [pd-address="[10.74.131.81:2379,10.74.131.82:2379,10.74.131.83:2379]"]
    [2022/02/14 15:51:13.055 +08:00] [INFO] [base_client.go:242] ["[pd] switch leader"] [new-leader=http://10.74.131.81:2379] [old-leader=]
    [2022/02/14 15:51:13.055 +08:00] [INFO] [base_client.go:92] ["[pd] init cluster id"] [cluster-id=6634740172973472258]
    [2022/02/14 15:51:13.055 +08:00] [INFO] [store.go:74] [“new store with retry success”]
    [2022/02/14 15:51:14.792 +08:00] [INFO] [client.go:127] ["[sarama] Initializing new client"]
    [2022/02/14 15:51:14.792 +08:00] [INFO] [client.go:174] ["[sarama] Successfully initialized new client"]
    [2022/02/14 15:51:14.792 +08:00] [INFO] [store.go:68] [“new store”] [path=“tikv://10.74.131.81:2379,10.74.131.82:2379,10.74.131.83:2379?disableGC=true”]
    [2022/02/14 15:51:14.792 +08:00] [INFO] [client.go:135] ["[pd] create pd client with endpoints"] [pd-address="[10.74.131.81:2379,10.74.131.82:2379,10.74.131.83:2379]"]
    [2022/02/14 15:51:14.795 +08:00] [INFO] [base_client.go:242] ["[pd] switch leader"] [new-leader=http://10.74.131.81:2379] [old-leader=]
    [2022/02/14 15:51:14.795 +08:00] [INFO] [base_client.go:92] ["[pd] init cluster id"] [cluster-id=6634740172973472258]
    [2022/02/14 15:51:14.796 +08:00] [INFO] [store.go:74] [“new store with retry success”]
    [2022/02/14 15:51:14.801 +08:00] [INFO] [server.go:265] [“register success”] [“drainer node id”=10.74.131.103:8253]
    [2022/02/14 15:51:14.801 +08:00] [INFO] [server.go:326] [“start to server request”] [addr=http://10.74.131.103:8253]
    [2022/02/14 15:51:14.802 +08:00] [INFO] [merge.go:222] [“merger add source”] [“source id”=host-tidb-pump01.ymt.io:8250]
    [2022/02/14 15:51:14.802 +08:00] [INFO] [merge.go:222] [“merger add source”] [“source id”=host-tidb-pump02.ymt.io:8250]
    [2022/02/14 15:51:14.802 +08:00] [INFO] [merge.go:222] [“merger add source”] [“source id”=host-tidb-pump03.ymt.io:8250]
    [2022/02/14 15:51:14.802 +08:00] [INFO] [pump.go:138] [“pump create pull binlogs client”] [id:8250=host-tidb-pump03.ymt.io]
    [2022/02/14 15:51:14.803 +08:00] [INFO] [pump.go:138] [“pump create pull binlogs client”] [id:8250=host-tidb-pump01.ymt.io]
    [2022/02/14 15:51:14.803 +08:00] [INFO] [pump.go:138] [“pump create pull binlogs client”] [id:8250=host-tidb-pump02.ymt.io]
    [2022/02/14 15:51:15.796 +08:00] [WARN] [schema.go:243] [“ddl job schema version is less than current version, skip this ddl job”] [job=“ID:60, Type:create table, State:cancelled, SchemaState:none, SchemaID:3, TableID:59, RowCount:0, ArgLen:0, start time: 2018-12-14 15:00:57.135 +0800 CST, Err:[schema:1050]Table ‘stats_histograms’ already exists, ErrCount:1, SnapshotVersion:0”] [currentVersion=0]
    [2022/02/14 15:51:15.796 +08:00] [WARN] [schema.go:243] [“ddl job schema version is less than current version, skip this ddl job”] [job=“ID:84, Type:create table, State:cancelled, SchemaState:none, SchemaID:3, TableID:83, RowCount:0, ArgLen:0, start time: 2018-12-14 15:01:05.685 +0800 CST, Err:[schema:1050]Table ‘stats_feedback’ already exists, ErrCount:1, SnapshotVersion:0”] [currentVersion=0]
    [2022/02/14 15:51:15.796 +08:00] [WARN] [schema.go:243] [“ddl job schema version is less than current version, skip this ddl job”] [job=“ID:6, Type:create schema, State:cancelled, SchemaState:none, SchemaID:5, TableID:0, RowCount:0, ArgLen:0, start time: 2018-12-14 15:00:39.985 +0800 CST, Err:[schema:1007]Can’t create database ‘mysql’; database exists, ErrCount:1, SnapshotVersion:0”] [currentVersion=0]
    [2022/02/14 15:51:15.796 +08:00] [WARN] [schema.go:243] [“ddl job schema version is less than current version, skip this ddl job”] [job=“ID:8, Type:create schema, State:cancelled, SchemaState:none, SchemaID:7, TableID:0, RowCount:0, ArgLen:0, start time: 2018-12-14 15:00:40.085 +0800 CST, Err:[schema:1007]Can’t create database ‘mysql’; database exists, ErrCount:1, SnapshotVersion:0”] [currentVersion=0]
    [2022/02/14 15:51:15.796 +08:00] [WARN] [schema.go:243] [“ddl job schema version is less than current version, skip this ddl job”] [job=“ID:74, Type:create table, State:cancelled, SchemaState:none, SchemaID:3, TableID:73, RowCount:0, ArgLen:0, start time: 2018-12-14 15:01:02.135 +0800 CST, Err:[schema:1050]Table ‘gc_delete_range’ already exists, ErrCount:1, SnapshotVersion:0”] [currentVersion=0]
    [2022/02/14 15:51:15.796 +08:00] [WARN] [schema.go:243] [“ddl job schema version is less than current version, skip this ddl job”] [job=“ID:12, Type:create table, State:cancelled, SchemaState:none, SchemaID:3, TableID:11, RowCount:0, ArgLen:0, start time: 2018-12-14 15:00:40.485 +0800 CST, Err:[schema:1050]Table ‘user’ already exists, ErrCount:1, SnapshotVersion:0”] [currentVersion=0]
    [2022/02/14 15:51:15.797 +08:00] [WARN] [schema.go:243] [“ddl job schema version is less than current version, skip this ddl job”] [job=“ID:14, Type:create table, State:cancelled, SchemaState:none, SchemaID:3, TableID:13, RowCount:0, ArgLen:0, start time: 2018-12-14 15:00:41.585 +0800 CST, Err:[schema:1050]Table ‘user’ already exists, ErrCount:1, SnapshotVersion:0”] [currentVersion=0]
    [2022/02/14 15:51:15.797 +08:00] [WARN] [schema.go:243] [“ddl job schema version is less than current version, skip this ddl job”] [job=“ID:80, Type:create table, State:cancelled, SchemaState:none, SchemaID:3, TableID:79, RowCount:0, ArgLen:0, start time: 2018-12-14 15:01:04.685 +0800 CST, Err:[schema:1050]Table ‘gc_delete_range_done’ already exists, ErrCount:1, SnapshotVersion:0”] [currentVersion=0]
    [2022/02/14 15:51:15.797 +08:00] [WARN] [schema.go:243] [“ddl job schema version is less than current version, skip this ddl job”] [job=“ID:18, Type:create table, State:cancelled, SchemaState:none, SchemaID:3, TableID:17, RowCount:0, ArgLen:0, start time: 2018-12-14 15:00:42.485 +0800 CST, Err:[schema:1050]Table ‘db’ already exists, ErrCount:1, SnapshotVersion:0”] [currentVersion=0]
    [2022/02/14 15:51:15.797 +08:00] [WARN] [schema.go:243] [“ddl job schema version is less than current version, skip this ddl job”] [job=“ID:20, Type:create table, State:cancelled, SchemaState:none, SchemaID:3, TableID:19, RowCount:0, ArgLen:0, start time: 2018-12-14 15:00:43.585 +0800 CST, Err:[schema:1050]Table ‘db’ already exists, ErrCount:1, SnapshotVersion:0”] [currentVersion=0]
    [2022/02/14 15:51:15.797 +08:00] [WARN] [schema.go:243] [“ddl job schema version is less than current version, skip this ddl job”] [job=“ID:78, Type:create table, State:cancelled, SchemaState:none, SchemaID:3, TableID:77, RowCount:0, ArgLen:0, start time: 2018-12-14 15:01:03.635 +0800 CST, Err:[schema:1050]Table ‘gc_delete_range_done’ already exists, ErrCount:1, SnapshotVersion:0”] [currentVersion=0]
    [2022/02/14 15:51:15.797 +08:00] [WARN] [schema.go:243] [“ddl job schema version is less than current version, skip this ddl job”] [job=“ID:24, Type:create table, State:cancelled, SchemaState:none, SchemaID:3, TableID:23, RowCount:0, ArgLen:0, start time: 2018-12-14 15:00:44.485 +0800 CST, Err:[schema:1050]Table ‘tables_priv’ already exists, ErrCount:1, SnapshotVersion:0”] [currentVersion=0]
    [2022/02/14 15:51:15.797 +08:00] [WARN] [schema.go:243] [“ddl job schema version is less than current version, skip this ddl job”] [job=“ID:26, Type:create table, State:cancelled, SchemaState:none, SchemaID:3, TableID:25, RowCount:0, ArgLen:0, start time: 2018-12-14 15:00:45.585 +0800 CST, Err:[schema:1050]Table ‘tables_priv’ already exists, ErrCount:1, SnapshotVersion:0”] [currentVersion=0]
    [2022/02/14 15:51:15.797 +08:00] [WARN] [schema.go:243] [“ddl job schema version is less than current version, skip this ddl job”] [job=“ID:5606, Type:add column, State:cancelled, SchemaState:none, SchemaID:5038, TableID:5541, RowCount:0, ArgLen:0, start time: 2019-11-02 11:35:13.902 +0800 CST, Err:[schema:1054]Unknown column ‘click_search_real_pv_15day’ in ‘supply_effect’, ErrCount:1, SnapshotVersion:0”] [currentVersion=0]
    [2022/02/14 15:51:15.797 +08:00] [WARN] [schema.go:243] [“ddl job schema version is less than current version, skip this ddl job”] [job=“ID:30, Type:create table, State:cancelled, SchemaState:none, SchemaID:3, TableID:29, RowCount:0, ArgLen:0, start time: 2018-12-14 15:00:46.485 +0800 CST, Err:[schema:1050]Table ‘columns_priv’ already exists, ErrCount:1, SnapshotVersion:0”] [currentVersion=0]
    [2022/02/14 15:51:15.797 +08:00] [WARN] [schema.go:243] [“ddl job schema version is less than current version, skip this ddl job”] [job=“ID:32, Type:create table, State:cancelled, SchemaState:none, SchemaID:3, TableID:31, RowCount:0, ArgLen:0, start time: 2018-12-14 15:00:47.585 +0800 CST, Err:[schema:1050]Table ‘columns_priv’ already exists, ErrCount:1, SnapshotVersion:0”] [currentVersion=0]
    [2022/02/14 15:51:15.797 +08:00] [WARN] [schema.go:243] [“ddl job schema version is less than current version, skip this ddl job”] [job=“ID:44, Type:create table, State:cancelled, SchemaState:none, SchemaID:3, TableID:43, RowCount:0, ArgLen:0, start time: 2018-12-14 15:00:52.135 +0800 CST, Err:[schema:1050]Table ‘tidb’ already exists, ErrCount:1, SnapshotVersion:0”] [currentVersion=0]
    [2022/02/14 15:51:15.797 +08:00] [WARN] [schema.go:243] [“ddl job schema version is less than current version, skip this ddl job”] [job=“ID:36, Type:create table, State:cancelled, SchemaState:none, SchemaID:3, TableID:35, RowCount:0, ArgLen:0, start time: 2018-12-14 15:00:49.085 +0800 CST, Err:[schema:1050]Table ‘GLOBAL_VARIABLES’ already exists, ErrCount:1, SnapshotVersion:0”] [currentVersion=0]
    [2022/02/14 15:51:15.797 +08:00] [WARN] [schema.go:243] [“ddl job schema version is less than current version, skip this ddl job”] [job=“ID:38, Type:create table, State:cancelled, SchemaState:none, SchemaID:3, TableID:37, RowCount:0, ArgLen:0, start time: 2018-12-14 15:00:49.585 +0800 CST, Err:[schema:1050]Table ‘GLOBAL_VARIABLES’ already exists, ErrCount:1, SnapshotVersion:0”] [currentVersion=0]
    [2022/02/14 15:51:15.797 +08:00] [WARN] [schema.go:243] [“ddl job schema version is less than current version, skip this ddl job”] [job=“ID:72, Type:create table, State:cancelled, SchemaState:none, SchemaID:3, TableID:71, RowCount:0, ArgLen:0, start time: 2018-12-14 15:01:01.535 +0800 CST, Err:[schema:1050]Table ‘gc_delete_range’ already exists, ErrCount:1, SnapshotVersion:0”] [currentVersion=0]
    [2022/02/14 15:51:15.797 +08:00] [WARN] [schema.go:243] [“ddl job schema version is less than current version, skip this ddl job”] [job=“ID:42, Type:create table, State:cancelled, SchemaState:none, SchemaID:3, TableID:41, RowCount:0, ArgLen:0, start time: 2018-12-14 15:00:51.085 +0800 CST, Err:[schema:1050]Table ‘tidb’ already exists, ErrCount:1, SnapshotVersion:0”] [currentVersion=0]
    [2022/02/14 15:51:15.797 +08:00] [WARN] [schema.go:243] [“ddl job schema version is less than current version, skip this ddl job”] [job=“ID:54, Type:create table, State:cancelled, SchemaState:none, SchemaID:3, TableID:53, RowCount:0, ArgLen:0, start time: 2018-12-14 15:00:55.135 +0800 CST, Err:[schema:1050]Table ‘stats_meta’ already exists, ErrCount:1, SnapshotVersion:0”] [currentVersion=0]
    [2022/02/14 15:51:15.797 +08:00] [WARN] [schema.go:243] [“ddl job schema version is less than current version, skip this ddl job”] [job=“ID:68, Type:create table, State:cancelled, SchemaState:none, SchemaID:3, TableID:67, RowCount:0, ArgLen:0, start time: 2018-12-14 15:01:00.135 +0800 CST, Err:[schema:1050]Table ‘stats_buckets’ already exists, ErrCount:1, SnapshotVersion:0”] [currentVersion=0]
    [2022/02/14 15:51:15.797 +08:00] [WARN] [schema.go:243] [“ddl job schema version is less than current version, skip this ddl job”] [job=“ID:48, Type:create table, State:cancelled, SchemaState:none, SchemaID:3, TableID:47, RowCount:0, ArgLen:0, start time: 2018-12-14 15:00:53.085 +0800 CST, Err:[schema:1050]Table ‘help_topic’ already exists, ErrCount:1, SnapshotVersion:0”] [currentVersion=0]
    [2022/02/14 15:51:15.797 +08:00] [WARN] [schema.go:243] [“ddl job schema version is less than current version, skip this ddl job”] [job=“ID:50, Type:create table, State:cancelled, SchemaState:none, SchemaID:3, TableID:49, RowCount:0, ArgLen:0, start time: 2018-12-14 15:00:54.135 +0800 CST, Err:[schema:1050]Table ‘help_topic’ already exists, ErrCount:1, SnapshotVersion:0”] [currentVersion=0]
    [2022/02/14 15:51:15.797 +08:00] [WARN] [schema.go:243] [“ddl job schema version is less than current version, skip this ddl job”] [job=“ID:66, Type:create table, State:cancelled, SchemaState:none, SchemaID:3, TableID:65, RowCount:0, ArgLen:0, start time: 2018-12-14 15:00:59.135 +0800 CST, Err:[schema:1050]Table ‘stats_buckets’ already exists, ErrCount:1, SnapshotVersion:0”] [currentVersion=0]
    [2022/02/14 15:51:15.797 +08:00] [WARN] [schema.go:243] [“ddl job schema version is less than current version, skip this ddl job”] [job=“ID:62, Type:create table, State:cancelled, SchemaState:none, SchemaID:3, TableID:61, RowCount:0, ArgLen:0, start time: 2018-12-14 15:00:58.135 +0800 CST, Err:[schema:1050]Table ‘stats_histograms’ already exists, ErrCount:1, SnapshotVersion:0”] [currentVersion=0]
    [2022/02/14 15:51:15.797 +08:00] [WARN] [schema.go:243] [“ddl job schema version is less than current version, skip this ddl job”] [job=“ID:56, Type:create table, State:cancelled, SchemaState:none, SchemaID:3, TableID:55, RowCount:0, ArgLen:0, start time: 2018-12-14 15:00:56.135 +0800 CST, Err:[schema:1050]Table ‘stats_meta’ already exists, ErrCount:1, SnapshotVersion:0”] [currentVersion=0]
    [2022/02/14 15:51:15.797 +08:00] [WARN] [schema.go:243] [“ddl job schema version is less than current version, skip this ddl job”] [job=“ID:5607, Type:add column, State:cancelled, SchemaState:none, SchemaID:5038, TableID:5541, RowCount:0, ArgLen:0, start time: 2019-11-02 11:35:39.102 +0800 CST, Err:[schema:1054]Unknown column ‘click_search_real_pv_30day’ in ‘supply_effect’, ErrCount:1, SnapshotVersion:0”] [currentVersion=0]
    [2022/02/14 15:51:15.797 +08:00] [WARN] [schema.go:243] [“ddl job schema version is less than current version, skip this ddl job”] [job=“ID:86, Type:create table, State:cancelled, SchemaState:none, SchemaID:3, TableID:85, RowCount:0, ArgLen:0, start time: 2018-12-14 15:01:06.685 +0800 CST, Err:[schema:1050]Table ‘stats_feedback’ already exists, ErrCount:1, SnapshotVersion:0”] [currentVersion=0]
    [2022/02/14 15:51:15.804 +08:00] [INFO] [schema.go:451] [“Finished dropping column”] [job=“ID:5555, Type:drop column, State:synced, SchemaState:none, SchemaID:5038, TableID:5543, RowCount:0, ArgLen:0, start time: 2019-10-24 14:32:44.052 +0800 CST, Err:, ErrCount:0, SnapshotVersion:0”]
    [2022/02/14 15:51:15.804 +08:00] [INFO] [schema.go:451] [“Finished dropping column”] [job=“ID:5558, Type:drop column, State:synced, SchemaState:none, SchemaID:5038, TableID:5543, RowCount:0, ArgLen:0, start time: 2019-10-24 14:45:40.352 +0800 CST, Err:, ErrCount:0, SnapshotVersion:0”]
    [2022/02/14 15:51:15.804 +08:00] [INFO] [schema.go:451] [“Finished dropping column”] [job=“ID:5561, Type:drop column, State:synced, SchemaState:none, SchemaID:5038, TableID:5543, RowCount:0, ArgLen:0, start time: 2019-10-24 14:45:59.902 +0800 CST, Err:, ErrCount:0, SnapshotVersion:0”]
    [2022/02/14 15:51:15.804 +08:00] [INFO] [schema.go:451] [“Finished dropping column”] [job=“ID:5564, Type:drop column, State:synced, SchemaState:none, SchemaID:5038, TableID:5543, RowCount:0, ArgLen:0, start time: 2019-10-24 14:48:15.502 +0800 CST, Err:, ErrCount:0, SnapshotVersion:0”]
    [2022/02/14 15:51:15.804 +08:00] [INFO] [schema.go:451] [“Finished dropping column”] [job=“ID:5567, Type:drop column, State:synced, SchemaState:none, SchemaID:5038, TableID:5543, RowCount:0, ArgLen:0, start time: 2019-10-24 14:48:29.402 +0800 CST, Err:, ErrCount:0, SnapshotVersion:0”]
    [2022/02/14 15:51:15.804 +08:00] [INFO] [schema.go:451] [“Finished dropping column”] [job=“ID:5570, Type:drop column, State:synced, SchemaState:none, SchemaID:5038, TableID:5543, RowCount:0, ArgLen:0, start time: 2019-10-24 14:49:48.752 +0800 CST, Err:, ErrCount:0, SnapshotVersion:0”]
    配置文件：
    [syncer]
    db-type = “kafka”
    ignore-schemas = “INFORMATION_SCHEMA,PERFORMANCE_SCHEMA,mysql”
    [[syncer.replicate-do-table]]
    db-name =“ymtdw”
    tbl-name = “dwd_growth_page_achieve_rank_m”
    [[syncer.replicate-do-table]]
    db-name =“ymtdw”
    tbl-name = “dwd_growth_page_achievement_m”
    [[syncer.replicate-do-table]]
    db-name =“ymtdw”
    tbl-name = “service_page_achieve_rank”
    [syncer.to]
    kafka-addrs = “host-kf001.ymt.io:9092,host-kf002.ymt.io:9092,host-kf003.ymt.io:9092”
    kafka-max-messages = 4096
    kafka-version = “2.6.0”
    topic-name = “drainer_kafka_dts_adb_ymtdw_special”
2

你上面的这个配置是什么，从打印出来的日志看设置的值是 0？

在 4.0 官网看见，该参数的值为：

image
image1574×594 124 KB

在更早版本 v2.1 中，这个参数的默认值是 0。你上面是 v4.0.9 版本，建议根据下面的文档配置下，观察下效果：
https://docs.pingcap.com/zh/tidb/v4.0/tidb-binlog-configuration-file#initial-commit-ts

另外，建议了解下 ticdc，ticdc 具有高可用，并且下游的生态更加丰富：
https://docs.pingcap.com/zh/tidb/v5.2/ticdc-overview/#ticdc-简介

image
image1584×724 148 KB

1 个赞
3

是不是问题不止一个，这个有关系么？这一块接触不深，会不会是在同步结构的时候也有问题

4

表结构的问题吧

5

此话题已在最后回复的 1 分钟后被自动关闭。不再允许新回复。