DM-worker切换上游MYSQL后无法获取gtid

  • 【TiDB 版本】:v4.0.0-rc
  • 【DM 版本】:v1.0.4-hotfix
  • 【问题描述】:由于上游mysql日志缺失,打算手动切换上游mysql实例。但是重启worker时报错

操作: 1、修改inventory.ini,手动指定新实例binlog和gtid:mysql_host ,relay_binlog_name,relay_binlog_gtid 2、更新dm-worker:ansible-playbook rolling_update.yml --tags=dm-worker

dm-worker日志: [2020/05/09 11:30:20.876 +08:00] [ERROR] [relay.go:395] [“the requested binlog files have purged in the master server or the master server have switched, currently DM do no support to handle this error”] [component=“relay log”] [“db host”=r3312_mydb.lzdb.com] [“db port”=3312] [error="[code=30015:class=relay-unit:scope=upstream:level=high] TCPReader get relay event with error: ERROR 1236 (HY000): The slave is connecting using CHANGE MASTER TO MASTER_AUTO_POSITION = 1, but the master has purged binary logs containing GTIDs that the slave requires."]

[2020/05/09 11:30:20.879 +08:00] [ERROR] [relay.go:264] [“fail to close binlog event reader”] [component=“relay log”] [error="[code=10001:class=database:scope=upstream:level=high] kill connection 3628260 for master r3312_mydb.lzdb.com:3312: database driver error: Error 1094: Unknown thread id: 3628260"] [errorVerbose="[code=10001:class=database:scope=upstream:level=high] kill connection 3628260 for master r3312_mydb.lzdb.com:3312: database driver error: Error 1094: Unknown thread id: 3628260\ngithub.com/pingcap/dm/pkg/terror.(*Error).Delegate\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/terror/terror.go:267\ngithub.com/pingcap/dm/pkg/terror.DBErrorAdaptArgs\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/terror/adapter.go:39\ github.com/pingcap/dm/pkg/terror.DBErrorAdapt\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/terror/adapter.go:46\ github.com/pingcap/dm/pkg/utils.KillConn\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/utils/db.go:356\ github.com/pingcap/dm/pkg/binlog/reader.(*TCPReader).Close\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/binlog/reader/tcp.go:131\ngithub.com/pingcap/dm/relay/reader.(*reader).Close\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/reader/reader.go:126\ngithub.com/pingcap/dm/relay.(*Relay).process.func1\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/relay.go:262\ngithub.com/pingcap/dm/relay.(*Relay).process\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/relay.go:296\ngithub.com/pingcap/dm/relay.(*Relay).Process\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/relay.go:191\ngithub.com/pingcap/dm/dm/worker.(*realRelayHolder).run\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/dm/worker/relay.go:167\ngithub.com/pingcap/dm/dm/worker.(*realRelayHolder).Start.func1\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/dm/worker/relay.go:143\ runtime.goexit\ \t/usr/local/go/src/runtime/asm_amd64.s:1357"] [2020/05/09 11:30:20.879 +08:00] [ERROR] [relay.go:194] [“process exit”] [component=“relay log”] [error="[code=30015:class=relay-unit:scope=upstream:level=high] TCPReader get relay event with error: ERROR 1236 (HY000): The slave is connecting using CHANGE MASTER TO MASTER_AUTO_POSITION = 1, but the master has purged binary logs containing GTIDs that the slave requires."] [errorVerbose="[code=30015:class=relay-unit:scope=upstream:level=high] TCPReader get relay event with error: ERROR 1236 (HY000): The slave is connecting using CHANGE MASTER TO MASTER_AUTO_POSITION = 1, but the master has purged binary logs containing GTIDs that the slave requires.\ngithub.com/pingcap/dm/pkg/terror.(*Error).Delegate\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/terror/terror.go:267\ngithub.com/pingcap/dm/pkg/binlog/reader.(*TCPReader).GetEvent\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/binlog/reader/tcp.go:151\ngithub.com/pingcap/dm/relay/reader.(*reader).GetEvent\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/reader/reader.go:144\ngithub.com/pingcap/dm/relay.(*Relay).handleEvents\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/relay.go:382\ngithub.com/pingcap/dm/relay.(*Relay).process\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/relay.go:292\ngithub.com/pingcap/dm/relay.(*Relay).Process\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/relay.go:191\ngithub.com/pingcap/dm/dm/worker.(*realRelayHolder).run\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/dm/worker/relay.go:167\ngithub.com/pingcap/dm/dm/worker.(*realRelayHolder).Start.func1\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/dm/worker/relay.go:143\ runtime.goexit\ \t/usr/local/go/src/runtime/asm_amd64.s:1357"] [2020/05/09 11:30:20.879 +08:00] [ERROR] [relay.go:173] [“process error”] [component=“relay holder”] [type=“msg:”[code=30015:class=relay-unit:scope=upstream:level=high] TCPReader get relay event with error: ERROR 1236 (HY000): The slave is connecting using CHANGE MASTER TO MASTER_AUTO_POSITION = 1, but the master has purged binary logs containing GTIDs that the slave requires.\ngithub.com/pingcap/dm/pkg/terror.(*Error).Delegate\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/terror/terror.go:267\ngithub.com/pingcap/dm/pkg/binlog/reader.(*TCPReader).GetEvent\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/binlog/reader/tcp.go:151\ngithub.com/pingcap/dm/relay/reader.(*reader).GetEvent\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/reader/reader.go:144\ngithub.com/pingcap/dm/relay.(*Relay).handleEvents\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/relay.go:382\ngithub.com/pingcap/dm/relay.(*Relay).process\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/relay.go:292\ngithub.com/pingcap/dm/relay.(*Relay).Process\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/relay.go:191\ngithub.com/pingcap/dm/dm/worker.(*realRelayHolder).run\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/dm/worker/relay.go:167\ngithub.com/pingcap/dm/dm/worker.(*realRelayHolder).Start.func1\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/dm/worker/relay.go:143\ runtime.goexit\ \t/usr/local/go/src/runtime/asm_amd64.s:1357" error:<ErrCode:30015 ErrClass:8 ErrScope:1 ErrLevel:3 Message:“TCPReader get relay event with error: ERROR 1236 (HY000): The slave is connecting using CHANGE MASTER TO MASTER_AUTO_POSITION = 1, but the master has purged binary logs containing GTIDs that the slave requires.” RawCause:“ERROR 1236 (HY000): The slave is connecting using CHANGE MASTER TO MASTER_AUTO_POSITION = 1, but the master has purged binary logs containing GTIDs that the slave requires.” > "]

你好,

你的问题已收到,正在分析,请稍等

你好,

从日志中看,该 gtid 已经被 purged

  1. 请确认上游 mysql 缺失 binlog 时间点
  2. 看下对应时间点的 mysql2 的日志是否存在
  3. 将 gtid 往前设置一点,开启 safe_mode 并 stop-task 、start-task,看是否可恢复数据。

原则就是

  1. 最好的结果就是将数据从断电就能接上。
  2. 再差一些就是将复制点位提前并开启 safe_mode 进行同步。如果判断数据已经没有冲突,记得将 safe_mode 修改为 false。

请检查下是否严格按照文档要求操作。

上游mysql1是缺失日志了。mysql2是确定包含该gtid信息的,目前卡在worker这一步,指定mysql2的gtid重启worker后就报找不到gtid。是否需要清除tidb中 dm_meta.table_checkpoint表中的记录

你好,

从目前的情况,建议修改 inventory 文件中 relay_binlog_gtid 设置为 mysql2 最早的一个点(在现存 binlog 文件找到最早的 gtid 点),开启 safe_mode 进行将数据补全,或者是复制。

以防数据丢失。因为你已经无法从 mysql 中确定具体的 xid 以及 mysql2 所对应的的 gtid 具体是什么。

PS:还是想问下具体的操作,因为如果操作不当可能导致数据丢失,是否严格按照文档操作进行的,是否存在偏差

具体是这样的:mysql1发生网络问题恢复后,dm并没有继续同步。当发现时mysql1的binlog已经被purge了。但是这个时候执行到的gtid是可以在query-status里面找到的。 第一次尝试使用文档的直接切换方式,结果日志报错找不到gtid

第二次尝试直接在dm-worker.toml中配置relay-binlog-name,relay-binlog-gtid。发现启动直接报错,也是找不到gtid。如果我不配置这两个参数,应该会从mysql2最新的binlog开始同步,这样可能会导致数据丢失

你好,

按照,上面提到的方式,将数据从 mysql2 最早的 binglog 开始同步如果还会丢数据,除非有备份,在 tidb 恢复下,在进行同步。

ps:在开启 enable_gtid ,relay_binlog_name 是不生效的。因为你已经开启了 gtid 复制模式。

【解决方案】

  1. stop-task
  2. 修改 inventory 文件中 relay_binlog_gtid 并将其设置为 mysql2 的最早的点(从最早的 binlog 中找下)
  3. 修改 task 中 safe_mode: true
  4. 滚动 dm-woker ,cat conf/dm-worker.toml 看配置是否生效
  5. start-task
  6. 将 safe_mode 修改回 false

谢谢,我先试试

第4步还是没有生效,我发现就算配置了relay_binlog_gtid。重启worker后还是会从上次中断的gtid开始读去,(relay.meta里面),结果还是会报一样的错

你好,

执行

ll {dm-worker_deploy_dir}/relay_log/
cat {dm-worker_deploy_dir}/relay_log/server-uuid.index

看下 .index 提示的目录下的 relay.meta 是否为需要的 gtid 点。

例子:

如果你已经将 gtid:mysql_host mysql_port 更换,就是已经切换了数据源,在 dm-worker/relay_log/ 下会增加一个文件夹,用来同步新的上游数据。

有生产新的文件夹,位置不是我设置的最早的binlog中的gtid,而是切换前执行到的gtid。

你好,

请帮忙确认

  1. 修改 inventory 文件之后,是否已经 rolling_update,将配置刷到 dm-worker 上
  2. 检查你的 dm-worker - conf - dm-worker.toml 是否已经更新了配置。
  3. 尝试 start-task 进行恢复。

1,2 都已经确认了。只是worker启动报错,也没有拉到relay log。所以没哟执行3

反馈下 dm-worker 的 log/dm-worker.log 日志,这边看下。

[2020/05/09 17:05:40.555 +08:00] [INFO] [server.go:284] [request=FetchDDLInfo]
[2020/05/09 17:23:11.510 +08:00] [INFO] [main.go:72] [“got signal to exit”] [signal=terminated]
[2020/05/09 17:23:11.510 +08:00] [ERROR] [metrics.go:87] [“fail to start status server return”] [error=“mux: listener closed”]
[2020/05/09 17:23:11.510 +08:00] [INFO] [worker.go:189] [“status print process exits!”] [component=“worker controller”]
[2020/05/09 17:23:11.510 +08:00] [INFO] [worker.go:781] [“handle task process exits!”] [component=“worker controller”]
[2020/05/09 17:23:11.510 +08:00] [INFO] [relay.go:668] [“relay unit is closing”] [component=“relay log”]
[2020/05/09 17:23:11.511 +08:00] [INFO] [relay.go:675] [“relay unit closed”] [component=“relay log”]
[2020/05/09 17:23:11.511 +08:00] [INFO] [purger.go:151] [“closing relay log purger”] [component=“relay purger”]
[2020/05/09 17:23:11.511 +08:00] [INFO] [task_checker.go:251] [“worker task checker exits”] [component=“task checker”]
[2020/05/09 17:23:11.511 +08:00] [INFO] [log.go:304] [“gc routine exits!”] [component=“operator log”]
[2020/05/09 17:23:11.511 +08:00] [INFO] [main.go:81] [“dm-worker exit”]
[2020/05/09 17:23:32.593 +08:00] [INFO] [printer.go:54] [“Welcome to dm-worker”] [“Release Version”=v1.0.4-hotfix] [“Git Commit Hash”=be5b54faec39c30e8161760db014224398f63156] [“Git Branch”=release-1.0] [“UTC Build Time”=“2020-03-19 09:30:23”] [“Go Version”=“go version go1.13 linux/amd64”]
[2020/05/09 17:23:32.593 +08:00] [INFO] [main.go:58] [“dm-worker config”=“{"log-level":"info","log-file":"/data_db1/dm/dm_worker1_1/log/dm-worker.log","log-rotate":"","worker-addr":":8262","enable-gtid":true,"auto-fix-gtid":false,"relay-dir":"/data_db1/dm/dm_worker1_1/relay_log","meta-dir":"./dm_worker_meta","server-id":429536752,"flavor":"mysql","charset":"","relay-binlog-name":"","relay-binlog-gtid":"db27cec9-0f05-11e9-84d2-6c92bf53d384:503232332","source-id":"mysql-replica-lizhipayment","from":{"host":"192.168.100.203","port":3312,"user":"tidbrepl","max-allowed-packet":67108864},"purge":{"interval":3600,"expires":0,"remain-space":15},"checker":{"check-enable":true,"backoff-rollback":{"duration":"5m0s"},"backoff-max":{"duration":"5m0s"}},"tracer":{"enable":false,"source":"mysql-replica-lizhipayment","tracer-addr":"","batch-size":20,"checksum":false},"config-file":"conf/dm-worker.toml"}”]
[2020/05/09 17:23:32.596 +08:00] [INFO] [db.go:54] [“kv config”=“{"block-cache-capacity":8388608,"block-restart-interval":16,"block-size":4096,"compaction-L0-trigger":8,"compaction-table-size":67108864,"compaction-total-size":536870912,"compaction-total-size-multiplier":8,"write-buffer":67108864,"write-L0-pause-trigger":24,"write-L0-slowdown-trigger":17}”]
[2020/05/09 17:23:32.605 +08:00] [INFO] [upgrade.go:169] [component=“bootstrap upgrade”] [“previous version”=“{"internal-no":1,"release-version":"v1.0.4-hotfix"}”]
[2020/05/09 17:23:32.605 +08:00] [INFO] [upgrade.go:174] [“previous and current versions are same, no need to upgrade”] [component=“bootstrap upgrade”] [version=“{"internal-no":1,"release-version":"v1.0.4-hotfix"}”]
[2020/05/09 17:23:32.606 +08:00] [INFO] [db.go:54] [“kv config”=“{"block-cache-capacity":8388608,"block-restart-interval":16,"block-size":4096,"compaction-L0-trigger":8,"compaction-table-size":67108864,"compaction-total-size":536870912,"compaction-total-size-multiplier":8,"write-buffer":67108864,"write-L0-pause-trigger":24,"write-L0-slowdown-trigger":17}”]
[2020/05/09 17:23:32.612 +08:00] [INFO] [log.go:199] [“finish initialization”] [component=“operator log”] [“handle pointer”=“{"Location":0}”] [“end pointer”=“{"Location":1}”]
[2020/05/09 17:23:32.612 +08:00] [INFO] [worker.go:146] [initialized] [component=“worker controller”]
[2020/05/09 17:23:32.612 +08:00] [INFO] [purger.go:115] [“starting relay log purger”] [component=“relay purger”] [config=“{"interval":3600,"expires":0,"remain-space":15}”]
[2020/05/09 17:23:32.612 +08:00] [INFO] [worker.go:182] [“start running”] [component=“worker controller”]
[2020/05/09 17:23:32.612 +08:00] [INFO] [server.go:112] [“start gRPC API”] [“listened address”=:8262]
[2020/05/09 17:23:32.614 +08:00] [WARN] [relay.go:320] [“no relay log file need to recover”] [component=“relay log”] [position=“(, 4)”] [“gtid set”=6a25d501-6991-11ea-9895-80615f0468d4:54697432]
[2020/05/09 17:23:32.614 +08:00] [INFO] [reader.go:159] [“start sync”] [component=“relay reader”] [master=192.168.100.203:3312] [“from GTID set”=6a25d501-6991-11ea-9895-80615f0468d4:54697432]
[2020/05/09 17:23:32.616 +08:00] [INFO] [reader.go:104] [“set up binlog reader”] [component=“relay reader”] [master=192.168.100.203:3312] [status=“{"stage":"prepared","connection":3642459}”]
[2020/05/09 17:23:32.616 +08:00] [INFO] [relay.go:617] [“started underlying reader”] [component=“relay log”] [UUID=5633ea06-14cc-11ea-afcf-50af73303989.000001]
[2020/05/09 17:23:32.616 +08:00] [INFO] [relay.go:634] [“started underlying writer”] [component=“relay log”] [UUID=5633ea06-14cc-11ea-afcf-50af73303989.000001] [config=“{"RelayDir":"/data_db1/dm/dm_worker1_1/relay_log/5633ea06-14cc-11ea-afcf-50af73303989.000001","Filename":""}”]
[2020/05/09 17:23:32.617 +08:00] [ERROR] [relay.go:395] [“the requested binlog files have purged in the master server or the master server have switched, currently DM do no support to handle this error”] [component=“relay log”] [“db host”=192.168.100.203] [“db port”=3312] [error=“[code=30015:class=relay-unit:scope=upstream:level=high] TCPReader get relay event with error: ERROR 1236 (HY000): The slave is connecting using CHANGE MASTER TO MASTER_AUTO_POSITION = 1, but the master has purged binary logs containing GTIDs that the slave requires.”]
[2020/05/09 17:23:32.617 +08:00] [INFO] [relay.go:400] [“current master status”] [component=“relay log”] [position=“(mysql-bin.000624, 522196882)”] [“GTID sets”=5633ea06-14cc-11ea-afcf-50af73303989:1-772,6a25d501-6991-11ea-9895-80615f0468d4:14-68374767,db27cec9-0f05-11e9-84d2-6c92bf53d384:369297016-478031801:479244719-504603615]
[2020/05/09 17:23:32.619 +08:00] [ERROR] [relay.go:264] [“fail to close binlog event reader”] [component=“relay log”] [error=“[code=10001:class=database:scope=upstream:level=high] kill connection 3642459 for master 192.168.100.203:3312: database driver error: Error 1094: Unknown thread id: 3642459”] [errorVerbose=“[code=10001:class=database:scope=upstream:level=high] kill connection 3642459 for master 192.168.100.203:3312: database driver error: Error 1094: Unknown thread id: 3642459
github.com/pingcap/dm/pkg/terror.(*Error).Delegate
\t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/terror/terror.go:267
github.com/pingcap/dm/pkg/terror.DBErrorAdaptArgs
\t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/terror/adapter.go:39
github.com/pingcap/dm/pkg/terror.DBErrorAdapt
\t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/terror/adapter.go:46
github.com/pingcap/dm/pkg/utils.KillConn
\t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/utils/db.go:356
github.com/pingcap/dm/pkg/binlog/reader.(*TCPReader).Close
\t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/binlog/reader/tcp.go:131
github.com/pingcap/dm/relay/reader.(*reader).Close
\t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/reader/reader.go:126
github.com/pingcap/dm/relay.(*Relay).process.func1
\t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/relay.go:262
github.com/pingcap/dm/relay.(*Relay).process
\t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/relay.go:296
github.com/pingcap/dm/relay.(*Relay).Process
\t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/relay.go:191
github.com/pingcap/dm/dm/worker.(*realRelayHolder).run
\t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/dm/worker/relay.go:167
github.com/pingcap/dm/dm/worker.(*realRelayHolder).Start.func1
\t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/dm/worker/relay.go:143
runtime.goexit
\t/usr/local/go/src/runtime/asm_amd64.s:1357”]
[2020/05/09 17:23:32.619 +08:00] [ERROR] [relay.go:194] [“process exit”] [component=“relay log”] [error=“[code=30015:class=relay-unit:scope=upstream:level=high] TCPReader get relay event with error: ERROR 1236 (HY000): The slave is connecting using CHANGE MASTER TO MASTER_AUTO_POSITION = 1, but the master has purged binary logs containing GTIDs that the slave requires.”] [errorVerbose=“[code=30015:class=relay-unit:scope=upstream:level=high] TCPReader get relay event with error: ERROR 1236 (HY000): The slave is connecting using CHANGE MASTER TO MASTER_AUTO_POSITION = 1, but the master has purged binary logs containing GTIDs that the slave requires.
github.com/pingcap/dm/pkg/terror.(*Error).Delegate
\t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/terror/terror.go:267
github.com/pingcap/dm/pkg/binlog/reader.(*TCPReader).GetEvent
\t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/binlog/reader/tcp.go:151
github.com/pingcap/dm/relay/reader.(*reader).GetEvent
\t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/reader/reader.go:144
github.com/pingcap/dm/relay.(*Relay).handleEvents
\t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/relay.go:382
github.com/pingcap/dm/relay.(*Relay).process
\t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/relay.go:292
github.com/pingcap/dm/relay.(*Relay).Process
\t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/relay.go:191
github.com/pingcap/dm/dm/worker.(*realRelayHolder).run
\t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/dm/worker/relay.go:167
github.com/pingcap/dm/dm/worker.(*realRelayHolder).Start.func1
\t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/dm/worker/relay.go:143
runtime.goexit
\t/usr/local/go/src/runtime/asm_amd64.s:1357”]
[2020/05/09 17:23:32.619 +08:00] [ERROR] [relay.go:173] [“process error”] [component=“relay holder”] [type="msg:"[code=30015:class=relay-unit:scope=upstream:level=high] TCPReader get relay event with error: ERROR 1236 (HY000): The slave is connecting using CHANGE MASTER TO MASTER_AUTO_POSITION = 1, but the master has purged binary logs containing GTIDs that the slave requires.\
github.com/pingcap/dm/pkg/terror.(*Error).Delegate\
\t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/terror/terror.go:267\
github.com/pingcap/dm/pkg/binlog/reader.(*TCPReader).GetEvent\
\t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/binlog/reader/tcp.go:151\
github.com/pingcap/dm/relay/reader.(*reader).GetEvent\
\t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/reader/reader.go:144\
github.com/pingcap/dm/relay.(*Relay).handleEvents\
\t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/relay.go:382\
github.com/pingcap/dm/relay.(*Relay).process\
\t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/relay.go:292\
github.com/pingcap/dm/relay.(*Relay).Process\
\t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/relay.go:191\
github.com/pingcap/dm/dm/worker.(*realRelayHolder).run\
\t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/dm/worker/relay.go:167\
github.com/pingcap/dm/dm/worker.(*realRelayHolder).Start.func1\
\t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/dm/worker/relay.go:143\
runtime.goexit\
\t/usr/local/go/src/runtime/asm_amd64.s:1357" error:<ErrCode:30015 ErrClass:8 ErrScope:1 ErrLevel:3 Message:"TCPReader get relay event with error: ERROR 1236 (HY000): The slave is connecting using CHANGE MASTER TO MASTER_AUTO_POSITION = 1, but the master has purged binary logs containing GTIDs that the slave requires." RawCause:"ERROR 1236 (HY000): The slave is connecting using CHANGE MASTER TO MASTER_AUTO_POSITION = 1, but the master has purged binary logs containing GTIDs that the slave requires." > "]
[2020/05/09 17:23:36.513 +08:00] [INFO] [server.go:284] [request=FetchDDLInfo]

image

image

你好,

  1. 检查 mysql2 中的 dm 复制的账号权限是否正确,检查 mysql2 和 dm-worker 的防火墙是否关闭 2.mv relay_log relay_log_bak ,在执行 rolling_update ,看下 dm-woker 是否正常启动

检查过了,还是不行。我先重新部署一次同步吧,后面再测试切换的

:sleepy::ok_hand: