DM 同步报错 TCPReader get relay event with error: ERROR 1236 (HY000): The slave is connecting using CHANGE MASTER

“msg”: “[code=30015:class=relay-unit:scope=upstream:level=high] TCPReader get relay event with error: ERROR 1236 (HY000): The slave is connecting using CHANGE MASTER TO MASTER_AUTO_POSITION = 1, but the master has purged binary logs containing GTIDs that the slave requires.\ngithub.com/pingcap/dm/pkg/terror.(*Error).Delegate\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/terror/terror.go:267\ngithub.com/pingcap/dm/pkg/binlog/reader.(*TCPReader).GetEvent\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/binlog/reader/tcp.go:151\ngithub.com/pingcap/dm/relay/reader.(*reader).GetEvent\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/reader/reader.go:144\ngithub.com/pingcap/dm/relay.(*Relay).handleEvents\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/relay.go:382\ngithub.com/pingcap/dm/relay.(*Relay).process\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/relay.go:292\ngithub.com/pingcap/dm/relay.(*Relay).Process\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/relay.go:191\ngithub.com/pingcap/dm/dm/worker.(*realRelayHolder).run\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/dm/worker/relay.go:167\ngithub.com/pingcap/dm/dm/worker.(*realRelayHolder).Start.func1\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/dm/worker/relay.go:143\ runtime.goexit\ \t/usr/local/go/src/runtime/asm_amd64.s:1357”,
“error”: null

提供下 dm 基本版本信息,上下游节点情况,反馈下详细的 query status 信息和 dm worker log 感谢

dm 105
详细信息
{
“result”: true,
“msg”: “”,
“workers”: [
{
“result”: true,
“worker”: “172.17.116.163:8266”,
“msg”: “”,
“subTaskStatus”: [
{
“name”: “galaxy”,
“stage”: “Running”,
“unit”: “Sync”,
“result”: null,
“unresolvedDDLLockID”: “”,
“sync”: {
“totalEvents”: “2184372”,
“totalTps”: “6589”,
“recentTps”: “0”,
“masterBinlog”: “(mysql-bin.024303, 353162314)”,
“masterBinlogGtid”: “d2aee377-0e66-11ea-b85a-7cd30aeb1510:1-17523587701,678e197a-f60e-11e9-94f9-b8599f37f3d0:1-264381,706de50e-075e-11ea-9909-7cd30adb2d50:1-100298”,
“syncerBinlog”: “(mysql-bin|000001.024291, 625629)”,
“syncerBinlogGtid”: “”,
“blockingDDLs”: [
],
“unresolvedGroups”: [
],
“synced”: false
}
}
],
“relayStatus”: {
“masterBinlog”: “(mysql-bin.024303, 353162314)”,
“masterBinlogGtid”: “706de50e-075e-11ea-9909-7cd30adb2d50:1-100298,d2aee377-0e66-11ea-b85a-7cd30aeb1510:1-17523587701,678e197a-f60e-11e9-94f9-b8599f37f3d0:1-264381”,
“relaySubDir”: “d2aee377-0e66-11ea-b85a-7cd30aeb1510.000001”,
“relayBinlog”: “(mysql-bin.024271, 4)”,
“relayBinlogGtid”: “706de50e-075e-11ea-9909-7cd30adb2d50:1-100298,d2aee377-0e66-11ea-b85a-7cd30aeb1510:1-17514179562”,
“relayCatchUpMaster”: false,
“stage”: “Paused”,
“result”: {
“isCanceled”: false,
“errors”: [
{
“Type”: “UnknownError”,
“msg”: “[code=30015:class=relay-unit:scope=upstream:level=high] TCPReader get relay event with error: ERROR 1236 (HY000): The slave is connecting using CHANGE MASTER TO MASTER_AUTO_POSITION = 1, but the master has purged binary logs containing GTIDs that the slave requires.\ngithub.com/pingcap/dm/pkg/terror.(*Error).Delegate\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/terror/terror.go:267\ngithub.com/pingcap/dm/pkg/binlog/reader.(*TCPReader).GetEvent\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/binlog/reader/tcp.go:151\ngithub.com/pingcap/dm/relay/reader.(*reader).GetEvent\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/reader/reader.go:144\ngithub.com/pingcap/dm/relay.(*Relay).handleEvents\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/relay.go:382\ngithub.com/pingcap/dm/relay.(*Relay).process\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/relay.go:292\ngithub.com/pingcap/dm/relay.(*Relay).Process\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/relay.go:191\ngithub.com/pingcap/dm/dm/worker.(*realRelayHolder).run\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/dm/worker/relay.go:167\ngithub.com/pingcap/dm/dm/worker.(*realRelayHolder).Start.func1\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/dm/worker/relay.go:143\ runtime.goexit\ \t/usr/local/go/src/runtime/asm_amd64.s:1357”,
“error”: null
}
],
“detail”: null
}
},
“sourceID”: “galaxy”
}
]
}

[errorVerbose="[code=40041:class=dm-worker:scope=internal:level=high] try to upgrade from any older version to {“internal-no”:1,“release-version”:“v1.0.5”}: open DB for /data/dm-worker-00/dm_worker_meta/kv: open kv db file: file missing [file=MANIFEST-000000]

请问你俩是一个问题吗?

这边做过什么操作,可以简述一下,从报错来看是上游 binlog 已经被清理掉了。下游不能继续拉去了。

什么操作都没做啊,主库上binlog看着都还在没清理啊

mysql:
show master status;
show variables like ‘%purged%’;
看下 mysql binlog

tidb:
看下 relay log 拉到多少了。

从上面的信息看,mysql binlog :mysql-bin.024303,353162314,有效的 gitid
:d2aee377-0e66-11ea-b85a-7cd30aeb1510:1-17523587701
dm-worker 拉去的 relay log 为:(mysql-bin.024271, 4),gtid:d2aee377-0e66-11ea-b85a-7cd30aeb1510:1-17523587701

差了不少

改成其他更新的的 再去拉也不行,024271 是改过的尝试重新拉取