dm同步报错handle a potential duplicate event

为提高效率,提问时请提供以下信息,问题描述清晰可优先响应。

  • 【TiDB 版本】:v4.0.0
  • 【问题描述】:remove-meta 已设置为 true,dm对task进程重新同步时报错,dm版本:v1.0.5,具体报错如下:
    » query-status hollycas
    {
    “result”: true,
    “msg”: “”,
    “workers”: [
    {
    “result”: true,
    “worker”: “xxx.xxx.xxx.4:8262”,
    “msg”: “”,
    “subTaskStatus”: [
    {
    “name”: “hollycas”,
    “stage”: “Running”,
    “unit”: “Load”,
    “result”: null,
    “unresolvedDDLLockID”: “”,
    “load”: {
    “finishedBytes”: “600368568504”,
    “totalBytes”: “628884418511”,
    “progress”: “95.47 %”,
    “metaBinlog”: “(mysql-bin.004944, 74104082)”
    }
    }
    ],
    “relayStatus”: {
    “masterBinlog”: “(mysql-bin.004971, 307766955)”,
    “masterBinlogGtid”: “”,
    “relaySubDir”: “89b73e86-22ad-11e9-81b6-70c7f218fe45.000001”,
    “relayBinlog”: “(mysql-bin.004963, 1744)”,
    “relayBinlogGtid”: “”,
    “relayCatchUpMaster”: false,
    “stage”: “Paused”,
    “result”: {
    “isCanceled”: false,
    “errors”: [
    {
    “Type”: “UnknownError”,
    “msg”: “[code=30029:class=relay-unit:scope=internal:level=high] handle a potential duplicate event \u0026{Timestamp:1591553976 EventType:DeleteRowsEventV2 ServerID:11 EventSize:7696 LogPos:9440 Flags:0} in mysql-bin.004963: check event \u0026{Timestamp:1591553976 EventType:DeleteRowsEventV2 ServerID:11 EventSize:7696 LogPos:9440 Flags:0} whether duplicate in /data/dm/deploy/relay_log/89b73e86-22ad-11e9-81b6-70c7f218fe45.000001/mysql-bin.004963: event from 1744 in /data/dm/deploy/relay_log/89b73e86-22ad-11e9-81b6-70c7f218fe45.000001/mysql-bin.004963 diff from passed-in event \u0026{Timestamp:1591553976 EventType:DeleteRowsEventV2 ServerID:11 EventSize:7696 LogPos:9440 Flags:0}\ngithub.com/pingcap/dm/pkg/terror.(*Error).Generatef\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/terror/terror.go:237\ngithub.com/pingcap/dm/relay/writer.checkIsDuplicateEvent\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/writer/file_util.go:157\ github.com/pingcap/dm/relay/writer.(*FileWriter).handleDuplicateEventsExist\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/writer/file.go:369\ngithub.com/pingcap/dm/relay/writer.(*FileWriter).handlePotentialHoleOrDuplicate\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/writer/file.go:312\ngithub.com/pingcap/dm/relay/writer.(*FileWriter).handleEventDefault\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/writer/file.go:287\ngithub.com/pingcap/dm/relay/writer.(*FileWriter).WriteEvent\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/writer/file.go:130\ngithub.com/pingcap/dm/relay.(*Relay).handleEvents\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/relay.go:438\ngithub.com/pingcap/dm/relay.(*Relay).process\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/relay.go:292\ngithub.com/pingcap/dm/relay.(*Relay).Process\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/relay.go:191\ngithub.com/pingcap/dm/dm/worker.(*realRelayHolder).run\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/dm/worker/relay.go:167\ngithub.com/pingcap/dm/dm/worker.(*realRelayHolder).Start.func1\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/dm/worker/relay.go:143\ runtime.goexit\ \t/usr/local/go/src/runtime/asm_amd64.s:1357”,
    “error”: null
    }
    ],
    “detail”: null
    }
    },
    “sourceID”: “mysql-replica-03”
    }
    ]
    }

dm-work.log日志如下:
[2020/06/08 02:32:28.194 +08:00] [ERROR] [relay.go:194] [“process exit”] [component=“relay log”] [error="[code=30029:class=relay-unit:scope=internal:level=high] handle a potential duplicate event &{Timestamp:1591553976 EventType:DeleteRowsEventV2 ServerID:11 EventSize:7696 LogPos:9440 Flags:0} in mysql-bin.004963: check event &{Timestamp:1591553976 EventType:DeleteRowsEventV2 ServerID:11 EventSize:7696 LogPos:9440 Flags:0} whether duplicate in /data/dm/deploy/relay_log/89b73e86-22ad-11e9-81b6-70c7f218fe45.000001/mysql-bin.004963: event from 1744 in /data/dm/deploy/relay_log/89b73e86-22ad-11e9-81b6-70c7f218fe45.000001/mysql-bin.004963 diff from passed-in event &{Timestamp:1591553976 EventType:DeleteRowsEventV2 ServerID:11 EventSize:7696 LogPos:9440 Flags:0}"] [errorVerbose="[code=30029:class=relay-unit:scope=internal:level=high] handle a potential duplicate event &{Timestamp:1591553976 EventType:DeleteRowsEventV2 ServerID:11 EventSize:7696 LogPos:9440 Flags:0} in mysql-bin.004963: check event &{Timestamp:1591553976 EventType:DeleteRowsEventV2 ServerID:11 EventSize:7696 LogPos:9440 Flags:0} whether duplicate in /data/dm/deploy/relay_log/89b73e86-22ad-11e9-81b6-70c7f218fe45.000001/mysql-bin.004963: event from 1744 in /data/dm/deploy/relay_log/89b73e86-22ad-11e9-81b6-70c7f218fe45.000001/mysql-bin.004963 diff from passed-in event &{Timestamp:1591553976 EventType:DeleteRowsEventV2 ServerID:11 EventSize:7696 LogPos:9440 Flags:0}\ngithub.com/pingcap/dm/pkg/terror.(*Error).Generatef\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/terror/terror.go:237\ngithub.com/pingcap/dm/relay/writer.checkIsDuplicateEvent\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/writer/file_util.go:157\ github.com/pingcap/dm/relay/writer.(*FileWriter).handleDuplicateEventsExist\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/writer/file.go:369\ngithub.com/pingcap/dm/relay/writer.(*FileWriter).handlePotentialHoleOrDuplicate\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/writer/file.go:312\ngithub.com/pingcap/dm/relay/writer.(*FileWriter).handleEventDefault\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/writer/file.go:287\ngithub.com/pingcap/dm/relay/writer.(*FileWriter).WriteEvent\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/writer/file.go:130\ngithub.com/pingcap/dm/relay.(*Relay).handleEvents\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/relay.go:438\ngithub.com/pingcap/dm/relay.(*Relay).process\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/relay.go:292\ngithub.com/pingcap/dm/relay.(*Relay).Process\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/relay.go:191\ngithub.com/pingcap/dm/dm/worker.(*realRelayHolder).run\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/dm/worker/relay.go:167\ngithub.com/pingcap/dm/dm/worker.(*realRelayHolder).Start.func1\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/dm/worker/relay.go:143\ runtime.goexit\ \t/usr/local/go/src/runtime/asm_amd64.s:1357"]
[2020/06/08 02:32:28.271 +08:00] [ERROR] [relay.go:173] [“process error”] [component=“relay holder”] [type=“msg:”[code=30029:class=relay-unit:scope=internal:level=high] handle a potential duplicate event &{Timestamp:1591553976 EventType:DeleteRowsEventV2 ServerID:11 EventSize:7696 LogPos:9440 Flags:0} in mysql-bin.004963: check event &{Timestamp:1591553976 EventType:DeleteRowsEventV2 ServerID:11 EventSize:7696 LogPos:9440 Flags:0} whether duplicate in /data/dm/deploy/relay_log/89b73e86-22ad-11e9-81b6-70c7f218fe45.000001/mysql-bin.004963: event from 1744 in /data/dm/deploy/relay_log/89b73e86-22ad-11e9-81b6-70c7f218fe45.000001/mysql-bin.004963 diff from passed-in event &{Timestamp:1591553976 EventType:DeleteRowsEventV2 ServerID:11 EventSize:7696 LogPos:9440 Flags:0}\ngithub.com/pingcap/dm/pkg/terror.(*Error).Generatef\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/terror/terror.go:237\ngithub.com/pingcap/dm/relay/writer.checkIsDuplicateEvent\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/writer/file_util.go:157\ github.com/pingcap/dm/relay/writer.(*FileWriter).handleDuplicateEventsExist\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/writer/file.go:369\ngithub.com/pingcap/dm/relay/writer.(*FileWriter).handlePotentialHoleOrDuplicate\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/writer/file.go:312\ngithub.com/pingcap/dm/relay/writer.(*FileWriter).handleEventDefault\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/writer/file.go:287\ngithub.com/pingcap/dm/relay/writer.(*FileWriter).WriteEvent\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/writer/file.go:130\ngithub.com/pingcap/dm/relay.(*Relay).handleEvents\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/relay.go:438\ngithub.com/pingcap/dm/relay.(*Relay).process\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/relay.go:292\ngithub.com/pingcap/dm/relay.(*Relay).Process\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/relay.go:191\ngithub.com/pingcap/dm/dm/worker.(*realRelayHolder).run\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/dm/worker/relay.go:167\ngithub.com/pingcap/dm/dm/worker.(*realRelayHolder).Start.func1\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/dm/worker/relay.go:143\ runtime.goexit\ \t/usr/local/go/src/runtime/asm_amd64.s:1357" error:<ErrCode:30029 ErrClass:8 ErrScope:3 ErrLevel:3 Message:“handle a potential duplicate event &{Timestamp:1591553976 EventType:DeleteRowsEventV2 ServerID:11 EventSize:7696 LogPos:9440 Flags:0} in mysql-bin.004963: check event &{Timestamp:1591553976 EventType:DeleteRowsEventV2 ServerID:11 EventSize:7696 LogPos:9440 Flags:0} whether duplicate in /data/dm/deploy/relay_log/89b73e86-22ad-11e9-81b6-70c7f218fe45.000001/mysql-bin.004963: event from 1744 in /data/dm/deploy/relay_log/89b73e86-22ad-11e9-81b6-70c7f218fe45.000001/mysql-bin.004963 diff from passed-in event &{Timestamp:1591553976 EventType:DeleteRowsEventV2 ServerID:11 EventSize:7696 LogPos:9440 Flags:0}” > "]

若提问为性能优化、故障排查类问题,请下载脚本运行。终端输出的打印结果,请务必全选并复制粘贴上传。

请参考文档

https://docs.pingcap.com/zh/tidb-data-migration/v1.0/error-handling#relay-处理单元报错-event-from--in--diff-from-passed-in-event--或同步任务中断并包含-get-binlog-error-error-1236-hy000binlog-checksum-mismatch-data-may-be-corrupted-等-binlog-获取或解析失败错误

查看并修复下,多谢。

我的binlog日志配置的是1G轮转,可能由于大事务,导致该binlog文件达到5G,按文中操作步骤执行之后搞定,感谢!

:+1:

此话题已在最后回复的 1 分钟后被自动关闭。不再允许新回复。