同步遇到未知错误，解决不了，如何解决

🪐 TiDB 技术问题 部署&运维管理
1

“msg”: “[code=11038:class=functional:scope=internal:level=high] parse relay log file mysql-bin.006890 from offset 4 in dir /data/tidb_deploy_dm_worker8/relay_log/e29d0ae7-da82-11e9-b77f-00163e10ec8b.000001: parse relay log file mysql-bin.006890 from offset 0 in dir /data/tidb_deploy_dm_worker8/relay_log/e29d0ae7-da82-11e9-b77f-00163e10ec8b.000001: parse relay log file /data/tidb_deploy_dm_worker8/relay_log/e29d0ae7-da82-11e9-b77f-00163e10ec8b.000001/mysql-bin.006890: binlog checksum mismatch, data may be corrupted\ngithub.com/pingcap/dm/pkg/terror.(*Error).Delegate\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/terror/terror.go:267\ngithub.com/pingcap/dm/pkg/streamer.(*BinlogReader).parseFile\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/streamer/reader.go:334\ngithub.com/pingcap/dm/pkg/streamer.(*BinlogReader).parseFileAsPossible\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/streamer/reader.go:241\ngithub.com/pingcap/dm/pkg/streamer.(*BinlogReader).parseDirAsPossible\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/streamer/reader.go:209\ngithub.com/pingcap/dm/pkg/streamer.(*BinlogReader).parseRelay\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/streamer/reader.go:138\ngithub.com/pingcap/dm/pkg/streamer.(*BinlogReader).StartSync.func1\ \t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/streamer/reader.go:111\ runtime.goexit\ \t/usr/local/go/src/runtime/asm_amd64.s:1337”

2

你好，

当前 dm-worker 的 relay log 可能已损坏

  1. 可否详细描述下操作过程，以便排查
  2. 上传下完整的 query-status task-name（文字版），观察下当前的同步进度。
  3. 上传下出错时间点对应的 dm-worker.log
3 
query-status task_hronehrcore
{
    "result": true,
    "msg": "",
    "workers": [
        {
            "result": true,
            "worker": "",
            "msg": "",
            "subTaskStatus": [
                {
                    "name": "task_hronehrcore",
                    "stage": "Paused",
                    "unit": "Sync",
                    "result": {
                        "isCanceled": false,
                        "errors": [
                            {
                                "Type": "UnknownError",
                                "msg": "[code=11038:class=functional:scope=internal:level=high] parse relay log file mysql-bin.006890 from offset 4 in dir /data/tidb_deploy_dm_worker8/relay_log/e29d0ae7-da82-11e9-b77f-00163e10ec8b.000001: parse relay log file mysql-bin.006890 from offset 0 in dir /data/tidb_deploy_dm_worker8/relay_log/e29d0ae7-da82-11e9-b77f-00163e10ec8b.000001: parse relay log file /data/tidb_deploy_dm_worker8/relay_log/e29d0ae7-da82-11e9-b77f-00163e10ec8b.000001/mysql-bin.006890: binlog checksum mismatch, data may be corrupted\
github.com/pingcap/dm/pkg/terror.(*Error).Delegate\
\t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/terror/terror.go:267\
github.com/pingcap/dm/pkg/streamer.(*BinlogReader).parseFile\
\t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/streamer/reader.go:334\
github.com/pingcap/dm/pkg/streamer.(*BinlogReader).parseFileAsPossible\
\t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/streamer/reader.go:241\
github.com/pingcap/dm/pkg/streamer.(*BinlogReader).parseDirAsPossible\
\t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/streamer/reader.go:209\
github.com/pingcap/dm/pkg/streamer.(*BinlogReader).parseRelay\
\t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/streamer/reader.go:138\
github.com/pingcap/dm/pkg/streamer.(*BinlogReader).StartSync.func1\
\t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/streamer/reader.go:111\
runtime.goexit\
\t/usr/local/go/src/runtime/asm_amd64.s:1337",
                                "error": null
                            }
                        ],
                        "detail": null
                    },
                    "unresolvedDDLLockID": "",
                    "sync": {
                        "totalEvents": "6668",
                        "totalTps": "0",
                        "recentTps": "0",
                        "masterBinlog": "(mysql-bin.006923, 41676759)",
                        "masterBinlogGtid": "5d423d66-d8f3-11e8-b61e-7cd30ad3a67a:1-262777612:262777614-266757386,7a168537-c56c-11e9-b9e6-7cd30ae0074a:1-286011790,842b816b-2835-11e8-89d5-7cd30ad3aa7a:1-886989694,95f4f8bb-2835-11e8-aff7-6c92bf5d7f08:1-65929843,274c2218-24f8-11e9-aecd-7cd30ab8a72a:1-8744496,2e5376e4-c57f-11e9-b83c-506b4b270b62:1-11304393",
                        "syncerBinlog": "(mysql-bin|000001.006890, 530096015)",
                        "syncerBinlogGtid": "",
                        "blockingDDLs": [
                        ],
                        "unresolvedGroups": [
                        ],
                        "synced": false
                    }
                }
            ],
            "relayStatus": {
                "masterBinlog": "(mysql-bin.006923, 41676759)",
                "masterBinlogGtid": "95f4f8bb-2835-11e8-aff7-6c92bf5d7f08:1-65929843,274c2218-24f8-11e9-aecd-7cd30ab8a72a:1-8744496,2e5376e4-c57f-11e9-b83c-506b4b270b62:1-11304393,5d423d66-d8f3-11e8-b61e-7cd30ad3a67a:1-262777612:262777614-266757386,7a168537-c56c-11e9-b9e6-7cd30ae0074a:1-286011790,842b816b-2835-11e8-89d5-7cd30ad3aa7a:1-886989694",
                "relaySubDir": "e29d0ae7-da82-11e9-b77f-00163e10ec8b.000001",
                "relayBinlog": "(mysql-bin.006890, 8057)",
                "relayBinlogGtid": "",
                "relayCatchUpMaster": false,
                "stage": "Paused",
                "result": {
                    "isCanceled": false,
                    "errors": [
                        {
                            "Type": "UnknownError",
                            "msg": "[code=30029:class=relay-unit:scope=internal:level=high] handle a potential duplicate event \u0026{Timestamp:1590575331 EventType:WriteRowsEventV2 ServerID:1 EventSize:8177 LogPos:16234 Flags:0} in mysql-bin.006890: check event \u0026{Timestamp:1590575331 EventType:WriteRowsEventV2 ServerID:1 EventSize:8177 LogPos:16234 Flags:0} whether duplicate in /data/tidb_deploy_dm_worker8/relay_log/e29d0ae7-da82-11e9-b77f-00163e10ec8b.000001/mysql-bin.006890: event from 8057 in /data/tidb_deploy_dm_worker8/relay_log/e29d0ae7-da82-11e9-b77f-00163e10ec8b.000001/mysql-bin.006890 diff from passed-in event \u0026{Timestamp:1590575331 EventType:WriteRowsEventV2 ServerID:1 EventSize:8177 LogPos:16234 Flags:0}\
github.com/pingcap/dm/pkg/terror.(*Error).Generatef\
\t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/terror/terror.go:237\
github.com/pingcap/dm/relay/writer.checkIsDuplicateEvent\
\t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/writer/file_util.go:157\
github.com/pingcap/dm/relay/writer.(*FileWriter).handleDuplicateEventsExist\
\t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/writer/file.go:369\
github.com/pingcap/dm/relay/writer.(*FileWriter).handlePotentialHoleOrDuplicate\
\t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/writer/file.go:312\
github.com/pingcap/dm/relay/writer.(*FileWriter).handleEventDefault\
\t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/writer/file.go:287\
github.com/pingcap/dm/relay/writer.(*FileWriter).WriteEvent\
\t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/writer/file.go:130\
github.com/pingcap/dm/relay.(*Relay).handleEvents\
\t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/relay.go:434\
github.com/pingcap/dm/relay.(*Relay).process\
\t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/relay.go:292\
github.com/pingcap/dm/relay.(*Relay).Process\
\t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/relay.go:191\
github.com/pingcap/dm/dm/worker.(*realRelayHolder).run\
\t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/dm/worker/relay.go:164\
github.com/pingcap/dm/dm/worker.(*realRelayHolder).Start.func1\
\t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/dm/worker/relay.go:140\
runtime.goexit\
\t/usr/local/go/src/runtime/asm_amd64.s:1337",
                            "error": null
                        }
                    ],
                    "detail": null
                }
            },
            "sourceID": "mysql-replica-08"
        }
    ]
}
4

你好，

请确认此 task 是否为首次同步，是否与之前 task 中 name 配置项重复，如果重复请确认是否可以修改 task 文件中 remove-meta 为 true 并重新开始同步。

image
image784×574 28.1 KB

5

image
image1578×694 38.7 KB

6

按照上面截图操作下，看是否可以恢复，

task 中的报错为 mysql-bin.006890：16234 和 /data/tidb_deploy_dm_worker8/relay_log/e29d0ae7-da82-11e9-b77f-00163e10ec8b.000001/mysql-bin.006890 的 event ：8057 重复。