【 TiDB 使用环境】生产环境Poc
【 TiDB 版本】6.1.7,MySQL版本5.7.21,dm版本2.0.3
问题描述,使用dm工具进行数据迁移,全量备份阶段,任务状态提示如下报错
{
"result": true,
"msg": "",
"sources": [
{
"result": true,
"msg": "",
"sourceStatus": {
"source": "source-xxxx-xxxx",
"worker": "dm-xx.xx.xx.xx-xxxx",
"result": null,
"relayStatus": null
},
"subTaskStatus": [
{
"name": "task-xxxx-xxxx",
"stage": "Paused",
"unit": "Dump",
"result": {
"isCanceled": false,
"errors": [
{
"ErrCode": 32001,
"ErrClass": "dump-unit",
"ErrScope": "internal",
"ErrLevel": "high",
"Message": "mydumper/dumpling runs with error, with output (may empty): ",
"RawCause": "sql: SELECT COLUMN_NAME,EXTRA FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_SCHEMA=? AND TABLE_NAME=? ORDER BY ORDINAL_POSITION;: driver: bad connection",
"Workaround": ""
}
],
"detail": null
},
"unresolvedDDLLockID": "",
"dump": {
}
}
]
}
]
}
根据官方文档的指导重启任务没法解决问题,重启任务后又会重新做全量备份。dm-worker.log日志报错如下
[2024/04/11 18:12:09.125 +08:00] [ERROR] [dumpling.go:138] ["dump data exits with error"] [task=task-xxxx-xxxx] [unit=dump] ["cost time"=3h55m25.147905887s] [error="ErrCode:32001 ErrClass:\"dump-unit\" ErrScope:\"internal\" ErrLevel:\"high\" Message:\"mydumper/dumpling runs with error, with output (may empty): \" RawCause:\"sql: SELECT COLUMN_NAME,EXTRA FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_SCHEMA=? AND TABLE_NAME=? ORDER BY ORDINAL_POSITION;: driver: bad connection\" "]
[2024/04/11 18:12:09.125 +08:00] [INFO] [subtask.go:289] ["unit process returned"] [subtask=task-xxxx-xxxx] [unit=Dump] [stage=Paused] [status={}]
[2024/04/11 18:12:09.125 +08:00] [ERROR] [subtask.go:308] ["unit process error"] [subtask=task-xxxx-xxxx] [unit=Dump] ["error information"="{\"ErrCode\":32001,\"ErrClass\":\"dump-unit\",\"ErrScope\":\"internal\",\"ErrLevel\":\"high\",\"Message\":\"mydumper/dumpling runs with error, with output (may empty): \",\"RawCause\":\"sql: SELECT COLUMN_NAME,EXTRA FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_SCHEMA=? AND TABLE_NAME=? ORDER BY ORDINAL_POSITION;: driver: bad connection\"}"]
已经排查确认过,MySQL端没有流量访问,不存在有任务去kill连接,且没有类似pt-kill这种工具的任务执行。