【 TiDB 使用环境】生产\测试环境\ POC
【 TiDB 版本】 5.3.0
【遇到的问题】
【复现路径】做过哪些操作出现的问题
【问题现象及影响】
【附件】
br restore 数据恢复报错
[2022/04/26 14:41:00.348 +08:00] [INFO] [client.go:666] ["[pd] exit tso dispatcher"] [dc-location=global]
[2022/04/26 14:41:00.348 +08:00] [INFO] [collector.go:66] [“Full restore failed summary”] [total-ranges=7942] [ranges-succeed=7942] [ranges-failed=0] [restore-checksum=2h22m37.383990292s] [split-region=6m19.828263084s] [restore-ranges=7831]
[2022/04/26 14:41:00.348 +08:00] [ERROR] [restore.go:34] [“failed to restore”] [error=“other error: Coprocessor task terminated due to exceeding the deadline”] [errorVerbose=“other error: Coprocessor task terminated due to exceeding the deadline\ngithub.com/pingcap/tidb/store/copr.(*copIteratorWorker).handleCopResponse\
\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/br/store/copr/coprocessor.go:913\ngithub.com/pingcap/tidb/store/copr.(*copIteratorWorker).handleTaskOnce\
\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/br/store/copr/coprocessor.go:755\ngithub.com/pingcap/tidb/store/copr.(*copIteratorWorker).handleTask\
\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/br/store/copr/coprocessor.go:645\ngithub.com/pingcap/tidb/store/copr.(*copIteratorWorker).run\
\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/br/store/copr/coprocessor.go:382\
runtime.goexit\
\t/usr/local/go/src/runtime/asm_amd64.s:1371”] [stack=“main.runRestoreCommand\
\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/br/br/cmd/br/restore.go:34\
main.newFullRestoreCommand.func1\
\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/br/br/cmd/br/restore.go:119\ngithub.com/spf13/cobra.(*Command).execute\
\t/nfs/cache/mod/github.com/spf13/cobra@v1.0.0/command.go:842\ngithub.com/spf13/cobra.(*Command).ExecuteC\
\t/nfs/cache/mod/github.com/spf13/cobra@v1.0.0/command.go:950\ngithub.com/spf13/cobra.(*Command).Execute\
\t/nfs/cache/mod/github.com/spf13/cobra@v1.0.0/command.go:887\
main.main\
\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/br/br/cmd/br/main.go:56\
runtime.main\
\t/usr/local/go/src/runtime/proc.go:225”]
[2022/04/26 14:41:00.348 +08:00] [ERROR] [main.go:58] [“br failed”] [error=“other error: Coprocessor task terminated due to exceeding the deadline”] [errorVerbose=“other error: Coprocessor task terminated due to exceeding the deadline\ngithub.com/pingcap/tidb/store/copr.(*copIteratorWorker).handleCopResponse\
\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/br/store/copr/coprocessor.go:913\ngithub.com/pingcap/tidb/store/copr.(*copIteratorWorker).handleTaskOnce\
\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/br/store/copr/coprocessor.go:755\ngithub.com/pingcap/tidb/store/copr.(*copIteratorWorker).handleTask\
\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/br/store/copr/coprocessor.go:645\ngithub.com/pingcap/tidb/store/copr.(*copIteratorWorker).run\
\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/br/store/copr/coprocessor.go:382\
runtime.goexit\
\t/usr/local/go/src/runtime/asm_amd64.s:1371”] [stack=“main.main\
\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/br/br/cmd/br/main.go:58\
runtime.main\
\t/usr/local/go/src/runtime/proc.go:225”]
问题发生在br恢复完成之后，checksum之前
Detail BR log in br-restore.log
Full restore <---------|…> Full restore <------------------------------------------------------------------> 100.00%
[2022/04/26 14:41:00.348 +08:00] [INFO] [collector.go:65] [“Full restore failed summary”] [total-ranges=7942] [ranges-succeed=7942] [ranges-failed=0] [restore-checksum=2h22m37.383990292s] [split-region=6m19.828263084s] [restore-ranges=7831]
Error: other error: Coprocessor task terminated due to exceeding the deadline
