tidb升级v.4.0.3失败

🪐 TiDB 技术问题
1

retry error: operation timed out after 1m0s
tikv xxxxx:20160 failed to start: timed out waiting for port 20160 to be started after 1m0s, please check the log of the instance

Error: failed to upgrade: failed to start xxxxx: tikv xxxxx:20160 failed to start: timed out waiting for port 20160 to be started after 1m0s, please check the log of the instance: timed out waiting for port 20160 to be started after 1m0s

Verbose debug logs has been written to /tidb-data/data/logs/tiup-cluster-debug-2020-07-28-17-41-00.log.
Error: run /home/tidb/.tiup/components/cluster/v1.0.6/tiup-cluster (wd:/home/tidb/.tiup/data/S61jj3O) failed: exit status 1

采用tiup升级

2
  1. 检查下 /tidb-data/data/logs/tiup-cluster-debug-2020-07-28-17-41-00.log
  2. 检查下 xxxxx:20160 的 tikv 日志
3

2020-07-28T17:41:00.818+0800 DEBUG retry error: operation timed out after 1m0s
2020-07-28T17:41:00.818+0800 ERROR tikv x.x.x.x:20160 failed to start: timed out waiting for port 20160 to be started after 1m0s, please check the log of the instance
2020-07-28T17:41:00.818+0800 DEBUG TaskFinish {“task”: “ClusterOperate: operation=UpgradeOperation, options={Roles:[] Nodes:[] Force:false SSHTimeout:5 OptTimeout:60 APITimeout:300 RetainDataRoles:[] RetainDataNodes:[]}”, “error”: “failed to upgrade: failed to start x.x.x.x: \ttikv x.x.x.x:20160 failed to start: timed out waiting for port 20160 to be started after 1m0s, please check the log of the instance: timed out waiting for port 20160 to be started after 1m0s”, “errorVerbose”: “timed out waiting for port 20160 to be started after 1m0s\ngithub.com/pingcap/tiup/pkg/cluster/module.(*WaitFor).Execute\ \tgithub.com/pingcap/tiup@/pkg/cluster/module/wait_for.go:90\ github.com/pingcap/tiup/pkg/cluster/meta.PortStarted\ \tgithub.com/pingcap/tiup@/pkg/cluster/meta/logic.go:112\ github.com/pingcap/tiup/pkg/cluster/meta.(*instance).Ready\ \tgithub.com/pingcap/tiup@/pkg/cluster/meta/logic.go:143\ github.com/pingcap/tiup/pkg/cluster/operation.startInstance\ \tgithub.com/pingcap/tiup@/pkg/cluster/operation/action.go:468\ github.com/pingcap/tiup/pkg/cluster/operation.Upgrade\ \tgithub.com/pingcap/tiup@/pkg/cluster/operation/upgrade.go:103\ github.com/pingcap/tiup/pkg/cluster/task.(*ClusterOperate).Execute\ \tgithub.com/pingcap/tiup@/pkg/cluster/task/action.go:53\ github.com/pingcap/tiup/pkg/cluster/task.(*Serial).Execute\ \tgithub.com/pingcap/tiup@/pkg/cluster/task/task.go:183\ github.com/pingcap/tiup/components/cluster/command.upgrade\ \tgithub.com/pingcap/tiup@/components/cluster/command/upgrade.go:171\ github.com/pingcap/tiup/components/cluster/command.newUpgradeCmd.func1\ \tgithub.com/pingcap/tiup@/components/cluster/command/upgrade.go:49\ github.com/spf13/cobra.(*Command).execute\ \tgithub.com/spf13/cobra@v1.0.0/command.go:842\ github.com/spf13/cobra.(*Command).ExecuteC\ \tgithub.com/spf13/cobra@v1.0.0/command.go:950\ github.com/spf13/cobra.(*Command).Execute\ \tgithub.com/spf13/cobra@v1.0.0/command.go:887\ github.com/pingcap/tiup/components/cluster/command.Execute\ \tgithub.com/pingcap/tiup@/components/cluster/command/root.go:220\ main.main\ \tgithub.com/pingcap/tiup@/components/cluster/main.go:19\ runtime.main\ \truntime/proc.go:203\ runtime.goexit\ \truntime/asm_amd64.s:1357\ \ttikv x.x.x.x:20160 failed to start: timed out waiting for port 20160 to be started after 1m0s, please check the log of the instance\ failed to start x.x.x.x\ failed to upgrade”}
2020-07-28T17:41:00.818+0800 INFO Execute command finished {“code”: 1, “error”: “failed to upgrade: failed to start x.x.x.x: \ttikv x.x.x.x:20160 failed to start: timed out waiting for port 20160 to be started after 1m0s, please check the log of the instance: timed out waiting for port 20160 to be started after 1m0s”, “errorVerbose”: “timed out waiting for port 20160 to be started after 1m0s\ngithub.com/pingcap/tiup/pkg/cluster/module.(*WaitFor).Execute\ \tgithub.com/pingcap/tiup@/pkg/cluster/module/wait_for.go:90\ github.com/pingcap/tiup/pkg/cluster/meta.PortStarted\ \tgithub.com/pingcap/tiup@/pkg/cluster/meta/logic.go:112\ github.com/pingcap/tiup/pkg/cluster/meta.(*instance).Ready\ \tgithub.com/pingcap/tiup@/pkg/cluster/meta/logic.go:143\ github.com/pingcap/tiup/pkg/cluster/operation.startInstance\ \tgithub.com/pingcap/tiup@/pkg/cluster/operation/action.go:468\ github.com/pingcap/tiup/pkg/cluster/operation.Upgrade\ \tgithub.com/pingcap/tiup@/pkg/cluster/operation/upgrade.go:103\ github.com/pingcap/tiup/pkg/cluster/task.(*ClusterOperate).Execute\ \tgithub.com/pingcap/tiup@/pkg/cluster/task/action.go:53\ github.com/pingcap/tiup/pkg/cluster/task.(*Serial).Execute\ \tgithub.com/pingcap/tiup@/pkg/cluster/task/task.go:183\ github.com/pingcap/tiup/components/cluster/command.upgrade\ \tgithub.com/pingcap/tiup@/components/cluster/command/upgrade.go:171\ github.com/pingcap/tiup/components/cluster/command.newUpgradeCmd.func1\ \tgithub.com/pingcap/tiup@/components/cluster/command/upgrade.go:49\ github.com/spf13/cobra.(*Command).execute\ \tgithub.com/spf13/cobra@v1.0.0/command.go:842\ github.com/spf13/cobra.(*Command).ExecuteC\ \tgithub.com/spf13/cobra@v1.0.0/command.go:950\ github.com/spf13/cobra.(*Command).Execute\ \tgithub.com/spf13/cobra@v1.0.0/command.go:887\ github.com/pingcap/tiup/components/cluster/command.Execute\ \tgithub.com/pingcap/tiup@/components/cluster/command/root.go:220\ main.main\ \tgithub.com/pingcap/tiup@/components/cluster/main.go:19\ runtime.main\ \truntime/proc.go:203\ runtime.goexit\ \truntime/asm_amd64.s:1357\ \ttikv x.x.x.x:20160 failed to start: timed out waiting for port 20160 to be started after 1m0s, please check the log of the instance\ failed to start x.x.x.x\ failed to upgrade”}

4

tikv 日志呢？有什么报错吗？