tiup upgrade ssh timeout

depengliu · 2020 年8 月 7 日 08:44

在从v4.0.1升级到v4.0.4的过程中，报错：

[root@tidb1 v4.0.4]# tiup cluster upgrade tidb v4.0.4
Starting component `cluster`: /root/.tiup/components/cluster/v1.0.9/tiup-cluster upgrade tidb v4.0.4
+ [ Serial ] - SSHKeySet: privateKey=/root/.tiup/storage/cluster/clusters/tidb/ssh/id_rsa, publicKey=/root/.tiup/storage/cluster/clusters/tidb/ssh/id_rsa.pub
+ [Parallel] - UserSSH: user=tidb, host=192.168.10.20
+ [Parallel] - UserSSH: user=tidb, host=192.168.10.20
+ [Parallel] - UserSSH: user=tidb, host=192.168.10.20
+ [Parallel] - UserSSH: user=tidb, host=192.168.10.20
+ [Parallel] - UserSSH: user=tidb, host=192.168.10.20
+ [Parallel] - UserSSH: user=tidb, host=192.168.10.20
+ [Parallel] - UserSSH: user=tidb, host=192.168.10.20
+ [Parallel] - UserSSH: user=tidb, host=192.168.10.20
+ [ Serial ] - Download: component=grafana, version=v4.0.4, os=linux, arch=amd64
+ [ Serial ] - Download: component=tikv, version=v4.0.4, os=linux, arch=amd64
+ [ Serial ] - Download: component=tiflash, version=v4.0.4, os=linux, arch=amd64
+ [ Serial ] - Download: component=pd, version=v4.0.4, os=linux, arch=amd64
+ [ Serial ] - Download: component=tidb, version=v4.0.4, os=linux, arch=amd64
+ [ Serial ] - Download: component=prometheus, version=v4.0.4, os=linux, arch=amd64
+ [ Serial ] - BackupComponent: component=grafana, currentVersion=v4.0.1, remote=192.168.10.20:/tidb-deploy/grafana-3000
+ [ Serial ] - BackupComponent: component=tikv, currentVersion=v4.0.1, remote=192.168.10.20:/tidb-deploy/tikv-20161
+ [ Serial ] - BackupComponent: component=pd, currentVersion=v4.0.1, remote=192.168.10.20:/tidb-deploy/pd-2379
+ [ Serial ] - BackupComponent: component=tiflash, currentVersion=v4.0.1, remote=192.168.10.20:/tidb-deploy/tiflash-9000
+ [ Serial ] - BackupComponent: component=tikv, currentVersion=v4.0.1, remote=192.168.10.20:/tidb-deploy/tikv-20160
+ [ Serial ] - BackupComponent: component=tikv, currentVersion=v4.0.1, remote=192.168.10.20:/tidb-deploy/tikv-20162
+ [ Serial ] - BackupComponent: component=tidb, currentVersion=v4.0.1, remote=192.168.10.20:/tidb-deploy/tidb-4000
+ [ Serial ] - BackupComponent: component=prometheus, currentVersion=v4.0.1, remote=192.168.10.20:/tidb-deploy/prometheus-9090
+ [ Serial ] - CopyComponent: component=pd, version=v4.0.4, remote=192.168.10.20:/tidb-deploy/pd-2379 os=linux, arch=amd64
+ [ Serial ] - CopyComponent: component=tiflash, version=v4.0.4, remote=192.168.10.20:/tidb-deploy/tiflash-9000 os=linux, arch=amd64
+ [ Serial ] - CopyComponent: component=tikv, version=v4.0.4, remote=192.168.10.20:/tidb-deploy/tikv-20161 os=linux, arch=amd64
+ [ Serial ] - CopyComponent: component=tikv, version=v4.0.4, remote=192.168.10.20:/tidb-deploy/tikv-20160 os=linux, arch=amd64
+ [ Serial ] - CopyComponent: component=tidb, version=v4.0.4, remote=192.168.10.20:/tidb-deploy/tidb-4000 os=linux, arch=amd64
+ [ Serial ] - CopyComponent: component=tikv, version=v4.0.4, remote=192.168.10.20:/tidb-deploy/tikv-20162 os=linux, arch=amd64
+ [ Serial ] - CopyComponent: component=grafana, version=v4.0.4, remote=192.168.10.20:/tidb-deploy/grafana-3000 os=linux, arch=amd64
+ [ Serial ] - CopyComponent: component=prometheus, version=v4.0.4, remote=192.168.10.20:/tidb-deploy/prometheus-9090 os=linux, arch=amd64
+ [ Serial ] - InitConfig: cluster=tidb, user=tidb, host=192.168.10.20, path=/root/.tiup/storage/cluster/clusters/tidb/config-cache/pd-2379.service, deploy_dir=/tidb-deploy/pd-2379, data_dir=[/tidb-data/pd-2379], log_dir=/tidb-deploy/pd-2379/log, cache_dir=/root/.tiup/storage/cluster/clusters/tidb/config-cache
+ [ Serial ] - InitConfig: cluster=tidb, user=tidb, host=192.168.10.20, path=/root/.tiup/storage/cluster/clusters/tidb/config-cache/tidb-4000.service, deploy_dir=/tidb-deploy/tidb-4000, data_dir=[], log_dir=/tidb-deploy/tidb-4000/log, cache_dir=/root/.tiup/storage/cluster/clusters/tidb/config-cache
+ [ Serial ] - InitConfig: cluster=tidb, user=tidb, host=192.168.10.20, path=/root/.tiup/storage/cluster/clusters/tidb/config-cache/prometheus-9090.service, deploy_dir=/tidb-deploy/prometheus-9090, data_dir=[/tidb-data/prometheus-9090], log_dir=/tidb-deploy/prometheus-9090/log, cache_dir=/root/.tiup/storage/cluster/clusters/tidb/config-cache
+ [ Serial ] - InitConfig: cluster=tidb, user=tidb, host=192.168.10.20, path=/root/.tiup/storage/cluster/clusters/tidb/config-cache/tikv-20160.service, deploy_dir=/tidb-deploy/tikv-20160, data_dir=[/tidb-data/tikv-20160], log_dir=/tidb-deploy/tikv-20160/log, cache_dir=/root/.tiup/storage/cluster/clusters/tidb/config-cache
+ [ Serial ] - InitConfig: cluster=tidb, user=tidb, host=192.168.10.20, path=/root/.tiup/storage/cluster/clusters/tidb/config-cache/tikv-20161.service, deploy_dir=/tidb-deploy/tikv-20161, data_dir=[/tidb-data/tikv-20161], log_dir=/tidb-deploy/tikv-20161/log, cache_dir=/root/.tiup/storage/cluster/clusters/tidb/config-cache
+ [ Serial ] - InitConfig: cluster=tidb, user=tidb, host=192.168.10.20, path=/root/.tiup/storage/cluster/clusters/tidb/config-cache/tikv-20162.service, deploy_dir=/tidb-deploy/tikv-20162, data_dir=[/tidb-data/tikv-20162], log_dir=/tidb-deploy/tikv-20162/log, cache_dir=/root/.tiup/storage/cluster/clusters/tidb/config-cache
+ [ Serial ] - InitConfig: cluster=tidb, user=tidb, host=192.168.10.20, path=/root/.tiup/storage/cluster/clusters/tidb/config-cache/tiflash-9000.service, deploy_dir=/tidb-deploy/tiflash-9000, data_dir=[/tidb-data/tiflash-9000], log_dir=/tidb-deploy/tiflash-9000/log, cache_dir=/root/.tiup/storage/cluster/clusters/tidb/config-cache

Error: stderr: Run Command Timeout!
: executor.ssh.execute_timedout: Execute command over SSH timedout for 'tidb@192.168.10.20:22' {ssh_stderr: Run Command Timeout!
, ssh_stdout: , ssh_command: export LANG=C; PATH=$PATH:/usr/bin:/usr/sbin tar -xzf /tidb-deploy/grafana-3000/bin/grafana-v4.0.4-linux-amd64.tar.gz -C /tidb-deploy/grafana-3000/bin && rm /tidb-deploy/grafana-3000/bin/grafana-v4.0.4-linux-amd64.tar.gz}

Verbose debug logs has been written to /root/.tiup/components/grafana/v4.0.4/logs/tiup-cluster-debug-2020-08-07-16-42-24.log.
Error: run `/root/.tiup/components/cluster/v1.0.9/tiup-cluster` (wd:/root/.tiup/data/S6xyVES) failed: exit status 1

ssh配置确认没有问题，ssh tidb@192.168.10.20 是成功的，但脚本里报错的是 ‘tidb@192.168.10.20:22’ ，感觉是升级脚本里多了":22"端口号这个导致的不成功。

来了老弟 · 2020 年8 月 7 日 08:51

ssh tidb@192.168.10.20 使用的端口是 22 吗。

可以使用 --ssh-timeout 和 --wait-timeout 设置一个较大的值绕过，并且可以在目标服务器执行下这个 command ，看具体耗时多久，有个预期

depengliu · 2020 年8 月 8 日 02:05

谢谢，请关闭此问题，可能是系统异常，重新复制的虚拟机upgrade没有遇到此问题。

GangShen · 2020 年8 月 8 日 02:49

收到，感谢反馈

system · 2022 年10 月 31 日 19:07

此话题已在最后回复的 1 分钟后被自动关闭。不再允许新回复。