启动报错：Error: failed to start pd: pd 192.168.127.130:2379 failed to start: timed out waiting for port 2379 to be started after 2m0s

🪐 TiDB 技术问题 部署&运维管理
1

你好！按照官方教程

我的操作系统center os 8.0
配置文件 topo.yaml

#############Global variables are applied to all deployments and used as the default value of

the deployments if a specific deployment value is missing.

global:
user: “tidb”
ssh_port: 22
deploy_dir: “/tidb-deploy”
data_dir: “/tidb-data”

Monitored variables are applied to all the machines.

monitored:
node_exporter_port: 9100
blackbox_exporter_port: 9115

server_configs:
tidb:
log.slow-threshold: 300
tikv:
readpool.storage.use-unified-pool: false
readpool.coprocessor.use-unified-pool: true
pd:
replication.enable-placement-rules: true
tiflash:
logger.level: “info”

pd_servers:

  • host: 192.168.127.130

tidb_servers:

  • host: 192.168.127.130

tikv_servers:

  • host: 192.168.127.130
    port: 20160
    status_port: 20180

  • host: 192.168.127.130
    port: 20161
    status_port: 20181

  • host: 192.168.127.130
    port: 20162
    status_port: 20182

tiflash_servers:

  • host: 192.168.127.130

monitoring_servers:

  • host: 192.168.127.130

grafana_servers:

  • host: 192.168.127.130

错误日志：

2020-10-02T03:37:33.249-0400 INFO Execute command {“command”: “tiup cluster start tidb_gdy2”}

2020-10-02T03:37:33.249-0400 DEBUG Environment variables {“env”: [“TIUP_HOME=/root/.tiup”, “TIUP_WORK_DIR=/home/tidb”, “TIUP_INSTANCE_DATA_DIR=/root/.tiup/data/SCFA6Ds”, “TIUP_COMPONENT_DATA_DIR=/root/.tiup/storage/cluster”, “TIUP_COMPONENT_INSTALL_DIR=/root/.tiup/components/cluster/v1.1.2”, “TIUP_TELEMETRY_STATUS=enable”, “TIUP_TELEMETRY_UUID=653b5bba-734e-4df3-ad6e-e677335e1a4c”, “TIUP_TAG=SCFA6Ds”, “LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=01;05;37;41:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:.tar=01;31:.tgz=01;31:.arc=01;31:.arj=01;31:.taz=01;31:.lha=01;31:.lz4=01;31:.lzh=01;31:.lzma=01;31:.tlz=01;31:.txz=01;31:.tzo=01;31:.t7z=01;31:.zip=01;31:.z=01;31:.dz=01;31:.gz=01;31:.lrz=01;31:.lz=01;31:.lzo=01;31:.xz=01;31:.zst=01;31:.tzst=01;31:.bz2=01;31:.bz=01;31:.tbz=01;31:.tbz2=01;31:.tz=01;31:.deb=01;31:.rpm=01;31:.jar=01;31:.war=01;31:.ear=01;31:.sar=01;31:.rar=01;31:.alz=01;31:.ace=01;31:.zoo=01;31:.cpio=01;31:.7z=01;31:.rz=01;31:.cab=01;31:.wim=01;31:.swm=01;31:.dwm=01;31:.esd=01;31:.jpg=01;35:.jpeg=01;35:.mjpg=01;35:.mjpeg=01;35:.gif=01;35:.bmp=01;35:.pbm=01;35:.pgm=01;35:.ppm=01;35:.tga=01;35:.xbm=01;35:.xpm=01;35:.tif=01;35:.tiff=01;35:.png=01;35:.svg=01;35:.svgz=01;35:.mng=01;35:.pcx=01;35:.mov=01;35:.mpg=01;35:.mpeg=01;35:.m2v=01;35:.mkv=01;35:.webm=01;35:.ogm=01;35:.mp4=01;35:.m4v=01;35:.mp4v=01;35:.vob=01;35:.qt=01;35:.nuv=01;35:.wmv=01;35:.asf=01;35:.rm=01;35:.rmvb=01;35:.flc=01;35:.avi=01;35:.fli=01;35:.flv=01;35:.gl=01;35:.dl=01;35:.xcf=01;35:.xwd=01;35:.yuv=01;35:.cgm=01;35:.emf=01;35:.ogv=01;35:.ogx=01;35:.aac=01;36:.au=01;36:.flac=01;36:.m4a=01;36:.mid=01;36:.midi=01;36:.mka=01;36:.mp3=01;36:.mpc=01;36:.ogg=01;36:.ra=01;36:.wav=01;36:.oga=01;36:.opus=01;36:.spx=01;36:*.xspf=01;36:”, “SSH_CONNECTION=192.168.127.1 51859 192.168.127.130 22”, “LANG=en_US.UTF-8”, “HISTCONTROL=ignoredups”, “HOSTNAME=localhost.localdomain”, “XDG_SESSION_ID=1”, “USER=root”, “SELINUX_ROLE_REQUESTED=”, “PWD=/home/tidb”, “HOME=/root”, “SSH_CLIENT=192.168.127.1 51859 22”, “SELINUX_LEVEL_REQUESTED=”, “SSH_TTY=/dev/pts/0”, “MAIL=/var/spool/mail/root”, “TERM=xterm”, “SHELL=/bin/bash”, “SELINUX_USE_CURRENT_RANGE=”, “SHLVL=1”, “LOGNAME=root”, “DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/0/bus”, “XDG_RUNTIME_DIR=/run/user/0”, “PATH=/root/.tiup/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin”, “HISTSIZE=1000”, “LESSOPEN=||/usr/bin/lesspipe.sh %s”, “_=/root/.tiup/bin/tiup”, “OLDPWD=/root”, “TIUP_MIRRORS=https://tiup-mirrors.pingcap.com”]}

2020-10-02T03:37:33.251-0400 INFO Starting cluster tidb_gdy2…

2020-10-02T03:37:33.252-0400 INFO + [ Serial ] - SSHKeySet: privateKey=/root/.tiup/storage/cluster/clusters/tidb_gdy2/ssh/id_rsa, publicKey=/root/.tiup/storage/cluster/clusters/tidb_gdy2/ssh/id_rsa.pub

2020-10-02T03:37:33.252-0400 DEBUG TaskBegin {“task”: “SSHKeySet: privateKey=/root/.tiup/storage/cluster/clusters/tidb_gdy2/ssh/id_rsa, publicKey=/root/.tiup/storage/cluster/clusters/tidb_gdy2/ssh/id_rsa.pub”}

2020-10-02T03:37:33.252-0400 DEBUG TaskFinish {“task”: “SSHKeySet: privateKey=/root/.tiup/storage/cluster/clusters/tidb_gdy2/ssh/id_rsa, publicKey=/root/.tiup/storage/cluster/clusters/tidb_gdy2/ssh/id_rsa.pub”}

2020-10-02T03:37:33.252-0400 DEBUG TaskBegin {“task”: “UserSSH: user=tidb, host=192.168.127.130\ UserSSH: user=tidb, host=192.168.127.130\ UserSSH: user=tidb, host=192.168.127.130\ UserSSH: user=tidb, host=192.168.127.130\ UserSSH: user=tidb, host=192.168.127.130\ UserSSH: user=tidb, host=192.168.127.130\ UserSSH: user=tidb, host=192.168.127.130\ UserSSH: user=tidb, host=192.168.127.130”}

2020-10-02T03:37:33.252-0400 INFO + [Parallel] - UserSSH: user=tidb, host=192.168.127.130

2020-10-02T03:37:33.252-0400 INFO + [Parallel] - UserSSH: user=tidb, host=192.168.127.130

2020-10-02T03:37:33.252-0400 INFO + [Parallel] - UserSSH: user=tidb, host=192.168.127.130

2020-10-02T03:37:33.252-0400 INFO + [Parallel] - UserSSH: user=tidb, host=192.168.127.130

2020-10-02T03:37:33.252-0400 INFO + [Parallel] - UserSSH: user=tidb, host=192.168.127.130

2020-10-02T03:37:33.252-0400 INFO + [Parallel] - UserSSH: user=tidb, host=192.168.127.130

2020-10-02T03:37:33.252-0400 DEBUG TaskBegin {“task”: “UserSSH: user=tidb, host=192.168.127.130”}

2020-10-02T03:37:33.252-0400 INFO + [Parallel] - UserSSH: user=tidb, host=192.168.127.130

2020-10-02T03:37:33.252-0400 DEBUG TaskBegin {“task”: “UserSSH: user=tidb, host=192.168.127.130”}

2020-10-02T03:37:33.252-0400 DEBUG TaskFinish {“task”: “UserSSH: user=tidb, host=192.168.127.130”}

2020-10-02T03:37:33.252-0400 DEBUG TaskFinish {“task”: “UserSSH: user=tidb, host=192.168.127.130”}

2020-10-02T03:37:33.252-0400 DEBUG TaskBegin {“task”: “UserSSH: user=tidb, host=192.168.127.130”}

2020-10-02T03:37:33.252-0400 DEBUG TaskFinish {“task”: “UserSSH: user=tidb, host=192.168.127.130”}

2020-10-02T03:37:33.252-0400 DEBUG TaskBegin {“task”: “UserSSH: user=tidb, host=192.168.127.130”}

2020-10-02T03:37:33.252-0400 DEBUG TaskFinish {“task”: “UserSSH: user=tidb, host=192.168.127.130”}

2020-10-02T03:37:33.252-0400 DEBUG TaskBegin {“task”: “UserSSH: user=tidb, host=192.168.127.130”}

2020-10-02T03:37:33.252-0400 DEBUG TaskFinish {“task”: “UserSSH: user=tidb, host=192.168.127.130”}

2020-10-02T03:37:33.252-0400 DEBUG TaskBegin {“task”: “UserSSH: user=tidb, host=192.168.127.130”}

2020-10-02T03:37:33.252-0400 DEBUG TaskFinish {“task”: “UserSSH: user=tidb, host=192.168.127.130”}

2020-10-02T03:37:33.252-0400 DEBUG TaskBegin {“task”: “UserSSH: user=tidb, host=192.168.127.130”}

2020-10-02T03:37:33.252-0400 DEBUG TaskFinish {“task”: “UserSSH: user=tidb, host=192.168.127.130”}

2020-10-02T03:37:33.252-0400 INFO + [Parallel] - UserSSH: user=tidb, host=192.168.127.130

2020-10-02T03:37:33.252-0400 DEBUG TaskBegin {“task”: “UserSSH: user=tidb, host=192.168.127.130”}

2020-10-02T03:37:33.252-0400 DEBUG TaskFinish {“task”: “UserSSH: user=tidb, host=192.168.127.130”}

2020-10-02T03:37:33.252-0400 DEBUG TaskFinish {“task”: “UserSSH: user=tidb, host=192.168.127.130\ UserSSH: user=tidb, host=192.168.127.130\ UserSSH: user=tidb, host=192.168.127.130\ UserSSH: user=tidb, host=192.168.127.130\ UserSSH: user=tidb, host=192.168.127.130\ UserSSH: user=tidb, host=192.168.127.130\ UserSSH: user=tidb, host=192.168.127.130\ UserSSH: user=tidb, host=192.168.127.130”}

2020-10-02T03:37:33.252-0400 INFO + [ Serial ] - StartCluster

2020-10-02T03:37:33.253-0400 DEBUG TaskBegin {“task”: “StartCluster”}

2020-10-02T03:37:33.253-0400 INFO Starting component pd

2020-10-02T03:37:33.253-0400 INFO Starting instance pd 192.168.127.130:2379

2020-10-02T03:37:34.276-0400 INFO SSHCommand {“host”: “192.168.127.130”, “port”: “22”, “cmd”: “export LANG=C; PATH=$PATH:/usr/bin:/usr/sbin sudo -H bash -c “systemctl daemon-reload && systemctl start pd-2379.service && systemctl enable pd-2379.service””, “stdout”: “”, “stderr”: “”}

2020-10-02T03:37:34.499-0400 INFO SSHCommand {“host”: “192.168.127.130”, “port”: “22”, “cmd”: “export LANG=C; PATH=$PATH:/usr/bin:/usr/sbin ss -ltn”, “stdout”: “State Recv-Q Send-Q Local Address:Port Peer Address:Port \ LISTEN 0 128 0.0.0.0:22 0.0.0.0:* \ LISTEN 0 128 [::]:22 [::]:* \ ”, “stderr”: “”}

2020-10-02T03:37:35.772-0400 INFO SSHCommand {“host”: “192.168.127.130”, “port”: “22”, “cmd”: “export LANG=C; PATH=$PATH:/usr/bin:/usr/sbin ss -ltn”, “stdout”: “State Recv-Q Send-Q Local Address:Port Peer Address:Port \ LISTEN 0 128 0.0.0.0:22 0.0.0.0:* \ LISTEN 0 128 [::]:22 [::]:* \ ”, “stderr”: “”}

2020-10-02T03:37:37.074-0400 INFO SSHCommand {“host”: “192.168.127.130”, “port”: “22”, “cmd”: “export LANG=C; PATH=$PATH:/usr/bin:/usr/sbin ss -ltn”, “stdout”: “State Recv-Q Send-Q Local Address:Port Peer Address:Port \ LISTEN 0 128 0.0.0.0:22 0.0.0.0:* \ LISTEN 0 128 [::]:22 [::]:* \ ”, “stderr”: “”}

2020-10-02T03:37:38.326-0400 INFO SSHCommand {“host”: “192.168.127.130”, “port”: “22”, “cmd”: “export LANG=C; PATH=$PATH:/usr/bin:/usr/sbin ss -ltn”, “stdout”: “State Recv-Q Send-Q Local Address:Port Peer Address:Port \ LISTEN 0 128 0.0.0.0:22 0.0.0.0:* \ LISTEN 0 128 [::]:22 [::]:* \ ”, “stderr”: “”}

2020-10-02T03:37:39.592-0400 INFO SSHCommand {“host”: “192.168.127.130”, “port”: “22”, “cmd”: “export LANG=C; PATH=$PATH:/usr/bin:/usr/sbin ss -ltn”, “stdout”: “State Recv-Q Send-Q Local Address:Port Peer Address:Port \ LISTEN 0 128 0.0.0.0:22 0.0.0.0:* \ LISTEN 0 128 [::]:22 [::]:* \ ”, “stderr”: “”}

2020-10-02T03:37:40.868-0400 INFO SSHCommand {“host”: “192.168.127.130”, “port”: “22”, “cmd”: “export LANG=C; PATH=$PATH:/usr/bin:/usr/sbin ss -ltn”, “stdout”: “State Recv-Q Send-Q Local Address:Port Peer Address:Port \ LISTEN 0 128 0.0.0.0:22 0.0.0.0:* \ LISTEN 0 128 [::]:22 [::]:* \ ”, “stderr”: “”}

2020-10-02T03:37:42.129-0400 INFO SSHCommand {“host”: “192.168.127.130”, “port”: “22”, “cmd”: “export LANG=C; PATH=$PATH:/usr/bin:/usr/sbin ss -ltn”, “stdout”: “State Recv-Q Send-Q Local Address:Port Peer Address:Port \ LISTEN 0 128 0.0.0.0:22 0.0.0.0:* \ LISTEN 0 128 [::]:22 [::]:* \ ”, “stderr”: “”}

2

请大神速度帮忙解决，正在犹豫是否要使用greenplum 6

3

看起来是 PD 服务启动超时，日志报错的意思是说让你检查一下 PD log 看一下为什么启动超时。另外看起来你这台机器是测试场景，可能存在测试环境混乱导致的机器端口、IP、目录冲突等因素。可以通过一下步骤逐一排查

# 登陆到目标主机，检查 PD server 启动的服务的 log 和 系统log 
journal -u pd-2379.service
less /var/log/messages
# 查看 PD log 和 stderr.log
less /tidb-deploy/log/pd.log
less /tidb-deploy/log/pd_stderr.log

注意：
报错关键字会在 log 中出现，tiup 启动日志返回的 pd server 服务启动超时报错