TIDB集群修改PD的IP地址启动不了

🪐 TiDB 技术问题 部署&运维管理
1

修改集群的pd的ip地址，但是修改之后，启动不了集群，此操作是完全按照 【SOP 系列 12】TiUP 修改集群 IP 、Port 和目录进行修改，也不行

环境说明：我的pd是3节点，分别是192.168.1.182、192.168.1.183、192.168.1.184 修改成192.168.1.192、192.168.1.193、192.168.1.194

1、停止集群 tiup cluster stop tidb-test

2、修改/home/tidb/.tiup/storage/cluster/clusters/tidb-test
[tidb@tidb180 tidb-test]$ ls -ll meta.yaml
-rw-r–r-- 1 tidb tidb 3065 Aug 19 21:44 meta.yaml
[tidb@tidb180 tidb-test]$
中的meta.yaml

3、修改ip地址，重新启动网络服务

4、启动集群报错：
[tidb@tidb180 tidb-test]$ tiup cluster:v1.10.3 reload tidb-test -R pd --force
Starting component cluster: /home/tidb/.tiup/components/cluster/v1.10.3/tiup-cluster reload tidb-test -R pd --force
Will reload the cluster tidb-test with restart policy is true, nodes: , roles: pd.
Do you want to continue? [y/N]:(default=N) Y

  • [ Serial ] - SSHKeySet: privateKey=/home/tidb/.tiup/storage/cluster/clusters/tidb-test/ssh/id_rsa, publicKey=/home/tidb/.tiup/storage/cluster/clusters/tidb-test/ssh/id_rsa.pub
  • [Parallel] - UserSSH: user=tidb, host=192.168.1.186
  • [Parallel] - UserSSH: user=tidb, host=192.168.1.187
  • [Parallel] - UserSSH: user=tidb, host=192.168.1.192
  • [Parallel] - UserSSH: user=tidb, host=192.168.1.180
  • [Parallel] - UserSSH: user=tidb, host=192.168.1.181
  • [Parallel] - UserSSH: user=tidb, host=192.168.1.180
  • [Parallel] - UserSSH: user=tidb, host=192.168.1.185
  • [Parallel] - UserSSH: user=tidb, host=192.168.1.194
  • [Parallel] - UserSSH: user=tidb, host=192.168.1.180
  • [Parallel] - UserSSH: user=tidb, host=192.168.1.193
  • [Parallel] - UserSSH: user=tidb, host=192.168.1.180
  • [ Serial ] - UpdateTopology: cluster=tidb-test
    {“level”:“warn”,“ts”:“2022-08-19T21:53:12.896+0800”,“logger”:“etcd-client”,“caller”:“v3@v3.5.4/retry_interceptor.go:62”,“msg”:“retrying of unary invoker failed”,“target”:“etcd-endpoints://0xc0005bce00/192.168.1.192:2379”,“attempt”:0,“error”:“rpc error: code = DeadlineExceeded desc = latest balancer error: last connection error: connection error: desc = “transport: Error while dialing dial tcp 192.168.1.194:2379: connect: connection refused””}

Error: context deadline exceeded

Verbose debug logs has been written to /home/tidb/.tiup/logs/tiup-cluster-debug-2022-08-19-21-53-13.log

2

192.168.1.192:2379
192.168.1.194:2379

你得检查下网络是否通畅，然后 这几个PD 节点的服务是否还是启动状态
如果不是，可以直接重启

[tidb@localhost ~]$ tiup cluster reload tidb-test
[tidb@localhost ~]$ tiup cluster restart tidb-test

3

您好！
1、网络肯定是通的
2、pd的服务的停止的，因为集群我停止了
3、[tidb@tidb180 tidb-test]$ tiup cluster reload tidb-test
tiup is checking updates for component cluster …
Starting component cluster: /home/tidb/.tiup/components/cluster/v1.10.3/tiup-cluster reload tidb-test
Will reload the cluster tidb-test with restart policy is true, nodes: , roles: .
Do you want to continue? [y/N]:(default=N) Y

  • [ Serial ] - SSHKeySet: privateKey=/home/tidb/.tiup/storage/cluster/clusters/tidb-test/ssh/id_rsa, publicKey=/home/tidb/.tiup/storage/cluster/clusters/tidb-test/ssh/id_rsa.pub
  • [Parallel] - UserSSH: user=tidb, host=192.168.1.186
  • [Parallel] - UserSSH: user=tidb, host=192.168.1.187
  • [Parallel] - UserSSH: user=tidb, host=192.168.1.193
  • [Parallel] - UserSSH: user=tidb, host=192.168.1.180
  • [Parallel] - UserSSH: user=tidb, host=192.168.1.181
  • [Parallel] - UserSSH: user=tidb, host=192.168.1.185
  • [Parallel] - UserSSH: user=tidb, host=192.168.1.180
  • [Parallel] - UserSSH: user=tidb, host=192.168.1.192
  • [Parallel] - UserSSH: user=tidb, host=192.168.1.180
  • [Parallel] - UserSSH: user=tidb, host=192.168.1.180
  • [Parallel] - UserSSH: user=tidb, host=192.168.1.194
  • [ Serial ] - UpdateTopology: cluster=tidb-test
    {“level”:“warn”,“ts”:“2022-08-19T22:10:27.732+0800”,“logger”:“etcd-client”,“caller”:“v3@v3.5.4/retry_interceptor.go:62”,“msg”:“retrying of unary invoker failed”,“target”:“etcd-endpoints://0xc0003be700/192.168.1.192:2379”,“attempt”:0,“error”:“rpc error: code = DeadlineExceeded desc = latest balancer error: last connection error: connection error: desc = “transport: Error while dialing dial tcp 192.168.1.192:2379: connect: connection refused””}

Error: context deadline exceeded
4、tiup cluster restart tidb-test 这个执行到下面这里就卡住，其他都是成功的
Starting component pd
Starting instance 192.168.1.192:2379
Starting instance 192.168.1.193:2379
Starting instance 192.168.1.194:2379

4

网络本身就是通的，请问对于PD的修改是不是哪里还要进行设置（其中meta.yaml 已经进行修改）

[tidb@tidb182 ~]$ ping 192.168.1.194
PING 192.168.1.194 (192.168.1.194) 56(84) bytes of data.
64 bytes from 192.168.1.194: icmp_seq=1 ttl=63 time=0.682 ms
64 bytes from 192.168.1.194: icmp_seq=2 ttl=63 time=0.641 ms
^C
— 192.168.1.194 ping statistics —
2 packets transmitted, 2 received, 0% packet loss, time 1000ms
rtt min/avg/max/mdev = 0.641/0.661/0.682/0.032 ms
[tidb@tidb182 ~]$ ping 192.168.1.193
PING 192.168.1.193 (192.168.1.193) 56(84) bytes of data.
64 bytes from 192.168.1.193: icmp_seq=1 ttl=63 time=0.556 ms
^C
— 192.168.1.193 ping statistics —
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.556/0.556/0.556/0.000 ms
[tidb@tidb182 ~]$ ifconfig
ens192: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.1.192 netmask 255.255.255.255 broadcast 192.168.1.192
inet6 fe80::1cd1:5684:7d8:a460 prefixlen 64 scopeid 0x20
inet6 fe80::df2b:7371:4a97:e1a0 prefixlen 64 scopeid 0x20

5

确认下防火墙吧…

你按照 tiup 的修改流程操作的，不会有啥问题的

6

防火墙是关的，我把IP地址修改回以前的就可以了、防火墙肯定是关掉的【不然为什么ip改成以前的就可以】、所以说是不是 【SOP 系列 12】TiUP 修改集群 IP 、Port 和目录上有为什么其他地方有修改的没有写上。

7

看下 中控机到 这台机器的互信是同的吗？

8

pd节点手动启动Pd run_pd_xx.sh（ip也修改下）
在启动集群试试呢