【 TiDB 使用环境】生产
【 TiDB 版本】5.4.0
【遇到的问题】pd无法加入集群
【复现路径】扩容pd副本数
【问题现象及影响】pd无法加入集群
【TiDB Operator 版本】:v1.2.7
【K8s 版本】:v1.15.12
生产环境开启了ipv6 DNS,wget默认使用ipv6,执行失败,没有生成/var/lib/pd/join文件
网络测试结果,不加参数
/ # ping www.baidu.com
ping: bad address 'www.baidu.com'
网络测试结果,使用IPv4
/ # ping -4 www.baidu.com
PING www.baidu.com (14.215.177.39): 56 data bytes
64 bytes from 14.215.177.39: seq=0 ttl=48 time=6.421 ms
初始化脚本执行结果
/ # sh -x /usr/local/bin/pd_start_script.sh
+ set -uo pipefail
+ ANNOTATIONS=/etc/podinfo/annotations
+ '[[' '!' -f /etc/podinfo/annotations ]]
+ source /etc/podinfo/annotations
+ runmode=normal
+ '[[' Xnormal '==' Xdebug ]]
+ POD_NAME=hmerp-pd-2
+ echo hmerp-pd-peer
+ sed s/-pd-peer//
+ cluster_name=hmerp
+ domain=hmerp-pd-2.hmerp-pd-peer.prod.svc
+ discovery_url=hmerp-discovery.prod.svc:10261
+ echo hmerp-pd-2.hmerp-pd-peer.prod.svc:2380
+ tr '\
' ' '
+ sed 's/ //g'
+ base64
+ encoded_domain_url=aG1lcnAtcGQtMi5obWVycC1wZC1wZWVyLnByb2Quc3ZjOjIzODAK
+ elapseTime=0
+ period=1
+ threshold=30
+ true
+ sleep 1
+ elapseTime=1
+ '[[' 1 -ge 30 ]]
+ dig hmerp-pd-2.hmerp-pd-peer.prod.svc A hmerp-pd-2.hmerp-pd-peer.prod.svc AAAA +search +short
+ digRes=10.251.16.22
+ '[' 0 -ne 0 ]
+ '[' -z 10.251.16.22 ]
+ echo 'domain resolve hmerp-pd-2.hmerp-pd-peer.prod.svc success'
domain resolve hmerp-pd-2.hmerp-pd-peer.prod.svc success
+ echo 10.251.16.22
10.251.16.22
+ break
+ ARGS='--data-dir=/var/lib/pd --name=hmerp-pd-2 --peer-urls=http://0.0.0.0:2380 --advertise-peer-urls=http://hmerp-pd-2.hmerp-pd-peer.prod.svc:2380 --client-urls=http://0.0.0.0:2379 --advertise-client-urls=http://hmerp-pd-2.hmerp-pd-peer.prod.svc:2379 --config=/etc/pd/pd.toml '
+ '[[' -f /var/lib/pd/join ]]
+ '[[' '!' -d /var/lib/pd/member/wal ]]
+ wget -qO- -T 3 http://hmerp-discovery.prod.svc:10261/new/aG1lcnAtcGQtMi5obWVycC1wZC1wZWVyLnByb2Quc3ZjOjIzODAK
+ result=
+ echo 'waiting for discovery service to return start args ...'
waiting for discovery service to return start args ...
+ sleep 0
+ wget -qO- -T 3 http://hmerp-discovery.prod.svc:10261/new/aG1lcnAtcGQtMi5obWVycC1wZC1wZWVyLnByb2Quc3ZjOjIzODAK
+ result=
+ echo 'waiting for discovery service to return start args ...'
waiting for discovery service to return start args ...
+ sleep 3
wget 执行结果
/ # wget -qO- -T 3 http://hmerp-discovery.prod.svc:10261/new/aG1lcnAtcGQtMi5obWVycC1wZC1wZWVyLnByb2Quc3ZjOjIzODAK
wget: bad address 'hmerp-discovery.prod.svc:10261'