tiup cluster scale-in 加--force有什么风险吗?

【 TiDB 使用环境】生产环境
【 TiDB 版本】7.2
【复现路径】做过哪些操作出现的问题
【遇到的问题:问题现象及影响】
【资源配置】进入到 TiDB Dashboard -集群信息 (Cluster Info) -主机(Hosts) 截图此页面
【附件:截图/日志/监控】
tidb的pd节点做了扩缩容之后,依然是无法启动,查了网上说需要带–force参数,但执行的时候有安全风险。问下这个风险是啥?

tiup cluster scale-in tidb --node 192.168.209.5:22379 --force


补充,这是当前服务状态的截图

这个是高危操作,会直接强制抹除目标节点的部署目录和数据,谨慎执行。

先确认下在操作的过程中出现了什么问题,根据具体问题来处理

1 个赞

关键是加了–force能不能解决,pd依然是down的问题。

force缩容执行成功了吗?正常是这个节点会被抹除掉的。

带–force的缩容没敢执行,不带–force的缩容、扩容都执行成功了,但这个节点的pd就是无法启动。

风险挺大的

无法启动看日志啊,可以看看systemctl的日志

缩容down掉的节点确实是只能通过–force的方式(我看强制缩容的是PD,并且leader已经完成过切换集群目前状态正常的,我认为影响不是很大),或者我们可以先确认PD宕掉的原因?看看是否可以恢复up状态
可以去DOWN掉的pd目录手动执行一下启动,看看报错是什么。

不过建议先扩容进来一台PD,然后在进行强制缩容

你查看了pd的日志了没,去确认一下

[root@host-192-168-209-5 log]# tail -n 100 pd.log
[2024/04/17 16:31:54.657 +08:00] [INFO] [versioninfo.go:89] [“Welcome to Placement Driver (PD)”]
[2024/04/17 16:31:54.657 +08:00] [INFO] [versioninfo.go:90] [PD] [release-version=v7.1.0]
[2024/04/17 16:31:54.657 +08:00] [INFO] [versioninfo.go:91] [PD] [edition=Community]
[2024/04/17 16:31:54.657 +08:00] [INFO] [versioninfo.go:92] [PD] [git-hash=1ff614d90412396c9ebaad76a30d31e683c34adc]
[2024/04/17 16:31:54.657 +08:00] [INFO] [versioninfo.go:93] [PD] [git-branch=heads/refs/tags/v7.1.0]
[2024/04/17 16:31:54.657 +08:00] [INFO] [versioninfo.go:94] [PD] [utc-build-time=“2023-05-25 02:10:43”]
[2024/04/17 16:31:54.657 +08:00] [INFO] [metricutil.go:86] [“disable Prometheus push client”]
[2024/04/17 16:31:54.657 +08:00] [INFO] [join.go:218] [“failed to open directory, maybe start for the first time”] [error=“open /data/software/tidb-data/tidb/tidb-data/pd-22379/member: no such file or directory”]
[2024/04/17 16:31:54.667 +08:00] [INFO] [server.go:242] [“PD config”] [config=“{"client-urls":"http://0.0.0.0:22379","peer-urls":"http://0.0.0.0:22380","advertise-client-urls":"http://192.168.209.5:22379","advertise-peer-urls":"http://192.168.209.5:22380","name":"pd-1","data-dir":"/data/software/tidb-data/tidb/tidb-data/pd-22379","force-new-cluster":false,"enable-grpc-gateway":true,"initial-cluster":"pd-192.168.209.7-22379=http://192.168.209.7:22380,pd-1=http://192.168.209.5:22380,pd-192.168.209.6-22379=http://192.168.209.6:22380","initial-cluster-state":"existing","initial-cluster-token":"pd-cluster","join":"http://192.168.209.6:22379,http://192.168.209.7:22379","lease":3,"log":{"level":"info","format":"text","disable-timestamp":false,"file":{"filename":"/data/software/tidb-7.1.0/tidb-deploy/pd-22379/log/pd.log","max-size":0,"max-days":0,"max-backups":0},"development":false,"disable-caller":false,"disable-stacktrace":false,"disable-error-verbose":true,"sampling":null,"error-output-path":""},"tso-save-interval":"3s","tso-update-physical-interval":"50ms","enable-local-tso":false,"metric":{"job":"pd-1","address":"","interval":"15s"},"schedule":{"max-snapshot-count":64,"max-pending-peer-count":64,"max-merge-region-size":20,"max-merge-region-keys":0,"split-merge-interval":"1h0m0s","swtich-witness-interval":"1h0m0s","enable-one-way-merge":"false","enable-cross-table-merge":"true","patrol-region-interval":"10ms","max-store-down-time":"30m0s","max-store-preparing-time":"48h0m0s","leader-schedule-limit":4,"leader-schedule-policy":"count","region-schedule-limit":2048,"witness-schedule-limit":4,"replica-schedule-limit":64,"merge-schedule-limit":8,"hot-region-schedule-limit":4,"hot-region-cache-hits-threshold":3,"store-limit":{},"tolerant-size-ratio":0,"low-space-ratio":0.8,"high-space-ratio":0.7,"region-score-formula-version":"v2","scheduler-max-waiting-operator":5,"enable-remove-down-replica":"true","enable-replace-offline-replica":"true","enable-make-up-replica":"true","enable-remove-extra-replica":"true","enable-location-replacement":"true","enable-debug-metrics":"false","enable-joint-consensus":"true","enable-tikv-split-region":"true","schedulers-v2":[{"type":"balance-region","args":null,"disable":false,"args-payload":""},{"type":"balance-leader","args":null,"disable":false,"args-payload":""},{"type":"balance-witness","args":null,"disable":false,"args-payload":""},{"type":"hot-region","args":null,"disable":false,"args-payload":""},{"type":"transfer-witness-leader","args":null,"disable":false,"args-payload":""}],"schedulers-payload":null,"store-limit-mode":"manual","hot-regions-write-interval":"10m0s","hot-regions-reserved-days":7,"enable-diagnostic":"true","enable-witness":"false","slow-store-evicting-affected-store-ratio-threshold":0.3,"store-limit-version":"v1"},"replication":{"max-replicas":3,"location-labels":"","strictly-match-label":"false","enable-placement-rules":"true","enable-placement-rules-cache":"false","isolation-level":""},"pd-server":{"use-region-storage":"true","max-gap-reset-ts":"24h0m0s","key-type":"table","runtime-services":"","metric-storage":"","dashboard-address":"auto","trace-region-flow":"true","flow-round-by-digit":3,"min-resolved-ts-persistence-interval":"1s","server-memory-limit":0,"server-memory-limit-gc-trigger":0.7,"enable-gogc-tuner":"false","gc-tuner-threshold":0.6},"cluster-version":"0.0.0","labels":{},"quota-backend-bytes":"8GiB","auto-compaction-mode":"periodic","auto-compaction-retention-v2":"1h","TickInterval":"500ms","ElectionInterval":"3s","PreVote":true,"max-request-bytes":157286400,"security":{"cacert-path":"","cert-path":"","key-path":"","cert-allowed-cn":null,"SSLCABytes":null,"SSLCertBytes":null,"SSLKEYBytes":null,"redact-info-log":false,"encryption":{"data-encryption-method":"plaintext","data-key-rotation-period":"168h0m0s","master-key":{"type":"plaintext","key-id":"","region":"","endpoint":"","path":""}}},"label-property":null,"WarningMsgs":null,"DisableStrictReconfigCheck":false,"HeartbeatStreamBindInterval":"1m0s","LeaderPriorityCheckInterval":"1m0s","dashboard":{"tidb-cacert-path":"","tidb-cert-path":"","tidb-key-path":"","public-path-prefix":"","internal-proxy":false,"enable-telemetry":false,"enable-experimental":false},"replication-mode":{"replication-mode":"majority","dr-auto-sync":{"label-key":"","primary":"","dr":"","primary-replicas":0,"dr-replicas":0,"wait-store-timeout":"1m0s","pause-region-split":"false"}},"keyspace":{"pre-alloc":null},"controller":{"degraded-mode-wait-duration":"0s","request-unit":{"read-base-cost":0.25,"read-cost-per-byte":0.0000152587890625,"write-base-cost":1,"write-cost-per-byte":0.0009765625,"read-cpu-ms-cost":0.3333333333333333}}}”]
[2024/04/17 16:31:54.673 +08:00] [INFO] [apiutil.go:378] [“register REST path”] [path=/pd/api/v1]
[2024/04/17 16:31:54.673 +08:00] [INFO] [apiutil.go:378] [“register REST path”] [path=/pd/api/v2/]
[2024/04/17 16:31:54.673 +08:00] [INFO] [apiutil.go:378] [“register REST path”] [path=/swagger/]
[2024/04/17 16:31:54.673 +08:00] [INFO] [apiutil.go:378] [“register REST path”] [path=/autoscaling]
[2024/04/17 16:31:54.673 +08:00] [INFO] [distro.go:51] [“using distribution strings”] [strings={}]
[2024/04/17 16:31:54.674 +08:00] [INFO] [apiutil.go:378] [“register REST path”] [path=/dashboard/api/]
[2024/04/17 16:31:54.674 +08:00] [INFO] [apiutil.go:378] [“register REST path”] [path=/dashboard/]
[2024/04/17 16:31:54.674 +08:00] [INFO] [apiutil.go:378] [“register REST path”] [path=/resource-manager/api/v1/]
[2024/04/17 16:31:54.674 +08:00] [INFO] [registry.go:92] [“restful API service registered successfully”] [prefix=pd-1] [service-name=ResourceManager]
[2024/04/17 16:31:54.674 +08:00] [INFO] [registry.go:92] [“restful API service registered successfully”] [prefix=pd-1] [service-name=MetaStorage]
[2024/04/17 16:31:54.675 +08:00] [INFO] [etcd.go:117] [“configuring peer listeners”] [listen-peer-urls=“[http://0.0.0.0:22380]”]
[2024/04/17 16:31:54.675 +08:00] [INFO] [systimemon.go:30] [“start system time monitor”]
[2024/04/17 16:31:54.675 +08:00] [INFO] [etcd.go:127] [“configuring client listeners”] [listen-client-urls=“[http://0.0.0.0:22379]”]
[2024/04/17 16:31:54.675 +08:00] [INFO] [etcd.go:611] [“pprof is enabled”] [path=/debug/pprof]
[2024/04/17 16:31:54.675 +08:00] [INFO] [etcd.go:305] [“starting an etcd server”] [etcd-version=3.4.21] [git-sha=“Not provided (use ./build instead of go build)”] [go-version=go1.20.3] [go-os=linux] [go-arch=amd64] [max-cpu-set=16] [max-cpu-available=16] [member-initialized=false] [name=pd-1] [data-dir=/data/software/tidb-data/tidb/tidb-data/pd-22379] [wal-dir=] [wal-dir-dedicated=] [member-dir=/data/software/tidb-data/tidb/tidb-data/pd-22379/member] [force-new-cluster=false] [heartbeat-interval=500ms] [election-timeout=3s] [initial-election-tick-advance=true] [snapshot-count=100000] [snapshot-catchup-entries=5000] [initial-advertise-peer-urls=“[http://192.168.209.5:22380]”] [listen-peer-urls=“[http://0.0.0.0:22380]”] [advertise-client-urls=“[http://192.168.209.5:22379]”] [listen-client-urls=“[http://0.0.0.0:22379]”] [listen-metrics-urls=“”] [cors=“[]“] [host-whitelist=”[]”] [initial-cluster=“pd-192.168.209.6-22379=http://192.168.209.6:22380,pd-192.168.209.7-22379=http://192.168.209.7:22380,pd-1=http://192.168.209.5:22380”] [initial-cluster-state=existing] [initial-cluster-token=pd-cluster] [quota-backend-bytes=8589934592] [max-request-bytes=157286400] [max-concurrent-streams=4294967295] [pre-vote=true] [initial-corrupt-check=false] [corrupt-check-time-interval=0s] [auto-compaction-mode=periodic] [auto-compaction-retention=1h0m0s] [auto-compaction-interval=1h0m0s] [discovery-url=] [discovery-proxy=]
[2024/04/17 16:31:54.675 +08:00] [WARN] [server.go:297] [“exceeded recommended request limit”] [max-request-bytes=157286400] [max-request-size=“157 MB”] [recommended-request-bytes=10485760] [recommended-request-size=“10 MB”]
[2024/04/17 16:31:54.680 +08:00] [INFO] [backend.go:80] [“opened backend db”] [path=/data/software/tidb-data/tidb/tidb-data/pd-22379/member/snap/db] [took=5.299165ms]
[2024/04/17 16:31:54.691 +08:00] [INFO] [raft.go:536] [“starting local member”] [local-member-id=454254c164d8c6cf] [cluster-id=2c0580342200cbf5]
[2024/04/17 16:31:54.691 +08:00] [INFO] [raft.go:1523] [“454254c164d8c6cf switched to configuration voters=()”]
[2024/04/17 16:31:54.691 +08:00] [INFO] [raft.go:706] [“454254c164d8c6cf became follower at term 0”]
[2024/04/17 16:31:54.691 +08:00] [INFO] [raft.go:389] [“newRaft 454254c164d8c6cf [peers: , term: 0, commit: 0, applied: 0, lastindex: 0, lastterm: 0]”]
[2024/04/17 16:31:54.696 +08:00] [WARN] [store.go:1379] [“simple token is not cryptographically signed”]
[2024/04/17 16:31:54.702 +08:00] [INFO] [quota.go:126] [“enabled backend quota”] [quota-name=v3-applier] [quota-size-bytes=8589934592] [quota-size=“8.6 GB”]
[2024/04/17 16:31:54.705 +08:00] [INFO] [pipeline.go:71] [“started HTTP pipelining with remote peer”] [local-member-id=454254c164d8c6cf] [remote-peer-id=359d5f2f171f90a4]
[2024/04/17 16:31:54.705 +08:00] [INFO] [transport.go:294] [“added new remote peer”] [local-member-id=454254c164d8c6cf] [remote-peer-id=359d5f2f171f90a4] [remote-peer-urls=“[http://192.168.209.7:22380]”]
[2024/04/17 16:31:54.705 +08:00] [INFO] [pipeline.go:71] [“started HTTP pipelining with remote peer”] [local-member-id=454254c164d8c6cf] [remote-peer-id=6abe5923309025a4]
[2024/04/17 16:31:54.705 +08:00] [INFO] [transport.go:294] [“added new remote peer”] [local-member-id=454254c164d8c6cf] [remote-peer-id=6abe5923309025a4] [remote-peer-urls=“[http://192.168.209.6:22380]”]
[2024/04/17 16:31:54.705 +08:00] [INFO] [peer.go:128] [“starting remote peer”] [remote-peer-id=359d5f2f171f90a4]
[2024/04/17 16:31:54.705 +08:00] [INFO] [pipeline.go:71] [“started HTTP pipelining with remote peer”] [local-member-id=454254c164d8c6cf] [remote-peer-id=359d5f2f171f90a4]
[2024/04/17 16:31:54.705 +08:00] [INFO] [stream.go:166] [“started stream writer with remote peer”] [local-member-id=454254c164d8c6cf] [remote-peer-id=359d5f2f171f90a4]
[2024/04/17 16:31:54.706 +08:00] [INFO] [peer.go:134] [“started remote peer”] [remote-peer-id=359d5f2f171f90a4]
[2024/04/17 16:31:54.706 +08:00] [INFO] [transport.go:327] [“added remote peer”] [local-member-id=454254c164d8c6cf] [remote-peer-id=359d5f2f171f90a4] [remote-peer-urls=“[http://192.168.209.7:22380]”]
[2024/04/17 16:31:54.707 +08:00] [INFO] [peer.go:128] [“starting remote peer”] [remote-peer-id=6abe5923309025a4]
[2024/04/17 16:31:54.707 +08:00] [INFO] [pipeline.go:71] [“started HTTP pipelining with remote peer”] [local-member-id=454254c164d8c6cf] [remote-peer-id=6abe5923309025a4]
[2024/04/17 16:31:54.707 +08:00] [INFO] [stream.go:406] [“started stream reader with remote peer”] [stream-reader-type=“stream Message”] [local-member-id=454254c164d8c6cf] [remote-peer-id=359d5f2f171f90a4]
[2024/04/17 16:31:54.705 +08:00] [INFO] [stream.go:166] [“started stream writer with remote peer”] [local-member-id=454254c164d8c6cf] [remote-peer-id=359d5f2f171f90a4]
[2024/04/17 16:31:54.708 +08:00] [INFO] [stream.go:406] [“started stream reader with remote peer”] [stream-reader-type=“stream MsgApp v2”] [local-member-id=454254c164d8c6cf] [remote-peer-id=359d5f2f171f90a4]
[2024/04/17 16:31:54.708 +08:00] [INFO] [stream.go:166] [“started stream writer with remote peer”] [local-member-id=454254c164d8c6cf] [remote-peer-id=6abe5923309025a4]
[2024/04/17 16:31:54.708 +08:00] [INFO] [stream.go:166] [“started stream writer with remote peer”] [local-member-id=454254c164d8c6cf] [remote-peer-id=6abe5923309025a4]
[2024/04/17 16:31:54.709 +08:00] [INFO] [peer.go:134] [“started remote peer”] [remote-peer-id=6abe5923309025a4]
[2024/04/17 16:31:54.709 +08:00] [INFO] [stream.go:406] [“started stream reader with remote peer”] [stream-reader-type=“stream MsgApp v2”] [local-member-id=454254c164d8c6cf] [remote-peer-id=6abe5923309025a4]
[2024/04/17 16:31:54.709 +08:00] [INFO] [transport.go:327] [“added remote peer”] [local-member-id=454254c164d8c6cf] [remote-peer-id=6abe5923309025a4] [remote-peer-urls=“[http://192.168.209.6:22380]”]
[2024/04/17 16:31:54.709 +08:00] [INFO] [server.go:816] [“starting etcd server”] [local-member-id=454254c164d8c6cf] [local-server-version=3.4.21] [cluster-version=to_be_decided]
[2024/04/17 16:31:54.709 +08:00] [INFO] [server.go:704] [“starting initial election tick advance”] [election-ticks=6]
[2024/04/17 16:31:54.710 +08:00] [INFO] [peer_status.go:51] [“peer became active”] [peer-id=359d5f2f171f90a4]
[2024/04/17 16:31:54.710 +08:00] [INFO] [stream.go:425] [“established TCP streaming connection with remote peer”] [stream-reader-type=“stream Message”] [local-member-id=454254c164d8c6cf] [remote-peer-id=359d5f2f171f90a4]
[2024/04/17 16:31:54.710 +08:00] [INFO] [stream.go:425] [“established TCP streaming connection with remote peer”] [stream-reader-type=“stream MsgApp v2”] [local-member-id=454254c164d8c6cf] [remote-peer-id=359d5f2f171f90a4]
[2024/04/17 16:31:54.712 +08:00] [INFO] [etcd.go:247] [“now serving peer/client/metrics”] [local-member-id=454254c164d8c6cf] [initial-advertise-peer-urls=“[http://192.168.209.5:22380]”] [listen-peer-urls=“[http://0.0.0.0:22380]”] [advertise-client-urls=“[http://192.168.209.5:22379]”] [listen-client-urls=“[http://0.0.0.0:22379]”] [listen-metrics-urls=“”]
[2024/04/17 16:31:54.713 +08:00] [INFO] [stream.go:406] [“started stream reader with remote peer”] [stream-reader-type=“stream Message”] [local-member-id=454254c164d8c6cf] [remote-peer-id=6abe5923309025a4]
[2024/04/17 16:31:54.713 +08:00] [INFO] [etcd.go:585] [“serving peer traffic”] [address=“[::]:22380”]
[2024/04/17 16:31:54.714 +08:00] [INFO] [peer_status.go:51] [“peer became active”] [peer-id=6abe5923309025a4]
[2024/04/17 16:31:54.714 +08:00] [INFO] [stream.go:425] [“established TCP streaming connection with remote peer”] [stream-reader-type=“stream MsgApp v2”] [local-member-id=454254c164d8c6cf] [remote-peer-id=6abe5923309025a4]
[2024/04/17 16:31:54.714 +08:00] [INFO] [stream.go:425] [“established TCP streaming connection with remote peer”] [stream-reader-type=“stream Message”] [local-member-id=454254c164d8c6cf] [remote-peer-id=6abe5923309025a4]
[2024/04/17 16:31:54.759 +08:00] [INFO] [server.go:729] [“initialized peer connections; fast-forwarding election ticks”] [local-member-id=454254c164d8c6cf] [forward-ticks=4] [forward-duration=2s] [election-ticks=6] [election-timeout=3s] [active-remote-members=2]
[2024/04/17 16:31:55.046 +08:00] [INFO] [raft.go:865] [“454254c164d8c6cf [term: 0] received a MsgHeartbeat message with higher term from 359d5f2f171f90a4 [term: 4]”]
[2024/04/17 16:31:55.046 +08:00] [INFO] [raft.go:706] [“454254c164d8c6cf became follower at term 4”]
[2024/04/17 16:31:55.046 +08:00] [INFO] [node.go:327] [“raft.node: 454254c164d8c6cf elected leader 359d5f2f171f90a4 at term 4”]
[2024/04/17 16:32:05.711 +08:00] [WARN] [server.go:2098] [“failed to publish local member to cluster through raft”] [local-member-id=454254c164d8c6cf] [local-member-attributes=“{Name:pd-1 ClientURLs:[http://192.168.209.5:22379]}”] [request-path=/0/members/454254c164d8c6cf/attributes] [publish-timeout=11s] [error=“etcdserver: request timed out, possibly due to connection lost”]
[2024/04/17 16:32:16.711 +08:00] [WARN] [server.go:2098] [“failed to publish local member to cluster through raft”] [local-member-id=454254c164d8c6cf] [local-member-attributes=“{Name:pd-1 ClientURLs:[http://192.168.209.5:22379]}”] [request-path=/0/members/454254c164d8c6cf/attributes] [publish-timeout=11s] [error=“etcdserver: request timed out”]
[2024/04/17 16:32:27.711 +08:00] [WARN] [server.go:2098] [“failed to publish local member to cluster through raft”] [local-member-id=454254c164d8c6cf] [local-member-attributes=“{Name:pd-1 ClientURLs:[http://192.168.209.5:22379]}”] [request-path=/0/members/454254c164d8c6cf/attributes] [publish-timeout=11s] [error=“etcdserver: request timed out”]
[2024/04/17 16:32:38.712 +08:00] [WARN] [server.go:2098] [“failed to publish local member to cluster through raft”] [local-member-id=454254c164d8c6cf] [local-member-attributes=“{Name:pd-1 ClientURLs:[http://192.168.209.5:22379]}”] [request-path=/0/members/454254c164d8c6cf/attributes] [publish-timeout=11s] [error=“etcdserver: request timed out”]
[2024/04/17 16:32:49.713 +08:00] [WARN] [server.go:2098] [“failed to publish local member to cluster through raft”] [local-member-id=454254c164d8c6cf] [local-member-attributes=“{Name:pd-1 ClientURLs:[http://192.168.209.5:22379]}”] [request-path=/0/members/454254c164d8c6cf/attributes] [publish-timeout=11s] [error=“etcdserver: request timed out”]
[2024/04/17 16:33:00.713 +08:00] [WARN] [server.go:2098] [“failed to publish local member to cluster through raft”] [local-member-id=454254c164d8c6cf] [local-member-attributes=“{Name:pd-1 ClientURLs:[http://192.168.209.5:22379]}”] [request-path=/0/members/454254c164d8c6cf/attributes] [publish-timeout=11s] [error=“etcdserver: request timed out”]
[2024/04/17 16:33:11.714 +08:00] [WARN] [server.go:2098] [“failed to publish local member to cluster through raft”] [local-member-id=454254c164d8c6cf] [local-member-attributes=“{Name:pd-1 ClientURLs:[http://192.168.209.5:22379]}”] [request-path=/0/members/454254c164d8c6cf/attributes] [publish-timeout=11s] [error=“etcdserver: request timed out”]
[2024/04/17 16:33:22.715 +08:00] [WARN] [server.go:2098] [“failed to publish local member to cluster through raft”] [local-member-id=454254c164d8c6cf] [local-member-attributes=“{Name:pd-1 ClientURLs:[http://192.168.209.5:22379]}”] [request-path=/0/members/454254c164d8c6cf/attributes] [publish-timeout=11s] [error=“etcdserver: request timed out”]
[2024/04/17 16:33:33.715 +08:00] [WARN] [server.go:2098] [“failed to publish local member to cluster through raft”] [local-member-id=454254c164d8c6cf] [local-member-attributes=“{Name:pd-1 ClientURLs:[http://192.168.209.5:22379]}”] [request-path=/0/members/454254c164d8c6cf/attributes] [publish-timeout=11s] [error=“etcdserver: request timed out”]
[2024/04/17 16:33:44.715 +08:00] [WARN] [server.go:2098] [“failed to publish local member to cluster through raft”] [local-member-id=454254c164d8c6cf] [local-member-attributes=“{Name:pd-1 ClientURLs:[http://192.168.209.5:22379]}”] [request-path=/0/members/454254c164d8c6cf/attributes] [publish-timeout=11s] [error=“etcdserver: request timed out”]
[2024/04/17 16:33:55.716 +08:00] [WARN] [server.go:2098] [“failed to publish local member to cluster through raft”] [local-member-id=454254c164d8c6cf] [local-member-attributes=“{Name:pd-1 ClientURLs:[http://192.168.209.5:22379]}”] [request-path=/0/members/454254c164d8c6cf/attributes] [publish-timeout=11s] [error=“etcdserver: request timed out”]
[2024/04/17 16:34:06.717 +08:00] [WARN] [server.go:2098] [“failed to publish local member to cluster through raft”] [local-member-id=454254c164d8c6cf] [local-member-attributes=“{Name:pd-1 ClientURLs:[http://192.168.209.5:22379]}”] [request-path=/0/members/454254c164d8c6cf/attributes] [publish-timeout=11s] [error=“etcdserver: request timed out”]
[2024/04/17 16:34:17.718 +08:00] [WARN] [server.go:2098] [“failed to publish local member to cluster through raft”] [local-member-id=454254c164d8c6cf] [local-member-attributes=“{Name:pd-1 ClientURLs:[http://192.168.209.5:22379]}”] [request-path=/0/members/454254c164d8c6cf/attributes] [publish-timeout=11s] [error=“etcdserver: request timed out”]
[2024/04/17 16:34:28.719 +08:00] [WARN] [server.go:2098] [“failed to publish local member to cluster through raft”] [local-member-id=454254c164d8c6cf] [local-member-attributes=“{Name:pd-1 ClientURLs:[http://192.168.209.5:22379]}”] [request-path=/0/members/454254c164d8c6cf/attributes] [publish-timeout=11s] [error=“etcdserver: request timed out”]
[2024/04/17 16:34:39.719 +08:00] [WARN] [server.go:2098] [“failed to publish local member to cluster through raft”] [local-member-id=454254c164d8c6cf] [local-member-attributes=“{Name:pd-1 ClientURLs:[http://192.168.209.5:22379]}”] [request-path=/0/members/454254c164d8c6cf/attributes] [publish-timeout=11s] [error=“etcdserver: request timed out”]
[2024/04/17 16:34:50.721 +08:00] [WARN] [server.go:2098] [“failed to publish local member to cluster through raft”] [local-member-id=454254c164d8c6cf] [local-member-attributes=“{Name:pd-1 ClientURLs:[http://192.168.209.5:22379]}”] [request-path=/0/members/454254c164d8c6cf/attributes] [publish-timeout=11s] [error=“etcdserver: request timed out”]
[2024/04/17 16:35:01.721 +08:00] [WARN] [server.go:2098] [“failed to publish local member to cluster through raft”] [local-member-id=454254c164d8c6cf] [local-member-attributes=“{Name:pd-1 ClientURLs:[http://192.168.209.5:22379]}”] [request-path=/0/members/454254c164d8c6cf/attributes] [publish-timeout=11s] [error=“etcdserver: request timed out”]
[2024/04/17 16:35:12.722 +08:00] [WARN] [server.go:2098] [“failed to publish local member to cluster through raft”] [local-member-id=454254c164d8c6cf] [local-member-attributes=“{Name:pd-1 ClientURLs:[http://192.168.209.5:22379]}”] [request-path=/0/members/454254c164d8c6cf/attributes] [publish-timeout=11s] [error=“etcdserver: request timed out”]
[2024/04/17 16:35:23.722 +08:00] [WARN] [server.go:2098] [“failed to publish local member to cluster through raft”] [local-member-id=454254c164d8c6cf] [local-member-attributes=“{Name:pd-1 ClientURLs:[http://192.168.209.5:22379]}”] [request-path=/0/members/454254c164d8c6cf/attributes] [publish-timeout=11s] [error=“etcdserver: request timed out”]
[2024/04/17 16:35:34.723 +08:00] [WARN] [server.go:2098] [“failed to publish local member to cluster through raft”] [local-member-id=454254c164d8c6cf] [local-member-attributes=“{Name:pd-1 ClientURLs:[http://192.168.209.5:22379]}”] [request-path=/0/members/454254c164d8c6cf/attributes] [publish-timeout=11s] [error=“etcdserver: request timed out”]
[2024/04/17 16:35:45.724 +08:00] [WARN] [server.go:2098] [“failed to publish local member to cluster through raft”] [local-member-id=454254c164d8c6cf] [local-member-attributes=“{Name:pd-1 ClientURLs:[http://192.168.209.5:22379]}”] [request-path=/0/members/454254c164d8c6cf/attributes] [publish-timeout=11s] [error=“etcdserver: request timed out”]
[2024/04/17 16:35:56.724 +08:00] [WARN] [server.go:2098] [“failed to publish local member to cluster through raft”] [local-member-id=454254c164d8c6cf] [local-member-attributes=“{Name:pd-1 ClientURLs:[http://192.168.209.5:22379]}”] [request-path=/0/members/454254c164d8c6cf/attributes] [publish-timeout=11s] [error=“etcdserver: request timed out”]
[2024/04/17 16:36:07.725 +08:00] [WARN] [server.go:2098] [“failed to publish local member to cluster through raft”] [local-member-id=454254c164d8c6cf] [local-member-attributes=“{Name:pd-1 ClientURLs:[http://192.168.209.5:22379]}”] [request-path=/0/members/454254c164d8c6cf/attributes] [publish-timeout=11s] [error=“etcdserver: request timed out”]
[2024/04/17 16:36:18.726 +08:00] [WARN] [server.go:2098] [“failed to publish local member to cluster through raft”] [local-member-id=454254c164d8c6cf] [local-member-attributes=“{Name:pd-1 ClientURLs:[http://192.168.209.5:22379]}”] [request-path=/0/members/454254c164d8c6cf/attributes] [publish-timeout=11s] [error=“etcdserver: request timed out”]
[2024/04/17 16:36:29.726 +08:00] [WARN] [server.go:2098] [“failed to publish local member to cluster through raft”] [local-member-id=454254c164d8c6cf] [local-member-attributes=“{Name:pd-1 ClientURLs:[http://192.168.209.5:22379]}”] [request-path=/0/members/454254c164d8c6cf/attributes] [publish-timeout=11s] [error=“etcdserver: request timed out”]
[2024/04/17 16:36:40.727 +08:00] [WARN] [server.go:2098] [“failed to publish local member to cluster through raft”] [local-member-id=454254c164d8c6cf] [local-member-attributes=“{Name:pd-1 ClientURLs:[http://192.168.209.5:22379]}”] [request-path=/0/members/454254c164d8c6cf/attributes] [publish-timeout=11s] [error=“etcdserver: request timed out”]
[2024/04/17 16:36:51.728 +08:00] [WARN] [server.go:2098] [“failed to publish local member to cluster through raft”] [local-member-id=454254c164d8c6cf] [local-member-attributes=“{Name:pd-1 ClientURLs:[http://192.168.209.5:22379]}”] [request-path=/0/members/454254c164d8c6cf/attributes] [publish-timeout=11s] [error=“etcdserver: request timed out”]
[2024/04/17 16:36:54.675 +08:00] [FATAL] [main.go:232] [“run server failed”] [error=“[PD:server:ErrCancelStartEtcd]etcd start canceled”] [stack=“main.start\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/pd/cmd/pd-server/main.go:232\nmain.createServerWrapper\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/pd/cmd/pd-server/main.go:147\ngithub.com/spf13/cobra.(*Command).execute\n\t/go/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:846\ngithub.com/spf13/cobra.(*Command).ExecuteC\n\t/go/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:950\ngithub.com/spf13/cobra.(*Command).Execute\n\t/go/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:887\nmain.main\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/pd/cmd/pd-server/main.go:56\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:250”]
[root@host-192-168-209-5 log]#

参照这个看看呢?

报错的原因是这里,出现了FATAL的异常日志。

搜索关键字错误,看到有类似的帖子,可以看看是否有帮助:

目前集群状态各个节点正常吗?pd有几个节点,除了这个外是否都正常?

如果我想干净的清理这个pd,scale-in的时候必须加–force吗?

不加 - force ,也是干净的清理。 - force 只是用于节点失联无法正常缩容的情况。

还是等待缩容执行成功

集群的其他节点的服务状态都是正常的

  • force除非那个节点机器坏了,一般用不上
1 个赞