tikv节点store一直处于offline状态,无法下线

另一个简单些的方式,br 恢复还原重新来一遍,但需要停业务

方便发下 pd-ctl ,store 命令输出信息么。
还有 pd-ctl 中,config show all 的信息。

当前数据库还能否正常使用?

这个命令,其实可以改为 tomestore,强制修改状态为墓碑。最后手段吧,先别用。

大哥,怎么手动调度leader。
我用了delete删过了,一直处于offline,不会变成墓碑

还能用

在我这个版本已经不支持直接切换到墓碑状态了

./pd-ctl store

{
“count”: 5,
“stores”: [
{
“store”: {
“id”: 72223,
“address”: “pre-test-tikv-4.pre-test-tikv-peer.tidb-cluster.svc:20160”,
“labels”: [
{
“key”: “host”,
“value”: “amj-3”
}
],
“version”: “6.1.0”,
“status_address”: “0.0.0.0:20180”,
“git_hash”: “080d086832ae5ce2495352dccaf8df5d40f30687”,
“start_timestamp”: 1670817813,
“deploy_path”: “/”,
“last_heartbeat”: 1670923125180487806,
“state_name”: “Up”
},
“status”: {
“capacity”: “107.8GiB”,
“available”: “106.6GiB”,
“used_size”: “287.5MiB”,
“leader_count”: 0,
“leader_weight”: 1,
“leader_score”: 0,
“leader_size”: 0,
“region_count”: 0,
“region_weight”: 1,
“region_score”: 0,
“region_size”: 0,
“slow_score”: 1,
“start_ts”: “2022-12-12T04:03:33Z”,
“last_heartbeat_ts”: “2022-12-13T09:18:45.180487806Z”,
“uptime”: “29h15m12.180487806s”
}
},
{
“store”: {
“id”: 1,
“address”: “pre-test-tikv-0.pre-test-tikv-peer.tidb-cluster.svc:20160”,
“labels”: [
{
“key”: “host”,
“value”: “amj-4”
}
],
“version”: “6.1.0”,
“status_address”: “0.0.0.0:20180”,
“git_hash”: “080d086832ae5ce2495352dccaf8df5d40f30687”,
“start_timestamp”: 1670817637,
“deploy_path”: “/”,
“last_heartbeat”: 1670923119967515584,
“state_name”: “Up”
},
“status”: {
“capacity”: “195.9GiB”,
“available”: “192.6GiB”,
“used_size”: “1.384GiB”,
“leader_count”: 4,
“leader_weight”: 1,
“leader_score”: 4,
“leader_size”: 388,
“region_count”: 14,
“region_weight”: 1,
“region_score”: 2250.7984989599654,
“region_size”: 893,
“slow_score”: 1,
“start_ts”: “2022-12-12T04:00:37Z”,
“last_heartbeat_ts”: “2022-12-13T09:18:39.967515584Z”,
“uptime”: “29h18m2.967515584s”
}
},
{
“store”: {
“id”: 4,
“address”: “pre-test-tikv-1.pre-test-tikv-peer.tidb-cluster.svc:20160”,
“labels”: [
{
“key”: “host”,
“value”: “amj-6”
}
],
“version”: “6.1.0”,
“status_address”: “0.0.0.0:20180”,
“git_hash”: “080d086832ae5ce2495352dccaf8df5d40f30687”,
“start_timestamp”: 1670566266,
“deploy_path”: “/”,
“last_heartbeat”: 1670567206781877475,
“state_name”: “Down”
},
“status”: {
“capacity”: “195.9GiB”,
“available”: “192.5GiB”,
“used_size”: “1.382GiB”,
“leader_count”: 8,
“leader_weight”: 1,
“leader_score”: 8,
“leader_size”: 449,
“region_count”: 14,
“region_weight”: 1,
“region_score”: 2251.254041591136,
“region_size”: 893,
“slow_score”: 1,
“start_ts”: “2022-12-09T06:11:06Z”,
“last_heartbeat_ts”: “2022-12-09T06:26:46.781877475Z”,
“uptime”: “15m40.781877475s”
}
},
{
“store”: {
“id”: 5,
“address”: “pre-test-tikv-2.pre-test-tikv-peer.tidb-cluster.svc:20160”,
“labels”: [
{
“key”: “host”,
“value”: “amj-2”
}
],
“version”: “6.1.0”,
“status_address”: “0.0.0.0:20180”,
“git_hash”: “080d086832ae5ce2495352dccaf8df5d40f30687”,
“start_timestamp”: 1670566259,
“deploy_path”: “/”,
“last_heartbeat”: 1670567200007429766,
“state_name”: “Down”
},
“status”: {
“capacity”: “195.9GiB”,
“available”: “192.5GiB”,
“used_size”: “1.384GiB”,
“leader_count”: 2,
“leader_weight”: 1,
“leader_score”: 2,
“leader_size”: 56,
“region_count”: 14,
“region_weight”: 1,
“region_score”: 2251.2701320660067,
“region_size”: 893,
“slow_score”: 1,
“start_ts”: “2022-12-09T06:10:59Z”,
“last_heartbeat_ts”: “2022-12-09T06:26:40.007429766Z”,
“uptime”: “15m41.007429766s”
}
},
{
“store”: {
“id”: 72180,
“address”: “pre-test-tikv-3.pre-test-tikv-peer.tidb-cluster.svc:20160”,
“labels”: [
{
“key”: “host”,
“value”: “amj-5”
}
],
“version”: “6.1.0”,
“status_address”: “0.0.0.0:20180”,
“git_hash”: “080d086832ae5ce2495352dccaf8df5d40f30687”,
“start_timestamp”: 1670817812,
“deploy_path”: “/”,
“last_heartbeat”: 1670923124506811265,
“state_name”: “Up”
},
“status”: {
“capacity”: “107.8GiB”,
“available”: “106.6GiB”,
“used_size”: “287.5MiB”,
“leader_count”: 0,
“leader_weight”: 1,
“leader_score”: 0,
“leader_size”: 0,
“region_count”: 0,
“region_weight”: 1,
“region_score”: 0,
“region_size”: 0,
“slow_score”: 1,
“start_ts”: “2022-12-12T04:03:32Z”,
“last_heartbeat_ts”: “2022-12-13T09:18:44.506811265Z”,
“uptime”: “29h15m12.506811265s”
}
}
]
}

./pd-ctl config show all

{
“client-urls”: “http://0.0.0.0:2379”,
“peer-urls”: “http://0.0.0.0:2380”,
“advertise-client-urls”: “http://pre-test-pd-2.pre-test-pd-peer.tidb-cluster.svc:2379”,
“advertise-peer-urls”: “http://pre-test-pd-2.pre-test-pd-peer.tidb-cluster.svc:2380”,
“name”: “pre-test-pd-2”,
“data-dir”: “/var/lib/pd”,
“force-new-cluster”: false,
“enable-grpc-gateway”: true,
“initial-cluster”: “pre-test-pd-2=http://pre-test-pd-2.pre-test-pd-peer.tidb-cluster.svc:2380”,
“initial-cluster-state”: “new”,
“initial-cluster-token”: “pd-cluster”,
“join”: “”,
“lease”: 3,
“log”: {
“level”: “info”,
“format”: “text”,
“disable-timestamp”: false,
“file”: {
“filename”: “”,
“max-size”: 0,
“max-days”: 0,
“max-backups”: 0
},
“development”: false,
“disable-caller”: false,
“disable-stacktrace”: false,
“disable-error-verbose”: true,
“sampling”: null
},
“tso-save-interval”: “3s”,
“tso-update-physical-interval”: “50ms”,
“enable-local-tso”: false,
“metric”: {
“job”: “pre-test-pd-2”,
“address”: “”,
“interval”: “15s”
},
“schedule”: {
“max-snapshot-count”: 3,
“max-pending-peer-count”: 16,
“max-merge-region-size”: 20,
“max-merge-region-keys”: 200000,
“split-merge-interval”: “1h0m0s”,
“enable-one-way-merge”: “false”,
“enable-cross-table-merge”: “true”,
“patrol-region-interval”: “100ms”,
“max-store-down-time”: “30m0s”,
“max-store-preparing-time”: “48h0m0s”,
“leader-schedule-limit”: 4,
“leader-schedule-policy”: “count”,
“region-schedule-limit”: 2048,
“replica-schedule-limit”: 64,
“merge-schedule-limit”: 8,
“hot-region-schedule-limit”: 4,
“hot-region-cache-hits-threshold”: 3,
“store-limit”: {
“1”: {
“add-peer”: 15,
“remove-peer”: 15
},
“4”: {
“add-peer”: 15,
“remove-peer”: 15
},
“5”: {
“add-peer”: 15,
“remove-peer”: 15
},
“72180”: {
“add-peer”: 15,
“remove-peer”: 15
},
“72223”: {
“add-peer”: 15,
“remove-peer”: 15
}
},
“tolerant-size-ratio”: 0,
“low-space-ratio”: 0.8,
“high-space-ratio”: 0.7,
“region-score-formula-version”: “v2”,
“scheduler-max-waiting-operator”: 5,
“enable-remove-down-replica”: “true”,
“enable-replace-offline-replica”: “true”,
“enable-make-up-replica”: “true”,
“enable-remove-extra-replica”: “true”,
“enable-location-replacement”: “true”,
“enable-debug-metrics”: “false”,
“enable-joint-consensus”: “true”,
“schedulers-v2”: [
{
“type”: “balance-region”,
“args”: null,
“disable”: false,
“args-payload”: “”
},
{
“type”: “balance-leader”,
“args”: null,
“disable”: false,
“args-payload”: “”
},
{
“type”: “hot-region”,
“args”: null,
“disable”: false,
“args-payload”: “”
},
{
“type”: “label”,
“args”: null,
“disable”: false,
“args-payload”: “”
},
{
“type”: “split-bucket”,
“args”: null,
“disable”: false,
“args-payload”: “”
}
],
“schedulers-payload”: {
“balance-hot-region-scheduler”: null,
“balance-leader-scheduler”: {
“batch”: 4,
“ranges”: [
{
“end-key”: “”,
“start-key”: “”
}
]
},
“balance-region-scheduler”: {
“name”: “balance-region-scheduler”,
“ranges”: [
{
“end-key”: “”,
“start-key”: “”
}
]
},
“label-scheduler”: {
“name”: “label-scheduler”,
“ranges”: [
{
“end-key”: “”,
“start-key”: “”
}
]
},
“split-bucket-scheduler”: null
},
“store-limit-mode”: “manual”,
“hot-regions-write-interval”: “10m0s”,
“hot-regions-reserved-days”: 7
},
“replication”: {
“max-replicas”: 3,
“location-labels”: “region,zone,rack,host”,
“strictly-match-label”: “false”,
“enable-placement-rules”: “true”,
“enable-placement-rules-cache”: “false”,
“isolation-level”: “”
},
“pd-server”: {
“use-region-storage”: “true”,
“max-gap-reset-ts”: “24h0m0s”,
“key-type”: “table”,
“runtime-services”: “”,
“metric-storage”: “”,
“dashboard-address”: “http://pre-test-pd-2.pre-test-pd-peer.tidb-cluster.svc:2379”,
“flow-round-by-digit”: 3,
“min-resolved-ts-persistence-interval”: “0s”
},
“cluster-version”: “6.1.0”,
“labels”: {},
“quota-backend-bytes”: “8GiB”,
“auto-compaction-mode”: “periodic”,
“auto-compaction-retention-v2”: “1h”,
“TickInterval”: “500ms”,
“ElectionInterval”: “3s”,
“PreVote”: true,
“max-request-bytes”: 157286400,
“security”: {
“cacert-path”: “”,
“cert-path”: “”,
“key-path”: “”,
“cert-allowed-cn”: null,
“SSLCABytes”: null,
“SSLCertBytes”: null,
“SSLKEYBytes”: null,
“redact-info-log”: false,
“encryption”: {
“data-encryption-method”: “plaintext”,
“data-key-rotation-period”: “168h0m0s”,
“master-key”: {
“type”: “plaintext”,
“key-id”: “”,
“region”: “”,
“endpoint”: “”,
“path”: “”
}
}
},
“label-property”: {},
“WarningMsgs”: null,
“DisableStrictReconfigCheck”: false,
“HeartbeatStreamBindInterval”: “1m0s”,
“LeaderPriorityCheckInterval”: “1m0s”,
“dashboard”: {
“tidb-cacert-path”: “”,
“tidb-cert-path”: “”,
“tidb-key-path”: “”,
“public-path-prefix”: “”,
“internal-proxy”: false,
“enable-telemetry”: true,
“enable-experimental”: false
},
“replication-mode”: {
“replication-mode”: “majority”,
“dr-auto-sync”: {
“label-key”: “”,
“primary”: “”,
“dr”: “”,
“primary-replicas”: 0,
“dr-replicas”: 0,
“wait-store-timeout”: “1m0s”,
“wait-sync-timeout”: “1m0s”,
“wait-async-timeout”: “2m0s”,
“pause-region-split”: “false”
}
}
}

pd store 信息中,为什么是两个 节点为 down 状态,如果进程还存在 下线过程应该是 offline 状态。和你描述不符。

如果在扩容前,3 个 pod ,两个 pod 是 down 状态(相当于失联了 30min 以上),非 offline 状态, 相当于你 3 副本挂了两个副本。不满足 raft 多副本了。实际上你整个集群不可用。

你需要先将 down 的 pod 启动起来。感觉你整个操作流程得先梳理下。

我执行 ./pd-ctl store delete 4或者5 后就变成offline了。因为在offline状态他一直不转移region和leader,我就用

curl -X POST http://127.0.0.1:2379/pd/api/v1/store/${store_id}/state?state=Up
这个命令,看能不能将store恢复为up状态,但是事与愿违,直接就变成down了,我现在起不来故障的pod,因为故障的pod报错显示storeid已经被注册

./pd-ctl store
{
“count”: 5,
“stores”: [
{
“store”: {
“id”: 1,
“address”: “pre-test-tikv-0.pre-test-tikv-peer.tidb-cluster.svc:20160”,
“labels”: [
{
“key”: “host”,
“value”: “amj-4”
}
],
“version”: “6.1.0”,
“status_address”: “0.0.0.0:20180”,
“git_hash”: “080d086832ae5ce2495352dccaf8df5d40f30687”,
“start_timestamp”: 1670817637,
“deploy_path”: “/”,
“last_heartbeat”: 1670923430002349231,
“state_name”: “Up”
},
“status”: {
“capacity”: “195.9GiB”,
“available”: “192.6GiB”,
“used_size”: “1.384GiB”,
“leader_count”: 4,
“leader_weight”: 1,
“leader_score”: 4,
“leader_size”: 388,
“region_count”: 14,
“region_weight”: 1,
“region_score”: 2250.798668582434,
“region_size”: 893,
“slow_score”: 1,
“start_ts”: “2022-12-12T04:00:37Z”,
“last_heartbeat_ts”: “2022-12-13T09:23:50.002349231Z”,
“uptime”: “29h23m13.002349231s”
}
},
{
“store”: {
“id”: 4,
“address”: “pre-test-tikv-1.pre-test-tikv-peer.tidb-cluster.svc:20160”,
“labels”: [
{
“key”: “host”,
“value”: “amj-6”
}
],
“version”: “6.1.0”,
“status_address”: “0.0.0.0:20180”,
“git_hash”: “080d086832ae5ce2495352dccaf8df5d40f30687”,
“start_timestamp”: 1670566266,
“deploy_path”: “/”,
“last_heartbeat”: 1670567206781877475,
“state_name”: “Offline”
},
“status”: {
“capacity”: “195.9GiB”,
“available”: “192.5GiB”,
“used_size”: “1.382GiB”,
“leader_count”: 8,
“leader_weight”: 1,
“leader_score”: 8,
“leader_size”: 449,
“region_count”: 14,
“region_weight”: 1,
“region_score”: 2251.254041591136,
“region_size”: 893,
“slow_score”: 1,
“start_ts”: “2022-12-09T06:11:06Z”,
“last_heartbeat_ts”: “2022-12-09T06:26:46.781877475Z”,
“uptime”: “15m40.781877475s”
}
},
{
“store”: {
“id”: 5,
“address”: “pre-test-tikv-2.pre-test-tikv-peer.tidb-cluster.svc:20160”,
“labels”: [
{
“key”: “host”,
“value”: “amj-2”
}
],
“version”: “6.1.0”,
“status_address”: “0.0.0.0:20180”,
“git_hash”: “080d086832ae5ce2495352dccaf8df5d40f30687”,
“start_timestamp”: 1670566259,
“deploy_path”: “/”,
“last_heartbeat”: 1670567200007429766,
“state_name”: “Offline”
},
“status”: {
“capacity”: “195.9GiB”,
“available”: “192.5GiB”,
“used_size”: “1.384GiB”,
“leader_count”: 2,
“leader_weight”: 1,
“leader_score”: 2,
“leader_size”: 56,
“region_count”: 14,
“region_weight”: 1,
“region_score”: 2251.2701320660067,
“region_size”: 893,
“slow_score”: 1,
“start_ts”: “2022-12-09T06:10:59Z”,
“last_heartbeat_ts”: “2022-12-09T06:26:40.007429766Z”,
“uptime”: “15m41.007429766s”
}
},
{
“store”: {
“id”: 72180,
“address”: “pre-test-tikv-3.pre-test-tikv-peer.tidb-cluster.svc:20160”,
“labels”: [
{
“key”: “host”,
“value”: “amj-5”
}
],
“version”: “6.1.0”,
“status_address”: “0.0.0.0:20180”,
“git_hash”: “080d086832ae5ce2495352dccaf8df5d40f30687”,
“start_timestamp”: 1670817812,
“deploy_path”: “/”,
“last_heartbeat”: 1670923434540157427,
“state_name”: “Up”
},
“status”: {
“capacity”: “107.8GiB”,
“available”: “106.6GiB”,
“used_size”: “287.5MiB”,
“leader_count”: 0,
“leader_weight”: 1,
“leader_score”: 0,
“leader_size”: 0,
“region_count”: 0,
“region_weight”: 1,
“region_score”: 0,
“region_size”: 0,
“slow_score”: 1,
“start_ts”: “2022-12-12T04:03:32Z”,
“last_heartbeat_ts”: “2022-12-13T09:23:54.540157427Z”,
“uptime”: “29h20m22.540157427s”
}
},
{
“store”: {
“id”: 72223,
“address”: “pre-test-tikv-4.pre-test-tikv-peer.tidb-cluster.svc:20160”,
“labels”: [
{
“key”: “host”,
“value”: “amj-3”
}
],
“version”: “6.1.0”,
“status_address”: “0.0.0.0:20180”,
“git_hash”: “080d086832ae5ce2495352dccaf8df5d40f30687”,
“start_timestamp”: 1670817813,
“deploy_path”: “/”,
“last_heartbeat”: 1670923435213926100,
“state_name”: “Up”
},
“status”: {
/ # ./pd-ctl store
{
“count”: 5,
“stores”: [
{
“store”: {
“id”: 72223,
“address”: “pre-test-tikv-4.pre-test-tikv-peer.tidb-cluster.svc:20160”,
“labels”: [
{
“key”: “host”,
“value”: “amj-3”
}
],
“version”: “6.1.0”,
“status_address”: “0.0.0.0:20180”,
“git_hash”: “080d086832ae5ce2495352dccaf8df5d40f30687”,
“start_timestamp”: 1670817813,
“deploy_path”: “/”,
“last_heartbeat”: 1670923795252747665,
“state_name”: “Up”
},
“status”: {
“capacity”: “107.8GiB”,
“available”: “106.6GiB”,
“used_size”: “287.5MiB”,
“leader_count”: 0,
“leader_weight”: 1,
“leader_score”: 0,
“leader_size”: 0,
“region_count”: 0,
“region_weight”: 1,
“region_score”: 0,
“region_size”: 0,
“slow_score”: 1,
“start_ts”: “2022-12-12T04:03:33Z”,
“last_heartbeat_ts”: “2022-12-13T09:29:55.252747665Z”,
“uptime”: “29h26m22.252747665s”
}
},
{
“store”: {
“id”: 1,
“address”: “pre-test-tikv-0.pre-test-tikv-peer.tidb-cluster.svc:20160”,
“labels”: [
{
“key”: “host”,
“value”: “amj-4”
}
],
“version”: “6.1.0”,
“status_address”: “0.0.0.0:20180”,
“git_hash”: “080d086832ae5ce2495352dccaf8df5d40f30687”,
“start_timestamp”: 1670817637,
“deploy_path”: “/”,
“last_heartbeat”: 1670923800046910405,
“state_name”: “Up”
},
“status”: {
“capacity”: “195.9GiB”,
“available”: “192.6GiB”,
“used_size”: “1.384GiB”,
“leader_count”: 4,
“leader_weight”: 1,
“leader_score”: 4,
“leader_size”: 388,
“region_count”: 14,
“region_weight”: 1,
“region_score”: 2250.798646523492,
“region_size”: 893,
“slow_score”: 1,
“start_ts”: “2022-12-12T04:00:37Z”,
“last_heartbeat_ts”: “2022-12-13T09:30:00.046910405Z”,
“uptime”: “29h29m23.046910405s”
}
},
{
“store”: {
“id”: 4,
“address”: “pre-test-tikv-1.pre-test-tikv-peer.tidb-cluster.svc:20160”,
“labels”: [
{
“key”: “host”,
“value”: “amj-6”
}
],
“version”: “6.1.0”,
“status_address”: “0.0.0.0:20180”,
“git_hash”: “080d086832ae5ce2495352dccaf8df5d40f30687”,
“start_timestamp”: 1670566266,
“deploy_path”: “/”,
“last_heartbeat”: 1670567206781877475,
“state_name”: “Offline”
},
“status”: {
“capacity”: “195.9GiB”,
“available”: “192.5GiB”,
“used_size”: “1.382GiB”,
“leader_count”: 8,
“leader_weight”: 1,
“leader_score”: 8,
“leader_size”: 449,
“region_count”: 14,
“region_weight”: 1,
“region_score”: 2251.254041591136,
“region_size”: 893,
“slow_score”: 1,
“start_ts”: “2022-12-09T06:11:06Z”,
“last_heartbeat_ts”: “2022-12-09T06:26:46.781877475Z”,
“uptime”: “15m40.781877475s”
}
},
{
“store”: {
“id”: 5,
“address”: “pre-test-tikv-2.pre-test-tikv-peer.tidb-cluster.svc:20160”,
“labels”: [
{
“key”: “host”,
“value”: “amj-2”
}
],
“version”: “6.1.0”,
“status_address”: “0.0.0.0:20180”,
“git_hash”: “080d086832ae5ce2495352dccaf8df5d40f30687”,
“start_timestamp”: 1670566259,
“deploy_path”: “/”,
“last_heartbeat”: 1670567200007429766,
“state_name”: “Offline”
},
“status”: {
“capacity”: “195.9GiB”,
“available”: “192.5GiB”,
“used_size”: “1.384GiB”,
“leader_count”: 2,
“leader_weight”: 1,
“leader_score”: 2,
“leader_size”: 56,
“region_count”: 14,
“region_weight”: 1,
“region_score”: 2251.2701320660067,
“region_size”: 893,
“slow_score”: 1,
“start_ts”: “2022-12-09T06:10:59Z”,
“last_heartbeat_ts”: “2022-12-09T06:26:40.007429766Z”,
“uptime”: “15m41.007429766s”
}
},
{
“store”: {
“id”: 72180,
“address”: “pre-test-tikv-3.pre-test-tikv-peer.tidb-cluster.svc:20160”,
“labels”: [
{
“key”: “host”,
“value”: “amj-5”
}
],
“version”: “6.1.0”,
“status_address”: “0.0.0.0:20180”,
“git_hash”: “080d086832ae5ce2495352dccaf8df5d40f30687”,
“start_timestamp”: 1670817812,
“deploy_path”: “/”,
“last_heartbeat”: 1670923794578865040,
“state_name”: “Up”
},
“status”: {
“capacity”: “107.8GiB”,
“available”: “106.6GiB”,
“used_size”: “287.5MiB”,
“leader_count”: 0,
“leader_weight”: 1,
“leader_score”: 0,
“leader_size”: 0,
“region_count”: 0,
“region_weight”: 1,
“region_score”: 0,
“region_size”: 0,
“slow_score”: 1,
“start_ts”: “2022-12-12T04:03:32Z”,
“last_heartbeat_ts”: “2022-12-13T09:29:54.57886504Z”,
“uptime”: “29h26m22.57886504s”
}
}
]
}


如果 store id 重复加不回来,那 store-4 和 store-5 数据可能已经找不回来,这个取决于之前具体的操作流程。
如果 store-4 和 store-5 的数据可以不要,那就直接从第 2 步开始

数据我不要了,我就要把4和5的store信息删掉,但是从第二步开始后就一直是offline状态

大佬听你的意思是我之前的操作有问题?那么如果我有一个需求,就是重建我目前的三个tikv的pod(因为我的PVC给的太大了,我想对PVC缩容,但是删除正在使用的PVC必须先把pod删掉,我之前的操作就是为了删除PVC,就直接把pod的数量减到0了,然后再起的pod就出现了这些情况,如果正常操作该怎么操作呢)

上面的 curl 强制修改 <— 这个 store id 就回不来了,慎重

如果数据可以不要,可以考虑直接删除集群重建。

不过他这个操作不应该会导致 pod down,不太合理。

大佬,我有两个tidb集群,这个集群重建不要紧,但是,我另外一个集群有同样的问题,我PV和PVC给的太大了,我想从270G缩到110G,我该怎么操作才能在不停止业务的情况下,把这个PV和PVC缩小下来。我之前就是为了删PVC把三个pod直接减到0,然后再提上来就报storeid被注册,我就只能再往上提,提到6个pod,3个up,然后开始删除最原始三个的store信息,但是只删除成功了一个,也就是tikv-0,1和2死都删不掉

临时停一会儿业务也可以,不重建集群就行

你这个操作有问题。理论上应该做不到 pod 缩容为 0

我理解你应该先扩容,然后用 asts 指定节点缩容才对。

如果是重要集群,按照我说的这个方法先测试下。

做之前做下备份。

或者搭建两个集群 做个主从灾备。生产要小心 小心 再小心啊