tidb on k8s tikv : Found raft data set when it should not exist

【 TiDB 使用环境】 测试
【 TiDB 版本】
【复现路径】使用nfs 作为 storageclass tikv 部署出现问题
【遇到的问题:问题现象及影响】
tikv 启动失败
CrashLoopBackOff

【资源配置】
【附件:截图/日志/监控】

[root@tidb06 ~]# kubectl logs basic-tikv-0 -n tidb-cluster
starting tikv-server ...
/tikv-server --pd=http://basic-pd:2379 --advertise-addr=basic-tikv-0.basic-tikv-peer.tidb-cluster.svc:20160 --addr=0.0.0.0:20160 --status-addr=0.0.0.0:20180 --advertise-status-addr=basic-tikv-0.basic-tikv-peer.tidb-cluster.svc:20180 --data-dir=/var/lib/tikv --capacity=0 --config=/etc/tikv/tikv.toml

[2022/12/29 02:07:11.977 +00:00] [INFO] [lib.rs:79] ["Welcome to TiKV"]
[2022/12/29 02:07:11.978 +00:00] [INFO] [lib.rs:84] ["Release Version:   6.1.0"]
[2022/12/29 02:07:11.978 +00:00] [INFO] [lib.rs:84] ["Edition:           Community"]
[2022/12/29 02:07:11.978 +00:00] [INFO] [lib.rs:84] ["Git Commit Hash:   080d086832ae5ce2495352dccaf8df5d40f30687"]
[2022/12/29 02:07:11.978 +00:00] [INFO] [lib.rs:84] ["Git Commit Branch: heads/refs/tags/v6.1.0"]
[2022/12/29 02:07:11.978 +00:00] [INFO] [lib.rs:84] ["UTC Build Time:    Unknown (env var does not exist when building)"]
[2022/12/29 02:07:11.978 +00:00] [INFO] [lib.rs:84] ["Rust Version:      rustc 1.60.0-nightly (1e12aef3f 2022-02-13)"]
[2022/12/29 02:07:11.978 +00:00] [INFO] [lib.rs:84] ["Enable Features:   jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure"]
[2022/12/29 02:07:11.978 +00:00] [INFO] [lib.rs:84] ["Profile:           dist_release"]
[2022/12/29 02:07:11.979 +00:00] [INFO] [mod.rs:74] ["cgroup quota: memory=Some(9223372036854771712), cpu=None, cores={7, 31, 11, 9, 15, 22, 27, 14, 21, 4, 20, 12, 30, 6, 26, 5, 8, 28, 1, 13, 29, 10, 2, 23, 17, 24, 18, 0, 19, 3, 16, 25}"]
[2022/12/29 02:07:11.995 +00:00] [INFO] [mod.rs:81] ["memory limit in bytes: 34367975424, cpu cores quota: 32"]
[2022/12/29 02:07:11.995 +00:00] [INFO] [config.rs:705] ["kernel parameters"] [value=0] [param=vm.swappiness]
[2022/12/29 02:07:11.995 +00:00] [WARN] [server.rs:1533] ["check: kernel"] [err="kernel parameters net.core.somaxconn got 128, expect 32768"]
[2022/12/29 02:07:11.996 +00:00] [WARN] [server.rs:1533] ["check: kernel"] [err="check_kernel_params failed No such file or directory (os error 2)"]
[2022/12/29 02:07:12.003 +00:00] [INFO] [util.rs:575] ["connecting to PD endpoint"] [endpoints=http://basic-pd:2379]
[2022/12/29 02:07:12.010 +00:00] [INFO] [<unknown>] ["TCP_USER_TIMEOUT is available. TCP_USER_TIMEOUT will be used thereafter"]
[2022/12/29 02:07:12.015 +00:00] [INFO] [util.rs:575] ["connecting to PD endpoint"] [endpoints=http://basic-pd-2.basic-pd-peer.tidb-cluster.svc:2379]
[2022/12/29 02:07:12.027 +00:00] [INFO] [util.rs:575] ["connecting to PD endpoint"] [endpoints=http://basic-pd-0.basic-pd-peer.tidb-cluster.svc:2379]
[2022/12/29 02:07:12.038 +00:00] [INFO] [util.rs:701] ["connected to PD member"] [endpoints=http://basic-pd-0.basic-pd-peer.tidb-cluster.svc:2379]
[2022/12/29 02:07:12.038 +00:00] [INFO] [util.rs:567] ["all PD endpoints are consistent"] [endpoints="[\"http://basic-pd:2379\"]"]
[2022/12/29 02:07:12.041 +00:00] [INFO] [server.rs:369] ["connect to PD cluster"] [cluster_id=7182053587741419436]
[2022/12/29 02:07:12.041 +00:00] [INFO] [config.rs:2041] ["readpool.storage.use-unified-pool is not set, set to true by default"]
[2022/12/29 02:07:12.041 +00:00] [INFO] [config.rs:2064] ["readpool.coprocessor.use-unified-pool is not set, set to true by default"]
[2022/12/29 02:07:12.042 +00:00] [FATAL] [setup.rs:304] ["invalid configuration: Found raft data set when it should not exist."]

参考一下实例,确认一下是否正确的安装 storage-provisioner https://docs.pingcap.com/zh/tidb-in-kubernetes/dev/configure-storage-class#示例

这几篇可以看一下:
【SOP 系列 39】通过 TiOperator 部署 TiDB
【SOP 系列 40 】TiDB Operator部署TiDB集群的监控与告警
【SOP 系列 15】如何在 Kubernetes 上部署 TiDB-Operator (上)
【SOP 系列 15】如何在 Kubernetes 上部署 TiDB-Operator (下)

我这里使用的是 nfs client 的nfs-client-provisioner ,但是没有在同一个命名空间下,这个影响吗?

nfs-client-provisioner-76cfd5698-xwm29   1/1     Running   0              26h

tikv对应的pvc和pv的状态都是啥?

check_kernel_params failed No such file or directory (os error 2)"

直接反馈没找到目录…

我感觉是不支持

nfs就很明确了,nfs的pv删掉后,数据不会删掉。
你的问题明确的就是因为上面说的重用了,但是k8s的pv命名uuid,一般不会重复的啊。每个pv挂载在nfs的不同目录下。你看看是这样的吗?

找个主机mount下nfs,看看目录结构。

我是用 local storageclass 是没有问题的,大概率是您说的这个问题导致的

按照大家建议的方式试一下看看~

此话题已在最后回复的 60 天后被自动关闭。不再允许新回复。