tidb k8s模式启动失败

pvc pv 没有绑定tidb的资源了

您是说这个地方吗?


那我是不是在配置文件里面把需要的磁盘改大点就行了

需要没有绑定的pvc

我不是很理解,具体需要怎么做呢,麻烦了

是要手动创建一个pvc吗

describe看下
kubectl describe tidbcluster basic -n tidb-cluster

tidb.yaml (8.2 KB)

发您了,麻烦看下,谢谢

local-volume-provisioner-xxxx这个pod的日志看一下

几个pv是211天前的,把pv清理了,重来一次

pv删了数据不是没了吗?
provisioner日志:


另外我昨天成功跑起来了,发现把tiflash注释掉就能正常启动,tiflash是新加的,之前没有运行,是不是因为启动了tiflash导致tikv启动失败?
这是我集群的配置,麻烦帮忙看下
tidb-cluster-local.yaml (36.5 KB)

kubectl describe tc advanced-tidb
这个看看

Name: advanced-tidb
Namespace: default
Labels:
Annotations:
API Version: pingcap.com/v1alpha1
Kind: TidbCluster
Metadata:
Creation Timestamp: 2022-10-14T08:19:59Z
Generation: 28
Managed Fields:
API Version: pingcap.com/v1alpha1
Fields Type: FieldsV1
fieldsV1:
f:spec:
.:
f:configUpdateStrategy:
f:enableDynamicConfiguration:
f:helper:
.:
f:image:
f:imagePullPolicy:
f:pd:
.:
f:baseImage:
f:maxFailoverCount:
f:mountClusterClientSecret:
f:replicas:
f:requests:
.:
f:storage:
f:service:
.:
f:clusterIP:
f:type:
f:pvReclaimPolicy:
f:ticdc:
.:
f:baseImage:
f:limits:
.:
f:memory:
f:replicas:
f:requests:
.:
f:memory:
f:serviceAccount:
f:version:
f:tidb:
.:
f:baseImage:
f:maxFailoverCount:
f:replicas:
f:service:
.:
f:externalTrafficPolicy:
f:mysqlNodePort:
f:type:
f:tiflash:
.:
f:baseImage:
f:config:
f:limits:
.:
f:memory:
f:maxFailoverCount:
f:replicas:
f:storageClaims:
f:version:
f:tikv:
.:
f:baseImage:
f:config:
f:limits:
.:
f:memory:
f:maxFailoverCount:
f:mountClusterClientSecret:
f:replicas:
f:requests:
.:
f:cpu:
f:memory:
f:storage:
f:timezone:
f:version:
Manager: kubectl
Operation: Update
Time: 2022-10-14T08:19:59Z
API Version: pingcap.com/v1alpha1
Fields Type: FieldsV1
fieldsV1:
f:spec:
f:discovery:
f:enablePVReclaim:
f:pd:
f:config:
f:ticdc:
f:limits:
f:cpu:
f:requests:
f:cpu:
f:tidb:
f:config:
f:tiflash:
f:config:
f:config:
f:limits:
f:cpu:
f:tikv:
f:limits:
f:cpu:
f:tlsCluster:
f:status:
.:
f:clusterID:
f:conditions:
f:pd:
.:
f:image:
f:leader:
.:
f:clientURL:
f:health:
f:id:
f:lastTransitionTime:
f:name:
f:members:
.:
f:advanced-tidb-pd-0:
.:
f:clientURL:
f:health:
f:id:
f:lastTransitionTime:
f:name:
f:advanced-tidb-pd-1:
.:
f:clientURL:
f:health:
f:id:
f:lastTransitionTime:
f:name:
f:phase:
f:statefulSet:
.:
f:collisionCount:
f:currentReplicas:
f:currentRevision:
f:observedGeneration:
f:readyReplicas:
f:replicas:
f:updateRevision:
f:updatedReplicas:
f:synced:
f:pump:
f:ticdc:
f:tidb:
f:tiflash:
f:tikv:
Manager: tidb-controller-manager
Operation: Update
Time: 2022-10-14T09:45:03Z
Resource Version: 99335860
Self Link: /apis/pingcap.com/v1alpha1/namespaces/default/tidbclusters/advanced-tidb
UID: 2779917b-bed4-4d6d-a47a-cdfce97fe561
Spec:
Config Update Strategy: RollingUpdate
Discovery:
Enable Dynamic Configuration: true
Enable PV Reclaim: false
Helper:
Image: busybox:1.34.1
Image Pull Policy: IfNotPresent
Pd:
Base Image: pingcap/pd
Config: [dashboard]
internal-proxy = true

[replication]
enable-placement-rules = true

Max Failover Count:           0
Mount Cluster Client Secret:  true
Replicas:                     2
Requests:
  Storage:  10Gi
Service:
  Cluster IP:     10.97.24.189
  Type:           NodePort

Pv Reclaim Policy: Retain
Ticdc:
Base Image: pingcap/ticdc
Limits:
Cpu: 2
Memory: 2Gi
Replicas: 2
Requests:
Cpu: 1
Memory: 1Gi
Service Account: advanced-tidb-ticdc
Version: v5.4.0
Tidb:
Base Image: pingcap/tidb
Config: [log]
[log.file]
max-backups = 3

[performance]
tcp-keep-alive = true

Max Failover Count:  0
Replicas:            2
Service:
  External Traffic Policy:  Local
  Mysql Node Port:          23587
  Type:                     NodePort

Tiflash:
Base Image: pingcap/tiflash
Config:
Config: [flash]
[flash.flash_cluster]
log = “/data0/logs/flash_cluster_manager.log”

[logger]
count = 10
errorlog = “/data0/logs/error.log”
level = “information”
log = “/data0/logs/server.log”

Limits:
  Cpu:               2
  Memory:            2Gi
Max Failover Count:  3
Replicas:            1
Storage Claims:
  Resources:
    Requests:
      Storage:         100Gi
  Storage Class Name:  local-path
Version:               v5.4.0

Tikv:
Base Image: pingcap/tikv
Config: log-level = “info”

Limits:
  Cpu:                        1
  Memory:                     6Gi
Max Failover Count:           0
Mount Cluster Client Secret:  true
Replicas:                     2
Requests:
  Cpu:      100m
  Memory:   500Mi
  Storage:  100Gi

Timezone: Asia/Shanghai
Tls Cluster:
Version: v5.4.0
Status:
Cluster ID: 7075532306182605996
Conditions:
Last Transition Time: 2022-10-14T08:20:04Z
Last Update Time: 2022-10-14T09:45:03Z
Message: TiKV store(s) are not up
Reason: TiKVStoreNotUp
Status: False
Type: Ready
Pd:
Image: pingcap/pd:v5.4.0
Leader:
Client URL: http://advanced-tidb-pd-0.advanced-tidb-pd-peer.default.svc:2379
Health: true
Id: 17033634296287288362
Last Transition Time: 2022-10-14T09:44:37Z
Name: advanced-tidb-pd-0
Members:
advanced-tidb-pd-0:
Client URL: http://advanced-tidb-pd-0.advanced-tidb-pd-peer.default.svc:2379
Health: true
Id: 17033634296287288362
Last Transition Time: 2022-10-14T09:44:37Z
Name: advanced-tidb-pd-0
advanced-tidb-pd-1:
Client URL: http://advanced-tidb-pd-1.advanced-tidb-pd-peer.default.svc:2379
Health: true
Id: 12507838731813631288
Last Transition Time: 2022-10-14T09:45:03Z
Name: advanced-tidb-pd-1
Phase: Normal
Stateful Set:
Collision Count: 0
Current Replicas: 2
Current Revision: advanced-tidb-pd-69ccd959f9
Observed Generation: 1
Ready Replicas: 2
Replicas: 2
Update Revision: advanced-tidb-pd-69ccd959f9
Updated Replicas: 2
Synced: true
Pump:
Ticdc:
Tidb:
Tiflash:
Tikv:
Events:
Type Reason Age From Message


Warning PDMemberUnhealthy 30m (x2 over 65m) tidb-controller-manager default/advanced-tidb-pd-0(17033634296287288362) is unhealthy
Warning PDMemberUnhealthy 29m (x10 over 99m) tidb-controller-manager default/advanced-tidb-pd-1(12507838731813631288) is unhealthy

这是pd的日志

我看tidb-controller-manager的日志也有问题,和这个有关系吗?:

kubect get tc -A 看一下,
tidb-cluster-local.yaml这个配置文件tiflash storageClaims 都去掉看看能不能起来。

这几个日志都是Pd 没起来报的错

error: error validating “tidb-cluster-tiflash.yaml”: error validating data: ValidationError(TidbCluster.spec.tiflash): missing required field “storageClaims” in com.pingcap.v1alpha1.TidbCluster.spec.tiflash; if you choose to ignore these errors, turn validation off with --validate=false
关了在集群启动的时候就会报错

tc 里 pd 和 tikv 都开启了 mountClusterClientSecret: true, 请参考官网文档同时开启集群组件间 TLS 支持,该步骤会创建报错里提示缺失的 Secret 对象

tidb-controller-manager 的日志报错原因是 tc 的 spec.pd.service.clusterIP 没有用字符串,需要用双引号引起来