【 TiDB 使用环境】生产环境
【 TiDB 版本】7.1.0
【复现路径】无
【遇到的问题:问题现象及影响】
PD节点leader oom,tidb访问pd出现故障。日志:
[2024/01/17 15:30:11.960 +08:00] [WARN] [util.go:163] ["apply request took too long"] [took=182.652093ms] [expected-duration=100ms] [prefix=] [request="header:<ID:15889696433221208426 > txn:<compare:<target:MOD key:\"/tidb/cdc/default/default/changefeed/status/broker-b-broker-bills-enterprise-collection\" mod_revision:7048499856 target:MOD key:\"/tidb/cdc/default/default/changefeed/status/prod-tidb-rd-stat-deal-suffix\" mod_revision:7048499856 target:MOD key:\"/tidb/cdc/default/default/changefeed/status/broker-b-broker-bills-corporate\" mod_revision:7048499856 target:MOD key:\"/tidb/cdc/default/default/changefeed/status/broker-b-broker-bills-realtime\" mod_revision:7048499856 target:MOD key:\"/tidb/cdc/default/default/changefeed/status/broker-b-broker-bills-wxpay-datalake\" mod_revision:7048499856 target:MOD key:\"/tidb/cdc/default/default/changefeed/status/ticdc-v504-rd-payment-orders-many-prd\" mod_revision:7048499857 target:MOD key:\"/tidb/cdc/default/default/changefeed/status/broker-b-broker-bills-alipay\" mod_revision:7048499856 target:MOD key:\"/tidb/cdc/default/default/changefeed/status/ticdc-v504-stat-ads-adi-prd\" mod_revision:7048499856 target:MOD key:\"/tidb/cdc/default/default/changefeed/status/broker-mutex-user-remit-period\" mod_revision:7048499856 target:MOD key:\"/tidb/cdc/default/default/changefeed/status/ticdc-v504-broker-stat-v3-income-record-prd\" mod_revision:7048499856 target:MOD key:\"/tidb/cdc/default/default/changefeed/status/broker-b-broker-bills-enterprise\" mod_revision:7048499856 target:MOD key:\"/tidb/cdc/default/default/changefeed/status/ticdc-v504-rd-broker-bills-many-prod\" mod_revision:7048499857 target:MOD key:\"/tidb/cdc/default/default/changefeed/status/prod-tidb-broker-vol3\" mod_revision:7048499856 target:MOD key:\"/tidb/cdc/default/default/changefeed/status/ticdc-v504-stat-online-bp-many-prod-c\" mod_revision:7048499856 target:MOD key:\"/tidb/cdc/default/default/changefeed/status/broker-b-broker-bills-external\" mod_revision:7048499856 target:VALUE key:\"/tidb/cdc/default/__cdc_meta__/meta/ticdc-delete-etcd-key-count\" value_size:2 > success:<request_put:<key:\"/tidb/cdc/default/default/changefeed/status/broker-b-broker-bills-enterprise-collection\" value_size:130 > request_put:<key:\"/tidb/cdc/default/default/changefeed/status/prod-tidb-rd-stat-deal-suffix\" value_size:130 > request_put:<key:\"/tidb/cdc/default/default/changefeed/status/broker-b-broker-bills-corporate\" value_size:130 > request_put:<key:\"/tidb/cdc/default/default/changefeed/status/broker-b-broker-bills-realtime\" value_size:130 > request_put:<key:\"/tidb/cdc/default/default/changefeed/status/broker-b-broker-bills-wxpay-datalake\" value_size:130 > request_put:<key:\"/tidb/cdc/default/default/changefeed/status/ticdc-v504-rd-payment-orders-many-prd\" value_size:130 > request_put:<key:\"/tidb/cdc/default/default/changefeed/status/broker-b-broker-bills-alipay\" value_size:130 > request_put:<key:\"/tidb/cdc/default/default/changefeed/status/ticdc-v504-stat-ads-adi-prd\" value_size:130 > request_put:<key:\"/tidb/cdc/default/default/changefeed/status/broker-mutex-user-remit-period\" value_size:130 > request_put:<key:\"/tidb/cdc/default/default/changefeed/status/ticdc-v504-broker-stat-v3-income-record-prd\" value_size:130 > request_put:<key:\"/tidb/cdc/default/default/changefeed/status/broker-b-broker-bills-enterprise\" value_size:130 > request_put:<key:\"/tidb/cdc/default/default/changefeed/status/ticdc-v504-rd-broker-bills-many-prod\" value_size:130 > request_put:<key:\"/tidb/cdc/default/default/changefeed/status/prod-tidb-broker-vol3\" value_size:130 > request_put:<key:\"/tidb/cdc/default/default/changefeed/status/ticdc-v504-stat-online-bp-many-prod-c\" value_size:130 > request_put:<key:\"/tidb/cdc/default/default/changefeed/status/broker-b-broker-bills-external\" value_size:130 >> failure:<>>"] [response=size:190] []