为提高效率,提问时请提供以下信息,问题描述清晰可优先响应。
- 【TiDB 版本】:v3.0.11
- 【问题描述】:
{‘tidb_log_dir’: ‘{{ deploy_dir }}/log’, ‘dummy’: None, ‘tidb_port’: 4000, ‘tidb_status_port’: 10080, ‘tidb_cert_dir’: ‘{{ deploy_dir }}/conf/ssl’}
系统信息
+--------------+-----------------------------+
| Host | Release |
+--------------+-----------------------------+
| tidb-mon | 3.10.0-1062.12.1.el7.x86_64 |
| tidb-kv1 | 3.10.0-1062.12.1.el7.x86_64 |
| tidb-kv2 | 3.10.0-1062.12.1.el7.x86_64 |
| tidb-kv3 | 3.10.0-1062.12.1.el7.x86_64 |
| tidb-pd1-db1 | 3.10.0-1062.12.1.el7.x86_64 |
| tidb-pd2-db2 | 3.10.0-1062.12.1.el7.x86_64 |
| tidb-pd3 | 3.10.0-1062.12.1.el7.x86_64 |
+--------------+-----------------------------+
TiDB 集群信息
+---------------------+--------------+------+----+------+
| TiDB_version | Clu_replicas | TiDB | PD | TiKV |
+---------------------+--------------+------+----+------+
| 5.7.25-TiDB-v3.0.11 | 3 | 2 | 3 | 3 |
+---------------------+--------------+------+----+------+
集群节点信息
+------------+-------------+
| Node_IP | Server_info |
+------------+-------------+
| instance_0 | tikv |
| instance_1 | tikv |
| instance_2 | tikv |
| instance_3 | pd+tidb |
| instance_4 | tidb+pd |
| instance_5 | pd |
+------------+-------------+
容量 & region 数量
+---------------------+-----------------+--------------+
| Storage_capacity_GB | Storage_uesd_GB | Region_count |
+---------------------+-----------------+--------------+
| 4535.30 | 1445.54 | 103269 |
+---------------------+-----------------+--------------+
QPS
+---------+----------------+-----------------+
| Clu_QPS | Duration_99_MS | Duration_999_MS |
+---------+----------------+-----------------+
| 3833.89 | 63.21 | 109.66 |
+---------+----------------+-----------------+
热点 region 信息
+---------+----------+-----------+
| Store | Hot_read | Hot_write |
+---------+----------+-----------+
| store-1 | 6 | 4 |
| store-5 | 5 | 5 |
| store-4 | 5 | 5 |
+---------+----------+-----------+
磁盘延迟信息
+--------+------------+-------------+--------------+
| Device | Instance | Read_lat_MS | Write_lat_MS |
+--------+------------+-------------+--------------+
| dm-0 | instance_0 | nan | nan |
| dm-0 | instance_1 | nan | nan |
| dm-0 | instance_2 | nan | 0.74 |
| dm-0 | instance_3 | nan | 0.00 |
| dm-0 | instance_4 | nan | 0.00 |
| dm-0 | instance_5 | nan | 0.00 |
| dm-1 | instance_0 | nan | nan |
| dm-1 | instance_1 | nan | nan |
| dm-1 | instance_2 | nan | nan |
| dm-1 | instance_3 | nan | nan |
| dm-1 | instance_4 | nan | nan |
| dm-1 | instance_5 | nan | nan |
| sda | instance_0 | nan | nan |
| sda | instance_1 | nan | nan |
| sda | instance_2 | nan | 0.75 |
| sda | instance_3 | nan | 0.00 |
| sda | instance_4 | nan | 0.00 |
| sda | instance_5 | nan | 0.00 |
| sdb | instance_0 | 0.19 | 0.12 |
| sdb | instance_1 | 0.28 | 0.12 |
| sdb | instance_2 | 0.16 | 0.11 |
| sdb | instance_3 | nan | 0.14 |
| sdb | instance_4 | nan | 0.08 |
| sdb | instance_5 | nan | 0.14 |
| sr0 | instance_0 | nan | nan |
| sr0 | instance_1 | nan | nan |
| sr0 | instance_2 | nan | nan |
| sr0 | instance_3 | nan | nan |
| sr0 | instance_4 | nan | nan |
| sr0 | instance_5 | nan | nan |
+--------+------------+-------------+--------------+
从昨天下午14:12分钟左右开始,3个tikv节点cpu使用一直处于繁忙状态,如下图
pd监控图
这种现象是热点问题吗?要怎么排查?