【 TiDB 使用环境】生产环境 /测试/ Poc
【 TiDB 版本】
【复现路径】做过哪些操作出现的问题
【遇到的问题:问题现象及影响】
【资源配置】
表结构是
Create Table: CREATE TABLE `data` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
`req_id` char(32) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL ,
`source_from` varchar(32) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL DEFAULT '' COMMENT '数据来源',
`source_type` varchar(32) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL DEFAULT '1' COMMENT '数据类型',
`create_at` bigint(20) unsigned NOT NULL DEFAULT '0' COMMENT '创建时间',
`update_at` bigint(20) unsigned NOT NULL DEFAULT '0' COMMENT '更新时间',
`upload_time` datetime DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT '上传时间',
PRIMARY KEY (`id`) /*T![clustered_index] CLUSTERED */,
KEY `idx_created_at` (`create_at`),
KEY `idx_updated_at` (`update_at`),
KEY `i_req_id` (`req_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin AUTO_INCREMENT=3256369687
1 row in set (0.00 sec)
查询为
SELECT
min(id)
FROM
data;
执行计划
mysql> explain SELECT max(id) FROM data;
+------------------------------+---------+-----------+-----------------+----------------------------------------------+
| id | estRows | task | access object | operator info |
+------------------------------+---------+-----------+-----------------+----------------------------------------------+
| StreamAgg_9 | 1.00 | root | | funcs:max(order.data.id)->Column#12 |
| └─Limit_13 | 1.00 | root | | offset:0, count:1 |
| └─TableReader_24 | 1.00 | root | | data:Limit_23 |
| └─Limit_23 | 1.00 | cop[tikv] | | offset:0, count:1 |
| └─TableFullScan_22 | 1.00 | cop[tikv] | table:data | keep order:true, desc |
+------------------------------+---------+-----------+-----------------+----------------------------------------------+
5 rows in set (0.00 sec)
小龙虾爱大龙虾
(Minghao Ren)
2
实际并不是全表,只是算子显示是TableFullScan(因为没有别的算子表示扫表了 ),实际上只拿到最大值就结束了,从执行计划中可以看到上面有limit算子中count:1,并且TableFullScan中有keep order:true。
min/max优化参考:https://docs.pingcap.com/zh/tidb/stable/max-min-eliminate
SQL在进行逻辑优化时会被改写,min会杯改为order by id limit 1
正常情况下是可以用到索引列的,请问这个表的数据有多少行
zhanggame1
(Ti D Ber G I13ecx U)
9
执行计划写行数是1,聚簇表有顺序,直接去表里面取最小的就行,执行慢应该是有问题,是不是delete过大量数据
dba远航
(Ti D Ber M Lo7 Bqhk)
10
使用explain analyze SELECT max(id) FROM data; 看一下实际的执行计划
Kongdom
(Kongdom)
12
1、执行explain analyze select max(id) from data看看,explain analyze是直接执行计划,explain是预估执行计划,预估执行计划可能不准。
2、验证了一下,确实是走的tablefullscan,分裂成两个region也是走的tablefullscan,那就只能有一个解释,是我的数据量太小,插入了35w数据,查询分析器可能认为全表扫描性能更快。
https://docs.pingcap.com/zh/tidb/stable/max-min-eliminate#只有一个-maxmin-函数时的优化规则
随缘天空
(Ti D Ber Ivw R7o Pj)
13
这个数据量耗时40多秒确实不正常,是不是出现读热点问题了,你把id
bigint(20) unsigned NOT NULL AUTO_INCREMENT,主键的类型修改一下,将AUTO_INCREMENT改为AUTO_RANDOM试试
+------------------------------+---------+---------+-----------+-----------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------+-----------+------+
| id | estRows | actRows | task | access object | execution info | operator info | memory | disk |
+------------------------------+---------+---------+-----------+-----------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------+-----------+------+
| StreamAgg_9 | 1.00 | 1 | root | | time:1m0.7s, loops:2 | funcs:min(order.data.id)->Column#12 | 160 Bytes | N/A |
| └─Limit_13 | 1.00 | 1 | root | | time:1m0.7s, loops:2 | offset:0, count:1 | N/A | N/A |
| └─TableReader_24 | 1.00 | 1 | root | | time:1m0.7s, loops:1, cop_task: {num: 221, max: 491.7ms, min: 584.9µs, avg: 274.4ms, p95: 419.9ms, max_proc_keys: 1, p95_proc_keys: 0, tot_proc: 1m0.3s, tot_wait: 23ms, rpc_num: 221, rpc_time: 1m0.6s, copr_cache_hit_ratio: 0.00, distsql_concurrency: 1} | data:Limit_23 | 547 Bytes | N/A |
| └─Limit_23 | 1.00 | 1 | cop[tikv] | | tikv_task:{proc max:491ms, min:0s, avg: 273.5ms, p80:409ms, p95:419ms, iters:221, tasks:221}, scan_detail: {total_process_keys: 1, total_process_keys_size: 27, total_keys: 153800520, get_snapshot_time: 108ms, rocksdb: {delete_skipped_count: 14402724, key_skipped_count: 168203023, block: {cache_hit_count: 2191, read_count: 240664, read_byte: 2.66 GB, read_time: 11.2s}}} | offset:0, count:1 | N/A | N/A |
| └─TableFullScan_22 | 1.00 | 1 | cop[tikv] | table:data | tikv_task:{proc max:491ms, min:0s, avg: 273.5ms, p80:409ms, p95:419ms, iters:221, tasks:221} | keep order:true | N/A | N/A |
+------------------------------+---------+---------+-----------+-----------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------+-----------+------+
5 rows in set (1 min 0.77 sec)
倒是跑着一个pt-archiver工具,在删除过期数据。就第一次运行很快,后面运行就很快
有猫万事足
16
在没有where条件的情况下,直接from t,只会有两种结果,不是table full scan就是index full scan。
https://docs.pingcap.com/zh/tidb/stable/clustered-indexes#创建聚簇索引表
CLUSTERED
,表示该表的主键为聚簇索引。在聚簇索引表中,行数据的键由用户给定的主键列数据构成,因此聚簇索引表存储一行至少只要一个键值对,即
你建立的是个聚簇索引表。没有单独的主键索引,主键就是row_id.所以会直接table full scan。
buddyyuan
(Buddyyuan)
20
你把 distsql_concurrency 改成15试试呢 ?