为提高效率,请提供以下信息,问题描述清晰能够更快得到解决:
【 TiDB 使用环境】
【概述】 场景 + 问题概述
文件表,目前单表1.1T,7.6亿数据,已经走索引 KEY idx_fta
(file_extension
,transition_state
,addtime
), 但是还是需要500毫秒左右,GC时间设置是24小时, 发现扫描Key高:scan_detail: {total_process_keys: 706, total_keys: 4277610, rocksdb: {delete_skipped_count: 2122, key_skipped_count: 4580802
【背景】 做过哪些操作
【现象】 业务和数据库现象
【问题】 当前遇到的问题
【业务影响】
【TiDB 版本】 v5.2.2
【应用软件及版本】
【附件】 相关日志及配置信息
- TiUP Cluster Display 信息
- TiUP CLuster Edit config 信息
监控(https://metricstool.pingcap.com/)
- TiDB-Overview Grafana监控
- TiDB Grafana 监控
- TiKV Grafana 监控
- PD Grafana 监控
- 对应模块日志(包含问题前后 1 小时日志)
SQL: desc analyze SELECT count(*) AS sum FROM eeo_files WHERE addtime > 1662974822 AND transition_state IN(0, 1, 4) AND file_extension IN(‘jpeg’, ‘jpg’, ‘png’, ‘bmp’) AND system_del = 0;
id task estRows operator info actRows execution info memory disk
HashAgg_18 root 1 funcs:count(Column#29)->Column#26 1 time:413.5ms, loops:2, partial_worker:{wall_time:413.481424ms, concurrency:5, task_num:1, tot_wait:2.067209901s, tot_exec:26.056µs, tot_time:2.067243399s, max:413.460881ms, p95:413.460881ms}, final_worker:{wall_time:413.538036ms, concurrency:5, task_num:1, tot_wait:2.067462191s, tot_exec:24.719µs, tot_time:2.067489543s, max:413.510606ms, p95:413.510606ms} 9.55 KB N/A
└─IndexLookUp_19 root 1 81 time:413.4ms, loops:2, index_task: {total_time: 396.6ms, fetch_handle: 396.6ms, build: 1.26µs, wait: 3.81µs}, table_task: {total_time: 2s, num: 1, concurrency: 5} 20.5 KB N/A
├─IndexRangeScan_15 cop[tikv] 14.11 table:eeo_files, index:idx_fta(file_extension, transition_state, addtime), range:("bmp" 0 1662988682,"bmp" 0 +inf], ("bmp" 1 1662988682,"bmp" 1 +inf], ("bmp" 4 1662988682,"bmp" 4 +inf], ("jpeg" 0 1662988682,"jpeg" 0 +inf], ("jpeg" 1 1662988682,"jpeg" 1 +inf], ("jpeg" 4 1662988682,"jpeg" 4 +inf], ("jpg" 0 1662988682,"jpg" 0 +inf], ("jpg" 1 1662988682,"jpg" 1 +inf], ("jpg" 4 1662988682,"jpg" 4 +inf], ("png" 0 1662988682,"png" 0 +inf], ("png" 1 1662988682,"png" 1 +inf], ("png" 4 1662988682,"png" 4 +inf], keep order:false 726 time:396.6ms, loops:3, cop_task: {num: 14, max: 396.4ms, min: 842.1µs, avg: 90.3ms, p95: 396.4ms, max_proc_keys: 315, p95_proc_keys: 315, tot_proc: 1.25s, tot_wait: 8ms, rpc_num: 14, rpc_time: 1.26s, copr_cache_hit_ratio: 0.29}, tikv_task:{proc max:438ms, min:0s, p80:284ms, p95:438ms, iters:22, tasks:14}, scan_detail: {total_process_keys: 706, total_keys: 4277610, rocksdb: {delete_skipped_count: 2122, key_skipped_count: 4580802, block: {cache_hit_count: 3655, read_count: 0, read_byte: 0 Bytes}}} N/A N/A
└─HashAgg_7 cop[tikv] 1 funcs:count(1)->Column#29 81 time:16ms, loops:2, cop_task: {num: 81, max: 10.4ms, min: 547.8µs, avg: 1.23ms, p95: 1.98ms, max_proc_keys: 301, p95_proc_keys: 12, tot_proc: 36ms, tot_wait: 16ms, rpc_num: 81, rpc_time: 98.8ms, copr_cache_hit_ratio: 0.00}, tikv_task:{proc max:9ms, min:0s, p80:1ms, p95:1ms, iters:81, tasks:81}, scan_detail: {total_process_keys: 726, total_keys: 1306, rocksdb: {delete_skipped_count: 330, key_skipped_count: 1834, block: {cache_hit_count: 4349, read_count: 0, read_byte: 0 Bytes}}} N/A N/A
└─Selection_17 cop[tikv] 14.11 eq(eo_osfile.eeo_files.system_del, 0) 726 tikv_task:{proc max:9ms, min:0s, p80:1ms, p95:1ms, iters:81, tasks:81} N/A N/A
└─TableRowIDScan_16 cop[tikv] 14.11 table:eeo_files, keep order:false 726 tikv_task:{proc max:9ms, min:0s, p80:1ms, p95:1ms, iters:81, tasks:81} N/A N/A