分页导出场景-JOIN关联查询分页处理速度慢

为提高效率,请提供以下信息,问题描述清晰能够更快得到解决:
【 TiDB 使用环境】
image

【概述】 场景 + 问题概述
导出分页查询,采用游标的方式操作增加页数查询,耗时达到2-5s不等,由于有orderby 好像无法下推到cro
【表1结构】:
– auto-generated definition
create table sxt_order
(
id bigint auto_increment comment ‘主键id’
primary key,
task_id bigint null comment ‘投放任务id’,
order_id varchar(64) not null comment ‘随心推订单id’,
is_out int default 0 null comment ‘是否是外部订单。1 是,0 不是’,
video_id varchar(64) not null comment ‘被投视频id’,
video_title varchar(200) null comment ‘视频标题’,
video_cover_url varchar(500) null comment ‘视频封面地址’,
target_douyin_id varchar(64) null comment ‘被投视频的抖音ID’,
target_douyin_name varchar(100) null comment ‘被投抖音名称’,
paid_douyin_id varchar(64) not null comment ‘付款抖音ID’,
qc_order_id varchar(64) null comment ‘千川内的订单id’,
qc_order_name varchar(200) null comment ‘千川内的订单名称’,
order_create_time datetime null comment ‘订单创建时间’,
order_status int not null comment ‘订单状态:1-未支付、3-创建中、5-审核中、7-投放中(部分审核未通过)、8-投放终止、10-投放结束、999-终止中’,
status_desc varchar(32) null comment ‘订单状态说明’,
reject_reason varchar(255) null comment ‘审核失败原因’,
order_amount int not null comment ‘投放金额,单位(厘)’,
aim int null comment ‘优化目标:96-商品购买,269-粉丝提示,270-点赞评论’,
duration int null comment ‘投放时长(小时)2/6/12/24’,
delivery_type int null comment ‘视频推荐方式:1-系统智能推荐,2-自定义,3-相似达人粉丝推荐’,
gender varchar(10) null comment ‘性别:0-不限,1-男,2-女。默认为0’,
age_range_codes varchar(64) null comment ‘年龄(多选按逗号隔开):2-18~23岁,3-24~30岁,4-31~40岁,5-41~50岁’,
district int null comment ‘地域标签:0-全国、1-区县、2-省市、3-商圈或附近区域’,
provinces varchar(2000) null comment ‘省份’,
area varchar(2000) null comment ‘地域中文列表’,
cities varchar(2000) null comment ‘区域编码(district为1时代表区县的区域编码、为2时代表城市的区域编码),多个用逗号隔开’,
businesses varchar(2000) null comment ‘商圈编码,多个用逗号隔开’,
interests varchar(255) null comment ‘兴趣,逗号分隔’,
authors varchar(2000) null comment ‘达人uid,多个用逗号隔开’,
pricing_target int null comment ‘出价目标:1-按播放量出价,2-按优化目标手动出价(需级联bid字段),3-按优化目标自动出价’,
bid int null comment ‘转化效果,厘’,
estimate_show_min int null comment ‘预估(出价目标,如展示粉丝量云云)数(最小值)’,
estimate_show_max int null comment ‘预估(出价目标,如展示粉丝量云云)数(最大值)’,
abandon_time datetime null comment ‘随心推执行终止时间’,
pull_date datetime null comment ‘拉取日期(该日期为还需要拉取的日期的零点整)’,
report_finish int default 0 null comment ‘报表数据更新完成,1完成 0 未完成’,
del_flag tinyint(4) unsigned default 0 null comment ‘删除标识 0:未删除 1:删除’,
create_time datetime default CURRENT_TIMESTAMP not null comment ‘创建时间’,
update_time datetime default CURRENT_TIMESTAMP not null on update CURRENT_TIMESTAMP comment ‘更新时间’,
constraint order_id
unique (order_id)
)
comment ‘随心推订单表(新)’ collate = utf8mb4_general_ci;

create index create_time
on sxt_order (create_time);

create index idx_targetdouyinid_paiddouyinid
on sxt_order (target_douyin_id, paid_douyin_id);

create index sxt_order_order_create_time_order_status_report_finish_index
on sxt_order (order_create_time, order_status, report_finish);

create index sxt_order_paid_douyin_id_order_create_time_index
on sxt_order (paid_douyin_id, order_create_time);

create index sxt_order_update_time_index
on sxt_order (update_time);

create index sxt_order_video_id_index
on sxt_order (video_id);

create index task_id
on sxt_order (task_id);

【表2结构】:
– auto-generated definition
create table sxt_qc_order_hour_interval_report
(
id bigint auto_increment comment ‘主键ID’
primary key,
sxt_order_id varchar(64) not null comment ‘随心推订单Id’,
video_id varchar(64) null comment ‘视频Id’,
paid_douyin_id varchar(64) null,
stat_time_day datetime not null comment ‘时段开始时间,秒数unix时间戳’,
stat_cost float default 0 null comment ‘已消耗金额’,
pay_order_count int default 0 null comment ‘成交订单数’,
pay_order_amount float default 0 null comment ‘成交订单金额’,
pay_order_roi float default 0 null comment ‘成交订单ROI’,
create_order_count int default 0 null comment ‘下单数’,
create_order_amount float default 0 null comment ‘下单金额’,
create_order_roi float default 0 null comment ‘下单roi’,
show_cnt int default 0 null comment ‘展示次数’,
click_cnt int default 0 null comment ‘点击次数’,
ctr float default 0 null comment ‘点击率’,
cpm float default 0 null comment ‘平均千展费用’,
convert_cnt int default 0 null comment ‘转化数’,
convert_cost float default 0 null comment ‘转化成本’,
convert_rate float default 0 null comment ‘转化率’,
dy_comment int default 0 null comment ‘评论数’,
total_play int default 0 null comment ‘播放数’,
dy_like int default 0 null comment ‘点赞数’,
dy_share int default 0 null comment ‘分享数’,
dy_follow int default 0 null comment ‘新增粉丝数’,
play_duration_3s int default 0 null comment ‘三秒播放数’,
play_25_feed_break int default 0 null comment ‘25%进度播放数’,
play_50_feed_break int default 0 null comment ‘50%进度播放数’,
play_75_feed_break int default 0 null comment ‘75%进度播放数’,
play_over int default 0 null comment ‘完播数’,
play_over_rate float default 0 null comment ‘完播率’,
del_flag tinyint(4) unsigned default 0 null comment ‘删除标识 0:未删除 1:删除’,
create_time datetime default CURRENT_TIMESTAMP null comment ‘创建时间 创建时间’,
update_time datetime default CURRENT_TIMESTAMP null on update CURRENT_TIMESTAMP comment ‘更新时间 修改时间’,
constraint sxt_order_id_stat_time_day
unique (sxt_order_id, stat_time_day)
)
comment ‘随心推订单时段数据’ collate = utf8mb4_general_ci;

create index idx_sxtorderid_paiddouyinid_stattimeday
on sxt_qc_order_hour_interval_report (sxt_order_id, paid_douyin_id, stat_time_day);

create index idx_videoid_stattimeday
on sxt_qc_order_hour_interval_report (video_id, stat_time_day);

create index sxt_qc_order_hour_interval_report_bk_paid_day_index
on sxt_qc_order_hour_interval_report (paid_douyin_id, stat_time_day);

create index sxt_qc_order_hour_interval_report_bk_stat_time_day_index
on sxt_qc_order_hour_interval_report (stat_time_day);

【SQL】:explain analyze
SELECT
a.id id,
a.order_id orderId,
a.reject_reason rejectReason,
a.is_out isOut,
a.order_status orderStatus,
a.order_create_time orderCreateTime,
a.video_title videoName,
a.video_cover_url coverImgeUrl,
a.status_desc statusDesc,
a.paid_douyin_id paidDouyinId,
a.target_douyin_id targetDouyinId,
a.target_douyin_name targetDouyinName,
a.video_id videoId,
IF(
a.bid IS NULL,
0,
CONVERT(a.bid / 1000, decimal(16, 2))
) bid,
IF(
SUM(b.pay_order_amount) IS NULL,
0,
CONVERT(SUM(b.pay_order_amount), decimal(16, 2))
) / IF(
SUM(b.stat_cost) IS NULL,
0,
CONVERT(SUM(b.stat_cost), decimal(16, 2))
) orderRoi,
CONVERT(a.order_amount / 1000, decimal(16, 2)) orderAmount,
IF(
SUM(b.stat_cost) IS NULL,
0,
CONVERT(SUM(b.stat_cost), decimal(16, 2))
) costInt,
IF(SUM(b.show_cnt) IS NULL, 0, SUM(b.show_cnt)) showCount,
IF(SUM(b.click_cnt) IS NULL, 0, SUM(b.click_cnt)) clickCount,
IF(
SUM(b.pay_order_count) IS NULL,
0,
SUM(b.pay_order_count)
) payOrderCount,
IF(
SUM(b.pay_order_amount) IS NULL,
0,
CONVERT(SUM(b.pay_order_amount), decimal(16, 2))
) payOrderAmount,
IF(
SUM(b.create_order_count) IS NULL,
0,
SUM(b.create_order_count)
) createOrderCount,
IF(
SUM(b.create_order_amount) IS NULL,
0,
CONVERT(SUM(b.create_order_amount), decimal(16, 2))
) createOrderAmount,
IF(
SUM(b.play_duration_3s) IS NULL,
0,
SUM(b.play_duration_3s)
) threeSecondsPlayCount,
IF(SUM(b.total_play) IS NULL, 0, SUM(b.total_play)) play,
IF(SUM(b.dy_follow) IS NULL, 0, SUM(b.dy_follow)) addFansCount,
IF(SUM(b.dy_like) IS NULL, 0, SUM(b.dy_like)) fabulousCount,
IF(SUM(b.dy_share) IS NULL, 0, SUM(b.dy_share)) shareCount,
IF(SUM(b.dy_comment) IS NULL, 0, SUM(b.dy_comment)) commentCount
FROM
sxt_order a
JOIN sxt_qc_order_hour_interval_report b use index (sxt_order_id_stat_time_day) on a.order_id = b.sxt_order_id and a.paid_douyin_id=b.paid_douyin_id and b.stat_cost > 0 and
b.stat_time_day BETWEEN ‘2021-09-18 00:00:00’ AND ‘2021-10-19 23:59:59’
WHERE
a.id >2118806 and
a.paid_douyin_id IN
( ‘1662108471658904’ ,‘2066737168409358’ ,‘61317543460’ ,‘1930393900817603’ ,‘2717648042858318’ ,‘3325353093234775’ ,‘3888301724678888’ ,‘2356999089034912’ ,‘3227820456683768’ ,‘598563211975470’ ,‘78111457015’ ,‘96390118087’ ,‘923215435150103’ ,‘93143676232’ ,‘2189832380756879’ ,‘2805597364302079’ ,‘892459574371576’ ,‘4125011064914158’ ,
‘968031366559111’ ,‘3139827325221119’ )
AND a.order_create_time
BETWEEN date_sub(‘2021-09-18 00:00:00.0’, INTERVAL 7 DAY)
AND ‘2021-10-19 23:59:59.999’

GROUP BY a.order_id
order by a.id
LIMIT 500;

TIDB执行计划:

Projection_11,500.00,500,root,“time:2.25s, loops:2, Concurrency:OFF”,“doujia.sxt_order.id, doujia.sxt_order.order_id, doujia.sxt_order.reject_reason, doujia.sxt_order.is_out, doujia.sxt_order.order_status, doujia.sxt_order.order_create_time, doujia.sxt_order.video_title, doujia.sxt_order.video_cover_url, doujia.sxt_order.status_desc, doujia.sxt_order.paid_douyin_id, doujia.sxt_order.target_douyin_id, doujia.sxt_order.target_douyin_name, doujia.sxt_order.video_id, if(isnull(doujia.sxt_order.bid), 0, cast(div(cast(doujia.sxt_order.bid, decimal(20,0) BINARY), 1000), decimal(16,2) BINARY))->Column#86, div(if(isnull(Column#73), 0, cast(Column#73, decimal(16,2) BINARY)), if(isnull(Column#74), 0, cast(Column#74, decimal(16,2) BINARY)))->Column#87, cast(div(cast(doujia.sxt_order.order_amount, decimal(20,0) BINARY), 1000), decimal(16,2) BINARY)->Column#88, if(isnull(Column#74), 0, cast(Column#74, decimal(16,2) BINARY))->Column#89, if(isnull(Column#75), 0, Column#75)->Column#90, if(isnull(Column#76), 0, Column#76)->Column#91, if(isnull(Column#77), 0, Column#77)->Column#92, if(isnull(Column#73), 0, cast(Column#73, decimal(16,2) BINARY))->Column#93, if(isnull(Column#78), 0, Column#78)->Column#94, if(isnull(Column#79), 0, cast(Column#79, decimal(16,2) BINARY))->Column#95, if(isnull(Column#80), 0, Column#80)->Column#96, if(isnull(Column#81), 0, Column#81)->Column#97, if(isnull(Column#82), 0, Column#82)->Column#98, if(isnull(Column#83), 0, Column#83)->Column#99, if(isnull(Column#84), 0, Column#84)->Column#100, if(isnull(Column#85), 0, Column#85)->Column#101”,444.3 KB,N/A
└─TopN_14,500.00,500,root,“time:2.23s, loops:2”,“doujia.sxt_order.id:asc, offset:0, count:500”,2.64 MB,N/A
└─HashAgg_18,18998.33,70858,root,“time:2.21s, loops:73, PartialConcurrency:4, FinalConcurrency:4”,“group by:Column#149, funcs:sum(Column#121)->Column#73, funcs:sum(Column#122)->Column#74, funcs:sum(Column#123)->Column#75, funcs:sum(Column#124)->Column#76, funcs:sum(Column#125)->Column#77, funcs:sum(Column#126)->Column#78, funcs:sum(Column#127)->Column#79, funcs:sum(Column#128)->Column#80, funcs:sum(Column#129)->Column#81, funcs:sum(Column#130)->Column#82, funcs:sum(Column#131)->Column#83, funcs:sum(Column#132)->Column#84, funcs:sum(Column#133)->Column#85, funcs:firstrow(Column#134)->doujia.sxt_order.id, funcs:firstrow(Column#135)->doujia.sxt_order.order_id, funcs:firstrow(Column#136)->doujia.sxt_order.is_out, funcs:firstrow(Column#137)->doujia.sxt_order.video_id, funcs:firstrow(Column#138)->doujia.sxt_order.video_title, funcs:firstrow(Column#139)->doujia.sxt_order.video_cover_url, funcs:firstrow(Column#140)->doujia.sxt_order.target_douyin_id, funcs:firstrow(Column#141)->doujia.sxt_order.target_douyin_name, funcs:firstrow(Column#142)->doujia.sxt_order.paid_douyin_id, funcs:firstrow(Column#143)->doujia.sxt_order.order_create_time, funcs:firstrow(Column#144)->doujia.sxt_order.order_status, funcs:firstrow(Column#145)->doujia.sxt_order.status_desc, funcs:firstrow(Column#146)->doujia.sxt_order.reject_reason, funcs:firstrow(Column#147)->doujia.sxt_order.order_amount, funcs:firstrow(Column#148)->doujia.sxt_order.bid”,7.88 MB,N/A
└─Projection_60,141730.41,248642,root,“time:1.97s, loops:246, Concurrency:4”,“doujia.sxt_qc_order_hour_interval_report.pay_order_amount, doujia.sxt_qc_order_hour_interval_report.stat_cost, cast(doujia.sxt_qc_order_hour_interval_report.show_cnt, decimal(32,0) BINARY)->Column#123, cast(doujia.sxt_qc_order_hour_interval_report.click_cnt, decimal(32,0) BINARY)->Column#124, cast(doujia.sxt_qc_order_hour_interval_report.pay_order_count, decimal(32,0) BINARY)->Column#125, cast(doujia.sxt_qc_order_hour_interval_report.create_order_count, decimal(32,0) BINARY)->Column#126, doujia.sxt_qc_order_hour_interval_report.create_order_amount, cast(doujia.sxt_qc_order_hour_interval_report.play_duration_3s, decimal(32,0) BINARY)->Column#128, cast(doujia.sxt_qc_order_hour_interval_report.total_play, decimal(32,0) BINARY)->Column#129, cast(doujia.sxt_qc_order_hour_interval_report.dy_follow, decimal(32,0) BINARY)->Column#130, cast(doujia.sxt_qc_order_hour_interval_report.dy_like, decimal(32,0) BINARY)->Column#131, cast(doujia.sxt_qc_order_hour_interval_report.dy_share, decimal(32,0) BINARY)->Column#132, cast(doujia.sxt_qc_order_hour_interval_report.dy_comment, decimal(32,0) BINARY)->Column#133, doujia.sxt_order.id, doujia.sxt_order.order_id, doujia.sxt_order.is_out, doujia.sxt_order.video_id, doujia.sxt_order.video_title, doujia.sxt_order.video_cover_url, doujia.sxt_order.target_douyin_id, doujia.sxt_order.target_douyin_name, doujia.sxt_order.paid_douyin_id, doujia.sxt_order.order_create_time, doujia.sxt_order.order_status, doujia.sxt_order.status_desc, doujia.sxt_order.reject_reason, doujia.sxt_order.order_amount, doujia.sxt_order.bid, doujia.sxt_order.order_id”,6.57 MB,N/A
└─IndexHashJoin_26,141730.41,248642,root,“time:1.97s, loops:246, inner:{total:5.72s, concurrency:4, task:11, construct:156.2ms, fetch:4.77s, build:35.2ms, join:796.2ms}”,“inner join, inner:IndexLookUp_23, outer key:doujia.sxt_order.order_id, inner key:doujia.sxt_qc_order_hour_interval_report.sxt_order_id, equal cond:eq(doujia.sxt_order.order_id, doujia.sxt_qc_order_hour_interval_report.sxt_order_id), eq(doujia.sxt_order.paid_douyin_id, doujia.sxt_qc_order_hour_interval_report.paid_douyin_id)”,81.8 MB,N/A
├─IndexLookUp_42(Build),18998.33,74485,root,“time:363.9ms, loops:78, index_task: {total_time: 48ms, fetch_handle: 18.3ms, build: 11.1µs, wait: 29.6ms}, table_task: {total_time: 866.2ms, num: 11, concurrency: 4}”,22.2 MB,N/A
│ ├─Selection_41(Build),18998.33,74485,cop[tikv],“time:17.6ms, loops:80, cop_task: {num: 3, max: 25.2ms, min: 9.82ms, avg: 19ms, p95: 25.2ms, max_proc_keys: 36354, p95_proc_keys: 36354, tot_proc: 56ms, rpc_num: 3, rpc_time: 56.9ms, copr_cache: disabled}, tikv_task:{proc max:22ms, min:8ms, p80:22ms, p95:22ms, iters:91, tasks:3}, scan_detail: {total_process_keys: 79133, total_keys: 79153}”,“gt(doujia.sxt_order.id, 2118806)”,N/A,N/A
│ │ └─IndexRangeScan_39,45561.35,79133,cop[tikv],“tikv_task:{proc max:22ms, min:8ms, p80:22ms, p95:22ms, iters:91, tasks:3}”,“table:a, index:sxt_order_paid_douyin_id_order_create_time_index(paid_douyin_id, order_create_time)”,“range:[”“1662108471658904"” 2021-09-11 00:00:00,““1662108471658904"” 2021-10-20 00:00:00), [”“1930393900817603"” 2021-09-11 00:00:00,““1930393900817603"” 2021-10-20 00:00:00), [”“2066737168409358"” 2021-09-11 00:00:00,““2066737168409358"” 2021-10-20 00:00:00), [”“2189832380756879"” 2021-09-11 00:00:00,““2189832380756879"” 2021-10-20 00:00:00), [”“2356999089034912"” 2021-09-11 00:00:00,““2356999089034912"” 2021-10-20 00:00:00), [”“2717648042858318"” 2021-09-11 00:00:00,““2717648042858318"” 2021-10-20 00:00:00), [”“2805597364302079"” 2021-09-11 00:00:00,““2805597364302079"” 2021-10-20 00:00:00), [”“3139827325221119"” 2021-09-11 00:00:00,““3139827325221119"” 2021-10-20 00:00:00), [”“3227820456683768"” 2021-09-11 00:00:00,““3227820456683768"” 2021-10-20 00:00:00), [”“3325353093234775"” 2021-09-11 00:00:00,““3325353093234775"” 2021-10-20 00:00:00), [”“3888301724678888"” 2021-09-11 00:00:00,““3888301724678888"” 2021-10-20 00:00:00), [”“4125011064914158"” 2021-09-11 00:00:00,““4125011064914158"” 2021-10-20 00:00:00), [”“598563211975470"” 2021-09-11 00:00:00,““598563211975470"” 2021-10-20 00:00:00), [”“61317543460"” 2021-09-11 00:00:00,““61317543460"” 2021-10-20 00:00:00), [”“78111457015"” 2021-09-11 00:00:00,““78111457015"” 2021-10-20 00:00:00), [”“892459574371576"” 2021-09-11 00:00:00,““892459574371576"” 2021-10-20 00:00:00), [”“923215435150103"” 2021-09-11 00:00:00,““923215435150103"” 2021-10-20 00:00:00), [”“93143676232"” 2021-09-11 00:00:00,““93143676232"” 2021-10-20 00:00:00), [”“96390118087"” 2021-09-11 00:00:00,““96390118087"” 2021-10-20 00:00:00), [”“968031366559111"” 2021-09-11 00:00:00,““968031366559111"” 2021-10-20 00:00:00), keep order:false”,N/A,N/A
│ └─TableRowIDScan_40(Probe),18998.33,74485,cop[tikv],“time:796.2ms, loops:98, cop_task: {num: 55, max: 200.4ms, min: 2.49ms, avg: 63.3ms, p95: 162.7ms, max_proc_keys: 5384, p95_proc_keys: 3728, tot_proc: 2.28s, tot_wait: 518ms, rpc_num: 55, rpc_time: 3.48s, copr_cache: disabled}, tikv_task:{proc max:84ms, min:0s, p80:47ms, p95:75ms, iters:275, tasks:55}, scan_detail: {total_process_keys: 74485, total_keys: 77926}”,table:a,keep order:false,N/A,N/A
└─IndexLookUp_23(Probe),7.46,248642,root,“time:4.58s, loops:260, index_task: {total_time: 3.7s, fetch_handle: 1.09s, build: 747.5µs, wait: 2.62s}, table_task: {total_time: 17.1s, num: 139, concurrency: 44}”,42.5 KB,N/A
├─IndexRangeScan_20(Build),2351.06,2088456,cop[tikv],“time:967.7ms, loops:2132, cop_task: {num: 160, max: 318.7ms, min: 3.24ms, avg: 97.7ms, p95: 229.4ms, max_proc_keys: 51864, p95_proc_keys: 36504, tot_proc: 11.4s, tot_wait: 2.79s, rpc_num: 160, rpc_time: 15.6s, copr_cache: disabled}, tikv_task:{proc max:78ms, min:0s, p80:38ms, p95:58ms, iters:2747, tasks:160}, scan_detail: {total_process_keys: 2088456, total_keys: 2162943}”,“table:b, index:sxt_order_id_stat_time_day(sxt_order_id, stat_time_day)”,“range: decided by [eq(doujia.sxt_qc_order_hour_interval_report.sxt_order_id, doujia.sxt_order.order_id) ge(doujia.sxt_qc_order_hour_interval_report.stat_time_day, 2021-09-18 00:00:00.000000) le(doujia.sxt_qc_order_hour_interval_report.stat_time_day, 2021-10-19 23:59:59.000000)], keep order:false”,N/A,N/A
└─Selection_22(Probe),7.46,248642,cop[tikv],“time:14.7s, loops:463, cop_task: {num: 1915, max: 357.8ms, min: 552.9µs, avg: 37.2ms, p95: 116.4ms, max_proc_keys: 17760, p95_proc_keys: 5352, tot_proc: 35.5s, tot_wait: 27.2s, rpc_num: 1915, rpc_time: 1m11.1s, copr_cache: disabled}, tikv_task:{proc max:75ms, min:0s, p80:7ms, p95:20ms, iters:8019, tasks:1915}, scan_detail: {total_process_keys: 2088456, total_keys: 2148534}”,"gt(doujia.sxt_qc_order_hour_interval_report.stat_cost, 0), in(doujia.sxt_qc_order_hour_interval_report.paid_douyin_id, "“1662108471658904"”, "“2066737168409358"”, "“61317543460"”, "“1930393900817603"”, "“2717648042858318"”, "“3325353093234775"”, "“3888301724678888"”, "“2356999089034912"”, "“3227820456683768"”, "“598563211975470"”, "“78111457015"”, "“96390118087"”, "“923215435150103"”, "“93143676232"”, "“2189832380756879"”, "“2805597364302079"”, "“892459574371576"”, "“4125011064914158"”, "“968031366559111"”, ““3139827325221119"”), not(isnull(doujia.sxt_qc_order_hour_interval_report.paid_douyin_id))”,N/A,N/A
└─TableRowIDScan_21,2351.06,2088456,cop[tikv],“tikv_task:{proc max:74ms, min:0s, p80:7ms, p95:19ms, iters:8019, tasks:1915}”,table:b,keep order:false,N/A,N/A

【背景】 做过哪些操作

【现象】 业务和数据库现象

【问题】 当前遇到的问题

【业务影响】

【TiDB 版本】
4
【应用软件及版本】

【附件】 相关日志及配置信息

  • TiUP Cluster Display 信息
  • TiUP CLuster Edit config 信息

监控(https://metricstool.pingcap.com/)

  • TiDB-Overview Grafana监控
  • TiDB Grafana 监控
  • TiKV Grafana 监控
  • PD Grafana 监控
  • 对应模块日志(包含问题前后 1 小时日志)

若提问为性能优化、故障排查类问题,请下载脚本运行。终端输出的打印结果,请务必全选并复制粘贴上传。

2 个赞
  1. 只有两个 tikv 吗?
  2. 请问机器配置是什么?包括磁盘配置。
  3. 2s多不满足条件,期望多长时间?
1 个赞
  1. 是的,两个KV。整体的数据量没有那么大,就是这个场景中是一个500w的表join 1亿数据量的表,其他都是小表

  2. 两个kv 都是16核32G。其中一个kv上部署了pd和db。。配置的话平时正常业务cpu和内存的占比分别在30+和50+
    磁盘配置是ESSD

  3. 因为这个场景中是分页导出几万条数据,单这条sql2s整体会很长,其次最主要的是这个sql跑的时间长了会把整个机器的CPU占满,但是业务直接奔溃

1 个赞

我调整了SQL,最后排序换成分组条件是可以到1s内的,但是如果排序换成Id 还是很慢,这个是什么原因。

1 个赞

而且目前我们是没有安装TIFLASH的,这个是否会对现有的查询有改善

1 个赞

尝试强制使用索引:idx_sxtorderid_paiddouyinid_stattimeday,看看执行时间与执行计划


用过了还是不行

大佬,我有个问题,为什么我把order by换成 order_id时 或者不用order by 效率很高。。但是用 order by id 会 很慢。

大佬,我有个问题,为什么我把order by换成 order_id时 或者不用order by 效率很高。。但是用 order by id 会 很慢。

因为你的group by就包含order_id,已经是排序的了

重新排序会变慢很多吗?这个必须要按照非分组条件进行排序,有别的方式可以调整吗

按照stat_time_day排序可以吗?如果可以可以看下执行时间和执行计划

还是一样的 3s+,只有用order_id 排序可以:joy:

单机混合部署估计是要调整限制一下节点资源争用

https://docs.pingcap.com/zh/tidb/stable/three-nodes-hybrid-deployment#参数调整

好的 多谢

此话题已在最后回复的 1 分钟后被自动关闭。不再允许新回复。