ON DUPLICATE KEY UPDATE 慢查询

【 TiDB 使用环境】生产环境 /测试/ Poc
【 TiDB 版本】
【复现路径】做过哪些操作出现的问题
【遇到的问题:问题现象及影响】
【资源配置】进入到 TiDB Dashboard -集群信息 (Cluster Info) -主机(Hosts) 截图此页面

INSERT INTO
  db.table (
    account_id,
    access_token,
    imei,
    gaid,
    uuid,
    server_time,
    ip,
    base
  ) value (
    230606010169920127,
    'b01732b1aa0dc3b223e449e4fd1c33e4',
    '33df5233f09bf0198f9c62a98d0764',
    'bbf987c6-ed5a-44eb-8dbf-e973c9f114e0',
    'U_796822c055e54efbbe15d46a16024f3b',
    1695113568000,
    '192.188.80.163',
    '{"app_avaliable_memory":"16.75M","voice":[{"current":9,"max":9,"type":"call"},{"current":0,"max":16,"type":"system"},{"current":5,"max":16,"type":"music"},{"current":2,"max":16,"type":"alarm"}],"gaid":"bbf987c6-ed5a-44eb-8dbf-e973c9f114e0","tz":"Asia/Jakarta","is_tt":"0","lon":"","ui_version":"10048","uuid":"U_796822c055e54efbbe15d46a16024f3b","manufacturer":"realme","app_free_memory":"5.99M","mem":"5.60GB","total_boot_time_wake":"32441556","model":"RMX3710","phone_num":"","net":4,"brand":"realme","lat":"","height":2158,"product":"RMX3710","battery_level":64,"os":"android 13","scene_id":9,"version_code":"454","cpu_num":8,"battery_max":100,"is_sim":"0","uptime":1695113567,"access_token":"b01732b1aa0dc3b223e449e4fd1c33e4","total_boot_time":"41724397","account_id":"230606010169920127","event_id":10000,"hasNfc":true,"width":1080,"imei":"33df5233f09bf0198f9c62a98d0764","rcid":"33df5233f09bf0198f9c62a98d0764","device":"REE2ADL1","fontScale":1,"board":"RM6769","app_max_memory":"384.00M","cid":104488}'
  ) ON DUPLICATE KEY
UPDATE
  account_id = 230606010169920127,
  access_token = 'b01732b1aa0dc3b223e449e4fd1c33e4',
  imei = '33df5233f09bf0198f9c62a98d0764',
  gaid = 'bbf987c6-ed5a-44eb-8dbf-e973c9f114e0',
  uuid = 'U_796822c055e54efbbe15d46a16024f3b',
  server_time = 1695113568000,
  ip = '192.188.80.163',
  base = '{"app_avaliable_memory":"16.75M","voice":[{"current":9,"max":9,"type":"call"},{"current":0,"max":16,"type":"system"},{"current":5,"max":16,"type":"music"},{"current":2,"max":16,"type":"alarm"}],"gaid":"bbf987c6-ed5a-44eb-8dbf-e973c9f114e0","tz":"Asia/Jakarta","is_tt":"0","lon":"","ui_version":"10048","uuid":"U_796822c055e54efbbe15d46a16024f3b","manufacturer":"realme","app_free_memory":"5.99M","mem":"5.60GB","total_boot_time_wake":"32441556","model":"RMX3710","phone_num":"","net":4,"brand":"realme","lat":"","height":2158,"product":"RMX3710","battery_level":64,"os":"android 13","scene_id":9,"version_code":"454","cpu_num":8,"battery_max":100,"is_sim":"0","uptime":1695113567,"access_token":"b01732b1aa0dc3b223e449e4fd1c33e4","total_boot_time":"41724397","account_id":"230606010169920127","event_id":10000,"hasNfc":true,"width":1080,"imei":"33df5233f09bf0198f9c62a98d0764","rcid":"33df5233f09bf0198f9c62a98d0764","device":"REE2ADL1","fontScale":1,"board":"RM6769","app_max_memory":"384.00M","cid":104488}';
	id      	task	estRows	operator info	actRows	execution info                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          	memory 	disk
	Insert_1	root	0      	N/A          	0      	time:324.7ms, loops:1, prepare: 100.8µs, check_insert: {total_time: 324.6ms, mem_insert_time: 130.2µs, prefetch: 324.5ms, rpc:{BatchGet:{num_rpc:1, total_time:324.5ms}, total_wait_time: 324ms, tikv_wall_time: 324.3ms, scan_detail: {total_process_keys: 1, total_process_keys_size: 1724, total_keys: 1, get_snapshot_time: 324.1ms, rocksdb: {block: {cache_hit_count: 14}}}}}, commit_txn: {prewrite:4.79ms, slowest_prewrite_rpc: {total: 0.005s, region_id: 294078, store: 10.95.6.86:20160, tikv_wall_time: 4.38ms, scan_detail: {get_snapshot_time: 10µs, rocksdb: {block: {cache_hit_count: 13}}}, write_detail: {store_batch_wait: 1.58ms, propose_send_wait: 0s, persist_log: {total: 1.48ms, write_leader_wait: 336.7µs, sync_log: 759.1µs, write_memtable: 18.5µs}, commit_log: 2.48ms, apply_batch_wait: 19.6µs, apply: {total:124.3µs, mutex_lock: 0s, write_leader_wait: 14.8µs, write_wal: 19.6µs, write_memtable: 36.8µs}}}, region_num:3, write_keys:5, write_byte:1866}	2.25 KB	N/A


为什么一个插入就这么慢,如何排查是什么原因造成的?

这也不是普通插入,要比较所有字段的

account_id是主键,我一直理解的是只比较account_id就可以,怎么验证是不是也比较了其它的字段,如果这样的话,是不是完全可以用replace into去代替

可能我搞错了,应该不用比较所有字段,那个base是json类型吗,json比较慢

插入语句涉及到了主键或唯一键冲突,这会导致TiDB需要先检查是否存在冲突的记录,然后再执行相应的插入或更新操作。这个过程可能会增加磁盘寻道的开销,尤其是当数据集很大或者主键或唯一键不是连续的时候

有读写热点么?

prefetch: 324.5ms

prefetch:从 TiKV 中获取需要检查冲突的数据的耗时,该步骤主要是向 TiKV 发送 BatchGet 类型的 RPC 请求的获取数据。

get_snapshot_time: 324.1ms

2 个赞

流量可视化上看没有读写热点

为什么是BatchGet,我的理解应该是 Batch_Point_Get吧

mysql上replace into是先删除一行,再insert。tidb是不是不需要删除,直接插入就可以了,也避免比较操作了。

我记得好像这种写法比replace好一些

1 个赞

看看这个呢 get_snapshot_time: 324.1ms

有写压力么?

2 个赞

那没有什么好思路了。
或者可以验证一下这三种情况的时间和执行计划
1、数据没有主键冲突
2、数据有主键冲突
3、数据没有主键冲突且不带ON DUPLICATE KEY

1 个赞

如果是json字段数据库还要做检验

replace呢