tidb查询数据混乱

【 TiDB 使用环境】测试
【 TiDB 版本】5.3
【遇到的问题】
这里表的联合主键是item_id和erpsource,
根据delivery_id查询出现两条4025931的数据,这两条的联合主键值一样!
SELECT * from erp_sdb_wms_delivery_items where delivery_id=4025931

通过item_id查询出现一条delivery_id=4025931的数据,没有出现重复的联合主键记录
SELECT * from erp_sdb_wms_delivery_items where item_id=7981775


【问题现象及影响】
表结构:
CREATE TABLE erp_sdb_wms_delivery_items (
item_id bigint(20) NOT NULL,
order_item_id bigint(20) NULL DEFAULT NULL,
delivery_id bigint(20) NULL DEFAULT NULL,
product_id bigint(20) NULL DEFAULT NULL,
shop_product_id varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
bn varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
product_name varchar(500) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
number bigint(20) NULL DEFAULT NULL,
pick_number bigint(20) NULL DEFAULT NULL,
addon longtext CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL,
sell_code varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
iscustomitem varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
unique_code_status varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
unique_code_status_lasttime varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
unique_code_status_op_id bigint(20) NULL DEFAULT NULL,
is_freeze varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
freeze_time bigint(20) NULL DEFAULT NULL,
unique_code varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
picurl varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
photo text CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL,
photo_design text CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL,
photo_ai varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
photo_append longtext CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL,
word_photo varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
word_append longtext CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL,
letter_photo varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
letter_append longtext CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL,
verify varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
verify_num bigint(20) NULL DEFAULT NULL,
price double NULL DEFAULT NULL,
sale_price double NULL DEFAULT NULL,
pmt_price double NULL DEFAULT NULL,
custom_type int(11) NULL DEFAULT NULL,
ispickup varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
checknum bigint(20) NULL DEFAULT NULL,
photoimgispull varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
wordphotoimgispull varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
letterphotoimgispull varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
glasses_id varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
glasses_sendtime bigint(20) NULL DEFAULT NULL,
defective_goods int(11) NULL DEFAULT NULL,
defective_memo varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
defective_supplier bigint(20) NULL DEFAULT NULL,
defective_time bigint(20) NULL DEFAULT NULL,
clarity_photo varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
check_clarity bigint(20) NULL DEFAULT NULL,
clarity_time bigint(20) NULL DEFAULT NULL,
clarity_type bigint(20) NULL DEFAULT NULL,
is_urgent int(11) NULL DEFAULT NULL,
urgent_opid bigint(20) NULL DEFAULT NULL,
gspdf_status varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
gspdf_memo text CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL,
gspdf_time bigint(20) NULL DEFAULT NULL,
myjit_pull_status varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
myjit_pull_time bigint(20) NULL DEFAULT NULL,
gspdf_order_sn varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
gspdf_invoice_no varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
zworders_status varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
zworders_memo text CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL,
zworders_time bigint(20) NULL DEFAULT NULL,
zworders_sn varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
zworders_no varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
is_only_product varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
clarity_intimes varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
erpsource varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
PRIMARY KEY (item_id, erpsource) USING BTREE,
INDEX index_unique_code(unique_code) USING BTREE,
INDEX index_order_item_id(order_item_id) USING BTREE,
INDEX index_gspdf_status(gspdf_status) USING BTREE,
INDEX ind_ido(ispickup, delivery_id, order_item_id) USING BTREE,
INDEX ind_number_delivery(delivery_id, number) USING BTREE,
INDEX ind_zworders_status(zworders_status) USING BTREE,
INDEX ind_bn_status_code(bn, delivery_id, unique_code) USING BTREE,
INDEX ind_unique_code_status(unique_code_status) USING BTREE,
INDEX idx_deliveryid_customtype_isfreeze_itemid_productid(delivery_id, custom_type, is_freeze, item_id, product_id) USING BTREE,
INDEX idx_deliveryid_uniquecode_productid(delivery_id, unique_code, product_id) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Compact;
【附件】

请提供各个组件的 version 信息,如 cdc/tikv,可通过执行 cdc version/tikv-server --version 获取。

数据写入用的tispark 版本是2.4.1

是否可以考虑升级 tispark 到 3.0 版本试一下?-

不打算升级,有什么解决办法呢

加 order by 就好了啊,这里的查询结果集没有强制排序。

也可参考这个问题。

咱们说的不是一个问题吧:sweat:

这个数据是不是本身在发生变更啊,如果不是,能把数据脱敏后,给两条这个数据的示例验证下

4.0.16模拟未复现

用tispark写入的,数据量一千多万条

执行查询SELECT * from erp_sdb_wms_delivery_items where item_id >7950000 and item_id <=7980000;会报错
inconsistent index PRIMARY handle count 8192 isn’t equal to value count 8191

你模拟的过程是什么,我这边是用tispark写入代码:
df.write.
format(“tidb”).
option(“tidb.addr”, “172.xxx”).
option(“tidb.port”, “4000”).
option(“tidb.user”, “bdata”).
option(“tidb.password”, “”).
option(“database”, “bdata”).
option(“table”,tblName).
option(“replace”,true).
option(“isolationLevel”,“NONE”).
option(JDBCOptions.JDBC_BATCH_INSERT_SIZE,1000).
mode(“append”).
save()

这个问题看起来有点严重啊。关注下…坐等官方结论。

是的,现在sql执行的结果全是错的

确定两次查询之间没有对数据修改吗?如果是这样的话,等官方有结论的话通知我一声,谢谢:smile:

没有做过修改,我怀疑是索引和数据不一致的问题

分布式下这个问题都存在的啊,结果集不稳定问题

建议按照以下建议,修复一下,修复之后,看看上述现象是否能够复现。
数据修复官方提供了下面两个命令:admin cleanup、admin recover。前者适用于 index 数据多于 data 行数据的情况,会将多余的 index 数据删除掉,后者适用于 data 行数据多于 index 行数据的情况,会将缺少的 index 数据填充完整。
https://docs.pingcap.com/zh/tidb/stable/sql-statement-admin-checksum-table#语法图

1 个赞

灵异事件,之前mysql,也遇到过,那张表数据量特别大,偶尔发现有几条数据,就是查询不一致。

你说的这个情况,我是不是可以概括成使用主键查询和使用非主键查询,结果不一致?