TiDB 7.1.0 Can't find column Column#40 in schema Column: [Column#37,Column#38] Unique key: []

Bug 反馈
【 TiDB 版本】7.1.0
【 Bug 的影响】
查询无法执行,影响AP业务

【可能的问题复现步骤】

  • 表结构
CREATE TABLE `dws_gs_bill_i18n_user_amt_mdi` (
  `I_ID` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT '自增ID',
  `I_DATE` int(11) NOT NULL DEFAULT '0' COMMENT '',
  `CH_ID_CARD_ENCRYPT` varchar(160) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '' COMMENT '',
  `I_AMOUNT` decimal(20,4) NOT NULL DEFAULT '0.0000' COMMENT '',
  `D_CREATED_AT` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '',
  `D_UPDATED_AT` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT '',
  PRIMARY KEY (`I_ID`) /*T![clustered_index] CLUSTERED */,
  UNIQUE KEY `UNIQ_DT_CH_ID_CARD_ENCRYPT` (`I_DATE`,`CH_ID_CARD_ENCRYPT`),
  KEY `IDX_CH_ID_CARD_ENCRYPT` (`CH_ID_CARD_ENCRYPT`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci AUTO_INCREMENT=90001 COMMENT=''

CREATE TABLE `redpacket_white_user_bankcards` (
  `I_ID` bigint(20) NOT NULL AUTO_INCREMENT COMMENT '自增主键',
  `CH_ID_CARD_ENCRYPT` varchar(160) NOT NULL DEFAULT '' COMMENT '',
  `I_STATUS` tinyint(4) NOT NULL DEFAULT '0' COMMENT '',
  `B_ACTIVE` tinyint(1) NOT NULL DEFAULT '1' COMMENT '',
  `CH_COUNTRY_CODE` varchar(3) NOT NULL DEFAULT '' COMMENT '',
  PRIMARY KEY (`I_ID`) /*T![clustered_index] CLUSTERED */,
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin AUTO_INCREMENT=485514 COMMENT='';
  • sql 语句
SELECT group_concat(concat('{"key":"', CH_DATA_KEY, '","value":[', '{"key":"name1","value":"', I_AMOUNT, '"}', ']}') order by I_AMOUNT) CH_DATA_KEY 
FROM (
  select 'name2' as CH_DATA_KEY, round(sum(I_AMOUNT) / 100 / count(distinct a.CH_ID_CARD_ENCRYPT), 2) as I_AMOUNT      
  from dws_gs_bill_i18n_user_amt_mdi as a 
  join (
    select distinct u.CH_ID_CARD_ENCRYPT from broker.redpacket_white_user_bankcards as u 
    where CH_COUNTRY_CODE != '' and CH_COUNTRY_CODE != 'CHN' and I_STATUS IN (1, 4) and B_ACTIVE = 1) as w 
  on a.CH_ID_CARD_ENCRYPT = w.CH_ID_CARD_ENCRYPT 
  where a.I_DATE = 202304 and a.I_AMOUNT >= 100 * 100) sql_fmt_base_0;

【看到的非预期行为】

补充说明:
字符集都改成utf8mb4,sql 依然报错

root@127.0.0.1 11:53:50 [test]> ALTER TABLE redpacket_white_user_bankcards CONVERT TO CHARACTER SET utf8mb4;
Query OK, 0 rows affected (0.18 sec)

root@127.0.0.1 12:04:00 [test]> SELECT group_concat(concat('{"key":"', CH_DATA_KEY, '","value":[', '{"key":"name1","value":"', I_AMOUNT, '"}', ']}') order by I_AMOUNT) CH_DATA_KEY  FROM (   select 'name2' as CH_DATA_KEY, round(sum(I_AMOUNT) / 100 / count(distinct a.CH_ID_CARD_ENCRYPT), 2) as I_AMOUNT         from dws_gs_bill_i18n_user_amt_mdi as a    join (     select distinct u.CH_ID_CARD_ENCRYPT from redpacket_white_user_bankcards as u      where CH_COUNTRY_CODE != '' and CH_COUNTRY_CODE != 'CHN' and I_STATUS IN (1, 4) and B_ACTIVE = 1) as w    on a.CH_ID_CARD_ENCRYPT = w.CH_ID_CARD_ENCRYPT    where a.I_DATE = 202304 and a.I_AMOUNT >= 100 * 100) sql_fmt_base_0;
ERROR 1105 (HY000): Can't find column Column#40 in schema Column: [Column#37,Column#38] Unique key: []
  • 给表join字段加索引,执行失败
root@127.0.0.1 12:08:01 [test]> ALTER TABLE redpacket_white_user_bankcards add key idx_card_enc(CH_ID_CARD_ENCRYPT);
Query OK, 0 rows affected (1.38 sec)

root@127.0.0.1 12:08:25 [test]> SELECT group_concat(concat('{"key":"', CH_DATA_KEY, '","value":[', '{"key":"name1","value":"', I_AMOUNT, '"}', ']}') order by I_AMOUNT) CH_DATA_KEY  FROM (   select 'name2' as CH_DATA_KEY, round(sum(I_AMOUNT) / 100 / count(distinct a.CH_ID_CARD_ENCRYPT), 2) as I_AMOUNT         from dws_gs_bill_i18n_user_amt_mdi as a    join (     select distinct u.CH_ID_CARD_ENCRYPT from redpacket_white_user_bankcards as u      where CH_COUNTRY_CODE != '' and CH_COUNTRY_CODE != 'CHN' and I_STATUS IN (1, 4) and B_ACTIVE = 1) as w    on a.CH_ID_CARD_ENCRYPT = w.CH_ID_CARD_ENCRYPT    where a.I_DATE = 202304 and a.I_AMOUNT >= 100 * 100) sql_fmt_base_0;
ERROR 1105 (HY000): Can't find column Column#40 in schema Column: [Column#37,Column#38] Unique key: []

简化下测试用例(测试版本 7.2.0):

CREATE TABLE `t2` (
  `I_ID` bigint(20) NOT NULL AUTO_INCREMENT COMMENT '',
  `c` varchar(160) NOT NULL DEFAULT '' COMMENT '',
  PRIMARY KEY (`I_ID`) /*T![clustered_index] CLUSTERED */
) ;


SELECT group_concat(concat('{"key":"', 'name2', '","value":[', '{"key":"name1","value":"', 1, '"}', ']}') order by a) CH_DATA_KEY
FROM (
  select round(1 / count(distinct '1'), 2) as a
  from t2
  join (
    select distinct u.c from t2 as u) as w
  ) t;

SELECT group_concat(concat('{"key":"', 'name2', '","value":[', '{"key":"name1","value":"', 1, '"}', ']}') ) CH_DATA_KEY
FROM (
  select round(1 / count(distinct '1'), 2) as a
  from t2
  join (
    select distinct u.c from t2 as u) as w
  ) t;  

这里有 order by a 会报错

1 个赞

https://github.com/pingcap/tidb/issues/41986

好像github上已经有类似的issue了,可以追踪一下修复进度。

1 个赞

了解,多谢,我们看看业务层面怎么解决。

最近在升级,遇到好多sql不兼容问题。为啥不向前兼容咧~ :smiling_face_with_tear:

你之前是哪个版本呢, 之前的版本上可以执行成功么

5.0.4版本,可以执行。
主要这个问题影响了核心功能。

发现结果集不稳定提到了 GROUP_CONCAT( customer_id order by customer_id SEPARATOR ',' )

不知道是否与这个有关

https://docs.pingcap.com/zh/tidb/stable/dev-guide-unstable-result-set#由于-group_concat-中没有使用-order-by-导致结果集不稳定

大佬,官方标明使用GROUP_CONCAT order by ,可以解决结果集不稳定问题。有些奇怪。。

https://docs.pingcap.com/zh/tidb/stable/dev-guide-unstable-result-set#由于-group_concat-中没有使用-order-by-导致结果集不稳定

是中了那个问题么。。。

是的~ :rofl:

大佬,有没有其他sql改造的方法?
业务层面需要 order by 这个特性。

修复方法:在最外层子查询增加 order by 子查询

SELECT group_concat(concat('{"key":"', 'name2', '","value":[', '{"key":"name1","value":"', 1, '"}', ']}') order by a) CH_DATA_KEY
FROM (
  select round(1 / count(distinct '1'), 2) as a
  from t2
  join (
    select distinct u.c from t2 as u) as w
  order by a ) t;

此话题已在最后回复的 60 天后被自动关闭。不再允许新回复。