查询大表时报错:ERROR 1105 (HY000): Storage engine DeltaMergedoesn't support lossy data type modify from Nullable(Int32) to Nullable(Int8)

为提高效率,请提供以下信息,问题描述清晰能够更快得到解决:
【 TiDB 使用环境】

【概述】查询大表时,提示如标题报错

【背景】数据量182899658条 大小:9.14GB

【现象】查询时,提示:ERROR 1105 (HY000): Storage engine DeltaMergedoesn’t support lossy data type modify from Nullable(Int32) to Nullable(Int8)

【业务影响】数据表无法打开

【TiDB 版本】Tidbv4.0

【附件】

  1. TiUP Cluster Display 信息

  2. TiUP Cluster Edit Config 信息

  3. TiDB- Overview 监控

  • 对应模块日志(包含问题前后1小时日志)

若提问为性能优化、故障排查类问题,请下载脚本运行。终端输出的打印结果,请务必全选并复制粘贴上传。

2 个赞

执行的 sql 和 table schema 提供看下,是通过什么前端查询的?

客户端:navicat 和mysql客户端 报同样的错
table schema:
CREATE TABLE account_log (
id bigint(20) NOT NULL AUTO_INCREMENT,
type int(11) NOT NULL COMMENT ‘10-收入 20-支出’,
account_id bigint(20) NOT NULL,
user_id bigint(20) NOT NULL COMMENT ‘用户id’,
user_type bigint(20) NOT NULL COMMENT ‘用户类型’,
refer_id bigint(20) NOT NULL COMMENT ‘单据ID’,
refer_type varchar(255) NOT NULL COMMENT ‘单据类型’,
charge_money int(11) NOT NULL COMMENT ‘操作前充值金额’,
check_money int(11) NOT NULL COMMENT ‘操作前核销金额’,
profit_money int(11) NOT NULL COMMENT ‘操作前分润金额’,
amount int(11) NOT NULL COMMENT ‘金额’,
remain_charge_money int(11) NOT NULL COMMENT ‘操作后充值金额’,
remain_check_money int(11) NOT NULL COMMENT ‘操作后核销金额’,
remain_profit_money int(11) NOT NULL COMMENT ‘操作后分润金额’,
create_time datetime NOT NULL COMMENT ‘创建时间’,
PRIMARY KEY (id),
KEY account_log_id_type (user_id,user_type,refer_id,refer_type),
KEY account_log_account_id (account_id)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin AUTO_INCREMENT=111420001;

tiflash报错:
2021.08.04 15:21:35.599493 [ 18 ] SchemaSyncService: DB::SchemaSyncService::SchemaSyncService(DB::Context&)::<lambda()>: Sync schemas failed by DB::Exception: Storage engine DeltaMergedoesn’t support lossy data type modify from Nullable(Int32) to Nullable(Int8)
stack : 0. bin/tiflash/tiflash(StackTrace::StackTrace()+0x16) [0x6c130b6]

  1. bin/tiflash/tiflash(DB::Exception::Exception(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, int)+0x26) [0x30453d6]
  2. bin/tiflash/tiflash(DB::StorageDeltaMerge::alterImpl(DB::AlterCommands const&, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::optional<std::reference_wrapper<TiDB::TableInfo const> >, DB::Context const&)+0x4a5) [0x6393175]
  3. bin/tiflash/tiflash(DB::StorageDeltaMerge::alterFromTiDB(DB::AlterCommands const&, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, TiDB::TableInfo const&, DB::SchemaNameMapper const&, DB::Context const&)+0xa5) [0x6394db5]
  4. bin/tiflash/tiflash(DB::SchemaBuilder<DB::SchemaGetter, DB::SchemaNameMapper>::applyAlterPhysicalTable(std::shared_ptrTiDB::DBInfo, std::shared_ptrTiDB::TableInfo, std::shared_ptrDB::IManageableStorage)+0x373) [0x696e4f3]
  5. bin/tiflash/tiflash(DB::SchemaBuilder<DB::SchemaGetter, DB::SchemaNameMapper>::applyAlterLogicalTable(std::shared_ptrTiDB::DBInfo, std::shared_ptrTiDB::TableInfo, std::shared_ptrDB::IManageableStorage)+0x6d) [0x696eb7d]
  6. bin/tiflash/tiflash(DB::SchemaBuilder<DB::SchemaGetter, DB::SchemaNameMapper>::syncAllSchema()+0xa48) [0x69700f8]
  7. bin/tiflash/tiflash(DB::TiDBSchemaSyncer::syncSchemas(DB::Context&)+0x422) [0x6630b02]
  8. bin/tiflash/tiflash(DB::SchemaSyncService::syncSchemas()+0x2f) [0x661fc1f]
  9. bin/tiflash/tiflash() [0x662286a]
  10. bin/tiflash/tiflash(DB::BackgroundProcessingPool::threadFunction()+0x857) [0x64b9477]
  11. bin/tiflash/tiflash() [0x797277f]
  12. /lib64/libpthread.so.0(+0x7e65) [0x7f790b36ce65]
  13. /lib64/libc.so.6(clone+0x6d) [0x7f790ab8f88d]

2021.08.04 15:21:36.407929 [ 24 ] KVStore: [region 2016] handle useless admin command CompactLog at [term: 66740, index: 259167839]
2021.08.04 15:21:36.414293 [ 24 ] KVStore: Start to persist [region 2016, applied: term 66740 index 259167839], cache size: 0 bytes
2021.08.04 15:21:37.402341 [ 22 ] PageStorage: db_137.t_861.log GC exit within 0.00 sec. PageFiles from [1,0,Legacy] to [3,0,Formal], size: 3, compact legacy archive files: 0, remove data files: 0, gc apply: 0 puts and 0 refs and 0 deletes and 0 upserts

查不出来的表都是走tiflash的,从tiflash移出来,查询就正常了,能帮忙分析下这个是什么原因导致的,突然就查询不了。

是否进行过 modify 的 ddl 操作把 int 列转为 tinyint?
使用的是哪个版本?

没有修改过表结构,版本是4.0稳定版。

4.0.0?
curl http://{TiDBIP}:10080/ddl/history > ddl.txt 我们看看吧

workaround:
如果不升级,需要 truncate 进行过有损变更 DDL 的表。重新设置 tiflash 副本数解决不了。
或者升级到 4.0.6+、或者 5.0.* / 5.1.* 的版本都可以

此话题已在最后回复的 1 分钟后被自动关闭。不再允许新回复。