5.3版本 ddl更改decimal格式 报#1105错误码 现在开启tiflash的表全部读不了 新建的表也无法读取

用那个tiup dumpling 想把表导出为sql,用了命令,也不生效,救救孩子吧:clown_face:
dumpling -u root -p *** -P 4000 -h *** --filetype sql -t 4 -o /jumpserver_data/tidb_data -F 256MiB -T data_center.product_pond_v4

重启tikv的时候,就报的下面这个错误。
[FATAL] [lib.rs:465] [“assertion failed: start < end”] [backtrace=" 0: tikv_util::set_panic_hook::{{closure}}\ at /home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tikv/components/tikv_util/src/lib.rs:464:18\ 1: std::panicking::rust_panic_with_hook\ at /rustc/2faabf579323f5252329264cc53ba9ff803429a3/library/std/src/panicking.rs:626:17\ 2: std::panicking::begin_panic_handler::{{closure}}\ at /rustc/2faabf579323f5252329264cc53ba9ff803429a3/library/std/src/panicking.rs:517:13\ 3: std::sys_common::backtrace::__rust_end_short_backtrace\ at /rustc/2faabf579323f5252329264cc53ba9ff803429a3/library/std/src/sys_common/backtrace.rs:141:18\ 4: rust_begin_unwind\ at /rustc/2faabf579323f5252329264cc53ba9ff803429a3/library/std/src/panicking.rs:515:5\ 5: core::panicking::panic_fmt\ at /rustc/2faabf579323f5252329264cc53ba9ff803429a3/library/core/src/panicking.rs:92:14\ 6: core::panicking::panic\ at /rustc/2faabf579323f5252329264cc53ba9ff803429a3/library/core/src/panicking.rs:50:5\ 7: engine_rocks::misc::::roughly_cleanup_ranges\ at /home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tikv/components/engine_rocks/src/misc.rs:231:13\ 8: raftstore::store::fsm::store::RaftPollerBuilder<EK,ER,T>::clear_stale_data\ at /home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tikv/components/raftstore/src/store/fsm/store.rs:1106:9\ raftstore::store::fsm::store::RaftPollerBuilder<EK,ER,T>::init\ at /home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tikv/components/raftstore/src/store/fsm/store.rs:1070:9\ 9: raftstore::store::fsm::store::RaftBatchSystem<EK,ER>::spawn\ at /home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tikv/components/raftstore/src/store/fsm/store.rs:1332:28\ 10: tikv::server::node::Node<C,EK,ER>::start_store\ at /home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tikv/src/server/node.rs:456:9\ tikv::server::node::Node<C,EK,ER>::start\ at /home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tikv/src/server/node.rs:199:9\ 11: server::server::TiKVServer::init_servers\ at /home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tikv/components/server/src/server.rs:831:9\ 12: server::server::run_tikv\ at /home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tikv/components/server/src/server.rs:156:9\ 13: tikv_server::main\ at /home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tikv/cmd/tikv-server/src/main.rs:181:5\ 14: core::ops::function::FnOnce::call_once\ at /rustc/2faabf579323f5252329264cc53ba9ff803429a3/library/core/src/ops/function.rs:227:5\ std::sys_common::backtrace::__rust_begin_short_backtrace\ at /rustc/2faabf579323f5252329264cc53ba9ff803429a3/library/std/src/sys_common/backtrace.rs:125:18\ 15: main\ 16: __libc_start_main\ 17: \ "] [location=components/engine_rocks/src/misc.rs:231] [thread_name=main]

1 个赞

能上传一份完整的tikv.log和tidb.log吗,方便分析
p.s:我只是热心群众:joy:

2 个赞
    • 如果你需要获得快速 “加急”处理问题的权限,加快问题响应速度, 点击完成认证,获得“加急”处理问题的权限,方便你更快速地解决问题。
      认证后在 导航栏:我的团队-全部主题-加急,直接加急你的问题

发现反馈了好几个问题,现在状况是啥,建议提供一下日志之类的信息,不然只提供现象,没法排查

没有看到 tiflash 的日志,无法 100% 确认问题。但从描述的操作以及报错信息来看,判断5楼及之前的问题是这个 bug 引起的:https://github.com/pingcap/tidb/issues/31048

在需要对列数据进行重写的 ddl 操作执行期间,再对同一列进行 ddl 操作,会触发一个 bug,导致 tidb 的表信息错误。进而导致 tiflash 同步时发生异常,这个发生之后,会导致第 5 楼及之前描述的 “tiflash 的表全部读不了,新建的 tiflash 表也无法读取” 的问题。
在 [5.1.0~5.1.3], [5.2.0~5.2.3], [5.3.0] 上都存在这个问题,在升级到修复版本之前,建议对 ddl 权限进行限制。目前在 master 上已经修复,下一个小版本会带上修复。

可以先通过 set @@session.tidb_isolation_read_engines = "tikv" tidb_isolation_read_engines 来让 SQL 查询都走 tikv 进行查询,确认下 tikv 上的数据能否正常读写。

1 个赞

此话题已在最后回复的 1 分钟后被自动关闭。不再允许新回复。