tiflash down掉之后无法启动

【 TiDB 使用环境`】生产环境
【 TiDB 版本】4.0.11
【遇到的问题】
今天0点,tiflash 突然down掉,并且无法启动
【复现路径】无
【问题现象及影响】
error 关键字如下:
[2022/09/28 10:47:53.846 +08:00] [ERROR] [] [“Application: Bootstrap failed because sync schema error: Syntax error: Not a valid integer: 0.00\ We will sleep for 3 seconds and try again.”] [thread_id=1]

日志里面出现了大量的synced信息:
[2022/09/28 10:52:15.061 +08:00] [INFO] [] [“SchemaBuilder: No schema change detected for table qrp(45756).users_to_role(45954), not altering”] [thread_id=1]
[2022/09/28 10:52:15.061 +08:00] [DEBUG] [] [“SchemaBuilder: Table qrp(45756).users_to_role(45954) synced during sync all schemas”] [thread_id=1]
[2022/09/28 10:52:15.061 +08:00] [DEBUG] [] [“SchemaBuilder: Table qrp(45756).trade_jnls_201904(45956) syncing during sync all schemas”] [thread_id=1]
[2022/09/28 10:52:15.061 +08:00] [DEBUG] [] [“SchemaBuilder: Table qrp(45756).trade_jnls_201904(45956) name identical, not renaming.”] [thread_id=1]
[2022/09/28 10:52:15.061 +08:00] [INFO] [] [“SchemaBuilder: Altering table qrp(45756).trade_jnls_201904(45956)”] [thread_id=1]
[2022/09/28 10:52:15.061 +08:00] [INFO] [] [“SchemaBuilder: No schema change detected for table qrp(45756).trade_jnls_201904(45956), not altering”] [thread_id=1]
[2022/09/28 10:52:15.061 +08:00] [DEBUG] [] [“SchemaBuilder: Table qrp(45756).trade_jnls_201904(45956) synced during sync all schemas”] [thread_id=1]
[2022/09/28 10:52:15.061 +08:00] [DEBUG] [] [“SchemaBuilder: Table qrp(45756).trade_jnls_201905(45958) syncing during sync all schemas”] [thread_id=1]
[2022/09/28 10:52:15.061 +08:00] [DEBUG] [] [“SchemaBuilder: Table qrp(45756).trade_jnls_201905(45958) name identical, not renaming.”] [thread_id=1]
[2022/09/28 10:52:15.061 +08:00] [INFO] [] [“SchemaBuilder: Altering table qrp(45756).trade_jnls_201905(45958)”] [thread_id=1]
[2022/09/28 10:52:15.061 +08:00] [INFO] [] [“SchemaBuilder: No schema change detected for table qrp(45756).trade_jnls_201905(45958), not altering”] [thread_id=1]
[2022/09/28 10:52:15.061 +08:00] [DEBUG] [] [“SchemaBuilder: Table qrp(45756).trade_jnls_201905(45958) synced during sync all schemas”] [thread_id=1]
[2022/09/28 10:52:15.061 +08:00] [DEBUG] [] [“SchemaBuilder: Table qrp(45756).channel_lines_limit(45960) syncing during sync all schemas”] [thread_id=1]
[2022/09/28 10:52:15.061 +08:00] [DEBUG] [] [“SchemaBuilder: Table qrp(45756).channel_lines_limit(45960) name identical, not renaming.”] [thread_id=1]
[2022/09/28 10:52:15.061 +08:00] [INFO] [] [“SchemaBuilder: Altering table qrp(45756).channel_lines_limit(45960)”] [thread_id=1]
[2022/09/28 10:52:15.061 +08:00] [INFO] [] [“SchemaBuilder: No schema change detected for table qrp(45756).channel_lines_limit(45960), not altering”] [thread_id=1]
[2022/09/28 10:52:15.061 +08:00] [DEBUG] [] [“SchemaBuilder: Table qrp(45756).channel_lines_limit(45960) synced during sync all schemas”] [thread_id=1]
[2022/09/28 10:52:15.061 +08:00] [DEBUG] [] [“SchemaBuilder: Table qrp(45756).tmp_buxiaofan_220206(45965) syncing during sync all schemas”] [thread_id=1]
[2022/09/28 10:52:15.061 +08:00] [DEBUG] [] [“SchemaBuilder: Table qrp(45756).tmp_buxiaofan_220206(45965) name identical, not renaming.”] [thread_id=1]
[2022/09/28 10:52:15.061 +08:00] [INFO] [] [“SchemaBuilder: Altering table qrp(45756).tmp_buxiaofan_220206(45965)”] [thread_id=1]
[2022/09/28 10:52:15.061 +08:00] [INFO] [] [“SchemaBuilder: No schema change detected for table qrp(45756).tmp_buxiaofan_220206(45965), not altering”] [thread_id=1]
[2022/09/28 10:52:15.061 +08:00] [DEBUG] [] [“SchemaBuilder: Table qrp(45756).tmp_buxiaofan_220206(45965) synced during sync all schemas”] [thread_id=1]
[2022/09/28 10:52:15.061 +08:00] [DEBUG] [] [“SchemaBuilder: Table qrp(45756).trade_jnls_202202(46189) syncing during sync all schemas”] [thread_id=1]
[2022/09/28 10:52:15.061 +08:00] [DEBUG] [] [“SchemaBuilder: Table qrp(45756).trade_jnls_202202(46189) name identical, not renaming.”] [thread_id=1]
[2022/09/28 10:52:15.061 +08:00] [INFO] [] [“SchemaBuilder: Altering table qrp(45756).trade_jnls_202202(46189)”] [thread_id=1]
[2022/09/28 10:52:15.061 +08:00] [INFO] [] [“SchemaBuilder: No schema change detected for table qrp(45756).trade_jnls_202202(46189), not altering”] [thread_id=1]
[2022/09/28 10:52:15.061 +08:00] [DEBUG] [] [“SchemaBuilder: Table qrp(45756).trade_jnls_202202(46189) synced during sync all schemas”] [thread_id=1]
[2022/09/28 10:52:15.061 +08:00] [DEBUG] [] [“SchemaBuilder: Table qrp(45756).trade_jnls_202203(54001) syncing during sync all schemas”] [thread_id=1]
[2022/09/28 10:52:15.061 +08:00] [DEBUG] [] [“SchemaBuilder: Table qrp(45756).trade_jnls_202203(54001) name identical, not renaming.”] [thread_id=1]
[2022/09/28 10:52:15.061 +08:00] [INFO] [] [“SchemaBuilder: Altering table qrp(45756).trade_jnls_202203(54001)”] [thread_id=1]
[2022/09/28 10:52:15.061 +08:00] [INFO] [] [“SchemaBuilder: No schema change detected for table qrp(45756).trade_jnls_202203(54001), not altering”] [thread_id=1]
[2022/09/28 10:52:15.061 +08:00] [DEBUG] [] [“SchemaBuilder: Table qrp(45756).trade_jnls_202203(54001) synced during sync all schemas”] [thread_id=1]
[2022/09/28 10:52:15.061 +08:00] [DEBUG] [] [“SchemaBuilder: Table qrp(45756).trade_jnls_202204(55433) syncing during sync all schemas”] [thread_id=1]
[2022/09/28 10:52:15.061 +08:00] [DEBUG] [] [“SchemaBuilder: Table qrp(45756).trade_jnls_202204(55433) name identical, not renaming.”] [thread_id=1]
[2022/09/28 10:52:15.061 +08:00] [INFO] [] [“SchemaBuilder: Altering table qrp(45756).trade_jnls_202204(55433)”] [thread_id=1]
[2022/09/28 10:52:15.061 +08:00] [INFO] [] [“SchemaBuilder: No schema change detected for table qrp(45756).trade_jnls_202204(55433), not altering”] [thread_id=1]
[2022/09/28 10:52:15.061 +08:00] [DEBUG] [] [“SchemaBuilder: Table qrp(45756).trade_jnls_202204(55433) synced during sync all schemas”] [thread_id=1]
[2022/09/28 10:52:15.061 +08:00] [DEBUG] [] [“SchemaBuilder: Table qrp(45756).trade_jnls_202205(62487) syncing during sync all schemas”] [thread_id=1]
[2022/09/28 10:52:15.061 +08:00] [DEBUG] [] [“SchemaBuilder: Table qrp(45756).trade_jnls_202205(62487) name identical, not renaming.”] [thread_id=1]
[2022/09/28 10:52:15.061 +08:00] [INFO] [] [“SchemaBuilder: Altering table qrp(45756).trade_jnls_202205(62487)”] [thread_id=1]
[2022/09/28 10:52:15.061 +08:00] [INFO] [] [“SchemaBuilder: No schema change detected for table qrp(45756).trade_jnls_202205(62487), not altering”] [thread_id=1]
[2022/09/28 10:52:15.061 +08:00] [DEBUG] [] [“SchemaBuilder: Table qrp(45756).trade_jnls_202205(62487) synced during sync all schemas”] [thread_id=1]
[2022/09/28 10:52:15.061 +08:00] [DEBUG] [] [“SchemaBuilder: Table qrp(45756).trade_jnls_202206(63616) syncing during sync all schemas”] [thread_id=1]
[2022/09/28 10:52:15.061 +08:00] [DEBUG] [] [“SchemaBuilder: Table qrp(45756).trade_jnls_202206(63616) name identical, not renaming.”] [thread_id=1]
[2022/09/28 10:52:15.061 +08:00] [INFO] [] [“SchemaBuilder: Altering table qrp(45756).trade_jnls_202206(63616)”] [thread_id=1]
[2022/09/28 10:52:15.061 +08:00] [INFO] [] [“SchemaBuilder: No schema change detected for table qrp(45756).trade_jnls_202206(63616), not altering”] [thread_id=1]
[2022/09/28 10:52:15.061 +08:00] [DEBUG] [] [“SchemaBuilder: Table qrp(45756).trade_jnls_202206(63616) synced during sync all schemas”] [thread_id=1]
[2022/09/28 10:52:15.061 +08:00] [DEBUG] [] [“SchemaBuilder: Table qrp(45756).trade_jnls_202207(68398) syncing during sync all schemas”] [thread_id=1]
[2022/09/28 10:52:15.061 +08:00] [DEBUG] [] [“SchemaBuilder: Table qrp(45756).trade_jnls_202207(68398) name identical, not renaming.”] [thread_id=1]
[2022/09/28 10:52:15.061 +08:00] [INFO] [] [“SchemaBuilder: Altering table qrp(45756).trade_jnls_202207(68398)”] [thread_id=1]
[2022/09/28 10:52:15.062 +08:00] [INFO] [] [“SchemaBuilder: No schema change detected for table qrp(45756).trade_jnls_202207(68398), not altering”] [thread_id=1]
[2022/09/28 10:52:15.062 +08:00] [DEBUG] [] [“SchemaBuilder: Table qrp(45756).trade_jnls_202207(68398) synced during sync all schemas”] [thread_id=1]
[2022/09/28 10:52:15.062 +08:00] [DEBUG] [] [“SchemaBuilder: Table qrp(45756).trade_jnls_202208(73754) syncing during sync all schemas”] [thread_id=1]
[2022/09/28 10:52:15.062 +08:00] [DEBUG] [] [“SchemaBuilder: Table qrp(45756).trade_jnls_202208(73754) name identical, not renaming.”] [thread_id=1]
[2022/09/28 10:52:15.062 +08:00] [INFO] [] [“SchemaBuilder: Altering table qrp(45756).trade_jnls_202208(73754)”] [thread_id=1]
[2022/09/28 10:52:15.062 +08:00] [INFO] [] [“SchemaBuilder: No schema change detected for table qrp(45756).trade_jnls_202208(73754), not altering”] [thread_id=1]
[2022/09/28 10:52:15.062 +08:00] [DEBUG] [] [“SchemaBuilder: Table qrp(45756).trade_jnls_202208(73754) synced during sync all schemas”] [thread_id=1]
[2022/09/28 10:52:15.062 +08:00] [DEBUG] [] [“SchemaBuilder: Database qrp(45756) synced during sync all schemas”] [thread_id=1]

【附件】 相关日志及监控(https://metricstool.pingcap.com/)

参下这个 现了 int 类型列的 default value 为 0.00 ,最后重建的表

此话题已在最后回复的 60 天后被自动关闭。不再允许新回复。