查询引擎只配置为tiflash、tidb，而不配置tikv，查询报错

wenyi · 2023 年5 月 9 日 06:50

【 TiDB 使用环境】测试
【 TiDB 版本】6.5.2
set @@session.tidb_isolation_read_engines = “tiflash,tidb”;
【资源配置】
3个pd、tidb，部署在一起，每个节点是32核cpu、128G内存，机械磁盘，即每个节点部署了tidb与pd
3个tikv，每个节点是32核cpu、128G内存，SSD磁盘
1个tiflash，每个节点是32核cpu、256G内存，SSD磁盘
查询报错

具体日志：
[2023/05/09 14:50:10.630 +08:00] [ERROR] [MPPTask.cpp:429] [“task running meets error: Code: 0, e.displayText() = DB::Exception: write to tunnel which is already closed,Receiver cancelled, reason: Push mpp packet failed. Receiver state: CLOSED, e.what() = DB::Exception, Stack trace:\n\n\n 0x17221ce\tDB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&, int) [tiflash+24256974]\n \tdbms/src/Common/Exception.h:46\n 0x6e23f34\tDB::MPPTunnel::write(std::__1::shared_ptrDB::TrackedMppDataPacket&&) [tiflash+115490612]\n \tdbms/src/Flash/Mpp/MPPTunnel.cpp:168\n 0x16f5d6c\tDB::HashPartitionWriter<std::__1::shared_ptrDB::MPPTunnelSet >::partitionAndEncodeThenWriteBlocks() [tiflash+24075628]\n \tdbms/src/Flash/Mpp/HashPartitionWriter.cpp:103\n 0x6dc7042\tDB::ExchangeSenderBlockInputStream::readImpl() [tiflash+115109954]\n \tdbms/src/DataStreams/ExchangeSenderBlockInputStream.cpp:43\n 0x61609a5\tDB::IProfilingBlockInputStream::read(DB::PODArray<unsigned char, 4096ul, Allocator, 15ul, 16ul>&, bool) [tiflash+102107557]\n \tdbms/src/DataStreams/IProfilingBlockInputStream.cpp:75\n 0x6160695\tDB::IProfilingBlockInputStream::read() [tiflash+102106773]\n \tdbms/src/DataStreams/IProfilingBlockInputStream.cpp:43\n 0x6dcd95e\tDB::ParallelInputsProcessor<DB::UnionBlockInputStream<(DB::StreamUnionMode)0, true>::Handler, (DB::StreamUnionMode)0>::work(unsigned long, DB::ParallelInputsProcessor<DB::UnionBlockInputStream<(DB::StreamUnionMode)0, true>::Handler, (DB::StreamUnionMode)0>::WorkingInputs&) [tiflash+115136862]\n \tdbms/src/DataStreams/ParallelInputsProcessor.h:270\n 0x6dcd476\tstd::__1::__function::__func<DB::ParallelInputsProcessor<DB::UnionBlockInputStream<(DB::StreamUnionMode)0, true>::Handler, (DB::StreamUnionMode)0>::process()::‘lambda’(), std::__1::allocator<DB::ParallelInputsProcessor<DB::UnionBlockInputStream<(DB::StreamUnionMode)0, true>::Handler, (DB::StreamUnionMode)0>::process()::‘lambda’()>, void ()>::operator()() [tiflash+115135606]\n \t/usr/local/bin/…/include/c++/v1/__functional/function.h:345\n 0x1804c9b\tDB::ExecutableTask<std::__1::packaged_task<void ()> >::execute() [tiflash+25185435]\n \tdbms/src/Common/ExecutableTask.h:52\n 0x18081e3\tDB::DynamicThreadPool::executeTask(std::__1::unique_ptr<DB::IExecutableTask, std::__1::default_deleteDB::IExecutableTask >&) [tiflash+25199075]\n \tdbms/src/Common/DynamicThreadPool.cpp:101\n 0x1807840\tDB::DynamicThreadPool::fixedWork(unsigned long) [tiflash+25196608]\n \tdbms/src/Common/DynamicThreadPool.cpp:115\n 0x1808932\tvoid std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_deletestd::__1::__thread_struct >, std::__1::thread DB::ThreadFactory::newThread<void (DB::DynamicThreadPool::)(unsigned long), DB::DynamicThreadPool, unsigned long&>(bool, std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator >, void (DB::DynamicThreadPool::&&)(unsigned long), DB::DynamicThreadPool&&, unsigned long&)::‘lambda’(auto&&…), DB::DynamicThreadPool*, unsigned long> >(void*) [tiflash+25200946]\n \t/usr/local/bin/…/include/c++/v1/thread:291\n 0x7f177819eea5\tstart_thread [libpthread.so.0+32421]\n 0x7f17775a396d\t__clone [libc.so.6+1042797]”] [source=MPPquery:441349571902701578:3,task] [thread_id=281]
[2023/05/09 14:50:10.630 +08:00] [WARN] [MPPTaskManager.cpp:152] [“Begin to abort query: 441349571902701578, abort type: ONERROR, reason: From MPPquery:441349571902701578:3,task: Code: 0, e.displayText() = DB::Exception: write to tunnel which is already closed,Receiver cancelled, reason: Push mpp packet failed. Receiver state: CLOSED, e.what() = DB::Exception,”] [thread_id=281]
[2023/05/09 14:50:10.630 +08:00] [WARN] [MPPTaskManager.cpp:195] ["Remaining task in query 441349571902701578 are: MPPquery:441349571902701578:1,task MPPquery:441349571902701578:12,task MPPquery:441349571902701578:11,task MPPquery:441349571902701578:13,task MPPquery:441349571902701578:3,task "] [thread_id=281]
[2023/05/09 14:50:10.630 +08:00] [WARN] [MPPTask.cpp:471] [“Begin abort task: MPPquery:441349571902701578:1,task, abort type: ONERROR”] [source=MPPquery:441349571902701578:1,task] [thread_id=281]
[2023/05/09 14:50:10.630 +08:00] [WARN] [MPPTask.cpp:500] [“Finish abort task from running”] [source=MPPquery:441349571902701578:1,task] [thread_id=281]
[2023/05/09 14:50:10.630 +08:00] [WARN] [MPPTask.cpp:471] [“Begin abort task: MPPquery:441349571902701578:12,task, abort type: ONERROR”] [source=MPPquery:441349571902701578:12,task] [thread_id=281]
[2023/05/09 14:50:10.630 +08:00] [WARN] [MPPTask.cpp:500] [“Finish abort task from running”] [source=MPPquery:441349571902701578:12,task] [thread_id=281]
[2023/05/09 14:50:10.631 +08:00] [WARN] [MPPTask.cpp:471] [“Begin abort task: MPPquery:441349571902701578:11,task, abort type: ONERROR”] [source=MPPquery:441349571902701578:11,task] [thread_id=281]
[2023/05/09 14:50:10.632 +08:00] [WARN] [MPPTask.cpp:500] [“Finish abort task from running”] [source=MPPquery:441349571902701578:11,task] [thread_id=281]
[2023/05/09 14:50:10.632 +08:00] [WARN] [MPPTask.cpp:471] [“Begin abort task: MPPquery:441349571902701578:13,task, abort type: ONERROR”] [source=MPPquery:441349571902701578:13,task] [thread_id=281]
[2023/05/09 14:50:10.632 +08:00] [WARN] [MPPTask.cpp:500] [“Finish abort task from running”] [source=MPPquery:441349571902701578:13,task] [thread_id=281]
[2023/05/09 14:50:10.633 +08:00] [WARN] [MPPTask.cpp:471] [“Begin abort task: MPPquery:441349571902701578:3,task, abort type: ONERROR”] [source=MPPquery:441349571902701578:3,task] [thread_id=281]
[2023/05/09 14:50:10.633 +08:00] [WARN] [MPPTask.cpp:500] [“Finish abort task from running”] [source=MPPquery:441349571902701578:3,task] [thread_id=281]
[2023/05/09 14:50:10.633 +08:00] [WARN] [MPPTaskManager.cpp:207] [“Finish abort query: 441349571902701578”] [thread_id=281]
[2023/05/09 14:50:10.647 +08:00] [WARN] [MPPTaskManager.cpp:152] [“Begin to abort query: 441349571902701578, abort type: ONCANCELLATION, reason: Receive cancel request from TiDB”] [thread_id=140]
[2023/05/09 14:50:10.647 +08:00] [WARN] [MPPTaskManager.cpp:167] [“441349571902701578 already in abort process, skip abort”] [thread_id=140]

WalterWj · 2023 年5 月 9 日 06:52

去问题反馈区反馈下感觉不预期。

wenyi · 2023 年5 月 9 日 06:53

感觉不逾期这么理解？

TiDBer_UUTlqVvZ · 2023 年5 月 9 日 08:00

我来试试。根据您提供的报错信息，这是一个 MPP 相关的错误。可能是由于 TiFlash 节点的 MPP Stream 发生了异常，导致查询失败。
TiFlash 节点的 MPP Stream 发生异常，导致查询失败的可能原因有很多，以下是一些常见的原因：

网络连接问题：TiFlash 节点与 TiDB/TiKV 节点之间的网络连接不稳定或者带宽不足，导致 MPP Stream 传输失败。
资源限制问题：TiFlash 节点的资源限制不足，例如内存、CPU 等，导致 MPP Stream 处理失败。
TiFlash 版本问题：TiFlash 版本过旧或者存在已知的 Bug，导致 MPP Stream 处理失败。
查询语句问题：查询语句本身存在问题，例如语法错误、数据类型不匹配等，导致 MPP Stream 处理失败。
TiDB 集群配置问题：TiDB 集群的配置存在问题，例如 TiDB/TiKV/TiFlash 节点的版本不兼容、参数配置不正确等，导致 MPP Stream 处理失败。
针对这些问题，您可以逐一排查，以确定具体的原因。

system · 2023 年7 月 8 日 08:01

此话题已在最后回复的 60 天后被自动关闭。不再允许新回复。