tidb-lightning导入数据more than 1000 empty regions

tidb版本:5.2
部署方式:k8s
问题:
使用tidb-lightning导入数据,报错:
TiKV stores (4, 5, 1) contains more than 1000 empty regions respectively, which will greatly affect the import speed and success rate

查询INFORMATION_SCHEMA.TIKV_REGION_STATUS确实有很多 empty regions
image

根据官方建议的,默认是开启region merge:

./pd-ctl config show
{
“replication”: {
“enable-placement-rules”: “true”,
“isolation-level”: “”,
“location-labels”: “”,
“max-replicas”: 3,
“strictly-match-label”: “false”
},
“schedule”: {
“enable-cross-table-merge”: “true”,
“enable-joint-consensus”: “true”,
“high-space-ratio”: 0.7,
“hot-region-cache-hits-threshold”: 3,
“hot-region-schedule-limit”: 4,
“leader-schedule-limit”: 4,
“leader-schedule-policy”: “count”,
“low-space-ratio”: 0.8,
“max-merge-region-keys”: 200000,
“max-merge-region-size”: 20,
“max-pending-peer-count”: 16,
“max-snapshot-count”: 3,
“max-store-down-time”: “30m0s”,
“merge-schedule-limit”: 8,
“patrol-region-interval”: “100ms”,
“region-schedule-limit”: 2048,
“region-score-formula-version”: “v2”,
“replica-schedule-limit”: 64,
“split-merge-interval”: “1h0m0s”,
“tolerant-size-ratio”: 0
}

./pd-ctl config show all | grep enable
“enable-grpc-gateway”: true,
“enable-local-tso”: false,
“enable-one-way-merge”: “true”,
“enable-cross-table-merge”: “true”,
“enable-remove-down-replica”: “true”,
“enable-replace-offline-replica”: “true”,
“enable-make-up-replica”: “true”,
“enable-remove-extra-replica”: “true”,
“enable-location-replacement”: “true”,
“enable-debug-metrics”: “false”,
“enable-joint-consensus”: “true”,
“enable-placement-rules”: “true”,
“enable-telemetry”: true,
“enable-experimental”: false

TIDB集群目前只是用作一个测试使用,基本没连接和数据

如何解决这个问题呢?

1 个赞

没连接也数据,也需要将 空的region做合并, 空的region 存在就会额外的消耗计算资源和网络资源

你可以先通过调度来合并掉空的region,然后在 import data…

2 个赞

这个要怎么调度合并空的region呢?

1 个赞

搜下就能找到了

参考下面的
https://docs.pingcap.com/zh/tidb/stable/pd-scheduling-best-practices#region-merge
https://docs.pingcap.com/zh/tidb/stable/pd-scheduling-best-practices#region-merge-速度慢

1 个赞

把这两个值调小一点加速空region合并。
还有merge-schedule-limit可以适当调大点。

1 个赞

好,我试下

1 个赞

用完记得调回默认值

1 个赞

此话题已在最后回复的 1 分钟后被自动关闭。不再允许新回复。