背景:使用数据校验工具进行上下游mysql 一致性校验
上下游mysql 使用的是dm 进行数据同步
######################### Global config #########################
# 检查数据的线程数量,上下游数据库的连接数会略大于该值
check-thread-count = 8
# 如果开启,若表存在不一致,则输出用于修复的 SQL 语句。
export-fix-sql = true
# 只对比表结构而不对比数据
check-struct-only = false
######################### Datasource config #########################
[data-sources]
[data-sources.mysql1]
host = "10.105.xxxx"
port = 4012
user = "tidb_worker"
password = "xxxxxxxx"
route-rules = ["rule1"]
[data-sources.tidb0]
host = "10.105.xxxx"
port = 3862
user = "test123_app"
password = "58IR_rxxxx"
########################### Routes ###########################
[routes]
[routes.rule1]
schema-pattern = "tidb_sync"
target-schema = "test123"
######################### Task config #########################
# 配置需要对比的*目标数据库*中的表
[task]
output-dir = "/home/tidb/output"
source-instances = ["mysql1"]
target-instance = "tidb0"
target-check-tables = ["test123.*"]
在静态的情况下可以通过校验,但是在上游有写入的时候,校验同步进度就会卡主。
请问此工具是否可以进行mysql → mysql 的数据实时同步 ?
上面的配置是否有需要调整的地方 ?
===> 在热心网友的建议下,这次降低了上游的请求并发,检验可以跑一会,但是最终还是以失败停止。
前台日志如下
报错信息如下:
[2022/06/13 18:28:01.477 +08:00] [FATAL] [diff.go:565] ["the isEqual1 and isEqual2 cannot be both true"] [stack="main.(*Diff).binSearch\
\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb-tools/sync_diff_inspector/diff.go:565\
main.(*Diff).BinGenerate\
\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb-tools/sync_diff_inspector/diff.go:496\
main.(*Diff).consume\
\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb-tools/sync_diff_inspector/diff.go:435\
main.(*Diff).Equal.func2\
\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb-tools/sync_diff_inspector/diff.go:284\
github.com/pingcap/tidb-tools/sync_diff_inspector/utils.(*WorkerPool).Apply.func1\
\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb-tools/sync_diff_inspector/utils/utils.go:75"]