DM配置

为提高效率,提问时请尽量提供详细背景信息,问题描述清晰可优先响应。以下信息点请尽量提供:

  • 系统版本 & kernel 版本】centos7
  • TiDB 版本】3.0.2
  • 磁盘型号】ssd
  • 集群节点分布】DM集群在一台主机上面配置
  • 数据量 & region 数量 & 副本数】4万多
  • 问题描述(我做了什么)】由于资源有限,DM使用ansible 安装在tidb集群的ansible主机上面。目前仅实现将上游mysql实例的库,同步到下游tidb中。
  • 关键词】 task.yaml配置文件 [tidb@al-hb2-g-mysql-3-136 dmctl]$ cat task.yaml name: “one-tidb-slave” task-mode: all meta-schema: “dm_meta” remove-meta: false

target-database: host: “10.243.0.0” port: 4000 user: “root” password: “hM=”

mysql-instances:

source-id: "mysql-replica-01"

route-rules: [“instance-1-user-rule”]

filter-rules: [“log-filter-rule”, “user-filter-rule” , “store-filter-rule”]

black-white-list: “log-ignored”

mydumper-config-name: "global"
loader-config-name: "global"
syncer-config-name: "global"

#所有实例的共有配置 routes: instance-1-user-rule: schema-pattern: “wbcomic_logs” target-schema: “wbcomic_logs”

#filters:

log-filter-rule:

schema-pattern: “user”

table-pattern: “log”

events: [“truncate table”, “drop table”, “delete”]

action: Ignore

user-filter-rule:

schema-pattern: “user”

events: [“drop database”]

action: Ignore

store-filter-rule:

schema-pattern: “store”

events: [“drop database”, “truncate table”, “drop table”, “delete”]

action: Ignore

black-white-list: log-ignored: ignore-dbs: [“ceshi,mysql”]

mydumpers: global: threads: 4 chunk-filesize: 6 skip-tz-utc: true

loaders: global: pool-size: 16 dir: “./dumped_data”

syncers: global: worker-count: 16 batch: 100 max-retry: 100

dm-master.yaml

[tidb@al-hb2-g-mysql-3-136 conf]$ cat dm-master.toml

Master Configuration.

[[deploy]] source-id = “mysql-replica-01” dm-worker = “10.243.0.0:8262”

[[deploy]] source-id = “mysql-replica-02” dm-worker = “10.243.3.136:8262”

===================================== inventry.ini

[tidb@al-hb2-g-mysql-3-136 dm-ansible]$ cat inventory.ini

DM modules

[dm_master_servers] dm_master ansible_host=10.243.1.0

[dm_worker_servers] dm-worker1 ansible_host=10.243.1.0 source_id=“mysql-replica-01” server_id=101 mysql_host=10.243.9.128 mysql_user=root mysql_password=5 mysql_port=171011 #dm-worker1 ansible_host=10.243.1.0 source_id=“mysql-replica-02” server_id=102 mysql_host=10.243.9.128 mysql_user=root mysql_password=5 mysql_port=17102

[dm_portal_servers] dm_portal ansible_host=10.243.1.0

Monitoring modules

[prometheus_servers] prometheus ansible_host=10.243.1.0 #prometheus prometheus_port=9091 [grafana_servers] grafana ansible_host=10.243.1.0 #grafana grafana_port=3001 [alertmanager_servers] alertmanager ansible_host=10.243.1.0

Global variables

[all:vars] cluster_name = Vmanhua-cluster

ansible_user = tidb

dm_version = latest

deploy_dir = /home/tidb/dm

grafana_admin_user = “admin” grafana_admin_password = “”

dm-master.toml 配置文件中:

source_id 映射到 worker 的是 10.243.0.0:8262 不是你目标配置的 worker 的地址和端口,从 ini 文件里面看到,worker 配置的地址是:10.243.1.0。这个需要修正 dm-master 配置文件 worker 为正确的 ip 地址和端口。

修改后,再尝试启动看下~