AWS_tidb-部署_扩容DM

🪐 TiDB 技术问题
,
1

【TiDB 使用环境】生产环境 /测试/ Poc
【TiDB 版本】8.5.4
【操作系统】 aws linux 2023
【部署方式】AWS
【集群数据量】 30G
【集群节点数】8
【问题复现路径】做过哪些操作出现的问题
【遇到的问题：问题现象及影响】

----装好之后就是迁移数据了,结果发现少了个重要的东西DM,于是… 考虑在编扩容,以下为操作记录:

[ec2-user@tidb-adm ~]$ tiup cluster display tidb-test
Cluster type: tidb
Cluster name: tidb-test
Cluster version: v8.5.4
Deploy user: tidb
SSH type: builtin
Dashboard URL: http://172.244.0.4:2379/dashboard
Dashboard URLs: http://172.244.0.4:2379/dashboard
Grafana URL: http://172.244.0.76:3000
ID Role Host Ports OS/Arch Status Data Dir Deploy Dir

172.244.0.76:9093 alertmanager 172.244.0.76 9093/9094 linux/x86_64 Up /tidb-data/alertmanager-9093 /tidb-deploy/alertmanager-9093
172.244.0.83:8300 cdc 172.244.0.83 8300 linux/x86_64 Up /tidb-data/cdc-8300 /tidb-deploy/cdc-8300
172.244.0.76:3000 grafana 172.244.0.76 3000 linux/x86_64 Up - /tidb-deploy/grafana-3000
172.244.0.109:2379 pd 172.244.0.109 2379/2380 linux/x86_64 Up|L /tidb-data/pd-2379 /tidb-deploy/pd-2379
172.244.0.4:2379 pd 172.244.0.4 2379/2380 linux/x86_64 Up|UI /tidb-data/pd-2379 /tidb-deploy/pd-2379
172.244.0.62:2379 pd 172.244.0.62 2379/2380 linux/x86_64 Up /tidb-data/pd-2379 /tidb-deploy/pd-2379
172.244.0.76:9090 prometheus 172.244.0.76 9090/9115/9100/12020 linux/x86_64 Up /tidb-data/prometheus-9090 /tidb-deploy/prometheus-9090
172.244.0.33:4000 tidb 172.244.0.33 4000/10080 linux/x86_64 Up - /tidb-deploy/tidb-4000
172.244.0.99:9000 tiflash 172.244.0.99 9000/3930/20170/20292/8234/8123 linux/x86_64 Up /tidb-data/tiflash-9000 /tidb-deploy/tiflash-9000
172.244.0.122:20160 tikv 172.244.0.122 20160/20180 linux/x86_64 Up /tidb-data/tikv-20160 /tidb-deploy/tikv-20160
172.244.0.72:20160 tikv 172.244.0.72 20160/20180 linux/x86_64 Up /tidb-data/tikv-20160 /tidb-deploy/tikv-20160
172.244.0.95:20160 tikv 172.244.0.95 20160/20180 linux/x86_64 Up /tidb-data/tikv-20160 /tidb-deploy/tikv-20160
Total nodes: 12

—这次打算把172.244.0.83扩容为dm.

–查看 TiUP 支持的最新dm版本
[ec2-user@tidb-adm ~]$ tiup list dm-master
Available versions for dm-master:
Version Installed Release Platforms

nightly → v9.0.0-beta.2.pre-nightly 2025-12-12T08:10:30Z linux/amd64,linux/arm64,darwin/amd64,darwin/arm64
v2.0.0-rc 2020-08-21T17:49:08+08:00 linux/amd64,linux/arm64
v2.0.0-rc.2 2020-09-01T20:51:29+08:00 linux/amd64,linux/arm64
v2.0.0 2020-10-30T16:10:58+08:00 linux/amd64,linux/arm64
v2.0.1 2020-12-25T13:22:29+08:00 linux/amd64,linux/arm64
v2.0.3 2021-05-11T22:14:31+08:00 linux/amd64,linux/arm64
v2.0.4 2021-06-18T16:34:30+08:00 linux/amd64,linux/arm64
…
v8.5.1 2025-01-17T07:56:44Z linux/amd64,linux/arm64,darwin/amd64,darwin/arm64
v8.5.2 2025-06-04T04:47:56Z linux/amd64,linux/arm64,darwin/amd64,darwin/arm64
v8.5.3 2025-08-22T08:05:38Z linux/amd64,linux/arm64,darwin/amd64,darwin/arm64
v8.5.4 2025-11-27T08:15:57Z linux/amd64,linux/arm64,darwin/amd64,darwin/arm64
v9.0.0-beta.1 2025-03-31T05:21:57Z linux/amd64,linux/arm64,darwin/amd64,darwin/arm64
v9.0.0-beta.2.pre-nightly 2025-12-12T08:10:30Z linux/amd64,linux/arm64,darwin/amd64,darwin/arm64
[ec2-user@tidb-adm ~]$

[ec2-user@tidb-adm ~]$ tiup dm deploy dm-test v8.5.4 ./dm-topology.yaml --user ec2-user -i wazuh.pem
Starting component dm: /home/ec2-user/.tiup/components/dm/v1.16.4/tiup-dm deploy dm-test v8.5.4 ./dm-topology.yaml --user ec2-user -i wazuh.pem

  • Detect CPU Arch Name

    • Detecting node 172.244.0.83 Arch info … Done

  • Detect CPU OS Name

    • Detecting node 172.244.0.83 OS info … Done
      Please confirm your topology:
      Cluster type: dm
      Cluster name: dm-test
      Cluster version: v8.5.4
      Role Host Ports OS/Arch Directories

dm-master 172.244.0.83 8261/8291 linux/x86_64 /tidb-data/dm/deploy/dm-master-8261,/tidb-data/dm/data/dm-master-8261
dm-worker 172.244.0.83 8262 linux/x86_64 /tidb-data/dm/deploy/dm-worker-8262,/tidb-data/dm/data/dm-worker-8262
Attention:
1. If the topology is not what you expected, check your yaml file.
2. Please confirm there is no port/directory conflicts in same host.
Do you want to continue? [y/N]: (default=N) y

  • Generate SSH keys … Done
  • Download TiDB components
  • Download TiDB components
    • Download dm-master:v8.5.4 (linux/amd64) … Done
    • Download dm-worker:v8.5.4 (linux/amd64) … Done
  • Initialize target host environments
    • Prepare 172.244.0.83:22 … Done
  • Deploy TiDB instance
    • Copy dm-master → 172.244.0.83 … Done
    • Copy dm-worker → 172.244.0.83 … Done
  • Copy certificate to remote host
  • Init instance configs
    • Generate config dm-master → 172.244.0.83:8261 … Done
    • Generate config dm-worker → 172.244.0.83:8262 … Done
  • Init monitor configs
    Enabling component dm-master
    Enabling instance 172.244.0.83:8261
    download https://tiup-mirrors.pingcap.com/dm-worker-v8.5.4-linux-amd64.tar.gz 752.00 KiB / 80.88 MiB 0.91% ? MiB/s Enable instance 172.244.0.83:8261 success
    Enabling component dm-worker
    Enabling instance 172.244.0.83:8262
    download https://tiup-mirrors.pingcap.com/dm-worker-v8.5.4-linux-amd64.tar.gz 752.00 KiB / 80.88 MiB 0.91% ? MiB/s Enable instance 172.244.0.83:8262 success
    Cluster dm-test deployed successfully, you can start it with command: tiup dm start dm-test
    [ec2-user@tidb-adm ~]$

[ec2-user@tidb-adm ~]$ cat dm-topology.yaml

The topology template is used deploy a minimal DM cluster, which suitable

for scenarios with only three machinescontains. The minimal cluster contains

- 3 master nodes

- 3 worker nodes

You can change the hosts according your environment

global:
user: “tidb”

systemd_mode: “system”

ssh_port: 22
deploy_dir: “/tidb-data/dm/deploy”
data_dir: “/tidb-data/dm/data”

arch: “amd64”

master_servers:

  • host: 172.244.0.83

worker_servers:

  • host: 172.244.0.83

第 5 步：检查部署的 DM 集群情况

[ec2-user@tidb-adm ~]$ tiup dm list
Starting component dm: /home/ec2-user/.tiup/components/dm/v1.16.4/tiup-dm list
Name User Version Path PrivateKey

dm-test tidb v8.5.4 /home/ec2-user/.tiup/storage/dm/clusters/dm-test /home/ec2-user/.tiup/storage/dm/clusters/dm-test/ssh/id_rsa

[ec2-user@tidb-adm ~]$ tiup dm display dm-test
Starting component dm: /home/ec2-user/.tiup/components/dm/v1.16.4/tiup-dm display dm-test
Cluster type: dm
Cluster name: dm-test
Cluster version: v8.5.4
Deploy user: tidb
SSH type: builtin
ID Role Host Ports OS/Arch Status Data Dir Deploy Dir

172.244.0.83:8261 dm-master 172.244.0.83 8261/8291 linux/x86_64 Down /tidb-data/dm/data/dm-master-8261 /tidb-data/dm/deploy/dm-master-8261
172.244.0.83:8262 dm-worker 172.244.0.83 8262 linux/x86_64 N/A /tidb-data/dm/data/dm-worker-8262 /tidb-data/dm/deploy/dm-worker-8262
Total nodes: 2
[ec2-user@tidb-adm ~]$

第 6 步：启动集群
[ec2-user@tidb-adm ~]$ tiup dm start dm-test
Starting component dm: /home/ec2-user/.tiup/components/dm/v1.16.4/tiup-dm start dm-test
Starting cluster dm-test…

  • [ Serial ] - SSHKeySet: privateKey=/home/ec2-user/.tiup/storage/dm/clusters/dm-test/ssh/id_rsa, publicKey=/home/ec2-user/.tiup/storage/dm/clusters/dm-test/ssh/id_rsa.pub
  • [Parallel] - UserSSH: user=tidb, host=172.244.0.83
  • [Parallel] - UserSSH: user=tidb, host=172.244.0.83
  • [ Serial ] - StartCluster
    Starting component dm-master
    Starting instance 172.244.0.83:8261
    Start instance 172.244.0.83:8261 success
    Starting component dm-worker
    Starting instance 172.244.0.83:8262
    Start instance 172.244.0.83:8262 success
    Started cluster dm-test successfully
    [ec2-user@tidb-adm ~]$

[ec2-user@tidb-adm ~]$ tiup dm display dm-test
Starting component dm: /home/ec2-user/.tiup/components/dm/v1.16.4/tiup-dm display dm-test
Cluster type: dm
Cluster name: dm-test
Cluster version: v8.5.4
Deploy user: tidb
SSH type: builtin
ID Role Host Ports OS/Arch Status Data Dir Deploy Dir

172.244.0.83:8261 dm-master 172.244.0.83 8261/8291 linux/x86_64 Healthy|L /tidb-data/dm/data/dm-master-8261 /tidb-data/dm/deploy/dm-master-8261
172.244.0.83:8262 dm-worker 172.244.0.83 8262 linux/x86_64 Free /tidb-data/dm/data/dm-worker-8262 /tidb-data/dm/deploy/dm-worker-8262
Total nodes: 2
[ec2-user@tidb-adm ~]$

2 个赞
2

你的问题是什么？？？需要帮你解决什么问题？

3

这个是个笔记吧

4

嗯,记录得很详细. 这是什么的日志?

5

数据量30G？能讲下tikv节点和数据量大小的比例多少是最优配置？

6

当前集群状态总结

组件 节点 IP 端口 状态 说明
dm-master 172.244.0.83 8261/8291 Healthy L 主节点正常运行（Leader）
dm-worker 172.244.0.83 8262 Free 工作节点空闲（未绑定数据源
1 个赞
7
  1. dm-master 状态 Healthy|L：
  • Healthy 表示组件进程正常、端口监听正常、集群心跳正常；
  • L 表示该节点是 dm-master 集群的 Leader 节点（单节点部署时默认是 Leader）。
  1. dm-worker 状态 Free：
  • 表示 worker 节点已启动但尚未绑定任何上游数据源（MySQL/MariaDB 等），这是初始正常状态。
8

日志信息好多啊

9

是的,真实的记录

10

部署记录超详细！DM 集群已成功启动，dm-master 状态健康、dm-worker 为 Free。接下来要配置数据源和同步任务了吧。

1 个赞
12

对的,配置了上游数据库,目前在dm同步中;

1 个赞
13

[ec2-user@tidb-adm ~]$ cat dm-topology.yaml

The topology template is used deploy a minimal DM cluster, which suitable

for scenarios with only three machinescontains. The minimal cluster contains

- 3 master nodes

- 3 worker nodes

You can change the hosts according your environment

global:
user: “tidb”

systemd_mode: “system”

ssh_port: 22
deploy_dir: “/tidb-data/dm1/deploy”
data_dir: “/tidb-data/dm1/data”

arch: “amd64”

master_servers:

  • host: 172.244.0.83

worker_servers:

  • host: 172.244.0.83

monitoring_servers:

  • host: 172.244.0.83

grafana_servers:

  • host: 172.244.0.83

alertmanager_servers:

  • host: 172.244.0.83

[ec2-user@tidb-adm ~]$
我重新更新了配置文件.将页面展示出来,想看看同步时的情况;是可以放到同一台主机上.

1 个赞
14

[ec2-user@tidb-adm ~]$ tiup dm display dm-tt
Starting component dm: /home/ec2-user/.tiup/components/dm/v1.16.4/tiup-dm display dm-tt
Cluster type: dm
Cluster name: dm-tt
Cluster version: v8.5.4
Deploy user: tidb
SSH type: builtin
Grafana URL: http://172.244.0.83:3000
ID Role Host Ports OS/Arch Status Data Dir Deploy Dir

172.244.0.83:9093 alertmanager 172.244.0.83 9093/9094 linux/x86_64 Up /tidb-data/dm1/data/alertmanager-9093 /tidb-data/dm1/deploy/alertmanager-9093
172.244.0.83:8261 dm-master 172.244.0.83 8261/8291 linux/x86_64 Healthy|L /tidb-data/dm1/data/dm-master-8261 /tidb-data/dm1/deploy/dm-master-8261
172.244.0.83:8262 dm-worker 172.244.0.83 8262 linux/x86_64 Bound /tidb-data/dm1/data/dm-worker-8262 /tidb-data/dm1/deploy/dm-worker-8262
172.244.0.83:3000 grafana 172.244.0.83 3000 linux/x86_64 Up - /tidb-data/dm1/deploy/grafana-3000
172.244.0.83:9090 prometheus 172.244.0.83 9090 linux/x86_64 Up /tidb-data/dm1/data/prometheus-9090 /tidb-data/dm1/deploy/prometheus-9090
Total nodes: 5

1 个赞
15

社区因为有这样的高质量分享而更有价值！

1 个赞
16

从你的操作记录能清晰看到，DM 集群从部署、启动到后续配置监控组件、绑定数据源的全流程都顺利，每一步操作都记录得详细完整，给有类似扩容需求的用户提供了很实用的参考！ 点赞！！！

17

没看明白啥意思,特别是DM都没有明白啥意思