DM-work报错[ERROR] [subtask.go:419] ["wait condition"] [subtask=test5] [error="[code=40067:class=dm-worker:scope=internal:level=high] wait relay catchup timeout, loader end binlog pos: (mysql-bin.000005, 18712), relay binlog pos: (, 4)"]

sanger · 2019 年11 月 13 日 04:54

为提高效率，提问时请尽量提供详细背景信息，问题描述清晰可优先响应。以下信息点请尽量提供：

【系统版本 & kernel 版本】centos7
【TiDB 版本】v3.0.1
【磁盘型号】sata
【集群节点分布】3pd 3tikv 2tidb k8s部署
【数据量 & region 数量 & 副本数】测试dm用的，数据量不大
【问题描述（我做了什么）】 dm安装是通过ansible的方式创建一个同步任务通过web-ui，任务报错

name: test5
task-mode: all
is-sharding: false
target-database:
  host: tidb.xxx.xx
  port: 3306
  user: root
  password: xxx
mysql-instances:
- source-id: mysql-replica-01
  meta:
    binlog-name: mysql-bin.0000001
    binlog-pos: 4
  filter-rules: []
  route-rules:
  - mysql-replica-01.route_rules.1
  - mysql-replica-01.route_rules.2
  black-white-list: mysql-replica-01.bw_list.1
  mydumper-config-name: mysql-replica-01.dump
routes:
  mysql-replica-01.route_rules.1:
    schema-pattern: dm_test
    table-pattern: ""
    target-schema: dm_test
    target-table: ""
  mysql-replica-01.route_rules.2:
    schema-pattern: dm_test
    table-pattern: t1
    target-schema: dm_test
    target-table: t1
filters: {}
black-white-list:
  mysql-replica-01.bw_list.1:
    do-tables:
    - db-name: dm_test
      tbl-name: t1
    do-dbs: []
    ignore-tables: []
    ignore-dbs: []
mydumpers:
  mysql-replica-01.dump:
    mydumper-path: bin/mydumper
    threads: 4
    chunk-filesize: 64
    skip-tz-utc: true
    extra-args: -T dm_test.t1

【关键词】

» query-status test5
{
    "result": true,
    "msg": "",
    "workers": [
        {
            "result": true,
            "worker": "10.125.0.34:8262",
            "msg": "",
            "subTaskStatus": [
                {
                    "name": "test5",
                    "stage": "Paused",
                    "unit": "Load",
                    "result": {
                        "isCanceled": false,
                        "errors": [
                            {
                                "Type": "ExecSQL",
                                "msg": "[code=10006:class=database:scope=downstream:level=high] file dm_test.t1.sql: execute statement failed: commit: Error 1062: Duplicate entry '1' for key 'PRIMARY'
github.com/pingcap/dm/pkg/terror.(*Error).Delegate
	/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/terror/terror.go:267
github.com/pingcap/dm/pkg/conn.(*BaseConn).ExecuteSQLWithIgnoreError
	/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/conn/baseconn.go:164
github.com/pingcap/dm/pkg/conn.(*BaseConn).ExecuteSQL
	/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/conn/baseconn.go:174
github.com/pingcap/dm/loader.(*DBConn).executeSQL.func2
	/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/loader/db.go:149
github.com/pingcap/dm/pkg/retry.(*FiniteRetryStrategy).Apply
	/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/retry/strategy.go:71
github.com/pingcap/dm/pkg/conn.(*BaseConn).ApplyRetryStrategy
	/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/conn/baseconn.go:180
github.com/pingcap/dm/loader.(*DBConn).executeSQL
	/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/loader/db.go:144
github.com/pingcap/dm/loader.(*Worker).run.func1
	/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/loader/loader.go:152
runtime.goexit
	/usr/local/go/src/runtime/asm_amd64.s:1337",
                                "error": null
                            }
                        ],
                        "detail": null
                    },
                    "unresolvedDDLLockID": "",
                    "load": {
                        "finishedBytes": "0",
                        "totalBytes": "110",
                        "progress": "0.00 %",
                        "metaBinlog": "(mysql-bin.000005, 64313)"
                    }
                }
            ],
            "relayStatus": {
                "masterBinlog": "(mysql-bin.000005, 65063)",
                "masterBinlogGtid": "561d10a3-7614-11e9-8f94-246e96757168:1-1223137",
                "relaySubDir": "561d10a3-7614-11e9-8f94-246e96757168.000001",
                "relayBinlog": "(, 4)",
                "relayBinlogGtid": "",
                "relayCatchUpMaster": false,
                "stage": "Paused",
                "result": {
                    "isCanceled": false,
                    "errors": [
                        {
                            "Type": "UnknownError",
                            "msg": "[code=30015:class=relay-unit:scope=upstream:level=high] TCPReader get event: ERROR 1236 (HY000): The slave is connecting using CHANGE MASTER TO MASTER_AUTO_POSITION = 1, but the master has purged binary logs containing GTIDs that the slave requires.
github.com/pingcap/dm/pkg/terror.(*Error).Delegate
	/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/terror/terror.go:267
github.com/pingcap/dm/pkg/binlog/reader.(*TCPReader).GetEvent
	/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/binlog/reader/tcp.go:151
github.com/pingcap/dm/relay/reader.(*reader).GetEvent
	/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/reader/reader.go:144
github.com/pingcap/dm/relay.(*Relay).handleEvents
	/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/relay.go:378
github.com/pingcap/dm/relay.(*Relay).process
	/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/relay.go:292
github.com/pingcap/dm/relay.(*Relay).Process
	/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/relay.go:191
github.com/pingcap/dm/dm/worker.(*realRelayHolder).run
	/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/dm/worker/relay.go:164
github.com/pingcap/dm/dm/worker.(*realRelayHolder).Start.func1
	/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/dm/worker/relay.go:140
runtime.goexit
	/usr/local/go/src/runtime/asm_amd64.s:1337",
                            "error": null
                        }
                    ],
                    "detail": null
                }
            },
            "sourceID": "mysql-replica-01"
        }
    ]
}

dm-work报的错

[2019/11/13 11:48:46.103 +08:00] [WARN] [task_checker.go:348] ["task can't auto resume"] [component="task checker"] [task=test5] ["paused duration"=14m55.000061086s]
[2019/11/13 11:48:49.204 +08:00] [ERROR] [subtask.go:419] ["wait condition"] [subtask=test4] [error="[code=40067:class=dm-worker:scope=internal:level=high] wait relay catchup timeout, loader end binlog pos: (mysql-bin.000005, 18712), relay binlog pos: (, 4)"]
[2019/11/13 11:48:49.204 +08:00] [INFO] [worker.go:883] ["end to execute operation"] [component="worker controller"] ["oplog ID"=149] [error="[code=40067:class=dm-worker:scope=internal:level=high] wait relay catchup timeout, loader end binlog pos: (mysql-bin.000005, 18712), relay binlog pos: (, 4)"]

yilong · 2019 年11 月 13 日 05:29

您好： 1. dm使用的哪个版本？ 2. mysql是否有主从配置，是使用主库还是从库同步？ mysql是否有配置gtid？ 3. 将inventory.ini 的配置也发送下，多谢.

ericsyh · 2019 年11 月 13 日 05:29

你好，请问 worker 上 relay log 配置开启了么？看报错是全量导完后，relay log 的 pos 没有追上。

sanger · 2019 年11 月 13 日 06:07

你好，怎么看开启了没哦

# Worker Configuration.

server-id = 101
source-id = "mysql-replica-01"
flavor = "mysql"
enable-gtid = true
relay-binlog-name = ""
relay-binlog-gtid = "561d10a3-7614-11e9-8f94-246e96757168:1-1221235"
#charset of DSN of source mysql/mariadb instance
charset = ""
meta-dir = ""

[from]
host = "10.125.0.49"
user = "test"
password = "ndzYxh+L/P6hXPTV5ZT+u0B9xziKiNAmTw=="
port = 3306

#relay log purge strategy
[purge]
interval = 3600
expires = 0
remain-space = 15

sanger · 2019 年11 月 13 日 06:31

dm版本：v1.1.0-alpha-29-gb95de78 mysql在有主从，使用的主库同步的，是 gtid

## DM modules
[dm_master_servers]
dm_master ansible_host=10.125.0.21

[dm_worker_servers]

dm-worker1 ansible_host=10.125.0.34 source_id="mysql-replica-01" server_id=101 enable_gtid=true relay_binlog_gtid="561d10a3-7614-11e9-8f94-246e96757168:1-1221235" mysql_host=10.125.0.49 mysql_user=test mysql_password='ndzYxh+L/P6hXPTV5ZT+u0B9xziKiNAmTw==' mysql_port=3306

dm-worker2 ansible_host=10.125.0.45 source_id="mysql-replica-02" server_id=102 enable_gtid=true relay_binlog_gtid="561d10a3-7614-11e9-8f94-246e96757168:1-1221235" mysql_host=10.125.0.49 mysql_user=test mysql_password='ndzYxh+L/P6hXPTV5ZT+u0B9xziKiNAmTw==' mysql_port=3306

[dm_portal_servers]
dm_portal ansible_host=10.125.0.21

## Monitoring modules
[prometheus_servers]
prometheus ansible_host=10.125.0.21

[grafana_servers]
grafana ansible_host=10.125.0.21

[alertmanager_servers]
alertmanager ansible_host=10.125.0.21

## Global variables
[all:vars]
cluster_name = test-cluster

ansible_user = tidb

dm_version = latest

deploy_dir = /home/tidb/deploy

grafana_admin_user = "admin"
grafana_admin_password = "admin"

不好意思，回复慢了

yilong · 2019 年11 月 13 日 06:44

你好：这里感觉配置的没问题，请将dm-master,dm-worker, *.task 文件都反馈下，dm的报错日志完整上传下，多谢

sanger · 2019 年11 月 13 日 07:17

现在又做了一次task，同步dm-test库到tidb
因为看dm-master中有描述
“metadata will not be used. for Full mode, incremental sync will never occur; for All mode, the meta dumped by MyDumper will be used”
所以用了增量的方式，貌似还是不行

这个是上游库
下面是下游库

test5-task.yaml

name: test5
task-mode: incremental
is-sharding: false
target-database:
  host: tidb.xxx
  port: 3306
  user: root
  password: h46xBDWEdjCj+uy5kJxeMUAB4z4KJDaJG2U=
mysql-instances:
- source-id: mysql-replica-01
  meta:
    binlog-name: mysql-bin.000005
    binlog-pos: 72150
  filter-rules: []
  route-rules:
  - mysql-replica-1.route_rules.1
  - mysql-replica-1.route_rules.2
  black-white-list: mysql-replica-1.bw_list.1
  mydumper-config-name: mysql-replica-1.dump
routes:
  mysql-replica-1.route_rules.1:
    schema-pattern: dm_test
    table-pattern: ""
    target-schema: dm_test
    target-table: ""
  mysql-replica-1.route_rules.2:
    schema-pattern: dm_test
    table-pattern: t1
    target-schema: dm_test
    target-table: t1
filters: {}
black-white-list:
  mysql-replica-1.bw_list.1:
    do-tables:
    - db-name: dm_test
      tbl-name: t1
    do-dbs: []
    ignore-tables:
    - db-name: drone
      tbl-name: builds
    - db-name: drone
      tbl-name: cron
    - db-name: drone
      tbl-name: logs
    - db-name: drone
      tbl-name: migrations
    - db-name: drone
      tbl-name: nodes
    - db-name: drone
      tbl-name: perms
    - db-name: drone
      tbl-name: repos
    - db-name: drone
      tbl-name: secrets
    - db-name: drone
      tbl-name: stages
    - db-name: drone
      tbl-name: stages_unfinished
    - db-name: drone
      tbl-name: steps
    - db-name: drone
      tbl-name: users
    - db-name: grafana
      tbl-name: alert
    - db-name: grafana
      tbl-name: alert_notification
    - db-name: grafana
      tbl-name: alert_notification_state
    - db-name: grafana
      tbl-name: annotation
    - db-name: grafana
      tbl-name: annotation_tag
    - db-name: grafana
      tbl-name: api_key
    - db-name: grafana
      tbl-name: cache_data
    - db-name: grafana
      tbl-name: dashboard
    - db-name: grafana
      tbl-name: dashboard_acl
    - db-name: grafana
      tbl-name: dashboard_provisioning
    - db-name: grafana
      tbl-name: dashboard_snapshot
    - db-name: grafana
      tbl-name: dashboard_tag
    - db-name: grafana
      tbl-name: dashboard_version
    - db-name: grafana
      tbl-name: data_source
    - db-name: grafana
      tbl-name: login_attempt
    - db-name: grafana
      tbl-name: migration_log
    - db-name: grafana
      tbl-name: org
    - db-name: grafana
      tbl-name: org_user
    - db-name: grafana
      tbl-name: playlist
    - db-name: grafana
      tbl-name: playlist_item
    - db-name: grafana
      tbl-name: plugin_setting
    - db-name: grafana
      tbl-name: preferences
    - db-name: grafana
      tbl-name: quota
    - db-name: grafana
      tbl-name: server_lock
    - db-name: grafana
      tbl-name: session
    - db-name: grafana
      tbl-name: star
    - db-name: grafana
      tbl-name: tag
    - db-name: grafana
      tbl-name: team
    - db-name: grafana
      tbl-name: team_member
    - db-name: grafana
      tbl-name: temp_user
    - db-name: grafana
      tbl-name: test_data
    - db-name: grafana
      tbl-name: user
    - db-name: grafana
      tbl-name: user_auth
    - db-name: grafana
      tbl-name: user_auth_token
    ignore-dbs: []
mydumpers:
  mysql-replica-1.dump:
    mydumper-path: bin/mydumper
    threads: 4
    chunk-filesize: 64
    skip-tz-utc: true
    extra-args: -T dm_test.t1

通过dmctl执行任务后，查询报错

» start-task /tmp/test5-task.yaml
{
    "result": true,
    "msg": "",
    "workers": [
        {
            "result": true,
            "worker": "10.125.0.34:8262",
            "msg": ""
        }
    ]
}
» query-status test5
{
    "result": true,
    "msg": "",
    "workers": [
        {
            "result": true,
            "worker": "10.125.0.34:8262",
            "msg": "",
            "subTaskStatus": [
                {
                    "name": "test5",
                    "stage": "Paused",
                    "unit": "Sync",
                    "result": {
                        "isCanceled": false,
                        "errors": [
                            {
                                "Type": "UnknownError",
                                "msg": "[code=11025:class=functional:scope=internal:level=high] parse relay dir /home/tidb/deploy/relay_log/561d10a3-7614-11e9-8f94-246e96757168.000001 with pos (mysql-bin.000005, 72150): base file mysql-bin.000005 in directory /home/tidb/deploy/relay_log/561d10a3-7614-11e9-8f94-246e96757168.000001 not found
github.com/pingcap/dm/pkg/terror.(*Error).Generate
	/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/terror/terror.go:232
github.com/pingcap/dm/pkg/streamer.CollectBinlogFilesCmp
	/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/streamer/file.go:78
github.com/pingcap/dm/pkg/streamer.(*BinlogReader).parseDirAsPossible
	/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/streamer/reader.go:181
github.com/pingcap/dm/pkg/streamer.(*BinlogReader).parseRelay
	/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/streamer/reader.go:138
github.com/pingcap/dm/pkg/streamer.(*BinlogReader).StartSync.func1
	/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/streamer/reader.go:111
runtime.goexit
	/usr/local/go/src/runtime/asm_amd64.s:1337",
                                "error": null
                            }
                        ],
                        "detail": null
                    },
                    "unresolvedDDLLockID": "",
                    "sync": {
                        "totalEvents": "0",
                        "totalTps": "0",
                        "recentTps": "0",
                        "masterBinlog": "(mysql-bin.000005, 72523)",
                        "masterBinlogGtid": "561d10a3-7614-11e9-8f94-246e96757168:1-1223157",
                        "syncerBinlog": "(mysql-bin.000005, 72150)",
                        "syncerBinlogGtid": "",
                        "blockingDDLs": [
                        ],
                        "unresolvedGroups": [
                        ],
                        "synced": false
                    }
                }
            ],
            "relayStatus": {
                "masterBinlog": "(mysql-bin.000005, 72523)",
                "masterBinlogGtid": "561d10a3-7614-11e9-8f94-246e96757168:1-1223157",
                "relaySubDir": "561d10a3-7614-11e9-8f94-246e96757168.000001",
                "relayBinlog": "(, 4)",
                "relayBinlogGtid": "",
                "relayCatchUpMaster": false,
                "stage": "Paused",
                "result": {
                    "isCanceled": false,
                    "errors": [
                        {
                            "Type": "UnknownError",
                            "msg": "[code=30015:class=relay-unit:scope=upstream:level=high] TCPReader get event: ERROR 1236 (HY000): The slave is connecting using CHANGE MASTER TO MASTER_AUTO_POSITION = 1, but the master has purged binary logs containing GTIDs that the slave requires.
github.com/pingcap/dm/pkg/terror.(*Error).Delegate
	/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/terror/terror.go:267
github.com/pingcap/dm/pkg/binlog/reader.(*TCPReader).GetEvent
	/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/binlog/reader/tcp.go:151
github.com/pingcap/dm/relay/reader.(*reader).GetEvent
	/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/reader/reader.go:144
github.com/pingcap/dm/relay.(*Relay).handleEvents
	/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/relay.go:378
github.com/pingcap/dm/relay.(*Relay).process
	/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/relay.go:292
github.com/pingcap/dm/relay.(*Relay).Process
	/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/relay.go:191
github.com/pingcap/dm/dm/worker.(*realRelayHolder).run
	/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/dm/worker/relay.go:164
github.com/pingcap/dm/dm/worker.(*realRelayHolder).Start.func1
	/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/dm/worker/relay.go:140
runtime.goexit
	/usr/local/go/src/runtime/asm_amd64.s:1337",
                            "error": null
                        }
                    ],
                    "detail": null
                }
            },
            "sourceID": "mysql-replica-01"
        }
    ]
}

dm-master日志

[2019/11/13 14:56:38.017 +08:00] [INFO] [server.go:228] [payload="task:"name: test5\
task-mode: incremental\
is-sharding: false\
target-database:\
  host: tidb.xxx\
  port: 3306\
  user: root\
  password: h46xBDWEdjCj+uy5kJxeMUAB4z4KJDaJG2U=\
mysql-instances:\
- source-id: mysql-replica-1\
  meta:\
    binlog-name: mysql-bin.000005\
    binlog-pos: 72150\
  filter-rules: []\
  route-rules:\
  - mysql-replica-1.route_rules.1\
  - mysql-replica-1.route_rules.2\
  black-white-list: mysql-replica-1.bw_list.1\
  mydumper-config-name: mysql-replica-1.dump\
routes:\
  mysql-replica-1.route_rules.1:\
    schema-pattern: dm_test\
    table-pattern: \"\"\
    target-schema: dm_test\
    target-table: \"\"\
  mysql-replica-1.route_rules.2:\
    schema-pattern: dm_test\
    table-pattern: t1\
    target-schema: dm_test\
    target-table: t1\
filters: {}\
black-white-list:\
  mysql-replica-1.bw_list.1:\
    do-tables:\
    - db-name: dm_test\
      tbl-name: t1\
    do-dbs: []\
    ignore-tables:\
    - db-name: drone\
      tbl-name: builds\
    - db-name: drone\
      tbl-name: cron\
    - db-name: drone\
      tbl-name: logs\
    - db-name: drone\
      tbl-name: migrations\
    - db-name: drone\
      tbl-name: nodes\
    - db-name: drone\
      tbl-name: perms\
    - db-name: drone\
      tbl-name: repos\
    - db-name: drone\
      tbl-name: secrets\
    - db-name: drone\
      tbl-name: stages\
    - db-name: drone\
      tbl-name: stages_unfinished\
    - db-name: drone\
      tbl-name: steps\
    - db-name: drone\
      tbl-name: users\
    - db-name: grafana\
      tbl-name: alert\
    - db-name: grafana\
      tbl-name: alert_notification\
    - db-name: grafana\
      tbl-name: alert_notification_state\
    - db-name: grafana\
      tbl-name: annotation\
    - db-name: grafana\
      tbl-name: annotation_tag\
    - db-name: grafana\
      tbl-name: api_key\
    - db-name: grafana\
      tbl-name: cache_data\
    - db-name: grafana\
      tbl-name: dashboard\
    - db-name: grafana\
      tbl-name: dashboard_acl\
    - db-name: grafana\
      tbl-name: dashboard_provisioning\
    - db-name: grafana\
      tbl-name: dashboard_snapshot\
    - db-name: grafana\
      tbl-name: dashboard_tag\
    - db-name: grafana\
      tbl-name: dashboard_version\
    - db-name: grafana\
      tbl-name: data_source\
    - db-name: grafana\
      tbl-name: login_attempt\
    - db-name: grafana\
      tbl-name: migration_log\
    - db-name: grafana\
      tbl-name: org\
    - db-name: grafana\
      tbl-name: org_user\
    - db-name: grafana\
      tbl-name: playlist\
    - db-name: grafana\
      tbl-name: playlist_item\
    - db-name: grafana\
      tbl-name: plugin_setting\
    - db-name: grafana\
      tbl-name: preferences\
    - db-name: grafana\
      tbl-name: quota\
    - db-name: grafana\
      tbl-name: server_lock\
    - db-name: grafana\
      tbl-name: session\
    - db-name: grafana\
      tbl-name: star\
    - db-name: grafana\
      tbl-name: tag\
    - db-name: grafana\
      tbl-name: team\
    - db-name: grafana\
      tbl-name: team_member\
    - db-name: grafana\
      tbl-name: temp_user\
    - db-name: grafana\
      tbl-name: test_data\
    - db-name: grafana\
      tbl-name: user\
    - db-name: grafana\
      tbl-name: user_auth\
    - db-name: grafana\
      tbl-name: user_auth_token\
    ignore-dbs: []\
mydumpers:\
  mysql-replica-1.dump:\
    mydumper-path: bin/mydumper\
    threads: 4\
    chunk-filesize: 64\
    skip-tz-utc: true\
    extra-args: -T dm_test.t1\
" "] [request=StartTask]
[2019/11/13 14:57:27.718 +08:00] [INFO] [server.go:228] [payload="task:"name: test5\
task-mode: incremental\
is-sharding: false\
target-database:\
  host: tidb.xxx\
  port: 3306\
  user: root\
  password: h46xBDWEdjCj+uy5kJxeMUAB4z4KJDaJG2U=\
mysql-instances:\
- source-id: mysql-replica-01\
  meta:\
    binlog-name: mysql-bin.000005\
    binlog-pos: 72150\
  filter-rules: []\
  route-rules:\
  - mysql-replica-1.route_rules.1\
  - mysql-replica-1.route_rules.2\
  black-white-list: mysql-replica-1.bw_list.1\
  mydumper-config-name: mysql-replica-1.dump\
routes:\
  mysql-replica-1.route_rules.1:\
    schema-pattern: dm_test\
    table-pattern: \"\"\
    target-schema: dm_test\
    target-table: \"\"\
  mysql-replica-1.route_rules.2:\
    schema-pattern: dm_test\
    table-pattern: t1\
    target-schema: dm_test\
    target-table: t1\
filters: {}\
black-white-list:\
  mysql-replica-1.bw_list.1:\
    do-tables:\
    - db-name: dm_test\
      tbl-name: t1\
    do-dbs: []\
    ignore-tables:\
    - db-name: drone\
      tbl-name: builds\
    - db-name: drone\
      tbl-name: cron\
    - db-name: drone\
      tbl-name: logs\
    - db-name: drone\
      tbl-name: migrations\
    - db-name: drone\
      tbl-name: nodes\
    - db-name: drone\
      tbl-name: perms\
    - db-name: drone\
      tbl-name: repos\
    - db-name: drone\
      tbl-name: secrets\
    - db-name: drone\
      tbl-name: stages\
    - db-name: drone\
      tbl-name: stages_unfinished\
    - db-name: drone\
      tbl-name: steps\
    - db-name: drone\
      tbl-name: users\
    - db-name: grafana\
      tbl-name: alert\
    - db-name: grafana\
      tbl-name: alert_notification\
    - db-name: grafana\
      tbl-name: alert_notification_state\
    - db-name: grafana\
      tbl-name: annotation\
    - db-name: grafana\
      tbl-name: annotation_tag\
    - db-name: grafana\
      tbl-name: api_key\
    - db-name: grafana\
      tbl-name: cache_data\
    - db-name: grafana\
      tbl-name: dashboard\
    - db-name: grafana\
      tbl-name: dashboard_acl\
    - db-name: grafana\
      tbl-name: dashboard_provisioning\
    - db-name: grafana\
      tbl-name: dashboard_snapshot\
    - db-name: grafana\
      tbl-name: dashboard_tag\
    - db-name: grafana\
      tbl-name: dashboard_version\
    - db-name: grafana\
      tbl-name: data_source\
    - db-name: grafana\
      tbl-name: login_attempt\
    - db-name: grafana\
      tbl-name: migration_log\
    - db-name: grafana\
      tbl-name: org\
    - db-name: grafana\
      tbl-name: org_user\
    - db-name: grafana\
      tbl-name: playlist\
    - db-name: grafana\
      tbl-name: playlist_item\
    - db-name: grafana\
      tbl-name: plugin_setting\
    - db-name: grafana\
      tbl-name: preferences\
    - db-name: grafana\
      tbl-name: quota\
    - db-name: grafana\
      tbl-name: server_lock\
    - db-name: grafana\
      tbl-name: session\
    - db-name: grafana\
      tbl-name: star\
    - db-name: grafana\
      tbl-name: tag\
    - db-name: grafana\
      tbl-name: team\
    - db-name: grafana\
      tbl-name: team_member\
    - db-name: grafana\
      tbl-name: temp_user\
    - db-name: grafana\
      tbl-name: test_data\
    - db-name: grafana\
      tbl-name: user\
    - db-name: grafana\
      tbl-name: user_auth\
    - db-name: grafana\
      tbl-name: user_auth_token\
    ignore-dbs: []\
mydumpers:\
  mysql-replica-1.dump:\
    mydumper-path: bin/mydumper\
    threads: 4\
    chunk-filesize: 64\
    skip-tz-utc: true\
    extra-args: -T dm_test.t1\
" "] [request=StartTask]
[2019/11/13 14:57:27.727 +08:00] [INFO] [common.go:109] ["no tables need to sync"] [schema=drone]
[2019/11/13 14:57:27.728 +08:00] [INFO] [common.go:109] ["no tables need to sync"] [schema=grafana]
[2019/11/13 14:57:27.728 +08:00] [INFO] [checker.go:225] ["
************ task test5 checking items ************
mysql_version
mysql_binlog_enable
mysql_binlog_format
mysql_binlog_row_image
source db dump privilege chcker
source db replication privilege chcker
table structure compatibility check
************ task test5 checking items ************"] [unit="task check"]
[2019/11/13 14:57:27.735 +08:00] [INFO] [server.go:237] ["task name"=test5] [task="name: test5
task-mode: incremental
is-sharding: false
ignore-checking-items: []
meta-schema: dm_meta
remove-meta: false
disable-heartbeat: true
enable-heartbeat: false
heartbeat-update-interval: 1
heartbeat-report-interval: 10
timezone: ""
case-sensitive: false
target-database:
  host: tidb.xxx
  port: 3306
  user: root
  password: h46xBDWEdjCj+uy5kJxeMUAB4z4KJDaJG2U=
  max-allowed-packet: null
mysql-instances:
- source-id: mysql-replica-01
  meta:
    binlog-name: mysql-bin.000005
    binlog-pos: 72150
  filter-rules: []
  column-mapping-rules: []
  route-rules:
  - mysql-replica-1.route_rules.1
  - mysql-replica-1.route_rules.2
  black-white-list: mysql-replica-1.bw_list.1
  mydumper-config-name: mysql-replica-1.dump
  mydumper:
    mydumper-path: bin/mydumper
    threads: 4
    chunk-filesize: 64
    skip-tz-utc: true
    extra-args: -T dm_test.t1
  mydumper-thread: 0
  loader-config-name: ""
  loader:
    pool-size: 16
    dir: ./dumped_data
  loader-thread: 0
  syncer-config-name: ""
  syncer:
    meta-file: ""
    worker-count: 16
    batch: 100
    max-retry: 0
    auto-fix-gtid: false
    enable-gtid: false
    disable-detect: false
    safe-mode: false
    enable-ansi-quotes: false
  syncer-thread: 0
online-ddl-scheme: ""
routes:
  mysql-replica-1.route_rules.1:
    schema-pattern: dm_test
    table-pattern: ""
    target-schema: dm_test
    target-table: ""
  mysql-replica-1.route_rules.2:
    schema-pattern: dm_test
    table-pattern: t1
    target-schema: dm_test
    target-table: t1
filters: {}
column-mappings: {}
black-white-list:
  mysql-replica-1.bw_list.1:
    do-tables:
    - db-name: dm_test
      tbl-name: t1
    do-dbs: []
    ignore-tables:
    - db-name: drone
      tbl-name: builds
    - db-name: drone
      tbl-name: cron
    - db-name: drone
      tbl-name: logs
    - db-name: drone
      tbl-name: migrations
    - db-name: drone
      tbl-name: nodes
    - db-name: drone
      tbl-name: perms
    - db-name: drone
      tbl-name: repos
    - db-name: drone
      tbl-name: secrets
    - db-name: drone
      tbl-name: stages
    - db-name: drone
      tbl-name: stages_unfinished
    - db-name: drone
      tbl-name: steps
    - db-name: drone
      tbl-name: users
    - db-name: grafana
      tbl-name: alert
    - db-name: grafana
      tbl-name: alert_notification
    - db-name: grafana
      tbl-name: alert_notification_state
    - db-name: grafana
      tbl-name: annotation
    - db-name: grafana
      tbl-name: annotation_tag
    - db-name: grafana
      tbl-name: api_key
    - db-name: grafana
      tbl-name: cache_data
    - db-name: grafana
      tbl-name: dashboard
    - db-name: grafana
      tbl-name: dashboard_acl
    - db-name: grafana
      tbl-name: dashboard_provisioning
    - db-name: grafana
      tbl-name: dashboard_snapshot
    - db-name: grafana
      tbl-name: dashboard_tag
    - db-name: grafana
      tbl-name: dashboard_version
    - db-name: grafana
      tbl-name: data_source
    - db-name: grafana
      tbl-name: login_attempt
    - db-name: grafana
      tbl-name: migration_log
    - db-name: grafana
      tbl-name: org
    - db-name: grafana
      tbl-name: org_user
    - db-name: grafana
      tbl-name: playlist
    - db-name: grafana
      tbl-name: playlist_item
    - db-name: grafana
      tbl-name: plugin_setting
    - db-name: grafana
      tbl-name: preferences
    - db-name: grafana
      tbl-name: quota
    - db-name: grafana
      tbl-name: server_lock
    - db-name: grafana
      tbl-name: session
    - db-name: grafana
      tbl-name: star
    - db-name: grafana
      tbl-name: tag
    - db-name: grafana
      tbl-name: team
    - db-name: grafana
      tbl-name: team_member
    - db-name: grafana
      tbl-name: temp_user
    - db-name: grafana
      tbl-name: test_data
    - db-name: grafana
      tbl-name: user
    - db-name: grafana
      tbl-name: user_auth
    - db-name: grafana
      tbl-name: user_auth_token
    ignore-dbs: []
mydumpers:
  mysql-replica-1.dump:
    mydumper-path: bin/mydumper
    threads: 4
    chunk-filesize: 64
    skip-tz-utc: true
    extra-args: -T dm_test.t1
loaders: {}
syncers: {}
"] [request=StartTask]
[2019/11/13 14:57:27.742 +08:00] [INFO] [server.go:1913] ["wait op log result"] [task=test5] ["operation log ID"=166] [result="meta:<result:true > log:<id:166 task:<op:Start name:"test5" task:"is-sharding = false\
online-ddl-scheme = \"\"\
case-sensitive = false\
name = \"test5\"\
mode = \"incremental\"\
source-id = \"mysql-replica-01\"\
server-id = 101\
flavor = \"mysql\"\
meta-schema = \"dm_meta\"\
remove-meta = false\
disable-heartbeat = true\
heartbeat-update-interval = 1\
heartbeat-report-interval = 10\
enable-heartbeat = false\
timezone = \"\"\
binlog-type = \"local\"\
relay-dir = \"/home/tidb/deploy/relay_log\"\
filter-rules = []\
mapping-rule = []\
mydumper-path = \"bin/mydumper\"\
threads = 4\
chunk-filesize = 64\
skip-tz-utc = true\
extra-args = \"-T dm_test.t1\"\
pool-size = 16\
dir = \"./dumped_data.test5\"\
meta-file = \"\"\
worker-count = 16\
batch = 100\
max-retry = 0\
auto-fix-gtid = false\
enable-gtid = true\
disable-detect = false\
safe-mode = false\
enable-ansi-quotes = false\
log-level = \"info\"\
log-file = \"/home/tidb/deploy/log/dm-worker.log\"\
log-rotate = \"\"\
pprof-addr = \"\"\
status-addr = \"\"\
\
[meta]\
  BinLogName = \"mysql-bin.000005\"\
  BinLogPos = 72150\
\
[from]\
  host = \"10.125.0.49\"\
  port = 3306\
  user = \"test\"\
  password = \"ndzYxh+L/P6hXPTV5ZT+u0B9xziKiNAmTw==\"\
  max-allowed-packet = 67108864\
\
[to]\
  host = \"tidb.xxx\"\
  port = 3306\
  user = \"root\"\
  password = \"h46xBDWEdjCj+uy5kJxeMUAB4z4KJDaJG2U=\"\
  max-allowed-packet = 67108864\
\
[[route-rules]]\
  schema-pattern = \"dm_test\"\
  table-pattern = \"\"\
  target-schema = \"dm_test\"\
  target-table = \"\"\
\
[[route-rules]]\
  schema-pattern = \"dm_test\"\
  table-pattern = \"t1\"\
  target-schema = \"dm_test\"\
  target-table = \"t1\"\
\
[black-white-list]\
  do-dbs = []\
  ignore-dbs = []\
\
  [[black-white-list.do-tables]]\
    db-name = \"dm_test\"\
    tbl-name = \"t1\"\
\
  [[black-white-list.ignore-tables]]\
    db-name = \"drone\"\
    tbl-name = \"builds\"\
\
  [[black-white-list.ignore-tables]]\
    db-name = \"drone\"\
    tbl-name = \"cron\"\
\
  [[black-white-list.ignore-tables]]\
    db-name = \"drone\"\
    tbl-name = \"logs\"\
\
  [[black-white-list.ignore-tables]]\
    db-name = \"drone\"\
    tbl-name = \"migrations\"\
\
  [[black-white-list.ignore-tables]]\
    db-name = \"drone\"\
    tbl-name = \"nodes\"\
\
  [[black-white-list.ignore-tables]]\
    db-name = \"drone\"\
    tbl-name = \"perms\"\
\
  [[black-white-list.ignore-tables]]\
    db-name = \"drone\"\
    tbl-name = \"repos\"\
\
  [[black-white-list.ignore-tables]]\
    db-name = \"drone\"\
    tbl-name = \"secrets\"\
\
  [[black-white-list.ignore-tables]]\
    db-name = \"drone\"\
    tbl-name = \"stages\"\
\
  [[black-white-list.ignore-tables]]\
    db-name = \"drone\"\
    tbl-name = \"stages_unfinished\"\
\
  [[black-white-list.ignore-tables]]\
    db-name = \"drone\"\
    tbl-name = \"steps\"\
\
  [[black-white-list.ignore-tables]]\
    db-name = \"drone\"\
    tbl-name = \"users\"\
\
  [[black-white-list.ignore-tables]]\
    db-name = \"grafana\"\
    tbl-name = \"alert\"\
\
  [[black-white-list.ignore-tables]]\
    db-name = \"grafana\"\
    tbl-name = \"alert_notification\"\
\
  [[black-white-list.ignore-tables]]\
    db-name = \"grafana\"\
    tbl-name = \"alert_notification_state\"\
\
  [[black-white-list.ignore-tables]]\
    db-name = \"grafana\"\
    tbl-name = \"annotation\"\
\
  [[black-white-list.ignore-tables]]\
    db-name = \"grafana\"\
    tbl-name = \"annotation_tag\"\
\
  [[black-white-list.ignore-tables]]\
    db-name = \"grafana\"\
    tbl-name = \"api_key\"\
\
  [[black-white-list.ignore-tables]]\
    db-name = \"grafana\"\
    tbl-name = \"cache_data\"\
\
  [[black-white-list.ignore-tables]]\
    db-name = \"grafana\"\
    tbl-name = \"dashboard\"\
\
  [[black-white-list.ignore-tables]]\
    db-name = \"grafana\"\
    tbl-name = \"dashboard_acl\"\
\
  [[black-white-list.ignore-tables]]\
    db-name = \"grafana\"\
    tbl-name = \"dashboard_provisioning\"\
\
  [[black-white-list.ignore-tables]]\
    db-name = \"grafana\"\
    tbl-name = \"dashboard_snapshot\"\
\
  [[black-white-list.ignore-tables]]\
    db-name = \"grafana\"\
    tbl-name = \"dashboard_tag\"\
\
  [[black-white-list.ignore-tables]]\
    db-name = \"grafana\"\
    tbl-name = \"dashboard_version\"\
\
  [[black-white-list.ignore-tables]]\
    db-name = \"grafana\"\
    tbl-name = \"data_source\"\
\
  [[black-white-list.ignore-tables]]\
    db-name = \"grafana\"\
    tbl-name = \"login_attempt\"\
\
  [[black-white-list.ignore-tables]]\
    db-name = \"grafana\"\
    tbl-name = \"migration_log\"\
\
  [[black-white-list.ignore-tables]]\
    db-name = \"grafana\"\
    tbl-name = \"org\"\
\
  [[black-white-list.ignore-tables]]\
    db-name = \"grafana\"\
    tbl-name = \"org_user\"\
\
  [[black-white-list.ignore-tables]]\
    db-name = \"grafana\"\
    tbl-name = \"playlist\"\
\
  [[black-white-list.ignore-tables]]\
    db-name = \"grafana\"\
    tbl-name = \"playlist_item\"\
\
  [[black-white-list.ignore-tables]]\
    db-name = \"grafana\"\
    tbl-name = \"plugin_setting\"\
\
  [[black-white-list.ignore-tables]]\
    db-name = \"grafana\"\
    tbl-name = \"preferences\"\
\
  [[black-white-list.ignore-tables]]\
    db-name = \"grafana\"\
    tbl-name = \"quota\"\
\
  [[black-white-list.ignore-tables]]\
    db-name = \"grafana\"\
    tbl-name = \"server_lock\"\
\
  [[black-white-list.ignore-tables]]\
    db-name = \"grafana\"\
    tbl-name = \"session\"\
\
  [[black-white-list.ignore-tables]]\
    db-name = \"grafana\"\
    tbl-name = \"star\"\
\
  [[black-white-list.ignore-tables]]\
    db-name = \"grafana\"\
    tbl-name = \"tag\"\
\
  [[black-white-list.ignore-tables]]\
    db-name = \"grafana\"\
    tbl-name = \"team\"\
\
  [[black-white-list.ignore-tables]]\
    db-name = \"grafana\"\
    tbl-name = \"team_member\"\
\
  [[black-white-list.ignore-tables]]\
    db-name = \"grafana\"\
    tbl-name = \"temp_user\"\
\
  [[black-white-list.ignore-tables]]\
    db-name = \"grafana\"\
    tbl-name = \"test_data\"\
\
  [[black-white-list.ignore-tables]]\
    db-name = \"grafana\"\
    tbl-name = \"user\"\
\
  [[black-white-list.ignore-tables]]\
    db-name = \"grafana\"\
    tbl-name = \"user_auth\"\
\
  [[black-white-list.ignore-tables]]\
    db-name = \"grafana\"\
    tbl-name = \"user_auth_token\"\
" > ts:1573628247743165775 > "]
[2019/11/13 14:57:28.743 +08:00] [INFO] [server.go:1041] ["update workers of task"] [task=test5] [workers="[10.125.0.34:8262]"]
[2019/11/13 14:57:37.993 +08:00] [INFO] [server.go:497] [payload="name:"test5" "] [request=QueryStatus]

dm-work.log
dm-work (58.6 KB)

yilong · 2019 年11 月 13 日 07:24

您好： 1. 根据报错：你配置的这个位点信息，没有找到，你改为all的方式，不要指定位点信息，尝试自动拉取吧.重启任务 binlog-name: mysql-bin.000005 binlog-pos: 72150 2. 查看下你的relay log目录下自动拉取binglog文件成功了吗？是否拉取了最新的binlog文件

sanger · 2019 年11 月 13 日 07:35

改成all，看日期binglog文件没有拉取到哦

dm-work日志

[2019/11/13 15:32:13.613 +08:00] [ERROR] [db.go:172] ["execute statements failed after retry"] [task=test5] [unit=load] [component="remote checkpoint"] [queries="[INSERT INTO `dm_meta`.`test5_loader_checkpoint` (`id`, `filename`, `cp_schema`, `cp_table`, `offset`, `end_pos`) VALUES(?,?,?,?,?,?)]"] [arguments="[[mysql-replica-01 dm_test.t1.sql dm_test t1 0 110]]"] [error="[code=10006:class=database:scope=not-set:level=high] execute statement failed: commit: Error 1062: Duplicate entry 'mysql-replica-01-dm_test.t1.sql' for key 'uk_id_f'"]
[2019/11/13 15:32:13.613 +08:00] [INFO] [checkpoint.go:294] ["checkpoint record already exists, skip it."] [task=test5] [unit=load] [component="remote checkpoint"] [id=mysql-replica-01] [filename=dm_test.t1.sql]
[2019/11/13 15:32:13.649 +08:00] [INFO] [loader.go:252] ["data are scanned finished."] [task=test5] [unit=load] ["worker ID"=0] ["data file"=dumped_data.test5/dm_test.t1.sql]
[2019/11/13 15:32:13.665 +08:00] [ERROR] [db.go:172] ["execute statements failed after retry"] [task=test5] [unit=load] ["worker ID"=0] [queries="[USE `dm_test`; INSERT INTO `t1` VALUES
(1),
(2),
(3),
(4); UPDATE `dm_meta`.`test5_loader_checkpoint` SET `offset`=110 WHERE `id` ='mysql-replica-01' AND `filename`='dm_test.t1.sql';]"] [arguments="[]"] [error="[code=10006:class=database:scope=not-set:level=high] execute statement failed: commit: Error 1062: Duplicate entry '1' for key 'PRIMARY'"]
[2019/11/13 15:32:13.665 +08:00] [INFO] [loader.go:201] ["finish to restore dump sql file"] [task=test5] [unit=load] ["worker ID"=0] ["data file"=dumped_data.test5/dm_test.t1.sql]
[2019/11/13 15:32:13.665 +08:00] [INFO] [loader.go:1119] ["all data files have been finished"] [task=test5] [unit=load] ["cost time"=76.342451ms]
[2019/11/13 15:32:13.665 +08:00] [INFO] [status.go:75] ["progress status of load"] [task=test5] [unit=load] [finished_bytes=0] [total_bytes=110] [total_file_count=3] [progress="0.00 %"]
[2019/11/13 15:32:13.665 +08:00] [INFO] [subtask.go:233] ["unit process returned"] [subtask=test5] [unit=Load] [stage=Paused] [status="{"totalBytes":110,"progress":"0.00 %","metaBinlog":"(mysql-bin.000005, 64313)"}"]
[2019/11/13 15:32:13.665 +08:00] [ERROR] [subtask.go:252] ["unit process error"] [subtask=test5] [unit=Load] ["error information"="{"Type":1,"msg":"[code=10006:class=database:scope=downstream:level=high] file dm_test.t1.sql: execute statement failed: commit: Error 1062: Duplicate entry '1' for key 'PRIMARY'\
github.com/pingcap/dm/pkg/terror.(*Error).Delegate\
\t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/terror/terror.go:267\
github.com/pingcap/dm/pkg/conn.(*BaseConn).ExecuteSQLWithIgnoreError\
\t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/conn/baseconn.go:164\
github.com/pingcap/dm/pkg/conn.(*BaseConn).ExecuteSQL\
\t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/conn/baseconn.go:174\
github.com/pingcap/dm/loader.(*DBConn).executeSQL.func2\
\t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/loader/db.go:149\
github.com/pingcap/dm/pkg/retry.(*FiniteRetryStrategy).Apply\
\t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/retry/strategy.go:71\
github.com/pingcap/dm/pkg/conn.(*BaseConn).ApplyRetryStrategy\
\t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/conn/baseconn.go:180\
github.com/pingcap/dm/loader.(*DBConn).executeSQL\
\t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/loader/db.go:144\
github.com/pingcap/dm/loader.(*Worker).run.func1\
\t/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/loader/loader.go:152\
runtime.goexit\
\t/usr/local/go/src/runtime/asm_amd64.s:1337","error":{"ErrCode":10006,"ErrClass":1,"ErrScope":2,"ErrLevel":3,"Message":"file dm_test.t1.sql: execute statement failed: commit: Error 1062: Duplicate entry '1' for key 'PRIMARY'","RawCause":"Error 1062: Duplicate entry '1' for key 'PRIMARY'"}}"]
[2019/11/13 15:32:14.244 +08:00] [INFO] [server.go:227] [request=QueryTaskOperation] [payload="name:"test5" logID:186 "]
[2019/11/13 15:32:16.103 +08:00] [WARN] [task_checker.go:351] ["task can't auto resume"] [component="task checker"] [task=test5]

yilong · 2019 年11 月 13 日 07:46

您好： 1. 这个是在测试对吧？如果库里的数据没有用，都删掉，再看下是否能够成功，当前看报主键错误。 2. 或者改为syncer，但是要指定到对的位点信息才行。 3. cd 56xxxx-.00001 , 看下目录下的relay log 文件生成了吗？

sanger · 2019 年11 月 13 日 08:06

表删了，同步是成功了但是还是在报错

query-task test5

{
    "result": true,
    "msg": "",
    "workers": [
        {
            "result": true,
            "worker": "10.125.0.34:8262",
            "msg": "",
            "subTaskStatus": [
                {
                    "name": "test5",
                    "stage": "Paused",
                    "unit": "Sync",
                    "result": {
                        "isCanceled": false,
                        "errors": [
                        ],
                        "detail": null
                    },
                    "unresolvedDDLLockID": "",
                    "sync": {
                        "totalEvents": "0",
                        "totalTps": "0",
                        "recentTps": "0",
                        "masterBinlog": "(mysql-bin.000005, 74761)",
                        "masterBinlogGtid": "561d10a3-7614-11e9-8f94-246e96757168:1-1223163",
                        "syncerBinlog": "(, 4)",
                        "syncerBinlogGtid": "",
                        "blockingDDLs": [
                        ],
                        "unresolvedGroups": [
                        ],
                        "synced": false
                    }
                }
            ],
            "relayStatus": {
                "masterBinlog": "(mysql-bin.000005, 74761)",
                "masterBinlogGtid": "561d10a3-7614-11e9-8f94-246e96757168:1-1223163",
                "relaySubDir": "561d10a3-7614-11e9-8f94-246e96757168.000001",
                "relayBinlog": "(, 4)",
                "relayBinlogGtid": "",
                "relayCatchUpMaster": false,
                "stage": "Paused",
                "result": {
                    "isCanceled": false,
                    "errors": [
                        {
                            "Type": "UnknownError",
                            "msg": "[code=30015:class=relay-unit:scope=upstream:level=high] TCPReader get event: ERROR 1236 (HY000): The slave is connecting using CHANGE MASTER TO MASTER_AUTO_POSITION = 1, but the master has purged binary logs containing GTIDs that the slave requires.
github.com/pingcap/dm/pkg/terror.(*Error).Delegate
	/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/terror/terror.go:267
github.com/pingcap/dm/pkg/binlog/reader.(*TCPReader).GetEvent
	/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/binlog/reader/tcp.go:151
github.com/pingcap/dm/relay/reader.(*reader).GetEvent
	/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/reader/reader.go:144
github.com/pingcap/dm/relay.(*Relay).handleEvents
	/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/relay.go:378
github.com/pingcap/dm/relay.(*Relay).process
	/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/relay.go:292
github.com/pingcap/dm/relay.(*Relay).Process
	/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/relay/relay.go:191
github.com/pingcap/dm/dm/worker.(*realRelayHolder).run
	/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/dm/worker/relay.go:164
github.com/pingcap/dm/dm/worker.(*realRelayHolder).Start.func1
	/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/dm/worker/relay.go:140
runtime.goexit
	/usr/local/go/src/runtime/asm_amd64.s:1337",
                            "error": null
                        }
                    ],
                    "detail": null
                }
            },
            "sourceID": "mysql-replica-01"
        }
    ]
}

dm-work

[2019/11/13 16:03:49.274 +08:00] [ERROR] [subtask.go:183] ["wait condition"] [subtask=test5] [error="[code=40067:class=dm-worker:scope=internal:level=high] wait relay catchup timeout, loader end binlog pos: (mysql-bin.000005, 64313), relay binlog pos: (, 4)"]
[2019/11/13 16:03:51.103 +08:00] [WARN] [task_checker.go:354] ["backoff skip auto resume task"] [component="task checker"] [task=test5] [latestResumeTime=2019/11/13 16:02:21.103 +08:00] [duration=2m16.455801104s]
[2019/11/13 16:03:56.103 +08:00] [WARN] [task_checker.go:354] ["backoff skip auto resume task"] [component="task checker"] [task=test5] [latestResumeTime=2019/11/13 16:02:21.103 +08:00] [duration=3m50.492712013s]
[2019/11/13 16:03:57.103 +08:00] [ERROR] [subtask.go:419] ["wait condition"] [subtask=test5] [error="[code=40067:class=dm-worker:scope=internal:level=high] wait relay catchup timeout, loader end binlog pos: (mysql-bin.000005, 64313), relay binlog pos: (, 4)"]
[2019/11/13 16:03:57.103 +08:00] [INFO] [worker.go:883] ["end to execute operation"] [component="worker controller"] ["oplog ID"=192] [error="[code=40067:class=dm-worker:scope=internal:level=high] wait relay catchup timeout, loader end binlog pos: (mysql-bin.000005, 64313), relay binlog pos: (, 4)"]
[2019/11/13 16:03:57.202 +08:00] [INFO] [worker.go:792] ["start to execute operation"] [component="worker controller"] [oplog="{"id":193,"task":{"op":6,"name":"test5"},"ts":1573631941103556898}"]
[2019/11/13 16:03:57.202 +08:00] [INFO] [worker.go:879] ["auto_resume sub task"] [component="worker controller"] [task=test5]
[2019/11/13 16:03:57.202 +08:00] [INFO] [subtask.go:624] ["wait condition between two units"] [subtask=test5] ["previous unit"=Load] [unit=Sync]

sanger · 2019 年11 月 13 日 08:09

每次新建同步任务都要指定最新的binlog-name, binlog-pos么

小王同学 · 2019 年11 月 13 日 08:10

看下 dm-worker 是否拉取上游的 binlog ？ relay.meta 中的记录当前是什么情况？

sanger · 2019 年11 月 13 日 08:11

小王同学 · 2019 年11 月 13 日 08:18

看起来是因为 dm-worker 没有从上游拉取 binlog 导致，和 task 同步任务如何配置没有关系。
可以按照以下方式恢复

停掉 dm-worker
修改 relay.meta 中的内容，需要从上游哪个 GTID 位点开始拉取
启动 dm-worker

另外 v1.1.0-alpha-29-gb95de78 ，alpha 版本不是稳定版，目前 DM 版本为 1.0.x 推荐使用 1.0.2 版本。

https://pingcap.com/docs-cn/stable/reference/tools/data-migration/releases/1.0.2/

sanger · 2019 年11 月 13 日 08:44

好像还是不行我换个稳定版安装下试试吧谢谢各位pingcap的大佬啦

小王同学 · 2019 年11 月 13 日 08:49

好的，换个稳定版本试下，启动 dm-worker 后，会去从上游拉取 binlog，拉取正常后，再启动 task 同步任务。如果还有其他问题，可以贴出来继续跟进。辛苦。

sanger · 2019 年11 月 14 日 06:34

换了稳定版本，测试task已经成功啦

小王同学 · 2019 年11 月 21 日 06:22

system · 2022 年10 月 31 日 19:12

此话题已在最后回复的 1 分钟后被自动关闭。不再允许新回复。