dm start-task 报错 check dump privileges of source DB

chaojiwudidashuaige · 2020 年4 月 10 日 06:56

数据源：mysql8.0.11

dm版本：dm-ansible-v1.0.4 dm已经启动成功，work节点可以看到拉取的binlog文件

task任务配置

$ grep -vE "#|^$" conf/task.yaml
name: "test"
task-mode: "all"
target-database:
  host: "xx.xx.xx.xx"
  port: 4000
  user: "root"
  password: "xxxxxxxxxxxxxxxxxx="
mysql-instances:
-
  source-id: "xxxx3306"
  black-white-list: "global"
black-white-list:
  global:
    do-dbs: ["db1","db2"]

启动报错

» start-task ./conf/task.yaml
{
    "result": false,
    "msg": "[code=26005:class=dm-master:scope=internal:level=medium] fail to check synchronization configuration with type CheckFailed: check was failed, please see detail
    	detail: {
		"results": [
			{
				"id": 4,
				"name": "source db dump privilege chcker",
				"desc": "check dump privileges of source DB",
				"state": "fail",
				"errorMsg": "line 1 column 83 near \"FILE, REFERENCES, INDEX, ALTER, SHOW DATABASES, SUPER, CREATE TEMPORARY TABLES, LOCK TABLES, EXECUTE, REPLICATION SLAVE, REPLICATION CLIENT, CREATE VIEW, SHOW VIEW, CREATE ROUTINE, ALTER ROUTINE, CREATE USER, EVENT, TRIGGER, CREATE TABLESPACE, CREATE ROLE, DROP ROLE ON *.* TO `root`@`%`\" \
github.com/pingcap/errors.AddStack\
\t/go/pkg/mod/github.com/pingcap/errors@v0.11.4/errors.go:174\
github.com/pingcap/errors.Trace\
\t/go/pkg/mod/github.com/pingcap/errors@v0.11.4/juju_adaptor.go:15\
github.com/pingcap/parser.(*Parser).Parse\
\t/go/pkg/mod/github.com/pingcap/parser@v0.0.0-20191112053614-3b43b46331d5/yy_parser.go:137\
github.com/pingcap/parser.(*Parser).ParseOneStmt\
\t/go/pkg/mod/github.com/pingcap/parser@v0.0.0-20191112053614-3b43b46331d5/yy_parser.go:156\
github.com/pingcap/tidb-tools/pkg/check.verifyPrivileges\
\t/go/pkg/mod/github.com/pingcap/tidb-tools@v3.0.7-0.20191202034632-451c58d281c7+incompatible/pkg/check/privilege.go:125\
github.com/pingcap/tidb-tools/pkg/check.(*SourceDumpPrivilegeChecker).Check\
\t/go/pkg/mod/github.com/pingcap/tidb-tools@v3.0.7-0.20191202034632-451c58d281c7+incompatible/pkg/check/privilege.go:58\
github.com/pingcap/tidb-tools/pkg/check.Do.func2\
\t/go/pkg/mod/github.com/pingcap/tidb-tools@v3.0.7-0.20191202034632-451c58d281c7+incompatible/pkg/check/check.go:118\
runtime.goexit\
\t/usr/local/go/src/runtime/asm_amd64.s:1357\
grants[0] GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, RELOAD, SHUTDOWN, PROCESS, FILE, REFERENCES, INDEX, ALTER, SHOW DATABASES, SUPER, CREATE TEMPORARY TABLES, LOCK TABLES, EXECUTE, REPLICATION SLAVE, REPLICATION CLIENT, CREATE VIEW, SHOW VIEW, CREATE ROUTINE, ALTER ROUTINE, CREATE USER, EVENT, TRIGGER, CREATE TABLESPACE, CREATE ROLE, DROP ROLE ON *.* TO `root`@`%`",
				"instruction": "",
				"extra": "address of db instance - xx.xx.xx.xx:3306"
			},
			{
				"id": 5,
				"name": "source db replication privilege chcker",
				"desc": "check replication privileges of source DB",
				"state": "fail",
				"errorMsg": "line 1 column 83 near \"FILE, REFERENCES, INDEX, ALTER, SHOW DATABASES, SUPER, CREATE TEMPORARY TABLES, LOCK TABLES, EXECUTE, REPLICATION SLAVE, REPLICATION CLIENT, CREATE VIEW, SHOW VIEW, CREATE ROUTINE, ALTER ROUTINE, CREATE USER, EVENT, TRIGGER, CREATE TABLESPACE, CREATE ROLE, DROP ROLE ON *.* TO `root`@`%`\" \
github.com/pingcap/errors.AddStack\
\t/go/pkg/mod/github.com/pingcap/errors@v0.11.4/errors.go:174\
github.com/pingcap/errors.Trace\
\t/go/pkg/mod/github.com/pingcap/errors@v0.11.4/juju_adaptor.go:15\
github.com/pingcap/parser.(*Parser).Parse\
\t/go/pkg/mod/github.com/pingcap/parser@v0.0.0-20191112053614-3b43b46331d5/yy_parser.go:137\
github.com/pingcap/parser.(*Parser).ParseOneStmt\
\t/go/pkg/mod/github.com/pingcap/parser@v0.0.0-20191112053614-3b43b46331d5/yy_parser.go:156\
github.com/pingcap/tidb-tools/pkg/check.verifyPrivileges\
\t/go/pkg/mod/github.com/pingcap/tidb-tools@v3.0.7-0.20191202034632-451c58d281c7+incompatible/pkg/check/privilege.go:125\
github.com/pingcap/tidb-tools/pkg/check.(*SourceReplicatePrivilegeChecker).Check\
\t/go/pkg/mod/github.com/pingcap/tidb-tools@v3.0.7-0.20191202034632-451c58d281c7+incompatible/pkg/check/privilege.go:96\
github.com/pingcap/tidb-tools/pkg/check.Do.func2\
\t/go/pkg/mod/github.com/pingcap/tidb-tools@v3.0.7-0.20191202034632-451c58d281c7+incompatible/pkg/check/check.go:118\
runtime.goexit\
\t/usr/local/go/src/runtime/asm_amd64.s:1357\
grants[0] GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, RELOAD, SHUTDOWN, PROCESS, FILE, REFERENCES, INDEX, ALTER, SHOW DATABASES, SUPER, CREATE TEMPORARY TABLES, LOCK TABLES, EXECUTE, REPLICATION SLAVE, REPLICATION CLIENT, CREATE VIEW, SHOW VIEW, CREATE ROUTINE, ALTER ROUTINE, CREATE USER, EVENT, TRIGGER, CREATE TABLESPACE, CREATE ROLE, DROP ROLE ON *.* TO `root`@`%`",
				"instruction": "",
				"extra": "address of db instance - xx.xx.xx.xx:3306"
			},
			{
				"id": 2,
				"name": "mysql_binlog_format",
				"desc": "check whether mysql binlog_format is ROW",
				"state": "success",
				"errorMsg": "",
				"instruction": "",
				"extra": "address of db instance - xx.xx.xx.xx:3306"
			},
			{
				"id": 1,
				"name": "mysql_binlog_enable",
				"desc": "check whether mysql binlog is enabled",
				"state": "success",
				"errorMsg": "",
				"instruction": "",
				"extra": "address of db instance - xx.xx.xx.xx:3306"
			},
			{
				"id": 0,
				"name": "mysql_version",
				"desc": "check whether mysql version is satisfied",
				"state": "fail",
				"errorMsg": "version required less than 5.8.0 but got 8.0.11",
				"instruction": "",
				"extra": "address of db instance - xx.xx.xx.xx:3306"
			},
			{
				"id": 3,
				"name": "mysql_binlog_row_image",
				"desc": "check whether mysql binlog_row_image is FULL",
				"state": "success",
				"errorMsg": "",
				"instruction": "",
				"extra": "address of db instance - xx.xx.xx.xx:3306"
			},
			{
				"id": 6,
				"name": "table structure compatibility check",
				"desc": "check compatibility of table structure",
				"state": "success",
				"errorMsg": "",
				"instruction": "",
				"extra": "address of db instance - xx.xx.xx.xx:3306"
			}
		],
		"summary": {
			"passed": false,
			"total": 7,
			"successful": 4,
			"failed": 3,
			"warning": 0
		}
	}
github.com/pingcap/dm/pkg/terror.(*Error).Generate
	/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/pkg/terror/terror.go:232
github.com/pingcap/dm/checker.CheckSyncConfig
	/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/checker/cmd.go:73
github.com/pingcap/dm/dm/master.(*Server).generateSubTask
	/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/dm/master/server.go:1876
github.com/pingcap/dm/dm/master.(*Server).StartTask
	/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/dm/master/server.go:230
github.com/pingcap/dm/dm/pb._Master_StartTask_Handler
	/home/jenkins/agent/workspace/build_dm_master/go/src/github.com/pingcap/dm/dm/pb/dmmaster.pb.go:2355
google.golang.org/grpc.(*Server).processUnaryRPC
	/go/pkg/mod/google.golang.org/grpc@v1.25.1/server.go:1007
google.golang.org/grpc.(*Server).handleStream
	/go/pkg/mod/google.golang.org/grpc@v1.25.1/server.go:1287
google.golang.org/grpc.(*Server).serveStreams.func1.1
	/go/pkg/mod/google.golang.org/grpc@v1.25.1/server.go:722
runtime.goexit
	/usr/local/go/src/runtime/asm_amd64.s:1357",
    "workers": [
    ]
}

另外个人疑问：

mydumper配置是否要写到task.yaml中？手册上有的有，有的没。
一个task任务是不是只能同步一个DB？如果数据源是单实例多DB，就需要多个task任务？

qizheng · 2020 年4 月 10 日 07:28

1、 check dump privileges of source DB 权限报错是因为有些权限如 CREATE TABLESPACE TiDB 不支持，按照文档说明授予最小权限即可

REPLICATION SLAVE
REPLICATION CLIENT
RELOAD
SELECT

2、check whether mysql version is satisfied 这里提示 version required less than 5.8.0 but got 8.0.11，目前 DM 还不支持上游 mysql 8.0 版本

chaojiwudidashuaige · 2020 年4 月 10 日 07:30

麻烦回答下两个疑问，辛苦，谢谢

另外个人疑问：

mydumper配置是否要写到task.yaml中？手册上有的有，有的没。

一个task任务是不是只能同步一个DB？如果数据源是单实例多DB，就需要多个task任务？

qizheng · 2020 年4 月 10 日 07:40

1、mydumper 是指 extra-args 这部分指定表名的配置吗，这个配置在 v1.0.2 版本 DM 会自动生成 table-list 配置，在其之前的版本仍然需要人工配置。

2、一个 task 任务可以同步多个 DB，配置好对应 DB 的 routes 规则即可。

详细的 task 配置信息参考 https://github.com/pingcap/dm/blob/master/dm/master/task_advanced.yaml

chaojiwudidashuaige · 2020 年4 月 10 日 07:44

感谢
另外，

"errorMsg": "version required less than 5.8.0 but got 8.0.11",

版本版本不支持

yilong · 2020 年4 月 10 日 09:17

当前DM不支持 Mysql 8.0 多谢

chaojiwudidashuaige · 2020 年4 月 14 日 04:22

请问，什么时候可以支持mysql8.0版本。谢谢

来了老弟 · 2020 年4 月 14 日 04:26

你好，

dm 支持 8.0 长远看是有计划的，可以持续关注下。

system · 2022 年10 月 31 日 19:05

此话题已在最后回复的 1 分钟后被自动关闭。不再允许新回复。