场景描述:
在搭建mysql、dm、tidb,并创建表,创建表后发现mysql未开启binlog,开启binlog后配置mydumper、myloader、syncer。在开启binlog后的创建表能同步,未开启binlog前的表进行insert提示 [“unit process error”] [subtask=syncer] [unit=Sync] [“error information”="{“msg”:"Error 1146: Table ‘hydee.t_stock_a’ doesn’t exist\ngithub.com/pingcap/errors.AddStack
是mydumper和myloader配置问题吗?
附(dm-task.yml):
name: “syncer”
task-mode: “all”
target-database:
host: “192.168.10.111”
port: 4000
user: “root”
password: “SYUDS9UPfUnc2jSCZ6vya0B+SVgPhNvWQP0=”
mysql-instances:
source-id: “mysql”
black-white-list: “global”
mydumper-config-name: “global”
loader-config-name: “global”
syncer-config-name: “global”
black-white-list:
global:
do-tables:
- db-name: “hydee”
tbl-name: “~^t.*”
mydumpers:
global:
mydumper-path: “./bin/mydumper”
extra-args: “-B hydee --regex=t.*”
loaders:
global:
pool-size: 16
dir: “./dumped_data”
syncers:
global:
worker-count: 5
batch: 5
max-retry: 10
mydumper导出参数默认–outputdir,./dumped_data.syncer
[2019/08/01 13:15:15.382 +08:00] [INFO] [mydumper.go:246] [“create mydumper”] [task=syncer] [unit=dump] [argument="[–host,192.168.10.200,–port,3306,–user,mysql,–outputdir,./dumped_data.syncer,–logfile,/dev/stderr,–verbose,3,–threads,4,–chunk-filesize,64,–skip-tz-utc]"]
执行的时候没有输出文件到./dumped_data.syncer
[2019/08/01 13:15:15.559 +08:00] [INFO] [mydumper.go:136] [“Thread 1 dumping data for hydee
.t_out_stock_a
”] [task=syncer] [unit=dump]
[2019/08/01 13:15:15.560 +08:00] [INFO] [mydumper.go:136] [“Thread 4 dumping data for hydee
.t_out_stock_c
”] [task=syncer] [unit=dump]
[2019/08/01 13:15:15.560 +08:00] [INFO] [mydumper.go:136] [“Thread 3 dumping data for hydee
.t_stock_a
”] [task=syncer] [unit=dump]
[2019/08/01 13:15:15.561 +08:00] [INFO] [mydumper.go:136] [“Thread 1 dumping data for hydee
.t_stock_c
”] [task=syncer] [unit=dump]
[2019/08/01 13:15:15.562 +08:00] [INFO] [mydumper.go:136] [“Empty table hydee.t_stock_a”] [task=syncer] [unit=dump]
[2019/08/01 13:15:15.562 +08:00] [INFO] [mydumper.go:136] [“Thread 3 dumping data for hydee
.t_stock_d
”] [task=syncer] [unit=dump]
[2019/08/01 13:15:15.562 +08:00] [INFO] [mydumper.go:136] [“Empty table hydee.t_out_stock_c”] [task=syncer] [unit=dump]
[2019/08/01 13:15:15.562 +08:00] [INFO] [mydumper.go:136] [“Thread 4 dumping data for mysql
.engine_cost
”] [task=syncer] [unit=dump]
[2019/08/01 13:15:15.562 +08:00] [INFO] [mydumper.go:136] [“Empty table hydee.t_stock_c”] [task=syncer] [unit=dump]
[2019/08/01 13:15:15.562 +08:00] [INFO] [mydumper.go:136] [“Thread 1 dumping data for mysql
.gtid_executed
”] [task=syncer] [unit=dump]
[2019/08/01 13:15:15.563 +08:00] [INFO] [mydumper.go:136] [“Empty table hydee.t_stock_d”] [task=syncer] [unit=dump]
loader时被忽略了
[2019/08/01 13:15:15.603 +08:00] [WARN] [loader.go:725] [“ignore table file”] [task=syncer] [unit=load] [“table file”=hydee.t_stock_d-schema.sql]
[2019/08/01 13:15:15.603 +08:00] [WARN] [loader.go:725] [“ignore table file”] [task=syncer] [unit=load] [“table file”=hydee.t_stock_a-schema.sql]
loader导入成功的
[2019/08/01 13:15:15.607 +08:00] [INFO] [loader.go:985] [“start to create schema”] [task=syncer] [unit=load] [“schema file”=./dumped_data.syncer/hydee-schema-create.sql]
[2019/08/01 13:15:15.609 +08:00] [WARN] [db.go:172] [“execute statement”] [task=syncer] [unit=load] [sqls=“CREATE DATABASE hydee
/*!40100 DEFAULT CHARACTER SET utf8 */;”] [error=“Error 1007: Can’t create database ‘hydee’; database exists”]
[2019/08/01 13:15:15.610 +08:00] [INFO] [loader.go:857] [“database already exists, skip it”] [task=syncer] [unit=load] [“db schema file”=./dumped_data.syncer/hydee-schema-create.sql]
[2019/08/01 13:15:15.610 +08:00] [INFO] [loader.go:990] [“finish to create schema”] [task=syncer] [unit=load] [“schema file”=./dumped_data.syncer/hydee-schema-create.sql]
[2019/08/01 13:15:15.610 +08:00] [INFO] [loader.go:1012] [“start to create table”] [task=syncer] [unit=load] [“table file”=./dumped_data.syncer/hydee.t_out_stock_a-schema.sql]
[2019/08/01 13:15:15.702 +08:00] [WARN] [db.go:146] [“transaction execute successfully”] [task=syncer] [unit=load] [“cost time”=91.122899ms]
[2019/08/01 13:15:15.702 +08:00] [INFO] [loader.go:1017] [“finish to create table”] [task=syncer] [unit=load] [“table file”=./dumped_data.syncer/hydee.t_out_stock_a-schema.sql]
[2019/08/01 13:15:15.702 +08:00] [INFO] [loader.go:1051] [“finish to create tables”] [task=syncer] [unit=load] [“cost time”=95.224711ms]
配置文件在window上写的,文件为doc格式,改为unix后就好了,应该是loaders解析出问题了
system
(system)
关闭
4
此话题已在最后回复的 1 分钟后被自动关闭。不再允许新回复。