[FAQ] tidb-lightning importer 模式导入数据失败, lightning-err 日志报错:403 Forbidden

【问题澄清】

  • tidb v3.0.15 & tools v3.0.15
  • 用户使用 tidb-lightning importer 模式导入数据失败,import log 中没有明显报错,仅为启动日志,lightning err log 中有如下报错信息:

[2020/06/23 10:50:14.318 +08:00] [INFO] [lightning.go:202] [“load data source start”]
[2020/06/23 10:50:14.318 +08:00] [INFO] [lightning.go:205] [“load data source completed”] [takeTime=222.867µs] []
[2020/06/23 10:50:14.318 +08:00] [INFO] [checkpoints.go:770] [“open checkpoint file failed, going to create a new one”] [path=/tmp/tidb_lightning_checkpoint.pb] [error=“open /tmp/tidb_lightning_checkpoint.pb: no such file or directory”]
[2020/06/23 10:50:14.325 +08:00] [INFO] [restore.go:255] [“the whole procedure start”]
[2020/06/23 10:50:14.336 +08:00] [INFO] [restore.go:293] [“restore table schema start”] [db=demo]
[2020/06/23 10:50:14.419 +08:00] [INFO] [tidb.go:104] [“create tables start”] [db=demo]
[2020/06/23 10:50:14.506 +08:00] [INFO] [tidb.go:122] [“create tables completed”] [db=demo] [takeTime=87.126301ms] []
[2020/06/23 10:50:14.506 +08:00] [INFO] [restore.go:301] [“restore table schema completed”] [db=demo] [takeTime=170.950998ms] []
[2020/06/23 10:50:14.510 +08:00] [INFO] [restore.go:575] [“restore all tables data start”]
[2020/06/23 10:50:14.510 +08:00] [INFO] [restore.go:596] [“restore table start”] [table=demo.A]
[2020/06/23 10:50:14.510 +08:00] [INFO] [restore.go:1261] [“load engines and files start”] [table=demo.A]
[2020/06/23 10:50:14.510 +08:00] [INFO] [restore.go:1290] [“load engines and files completed”] [table=demo.A] [enginesCnt=2] [filesCnt=7] [takeTime=34.869µs] []
[2020/06/23 10:50:14.511 +08:00] [ERROR] [restore.go:600] [“restore table failed”] [table=demo.A] [takeTime=564.429µs] [error="restore table demo.A failed: rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: connection error: desc = "transport: Error while dialing failed to do connect handshake, response: \"HTTP/1.1 403 Forbidden\\r\\nContent-Length: 3490\\r\\nConnection: keep-alive\\r\\nContent-Language: en\\r\\nContent-Type: text/html;charset=utf-8\\r\\nDate: Tue, 23 Jun 2020 02:50:14 GMT\\r\\nMime-Version: 1.0\\r\\n

【解决方案】

  • Nginx 代理配置错误,临时将其去掉解决。

【排查思路】

  • err log 中存在 403 Forbidden 类型报错并伴随 rpc error,一般为无法通信问题,可以从当前网络和链路上进行分析。
  • 如果使用 backend 模式,403 错误一般为无法连接到 TiKV,可以从此入手进行排查

【最佳案例】

【更多链接】