升级增加TiFlash后DDL报错:ERROR 1105 (HY000): invalid ddl job type: none

为提高效率,请提供以下信息,问题描述清晰能够更快得到解决:

【TiDB 版本】4.0.12

【问题描述】
使用tiup从3.0.0升级到4.0.12后,增加tiflash集群,集群正常。使用 ALTER TABLE table_name SET TIFLASH REPLICA count后报错ERROR 1105 (HY000): invalid ddl job type: none
版本及执行计划如下所示


个人怀疑TiDB解析问题。由于TiDB配置由ansible导入,比较长,不确定是否和配置有关,tidb部分配置如下

server_configs:
  tidb:
    binlog:
      ignore-error: false
      write-timeout: 15s
    compatible-kill-query: false
    enable-streaming: false
    enable-telemetry: false
    host: 0.0.0.0
    lease: 45s
    log:
      disable-timestamp: false
      expensive-threshold: 10000
      file:
        log-rotate: true
        max-backups: 0
        max-days: 0
        max-size: 300
      format: text
      level: info
      query-log-max-len: 2048
      slow-threshold: 300
    lower-case-table-names: 2
    oom-action: log
    opentracing:
      enable: false
      reporter:
        buffer-flush-interval: 0
        local-agent-host-port: ""
        log-spans: false
        queue-size: 0
      rpc-metrics: false
      sampler:
        max-operations: 0
        param: 1.0
        sampling-refresh-interval: 0
        sampling-server-url: ""
        type: const
    performance:
      cross-join: true
      feedback-probability: 0.05
      force-priority: NO_PRIORITY
      max-procs: 0
      pseudo-estimate-ratio: 0.8
      query-feedback-limit: 1024
      run-auto-analyze: true
      stats-lease: 3s
      stmt-count-limit: 5000
      tcp-keep-alive: true
    prepared-plan-cache:
      capacity: 150
      enabled: true
      memory-guard-ratio: 0.1
    proxy-protocol:
      header-timeout: 5
      networks: ""
    run-ddl: true
    split-table: true
    status:
      report-status: true
    store: tikv
    tikv-client:
      commit-timeout: 41s
      grpc-connection-count: 24
      grpc-keepalive-time: 10
      grpc-keepalive-timeout: 3
    token-limit: 1000
    txn-local-latches:
      capacity: 2048000
      enabled: true

若提问为性能优化、故障排查类问题,请下载脚本运行。终端输出的打印结果,请务必全选并复制粘贴上传。

  1. tiup cluster display 看下当前集群状态
  2. 麻烦反馈下表结构
  3. 执行报错后,麻烦看下日志是否有更多详细信息,反馈下 tidb.log 和 tiflash.log ,多谢。
  1. tiup cluster display 看下当前集群状态
Starting component `cluster`: /home/panda/.tiup/components/cluster/v1.4.1/tiup-cluster display tidb1
Cluster type:       tidb
Cluster name:       tidb1
Cluster version:    v4.0.12
SSH type:           builtin
Dashboard URL:      http://10.218.80.252:2379/dashboard
ID                    Role          Host            Ports                            OS/Arch       Status   Data Dir                                              Deploy Dir
--                    ----          ----            -----                            -------       ------   --------                                              ----------
10.204.64.161:9093    alertmanager  10.204.64.161   9093/9094                        linux/x86_64  Up       /data/applications/tidb/data.alertmanager             /data/applications/tidb
10.153.185.229:8300   cdc           10.153.185.229  8300                             linux/x86_64  Up       -                                                     /data/applications/tidb/cdc-8300
10.153.188.49:8300    cdc           10.153.188.49   8300                             linux/x86_64  Up       -                                                     /data/applications/tidb/cdc-8300
10.153.191.82:8300    cdc           10.153.191.82   8300                             linux/x86_64  Up       -                                                     /data/applications/tidb/cdc-8300
10.156.83.11:8300     cdc           10.156.83.11    8300                             linux/x86_64  Up       -                                                     /data/applications/tidb/cdc-8300
10.158.11.106:8300    cdc           10.158.11.106   8300                             linux/x86_64  Up       -                                                     /data/applications/tidb/cdc-8300
10.158.143.111:8300   cdc           10.158.143.111  8300                             linux/x86_64  Up       -                                                     /data/applications/tidb/cdc-8300
10.158.15.52:8300     cdc           10.158.15.52    8300                             linux/x86_64  Up       -                                                     /data/applications/tidb/cdc-8300
10.158.76.19:8300     cdc           10.158.76.19    8300                             linux/x86_64  Up       -                                                     /data/applications/tidb/cdc-8300
10.158.78.173:8300    cdc           10.158.78.173   8300                             linux/x86_64  Up       -                                                     /data/applications/tidb/cdc-8300
10.204.64.161:3000    grafana       10.204.64.161   3000                             linux/x86_64  Up       -                                                     /data/applications/tidb
10.190.139.33:2379    pd            10.190.139.33   2379/2380                        linux/x86_64  Up       /data/applications/tidb/data.pd                       /data/applications/tidb
10.204.120.22:2379    pd            10.204.120.22   2379/2380                        linux/x86_64  Up       /data/applications/tidb/data.pd                       /data/applications/tidb
10.156.189.235:4000   tidb          10.156.189.235  4000/10080                       linux/x86_64  Up       -                                                     /data/applications/tidb
10.156.190.97:4000    tidb          10.156.190.97   4000/10080                       linux/x86_64  Up       -                                                     /data/applications/tidb
10.190.185.14:4000    tidb          10.190.185.14   4000/10080                       linux/x86_64  Up       -                                                     /data/applications/tidb
10.195.210.93:4000    tidb          10.195.210.93   4000/10080                       linux/x86_64  Up       -                                                     /data/applications/tidb
10.206.61.166:4000    tidb          10.206.61.166   4000/10080                       linux/x86_64  Up       -                                                     /data/applications/tidb
10.206.82.10:4000     tidb          10.206.82.10    4000/10080                       linux/x86_64  Up       -                                                     /data/applications/tidb
10.218.10.221:4000    tidb          10.218.10.221   4000/10080                       linux/x86_64  Up       -                                                     /data/applications/tidb
10.218.2.123:4000     tidb          10.218.2.123    4000/10080                       linux/x86_64  Up       -                                                     /data/applications/tidb
10.218.2.124:4000     tidb          10.218.2.124    4000/10080                       linux/x86_64  Up       -                                                     /data/applications/tidb
10.218.2.125:4000     tidb          10.218.2.125    4000/10080                       linux/x86_64  Up       -                                                     /data/applications/tidb
10.218.22.106:4000    tidb          10.218.22.106   4000/10080                       linux/x86_64  Up       -                                                     /data/applications/tidb
10.218.22.107:4000    tidb          10.218.22.107   4000/10080                       linux/x86_64  Up       -                                                     /data/applications/tidb
10.218.69.67:4000     tidb          10.218.69.67    4000/10080                       linux/x86_64  Up       -                                                     /data/applications/tidb
10.218.73.147:4000    tidb          10.218.73.147   4000/10080                       linux/x86_64  Up       -                                                     /data/applications/tidb
10.218.77.32:4000     tidb          10.218.77.32    4000/10080                       linux/x86_64  Up       -                                                     /data/applications/tidb
10.218.82.80:4000     tidb          10.218.82.80    4000/10080                       linux/x86_64  Up       -                                                     /data/applications/tidb
10.72.36.120:4000     tidb          10.72.36.120    4000/10080                       linux/x86_64  Up       -                                                     /data/applications/tidb
10.72.65.92:4000      tidb          10.72.65.92     4000/10080                       linux/x86_64  Up       -                                                     /data/applications/tidb
10.105.32.164:9000    tiflash       10.105.32.164   9000/8123/3930/20170/20292/8234  linux/x86_64  Up       /data/applications/tidb/tiflash-9000/data             /data/applications/tidb/tiflash-9000
10.190.129.161:9000   tiflash       10.190.129.161  9000/8123/3930/20170/20292/8234  linux/x86_64  Up       /data/applications/tidb/tiflash-9000/data             /data/applications/tidb/tiflash-9000
10.190.185.132:9000   tiflash       10.190.185.132  9000/8123/3930/20170/20292/8234  linux/x86_64  Up       /data/applications/tidb/tiflash-9000/data             /data/applications/tidb/tiflash-9000
10.190.185.152:9000   tiflash       10.190.185.152  9000/8123/3930/20170/20292/8234  linux/x86_64  Up       /data/applications/tidb/tiflash-9000/data             /data/applications/tidb/tiflash-9000
10.190.185.16:9000    tiflash       10.190.185.16   9000/8123/3930/20170/20292/8234  linux/x86_64  Up       /data/applications/tidb/tiflash-9000/data             /data/applications/tidb/tiflash-9000
10.218.71.57:9000     tiflash       10.218.71.57    9000/8123/3930/20170/20292/8234  linux/x86_64  Up       /data/applications/tidb/tiflash-9000/data             /data/applications/tidb/tiflash-9000
10.72.36.44:9000      tiflash       10.72.36.44     9000/8123/3930/20170/20292/8234  linux/x86_64  Up       /data/applications/tidb/tiflash-9000/data             /data/applications/tidb/tiflash-9000
10.72.36.48:9000      tiflash       10.72.36.48     9000/8123/3930/20170/20292/8234  linux/x86_64  Up       /data/applications/tidb/tiflash-9000/data             /data/applications/tidb/tiflash-9000
10.72.36.50:9000      tiflash       10.72.36.50     9000/8123/3930/20170/20292/8234  linux/x86_64  Up       /data/applications/tidb/tiflash-9000/data             /data/applications/tidb/tiflash-9000
10.72.36.57:9000      tiflash       10.72.36.57     9000/8123/3930/20170/20292/8234  linux/x86_64  Up       /data/applications/tidb/tiflash-9000/data             /data/applications/tidb/tiflash-9000
10.190.184.104:20160  tikv          10.190.184.104  20160/20180                      linux/x86_64  Up       /data/applications/tidb/data                          /data/applications/tidb
10.190.185.126:20160  tikv          10.190.185.126  20160/20180                      linux/x86_64  Up       /data/applications/tidb/data                          /data/applications/tidb
10.190.185.127:20160  tikv          10.190.185.127  20160/20180                      linux/x86_64  Up       /data/applications/tidb/data                          /data/applications/tidb
10.212.171.248:20160  tikv          10.212.171.248  20160/20180                      linux/x86_64  Up       /data/applications/tidb/data                          /data/applications/tidb
10.212.171.249:20160  tikv          10.212.171.249  20160/20180                      linux/x86_64  Up       /data/applications/tidb/data                          /data/applications/tidb
10.212.171.251:20160  tikv          10.212.171.251  20160/20180                      linux/x86_64  Up       /data/applications/tidb/data                          /data/applications/tidb
10.212.196.124:20160  tikv          10.212.196.124  20160/20180                      linux/x86_64  Up       /data/applications/tidb/data                          /data/applications/tidb
10.212.196.125:20160  tikv          10.212.196.125  20160/20180                      linux/x86_64  Up       /data/applications/tidb/data                          /data/applications/tidb
10.212.196.126:20160  tikv          10.212.196.126  20160/20180                      linux/x86_64  Up       /data/applications/tidb/data                          /data/applications/tidb
10.212.196.71:20160   tikv          10.212.196.71   20160/20180                      linux/x86_64  Up       /data/applications/tidb/data                          /data/applications/tidb
10.212.77.38:20160    tikv          10.212.77.38    20160/20180                      linux/x86_64  Up       /data/applications/tidb/data                          /data/applications/tidb
10.212.77.39:20160    tikv          10.212.77.39    20160/20180                      linux/x86_64  Up       /data/applications/tidb/data                          /data/applications/tidb
10.218.20.251:20160   tikv          10.218.20.251   20160/20180                      linux/x86_64  Up       /data/applications/tidb/data                          /data/applications/tidb
10.218.73.89:20160    tikv          10.218.73.89    20160/20180                      linux/x86_64  Up       /data/applications/tidb/data                          /data/applications/tidb
10.218.79.69:20160    tikv          10.218.79.69    20160/20180                      linux/x86_64  Up       /data/applications/tidb/data                          /data/applications/tidb
10.218.81.227:20160   tikv          10.218.81.227   20160/20180                      linux/x86_64  Up       /data/applications/tidb/data                          /data/applications/tidb
10.218.81.250:20160   tikv          10.218.81.250   20160/20180                      linux/x86_64  Up       /data/applications/tidb/data                          /data/applications/tidb
10.218.81.251:20160   tikv          10.218.81.251   20160/20180                      linux/x86_64  Up       /data/applications/tidb/data                          /data/applications/tidb
10.218.81.253:20160   tikv          10.218.81.253   20160/20180                      linux/x86_64  Up       /data/applications/tidb/data                          /data/applications/tidb
10.218.82.21:20160    tikv          10.218.82.21    20160/20180                      linux/x86_64  Up       /data/applications/tidb/data                          /data/applications/tidb
10.218.82.22:20160    tikv          10.218.82.22    20160/20180                      linux/x86_64  Up       /data/applications/tidb/data                          /data/applications/tidb
10.218.82.23:20160    tikv          10.218.82.23    20160/20180                      linux/x86_64  Up       /data/applications/tidb/data                          /data/applications/tidb
10.218.82.25:20160    tikv          10.218.82.25    20160/20180                      linux/x86_64  Up       /data/applications/tidb/data                          /data/applications/tidb
10.22.240.223:20160   tikv          10.22.240.223   20160/20180                      linux/x86_64  Up       /data/applications/tidb/data                          /data/applications/tidb
10.22.240.224:20160   tikv          10.22.240.224   20160/20180                      linux/x86_64  Up       /data/applications/tidb/data                          /data/applications/tidb
10.72.64.61:20160     tikv          10.72.64.61     20160/20180                      linux/x86_64  Up       /data/applications/tidb/data                          /data/applications/tidb
10.72.65.86:20160     tikv          10.72.65.86     20160/20180                      linux/x86_64  Up       /data/applications/tidb/data                          /data/applications/tidb
  1. 麻烦反馈下表结构
mysql> desc uis.uis_item_store;
+---------------------------------+---------------+------+------+---------+-------+
| Field                           | Type          | Null | Key  | Default | Extra |
+---------------------------------+---------------+------+------+---------+-------+
| item_id                         | bigint(20)    | NO   | PRI  | NULL    |       |
| item_vrtn_id                    | bigint(20)    | NO   | PRI  | NULL    |       |
| item_site_id                    | int(11)       | NO   |      | NULL    |       |
| site_name                       | text          | YES  |      | NULL    |       |
| prod_ref_id                     | bigint(20)    | YES  | MUL  | NULL    |       |
| slr_id                          | bigint(20)    | NO   | MUL  | NULL    |       |
| slr_name                        | varchar(255)  | NO   | MUL  | NULL    |       |
| curnt_price                     | decimal(18,2) | YES  |      | NULL    |       |
| orgnl_price                     | decimal(18,2) | YES  |      | NULL    |       |
| currency_id                     | int(11)       | YES  |      | NULL    |       |
| item_cndtn_id                   | int(11)       | YES  |      | NULL    |       |
| raw_cndtn_txt                   | text          | YES  |      | NULL    |       |
| auct_type_code                  | int(11)       | YES  |      | NULL    |       |
| auct_end_date                   | bigint(20)    | YES  |      | NULL    |       |
| auct_start_date                 | bigint(20)    | YES  |      | NULL    |       |
| auct_title                      | text          | NO   |      | NULL    |       |
| item_cntry_id                   | int(11)       | NO   |      | NULL    |       |
| item_loc                        | text          | NO   |      | NULL    |       |
| item_quantity                   | bigint(20)    | YES  |      | NULL    |       |
| picture_url                     | text          | YES  |      | NULL    |       |
| gallery_url                     | text          | YES  |      | NULL    |       |
| shipping_fee                    | decimal(18,2) | YES  |      | NULL    |       |
| shipping_type                   | int(11)       | YES  |      | NULL    |       |
| zip_code                        | text          | YES  |      | NULL    |       |
| last_modified_date              | bigint(20)    | YES  |      | NULL    |       |
| gtc_up_flag                     | tinyint(1)    | YES  |      | NULL    |       |
| slr_feedback_score              | bigint(20)    | YES  |      | NULL    |       |
| etrs_flag_glb                   | text          | YES  |      | NULL    |       |
| etrs_flag_us                    | text          | YES  |      | NULL    |       |
| etrs_flag_uk                    | text          | YES  |      | NULL    |       |
| etrs_flag_de                    | text          | YES  |      | NULL    |       |
| slr_lvl_code                    | text          | YES  |      | NULL    |       |
| ean_txt                         | text          | YES  |      | NULL    |       |
| brand                           | text          | YES  |      | NULL    |       |
| color                           | text          | YES  |      | NULL    |       |
| gender                          | text          | YES  |      | NULL    |       |
| isbn_10                         | text          | YES  |      | NULL    |       |
| isbn_13                         | text          | YES  |      | NULL    |       |
| mpn                             | text          | YES  |      | NULL    |       |
| upc                             | text          | YES  |      | NULL    |       |
| size                            | text          | YES  |      | NULL    |       |
| manufacturer                    | text          | YES  |      | NULL    |       |
| material                        | text          | YES  |      | NULL    |       |
| unit_type                       | text          | YES  |      | NULL    |       |
| unit_quantity                   | text          | YES  |      | NULL    |       |
| enrgy_effcnt_cls                | text          | YES  |      | NULL    |       |
| leaf_categ_id                   | bigint(20)    | YES  | MUL  | NULL    |       |
| leaf_categ_name                 | text          | YES  |      | NULL    |       |
| meta_categ_id                   | bigint(20)    | YES  | MUL  | NULL    |       |
| meta_categ_name                 | text          | YES  |      | NULL    |       |
| l2_categ_id                     | bigint(20)    | YES  |      | NULL    |       |
| l2_categ_name                   | text          | YES  |      | NULL    |       |
| l3_categ_id                     | bigint(20)    | YES  |      | NULL    |       |
| l4_categ_id                     | bigint(20)    | YES  |      | NULL    |       |
| l5_categ_id                     | bigint(20)    | YES  |      | NULL    |       |
| l6_categ_id                     | bigint(20)    | YES  |      | NULL    |       |
| vertical_name                   | text          | YES  |      | NULL    |       |
| slr_feedback_pct                | decimal(18,2) | YES  |      | NULL    |       |
| cntry_code                      | text          | YES  |      | NULL    |       |
| cbt_site_id                     | int(11)       | YES  |      | NULL    |       |
| gallery_guid                    | text          | YES  |      | NULL    |       |
| picture_guids                   | text          | YES  |      | NULL    |       |
| auct_durtn_days                 | bigint(20)    | YES  |      | NULL    |       |
| start_price                     | decimal(18,2) | YES  |      | NULL    |       |
| tax_rate                        | bigint(20)    | YES  |      | NULL    |       |
| tax_threshold                   | bigint(20)    | YES  |      | NULL    |       |
| inclusive_price                 | decimal(18,2) | YES  |      | NULL    |       |
| update_ts                       | bigint(20)    | NO   |      | NULL    |       |
| price_update_ts                 | bigint(20)    | NO   |      | NULL    |       |
| enrgy_effcnt_range              | text          | YES  |      | NULL    |       |
| delayed_item_index_time         | bigint(20)    | YES  |      | NULL    |       |
| delayed_item_last_modified_date | bigint(20)    | YES  |      | NULL    |       |
| hidden_slr_flag                 | tinyint(1)    | YES  |      | NULL    |       |
+---------------------------------+---------------+------+------+---------+-------+
73 rows in set (0.00 sec)
 mysql> show create table uis.uis_item_store;
+----------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table          | Create Table
+----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------[57/509]-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------$-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------$-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------$---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| uis_item_store | CREATE TABLE `uis_item_store` (
  `item_id` bigint(20) NOT NULL,
  `item_vrtn_id` bigint(20) NOT NULL,
  `item_site_id` int(11) NOT NULL,
  `site_name` text DEFAULT NULL,
  `prod_ref_id` bigint(20) DEFAULT NULL,
  `slr_id` bigint(20) NOT NULL,
  `slr_name` varchar(255) NOT NULL,
  `curnt_price` decimal(18,2) DEFAULT NULL,
  `orgnl_price` decimal(18,2) DEFAULT NULL,
  `currency_id` int(11) DEFAULT NULL,
  `item_cndtn_id` int(11) DEFAULT NULL,
  `raw_cndtn_txt` text DEFAULT NULL,
  `auct_type_code` int(11) DEFAULT NULL,
  `auct_end_date` bigint(20) DEFAULT NULL,
  `auct_start_date` bigint(20) DEFAULT NULL,
  `auct_title` text NOT NULL,
  `item_cntry_id` int(11) NOT NULL,
  `item_loc` text NOT NULL,
  `item_quantity` bigint(20) DEFAULT NULL,
  `picture_url` text DEFAULT NULL,
  `gallery_url` text DEFAULT NULL,
  `shipping_fee` decimal(18,2) DEFAULT NULL,
  `shipping_type` int(11) DEFAULT NULL,
  `zip_code` text DEFAULT NULL,
  `last_modified_date` bigint(20) DEFAULT NULL,
  `gtc_up_flag` tinyint(1) DEFAULT NULL,
  `slr_feedback_score` bigint(20) DEFAULT NULL,
  `etrs_flag_glb` text DEFAULT NULL,
  `etrs_flag_us` text DEFAULT NULL,
  `etrs_flag_uk` text DEFAULT NULL,
  `etrs_flag_de` text DEFAULT NULL,
  `slr_lvl_code` text DEFAULT NULL,
  `ean_txt` text DEFAULT NULL,
  `brand` text DEFAULT NULL,
  `color` text DEFAULT NULL,
  `gender` text DEFAULT NULL,
  `isbn_10` text DEFAULT NULL,
  `isbn_13` text DEFAULT NULL,
  `mpn` text DEFAULT NULL,
  `upc` text DEFAULT NULL,
  `size` text DEFAULT NULL,
  `manufacturer` text DEFAULT NULL,
  `material` text DEFAULT NULL,
  `unit_type` text DEFAULT NULL,
  `unit_quantity` text DEFAULT NULL,
  `enrgy_effcnt_cls` text DEFAULT NULL,
  `leaf_categ_id` bigint(20) DEFAULT NULL,
  `leaf_categ_name` text DEFAULT NULL,
  `meta_categ_id` bigint(20) DEFAULT NULL,
  `meta_categ_name` text DEFAULT NULL,
  `l2_categ_id` bigint(20) DEFAULT NULL,
  `l2_categ_name` text DEFAULT NULL,
  `l3_categ_id` bigint(20) DEFAULT NULL,
  `l4_categ_id` bigint(20) DEFAULT NULL,
  `l5_categ_id` bigint(20) DEFAULT NULL,
  `l6_categ_id` bigint(20) DEFAULT NULL,
  `vertical_name` text DEFAULT NULL,
  `slr_feedback_pct` decimal(18,2) DEFAULT NULL,
  `cntry_code` text DEFAULT NULL,
  `cbt_site_id` int(11) DEFAULT NULL,
  `gallery_guid` text DEFAULT NULL,
  `picture_guids` text DEFAULT NULL,
  `auct_durtn_days` bigint(20) DEFAULT NULL,
  `start_price` decimal(18,2) DEFAULT NULL,
  `tax_rate` bigint(20) DEFAULT NULL,
  `tax_threshold` bigint(20) DEFAULT NULL,
  `inclusive_price` decimal(18,2) DEFAULT NULL,
  `update_ts` bigint(20) NOT NULL,
  `price_update_ts` bigint(20) NOT NULL,
  `enrgy_effcnt_range` text DEFAULT NULL,
  `delayed_item_index_time` bigint(20) DEFAULT NULL,
  `delayed_item_last_modified_date` bigint(20) DEFAULT NULL,
  `hidden_slr_flag` tinyint(1) DEFAULT NULL,
  PRIMARY KEY (`item_id`,`item_vrtn_id`),
  KEY `idx_epid` (`prod_ref_id`),
  KEY `idx_slr_id` (`slr_id`),
  KEY `idx_slr_name` (`slr_name`),
  KEY `idx_leaf_id` (`leaf_categ_id`),
  KEY `idx_cascade_categ` (`meta_categ_id`,`l2_categ_id`,`l3_categ_id`,`l4_categ_id`,`l5_categ_id`,`l6_categ_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin |
+----------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)
  1. 执行报错后,麻烦看下日志是否有更多详细信息,反馈下 tidb.log 和 tiflash.log ,多谢。
    执行报错之后,tidb.log
[2021/04/24 03:56:41.643 -07:00] [INFO] [session.go:2397] ["CRUCIAL OPERATION"] [conn=26115] [schemaVersion=1214] [cur_db=] [sql="ALTER TABLE uis.uis_item_store SET TIFLASH REPLICA 2"] [user=root@10.153.188.107]
[2021/04/24 03:56:41.654 -07:00] [INFO] [ddl_worker.go:262] ["[ddl] add DDL jobs"] ["batch count"=1] [jobs="ID:1676, Type:set tiflash replica, State:none, SchemaState:none, SchemaID:728, TableID:890, RowCount:0, ArgLen:1, start time: 2021-04-24 03:56:41.596 -0700 GMT+7, Err:<nil>, ErrCount:0, SnapshotVersion:0; "]
[2021/04/24 03:56:41.654 -07:00] [INFO] [ddl.go:537] ["[ddl] start DDL job"] [job="ID:1676, Type:set tiflash replica, State:none, SchemaState:none, SchemaID:728, TableID:890, RowCount:0, ArgLen:1, start time: 2021-04-24 03:56:41.596 -0700 GMT+7, Err:<nil>, ErrCount:0, SnapshotVersion:0"] [query="ALTER TABLE uis.uis_item_store SET TIFLASH REPLICA 2"]
[2021/04/24 03:56:42.155 -07:00] [INFO] [tidb.go:219] ["rollbackTxn for ddl/autocommit failed"]
[2021/04/24 03:56:42.156 -07:00] [WARN] [session.go:1383] ["run statement failed"] [conn=26115] [schemaVersion=1214] [error="[ddl:3]invalid ddl job type: none"] [session="{\n  \"currDBName\": \"\",\n  \"id\": 26115,\n  \"status\": 2,\n  \"strictMode\": true,\n  \"user\": {\n    \"Username\": \"root\",\n    \"Hostname\": \"10.153.188.107\",\n    \"CurrentUser\": false,\n    \"AuthUsername\": \"root\",\n    \"AuthHostname\": \"%\"\n  }\n}"]
[2021/04/24 03:56:42.156 -07:00] [INFO] [conn.go:797] ["command dispatched failed"] [conn=26115] [connInfo="id:26115, addr:10.153.188.107:49824 status:10, collation:latin1_swedish_ci, user:root"] [command=Query] [status="inTxn:0, autocommit:1"] [sql="ALTER TABLE uis.uis_item_store SET TIFLASH REPLICA 2"] [txn_mode=OPTIMISTIC] [err="[ddl:3]invalid ddl job type: none\ngithub.com/pingcap/errors.AddStack\n\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/pkg/mod/github.com/pingcap/errors@v0.11.5-0.20201126102027-b0a155152ca3/errors.go:174\ngithub.com/pingcap/errors.Trace\n\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/pkg/mod/github.com/pingcap/errors@v0.11.5-0.20201126102027-b0a155152ca3/juju_adaptor.go:15\ngithub.com/pingcap/tidb/ddl.(*ddl).doDDLJob\n\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb/ddl/ddl.go:578\ngithub.com/pingcap/tidb/ddl.(*ddl).AlterTableSetTiFlashReplica\n\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb/ddl/ddl_api.go:3472\ngithub.com/pingcap/tidb/ddl.(*ddl).AlterTable\n\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb/ddl/ddl_api.go:2199\ngithub.com/pingcap/tidb/executor.(*DDLExec).executeAlterTable\n\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb/executor/ddl.go:366\ngithub.com/pingcap/tidb/executor.(*DDLExec).Next\n\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb/executor/ddl.go:86\ngithub.com/pingcap/tidb/executor.Next\n\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb/executor/executor.go:262\ngithub.com/pingcap/tidb/executor.(*ExecStmt).handleNoDelayExecutor\n\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb/executor/adapter.go:531\ngithub.com/pingcap/tidb/executor.(*ExecStmt).handleNoDelay\n\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb/executor/adapter.go:413\ngithub.com/pingcap/tidb/executor.(*ExecStmt).Exec\n\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb/executor/adapter.go:366\ngithub.com/pingcap/tidb/session.runStmt\n\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb/session/tidb.go:308\ngithub.com/pingcap/tidb/session.(*session).ExecuteStmt\n\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb/session/session.go:1380\ngithub.com/pingcap/tidb/server.(*TiDBContext).ExecuteStmt\n\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb/server/driver_tidb.go:262\ngithub.com/pingcap/tidb/server.(*clientConn).handleStmt\n\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb/server/conn.go:1430\ngithub.com/pingcap/tidb/server.(*clientConn).handleQuery\n\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb/server/conn.go:1419\ngithub.com/pingcap/tidb/server.(*clientConn).dispatch\n\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb/server/conn.go:1009\ngithub.com/pingcap/tidb/server.(*clientConn).Run\n\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb/server/conn.go:782\ngithub.com/pingcap/tidb/server.(*Server).onConn\n\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb/server/server.go:449\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1357"]

执行报错后tiflash下的所有log没有任何变化(包括tiflash.log,tiflash_cluster_manager.log,tiflash_error.log,tiflash_stderr.log,tiflash_tikv.log)

1、麻烦测试或观察一下 集群当前 DDL 的运行情况(是否其他的 DDL 运行正常)
2、给一下这个命令的结果:curl http://xx.xx.xx.xx:10080/info 其中 xx.xx.xx.xx 替换成 tidb-server 的 ip 及状态端口

另外,你的标题是升级,你是升级时执行的命令还是升级后执行的,最好环境描述一下

1、麻烦测试或观察一下 集群当前 DDL 的运行情况(是否其他的 DDL 运行正常)
其他DDL正常

mysql> create table test(c1 int not null ,key(c1));
Query OK, 0 rows affected (0.53 sec)

mysql> alter table test add column c2 int default null;
Query OK, 0 rows affected (1.01 sec)

mysql> drop table test;
Query OK, 0 rows affected (1.01 sec)

mysql> ALTER TABLE uis_item_store SET TIFLASH REPLICA 2;
ERROR 1105 (HY000): invalid ddl job type: none

2、给一下这个命令的结果:curl http://xx.xx.xx.xx:10080/info 其中 xx.xx.xx.xx 替换成 tidb-server 的 ip 及状态端口

#curl http://10.218.2.123:10080/info
{
 "is_owner": false,
 "max_procs": 16,
 "gogc": 100,
 "version": "5.7.25-TiDB-v4.0.12",
 "git_hash": "11a9254bec522c1991033a6098f6dccadc6eb7c1",
 "ddl_id": "f9f3bd14-3a5b-4545-8f5d-6dd9eba6610c",
 "ip": "10.218.2.123",
 "listening_port": 4000,
 "status_port": 10080,
 "lease": "45s",
 "binlog_status": "Off",
 "start_timestamp": 1619190062
}

另外,你的标题是升级,你是升级时执行的命令还是升级后执行的,最好环境描述一下

使用tiup从3.0.0升级到4.0.12后执行命令

1.猜测是组件直接版本问题:请检查一下所有的 tidb/tiflash 是不是也是升级到 v4.0.12,tiflash 二进制查看版本方法
2.猜测是配置问题,请另外用tiup 启动一个 v4.0.12 的集群,然后测试这个语句是否有问题。应该是没有问题的,然后你再导入你的配置,再检查一下。

1.猜测是组件直接版本问题:请检查一下 tiflash 是不是也是升级到 v4.0.12,方法
组件版本无误

#LD_LIBRARY_PATH=./ ./tiflash version
TiFlash
Release Version: v4.0.12
Edition:         Community
Git Commit Hash: e5e3fcf251bae181b9b6c8676f57c51eb22106d8
Git Branch:      heads/refs/tags/v4.0.12
UTC Build Time:  2021-04-02 04:29:24
Profile:         RELWITHDEBINFO

Raft Proxy
Git Commit Hash:   cc9b586f415b6dc7738cfd62f2fd47b50c1e3507
Git Commit Branch: HEAD
UTC Build Time:    2021-04-02 04:08:16
Rust Version:      rustc 1.42.0-nightly (0de96d37f 2019-12-19)
Profile:           release

2.猜测是配置问题,请另外用tiup 启动一个 v4.0.12 的集群,然后测试这个语句是否有问题。应该是没有问题的,然后你再导入你的配置,再检查一下。
在另一个新建的4.0.12的集群上apply从ansible导入的tidb配置文件,可以使用ALTER TABLE test SET TIFLASH REPLICA 2

mysql> create table test(c1 int not null ,key(c1));
Query OK, 0 rows affected (1.56 sec)

mysql> ALTER TABLE test SET TIFLASH REPLICA 2;
Query OK, 0 rows affected (2.05 sec)

尝试将从ansible导入的tidb部分的配置全部清空(手动在机器上操作,原因详细描述在最后),重启后依然报错

mysql> ALTER TABLE test SET TIFLASH REPLICA 2;
ERROR 1105 (HY000): invalid ddl job type: none

看了一下tidb.log,发现两个有意思的点

  • TiKV Min Version为v3.0.0
  • 即使tidb.toml清空,之前进程已经被kill的情况下,原来的配置仍会加载(比如commit-timeout为41s,这个时间印象深刻),请问是否是从pd或者别的地方(systemctl)中读取?
[panda@xxxxxx tidb]$ ps -ef | grep tidb
panda    18061     1 99 22:44 ?        00:10:32 bin/tidb-server -P 4000 --status=10080 --host=0.0.0.0 --advertise-address=10.218.2.123 --store=tikv --path=10.218.80.252:2379,10.190.139.33:2379,10.204.120.22:2379 --log-slow-query=log/tidb_slow_query.log --config=conf/tidb.toml --log-file=/data/applications/tidb/log/tidb.log
panda    22856  5885  0 22:54 pts/0    00:00:00 grep --color=auto tidb
panda    26117 26116  0 Apr23 ?        00:00:00 /bin/bash /data/applications/tidb/scripts/run_node_exporter.sh
panda    26118 26117  0 Apr23 ?        00:00:00 tee -i -a /data/applications/tidb/log/node_exporter.log
panda    26178 26177  0 Apr23 ?        00:00:00 /bin/bash /data/applications/tidb/scripts/run_blackbox_exporter.sh
panda    26179 26178  0 Apr23 ?        00:00:00 tee -i -a /data/applications/tidb/log/blackbox_exporter.log
[panda@xxxxxx tidb]$ cat conf/tidb.toml
# WARNING: This file is auto-generated. Do not edit! All your modification will be overwritten!
# You can use 'tiup cluster edit-config' and 'tiup cluster reload' to update the configuration
# All configuration items you want to change can be added to:
# server_configs:
#   tidb:
#     aa.b1.c3: value
#     aa.b2.c4: value
[log]
level = "debug"
slow-query-file = "tidb-slow.log"
[panda@xxxxxx tidb]$ head log/tidb.log
[2021/04/24 22:44:11.925 -07:00] [INFO] [printer.go:33] ["Welcome to TiDB."] ["Release Version"=v4.0.12] [Edition=Community] ["Git Commit Hash"=11a9254bec522c1991033a6098f6dccadc6eb7c1] ["Git Branch"=heads/refs/tags/v4.0.12] ["UTC Build Time"="2021-04-02 03:23:24"] [GoVersion=go1.13] ["Race Enabled"=false] ["Check Table Before Drop"=false] ["TiKV Min Version"=v3.0.0-60965b006877ca7234adaced7890d7b029ed1306]
[2021/04/24 22:44:11.928 -07:00] [INFO] [printer.go:47] ["loaded config"] [config="{\"host\":\"0.0.0.0\",\"advertise-address\":\"10.218.2.123\",\"port\":4000,\"cors\":\"\",\"store\":\"tikv\",\"path\":\"10.218.80.252:2379,10.190.139.33:2379,10.204.120.22:2379\",\"socket\":\"\",\"lease\":\"45s\",\"run-ddl\":true,\"split-table\":true,\"token-limit\":1000,\"oom-use-tmp-storage\":true,\"tmp-storage-path\":\"/tmp/65640_tidb/MC4wLjAuMDo0MDAwLzAuMC4wLjA6MTAwODA=/tmp-storage\",\"oom-action\":\"log\",\"mem-quota-query\":1073741824,\"tmp-storage-quota\":-1,\"enable-streaming\":false,\"enable-batch-dml\":false,\"lower-case-table-names\":2,\"server-version\":\"\",\"log\":{\"level\":\"debug\",\"format\":\"text\",\"disable-timestamp\":null,\"enable-timestamp\":null,\"disable-error-stack\":null,\"enable-error-stack\":null,\"file\":{\"filename\":\"/data/applications/tidb/log/tidb.log\",\"max-size\":300,\"max-days\":0,\"max-backups\":0},\"enable-slow-log\":true,\"slow-query-file\":\"log/tidb_slow_query.log\",\"slow-threshold\":300,\"expensive-threshold\":10000,\"query-log-max-len\":4096,\"record-plan-in-slow-log\":1},\"security\":{\"skip-grant-table\":false,\"ssl-ca\":\"\",\"ssl-cert\":\"\",\"ssl-key\":\"\",\"require-secure-transport\":false,\"cluster-ssl-ca\":\"\",\"cluster-ssl-cert\":\"\",\"cluster-ssl-key\":\"\",\"cluster-verify-cn\":null},\"status\":{\"status-host\":\"0.0.0.0\",\"metrics-addr\":\"\",\"status-port\":10080,\"metrics-interval\":15,\"report-status\":true,\"record-db-qps\":false},\"performance\":{\"max-procs\":0,\"max-memory\":0,\"server-memory-quota\":0,\"memory-usage-alarm-ratio\":0.8,\"stats-lease\":\"3s\",\"stmt-count-limit\":5000,\"feedback-probability\":0,\"query-feedback-limit\":512,\"pseudo-estimate-ratio\":0.8,\"force-priority\":\"NO_PRIORITY\",\"bind-info-lease\":\"3s\",\"txn-entry-size-limit\":6291456,\"txn-total-size-limit\":104857600,\"tcp-keep-alive\":true,\"cross-join\":true,\"run-auto-analyze\":true,\"agg-push-down-join\":false,\"committer-concurrency\":16,\"max-txn-ttl\":600000,\"gogc\":100},\"prepared-plan-cache\":{\"enabled\":false,\"capacity\":100,\"memory-guard-ratio\":0.1},\"opentracing\":{\"enable\":false,\"rpc-metrics\":false,\"sampler\":{\"type\":\"const\",\"param\":1,\"sampling-server-url\":\"\",\"max-operations\":0,\"sampling-refresh-interval\":0},\"reporter\":{\"queue-size\":0,\"buffer-flush-interval\":0,\"log-spans\":false,\"local-agent-host-port\":\"\"}},\"proxy-protocol\":{\"networks\":\"\",\"header-timeout\":5},\"tikv-client\":{\"grpc-connection-count\":4,\"grpc-keepalive-time\":10,\"grpc-keepalive-timeout\":3,\"commit-timeout\":\"41s\",\"max-batch-size\":128,\"overload-threshold\":200,\"max-batch-wait-time\":0,\"batch-wait-size\":8,\"enable-chunk-rpc\":true,\"region-cache-ttl\":600,\"store-limit\":0,\"store-liveness-timeout\":\"5s\",\"copr-cache\":{\"enable\":false,\"capacity-mb\":1000,\"admission-max-ranges\":500,\"admission-max-result-mb\":10,\"admission-min-process-ms\":5}},\"binlog\":{\"enable\":false,\"ignore-error\":false,\"write-timeout\":\"15s\",\"binlog-socket\":\"\",\"strategy\":\"range\"},\"compatible-kill-query\":false,\"plugin\":{\"dir\":\"\",\"load\":\"\"},\"pessimistic-txn\":{\"enable\":true,\"max-retry-count\":256},\"check-mb4-value-in-utf8\":true,\"max-index-length\":3072,\"graceful-wait-before-shutdown\":0,\"alter-primary-key\":false,\"treat-old-version-utf8-as-utf8mb4\":true,\"enable-table-lock\":false,\"delay-clean-table-lock\":0,\"split-region-max-num\":1000,\"stmt-summary\":{\"enable\":true,\"enable-internal-query\":false,\"max-stmt-count\":200,\"max-sql-length\":4096,\"refresh-interval\":1800,\"history-size\":24},\"repair-mode\":false,\"repair-table-list\":[],\"isolation-read\":{\"engines\":[\"tikv\",\"tiflash\",\"tidb\"]},\"max-server-connections\":0,\"new_collations_enabled_on_first_bootstrap\":false,\"experimental\":{\"allow-expression-index\":false},\"enable-collect-execution-info\":true,\"skip-register-to-dashboard\":false,\"enable-telemetry\":true}"]

另外发现一个问题,使用tiup cluster edit-config删除所有从ansible导入的tidb部分的配置,之后reload的config,但并没有在tidb上生效,可能和主线问题有关联

猜测是升级还没有完成,角色为 DDL owner 的 TiDB 节点还处于旧版本。

可以将 curl http://xx.xx.xx.xx:10080/info 在所有 TiDB 上执行一遍,检查 is_owner: true 的 TiDB 是否已经升上了 4.0.12。如果没有,可以通过 curl -X POST http://{TiDBIP}:10080/ddl/owner/resign 驱逐到其他节点,直到 DDL owner 的 TiDB 版本为 4.0.12。

针对第一个问题,我后来修改了一下是: 所有的 tidb/tiflash 是不是也是升级到 v4.0.12?这个要把所有的 二进制都检查一下。
第二个问题我再看看你这里细节

@tangenta @fzhedu
一共18个tidb节点,所有version都是5.7.25-TiDB-v4.0.12,但是所有is_owner都是false
tiflash是升到4.0.12新增的,应该不存在别的版本的可能

{
 "is_owner": false,
 "max_procs": 16,
 "gogc": 100,
 "version": "5.7.25-TiDB-v4.0.12",
 "git_hash": "11a9254bec522c1991033a6098f6dccadc6eb7c1",
 "ddl_id": "c3f0d91b-33ed-4bf8-878d-a89393025f68",
 "ip": "10.156.189.235",
 "listening_port": 4000,
 "status_port": 10080,
 "lease": "45s",
 "binlog_status": "Off",
 "start_timestamp": 1619338883
}{
 "is_owner": false,
 "max_procs": 16,
 "gogc": 100,
 "version": "5.7.25-TiDB-v4.0.12",
 "git_hash": "11a9254bec522c1991033a6098f6dccadc6eb7c1",
 "ddl_id": "718c7e58-1c8c-471f-91f2-67d72da6d94e",
 "ip": "10.156.190.97",
 "listening_port": 4000,
 "status_port": 10080,
 "lease": "45s",
 "binlog_status": "Off",
 "start_timestamp": 1619338861
}{
 "is_owner": false,
 "max_procs": 12,
 "gogc": 100,
 "version": "5.7.25-TiDB-v4.0.12",
 "git_hash": "11a9254bec522c1991033a6098f6dccadc6eb7c1",
 "ddl_id": "7b2d60ea-602e-45f1-a0e8-984c970e7483",
 "ip": "10.190.185.14",
 "listening_port": 4000,
 "status_port": 10080,
 "lease": "45s",
 "binlog_status": "Off",
 "start_timestamp": 1619338820
}{
 "is_owner": false,
 "max_procs": 16,
 "gogc": 100,
 "version": "5.7.25-TiDB-v4.0.12",
 "git_hash": "11a9254bec522c1991033a6098f6dccadc6eb7c1",
 "ddl_id": "b26d6ee9-7fb4-47c2-92b7-80d5cbff46fb",
 "ip": "10.195.210.93",
 "listening_port": 4000,
 "status_port": 10080,
 "lease": "45s",
 "binlog_status": "Off",
 "start_timestamp": 1619338880
}{
 "is_owner": false,
 "max_procs": 16,
 "gogc": 100,
 "version": "5.7.25-TiDB-v4.0.12",
 "git_hash": "11a9254bec522c1991033a6098f6dccadc6eb7c1",
 "ddl_id": "7efd6f51-c1ba-4b2c-94dd-5e64b868e4ba",
 "ip": "10.206.61.166",
 "listening_port": 4000,
 "status_port": 10080,
 "lease": "45s",
 "binlog_status": "Off",
 "start_timestamp": 1619338853
}{
 "is_owner": false,
 "max_procs": 12,
 "gogc": 100,
 "version": "5.7.25-TiDB-v4.0.12",
 "git_hash": "11a9254bec522c1991033a6098f6dccadc6eb7c1",
 "ddl_id": "4734db9d-50a4-4515-96ea-04af9d58d8d9",
 "ip": "10.206.82.10",
 "listening_port": 4000,
 "status_port": 10080,
 "lease": "45s",
 "binlog_status": "Off",
 "start_timestamp": 1619338802
}{
 "is_owner": false,
 "max_procs": 12,
 "gogc": 100,
 "version": "5.7.25-TiDB-v4.0.12",
 "git_hash": "11a9254bec522c1991033a6098f6dccadc6eb7c1",
 "ddl_id": "eb86cb2f-45f0-4f67-a3ba-cde8a19fb786",
 "ip": "10.218.10.221",
 "listening_port": 4000,
 "status_port": 10080,
 "lease": "45s",
 "binlog_status": "Off",
 "start_timestamp": 1619338827
}{
 "is_owner": false,
 "max_procs": 16,
 "gogc": 100,
 "version": "5.7.25-TiDB-v4.0.12",
 "git_hash": "11a9254bec522c1991033a6098f6dccadc6eb7c1",
 "ddl_id": "6c0ddb86-d0eb-4744-93c5-7133caeb6605",
 "ip": "10.218.2.123",
 "listening_port": 4000,
 "status_port": 10080,
 "lease": "45s",
 "binlog_status": "Off",
 "start_timestamp": 1619338864
}{
 "is_owner": false,
 "max_procs": 16,
 "gogc": 100,
 "version": "5.7.25-TiDB-v4.0.12",
 "git_hash": "11a9254bec522c1991033a6098f6dccadc6eb7c1",
 "ddl_id": "ab5abae1-4cdf-4e1c-86a5-6a964786dfb9",
 "ip": "10.218.2.124",
 "listening_port": 4000,
 "status_port": 10080,
 "lease": "45s",
 "binlog_status": "Off",
 "start_timestamp": 1619338875
}{
 "is_owner": false,
 "max_procs": 16,
 "gogc": 100,
 "version": "5.7.25-TiDB-v4.0.12",
 "git_hash": "11a9254bec522c1991033a6098f6dccadc6eb7c1",
 "ddl_id": "bdff573a-b9b9-4d7c-9518-4377fef57921",
 "ip": "10.218.2.125",
 "listening_port": 4000,
 "status_port": 10080,
 "lease": "45s",
 "binlog_status": "Off",
 "start_timestamp": 1619338893
}{
 "is_owner": false,
 "max_procs": 12,
 "gogc": 100,
 "version": "5.7.25-TiDB-v4.0.12",
 "git_hash": "11a9254bec522c1991033a6098f6dccadc6eb7c1",
 "ddl_id": "8d2347be-f424-466e-af7d-3876b3f04f76",
 "ip": "10.218.22.106",
 "listening_port": 4000,
 "status_port": 10080,
 "lease": "45s",
 "binlog_status": "Off",
 "start_timestamp": 1619338837
}{
 "is_owner": false,
 "max_procs": 12,
 "gogc": 100,
 "version": "5.7.25-TiDB-v4.0.12",
 "git_hash": "11a9254bec522c1991033a6098f6dccadc6eb7c1",
 "ddl_id": "52edfe1e-565b-49a9-871f-36aa0fdb977f",
 "ip": "10.218.22.107",
 "listening_port": 4000,
 "status_port": 10080,
 "lease": "45s",
 "binlog_status": "Off",
 "start_timestamp": 1619338889
}{
 "is_owner": false,
 "max_procs": 12,
 "gogc": 100,
 "version": "5.7.25-TiDB-v4.0.12",
 "git_hash": "11a9254bec522c1991033a6098f6dccadc6eb7c1",
 "ddl_id": "2564bcb6-83fc-4ced-a15a-83e78affdfbd",
 "ip": "10.218.69.67",
 "listening_port": 4000,
 "status_port": 10080,
 "lease": "45s",
 "binlog_status": "Off",
 "start_timestamp": 1619338850
}{
 "is_owner": false,
 "max_procs": 12,
 "gogc": 100,
 "version": "5.7.25-TiDB-v4.0.12",
 "git_hash": "11a9254bec522c1991033a6098f6dccadc6eb7c1",
 "ddl_id": "dacbe784-b428-4c3a-ae23-c69c0a2e1324",
 "ip": "10.218.73.147",
 "listening_port": 4000,
 "status_port": 10080,
 "lease": "45s",
 "binlog_status": "Off",
 "start_timestamp": 1619338824
}{
 "is_owner": false,
 "max_procs": 12,
 "gogc": 100,
 "version": "5.7.25-TiDB-v4.0.12",
 "git_hash": "11a9254bec522c1991033a6098f6dccadc6eb7c1",
 "ddl_id": "b1ccadf9-2e82-4332-874a-cdd9ac3a96d8",
 "ip": "10.218.77.32",
 "listening_port": 4000,
 "status_port": 10080,
 "lease": "45s",
 "binlog_status": "Off",
 "start_timestamp": 1619338833
}{
 "is_owner": false,
 "max_procs": 12,
 "gogc": 100,
 "version": "5.7.25-TiDB-v4.0.12",
 "git_hash": "11a9254bec522c1991033a6098f6dccadc6eb7c1",
 "ddl_id": "42b7dfd9-a821-4b18-80e6-e5b8f60dcb7e",
 "ip": "10.218.82.80",
 "listening_port": 4000,
 "status_port": 10080,
 "lease": "45s",
 "binlog_status": "Off",
 "start_timestamp": 1619338856
}{
 "is_owner": false,
 "max_procs": 12,
 "gogc": 100,
 "version": "5.7.25-TiDB-v4.0.12",
 "git_hash": "11a9254bec522c1991033a6098f6dccadc6eb7c1",
 "ddl_id": "f80a8239-d219-4d76-8788-47b408d166d8",
 "ip": "10.72.36.120",
 "listening_port": 4000,
 "status_port": 10080,
 "lease": "45s",
 "binlog_status": "Off",
 "start_timestamp": 1619338886
}{
 "is_owner": false,
 "max_procs": 12,
 "gogc": 100,
 "version": "5.7.25-TiDB-v4.0.12",
 "git_hash": "11a9254bec522c1991033a6098f6dccadc6eb7c1",
 "ddl_id": "424770ec-9106-44aa-bca1-182d4507e53a",
 "ip": "10.72.65.92",
 "listening_port": 4000,
 "status_port": 10080,
 "lease": "45s",
 "binlog_status": "Off",
 "start_timestamp": 1619338830
}

另外重新试了一下reload,tiup返回Reloaded cluster tidb1 successfully, 但tidb.toml的配置依然是旧的没刷新

对于导入的集群,删配置是不行的,只能增加配置或修改导入的配置,若删配置,则会回退成导入时的配置(而不是新部署的集群的默认配置)

看看选一台 TiDB 执行 SQL 语句 admin show ddl 的结果呢

请看第一张图

这里是指 admin show ddl, 不是 admin show ddl jobs, 麻烦请再贴一下结果。

意思是说config里面的配置项不能删,如果要改成4.0默认值的话得把这个配置项设置成默认值?这是只针对ansible导入集群会这样,还是新建集群也是这样?

只针对导入的集群会这样,这个是做成 feature 的,导入的集群会在 ~/.tiup/storage/cluster/clusters//ansible-imported-configs 文件夹里面存放导入时候的配置作为默认值,如果不想要这样的行为,把这个文件夹下面的配置文件删了就好了

1赞

破案了,ddl owner在一台之前被remove的机器上,这台机器之前因为硬件问题被挪掉了,所以inventory内已经注掉了,导入tiup后这台机器不在display内,但这台机器上的tidb可能是ansible stop没有执行成功,或者之后又因为某些原因启动了,所以ddl走这个台机器之后报错

mysql> admin show ddl;
+------------+--------------------------------------+------------------+--------------+--------------------------------------+-------+
| SCHEMA_VER | OWNER_ID                             | OWNER_ADDRESS    | RUNNING_JOBS | SELF_ID                              | QUERY |
+------------+--------------------------------------+------------------+--------------+--------------------------------------+-------+
|       1225 | 49aa7631-9db1-477a-8621-4a31acb6c8ea | 10.206.82.9:4000 |              | 6c0ddb86-d0eb-4744-93c5-7133caeb6605 |       |
+------------+--------------------------------------+------------------+--------------+--------------------------------------+-------+
1 row in set (0.00 sec)

现在把这个tidb进程kill掉之后,owner切换到一台在tiup cluster display内的机器,可以SET TIFLASH REPLICA了,非常感谢support。
另外想问一下ddl owner这块设计有什么资料文档吗,之前没注意过这块

感谢反馈。

这里有一篇文章介绍 TiDB 的 DDL 原理:【SOP 系列 03】在线表结构变更(Online DDL)

1赞
  1. 我看了一下~/.tiup/storage/cluster/clusters//ansible-imported-configs下面时ansible导入的每台机器的config toml。假设我扩容集群,比如加了一个tidb节点,那么会不会出现新的tidb节点上的config从cluster edit-config里来,而老的tidb节点config从ansible里来,两者不一致?

  2. 再问一个不太相关的问题,现在tiup的所有操作(包括list, cluster, …)都要走http和https proxy外网代理之后操作,还要把内网机器加在no proxy里面,非常麻烦,如果不走代理的话tiup所有操作都会卡住,估计在尝试连pingcap.com。请问有什么办法能不连外网情况下也能操作tiup吗