【 TiDB 使用环境】生产环境
【 TiDB 版本】7.1.0
【复现路径】两张表，一张稍大表：etldr2125.tbl_adt_admits_discharges（大概400万） 一张小表 etldr21252023_08_14.pid_table（100万，只有一列id）。
1、在mysql客户端运行：
select tbl_adt_admits_discharges.* from etldr2125.tbl_adt_admits_discharges inner join etldr21252023_08_14.pid_table on tbl_adt_admits_discharges.patient_id = etldr21252023_08_14.pid_table.patient_id where etldr2125.tbl_adt_admits_discharges.yidu_t1_removed = ‘0’ or (etldr2125.tbl_adt_admits_discharges.yidu_t1_removed is null and etldr21252023_08_14.pid_table.patient_id=‘xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx’);
运行20秒左右，有两条完全一样的结果。
执行计划：
MySQL [hn3yjcyj]> explain analyze select tbl_adt_admits_discharges.* from etldr2125.tbl_adt_admits_discharges join etldr21252023_08_14.pid_table on tbl_adt_admits_discharges.patient_id = etldr21252023_08_14.pid_table.patient_id where etldr2125.tbl_adt_admits_discharges.yidu_t1_removed = '0' or (etldr2125.tbl_adt_admits_discharges.yidu_t1_removed is null and etldr21252023_08_14.pid_table.patient_id='xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx');
-------------------------------+------------+---------+-----------+---------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+------+
| id | estRows | actRows | task | access object | execution info | operator info | memory | disk |
+-------------------------------+------------+---------+-----------+---------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+------+
| IndexHashJoin_18 | 1560616.41 | 2 | root | | time:41s, loops:2, RU:62084.501854, inner:{total:3m20.4s, concurrency:5, task:51, construct:1.36s, fetch:3m16.8s, build:423.4ms, join:2.19s} | inner join, inner:IndexLookUp_15, outer key:etldr21252023_08_14.pid_table.patient_id, inner key:etldr2125.tbl_adt_admits_discharges.patient_id, equal cond:eq(etldr21252023_08_14.pid_table.patient_id, etldr2125.tbl_adt_admits_discharges.patient_id), other cond:or(eq(etldr2125.tbl_adt_admits_discharges.yidu_t1_removed, "0"), and(isnull(etldr2125.tbl_adt_admits_discharges.yidu_t1_removed), eq(etldr21252023_08_14.pid_table.patient_id, "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"))) | 143.7 MB | N/A |
| ├─IndexReader_29(Build) | 1093867.00 | 1093867 | root | | time:33ms, loops:1079, cop_task: {num: 39, max: 128.5ms, min: 434.6µs, avg: 5.7ms, p95: 28.5ms, max_proc_keys: 992, p95_proc_keys: 992, tot_proc: 5.7ms, tot_wait: 98.7ms, rpc_num: 39, rpc_time: 214.2ms, copr_cache_hit_ratio: 0.82, build_task_duration: 12.5ms, max_distsql_concurrency: 3} | index:IndexFullScan_28 | 3.74 MB | N/A |
| │ └─IndexFullScan_28 | 1093867.00 | 1093867 | cop[tikv] | table:pid_table, index:pid_table_pid(patient_id) | tikv_task:{proc max:50ms, min:0s, avg: 24.9ms, p80:50ms, p95:50ms, iters:1220, tasks:39}, scan_detail: {total_process_keys: 3392, total_process_keys_size: 467114, total_keys: 3399, get_snapshot_time: 96.5ms, rocksdb: {key_skipped_count: 3392, block: {cache_hit_count: 194, read_count: 8, read_byte: 175.3 KB, read_time: 117.6µs}}} | keep order:false | N/A | N/A |
| └─IndexLookUp_15(Probe) | 1560616.41 | 410583 | root | | time:3m14.9s, loops:478, index_task: {total_time: 2m36.8s, fetch_handle: 2m36.8s, build: 401.2µs, wait: 1.28ms}, table_task: {total_time: 49s, num: 135, concurrency: 5}, next: {wait_index: 2m12.6s, wait_table_lookup_build: 299.5ms, wait_table_lookup_resp: 42.1s} | | 89.8 KB | N/A |
| ├─Selection_13(Build) | 1560616.41 | 410583 | cop[tikv] | | time:2m55.6s, loops:538, cop_task: {num: 284, max: 7.27s, min: 1.38ms, avg: 636.5ms, p95: 2.86s, max_proc_keys: 17376, p95_proc_keys: 5088, tot_proc: 2m12s, tot_wait: 444.5ms, rpc_num: 284, rpc_time: 3m0.8s, copr_cache_hit_ratio: 0.01, build_task_duration: 120ms, max_distsql_concurrency: 3}, tikv_task:{proc max:7.17s, min:0s, avg: 600.2ms, p80:760ms, p95:2.82s, iters:1373, tasks:284}, scan_detail: {total_process_keys: 410015, total_process_keys_size: 48151015, total_keys: 1516310, get_snapshot_time: 426.2ms, rocksdb: {delete_skipped_count: 20, key_skipped_count: 423477, block: {cache_hit_count: 32121244, read_count: 383, read_byte: 4.82 MB, read_time: 62.2ms}}} | not(isnull(etldr2125.tbl_adt_admits_discharges.patient_id)) | N/A | N/A |
| │ └─IndexRangeScan_11 | 1561716.87 | 410583 | cop[tikv] | table:tbl_adt_admits_discharges, index:tbl_adt_admits_discharges_76d1d3337584a6cae15cd61503cba6c3(patient_id) | tikv_task:{proc max:7.17s, min:0s, avg: 599.9ms, p80:760ms, p95:2.82s, iters:1373, tasks:284} | range: decided by [eq(etldr2125.tbl_adt_admits_discharges.patient_id, etldr21252023_08_14.pid_table.patient_id)], keep order:false | N/A | N/A |
| └─Selection_14(Probe) | 1560616.41 | 410583 | cop[tikv] | | time:48.6s, loops:607, cop_task: {num: 239, max: 2.51s, min: 0s, avg: 242.4ms, p95: 964ms, max_proc_keys: 16145, p95_proc_keys: 7861, tot_proc: 44.5s, tot_wait: 236.1ms, rpc_num: 238, rpc_time: 57.9s, copr_cache_hit_ratio: 0.03, build_task_duration: 43.1ms, max_distsql_concurrency: 4, max_extra_concurrency: 1, store_batch_num: 1}, tikv_task:{proc max:2.43s, min:0s, avg: 220.9ms, p80:340ms, p95:920ms, iters:1294, tasks:239}, scan_detail: {total_process_keys: 409737, total_process_keys_size: 156453353, total_keys: 844528, get_snapshot_time: 222.6ms, rocksdb: {delete_skipped_count: 14, key_skipped_count: 435118, block: {cache_hit_count: 11360728, read_count: 250, read_byte: 4.19 MB, read_time: 11.9ms}}} | or(eq(etldr2125.tbl_adt_admits_discharges.yidu_t1_removed, "0"), isnull(etldr2125.tbl_adt_admits_discharges.yidu_t1_removed)) | N/A | N/A |
| └─TableRowIDScan_12 | 1560616.41 | 410583 | cop[tikv] | table:tbl_adt_admits_discharges | tikv_task:{proc max:2.43s, min:0s, avg: 220.7ms, p80:340ms, p95:920ms, iters:1294, tasks:239} | keep order:false | N/A | N/A |
+-------------------------------+------------+---------+-----------+---------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+------+
8 rows in set (40.976 sec)
2、去掉 第一个or 条件：etldr2125.tbl_adt_admits_discharges.yidu_t1_removed = ‘0’ 。
在mysql 客户端运行：
select tbl_adt_admits_discharges.* from etldr2125.tbl_adt_admits_discharges inner join etldr21252023_08_14.pid_table on tbl_adt_admits_discharges.patient_id = etldr21252023_08_14.pid_table.patient_id where (etldr2125.tbl_adt_admits_discharges.yidu_t1_removed is null and etldr21252023_08_14.pid_table.patient_id=‘xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx’);
只有一条，并且秒出结果。
执行计划：
+---------------------------------+---------+---------+-----------+---------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------+------+
| id | estRows | actRows | task | access object | execution info | operator info | memory | disk |
+---------------------------------+---------+---------+-----------+---------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------+------+
| Projection_10 | 1.43 | 1 | root | | time:4.08ms, loops:2, RU:1.009824, Concurrency:OFF | etldr2125.tbl_adt_admits_discharges.inpatient_visit_id, etldr2125.tbl_adt_admits_discharges.patient_id, etldr2125.tbl_adt_admits_discharges.dom_hospital_no, etldr2125.tbl_adt_admits_discharges.inpatient_no, etldr2125.tbl_adt_admits_discharges.inpatient_case_no, etldr2125.tbl_adt_admits_discharges.inpatient_visit_count, etldr2125.tbl_adt_admits_discharges.cdt_admit_department, etldr2125.tbl_adt_admits_discharges.cdt_admit_dept_speciality, etldr2125.tbl_adt_admits_discharges.rstd_admit_dept_name, etldr2125.tbl_adt_admits_discharges.rstd_admit_speciality, etldr2125.tbl_adt_admits_discharges.cdt_admit_ward, etldr2125.tbl_adt_admits_discharges.cdt_discharge_department, etldr2125.tbl_adt_admits_discharges.cdt_discharge_dept_speciality, etldr2125.tbl_adt_admits_discharges.rstd_discharge_dept_name, etldr2125.tbl_adt_admits_discharges.rstd_discharge_speciality, etldr2125.tbl_adt_admits_discharges.cdt_discharge_ward, etldr2125.tbl_adt_admits_discharges.admit_datetime, etldr2125.tbl_adt_admits_discharges.discharge_datetime, etldr2125.tbl_adt_admits_discharges.patient_age, etldr2125.tbl_adt_admits_discharges.cdt_fee_type, etldr2125.tbl_adt_admits_discharges.cdt_visit_type, etldr2125.tbl_adt_admits_discharges.cdt_attending_physician, etldr2125.tbl_adt_admits_discharges.cdt_attending_physician_id, etldr2125.tbl_adt_admits_discharges.primary_respon_doctor_name, etldr2125.tbl_adt_admits_discharges.primary_respon_doctor_id, etldr2125.tbl_adt_admits_discharges.cdt_adt_status, etldr2125.tbl_adt_admits_discharges.birth_date, etldr2125.tbl_adt_admits_discharges.admitted_time, etldr2125.tbl_adt_admits_discharges.yidu_mapping_source, etldr2125.tbl_adt_admits_discharges.yidu_adt_status, etldr2125.tbl_adt_admits_discharges.yidu_visit_type, etldr2125.tbl_adt_admits_discharges.yidu_data_credibility, etldr2125.tbl_adt_admits_discharges.admit_dept_code, etldr2125.tbl_adt_admits_discharges.rstd_admit_dept_code, etldr2125.tbl_adt_admits_discharges.admit_ward_code, etldr2125.tbl_adt_admits_discharges.discharge_dept_code, etldr2125.tbl_adt_admits_discharges.rstd_discharge_dept_code, etldr2125.tbl_adt_admits_discharges.discharge_ward_code, etldr2125.tbl_adt_admits_discharges.weight, etldr2125.tbl_adt_admits_discharges.special_flag, etldr2125.tbl_adt_admits_discharges.yidu_mapping_date, etldr2125.tbl_adt_admits_discharges.yidu_remarks, etldr2125.tbl_adt_admits_discharges.etldr_rowkey, etldr2125.tbl_adt_admits_discharges.yidu_t1_update_time, etldr2125.tbl_adt_admits_discharges.yidu_t1_removed, etldr21252023_08_14.pid_table.patient_id | 676.9 KB | N/A |
| └─IndexHashJoin_18 | 1.43 | 1 | root | | time:4.04ms, loops:2, inner:{total:2.97ms, concurrency:5, task:1, construct:25.9µs, fetch:2.92ms, build:24.6µs, join:13µs} | inner join, inner:IndexLookUp_15, outer key:etldr21252023_08_14.pid_table.patient_id, inner key:etldr2125.tbl_adt_admits_discharges.patient_id, equal cond:eq(etldr21252023_08_14.pid_table.patient_id, etldr2125.tbl_adt_admits_discharges.patient_id) | 714.5 KB | N/A |
| ├─IndexReader_29(Build) | 1.01 | 1 | root | | time:902.5µs, loops:3, cop_task: {num: 1, max: 1.03ms, proc_keys: 1, tot_proc: 132.3µs, tot_wait: 82µs, rpc_num: 1, rpc_time: 992µs, copr_cache_hit_ratio: 0.00, build_task_duration: 14µs, max_distsql_concurrency: 1} | index:IndexRangeScan_28 | 359 Bytes | N/A |
| │ └─IndexRangeScan_28 | 1.01 | 1 | cop[tikv] | table:pid_table, index:pid_table_pid(patient_id) | tikv_task:{time:0s, loops:1}, scan_detail: {total_process_keys: 1, total_process_keys_size: 157, total_keys: 2, get_snapshot_time: 28.8µs, rocksdb: {key_skipped_count: 1, block: {cache_hit_count: 8}}} | range:["xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx","xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"], keep order:false | N/A | N/A |
| └─IndexLookUp_15(Probe) | 1.43 | 1 | root | | time:2.82ms, loops:2, index_task: {total_time: 1.21ms, fetch_handle: 1.2ms, build: 1.73µs, wait: 1.98µs}, table_task: {total_time: 1.42ms, num: 1, concurrency: 5}, next: {wait_index: 1.36ms, wait_table_lookup_build: 71.9µs, wait_table_lookup_resp: 1.34ms} | | 71.2 KB | N/A |
| ├─Selection_13(Build) | 1.43 | 1 | cop[tikv] | | time:1.19ms, loops:3, cop_task: {num: 1, max: 1.11ms, proc_keys: 1, tot_proc: 254.1µs, tot_wait: 111.8µs, rpc_num: 1, rpc_time: 1.09ms, copr_cache_hit_ratio: 0.00, build_task_duration: 43.3µs, max_distsql_concurrency: 1}, tikv_task:{time:0s, loops:1}, scan_detail: {total_process_keys: 1, total_process_keys_size: 111, total_keys: 2, get_snapshot_time: 46.1µs, rocksdb: {key_skipped_count: 1, block: {cache_hit_count: 9}}} | not(isnull(etldr2125.tbl_adt_admits_discharges.patient_id)) | N/A | N/A |
| │ └─IndexRangeScan_11 | 1.43 | 1 | cop[tikv] | table:tbl_adt_admits_discharges, index:tbl_adt_admits_discharges_76d1d3337584a6cae15cd61503cba6c3(patient_id) | tikv_task:{time:0s, loops:1} | range: decided by [eq(etldr2125.tbl_adt_admits_discharges.patient_id, etldr21252023_08_14.pid_table.patient_id)], keep order:false | N/A | N/A |
| └─Selection_14(Probe) | 1.43 | 1 | cop[tikv] | | time:1.26ms, loops:2, cop_task: {num: 1, max: 1.17ms, proc_keys: 1, tot_proc: 365.3µs, tot_wait: 94.5µs, rpc_num: 1, rpc_time: 1.14ms, copr_cache_hit_ratio: 0.00, build_task_duration: 16.9µs, max_distsql_concurrency: 1, max_extra_concurrency: 1}, tikv_task:{time:0s, loops:1}, scan_detail: {total_process_keys: 1, total_process_keys_size: 340, total_keys: 2, get_snapshot_time: 43.2µs, rocksdb: {key_skipped_count: 1, block: {cache_hit_count: 18}}} | isnull(etldr2125.tbl_adt_admits_discharges.yidu_t1_removed) | N/A | N/A |
| └─TableRowIDScan_12 | 1.43 | 1 | cop[tikv] | table:tbl_adt_admits_discharges | tikv_task:{time:0s, loops:1} | keep order:false | N/A | N/A |
+---------------------------------+---------+---------+-----------+---------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------+------+
9 rows in set (0.007 sec)
3、去掉(etldr2125.tbl_adt_admits_discharges.yidu_t1_removed is null and etldr21252023_08_14.pid_table.patient_id=‘pidxxxxxx’)执行
select tbl_adt_admits_discharges.* from etldr2125.tbl_adt_admits_discharges inner join etldr21252023_08_14.pid_table on tbl_adt_admits_discharges.patient_id = etldr21252023_08_14.pid_table.patient_id where etldr2125.tbl_adt_admits_discharges.yidu_t1_removed = ‘0’ ;
0条数据
4、单独查询 pid_table 和tbl_adt_admits_discharges where条件patient_id=‘pidxxxxxx’ 时，都只有一条数据。
5、用tispark 查询1中的语句，为一条数据，符合预期
scala> spark.sql("select tbl_adt_admits_discharges.* from tidb_catalog.etldr2125.tbl_adt_admits_discharges join tidb_catalog.etldr21252023_08_14.pid_table on tbl_adt_admits_discharges.patient_id = etldr21252023_08_14.pid_table.patient_id where tidb_catalog.etldr2125.tbl_adt_admits_discharges.yidu_t1_removed = '0' or (tidb_catalog.etldr2125.tbl_adt_admits_discharges.yidu_t1_removed is null and tidb_catalog.etldr21252023_08_14.pid_table.patient_id='xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx')").show()
+--------------------+--------------------+---------------+----------------+-----------------+---------------------+--------------------+-------------------------+--------------------+---------------------+--------------+------------------------+-----------------------------+------------------------+-------------------------+------------------+-------------------+-------------------+-----------+------------+--------------+-----------------------+--------------------------+--------------------------+------------------------+--------------+-------------------+-------------+-------------------+---------------+---------------+---------------------+---------------+--------------------+---------------+-------------------+------------------------+-------------------+------+------------+-------------------+------------+------------+-------------------+---------------+
| inpatient_visit_id| patient_id|dom_hospital_no| inpatient_no|inpatient_case_no|inpatient_visit_count|cdt_admit_department|cdt_admit_dept_speciality|rstd_admit_dept_name|rstd_admit_speciality|cdt_admit_ward|cdt_discharge_department|cdt_discharge_dept_speciality|rstd_discharge_dept_name|rstd_discharge_speciality|cdt_discharge_ward| admit_datetime| discharge_datetime|patient_age|cdt_fee_type|cdt_visit_type|cdt_attending_physician|cdt_attending_physician_id|primary_respon_doctor_name|primary_respon_doctor_id|cdt_adt_status| birth_date|admitted_time|yidu_mapping_source|yidu_adt_status|yidu_visit_type|yidu_data_credibility|admit_dept_code|rstd_admit_dept_code|admit_ward_code|discharge_dept_code|rstd_discharge_dept_code|discharge_ward_code|weight|special_flag| yidu_mapping_date|yidu_remarks|etldr_rowkey|yidu_t1_update_time|yidu_t1_removed|
+--------------------+--------------------+---------------+----------------+-----------------+---------------------+--------------------+-------------------------+--------------------+---------------------+--------------+------------------------+-----------------------------+------------------------+-------------------------+------------------+-------------------+-------------------+-----------+------------+--------------+-----------------------+--------------------------+--------------------------+------------------------+--------------+-------------------+-------------+-------------------+---------------+---------------+---------------------+---------------+--------------------+---------------+-------------------+------------------------+-------------------+------+------------+-------------------+------------+------------+-------------------+---------------+
|xxxx...|xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| 11|11|11| 11|11| 11| 11| 11| null| null| 11| 11| 1111 null| null| 1|2023-08-11 16:04:00|2023-08-14 14:30:01| null| null| 11| null| null| null| null| 11|2015-01-10 00:00:00| null| null| null| null| null| null| null| null| null| null| null| null| null|2023-08-18 07:03:41| null| null|2023-08-19 00:59:38| null|
+--------------------+--------------------+---------------+----------------+-----------------+---------------------+--------------------+-------------------------+--------------------+---------------------+--------------+------------------------+-----------------------------+------------------------+-------------------------+------------------+-------------------+-------------------+-----------+------------+--------------+-----------------------+--------------------------+--------------------------+------------------------+--------------+-------------------+-------------+-------------------+---------------+---------------+---------------------+---------------+--------------------+---------------+-------------------+------------------------+-------------------+------+------------+-------------------+------------+------------+-------------------+---------------+
【遇到的问题：问题现象及影响】根据2,3的结果，1应该只有一条数据，但是确出现了两条。非常费解
同样的sql，用mysql客户端和用spark-shell执行结果也不一样。