Commit Graph

416 Commits

Author SHA1 Message Date
alexey-milovidov
4f9df21d3e
Merge pull request #10373 from azat/dist-SELECT-optimization
Optimize queries with LIMIT/LIMIT BY/ORDER BY for distributed with GROUP BY sharding_key
2020-09-04 01:38:56 +03:00
Azat Khuzhin
fffeeeba06 Force WithMergeableStateAfterAggregation via distributed_group_by_no_merge (convert to UInt64)
Possible values:
- 1 - Do not merge aggregation states from different servers for distributed query processing - in case it is for certain that there are different keys on different shards.
- 2 - same as 1 but also apply ORDER BY and LIMIT stages
2020-09-03 00:52:51 +03:00
Azat Khuzhin
4043be3121 Add new query processing stage WithMergeableStateAfterAggregation
Process query until the stage where the aggregate functions were
calculated and finalized.

It will be used for optimize_distributed_group_by_sharding_key.

v2: fix aliases
v3: Fix protocol ABI breakage due to WithMergeableStateAfterAggregation
    Conditions >= for QueryProcessingStage::Enum has been verified, and they
    are ok (in InterpreterSelectQuery).
2020-09-03 00:51:08 +03:00
BohuTANG
6dfab88156 ISSUES-14235 change string.rfind to string starts_with and add some tests 2020-09-02 08:31:51 +08:00
Alexander Kuzmenkov
fa04b39d32
Revert "Change query event filter and add integration test for empty GTID transaction" 2020-09-02 02:06:53 +03:00
Alexander Kuzmenkov
949a41f8b5
Merge pull request #14321 from BohuTANG/materialize_mysql_query_filter
Change query event filter and add integration test for empty GTID transaction
2020-09-01 17:44:58 +03:00
Artem Zuikov
d304d522cc
Refactoring: rename types to allow change bigint library (#14219) 2020-09-01 12:54:50 +03:00
BohuTANG
453fb837d8 ISSUES-14235 change string.rfind to stringstarts_with 2020-09-01 16:25:15 +08:00
BohuTANG
4d4b4b9532 Fix MaterializeMySQL empty GTID transaction issue #14235 2020-08-30 15:55:34 +08:00
Alexander Kuzmenkov
c4fc434a13
Merge pull request #14060 from azat/parallel_distributed_insert_select-2
Extend parallel_distributed_insert_select to run INSERT into local table
2020-08-26 20:37:36 +03:00
tavplubix
cff92c8ad3
Merge pull request #13820 from BohuTANG/mysql_replica_gtid_issue_4006
ISSUES-4006 support MySQL GTID based replication #4006
2020-08-26 15:02:53 +03:00
BohuTANG
e3bfd1a5b8 ISSUES-4006 fix clang-tidy error on boost::is_any_of 2020-08-26 14:33:05 +08:00
BohuTANG
3510afd149 ISSUES-4006 change UUID from MySQL big-endian to ClickHouse little-endian 2020-08-26 10:39:07 +08:00
BohuTANG
81e679dc22 ISSUES-4006 support gtid sets with whitespace 2020-08-26 09:04:33 +08:00
Azat Khuzhin
50a312534c Extend parallel_distributed_insert_select to run INSERT into local table
Before this patch there was:

- parallel_distributed_insert_select=1, that executes:

    INSERT INTO dist_out SELECT ... FROM underlying_dist_in

After this patch there will be:

- parallel_distributed_insert_select=2, that executes:

    INSERT INTO underlying_dist_out SELECT ... FROM underlying_dist_in

And cover the behaviour w/o integration test, by using the following
techincs:
- SYSTEM STOP DISTRIBUTED SENDS
- prefer_localhost_replica=0
2020-08-25 22:49:13 +03:00
BohuTANG
d01f4d38eb ISSUES-4006 fix fast build warning and remove binlog file name 2020-08-25 17:44:23 +08:00
BohuTANG
d8e88499a5 ISSUES-4006 remove old BinlogDump style 2020-08-25 16:01:51 +08:00
BohuTANG
80af056cb4 ISSUES-4006 change the error code from UNKNOWN_EXCEPTION to LOGICAL_ERROR 2020-08-25 15:49:47 +08:00
BohuTANG
0876ab63ec ISSUES-4006 change to UUID type 2020-08-25 15:44:38 +08:00
Alexey Milovidov
dfe870e38c Merge branch 'new-branch' of https://github.com/ucasFL/ClickHouse into ucasFL-new-branch 2020-08-24 16:26:08 +03:00
Artem Zuikov
becc186c91
Add support for extended precision integers and decimals (#13097) 2020-08-19 14:52:17 +03:00
BohuTANG
26525a5eb1 Merge remote-tracking branch 'origin/master' into mysql_replica_gtid_issue_4006 2020-08-19 08:04:02 +08:00
BohuTANG
30b8159bb0 ISSUES-4006 fix empty gtid format when sets parse 2020-08-18 23:01:21 +08:00
Alexey Milovidov
9615b1f06a Generate ya.make 2020-08-18 12:46:02 +03:00
BohuTANG
a78604db2b ISSUES-4006 add constructor to initialize number member 2020-08-18 11:56:40 +08:00
BohuTANG
34975d1dd3 ISSUES-4006 try to fix PVS check error 2020-08-18 08:56:18 +08:00
BohuTANG
8c9236fefd ISSUES-4006 adapted GTID replication to Materialize MySQL 2020-08-17 22:12:41 +08:00
BohuTANG
c3561cd7e8 ISSUES-4006 add BIT/SET field type support 2020-08-17 16:30:13 +08:00
BohuTANG
ee6ac28773 ISSUES-4006 fix up meta parse 2020-08-17 16:27:34 +08:00
BohuTANG
6557e706c9 ISSUES-4006 fix special build check errors #4006 2020-08-17 16:01:18 +08:00
BohuTANG
a8465cfce6 ISSUES-4006 move MySQL command to the PacketsGeneric.h #4006 2020-08-17 15:27:35 +08:00
BohuTANG
cfc1fea619 ISSUES-4006 add startBinlogDumpGTID api for MySQLClient #4006 2020-08-17 15:21:20 +08:00
BohuTANG
059bf166ae ISSUES-4006 add GTID unit tests #4006 2020-08-17 14:45:43 +08:00
BohuTANG
3ccc650995 ISSUES-4006 add position.Dump() #4006 2020-08-17 09:30:16 +08:00
BohuTANG
e90679707a ISSUES-4006 add MySQL replication GTIDEvent #4006 2020-08-17 09:23:54 +08:00
BohuTANG
b4d18b1a52 ISSUES-4006 add MySQL replication GTID module #4006 2020-08-17 09:02:59 +08:00
Vitaly Baranov
2909ed1bc0 Better initialization of access storages. Make list of access storages dynamic. 2020-08-16 19:15:34 +03:00
feng lv
aaedf02d2b fix conflict 2020-08-16 12:38:39 +08:00
zhang2014
fe49bd3414 Merge branch 'master' of github.com:yandex/ClickHouse into refactor/materialize_mysql_database 2020-08-15 16:12:46 +08:00
alexey-milovidov
ea79c53ed2
Merge pull request #13735 from azat/gcc10-build-fixes-20.7
Fix compilation with gcc10 and libstdc++
2020-08-15 11:01:14 +03:00
zhang2014
2f3042ba86 ISSUES-4006 try fix build 2020-08-15 11:16:55 +08:00
Azat Khuzhin
194d1d514c Fix compilation MySQLReplication with gcc10 and libstdc++
gcc10 reports:
    ../src/Core/MySQLReplication.cpp:62:46: error: use of deleted function ‘std::basic_ostream<char, _Traits>& std::operator<<(std::basic_ostream<char, _Traits>&, char8_t) [with _Traits = std::char_traits<char>]’
2020-08-14 22:15:17 +03:00
zhang2014
b49c3cebb5 ISSUES-4006 try fix build 2020-08-14 23:52:47 +08:00
Alexander Tokmakov
128b19fdba Merge branch 'master' into use_atomic_engine_for_system_db 2020-08-14 13:47:44 +03:00
zhang2014
4199baf69d Merge branch 'master' of github.com:yandex/ClickHouse into refactor/materialize_mysql_database 2020-08-14 13:30:12 +08:00
zhang2014
7ff62b090e ISSUES-4006 try fix build and style failure 2020-08-14 09:59:59 +08:00
alexey-milovidov
eeefcd6493
Merge pull request #13337 from ClickHouse/refactor-pipes-3
Refactor Pipe
2020-08-14 04:12:22 +03:00
zhang2014
e3b78c5a31 ISSUES-4006 try fix build and style failure 2020-08-14 00:51:51 +08:00
zhang2014
57bf007b82 ISSUES-4006 add ya make 2020-08-13 20:51:29 +08:00
zhang2014
96bd3ac34b ISSUES-4006 split mysql protocol 2020-08-13 20:41:36 +08:00