Commit Graph

20537 Commits

Author SHA1 Message Date
BohuTANG
691fe42586 Merge remote-tracking branch 'ck/master' into 20200409_bug_fix_mysql_handshake_scramble 2020-04-09 10:59:04 +08:00
alexey-milovidov
6ff4c3aa16
Merge pull request #10135 from azat/distributed-insert-converting
Convert blocks if structure does not match on INSERT into Distributed()
2020-04-09 05:51:36 +03:00
BohuTANG
e544edd726 Fix random scramble using seperator character issue during MySQL handshakes 2020-04-09 09:43:02 +08:00
Alexander Tokmakov
140cd88c20 fixes 2020-04-09 02:59:39 +03:00
Alexey Milovidov
540e9f7d0d Fix logged number of inserted rows into ReplicatedMergeTree 2020-04-09 01:58:15 +03:00
Konstantin Lebedev
b666f60af8 Optional secured communication between ClickHouse and Zookeeper 2020-04-09 01:40:54 +03:00
Azat Khuzhin
6d85207bfb Convert blocks if structure does not match on INSERT into Distributed()
Follow-up for: #10105
2020-04-08 23:46:01 +03:00
alexey-milovidov
db4270b60c
Merge pull request #10077 from zhang2014/fix/ISSUES-10056
ISSUES-10056 support identifier argument for MySQL Database engine
2020-04-08 22:59:09 +03:00
Artem Zuikov
ab12ebb5cf add allowDictJoin() 2020-04-08 22:58:27 +03:00
alexey-milovidov
0372a6119a
Merge pull request #10110 from amosbird/tsm
more monotonicity for toString()
2020-04-08 22:55:04 +03:00
alexey-milovidov
9a00eca1c5
Merge pull request #10121 from ClickHouse/fix_adaptive_granularity_compatibility
Fix adaptive granularity compatibility
2020-04-08 22:50:09 +03:00
alexey-milovidov
c389fee7e7
Update StorageReplicatedMergeTree.cpp 2020-04-08 22:48:01 +03:00
alexey-milovidov
80b73a209b
Merge pull request #10107 from ClickHouse/fix-globs-test
Fix unit test in debug build.
2020-04-08 22:46:44 +03:00
alexey-milovidov
de2fb3ea17
Merge pull request #10108 from ClickHouse/thread-fuzzer-disable-under-thread-memory-sanitizer
ThreadFuzzer: do not wrap pthread functions under thread and memory sanitizers
2020-04-08 22:46:07 +03:00
Alexander Tokmakov
dd1590830b Merge branch 'master' into database_atomic 2020-04-08 22:00:46 +03:00
Artem Zuikov
8d6f641796 join over dictionary concept 2020-04-08 21:59:52 +03:00
Nikolai Kochetov
458c7f516d Fix extremes for Processors. 2020-04-08 21:50:14 +03:00
Pavel Kovalenko
0ea0daa194 Merge remote-tracking branch 'origin/master' into replicated-merge-tree-s3 2020-04-08 21:47:17 +03:00
zhang2014
9eb96b87db ISSUES-10056 reused evaluateConstantExpressionOrIdentifierAsLiteral 2020-04-09 01:31:28 +08:00
zhang2014
9ed708b902 ISSUES-10056 add some check and support identifier argument for MySQL Database Engine 2020-04-09 01:31:27 +08:00
Anton Popov
c8c4dc8104 fix 'ALTER MODIFY COLUMN' with compact parts 2020-04-08 19:20:52 +03:00
Amos Bird
f1cb928737
more monotonicity for toString() 2020-04-08 20:13:44 +08:00
Pavel Kovalenko
3603c98c81 Move gtest for S3 URI to proper folder. 2020-04-08 14:59:24 +03:00
Pavel Kovalenko
89fe81ed62 Integration test for ReplicatedMergeTree over S3 fixes. 2020-04-08 14:56:31 +03:00
Pavel Kovalenko
bc349a563a Merge branch 'master' into replicated-merge-tree-s3 2020-04-08 13:53:30 +03:00
alesapin
ded306f016 Fix adaptive granularity compatibility 2020-04-08 12:51:04 +03:00
Pavel Kovalenko
9b1c5c8b51 Fix regex for S3 URI to pass clang-tidy check. 2020-04-08 11:44:26 +03:00
Pavel Kovalenko
5b2b8d38fa Download part trough disk interface. 2020-04-08 11:41:13 +03:00
Azat Khuzhin
3116b2d29b Make SHOW CREATE TABLE multiline (because it more readable and also mysql like) 2020-04-08 10:26:40 +03:00
Amos Bird
0b5cc80580
joinGetOrNull 2020-04-08 11:01:08 +08:00
alexey-milovidov
a46a61c970
Update KeyCondition.h 2020-04-08 05:56:25 +03:00
alexey-milovidov
a42d875a68
Update KeyCondition.h 2020-04-08 05:55:39 +03:00
alexey-milovidov
723a1f41e2
Update KeyCondition.h 2020-04-08 05:55:22 +03:00
alexey-milovidov
94a621060d
Update KeyCondition.h 2020-04-08 05:55:03 +03:00
alexey-milovidov
a6194364ea
Update MergeTreeDataSelectExecutor.cpp 2020-04-08 05:54:08 +03:00
alexey-milovidov
6d80ab1eed
Merge pull request #9811 from vitlibar/RBAC-8
RBAC-8
2020-04-08 05:47:55 +03:00
alexey-milovidov
dd333460d6
Merge pull request #10068 from ClickHouse/fix_check_table_with_index
Fix bug with uncompressed checksums in CHECK TABLE query
2020-04-08 05:37:18 +03:00
alexey-milovidov
dad892e9d4
Merge pull request #10105 from azat/distributed-insert-types-differs
Fix SIGSEGV on INSERT into Distributed on different struct with underlying
2020-04-08 05:36:02 +03:00
Alexey Milovidov
97c2d17e99 ThreadFuzzer: do not wrap pthread functions under thread and memory sanitizers 2020-04-08 05:32:33 +03:00
Anton Popov
cbdc3ef214 Merge remote-tracking branch 'upstream/master' into speedup-index 2020-04-08 04:15:48 +03:00
alexey-milovidov
bcb2899b0d
Merge pull request #10104 from ClickHouse/adjust-collisions-weak-hash
Remove unwanted output from unit tests
2020-04-08 04:10:08 +03:00
Alexey Milovidov
f8500cf5d5 Fix unit test 2020-04-08 04:09:01 +03:00
Alexander Tokmakov
135197aa75 fixes 2020-04-08 04:02:00 +03:00
alexey-milovidov
401c52524b
Merge pull request #9998 from Jokser/s3-uri-virtual-hosted-style-support
Virtual hosted-style support for S3 URI
2020-04-08 03:51:28 +03:00
alexey-milovidov
37fe63f8d8
Update S3Common.cpp 2020-04-08 03:49:09 +03:00
alexey-milovidov
09a397a68f
Merge pull request #9997 from azat/dist-on-dist-fixes
Fix Distributed-over-Distributed with one only shard in nested table
2020-04-08 03:45:29 +03:00
alexey-milovidov
ebe9ae4fab
Update StorageMerge.cpp 2020-04-08 03:45:11 +03:00
alexey-milovidov
1b704425b6
Update StorageBuffer.cpp 2020-04-08 03:42:47 +03:00
alexey-milovidov
a29393821d
Merge pull request #10099 from ClickHouse/fix-9826
Fix 9826
2020-04-08 02:41:11 +03:00
Azat Khuzhin
b2fa9d8750 Fix SIGSEGV on INSERT into Distributed on different struct with underlying 2020-04-08 02:35:31 +03:00
alexey-milovidov
c079fd7f1b
Merge pull request #10064 from ClickHouse/small_fixes_in_dict_parser
Small fixes in dictionaries DDL parser
2020-04-08 02:22:20 +03:00
alexey-milovidov
7212efbfbf
Merge pull request #10103 from ClickHouse/fix-zk-multi-async-flacky
Make the test "zk_multi_async" less flacky
2020-04-08 02:15:43 +03:00
alexey-milovidov
7bef5a6edb
Update checkDataPart.cpp 2020-04-08 02:13:53 +03:00
alexey-milovidov
491f4b2c60
Merge pull request #10071 from ClickHouse/add_conversion_stream
Add converting stream to localhost clickhouse dict
2020-04-08 02:13:17 +03:00
Vitaly Baranov
d6fa8d1aad
Merge pull request #10101 from vitlibar/external-loader-remove-useless-logging
Remove useless logging from ExternalLoader.
2020-04-08 02:12:50 +03:00
alexey-milovidov
d20dfea6ed
Merge pull request #10095 from ClickHouse/fix_replica_start_from_old_version
Fix replicated tables startup when updating from old version
2020-04-08 02:11:12 +03:00
alexey-milovidov
115dd1b5fe
Merge pull request #9989 from ClickHouse/globs-in-paths-leading-zeros
Fix behaviour of globs in filepath with leading zeros
2020-04-08 00:49:27 +03:00
Alexey Milovidov
78b0bb6a3f Remove unwanted output from unit tests 2020-04-08 00:38:31 +03:00
Vitaly Baranov
e573549945 Rework access rights for table functions. 2020-04-07 23:31:59 +03:00
Vitaly Baranov
42b8ed3ec6 Implement "ON CLUSTER" clause for access control SQL. 2020-04-07 23:31:59 +03:00
Vitaly Baranov
b77e0a5b4e Avoid writing "HOST ANY" if the host is any by default. 2020-04-07 23:31:59 +03:00
Vitaly Baranov
d0af31bbcf Use ALTER USER DROP HOST instead of ALTER USER REMOVE HOST for consistency. 2020-04-07 23:31:59 +03:00
Vitaly Baranov
d064ddfe13 Disable MemoryAccessStorage. 2020-04-07 23:31:59 +03:00
Alexey Milovidov
ccec944ff4 Fix flacky test, take 3 2020-04-07 23:21:43 +03:00
Vitaly Baranov
423fa5087a Add SHOW_USERS(SHOW ROLES, etc.) privileges. 2020-04-07 23:20:38 +03:00
Vitaly Baranov
b4d7ef390c Rename some access types: add ALTER and SYSTEM prefixes. 2020-04-07 23:20:38 +03:00
Vitaly Baranov
f53b4ad3a8 Replace access types "TRUNCATE_VIEW" and "TRUNCATE_TABLE" with "TRUNCATE". 2020-04-07 23:20:38 +03:00
Vitaly Baranov
e5d8f05251 Rename sql command "CREATE POLICY" -> "CREATE ROW POLICY", "CREATE POLICY" is now an alias. 2020-04-07 23:20:38 +03:00
Vitaly Baranov
c2f5e3c4ad Improve declaration of access rights: single place in code instead of three. 2020-04-07 23:20:38 +03:00
Vitaly Baranov
d8b76232fe Remove useless logging from ExternalLoader. 2020-04-07 21:39:34 +03:00
Nikolai Kochetov
6b5b22cc5c Fix build. 2020-04-07 20:53:14 +03:00
Nikolai Kochetov
2704d81aae Fix pipeline expansion in PipelineExecutor. 2020-04-07 20:38:48 +03:00
alexey-milovidov
ead277b717
Update StorageReplicatedMergeTree.cpp 2020-04-07 20:23:26 +03:00
alexey-milovidov
2654f131cc
Update checkDataPart.cpp 2020-04-07 20:08:46 +03:00
alesapin
27777c1d2b Better comment 2020-04-07 18:55:23 +03:00
Alexander Tokmakov
4c48b7dd80 better rename 2020-04-07 18:31:33 +03:00
alesapin
82a87bc0d2 Fix replicated tables startup when updating from old version 2020-04-07 18:28:29 +03:00
Alexander Kuzmenkov
385e8839dc
Quote expected strings in parse error messages 2020-04-07 17:15:14 +03:00
Olga Khvostikova
834e458f26 Fix behaviour of globs in filepath with leading zeros 2020-04-07 16:46:12 +03:00
alesapin
0d7cff2b25 Merge branch 'master' into fix_check_table_with_index 2020-04-07 16:36:09 +03:00
alesapin
33c2587213 Merge branch 'master' into add_conversion_stream 2020-04-07 16:35:40 +03:00
alesapin
06f135ee8d Merge branch 'master' into small_fixes_in_dict_parser 2020-04-07 16:35:16 +03:00
Nikita Mikhaylov
626856deff Merge branch 'master' of github.com:ClickHouse/ClickHouse into quorum_bug 2020-04-07 15:40:55 +03:00
alesapin
f1e0f15556
Merge pull request #9948 from ClickHouse/alter_rename_column
Alter rename column
2020-04-07 15:28:00 +03:00
Nikita Mikhaylov
3bf67b1380 typo 2020-04-07 15:02:07 +03:00
alesapin
9bc0ab2eed Merge branch 'master' into small_fixes_in_dict_parser 2020-04-07 14:59:08 +03:00
alesapin
d0bed6b204 Merge branch 'master' into fix_check_table_with_index 2020-04-07 14:58:45 +03:00
alesapin
5f3bcf198b Merge branch 'master' into add_conversion_stream 2020-04-07 14:57:58 +03:00
Alexander Kazakov
621d26bcf7 Better timeout diagnostics message 2020-04-07 14:34:35 +03:00
philip.han
53b5dade5e Implement transform_null_in 2020-04-07 20:08:13 +09:00
Nikita Mikhaylov
70c0646942 Merge branch 'master' of github.com:ClickHouse/ClickHouse into quorum_bug 2020-04-07 13:42:48 +03:00
Artem Zuikov
8297683972
renames: AnalyzedJoin -> TableJoin, Join -> HashJoin (#10065) 2020-04-07 12:48:47 +03:00
Alexander Kazakov
78f97a7a59 Build fixes 2020-04-07 12:23:08 +03:00
Ivan
f6b31f344d
Add cross-compile build for FreeBSD (#9643)
* Add toolchain to Docker image
2020-04-07 11:33:49 +03:00
Vasily Nemkov
ef07c6d06f Merge remote-tracking branch 'upstream/master' into SIMD_tokenization 2020-04-07 11:25:52 +03:00
Alexander Kazakov
2e76e4d1ed Made clang happy - fixed build 2020-04-07 10:15:59 +03:00
Alexander Kuzmenkov
35783af25d Merge remote-tracking branch 'origin/master' into HEAD 2020-04-07 03:34:39 +03:00
Alexander Kazakov
081187dd1f Unit test for RWLock is set to use timeout for avoiding deadlocks 2020-04-07 03:06:48 +03:00
Alexander Kazakov
09ce548376 All locks in IStorage have timeouts now 2020-04-07 02:45:51 +03:00
Alexander Kazakov
b98bc9afef Reworked RWLockImpl::getLock() + phase-fairness
Timeout param added to getLock() method
2020-04-07 02:44:45 +03:00
Alexander Tokmakov
a2101c9b1c better code 2020-04-07 02:22:44 +03:00
Alexander Kazakov
2951ed4f1d Corrected Common.RWLockDeadlock test 2020-04-07 00:35:20 +03:00
Nikita Mikhaylov
2cec261f3a Merge branch 'master' of github.com:ClickHouse/ClickHouse into quorum_bug 2020-04-06 23:15:11 +03:00
alesapin
107c5b7860
Merge pull request #10063 from ClickHouse/fix-GroupingAggregatedTransform-single-level
Fix GroupingAggregatedTransform for single-level aggregation. Add test.
2020-04-06 22:13:15 +03:00
alesapin
04e3e3179c Add converting stream to localhost clickhouse dict 2020-04-06 22:09:39 +03:00
Vasily Nemkov
738e8a7ef8 Minor test refactoring
* using string literal for binary strings
* sorted includes
2020-04-06 21:41:11 +03:00
Nikolai Kochetov
4fd5ef8bad Review fixes. 2020-04-06 21:05:45 +03:00
tavplubix
ea376b9739
Merge pull request #10054 from azat/detachDatabase-on-fail-at-start-deadlock
Fix deadlock on failed database attach at start with materialized view
2020-04-06 20:37:39 +03:00
Vasily Nemkov
3d471556fb Merge commit '73b0f8db8c327a1d63cc7ebcc56087a3f9866dae' into SIMD_tokenization 2020-04-06 20:05:02 +03:00
alesapin
c17fa34fa5 Fix bug with uncompressed checksums in CHECK TABLE query 2020-04-06 20:00:29 +03:00
alesapin
4be263c72f Clang-tidy fix 2020-04-06 17:43:36 +03:00
Artem Zuikov
174136a1e0
inflating cross join (#10029) 2020-04-06 16:39:57 +03:00
Anton Popov
2dc1eddfab fix FieldRef 2020-04-06 16:35:11 +03:00
Nikolai Kochetov
8986cf688b Added comment. 2020-04-06 16:28:46 +03:00
Alexander Tokmakov
08bae4668d Merge branch 'master' into database_atomic 2020-04-06 16:18:07 +03:00
alesapin
848678d656 Now layout type for dictionaries DDL with no arguments can be written without brackets 2020-04-06 14:02:17 +03:00
Anton Popov
5ada959853 improve performance of index analysis with monotonic functions 2020-04-06 13:37:34 +03:00
Anton Popov
79024d73a2 improve performance of index analysis with monotonic functions 2020-04-06 13:37:34 +03:00
Nikolai Kochetov
0117c194c5 Fix GroupingAggregatedTransform for single-level aggregation. Add test. 2020-04-06 13:33:59 +03:00
Pavel Kovalenko
8c1dda0105 Merge remote-tracking branch 'jokser/s3-uri-virtual-hosted-style-support' into s3-uri-virtual-hosted-style-support 2020-04-06 12:59:59 +03:00
Pavel Kovalenko
f9c98dd689 Merge remote-tracking branch 'origin/master' into s3-uri-virtual-hosted-style-support 2020-04-06 12:57:53 +03:00
Vxider
2fb9138505 merged with master 2020-04-06 17:36:44 +08:00
alesapin
dd5ddf19d9 Rename missed files 2020-04-06 11:55:59 +03:00
alesapin
1cb072d58a Merge branch 'master' into alter_rename_column 2020-04-06 11:40:27 +03:00
Alexey Milovidov
110eb599c1 Fix "check style" tool 2020-04-06 02:57:24 +03:00
Azat Khuzhin
e7bbd400c7 Fix deadlock on failed database attach at start with materialized view
This is not the problem for plain DROP DATABASE query since it first
remove tables.

  (gdb) bt
  0  __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:103
  1  0x00007f353c262774 in __GI___pthread_mutex_lock (mutex=mutex@entry=0x11049288 <DB::DatabaseCatalog::init(DB::Context const*)::database_catalog+8>) at ../nptl/pthread_mutex_lock.c:80
  2  0x0000000004fd5d1e in DB::pthread_mutex_lock (arg=arg@entry=0x11049288 <DB::DatabaseCatalog::init(DB::Context const*)::database_catalog+8>) at ../src/Common/ThreadFuzzer.cpp:253
  3  0x000000000c70b3a9 in std::__1::__libcpp_mutex_lock (__m=__m@entry=0x11049288 <DB::DatabaseCatalog::init(DB::Context const*)::database_catalog+8>) at ../contrib/libcxx/include/__threading_support:322
  4  std::__1::mutex::lock (this=this@entry=0x11049288 <DB::DatabaseCatalog::init(DB::Context const*)::database_catalog+8>) at ../contrib/libcxx/src/mutex.cpp:33
  5  0x0000000008acd4e2 in std::__1::lock_guard<std::__1::mutex>::lock_guard (__m=..., this=<synthetic pointer>) at ../contrib/libcxx/include/__mutex_base:90
  6  DB::DatabaseCatalog::removeDependency (this=this@entry=0x11049280 <DB::DatabaseCatalog::init(DB::Context const*)::database_catalog>, from=..., where=...) at ../src/Interpreters/DatabaseCatalog.cpp:388
  7  0x000000000975044b in DB::StorageMaterializedView::shutdown (this=0x7f338d178a00) at ../src/Storages/StorageMaterializedView.cpp:362
  8  0x0000000008a79602 in DB::DatabaseWithOwnTablesBase::shutdown (this=0x7f353be3cc60) at ../contrib/libcxx/include/__tree:184
  9  0x0000000009546196 in DB::DatabaseWithDictionaries::shutdown (this=0x7f353be3cc60) at ../src/Databases/DatabaseWithDictionaries.cpp:265
  10 0x0000000008acecdb in DB::DatabaseCatalog::detachDatabase (this=0x11049280 <DB::DatabaseCatalog::init(DB::Context const*)::database_catalog>, database_name=..., drop=drop@entry=false, check_empty=check_empty@entry=false) at ../contrib/libcxx/include/memory:3826
  11 0x0000000004bf0aa5 in DB::InterpreterCreateQuery::createDatabase (this=0x7ffd6e1bab80, create=...) at ../src/Interpreters/InterpreterCreateQuery.cpp:162
  12 0x0000000008b04327 in DB::InterpreterCreateQuery::execute (this=this@entry=0x7ffd6e1bab80) at ../src/Interpreters/InterpreterCreateQuery.cpp:722
  13 0x0000000008d0fdaa in DB::executeCreateQuery (has_force_restore_data_flag=false, file_name=..., database=..., context=..., query=...) at ../src/Interpreters/loadMetadata.cpp:48
  14 DB::loadDatabase (context=..., database=..., database_path=..., force_restore_data=<optimized out>) at ../src/Interpreters/loadMetadata.cpp:72
  15 0x0000000008d103c3 in DB::loadMetadata (context=...) at ../src/Interpreters/loadMetadata.cpp:111
  16 0x0000000004f4d25e in DB::Server::main (this=<optimized out>) at ../contrib/libcxx/include/memory:2582
  17 0x000000000bbc8963 in Poco::Util::Application::run (this=this@entry=0x7ffd6e1bcc60) at ../contrib/poco/Util/src/Application.cpp:334
  18 0x0000000004ffe1de in DB::Server::run (this=0x7ffd6e1bcc60) at ../programs/server/Server.cpp:178
  19 0x0000000004ff36cc in mainEntryClickHouseServer (argc=3, argv=0x7f353be58bc0) at ../programs/server/Server.cpp:1060
  20 0x0000000004f49b99 in main (argc_=<optimized out>, argv_=<optimized out>) at ../contrib/libcxx/include/vector:655
2020-04-05 14:16:16 +03:00
Vxider
21532f6a6d parallel insert for materialized view 2020-04-05 01:33:51 +08:00
Avogar
e635b0e9eb Fix build error 2020-04-04 20:04:41 +03:00
Nikolai Kochetov
e23fda80aa Merge with master 2020-04-04 19:02:17 +03:00
Avogar
092479397a Remove extra line. 2020-04-04 17:07:11 +03:00
Avogar
fbc11d3bf9 Move files 2020-04-04 17:04:20 +03:00
Avogar
7a96984dd8 Merge branch 'master' of github.com:ClickHouse/ClickHouse into msgpack_format 2020-04-04 16:44:44 +03:00
Azat Khuzhin
1232760f78 Fix Distributed-over-Distributed when nested table has only one shard 2020-04-04 13:47:35 +03:00
Azat Khuzhin
8d372b0be7 Call getQueryProcessingStage() once, since it is heavy for StorageDistributed
Refs: #9808
2020-04-04 13:47:34 +03:00
Ivan Lezhankin
8a5a73e76d Merge branch 'master' into dbms2src 2020-04-03 18:20:05 +03:00
Ivan Lezhankin
06446b4f08 dbms/ → src/ 2020-04-03 18:14:31 +03:00
Azat Khuzhin
f53c9a6b25 Fix "Block structure mismatch" for INSERT into Distributed
Add missing conversion (via ConvertingBlockInputStream) for INSERT into
remote nodes (for sync insert, async insert and async batch insert),
like for local nodes (in DistributedBlockOutputStream::writeBlockConverted).

This is required when the structure of the Distributed table differs
from the structure of the local table.

And also add a warning message, to highlight this in logs (since this
works slower).

Fixes: #19888
2021-02-02 10:16:41 +03:00