thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-01 20:12:02 +00:00
Code Issues Packages Projects Releases Wiki Activity
91,855 Commits 9,193 Branches 1,782 Tags 2.2 GiB
ea0996674f
Commit Graph

27917 Commits

Author SHA1 Message Date
kssenii
a3823c67ee Fix 2022-06-24 02:44:22 +02:00
mergify[bot]
a0eb831cd1
Merge branch 'master' into overcommit-deadlock-fix 2022-06-23 23:02:59 +00:00
Alexey Milovidov
812ab9bd6b
Merge pull request #36103 from sauliusvl/uring 
Add support for io_uring read method
 2022-06-24 00:34:29 +03:00
Alexander Tokmakov
3f4a09478d
Revert "Fix optimization in PartialSortingTransform (SIGSEGV and possible incorrect result)" 2022-06-23 23:01:11 +03:00
kssenii
bc9b56096f Fix 2022-06-23 21:52:57 +02:00
Alexander Tokmakov
74f38710a8 Merge branch 'master' into cleanup_garbage_in_store_dir 2022-06-23 21:43:28 +02:00
kssenii
6fbd49f554 Merge master 2022-06-23 21:40:01 +02:00
zvonand
dd8203038f updated exception handling 2022-06-24 00:36:57 +05:00
Kseniia Sumarokova
40a4742f93
Merge pull request #38319 from kssenii/fix-access-for-drop-fs-cache 
Add access check and on cluster option for system drop fs cache
 2022-06-23 20:56:44 +02:00
Vladimir Chebotarev
3333dc2da1
Update S3Capabilities.h 2022-06-23 20:59:44 +03:00
Alexey Milovidov
e7c8023ef5
Merge pull request #38201 from ClickHouse/revert-35914-FIPS_compliance 
Revert "ClickHouse's boringssl module updated to the official version of the FIPS compliant."
 2022-06-23 20:22:03 +03:00
kssenii
468c98ff42 Better 2022-06-23 17:46:27 +02:00
mergify[bot]
174d8577fa
Merge branch 'master' into merge_tree_reader 2022-06-23 15:16:51 +00:00
Maksim Kita
b7439a36f6
Merge pull request #38324 from azat/fix-partial-sort 
Fix optimization in PartialSortingTransform (SIGSEGV and possible incorrect result)
 2022-06-23 16:08:52 +02:00
Raúl Marín
7f7d082fb3 Add implicit_transaction setting 2022-06-23 15:23:37 +02:00
Roman Vasin
cb748cd8ec Fix code style in KerberosInit 2022-06-23 16:11:48 +03:00
Kseniia Sumarokova
2b883c2c01
Merge pull request #38285 from kssenii/extract-some-diff 
Extract some diff from #36171
 2022-06-23 14:38:03 +02:00
mergify[bot]
234f0c6399
Merge branch 'master' into revert-35914-FIPS_compliance 2022-06-23 12:06:17 +00:00
Antonio Andelic
b2776e3dcd
Merge pull request #38056 from azat/global-join-cte-fix 
Fix "Missing columns" for GLOBAL JOIN with CTE w/o alias
 2022-06-23 14:05:50 +02:00
zvonand
946117ec89 Merge branch 'master' of github.com:ClickHouse/ClickHouse into zvonand-base58 2022-06-23 17:04:40 +05:00
Anton Popov
7efbae7728
Merge pull request #38069 from CurtizJ/better-support-gcp 
Better support of GCP storage
 2022-06-23 13:19:14 +02:00
alesapin
ae2feacbd1
Merge pull request #38322 from ClickHouse/ban_projections_in_a_right_way 
Ban projections for zero-copy replication in a right way
 2022-06-23 11:52:30 +02:00
kssenii
bc4fd4c044 Support ON CLUSTER 2022-06-23 11:44:36 +02:00
Vitaly Baranov
18b4413df8
Merge pull request #38299 from vitlibar/backup-improvements-7 
Backup improvements 7
 2022-06-23 11:37:03 +02:00
Kseniia Sumarokova
af2150a68d
Merge pull request #38176 from kssenii/url-headers-in-ast 
Add ability to pass headers to url table function / storage via sql
 2022-06-23 10:15:07 +02:00
Kseniia Sumarokova
cce381a3ae
Merge pull request #38307 from azat/fix-insert-profile-events 
Fix INSERT into Distributed hung due to ProfileEvents
 2022-06-23 10:09:45 +02:00
mergify[bot]
b33ff9f254
Merge branch 'master' into extract-some-diff 2022-06-23 08:04:49 +00:00
Kseniia Sumarokova
5163ca5565
Merge pull request #38310 from ClickHouse/kssenii-patch-4 
Fix retries in postgres engine
 2022-06-23 10:04:17 +02:00
Roman Vasin
4bf1fc4718 Add error code and message displaying in exceptions of KerberosInit; Correct code style in KerberosInit 2022-06-23 10:28:31 +03:00
Vladimir Chebotarev
e2cdaa795b
Fixes comments. 2022-06-23 09:55:00 +03:00
Antonio Andelic
a5ee918bd2
Merge pull request #38047 from ClickHouse/keeper-sequential-consistency-reads 
Add `SYNC` command to internal ZooKeeper client
 2022-06-23 08:43:24 +02:00
alesapin
1b495ec8ad Fix style 2022-06-23 00:17:37 +02:00
Azat Khuzhin
9db64952c0 Fix SIGSEGV in optimization in PartialSortingTransform 
Fixes: #37992 (cc @kitaisreal)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-06-22 21:39:10 +03:00
Alexander Gololobov
0ee47363d4 Fixed includes 2022-06-22 19:08:18 +02:00
Alexander Gololobov
8138a3a36e Cleanups based on review comments 2022-06-22 19:08:01 +02:00
alesapin
ca0299e927 Ban projections for zero-copy replication in a right way 2022-06-22 19:01:46 +02:00
Vitaly Baranov
5ae8fce1ef Attach threads to thread groups better. 2022-06-22 18:51:41 +02:00
Alexander Tokmakov
ed8341025b make code less bad 2022-06-22 18:31:42 +02:00
kssenii
dce1c76270 Fix 2022-06-22 18:00:25 +02:00
Kruglov Pavel
a8757881da
Merge pull request #38306 from ClickHouse/Avogar-patch-1 
Fix misleading error message while s3 schema inference
 2022-06-22 17:24:59 +02:00
Alexander Gololobov
e5b55b965b Removed incorrect check 2022-06-22 17:23:09 +02:00
Alexander Gololobov
dbc6d1a159 Cleanups 2022-06-22 17:23:09 +02:00
Alexander Gololobov
aea5bc7527 Removed unused should_reorder flag 2022-06-22 17:23:09 +02:00
Alexander Gololobov
5427d5c51b Apply filter at row level security step 2022-06-22 17:23:09 +02:00
Alexander Gololobov
a5b3f83181 Always remove row filter column 2022-06-22 17:23:09 +02:00
Alexander Gololobov
b3922461b3 Properly handle empty actions 2022-06-22 17:23:09 +02:00
Alexander Gololobov
ba89c3954c Do not add the same vitrual if it has been added by prev_reader 2022-06-22 17:23:09 +02:00
Alexander Gololobov
e5c0ebc7b5 Handle read from wide part when no columns are requested 2022-06-22 17:23:09 +02:00
Alexander Gololobov
a9e3b8d29e Don't read the same columns again 2022-06-22 17:23:09 +02:00
Alexander Gololobov
4e426c63cc Debuging test failures 2022-06-22 17:23:09 +02:00
Alexander Gololobov
6a26325fab Test dirty hacks for multiple PREWHERE steps 2022-06-22 17:23:05 +02:00
Alexander Gololobov
4360fd9798 Vector of structs instead of multiple vectors 2022-06-22 17:17:42 +02:00
Alexander Gololobov
87b669f439 Intermediate changes 2022-06-22 17:17:42 +02:00
Alexander Gololobov
159ab765fb Moved estimateNumRows from lambda to function 2022-06-22 17:17:42 +02:00
Alexander Gololobov
538c8c96fc Dump row_level_filter as a part of prewhere_info 2022-06-22 17:17:42 +02:00
Alexander Gololobov
64a2f3734b Protect ReadResult internals from MergeTreeRangeReader clients 2022-06-22 17:17:42 +02:00
kssenii
0acc95fe22 Review fixes 2022-06-22 16:55:47 +02:00
Dmitry Novik
3544fbe5c4 cleanup 2022-06-22 14:05:20 +00:00
Kseniia Sumarokova
cc16854c04
Fix postgres execWithRetry 2022-06-22 15:44:50 +02:00
Roman Vasin
7bd65c8c24 Add comments to KerberosInit; Remove input cache and flags from KerberosInit 2022-06-22 16:31:48 +03:00
Nikita Taranov
41ba0118b5
Bring back #36396 (#38110) 
* Revert "Revert "More parallel execution for queries with `FINAL` (#36396)""

This reverts commit 5bfb15262c.

* fix tests

* fix review suggestions

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2022-06-22 15:05:07 +02:00
Kseniia Sumarokova
87b3f331e2
Update MetadataStorageFromDiskTransactionOperations.cpp 2022-06-22 14:57:29 +02:00
Azat Khuzhin
9eeb856519 Fix INSERT into Distributed hung due to ProfileEvents 
Right now RemoteInserter does not read ProfileEvents for INSERT, it
handles them only after sending the query or on finish.

But #37391 sends them for each INSERT block, but sometimes they can be
no ProfileEvents packet, since it sends only non-empty blocks.

And this adds too much complexity, and anyway ProfileEvents are useless
for the server, so let's send them only if the query is initial (i.e.
send by user).

Note, that it is okay to change the logic of sending ProfileEvents w/o
changing DBMS_TCP_PROTOCOL_VERSION, because there were no public
releases with the original patch included yet.

Fixes: #37391
Refs: #35075
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-06-22 15:41:15 +03:00
alesapin
cc2f01c41d Fix name clash 2022-06-22 14:28:27 +02:00
alesapin
531c29ae46 Merge branch 'master' into kssenii-extract-some-diff 2022-06-22 13:06:32 +02:00
alesapin
0f8eed98c4
Merge branch 'master' into better-support-gcp 2022-06-22 12:46:01 +02:00
Kruglov Pavel
e5a7f53775
Fix misleading error message while s3 schema inference 2022-06-22 12:36:09 +02:00
Alexander Tokmakov
31c4f83469
Merge pull request #38272 from ClickHouse/create_empty_as_select 
Support CREATE ... EMPTY AS SELECT
 2022-06-22 13:25:12 +03:00
alesapin
d3bc7c0190
Merge pull request #37971 from Avogar/fix-alter-columns-with-dots 
Fix ALTER column with column names with dots
 2022-06-22 12:20:30 +02:00
Nikolai Kochetov
cc6fdfe0eb
Merge pull request #36555 from ClickHouse/refactor-something-in-part-volumes 
Separate data storage abstraction for MergeTree
 2022-06-22 11:13:36 +02:00
Roman Vasin
f281199588 Fix code style in KerberosInit; Add anonymous namespace; Add comment about using kerberos_init 2022-06-22 11:28:00 +03:00
Alexey Milovidov
5855668514 Remove trash 2022-06-22 06:23:35 +02:00
Alexey Milovidov
0f8bc027fd Merge branch 'revert-35914-FIPS_compliance' of github.com:ClickHouse/ClickHouse into revert-35914-FIPS_compliance 2022-06-22 06:17:01 +02:00
Alexey Milovidov
da578ec890
Merge pull request #38204 from ClickHouse/enable-enable_positional_arguments 
Enable `enable_positional_arguments`
 2022-06-22 07:12:08 +03:00
Alexander Tokmakov
dc1f596326 fix 2022-06-22 00:50:16 +02:00
Anton Popov
d5d568e235
Merge pull request #37913 from azat/parts-refcnt 
Fix refcnt for unused MergeTree parts in SELECT queries
 2022-06-21 22:56:26 +02:00
Robert Schulze
0d80874d40
Merge pull request #38068 from ClickHouse/clang-tsa 
Support for Clang Thread Safety Analysis (TSA)
 2022-06-21 20:19:33 +02:00
kssenii
ac937c7fd6 Add comment 2022-06-21 19:21:36 +02:00
kssenii
ca09935773 Move into separate file 2022-06-21 18:58:05 +02:00
Alexander Tokmakov
8288b9fe60 fixes 2022-06-21 18:47:55 +02:00
Roman Vasin
1d6eea6cfa Replace LOG_DEBUG by LOG_TRACE in KerberosInit; Correct exception message; Prohibit making a copy of KerberosInit 2022-06-21 18:55:17 +03:00
Kseniia Sumarokova
7fb5217384
Update ASTDescribeCacheQuery.h 2022-06-21 16:18:18 +02:00
kssenii
481644939a Add describe cache command 2022-06-21 16:07:39 +02:00
Alexander Tokmakov
13e3d40db4 fix 2022-06-21 15:11:03 +02:00
Nikolai Kochetov
dcf6e0d060 Roll back some changes from IPartMetadataManager 2022-06-21 13:05:49 +00:00
Alexander Tokmakov
f3ea8ad745 support CREATE EMPTY AS SELECT 2022-06-21 15:03:58 +02:00
Kruglov Pavel
8bf78dbc95
Add comment 2022-06-21 14:32:12 +02:00
Nikolai Kochetov
854d148d73 Add some comments. Remove some commented code. 2022-06-21 12:31:02 +00:00
Dmitry Novik
099055c183 Explicitly forbid allocations in OvercommitTracker 2022-06-21 12:26:13 +00:00
kssenii
4656dd3de3 Headers for url in ast 2022-06-21 14:25:05 +02:00
Nikolai Kochetov
46c4fd6cf7 Fixing build. 2022-06-21 11:15:06 +00:00
Kruglov Pavel
3c5a627bd7
Merge pull request #38239 from CurtizJ/fix-reading-from-s3 
Fix reading from s3 in some corner cases
 2022-06-21 12:57:55 +02:00
mergify[bot]
f304fc9179
Merge branch 'master' into fix-alter-columns-with-dots 2022-06-21 10:40:53 +00:00
mergify[bot]
bb79eb73e6
Merge branch 'master' into fix-short-circuit 2022-06-21 10:40:07 +00:00
Nikolai Kochetov
1e8c9ecd4c Merge branch 'master' into refactor-something-in-part-volumes 2022-06-21 12:37:21 +02:00
Alexander Tokmakov
ba0fcec993 add background cleanup of store/ subdirs 2022-06-21 12:35:47 +02:00
Kruglov Pavel
b9b58b4305
Merge pull request #37759 from Avogar/fix-nothing-error 
Fix possible logical error with type Nothing in some functions
 2022-06-21 12:35:05 +02:00
alesapin
fafcb876be
Merge pull request #38182 from ClickHouse/disk_transaction 
Disk transaction
 2022-06-21 12:28:49 +02:00
Saulius Valatka
160c95a6b8 unpoisson memory read by io_uring, handle interrupted syscalls, do not throw from monitor thread 2022-06-21 13:20:21 +03:00
Saulius Valatka
7b5e13a9eb implement pending request queue to prevent CQ overflows, more error checking, reduce size of io_uring 2022-06-21 13:17:56 +03:00
Nikolai Kochetov
b8d27aa8dd
Merge pull request #37469 from azat/projections-optimize_aggregation_in_order 
Implement in order aggregation (optimize_aggregation_in_order) for projections for tables with fully materialized projections
 2022-06-21 12:17:35 +02:00
Dmitry Novik
e10f079bd3 Get rid of allocations in OvercommitTracker 2022-06-21 10:15:33 +00:00
Kseniia Sumarokova
71ee5dbbb2
Merge pull request #38227 from kssenii/diff-reduce 
buffer's getFileSize small changes
 2022-06-21 12:12:22 +02:00
Nikolai Kochetov
e45d552f4d Do not create empty folder for in-memory parts. 2022-06-21 09:59:46 +00:00
mergify[bot]
bdd5b94192
Merge branch 'master' into keeper-sequential-consistency-reads 2022-06-21 08:13:04 +00:00
Nikolai Kochetov
5232c17b89 Fix style 2022-06-21 07:30:45 +00:00
Nikolai Kochetov
3429a5e07a Fix 2022-06-21 07:26:43 +00:00
Saulius Valatka
6bbabf59d9 simplify build, add read method randomized tests, fix typos 2022-06-21 09:59:46 +03:00
Saulius Valatka
68eef98128 check for io_uring support, fail if not supported 2022-06-21 09:59:46 +03:00
Saulius Valatka
0dc7854af1 add initial io_uring support 2022-06-21 09:59:46 +03:00
Saulius Valatka
3f755a2505 add liburing to build 2022-06-21 09:59:46 +03:00
Alexander Tokmakov
75b8800b5c
Merge pull request #37822 from ClickHouse/fix_flaky_tests_with_transactions 
Try fix flaky tests with transactions
 2022-06-21 09:38:25 +03:00
Larry Luo
bbd73ba727 use utility methods to access x509 struct fields. 2022-06-20 21:27:33 -04:00
Dmitry Novik
9ca368ac20 Use do while(false) in macro 2022-06-20 23:10:40 +00:00
Anton Popov
dcc9cd0367
fix style check 2022-06-20 22:50:39 +02:00
Maksim Kita
cb018348cf
Merge pull request #38022 from kitaisreal/sorting-added-batch-queue-variants 
Sorting added batch queue variants
 2022-06-20 22:35:44 +02:00
zvonand
22af00b757 rename variable + fix handling of ENABLE_LIBRARIES 2022-06-20 23:53:47 +05:00
Nikolai Kochetov
dccf90b1ea Cleanup. 2022-06-20 18:18:17 +00:00
mergify[bot]
9bdd9e14a6
Merge branch 'master' into fix_flaky_tests_with_transactions 2022-06-20 18:11:30 +00:00
Dmitry Novik
f6692c34e6
Merge pull request #34632 from excitoon-favorites/optimizedprocessing 
Optimized processing of ORDER BY in window functions
 2022-06-20 20:03:26 +02:00
Dmitry Novik
53d656d89f Fix deadlock in OvercommitTracker logging 2022-06-20 17:35:24 +00:00
Kseniia Sumarokova
2d039b739c
Update ReadBufferFromFileDescriptor.cpp 2022-06-20 19:13:20 +02:00
Kseniia Sumarokova
e48ce50863
Update ArrowBufferedStreams.cpp 2022-06-20 19:12:51 +02:00
Anton Popov
464cb59920 slightly better code 2022-06-20 16:22:36 +00:00
Anton Popov
59a9ce6def fix reading from s3 in some corner cases 2022-06-20 15:24:38 +00:00
kssenii
cea81822bd Fix 2022-06-20 17:19:36 +02:00
mergify[bot]
b440ee84ae
Merge branch 'master' into fix-short-circuit 2022-06-20 15:14:19 +00:00
zvonand
d4e5686b99 minor: fix message for base64 2022-06-20 20:13:09 +05:00
Robert Schulze
55b39e709d
Merge remote-tracking branch 'origin/master' into clang-tsa 2022-06-20 16:39:32 +02:00
zvonand
78d55d6f46 small fixes 2022-06-20 19:30:54 +05:00
Antonio Andelic
0c76fc1121 Fix style 2022-06-20 14:21:59 +00:00
Robert Schulze
5a4f21c50f
Support for Clang Thread Safety Analysis (TSA) 
- TSA is a static analyzer build by Google which finds race conditions
  and deadlocks at compile time.

- It works by associating a shared member variable with a
  synchronization primitive that protects it. The compiler can then
  check at each access if proper locking happened before. A good
  introduction are [0] and [1].

- TSA requires some help by the programmer via annotations. Luckily,
  LLVM's libcxx already has annotations for std::mutex, std::lock_guard,
  std::shared_mutex and std::scoped_lock. This commit enables them
  (--> contrib/libcxx-cmake/CMakeLists.txt).

- Further, this commit adds convenience macros for the low-level
  annotations for use in ClickHouse (--> base/defines.h). For
  demonstration, they are leveraged in a few places.

- As we compile with "-Wall -Wextra -Weverything", the required compiler
  flag "-Wthread-safety-analysis" was already enabled. Negative checks
  are an experimental feature of TSA and disabled
  (--> cmake/warnings.cmake). Compile times did not increase noticeably.

- TSA is used in a few places with simple locking. I tried TSA also
  where locking is more complex. The problem was usually that it is
  unclear which data is protected by which lock :-(. But there was
  definitely some weird code where locking looked broken. So there is
  some potential to find bugs.

*** Limitations of TSA besides the ones listed in [1]:

- The programmer needs to know which lock protects which piece of shared
  data. This is not always easy for large classes.

- Two synchronization primitives used in ClickHouse are not annotated in
  libcxx:
  (1) std::unique_lock: A releaseable lock handle often together with
      std::condition_variable, e.g. in solve producer-consumer problems.
  (2) std::recursive_mutex: A re-entrant mutex variant. Its usage can be
      considered a design flaw + typically it is slower than a standard
      mutex. In this commit, one std::recursive_mutex was converted to
      std::mutex and annotated with TSA.

- For free-standing functions (e.g. helper functions) which are passed
  shared data members, it can be tricky to specify the associated lock.
  This is because the annotations use the normal C++ rules for symbol
  resolution.

[0] https://clang.llvm.org/docs/ThreadSafetyAnalysis.html
[1] https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/42958.pdf
 2022-06-20 16:13:25 +02:00
Antonio Andelic
777ae72259 Merge branch 'master' into keeper-sequential-consistency-reads 2022-06-20 14:06:45 +00:00
Antonio Andelic
df31b562ec Use sync after reconnect to avoid stale reads 2022-06-20 14:05:51 +00:00
Nikolai Kochetov
39b02d4ab8 Continure resolving conflicts. 2022-06-20 15:39:05 +02:00
Nikolai Kochetov
7452d04e3a Merge branch 'master' into refactor-something-in-part-volumes 2022-06-20 15:31:02 +02:00
kssenii
5dd1bb2fd8 improvements for getFileSize 2022-06-20 15:22:56 +02:00
Nikolai Kochetov
ef8b967c3f Fix more tests. 2022-06-20 13:07:14 +00:00
Antonio Andelic
6c225bb04d
Update src/Common/ZooKeeper/ZooKeeperImpl.cpp 
Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
 2022-06-20 14:55:59 +02:00
Dmitry Novik
77177917bd
Merge pull request #37848 from ClickHouse/window-function-expression 
Support expressions with window functions
 2022-06-20 14:21:13 +02:00
Vitaly Baranov
06c4082a7c
Merge pull request #37358 from vitlibar/backup-improvements-6 
Backup Improvements 6
 2022-06-20 14:13:30 +02:00
alesapin
7ae775d2db Review fixes 2022-06-20 14:01:05 +02:00
alesapin
e3184d264b Merge branch 'master' into disk_transaction 2022-06-20 13:47:30 +02:00
mergify[bot]
4ed606a275
Merge branch 'master' into fix_flaky_tests_with_transactions 2022-06-20 10:17:51 +00:00
Kseniia Sumarokova
a756b4be27
Merge pull request #37391 from azat/insert-profile-events-fix 
Send profile events for INSERT queries (previously only SELECT was supported)
 2022-06-20 12:16:29 +02:00
Kseniia Sumarokova
c83594284d
Merge pull request #38205 from azat/fix-window-view 
(Window View is an experimental feature) Fix LOGICAL_ERROR for WINDOW VIEW with incorrect structure
 2022-06-20 11:47:45 +02:00
Nikolai Kochetov
fab62513aa
Merge pull request #38029 from ClickHouse/fix-possible-crash-after-removing-replica-in-distributed 
Fix possible crash in Distributed async insert in case of removing a replica from config.
 2022-06-20 11:38:35 +02:00
Vitaly Baranov
a6fc0dea4e Fix clang-tidy more. 2022-06-20 11:04:37 +02:00
Vladimir Chebotarev
aef6fe6008 Rebase fix. 2022-06-20 05:15:08 +03:00
Vladimir Chebotarev
bfb67ded2c Rebase fix. 2022-06-20 05:15:08 +03:00