Alexey Milovidov
3de747a6e2
Merge pull request #39746 from ClickHouse/cleanup-projection-setting
...
Cleanup usages of `allow_experimental_projection_optimization` setting, part 1
2022-08-01 19:21:30 +03:00
Robert Schulze
bf574b9154
Merge pull request #39760 from ClickHouse/bit-fiddling
...
Use std::popcount, ::countl_zero, ::countr_zero functions
2022-08-01 17:04:51 +02:00
Antonio Andelic
9ec27c0ab4
Merge pull request #39757 from ClickHouse/fix-rocksdb-filter-with-params
...
Use params correctly for filtering by key in EmbeddedRocksDB
2022-08-01 12:17:10 +02:00
Dmitry Novik
ab5a147065
Merge pull request #38725 from azat/fix-order-by-projection
...
Fix ORDER BY that matches projections ORDER BY
2022-08-01 00:32:59 +02:00
Nikolai Kochetov
22fbfe19a4
Merge branch 'master' into use-dag-in-key-condition
2022-07-31 21:54:12 +02:00
kssenii
e5f4a619ed
Merge master
2022-07-31 20:24:40 +03:00
Robert Schulze
a7734672b9
Use std::popcount, ::countl_zero, ::countr_zero functions
...
- Introduced with the C++20 <bit> header
- The problem with __builtin_c(l|t)z() is that 0 as input has an
undefined result (*) and the code did not always check. The std::
versions do not have this issue.
- In some cases, we continue to use buildin_c(l|t)z(), (e.g. in
src/Common/BitHelpers.h) because the std:: versions only accept
unsigned inputs (and they also check that) and the casting would be
ugly.
(*) https://gcc.gnu.org/onlinedocs/gcc/Other-Builtins.html
2022-07-31 15:16:51 +00:00
Robert Schulze
52d08d9db4
Merge pull request #39520 from ClickHouse/no-split-binary
...
Remove SPLIT_BINARY
2022-07-31 14:23:31 +02:00
Antonio Andelic
eea1aaef2c
Use params for filtering by key in EmbeddedRocksDB
2022-07-31 10:44:01 +00:00
Igor Nikonov
7f0adb5eb0
Merge remote-tracking branch 'origin/master' into skipping_sorting_step
2022-07-31 07:07:36 +00:00
Anton Popov
74eb8119ef
Merge branch 'master' into fix-custom-serializations
2022-07-31 01:51:52 +02:00
Alexey Milovidov
8fb70abe3e
Merge pull request #39178 from azat/dist-insert-log
...
Add connection info for Distributed sends log message
2022-07-31 02:22:22 +03:00
Alexey Milovidov
441f2feb49
Cleanup usages of allow_experimental_projection_optimization
setting, part 1
2022-07-30 23:40:21 +02:00
Igor Nikonov
a7cfad105e
Merge branch 'master' into distinct_sorted_simplify
2022-07-30 21:57:53 +02:00
Igor Nikonov
7b0b38e997
DistinctSortedTransform works only if columns contains sort prefix of
...
sort description
2022-07-29 20:01:07 +00:00
Kruglov Pavel
ccd1e1bdb8
Merge branch 'master' into fix-cancel-insert-into-function
2022-07-29 20:27:32 +02:00
Maksim Kita
8fc6bad4f4
Join enums refactoring
2022-07-29 18:35:05 +02:00
Alexander Tokmakov
6f88065cc1
check metadata consistency
2022-07-29 18:33:16 +02:00
Nikolai Kochetov
59a11b32ad
Merge branch 'master' into use-dag-in-key-condition
2022-07-29 17:01:33 +02:00
Igor Nikonov
fb5799fe7f
More generic getInputOrderInfo() in query info
2022-07-29 14:36:36 +00:00
Igor Nikonov
ae82bfb7f3
Merge remote-tracking branch 'origin/master' into distinct_sorted_simplify
2022-07-29 13:12:03 +00:00
Robert Schulze
3d1797f75f
Merge remote-tracking branch 'origin/master' into no-split-binary
2022-07-29 12:17:43 +00:00
Anton Popov
57e4fb2e30
Merge remote-tracking branch 'upstream/master' into HEAD
2022-07-29 11:42:11 +00:00
Anton Popov
bb551566e7
fix build
2022-07-29 11:41:53 +00:00
Azat Khuzhin
498c8b3c52
Fix clang-tidy in utils/examples
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-29 11:36:51 +03:00
Robert Schulze
199e254777
Merge remote-tracking branch 'origin/master' into no-split-binary
2022-07-28 15:54:22 +00:00
Anton Popov
04fd72cdef
fix alter column rename
2022-07-28 13:19:23 +00:00
Vitaly Baranov
f8ebe76826
Merge pull request #39503 from vitlibar/improve-system-backups-table
...
Improve system.backups table
2022-07-27 20:25:39 +02:00
Vladimir C
115506356c
Merge branch 'master' into refactor-prepared-sets
2022-07-27 19:57:23 +02:00
Nikolai Kochetov
6206e468a4
Ignor materialize function for ActionsDAG in PK analysis.
2022-07-27 17:02:48 +00:00
Nikolai Kochetov
51c4c9c957
Fixing build.
2022-07-27 15:45:23 +00:00
Anton Popov
56039c8780
fix alter of LowCardinality
2022-07-27 15:35:09 +00:00
Anton Popov
4cf0353892
try to fix Nested
2022-07-27 14:05:16 +00:00
Filatenkov Artur
a218597546
Merge branch 'master' into annoy-2
2022-07-27 15:37:58 +03:00
FArthur-cmd
6b5e92e68a
change index policy
2022-07-27 15:36:07 +03:00
Nikolai Kochetov
f9a8a4cd32
Fixing read-in-order for Merge
2022-07-27 12:00:55 +00:00
vdimir
8eecb9ef82
upd PreparedSets: rename/change signature of methods, add comments
2022-07-27 11:22:16 +00:00
Alexander Tokmakov
8fc075a527
Merge pull request #39619 from ClickHouse/try-do-enable-global-allocations-for-attach
...
Replace MemoryTrackerBlockerInThread to LockMemoryExceptionInThread
2022-07-27 13:13:04 +03:00
Kruglov Pavel
381ea139c2
Merge branch 'master' into schema-inference-cache
2022-07-27 11:35:36 +02:00
Vitaly Baranov
794eeb5d51
Split "total_size" to "uncompressed_size" and "compressed_size".
2022-07-27 10:36:56 +02:00
Vitaly Baranov
a79fc17235
Add test for system.backups
2022-07-27 09:04:10 +02:00
Vitaly Baranov
1cfe0b10f7
Add columns "total_size" and "num_files" to system.backups
2022-07-27 09:04:10 +02:00
Vitaly Baranov
35c267b3b1
Replace column "status_changed_time" with columns "start_time" and "end_time".
2022-07-27 09:04:10 +02:00
Vitaly Baranov
fc16a15ecf
Rename column "uuid" -> "id" in system.backups and allow user to set it in a query.
2022-07-27 09:04:10 +02:00
Vitaly Baranov
131019ba49
Rename column "backup_name" -> "name" in system.backups.
2022-07-27 09:04:10 +02:00
Vitaly Baranov
afd0982187
Remove column "internal" from system.backups
2022-07-27 09:04:10 +02:00
Alexander Gololobov
a64aa00869
Merge pull request #37893 from zhangjmruc/feature/sql-standard-delete
...
Support SQL standard "delete from ... where ..." syntax and lightweight implementation on merge tree tables
2022-07-26 23:39:07 +02:00
Nikolai Kochetov
1adb021df6
Fixing tests.
2022-07-26 18:42:27 +00:00
vdimir
5ce2960f03
Get rid of SelectQueryInfoBase -> SelectQueryInfo
2022-07-26 18:39:09 +00:00
vdimir
d9928ac93d
Add methods to SubqueryForSet, do not use refernce to SetPtr
2022-07-26 18:39:09 +00:00
vdimir
1e3fa2e01f
Refactor PreparedSets/SubqueryForSet
2022-07-26 18:39:02 +00:00
Anton Popov
9321ca34cf
try to fix Nested
2022-07-26 17:32:37 +00:00
Nikolai Kochetov
33ce662d3e
Replace MemoryTrackerBlockerInThread to LockMemoryExceptionInThread in some places. Reduced MemoryTrackerBlockerInThread level to User.
2022-07-26 15:22:00 +00:00
Kruglov Pavel
c683cb252f
Merge pull request #39227 from amosbird/rename-log1
...
Rename log when rename merge tree tables
2022-07-26 17:12:44 +02:00
Nikolai Kochetov
d7cc3831ea
Push predicate over StorageMerge.
2022-07-26 15:01:39 +00:00
Nikolai Kochetov
c5110bb164
Add a query plan step for StorageMerge
2022-07-26 14:43:05 +00:00
jianmei zhang
c1834d183b
ReplicatedMergeTree support lightweight delete
2022-07-26 11:50:09 +08:00
Alexander Gololobov
25deba2c1b
Merge branch 'master' into feature/sql-standard-delete
2022-07-25 22:13:20 +02:00
Nikolai Kochetov
be9c7ed52c
Add ReadFromMerge step.
2022-07-25 19:41:43 +00:00
Roman Vasin
b462366415
Merge branch 'master' of github.com:ClickHouse/ClickHouse into ADQM-485
2022-07-25 17:55:47 +00:00
Roman Vasin
b412ea5f6d
Improve generateRandom() for Date32; fix tests 01087_table_function_generate, 01277_fromUnixTimestamp64, 01691_DateTime64_clamp and 01702_toDateTime_from_string_clamping
2022-07-25 17:06:11 +00:00
Alexander Tokmakov
778dcf6994
Merge pull request #39230 from Algunenano/improve_replicated_merge_logging
...
Improve logging around replicated merges
2022-07-25 18:42:12 +03:00
Alexander Gololobov
48de02a7b8
Capitalized const name
2022-07-25 16:32:16 +02:00
Alexander Gololobov
7b44950d1d
More detailed comment
2022-07-25 16:14:33 +02:00
Nikolai Kochetov
b70be40804
Merge branch 'master' into use-dag-in-key-condition
2022-07-25 14:30:22 +02:00
Nikolai Kochetov
1ce8e714d3
Fixing build.
2022-07-25 12:30:01 +00:00
Alexey Milovidov
388d06fda1
Merge pull request #39535 from ClickHouse/stringref
...
Less usage of StringRef
2022-07-25 04:06:11 +03:00
Alexander Gololobov
460950ecdc
Merge branch 'master' into feature/sql-standard-delete
2022-07-24 21:27:22 +02:00
Robert Schulze
4333750985
Less usage of StringRef
...
... replaced by std::string_view, see #39262
2022-07-24 18:33:52 +00:00
Alexey Milovidov
0209012e5e
Merge pull request #39517 from pkit/pkit/fix_examples
...
fix `-DENABLE_EXAMPLES=1` in master
2022-07-24 17:41:23 +03:00
Igor Nikonov
eacc57b6a4
Simplify DistinctSorted
...
+ use ordinary Distinct for deduplication in MergeTree when no sorting provided
2022-07-24 14:26:25 +00:00
Azat Khuzhin
22d8e532ed
Fix ORDER BY that matches projections ORDER BY
...
In case of projection is seleted and it is comlete (all parts has such
projection) and ORDER BY matches projection ORDER BY, and
optimize_read_in_order=1 (default), then the sorting is simply not done,
because projections has separate plan, and so
InterpreterSelectQuery::executeOrder() -> SortingStep has
pipeline.getNumStreams() == 0 and it cannot do sorting.
Fix this, by adding sorting when creating plan for reading from
projections.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-24 15:23:13 +03:00
Kseniia Sumarokova
04944ff6f5
Merge pull request #39509 from monadbobo/remove_duplicate_code
...
Removed some duplicate code.
2022-07-24 14:24:40 +03:00
Alexander Gololobov
6692770113
Allow to update "system" virtual columns such as _row_exists
2022-07-24 12:21:47 +02:00
Alexander Gololobov
c8b3c574a4
Disable lightweight delete if table has projections
2022-07-24 12:21:47 +02:00
Alexander Gololobov
be64b45583
Properly handle empty parts list
2022-07-24 12:21:47 +02:00
Alexander Gololobov
594195451e
Cleanups
2022-07-24 12:21:18 +02:00
Alexey Milovidov
071374b152
Remove SPLIT_BINARY
2022-07-24 01:15:54 +02:00
Kseniia Sumarokova
f146183400
Merge pull request #39491 from ClickHouse/remove_unused_constructor
...
Remove unused constructor
2022-07-23 21:16:39 +02:00
mergify[bot]
c01ff2d38a
Merge branch 'master' into improve_replicated_merge_logging
2022-07-23 19:10:10 +00:00
Constantine Peresypkin
d645d03e89
fix -DENABLE_EXAMPLES=1
in master
2022-07-23 19:00:52 +02:00
Kseniia Sumarokova
77ddb8eda0
Merge pull request #39506 from CurtizJ/fix-direct-io
...
Fix reading from async read buffer with direct io
2022-07-23 06:08:48 +02:00
Anton Popov
3d03b2714b
try to fix Nested
2022-07-22 23:05:11 +00:00
Nikolai Kochetov
dd65a36c59
Fix another one test.
2022-07-22 19:11:25 +00:00
Nikolai Kochetov
53175e43f3
Better support for projections.
2022-07-22 19:11:25 +00:00
Simon Liu
2305f282dc
Removed some duplicate code.
2022-07-23 02:27:17 +08:00
Anton Popov
49627aa554
Merge remote-tracking branch 'upstream/master' into HEAD
2022-07-22 17:16:06 +00:00
Anton Popov
c1a5514e46
fix reading from async read buffer with direct io
2022-07-22 16:58:30 +00:00
Alexander Tokmakov
bed2206ae9
Merge pull request #39460 from ClickHouse/remove_some_dead_and_commented_code
...
Remove some dead and commented code
2022-07-22 13:24:34 +03:00
alesapin
69984b28e0
Remove unused constructor
2022-07-22 12:21:10 +02:00
Vadim Volodin
8f14c60932
remove exception for old partitions, add server configuration option to enable system unfreeze
2022-07-22 13:10:08 +03:00
Robert Schulze
99579ab440
Merge pull request #39364 from ClickHouse/stringref-to-string_view
...
Even less usage of StringRef
2022-07-22 09:30:49 +02:00
Alexander Tokmakov
0441d3629e
Merge pull request #39440 from liangliangpan/optimize-cluster-table-func
...
optimize select query from cluster table function
2022-07-21 20:40:31 +03:00
Mikhail f. Shiryaev
cf0b01b189
Update version to 22.8.1.1
2022-07-21 19:21:28 +02:00
Kruglov Pavel
9252f42b4c
Merge branch 'master' into schema-inference-cache
2022-07-21 18:59:14 +02:00
Robert Schulze
ea0a3bf600
Merge branch 'master' into stringref-to-string_view
2022-07-21 18:33:06 +02:00
zhangxiao871
5549b64c06
Fix issues 39469
2022-07-21 23:52:33 +08:00
Anton Popov
10719fd61d
fix style check
2022-07-21 15:06:55 +00:00
Anton Popov
e0d2c8fb37
fix json type with sparse columns
2022-07-21 14:47:19 +00:00
Vadim Volodin
85bf022258
review zookeeper and partition matching
2022-07-21 16:57:28 +03:00
Alexander Tokmakov
8bd674186a
Update getStructureOfRemoteTable.cpp
2022-07-21 16:15:42 +03:00
Alexander Tokmakov
a8da5d96fc
remove some dead and commented code
2022-07-21 15:05:48 +02:00
avogar
52b3a87ed1
Fix typo
2022-07-21 12:38:18 +00:00
avogar
6b541aa98f
Fix WriteBuffer finalize when cancel insert into function
2022-07-21 12:18:37 +00:00
Alexander Tokmakov
f79f12408d
fixes
2022-07-21 13:57:24 +02:00
Alexander Gololobov
1ea9f143ff
Leave only _row_exists-based implementation of lightweight delete
2022-07-21 11:26:13 +02:00
Nikolai Kochetov
e15967e9db
Merge pull request #38475 from ClickHouse/additional-filters
...
Additional filters for a table (from setting)
2022-07-21 07:52:04 +02:00
Mingliang Pan
e019b6ba69
Update getStructureOfRemoteTable.cpp
2022-07-21 12:11:33 +08:00
Alexey Milovidov
844042fc18
Merge pull request #39433 from ClickHouse/revert-39396-try-fix-write-buffer-terminate
...
Revert "Fix WriteBuffer finalize in destructor when cacnel query"
2022-07-21 07:04:07 +03:00
Mingliang Pan
3f76c8d7fd
optimize select query from cluster table function
...
Use local node as first priority to get Structure Of Remote Table.
we have many distributed queries( like select xx from cluster('xx',view (xxxx) ) on a clickhouse cluster. we found that the first node (shard_num=1) have 2 times of query number compared to other shards.
The reason is that the getStructureOfRemoteTableInShard func always take the first shard to execute "DESC TABLE xx" query.
The better way is to use local node as first priority which save the network rpc and reduce the pressure of first shard .
2022-07-21 11:56:35 +08:00
Alexey Milovidov
fd691000b7
Merge pull request #39342 from azat/bg-executor-fix
...
Fix possible UB in MergeTreeBackgroundExecutor (leads to SIGSEGV on race with DROP/DETACH)
2022-07-21 05:22:51 +03:00
Kruglov Pavel
92995a832b
Revert "Fix WriteBuffer finalize in destructor when cacnel query"
2022-07-21 01:45:16 +02:00
Nikolai Kochetov
91043351aa
Fixing build.
2022-07-20 20:30:16 +00:00
Nikolai Kochetov
91aa2385ca
Merge branch 'master' into additional-filters
2022-07-20 18:56:24 +02:00
Kruglov Pavel
0efed4644d
Merge pull request #38957 from Avogar/compatibility-setting
...
Add compatibility setting and system.settings_changes system table
2022-07-20 17:46:06 +02:00
Nikolai Kochetov
4e8cd70b1d
Merge branch 'master' into use-dag-in-key-condition
2022-07-20 17:38:33 +02:00
Dmitry Novik
a09422de7b
Merge pull request #38642 from ClickHouse/group-by-use-nulls
...
Add group_by_use_nulls setting
2022-07-20 14:42:42 +02:00
Kruglov Pavel
3046cd6d29
Merge branch 'master' into schema-inference-cache
2022-07-20 13:30:42 +02:00
Vladimir C
7694c72154
Merge pull request #38887 from bigo-sg/hive_file_crash
...
fixed: crash caused by IHiveFile be shared among threads
2022-07-20 12:15:57 +02:00
Kruglov Pavel
7722b647b7
Merge pull request #39396 from Avogar/try-fix-write-buffer-terminate
...
Fix WriteBuffer finalize in destructor when cacnel query
2022-07-20 12:06:20 +02:00
lgbo-ustc
4b46d80169
update codes
2022-07-20 18:03:34 +08:00
Alexander Tokmakov
988569bdb1
Merge pull request #39366 from ClickHouse/fix_race_in_replication_queue
...
Do not enqueue uneeded parts for check
2022-07-20 00:35:02 +03:00
Nikolai Kochetov
f570cde815
Fixing build.
2022-07-19 20:19:57 +00:00
avogar
5c16d6b553
Fix WriteBuffer finalize in destructor when cacnel query
2022-07-19 19:21:30 +00:00
Nikolai Kochetov
a00bcd389f
Fixing build
2022-07-19 19:33:14 +02:00
Nikolai Kochetov
eaeb30a71a
Merge branch 'master' into use-dag-in-key-condition
2022-07-19 18:39:52 +02:00
Dmitry Novik
50989bdb68
Merge branch 'master' into group-by-use-nulls
2022-07-19 14:58:01 +02:00
Alexander Tokmakov
a761da9f1b
add comment
2022-07-19 13:58:59 +02:00
kssenii
21ebf8874e
Merge remote-tracking branch 'upstream/master' into make-cache-composable
2022-07-19 13:36:42 +02:00
Alexander Tokmakov
3e7414b356
add comment
2022-07-19 12:47:20 +02:00
Nikolai Kochetov
d0db7135cb
Merge pull request #38946 from amosbird/nullable-partition-fix1
...
Fix incorrect partition pruning when there is a nullable partition
2022-07-19 12:14:36 +02:00
Kseniia Sumarokova
f168e0c5d0
Merge pull request #39272 from kssenii/fixes-for-materialized-pg
...
Fix segfault in materialised postgresql
2022-07-19 11:56:10 +02:00
Robert Schulze
81ef1099cc
Even less usage of StringRef
...
--> see #39300
2022-07-19 07:01:06 +00:00
FArthur-cmd
3b30cd9259
improvements
2022-07-19 00:59:07 +03:00
Alexander Tokmakov
d82f378a9d
do not enqueue uneeded parts for check
2022-07-18 23:37:07 +02:00
Alexander Gololobov
614cb9a87f
Disable trivial count() optimization if _row_exists column is present
2022-07-18 20:08:01 +02:00
Alexander Gololobov
ae0d00083c
Renamed __row_exists to _row_exists
2022-07-18 20:07:36 +02:00
Alexander Gololobov
f324ca9921
Cleanups
2022-07-18 20:07:22 +02:00
Alexander Gololobov
9de72d995a
POC lightweight delete using __row_exists virtual column and prewhere-like filtering
2022-07-18 20:06:42 +02:00
Robert Schulze
6df3c9d799
Merge pull request #39300 from ClickHouse/stringref-to-stringview
...
First try at reducing the use of StringRef
2022-07-18 19:20:16 +02:00
Azat Khuzhin
c6f7698f9a
Fix possible UB in MergeTreeBackgroundExecutor
...
Lambda erase_from_active() captures the item (TaskRuntimeDataPtr), most
of the code path is OK, since it explicitly reset the item->task.
However one is not, when it moves the item to pending list, which will
be cleaned up when the table will be DROP/DETACH'ed, from
MergeTreeBackgroundExecutor::removeTasksCorrespondingToStorage(),
and in this case if IStorage will be already removed, then it will lead
to use-after-free on destroying the lambda, since it captures the item
by value.
And I belive that CI founds this issue here [1]:
<details>
<summary>stack trace</summary>
4 0x268d1354 in DB::ReplicatedMergeTreeQueue::CurrentlyExecuting::~CurrentlyExecuting() build_docker/../src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1510:33
5 0x268ead1a in std::__1::default_delete<DB::ReplicatedMergeTreeQueue::CurrentlyExecuting>::operator()(DB::ReplicatedMergeTreeQueue::CurrentlyExecuting*) const build_docker/../contrib/libcxx/include/__memory/unique_ptr.h:54:5
6 0x268ead1a in std::__1::unique_ptr<DB::ReplicatedMergeTreeQueue::CurrentlyExecuting, std::__1::default_delete<DB::ReplicatedMergeTreeQueue::CurrentlyExecuting> >::reset(DB::ReplicatedMergeTreeQueue::CurrentlyExecuting*) build_docker/../contrib/libcxx/include/__memory/unique_ptr.h:315:7
7 0x268ead1a in std::__1::unique_ptr<DB::ReplicatedMergeTreeQueue::CurrentlyExecuting, std::__1::default_delete<DB::ReplicatedMergeTreeQueue::CurrentlyExecuting> >::~unique_ptr() build_docker/../contrib/libcxx/include/__memory/unique_ptr.h:269:19
8 0x268ead1a in DB::ReplicatedMergeTreeQueue::SelectedEntry::~SelectedEntry() build_docker/../src/Storages/MergeTree/ReplicatedMergeTreeQueue.h:351:12
9 0x268ead1a in void std::__1::__destroy_at<DB::ReplicatedMergeTreeQueue::SelectedEntry, 0>(DB::ReplicatedMergeTreeQueue::SelectedEntry*) build_docker/../contrib/libcxx/include/__memory/construct_at.h:56:13
...
16 0x265e9abb in DB::MergeTreeBackgroundExecutor<DB::MergeMutateRuntimeQueue>::routine(std::__1::shared_ptr<DB::TaskRuntimeData>)::'lambda'()::~() build_docker/../src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:127:30
17 0x265e9abb in DB::MergeTreeBackgroundExecutor<DB::MergeMutateRuntimeQueue>::routine(std::__1::shared_ptr<DB::TaskRuntimeData>) build_docker/../src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:235:1
18 0x265ea730 in DB::MergeTreeBackgroundExecutor<DB::MergeMutateRuntimeQueue>::threadFunction() build_docker/../src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:261:13
</details>
[1]: https://s3.amazonaws.com/clickhouse-test-reports/0/533c7888038453c047df816f3f65292cca05a54f/stateless_tests__ubsan__actions_.html
And I also I think that the following issue will also be fixed [2]:
<details>
<summary>stack trace</summary>
[ 680 ] {} <Fatal> : Logical error: 'Memory tracker: allocations not allowed.'.
...
[ 23430 ] {} <Fatal> BaseDaemon: 23. /build/build_docker/../src/Common/formatReadable.cpp:46: formatReadableSizeWithBinarySuffix(double, int) @ 0x1713bb98 in /usr/bin/clickhouse
...
[ 23430 ] {} <Fatal> BaseDaemon: 31. ../src/Common/MemoryTracker.cpp:112: MemoryTracker::logPeakMemoryUsage() @ 0x170e0ab9 in /usr/bin/clickhouse
[ 23430 ] {} <Fatal> BaseDaemon: 32. /build/build_docker/../src/Common/MemoryTracker.cpp:98: MemoryTracker::~MemoryTracker() @ 0x170e063a in /usr/bin/clickhouse
[ 23430 ] {} <Fatal> BaseDaemon: 33. /build/build_docker/../src/Storages/MergeTree/MergeList.cpp:144: DB::MergeListElement::~MergeListElement() @ 0x279fb290 in /usr/bin/clickhouse
...
[ 23430 ] {} <Fatal> BaseDaemon: 38. /build/build_docker/../src/Storages/MergeTree/BackgroundProcessList.h:41: DB::BackgroundProcessListEntry<DB::MergeListElement, DB::MergeInfo>::~BackgroundProcessListEntry() @ 0x276ce6c7 in /usr/bin/clickhouse
[ 23430 ] {} <Fatal> BaseDaemon: 39. /build/build_docker/../contrib/libcxx/include/__memory/unique_ptr.h:54: std::__1::default_delete<DB::BackgroundProcessListEntry<DB::MergeListElement, DB::MergeInfo> >::operator()(DB::BackgroundProcessListEntry<DB::MergeListElement, DB::MergeInfo>*) const @ 0x276ce60b in /usr/bin/clickhouse
[ 23430 ] {} <Fatal> BaseDaemon: 40. /build/build_docker/../contrib/libcxx/include/__memory/unique_ptr.h:316: std::__1::unique_ptr<DB::BackgroundProcessListEntry<DB::MergeListElement, DB::MergeInfo>, std::__1::default_delete<DB::BackgroundProcessListEntry<DB::MergeListElement, DB::MergeInfo> > >::reset(DB::BackgroundProcessListEntry<DB::MergeListElement, DB::MergeInfo>*) @ 0x276ce57c in /usr/bin/clickhouse
[ 23430 ] {} <Fatal> BaseDaemon: 41. /build/build_docker/../contrib/libcxx/include/__memory/unique_ptr.h:269: std::__1::unique_ptr<DB::BackgroundProcessListEntry<DB::MergeListElement, DB::MergeInfo>, std::__1::default_delete<DB::BackgroundProcessListEntry<DB::MergeListElement, DB::MergeInfo> > >::~unique_ptr() @ 0x276ce399 in /usr/bin/clickhouse
[ 23430 ] {} <Fatal> BaseDaemon: 42. /build/build_docker/../src/Storages/MergeTree/MutatePlainMergeTreeTask.h:22: DB::MutatePlainMergeTreeTask::~MutatePlainMergeTreeTask() @ 0x27defceb in /usr/bin/clickhouse
[ 23430 ] {} <Fatal> BaseDaemon: 43. /build/build_docker/../contrib/libcxx/include/__memory/construct_at.h:57: void std::__1::__destroy_at<DB::MutatePlainMergeTreeTask, 0>(DB::MutatePlainMergeTreeTask*) @ 0x27dd69c1 in /usr/bin/clickhouse
[ 23430 ] {} <Fatal> BaseDaemon: 44. /build/build_docker/../contrib/libcxx/include/__memory/construct_at.h:82: void std::__1::destroy_at<DB::MutatePlainMergeTreeTask, 0>(DB::MutatePlainMergeTreeTask*) @ 0x27dd6955 in /usr/bin/clickhouse
[ 23430 ] {} <Fatal> BaseDaemon: Integrity check of the executable skipped because the reference checksum could not be read. (calculated checksum: 91F5937571C11255DFE73230B52CE9C0)
[ 602 ] {} <Fatal> Application: Child process was terminated by signal 6.
</details>
[2]: https://s3.amazonaws.com/clickhouse-test-reports/39222/a068c397dfd7943359a8b554566c3c70b78baf8d/stateless_tests__debug__actions__%5B1/3%5D.html
Refs: https://github.com/ClickHouse/ClickHouse/pull/29614#discussion_r720455032 (cc @nikitamikhaylov)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-18 19:01:41 +03:00
kssenii
0aed5a602a
Merge master
2022-07-18 16:16:28 +02:00
Kseniia Sumarokova
b4eae94889
Merge pull request #39259 from rnbondarenko/patch-1
...
removed default queue arguments
2022-07-18 15:32:31 +02:00
avogar
80b5bc5236
Add system table settings_changes
2022-07-18 13:19:41 +00:00
Anton Popov
5605b60c97
Merge pull request #39039 from CurtizJ/randomize-more-settings
...
Add more settings for randomization
2022-07-18 15:05:41 +02:00
kssenii
a6655bea1c
Some fixes
2022-07-18 14:23:08 +02:00
Robert Schulze
1ced1d11fd
Merge pull request #39285 from azat/bg-exe
...
[RFC] Remove superior atomic from MergeTreeBackgroundExecutor and annotations for TSA
2022-07-18 12:46:54 +02:00
Kseniia Sumarokova
eb23491957
Merge branch 'master' into fixes-for-materialized-pg
2022-07-18 12:00:58 +02:00
Alexey Milovidov
13a8f85ed0
Merge branch 'master' into nullable-partition-fix1
2022-07-18 00:34:58 +03:00
Robert Schulze
13482af4ee
First try at reducing the use of StringRef
...
- to be replaced by std::string_view
- suggested in #39262
2022-07-17 17:26:02 +00:00
kssenii
8ad26b3111
Merge master
2022-07-16 22:50:03 +02:00
Vitaly Baranov
e0fb03d1b6
Merge pull request #38970 from vitlibar/select-from-system-db-requires-grant
...
SELECT from the system database requires grant now
2022-07-16 22:37:47 +02:00
Kseniia Sumarokova
3f43641b18
Update MergedColumnOnlyOutputStream.cpp
2022-07-16 16:32:33 +02:00
Azat Khuzhin
accb0e38da
Remove superior atomic from MergeTreeBackgroundExecutor and annotations for TSA
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-16 14:00:39 +03:00
kssenii
9f1d6814fc
Fix
2022-07-15 18:01:54 +02:00
alesapin
5cf4bd50de
Merge pull request #39214 from azat/s3-fix-fetching-memory-part
...
Fix fetch of in-memory part with allow_remote_fs_zero_copy_replication
2022-07-15 17:52:05 +02:00
Alexander Tokmakov
5ef7f5f494
Merge pull request #39229 from azat/fix-noexcept-scope
...
[RFC] Fix NOEXCEPT_SCOPE (before it calls std::terminate and looses the exception)
2022-07-15 18:14:56 +03:00
Raúl Marín
d68b3dfd43
Fix destructor order
2022-07-15 15:48:35 +02:00
Raúl Marín
aea045f297
Improve logging around replicated merges
2022-07-15 15:48:35 +02:00
Vitaly Baranov
c7cef91d4d
Two ways to provide access to system.users: "GRANT SHOW USERS ON *.*" and "GRANT SELECT ON system.users"
2022-07-15 15:44:29 +02:00
Vitaly Baranov
fbb2e14d54
Add new table function viewIfPermitted().
2022-07-15 15:44:29 +02:00
Anton Popov
76eec62469
Merge branch 'master' into randomize-more-settings
2022-07-15 14:12:49 +02:00
Kruglov Pavel
a944a92d4c
Merge pull request #39224 from Avogar/string-view-by-value
...
Pass const std::string_view by value, not by reference
2022-07-15 13:05:56 +02:00
rnbondarenko
a7fe1cf5f0
removed default queue arguments
...
declare queue without default arguments x-max-length and x-overflow.
2022-07-15 13:46:52 +03:00
Alexander Tokmakov
11ea455632
Merge pull request #39245 from ClickHouse/tavplubix-patch-1
...
Remove incorrect assertion
2022-07-15 13:26:14 +03:00
jianmei zhang
197d3796ec
Killed mutation has empty commands may cause SegV
2022-07-15 15:37:36 +08:00
jianmei zhang
ca42f649da
Rewrite logic for loading deleted mask related to getDeletedMask()
2022-07-15 15:31:10 +08:00
jianmei zhang
0793add9b5
Fix clang tidy compile error, remove std::move for const getDeletedRowsPtr()
2022-07-15 12:32:42 +08:00
jianmei zhang
8b1114fa60
fix clang tidy compile error
2022-07-15 12:32:42 +08:00
jianmei zhang
d37152a5d6
Remove loadDeletedMask() and get deleted mask when needed
2022-07-15 12:32:42 +08:00
jianmei zhang
4268ac8b0e
Add supportLightweightDeleteMutate() in IMergeTreeDataPart to disable LWD for part with projections
2022-07-15 12:32:42 +08:00
jianmei zhang
9d1dff515c
code changes for strange name and adding exception for mutation entry
2022-07-15 12:32:42 +08:00
jianmei zhang
198674e994
Consider lightweight delete as ordinary for compact part in selectPartsToMutate()
2022-07-15 12:32:42 +08:00
jianmei zhang
780cdfb8f0
Code changes based on review opinions. Make ordinary single alter delete lightwight by default
2022-07-15 12:32:42 +08:00
jianmei zhang
6e6f77ef8a
Fix compile error in clang tidy build
2022-07-15 12:32:41 +08:00
jianmei zhang
7e433859ea
Change deleted rows mask from String to Native UInt8 format
2022-07-15 12:32:41 +08:00
jianmei zhang
dcc7367ac4
Fix code style error
2022-07-15 12:32:41 +08:00
jianmei zhang
6248486790
Fix lightweight delete bugs: add skip files and use source part's columns to avoid metadata updated cases
2022-07-15 12:32:41 +08:00
jianmei zhang
8ad2bb7c33
Code changes due to master new fixes, and update reference for mutations table
2022-07-15 12:32:41 +08:00
jianmei zhang
11fdea6e4b
Add missing code for deleted_mask_filter_holder
2022-07-15 12:32:41 +08:00
jianmei zhang
9d27af7ee2
For some columns mutations, skip to apply deleted mask when read some columns. Also add unit test case
2022-07-15 12:32:41 +08:00
jianmei zhang
2c74e9b866
Merge support to apply lightweight delete mask
2022-07-15 12:32:41 +08:00
jianmei zhang
b4a37e1e22
Disable optimizations for count() when lightweight delete exists, add hasLightweightDelete() function in IMergeTreeDataPart
2022-07-15 12:32:41 +08:00
jianmei zhang
8696319d62
Support lightweight delete execution using string as deleted rows mask,also part of select can handle LWD
2022-07-15 12:32:41 +08:00
jianmei zhang
8df7b7a030
Support new mutation type for lightweight
2022-07-15 12:32:41 +08:00
Sergei Trifonov
62f975da60
Merge pull request #39095 from ClickHouse/remote-io-bandwidth-limits
...
add max_remote_{read,write}_network_bandwidth_for_server settings
2022-07-14 23:14:53 +02:00
Alexander Tokmakov
435d98738d
Update ReplicatedMergeTreeQueue.cpp
2022-07-14 22:41:01 +03:00
avogar
9291d33080
Pass const std::string_view & by value, not by reference
2022-07-14 16:11:57 +00:00
Yakov Olkhovskiy
1f09303d54
Merge pull request #39181 from azat/part_log-merge_algorithm
...
Add merge_algorithm to system.part_log
2022-07-14 11:43:04 -04:00
Sergei Trifonov
b2317ff710
renaming to be consistent
2022-07-14 17:33:22 +02:00
Anton Popov
d4cb2184c2
Merge branch 'master' into randomize-more-settings
2022-07-14 17:01:07 +02:00
Nikolai Kochetov
b24eeedf53
Merge branch 'master' into nullable-partition-fix1
2022-07-14 16:41:38 +02:00
Yakov Olkhovskiy
9ecf19c883
Merge pull request #39212 from azat/stack_trace-filter-v2
...
Optimize accesses to system.stack_trace (filter by name before sending signal)
2022-07-14 10:33:26 -04:00
Azat Khuzhin
b20407fab9
Fix NOEXCEPT_SCOPE (before it calls std::terminate and looses the exception)
...
Current implementation of NOEXCEPT_SCOPE will not work, you cannot
rethrow exception outside the catch block, this will simply terminate
(via std::terminate) the program.
In other words NOEXCEPT_SCOPE macro will simply call std::terminate on
exception and will lost original exception.
But if NOEXCEPT_SCOPE will accept the code that should be runned w/o
exceptions, then it can catch exception and log it, rewrite it in this
way.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-14 17:16:18 +03:00
Amos Bird
96bb6e0cd2
Rename log when rename merge tree tables
2022-07-14 21:22:46 +08:00
Anton Popov
51533adee8
fix build
2022-07-14 11:33:23 +00:00
FArthur-cmd
ad459bc6e4
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into annoy-2
2022-07-14 14:10:44 +03:00
Sergei Trifonov
8a693469f0
Merge branch 'master' into remote-io-bandwidth-limits
2022-07-14 12:38:35 +02:00
Kruglov Pavel
b38241b08a
Merge branch 'master' into schema-inference-cache
2022-07-14 12:29:54 +02:00
Kseniia Sumarokova
530dac6487
Merge pull request #37213 from aaapetrenko/master
...
add TableFunctionMongoDB and tests
2022-07-14 12:09:07 +02:00
Azat Khuzhin
531e3b13ce
Optimize accesses to system.stack_trace (filter by name before sending signal)
...
Because of failed rebase/push this part of the patch was lost, so
resending separatelly (sigh).
Follow-up for: #39177 (cc @yakov-olkhovskiy)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-14 11:49:34 +03:00
Azat Khuzhin
3e77b80b88
Fix fetch of in-memory part with allow_remote_fs_zero_copy_replication
...
CI founds the following error during trying to fetch in-memory part [1]:
2022.07.13 08:06:54.231033 [ 141093 ] {} <Error> InterserverIOHTTPHandler: Code: 107. DB::ErrnoException: Cannot open file /var/lib/clickhouse/disks/s3/store/886/88635b40-e4e3-4fe6-a0a0-1b6755463358/all_0_0_0/data.bin, errno: 2, strerror: No such file or directory. (FILE_DOESNT_EXIST), Stack trace (when copying this message, always include the lines below):
<details>
<summary>stacktrace</summary>
2022.07.13 08:06:54.101825 [ 189342 ] {8dbd11b3-f38a-4d5d-9ded-148987adb71d} <Debug> executeQuery: (from [::1]:54570) (comment: 01643_replicated_merge_tree_fsync_smoke.sql) select 'memory in_memory_parts_insert_sync'; (stage: Complete)
2022.07.13 08:06:54.131309 [ 691 ] {} <Debug> test_26u6kx.rep_fsync_r2 (39c3823c-22e5-4c05-9dec-cdffd8872c40): Fetching part all_0_0_0 from /clickhouse/tables/test_26u6kx/rep_fsync/replicas/r1
2022.07.13 08:06:54.231033 [ 141093 ] {} <Error> InterserverIOHTTPHandler: Code: 107. DB::ErrnoException: Cannot open file /var/lib/clickhouse/disks/s3/store/886/88635b40-e4e3-4fe6-a0a0-1b6755463358/all_0_0_0/data.bin, errno: 2, strerror: No such file or directory. (FILE_DOESNT_EXIST), Stack trace (when copying this message, always include the lines below):
0. DB::Exception::Exception() @ 0xba0191a in /usr/bin/clickhouse
1. DB::throwFromErrnoWithPath() @ 0xba029ca in /usr/bin/clickhouse
2. DB::OpenedFile::open() const @ 0x156e7fb0 in /usr/bin/clickhouse
3. DB::OpenedFile::getFD() const @ 0x156e8003 in /usr/bin/clickhouse
4. DB::ReadBufferFromFilePReadWithDescriptorsCache::ReadBufferFromFilePReadWithDescriptorsCache() @ 0x156e5f23 in /usr/bin/clickhouse
5. ? @ 0x156e53f0 in /usr/bin/clickhouse
6. DB::createReadBufferFromFileBase() @ 0x156e52b5 in /usr/bin/clickhouse
7. DB::DiskLocal::readFile() const @ 0x15e45ea8 in /usr/bin/clickhouse
8. DB::MetadataStorageFromDisk::readFileToString() const @ 0x15e6ab8b in /usr/bin/clickhouse
9. DB::MetadataStorageFromDisk::readMetadataUnlocked() const @ 0x15e6cdeb in /usr/bin/clickhouse
10. DB::MetadataStorageFromDisk::getSerializedMetadata() const @ 0x15e6cfc4 in /usr/bin/clickhouse
11. DB::DiskObjectStorage::getSerializedMetadata() const @ 0x15e19e2e in /usr/bin/clickhouse
12. DB::DiskDecorator::getSerializedMetadata() const @ 0x15e54ed1 in /usr/bin/clickhouse
13. DB::DiskDecorator::getSerializedMetadata() const @ 0x15e54ed1 in /usr/bin/clickhouse
14. DB::DataPartsExchange::Service::sendPartFromDiskRemoteMeta() @ 0x1700bb9e in /usr/bin/clickhouse
15. DB::DataPartsExchange::Service::processQuery(DB::HTMLForm const&, DB::ReadBuffer&, DB::WriteBuffer&, DB::HTTPServerResponse&) @ 0x1700a649 in /usr/bin/clickhouse
16. DB::InterserverIOHTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTTPServerResponse&, DB::InterserverIOHTTPHandler::Output&) @ 0x17433c53 in /usr/bin/clickhouse
17. DB::InterserverIOHTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0x174344f1 in /usr/bin/clickhouse
18. DB::HTTPServerConnection::run() @ 0x1768714d in /usr/bin/clickhouse
19. Poco::Net::TCPServerConnection::start() @ 0x1a398093 in /usr/bin/clickhouse
20. Poco::Net::TCPServerDispatcher::run() @ 0x1a399411 in /usr/bin/clickhouse
21. Poco::PooledThread::run() @ 0x1a54b7bb in /usr/bin/clickhouse
22. Poco::ThreadImpl::runnableEntry(void*) @ 0x1a548ec0 in /usr/bin/clickhouse
23. ? @ 0x7fdf1c204609 in ?
24. clone @ 0x7fdf1c129133 in ?
(version 22.7.1.1781 (official build))
</details>
[1]: https://s3.amazonaws.com/clickhouse-test-reports/0/8b6e31cc615ca52c80724b6e5097777cb9514f07/stateless_tests__release__s3_storage__actions_.html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-14 11:43:35 +03:00
Yakov Olkhovskiy
d6a09acb38
Merge pull request #39177 from azat/stack_trace-filter
...
Optimize accesses to system.stack_trace
2022-07-14 01:47:14 -04:00
Anton Popov
089009c996
fix reading from StorageLog with mmap
2022-07-13 20:36:18 +00:00
Sergei Trifonov
15ab3bc99f
use context->getWriteSettings()
2022-07-13 19:48:57 +02:00
Sergei Trifonov
af4a5a5f3a
fix conflicts
2022-07-13 19:44:29 +02:00
kssenii
615221f73b
Minor changes
2022-07-13 19:22:47 +02:00
Sergei Trifonov
f854507729
merge master and resolve conflicts
2022-07-13 19:07:28 +02:00
Kseniia Sumarokova
71aba5b4c6
Merge pull request #38860 from kssenii/object-storages-refactor
...
Refactoring of code around object storages, added LocalObjectStorage (extracted this diff from PR #36171 )
2022-07-13 16:51:56 +02:00
kssenii
0944183eec
Fix merge
2022-07-13 16:50:31 +02:00
Azat Khuzhin
3720a3843c
Add merge_algorithm to system.part_log
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-13 17:16:25 +03:00
Azat Khuzhin
4f25a08b7c
Add connection info for Distributed sends log message
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-13 16:04:12 +03:00
Anton Popov
54a0985c42
randomize one more setting
2022-07-13 12:50:12 +00:00
Azat Khuzhin
8122c1b07f
Optimize accesses to system.stack_trace
...
- filter by thread_id
- filter by thread_name
- filter by requested columns (query_id/trace, thread_name)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-13 15:17:09 +03:00
FArthur-cmd
8ddc2578a2
refacroring
2022-07-13 14:48:02 +03:00
kssenii
0d2d3ac2b4
Merge master
2022-07-13 13:26:15 +02:00
kssenii
83a09fd961
Merge remote-tracking branch 'petrenko/master' into petrenko-master
2022-07-13 11:28:52 +02:00
kssenii
ae324622d3
Add one more test
2022-07-13 11:27:43 +02:00
Amos Bird
0795cdd36a
Slightly better interface of waitForMutation
2022-07-13 12:34:12 +08:00
Dmitry Novik
5f65b45269
Merge branch 'master' into group-by-use-nulls
2022-07-12 22:36:04 +02:00
kssenii
6f29eada80
Merge master
2022-07-12 19:28:18 +02:00
Kseniia Sumarokova
0fb35b773f
Update StorageMongoDB.cpp
2022-07-12 18:37:19 +02:00
Kseniia Sumarokova
5468a2fde9
Update StorageMongoDB.cpp
2022-07-12 17:20:20 +02:00
Kseniia Sumarokova
869da0d923
Update StorageMongoDB.cpp
2022-07-12 17:19:49 +02:00
kssenii
f46d641440
Fix style check and black check
...
t:wq# Date: Tue Jul 12 14:02:09 2022 +0200
2022-07-12 16:29:11 +02:00
Nikolai Kochetov
8efbe6d44d
Merge pull request #39119 from ClickHouse/removed-projection-and-old-part
...
Properly remove projection from part in case it was removed from table metadata.
2022-07-12 16:24:30 +02:00
Robert Schulze
69de9ee0e8
Merge pull request #39108 from ClickHouse/prefix_cmake_commands
...
Prefix overridden add_executable() command with "clickhouse_"
2022-07-12 15:05:48 +02:00
FArthur-cmd
6e2812490b
refactoring
2022-07-12 14:29:11 +03:00
Alexander Tokmakov
c99fe6e6c8
Merge branch 'master' into removed-projection-and-old-part
2022-07-12 14:00:55 +03:00
FArthur-cmd
0e89fbff53
refactoring
2022-07-12 13:57:16 +03:00
Alexander Tokmakov
b35ad0dbdc
Merge pull request #39091 from ClickHouse/tavplubix-patch-1
...
Follow-up to #39036
2022-07-12 13:47:37 +03:00
Nikolai Kochetov
945b5d4b3f
Merge pull request #38993 from azat/fix-fsync_part_directory-on-fetch
...
Fix fsync_part_directory for fetches
2022-07-12 11:48:42 +02:00
Nikolai Kochetov
cd4211298e
Fix typo.
2022-07-12 09:37:00 +00:00
Nikolai Kochetov
03d9301425
Fix log.
2022-07-12 08:31:09 +00:00
Nikolai Kochetov
631be9219b
Properly remove projection from part in case it was removed from table metadata.
2022-07-12 08:28:22 +00:00
FArthur-cmd
783e6e398c
merge with other working branch
2022-07-11 23:02:58 +03:00
aaapetrenko
aabf0ed67f
refactor Storage
2022-07-11 22:00:57 +02:00
FArthur-cmd
6ae227a6bb
add test
2022-07-11 22:50:10 +03:00
Robert Schulze
1a7727a254
Prefix overridden add_executable() command with "clickhouse_"
...
A simple HelloWorld program with zero includes except iostream triggers
a build of ca. 2000 source files. The reason is that ClickHouse's
top-level CMakeLists.txt overrides "add_executable()" to link all
binaries against "clickhouse_new_delete". This links against
"clickhouse_common_io", which in turn has lots of 3rd party library
dependencies ... Without linking "clickhouse_new_delete", the number of
compiled files for "HelloWorld" goes down to ca. 70.
As an example, the self-extracting-executable needs none of its current
dependencies but other programs may also benefit.
In order to restore access to the original "add_executable()", the
overriding version is now prefixed. There is precedence for a
"clickhouse_" prefix (as opposed to "ch_"), for example
"clickhouse_split_debug_symbols". In general prefixing makes sense also
because overriding CMake commands relies on undocumented behavior and is
considered not-so-great practice (*).
(*) https://crascit.com/2018/09/14/do-not-redefine-cmake-commands/
2022-07-11 19:36:18 +02:00
Sergei Trifonov
6ed777d2a8
add azure support
2022-07-11 19:19:57 +02:00
Sergei Trifonov
add0d1698e
respect new limits in HDFS
2022-07-11 17:30:23 +02:00
FArthur-cmd
868628d3fd
make stable prototype with arrays support
2022-07-11 18:02:50 +03:00
Sergei Trifonov
43779ec280
add max_remote_{read,write}_network_bandwidth_for_server settings
2022-07-11 14:59:39 +02:00
Alexander Tokmakov
e9ddbc247f
Merge pull request #39016 from Algunenano/clearOldLogs_zkerror
...
clearOldLogs: Don't report KEEPER_EXCEPTION on concurrent deletes
2022-07-11 15:16:18 +03:00
Alexander Tokmakov
048d27189e
Update MergeTreeData.cpp
2022-07-11 15:07:31 +03:00
FArthur-cmd
7b7a9521c7
change submodule path
2022-07-10 18:10:47 +03:00
FArthur-cmd
2f9f5f172a
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into annoy_3
2022-07-10 17:25:21 +03:00
kssenii
4c88527c8e
Review fixes
2022-07-09 19:35:07 +02:00
kssenii
bf1c99daab
Refactoring of object storages, add LocalObjectStorage
2022-07-09 12:39:31 +02:00
Frank Chen
da57a993e4
Fix CI
2022-07-09 13:43:10 +08:00
Alexander Tokmakov
51130d723c
Update MergeTreeData.cpp
2022-07-08 20:17:29 +03:00
Raúl Marín
a965438eea
clearOldLogs: Don't report errors on concurrent deletes
2022-07-08 13:57:53 +02:00
Azat Khuzhin
b2f7f9c0f0
Fix fsync_part_directory for fetches
...
Before it produces the following error:
2022.07.07 23:40:03.131044 [ 44869 ] {} <Error> default.rep_data: auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(DB::StorageReplicatedMergeTree::LogEntryPtr &) const: Code: 107. DB::ErrnoException: Cannot open file /var/lib/clickhouse/data/tmp-fetch_36_25458_25458_0, errno: 2, strerror: No such file or directory. (FILE_DOESNT_EXIST), Stack trace (when copying this message, always include the lines below):
0. Poco::Exception::Exception() @ 0x1764634c in /usr/bin/clickhouse
1. DB::Exception::Exception() @ 0xb42caba in /usr/bin/clickhouse
2. DB::throwFromErrnoWithPath() @ 0xb42d79b in /usr/bin/clickhouse
3. DB::LocalDirectorySyncGuard::LocalDirectorySyncGuard() @ 0x1404bbd4 in /usr/bin/clickhouse
4. DB::DiskLocal::getDirectorySyncGuard() const @ 0x140466a4 in /usr/bin/clickhouse
5. DB::DataPartsExchange::Fetcher::downloadPartToDisk() @ 0x15122af1 in /usr/bin/clickhouse
6. DB::DataPartsExchange::Fetcher::fetchPart() @ 0x1511de39 in /usr/bin/clickhouse
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-08 13:48:51 +03:00
lgbo-ustc
e01b6cf8a5
update codes
2022-07-08 09:25:45 +08:00
Alexander Tokmakov
b0f8970d07
Merge pull request #38961 from Algunenano/reduce_zk_eexists
...
Stop reporting Zookeeper "Node exists" exceptions in system.errors when they are expected
2022-07-07 23:16:46 +03:00
Dmitry Novik
d1df66687b
Merge branch 'master' into group-by-use-nulls
2022-07-07 20:54:38 +02:00
Raúl Marín
9b0c7714e5
Style
2022-07-07 17:15:23 +02:00
Kseniia Sumarokova
48220f64d6
Merge pull request #38214 from bigo-sg/storagehive_alter
...
support alter on `StorageHive` table
2022-07-07 17:06:43 +02:00
Raúl Marín
c3935000a8
Avoid reporting ZK EEXIST errors on checkPartChecksumsAndCommit
2022-07-07 16:10:05 +02:00
Raúl Marín
33ffb71453
Avoid 'Node exists' errors when creating Ephemeral Locks
2022-07-07 16:10:04 +02:00
Alexander Tokmakov
fdd2db4ace
Move check for denied allocations ( #38858 )
...
* move check for denied allocations
* remove include
* make hardening softer
2022-07-07 14:24:36 +02:00
alesapin
1bd97fa6fa
Merge pull request #38851 from Algunenano/rmarin_read_only_on_drop
...
Deactivate mutations_finalizing_task during shutdown
2022-07-07 12:55:50 +02:00
Frank Chen
93dc109e36
Fix code
2022-07-07 17:44:19 +08:00
Frank Chen
d3d89f59ca
Add tracing support to distributed insert
2022-07-07 17:43:09 +08:00
Frank Chen
57a7e4a7c9
Remove old API reference
2022-07-07 17:42:35 +08:00
FArthur-cmd
8968ff8995
small inmprovements
2022-07-07 11:33:06 +03:00
Amos Bird
19bd3d8d9f
Nullable partition pruning fix
2022-07-07 16:02:38 +08:00
lgbo-ustc
337f379bc3
update codes
2022-07-07 09:45:49 +08:00
lgbo-ustc
cda081dc57
update codes
2022-07-07 09:41:52 +08:00
Vitaly Baranov
ed27987646
Merge pull request #38861 from vitlibar/backup-improvements-9
...
Backup Improvements 9
2022-07-07 02:24:47 +02:00
FArthur-cmd
a3407ba6db
Merge branch 'master' of github.com:FArthur-cmd/ClickHouse into annoy-2
2022-07-06 19:42:55 +03:00
Vitaly Baranov
7f84cf3968
Fix style.
2022-07-06 16:36:59 +02:00
Maksim Kita
b94489d52c
Merge pull request #38859 from kitaisreal/merge-tree-merge-disable-batch-optimization
...
MergeTree merge disable batch optimization
2022-07-06 15:59:40 +02:00
Vladimir C
3c7fdb604d
Merge pull request #35363 from vdimir/kv-join
2022-07-06 15:57:30 +02:00
FArthur-cmd
c94d6a57ec
add arrays
2022-07-06 14:39:49 +03:00
Raúl Marín
86d1c319f4
Remove unnecessary log
2022-07-06 12:23:45 +02:00
lgbo-ustc
aeb4f9c6be
fixed: crash caused by IHiveFile be shared among threads
2022-07-06 16:56:17 +08:00
lgbo-ustc
946a9b8c63
update codes
2022-07-06 09:26:02 +08:00
Kseniia Sumarokova
a26643a7a6
Merge pull request #38363 from kssenii/pg-add-auto-close
...
Support `auto_close` option for postgres engine connection
2022-07-06 01:10:05 +02:00
kssenii
e55169ebad
Fix
2022-07-05 20:15:35 +02:00
Vitaly Baranov
f9204315b5
Store columns.txt in backups for the Memory table engine too.
2022-07-05 19:03:20 +02:00
Kseniia Sumarokova
3e803006a2
Merge pull request #37171 from tchepavel/nats-integration
...
Nats Integration
2022-07-05 17:47:02 +02:00
Maksim Kita
bdc21737d5
MergeTree merge disable batch optimization
2022-07-05 16:15:00 +02:00
Vitaly Baranov
43d35eec1b
Write unfinished mutations to backup.
2022-07-05 14:51:09 +02:00
Raúl Marín
9e17ae365b
Deactivate mutations_finalizing_task during shutdown
2022-07-05 14:28:20 +02:00
vdimir
17903117d2
better
2022-07-05 10:46:08 +00:00
kssenii
7a4be3b748
Fix build, disable for freebsd
2022-07-05 12:10:49 +02:00
Kruglov Pavel
a6a9824261
Merge pull request #38362 from Avogar/better-exception-messages
...
Better exception messages on wrong table engines/functions argument types
2022-07-05 11:21:44 +02:00
lgbo-ustc
76813ee3c0
update codes
2022-07-05 11:49:55 +08:00
kssenii
b52084265c
Merge master
2022-07-04 21:37:43 +02:00
Kseniia Sumarokova
6c3d648f49
Merge branch 'master' into nats-integration
2022-07-04 21:06:07 +02:00
Kseniia Sumarokova
0cd228cab3
Merge pull request #38365 from kssenii/better-rabbit-adddress-setting
...
Fix RabbitMQ configuration via connection string setting
2022-07-04 20:53:43 +02:00
vdimir
c0cb588e45
Fix build
2022-07-04 17:28:34 +00:00
vdimir
43afda408b
do not call setStorageJoin when setting isn't set
2022-07-04 17:28:15 +00:00
vdimir
65ac97ce89
direct join: handle nullable, add tests
2022-07-04 17:28:14 +00:00
vdimir
8359753a85
Move DirectJoin to separate file
2022-07-04 17:28:11 +00:00
vdimir
5ed0911ff4
Fix rocksdb::getSlicedKeys
2022-07-04 17:28:10 +00:00
vdimir
085e70c7a3
wip left kvjoin
2022-07-04 17:28:10 +00:00
vdimir
980000d04c
Change getByKeys, add StorageEmbeddedRocksDB::getByKeysImpl
2022-07-04 17:28:09 +00:00
vdimir
744c692be3
wip key value join
2022-07-04 17:28:09 +00:00
vdimir
8bce6451c6
add method StorageEmbeddedRocksDB::getByKeys
2022-07-04 17:27:37 +00:00
Anton Popov
8b356f0e1b
better tests and comments
2022-07-04 15:06:43 +00:00
Dmitry Novik
c66e2c8095
Merge pull request #37155 from amosbird/projection-fix-two
...
Fix intermediate header in aggregate projection
2022-07-04 16:32:34 +02:00
FArthur-cmd
21df8ed12f
refactoring
2022-07-04 17:21:50 +03:00
Kruglov Pavel
6c4b6fa834
Merge branch 'master' into better-exception-messages
2022-07-04 15:07:31 +02:00
kssenii
cfff7c4c28
Merge master
2022-07-04 14:13:26 +02:00
FArthur-cmd
21b99e7304
Merge branch 'master' of https://github.com/Vector-Similarity-Search-for-ClickHouse/ClickHouse into annoy-2
2022-07-04 14:54:21 +03:00
Vitaly Baranov
1dba3d9f9c
Make temporary files in better places.
2022-07-04 11:02:58 +02:00
lgbo-ustc
932e596689
Merge remote-tracking branch 'ck/master' into storagehive_alter
2022-07-04 09:56:00 +08:00
Raúl Marín
838fd1e41e
Fix replication after improper merge process
2022-07-03 15:11:25 +02:00
Vitaly Baranov
92e0ee0b6f
More detailed error messages.
2022-07-03 14:20:19 +02:00
alesapin
b2db49dbf1
Merge branch 'master' into better_data_part_storage_builder
2022-07-02 15:39:44 +02:00
kssenii
831d2253ce
Merge master
2022-07-01 18:53:20 +02:00
Anton Popov
ef87e1207c
better support of read_in_order in case of fixed prefix of sorting key
2022-07-01 16:45:01 +00:00
Dmitry Novik
81dd90893e
Merge remote-tracking branch 'origin/master' into group-by-use-nulls
2022-07-01 16:24:05 +00:00
alesapin
0d7298f3a9
Fix typos
2022-07-01 15:26:27 +02:00
alesapin
57284e6a9d
Fix possible deadlocks with MergeTreeData::Transaction
2022-07-01 15:16:32 +02:00
Vitaly Baranov
cadb496894
Merge pull request #38537 from vitlibar/backup-improvements-8
...
Backup Improvements 8
2022-07-01 11:20:46 +02:00
Amos Bird
d73f975849
Fix intermediate header in aggregate projection
2022-07-01 16:47:32 +08:00
alesapin
09faa1641e
Some comments
2022-06-30 23:09:32 +02:00
alesapin
979565edf0
Merge branch 'master' into better_data_part_storage_builder
2022-06-30 22:54:42 +02:00
alesapin
eb5046ab26
Simplify everything
2022-06-30 22:51:27 +02:00
Alexander Tokmakov
60dcae9af0
Merge pull request #38627 from Algunenano/rmarin_fix_replicated_part_2
...
Adapt some more nodes to avoid issues with pre-22.4 replicas
2022-06-30 21:11:35 +03:00
Alexander Tokmakov
d2cbdc7c53
Update ReplicatedMergeTreeQueue.cpp
2022-06-30 17:23:53 +03:00
Maksim Kita
a0e5768956
Merge pull request #38623 from loyispa/patch
...
Fix typo
2022-06-30 16:02:36 +02:00
Vitaly Baranov
e367d96964
Fix style.
2022-06-30 15:10:33 +02:00
Raúl Marín
cbcd740dc1
Adapt some more nodes to avoid issues with pre-22.4 replicas
2022-06-30 15:10:09 +02:00
avogar
ee54c4f9b7
Add some fixes and add settings in docs
2022-06-30 12:41:56 +00:00
alesapin
1363b478d4
Merge branch 'fix_disk_tx_write_op' into better_data_part_storage_builder
2022-06-30 14:38:32 +02:00
Anton Popov
7c721578c7
Merge pull request #38320 from CurtizJ/dynamic-columns-16
...
Improve performace of insertion to columns of type JSON
2022-06-30 14:18:20 +02:00
alesapin
cb90aca2ef
Some changes in merge tree
2022-06-30 14:12:45 +02:00
loyispa
109c9bcbd5
Fix typo
2022-06-30 18:37:19 +08:00
Vitaly Baranov
5456bde4a2
Improve gathering metadata for storing ACL in backups.
2022-06-30 09:46:37 +02:00
Antonio Andelic
de264117fd
Merge pull request #38118 from bigo-sg/storagehive_struct_type
...
Add struct type support in `StorageHive`
2022-06-30 09:11:13 +02:00
Vitaly Baranov
031ca28fdc
Add test for partition clause. More checks for data compatibility on restore.
2022-06-30 08:37:18 +02:00
Vitaly Baranov
11b51d2878
Implement storing UDF in backups.
2022-06-30 08:37:17 +02:00
Vitaly Baranov
efbee5e723
Improve gathering metadata for backup - part 7.
2022-06-30 08:37:17 +02:00
Vitaly Baranov
01921ce9a3
Move most code from setTableStructure() to a separate function.
2022-06-30 08:37:17 +02:00
Vitaly Baranov
7689e0c36f
Improve gathering metadata for backup - part 6.
2022-06-30 08:37:17 +02:00
Vitaly Baranov
461a31f237
Improve gathering metadata for backup - part 2.
2022-06-30 08:37:17 +02:00
Vitaly Baranov
64b51a3772
Improve gathering metadata for backup.
2022-06-30 08:37:17 +02:00
Vitaly Baranov
47ac47350b
Store projections in backups.
2022-06-30 08:37:17 +02:00
alesapin
67f2fa0ef0
Fix style
2022-06-29 15:18:48 +02:00
alesapin
2f03e821bd
Fix stupid bug
2022-06-29 15:08:16 +02:00
alesapin
ac963a7663
Remove redundant check
2022-06-29 13:55:20 +02:00
Alexander Tokmakov
8246e55002
Merge pull request #38486 from azat/fix-parts-removal
...
Fix parts removal after incorrect server shutdown
2022-06-29 14:17:59 +03:00
Alexander Tokmakov
32c6a944ee
Merge pull request #38541 from Algunenano/rmarin_fix_replicated_create_table_compat
...
Fix table creation to avoid replication issues with pre-22.4 replicas
2022-06-29 14:15:50 +03:00
mergify[bot]
1280fc39a0
Merge branch 'master' into better-exception-messages
2022-06-29 11:09:06 +00:00
Kruglov Pavel
b9813bcdea
Merge pull request #38356 from kssenii/fix-rabbitmq-with-peekable
...
Fix RabbitMQ with formats based on PeekableReadBuffer
2022-06-29 13:06:10 +02:00
Alexander Tokmakov
ceb66ade4b
Merge pull request #38335 from ClickHouse/deprecate_ordinary_database
...
Deprecate Ordinary database and old *MergeTree syntax
2022-06-29 13:42:59 +03:00
alesapin
c80a4c27be
Merge branch 'master' into better_data_part_storage_builder
2022-06-29 12:33:42 +02:00
Alexey Milovidov
4cda5491f6
Merge pull request #38542 from ClickHouse/unique_lock-to-lock_guard
...
Don't use std::unique_lock unless we need to
2022-06-29 09:40:56 +03:00
Nikita Taranov
f5d26572df
Quick fix for aggregation pipeline ( #38295 )
2022-06-29 01:16:30 +02:00
Robert Schulze
f692ead6ad
Don't use std::unique_lock unless we have to
...
Replace where possible by std::lock_guard which is more light-weight.
2022-06-28 19:19:06 +00:00
Raúl Marín
9b4da86e2e
Adapt table creation with replication to avoid issues with pre-22.4 replicas
2022-06-28 20:12:07 +02:00
avogar
106f92dcdb
Fix tests
2022-06-28 16:13:42 +00:00
Alexander Tokmakov
428628238f
Merge branch 'master' into deprecate_ordinary_database
2022-06-28 15:40:33 +02:00
alesapin
6429b72371
Fixes
2022-06-28 14:41:22 +02:00
alesapin
c7620b8217
Use builder instead of disk
2022-06-28 13:19:30 +02:00
kssenii
4c99f608b3
Fix
2022-06-28 13:18:21 +02:00
mergify[bot]
ea416c877f
Merge branch 'master' into better-exception-messages
2022-06-28 11:13:26 +00:00
alesapin
a68bf8fff2
Missed changes
2022-06-28 12:59:54 +02:00
alesapin
0a3fab1cb6
Some sad changes
2022-06-28 12:51:49 +02:00
alesapin
49add19842
Merge branch 'better_data_part_storage_builder' of github.com:ClickHouse/ClickHouse into better_data_part_storage_builder
2022-06-28 12:02:27 +02:00
alesapin
1a4aa8e52e
Merge branch 'master' into better_data_part_storage_builder
2022-06-28 11:51:06 +02:00
Alexander Tokmakov
851534c9f7
Merge branch 'master' into deprecate_ordinary_database
2022-06-27 23:47:49 +02:00
Azat Khuzhin
e70cff2cea
Fix parts removal after incorrect server shutdown
...
Before this patch, and after transaction support had been added #24258 ,
if the server had not removed old parts on shutdown, then they will left
forever.
Fixes : #24258 (cc @tavplubix)
v2: use Tx::PrehistoricTID over removePartsFromWorkingSet() from ctor
v3: apply black to new test
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-06-27 23:56:25 +03:00
Azat Khuzhin
f0271dc486
Refactor deactive_part() a little
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-06-27 23:42:39 +03:00
alesapin
d88cdcd5c1
AAA
2022-06-27 21:41:29 +02:00
Yakov Olkhovskiy
d5f65ece9b
Merge pull request #38105 from arenadata/ADQM-419
...
Add kerberosInit function as a replacement for kinit executable calls in Kafka and HDFS
2022-06-27 14:19:24 -04:00
alesapin
180b4cd64d
Merge branch 'master' into better_data_part_storage_builder
2022-06-27 19:46:17 +02:00
Alexander Tokmakov
88dbbcb15d
fix tests
2022-06-27 16:10:00 +02:00
avogar
12448f9400
Fix style
2022-06-27 13:56:43 +00:00
Alexander Tokmakov
bec921c9d9
Merge pull request #38265 from ClickHouse/cleanup_garbage_in_store_dir
...
Add background cleanup of store/ subdirs
2022-06-27 16:53:13 +03:00
Kseniia Sumarokova
be283d1c12
Update RabbitMQSource.cpp
2022-06-27 15:39:22 +02:00
Kseniia Sumarokova
cadc202b35
Update RabbitMQSource.cpp
2022-06-27 15:37:58 +02:00
Alexander Tokmakov
7c37079a96
Merge branch 'master' into deprecate_ordinary_database
2022-06-27 14:46:59 +02:00
avogar
5155262a16
Add some additional information to cache keys
2022-06-27 12:43:24 +00:00
kssenii
2c5aeaaa1a
Add auto close for postgres connection
2022-06-27 13:46:52 +02:00
kssenii
5c26f99bc5
Merge master
2022-06-27 13:36:41 +02:00
Nikita Taranov
2487ba7f00
Move updateInputStream
to ITransformingStep
( #37393 )
2022-06-27 13:16:52 +02:00
alesapin
50eb364a56
Merge pull request #38436 from kssenii/remote-path-separation-in-object-storage
...
more consistent work with paths in object storages
2022-06-27 10:58:01 +02:00
alesapin
83b584b2dd
Something
2022-06-26 18:43:28 +02:00
mergify[bot]
f63c959679
Merge branch 'master' into cleanup_garbage_in_store_dir
2022-06-26 13:35:10 +00:00
Alexey Milovidov
b3098822e0
Merge pull request #38171 from ClickHouse/hyper-to-vectorscan
...
Replace hyperscan by vectorscan
2022-06-26 10:01:45 +03:00
kssenii
725d80d470
get rid of path separation
2022-06-26 01:17:19 +02:00
kssenii
03f8b974ce
get rid of path separation
2022-06-25 20:30:36 +02:00
Kseniia Sumarokova
a6e868dddc
Update RabbitMQSource.cpp
2022-06-25 11:26:08 +02:00
alesapin
e2a779b315
Merge with new changes
2022-06-25 00:29:45 +02:00
alesapin
80d27f6381
Merge pull request #38380 from ClickHouse/simplify_parts_commit_merge_tree
...
Simplify parts commit methods
2022-06-25 00:12:52 +02:00
kssenii
708b93a03b
Merge master
2022-06-25 00:08:24 +02:00
alesapin
8ccc47c202
Revert some changes
2022-06-24 23:54:10 +02:00
Alexander Gololobov
189d0fffb8
Merge pull request #37165 from ClickHouse/merge_tree_reader
...
Merge tree reader support for multiple read/filter steps: row level filter, prewhere, ...
2022-06-24 21:32:31 +02:00
alesapin
a22c77e118
Merge branch 'master' into better_data_part_storage_builder
2022-06-24 19:52:41 +02:00
Vadim Volodin
3296ba2532
Fix SYSTEM UNFREEZE for ordinary database
2022-06-24 20:46:16 +03:00
Kseniia Sumarokova
5e86d6c55b
Merge pull request #38360 from kssenii/add-check-materialized-pg
...
Fix possible `Invalid number of rows in Chunk` in materialized pg
2022-06-24 19:33:15 +02:00
alesapin
2bc4dc4a1d
Fix accident change
2022-06-24 18:38:43 +02:00
Alexander Tokmakov
31dcc7634e
Merge branch 'master' into deprecate_ordinary_database
2022-06-24 18:16:07 +02:00
alesapin
d963f262f8
Fix style
2022-06-24 17:43:18 +02:00
alesapin
f685cf2268
Fix comment
2022-06-24 17:33:43 +02:00
alesapin
011d58d7a0
Simplify more
2022-06-24 17:19:59 +02:00
Kruglov Pavel
86e8f31ad4
Merge branch 'master' into schema-inference-cache
2022-06-24 16:10:25 +02:00
avogar
23022441a6
Fix style
2022-06-24 14:01:12 +00:00
avogar
ba69193db1
Fix build
2022-06-24 14:00:22 +00:00
alesapin
612c4571d5
Split method into smaller
2022-06-24 15:41:09 +02:00
alesapin
7517e1f4d5
Remove some complexity
2022-06-24 15:24:02 +02:00
mergify[bot]
36b93772b9
Merge branch 'master' into merge_tree_reader
2022-06-24 13:22:53 +00:00
alesapin
c57e07cb49
Style
2022-06-24 14:51:47 +02:00
alesapin
37310dc9df
Simpler
2022-06-24 14:10:15 +02:00
avogar
8b88df8b81
Fix build
2022-06-24 11:58:12 +00:00
alesapin
36b810b076
Remove unused parameter
2022-06-24 13:42:36 +02:00
alesapin
af1a9d18ab
Remove transaction argument
2022-06-24 13:34:00 +02:00
alesapin
9910395823
Simplify method signature
2022-06-24 13:19:29 +02:00
Robert Schulze
2c828338f4
Replace hyperscan by vectorscan
...
This commit migrates ClickHouse to Vectorscan. The first 10 min of
[0] explain the reasons for it.
(*) Addresses (but does not resolve) #38046
(*) Config parameter names (e.g. "max_hyperscan_regexp_length") are
preserved for compatibility. Likewise, error codes (e.g.
"ErrorCodes::HYPERSCAN_CANNOT_SCAN_TEXT") and function/class names (e.g.
"HyperscanDeleter") are preserved as vectorscan aims to be a drop-in
replacement.
[0] https://www.youtube.com/watch?v=KlZWmmflW6M
2022-06-24 10:47:52 +02:00
Kruglov Pavel
982fcfc68a
Fix build
2022-06-24 00:54:05 +02:00
avogar
59c1c472cb
Better exception messages on wrong table engines/functions argument types
2022-06-23 20:04:06 +00:00
kssenii
bc9b56096f
Fix
2022-06-23 21:52:57 +02:00
Alexander Tokmakov
74f38710a8
Merge branch 'master' into cleanup_garbage_in_store_dir
2022-06-23 21:43:28 +02:00
kssenii
6fbd49f554
Merge master
2022-06-23 21:40:01 +02:00
Alexander Tokmakov
dbf2763788
automatically convert system database to Atomic
2022-06-23 21:38:43 +02:00
kssenii
e362e4fff1
Fix
2022-06-23 20:48:32 +02:00
alesapin
2bdedf5e0c
Some changes
2022-06-23 18:21:46 +02:00
kssenii
468c98ff42
Better
2022-06-23 17:46:27 +02:00
mergify[bot]
174d8577fa
Merge branch 'master' into merge_tree_reader
2022-06-23 15:16:51 +00:00
kssenii
f7b329ee57
Merge master
2022-06-23 14:56:48 +02:00
mergify[bot]
234f0c6399
Merge branch 'master' into revert-35914-FIPS_compliance
2022-06-23 12:06:17 +00:00
alesapin
5661280ef6
Remove some read methods
2022-06-23 14:01:26 +02:00
Anton Popov
7efbae7728
Merge pull request #38069 from CurtizJ/better-support-gcp
...
Better support of GCP storage
2022-06-23 13:19:14 +02:00
alesapin
ae2feacbd1
Merge pull request #38322 from ClickHouse/ban_projections_in_a_right_way
...
Ban projections for zero-copy replication in a right way
2022-06-23 11:52:30 +02:00
lgbo-ustc
cd8e5c7c49
update headers
2022-06-23 17:43:54 +08:00
Alexander Tokmakov
f00e6b5a7a
deprecate old MergeTree syntax
2022-06-23 11:24:54 +02:00
Alexander Tokmakov
2c5a88faba
deprecate Ordinary database
2022-06-23 10:20:14 +02:00
Kseniia Sumarokova
af2150a68d
Merge pull request #38176 from kssenii/url-headers-in-ast
...
Add ability to pass headers to url table function / storage via sql
2022-06-23 10:15:07 +02:00
lgbo-ustc
c1770c22b9
Merge remote-tracking branch 'ck/master' into storagehive_struct_type
2022-06-23 15:54:20 +08:00
lgbo-ustc
0fba75b21d
Merge remote-tracking branch 'ck/master' into storagehive_alter
2022-06-23 09:38:06 +08:00
alesapin
1b495ec8ad
Fix style
2022-06-23 00:17:37 +02:00
kssenii
e40d9bcf55
Merge master
2022-06-22 23:28:52 +02:00
Anton Popov
52db1b35a1
improve performace of insertion to columns of type JSON
2022-06-22 17:45:51 +00:00
Alexander Gololobov
0ee47363d4
Fixed includes
2022-06-22 19:08:18 +02:00
Alexander Gololobov
8138a3a36e
Cleanups based on review comments
2022-06-22 19:08:01 +02:00
alesapin
ca0299e927
Ban projections for zero-copy replication in a right way
2022-06-22 19:01:46 +02:00
Alexander Tokmakov
ed8341025b
make code less bad
2022-06-22 18:31:42 +02:00
Alexander Gololobov
e5b55b965b
Removed incorrect check
2022-06-22 17:23:09 +02:00
Alexander Gololobov
dbc6d1a159
Cleanups
2022-06-22 17:23:09 +02:00
Alexander Gololobov
aea5bc7527
Removed unused should_reorder flag
2022-06-22 17:23:09 +02:00
Alexander Gololobov
5427d5c51b
Apply filter at row level security step
2022-06-22 17:23:09 +02:00
Alexander Gololobov
b3922461b3
Properly handle empty actions
2022-06-22 17:23:09 +02:00
Alexander Gololobov
ba89c3954c
Do not add the same vitrual if it has been added by prev_reader
2022-06-22 17:23:09 +02:00
Alexander Gololobov
e5c0ebc7b5
Handle read from wide part when no columns are requested
2022-06-22 17:23:09 +02:00
Alexander Gololobov
a9e3b8d29e
Don't read the same columns again
2022-06-22 17:23:09 +02:00
Alexander Gololobov
4e426c63cc
Debuging test failures
2022-06-22 17:23:09 +02:00
Alexander Gololobov
6a26325fab
Test dirty hacks for multiple PREWHERE steps
2022-06-22 17:23:05 +02:00
Alexander Gololobov
4360fd9798
Vector of structs instead of multiple vectors
2022-06-22 17:17:42 +02:00
Alexander Gololobov
87b669f439
Intermediate changes
2022-06-22 17:17:42 +02:00
Alexander Gololobov
159ab765fb
Moved estimateNumRows from lambda to function
2022-06-22 17:17:42 +02:00
Alexander Gololobov
538c8c96fc
Dump row_level_filter as a part of prewhere_info
2022-06-22 17:17:42 +02:00
Alexander Gololobov
64a2f3734b
Protect ReadResult internals from MergeTreeRangeReader clients
2022-06-22 17:17:42 +02:00
kssenii
0acc95fe22
Review fixes
2022-06-22 16:55:47 +02:00
alesapin
0f8eed98c4
Merge branch 'master' into better-support-gcp
2022-06-22 12:46:01 +02:00
Kruglov Pavel
e5a7f53775
Fix misleading error message while s3 schema inference
2022-06-22 12:36:09 +02:00
alesapin
d3bc7c0190
Merge pull request #37971 from Avogar/fix-alter-columns-with-dots
...
Fix ALTER column with column names with dots
2022-06-22 12:20:30 +02:00
Nikolai Kochetov
cc6fdfe0eb
Merge pull request #36555 from ClickHouse/refactor-something-in-part-volumes
...
Separate data storage abstraction for MergeTree
2022-06-22 11:13:36 +02:00
Alexey Milovidov
0f8bc027fd
Merge branch 'revert-35914-FIPS_compliance' of github.com:ClickHouse/ClickHouse into revert-35914-FIPS_compliance
2022-06-22 06:17:01 +02:00
Anton Popov
d5d568e235
Merge pull request #37913 from azat/parts-refcnt
...
Fix refcnt for unused MergeTree parts in SELECT queries
2022-06-21 22:56:26 +02:00
kssenii
4178abc0a5
Merge master
2022-06-21 22:38:51 +02:00
kssenii
90ce0b44f0
Fixes
2022-06-21 21:03:18 +02:00
Robert Schulze
0d80874d40
Merge pull request #38068 from ClickHouse/clang-tsa
...
Support for Clang Thread Safety Analysis (TSA)
2022-06-21 20:19:33 +02:00
avogar
13a05adcb9
Some fixes
2022-06-21 17:34:08 +00:00
avogar
c14364e3d9
Check last modification time for URL function too
2022-06-21 17:18:14 +00:00
Nikolai Kochetov
dcf6e0d060
Roll back some changes from IPartMetadataManager
2022-06-21 13:05:49 +00:00
avogar
d37ad2e6de
Implement cache for schema inference for file/s3/hdfs/url
2022-06-21 13:02:48 +00:00
Kruglov Pavel
8bf78dbc95
Add comment
2022-06-21 14:32:12 +02:00
Nikolai Kochetov
854d148d73
Add some comments. Remove some commented code.
2022-06-21 12:31:02 +00:00
kssenii
4656dd3de3
Headers for url in ast
2022-06-21 14:25:05 +02:00
Nikolai Kochetov
543782436d
Process additional filters.
2022-06-21 11:24:46 +00:00
Nikolai Kochetov
46c4fd6cf7
Fixing build.
2022-06-21 11:15:06 +00:00
Kruglov Pavel
3c5a627bd7
Merge pull request #38239 from CurtizJ/fix-reading-from-s3
...
Fix reading from s3 in some corner cases
2022-06-21 12:57:55 +02:00
mergify[bot]
f304fc9179
Merge branch 'master' into fix-alter-columns-with-dots
2022-06-21 10:40:53 +00:00
Nikolai Kochetov
1e8c9ecd4c
Merge branch 'master' into refactor-something-in-part-volumes
2022-06-21 12:37:21 +02:00
alesapin
fafcb876be
Merge pull request #38182 from ClickHouse/disk_transaction
...
Disk transaction
2022-06-21 12:28:49 +02:00
Nikolai Kochetov
b8d27aa8dd
Merge pull request #37469 from azat/projections-optimize_aggregation_in_order
...
Implement in order aggregation (optimize_aggregation_in_order) for projections for tables with fully materialized projections
2022-06-21 12:17:35 +02:00
Kseniia Sumarokova
71ee5dbbb2
Merge pull request #38227 from kssenii/diff-reduce
...
buffer's getFileSize small changes
2022-06-21 12:12:22 +02:00
Nikolai Kochetov
e45d552f4d
Do not create empty folder for in-memory parts.
2022-06-21 09:59:46 +00:00
Nikolai Kochetov
5232c17b89
Fix style
2022-06-21 07:30:45 +00:00
Nikolai Kochetov
3429a5e07a
Fix
2022-06-21 07:26:43 +00:00
lgbo-ustc
fc641d9ce4
some changes
2022-06-21 11:29:55 +08:00
Larry Luo
bbd73ba727
use utility methods to access x509 struct fields.
2022-06-20 21:27:33 -04:00
Nikolai Kochetov
dccf90b1ea
Cleanup.
2022-06-20 18:18:17 +00:00
mergify[bot]
9bdd9e14a6
Merge branch 'master' into fix_flaky_tests_with_transactions
2022-06-20 18:11:30 +00:00
Anton Popov
59a9ce6def
fix reading from s3 in some corner cases
2022-06-20 15:24:38 +00:00
Robert Schulze
55b39e709d
Merge remote-tracking branch 'origin/master' into clang-tsa
2022-06-20 16:39:32 +02:00
Robert Schulze
5a4f21c50f
Support for Clang Thread Safety Analysis (TSA)
...
- TSA is a static analyzer build by Google which finds race conditions
and deadlocks at compile time.
- It works by associating a shared member variable with a
synchronization primitive that protects it. The compiler can then
check at each access if proper locking happened before. A good
introduction are [0] and [1].
- TSA requires some help by the programmer via annotations. Luckily,
LLVM's libcxx already has annotations for std::mutex, std::lock_guard,
std::shared_mutex and std::scoped_lock. This commit enables them
(--> contrib/libcxx-cmake/CMakeLists.txt).
- Further, this commit adds convenience macros for the low-level
annotations for use in ClickHouse (--> base/defines.h). For
demonstration, they are leveraged in a few places.
- As we compile with "-Wall -Wextra -Weverything", the required compiler
flag "-Wthread-safety-analysis" was already enabled. Negative checks
are an experimental feature of TSA and disabled
(--> cmake/warnings.cmake). Compile times did not increase noticeably.
- TSA is used in a few places with simple locking. I tried TSA also
where locking is more complex. The problem was usually that it is
unclear which data is protected by which lock :-(. But there was
definitely some weird code where locking looked broken. So there is
some potential to find bugs.
*** Limitations of TSA besides the ones listed in [1]:
- The programmer needs to know which lock protects which piece of shared
data. This is not always easy for large classes.
- Two synchronization primitives used in ClickHouse are not annotated in
libcxx:
(1) std::unique_lock: A releaseable lock handle often together with
std::condition_variable, e.g. in solve producer-consumer problems.
(2) std::recursive_mutex: A re-entrant mutex variant. Its usage can be
considered a design flaw + typically it is slower than a standard
mutex. In this commit, one std::recursive_mutex was converted to
std::mutex and annotated with TSA.
- For free-standing functions (e.g. helper functions) which are passed
shared data members, it can be tricky to specify the associated lock.
This is because the annotations use the normal C++ rules for symbol
resolution.
[0] https://clang.llvm.org/docs/ThreadSafetyAnalysis.html
[1] https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/42958.pdf
2022-06-20 16:13:25 +02:00
Nikolai Kochetov
39b02d4ab8
Continure resolving conflicts.
2022-06-20 15:39:05 +02:00
Nikolai Kochetov
7452d04e3a
Merge branch 'master' into refactor-something-in-part-volumes
2022-06-20 15:31:02 +02:00
kssenii
5dd1bb2fd8
improvements for getFileSize
2022-06-20 15:22:56 +02:00
Nikolai Kochetov
ef8b967c3f
Fix more tests.
2022-06-20 13:07:14 +00:00
kssenii
90be49faa4
Merge master
2022-06-20 14:26:56 +02:00
Dmitry Novik
77177917bd
Merge pull request #37848 from ClickHouse/window-function-expression
...
Support expressions with window functions
2022-06-20 14:21:13 +02:00
Vitaly Baranov
06c4082a7c
Merge pull request #37358 from vitlibar/backup-improvements-6
...
Backup Improvements 6
2022-06-20 14:13:30 +02:00
alesapin
e3184d264b
Merge branch 'master' into disk_transaction
2022-06-20 13:47:30 +02:00
mergify[bot]
4ed606a275
Merge branch 'master' into fix_flaky_tests_with_transactions
2022-06-20 10:17:51 +00:00
Kseniia Sumarokova
c83594284d
Merge pull request #38205 from azat/fix-window-view
...
(Window View is an experimental feature) Fix LOGICAL_ERROR for WINDOW VIEW with incorrect structure
2022-06-20 11:47:45 +02:00
lgbo-ustc
efec6f9ac5
support alter on storagehive
2022-06-20 14:57:33 +08:00
Vitaly Baranov
638ea23399
Fix build.
2022-06-20 03:44:59 +02:00
mergify[bot]
670a63865e
Merge branch 'master' into window-function-expression
2022-06-19 22:14:54 +00:00
Alexey Milovidov
7700c26076
Merge pull request #38196 from ClickHouse/revert-38194-revert-37015-zstd_window_log_max
...
Revert "Revert "Add a setting to use more memory for zstd decompression""
2022-06-19 19:58:03 +03:00
Azat Khuzhin
bc3e73d776
Fix LOGICAL_ERROR for WINDOW VIEW with incorrect structure
...
Caching header of the source table in the WINDOW VIEW should not be
done, since there is no ability to get notification when it had been
changed (ALTER or CREATE/DROP).
And this fires on [CI], when the following tests had been executed in
order in stress tests:
- 01050_window_view_parser_tumble (leaves wm for mt)
- 01748_partition_id_pruning (cache input_header)
- 01188_attach_table_from_path (insert into mt with wm attached and
incorrect structure)
[CI]: https://s3.amazonaws.com/clickhouse-test-reports/38056/109980eb275c064d08bc031bfdc14d95b9a7272b/stress_test__undefined__actions_.html
Follow-up for: #37965 (@Vxider)
Fixes : #37815
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-06-19 19:16:39 +03:00
alesapin
119ce90e5b
Fix strange diff
2022-06-19 18:03:43 +02:00
alesapin
08b5dd091b
Fix double negative
2022-06-19 17:12:04 +02:00
Vitaly Baranov
8a7c970ce0
Fix style.
2022-06-19 15:58:26 +02:00
alesapin
50801e41c5
Merge branch 'master' into refactor-something-in-part-volumes
2022-06-19 14:05:46 +02:00
Alexey Milovidov
04b7a34870
Maybe fix error
2022-06-19 05:21:59 +02:00
Alexey Milovidov
73709b0488
Revert "Revert "Add a setting to use more memory for zstd decompression""
2022-06-18 15:55:35 +03:00
alesapin
16e8b85fbf
Revert "Add a setting to use more memory for zstd decompression"
2022-06-18 14:08:14 +02:00
Alexey Milovidov
e20259e9ca
Merge pull request #37015 from wuxiaobai24/zstd_window_log_max
...
Add a setting to use more memory for zstd decompression
2022-06-18 04:19:27 +03:00
Vitaly Baranov
a0c558a17e
Implement backup/restore for ACL system tables (system.users, system.roles, etc.)
2022-06-17 18:14:31 +02:00
Alexander Tokmakov
83adf56383
fix race
2022-06-17 18:13:57 +02:00
Alexander Tokmakov
e9c2157f01
Merge branch 'master' into fix_flaky_tests_with_transactions
2022-06-17 15:57:30 +02:00
Alexander Tokmakov
5ffc38d9b9
fix style check, add comments
2022-06-17 13:47:14 +02:00
alesapin
e5b405701a
Merge branch 'master' into disk_transaction
2022-06-17 12:16:12 +02:00
alesapin
1c7a7da487
Merge pull request #38088 from kssenii/diff-with-cache
...
Extract some diff from pr #36171
2022-06-17 12:07:31 +02:00
mergify[bot]
296905771f
Merge branch 'master' into better-support-gcp
2022-06-17 01:19:11 +00:00
Anton Popov
1523c9c9e5
fix filling of empty Nested + small refactoring
2022-06-17 01:10:52 +00:00
Alexander Tokmakov
39c0219c11
fixes
2022-06-16 19:41:32 +02:00
Dmitry Novik
376412e417
Small refactoring
2022-06-16 15:41:04 +00:00
Mikhail f. Shiryaev
2bef5257b7
Merge pull request #38147 from ClickHouse/22.7-prepare
...
Update version after release
2022-06-16 17:19:40 +02:00
Mikhail f. Shiryaev
06dd85f921
Update version to 22.7.1.1
2022-06-16 17:15:22 +02:00
Anton Popov
13ec7e3092
Merge pull request #37978 from CurtizJ/fix-sparse-s3
...
Fix reading of sparse columns from s3
2022-06-16 15:32:54 +02:00
alesapin
379fff7318
Merge branch 'master' into disk_transaction
2022-06-16 14:36:19 +02:00
Roman Vasin
ed3fe84b63
Fix runKinit() is called only for USE_KRB5
2022-06-16 14:45:27 +03:00
Roman Vasin
0ab6bfd5d8
Add warning about using krb5 parameter in StorageKafka.cpp
2022-06-16 14:25:37 +03:00
Roman Vasin
6e28275569
Add warnings about using krb5 parameters
2022-06-16 14:21:04 +03:00
Roman Vasin
d93fd3bd2d
Add complilation support for case when krb5 is not used
2022-06-16 09:30:40 +00:00
lgbo-ustc
a8b17fec84
fixed a bug
2022-06-16 16:53:11 +08:00
lgbo-ustc
35d534c213
nested struct in struct
2022-06-16 16:45:05 +08:00
Azat Khuzhin
4ff82eb9d5
Implement optimize_aggregation_in_order for projections
...
v2: use real column name instead of aliases from GROUP BY
Fixes the following error in 01710_projection_aggregation_in_order:
Not found column a in block. There are only columns: toStartOfHour(ts), sum(value). (NOT_FOUND_COLUMN_IN_BLOCK)
v2.1: Get back support for projected and non-projected parts
v2.2: merge tests and rename
v3: Reduce copy-paste for optimize_aggregation_in_order for projections
v4: rebase on top of QueryPlanResourceHolder
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-06-16 09:58:36 +03:00
Azat Khuzhin
14f9491619
MergeTreeData: preserve order for group_by_elements_order_descr
...
This is required for proper optimize_aggregation_in_order for
projections.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-06-16 09:58:36 +03:00
Azat Khuzhin
4694929623
Implement merging only for AggregatingStep
...
v2: fill AggregateColumnsConstData only for only_merge
(fixes 01291_aggregation_in_order and some other tests)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-06-16 09:58:36 +03:00
Azat Khuzhin
682c93e2e1
Use proper setting compile_aggregate_expressions in MergeTreeDataSelectExecutor
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-06-16 09:58:35 +03:00
lgbo-ustc
e115e3f731
remove unused header
2022-06-16 09:53:04 +08:00
lgbo-ustc
655e42c9bc
remove trace logs
2022-06-16 09:44:41 +08:00
lgbo-ustc
4f13521aa6
struct type support for storage hive
2022-06-16 09:35:34 +08:00
mergify[bot]
7f24574609
Merge branch 'master' into better-support-gcp
2022-06-15 23:18:20 +00:00
Anton Popov
5b01f0f341
Merge remote-tracking branch 'upstream/master' into HEAD
2022-06-15 23:01:22 +00:00
Alexander Tokmakov
6ddfec47d2
maybe fix livelock on queue processing
2022-06-15 22:22:27 +02:00
Alexander Tokmakov
5e2b42c85a
Merge branch 'master' into fix_flaky_tests_with_transactions
2022-06-15 20:47:02 +02:00
Vitaly Baranov
c2c35fad82
Refactoring of the code getting create table queries for backup.
2022-06-15 20:32:35 +02:00
Vitaly Baranov
1198e86295
Fix storing temporary tables and skipping system tables while making a backup.
2022-06-15 20:32:34 +02:00
Vitaly Baranov
21f3bed435
Simplify path calculations in backup.
2022-06-15 20:32:34 +02:00
Vitaly Baranov
592f568f83
Move backup/restore code to storages and databases - part 2.
2022-06-15 20:32:31 +02:00