alesapin
f608e62c27
Fix call
2022-08-03 14:17:31 +02:00
alesapin
56a4d26e87
Better total part size calculation on mutation
2022-08-03 14:15:45 +02:00
Maksim Kita
3e6b663020
ASTSelectQuery added hasJoin method
2022-08-03 11:25:45 +02:00
Anton Popov
a333cc4146
Merge remote-tracking branch 'upstream/master' into HEAD
2022-08-02 12:57:43 +00:00
Frank Chen
1c9b462c80
Merge remote-tracking branch 'origin/master' into tracing_context_propagation
2022-08-02 20:18:22 +08:00
Igor Nikonov
56697125d3
Merge remote-tracking branch 'origin/master' into distinct_sorted_simplify
2022-08-02 11:18:46 +00:00
Igor Nikonov
4bf42ce867
Merge remote-tracking branch 'origin/master' into skipping_sorting_step
2022-08-02 10:50:25 +00:00
Wangyang Guo
91e3e2f18b
KeyCondition: optimize applyFunction in multi-thread scenario
...
Construct and deconstruct args (ColumnsWithTypeAndName) will inc/dec
ref_count (actually this is a atomic lock inc/dec operation) to share_ptr,
which may share the same DataTypePtr among different threads. This will
have a lock contention issue in large parallel situation.
The patch try to minimize `args` scope and reduce unnecessary
construct/destory of instances. It will improve the performance in
multi-thread cases.
2022-08-02 16:16:16 +08:00
Frank Chen
7d8a8f26cd
Fix build check
2022-08-02 13:31:36 +08:00
Frank Chen
8eb254c0c8
Fix merge problem
2022-08-02 10:23:51 +08:00
Frank Chen
40c6e4c0d6
Merge remote-tracking branch 'origin/master' into tracing_context_propagation
2022-08-02 10:02:09 +08:00
Alexander Tokmakov
1567b65473
Merge branch 'master' into replicated_database_improvements
2022-08-01 19:12:37 +02:00
Alexey Milovidov
3de747a6e2
Merge pull request #39746 from ClickHouse/cleanup-projection-setting
...
Cleanup usages of `allow_experimental_projection_optimization` setting, part 1
2022-08-01 19:21:30 +03:00
Robert Schulze
bf574b9154
Merge pull request #39760 from ClickHouse/bit-fiddling
...
Use std::popcount, ::countl_zero, ::countr_zero functions
2022-08-01 17:04:51 +02:00
Antonio Andelic
9ec27c0ab4
Merge pull request #39757 from ClickHouse/fix-rocksdb-filter-with-params
...
Use params correctly for filtering by key in EmbeddedRocksDB
2022-08-01 12:17:10 +02:00
Dmitry Novik
ab5a147065
Merge pull request #38725 from azat/fix-order-by-projection
...
Fix ORDER BY that matches projections ORDER BY
2022-08-01 00:32:59 +02:00
Nikolai Kochetov
22fbfe19a4
Merge branch 'master' into use-dag-in-key-condition
2022-07-31 21:54:12 +02:00
kssenii
e5f4a619ed
Merge master
2022-07-31 20:24:40 +03:00
Robert Schulze
a7734672b9
Use std::popcount, ::countl_zero, ::countr_zero functions
...
- Introduced with the C++20 <bit> header
- The problem with __builtin_c(l|t)z() is that 0 as input has an
undefined result (*) and the code did not always check. The std::
versions do not have this issue.
- In some cases, we continue to use buildin_c(l|t)z(), (e.g. in
src/Common/BitHelpers.h) because the std:: versions only accept
unsigned inputs (and they also check that) and the casting would be
ugly.
(*) https://gcc.gnu.org/onlinedocs/gcc/Other-Builtins.html
2022-07-31 15:16:51 +00:00
Robert Schulze
52d08d9db4
Merge pull request #39520 from ClickHouse/no-split-binary
...
Remove SPLIT_BINARY
2022-07-31 14:23:31 +02:00
Antonio Andelic
eea1aaef2c
Use params for filtering by key in EmbeddedRocksDB
2022-07-31 10:44:01 +00:00
Igor Nikonov
7f0adb5eb0
Merge remote-tracking branch 'origin/master' into skipping_sorting_step
2022-07-31 07:07:36 +00:00
Anton Popov
74eb8119ef
Merge branch 'master' into fix-custom-serializations
2022-07-31 01:51:52 +02:00
Alexey Milovidov
8fb70abe3e
Merge pull request #39178 from azat/dist-insert-log
...
Add connection info for Distributed sends log message
2022-07-31 02:22:22 +03:00
Alexey Milovidov
441f2feb49
Cleanup usages of allow_experimental_projection_optimization
setting, part 1
2022-07-30 23:40:21 +02:00
Igor Nikonov
a7cfad105e
Merge branch 'master' into distinct_sorted_simplify
2022-07-30 21:57:53 +02:00
Igor Nikonov
7b0b38e997
DistinctSortedTransform works only if columns contains sort prefix of
...
sort description
2022-07-29 20:01:07 +00:00
Kruglov Pavel
ccd1e1bdb8
Merge branch 'master' into fix-cancel-insert-into-function
2022-07-29 20:27:32 +02:00
Maksim Kita
8fc6bad4f4
Join enums refactoring
2022-07-29 18:35:05 +02:00
Alexander Tokmakov
6f88065cc1
check metadata consistency
2022-07-29 18:33:16 +02:00
Nikolai Kochetov
59a11b32ad
Merge branch 'master' into use-dag-in-key-condition
2022-07-29 17:01:33 +02:00
Igor Nikonov
fb5799fe7f
More generic getInputOrderInfo() in query info
2022-07-29 14:36:36 +00:00
Igor Nikonov
ae82bfb7f3
Merge remote-tracking branch 'origin/master' into distinct_sorted_simplify
2022-07-29 13:12:03 +00:00
Robert Schulze
3d1797f75f
Merge remote-tracking branch 'origin/master' into no-split-binary
2022-07-29 12:17:43 +00:00
Anton Popov
57e4fb2e30
Merge remote-tracking branch 'upstream/master' into HEAD
2022-07-29 11:42:11 +00:00
Anton Popov
bb551566e7
fix build
2022-07-29 11:41:53 +00:00
Azat Khuzhin
498c8b3c52
Fix clang-tidy in utils/examples
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-29 11:36:51 +03:00
Robert Schulze
199e254777
Merge remote-tracking branch 'origin/master' into no-split-binary
2022-07-28 15:54:22 +00:00
Anton Popov
04fd72cdef
fix alter column rename
2022-07-28 13:19:23 +00:00
Vitaly Baranov
f8ebe76826
Merge pull request #39503 from vitlibar/improve-system-backups-table
...
Improve system.backups table
2022-07-27 20:25:39 +02:00
Vladimir C
115506356c
Merge branch 'master' into refactor-prepared-sets
2022-07-27 19:57:23 +02:00
Nikolai Kochetov
6206e468a4
Ignor materialize function for ActionsDAG in PK analysis.
2022-07-27 17:02:48 +00:00
Nikolai Kochetov
51c4c9c957
Fixing build.
2022-07-27 15:45:23 +00:00
Anton Popov
56039c8780
fix alter of LowCardinality
2022-07-27 15:35:09 +00:00
Anton Popov
4cf0353892
try to fix Nested
2022-07-27 14:05:16 +00:00
Filatenkov Artur
a218597546
Merge branch 'master' into annoy-2
2022-07-27 15:37:58 +03:00
FArthur-cmd
6b5e92e68a
change index policy
2022-07-27 15:36:07 +03:00
Nikolai Kochetov
f9a8a4cd32
Fixing read-in-order for Merge
2022-07-27 12:00:55 +00:00
vdimir
8eecb9ef82
upd PreparedSets: rename/change signature of methods, add comments
2022-07-27 11:22:16 +00:00
Alexander Tokmakov
8fc075a527
Merge pull request #39619 from ClickHouse/try-do-enable-global-allocations-for-attach
...
Replace MemoryTrackerBlockerInThread to LockMemoryExceptionInThread
2022-07-27 13:13:04 +03:00
Kruglov Pavel
381ea139c2
Merge branch 'master' into schema-inference-cache
2022-07-27 11:35:36 +02:00
Vitaly Baranov
794eeb5d51
Split "total_size" to "uncompressed_size" and "compressed_size".
2022-07-27 10:36:56 +02:00
Vitaly Baranov
a79fc17235
Add test for system.backups
2022-07-27 09:04:10 +02:00
Vitaly Baranov
1cfe0b10f7
Add columns "total_size" and "num_files" to system.backups
2022-07-27 09:04:10 +02:00
Vitaly Baranov
35c267b3b1
Replace column "status_changed_time" with columns "start_time" and "end_time".
2022-07-27 09:04:10 +02:00
Vitaly Baranov
fc16a15ecf
Rename column "uuid" -> "id" in system.backups and allow user to set it in a query.
2022-07-27 09:04:10 +02:00
Vitaly Baranov
131019ba49
Rename column "backup_name" -> "name" in system.backups.
2022-07-27 09:04:10 +02:00
Vitaly Baranov
afd0982187
Remove column "internal" from system.backups
2022-07-27 09:04:10 +02:00
Alexander Gololobov
a64aa00869
Merge pull request #37893 from zhangjmruc/feature/sql-standard-delete
...
Support SQL standard "delete from ... where ..." syntax and lightweight implementation on merge tree tables
2022-07-26 23:39:07 +02:00
Nikolai Kochetov
1adb021df6
Fixing tests.
2022-07-26 18:42:27 +00:00
vdimir
5ce2960f03
Get rid of SelectQueryInfoBase -> SelectQueryInfo
2022-07-26 18:39:09 +00:00
vdimir
d9928ac93d
Add methods to SubqueryForSet, do not use refernce to SetPtr
2022-07-26 18:39:09 +00:00
vdimir
1e3fa2e01f
Refactor PreparedSets/SubqueryForSet
2022-07-26 18:39:02 +00:00
Anton Popov
9321ca34cf
try to fix Nested
2022-07-26 17:32:37 +00:00
Nikolai Kochetov
33ce662d3e
Replace MemoryTrackerBlockerInThread to LockMemoryExceptionInThread in some places. Reduced MemoryTrackerBlockerInThread level to User.
2022-07-26 15:22:00 +00:00
Kruglov Pavel
c683cb252f
Merge pull request #39227 from amosbird/rename-log1
...
Rename log when rename merge tree tables
2022-07-26 17:12:44 +02:00
Nikolai Kochetov
d7cc3831ea
Push predicate over StorageMerge.
2022-07-26 15:01:39 +00:00
Nikolai Kochetov
c5110bb164
Add a query plan step for StorageMerge
2022-07-26 14:43:05 +00:00
jianmei zhang
c1834d183b
ReplicatedMergeTree support lightweight delete
2022-07-26 11:50:09 +08:00
Alexander Gololobov
25deba2c1b
Merge branch 'master' into feature/sql-standard-delete
2022-07-25 22:13:20 +02:00
Nikolai Kochetov
be9c7ed52c
Add ReadFromMerge step.
2022-07-25 19:41:43 +00:00
Roman Vasin
b462366415
Merge branch 'master' of github.com:ClickHouse/ClickHouse into ADQM-485
2022-07-25 17:55:47 +00:00
Roman Vasin
b412ea5f6d
Improve generateRandom() for Date32; fix tests 01087_table_function_generate, 01277_fromUnixTimestamp64, 01691_DateTime64_clamp and 01702_toDateTime_from_string_clamping
2022-07-25 17:06:11 +00:00
Alexander Tokmakov
778dcf6994
Merge pull request #39230 from Algunenano/improve_replicated_merge_logging
...
Improve logging around replicated merges
2022-07-25 18:42:12 +03:00
Alexander Gololobov
48de02a7b8
Capitalized const name
2022-07-25 16:32:16 +02:00
Alexander Gololobov
7b44950d1d
More detailed comment
2022-07-25 16:14:33 +02:00
Nikolai Kochetov
b70be40804
Merge branch 'master' into use-dag-in-key-condition
2022-07-25 14:30:22 +02:00
Nikolai Kochetov
1ce8e714d3
Fixing build.
2022-07-25 12:30:01 +00:00
Alexey Milovidov
388d06fda1
Merge pull request #39535 from ClickHouse/stringref
...
Less usage of StringRef
2022-07-25 04:06:11 +03:00
Alexander Gololobov
460950ecdc
Merge branch 'master' into feature/sql-standard-delete
2022-07-24 21:27:22 +02:00
Robert Schulze
4333750985
Less usage of StringRef
...
... replaced by std::string_view, see #39262
2022-07-24 18:33:52 +00:00
Alexey Milovidov
0209012e5e
Merge pull request #39517 from pkit/pkit/fix_examples
...
fix `-DENABLE_EXAMPLES=1` in master
2022-07-24 17:41:23 +03:00
Igor Nikonov
eacc57b6a4
Simplify DistinctSorted
...
+ use ordinary Distinct for deduplication in MergeTree when no sorting provided
2022-07-24 14:26:25 +00:00
Azat Khuzhin
22d8e532ed
Fix ORDER BY that matches projections ORDER BY
...
In case of projection is seleted and it is comlete (all parts has such
projection) and ORDER BY matches projection ORDER BY, and
optimize_read_in_order=1 (default), then the sorting is simply not done,
because projections has separate plan, and so
InterpreterSelectQuery::executeOrder() -> SortingStep has
pipeline.getNumStreams() == 0 and it cannot do sorting.
Fix this, by adding sorting when creating plan for reading from
projections.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-24 15:23:13 +03:00
Kseniia Sumarokova
04944ff6f5
Merge pull request #39509 from monadbobo/remove_duplicate_code
...
Removed some duplicate code.
2022-07-24 14:24:40 +03:00
Alexander Gololobov
6692770113
Allow to update "system" virtual columns such as _row_exists
2022-07-24 12:21:47 +02:00
Alexander Gololobov
c8b3c574a4
Disable lightweight delete if table has projections
2022-07-24 12:21:47 +02:00
Alexander Gololobov
be64b45583
Properly handle empty parts list
2022-07-24 12:21:47 +02:00
Alexander Gololobov
594195451e
Cleanups
2022-07-24 12:21:18 +02:00
Alexey Milovidov
071374b152
Remove SPLIT_BINARY
2022-07-24 01:15:54 +02:00
Kseniia Sumarokova
f146183400
Merge pull request #39491 from ClickHouse/remove_unused_constructor
...
Remove unused constructor
2022-07-23 21:16:39 +02:00
mergify[bot]
c01ff2d38a
Merge branch 'master' into improve_replicated_merge_logging
2022-07-23 19:10:10 +00:00
Constantine Peresypkin
d645d03e89
fix -DENABLE_EXAMPLES=1
in master
2022-07-23 19:00:52 +02:00
Kseniia Sumarokova
77ddb8eda0
Merge pull request #39506 from CurtizJ/fix-direct-io
...
Fix reading from async read buffer with direct io
2022-07-23 06:08:48 +02:00
Anton Popov
3d03b2714b
try to fix Nested
2022-07-22 23:05:11 +00:00
Nikolai Kochetov
dd65a36c59
Fix another one test.
2022-07-22 19:11:25 +00:00
Nikolai Kochetov
53175e43f3
Better support for projections.
2022-07-22 19:11:25 +00:00
Simon Liu
2305f282dc
Removed some duplicate code.
2022-07-23 02:27:17 +08:00
Anton Popov
49627aa554
Merge remote-tracking branch 'upstream/master' into HEAD
2022-07-22 17:16:06 +00:00
Anton Popov
c1a5514e46
fix reading from async read buffer with direct io
2022-07-22 16:58:30 +00:00
Alexander Tokmakov
bed2206ae9
Merge pull request #39460 from ClickHouse/remove_some_dead_and_commented_code
...
Remove some dead and commented code
2022-07-22 13:24:34 +03:00
alesapin
69984b28e0
Remove unused constructor
2022-07-22 12:21:10 +02:00
Robert Schulze
99579ab440
Merge pull request #39364 from ClickHouse/stringref-to-string_view
...
Even less usage of StringRef
2022-07-22 09:30:49 +02:00
Alexander Tokmakov
0441d3629e
Merge pull request #39440 from liangliangpan/optimize-cluster-table-func
...
optimize select query from cluster table function
2022-07-21 20:40:31 +03:00
Mikhail f. Shiryaev
cf0b01b189
Update version to 22.8.1.1
2022-07-21 19:21:28 +02:00
Kruglov Pavel
9252f42b4c
Merge branch 'master' into schema-inference-cache
2022-07-21 18:59:14 +02:00
Robert Schulze
ea0a3bf600
Merge branch 'master' into stringref-to-string_view
2022-07-21 18:33:06 +02:00
zhangxiao871
5549b64c06
Fix issues 39469
2022-07-21 23:52:33 +08:00
Anton Popov
10719fd61d
fix style check
2022-07-21 15:06:55 +00:00
Anton Popov
e0d2c8fb37
fix json type with sparse columns
2022-07-21 14:47:19 +00:00
Alexander Tokmakov
8bd674186a
Update getStructureOfRemoteTable.cpp
2022-07-21 16:15:42 +03:00
Alexander Tokmakov
a8da5d96fc
remove some dead and commented code
2022-07-21 15:05:48 +02:00
avogar
52b3a87ed1
Fix typo
2022-07-21 12:38:18 +00:00
avogar
6b541aa98f
Fix WriteBuffer finalize when cancel insert into function
2022-07-21 12:18:37 +00:00
Alexander Tokmakov
f79f12408d
fixes
2022-07-21 13:57:24 +02:00
Alexander Gololobov
1ea9f143ff
Leave only _row_exists-based implementation of lightweight delete
2022-07-21 11:26:13 +02:00
Nikolai Kochetov
e15967e9db
Merge pull request #38475 from ClickHouse/additional-filters
...
Additional filters for a table (from setting)
2022-07-21 07:52:04 +02:00
Mingliang Pan
e019b6ba69
Update getStructureOfRemoteTable.cpp
2022-07-21 12:11:33 +08:00
Alexey Milovidov
844042fc18
Merge pull request #39433 from ClickHouse/revert-39396-try-fix-write-buffer-terminate
...
Revert "Fix WriteBuffer finalize in destructor when cacnel query"
2022-07-21 07:04:07 +03:00
Mingliang Pan
3f76c8d7fd
optimize select query from cluster table function
...
Use local node as first priority to get Structure Of Remote Table.
we have many distributed queries( like select xx from cluster('xx',view (xxxx) ) on a clickhouse cluster. we found that the first node (shard_num=1) have 2 times of query number compared to other shards.
The reason is that the getStructureOfRemoteTableInShard func always take the first shard to execute "DESC TABLE xx" query.
The better way is to use local node as first priority which save the network rpc and reduce the pressure of first shard .
2022-07-21 11:56:35 +08:00
Alexey Milovidov
fd691000b7
Merge pull request #39342 from azat/bg-executor-fix
...
Fix possible UB in MergeTreeBackgroundExecutor (leads to SIGSEGV on race with DROP/DETACH)
2022-07-21 05:22:51 +03:00
Kruglov Pavel
92995a832b
Revert "Fix WriteBuffer finalize in destructor when cacnel query"
2022-07-21 01:45:16 +02:00
Nikolai Kochetov
91043351aa
Fixing build.
2022-07-20 20:30:16 +00:00
Nikolai Kochetov
91aa2385ca
Merge branch 'master' into additional-filters
2022-07-20 18:56:24 +02:00
Kruglov Pavel
0efed4644d
Merge pull request #38957 from Avogar/compatibility-setting
...
Add compatibility setting and system.settings_changes system table
2022-07-20 17:46:06 +02:00
Nikolai Kochetov
4e8cd70b1d
Merge branch 'master' into use-dag-in-key-condition
2022-07-20 17:38:33 +02:00
Dmitry Novik
a09422de7b
Merge pull request #38642 from ClickHouse/group-by-use-nulls
...
Add group_by_use_nulls setting
2022-07-20 14:42:42 +02:00
Kruglov Pavel
3046cd6d29
Merge branch 'master' into schema-inference-cache
2022-07-20 13:30:42 +02:00
Vladimir C
7694c72154
Merge pull request #38887 from bigo-sg/hive_file_crash
...
fixed: crash caused by IHiveFile be shared among threads
2022-07-20 12:15:57 +02:00
Kruglov Pavel
7722b647b7
Merge pull request #39396 from Avogar/try-fix-write-buffer-terminate
...
Fix WriteBuffer finalize in destructor when cacnel query
2022-07-20 12:06:20 +02:00
lgbo-ustc
4b46d80169
update codes
2022-07-20 18:03:34 +08:00
Alexander Tokmakov
988569bdb1
Merge pull request #39366 from ClickHouse/fix_race_in_replication_queue
...
Do not enqueue uneeded parts for check
2022-07-20 00:35:02 +03:00
Nikolai Kochetov
f570cde815
Fixing build.
2022-07-19 20:19:57 +00:00
avogar
5c16d6b553
Fix WriteBuffer finalize in destructor when cacnel query
2022-07-19 19:21:30 +00:00
Nikolai Kochetov
a00bcd389f
Fixing build
2022-07-19 19:33:14 +02:00
Nikolai Kochetov
eaeb30a71a
Merge branch 'master' into use-dag-in-key-condition
2022-07-19 18:39:52 +02:00
Dmitry Novik
50989bdb68
Merge branch 'master' into group-by-use-nulls
2022-07-19 14:58:01 +02:00
Alexander Tokmakov
a761da9f1b
add comment
2022-07-19 13:58:59 +02:00
kssenii
21ebf8874e
Merge remote-tracking branch 'upstream/master' into make-cache-composable
2022-07-19 13:36:42 +02:00
Alexander Tokmakov
3e7414b356
add comment
2022-07-19 12:47:20 +02:00
Nikolai Kochetov
d0db7135cb
Merge pull request #38946 from amosbird/nullable-partition-fix1
...
Fix incorrect partition pruning when there is a nullable partition
2022-07-19 12:14:36 +02:00
Kseniia Sumarokova
f168e0c5d0
Merge pull request #39272 from kssenii/fixes-for-materialized-pg
...
Fix segfault in materialised postgresql
2022-07-19 11:56:10 +02:00
Robert Schulze
81ef1099cc
Even less usage of StringRef
...
--> see #39300
2022-07-19 07:01:06 +00:00
FArthur-cmd
3b30cd9259
improvements
2022-07-19 00:59:07 +03:00
Alexander Tokmakov
d82f378a9d
do not enqueue uneeded parts for check
2022-07-18 23:37:07 +02:00
Alexander Gololobov
614cb9a87f
Disable trivial count() optimization if _row_exists column is present
2022-07-18 20:08:01 +02:00
Alexander Gololobov
ae0d00083c
Renamed __row_exists to _row_exists
2022-07-18 20:07:36 +02:00
Alexander Gololobov
f324ca9921
Cleanups
2022-07-18 20:07:22 +02:00
Alexander Gololobov
9de72d995a
POC lightweight delete using __row_exists virtual column and prewhere-like filtering
2022-07-18 20:06:42 +02:00
Robert Schulze
6df3c9d799
Merge pull request #39300 from ClickHouse/stringref-to-stringview
...
First try at reducing the use of StringRef
2022-07-18 19:20:16 +02:00
Azat Khuzhin
c6f7698f9a
Fix possible UB in MergeTreeBackgroundExecutor
...
Lambda erase_from_active() captures the item (TaskRuntimeDataPtr), most
of the code path is OK, since it explicitly reset the item->task.
However one is not, when it moves the item to pending list, which will
be cleaned up when the table will be DROP/DETACH'ed, from
MergeTreeBackgroundExecutor::removeTasksCorrespondingToStorage(),
and in this case if IStorage will be already removed, then it will lead
to use-after-free on destroying the lambda, since it captures the item
by value.
And I belive that CI founds this issue here [1]:
<details>
<summary>stack trace</summary>
4 0x268d1354 in DB::ReplicatedMergeTreeQueue::CurrentlyExecuting::~CurrentlyExecuting() build_docker/../src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1510:33
5 0x268ead1a in std::__1::default_delete<DB::ReplicatedMergeTreeQueue::CurrentlyExecuting>::operator()(DB::ReplicatedMergeTreeQueue::CurrentlyExecuting*) const build_docker/../contrib/libcxx/include/__memory/unique_ptr.h:54:5
6 0x268ead1a in std::__1::unique_ptr<DB::ReplicatedMergeTreeQueue::CurrentlyExecuting, std::__1::default_delete<DB::ReplicatedMergeTreeQueue::CurrentlyExecuting> >::reset(DB::ReplicatedMergeTreeQueue::CurrentlyExecuting*) build_docker/../contrib/libcxx/include/__memory/unique_ptr.h:315:7
7 0x268ead1a in std::__1::unique_ptr<DB::ReplicatedMergeTreeQueue::CurrentlyExecuting, std::__1::default_delete<DB::ReplicatedMergeTreeQueue::CurrentlyExecuting> >::~unique_ptr() build_docker/../contrib/libcxx/include/__memory/unique_ptr.h:269:19
8 0x268ead1a in DB::ReplicatedMergeTreeQueue::SelectedEntry::~SelectedEntry() build_docker/../src/Storages/MergeTree/ReplicatedMergeTreeQueue.h:351:12
9 0x268ead1a in void std::__1::__destroy_at<DB::ReplicatedMergeTreeQueue::SelectedEntry, 0>(DB::ReplicatedMergeTreeQueue::SelectedEntry*) build_docker/../contrib/libcxx/include/__memory/construct_at.h:56:13
...
16 0x265e9abb in DB::MergeTreeBackgroundExecutor<DB::MergeMutateRuntimeQueue>::routine(std::__1::shared_ptr<DB::TaskRuntimeData>)::'lambda'()::~() build_docker/../src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:127:30
17 0x265e9abb in DB::MergeTreeBackgroundExecutor<DB::MergeMutateRuntimeQueue>::routine(std::__1::shared_ptr<DB::TaskRuntimeData>) build_docker/../src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:235:1
18 0x265ea730 in DB::MergeTreeBackgroundExecutor<DB::MergeMutateRuntimeQueue>::threadFunction() build_docker/../src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:261:13
</details>
[1]: https://s3.amazonaws.com/clickhouse-test-reports/0/533c7888038453c047df816f3f65292cca05a54f/stateless_tests__ubsan__actions_.html
And I also I think that the following issue will also be fixed [2]:
<details>
<summary>stack trace</summary>
[ 680 ] {} <Fatal> : Logical error: 'Memory tracker: allocations not allowed.'.
...
[ 23430 ] {} <Fatal> BaseDaemon: 23. /build/build_docker/../src/Common/formatReadable.cpp:46: formatReadableSizeWithBinarySuffix(double, int) @ 0x1713bb98 in /usr/bin/clickhouse
...
[ 23430 ] {} <Fatal> BaseDaemon: 31. ../src/Common/MemoryTracker.cpp:112: MemoryTracker::logPeakMemoryUsage() @ 0x170e0ab9 in /usr/bin/clickhouse
[ 23430 ] {} <Fatal> BaseDaemon: 32. /build/build_docker/../src/Common/MemoryTracker.cpp:98: MemoryTracker::~MemoryTracker() @ 0x170e063a in /usr/bin/clickhouse
[ 23430 ] {} <Fatal> BaseDaemon: 33. /build/build_docker/../src/Storages/MergeTree/MergeList.cpp:144: DB::MergeListElement::~MergeListElement() @ 0x279fb290 in /usr/bin/clickhouse
...
[ 23430 ] {} <Fatal> BaseDaemon: 38. /build/build_docker/../src/Storages/MergeTree/BackgroundProcessList.h:41: DB::BackgroundProcessListEntry<DB::MergeListElement, DB::MergeInfo>::~BackgroundProcessListEntry() @ 0x276ce6c7 in /usr/bin/clickhouse
[ 23430 ] {} <Fatal> BaseDaemon: 39. /build/build_docker/../contrib/libcxx/include/__memory/unique_ptr.h:54: std::__1::default_delete<DB::BackgroundProcessListEntry<DB::MergeListElement, DB::MergeInfo> >::operator()(DB::BackgroundProcessListEntry<DB::MergeListElement, DB::MergeInfo>*) const @ 0x276ce60b in /usr/bin/clickhouse
[ 23430 ] {} <Fatal> BaseDaemon: 40. /build/build_docker/../contrib/libcxx/include/__memory/unique_ptr.h:316: std::__1::unique_ptr<DB::BackgroundProcessListEntry<DB::MergeListElement, DB::MergeInfo>, std::__1::default_delete<DB::BackgroundProcessListEntry<DB::MergeListElement, DB::MergeInfo> > >::reset(DB::BackgroundProcessListEntry<DB::MergeListElement, DB::MergeInfo>*) @ 0x276ce57c in /usr/bin/clickhouse
[ 23430 ] {} <Fatal> BaseDaemon: 41. /build/build_docker/../contrib/libcxx/include/__memory/unique_ptr.h:269: std::__1::unique_ptr<DB::BackgroundProcessListEntry<DB::MergeListElement, DB::MergeInfo>, std::__1::default_delete<DB::BackgroundProcessListEntry<DB::MergeListElement, DB::MergeInfo> > >::~unique_ptr() @ 0x276ce399 in /usr/bin/clickhouse
[ 23430 ] {} <Fatal> BaseDaemon: 42. /build/build_docker/../src/Storages/MergeTree/MutatePlainMergeTreeTask.h:22: DB::MutatePlainMergeTreeTask::~MutatePlainMergeTreeTask() @ 0x27defceb in /usr/bin/clickhouse
[ 23430 ] {} <Fatal> BaseDaemon: 43. /build/build_docker/../contrib/libcxx/include/__memory/construct_at.h:57: void std::__1::__destroy_at<DB::MutatePlainMergeTreeTask, 0>(DB::MutatePlainMergeTreeTask*) @ 0x27dd69c1 in /usr/bin/clickhouse
[ 23430 ] {} <Fatal> BaseDaemon: 44. /build/build_docker/../contrib/libcxx/include/__memory/construct_at.h:82: void std::__1::destroy_at<DB::MutatePlainMergeTreeTask, 0>(DB::MutatePlainMergeTreeTask*) @ 0x27dd6955 in /usr/bin/clickhouse
[ 23430 ] {} <Fatal> BaseDaemon: Integrity check of the executable skipped because the reference checksum could not be read. (calculated checksum: 91F5937571C11255DFE73230B52CE9C0)
[ 602 ] {} <Fatal> Application: Child process was terminated by signal 6.
</details>
[2]: https://s3.amazonaws.com/clickhouse-test-reports/39222/a068c397dfd7943359a8b554566c3c70b78baf8d/stateless_tests__debug__actions__%5B1/3%5D.html
Refs: https://github.com/ClickHouse/ClickHouse/pull/29614#discussion_r720455032 (cc @nikitamikhaylov)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-18 19:01:41 +03:00
kssenii
0aed5a602a
Merge master
2022-07-18 16:16:28 +02:00
Kseniia Sumarokova
b4eae94889
Merge pull request #39259 from rnbondarenko/patch-1
...
removed default queue arguments
2022-07-18 15:32:31 +02:00
avogar
80b5bc5236
Add system table settings_changes
2022-07-18 13:19:41 +00:00
Anton Popov
5605b60c97
Merge pull request #39039 from CurtizJ/randomize-more-settings
...
Add more settings for randomization
2022-07-18 15:05:41 +02:00
kssenii
a6655bea1c
Some fixes
2022-07-18 14:23:08 +02:00
Robert Schulze
1ced1d11fd
Merge pull request #39285 from azat/bg-exe
...
[RFC] Remove superior atomic from MergeTreeBackgroundExecutor and annotations for TSA
2022-07-18 12:46:54 +02:00
Kseniia Sumarokova
eb23491957
Merge branch 'master' into fixes-for-materialized-pg
2022-07-18 12:00:58 +02:00
Alexey Milovidov
13a8f85ed0
Merge branch 'master' into nullable-partition-fix1
2022-07-18 00:34:58 +03:00
Robert Schulze
13482af4ee
First try at reducing the use of StringRef
...
- to be replaced by std::string_view
- suggested in #39262
2022-07-17 17:26:02 +00:00
kssenii
8ad26b3111
Merge master
2022-07-16 22:50:03 +02:00
Vitaly Baranov
e0fb03d1b6
Merge pull request #38970 from vitlibar/select-from-system-db-requires-grant
...
SELECT from the system database requires grant now
2022-07-16 22:37:47 +02:00
Kseniia Sumarokova
3f43641b18
Update MergedColumnOnlyOutputStream.cpp
2022-07-16 16:32:33 +02:00
Azat Khuzhin
accb0e38da
Remove superior atomic from MergeTreeBackgroundExecutor and annotations for TSA
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-16 14:00:39 +03:00
kssenii
9f1d6814fc
Fix
2022-07-15 18:01:54 +02:00
alesapin
5cf4bd50de
Merge pull request #39214 from azat/s3-fix-fetching-memory-part
...
Fix fetch of in-memory part with allow_remote_fs_zero_copy_replication
2022-07-15 17:52:05 +02:00
Alexander Tokmakov
5ef7f5f494
Merge pull request #39229 from azat/fix-noexcept-scope
...
[RFC] Fix NOEXCEPT_SCOPE (before it calls std::terminate and looses the exception)
2022-07-15 18:14:56 +03:00
Raúl Marín
d68b3dfd43
Fix destructor order
2022-07-15 15:48:35 +02:00
Raúl Marín
aea045f297
Improve logging around replicated merges
2022-07-15 15:48:35 +02:00
Vitaly Baranov
c7cef91d4d
Two ways to provide access to system.users: "GRANT SHOW USERS ON *.*" and "GRANT SELECT ON system.users"
2022-07-15 15:44:29 +02:00
Vitaly Baranov
fbb2e14d54
Add new table function viewIfPermitted().
2022-07-15 15:44:29 +02:00
Anton Popov
76eec62469
Merge branch 'master' into randomize-more-settings
2022-07-15 14:12:49 +02:00
Kruglov Pavel
a944a92d4c
Merge pull request #39224 from Avogar/string-view-by-value
...
Pass const std::string_view by value, not by reference
2022-07-15 13:05:56 +02:00
rnbondarenko
a7fe1cf5f0
removed default queue arguments
...
declare queue without default arguments x-max-length and x-overflow.
2022-07-15 13:46:52 +03:00
Alexander Tokmakov
11ea455632
Merge pull request #39245 from ClickHouse/tavplubix-patch-1
...
Remove incorrect assertion
2022-07-15 13:26:14 +03:00
jianmei zhang
197d3796ec
Killed mutation has empty commands may cause SegV
2022-07-15 15:37:36 +08:00
jianmei zhang
ca42f649da
Rewrite logic for loading deleted mask related to getDeletedMask()
2022-07-15 15:31:10 +08:00
jianmei zhang
0793add9b5
Fix clang tidy compile error, remove std::move for const getDeletedRowsPtr()
2022-07-15 12:32:42 +08:00
jianmei zhang
8b1114fa60
fix clang tidy compile error
2022-07-15 12:32:42 +08:00
jianmei zhang
d37152a5d6
Remove loadDeletedMask() and get deleted mask when needed
2022-07-15 12:32:42 +08:00
jianmei zhang
4268ac8b0e
Add supportLightweightDeleteMutate() in IMergeTreeDataPart to disable LWD for part with projections
2022-07-15 12:32:42 +08:00
jianmei zhang
9d1dff515c
code changes for strange name and adding exception for mutation entry
2022-07-15 12:32:42 +08:00
jianmei zhang
198674e994
Consider lightweight delete as ordinary for compact part in selectPartsToMutate()
2022-07-15 12:32:42 +08:00
jianmei zhang
780cdfb8f0
Code changes based on review opinions. Make ordinary single alter delete lightwight by default
2022-07-15 12:32:42 +08:00
jianmei zhang
6e6f77ef8a
Fix compile error in clang tidy build
2022-07-15 12:32:41 +08:00
jianmei zhang
7e433859ea
Change deleted rows mask from String to Native UInt8 format
2022-07-15 12:32:41 +08:00
jianmei zhang
dcc7367ac4
Fix code style error
2022-07-15 12:32:41 +08:00
jianmei zhang
6248486790
Fix lightweight delete bugs: add skip files and use source part's columns to avoid metadata updated cases
2022-07-15 12:32:41 +08:00
jianmei zhang
8ad2bb7c33
Code changes due to master new fixes, and update reference for mutations table
2022-07-15 12:32:41 +08:00
jianmei zhang
11fdea6e4b
Add missing code for deleted_mask_filter_holder
2022-07-15 12:32:41 +08:00
jianmei zhang
9d27af7ee2
For some columns mutations, skip to apply deleted mask when read some columns. Also add unit test case
2022-07-15 12:32:41 +08:00
jianmei zhang
2c74e9b866
Merge support to apply lightweight delete mask
2022-07-15 12:32:41 +08:00
jianmei zhang
b4a37e1e22
Disable optimizations for count() when lightweight delete exists, add hasLightweightDelete() function in IMergeTreeDataPart
2022-07-15 12:32:41 +08:00
jianmei zhang
8696319d62
Support lightweight delete execution using string as deleted rows mask,also part of select can handle LWD
2022-07-15 12:32:41 +08:00
jianmei zhang
8df7b7a030
Support new mutation type for lightweight
2022-07-15 12:32:41 +08:00
Sergei Trifonov
62f975da60
Merge pull request #39095 from ClickHouse/remote-io-bandwidth-limits
...
add max_remote_{read,write}_network_bandwidth_for_server settings
2022-07-14 23:14:53 +02:00
Alexander Tokmakov
435d98738d
Update ReplicatedMergeTreeQueue.cpp
2022-07-14 22:41:01 +03:00
avogar
9291d33080
Pass const std::string_view & by value, not by reference
2022-07-14 16:11:57 +00:00
Yakov Olkhovskiy
1f09303d54
Merge pull request #39181 from azat/part_log-merge_algorithm
...
Add merge_algorithm to system.part_log
2022-07-14 11:43:04 -04:00
Sergei Trifonov
b2317ff710
renaming to be consistent
2022-07-14 17:33:22 +02:00
Anton Popov
d4cb2184c2
Merge branch 'master' into randomize-more-settings
2022-07-14 17:01:07 +02:00
Nikolai Kochetov
b24eeedf53
Merge branch 'master' into nullable-partition-fix1
2022-07-14 16:41:38 +02:00
Yakov Olkhovskiy
9ecf19c883
Merge pull request #39212 from azat/stack_trace-filter-v2
...
Optimize accesses to system.stack_trace (filter by name before sending signal)
2022-07-14 10:33:26 -04:00
Azat Khuzhin
b20407fab9
Fix NOEXCEPT_SCOPE (before it calls std::terminate and looses the exception)
...
Current implementation of NOEXCEPT_SCOPE will not work, you cannot
rethrow exception outside the catch block, this will simply terminate
(via std::terminate) the program.
In other words NOEXCEPT_SCOPE macro will simply call std::terminate on
exception and will lost original exception.
But if NOEXCEPT_SCOPE will accept the code that should be runned w/o
exceptions, then it can catch exception and log it, rewrite it in this
way.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-14 17:16:18 +03:00
Amos Bird
96bb6e0cd2
Rename log when rename merge tree tables
2022-07-14 21:22:46 +08:00
Anton Popov
51533adee8
fix build
2022-07-14 11:33:23 +00:00
FArthur-cmd
ad459bc6e4
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into annoy-2
2022-07-14 14:10:44 +03:00
Sergei Trifonov
8a693469f0
Merge branch 'master' into remote-io-bandwidth-limits
2022-07-14 12:38:35 +02:00
Kruglov Pavel
b38241b08a
Merge branch 'master' into schema-inference-cache
2022-07-14 12:29:54 +02:00
Kseniia Sumarokova
530dac6487
Merge pull request #37213 from aaapetrenko/master
...
add TableFunctionMongoDB and tests
2022-07-14 12:09:07 +02:00
Azat Khuzhin
531e3b13ce
Optimize accesses to system.stack_trace (filter by name before sending signal)
...
Because of failed rebase/push this part of the patch was lost, so
resending separatelly (sigh).
Follow-up for: #39177 (cc @yakov-olkhovskiy)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-14 11:49:34 +03:00
Azat Khuzhin
3e77b80b88
Fix fetch of in-memory part with allow_remote_fs_zero_copy_replication
...
CI founds the following error during trying to fetch in-memory part [1]:
2022.07.13 08:06:54.231033 [ 141093 ] {} <Error> InterserverIOHTTPHandler: Code: 107. DB::ErrnoException: Cannot open file /var/lib/clickhouse/disks/s3/store/886/88635b40-e4e3-4fe6-a0a0-1b6755463358/all_0_0_0/data.bin, errno: 2, strerror: No such file or directory. (FILE_DOESNT_EXIST), Stack trace (when copying this message, always include the lines below):
<details>
<summary>stacktrace</summary>
2022.07.13 08:06:54.101825 [ 189342 ] {8dbd11b3-f38a-4d5d-9ded-148987adb71d} <Debug> executeQuery: (from [::1]:54570) (comment: 01643_replicated_merge_tree_fsync_smoke.sql) select 'memory in_memory_parts_insert_sync'; (stage: Complete)
2022.07.13 08:06:54.131309 [ 691 ] {} <Debug> test_26u6kx.rep_fsync_r2 (39c3823c-22e5-4c05-9dec-cdffd8872c40): Fetching part all_0_0_0 from /clickhouse/tables/test_26u6kx/rep_fsync/replicas/r1
2022.07.13 08:06:54.231033 [ 141093 ] {} <Error> InterserverIOHTTPHandler: Code: 107. DB::ErrnoException: Cannot open file /var/lib/clickhouse/disks/s3/store/886/88635b40-e4e3-4fe6-a0a0-1b6755463358/all_0_0_0/data.bin, errno: 2, strerror: No such file or directory. (FILE_DOESNT_EXIST), Stack trace (when copying this message, always include the lines below):
0. DB::Exception::Exception() @ 0xba0191a in /usr/bin/clickhouse
1. DB::throwFromErrnoWithPath() @ 0xba029ca in /usr/bin/clickhouse
2. DB::OpenedFile::open() const @ 0x156e7fb0 in /usr/bin/clickhouse
3. DB::OpenedFile::getFD() const @ 0x156e8003 in /usr/bin/clickhouse
4. DB::ReadBufferFromFilePReadWithDescriptorsCache::ReadBufferFromFilePReadWithDescriptorsCache() @ 0x156e5f23 in /usr/bin/clickhouse
5. ? @ 0x156e53f0 in /usr/bin/clickhouse
6. DB::createReadBufferFromFileBase() @ 0x156e52b5 in /usr/bin/clickhouse
7. DB::DiskLocal::readFile() const @ 0x15e45ea8 in /usr/bin/clickhouse
8. DB::MetadataStorageFromDisk::readFileToString() const @ 0x15e6ab8b in /usr/bin/clickhouse
9. DB::MetadataStorageFromDisk::readMetadataUnlocked() const @ 0x15e6cdeb in /usr/bin/clickhouse
10. DB::MetadataStorageFromDisk::getSerializedMetadata() const @ 0x15e6cfc4 in /usr/bin/clickhouse
11. DB::DiskObjectStorage::getSerializedMetadata() const @ 0x15e19e2e in /usr/bin/clickhouse
12. DB::DiskDecorator::getSerializedMetadata() const @ 0x15e54ed1 in /usr/bin/clickhouse
13. DB::DiskDecorator::getSerializedMetadata() const @ 0x15e54ed1 in /usr/bin/clickhouse
14. DB::DataPartsExchange::Service::sendPartFromDiskRemoteMeta() @ 0x1700bb9e in /usr/bin/clickhouse
15. DB::DataPartsExchange::Service::processQuery(DB::HTMLForm const&, DB::ReadBuffer&, DB::WriteBuffer&, DB::HTTPServerResponse&) @ 0x1700a649 in /usr/bin/clickhouse
16. DB::InterserverIOHTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTTPServerResponse&, DB::InterserverIOHTTPHandler::Output&) @ 0x17433c53 in /usr/bin/clickhouse
17. DB::InterserverIOHTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0x174344f1 in /usr/bin/clickhouse
18. DB::HTTPServerConnection::run() @ 0x1768714d in /usr/bin/clickhouse
19. Poco::Net::TCPServerConnection::start() @ 0x1a398093 in /usr/bin/clickhouse
20. Poco::Net::TCPServerDispatcher::run() @ 0x1a399411 in /usr/bin/clickhouse
21. Poco::PooledThread::run() @ 0x1a54b7bb in /usr/bin/clickhouse
22. Poco::ThreadImpl::runnableEntry(void*) @ 0x1a548ec0 in /usr/bin/clickhouse
23. ? @ 0x7fdf1c204609 in ?
24. clone @ 0x7fdf1c129133 in ?
(version 22.7.1.1781 (official build))
</details>
[1]: https://s3.amazonaws.com/clickhouse-test-reports/0/8b6e31cc615ca52c80724b6e5097777cb9514f07/stateless_tests__release__s3_storage__actions_.html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-14 11:43:35 +03:00
Yakov Olkhovskiy
d6a09acb38
Merge pull request #39177 from azat/stack_trace-filter
...
Optimize accesses to system.stack_trace
2022-07-14 01:47:14 -04:00
Anton Popov
089009c996
fix reading from StorageLog with mmap
2022-07-13 20:36:18 +00:00
Sergei Trifonov
15ab3bc99f
use context->getWriteSettings()
2022-07-13 19:48:57 +02:00
Sergei Trifonov
af4a5a5f3a
fix conflicts
2022-07-13 19:44:29 +02:00
kssenii
615221f73b
Minor changes
2022-07-13 19:22:47 +02:00
Sergei Trifonov
f854507729
merge master and resolve conflicts
2022-07-13 19:07:28 +02:00
Kseniia Sumarokova
71aba5b4c6
Merge pull request #38860 from kssenii/object-storages-refactor
...
Refactoring of code around object storages, added LocalObjectStorage (extracted this diff from PR #36171 )
2022-07-13 16:51:56 +02:00
kssenii
0944183eec
Fix merge
2022-07-13 16:50:31 +02:00
Azat Khuzhin
3720a3843c
Add merge_algorithm to system.part_log
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-13 17:16:25 +03:00
Azat Khuzhin
4f25a08b7c
Add connection info for Distributed sends log message
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-13 16:04:12 +03:00
Anton Popov
54a0985c42
randomize one more setting
2022-07-13 12:50:12 +00:00
Azat Khuzhin
8122c1b07f
Optimize accesses to system.stack_trace
...
- filter by thread_id
- filter by thread_name
- filter by requested columns (query_id/trace, thread_name)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-13 15:17:09 +03:00
FArthur-cmd
8ddc2578a2
refacroring
2022-07-13 14:48:02 +03:00
kssenii
0d2d3ac2b4
Merge master
2022-07-13 13:26:15 +02:00
kssenii
83a09fd961
Merge remote-tracking branch 'petrenko/master' into petrenko-master
2022-07-13 11:28:52 +02:00
kssenii
ae324622d3
Add one more test
2022-07-13 11:27:43 +02:00
Amos Bird
0795cdd36a
Slightly better interface of waitForMutation
2022-07-13 12:34:12 +08:00
Dmitry Novik
5f65b45269
Merge branch 'master' into group-by-use-nulls
2022-07-12 22:36:04 +02:00
kssenii
6f29eada80
Merge master
2022-07-12 19:28:18 +02:00
Kseniia Sumarokova
0fb35b773f
Update StorageMongoDB.cpp
2022-07-12 18:37:19 +02:00
Kseniia Sumarokova
5468a2fde9
Update StorageMongoDB.cpp
2022-07-12 17:20:20 +02:00
Kseniia Sumarokova
869da0d923
Update StorageMongoDB.cpp
2022-07-12 17:19:49 +02:00
kssenii
f46d641440
Fix style check and black check
...
t:wq# Date: Tue Jul 12 14:02:09 2022 +0200
2022-07-12 16:29:11 +02:00
Nikolai Kochetov
8efbe6d44d
Merge pull request #39119 from ClickHouse/removed-projection-and-old-part
...
Properly remove projection from part in case it was removed from table metadata.
2022-07-12 16:24:30 +02:00
Robert Schulze
69de9ee0e8
Merge pull request #39108 from ClickHouse/prefix_cmake_commands
...
Prefix overridden add_executable() command with "clickhouse_"
2022-07-12 15:05:48 +02:00
FArthur-cmd
6e2812490b
refactoring
2022-07-12 14:29:11 +03:00
Alexander Tokmakov
c99fe6e6c8
Merge branch 'master' into removed-projection-and-old-part
2022-07-12 14:00:55 +03:00
FArthur-cmd
0e89fbff53
refactoring
2022-07-12 13:57:16 +03:00
Alexander Tokmakov
b35ad0dbdc
Merge pull request #39091 from ClickHouse/tavplubix-patch-1
...
Follow-up to #39036
2022-07-12 13:47:37 +03:00
Nikolai Kochetov
945b5d4b3f
Merge pull request #38993 from azat/fix-fsync_part_directory-on-fetch
...
Fix fsync_part_directory for fetches
2022-07-12 11:48:42 +02:00
Nikolai Kochetov
cd4211298e
Fix typo.
2022-07-12 09:37:00 +00:00
Nikolai Kochetov
03d9301425
Fix log.
2022-07-12 08:31:09 +00:00
Nikolai Kochetov
631be9219b
Properly remove projection from part in case it was removed from table metadata.
2022-07-12 08:28:22 +00:00
FArthur-cmd
783e6e398c
merge with other working branch
2022-07-11 23:02:58 +03:00
aaapetrenko
aabf0ed67f
refactor Storage
2022-07-11 22:00:57 +02:00
FArthur-cmd
6ae227a6bb
add test
2022-07-11 22:50:10 +03:00
Robert Schulze
1a7727a254
Prefix overridden add_executable() command with "clickhouse_"
...
A simple HelloWorld program with zero includes except iostream triggers
a build of ca. 2000 source files. The reason is that ClickHouse's
top-level CMakeLists.txt overrides "add_executable()" to link all
binaries against "clickhouse_new_delete". This links against
"clickhouse_common_io", which in turn has lots of 3rd party library
dependencies ... Without linking "clickhouse_new_delete", the number of
compiled files for "HelloWorld" goes down to ca. 70.
As an example, the self-extracting-executable needs none of its current
dependencies but other programs may also benefit.
In order to restore access to the original "add_executable()", the
overriding version is now prefixed. There is precedence for a
"clickhouse_" prefix (as opposed to "ch_"), for example
"clickhouse_split_debug_symbols". In general prefixing makes sense also
because overriding CMake commands relies on undocumented behavior and is
considered not-so-great practice (*).
(*) https://crascit.com/2018/09/14/do-not-redefine-cmake-commands/
2022-07-11 19:36:18 +02:00
Sergei Trifonov
6ed777d2a8
add azure support
2022-07-11 19:19:57 +02:00
Sergei Trifonov
add0d1698e
respect new limits in HDFS
2022-07-11 17:30:23 +02:00
FArthur-cmd
868628d3fd
make stable prototype with arrays support
2022-07-11 18:02:50 +03:00
Sergei Trifonov
43779ec280
add max_remote_{read,write}_network_bandwidth_for_server settings
2022-07-11 14:59:39 +02:00
Alexander Tokmakov
e9ddbc247f
Merge pull request #39016 from Algunenano/clearOldLogs_zkerror
...
clearOldLogs: Don't report KEEPER_EXCEPTION on concurrent deletes
2022-07-11 15:16:18 +03:00
Alexander Tokmakov
048d27189e
Update MergeTreeData.cpp
2022-07-11 15:07:31 +03:00
FArthur-cmd
7b7a9521c7
change submodule path
2022-07-10 18:10:47 +03:00
FArthur-cmd
2f9f5f172a
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into annoy_3
2022-07-10 17:25:21 +03:00
kssenii
4c88527c8e
Review fixes
2022-07-09 19:35:07 +02:00
kssenii
bf1c99daab
Refactoring of object storages, add LocalObjectStorage
2022-07-09 12:39:31 +02:00
Frank Chen
da57a993e4
Fix CI
2022-07-09 13:43:10 +08:00
Alexander Tokmakov
51130d723c
Update MergeTreeData.cpp
2022-07-08 20:17:29 +03:00
Raúl Marín
a965438eea
clearOldLogs: Don't report errors on concurrent deletes
2022-07-08 13:57:53 +02:00
Azat Khuzhin
b2f7f9c0f0
Fix fsync_part_directory for fetches
...
Before it produces the following error:
2022.07.07 23:40:03.131044 [ 44869 ] {} <Error> default.rep_data: auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(DB::StorageReplicatedMergeTree::LogEntryPtr &) const: Code: 107. DB::ErrnoException: Cannot open file /var/lib/clickhouse/data/tmp-fetch_36_25458_25458_0, errno: 2, strerror: No such file or directory. (FILE_DOESNT_EXIST), Stack trace (when copying this message, always include the lines below):
0. Poco::Exception::Exception() @ 0x1764634c in /usr/bin/clickhouse
1. DB::Exception::Exception() @ 0xb42caba in /usr/bin/clickhouse
2. DB::throwFromErrnoWithPath() @ 0xb42d79b in /usr/bin/clickhouse
3. DB::LocalDirectorySyncGuard::LocalDirectorySyncGuard() @ 0x1404bbd4 in /usr/bin/clickhouse
4. DB::DiskLocal::getDirectorySyncGuard() const @ 0x140466a4 in /usr/bin/clickhouse
5. DB::DataPartsExchange::Fetcher::downloadPartToDisk() @ 0x15122af1 in /usr/bin/clickhouse
6. DB::DataPartsExchange::Fetcher::fetchPart() @ 0x1511de39 in /usr/bin/clickhouse
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-08 13:48:51 +03:00
lgbo-ustc
e01b6cf8a5
update codes
2022-07-08 09:25:45 +08:00
Alexander Tokmakov
b0f8970d07
Merge pull request #38961 from Algunenano/reduce_zk_eexists
...
Stop reporting Zookeeper "Node exists" exceptions in system.errors when they are expected
2022-07-07 23:16:46 +03:00
Dmitry Novik
d1df66687b
Merge branch 'master' into group-by-use-nulls
2022-07-07 20:54:38 +02:00
Raúl Marín
9b0c7714e5
Style
2022-07-07 17:15:23 +02:00
Kseniia Sumarokova
48220f64d6
Merge pull request #38214 from bigo-sg/storagehive_alter
...
support alter on `StorageHive` table
2022-07-07 17:06:43 +02:00
Raúl Marín
c3935000a8
Avoid reporting ZK EEXIST errors on checkPartChecksumsAndCommit
2022-07-07 16:10:05 +02:00
Raúl Marín
33ffb71453
Avoid 'Node exists' errors when creating Ephemeral Locks
2022-07-07 16:10:04 +02:00
Alexander Tokmakov
fdd2db4ace
Move check for denied allocations ( #38858 )
...
* move check for denied allocations
* remove include
* make hardening softer
2022-07-07 14:24:36 +02:00
alesapin
1bd97fa6fa
Merge pull request #38851 from Algunenano/rmarin_read_only_on_drop
...
Deactivate mutations_finalizing_task during shutdown
2022-07-07 12:55:50 +02:00
Frank Chen
93dc109e36
Fix code
2022-07-07 17:44:19 +08:00
Frank Chen
d3d89f59ca
Add tracing support to distributed insert
2022-07-07 17:43:09 +08:00
Frank Chen
57a7e4a7c9
Remove old API reference
2022-07-07 17:42:35 +08:00
FArthur-cmd
8968ff8995
small inmprovements
2022-07-07 11:33:06 +03:00
Amos Bird
19bd3d8d9f
Nullable partition pruning fix
2022-07-07 16:02:38 +08:00
lgbo-ustc
337f379bc3
update codes
2022-07-07 09:45:49 +08:00
lgbo-ustc
cda081dc57
update codes
2022-07-07 09:41:52 +08:00
Vitaly Baranov
ed27987646
Merge pull request #38861 from vitlibar/backup-improvements-9
...
Backup Improvements 9
2022-07-07 02:24:47 +02:00
FArthur-cmd
a3407ba6db
Merge branch 'master' of github.com:FArthur-cmd/ClickHouse into annoy-2
2022-07-06 19:42:55 +03:00
Vitaly Baranov
7f84cf3968
Fix style.
2022-07-06 16:36:59 +02:00
Maksim Kita
b94489d52c
Merge pull request #38859 from kitaisreal/merge-tree-merge-disable-batch-optimization
...
MergeTree merge disable batch optimization
2022-07-06 15:59:40 +02:00
Vladimir C
3c7fdb604d
Merge pull request #35363 from vdimir/kv-join
2022-07-06 15:57:30 +02:00
FArthur-cmd
c94d6a57ec
add arrays
2022-07-06 14:39:49 +03:00
Raúl Marín
86d1c319f4
Remove unnecessary log
2022-07-06 12:23:45 +02:00
lgbo-ustc
aeb4f9c6be
fixed: crash caused by IHiveFile be shared among threads
2022-07-06 16:56:17 +08:00
lgbo-ustc
946a9b8c63
update codes
2022-07-06 09:26:02 +08:00
Kseniia Sumarokova
a26643a7a6
Merge pull request #38363 from kssenii/pg-add-auto-close
...
Support `auto_close` option for postgres engine connection
2022-07-06 01:10:05 +02:00
kssenii
e55169ebad
Fix
2022-07-05 20:15:35 +02:00
Vitaly Baranov
f9204315b5
Store columns.txt in backups for the Memory table engine too.
2022-07-05 19:03:20 +02:00
Kseniia Sumarokova
3e803006a2
Merge pull request #37171 from tchepavel/nats-integration
...
Nats Integration
2022-07-05 17:47:02 +02:00
Maksim Kita
bdc21737d5
MergeTree merge disable batch optimization
2022-07-05 16:15:00 +02:00
Vitaly Baranov
43d35eec1b
Write unfinished mutations to backup.
2022-07-05 14:51:09 +02:00
Raúl Marín
9e17ae365b
Deactivate mutations_finalizing_task during shutdown
2022-07-05 14:28:20 +02:00
vdimir
17903117d2
better
2022-07-05 10:46:08 +00:00
kssenii
7a4be3b748
Fix build, disable for freebsd
2022-07-05 12:10:49 +02:00
Kruglov Pavel
a6a9824261
Merge pull request #38362 from Avogar/better-exception-messages
...
Better exception messages on wrong table engines/functions argument types
2022-07-05 11:21:44 +02:00
lgbo-ustc
76813ee3c0
update codes
2022-07-05 11:49:55 +08:00
kssenii
b52084265c
Merge master
2022-07-04 21:37:43 +02:00
Kseniia Sumarokova
6c3d648f49
Merge branch 'master' into nats-integration
2022-07-04 21:06:07 +02:00
Kseniia Sumarokova
0cd228cab3
Merge pull request #38365 from kssenii/better-rabbit-adddress-setting
...
Fix RabbitMQ configuration via connection string setting
2022-07-04 20:53:43 +02:00
vdimir
c0cb588e45
Fix build
2022-07-04 17:28:34 +00:00
vdimir
43afda408b
do not call setStorageJoin when setting isn't set
2022-07-04 17:28:15 +00:00
vdimir
65ac97ce89
direct join: handle nullable, add tests
2022-07-04 17:28:14 +00:00
vdimir
8359753a85
Move DirectJoin to separate file
2022-07-04 17:28:11 +00:00
vdimir
5ed0911ff4
Fix rocksdb::getSlicedKeys
2022-07-04 17:28:10 +00:00
vdimir
085e70c7a3
wip left kvjoin
2022-07-04 17:28:10 +00:00
vdimir
980000d04c
Change getByKeys, add StorageEmbeddedRocksDB::getByKeysImpl
2022-07-04 17:28:09 +00:00
vdimir
744c692be3
wip key value join
2022-07-04 17:28:09 +00:00
vdimir
8bce6451c6
add method StorageEmbeddedRocksDB::getByKeys
2022-07-04 17:27:37 +00:00
Anton Popov
8b356f0e1b
better tests and comments
2022-07-04 15:06:43 +00:00
Dmitry Novik
c66e2c8095
Merge pull request #37155 from amosbird/projection-fix-two
...
Fix intermediate header in aggregate projection
2022-07-04 16:32:34 +02:00
FArthur-cmd
21df8ed12f
refactoring
2022-07-04 17:21:50 +03:00
Kruglov Pavel
6c4b6fa834
Merge branch 'master' into better-exception-messages
2022-07-04 15:07:31 +02:00
kssenii
cfff7c4c28
Merge master
2022-07-04 14:13:26 +02:00
FArthur-cmd
21b99e7304
Merge branch 'master' of https://github.com/Vector-Similarity-Search-for-ClickHouse/ClickHouse into annoy-2
2022-07-04 14:54:21 +03:00
Vitaly Baranov
1dba3d9f9c
Make temporary files in better places.
2022-07-04 11:02:58 +02:00
lgbo-ustc
932e596689
Merge remote-tracking branch 'ck/master' into storagehive_alter
2022-07-04 09:56:00 +08:00
Raúl Marín
838fd1e41e
Fix replication after improper merge process
2022-07-03 15:11:25 +02:00
Vitaly Baranov
92e0ee0b6f
More detailed error messages.
2022-07-03 14:20:19 +02:00
alesapin
b2db49dbf1
Merge branch 'master' into better_data_part_storage_builder
2022-07-02 15:39:44 +02:00
kssenii
831d2253ce
Merge master
2022-07-01 18:53:20 +02:00
Anton Popov
ef87e1207c
better support of read_in_order in case of fixed prefix of sorting key
2022-07-01 16:45:01 +00:00
Dmitry Novik
81dd90893e
Merge remote-tracking branch 'origin/master' into group-by-use-nulls
2022-07-01 16:24:05 +00:00
alesapin
0d7298f3a9
Fix typos
2022-07-01 15:26:27 +02:00
alesapin
57284e6a9d
Fix possible deadlocks with MergeTreeData::Transaction
2022-07-01 15:16:32 +02:00
Vitaly Baranov
cadb496894
Merge pull request #38537 from vitlibar/backup-improvements-8
...
Backup Improvements 8
2022-07-01 11:20:46 +02:00
Amos Bird
d73f975849
Fix intermediate header in aggregate projection
2022-07-01 16:47:32 +08:00
alesapin
09faa1641e
Some comments
2022-06-30 23:09:32 +02:00
alesapin
979565edf0
Merge branch 'master' into better_data_part_storage_builder
2022-06-30 22:54:42 +02:00
alesapin
eb5046ab26
Simplify everything
2022-06-30 22:51:27 +02:00
Alexander Tokmakov
60dcae9af0
Merge pull request #38627 from Algunenano/rmarin_fix_replicated_part_2
...
Adapt some more nodes to avoid issues with pre-22.4 replicas
2022-06-30 21:11:35 +03:00
Alexander Tokmakov
d2cbdc7c53
Update ReplicatedMergeTreeQueue.cpp
2022-06-30 17:23:53 +03:00
Maksim Kita
a0e5768956
Merge pull request #38623 from loyispa/patch
...
Fix typo
2022-06-30 16:02:36 +02:00
Vitaly Baranov
e367d96964
Fix style.
2022-06-30 15:10:33 +02:00
Raúl Marín
cbcd740dc1
Adapt some more nodes to avoid issues with pre-22.4 replicas
2022-06-30 15:10:09 +02:00
avogar
ee54c4f9b7
Add some fixes and add settings in docs
2022-06-30 12:41:56 +00:00
alesapin
1363b478d4
Merge branch 'fix_disk_tx_write_op' into better_data_part_storage_builder
2022-06-30 14:38:32 +02:00
Anton Popov
7c721578c7
Merge pull request #38320 from CurtizJ/dynamic-columns-16
...
Improve performace of insertion to columns of type JSON
2022-06-30 14:18:20 +02:00
alesapin
cb90aca2ef
Some changes in merge tree
2022-06-30 14:12:45 +02:00
loyispa
109c9bcbd5
Fix typo
2022-06-30 18:37:19 +08:00
Vitaly Baranov
5456bde4a2
Improve gathering metadata for storing ACL in backups.
2022-06-30 09:46:37 +02:00
Antonio Andelic
de264117fd
Merge pull request #38118 from bigo-sg/storagehive_struct_type
...
Add struct type support in `StorageHive`
2022-06-30 09:11:13 +02:00
Vitaly Baranov
031ca28fdc
Add test for partition clause. More checks for data compatibility on restore.
2022-06-30 08:37:18 +02:00
Vitaly Baranov
11b51d2878
Implement storing UDF in backups.
2022-06-30 08:37:17 +02:00
Vitaly Baranov
efbee5e723
Improve gathering metadata for backup - part 7.
2022-06-30 08:37:17 +02:00
Vitaly Baranov
01921ce9a3
Move most code from setTableStructure() to a separate function.
2022-06-30 08:37:17 +02:00
Vitaly Baranov
7689e0c36f
Improve gathering metadata for backup - part 6.
2022-06-30 08:37:17 +02:00
Vitaly Baranov
461a31f237
Improve gathering metadata for backup - part 2.
2022-06-30 08:37:17 +02:00