thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-02 20:42:04 +00:00
Code Issues Packages Projects Releases Wiki Activity
95,891 Commits 9,183 Branches 1,782 Tags 2.2 GiB
1e24a731f7
Commit Graph

9128 Commits

Author SHA1 Message Date
Nikolai Kochetov
fdb1c2545f
Merge pull request #39990 from ClickHouse/fetch_huge_column_memory_consumption 
Fix big memory usage for fetch.
 2022-08-09 21:10:22 +02:00
vdimir
ae1db8386b
Change type of insert_quorum to UInt64WithAuto 2022-08-09 17:14:02 +00:00
Maksim Kita
acbfcf440b
Merge branch 'master' into actions-dag-refactoring 2022-08-09 18:52:08 +02:00
Alexander Tokmakov
fe572104aa fix old tmp dirs cleanup 2022-08-09 18:44:51 +02:00
Lloyd-Pottiger
bb861bdb03 support for user setting 
Signed-off-by: Lloyd-Pottiger <yan1579196623@gamil.com>
 2022-08-09 21:42:51 +08:00
Maksim Kita
a576a55375 Fixed build 2022-08-09 15:03:59 +02:00
Kseniia Sumarokova
a1ecbefcdb
Merge pull request #39943 from ucasfl/file 
Fix read of StorageFile with virtual columns
 2022-08-09 10:57:51 +02:00
zhangxiao871
8bae5f24af Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into yandex-master 2022-08-09 15:49:03 +08:00
alexX512
c9c26d4898 Fix review 2022-08-08 21:34:24 +00:00
Yakov Olkhovskiy
d39e9f65de
Merge branch 'master' into fix-quota-key 2022-08-08 11:54:21 -04:00
Alexander Tokmakov
2c5da1466d Merge branch 'master' into replicated_database_improvements 2022-08-08 16:08:27 +02:00
Nikolai Kochetov
540ae64f37 Fix big memory usage for fetch. 2022-08-08 13:32:49 +00:00
Lloyd-Pottiger
3fd7c336c8 remove useless header 
Signed-off-by: Lloyd-Pottiger <yan1579196623@gamil.com>
 2022-08-08 21:03:14 +08:00
Lloyd-Pottiger
ff1931c735 support ttl for embeddedrocksdb 
Signed-off-by: Lloyd-Pottiger <yan1579196623@gamil.com>
 2022-08-08 20:51:59 +08:00
Igor Nikonov
86c5280db3 Merge remote-tracking branch 'origin/master' into skipping_sorting_step 2022-08-08 12:43:06 +00:00
vdimir
708747ca0b
Merge branch 'master' into refactor-prepared-sets 2022-08-08 14:27:18 +02:00
avogar
1cfb5b7cf1 Fix build 2022-08-08 12:17:18 +00:00
Kseniia Sumarokova
d1b436597f
Merge pull request #39940 from azat/file-log-fix 
Create metadata directory on CREATE for FileLog engine
 2022-08-08 11:39:17 +02:00
Kseniia Sumarokova
ec4a4d31ea
Fix style check 2022-08-08 11:23:57 +02:00
Alexey Milovidov
0921548a37
Merge pull request #39527 from pkit/pkit/fix_nats_tls 
fix nats-io TLS support
 2022-08-08 08:28:14 +03:00
alexX512
62f84b88eb Review fixes 2022-08-07 20:02:21 +00:00
alexX512
2be72560f4 Add loading of mark cache policy 2022-08-07 19:59:39 +00:00
alexX512
6bf29cb610 Change class LRUCache to class CachBase. Check running CacheBase with default pcahce policy SLRU 2022-08-07 19:59:30 +00:00
Kseniia Sumarokova
895639644e
Update src/Storages/StorageFile.cpp 2022-08-07 14:17:42 +02:00
flynn
4fa1762f96
Merge branch 'master' into file 2022-08-07 14:22:08 +08:00
flynn
384a7ae901 Fix read of StorageFile with virtual columns 2022-08-06 17:29:33 +00:00
Azat Khuzhin
d0189e8f66 Create metadata directory on CREATE for FileLog engine 
This will allow to distinguish really corrupted data, since right now if
you will CREATE/DETACH/ATTACH such engine you will have the following
error [1]:

    2022.08.05 20:02:20.726398 [ 696405 ] {} <Error> StorageFileLog (file_log): Metadata files of table file_log are lost.

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/39926/72961328f68b1ec05300d6dc4411a87618a2f46b/stress_test__debug_.html

Likely that previously it was not created to avoid creating empty
directories, however this should be a problem I guess.

Refs: #25969 (@ucasfl)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-08-06 13:54:49 +03:00
Alexander Tokmakov
f2c4cad255 add flag that enables automatic canversion from Ordinary to Atomic 2022-08-05 21:41:25 +02:00
Alexander Gololobov
b57df6d71d
Merge branch 'master' into wait_for_mutation_race 2022-08-05 18:51:39 +02:00
avogar
9b1a267203 Refactor, remove TTL, add size limit, add system table and system query 2022-08-05 16:20:15 +00:00
Alexander Tokmakov
25a9bf5deb
Merge pull request #39922 from ClickHouse/fix_flaky_tests_commit_obsolete_part 
Fix flaky tests (`Tried to commit obsolete part`)
 2022-08-05 14:36:43 +03:00
kssenii
91f4143b6e Merge master 2022-08-05 12:14:08 +02:00
Alexander Tokmakov
4bf7a2ca3a fix warning 2022-08-05 12:05:50 +02:00
Nikolai Kochetov
2fe893ba67
Merge pull request #25563 from ClickHouse/use-dag-in-key-condition 
Use ActionsDAG in KeyCondition
 2022-08-05 11:22:46 +02:00
Constantine Peresypkin
10c76917ea fix nats-io TLS support 
nats-io library needs `NATS_HAS_TLS` define to correctly compile-in TLS
support

fixes #39525
 2022-08-05 11:20:05 +02:00
kssenii
7a9b0bc47f Merge master 2022-08-05 01:48:52 +02:00
Anton Popov
0492ef63a4
Merge pull request #39464 from CurtizJ/fix-custom-serializations 
Fix columns of type `Object` with sparse serialization
 2022-08-05 00:55:41 +02:00
Igor Nikonov
246f7b71b6
Merge branch 'master' into skipping_sorting_step 2022-08-05 00:03:54 +02:00
Amos Bird
6a3a9dcf96
Fix building aggregate projections when external aggregation is on (#39671) 2022-08-04 23:29:26 +02:00
Alexander Gololobov
eb3e2ed092
Merge branch 'master' into wait_for_mutation_race 2022-08-04 20:27:09 +02:00
Alexander Gololobov
f5e358ecc5 Make sure that the current replica is the first on the list 2022-08-04 20:26:39 +02:00
Yakov Olkhovskiy
23037daf17
Merge branch 'master' into fix-quota-key 2022-08-04 12:14:49 -04:00
Nikolai Kochetov
60599197b2 Review fixes. 2022-08-04 15:23:10 +00:00
Nikolai Kochetov
658a269d56
Merge branch 'master' into use-dag-in-key-condition 2022-08-04 16:18:40 +02:00
alesapin
1eb28f5f5f
Merge pull request #39860 from ClickHouse/avoid_additional_disk_touch 
Better total part size calculation on mutation
 2022-08-04 14:21:31 +02:00
Alexander Gololobov
7caf4c210e Update local queue to see current mutation entry 2022-08-04 14:06:19 +02:00
Vladimir C
a627b00c43
Merge branch 'master' into refactor-prepared-sets 2022-08-04 13:27:38 +02:00
Kruglov Pavel
235649cb98
Merge pull request #39458 from Avogar/fix-cancel-insert-into-function 
Fix WriteBuffer finalize when cancel insert into function
 2022-08-04 13:02:08 +02:00
Nikolay Degterinsky
71cb055ecc
Merge pull request #39812 from guowangy/applyFunction-multi-thread 
KeyCondition: optimize applyFunction in multi-thread scenario
 2022-08-04 01:36:47 +02:00
Yakov Olkhovskiy
2e34b384c1 update tcp protocol, add quota_key 2022-08-03 15:44:08 -04:00
Anton Popov
04f6a85070 fix mutations 2022-08-03 17:34:46 +00:00
Maksim Kita
e6efb47aa3
Merge pull request #39850 from kitaisreal/select-query-has-join-method 
Add hasJoin method into ASTSelectQuery
 2022-08-03 18:50:25 +02:00
Anton Popov
5da32fafb3 fix ALTER MODIFY COLUMN from nested 2022-08-03 15:08:38 +00:00
alesapin
f608e62c27 Fix call 2022-08-03 14:17:31 +02:00
alesapin
56a4d26e87 Better total part size calculation on mutation 2022-08-03 14:15:45 +02:00
Maksim Kita
3e6b663020 ASTSelectQuery added hasJoin method 2022-08-03 11:25:45 +02:00
Anton Popov
a333cc4146 Merge remote-tracking branch 'upstream/master' into HEAD 2022-08-02 12:57:43 +00:00
Igor Nikonov
56697125d3 Merge remote-tracking branch 'origin/master' into distinct_sorted_simplify 2022-08-02 11:18:46 +00:00
Igor Nikonov
4bf42ce867 Merge remote-tracking branch 'origin/master' into skipping_sorting_step 2022-08-02 10:50:25 +00:00
Wangyang Guo
91e3e2f18b KeyCondition: optimize applyFunction in multi-thread scenario 
Construct and deconstruct args (ColumnsWithTypeAndName) will inc/dec
ref_count (actually this is a atomic lock inc/dec operation) to share_ptr,
which may share the same DataTypePtr among different threads. This will
have a lock contention issue in large parallel situation.

The patch try to minimize `args` scope and reduce unnecessary
construct/destory of instances. It will improve the performance in
multi-thread cases.
 2022-08-02 16:16:16 +08:00
Alexander Tokmakov
1567b65473 Merge branch 'master' into replicated_database_improvements 2022-08-01 19:12:37 +02:00
Alexey Milovidov
3de747a6e2
Merge pull request #39746 from ClickHouse/cleanup-projection-setting 
Cleanup usages of `allow_experimental_projection_optimization` setting, part 1
 2022-08-01 19:21:30 +03:00
Robert Schulze
bf574b9154
Merge pull request #39760 from ClickHouse/bit-fiddling 
Use std::popcount, ::countl_zero, ::countr_zero functions
 2022-08-01 17:04:51 +02:00
Antonio Andelic
9ec27c0ab4
Merge pull request #39757 from ClickHouse/fix-rocksdb-filter-with-params 
Use params correctly for filtering by key in EmbeddedRocksDB
 2022-08-01 12:17:10 +02:00
Dmitry Novik
ab5a147065
Merge pull request #38725 from azat/fix-order-by-projection 
Fix ORDER BY that matches projections ORDER BY
 2022-08-01 00:32:59 +02:00
Nikolai Kochetov
22fbfe19a4 Merge branch 'master' into use-dag-in-key-condition 2022-07-31 21:54:12 +02:00
kssenii
e5f4a619ed Merge master 2022-07-31 20:24:40 +03:00
Robert Schulze
a7734672b9
Use std::popcount, ::countl_zero, ::countr_zero functions 
- Introduced with the C++20 <bit> header

- The problem with __builtin_c(l|t)z() is that 0 as input has an
  undefined result (*) and the code did not always check. The std::
  versions do not have this issue.

- In some cases, we continue to use buildin_c(l|t)z(), (e.g. in
  src/Common/BitHelpers.h) because the std:: versions only accept
  unsigned inputs (and they also check that) and the casting would be
  ugly.

(*) https://gcc.gnu.org/onlinedocs/gcc/Other-Builtins.html
 2022-07-31 15:16:51 +00:00
Robert Schulze
52d08d9db4
Merge pull request #39520 from ClickHouse/no-split-binary 
Remove SPLIT_BINARY
 2022-07-31 14:23:31 +02:00
Antonio Andelic
eea1aaef2c Use params for filtering by key in EmbeddedRocksDB 2022-07-31 10:44:01 +00:00
Igor Nikonov
7f0adb5eb0 Merge remote-tracking branch 'origin/master' into skipping_sorting_step 2022-07-31 07:07:36 +00:00
Anton Popov
74eb8119ef
Merge branch 'master' into fix-custom-serializations 2022-07-31 01:51:52 +02:00
Alexey Milovidov
8fb70abe3e
Merge pull request #39178 from azat/dist-insert-log 
Add connection info for Distributed sends log message
 2022-07-31 02:22:22 +03:00
Alexey Milovidov
441f2feb49 Cleanup usages of allow_experimental_projection_optimization setting, part 1 2022-07-30 23:40:21 +02:00
Igor Nikonov
a7cfad105e
Merge branch 'master' into distinct_sorted_simplify 2022-07-30 21:57:53 +02:00
Igor Nikonov
7b0b38e997 DistinctSortedTransform works only if columns contains sort prefix of 
sort description
 2022-07-29 20:01:07 +00:00
Kruglov Pavel
ccd1e1bdb8
Merge branch 'master' into fix-cancel-insert-into-function 2022-07-29 20:27:32 +02:00
Maksim Kita
8fc6bad4f4 Join enums refactoring 2022-07-29 18:35:05 +02:00
Alexander Tokmakov
6f88065cc1 check metadata consistency 2022-07-29 18:33:16 +02:00
Nikolai Kochetov
59a11b32ad
Merge branch 'master' into use-dag-in-key-condition 2022-07-29 17:01:33 +02:00
Igor Nikonov
fb5799fe7f More generic getInputOrderInfo() in query info 2022-07-29 14:36:36 +00:00
Igor Nikonov
ae82bfb7f3 Merge remote-tracking branch 'origin/master' into distinct_sorted_simplify 2022-07-29 13:12:03 +00:00
Robert Schulze
3d1797f75f
Merge remote-tracking branch 'origin/master' into no-split-binary 2022-07-29 12:17:43 +00:00
Anton Popov
57e4fb2e30 Merge remote-tracking branch 'upstream/master' into HEAD 2022-07-29 11:42:11 +00:00
Anton Popov
bb551566e7 fix build 2022-07-29 11:41:53 +00:00
Azat Khuzhin
498c8b3c52 Fix clang-tidy in utils/examples 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-07-29 11:36:51 +03:00
Robert Schulze
199e254777
Merge remote-tracking branch 'origin/master' into no-split-binary 2022-07-28 15:54:22 +00:00
Anton Popov
04fd72cdef fix alter column rename 2022-07-28 13:19:23 +00:00
Vitaly Baranov
f8ebe76826
Merge pull request #39503 from vitlibar/improve-system-backups-table 
Improve system.backups table
 2022-07-27 20:25:39 +02:00
Vladimir C
115506356c
Merge branch 'master' into refactor-prepared-sets 2022-07-27 19:57:23 +02:00
Nikolai Kochetov
6206e468a4 Ignor materialize function for ActionsDAG in PK analysis. 2022-07-27 17:02:48 +00:00
Nikolai Kochetov
51c4c9c957 Fixing build. 2022-07-27 15:45:23 +00:00
Anton Popov
56039c8780 fix alter of LowCardinality 2022-07-27 15:35:09 +00:00
Anton Popov
4cf0353892 try to fix Nested 2022-07-27 14:05:16 +00:00
Nikolai Kochetov
f9a8a4cd32 Fixing read-in-order for Merge 2022-07-27 12:00:55 +00:00
vdimir
8eecb9ef82
upd PreparedSets: rename/change signature of methods, add comments 2022-07-27 11:22:16 +00:00
Alexander Tokmakov
8fc075a527
Merge pull request #39619 from ClickHouse/try-do-enable-global-allocations-for-attach 
Replace MemoryTrackerBlockerInThread to LockMemoryExceptionInThread
 2022-07-27 13:13:04 +03:00
Kruglov Pavel
381ea139c2
Merge branch 'master' into schema-inference-cache 2022-07-27 11:35:36 +02:00
Vitaly Baranov
794eeb5d51 Split "total_size" to "uncompressed_size" and "compressed_size". 2022-07-27 10:36:56 +02:00
Vitaly Baranov
a79fc17235 Add test for system.backups 2022-07-27 09:04:10 +02:00
Vitaly Baranov
1cfe0b10f7 Add columns "total_size" and "num_files" to system.backups 2022-07-27 09:04:10 +02:00
Vitaly Baranov
35c267b3b1 Replace column "status_changed_time" with columns "start_time" and "end_time". 2022-07-27 09:04:10 +02:00
Vitaly Baranov
fc16a15ecf Rename column "uuid" -> "id" in system.backups and allow user to set it in a query. 2022-07-27 09:04:10 +02:00
Vitaly Baranov
131019ba49 Rename column "backup_name" -> "name" in system.backups. 2022-07-27 09:04:10 +02:00
Vitaly Baranov
afd0982187 Remove column "internal" from system.backups 2022-07-27 09:04:10 +02:00
Alexander Gololobov
a64aa00869
Merge pull request #37893 from zhangjmruc/feature/sql-standard-delete 
Support SQL standard "delete from ... where ..." syntax and lightweight implementation on merge tree tables
 2022-07-26 23:39:07 +02:00
Nikolai Kochetov
1adb021df6 Fixing tests. 2022-07-26 18:42:27 +00:00
vdimir
5ce2960f03
Get rid of SelectQueryInfoBase -> SelectQueryInfo 2022-07-26 18:39:09 +00:00
vdimir
d9928ac93d
Add methods to SubqueryForSet, do not use refernce to SetPtr 2022-07-26 18:39:09 +00:00
vdimir
1e3fa2e01f
Refactor PreparedSets/SubqueryForSet 2022-07-26 18:39:02 +00:00
Anton Popov
9321ca34cf try to fix Nested 2022-07-26 17:32:37 +00:00
Nikolai Kochetov
33ce662d3e Replace MemoryTrackerBlockerInThread to LockMemoryExceptionInThread in some places. Reduced MemoryTrackerBlockerInThread level to User. 2022-07-26 15:22:00 +00:00
Kruglov Pavel
c683cb252f
Merge pull request #39227 from amosbird/rename-log1 
Rename log when rename merge tree tables
 2022-07-26 17:12:44 +02:00
Nikolai Kochetov
d7cc3831ea Push predicate over StorageMerge. 2022-07-26 15:01:39 +00:00
Nikolai Kochetov
c5110bb164 Add a query plan step for StorageMerge 2022-07-26 14:43:05 +00:00
jianmei zhang
c1834d183b ReplicatedMergeTree support lightweight delete 2022-07-26 11:50:09 +08:00
Alexander Gololobov
25deba2c1b
Merge branch 'master' into feature/sql-standard-delete 2022-07-25 22:13:20 +02:00
Nikolai Kochetov
be9c7ed52c Add ReadFromMerge step. 2022-07-25 19:41:43 +00:00
Roman Vasin
b462366415 Merge branch 'master' of github.com:ClickHouse/ClickHouse into ADQM-485 2022-07-25 17:55:47 +00:00
Roman Vasin
b412ea5f6d Improve generateRandom() for Date32; fix tests 01087_table_function_generate, 01277_fromUnixTimestamp64, 01691_DateTime64_clamp and 01702_toDateTime_from_string_clamping 2022-07-25 17:06:11 +00:00
Alexander Tokmakov
778dcf6994
Merge pull request #39230 from Algunenano/improve_replicated_merge_logging 
Improve logging around replicated merges
 2022-07-25 18:42:12 +03:00
Alexander Gololobov
48de02a7b8 Capitalized const name 2022-07-25 16:32:16 +02:00
Alexander Gololobov
7b44950d1d More detailed comment 2022-07-25 16:14:33 +02:00
Nikolai Kochetov
b70be40804
Merge branch 'master' into use-dag-in-key-condition 2022-07-25 14:30:22 +02:00
Nikolai Kochetov
1ce8e714d3 Fixing build. 2022-07-25 12:30:01 +00:00
Alexey Milovidov
388d06fda1
Merge pull request #39535 from ClickHouse/stringref 
Less usage of StringRef
 2022-07-25 04:06:11 +03:00
Alexander Gololobov
460950ecdc
Merge branch 'master' into feature/sql-standard-delete 2022-07-24 21:27:22 +02:00
Robert Schulze
4333750985
Less usage of StringRef 
... replaced by std::string_view, see #39262
 2022-07-24 18:33:52 +00:00
Alexey Milovidov
0209012e5e
Merge pull request #39517 from pkit/pkit/fix_examples 
fix `-DENABLE_EXAMPLES=1` in master
 2022-07-24 17:41:23 +03:00
Igor Nikonov
eacc57b6a4 Simplify DistinctSorted 
+ use ordinary Distinct for deduplication in MergeTree when no sorting provided
 2022-07-24 14:26:25 +00:00
Azat Khuzhin
22d8e532ed Fix ORDER BY that matches projections ORDER BY 
In case of projection is seleted and it is comlete (all parts has such
projection) and ORDER BY matches projection ORDER BY, and
optimize_read_in_order=1 (default), then the sorting is simply not done,
because projections has separate plan, and so
InterpreterSelectQuery::executeOrder() -> SortingStep has
pipeline.getNumStreams() == 0 and it cannot do sorting.

Fix this, by adding sorting when creating plan for reading from
projections.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-07-24 15:23:13 +03:00
Kseniia Sumarokova
04944ff6f5
Merge pull request #39509 from monadbobo/remove_duplicate_code 
Removed some duplicate code.
 2022-07-24 14:24:40 +03:00
Alexander Gololobov
6692770113 Allow to update "system" virtual columns such as _row_exists 2022-07-24 12:21:47 +02:00
Alexander Gololobov
c8b3c574a4 Disable lightweight delete if table has projections 2022-07-24 12:21:47 +02:00
Alexander Gololobov
be64b45583 Properly handle empty parts list 2022-07-24 12:21:47 +02:00
Alexander Gololobov
594195451e Cleanups 2022-07-24 12:21:18 +02:00
Alexey Milovidov
071374b152 Remove SPLIT_BINARY 2022-07-24 01:15:54 +02:00
Kseniia Sumarokova
f146183400
Merge pull request #39491 from ClickHouse/remove_unused_constructor 
Remove unused constructor
 2022-07-23 21:16:39 +02:00
mergify[bot]
c01ff2d38a
Merge branch 'master' into improve_replicated_merge_logging 2022-07-23 19:10:10 +00:00
Constantine Peresypkin
d645d03e89 fix -DENABLE_EXAMPLES=1 in master 2022-07-23 19:00:52 +02:00
Kseniia Sumarokova
77ddb8eda0
Merge pull request #39506 from CurtizJ/fix-direct-io 
Fix reading from async read buffer with direct io
 2022-07-23 06:08:48 +02:00
Anton Popov
3d03b2714b try to fix Nested 2022-07-22 23:05:11 +00:00
Nikolai Kochetov
dd65a36c59 Fix another one test. 2022-07-22 19:11:25 +00:00
Nikolai Kochetov
53175e43f3 Better support for projections. 2022-07-22 19:11:25 +00:00
Simon Liu
2305f282dc Removed some duplicate code. 2022-07-23 02:27:17 +08:00
Anton Popov
49627aa554 Merge remote-tracking branch 'upstream/master' into HEAD 2022-07-22 17:16:06 +00:00
Anton Popov
c1a5514e46 fix reading from async read buffer with direct io 2022-07-22 16:58:30 +00:00
Alexander Tokmakov
bed2206ae9
Merge pull request #39460 from ClickHouse/remove_some_dead_and_commented_code 
Remove some dead and commented code
 2022-07-22 13:24:34 +03:00
alesapin
69984b28e0 Remove unused constructor 2022-07-22 12:21:10 +02:00
Robert Schulze
99579ab440
Merge pull request #39364 from ClickHouse/stringref-to-string_view 
Even less usage of StringRef
 2022-07-22 09:30:49 +02:00