thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-15 19:02:04 +00:00
Code Issues Packages Projects Releases Wiki Activity
96,998 Commits 9,326 Branches 1,783 Tags 2.1 GiB
7f46b6a48b
Commit Graph

9332 Commits

Author SHA1 Message Date
Igor Nikonov
d83bea626c Merge remote-tracking branch 'origin/master' into skipping_sorting_step 2022-08-13 21:46:34 +00:00
Alexey Milovidov
201cf69854
Merge pull request #39470 from nicelulu/issues_39469 
Fix issues 39469
 2022-08-13 03:13:28 +03:00
Alexey Milovidov
0e450787f3 Edit the documentation 2022-08-12 21:24:27 +02:00
Alexey Milovidov
a88e713102 Disable zero-copy replication by default 2022-08-12 21:22:11 +02:00
Alexander Tokmakov
46aaf842f8
Merge pull request #40031 from ClickHouse/fix_old_tmp_dirs_cleanup 
Fix old temporary directories cleanup
 2022-08-12 20:31:20 +03:00
Kseniia Sumarokova
a6cfc7bc3b
Merge pull request #34651 from alexX512/master 
New caching strategies
 2022-08-12 17:23:37 +02:00
Anton Popov
4bd50bb06c
Merge branch 'master' into distinct_sorted_simplify 2022-08-12 17:11:18 +02:00
Antonio Andelic
c6b3a897d2 Test replicated table without ZK, small fixes 2022-08-12 11:56:46 +00:00
Alexander Tokmakov
e691888267 fix build 2022-08-12 13:03:57 +02:00
Alexander Tokmakov
d24b9874bc fix Cannot quickly remove directory 2022-08-12 12:47:25 +02:00
Nikolai Kochetov
86061d945e
Merge pull request #39826 from amosbird/projection-row-policy-fix 
Primary key analysis for row_policy_filter and additional_filter
 2022-08-12 11:32:58 +02:00
Antonio Andelic
1c0f85ccdb Attach table in background 2022-08-12 09:32:13 +00:00
kssenii
768a207176 Merge master 2022-08-11 22:25:53 +02:00
Kseniia Sumarokova
be69169f97
Merge pull request #38048 from KinderRiven/decoupling_cache_function_and_algorithm 
Decoupling local cache function and cache algorithm
 2022-08-11 16:49:56 +02:00
Igor Nikonov
75f6fcfa70 Merge remote-tracking branch 'origin/master' into skipping_sorting_step 2022-08-11 12:35:55 +00:00
Vladimir C
1b44cb5c97
Merge pull request #39974 from ClickHouse/vdimir/setting-num-with-auto 
Add setting type to support special 'auto' value
 2022-08-11 14:25:10 +02:00
Amos Bird
fa8fab2e8f
Fix KeyCondition with other filters 2022-08-11 19:20:44 +08:00
Jianmei Zhang
d1051d822c
Use getSerializedFileExtension() to get correct file extension for index (#40095) 2022-08-11 12:39:40 +02:00
Maksim Kita
6bec0f5854
Merge pull request #38956 from vdimir/dict-join-refactoring 
Join with dictionary refactoring
 2022-08-11 11:54:11 +02:00
Vladimir C
2d44e6c458
Merge pull request #39343 from vdimir/refactor-prepared-sets 
Refactor PreparedSets/SubqueryForSet
 2022-08-11 11:19:18 +02:00
Alexey Milovidov
8374f31306
Merge pull request #39425 from arenadata/ADQM-485 
Add support of dates from year 1900 to 2299 for Date32 and DateTime64
 2022-08-11 05:01:53 +03:00
Alexey Milovidov
38fc0cc783
Merge branch 'master' into issues_39469 2022-08-11 04:45:06 +03:00
KinderRiven
9d83b93e88 fix rebase 2022-08-11 00:11:40 +08:00
vdimir
5fea2091ac
Embed IKeyValue impl into IDictionary.h 2022-08-10 15:58:15 +00:00
vdimir
90fa2ed8c1
better code for join with dict 2022-08-10 14:20:29 +00:00
vdimir
44c688332a
IKeyValueEntity is not inheritor of IStorage 2022-08-10 14:20:28 +00:00
vdimir
ad91c16ba0
Rename join_common -> JoinUtils 2022-08-10 14:20:28 +00:00
vdimir
f7b130b0cb
Rename IKVStorage.h -> IKeyValueStorage.h 2022-08-10 14:20:28 +00:00
Alexander Tokmakov
fff903ee81 fix 2022-08-10 15:48:56 +02:00
vdimir
b7c5c54181
Fix build 2022-08-10 13:43:55 +00:00
Lloyd-Pottiger
ee88aed42b style check 
Signed-off-by: Lloyd-Pottiger <yan1579196623@gamil.com>
 2022-08-10 21:39:48 +08:00
Lloyd-Pottiger
5488a392ab address comments 
Signed-off-by: Lloyd-Pottiger <yan1579196623@gamil.com>
 2022-08-10 21:20:25 +08:00
vdimir
5eb4cd39e0
Merge branch 'master' into refactor-prepared-sets 2022-08-10 11:47:49 +00:00
Alexander Tokmakov
54614f5b98 Merge branch 'master' into fix_old_tmp_dirs_cleanup 2022-08-10 13:13:04 +02:00
kssenii
5c3227ba56 Merge master 2022-08-10 12:00:34 +02:00
FArthur-cmd
0ab937afbb small update 2022-08-10 09:55:07 +00:00
Maksim Kita
aff8149f5c
Merge pull request #39998 from kitaisreal/actions-dag-refactoring 
ActionsDAG rename index to outputs
 2022-08-10 11:44:18 +02:00
Alexey Milovidov
ddd79e6c05
Merge pull request #40038 from ClickHouse/non-significant-changes 
Non-significant changes
 2022-08-10 07:19:03 +03:00
Yakov Olkhovskiy
ffbaf72bab
Merge pull request #39874 from ClickHouse/fix-quota-key 
Fix quota_key
 2022-08-10 00:12:44 -04:00
Lloyd-Pottiger
9e04d0593b style check 
Signed-off-by: Lloyd-Pottiger <yan1579196623@gamil.com>
 2022-08-10 10:34:37 +08:00
Igor Nikonov
754a9fb096 Merge remote-tracking branch 'origin/master' into skipping_sorting_step 2022-08-09 22:20:17 +00:00
kssenii
0dda03c94b Fix checks 2022-08-10 00:06:58 +02:00
Alexander Tokmakov
060b270929 fix 2022-08-09 23:16:08 +02:00
Alexey Milovidov
78d3079f0c Non-significant changes 2022-08-09 21:32:42 +02:00
Nikolai Kochetov
fdb1c2545f
Merge pull request #39990 from ClickHouse/fetch_huge_column_memory_consumption 
Fix big memory usage for fetch.
 2022-08-09 21:10:22 +02:00
vdimir
ae1db8386b
Change type of insert_quorum to UInt64WithAuto 2022-08-09 17:14:02 +00:00
Maksim Kita
acbfcf440b
Merge branch 'master' into actions-dag-refactoring 2022-08-09 18:52:08 +02:00
Alexander Tokmakov
fe572104aa fix old tmp dirs cleanup 2022-08-09 18:44:51 +02:00
Lloyd-Pottiger
bb861bdb03 support for user setting 
Signed-off-by: Lloyd-Pottiger <yan1579196623@gamil.com>
 2022-08-09 21:42:51 +08:00
Maksim Kita
a576a55375 Fixed build 2022-08-09 15:03:59 +02:00
Kseniia Sumarokova
a1ecbefcdb
Merge pull request #39943 from ucasfl/file 
Fix read of StorageFile with virtual columns
 2022-08-09 10:57:51 +02:00
zhangxiao871
8bae5f24af Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into yandex-master 2022-08-09 15:49:03 +08:00
alexX512
c9c26d4898 Fix review 2022-08-08 21:34:24 +00:00
Yakov Olkhovskiy
d39e9f65de
Merge branch 'master' into fix-quota-key 2022-08-08 11:54:21 -04:00
Alexander Tokmakov
2c5da1466d Merge branch 'master' into replicated_database_improvements 2022-08-08 16:08:27 +02:00
Nikolai Kochetov
540ae64f37 Fix big memory usage for fetch. 2022-08-08 13:32:49 +00:00
Lloyd-Pottiger
3fd7c336c8 remove useless header 
Signed-off-by: Lloyd-Pottiger <yan1579196623@gamil.com>
 2022-08-08 21:03:14 +08:00
Lloyd-Pottiger
ff1931c735 support ttl for embeddedrocksdb 
Signed-off-by: Lloyd-Pottiger <yan1579196623@gamil.com>
 2022-08-08 20:51:59 +08:00
Igor Nikonov
86c5280db3 Merge remote-tracking branch 'origin/master' into skipping_sorting_step 2022-08-08 12:43:06 +00:00
vdimir
708747ca0b
Merge branch 'master' into refactor-prepared-sets 2022-08-08 14:27:18 +02:00
avogar
1cfb5b7cf1 Fix build 2022-08-08 12:17:18 +00:00
Kseniia Sumarokova
d1b436597f
Merge pull request #39940 from azat/file-log-fix 
Create metadata directory on CREATE for FileLog engine
 2022-08-08 11:39:17 +02:00
Kseniia Sumarokova
ec4a4d31ea
Fix style check 2022-08-08 11:23:57 +02:00
Alexey Milovidov
0921548a37
Merge pull request #39527 from pkit/pkit/fix_nats_tls 
fix nats-io TLS support
 2022-08-08 08:28:14 +03:00
alexX512
62f84b88eb Review fixes 2022-08-07 20:02:21 +00:00
alexX512
2be72560f4 Add loading of mark cache policy 2022-08-07 19:59:39 +00:00
alexX512
6bf29cb610 Change class LRUCache to class CachBase. Check running CacheBase with default pcahce policy SLRU 2022-08-07 19:59:30 +00:00
Kseniia Sumarokova
895639644e
Update src/Storages/StorageFile.cpp 2022-08-07 14:17:42 +02:00
flynn
4fa1762f96
Merge branch 'master' into file 2022-08-07 14:22:08 +08:00
flynn
384a7ae901 Fix read of StorageFile with virtual columns 2022-08-06 17:29:33 +00:00
Azat Khuzhin
d0189e8f66 Create metadata directory on CREATE for FileLog engine 
This will allow to distinguish really corrupted data, since right now if
you will CREATE/DETACH/ATTACH such engine you will have the following
error [1]:

    2022.08.05 20:02:20.726398 [ 696405 ] {} <Error> StorageFileLog (file_log): Metadata files of table file_log are lost.

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/39926/72961328f68b1ec05300d6dc4411a87618a2f46b/stress_test__debug_.html

Likely that previously it was not created to avoid creating empty
directories, however this should be a problem I guess.

Refs: #25969 (@ucasfl)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-08-06 13:54:49 +03:00
Alexander Tokmakov
f2c4cad255 add flag that enables automatic canversion from Ordinary to Atomic 2022-08-05 21:41:25 +02:00
Alexander Gololobov
b57df6d71d
Merge branch 'master' into wait_for_mutation_race 2022-08-05 18:51:39 +02:00
avogar
9b1a267203 Refactor, remove TTL, add size limit, add system table and system query 2022-08-05 16:20:15 +00:00
Alexander Tokmakov
25a9bf5deb
Merge pull request #39922 from ClickHouse/fix_flaky_tests_commit_obsolete_part 
Fix flaky tests (`Tried to commit obsolete part`)
 2022-08-05 14:36:43 +03:00
kssenii
91f4143b6e Merge master 2022-08-05 12:14:08 +02:00
Alexander Tokmakov
4bf7a2ca3a fix warning 2022-08-05 12:05:50 +02:00
Nikolai Kochetov
2fe893ba67
Merge pull request #25563 from ClickHouse/use-dag-in-key-condition 
Use ActionsDAG in KeyCondition
 2022-08-05 11:22:46 +02:00
Constantine Peresypkin
10c76917ea fix nats-io TLS support 
nats-io library needs `NATS_HAS_TLS` define to correctly compile-in TLS
support

fixes #39525
 2022-08-05 11:20:05 +02:00
kssenii
7a9b0bc47f Merge master 2022-08-05 01:48:52 +02:00
Anton Popov
0492ef63a4
Merge pull request #39464 from CurtizJ/fix-custom-serializations 
Fix columns of type `Object` with sparse serialization
 2022-08-05 00:55:41 +02:00
Igor Nikonov
246f7b71b6
Merge branch 'master' into skipping_sorting_step 2022-08-05 00:03:54 +02:00
Amos Bird
6a3a9dcf96
Fix building aggregate projections when external aggregation is on (#39671) 2022-08-04 23:29:26 +02:00
Alexander Gololobov
eb3e2ed092
Merge branch 'master' into wait_for_mutation_race 2022-08-04 20:27:09 +02:00
Alexander Gololobov
f5e358ecc5 Make sure that the current replica is the first on the list 2022-08-04 20:26:39 +02:00
Yakov Olkhovskiy
23037daf17
Merge branch 'master' into fix-quota-key 2022-08-04 12:14:49 -04:00
Nikolai Kochetov
60599197b2 Review fixes. 2022-08-04 15:23:10 +00:00
Nikolai Kochetov
658a269d56
Merge branch 'master' into use-dag-in-key-condition 2022-08-04 16:18:40 +02:00
FArthur-cmd
a2eafae157 fix bug with limit in order by 2022-08-04 12:33:18 +00:00
alesapin
1eb28f5f5f
Merge pull request #39860 from ClickHouse/avoid_additional_disk_touch 
Better total part size calculation on mutation
 2022-08-04 14:21:31 +02:00
Alexander Gololobov
7caf4c210e Update local queue to see current mutation entry 2022-08-04 14:06:19 +02:00
Vladimir C
a627b00c43
Merge branch 'master' into refactor-prepared-sets 2022-08-04 13:27:38 +02:00
Kruglov Pavel
235649cb98
Merge pull request #39458 from Avogar/fix-cancel-insert-into-function 
Fix WriteBuffer finalize when cancel insert into function
 2022-08-04 13:02:08 +02:00
Nikolay Degterinsky
71cb055ecc
Merge pull request #39812 from guowangy/applyFunction-multi-thread 
KeyCondition: optimize applyFunction in multi-thread scenario
 2022-08-04 01:36:47 +02:00
Yakov Olkhovskiy
2e34b384c1 update tcp protocol, add quota_key 2022-08-03 15:44:08 -04:00
Anton Popov
04f6a85070 fix mutations 2022-08-03 17:34:46 +00:00
Maksim Kita
e6efb47aa3
Merge pull request #39850 from kitaisreal/select-query-has-join-method 
Add hasJoin method into ASTSelectQuery
 2022-08-03 18:50:25 +02:00
Anton Popov
5da32fafb3 fix ALTER MODIFY COLUMN from nested 2022-08-03 15:08:38 +00:00
alesapin
f608e62c27 Fix call 2022-08-03 14:17:31 +02:00
alesapin
56a4d26e87 Better total part size calculation on mutation 2022-08-03 14:15:45 +02:00
Maksim Kita
3e6b663020 ASTSelectQuery added hasJoin method 2022-08-03 11:25:45 +02:00
Anton Popov
a333cc4146 Merge remote-tracking branch 'upstream/master' into HEAD 2022-08-02 12:57:43 +00:00
Frank Chen
1c9b462c80 Merge remote-tracking branch 'origin/master' into tracing_context_propagation 2022-08-02 20:18:22 +08:00
Igor Nikonov
56697125d3 Merge remote-tracking branch 'origin/master' into distinct_sorted_simplify 2022-08-02 11:18:46 +00:00
Igor Nikonov
4bf42ce867 Merge remote-tracking branch 'origin/master' into skipping_sorting_step 2022-08-02 10:50:25 +00:00
Wangyang Guo
91e3e2f18b KeyCondition: optimize applyFunction in multi-thread scenario 
Construct and deconstruct args (ColumnsWithTypeAndName) will inc/dec
ref_count (actually this is a atomic lock inc/dec operation) to share_ptr,
which may share the same DataTypePtr among different threads. This will
have a lock contention issue in large parallel situation.

The patch try to minimize `args` scope and reduce unnecessary
construct/destory of instances. It will improve the performance in
multi-thread cases.
 2022-08-02 16:16:16 +08:00
Frank Chen
7d8a8f26cd Fix build check 2022-08-02 13:31:36 +08:00
Frank Chen
8eb254c0c8 Fix merge problem 2022-08-02 10:23:51 +08:00
Frank Chen
40c6e4c0d6 Merge remote-tracking branch 'origin/master' into tracing_context_propagation 2022-08-02 10:02:09 +08:00
Alexander Tokmakov
1567b65473 Merge branch 'master' into replicated_database_improvements 2022-08-01 19:12:37 +02:00
Alexey Milovidov
3de747a6e2
Merge pull request #39746 from ClickHouse/cleanup-projection-setting 
Cleanup usages of `allow_experimental_projection_optimization` setting, part 1
 2022-08-01 19:21:30 +03:00
Robert Schulze
bf574b9154
Merge pull request #39760 from ClickHouse/bit-fiddling 
Use std::popcount, ::countl_zero, ::countr_zero functions
 2022-08-01 17:04:51 +02:00
Antonio Andelic
9ec27c0ab4
Merge pull request #39757 from ClickHouse/fix-rocksdb-filter-with-params 
Use params correctly for filtering by key in EmbeddedRocksDB
 2022-08-01 12:17:10 +02:00
Dmitry Novik
ab5a147065
Merge pull request #38725 from azat/fix-order-by-projection 
Fix ORDER BY that matches projections ORDER BY
 2022-08-01 00:32:59 +02:00
Nikolai Kochetov
22fbfe19a4 Merge branch 'master' into use-dag-in-key-condition 2022-07-31 21:54:12 +02:00
kssenii
e5f4a619ed Merge master 2022-07-31 20:24:40 +03:00
Robert Schulze
a7734672b9
Use std::popcount, ::countl_zero, ::countr_zero functions 
- Introduced with the C++20 <bit> header

- The problem with __builtin_c(l|t)z() is that 0 as input has an
  undefined result (*) and the code did not always check. The std::
  versions do not have this issue.

- In some cases, we continue to use buildin_c(l|t)z(), (e.g. in
  src/Common/BitHelpers.h) because the std:: versions only accept
  unsigned inputs (and they also check that) and the casting would be
  ugly.

(*) https://gcc.gnu.org/onlinedocs/gcc/Other-Builtins.html
 2022-07-31 15:16:51 +00:00
Robert Schulze
52d08d9db4
Merge pull request #39520 from ClickHouse/no-split-binary 
Remove SPLIT_BINARY
 2022-07-31 14:23:31 +02:00
Antonio Andelic
eea1aaef2c Use params for filtering by key in EmbeddedRocksDB 2022-07-31 10:44:01 +00:00
Igor Nikonov
7f0adb5eb0 Merge remote-tracking branch 'origin/master' into skipping_sorting_step 2022-07-31 07:07:36 +00:00
Anton Popov
74eb8119ef
Merge branch 'master' into fix-custom-serializations 2022-07-31 01:51:52 +02:00
Alexey Milovidov
8fb70abe3e
Merge pull request #39178 from azat/dist-insert-log 
Add connection info for Distributed sends log message
 2022-07-31 02:22:22 +03:00
Alexey Milovidov
441f2feb49 Cleanup usages of allow_experimental_projection_optimization setting, part 1 2022-07-30 23:40:21 +02:00
Igor Nikonov
a7cfad105e
Merge branch 'master' into distinct_sorted_simplify 2022-07-30 21:57:53 +02:00
Igor Nikonov
7b0b38e997 DistinctSortedTransform works only if columns contains sort prefix of 
sort description
 2022-07-29 20:01:07 +00:00
Kruglov Pavel
ccd1e1bdb8
Merge branch 'master' into fix-cancel-insert-into-function 2022-07-29 20:27:32 +02:00
Maksim Kita
8fc6bad4f4 Join enums refactoring 2022-07-29 18:35:05 +02:00
Alexander Tokmakov
6f88065cc1 check metadata consistency 2022-07-29 18:33:16 +02:00
Nikolai Kochetov
59a11b32ad
Merge branch 'master' into use-dag-in-key-condition 2022-07-29 17:01:33 +02:00
Igor Nikonov
fb5799fe7f More generic getInputOrderInfo() in query info 2022-07-29 14:36:36 +00:00
Igor Nikonov
ae82bfb7f3 Merge remote-tracking branch 'origin/master' into distinct_sorted_simplify 2022-07-29 13:12:03 +00:00
Robert Schulze
3d1797f75f
Merge remote-tracking branch 'origin/master' into no-split-binary 2022-07-29 12:17:43 +00:00
Anton Popov
57e4fb2e30 Merge remote-tracking branch 'upstream/master' into HEAD 2022-07-29 11:42:11 +00:00
Anton Popov
bb551566e7 fix build 2022-07-29 11:41:53 +00:00
Azat Khuzhin
498c8b3c52 Fix clang-tidy in utils/examples 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-07-29 11:36:51 +03:00
Robert Schulze
199e254777
Merge remote-tracking branch 'origin/master' into no-split-binary 2022-07-28 15:54:22 +00:00
Anton Popov
04fd72cdef fix alter column rename 2022-07-28 13:19:23 +00:00
Vitaly Baranov
f8ebe76826
Merge pull request #39503 from vitlibar/improve-system-backups-table 
Improve system.backups table
 2022-07-27 20:25:39 +02:00
Vladimir C
115506356c
Merge branch 'master' into refactor-prepared-sets 2022-07-27 19:57:23 +02:00
Nikolai Kochetov
6206e468a4 Ignor materialize function for ActionsDAG in PK analysis. 2022-07-27 17:02:48 +00:00
Nikolai Kochetov
51c4c9c957 Fixing build. 2022-07-27 15:45:23 +00:00
Anton Popov
56039c8780 fix alter of LowCardinality 2022-07-27 15:35:09 +00:00
Anton Popov
4cf0353892 try to fix Nested 2022-07-27 14:05:16 +00:00
Filatenkov Artur
a218597546
Merge branch 'master' into annoy-2 2022-07-27 15:37:58 +03:00
FArthur-cmd
6b5e92e68a change index policy 2022-07-27 15:36:07 +03:00
Nikolai Kochetov
f9a8a4cd32 Fixing read-in-order for Merge 2022-07-27 12:00:55 +00:00
vdimir
8eecb9ef82
upd PreparedSets: rename/change signature of methods, add comments 2022-07-27 11:22:16 +00:00
Alexander Tokmakov
8fc075a527
Merge pull request #39619 from ClickHouse/try-do-enable-global-allocations-for-attach 
Replace MemoryTrackerBlockerInThread to LockMemoryExceptionInThread
 2022-07-27 13:13:04 +03:00
Kruglov Pavel
381ea139c2
Merge branch 'master' into schema-inference-cache 2022-07-27 11:35:36 +02:00
Vitaly Baranov
794eeb5d51 Split "total_size" to "uncompressed_size" and "compressed_size". 2022-07-27 10:36:56 +02:00
Vitaly Baranov
a79fc17235 Add test for system.backups 2022-07-27 09:04:10 +02:00
Vitaly Baranov
1cfe0b10f7 Add columns "total_size" and "num_files" to system.backups 2022-07-27 09:04:10 +02:00
Vitaly Baranov
35c267b3b1 Replace column "status_changed_time" with columns "start_time" and "end_time". 2022-07-27 09:04:10 +02:00
Vitaly Baranov
fc16a15ecf Rename column "uuid" -> "id" in system.backups and allow user to set it in a query. 2022-07-27 09:04:10 +02:00
Vitaly Baranov
131019ba49 Rename column "backup_name" -> "name" in system.backups. 2022-07-27 09:04:10 +02:00
Vitaly Baranov
afd0982187 Remove column "internal" from system.backups 2022-07-27 09:04:10 +02:00
Alexander Gololobov
a64aa00869
Merge pull request #37893 from zhangjmruc/feature/sql-standard-delete 
Support SQL standard "delete from ... where ..." syntax and lightweight implementation on merge tree tables
 2022-07-26 23:39:07 +02:00
Nikolai Kochetov
1adb021df6 Fixing tests. 2022-07-26 18:42:27 +00:00
vdimir
5ce2960f03
Get rid of SelectQueryInfoBase -> SelectQueryInfo 2022-07-26 18:39:09 +00:00
vdimir
d9928ac93d
Add methods to SubqueryForSet, do not use refernce to SetPtr 2022-07-26 18:39:09 +00:00
vdimir
1e3fa2e01f
Refactor PreparedSets/SubqueryForSet 2022-07-26 18:39:02 +00:00
Anton Popov
9321ca34cf try to fix Nested 2022-07-26 17:32:37 +00:00
Nikolai Kochetov
33ce662d3e Replace MemoryTrackerBlockerInThread to LockMemoryExceptionInThread in some places. Reduced MemoryTrackerBlockerInThread level to User. 2022-07-26 15:22:00 +00:00
Kruglov Pavel
c683cb252f
Merge pull request #39227 from amosbird/rename-log1 
Rename log when rename merge tree tables
 2022-07-26 17:12:44 +02:00
Nikolai Kochetov
d7cc3831ea Push predicate over StorageMerge. 2022-07-26 15:01:39 +00:00
Nikolai Kochetov
c5110bb164 Add a query plan step for StorageMerge 2022-07-26 14:43:05 +00:00
jianmei zhang
c1834d183b ReplicatedMergeTree support lightweight delete 2022-07-26 11:50:09 +08:00
Alexander Gololobov
25deba2c1b
Merge branch 'master' into feature/sql-standard-delete 2022-07-25 22:13:20 +02:00
Nikolai Kochetov
be9c7ed52c Add ReadFromMerge step. 2022-07-25 19:41:43 +00:00
Roman Vasin
b462366415 Merge branch 'master' of github.com:ClickHouse/ClickHouse into ADQM-485 2022-07-25 17:55:47 +00:00
Roman Vasin
b412ea5f6d Improve generateRandom() for Date32; fix tests 01087_table_function_generate, 01277_fromUnixTimestamp64, 01691_DateTime64_clamp and 01702_toDateTime_from_string_clamping 2022-07-25 17:06:11 +00:00
Alexander Tokmakov
778dcf6994
Merge pull request #39230 from Algunenano/improve_replicated_merge_logging 
Improve logging around replicated merges
 2022-07-25 18:42:12 +03:00
Alexander Gololobov
48de02a7b8 Capitalized const name 2022-07-25 16:32:16 +02:00
Alexander Gololobov
7b44950d1d More detailed comment 2022-07-25 16:14:33 +02:00
Nikolai Kochetov
b70be40804
Merge branch 'master' into use-dag-in-key-condition 2022-07-25 14:30:22 +02:00
Nikolai Kochetov
1ce8e714d3 Fixing build. 2022-07-25 12:30:01 +00:00
Alexey Milovidov
388d06fda1
Merge pull request #39535 from ClickHouse/stringref 
Less usage of StringRef
 2022-07-25 04:06:11 +03:00
Alexander Gololobov
460950ecdc
Merge branch 'master' into feature/sql-standard-delete 2022-07-24 21:27:22 +02:00
Robert Schulze
4333750985
Less usage of StringRef 
... replaced by std::string_view, see #39262
 2022-07-24 18:33:52 +00:00
Alexey Milovidov
0209012e5e
Merge pull request #39517 from pkit/pkit/fix_examples 
fix `-DENABLE_EXAMPLES=1` in master
 2022-07-24 17:41:23 +03:00
Igor Nikonov
eacc57b6a4 Simplify DistinctSorted 
+ use ordinary Distinct for deduplication in MergeTree when no sorting provided
 2022-07-24 14:26:25 +00:00
Azat Khuzhin
22d8e532ed Fix ORDER BY that matches projections ORDER BY 
In case of projection is seleted and it is comlete (all parts has such
projection) and ORDER BY matches projection ORDER BY, and
optimize_read_in_order=1 (default), then the sorting is simply not done,
because projections has separate plan, and so
InterpreterSelectQuery::executeOrder() -> SortingStep has
pipeline.getNumStreams() == 0 and it cannot do sorting.

Fix this, by adding sorting when creating plan for reading from
projections.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-07-24 15:23:13 +03:00
Kseniia Sumarokova
04944ff6f5
Merge pull request #39509 from monadbobo/remove_duplicate_code 
Removed some duplicate code.
 2022-07-24 14:24:40 +03:00
Alexander Gololobov
6692770113 Allow to update "system" virtual columns such as _row_exists 2022-07-24 12:21:47 +02:00
Alexander Gololobov
c8b3c574a4 Disable lightweight delete if table has projections 2022-07-24 12:21:47 +02:00
Alexander Gololobov
be64b45583 Properly handle empty parts list 2022-07-24 12:21:47 +02:00
Alexander Gololobov
594195451e Cleanups 2022-07-24 12:21:18 +02:00
Alexey Milovidov
071374b152 Remove SPLIT_BINARY 2022-07-24 01:15:54 +02:00
Kseniia Sumarokova
f146183400
Merge pull request #39491 from ClickHouse/remove_unused_constructor 
Remove unused constructor
 2022-07-23 21:16:39 +02:00
mergify[bot]
c01ff2d38a
Merge branch 'master' into improve_replicated_merge_logging 2022-07-23 19:10:10 +00:00
Constantine Peresypkin
d645d03e89 fix -DENABLE_EXAMPLES=1 in master 2022-07-23 19:00:52 +02:00
Kseniia Sumarokova
77ddb8eda0
Merge pull request #39506 from CurtizJ/fix-direct-io 
Fix reading from async read buffer with direct io
 2022-07-23 06:08:48 +02:00
Anton Popov
3d03b2714b try to fix Nested 2022-07-22 23:05:11 +00:00
Nikolai Kochetov
dd65a36c59 Fix another one test. 2022-07-22 19:11:25 +00:00
Nikolai Kochetov
53175e43f3 Better support for projections. 2022-07-22 19:11:25 +00:00
Simon Liu
2305f282dc Removed some duplicate code. 2022-07-23 02:27:17 +08:00
Anton Popov
49627aa554 Merge remote-tracking branch 'upstream/master' into HEAD 2022-07-22 17:16:06 +00:00
Anton Popov
c1a5514e46 fix reading from async read buffer with direct io 2022-07-22 16:58:30 +00:00
Alexander Tokmakov
bed2206ae9
Merge pull request #39460 from ClickHouse/remove_some_dead_and_commented_code 
Remove some dead and commented code
 2022-07-22 13:24:34 +03:00
alesapin
69984b28e0 Remove unused constructor 2022-07-22 12:21:10 +02:00
Robert Schulze
99579ab440
Merge pull request #39364 from ClickHouse/stringref-to-string_view 
Even less usage of StringRef
 2022-07-22 09:30:49 +02:00
Alexander Tokmakov
0441d3629e
Merge pull request #39440 from liangliangpan/optimize-cluster-table-func 
optimize select query from cluster table function
 2022-07-21 20:40:31 +03:00
Mikhail f. Shiryaev
cf0b01b189
Update version to 22.8.1.1 2022-07-21 19:21:28 +02:00
Kruglov Pavel
9252f42b4c
Merge branch 'master' into schema-inference-cache 2022-07-21 18:59:14 +02:00
Robert Schulze
ea0a3bf600
Merge branch 'master' into stringref-to-string_view 2022-07-21 18:33:06 +02:00
zhangxiao871
5549b64c06 Fix issues 39469 2022-07-21 23:52:33 +08:00
Anton Popov
10719fd61d fix style check 2022-07-21 15:06:55 +00:00
Anton Popov
e0d2c8fb37 fix json type with sparse columns 2022-07-21 14:47:19 +00:00
Alexander Tokmakov
8bd674186a
Update getStructureOfRemoteTable.cpp 2022-07-21 16:15:42 +03:00
Alexander Tokmakov
a8da5d96fc remove some dead and commented code 2022-07-21 15:05:48 +02:00
avogar
52b3a87ed1 Fix typo 2022-07-21 12:38:18 +00:00
avogar
6b541aa98f Fix WriteBuffer finalize when cancel insert into function 2022-07-21 12:18:37 +00:00
Alexander Tokmakov
f79f12408d fixes 2022-07-21 13:57:24 +02:00
Alexander Gololobov
1ea9f143ff Leave only _row_exists-based implementation of lightweight delete 2022-07-21 11:26:13 +02:00
Nikolai Kochetov
e15967e9db
Merge pull request #38475 from ClickHouse/additional-filters 
Additional filters for a table (from setting)
 2022-07-21 07:52:04 +02:00
Mingliang Pan
e019b6ba69
Update getStructureOfRemoteTable.cpp 2022-07-21 12:11:33 +08:00
Alexey Milovidov
844042fc18
Merge pull request #39433 from ClickHouse/revert-39396-try-fix-write-buffer-terminate 
Revert "Fix WriteBuffer finalize in destructor when cacnel query"
 2022-07-21 07:04:07 +03:00
Mingliang Pan
3f76c8d7fd
optimize select query from cluster table function 
Use local node as first priority to get Structure Of Remote Table.
we have many distributed queries( like   select xx from cluster('xx',view  (xxxx)   ) on a clickhouse cluster.  we found that the first node (shard_num=1)  have 2 times of query number compared to other shards.
The reason is that  the getStructureOfRemoteTableInShard func always take the first shard to execute  "DESC TABLE xx" query.
The better way is to  use local node as first priority which save the network rpc and reduce the pressure of first shard .
 2022-07-21 11:56:35 +08:00
Alexey Milovidov
fd691000b7
Merge pull request #39342 from azat/bg-executor-fix 
Fix possible UB in MergeTreeBackgroundExecutor (leads to SIGSEGV on race with DROP/DETACH)
 2022-07-21 05:22:51 +03:00
Kruglov Pavel
92995a832b
Revert "Fix WriteBuffer finalize in destructor when cacnel query" 2022-07-21 01:45:16 +02:00
Nikolai Kochetov
91043351aa Fixing build. 2022-07-20 20:30:16 +00:00
Nikolai Kochetov
91aa2385ca
Merge branch 'master' into additional-filters 2022-07-20 18:56:24 +02:00
Kruglov Pavel
0efed4644d
Merge pull request #38957 from Avogar/compatibility-setting 
Add compatibility setting and system.settings_changes system table
 2022-07-20 17:46:06 +02:00
Nikolai Kochetov
4e8cd70b1d
Merge branch 'master' into use-dag-in-key-condition 2022-07-20 17:38:33 +02:00
Dmitry Novik
a09422de7b
Merge pull request #38642 from ClickHouse/group-by-use-nulls 
Add group_by_use_nulls setting
 2022-07-20 14:42:42 +02:00
Kruglov Pavel
3046cd6d29
Merge branch 'master' into schema-inference-cache 2022-07-20 13:30:42 +02:00
Vladimir C
7694c72154
Merge pull request #38887 from bigo-sg/hive_file_crash 
fixed: crash caused by IHiveFile be shared among threads
 2022-07-20 12:15:57 +02:00
Kruglov Pavel
7722b647b7
Merge pull request #39396 from Avogar/try-fix-write-buffer-terminate 
Fix WriteBuffer finalize in destructor when cacnel query
 2022-07-20 12:06:20 +02:00
lgbo-ustc
4b46d80169 update codes 2022-07-20 18:03:34 +08:00
Alexander Tokmakov
988569bdb1
Merge pull request #39366 from ClickHouse/fix_race_in_replication_queue 
Do not enqueue uneeded parts for check
 2022-07-20 00:35:02 +03:00
Nikolai Kochetov
f570cde815 Fixing build. 2022-07-19 20:19:57 +00:00
avogar
5c16d6b553 Fix WriteBuffer finalize in destructor when cacnel query 2022-07-19 19:21:30 +00:00
Nikolai Kochetov
a00bcd389f Fixing build 2022-07-19 19:33:14 +02:00
Nikolai Kochetov
eaeb30a71a Merge branch 'master' into use-dag-in-key-condition 2022-07-19 18:39:52 +02:00
Dmitry Novik
50989bdb68
Merge branch 'master' into group-by-use-nulls 2022-07-19 14:58:01 +02:00
Alexander Tokmakov
a761da9f1b add comment 2022-07-19 13:58:59 +02:00
kssenii
21ebf8874e Merge remote-tracking branch 'upstream/master' into make-cache-composable 2022-07-19 13:36:42 +02:00
Alexander Tokmakov
3e7414b356 add comment 2022-07-19 12:47:20 +02:00
Nikolai Kochetov
d0db7135cb
Merge pull request #38946 from amosbird/nullable-partition-fix1 
Fix incorrect partition pruning when there is a nullable partition
 2022-07-19 12:14:36 +02:00
Kseniia Sumarokova
f168e0c5d0
Merge pull request #39272 from kssenii/fixes-for-materialized-pg 
Fix segfault in materialised postgresql
 2022-07-19 11:56:10 +02:00
Robert Schulze
81ef1099cc
Even less usage of StringRef 
--> see #39300
 2022-07-19 07:01:06 +00:00
FArthur-cmd
3b30cd9259 improvements 2022-07-19 00:59:07 +03:00
Alexander Tokmakov
d82f378a9d do not enqueue uneeded parts for check 2022-07-18 23:37:07 +02:00
Alexander Gololobov
614cb9a87f Disable trivial count() optimization if _row_exists column is present 2022-07-18 20:08:01 +02:00
Alexander Gololobov
ae0d00083c Renamed __row_exists to _row_exists 2022-07-18 20:07:36 +02:00
Alexander Gololobov
f324ca9921 Cleanups 2022-07-18 20:07:22 +02:00
Alexander Gololobov
9de72d995a POC lightweight delete using __row_exists virtual column and prewhere-like filtering 2022-07-18 20:06:42 +02:00
Robert Schulze
6df3c9d799
Merge pull request #39300 from ClickHouse/stringref-to-stringview 
First try at reducing the use of StringRef
 2022-07-18 19:20:16 +02:00
Azat Khuzhin
c6f7698f9a Fix possible UB in MergeTreeBackgroundExecutor 
Lambda erase_from_active() captures the item (TaskRuntimeDataPtr), most
of the code path is OK, since it explicitly reset the item->task.

However one is not, when it moves the item to pending list, which will
be cleaned up when the table will be DROP/DETACH'ed, from
MergeTreeBackgroundExecutor::removeTasksCorrespondingToStorage(),
and in this case if IStorage will be already removed, then it will lead
to use-after-free on destroying the lambda, since it captures the item
by value.

And I belive that CI founds this issue here [1]:

<details>

<summary>stack trace</summary>

    4 0x268d1354 in DB::ReplicatedMergeTreeQueue::CurrentlyExecuting::~CurrentlyExecuting() build_docker/../src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1510:33
    5 0x268ead1a in std::__1::default_delete<DB::ReplicatedMergeTreeQueue::CurrentlyExecuting>::operator()(DB::ReplicatedMergeTreeQueue::CurrentlyExecuting*) const build_docker/../contrib/libcxx/include/__memory/unique_ptr.h:54:5
    6 0x268ead1a in std::__1::unique_ptr<DB::ReplicatedMergeTreeQueue::CurrentlyExecuting, std::__1::default_delete<DB::ReplicatedMergeTreeQueue::CurrentlyExecuting> >::reset(DB::ReplicatedMergeTreeQueue::CurrentlyExecuting*) build_docker/../contrib/libcxx/include/__memory/unique_ptr.h:315:7
    7 0x268ead1a in std::__1::unique_ptr<DB::ReplicatedMergeTreeQueue::CurrentlyExecuting, std::__1::default_delete<DB::ReplicatedMergeTreeQueue::CurrentlyExecuting> >::~unique_ptr() build_docker/../contrib/libcxx/include/__memory/unique_ptr.h:269:19
    8 0x268ead1a in DB::ReplicatedMergeTreeQueue::SelectedEntry::~SelectedEntry() build_docker/../src/Storages/MergeTree/ReplicatedMergeTreeQueue.h:351:12
    9 0x268ead1a in void std::__1::__destroy_at<DB::ReplicatedMergeTreeQueue::SelectedEntry, 0>(DB::ReplicatedMergeTreeQueue::SelectedEntry*) build_docker/../contrib/libcxx/include/__memory/construct_at.h:56:13
    ...
    16 0x265e9abb in DB::MergeTreeBackgroundExecutor<DB::MergeMutateRuntimeQueue>::routine(std::__1::shared_ptr<DB::TaskRuntimeData>)::'lambda'()::~() build_docker/../src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:127:30
    17 0x265e9abb in DB::MergeTreeBackgroundExecutor<DB::MergeMutateRuntimeQueue>::routine(std::__1::shared_ptr<DB::TaskRuntimeData>) build_docker/../src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:235:1
    18 0x265ea730 in DB::MergeTreeBackgroundExecutor<DB::MergeMutateRuntimeQueue>::threadFunction() build_docker/../src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:261:13

</details>

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/0/533c7888038453c047df816f3f65292cca05a54f/stateless_tests__ubsan__actions_.html

And I also I think that the following issue will also be fixed [2]:

<details>

<summary>stack trace</summary>

    [ 680 ] {} <Fatal> : Logical error: 'Memory tracker: allocations not allowed.'.
    ...
    [ 23430 ] {} <Fatal> BaseDaemon: 23. /build/build_docker/../src/Common/formatReadable.cpp:46: formatReadableSizeWithBinarySuffix(double, int) @ 0x1713bb98 in /usr/bin/clickhouse
    ...
    [ 23430 ] {} <Fatal> BaseDaemon: 31. ../src/Common/MemoryTracker.cpp:112: MemoryTracker::logPeakMemoryUsage() @ 0x170e0ab9 in /usr/bin/clickhouse
    [ 23430 ] {} <Fatal> BaseDaemon: 32. /build/build_docker/../src/Common/MemoryTracker.cpp:98: MemoryTracker::~MemoryTracker() @ 0x170e063a in /usr/bin/clickhouse
    [ 23430 ] {} <Fatal> BaseDaemon: 33. /build/build_docker/../src/Storages/MergeTree/MergeList.cpp:144: DB::MergeListElement::~MergeListElement() @ 0x279fb290 in /usr/bin/clickhouse
    ...
    [ 23430 ] {} <Fatal> BaseDaemon: 38. /build/build_docker/../src/Storages/MergeTree/BackgroundProcessList.h:41: DB::BackgroundProcessListEntry<DB::MergeListElement, DB::MergeInfo>::~BackgroundProcessListEntry() @ 0x276ce6c7 in /usr/bin/clickhouse
    [ 23430 ] {} <Fatal> BaseDaemon: 39. /build/build_docker/../contrib/libcxx/include/__memory/unique_ptr.h:54: std::__1::default_delete<DB::BackgroundProcessListEntry<DB::MergeListElement, DB::MergeInfo> >::operator()(DB::BackgroundProcessListEntry<DB::MergeListElement, DB::MergeInfo>*) const @ 0x276ce60b in /usr/bin/clickhouse
    [ 23430 ] {} <Fatal> BaseDaemon: 40. /build/build_docker/../contrib/libcxx/include/__memory/unique_ptr.h:316: std::__1::unique_ptr<DB::BackgroundProcessListEntry<DB::MergeListElement, DB::MergeInfo>, std::__1::default_delete<DB::BackgroundProcessListEntry<DB::MergeListElement, DB::MergeInfo> > >::reset(DB::BackgroundProcessListEntry<DB::MergeListElement, DB::MergeInfo>*) @ 0x276ce57c in /usr/bin/clickhouse
    [ 23430 ] {} <Fatal> BaseDaemon: 41. /build/build_docker/../contrib/libcxx/include/__memory/unique_ptr.h:269: std::__1::unique_ptr<DB::BackgroundProcessListEntry<DB::MergeListElement, DB::MergeInfo>, std::__1::default_delete<DB::BackgroundProcessListEntry<DB::MergeListElement, DB::MergeInfo> > >::~unique_ptr() @ 0x276ce399 in /usr/bin/clickhouse
    [ 23430 ] {} <Fatal> BaseDaemon: 42. /build/build_docker/../src/Storages/MergeTree/MutatePlainMergeTreeTask.h:22: DB::MutatePlainMergeTreeTask::~MutatePlainMergeTreeTask() @ 0x27defceb in /usr/bin/clickhouse
    [ 23430 ] {} <Fatal> BaseDaemon: 43. /build/build_docker/../contrib/libcxx/include/__memory/construct_at.h:57: void std::__1::__destroy_at<DB::MutatePlainMergeTreeTask, 0>(DB::MutatePlainMergeTreeTask*) @ 0x27dd69c1 in /usr/bin/clickhouse
    [ 23430 ] {} <Fatal> BaseDaemon: 44. /build/build_docker/../contrib/libcxx/include/__memory/construct_at.h:82: void std::__1::destroy_at<DB::MutatePlainMergeTreeTask, 0>(DB::MutatePlainMergeTreeTask*) @ 0x27dd6955 in /usr/bin/clickhouse
    [ 23430 ] {} <Fatal> BaseDaemon: Integrity check of the executable skipped because the reference checksum could not be read. (calculated checksum: 91F5937571C11255DFE73230B52CE9C0)
    [ 602 ] {} <Fatal> Application: Child process was terminated by signal 6.

</details>

  [2]: https://s3.amazonaws.com/clickhouse-test-reports/39222/a068c397dfd7943359a8b554566c3c70b78baf8d/stateless_tests__debug__actions__%5B1/3%5D.html

Refs: https://github.com/ClickHouse/ClickHouse/pull/29614#discussion_r720455032 (cc @nikitamikhaylov)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-07-18 19:01:41 +03:00
kssenii
0aed5a602a Merge master 2022-07-18 16:16:28 +02:00