thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-16 19:32:07 +00:00
Code Issues Packages Projects Releases Wiki Activity
102,777 Commits 9,319 Branches 1,783 Tags 2.1 GiB
6740ae9fd7
Commit Graph

10408 Commits

Author SHA1 Message Date
Sergei Trifonov
159743edd6
Merge pull request #43014 from ClickHouse/disk-s3-throttler 
S3 request per second rate throttling
 2022-11-16 18:51:06 +01:00
Han Fei
221c4e5912 tiny changes 2022-11-16 17:59:08 +01:00
kssenii
f0ce28349d Better 2022-11-16 17:58:21 +01:00
Han Fei
14701670b5 Merge branch 'master' into hanfei/aysnc-insert 2022-11-16 17:24:36 +01:00
Nikolai Kochetov
17b75d3f5c Fix metrics and progress. 2022-11-16 14:10:56 +00:00
Antonio Andelic
11aba7597e Check if initialization is done on write 2022-11-16 13:33:22 +00:00
Smita Kulkarni
040f2b62d9 Enable compress marks and primary key 
Implementation:
* Set compress_marks and compress_primary_key to true by default.
* Updated getIndexExtensionFromFilesystem to only check for compressed file extension , else return default. When both are present, this function was returning uncompressed file extension giving rise to error as file got removed.
Testing:
* Updated tests where bytes size or compressed data is checked to check with compress_marks and compress_primary_key set to false.
 2022-11-16 14:23:58 +01:00
filimonov
dcb76f6bbf
Allow autoremoval of old parts if detach_not_byte_identical_parts enabled 
Allow autoremoval of old & detached parts if detach_not_byte_identical_parts enabled.

See also #28708 #37975
 2022-11-16 14:16:42 +01:00
alesapin
2012f4bd36 Fix use-after-free 2022-11-16 11:50:51 +01:00
Alexey Milovidov
04687c0d69
Merge pull request #43206 from ClickHouse/miscellaneous-changes-parallel-replicas 
Miscellaneous changes
 2022-11-16 05:30:03 +01:00
Vitaly Baranov
d365570b27 Hide passwords in SHOW CREATE TABLE and system.tables. 2022-11-16 01:35:11 +01:00
Nikolai Kochetov
1b1d23c100 Clumsy attempt to fix progress. 2022-11-15 21:23:18 +00:00
Alexander Tokmakov
69b067b81c
Merge pull request #43244 from ClickHouse/fix-replicated-merge-tree-shared-id 
Don't throw if shared ID already created in `StorageReplicatedMergeTree`
 2022-11-15 23:31:51 +03:00
Nikolai Kochetov
cf1afbfb84 Temporarily enable allow_asynchronous_read_from_io_pool_for_merge_tree by default to see thest results. 2022-11-15 17:47:26 +00:00
Nikolai Kochetov
29098045d8 Try read from MergeTree in background pool. 2022-11-15 17:38:37 +00:00
kssenii
234f6ee54d Add commands CREATE/ALTER/DROP NAMED COLLECTION 2022-11-15 15:51:56 +01:00
Sema Checherinda
7e73b187cc
Merge pull request #43159 from Algunenano/over-read 
Fix several buffer over-reads
 2022-11-15 15:04:24 +01:00
alesapin
a12f2d14dc Simpler check 2022-11-15 14:46:38 +01:00
Anton Popov
8c66c759d4
Merge pull request #43133 from amosbird/issue_43111 
Fix Nullable keys in hyperrectangle
 2022-11-15 14:30:13 +01:00
alesapin
4ec11d0f58 Remove redundant branch 2022-11-15 13:58:53 +01:00
Antonio Andelic
0e5e58bed9 Remove exception if shared ID already created 2022-11-15 12:18:24 +00:00
Antonio Andelic
96838f5be7 Merge branch 'master' into keeper-prometheus 2022-11-15 12:00:02 +00:00
Kseniia Sumarokova
59cf5def67
Merge branch 'master' into disk-s3-throttler 2022-11-15 12:13:37 +01:00
xiedeyantu
ec6698395e fix skip_unavailable_shards does not work using hdfsCluster table function 2022-11-15 13:25:15 +08:00
zhongyuankai
35690c1972 Merge branch 'master' into optimize_ttl 2022-11-15 08:49:37 +08:00
zhongyuankai
16f697f61d fix test 2022-11-15 08:47:51 +08:00
Alexey Milovidov
6e01b2b2a1
Merge pull request #43203 from ClickHouse/fix-typo-2 
Fix typo
 2022-11-15 01:39:34 +01:00
Alexey Milovidov
3cab3ce484
Merge pull request #43198 from FArthur-cmd/MySQL_clickhouse_connect 
add columns to information schema
 2022-11-15 01:11:29 +01:00
alesapin
6a50a32cfc Fix parts_to_do 2022-11-15 00:53:20 +01:00
alesapin
f53ada2bbd Revert "Fix bug in replication queue which can lead to premature mutation finish" 
This reverts commit 3b5424811c.
 2022-11-14 22:59:09 +01:00
alesapin
3b5424811c Fix bug in replication queue which can lead to premature mutation finish 2022-11-14 21:53:16 +01:00
Kruglov Pavel
386b69acae
Merge pull request #43131 from xiedeyantu/fix_s3cluster_connection_fail 
fix skip_unavailable_shards does not work using s3Cluster table function
 2022-11-14 20:48:00 +01:00
Vitaly Baranov
8e99f5fea3 Move maskSensitiveInfoInQueryForLogging() to src/Parsers/ 2022-11-14 18:55:19 +01:00
Alexander Tokmakov
c64109a2de
Merge branch 'master' into fetch-remove-covered 2022-11-14 18:23:16 +03:00
zhongyuankai
ea11d80536 Priority is given to deleting completely expired Parts 2022-11-14 23:03:56 +08:00
Han Fei
11f93dec3f tmp commit 2022-11-14 15:53:45 +01:00
Raúl Marín
4988e387f2 Merge remote-tracking branch 'blessed/master' into over-read 2022-11-14 12:28:31 +01:00
Kseniia Sumarokova
b7df9aa3da
Merge pull request #43155 from zzsmdfj/add_oss_function_and_StorageOSS 
to add_oss_function_and_StorageOSS
 2022-11-14 12:23:57 +01:00
Kseniia Sumarokova
e58a7140be
Merge pull request #43194 from ClickHouse/async-metrics-documentation 
Add documentation for `system.asynchronous_metrics`.
 2022-11-14 12:21:34 +01:00
Kseniia Sumarokova
79206c6eb0
Merge pull request #43147 from kssenii/named-collections-refactoring 
Add generic implementation for arbitrary structured named collections, access type and system.named_collections
 2022-11-14 11:57:17 +01:00
Antonio Andelic
ee182776da
Merge branch 'master' into keeper-prometheus 2022-11-14 09:14:13 +01:00
Alexey Milovidov
4edd8b0135 Remove unused method 2022-11-14 08:07:27 +01:00
Alexey Milovidov
17693de053 Better logs 2022-11-14 08:01:48 +01:00
Alexey Milovidov
9c48664f11 Better logs 2022-11-14 07:48:50 +01:00
Alexey Milovidov
582db233c4 Better logs 2022-11-14 07:22:17 +01:00
Alexey Milovidov
99f84d8095 Better logs 2022-11-14 07:22:02 +01:00
Alexey Milovidov
df01689ccc Better logs 2022-11-14 07:13:42 +01:00
Alexey Milovidov
a77032fafa Miscellaneous changes 2022-11-14 06:54:58 +01:00
Alexey Milovidov
15202833b1 Miscellaneous changes 2022-11-14 06:48:49 +01:00
Alexey Milovidov
53ee7446dc Miscellaneous changes 2022-11-14 06:46:37 +01:00
Alexey Milovidov
cc1855ea33 Miscellaneous changes 2022-11-14 06:45:52 +01:00
Alexey Milovidov
4966556f47 Miscellaneous changes 2022-11-14 06:41:51 +01:00
Alexey Milovidov
85dc692f7c Miscellaneous changes 2022-11-14 06:37:55 +01:00
Alexey Milovidov
60c96072e1 Miscellaneous changes 2022-11-14 06:37:19 +01:00
Alexey Milovidov
e080161ba5 Miscellaneous changes 2022-11-14 06:32:18 +01:00
Alexey Milovidov
bfea3deed9 Miscellaneous changes 2022-11-14 06:09:03 +01:00
Alexey Milovidov
3c603c3d8c
Update columns.sql 2022-11-14 02:42:30 +03:00
Alexey Milovidov
b9471e492a Fix typo 2022-11-13 22:52:21 +01:00
FArthur-cmd
f6b334fb3a add columns to information scheme 2022-11-13 12:03:46 +00:00
Alexey Milovidov
ba05b7dd2c Add documentation for AsynchronousMetrics 2022-11-13 03:36:20 +01:00
Filatenkov Artur
c7c52a570f
Update MergeTreeIndexAnnoy.cpp 2022-11-12 12:43:25 +03:00
FArthur-cmd
5eebc27d24 improve test 2022-11-12 09:23:49 +00:00
Nikolai Kochetov
9106910de8 Try read from MergeTree in background pool. 2022-11-11 20:47:11 +00:00
Igor Nikonov
ee5756b5b4
Merge pull request #43122 from ClickHouse/revert-43116-revert-42607-igor/insert_zk_retries_retry 
Revert "Revert " Keeper retries during insert (clean)""
 2022-11-11 18:22:38 +01:00
xiedeyantu
5504f3af9b fix skip_unavailable_shards does not work using s3Cluster table function 2022-11-12 00:03:36 +08:00
kssenii
616da1f695 Review fixed, style check fix 2022-11-11 16:50:02 +01:00
Antonio Andelic
cbe92bf8c8
Merge branch 'master' into keeper-prometheus 2022-11-11 13:56:27 +01:00
Nikolai Kochetov
63d06c8b53
Merge pull request #42829 from ClickHouse/read-in-order-from-query-plan 
Read-in-order over query plan
 2022-11-11 12:15:20 +01:00
Raúl Marín
54db7c6520 Enforce checking read output 2022-11-11 10:56:18 +01:00
Azat Khuzhin
3b2ceee753 Apply connection timeouts settings for Distributed async INSERT from the query 
Previosly connection related settings (connect_timeout_with_failover_ms,
connect_timeout_with_failover_secure_ms) was applied from the query only
for the case insert_distributed_sync=1, and in case of async INSERT it
uses global settings.

Note that this changes how connections is allocated, so now
split_batch_on_failure will create it's own connection, and this can
introduce more duplicates since in case of split_batch_on_failure is
enabled it may send files to different server, but this should not be a
problem because:
- it does not resend batch if it has only one file, when deduplication
  will work
- and in all other cases deduplication will not work since checksum
  should be different

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-11 09:54:07 +01:00
Azat Khuzhin
d1cee3e1ff Do not resend batch as separate files if there is only one file 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-11 09:48:44 +01:00
zzsmdfj
3835373644 to add_oss_function_and_StorageOSS 2022-11-11 16:40:10 +08:00
Amos Bird
661c541e57
Fix Nullable keys in hyperrectangle. 2022-11-11 11:14:05 +08:00
kssenii
63de577172 Better, add tests 2022-11-10 22:47:30 +01:00
avogar
9e89af28c6 Refactor BSONEachRow format, fix bugs, support more data types, support parallel parsing and schema inference 2022-11-10 20:15:14 +00:00
Alexander Tokmakov
bb00a246f1 priority for DROP_RANGEs, assertion for others 2022-11-10 20:53:05 +01:00
kssenii
a5d06cd909 Better Impl 2022-11-10 19:59:52 +01:00
Alexander Tokmakov
e415fc6e95
Revert "Optimize TTL merge, completely expired parts can be removed in time" 2022-11-10 19:38:55 +03:00
kssenii
a47aac3e2b Better system table structure 2022-11-10 17:07:17 +01:00
kssenii
bed7ac9442 Support arbitrary structure collections 2022-11-10 17:05:24 +01:00
kssenii
c4f522b414 Support arbintrary structure collections 2022-11-10 16:32:15 +01:00
Igor Nikonov
f8c907c463
Revert "Revert " Keeper retries during insert (clean)"" 2022-11-10 13:14:04 +01:00
Alexander Tokmakov
fb8c527a2f
Revert " Keeper retries during insert (clean)" 2022-11-10 14:12:24 +03:00
Igor Nikonov
1b9be27fdb
Merge pull request #42607 from ClickHouse/igor/insert_zk_retries_retry 
Keeper retries during insert (clean)
 2022-11-10 12:01:00 +01:00
Antonio Andelic
1954ff385d
Merge pull request #43076 from canhld94/ch_canh_minor 
Better logging for mark range filtering on projection parts
 2022-11-10 09:33:12 +01:00
Antonio Andelic
f2a9b5933d Merge branch 'master' into keeper-prometheus 2022-11-10 09:28:06 +01:00
Nikolai Kochetov
f2f5c170e4
Merge branch 'master' into read-in-order-from-query-plan 2022-11-09 22:36:36 +01:00
alesapin
9e1627fcdd
Merge pull request #43060 from ClickHouse/fix_projection_part 
Fix projection part removal with zero-copy replication
 2022-11-09 18:15:49 +01:00
kssenii
e94e087187 Add generic named collections class and factory 2022-11-09 14:08:16 +01:00
Igor Nikonov
379113f8f1 Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry 2022-11-09 13:03:32 +00:00
Antonio Andelic
c92ae5a385 Add suport for AsyncMetrics in standalone Keeper 2022-11-09 13:46:03 +01:00
Alexander Tokmakov
1c6a617207
Merge branch 'master' into fix_intersecting_parts2 2022-11-09 14:38:18 +03:00
Maksim Kita
768ada118b Analyzer AST key condition crash fix 2022-11-09 11:41:39 +01:00
Vladimir C
82bf992099
Merge pull request #42869 from zhongyuankai/optimze_ttl_merge 2022-11-09 11:26:46 +01:00
Duc Canh Le
b9a3d090ed use const string & 2022-11-09 13:07:24 +08:00
Duc Canh Le
901e1e2fef better logging for mark range filtering on projection parts 2022-11-09 12:01:51 +08:00
alesapin
761274d857 Fix wrong logic 2022-11-09 01:14:03 +01:00
Igor Nikonov
345304abe0 Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry 2022-11-08 20:26:50 +00:00
alesapin
a31a79020f Add comment 2022-11-08 16:27:26 +01:00
alesapin
6f059be748 Fix bug with temporary projection parts deletion 2022-11-08 15:37:03 +01:00
serxa
2de26daa56 fix build 2022-11-08 14:31:29 +00:00
SmitaRKulkarni
658036d179
Merge pull request #42708 from ClickHouse/42414_Enable_functions_in_readonly_mode 
42414 enable functions in readonly mode
 2022-11-08 15:22:31 +01:00
Alexander Tokmakov
eb19df0094 fix 2022-11-08 15:21:25 +01:00
Sergei Trifonov
8eedd1e046
Merge branch 'master' into disk-s3-throttler 2022-11-08 15:00:56 +01:00
SmitaRKulkarni
4f2e5596e3
Merge pull request #42933 from yakkomajuri/patch-2 
fix(typo): Passible -> Possible
 2022-11-08 14:54:50 +01:00
serxa
6d5d9ff421 rename ReadWriteSettings -> RequestSettings 2022-11-08 13:48:23 +00:00
Maksim Kita
0aa4fd3c2d Fixed tests 2022-11-08 14:29:17 +01:00
Kseniia Sumarokova
d2b777c059
Merge pull request #41054 from rubin-do/hudi 
Add Hudi and DeltaLake table engines
 2022-11-08 12:03:06 +01:00
Kseniia Sumarokova
92b5ca326c
Merge pull request #42303 from xiedeyantu/system_table_add_column 
system.detached_parts add column bytes_on_disk and path
 2022-11-08 12:00:56 +01:00
Robert Schulze
2c568df714
Merge pull request #42968 from ClickHouse/bitcast 
Less use of CH-specific bit_cast()
 2022-11-08 10:54:10 +01:00
Robert Schulze
bb507356ef
Merge pull request #42209 from canhld94/ch_canh_fix_prefix_not_like 
Add `notLike` to key condition atom map
 2022-11-08 10:07:21 +01:00
Antonio Andelic
7067f9e0e1
Merge pull request #42947 from azat/mergetree/part-load-tiny 
Attach thread pool for loading parts to the query
 2022-11-08 09:17:32 +01:00
zhongyuankai
37099e57f4
Merge branch 'master' into optimze_ttl_merge 2022-11-08 12:26:46 +08:00
Azat Khuzhin
2bd4a71150
Merge branch 'master' into fetch-remove-covered 2022-11-08 00:55:00 +01:00
Kseniia Sumarokova
4fc7926c16
Merge branch 'master' into system_table_add_column 2022-11-07 23:36:57 +01:00
Kseniia Sumarokova
d695240d53
Update StorageHudi.cpp 2022-11-07 23:07:00 +01:00
serxa
ea17b30358 Merge branch 'storage-system-moves' of github.com:ClickHouse/ClickHouse into storage-system-moves 2022-11-07 20:33:31 +00:00
serxa
59408ebf57 fill more fields 2022-11-07 20:33:25 +00:00
Sergei Trifonov
6d3b96ee42
Merge branch 'master' into storage-system-moves 2022-11-07 21:31:27 +01:00
alesapin
257e766e0e Trying to fix two flaky tests 2022-11-07 20:49:41 +01:00
Alexander Tokmakov
9210e586d2 fix 2022-11-07 20:27:18 +01:00
Alexander Tokmakov
b8174a63a8 Merge branch 'master' into fix_intersecting_parts2 2022-11-07 20:26:36 +01:00
serxa
2daec0b45e S3 request per second rate throttling + refactoring 2022-11-07 18:05:40 +00:00
xiedeyantu
d4cdc38b71 better 2022-11-07 22:49:58 +08:00
Kseniia Sumarokova
891e167509
Update StorageHudi.cpp 2022-11-07 15:17:16 +01:00
Kruglov Pavel
48163ff9c2
Merge pull request #42938 from amosbird/issue_42771 
Fix NOT_FOUND_COLUMN_IN_BLOCK in some rare case
 2022-11-07 13:10:12 +01:00
kssenii
897811371d Minor changes 2022-11-07 12:45:26 +01:00
kssenii
819be6bb32 Merge remote-tracking branch 'upstream/master' into hudi 2022-11-07 11:30:53 +01:00
Duc Canh Le
bd2bd7149d fix typo 2022-11-07 18:22:50 +08:00
SmitaRKulkarni
c563e124c2
Merge branch 'master' into 42414_Enable_functions_in_readonly_mode 2022-11-07 09:50:05 +01:00
Duc Canh Le
f7e65ebd72 make comments more clear 2022-11-07 16:25:36 +08:00
Robert Schulze
cdaf0becfe
Merge branch 'master' into bitcast 2022-11-07 09:24:52 +01:00
Robert Schulze
f3f50530d1
Merge branch 'master' into ch_canh_fix_prefix_not_like 2022-11-07 09:14:02 +01:00
Antonio Andelic
e08f94d0f8
Merge pull request #42659 from ClickHouse/optimize-entire-partition 
Support for optimizing old parts for entire partition only
 2022-11-07 09:06:11 +01:00
Duc Canh Le
69d6d42443 NOT LIKE only work for 'prefix%' 2022-11-06 15:26:19 +08:00
zhongyuankai
ef6065b331 fix test 2022-11-06 13:40:26 +08:00
Duc Canh Le
a8bce28c04 Merge branch 'master' into ch_canh_fix_prefix_not_like 2022-11-06 13:04:08 +08:00
Igor Nikonov
e94b9cd003 Fixes 
+ enable retries in integration test
+ typo in fault injection traces
 2022-11-05 22:28:32 +00:00
Igor Nikonov
859022660e Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry 2022-11-04 23:31:51 +00:00
Daniil Rubin
7500cc38c0 Apply clang-format 2022-11-04 20:51:44 +00:00
Daniil Rubin
37048e986a Add comments, fix review issues 2022-11-04 20:49:24 +00:00
Daniil Rubin
d6bbd65dde Fix broken include 2022-11-04 16:03:12 +00:00
Robert Schulze
9c066e964d
Less use of CH-specific bit_cast() 
Converted usage of CH-custom bit_cast to std::bit_cast if possible, i.e.
when
  sizeof(From) == sizeof(To).
(The CH-custom bit_cast is able to deal with sizeof(From) != sizeof(To).)

Motivation for this came from #42847 where it is not clear how the
internal bit_cast should behave on big endian systems, so we better
avoid that situation as much as possible.
 2022-11-04 15:52:48 +00:00
Vladimir C
f359411668
Merge branch 'master' into optimze_ttl_merge 2022-11-04 16:52:20 +01:00
Vitaly Baranov
52b1f4aed9
Merge pull request #42484 from vitlibar/mask-sensitive-info-in-logs 
Mask some information in logs
 2022-11-04 14:09:38 +01:00
zhongyuankai
1345f7e73d Merge branch 'master' into optimze_ttl_merge 2022-11-04 20:54:46 +08:00
alesapin
82729a88c8
Merge pull request #42833 from vitlibar/backup-s3-increase-upload-part-size-exponentially 
Increase the size of upload part exponentially for backup to S3.
 2022-11-04 13:41:10 +01:00
Kseniia Sumarokova
ed44b20694
Merge pull request #42224 from kssenii/fit-http-buffer-retries 
Fix http buffer retries
 2022-11-04 11:50:17 +01:00
Amos Bird
d901ead1bc
Done 2022-11-04 17:26:44 +08:00
zhongyuankai
049a593272 Merge branch 'master' into optimze_ttl_merge 2022-11-04 17:22:38 +08:00
zhongyuankai
e773eb2fd1 batter 2022-11-04 17:19:34 +08:00
Azat Khuzhin
7032b2b98a Attach thread pool for loading parts to the query 
Detach should be done via CurrentThread::defaultThreadDeleter()

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-04 09:06:12 +01:00
Azat Khuzhin
cdd09fd72a Improve messages for loading data parts (add part type) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-04 09:05:06 +01:00
Daniil Rubin
99d57325bb Merge branch 'master' into hudi 2022-11-03 19:29:23 +00:00
Daniil Rubin
a573d8aef7 Apply clang-format 2022-11-03 18:17:39 +00:00
Daniil Rubin
7b57e94d9f Fix StorageHudi 2022-11-03 18:14:10 +00:00
Daniil Rubin
a9f8948c8d Fix build 2022-11-03 17:28:41 +00:00
Amos Bird
7186898ffa
Experiment 2022-11-04 00:29:17 +08:00
Yakko Majuri
e68a3b0c7a
fix(typo): Passible -> Possible 2022-11-03 11:25:07 -03:00
kssenii
4fe4a07600 Add test 2022-11-03 12:14:08 +01:00
Antonio Andelic
66ffdda973
Merge branch 'master' into optimize-entire-partition 2022-11-03 09:52:13 +01:00
Alexander Tokmakov
e67d05604c Merge branch 'master' into fix_intersecting_parts2 2022-11-02 19:30:27 +01:00
Alexander Tokmakov
d8b3a2af84 fix race between INSERT and DROP 2022-11-02 19:26:50 +01:00
Vitaly Baranov
32194c1200 Add max limitation for the size of an uploaded part. 2022-11-02 17:53:54 +01:00
Igor Nikonov
9a315b7899 Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry 2022-11-02 15:32:54 +00:00
Vitaly Baranov
e0133688bc
Merge branch 'master' into mask-sensitive-info-in-logs 2022-11-02 16:26:13 +01:00
Alexander Tokmakov
00c9e50ee3 fix race between INSERT and ALTER PARTITION 2022-11-02 16:25:26 +01:00
Alexander Tokmakov
acbad99b24 remove abandonable_lock part 2 2022-11-02 14:28:49 +01:00
Antonio Andelic
c4b717b343
Update src/Storages/MergeTree/MergeTreeDataMergerMutator.cpp 
Co-authored-by: Sergei Trifonov <sergei@clickhouse.com>
 2022-11-02 08:24:54 +01:00
Duc Canh Le
ec382d7903 add comment 2022-11-02 09:50:41 +08:00
Duc Canh Le
c6598dc66d
Merge branch 'master' into ch_canh_fix_prefix_not_like 2022-11-02 09:42:01 +08:00
Alexander Tokmakov
4f50a999bb remove abandonable_lock part 1 2022-11-01 21:35:25 +01:00
Nikolai Kochetov
478d3079c8 Merge branch 'master' into read-in-order-from-query-plan 2022-11-01 21:02:22 +01:00
Nikolai Kochetov
9ffebf4a0c Fixing read-in-order for special storages. 2022-11-01 19:51:52 +00:00
Igor Nikonov
e76c3c381f Better handling tables in read-only mode during insert 
+ throw immediately if table has no metadata in zk
+ stop retries in case of shutdown
+ check if table is readonly at the begining of every retry
 2022-11-01 18:47:51 +00:00
Anton Popov
88674ef964
Merge pull request #42794 from CurtizJ/better-code-in-check-part 
Slightly better code with projections
 2022-11-01 16:05:27 +01:00
zhongyuankai
c1e2b2af74 Optimize TTL merge, completely expired parts can be removed in time 2022-11-01 22:58:15 +08:00
Anton Popov
876dca48da
Merge pull request #36969 from CurtizJ/dynamic-columns-14 
Support `Object` type inside other types
 2022-11-01 15:20:02 +01:00
Duc Canh Le
07d9720ed5
Merge branch 'master' into ch_canh_fix_prefix_not_like 2022-11-01 22:16:09 +08:00
Kruglov Pavel
21d50f76ea
Merge pull request #41979 from Avogar/s3-cluster-schema-inference 
Fix schema inference in s3Cluster and improve in hdfsCluster
 2022-11-01 14:00:21 +01:00
Vitaly Baranov
b9f2f17331 Add test and logging. 2022-11-01 12:23:20 +01:00
Maksim Kita
fe48a1ce7e
Merge pull request #42681 from kitaisreal/merge-tree-indexes-use-actions-dag 
MergeTree indexes use ActionsDAG
 2022-11-01 13:55:17 +03:00
Duc Canh Le
3545db0a6c
Merge branch 'master' into ch_canh_fix_prefix_not_like 2022-11-01 16:54:26 +08:00
Yakov Olkhovskiy
bab0e06e3d
Merge pull request #42458 from Clark0/match-func-use-index 
match function can use index if it's a condition on string prefix #37333
 2022-10-31 17:33:58 -04:00
Igor Nikonov
9cc64a0fcc Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry 2022-10-31 18:21:21 +00:00
Igor Nikonov
2d5050ea66 Fix review comments 2022-10-31 18:20:58 +00:00
Anton Popov
2ae3cfa9e0
Merge branch 'master' into dynamic-columns-14 2022-10-31 16:15:19 +01:00
Alexander Tokmakov
599ccb9939
Update MergeTreeData.h 2022-10-31 16:18:17 +03:00
Alexander Tokmakov
b2b9479afa
Update src/Storages/MergeTree/MergeTreeData.h 
Co-authored-by: Sergei Trifonov <sergei@clickhouse.com>
 2022-10-31 16:15:35 +03:00
SmitaRKulkarni
54035270b2
Merge branch 'master' into 42414_Enable_functions_in_readonly_mode 2022-10-31 14:15:29 +01:00
Maksim Kita
bca22ec5f5 Fixed code review issues 2022-10-31 12:30:00 +01:00
Maksim Kita
115fcaffc5 Fixed tests 2022-10-31 11:30:56 +01:00
Vitaly Baranov
5d2a222fe4 Mask sensitive information in logs. 2022-10-31 10:50:33 +01:00
Duc Canh Le
d309e8793a
Merge branch 'master' into ch_canh_fix_prefix_not_like 2022-10-31 15:13:39 +08:00
Alexander Tokmakov
f53df7870c fix race between drop and failed insert 2022-10-30 17:59:47 +01:00
Alexander Tokmakov
c68ab231f9 fix accessing part in Deleting state 2022-10-30 17:30:51 +01:00
FArthur-cmd
607b81989d fix build 2022-10-30 14:57:10 +00:00
Anton Popov
8de75bf29c better code with projections 2022-10-29 14:39:59 +00:00
clarkcaoliu
ca5dbe88db match function can use index if prefix 2022-10-29 18:25:29 +08:00
Filatenkov Artur
21c024b043
Merge branch 'master' into annoy_cosine 2022-10-28 20:15:53 +03:00
FArthur-cmd
f187d4e1d4 add cosine distance for annoy and fix docs for cosineDistance 2022-10-28 17:03:35 +00:00
Igor Nikonov
d09deaf10f Fix style check 2022-10-28 15:17:05 +00:00
Igor Nikonov
937dd44399 Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry 2022-10-28 15:09:23 +00:00
Igor Nikonov
c53b96a11c Add more comments 2022-10-28 15:09:01 +00:00
Alexey Milovidov
f64e632707
Merge pull request #42469 from amosbird/issue_42456 
Fix misbehavior of key analysis
 2022-10-28 16:18:00 +02:00
Antonio Andelic
5fae8b0cc9 Merge branch 'master' into optimize-entire-partition 2022-10-28 11:44:10 +00:00
serxa
acc00218d9 fix build 2022-10-28 11:23:49 +00:00
Sergei Trifonov
836595c7ae
Merge branch 'master' into storage-system-moves 2022-10-28 11:39:38 +02:00
Duc Canh Le
848f5b5d65
Merge branch 'master' into ch_canh_fix_prefix_not_like 2022-10-28 13:40:17 +08:00
Duc Canh Le
c080964391 correct 'notLike' in key condition 2022-10-28 13:39:01 +08:00
Duc Canh Le
69311bd5cd Merge branch 'master' into ch_canh_fix_prefix_not_like 2022-10-28 11:59:50 +08:00
Igor Nikonov
3682913172 Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry 2022-10-27 22:01:53 +00:00
Igor Nikonov
a601c166ad Add clean up for faults injected before zk request 
+ for multi-op request, it create responses in multi response
+ some polishing
 2022-10-27 21:35:13 +00:00
Anton Popov
7715afa595
Merge pull request #42618 from CurtizJ/refactor-data-part 
Try to save `IDataPartStorage` interface
 2022-10-27 22:20:44 +02:00
Igor Nikonov
f773436de5 Fix flaky test 2022-10-27 13:34:07 +00:00
Nikolai Kochetov
24e1aaf324
Merge pull request #42576 from ClickHouse/table-function-merge-invalid-where-type 
Invalid type in where for Merge table (logical error)
 2022-10-27 13:03:27 +02:00
Antonio Andelic
10aa7091ca
Merge pull request #42605 from ClickHouse/correctly-handle-prewhere-type 
Correctly handle non-native integers in PREWHERE
 2022-10-27 09:25:38 +02:00
SmitaRKulkarni
e860ecc80c
Merge branch 'master' into 42414_Enable_functions_in_readonly_mode 2022-10-27 08:26:44 +02:00
Nikolai Kochetov
313cbb1058 Fix test. 2022-10-26 17:49:36 +00:00
Igor Nikonov
649e8d302c Sync with retiring branch igor/insert_zk_retries 2022-10-26 17:41:44 +00:00
Igor Nikonov
8435cba352 Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry 2022-10-26 17:35:47 +00:00
Maksim Kita
f04f663818 Fixed tests 2022-10-26 18:55:14 +02:00
Smita Kulkarni
6a8fa2d4a5 Added new field allow_readonly in system.table_functions to allow using table functions in readonly mode. 
Implementation:
* Added a new field allow_readonly to table system.table_functions.
* Updated to use new field allow_readonly to allow using table functions in readonly mode.
* Added TableFunctionProperties similar to AggregateFunctionProperties.
* The functions allowed in readonly mode are in this set table_functions_allowed_in_readonly_mode.
Testing:
* Added a test for filesystem tests/queries/0_stateless/02473_functions_in_readonly_mode.sh
Documentation:
* Updated the english documentation for Table Functions.
 2022-10-26 18:45:23 +02:00
Nikolai Kochetov
1520bcd53c Fix error code. 2022-10-26 15:50:10 +00:00
Anton Popov
ca4df62e37
Merge branch 'master' into refactor-data-part 2022-10-26 17:49:20 +02:00
Mikhail f. Shiryaev
fdecac65b8
Merge pull request #42699 from ClickHouse/22.11-prepare 
Update version after release
 2022-10-26 16:31:48 +02:00
Maksim Kita
1c17e9d454 Fixed tests 2022-10-26 15:54:37 +02:00
Mikhail f. Shiryaev
d3461abd1f
Update version to 22.11.1.1 2022-10-26 15:27:09 +02:00
Nikolai Kochetov
db422fa903
Merge pull request #42410 from ClickHouse/fix-multi-arg-monotonicity 
Fix possible LOGICAL_ERROR in binary ariphmetics monotonicity.
 2022-10-26 14:11:23 +02:00
Kruglov Pavel
219553df8d
Merge pull request #42580 from hanfei1991/fei/limit_max_rows 
fix behaviour of max_rows_to_read for trival limit queries
 2022-10-26 14:09:59 +02:00
Maksim Kita
c47ca522f4 Fixed style check 2022-10-26 13:27:14 +02:00
Antonio Andelic
35a9672704 Merge branch 'master' into optimize-entire-partition 2022-10-26 11:09:25 +00:00
Maksim Kita
1b6293f6db MergeTree indexes use ActionsDAG 2022-10-26 12:44:37 +02:00
Antonio Andelic
97aaebfa18 Address PR comments 2022-10-26 10:06:56 +00:00
SmitaRKulkarni
5df138da82
Merge pull request #42376 from ClickHouse/36316_Support_glob_for_recursive_directory_traversal 
Added ** glob support for recursive directory traversal to filesystem…
 2022-10-26 11:49:20 +02:00
Antonio Andelic
979d180931
Merge branch 'master' into correctly-handle-prewhere-type 2022-10-26 11:13:16 +02:00
Azat Khuzhin
8db31beee6 Remove covered parts for fetched part 
Here is an example that I found on production, simplified.

Consider the following queue (nothing of this had been processed on this
replica):

- GET_PART all_0_0_0 (queue-0000000001)
- GET_PART all_1_1_0 (queue-0000000002)
...
- GET_PART all_0_1_1 (queue-0000000003)
- GET_PART all_2_2_0 (queue-0000000004)
...
- MERGE_PARTS from [all_0_1_1, all_2_2_0] to all_0_2_2 (queue-0000000005)

And now queue-0000000005 started to executing (either because
of reording, or because at that time GET_PART fails), and it
does not have any required parts, so it will fetch them, but
not all_0_0_0 and all_1_1_0, so this replica delay will set to
the time of min(queue-0000000001, queue-0000000002), while it
is not true, since it already have parts that covers those
parts.

and since MERGE_PARTS takes 30min, it increased the replica delay
eventually to 30min, for the time range of 30min, which is pretty huge.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-26 11:10:50 +03:00
Anton Popov
79889198b1 fix style check 2022-10-25 23:03:37 +00:00
Anton Popov
d8b09b430f add comments 2022-10-25 22:27:11 +00:00
Anton Popov
c4d4f2dbed better interface 2022-10-25 22:14:06 +00:00
Sergei Trifonov
34b43f8b4b
Merge branch 'master' into storage-system-moves 2022-10-25 22:56:45 +02:00
Nikolai Kochetov
bfb5e6eae4 Fix logical error for invalid prewhere type in StorageMerge. 2022-10-25 13:58:29 +00:00
serxa
55a86bc2de Add active moves system table 2022-10-25 13:56:46 +00:00
Antonio Andelic
611c2e2bd7 Support for optimizing old parts for entire partition only 2022-10-25 13:40:18 +00:00
Anton Popov
c8199bc125
Merge branch 'master' into refactor-data-part 2022-10-25 14:31:05 +02:00
Raúl Marín
0016bc2b73
Improve the time to recover keeper connections (#42541) 
Co-authored-by: Nikita Mikhaylov <nikitamikhaylov@clickhouse.com>
Co-authored-by: robot-clickhouse <robot-clickhouse@clickhouse.com>
Co-authored-by: Alexey Milovidov <milovidov@clickhouse.com>
 2022-10-25 13:26:29 +02:00
Antonio Andelic
af0c54e055 Small fix 2022-10-25 11:13:41 +00:00
Antonio Andelic
451a50d9e9 Merge branch 'master' into correctly-handle-prewhere-type 2022-10-25 10:48:35 +00:00
Maksim Kita
06fe6f3c8b
Merge pull request #31796 from kitaisreal/identifier-resolver 
Added Analyzer
 2022-10-25 12:36:08 +03:00
Anton Popov
cbfa887e52 fix merge of projections 2022-10-24 23:51:57 +00:00