thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-15 19:02:04 +00:00
Code Issues Packages Projects Releases Wiki Activity
103,893 Commits 9,326 Branches 1,783 Tags 2.1 GiB
7df137e460
Commit Graph

10614 Commits

Author SHA1 Message Date
Azat Khuzhin
d0ce500f89 Add table_uuid to system.parts 
Can be useful if the table constantly recreated, i.e. in some tests.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-21 12:11:51 +01:00
alesapin
b78b602ea1
Merge branch 'master' into Enable_compress_marks_and_primary_key 2022-11-21 12:05:04 +01:00
Nikolai Kochetov
be1a8054c7
Merge branch 'master' into read-from-mt-in-io-pool 2022-11-21 11:27:41 +01:00
Nikolai Kochetov
4a1774c5ac Fixing build. 2022-11-21 10:27:09 +00:00
Antonio Andelic
9a8d82f0fc Merge branch 'master' into fix-describe-delta-lake-hudi 2022-11-21 08:23:33 +00:00
Antonio Andelic
e78761b66c
Merge branch 'master' into keeper-prometheus 2022-11-21 09:10:10 +01:00
Guillaume Tassery
7267473aec
Merge branch 'master' into gt/add_expression 2022-11-21 09:07:12 +01:00
Guillaume Tassery
1f73c8dabd Merge branch 'gt/add_expression' of github.com:YiuRULE/ClickHouse into gt/add_expression 2022-11-21 09:06:25 +01:00
Guillaume Tassery
702768469b cosmetic 2022-11-21 09:04:44 +01:00
Kseniia Sumarokova
c12cfab953
Merge pull request #43343 from azat/disks/write-once 
Allow to "drop tables" from s3_plain disk (so as from web disk)
 2022-11-20 16:04:50 +01:00
Azat Khuzhin
177cbbac4b Avoid race condition for updating system.distribution_queue values 
Previously it was possible to have a race while updating
files_count/bytes_count, since INSERT updates it those counters from one
thread and the same metrics are updated from filesystem in a separate
thread, and even though the access is synchronized with the mutex it
avoids the race only for accessing the variables not the logical race,
since it is possible that getFiles() from a separate thread will
increment counters and later addAndSchedule() will increment them again.

Here you can find an example of this race [1].

  [1]: https://pastila.nl/?00950e00/41a3c7bbb0a7e75bd3f2922c58b02334

Note, that I analyzed logs from production system with lots of async
Distributed INSERT and everything is OK there, even though the logs
contains the following:

    2022.11.20 02:21:15.459483 [ 11528 ] {} <Trace> v21.dist_out.DirectoryMonitor: Files set to 35 (was 34)
    2022.11.20 02:21:15.459515 [ 11528 ] {} <Trace> v21.dist_out.DirectoryMonitor: Bytes set to 4035418 (was 3929008)
    2022.11.20 02:21:15.819488 [ 11528 ] {} <Trace> v21.dist_out.DirectoryMonitor: Files set to 1 (was 2)
    2022.11.20 02:21:15.819502 [ 11528 ] {} <Trace> v21.dist_out.DirectoryMonitor: Bytes set to 190072 (was 296482)

As you may see it first increases the counters and next update
decreases (and 4035418-3929008 == 296482-190072)

Refs: #23885
Reported-by: @tavplubix
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-20 14:13:01 +01:00
Alexey Milovidov
81ca8e91e6
Update StorageSystemDataSkippingIndices.cpp 2022-11-20 14:27:52 +03:00
Azat Khuzhin
4f6703c972 Add is_read_only/is_write_once/is_remote/is_broken to system.disks 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-19 10:11:39 +01:00
Azat Khuzhin
c029549859 Allow to drop tables from s3_plain disk (so as from web disk) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-19 10:10:27 +01:00
zhongyuankai
a3a3261745
Merge branch 'master' into optimize_ttl 2022-11-19 16:32:26 +08:00
zhongyuankai
ce9b4866ea
Update MergeTreeDataMergerMutator.cpp 2022-11-19 16:31:47 +08:00
Alexander Gololobov
f004eea413 Add columns required fro defaults calculation 2022-11-18 23:42:45 +01:00
alesapin
c7d648174b
Merge pull request #43328 from ClickHouse/better-format-version-check 
Check content of `format_version` file in `MergeTreeData`
 2022-11-18 22:34:37 +01:00
Kruglov Pavel
82d6f5357c
Merge pull request #43156 from azat/dist/insert-apply-timeouts 
Apply connection timeouts settings for Distributed async INSERT from the query
 2022-11-18 21:27:57 +01:00
Nikolai Kochetov
5da1d893fd
Merge branch 'master' into read-from-mt-in-io-pool 2022-11-18 21:10:45 +01:00
Nikolai Kochetov
03c35e146b Refactor a bit. 2022-11-18 20:09:20 +00:00
serxa
8e65b799c2 fix test + add one more column 2022-11-18 20:04:11 +00:00
Han Fei
b80a2d6c89 address comments 2022-11-18 17:22:05 +01:00
serxa
a05a1b0133 Merge branch 'storage-system-moves' of github.com:ClickHouse/ClickHouse into storage-system-moves 2022-11-18 15:35:00 +00:00
Sergei Trifonov
77100896e3
Merge branch 'master' into storage-system-moves 2022-11-18 16:33:43 +01:00
Sergei Trifonov
d05223e70b
Merge pull request #43335 from ClickHouse/revert-43306-revert-43014-disk-s3-throttler 
Revert "Revert "S3 request per second rate throttling""
 2022-11-18 16:22:30 +01:00
Antonio Andelic
25697fd87d
Merge pull request #43289 from ClickHouse/fix-another-race-rmt 
Fix race on write in `ReplicatedMergeTree`
 2022-11-18 16:09:50 +01:00
alesapin
0f3eb51aed Disable compressed marks by default 2022-11-18 15:54:48 +01:00
Vitaly Baranov
a348332eab
Merge pull request #43227 from vitlibar/improve-masking-sensitive-info 
Improve masking sensitive info
 2022-11-18 15:37:50 +01:00
Han Fei
74c9ba8df0
Update src/Storages/MergeTree/ReplicatedMergeTreeSink.cpp 
Co-authored-by: alesapin <alesapin@gmail.com>
 2022-11-18 11:16:38 +01:00
Han Fei
3dc72b6c56
Update src/Storages/MergeTree/ReplicatedMergeTreeSink.cpp 
Co-authored-by: alesapin <alesapin@gmail.com>
 2022-11-18 11:15:51 +01:00
Han Fei
8c823bcaab
Update src/Storages/MergeTree/ReplicatedMergeTreeSink.cpp 
Co-authored-by: alesapin <alesapin@gmail.com>
 2022-11-18 11:15:45 +01:00
Han Fei
035e933196
Update src/Storages/MergeTree/ReplicatedMergeTreeSink.cpp 
Co-authored-by: alesapin <alesapin@gmail.com>
 2022-11-18 11:15:35 +01:00
Han Fei
f8ac75a054
Update src/Storages/MergeTree/ReplicatedMergeTreeSink.cpp 
Co-authored-by: alesapin <alesapin@gmail.com>
 2022-11-18 11:15:14 +01:00
Han Fei
d41094207d
Update src/Storages/MergeTree/EphemeralLockInZooKeeper.cpp 
Co-authored-by: alesapin <alesapin@gmail.com>
 2022-11-18 11:14:56 +01:00
Mikhail f. Shiryaev
f942ab1bd2
Update version to 22.12.1.1 2022-11-17 22:15:12 +01:00
Nikolai Kochetov
679591cf0a Try fix race. 2022-11-17 19:51:47 +00:00
Azat Khuzhin
aa77beabc9 Fix possible "No column to rollback" logical error during INSERT into Buffer 
In case of concurrent INSERT and SELECT, and memory pressure it is
possible to get this error, if you are very lucky.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-17 19:12:58 +01:00
Vitaly Baranov
050df6ac7f Move InDepthNodeVisitor.h back to src/Interpreters. 2022-11-17 18:16:32 +01:00
Azat Khuzhin
e67fc54fc9 StorageBuffer: log exception before rollback 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-17 18:10:11 +01:00
Sergei Trifonov
f2f0676bcc
Revert "Revert "S3 request per second rate throttling"" 2022-11-17 17:35:04 +01:00
Vitaly Baranov
2b9ed89721 Rename formatWithHiddenSecrets() -> formatWithSecretsHidden(). 2022-11-17 17:30:37 +01:00
kssenii
4e2fd5e22a Slightly better loading of metadata 2022-11-17 16:35:05 +01:00
alesapin
29a08f64af Merge branch 'master' into Enable_compress_marks_and_primary_key 2022-11-17 15:53:37 +01:00
Antonio Andelic
d324e8cc3e Write to first non-readonly disk 2022-11-17 14:52:35 +00:00
kssenii
5d9b34b59d More tests 2022-11-17 15:34:35 +01:00
Antonio Andelic
d41af8b741 Check and compare content of format_version file 2022-11-17 13:41:36 +00:00
alesapin
06f395d946 Add comments 2022-11-17 13:10:14 +01:00
Kruglov Pavel
a406557263
Merge branch 'master' into dist/insert-apply-timeouts 2022-11-17 12:51:47 +01:00
Antonio Andelic
d5848d53cc Fix DESCRIBE for deltaLake and hudi table functions 2022-11-17 11:46:17 +00:00
alesapin
a79d83549f Merge branch 'master' into fix_parts_to_do_detection 2022-11-17 12:30:05 +01:00
Guillaume Tassery
c47f90d197 add expression index in system.data_skipping_indices 2022-11-16 20:54:55 +01:00
Alexander Tokmakov
9011a18234
Revert "S3 request per second rate throttling" 2022-11-16 22:33:48 +03:00
Sergei Trifonov
159743edd6
Merge pull request #43014 from ClickHouse/disk-s3-throttler 
S3 request per second rate throttling
 2022-11-16 18:51:06 +01:00
Han Fei
221c4e5912 tiny changes 2022-11-16 17:59:08 +01:00
kssenii
f0ce28349d Better 2022-11-16 17:58:21 +01:00
Han Fei
14701670b5 Merge branch 'master' into hanfei/aysnc-insert 2022-11-16 17:24:36 +01:00
Nikolai Kochetov
17b75d3f5c Fix metrics and progress. 2022-11-16 14:10:56 +00:00
Antonio Andelic
11aba7597e Check if initialization is done on write 2022-11-16 13:33:22 +00:00
Smita Kulkarni
040f2b62d9 Enable compress marks and primary key 
Implementation:
* Set compress_marks and compress_primary_key to true by default.
* Updated getIndexExtensionFromFilesystem to only check for compressed file extension , else return default. When both are present, this function was returning uncompressed file extension giving rise to error as file got removed.
Testing:
* Updated tests where bytes size or compressed data is checked to check with compress_marks and compress_primary_key set to false.
 2022-11-16 14:23:58 +01:00
filimonov
dcb76f6bbf
Allow autoremoval of old parts if detach_not_byte_identical_parts enabled 
Allow autoremoval of old & detached parts if detach_not_byte_identical_parts enabled.

See also #28708 #37975
 2022-11-16 14:16:42 +01:00
alesapin
2012f4bd36 Fix use-after-free 2022-11-16 11:50:51 +01:00
Alexey Milovidov
04687c0d69
Merge pull request #43206 from ClickHouse/miscellaneous-changes-parallel-replicas 
Miscellaneous changes
 2022-11-16 05:30:03 +01:00
Vitaly Baranov
d365570b27 Hide passwords in SHOW CREATE TABLE and system.tables. 2022-11-16 01:35:11 +01:00
Nikolai Kochetov
1b1d23c100 Clumsy attempt to fix progress. 2022-11-15 21:23:18 +00:00
Alexander Tokmakov
69b067b81c
Merge pull request #43244 from ClickHouse/fix-replicated-merge-tree-shared-id 
Don't throw if shared ID already created in `StorageReplicatedMergeTree`
 2022-11-15 23:31:51 +03:00
Nikolai Kochetov
cf1afbfb84 Temporarily enable allow_asynchronous_read_from_io_pool_for_merge_tree by default to see thest results. 2022-11-15 17:47:26 +00:00
Nikolai Kochetov
29098045d8 Try read from MergeTree in background pool. 2022-11-15 17:38:37 +00:00
kssenii
234f6ee54d Add commands CREATE/ALTER/DROP NAMED COLLECTION 2022-11-15 15:51:56 +01:00
Sema Checherinda
7e73b187cc
Merge pull request #43159 from Algunenano/over-read 
Fix several buffer over-reads
 2022-11-15 15:04:24 +01:00
alesapin
a12f2d14dc Simpler check 2022-11-15 14:46:38 +01:00
Anton Popov
8c66c759d4
Merge pull request #43133 from amosbird/issue_43111 
Fix Nullable keys in hyperrectangle
 2022-11-15 14:30:13 +01:00
alesapin
4ec11d0f58 Remove redundant branch 2022-11-15 13:58:53 +01:00
Antonio Andelic
0e5e58bed9 Remove exception if shared ID already created 2022-11-15 12:18:24 +00:00
Antonio Andelic
96838f5be7 Merge branch 'master' into keeper-prometheus 2022-11-15 12:00:02 +00:00
Kseniia Sumarokova
59cf5def67
Merge branch 'master' into disk-s3-throttler 2022-11-15 12:13:37 +01:00
xiedeyantu
ec6698395e fix skip_unavailable_shards does not work using hdfsCluster table function 2022-11-15 13:25:15 +08:00
zhongyuankai
35690c1972 Merge branch 'master' into optimize_ttl 2022-11-15 08:49:37 +08:00
zhongyuankai
16f697f61d fix test 2022-11-15 08:47:51 +08:00
Alexey Milovidov
6e01b2b2a1
Merge pull request #43203 from ClickHouse/fix-typo-2 
Fix typo
 2022-11-15 01:39:34 +01:00
Alexey Milovidov
3cab3ce484
Merge pull request #43198 from FArthur-cmd/MySQL_clickhouse_connect 
add columns to information schema
 2022-11-15 01:11:29 +01:00
alesapin
6a50a32cfc Fix parts_to_do 2022-11-15 00:53:20 +01:00
alesapin
f53ada2bbd Revert "Fix bug in replication queue which can lead to premature mutation finish" 
This reverts commit 3b5424811c.
 2022-11-14 22:59:09 +01:00
alesapin
3b5424811c Fix bug in replication queue which can lead to premature mutation finish 2022-11-14 21:53:16 +01:00
Kruglov Pavel
386b69acae
Merge pull request #43131 from xiedeyantu/fix_s3cluster_connection_fail 
fix skip_unavailable_shards does not work using s3Cluster table function
 2022-11-14 20:48:00 +01:00
Vitaly Baranov
8e99f5fea3 Move maskSensitiveInfoInQueryForLogging() to src/Parsers/ 2022-11-14 18:55:19 +01:00
Alexander Tokmakov
c64109a2de
Merge branch 'master' into fetch-remove-covered 2022-11-14 18:23:16 +03:00
zhongyuankai
ea11d80536 Priority is given to deleting completely expired Parts 2022-11-14 23:03:56 +08:00
Han Fei
11f93dec3f tmp commit 2022-11-14 15:53:45 +01:00
Raúl Marín
4988e387f2 Merge remote-tracking branch 'blessed/master' into over-read 2022-11-14 12:28:31 +01:00
Kseniia Sumarokova
b7df9aa3da
Merge pull request #43155 from zzsmdfj/add_oss_function_and_StorageOSS 
to add_oss_function_and_StorageOSS
 2022-11-14 12:23:57 +01:00
Kseniia Sumarokova
e58a7140be
Merge pull request #43194 from ClickHouse/async-metrics-documentation 
Add documentation for `system.asynchronous_metrics`.
 2022-11-14 12:21:34 +01:00
Kseniia Sumarokova
79206c6eb0
Merge pull request #43147 from kssenii/named-collections-refactoring 
Add generic implementation for arbitrary structured named collections, access type and system.named_collections
 2022-11-14 11:57:17 +01:00
Antonio Andelic
ee182776da
Merge branch 'master' into keeper-prometheus 2022-11-14 09:14:13 +01:00
Alexey Milovidov
4edd8b0135 Remove unused method 2022-11-14 08:07:27 +01:00
Alexey Milovidov
17693de053 Better logs 2022-11-14 08:01:48 +01:00
Alexey Milovidov
9c48664f11 Better logs 2022-11-14 07:48:50 +01:00
Alexey Milovidov
582db233c4 Better logs 2022-11-14 07:22:17 +01:00
Alexey Milovidov
99f84d8095 Better logs 2022-11-14 07:22:02 +01:00
Alexey Milovidov
df01689ccc Better logs 2022-11-14 07:13:42 +01:00
Alexey Milovidov
a77032fafa Miscellaneous changes 2022-11-14 06:54:58 +01:00
Alexey Milovidov
15202833b1 Miscellaneous changes 2022-11-14 06:48:49 +01:00
Alexey Milovidov
53ee7446dc Miscellaneous changes 2022-11-14 06:46:37 +01:00
Alexey Milovidov
cc1855ea33 Miscellaneous changes 2022-11-14 06:45:52 +01:00
Alexey Milovidov
4966556f47 Miscellaneous changes 2022-11-14 06:41:51 +01:00
Alexey Milovidov
85dc692f7c Miscellaneous changes 2022-11-14 06:37:55 +01:00
Alexey Milovidov
60c96072e1 Miscellaneous changes 2022-11-14 06:37:19 +01:00
Alexey Milovidov
e080161ba5 Miscellaneous changes 2022-11-14 06:32:18 +01:00
Alexey Milovidov
bfea3deed9 Miscellaneous changes 2022-11-14 06:09:03 +01:00
Alexey Milovidov
3c603c3d8c
Update columns.sql 2022-11-14 02:42:30 +03:00
Alexey Milovidov
b9471e492a Fix typo 2022-11-13 22:52:21 +01:00
FArthur-cmd
f6b334fb3a add columns to information scheme 2022-11-13 12:03:46 +00:00
Alexey Milovidov
ba05b7dd2c Add documentation for AsynchronousMetrics 2022-11-13 03:36:20 +01:00
Filatenkov Artur
c7c52a570f
Update MergeTreeIndexAnnoy.cpp 2022-11-12 12:43:25 +03:00
FArthur-cmd
5eebc27d24 improve test 2022-11-12 09:23:49 +00:00
Nikolai Kochetov
9106910de8 Try read from MergeTree in background pool. 2022-11-11 20:47:11 +00:00
Igor Nikonov
ee5756b5b4
Merge pull request #43122 from ClickHouse/revert-43116-revert-42607-igor/insert_zk_retries_retry 
Revert "Revert " Keeper retries during insert (clean)""
 2022-11-11 18:22:38 +01:00
xiedeyantu
5504f3af9b fix skip_unavailable_shards does not work using s3Cluster table function 2022-11-12 00:03:36 +08:00
kssenii
616da1f695 Review fixed, style check fix 2022-11-11 16:50:02 +01:00
Antonio Andelic
cbe92bf8c8
Merge branch 'master' into keeper-prometheus 2022-11-11 13:56:27 +01:00
Nikolai Kochetov
63d06c8b53
Merge pull request #42829 from ClickHouse/read-in-order-from-query-plan 
Read-in-order over query plan
 2022-11-11 12:15:20 +01:00
Raúl Marín
54db7c6520 Enforce checking read output 2022-11-11 10:56:18 +01:00
Azat Khuzhin
3b2ceee753 Apply connection timeouts settings for Distributed async INSERT from the query 
Previosly connection related settings (connect_timeout_with_failover_ms,
connect_timeout_with_failover_secure_ms) was applied from the query only
for the case insert_distributed_sync=1, and in case of async INSERT it
uses global settings.

Note that this changes how connections is allocated, so now
split_batch_on_failure will create it's own connection, and this can
introduce more duplicates since in case of split_batch_on_failure is
enabled it may send files to different server, but this should not be a
problem because:
- it does not resend batch if it has only one file, when deduplication
  will work
- and in all other cases deduplication will not work since checksum
  should be different

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-11 09:54:07 +01:00
Azat Khuzhin
d1cee3e1ff Do not resend batch as separate files if there is only one file 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-11 09:48:44 +01:00
zzsmdfj
3835373644 to add_oss_function_and_StorageOSS 2022-11-11 16:40:10 +08:00
Amos Bird
661c541e57
Fix Nullable keys in hyperrectangle. 2022-11-11 11:14:05 +08:00
kssenii
63de577172 Better, add tests 2022-11-10 22:47:30 +01:00
avogar
9e89af28c6 Refactor BSONEachRow format, fix bugs, support more data types, support parallel parsing and schema inference 2022-11-10 20:15:14 +00:00
Alexander Tokmakov
bb00a246f1 priority for DROP_RANGEs, assertion for others 2022-11-10 20:53:05 +01:00
kssenii
a5d06cd909 Better Impl 2022-11-10 19:59:52 +01:00
Alexander Tokmakov
e415fc6e95
Revert "Optimize TTL merge, completely expired parts can be removed in time" 2022-11-10 19:38:55 +03:00
kssenii
a47aac3e2b Better system table structure 2022-11-10 17:07:17 +01:00
kssenii
bed7ac9442 Support arbitrary structure collections 2022-11-10 17:05:24 +01:00
kssenii
c4f522b414 Support arbintrary structure collections 2022-11-10 16:32:15 +01:00
Igor Nikonov
f8c907c463
Revert "Revert " Keeper retries during insert (clean)"" 2022-11-10 13:14:04 +01:00
Alexander Tokmakov
fb8c527a2f
Revert " Keeper retries during insert (clean)" 2022-11-10 14:12:24 +03:00
Igor Nikonov
1b9be27fdb
Merge pull request #42607 from ClickHouse/igor/insert_zk_retries_retry 
Keeper retries during insert (clean)
 2022-11-10 12:01:00 +01:00
Antonio Andelic
1954ff385d
Merge pull request #43076 from canhld94/ch_canh_minor 
Better logging for mark range filtering on projection parts
 2022-11-10 09:33:12 +01:00
Antonio Andelic
f2a9b5933d Merge branch 'master' into keeper-prometheus 2022-11-10 09:28:06 +01:00
Nikolai Kochetov
f2f5c170e4
Merge branch 'master' into read-in-order-from-query-plan 2022-11-09 22:36:36 +01:00
alesapin
9e1627fcdd
Merge pull request #43060 from ClickHouse/fix_projection_part 
Fix projection part removal with zero-copy replication
 2022-11-09 18:15:49 +01:00
kssenii
e94e087187 Add generic named collections class and factory 2022-11-09 14:08:16 +01:00
Igor Nikonov
379113f8f1 Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry 2022-11-09 13:03:32 +00:00
Antonio Andelic
c92ae5a385 Add suport for AsyncMetrics in standalone Keeper 2022-11-09 13:46:03 +01:00
Alexander Tokmakov
1c6a617207
Merge branch 'master' into fix_intersecting_parts2 2022-11-09 14:38:18 +03:00
Maksim Kita
768ada118b Analyzer AST key condition crash fix 2022-11-09 11:41:39 +01:00
Vladimir C
82bf992099
Merge pull request #42869 from zhongyuankai/optimze_ttl_merge 2022-11-09 11:26:46 +01:00
Duc Canh Le
b9a3d090ed use const string & 2022-11-09 13:07:24 +08:00
Duc Canh Le
901e1e2fef better logging for mark range filtering on projection parts 2022-11-09 12:01:51 +08:00
alesapin
761274d857 Fix wrong logic 2022-11-09 01:14:03 +01:00
Igor Nikonov
345304abe0 Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry 2022-11-08 20:26:50 +00:00
alesapin
a31a79020f Add comment 2022-11-08 16:27:26 +01:00
alesapin
6f059be748 Fix bug with temporary projection parts deletion 2022-11-08 15:37:03 +01:00
serxa
2de26daa56 fix build 2022-11-08 14:31:29 +00:00
SmitaRKulkarni
658036d179
Merge pull request #42708 from ClickHouse/42414_Enable_functions_in_readonly_mode 
42414 enable functions in readonly mode
 2022-11-08 15:22:31 +01:00
Alexander Tokmakov
eb19df0094 fix 2022-11-08 15:21:25 +01:00
Sergei Trifonov
8eedd1e046
Merge branch 'master' into disk-s3-throttler 2022-11-08 15:00:56 +01:00
SmitaRKulkarni
4f2e5596e3
Merge pull request #42933 from yakkomajuri/patch-2 
fix(typo): Passible -> Possible
 2022-11-08 14:54:50 +01:00
serxa
6d5d9ff421 rename ReadWriteSettings -> RequestSettings 2022-11-08 13:48:23 +00:00
Maksim Kita
0aa4fd3c2d Fixed tests 2022-11-08 14:29:17 +01:00
Kseniia Sumarokova
d2b777c059
Merge pull request #41054 from rubin-do/hudi 
Add Hudi and DeltaLake table engines
 2022-11-08 12:03:06 +01:00
Kseniia Sumarokova
92b5ca326c
Merge pull request #42303 from xiedeyantu/system_table_add_column 
system.detached_parts add column bytes_on_disk and path
 2022-11-08 12:00:56 +01:00
Robert Schulze
2c568df714
Merge pull request #42968 from ClickHouse/bitcast 
Less use of CH-specific bit_cast()
 2022-11-08 10:54:10 +01:00
Robert Schulze
bb507356ef
Merge pull request #42209 from canhld94/ch_canh_fix_prefix_not_like 
Add `notLike` to key condition atom map
 2022-11-08 10:07:21 +01:00
Antonio Andelic
7067f9e0e1
Merge pull request #42947 from azat/mergetree/part-load-tiny 
Attach thread pool for loading parts to the query
 2022-11-08 09:17:32 +01:00
zhongyuankai
37099e57f4
Merge branch 'master' into optimze_ttl_merge 2022-11-08 12:26:46 +08:00
Azat Khuzhin
2bd4a71150
Merge branch 'master' into fetch-remove-covered 2022-11-08 00:55:00 +01:00
Kseniia Sumarokova
4fc7926c16
Merge branch 'master' into system_table_add_column 2022-11-07 23:36:57 +01:00
Kseniia Sumarokova
d695240d53
Update StorageHudi.cpp 2022-11-07 23:07:00 +01:00
serxa
ea17b30358 Merge branch 'storage-system-moves' of github.com:ClickHouse/ClickHouse into storage-system-moves 2022-11-07 20:33:31 +00:00
serxa
59408ebf57 fill more fields 2022-11-07 20:33:25 +00:00
Sergei Trifonov
6d3b96ee42
Merge branch 'master' into storage-system-moves 2022-11-07 21:31:27 +01:00
alesapin
257e766e0e Trying to fix two flaky tests 2022-11-07 20:49:41 +01:00
Alexander Tokmakov
9210e586d2 fix 2022-11-07 20:27:18 +01:00
Alexander Tokmakov
b8174a63a8 Merge branch 'master' into fix_intersecting_parts2 2022-11-07 20:26:36 +01:00
serxa
2daec0b45e S3 request per second rate throttling + refactoring 2022-11-07 18:05:40 +00:00
xiedeyantu
d4cdc38b71 better 2022-11-07 22:49:58 +08:00
Kseniia Sumarokova
891e167509
Update StorageHudi.cpp 2022-11-07 15:17:16 +01:00
Kruglov Pavel
48163ff9c2
Merge pull request #42938 from amosbird/issue_42771 
Fix NOT_FOUND_COLUMN_IN_BLOCK in some rare case
 2022-11-07 13:10:12 +01:00
kssenii
897811371d Minor changes 2022-11-07 12:45:26 +01:00
kssenii
819be6bb32 Merge remote-tracking branch 'upstream/master' into hudi 2022-11-07 11:30:53 +01:00
Duc Canh Le
bd2bd7149d fix typo 2022-11-07 18:22:50 +08:00
SmitaRKulkarni
c563e124c2
Merge branch 'master' into 42414_Enable_functions_in_readonly_mode 2022-11-07 09:50:05 +01:00
Duc Canh Le
f7e65ebd72 make comments more clear 2022-11-07 16:25:36 +08:00
Robert Schulze
cdaf0becfe
Merge branch 'master' into bitcast 2022-11-07 09:24:52 +01:00
Robert Schulze
f3f50530d1
Merge branch 'master' into ch_canh_fix_prefix_not_like 2022-11-07 09:14:02 +01:00
Antonio Andelic
e08f94d0f8
Merge pull request #42659 from ClickHouse/optimize-entire-partition 
Support for optimizing old parts for entire partition only
 2022-11-07 09:06:11 +01:00
Duc Canh Le
69d6d42443 NOT LIKE only work for 'prefix%' 2022-11-06 15:26:19 +08:00
zhongyuankai
ef6065b331 fix test 2022-11-06 13:40:26 +08:00
Duc Canh Le
a8bce28c04 Merge branch 'master' into ch_canh_fix_prefix_not_like 2022-11-06 13:04:08 +08:00
Igor Nikonov
e94b9cd003 Fixes 
+ enable retries in integration test
+ typo in fault injection traces
 2022-11-05 22:28:32 +00:00
Igor Nikonov
859022660e Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry 2022-11-04 23:31:51 +00:00
Daniil Rubin
7500cc38c0 Apply clang-format 2022-11-04 20:51:44 +00:00
Daniil Rubin
37048e986a Add comments, fix review issues 2022-11-04 20:49:24 +00:00
Daniil Rubin
d6bbd65dde Fix broken include 2022-11-04 16:03:12 +00:00
Robert Schulze
9c066e964d
Less use of CH-specific bit_cast() 
Converted usage of CH-custom bit_cast to std::bit_cast if possible, i.e.
when
  sizeof(From) == sizeof(To).
(The CH-custom bit_cast is able to deal with sizeof(From) != sizeof(To).)

Motivation for this came from #42847 where it is not clear how the
internal bit_cast should behave on big endian systems, so we better
avoid that situation as much as possible.
 2022-11-04 15:52:48 +00:00
Vladimir C
f359411668
Merge branch 'master' into optimze_ttl_merge 2022-11-04 16:52:20 +01:00
Vitaly Baranov
52b1f4aed9
Merge pull request #42484 from vitlibar/mask-sensitive-info-in-logs 
Mask some information in logs
 2022-11-04 14:09:38 +01:00
zhongyuankai
1345f7e73d Merge branch 'master' into optimze_ttl_merge 2022-11-04 20:54:46 +08:00
alesapin
82729a88c8
Merge pull request #42833 from vitlibar/backup-s3-increase-upload-part-size-exponentially 
Increase the size of upload part exponentially for backup to S3.
 2022-11-04 13:41:10 +01:00
Kseniia Sumarokova
ed44b20694
Merge pull request #42224 from kssenii/fit-http-buffer-retries 
Fix http buffer retries
 2022-11-04 11:50:17 +01:00
Amos Bird
d901ead1bc
Done 2022-11-04 17:26:44 +08:00
zhongyuankai
049a593272 Merge branch 'master' into optimze_ttl_merge 2022-11-04 17:22:38 +08:00
zhongyuankai
e773eb2fd1 batter 2022-11-04 17:19:34 +08:00
Azat Khuzhin
7032b2b98a Attach thread pool for loading parts to the query 
Detach should be done via CurrentThread::defaultThreadDeleter()

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-04 09:06:12 +01:00
Azat Khuzhin
cdd09fd72a Improve messages for loading data parts (add part type) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-04 09:05:06 +01:00
Daniil Rubin
99d57325bb Merge branch 'master' into hudi 2022-11-03 19:29:23 +00:00
Daniil Rubin
a573d8aef7 Apply clang-format 2022-11-03 18:17:39 +00:00
Daniil Rubin
7b57e94d9f Fix StorageHudi 2022-11-03 18:14:10 +00:00
Daniil Rubin
a9f8948c8d Fix build 2022-11-03 17:28:41 +00:00
Amos Bird
7186898ffa
Experiment 2022-11-04 00:29:17 +08:00
Yakko Majuri
e68a3b0c7a
fix(typo): Passible -> Possible 2022-11-03 11:25:07 -03:00
kssenii
4fe4a07600 Add test 2022-11-03 12:14:08 +01:00
Antonio Andelic
66ffdda973
Merge branch 'master' into optimize-entire-partition 2022-11-03 09:52:13 +01:00
Alexander Tokmakov
e67d05604c Merge branch 'master' into fix_intersecting_parts2 2022-11-02 19:30:27 +01:00
Alexander Tokmakov
d8b3a2af84 fix race between INSERT and DROP 2022-11-02 19:26:50 +01:00
Vitaly Baranov
32194c1200 Add max limitation for the size of an uploaded part. 2022-11-02 17:53:54 +01:00
Igor Nikonov
9a315b7899 Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry 2022-11-02 15:32:54 +00:00
Vitaly Baranov
e0133688bc
Merge branch 'master' into mask-sensitive-info-in-logs 2022-11-02 16:26:13 +01:00
Alexander Tokmakov
00c9e50ee3 fix race between INSERT and ALTER PARTITION 2022-11-02 16:25:26 +01:00
Alexander Tokmakov
acbad99b24 remove abandonable_lock part 2 2022-11-02 14:28:49 +01:00
Antonio Andelic
c4b717b343
Update src/Storages/MergeTree/MergeTreeDataMergerMutator.cpp 
Co-authored-by: Sergei Trifonov <sergei@clickhouse.com>
 2022-11-02 08:24:54 +01:00
Duc Canh Le
ec382d7903 add comment 2022-11-02 09:50:41 +08:00
Duc Canh Le
c6598dc66d
Merge branch 'master' into ch_canh_fix_prefix_not_like 2022-11-02 09:42:01 +08:00
Alexander Tokmakov
4f50a999bb remove abandonable_lock part 1 2022-11-01 21:35:25 +01:00
Nikolai Kochetov
478d3079c8 Merge branch 'master' into read-in-order-from-query-plan 2022-11-01 21:02:22 +01:00
Nikolai Kochetov
9ffebf4a0c Fixing read-in-order for special storages. 2022-11-01 19:51:52 +00:00
Igor Nikonov
e76c3c381f Better handling tables in read-only mode during insert 
+ throw immediately if table has no metadata in zk
+ stop retries in case of shutdown
+ check if table is readonly at the begining of every retry
 2022-11-01 18:47:51 +00:00
Anton Popov
88674ef964
Merge pull request #42794 from CurtizJ/better-code-in-check-part 
Slightly better code with projections
 2022-11-01 16:05:27 +01:00
zhongyuankai
c1e2b2af74 Optimize TTL merge, completely expired parts can be removed in time 2022-11-01 22:58:15 +08:00
Anton Popov
876dca48da
Merge pull request #36969 from CurtizJ/dynamic-columns-14 
Support `Object` type inside other types
 2022-11-01 15:20:02 +01:00
Duc Canh Le
07d9720ed5
Merge branch 'master' into ch_canh_fix_prefix_not_like 2022-11-01 22:16:09 +08:00
Kruglov Pavel
21d50f76ea
Merge pull request #41979 from Avogar/s3-cluster-schema-inference 
Fix schema inference in s3Cluster and improve in hdfsCluster
 2022-11-01 14:00:21 +01:00
Vitaly Baranov
b9f2f17331 Add test and logging. 2022-11-01 12:23:20 +01:00
Maksim Kita
fe48a1ce7e
Merge pull request #42681 from kitaisreal/merge-tree-indexes-use-actions-dag 
MergeTree indexes use ActionsDAG
 2022-11-01 13:55:17 +03:00
Duc Canh Le
3545db0a6c
Merge branch 'master' into ch_canh_fix_prefix_not_like 2022-11-01 16:54:26 +08:00
Yakov Olkhovskiy
bab0e06e3d
Merge pull request #42458 from Clark0/match-func-use-index 
match function can use index if it's a condition on string prefix #37333
 2022-10-31 17:33:58 -04:00
Igor Nikonov
9cc64a0fcc Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry 2022-10-31 18:21:21 +00:00
Igor Nikonov
2d5050ea66 Fix review comments 2022-10-31 18:20:58 +00:00
Anton Popov
2ae3cfa9e0
Merge branch 'master' into dynamic-columns-14 2022-10-31 16:15:19 +01:00
Alexander Tokmakov
599ccb9939
Update MergeTreeData.h 2022-10-31 16:18:17 +03:00
Alexander Tokmakov
b2b9479afa
Update src/Storages/MergeTree/MergeTreeData.h 
Co-authored-by: Sergei Trifonov <sergei@clickhouse.com>
 2022-10-31 16:15:35 +03:00
SmitaRKulkarni
54035270b2
Merge branch 'master' into 42414_Enable_functions_in_readonly_mode 2022-10-31 14:15:29 +01:00
Maksim Kita
bca22ec5f5 Fixed code review issues 2022-10-31 12:30:00 +01:00
Maksim Kita
115fcaffc5 Fixed tests 2022-10-31 11:30:56 +01:00
Vitaly Baranov
5d2a222fe4 Mask sensitive information in logs. 2022-10-31 10:50:33 +01:00
Duc Canh Le
d309e8793a
Merge branch 'master' into ch_canh_fix_prefix_not_like 2022-10-31 15:13:39 +08:00
Alexander Tokmakov
f53df7870c fix race between drop and failed insert 2022-10-30 17:59:47 +01:00
Alexander Tokmakov
c68ab231f9 fix accessing part in Deleting state 2022-10-30 17:30:51 +01:00
FArthur-cmd
607b81989d fix build 2022-10-30 14:57:10 +00:00
Anton Popov
8de75bf29c better code with projections 2022-10-29 14:39:59 +00:00
clarkcaoliu
ca5dbe88db match function can use index if prefix 2022-10-29 18:25:29 +08:00
Filatenkov Artur
21c024b043
Merge branch 'master' into annoy_cosine 2022-10-28 20:15:53 +03:00
FArthur-cmd
f187d4e1d4 add cosine distance for annoy and fix docs for cosineDistance 2022-10-28 17:03:35 +00:00
Igor Nikonov
d09deaf10f Fix style check 2022-10-28 15:17:05 +00:00
Igor Nikonov
937dd44399 Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry 2022-10-28 15:09:23 +00:00
Igor Nikonov
c53b96a11c Add more comments 2022-10-28 15:09:01 +00:00
Alexey Milovidov
f64e632707
Merge pull request #42469 from amosbird/issue_42456 
Fix misbehavior of key analysis
 2022-10-28 16:18:00 +02:00
Antonio Andelic
5fae8b0cc9 Merge branch 'master' into optimize-entire-partition 2022-10-28 11:44:10 +00:00
serxa
acc00218d9 fix build 2022-10-28 11:23:49 +00:00
Sergei Trifonov
836595c7ae
Merge branch 'master' into storage-system-moves 2022-10-28 11:39:38 +02:00
Duc Canh Le
848f5b5d65
Merge branch 'master' into ch_canh_fix_prefix_not_like 2022-10-28 13:40:17 +08:00
Duc Canh Le
c080964391 correct 'notLike' in key condition 2022-10-28 13:39:01 +08:00
Duc Canh Le
69311bd5cd Merge branch 'master' into ch_canh_fix_prefix_not_like 2022-10-28 11:59:50 +08:00
Igor Nikonov
3682913172 Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry 2022-10-27 22:01:53 +00:00
Igor Nikonov
a601c166ad Add clean up for faults injected before zk request 
+ for multi-op request, it create responses in multi response
+ some polishing
 2022-10-27 21:35:13 +00:00
Anton Popov
7715afa595
Merge pull request #42618 from CurtizJ/refactor-data-part 
Try to save `IDataPartStorage` interface
 2022-10-27 22:20:44 +02:00
Igor Nikonov
f773436de5 Fix flaky test 2022-10-27 13:34:07 +00:00
Nikolai Kochetov
24e1aaf324
Merge pull request #42576 from ClickHouse/table-function-merge-invalid-where-type 
Invalid type in where for Merge table (logical error)
 2022-10-27 13:03:27 +02:00
Antonio Andelic
10aa7091ca
Merge pull request #42605 from ClickHouse/correctly-handle-prewhere-type 
Correctly handle non-native integers in PREWHERE
 2022-10-27 09:25:38 +02:00
SmitaRKulkarni
e860ecc80c
Merge branch 'master' into 42414_Enable_functions_in_readonly_mode 2022-10-27 08:26:44 +02:00
Nikolai Kochetov
313cbb1058 Fix test. 2022-10-26 17:49:36 +00:00
Igor Nikonov
649e8d302c Sync with retiring branch igor/insert_zk_retries 2022-10-26 17:41:44 +00:00
Igor Nikonov
8435cba352 Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry 2022-10-26 17:35:47 +00:00
Maksim Kita
f04f663818 Fixed tests 2022-10-26 18:55:14 +02:00
Smita Kulkarni
6a8fa2d4a5 Added new field allow_readonly in system.table_functions to allow using table functions in readonly mode. 
Implementation:
* Added a new field allow_readonly to table system.table_functions.
* Updated to use new field allow_readonly to allow using table functions in readonly mode.
* Added TableFunctionProperties similar to AggregateFunctionProperties.
* The functions allowed in readonly mode are in this set table_functions_allowed_in_readonly_mode.
Testing:
* Added a test for filesystem tests/queries/0_stateless/02473_functions_in_readonly_mode.sh
Documentation:
* Updated the english documentation for Table Functions.
 2022-10-26 18:45:23 +02:00
Nikolai Kochetov
1520bcd53c Fix error code. 2022-10-26 15:50:10 +00:00
Anton Popov
ca4df62e37
Merge branch 'master' into refactor-data-part 2022-10-26 17:49:20 +02:00
Mikhail f. Shiryaev
fdecac65b8
Merge pull request #42699 from ClickHouse/22.11-prepare 
Update version after release
 2022-10-26 16:31:48 +02:00
Maksim Kita
1c17e9d454 Fixed tests 2022-10-26 15:54:37 +02:00
Mikhail f. Shiryaev
d3461abd1f
Update version to 22.11.1.1 2022-10-26 15:27:09 +02:00
Nikolai Kochetov
db422fa903
Merge pull request #42410 from ClickHouse/fix-multi-arg-monotonicity 
Fix possible LOGICAL_ERROR in binary ariphmetics monotonicity.
 2022-10-26 14:11:23 +02:00
Kruglov Pavel
219553df8d
Merge pull request #42580 from hanfei1991/fei/limit_max_rows 
fix behaviour of max_rows_to_read for trival limit queries
 2022-10-26 14:09:59 +02:00
Maksim Kita
c47ca522f4 Fixed style check 2022-10-26 13:27:14 +02:00
Antonio Andelic
35a9672704 Merge branch 'master' into optimize-entire-partition 2022-10-26 11:09:25 +00:00
Maksim Kita
1b6293f6db MergeTree indexes use ActionsDAG 2022-10-26 12:44:37 +02:00
Antonio Andelic
97aaebfa18 Address PR comments 2022-10-26 10:06:56 +00:00
SmitaRKulkarni
5df138da82
Merge pull request #42376 from ClickHouse/36316_Support_glob_for_recursive_directory_traversal 
Added ** glob support for recursive directory traversal to filesystem…
 2022-10-26 11:49:20 +02:00
Antonio Andelic
979d180931
Merge branch 'master' into correctly-handle-prewhere-type 2022-10-26 11:13:16 +02:00
Azat Khuzhin
8db31beee6 Remove covered parts for fetched part 
Here is an example that I found on production, simplified.

Consider the following queue (nothing of this had been processed on this
replica):

- GET_PART all_0_0_0 (queue-0000000001)
- GET_PART all_1_1_0 (queue-0000000002)
...
- GET_PART all_0_1_1 (queue-0000000003)
- GET_PART all_2_2_0 (queue-0000000004)
...
- MERGE_PARTS from [all_0_1_1, all_2_2_0] to all_0_2_2 (queue-0000000005)

And now queue-0000000005 started to executing (either because
of reording, or because at that time GET_PART fails), and it
does not have any required parts, so it will fetch them, but
not all_0_0_0 and all_1_1_0, so this replica delay will set to
the time of min(queue-0000000001, queue-0000000002), while it
is not true, since it already have parts that covers those
parts.

and since MERGE_PARTS takes 30min, it increased the replica delay
eventually to 30min, for the time range of 30min, which is pretty huge.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-26 11:10:50 +03:00
Anton Popov
79889198b1 fix style check 2022-10-25 23:03:37 +00:00
Anton Popov
d8b09b430f add comments 2022-10-25 22:27:11 +00:00
Anton Popov
c4d4f2dbed better interface 2022-10-25 22:14:06 +00:00
Sergei Trifonov
34b43f8b4b
Merge branch 'master' into storage-system-moves 2022-10-25 22:56:45 +02:00
Nikolai Kochetov
bfb5e6eae4 Fix logical error for invalid prewhere type in StorageMerge. 2022-10-25 13:58:29 +00:00
serxa
55a86bc2de Add active moves system table 2022-10-25 13:56:46 +00:00
Antonio Andelic
611c2e2bd7 Support for optimizing old parts for entire partition only 2022-10-25 13:40:18 +00:00
Anton Popov
c8199bc125
Merge branch 'master' into refactor-data-part 2022-10-25 14:31:05 +02:00
Raúl Marín
0016bc2b73
Improve the time to recover keeper connections (#42541) 
Co-authored-by: Nikita Mikhaylov <nikitamikhaylov@clickhouse.com>
Co-authored-by: robot-clickhouse <robot-clickhouse@clickhouse.com>
Co-authored-by: Alexey Milovidov <milovidov@clickhouse.com>
 2022-10-25 13:26:29 +02:00
Antonio Andelic
af0c54e055 Small fix 2022-10-25 11:13:41 +00:00
Antonio Andelic
451a50d9e9 Merge branch 'master' into correctly-handle-prewhere-type 2022-10-25 10:48:35 +00:00
Maksim Kita
06fe6f3c8b
Merge pull request #31796 from kitaisreal/identifier-resolver 
Added Analyzer
 2022-10-25 12:36:08 +03:00
Anton Popov
cbfa887e52 fix merge of projections 2022-10-24 23:51:57 +00:00
Anton Popov
f865b28da4 fix drop of MergeTree tables 2022-10-24 22:38:53 +00:00
Alexey Milovidov
9ee7131f67
Merge pull request #42423 from ClickHouse/optimize-partitions-in-background 
Optimize partitions in background
 2022-10-24 19:41:50 +02:00
SmitaRKulkarni
96c8260230
Merge branch 'master' into 36316_Support_glob_for_recursive_directory_traversal 2022-10-24 18:34:19 +02:00
Anton Popov
dcb7667808 share transaction for projections 2022-10-24 16:28:19 +00:00
Anton Popov
128fdd785e better interface 2022-10-24 14:44:22 +00:00
Antonio Andelic
5579d139eb Avoid crash with big int in prewhere 2022-10-24 08:37:52 +00:00
Maksim Kita
d083200d65 Fixed code review issues 2022-10-24 10:22:38 +02:00
Maksim Kita
10438aea01 Added JOIN support for different algorithm 2022-10-24 10:22:21 +02:00
Maksim Kita
890d9678b9 Fixed tests 2022-10-24 10:22:20 +02:00
Maksim Kita
ca93ee7479 Fixed tests 2022-10-24 10:22:20 +02:00
Maksim Kita
fed146e198 Added SAMPLE BY support. Added SAMPLE BY, FINAL support for JOINS. 2022-10-24 10:22:20 +02:00
Igor Nikonov
54013009d3 Code migration from #41734 to align diff and git blame. 
Initial implementation was different and it changed the entire
ReplicatedMergeTreeSink::commitPart() which change history provided by git blame.
Then RetriesControl.retryLoop() was introduced later which significantly reduces
the diff since it's like while() used before.
So, check outing the current version will keep more original history in
git blame, which is useful here
 2022-10-24 08:20:11 +00:00
xiedeyantu
61563d4f19 better 2022-10-24 11:52:20 +08:00
Anton Popov
cf375c9732 better interface 2022-10-23 22:29:24 +00:00
Han Fei
4a8326ff0c fix typo 2022-10-23 23:22:52 +02:00
Anton Popov
b40d9200d2 better semantic of constsness of DataPartStorage 2022-10-23 15:24:20 +00:00
Igor Nikonov
c0d1be9cab
Merge pull request #42560 from ClickHouse/igor/replicated_merge_tree_some_polishing 
Some polishing: replicated merge tree
 2022-10-23 15:15:24 +02:00
Anton Popov
56e5daba0c remove DataPartStorageBuilder 2022-10-23 00:23:15 +00:00
Igor Nikonov
730a3ba849
Merge branch 'master' into igor/replicated_merge_tree_some_polishing 2022-10-22 19:54:09 +02:00
Han Fei
2fc91fd338 fix behaviour of max_rows_to_read for trival limit queries 2022-10-22 18:27:24 +02:00
Amos Bird
f104dd08cd
Only fix LowCardinality for now 2022-10-22 15:47:13 +08:00
Azat Khuzhin
905a95e166 Review fixes 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-21 22:40:13 +02:00
Igor Nikonov
851f556d65 Remove unused parameter 2022-10-21 14:35:37 +02:00
Igor Nikonov
13d31eefbc Replicated merge tree polishing 
+ make read-only members const
+ reduce memory allocations
 2022-10-21 14:02:52 +02:00
Azat Khuzhin
784f7d988c Fix -Wshorten-64-to-32 in examples 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-21 13:25:43 +02:00
Azat Khuzhin
4e76629aaf Fixes for -Wshorten-64-to-32 
- lots of static_cast
- add safe_cast
- types adjustments
  - config
  - IStorage::read/watch
  - ...
- some TODO's (to convert types in future)

P.S. That was quite a journey...

v2: fixes after rebase
v3: fix conflicts after #42308 merged
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-21 13:25:19 +02:00
Antonio Andelic
3091d609fa
Merge branch 'master' into optimize-partitions-in-background 2022-10-21 11:01:41 +02:00
Alexander Tokmakov
0d8a814d80
Merge pull request #42254 from xinhuitian/optimize-selectPartsToMutate 
Early return of corner cases in selectPartsToMutate function
 2022-10-20 18:13:37 +03:00
Alexander Tokmakov
5b03bbce2e
Merge pull request #41981 from ClickHouse/fixes_for_replicated_merge_tree2 
Some fixes for ReplicatedMergeTree
 2022-10-20 16:57:51 +03:00
alesapin
24d02b5516
Merge pull request #42441 from ClickHouse/check_readonly_in_background_tasks 
Deactivate tasks in ReplicatedMergeTree until startup
 2022-10-20 15:36:20 +02:00
Robert Schulze
469d6ad624
Merge pull request #42485 from ClickHouse/fix-bad_cast-in-annoy 
Fix bad_cast in Annoy index
 2022-10-20 15:27:58 +02:00
Alexander Tokmakov
d668a82829
Merge branch 'master' into improve_replica_recovery 2022-10-20 14:25:09 +03:00
Amos Bird
2499ab3b3a
Use convertFieldToType 2022-10-20 17:53:52 +08:00
Antonio Andelic
9d7e44409a
Merge pull request #41342 from ClickHouse/keeper-upload-snapshot-to-s3 
Upload Keeper snapshots to S3
 2022-10-20 09:14:59 +02:00
Antonio Andelic
77eb353839
Merge pull request #42243 from ClickHouse/use-multiread-more 
Use MultiRead where possible
 2022-10-20 09:04:54 +02:00
Kruglov Pavel
df96acc80f
Fix style 2022-10-19 21:48:45 +02:00
Kruglov Pavel
ae3fa70594
Fix style 2022-10-19 21:48:20 +02:00
Kruglov Pavel
830ab3a5a9
Fix style 2022-10-19 21:48:01 +02:00
Nikolai Kochetov
d85bfe6ea3
Merge pull request #42322 from ClickHouse/additional-filters-and-distributed 
Fix additional_table_filters for Distributed.
 2022-10-19 17:09:51 +02:00
Alexander Tokmakov
84ef32e33f Merge branch 'master' into improve_replica_recovery 2022-10-19 16:28:01 +02:00
Alexander Tokmakov
f958c155f3
Merge branch 'master' into fixes_for_replicated_merge_tree2 2022-10-19 17:25:24 +03:00
Robert Schulze
42f680849c
Fix bad_cast in Annoy index 
- Problem originally found by data type fuzzer
  https://s3.amazonaws.com/clickhouse-test-reports/42180/2f83d8790581dce0ffeec56c137b1d13160cfa7b/fuzzer_astfuzzermsan//report.html

- This commit restricts which data types are allowed for Annoy indexes
  (similar things are done for other index types).
 2022-10-19 13:40:39 +00:00
Antonio Andelic
05e6b94b54 Polishing 2022-10-19 12:59:28 +00:00
Antonio Andelic
e5408aac99 Simplify logic 2022-10-19 12:34:20 +00:00
xiedeyantu
330777a0cc fix 2022-10-19 17:44:36 +08:00
xiedeyantu
32ad28cbe5 optimize code 2022-10-19 17:37:26 +08:00
Amos Bird
549597fe80
Fix misbehavior of key analysis. 
When key types cannot be inside Nullable, it can break key analysis. See
https://github.com/ClickHouse/ClickHouse/issues/42456.
 2022-10-19 12:15:27 +08:00
Alexey Milovidov
46917c023c
Update AlterCommands.cpp 2022-10-19 00:25:21 +03:00
Alexander Tokmakov
1d9d753167 fix 2022-10-18 18:13:18 +02:00
Nikolai Kochetov
216d671cdf
Update AlterCommands.cpp 2022-10-18 17:56:34 +02:00
Alexander Tokmakov
ea77ff9022 Merge branch 'master' into fixes_for_replicated_merge_tree2 2022-10-18 17:25:25 +02:00
Kruglov Pavel
ba3e213a83
Remove outdated comment 2022-10-18 17:03:04 +02:00
Nikolai Kochetov
b109767dda Merge branch 'master' into fix-multi-arg-monotonicity 2022-10-18 14:47:33 +00:00
Alexey Milovidov
7befe2825c
Update StorageReplicatedMergeTree.cpp 2022-10-18 17:36:11 +03:00
Alexey Milovidov
0623ad4e37
Update StorageReplicatedMergeTree.cpp 2022-10-18 17:35:44 +03:00
Nikolai Kochetov
a5711fda44 Revert "Revert #27787" 
This reverts commit c1cc04d44d.
 2022-10-18 14:00:50 +00:00
alesapin
9769057767 Deactivate all tasks 2022-10-18 15:53:52 +02:00
alesapin
ab8da5a539 Deactivate mutationsFinalizingTask until startup 2022-10-18 15:50:02 +02:00
Kruglov Pavel
25e13bdd2f
Merge pull request #41107 from Avogar/improve-combinators 
Support all combinators combination in WindowTransform/arratReduce*/initializeAggregation/aggregate functions versioning
 2022-10-18 15:24:49 +02:00
avogar
3bb1161859 Better naming 2022-10-18 13:05:40 +00:00
avogar
2c2f977096 Make better 2022-10-18 13:03:55 +00:00
Antonio Andelic
d105796c91 Merge branch 'master' into optimize-partitions-in-background 2022-10-18 12:24:03 +00:00
Antonio Andelic
e828dce331 Disable auto optimize by default 2022-10-18 12:11:23 +00:00
avogar
75c4ef0be7 Merge branch 'master' of github.com:ClickHouse/ClickHouse into s3-cluster-schema-inference 2022-10-18 11:58:12 +00:00
Tian Xinhui
45c6974f7d
Use alias for std::shared_ptr<MergeMutateSelectedEntry> (#42211) 
* use alias for MergeMutateSelectedEntry share ptr

* fix StorageMergeTree.cpp
 2022-10-18 13:35:34 +02:00
alesapin
673adfbd5c
Merge pull request #42317 from ClickHouse/fix_another_trash_with_zero_cop 
Fix another trash in zero-copy replication
 2022-10-18 12:51:16 +02:00
Antonio Andelic
06b2eeb5a7 Small changes to settings 2022-10-18 09:31:16 +00:00
Antonio Andelic
bd2522fdd0 Merge branch 'master' into optimize-partitions-in-background 2022-10-18 09:13:59 +00:00
Antonio Andelic
ee2f5977d5
Merge branch 'master' into keeper-upload-snapshot-to-s3 2022-10-18 10:02:50 +02:00
Ilya Yatsishin
36b63badb3
Merge pull request #42223 from qoega/hdfs-empty-url 2022-10-18 09:34:10 +02:00
Nikolai Kochetov
3d1ed2969a Fix possible LOGICAL_ERROR in binary ariphmetics monotonicity. Fix invalid evaluation of binary monotonic function in KeyCondition. 2022-10-17 18:21:03 +00:00
Alexander Tokmakov
57f6e27300 Merge branch 'master' into improve_replica_recovery 2022-10-17 18:37:31 +02:00
xiedeyantu
b26361c647 fix test 2022-10-17 23:02:39 +08:00
xiedeyantu
9e62570d61 fix test 2022-10-17 22:57:11 +08:00
Duc Canh Le
9af817bb43
Fix read from buffer with read in order (#42236) 2022-10-17 14:25:31 +02:00
Antonio Andelic
7c7395575f Merge branch 'master' into keeper-upload-snapshot-to-s3 2022-10-17 11:02:36 +00:00
Ilya Yatsishin
dd0fc2435e Merge remote-tracking branch 'origin/master' into hdfs-empty-url 2022-10-17 12:48:42 +02:00
Nikolai Kochetov
c9aa6a655c
Merge pull request #42319 from ClickHouse/fix-alter-update-wrong-type 
Fix an invalid type of a column after attach and alter.
 2022-10-17 12:44:16 +02:00
alesapin
051e3a0db5 Fix if 2022-10-17 12:27:05 +02:00
alesapin
cf6442fa65
Merge pull request #42222 from ClickHouse/disable_concurrent_parts_removal 
Disable concurrent parts removal
 2022-10-17 12:17:55 +02:00
Antonio Andelic
125501b36a Merge branch 'master' into use-multiread-more 2022-10-17 07:32:30 +00:00
Antonio Andelic
f3c4d0eff2 Fix keeperMap test 2022-10-17 07:29:22 +00:00
Smita Kulkarni
91433e5b9c Added ** glob support for recursive directory traversal to filesystem and S3. 
Implementation:
* Updated parseGlob to not add ‘/‘ restriction when ** is used.
* Updated S3 & filesystem to fetch files and not use regex match if glob is **.
Testing:
* Added a test for filesystem tests/queries/0_stateless/02459_glob_for_recursive_directory_traversal.sh
 2022-10-17 09:04:25 +02:00
xiedeyantu
9ec206f8f7 fix test 2022-10-16 19:52:53 +08:00
xiedeyantu
2a356715da fix test 2022-10-16 16:46:00 +08:00
chen
0e86976e2b
Merge branch 'master' into system_table_add_column 2022-10-16 13:17:45 +08:00
Ilya Yatsishin
165785f716 mErge remote-tracking branch 'origin/master' into hdfs-empty-url 2022-10-15 19:13:59 +02:00
Vitaly Baranov
91c438eebe
Merge pull request #42263 from vitlibar/refactor-udf 
Refactor the implementation of user-defined functions
 2022-10-15 12:33:10 +02:00
Kruglov Pavel
3d9f46a1e7
Merge branch 'master' into s3-cluster-schema-inference 2022-10-14 22:07:54 +02:00
avogar
52427e6028 Remove code duplication 2022-10-14 18:07:02 +00:00
Nikolai Kochetov
b98b1759c8 Fix additional_table_filters for Distributed. 2022-10-14 17:04:42 +00:00
Kruglov Pavel
ff11904850
Merge branch 'master' into improve-combinators 2022-10-14 17:19:31 +02:00
Nikolai Kochetov
b18c6fd8e6 Fix an invalid type of a column after attach and alter. 2022-10-14 14:52:26 +00:00
alesapin
e0c2518416 Fix another trash in zero-copy replication 2022-10-14 16:50:48 +02:00
Kseniia Sumarokova
555d275920
Update src/Storages/MergeTree/DataPartStorageOnDisk.h 2022-10-14 15:18:53 +02:00
Ilya Yatsishin
3361b27965 less 2022-10-14 14:27:52 +02:00
alesapin
fd718c0d00 Merge branch 'master' into disable_concurrent_parts_removal 2022-10-14 13:32:11 +02:00