thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-15 02:41:59 +00:00
Code Issues Packages Projects Releases Wiki Activity
103,893 Commits 9,328 Branches 1,783 Tags 2.1 GiB
7df137e460
Commit Graph

10614 Commits

Author SHA1 Message Date
Sema Checherinda
a29437e179 delete verbose logging about cleaning old parts 2022-11-23 15:16:09 +00:00
Sema Checherinda
e54ca534b0 test 01167_isolation_hermitage uses fallback to recreate table 2022-11-23 15:16:09 +00:00
Sema Checherinda
f49936b9a0 remove tmp dir from inmemory part when freeze, more logs about old parts 2022-11-23 15:16:09 +00:00
Sema Checherinda
3a8024002d fix race in read full path from deleting part, add more logs in grabOldParts 2022-11-23 15:16:09 +00:00
Sema Checherinda
4f592bc208 fix isolation breach, droping outdated part should't be ignored 2022-11-23 15:16:09 +00:00
Sema Checherinda
6b4cb25e25 fixed 01130_in_memory_parts_partitons, remove flushed tmp part from in memory part 2022-11-23 15:16:09 +00:00
Sema Checherinda
c976b28104 less pedantic checks in operations, rely on existed tx functionality 2022-11-23 15:16:09 +00:00
Sema Checherinda
a4d1adf945 in order to restore partition do not write empty inmemory part 2022-11-23 15:16:09 +00:00
Sema Checherinda
00e7b4e195 fix race on remove, add creation csn for removed parts 2022-11-23 15:16:09 +00:00
Sema Checherinda
c1e6b560f0 no groupping parts for cover with empty 2022-11-23 15:16:08 +00:00
Sema Checherinda
597421014e fix style check 2022-11-23 15:16:08 +00:00
Sema Checherinda
db86cd0f0a fix several critical races, fix tests 2022-11-23 15:16:08 +00:00
Sema Checherinda
8977e499c2 address 01825_type_json_18 test 2022-11-23 15:16:08 +00:00
Sema Checherinda
f2b9f80f2f fix tidy build 2022-11-23 15:16:08 +00:00
Sema Checherinda
6d929266e0 address 01825_type_json_schema_race_long test 2022-11-23 15:16:08 +00:00
Sema Checherinda
15012c7070 address the review comments 2022-11-23 15:16:08 +00:00
Sema Checherinda
fb86019db9 Delete parts over replacing them with empty parts 2022-11-23 15:16:08 +00:00
Alexander Tokmakov
c0133199c8
Revert "Add table_uuid to system.parts" 2022-11-23 16:41:43 +03:00
Antonio Andelic
981e8a3a27 Modifiy visitor 2022-11-23 13:32:36 +00:00
Alexander Tokmakov
ad74961075
Merge branch 'master' into tavplubix-patch-5 2022-11-23 15:57:11 +03:00
Alexander Gololobov
1dec7910c3
Merge pull request #43336 from azat/buffer/fix-rollback 
Fix possible "No column to rollback" logical error during INSERT into Buffer
 2022-11-23 13:00:32 +01:00
kssenii
7ea1f4a3dd Merge remote-tracking branch 'origin/named-collections-sql-commands' into named-collections-sql-commands 2022-11-23 12:02:00 +01:00
kssenii
6044a9257c Merge remote-tracking branch 'upstream/master' into named-collections-sql-commands 2022-11-23 12:00:55 +01:00
Antonio Andelic
84884df595 Substitute UDFs in CREATE queries 2022-11-23 09:28:08 +00:00
Alexey Milovidov
c1283a6bb5
Update StorageSystemDataSkippingIndices.cpp 2022-11-22 22:17:20 +03:00
Alexey Milovidov
bedfe68503
Merge branch 'master' into nv/fix-mutate-fetch 2022-11-22 20:06:56 +01:00
Azat Khuzhin
2cfbc27f13 Fix incorrect memory accounting because of merges/mutations 
In case of merges/mutations, separate MemoryTracker object is used, to
track memory separately, to report this metrics.

However, the tail of those allocations hadn't been taken into account,
and in case of lots of mutations/merges it can be significant.

100% reproducible with the following example:

Start the server w/o frequent sync with RSS:

    $ clickhouse-server -C /src/programs/server/config.xml -- --asynchronous_metrics_update_period_s=86400

Run the following snippet:

    create table data (key Int, value Nullable(Decimal(38, 0))) engine=ReplicatedMergeTree('/ch/data', '1') order by key
    insert into data values (1, -13028236692093846346337460743176821145)
    alter table data modify column value UInt256
    Code: 341. DB::Exception: Received from localhost:9000. DB::Exception: Exception happened during execution of mutation 'mutation_2.txt' with part 'all_1_1_0' reason: 'Code: 407. DB::Exception: Convert overflow: while executing 'FUNCTION _CAST(value :: 1, 'UInt256' :: 2) -> _CAST(value, 'UInt256') UInt256 : 3': (while reading from part ./store/dd9/dd9ef183-1270-4877-af22-8a000bdcc1b8/all_1_1_0/): While executing MergeTreeInOrder. (DECIMAL_OVERFLOW) (version 22.12.1.1)'. This error maybe retryable or not. In case of unretryable error, mutation can be killed with KILL MUTATION query. (UNFINISHED)

And track the memory:

    select value/1e9 from system.metrics where metric like 'MemoryTracking'

Reported-by: @filimonov
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-22 19:40:35 +01:00
Azat Khuzhin
5da2f52722 Use Int64 over UInt64 for prev_untracked_memory* in MemoryTrackerThreadSwitcher 
Since those types originally is Int64

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-22 19:40:35 +01:00
Azat Khuzhin
05f30aaad4 Fix logger name for MutateFromLogEntryTask 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-22 19:40:35 +01:00
Alexey Milovidov
b7e4547f53
Merge branch 'master' into fix-typo-3 2022-11-22 18:30:06 +01:00
Han Fei
32c7fbec55 use another dedup path for async inserts 2022-11-22 15:12:00 +01:00
Sergei Trifonov
59f422f10c
Merge branch 'master' into storage-system-moves 2022-11-22 14:46:00 +01:00
Kruglov Pavel
98d6b96c82
Merge pull request #42033 from mark-polokhov/BSONEachRow 
Add BSONEachRow input/output format
 2022-11-22 14:45:21 +01:00
Kseniia Sumarokova
392963012a
Merge pull request #43404 from azat/part.log.uuid 
Add table_uuid to system.parts
 2022-11-22 14:25:34 +01:00
Alexander Tokmakov
91b2cdb94a
Merge branch 'master' into tavplubix-patch-5 2022-11-22 16:17:55 +03:00
Kseniia Sumarokova
ff630e3f7c
Update src/Storages/NamedCollectionUtils.cpp 
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
 2022-11-22 14:04:44 +01:00
Nikolai Kochetov
9e84a351c7 Fixing special build. Disable setting by default. 2022-11-22 11:24:24 +00:00
Antonio Andelic
843401c3a3
Merge pull request #43323 from ClickHouse/fix-describe-delta-lake-hudi 
Fix `DESCRIBE` for `deltaLake` and `hudi` table functions
 2022-11-22 09:09:39 +01:00
Anton Popov
0479d8d9f5
Merge pull request #43387 from ClickHouse/fix_default_value_used_in_row_level_filter 
Fix default value used in row level filter
 2022-11-22 01:46:45 +01:00
kssenii
1073626f8e Mask secrets in log 2022-11-21 22:45:43 +01:00
kssenii
155ddd5521 Fix review comments 2022-11-21 21:50:24 +01:00
kssenii
a279bac767 Merge remote-tracking branch 'upstream/master' into named-collections-sql-commands 2022-11-21 21:49:34 +01:00
Sergei Trifonov
94a7614c45
Merge branch 'master' into storage-system-moves 2022-11-21 20:10:12 +01:00
Alexander Tokmakov
a7efdd7f4e
Update StorageReplicatedMergeTree.cpp 2022-11-21 21:52:48 +03:00
Nikolai Kochetov
c305afd77a Fix race. 2022-11-21 18:26:31 +00:00
Kruglov Pavel
581e57be9f
Merge pull request #43236 from xiedeyantu/fix_hdfscluster_connection_fail 
fix skip_unavailable_shards does not work using hdfsCluster table function
 2022-11-21 19:22:16 +01:00
serxa
faef2e9b00 fix moves list entry creation 2022-11-21 17:08:54 +00:00
kssenii
5e01441f61 Show progress bar while reading from s3 table function 2022-11-21 17:56:02 +01:00
serxa
c425da06b2 fix column name 2022-11-21 16:16:53 +00:00
Alexey Milovidov
56f8c0b84b Fix typo 2022-11-21 15:10:15 +01:00
Azat Khuzhin
d0ce500f89 Add table_uuid to system.parts 
Can be useful if the table constantly recreated, i.e. in some tests.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-21 12:11:51 +01:00
alesapin
b78b602ea1
Merge branch 'master' into Enable_compress_marks_and_primary_key 2022-11-21 12:05:04 +01:00
Nikolai Kochetov
be1a8054c7
Merge branch 'master' into read-from-mt-in-io-pool 2022-11-21 11:27:41 +01:00
Nikolai Kochetov
4a1774c5ac Fixing build. 2022-11-21 10:27:09 +00:00
Antonio Andelic
9a8d82f0fc Merge branch 'master' into fix-describe-delta-lake-hudi 2022-11-21 08:23:33 +00:00
Antonio Andelic
e78761b66c
Merge branch 'master' into keeper-prometheus 2022-11-21 09:10:10 +01:00
Guillaume Tassery
7267473aec
Merge branch 'master' into gt/add_expression 2022-11-21 09:07:12 +01:00
Guillaume Tassery
1f73c8dabd Merge branch 'gt/add_expression' of github.com:YiuRULE/ClickHouse into gt/add_expression 2022-11-21 09:06:25 +01:00
Guillaume Tassery
702768469b cosmetic 2022-11-21 09:04:44 +01:00
Kseniia Sumarokova
c12cfab953
Merge pull request #43343 from azat/disks/write-once 
Allow to "drop tables" from s3_plain disk (so as from web disk)
 2022-11-20 16:04:50 +01:00
Azat Khuzhin
177cbbac4b Avoid race condition for updating system.distribution_queue values 
Previously it was possible to have a race while updating
files_count/bytes_count, since INSERT updates it those counters from one
thread and the same metrics are updated from filesystem in a separate
thread, and even though the access is synchronized with the mutex it
avoids the race only for accessing the variables not the logical race,
since it is possible that getFiles() from a separate thread will
increment counters and later addAndSchedule() will increment them again.

Here you can find an example of this race [1].

  [1]: https://pastila.nl/?00950e00/41a3c7bbb0a7e75bd3f2922c58b02334

Note, that I analyzed logs from production system with lots of async
Distributed INSERT and everything is OK there, even though the logs
contains the following:

    2022.11.20 02:21:15.459483 [ 11528 ] {} <Trace> v21.dist_out.DirectoryMonitor: Files set to 35 (was 34)
    2022.11.20 02:21:15.459515 [ 11528 ] {} <Trace> v21.dist_out.DirectoryMonitor: Bytes set to 4035418 (was 3929008)
    2022.11.20 02:21:15.819488 [ 11528 ] {} <Trace> v21.dist_out.DirectoryMonitor: Files set to 1 (was 2)
    2022.11.20 02:21:15.819502 [ 11528 ] {} <Trace> v21.dist_out.DirectoryMonitor: Bytes set to 190072 (was 296482)

As you may see it first increases the counters and next update
decreases (and 4035418-3929008 == 296482-190072)

Refs: #23885
Reported-by: @tavplubix
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-20 14:13:01 +01:00
Alexey Milovidov
81ca8e91e6
Update StorageSystemDataSkippingIndices.cpp 2022-11-20 14:27:52 +03:00
Azat Khuzhin
4f6703c972 Add is_read_only/is_write_once/is_remote/is_broken to system.disks 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-19 10:11:39 +01:00
Azat Khuzhin
c029549859 Allow to drop tables from s3_plain disk (so as from web disk) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-19 10:10:27 +01:00
zhongyuankai
a3a3261745
Merge branch 'master' into optimize_ttl 2022-11-19 16:32:26 +08:00
zhongyuankai
ce9b4866ea
Update MergeTreeDataMergerMutator.cpp 2022-11-19 16:31:47 +08:00
Alexander Gololobov
f004eea413 Add columns required fro defaults calculation 2022-11-18 23:42:45 +01:00
alesapin
c7d648174b
Merge pull request #43328 from ClickHouse/better-format-version-check 
Check content of `format_version` file in `MergeTreeData`
 2022-11-18 22:34:37 +01:00
Kruglov Pavel
82d6f5357c
Merge pull request #43156 from azat/dist/insert-apply-timeouts 
Apply connection timeouts settings for Distributed async INSERT from the query
 2022-11-18 21:27:57 +01:00
Nikolai Kochetov
5da1d893fd
Merge branch 'master' into read-from-mt-in-io-pool 2022-11-18 21:10:45 +01:00
Nikolai Kochetov
03c35e146b Refactor a bit. 2022-11-18 20:09:20 +00:00
serxa
8e65b799c2 fix test + add one more column 2022-11-18 20:04:11 +00:00
Han Fei
b80a2d6c89 address comments 2022-11-18 17:22:05 +01:00
serxa
a05a1b0133 Merge branch 'storage-system-moves' of github.com:ClickHouse/ClickHouse into storage-system-moves 2022-11-18 15:35:00 +00:00
Sergei Trifonov
77100896e3
Merge branch 'master' into storage-system-moves 2022-11-18 16:33:43 +01:00
Sergei Trifonov
d05223e70b
Merge pull request #43335 from ClickHouse/revert-43306-revert-43014-disk-s3-throttler 
Revert "Revert "S3 request per second rate throttling""
 2022-11-18 16:22:30 +01:00
Antonio Andelic
25697fd87d
Merge pull request #43289 from ClickHouse/fix-another-race-rmt 
Fix race on write in `ReplicatedMergeTree`
 2022-11-18 16:09:50 +01:00
alesapin
0f3eb51aed Disable compressed marks by default 2022-11-18 15:54:48 +01:00
Vitaly Baranov
a348332eab
Merge pull request #43227 from vitlibar/improve-masking-sensitive-info 
Improve masking sensitive info
 2022-11-18 15:37:50 +01:00
Han Fei
74c9ba8df0
Update src/Storages/MergeTree/ReplicatedMergeTreeSink.cpp 
Co-authored-by: alesapin <alesapin@gmail.com>
 2022-11-18 11:16:38 +01:00
Han Fei
3dc72b6c56
Update src/Storages/MergeTree/ReplicatedMergeTreeSink.cpp 
Co-authored-by: alesapin <alesapin@gmail.com>
 2022-11-18 11:15:51 +01:00
Han Fei
8c823bcaab
Update src/Storages/MergeTree/ReplicatedMergeTreeSink.cpp 
Co-authored-by: alesapin <alesapin@gmail.com>
 2022-11-18 11:15:45 +01:00
Han Fei
035e933196
Update src/Storages/MergeTree/ReplicatedMergeTreeSink.cpp 
Co-authored-by: alesapin <alesapin@gmail.com>
 2022-11-18 11:15:35 +01:00
Han Fei
f8ac75a054
Update src/Storages/MergeTree/ReplicatedMergeTreeSink.cpp 
Co-authored-by: alesapin <alesapin@gmail.com>
 2022-11-18 11:15:14 +01:00
Han Fei
d41094207d
Update src/Storages/MergeTree/EphemeralLockInZooKeeper.cpp 
Co-authored-by: alesapin <alesapin@gmail.com>
 2022-11-18 11:14:56 +01:00
Mikhail f. Shiryaev
f942ab1bd2
Update version to 22.12.1.1 2022-11-17 22:15:12 +01:00
Nikolai Kochetov
679591cf0a Try fix race. 2022-11-17 19:51:47 +00:00
Azat Khuzhin
aa77beabc9 Fix possible "No column to rollback" logical error during INSERT into Buffer 
In case of concurrent INSERT and SELECT, and memory pressure it is
possible to get this error, if you are very lucky.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-17 19:12:58 +01:00
Vitaly Baranov
050df6ac7f Move InDepthNodeVisitor.h back to src/Interpreters. 2022-11-17 18:16:32 +01:00
Azat Khuzhin
e67fc54fc9 StorageBuffer: log exception before rollback 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-17 18:10:11 +01:00
Sergei Trifonov
f2f0676bcc
Revert "Revert "S3 request per second rate throttling"" 2022-11-17 17:35:04 +01:00
Vitaly Baranov
2b9ed89721 Rename formatWithHiddenSecrets() -> formatWithSecretsHidden(). 2022-11-17 17:30:37 +01:00
kssenii
4e2fd5e22a Slightly better loading of metadata 2022-11-17 16:35:05 +01:00
alesapin
29a08f64af Merge branch 'master' into Enable_compress_marks_and_primary_key 2022-11-17 15:53:37 +01:00
Antonio Andelic
d324e8cc3e Write to first non-readonly disk 2022-11-17 14:52:35 +00:00
kssenii
5d9b34b59d More tests 2022-11-17 15:34:35 +01:00
Antonio Andelic
d41af8b741 Check and compare content of format_version file 2022-11-17 13:41:36 +00:00
alesapin
06f395d946 Add comments 2022-11-17 13:10:14 +01:00
Kruglov Pavel
a406557263
Merge branch 'master' into dist/insert-apply-timeouts 2022-11-17 12:51:47 +01:00
Antonio Andelic
d5848d53cc Fix DESCRIBE for deltaLake and hudi table functions 2022-11-17 11:46:17 +00:00
alesapin
a79d83549f Merge branch 'master' into fix_parts_to_do_detection 2022-11-17 12:30:05 +01:00
Guillaume Tassery
c47f90d197 add expression index in system.data_skipping_indices 2022-11-16 20:54:55 +01:00
Alexander Tokmakov
9011a18234
Revert "S3 request per second rate throttling" 2022-11-16 22:33:48 +03:00
Sergei Trifonov
159743edd6
Merge pull request #43014 from ClickHouse/disk-s3-throttler 
S3 request per second rate throttling
 2022-11-16 18:51:06 +01:00
Han Fei
221c4e5912 tiny changes 2022-11-16 17:59:08 +01:00
kssenii
f0ce28349d Better 2022-11-16 17:58:21 +01:00
Han Fei
14701670b5 Merge branch 'master' into hanfei/aysnc-insert 2022-11-16 17:24:36 +01:00
Nikolai Kochetov
17b75d3f5c Fix metrics and progress. 2022-11-16 14:10:56 +00:00
Antonio Andelic
11aba7597e Check if initialization is done on write 2022-11-16 13:33:22 +00:00
Smita Kulkarni
040f2b62d9 Enable compress marks and primary key 
Implementation:
* Set compress_marks and compress_primary_key to true by default.
* Updated getIndexExtensionFromFilesystem to only check for compressed file extension , else return default. When both are present, this function was returning uncompressed file extension giving rise to error as file got removed.
Testing:
* Updated tests where bytes size or compressed data is checked to check with compress_marks and compress_primary_key set to false.
 2022-11-16 14:23:58 +01:00
filimonov
dcb76f6bbf
Allow autoremoval of old parts if detach_not_byte_identical_parts enabled 
Allow autoremoval of old & detached parts if detach_not_byte_identical_parts enabled.

See also #28708 #37975
 2022-11-16 14:16:42 +01:00
alesapin
2012f4bd36 Fix use-after-free 2022-11-16 11:50:51 +01:00
Alexey Milovidov
04687c0d69
Merge pull request #43206 from ClickHouse/miscellaneous-changes-parallel-replicas 
Miscellaneous changes
 2022-11-16 05:30:03 +01:00
Vitaly Baranov
d365570b27 Hide passwords in SHOW CREATE TABLE and system.tables. 2022-11-16 01:35:11 +01:00
Nikolai Kochetov
1b1d23c100 Clumsy attempt to fix progress. 2022-11-15 21:23:18 +00:00
Alexander Tokmakov
69b067b81c
Merge pull request #43244 from ClickHouse/fix-replicated-merge-tree-shared-id 
Don't throw if shared ID already created in `StorageReplicatedMergeTree`
 2022-11-15 23:31:51 +03:00
Nikolai Kochetov
cf1afbfb84 Temporarily enable allow_asynchronous_read_from_io_pool_for_merge_tree by default to see thest results. 2022-11-15 17:47:26 +00:00
Nikolai Kochetov
29098045d8 Try read from MergeTree in background pool. 2022-11-15 17:38:37 +00:00
kssenii
234f6ee54d Add commands CREATE/ALTER/DROP NAMED COLLECTION 2022-11-15 15:51:56 +01:00
Sema Checherinda
7e73b187cc
Merge pull request #43159 from Algunenano/over-read 
Fix several buffer over-reads
 2022-11-15 15:04:24 +01:00
alesapin
a12f2d14dc Simpler check 2022-11-15 14:46:38 +01:00
Anton Popov
8c66c759d4
Merge pull request #43133 from amosbird/issue_43111 
Fix Nullable keys in hyperrectangle
 2022-11-15 14:30:13 +01:00
alesapin
4ec11d0f58 Remove redundant branch 2022-11-15 13:58:53 +01:00
Antonio Andelic
0e5e58bed9 Remove exception if shared ID already created 2022-11-15 12:18:24 +00:00
Antonio Andelic
96838f5be7 Merge branch 'master' into keeper-prometheus 2022-11-15 12:00:02 +00:00
Kseniia Sumarokova
59cf5def67
Merge branch 'master' into disk-s3-throttler 2022-11-15 12:13:37 +01:00
xiedeyantu
ec6698395e fix skip_unavailable_shards does not work using hdfsCluster table function 2022-11-15 13:25:15 +08:00
zhongyuankai
35690c1972 Merge branch 'master' into optimize_ttl 2022-11-15 08:49:37 +08:00
zhongyuankai
16f697f61d fix test 2022-11-15 08:47:51 +08:00
Alexey Milovidov
6e01b2b2a1
Merge pull request #43203 from ClickHouse/fix-typo-2 
Fix typo
 2022-11-15 01:39:34 +01:00
Alexey Milovidov
3cab3ce484
Merge pull request #43198 from FArthur-cmd/MySQL_clickhouse_connect 
add columns to information schema
 2022-11-15 01:11:29 +01:00
alesapin
6a50a32cfc Fix parts_to_do 2022-11-15 00:53:20 +01:00
alesapin
f53ada2bbd Revert "Fix bug in replication queue which can lead to premature mutation finish" 
This reverts commit 3b5424811c.
 2022-11-14 22:59:09 +01:00
alesapin
3b5424811c Fix bug in replication queue which can lead to premature mutation finish 2022-11-14 21:53:16 +01:00
Kruglov Pavel
386b69acae
Merge pull request #43131 from xiedeyantu/fix_s3cluster_connection_fail 
fix skip_unavailable_shards does not work using s3Cluster table function
 2022-11-14 20:48:00 +01:00
Vitaly Baranov
8e99f5fea3 Move maskSensitiveInfoInQueryForLogging() to src/Parsers/ 2022-11-14 18:55:19 +01:00
Alexander Tokmakov
c64109a2de
Merge branch 'master' into fetch-remove-covered 2022-11-14 18:23:16 +03:00
zhongyuankai
ea11d80536 Priority is given to deleting completely expired Parts 2022-11-14 23:03:56 +08:00
Han Fei
11f93dec3f tmp commit 2022-11-14 15:53:45 +01:00
Raúl Marín
4988e387f2 Merge remote-tracking branch 'blessed/master' into over-read 2022-11-14 12:28:31 +01:00
Kseniia Sumarokova
b7df9aa3da
Merge pull request #43155 from zzsmdfj/add_oss_function_and_StorageOSS 
to add_oss_function_and_StorageOSS
 2022-11-14 12:23:57 +01:00
Kseniia Sumarokova
e58a7140be
Merge pull request #43194 from ClickHouse/async-metrics-documentation 
Add documentation for `system.asynchronous_metrics`.
 2022-11-14 12:21:34 +01:00
Kseniia Sumarokova
79206c6eb0
Merge pull request #43147 from kssenii/named-collections-refactoring 
Add generic implementation for arbitrary structured named collections, access type and system.named_collections
 2022-11-14 11:57:17 +01:00
Antonio Andelic
ee182776da
Merge branch 'master' into keeper-prometheus 2022-11-14 09:14:13 +01:00
Alexey Milovidov
4edd8b0135 Remove unused method 2022-11-14 08:07:27 +01:00
Alexey Milovidov
17693de053 Better logs 2022-11-14 08:01:48 +01:00
Alexey Milovidov
9c48664f11 Better logs 2022-11-14 07:48:50 +01:00
Alexey Milovidov
582db233c4 Better logs 2022-11-14 07:22:17 +01:00
Alexey Milovidov
99f84d8095 Better logs 2022-11-14 07:22:02 +01:00
Alexey Milovidov
df01689ccc Better logs 2022-11-14 07:13:42 +01:00
Alexey Milovidov
a77032fafa Miscellaneous changes 2022-11-14 06:54:58 +01:00
Alexey Milovidov
15202833b1 Miscellaneous changes 2022-11-14 06:48:49 +01:00
Alexey Milovidov
53ee7446dc Miscellaneous changes 2022-11-14 06:46:37 +01:00
Alexey Milovidov
cc1855ea33 Miscellaneous changes 2022-11-14 06:45:52 +01:00
Alexey Milovidov
4966556f47 Miscellaneous changes 2022-11-14 06:41:51 +01:00
Alexey Milovidov
85dc692f7c Miscellaneous changes 2022-11-14 06:37:55 +01:00
Alexey Milovidov
60c96072e1 Miscellaneous changes 2022-11-14 06:37:19 +01:00
Alexey Milovidov
e080161ba5 Miscellaneous changes 2022-11-14 06:32:18 +01:00
Alexey Milovidov
bfea3deed9 Miscellaneous changes 2022-11-14 06:09:03 +01:00
Alexey Milovidov
3c603c3d8c
Update columns.sql 2022-11-14 02:42:30 +03:00
Alexey Milovidov
b9471e492a Fix typo 2022-11-13 22:52:21 +01:00
FArthur-cmd
f6b334fb3a add columns to information scheme 2022-11-13 12:03:46 +00:00
Alexey Milovidov
ba05b7dd2c Add documentation for AsynchronousMetrics 2022-11-13 03:36:20 +01:00
Filatenkov Artur
c7c52a570f
Update MergeTreeIndexAnnoy.cpp 2022-11-12 12:43:25 +03:00
FArthur-cmd
5eebc27d24 improve test 2022-11-12 09:23:49 +00:00
Nikolai Kochetov
9106910de8 Try read from MergeTree in background pool. 2022-11-11 20:47:11 +00:00
Igor Nikonov
ee5756b5b4
Merge pull request #43122 from ClickHouse/revert-43116-revert-42607-igor/insert_zk_retries_retry 
Revert "Revert " Keeper retries during insert (clean)""
 2022-11-11 18:22:38 +01:00
xiedeyantu
5504f3af9b fix skip_unavailable_shards does not work using s3Cluster table function 2022-11-12 00:03:36 +08:00
kssenii
616da1f695 Review fixed, style check fix 2022-11-11 16:50:02 +01:00
Antonio Andelic
cbe92bf8c8
Merge branch 'master' into keeper-prometheus 2022-11-11 13:56:27 +01:00
Nikolai Kochetov
63d06c8b53
Merge pull request #42829 from ClickHouse/read-in-order-from-query-plan 
Read-in-order over query plan
 2022-11-11 12:15:20 +01:00
Raúl Marín
54db7c6520 Enforce checking read output 2022-11-11 10:56:18 +01:00
Azat Khuzhin
3b2ceee753 Apply connection timeouts settings for Distributed async INSERT from the query 
Previosly connection related settings (connect_timeout_with_failover_ms,
connect_timeout_with_failover_secure_ms) was applied from the query only
for the case insert_distributed_sync=1, and in case of async INSERT it
uses global settings.

Note that this changes how connections is allocated, so now
split_batch_on_failure will create it's own connection, and this can
introduce more duplicates since in case of split_batch_on_failure is
enabled it may send files to different server, but this should not be a
problem because:
- it does not resend batch if it has only one file, when deduplication
  will work
- and in all other cases deduplication will not work since checksum
  should be different

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-11 09:54:07 +01:00
Azat Khuzhin
d1cee3e1ff Do not resend batch as separate files if there is only one file 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-11 09:48:44 +01:00
zzsmdfj
3835373644 to add_oss_function_and_StorageOSS 2022-11-11 16:40:10 +08:00
Amos Bird
661c541e57
Fix Nullable keys in hyperrectangle. 2022-11-11 11:14:05 +08:00
kssenii
63de577172 Better, add tests 2022-11-10 22:47:30 +01:00
avogar
9e89af28c6 Refactor BSONEachRow format, fix bugs, support more data types, support parallel parsing and schema inference 2022-11-10 20:15:14 +00:00
Alexander Tokmakov
bb00a246f1 priority for DROP_RANGEs, assertion for others 2022-11-10 20:53:05 +01:00
kssenii
a5d06cd909 Better Impl 2022-11-10 19:59:52 +01:00
Alexander Tokmakov
e415fc6e95
Revert "Optimize TTL merge, completely expired parts can be removed in time" 2022-11-10 19:38:55 +03:00
kssenii
a47aac3e2b Better system table structure 2022-11-10 17:07:17 +01:00
kssenii
bed7ac9442 Support arbitrary structure collections 2022-11-10 17:05:24 +01:00
kssenii
c4f522b414 Support arbintrary structure collections 2022-11-10 16:32:15 +01:00
Igor Nikonov
f8c907c463
Revert "Revert " Keeper retries during insert (clean)"" 2022-11-10 13:14:04 +01:00
Alexander Tokmakov
fb8c527a2f
Revert " Keeper retries during insert (clean)" 2022-11-10 14:12:24 +03:00
Igor Nikonov
1b9be27fdb
Merge pull request #42607 from ClickHouse/igor/insert_zk_retries_retry 
Keeper retries during insert (clean)
 2022-11-10 12:01:00 +01:00
Antonio Andelic
1954ff385d
Merge pull request #43076 from canhld94/ch_canh_minor 
Better logging for mark range filtering on projection parts
 2022-11-10 09:33:12 +01:00
Antonio Andelic
f2a9b5933d Merge branch 'master' into keeper-prometheus 2022-11-10 09:28:06 +01:00
Nikolai Kochetov
f2f5c170e4
Merge branch 'master' into read-in-order-from-query-plan 2022-11-09 22:36:36 +01:00
alesapin
9e1627fcdd
Merge pull request #43060 from ClickHouse/fix_projection_part 
Fix projection part removal with zero-copy replication
 2022-11-09 18:15:49 +01:00
kssenii
e94e087187 Add generic named collections class and factory 2022-11-09 14:08:16 +01:00
Igor Nikonov
379113f8f1 Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry 2022-11-09 13:03:32 +00:00
Antonio Andelic
c92ae5a385 Add suport for AsyncMetrics in standalone Keeper 2022-11-09 13:46:03 +01:00
Alexander Tokmakov
1c6a617207
Merge branch 'master' into fix_intersecting_parts2 2022-11-09 14:38:18 +03:00
Maksim Kita
768ada118b Analyzer AST key condition crash fix 2022-11-09 11:41:39 +01:00
Vladimir C
82bf992099
Merge pull request #42869 from zhongyuankai/optimze_ttl_merge 2022-11-09 11:26:46 +01:00
Duc Canh Le
b9a3d090ed use const string & 2022-11-09 13:07:24 +08:00
Duc Canh Le
901e1e2fef better logging for mark range filtering on projection parts 2022-11-09 12:01:51 +08:00
alesapin
761274d857 Fix wrong logic 2022-11-09 01:14:03 +01:00
Igor Nikonov
345304abe0 Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry 2022-11-08 20:26:50 +00:00
alesapin
a31a79020f Add comment 2022-11-08 16:27:26 +01:00
alesapin
6f059be748 Fix bug with temporary projection parts deletion 2022-11-08 15:37:03 +01:00
serxa
2de26daa56 fix build 2022-11-08 14:31:29 +00:00
SmitaRKulkarni
658036d179
Merge pull request #42708 from ClickHouse/42414_Enable_functions_in_readonly_mode 
42414 enable functions in readonly mode
 2022-11-08 15:22:31 +01:00
Alexander Tokmakov
eb19df0094 fix 2022-11-08 15:21:25 +01:00
Sergei Trifonov
8eedd1e046
Merge branch 'master' into disk-s3-throttler 2022-11-08 15:00:56 +01:00
SmitaRKulkarni
4f2e5596e3
Merge pull request #42933 from yakkomajuri/patch-2 
fix(typo): Passible -> Possible
 2022-11-08 14:54:50 +01:00
serxa
6d5d9ff421 rename ReadWriteSettings -> RequestSettings 2022-11-08 13:48:23 +00:00
Maksim Kita
0aa4fd3c2d Fixed tests 2022-11-08 14:29:17 +01:00
Kseniia Sumarokova
d2b777c059
Merge pull request #41054 from rubin-do/hudi 
Add Hudi and DeltaLake table engines
 2022-11-08 12:03:06 +01:00
Kseniia Sumarokova
92b5ca326c
Merge pull request #42303 from xiedeyantu/system_table_add_column 
system.detached_parts add column bytes_on_disk and path
 2022-11-08 12:00:56 +01:00
Robert Schulze
2c568df714
Merge pull request #42968 from ClickHouse/bitcast 
Less use of CH-specific bit_cast()
 2022-11-08 10:54:10 +01:00
Robert Schulze
bb507356ef
Merge pull request #42209 from canhld94/ch_canh_fix_prefix_not_like 
Add `notLike` to key condition atom map
 2022-11-08 10:07:21 +01:00
Antonio Andelic
7067f9e0e1
Merge pull request #42947 from azat/mergetree/part-load-tiny 
Attach thread pool for loading parts to the query
 2022-11-08 09:17:32 +01:00
zhongyuankai
37099e57f4
Merge branch 'master' into optimze_ttl_merge 2022-11-08 12:26:46 +08:00
Azat Khuzhin
2bd4a71150
Merge branch 'master' into fetch-remove-covered 2022-11-08 00:55:00 +01:00
Kseniia Sumarokova
4fc7926c16
Merge branch 'master' into system_table_add_column 2022-11-07 23:36:57 +01:00
Kseniia Sumarokova
d695240d53
Update StorageHudi.cpp 2022-11-07 23:07:00 +01:00
serxa
ea17b30358 Merge branch 'storage-system-moves' of github.com:ClickHouse/ClickHouse into storage-system-moves 2022-11-07 20:33:31 +00:00
serxa
59408ebf57 fill more fields 2022-11-07 20:33:25 +00:00
Sergei Trifonov
6d3b96ee42
Merge branch 'master' into storage-system-moves 2022-11-07 21:31:27 +01:00
alesapin
257e766e0e Trying to fix two flaky tests 2022-11-07 20:49:41 +01:00
Alexander Tokmakov
9210e586d2 fix 2022-11-07 20:27:18 +01:00
Alexander Tokmakov
b8174a63a8 Merge branch 'master' into fix_intersecting_parts2 2022-11-07 20:26:36 +01:00
serxa
2daec0b45e S3 request per second rate throttling + refactoring 2022-11-07 18:05:40 +00:00
xiedeyantu
d4cdc38b71 better 2022-11-07 22:49:58 +08:00
Kseniia Sumarokova
891e167509
Update StorageHudi.cpp 2022-11-07 15:17:16 +01:00
Kruglov Pavel
48163ff9c2
Merge pull request #42938 from amosbird/issue_42771 
Fix NOT_FOUND_COLUMN_IN_BLOCK in some rare case
 2022-11-07 13:10:12 +01:00
kssenii
897811371d Minor changes 2022-11-07 12:45:26 +01:00
kssenii
819be6bb32 Merge remote-tracking branch 'upstream/master' into hudi 2022-11-07 11:30:53 +01:00
Duc Canh Le
bd2bd7149d fix typo 2022-11-07 18:22:50 +08:00
SmitaRKulkarni
c563e124c2
Merge branch 'master' into 42414_Enable_functions_in_readonly_mode 2022-11-07 09:50:05 +01:00
Duc Canh Le
f7e65ebd72 make comments more clear 2022-11-07 16:25:36 +08:00
Robert Schulze
cdaf0becfe
Merge branch 'master' into bitcast 2022-11-07 09:24:52 +01:00
Robert Schulze
f3f50530d1
Merge branch 'master' into ch_canh_fix_prefix_not_like 2022-11-07 09:14:02 +01:00
Antonio Andelic
e08f94d0f8
Merge pull request #42659 from ClickHouse/optimize-entire-partition 
Support for optimizing old parts for entire partition only
 2022-11-07 09:06:11 +01:00
Duc Canh Le
69d6d42443 NOT LIKE only work for 'prefix%' 2022-11-06 15:26:19 +08:00
zhongyuankai
ef6065b331 fix test 2022-11-06 13:40:26 +08:00
Duc Canh Le
a8bce28c04 Merge branch 'master' into ch_canh_fix_prefix_not_like 2022-11-06 13:04:08 +08:00
Igor Nikonov
e94b9cd003 Fixes 
+ enable retries in integration test
+ typo in fault injection traces
 2022-11-05 22:28:32 +00:00
Igor Nikonov
859022660e Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry 2022-11-04 23:31:51 +00:00
Daniil Rubin
7500cc38c0 Apply clang-format 2022-11-04 20:51:44 +00:00
Daniil Rubin
37048e986a Add comments, fix review issues 2022-11-04 20:49:24 +00:00
Daniil Rubin
d6bbd65dde Fix broken include 2022-11-04 16:03:12 +00:00
Robert Schulze
9c066e964d
Less use of CH-specific bit_cast() 
Converted usage of CH-custom bit_cast to std::bit_cast if possible, i.e.
when
  sizeof(From) == sizeof(To).
(The CH-custom bit_cast is able to deal with sizeof(From) != sizeof(To).)

Motivation for this came from #42847 where it is not clear how the
internal bit_cast should behave on big endian systems, so we better
avoid that situation as much as possible.
 2022-11-04 15:52:48 +00:00
Vladimir C
f359411668
Merge branch 'master' into optimze_ttl_merge 2022-11-04 16:52:20 +01:00
Vitaly Baranov
52b1f4aed9
Merge pull request #42484 from vitlibar/mask-sensitive-info-in-logs 
Mask some information in logs
 2022-11-04 14:09:38 +01:00
zhongyuankai
1345f7e73d Merge branch 'master' into optimze_ttl_merge 2022-11-04 20:54:46 +08:00
alesapin
82729a88c8
Merge pull request #42833 from vitlibar/backup-s3-increase-upload-part-size-exponentially 
Increase the size of upload part exponentially for backup to S3.
 2022-11-04 13:41:10 +01:00
Kseniia Sumarokova
ed44b20694
Merge pull request #42224 from kssenii/fit-http-buffer-retries 
Fix http buffer retries
 2022-11-04 11:50:17 +01:00
Amos Bird
d901ead1bc
Done 2022-11-04 17:26:44 +08:00
zhongyuankai
049a593272 Merge branch 'master' into optimze_ttl_merge 2022-11-04 17:22:38 +08:00
zhongyuankai
e773eb2fd1 batter 2022-11-04 17:19:34 +08:00
Azat Khuzhin
7032b2b98a Attach thread pool for loading parts to the query 
Detach should be done via CurrentThread::defaultThreadDeleter()

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-04 09:06:12 +01:00
Azat Khuzhin
cdd09fd72a Improve messages for loading data parts (add part type) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-04 09:05:06 +01:00
Daniil Rubin
99d57325bb Merge branch 'master' into hudi 2022-11-03 19:29:23 +00:00
Daniil Rubin
a573d8aef7 Apply clang-format 2022-11-03 18:17:39 +00:00
Daniil Rubin
7b57e94d9f Fix StorageHudi 2022-11-03 18:14:10 +00:00
Daniil Rubin
a9f8948c8d Fix build 2022-11-03 17:28:41 +00:00
Amos Bird
7186898ffa
Experiment 2022-11-04 00:29:17 +08:00
Yakko Majuri
e68a3b0c7a
fix(typo): Passible -> Possible 2022-11-03 11:25:07 -03:00
kssenii
4fe4a07600 Add test 2022-11-03 12:14:08 +01:00
Antonio Andelic
66ffdda973
Merge branch 'master' into optimize-entire-partition 2022-11-03 09:52:13 +01:00
Alexander Tokmakov
e67d05604c Merge branch 'master' into fix_intersecting_parts2 2022-11-02 19:30:27 +01:00
Alexander Tokmakov
d8b3a2af84 fix race between INSERT and DROP 2022-11-02 19:26:50 +01:00
Vitaly Baranov
32194c1200 Add max limitation for the size of an uploaded part. 2022-11-02 17:53:54 +01:00
Igor Nikonov
9a315b7899 Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry 2022-11-02 15:32:54 +00:00
Vitaly Baranov
e0133688bc
Merge branch 'master' into mask-sensitive-info-in-logs 2022-11-02 16:26:13 +01:00
Alexander Tokmakov
00c9e50ee3 fix race between INSERT and ALTER PARTITION 2022-11-02 16:25:26 +01:00
Alexander Tokmakov
acbad99b24 remove abandonable_lock part 2 2022-11-02 14:28:49 +01:00
Antonio Andelic
c4b717b343
Update src/Storages/MergeTree/MergeTreeDataMergerMutator.cpp 
Co-authored-by: Sergei Trifonov <sergei@clickhouse.com>
 2022-11-02 08:24:54 +01:00
Duc Canh Le
ec382d7903 add comment 2022-11-02 09:50:41 +08:00
Duc Canh Le
c6598dc66d
Merge branch 'master' into ch_canh_fix_prefix_not_like 2022-11-02 09:42:01 +08:00
Alexander Tokmakov
4f50a999bb remove abandonable_lock part 1 2022-11-01 21:35:25 +01:00
Nikolai Kochetov
478d3079c8 Merge branch 'master' into read-in-order-from-query-plan 2022-11-01 21:02:22 +01:00
Nikolai Kochetov
9ffebf4a0c Fixing read-in-order for special storages. 2022-11-01 19:51:52 +00:00
Igor Nikonov
e76c3c381f Better handling tables in read-only mode during insert 
+ throw immediately if table has no metadata in zk
+ stop retries in case of shutdown
+ check if table is readonly at the begining of every retry
 2022-11-01 18:47:51 +00:00
Anton Popov
88674ef964
Merge pull request #42794 from CurtizJ/better-code-in-check-part 
Slightly better code with projections
 2022-11-01 16:05:27 +01:00
zhongyuankai
c1e2b2af74 Optimize TTL merge, completely expired parts can be removed in time 2022-11-01 22:58:15 +08:00
Anton Popov
876dca48da
Merge pull request #36969 from CurtizJ/dynamic-columns-14 
Support `Object` type inside other types
 2022-11-01 15:20:02 +01:00
Duc Canh Le
07d9720ed5
Merge branch 'master' into ch_canh_fix_prefix_not_like 2022-11-01 22:16:09 +08:00
Kruglov Pavel
21d50f76ea
Merge pull request #41979 from Avogar/s3-cluster-schema-inference 
Fix schema inference in s3Cluster and improve in hdfsCluster
 2022-11-01 14:00:21 +01:00
Vitaly Baranov
b9f2f17331 Add test and logging. 2022-11-01 12:23:20 +01:00
Maksim Kita
fe48a1ce7e
Merge pull request #42681 from kitaisreal/merge-tree-indexes-use-actions-dag 
MergeTree indexes use ActionsDAG
 2022-11-01 13:55:17 +03:00
Duc Canh Le
3545db0a6c
Merge branch 'master' into ch_canh_fix_prefix_not_like 2022-11-01 16:54:26 +08:00
Yakov Olkhovskiy
bab0e06e3d
Merge pull request #42458 from Clark0/match-func-use-index 
match function can use index if it's a condition on string prefix #37333
 2022-10-31 17:33:58 -04:00
Igor Nikonov
9cc64a0fcc Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry 2022-10-31 18:21:21 +00:00
Igor Nikonov
2d5050ea66 Fix review comments 2022-10-31 18:20:58 +00:00
Anton Popov
2ae3cfa9e0
Merge branch 'master' into dynamic-columns-14 2022-10-31 16:15:19 +01:00
Alexander Tokmakov
599ccb9939
Update MergeTreeData.h 2022-10-31 16:18:17 +03:00
Alexander Tokmakov
b2b9479afa
Update src/Storages/MergeTree/MergeTreeData.h 
Co-authored-by: Sergei Trifonov <sergei@clickhouse.com>
 2022-10-31 16:15:35 +03:00
SmitaRKulkarni
54035270b2
Merge branch 'master' into 42414_Enable_functions_in_readonly_mode 2022-10-31 14:15:29 +01:00
Maksim Kita
bca22ec5f5 Fixed code review issues 2022-10-31 12:30:00 +01:00
Maksim Kita
115fcaffc5 Fixed tests 2022-10-31 11:30:56 +01:00
Vitaly Baranov
5d2a222fe4 Mask sensitive information in logs. 2022-10-31 10:50:33 +01:00
Duc Canh Le
d309e8793a
Merge branch 'master' into ch_canh_fix_prefix_not_like 2022-10-31 15:13:39 +08:00
Alexander Tokmakov
f53df7870c fix race between drop and failed insert 2022-10-30 17:59:47 +01:00
Alexander Tokmakov
c68ab231f9 fix accessing part in Deleting state 2022-10-30 17:30:51 +01:00
FArthur-cmd
607b81989d fix build 2022-10-30 14:57:10 +00:00
Anton Popov
8de75bf29c better code with projections 2022-10-29 14:39:59 +00:00
clarkcaoliu
ca5dbe88db match function can use index if prefix 2022-10-29 18:25:29 +08:00
Filatenkov Artur
21c024b043
Merge branch 'master' into annoy_cosine 2022-10-28 20:15:53 +03:00
FArthur-cmd
f187d4e1d4 add cosine distance for annoy and fix docs for cosineDistance 2022-10-28 17:03:35 +00:00
Igor Nikonov
d09deaf10f Fix style check 2022-10-28 15:17:05 +00:00
Igor Nikonov
937dd44399 Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry 2022-10-28 15:09:23 +00:00
Igor Nikonov
c53b96a11c Add more comments 2022-10-28 15:09:01 +00:00
Alexey Milovidov
f64e632707
Merge pull request #42469 from amosbird/issue_42456 
Fix misbehavior of key analysis
 2022-10-28 16:18:00 +02:00
Antonio Andelic
5fae8b0cc9 Merge branch 'master' into optimize-entire-partition 2022-10-28 11:44:10 +00:00
serxa
acc00218d9 fix build 2022-10-28 11:23:49 +00:00
Sergei Trifonov
836595c7ae
Merge branch 'master' into storage-system-moves 2022-10-28 11:39:38 +02:00
Duc Canh Le
848f5b5d65
Merge branch 'master' into ch_canh_fix_prefix_not_like 2022-10-28 13:40:17 +08:00
Duc Canh Le
c080964391 correct 'notLike' in key condition 2022-10-28 13:39:01 +08:00
Duc Canh Le
69311bd5cd Merge branch 'master' into ch_canh_fix_prefix_not_like 2022-10-28 11:59:50 +08:00
Igor Nikonov
3682913172 Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry 2022-10-27 22:01:53 +00:00
Igor Nikonov
a601c166ad Add clean up for faults injected before zk request 
+ for multi-op request, it create responses in multi response
+ some polishing
 2022-10-27 21:35:13 +00:00
Anton Popov
7715afa595
Merge pull request #42618 from CurtizJ/refactor-data-part 
Try to save `IDataPartStorage` interface
 2022-10-27 22:20:44 +02:00
Igor Nikonov
f773436de5 Fix flaky test 2022-10-27 13:34:07 +00:00
Nikolai Kochetov
24e1aaf324
Merge pull request #42576 from ClickHouse/table-function-merge-invalid-where-type 
Invalid type in where for Merge table (logical error)
 2022-10-27 13:03:27 +02:00
Antonio Andelic
10aa7091ca
Merge pull request #42605 from ClickHouse/correctly-handle-prewhere-type 
Correctly handle non-native integers in PREWHERE
 2022-10-27 09:25:38 +02:00
SmitaRKulkarni
e860ecc80c
Merge branch 'master' into 42414_Enable_functions_in_readonly_mode 2022-10-27 08:26:44 +02:00
Nikolai Kochetov
313cbb1058 Fix test. 2022-10-26 17:49:36 +00:00
Igor Nikonov
649e8d302c Sync with retiring branch igor/insert_zk_retries 2022-10-26 17:41:44 +00:00
Igor Nikonov
8435cba352 Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry 2022-10-26 17:35:47 +00:00
Maksim Kita
f04f663818 Fixed tests 2022-10-26 18:55:14 +02:00
Smita Kulkarni
6a8fa2d4a5 Added new field allow_readonly in system.table_functions to allow using table functions in readonly mode. 
Implementation:
* Added a new field allow_readonly to table system.table_functions.
* Updated to use new field allow_readonly to allow using table functions in readonly mode.
* Added TableFunctionProperties similar to AggregateFunctionProperties.
* The functions allowed in readonly mode are in this set table_functions_allowed_in_readonly_mode.
Testing:
* Added a test for filesystem tests/queries/0_stateless/02473_functions_in_readonly_mode.sh
Documentation:
* Updated the english documentation for Table Functions.
 2022-10-26 18:45:23 +02:00
Nikolai Kochetov
1520bcd53c Fix error code. 2022-10-26 15:50:10 +00:00
Anton Popov
ca4df62e37
Merge branch 'master' into refactor-data-part 2022-10-26 17:49:20 +02:00
Mikhail f. Shiryaev
fdecac65b8
Merge pull request #42699 from ClickHouse/22.11-prepare 
Update version after release
 2022-10-26 16:31:48 +02:00
Maksim Kita
1c17e9d454 Fixed tests 2022-10-26 15:54:37 +02:00
Mikhail f. Shiryaev
d3461abd1f
Update version to 22.11.1.1 2022-10-26 15:27:09 +02:00
Nikolai Kochetov
db422fa903
Merge pull request #42410 from ClickHouse/fix-multi-arg-monotonicity 
Fix possible LOGICAL_ERROR in binary ariphmetics monotonicity.
 2022-10-26 14:11:23 +02:00
Kruglov Pavel
219553df8d
Merge pull request #42580 from hanfei1991/fei/limit_max_rows 
fix behaviour of max_rows_to_read for trival limit queries
 2022-10-26 14:09:59 +02:00
Maksim Kita
c47ca522f4 Fixed style check 2022-10-26 13:27:14 +02:00
Antonio Andelic
35a9672704 Merge branch 'master' into optimize-entire-partition 2022-10-26 11:09:25 +00:00
Maksim Kita
1b6293f6db MergeTree indexes use ActionsDAG 2022-10-26 12:44:37 +02:00
Antonio Andelic
97aaebfa18 Address PR comments 2022-10-26 10:06:56 +00:00
SmitaRKulkarni
5df138da82
Merge pull request #42376 from ClickHouse/36316_Support_glob_for_recursive_directory_traversal 
Added ** glob support for recursive directory traversal to filesystem…
 2022-10-26 11:49:20 +02:00
Antonio Andelic
979d180931
Merge branch 'master' into correctly-handle-prewhere-type 2022-10-26 11:13:16 +02:00
Azat Khuzhin
8db31beee6 Remove covered parts for fetched part 
Here is an example that I found on production, simplified.

Consider the following queue (nothing of this had been processed on this
replica):

- GET_PART all_0_0_0 (queue-0000000001)
- GET_PART all_1_1_0 (queue-0000000002)
...
- GET_PART all_0_1_1 (queue-0000000003)
- GET_PART all_2_2_0 (queue-0000000004)
...
- MERGE_PARTS from [all_0_1_1, all_2_2_0] to all_0_2_2 (queue-0000000005)

And now queue-0000000005 started to executing (either because
of reording, or because at that time GET_PART fails), and it
does not have any required parts, so it will fetch them, but
not all_0_0_0 and all_1_1_0, so this replica delay will set to
the time of min(queue-0000000001, queue-0000000002), while it
is not true, since it already have parts that covers those
parts.

and since MERGE_PARTS takes 30min, it increased the replica delay
eventually to 30min, for the time range of 30min, which is pretty huge.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-26 11:10:50 +03:00
Anton Popov
79889198b1 fix style check 2022-10-25 23:03:37 +00:00
Anton Popov
d8b09b430f add comments 2022-10-25 22:27:11 +00:00
Anton Popov
c4d4f2dbed better interface 2022-10-25 22:14:06 +00:00
Sergei Trifonov
34b43f8b4b
Merge branch 'master' into storage-system-moves 2022-10-25 22:56:45 +02:00
Nikolai Kochetov
bfb5e6eae4 Fix logical error for invalid prewhere type in StorageMerge. 2022-10-25 13:58:29 +00:00
serxa
55a86bc2de Add active moves system table 2022-10-25 13:56:46 +00:00
Antonio Andelic
611c2e2bd7 Support for optimizing old parts for entire partition only 2022-10-25 13:40:18 +00:00
Anton Popov
c8199bc125
Merge branch 'master' into refactor-data-part 2022-10-25 14:31:05 +02:00
Raúl Marín
0016bc2b73
Improve the time to recover keeper connections (#42541) 
Co-authored-by: Nikita Mikhaylov <nikitamikhaylov@clickhouse.com>
Co-authored-by: robot-clickhouse <robot-clickhouse@clickhouse.com>
Co-authored-by: Alexey Milovidov <milovidov@clickhouse.com>
 2022-10-25 13:26:29 +02:00
Antonio Andelic
af0c54e055 Small fix 2022-10-25 11:13:41 +00:00