thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-15 10:52:30 +00:00
Code Issues Packages Projects Releases Wiki Activity
131,500 Commits 9,328 Branches 1,783 Tags 2.1 GiB
dcbd3b9c26
Commit Graph

15578 Commits

Author SHA1 Message Date
Nikolai Kochetov
08a7575984 Re-implement filtering a bit. 2023-11-28 16:17:35 +00:00
Han Fei
e4421e2808 remove wrong code 2023-11-28 16:46:55 +01:00
Nikita Taranov
52f644c0df Merge branch 'master' into optimize_reading3 2023-11-28 16:36:38 +01:00
Han Fei
6733438adc Merge branch 'master' into hanfei/statistic 2023-11-28 16:35:00 +01:00
Han Fei
8e84c734bf rename some code 2023-11-28 16:32:47 +01:00
zvonand
c306d21b54 merge master + resolve conflicts 2023-11-28 15:51:21 +01:00
Mikhail Koviazin
daaa575106
Remove unused Strings from MergeTreeData 
Compiler does not emit a warning when there is an unused variable of
a type with a non-trivial constructor. std::string is one of them.
Hence, these two variables were overlooked. This commit removes them
 2023-11-28 16:33:44 +02:00
Alexander Tokmakov
e40c71a74c
Merge pull request #56533 from ClickHouse/rmt_check_shutdown_flags_in_retry_loops 
ReplicatedMergeTree: check shutdown flags in retry loops
 2023-11-28 15:24:40 +01:00
Alexander Sapin
2cd9f5f3cc Fix known issue 2023-11-28 15:13:12 +01:00
Alexander Sapin
e6855cef4d Followup 2023-11-28 14:01:31 +01:00
Alexander Sapin
b5404f4a9f Followup 2023-11-28 13:34:05 +01:00
Alexander Sapin
e1aa62d9cd Make alter materialized view non expiremental and deprecate allow_experimental_alter_materialized_view_structure obsolete 2023-11-28 13:17:55 +01:00
Antonio Andelic
636ac3f349 Fix assert 2023-11-28 12:07:05 +00:00
vdimir
af81cb8252
Setting JoinAlgorithm respect specified order 2023-11-28 11:01:35 +00:00
Antonio Andelic
9dd4783675 Merge branch 'master' into fix-readonly-replica 2023-11-28 10:43:29 +00:00
Robert Schulze
d459fa1fc6
Merge remote-tracking branch 'ClickHouse/master' into alias_column_in_indices 2023-11-28 10:15:25 +00:00
Raúl Marín
ff0df49e2b Fix comment in MergeTreeDataPartWide::checkConsistency 2023-11-28 10:09:03 +00:00
Nikolai Kochetov
7e1f625179 Revert system tables to master. 2023-11-28 10:07:36 +00:00
Julia Kartseva
57a4492275
Merge pull request #56490 from jkartseva/fuzz_json 
Fuzz JSON table function
 2023-11-27 21:47:53 -08:00
Ubuntu
a85cf758b0 Fuzz JSON table function 
Create a table function `fuzzJSON`

An example query:
```
CREATE NAMED COLLECTION json_fuzzer AS json_str={};

SELECT *
FROM fuzzJSON(json_fuzzer, json_str = '{"students" : ["Alice", "Bob"]}', random_seed = 666, max_output_length = 128, probability = 0.9)
LIMIT 3

Query id: 7f802052-efb0-41b4-87fa-03b7dd290e9d

┌─json──────────────────────────────────────────────────────────────────────────────────┐
│ {"ade0yX":[9200406572736542991, true, "sm"]}                                          │
│ {"students":["Alice", "eSN3WO#a6NYTBe0$#OWwyIQ"], "cVoP2BuQugQ":17384271928263249403} │
│ {"students":["Alice", "Bob", "T+-k4+PJGkL%XRRaF2BoeN@&A"]}                            │
└───────────────────────────────────────────────────────────────────────────────────────┘
```

Next step:
* Generate invalid string

Fixes #35962

add Object('json')

use named collection
 2023-11-28 01:58:24 +00:00
Han Fei
7bd2ea3add Merge branch 'master' into hanfei/statistic 2023-11-27 23:31:23 +01:00
Antonio Andelic
544f6081eb
Merge pull request #57259 from ClickHouse/ignore-column-comments 
Ignore comments when comparing column descriptions
 2023-11-27 20:43:39 +01:00
Nikolai Kochetov
e1b00b3ee0 Merge branch 'master' into SystemTablesFilterEngine 2023-11-27 18:54:21 +00:00
Antonio Andelic
9077550bb1 Better Readonly metric 2023-11-27 14:25:45 +00:00
Antonio Andelic
9707796869 Ignore comments when comparing column descriptions 2023-11-27 12:25:41 +00:00
vdimir
bbc202a891
Merge pull request #56816 from AVMusorin/improve-settings-rocksdb 
Show total_bytes and total_rows in system tables for RocksDB storage
 2023-11-27 11:42:15 +01:00
Kruglov Pavel
b10e46b2bc
Merge pull request #57126 from Avogar/size-virtual-column 
Add _size virtual column to s3/file/hdfs/url/azureBlobStorage engines
 2023-11-27 10:12:18 +01:00
flynn
aa15a4e46d Merge branch 'master' of github.com:ClickHouse/ClickHouse into alias_column_in_indices 2023-11-27 03:52:00 +00:00
flynn
f995ac87c5 review fix 2023-11-27 03:50:34 +00:00
Azat Khuzhin
4a02de4674 Add ability to disable checksums for S3 to avoid excessive input file read 
AWS S3 client can read file multiple times, this is required for:
- calculate checksums
- calculate signature (done only for HTTP, since ClickHouse uses
  PayloadSigningPolicy::Never)

So this means that for HTTP, to send file to S3 it will be read 3x
times, and for HTTPS 2x times.

By overriding GetChecksumAlgorithmName() to return empty string,
checksums can be disabled, and the input file will be read only once.

And even though additional https layer adds extra integrity layer,
someone still may find this too risky I guess, even though ClickHouse
internal format (for MergeTree) has checksums, and more.

Here is an example stacktrace of this excessive read:

<details>

<summary>stacktrace</summary>

    (lldb) bt
    * thread 383, name = 'BackupWorker', stop reason = breakpoint 1.1
      * frame 0: 0x00000000103c5fc0 clickhouse`DB::StdStreamBufFromReadBuffer::seekpos() + 32 at StdStreamBufFromReadBuffer.cpp:67
        frame 1: 0x000000001777f7f8 clickhouse`std::__1::basic_istream<char, std::__1::char_traits<char>>::tellg() [inlined] std::__1::basic_streambuf<char, std::__1::char_traits<char>>::pubseekoff[abi:v15000](this=<unavailable>, __off=0, __way=cur, __which=8) + 120 at streambuf:162
        frame 2: 0x000000001777f7e3 clickhouse`std::__1::basic_istream<char, std::__1::char_traits<char>>::tellg() + 99 at istream:1249
        frame 3: 0x00000000152e4979 clickhouse`Aws::Utils::Crypto::MD5OpenSSLImpl::Calculate() + 57 at CryptoImpl.cpp:223
        frame 4: 0x00000000152dedee clickhouse`Aws::Utils::Crypto::MD5::Calculate() + 14 at MD5.cpp:30
        frame 5: 0x00000000152db5ac clickhouse`Aws::Utils::HashingUtils::CalculateMD5() + 44 at HashingUtils.cpp:235
        frame 6: 0x000000001528b97b clickhouse`Aws::Client::AWSClient::AddChecksumToRequest() const + 507 at AWSClient.cpp:772
        frame 7: 0x000000001528ded2 clickhouse`Aws::Client::AWSClient::BuildHttpRequest() const + 1682 at AWSClient.cpp:930
        frame 8: 0x00000000100b864f clickhouse`DB::S3::Client::BuildHttpRequest() const + 15 at Client.cpp:622
        frame 9: 0x0000000015286a41 clickhouse`Aws::Client::AWSClient::AttemptOneRequest(this=0x00007ffde2f8f000, httpRequest=<unavailable>, request=<unavailable>, signerName=<unavailable>, signerRegionOverride=<unavailable>, signerServiceNameOverride="s3") const + 65 at AWSClient.cpp:491
        frame 10: 0x00000000152845b9 clickhouse`Aws::Client::AWSClient::AttemptExhaustively(this=0x00007ffde2f8f000, uri=0x00007ffdd4d44f38, request=0x00007ffdd4d45d10, method=HTTP_PUT, signerName="SignatureV4", signerRegionOverride="us-east-1", signerServiceNameOverride="s3") const + 1337 at AWSClient.cpp:272
        frame 11: 0x0000000015298d0d clickhouse`Aws::Client::AWSXMLClient::MakeRequest() const + 45 at AWSXmlClient.cpp:99
        frame 12: 0x0000000015298cb5 clickhouse`Aws::Client::AWSXMLClient::MakeRequest() const + 309 at AWSXmlClient.cpp:66
        frame 13: 0x0000000015354b23 clickhouse`Aws::S3::S3Client::PutObject(this=0x00007ffde2f8f000, request=0x00007ffdd4d45d10) const + 2659 at S3Client.cpp:1731
        frame 14: 0x00000000100b174f clickhouse`DB::S3::Client::PutObject(DB::S3::ExtendedRequest<Aws::S3::Model::PutObjectRequest> const&) const [inlined]
        frame 15: 0x00000000100b173a clickhouse`DB::S3::Client::PutObject(DB::S3::ExtendedRequest<Aws::S3::Model::PutObjectRequest> const&) const + 41 at Client.cpp:578
        frame 16: 0x00000000100b1711 clickhouse`DB::S3::Client::PutObject(DB::S3::ExtendedRequest<Aws::S3::Model::PutObjectRequest> const&) const + 981 at Client.cpp:508
        frame 17: 0x00000000100b133c clickhouse`DB::S3::Client::PutObject(DB::S3::ExtendedRequest<Aws::S3::Model::PutObjectRequest> const&) const [inlined]
        frame 18: 0x00000000100b133c clickhouse`DB::S3::Client::PutObject() const + 28 at Client.cpp:418
        frame 19: 0x00000000103b96d6 clickhouse`DB::copyDataToS3File()

</details>

This new behaviour could be enabled with `s3_disable_checksum=true`.

Note, that I've checked this implementation with GCS/R2/S3/MinIO and it
works everywhere.
 2023-11-26 19:20:19 +01:00
Alexey Milovidov
a9bb76378f
Merge pull request #56724 from canhld94/ch_replicated_column_mismatch 
Do not allow tables on different replicas have different aggregate functions in SimpleAggregateFunction column
 2023-11-26 17:20:10 +01:00
Alexey Milovidov
1cc33f3430
Merge pull request #56164 from azat/not-byte-identical-message 
Add more details to "Data after merge is not byte-identical to data on another replicas"
 2023-11-26 04:14:15 +01:00
Alexey Milovidov
304d6375be
Merge pull request #56225 from azat/rocksdb-compact 
Allow manual compaction of rocksdb via OPTIMIZE query
 2023-11-26 03:59:54 +01:00
Robert Schulze
5b7d2a903d
Merge pull request #57152 from rschu1ze/fine-granular-plan-opt-settings 
Fine-granular enablement/disabling of plan-level optimizations
 2023-11-25 20:44:23 +01:00
flynn
9f174b1c5e Support use alias column in indices 2023-11-25 15:40:22 +00:00
Vitaly Baranov
2e7f314599
Merge pull request #50209 from ilejn/merge_row_policy 
Engine Merge obeys row policy
 2023-11-25 10:34:22 +01:00
Sema Checherinda
bd36dce81b
Merge pull request #57129 from ClickHouse/fix_something 
Fix something in ReplicatedMergeTree
 2023-11-24 20:19:25 +01:00
Alexander Tokmakov
deda8483be
Merge pull request #56536 from ClickHouse/move-partition-inactive-replica 
Fix race between REPLACE_RANGE and GET_PART (set actual part name when fetching)
 2023-11-24 20:09:24 +01:00
Aleksandr Musorin
c3b617bd7b Move class functions upper 2023-11-24 19:00:00 +01:00
Raúl Marín
63fe821f7b Merge remote-tracking branch 'blessed/master' into backup_1 2023-11-24 16:21:59 +01:00
Raúl Marín
2539100afb Review improvements 2023-11-24 16:21:56 +01:00
Kseniia Sumarokova
13187bf4f1
Merge pull request #56959 from melvynator/master 
Add experimental flag for MaterializedPostgreSQL table engine.
 2023-11-24 16:16:42 +01:00
Kseniia Sumarokova
c25c6ec403
Merge pull request #57031 from aiven-sal/aiven-sal/fix_namedcoll2 
NamedCollections: make exception message more informative.
 2023-11-24 16:15:16 +01:00
alesapin
a3c2896e9d
Merge pull request #56846 from aalexfvk/add_cancellation_point_for_moving_background_operation 
Add cancellation hook for moving background operation
 2023-11-24 12:12:47 +01:00
Chen Lixiang
2b35e98ef8 add comments and doc 2023-11-24 16:40:19 +08:00
Chen Lixiang
89fa729235 show uncompressed size in system.tables 2023-11-24 15:54:27 +08:00
Alexey Milovidov
81a50703c3 Fix tests 2023-11-23 23:39:40 +01:00
Alexey Milovidov
ade67cc57e Merge branch 'master' of github.com:ClickHouse/ClickHouse into CurtizJ-fix-ttl-non-deterministic 2023-11-23 23:04:11 +01:00
Alexey Milovidov
d72bc854d1
Merge pull request #57074 from ClickHouse/less-allocation-in-arenas 
Avoid excessive allocation in Arena
 2023-11-23 23:02:26 +01:00
Igor Nikonov
0434044a7e
Merge pull request #57054 from ClickHouse/pr-cleanup-narrow-dependency 
Parallel replicas: cleanup, narrow dependency
 2023-11-23 20:12:37 +01:00
Robert Schulze
77d48e4d55
"optimise" --> "optimize". Sorry, friends from the UK. 2023-11-23 18:41:35 +00:00
Alexey Milovidov
c5fabcc472 Fix tests 2023-11-23 17:50:50 +01:00
Alexey Milovidov
dc0d6e432c Fix build 2023-11-23 16:11:54 +01:00
Alexey Milovidov
6bbf449808 Fix build 2023-11-23 16:06:35 +01:00
Alexey Milovidov
39ea0c426a Fix build 2023-11-23 16:05:15 +01:00
Alexey Milovidov
89656723fb Fix tests 2023-11-23 16:02:33 +01:00
Alexey Milovidov
93b789169b Merge branch 'master' into CurtizJ-fix-ttl-non-deterministic 2023-11-23 15:46:01 +01:00
Aleksei Filatov
1a03f5f7f4 Merge remote-tracking branch 'upstream/master' into add_cancellation_point_for_moving_background_operation 2023-11-23 16:43:33 +03:00
serxa
663c8cd944 Merge branch 'master' into dashboards-table 2023-11-23 10:29:48 +00:00
serxa
0fc25fbda8 Merge branch 'master' into async-loader-integration 2023-11-23 09:32:43 +00:00
Alexander Tokmakov
a0ce9f46cf fix 2023-11-22 23:15:20 +01:00
Alexey Milovidov
2ba2f48051 Merge branch 'master' of github.com:ClickHouse/ClickHouse into less-allocation-in-arenas 2023-11-22 22:11:59 +01:00
avogar
70f45c11bd Fix 2023-11-22 19:56:42 +00:00
Alexander Tokmakov
ef3e558a65 fix no such key for empty parts 2023-11-22 20:51:10 +01:00
Alexander Tokmakov
d9eb68ba35 fix unexpected outdated parts 2023-11-22 20:29:57 +01:00
Alexander Tokmakov
9f1a9826af fix replace/move partition 2023-11-22 20:04:14 +01:00
Alexander Tokmakov
86f7732c24
Update StorageReplicatedMergeTree.cpp 2023-11-22 19:41:27 +01:00
avogar
031f030102 Remove unused method 2023-11-22 18:27:32 +00:00
Kruglov Pavel
b84e3cf683
Merge branch 'master' into size-virtual-column 2023-11-22 19:25:00 +01:00
avogar
4a86f4a7b9 Fix style changes 2023-11-22 18:24:34 +00:00
avogar
007353a2dd Add _size virtual column to s3/file/hdfs/url/azureBlobStorage engines 2023-11-22 18:12:36 +00:00
Alexander Tokmakov
d277bfb225 setActualPartName for executeReplaceRange 2023-11-22 18:48:25 +01:00
vdimir
ffbe85d3a0
Merge pull request #56668 from ClickHouse/vdimir/analyzer_s3_partition_pruning 
Analyzer: filtering by virtual columns for StorageS3
 2023-11-22 16:44:44 +01:00
Duc Canh Le
44a3dc1cd8 add new method to check if two types are exactly identical 
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
 2023-11-22 15:02:34 +00:00
Dmitry Novik
7539928814
Merge pull request #56079 from ZhiguoZh/20231027-combine-filter-avx512 
Optimize DB::combineFilters with AVX512_VBMI2 intrinsic
 2023-11-22 15:42:33 +01:00
zvonand
5153798aeb Introduced fileCluster table function 
Added fileCluster function
Added test and docs
 2023-11-22 15:06:04 +01:00
Kruglov Pavel
9fb1acc947
Merge pull request #56810 from Avogar/iceberg-metadata-files 
Iceberg metadata files
 2023-11-22 14:40:04 +01:00
Alexey Milovidov
9cf7ebeced Minor changes 2023-11-22 12:13:47 +01:00
Raúl Marín
281060329f Merge remote-tracking branch 'blessed/master' into backup_1 2023-11-22 10:35:18 +01:00
Ilya Golshtein
c2816ec589 merge_row_policy: make clang-tidy happy 2023-11-21 22:06:18 +00:00
Dmitry Novik
1e0b4f979c
Add extra test 2023-11-21 22:55:55 +01:00
Kseniia Sumarokova
04aa0b10fc
Merge pull request #56990 from ClickHouse/s3queue-fix-incorrect-ref-count-from-exception-in-ctor 
S3Queue: fix metadata reference increment
 2023-11-21 21:37:23 +01:00
Alexander Tokmakov
990e3ea1c8 Merge branch 'master' into rmt_check_shutdown_flags_in_retry_loops 2023-11-21 18:01:16 +01:00
vdimir
a139ae97eb
Merge pull request #52918 from ClickHouse/vdimir/s3_blob_log 
Add system table with blob storage operations log
 2023-11-21 17:40:42 +01:00
Igor Nikonov
63a1625b77 Merge remote-tracking branch 'origin/master' into pr-cleanup-narrow-dependency 2023-11-21 16:05:48 +00:00
Igor Nikonov
ce98dfb251 Settings pointer to reference 2023-11-21 16:04:54 +00:00
Igor Nikonov
66f6a6575f Cleanup iteration: settings usage 2023-11-21 13:29:04 +00:00
Igor Nikonov
83c6ac4f6c Remove storage as parameter dependency, only columns are needed 2023-11-21 12:14:12 +00:00
alesapin
227bb30d02
Merge pull request #56986 from kevinmingtarja/implicit-constraint-collapsing-merged-tree 
Add implicit constraint for CollapsingMergeTree sign column
 2023-11-21 12:43:51 +01:00
kssenii
474f1ae0b6 Better 2023-11-21 11:42:23 +01:00
Aleksandr Musorin
ee0ae17ce2 Added optimize_trivial_approximate_count_query documentatiosn for RocksDB 2023-11-21 11:31:56 +01:00
Salvatore Mesoraca
f96cd4cfb8
Update src/Storages/NamedCollectionsHelpers.cpp 
Co-authored-by: Nikolay Degterinsky <43110995+evillique@users.noreply.github.com>
 2023-11-21 11:14:48 +01:00
Salvatore Mesoraca
970e34f32e
NamedCollections: make exception message more informative. 
Fix test 00002_log_and_exception_messages_formatting
 2023-11-21 10:21:28 +01:00
vdimir
15234474d7
Implement system table blob_storage_log 2023-11-21 09:18:25 +00:00
Amos Bird
ee1cec4d38
Allow RENAME of index columns. 2023-11-21 15:42:54 +08:00
Amos Bird
6581aaf9e8
Prevent incompatible ALTER of projection columns. 2023-11-21 15:42:54 +08:00
kevinyhzou
3adc8fdf78 Fix ci 2023-11-21 11:22:12 +08:00
Raúl Marín
04f966c471 Recover special handling of ephemeral nodes in ZooKeeperWithFaultInjection 2023-11-20 22:17:34 +01:00
Kseniia Sumarokova
2beee5d9cb
Merge branch 'master' into s3queue-fix-empty-key-check 2023-11-20 20:17:47 +01:00
jsc0218
3d5265c1d3 Merge remote-tracking branch 'origin/master' into SystemTablesFilterEngine 
merge master
 2023-11-20 18:10:54 +00:00
vdimir
31a6c7c1c4
Style changes around filterKeysForPartitionPruning 2023-11-20 18:08:45 +00:00
vdimir
95e9a27417
Remove ast based code from filterKeysForPartitionPruning 2023-11-20 17:59:58 +00:00
vdimir
a915eeded8
StorageS3 use filters from SourceStepWithFilter 2023-11-20 17:59:58 +00:00
vdimir
1cddfb1e6b
rewrite getBlockWithVirtuals for S3Storage 2023-11-20 17:59:57 +00:00
vdimir
cbb2e02c03
Analyzer: partition pruning for S3 2023-11-20 17:59:53 +00:00
Aleksandr Musorin
38133692bb Add total_bytes for RocksDB storage 
Show total_bytes in system tables for RocksDB storage

Previously it did not work because total_rows was calculated only when optimize_trivial_approximate_count_query=1
 2023-11-20 17:08:35 +01:00
Kevin Mingtarja
1d668264e1 Fix style 2023-11-20 23:54:36 +08:00
Kevin Mingtarja
d84d5692ef Enable implicit constraint for VersionedCollapsing 2023-11-20 23:52:22 +08:00
Zhiguo Zhou
be0f0b6879 Fix code style 2023-11-20 23:34:52 +08:00
kssenii
9b5e180995 Fix 2023-11-20 16:27:02 +01:00
avogar
f537bad469 Merge branch 'master' of github.com:ClickHouse/ClickHouse into schema-inference-union 2023-11-20 14:32:50 +00:00
avogar
872556a5d4 Merge branch 'master' of github.com:ClickHouse/ClickHouse into schema-inference-union 2023-11-20 14:03:36 +00:00
Sema Checherinda
f999337dae
Revert "Revert "s3 adaptive timeouts"" 2023-11-20 14:53:22 +01:00
Alexander Tokmakov
5031f239c3
Revert "s3 adaptive timeouts" 2023-11-20 14:28:59 +01:00
Raúl Marín
8217915064 Replace part_committed_locally_but_zookeeper with retries 2023-11-20 13:55:50 +01:00
kssenii
1ade4b797b Add a comment 2023-11-20 12:48:49 +01:00
kssenii
10fb40ece9 Fix 2023-11-20 12:44:17 +01:00
alesapin
857ce2d880
Merge branch 'master' into implicit-constraint-collapsing-merged-tree 2023-11-20 12:28:55 +01:00
Raúl Marín
aadb786b2b Merge remote-tracking branch 'blessed/master' into backup_1 2023-11-20 10:53:31 +01:00
Zhiguo Zhou
1f044b11e3 Enhance unittest MergeTree.CombineFilters 2023-11-20 16:00:14 +08:00
Kevin Mingtarja
af0c954c9e Add implicit constraint for CollapsingMergeTree sign column 2023-11-20 15:05:12 +08:00
melvynator
cfdb663892 Remove a tab 2023-11-20 00:46:14 +01:00
Nikolay Degterinsky
9683adfb03
Merge pull request #56830 from evillique/better-add-column 
Fix flattening existing Nested columns during ADD COLUMN
 2023-11-19 16:21:56 +01:00
Sema Checherinda
a950595c24
Merge pull request #56314 from CheSema/s3-aggressive-timeouts 
s3 adaptive timeouts
 2023-11-19 14:12:14 +01:00
serxa
9023f61349 Merge branch 'master' into async-loader-integration 2023-11-19 13:05:58 +00:00
Alexey Milovidov
edc3b2fe48
Merge pull request #56958 from ClickHouse/metric-queued-jobs 
Add metrics for the number of queued jobs, which is useful for the IO thread pool
 2023-11-19 10:37:18 +01:00
jsc0218
0171d61fa5 Add MySQL Engine Filter 2023-11-19 02:34:14 +00:00
Sema Checherinda
c465933b1d
Merge pull request #56808 from CheSema/retry-fetch-part 
Fetching a part waits when that part is fully committed on remote replica
 2023-11-19 01:57:25 +01:00
melvynator
3ade60a261 Keep trying to fix style 2023-11-19 01:31:21 +01:00
melvynator
88b0525306 Style check 2023-11-19 01:10:25 +01:00
melvynator
5f0f9b0ad4 Test check style 2023-11-19 01:05:11 +01:00
Ubuntu
ddd9a23116 Fix newline 2023-11-19 00:02:33 +00:00
Ubuntu
89b93adf82 Fix trailing whitespace 2023-11-18 23:55:46 +00:00
Ilya Golshtein
a8abec6cac merge_row_policy: cleanup after merge 2023-11-18 21:45:17 +00:00
Antonio Andelic
274f2c71f2
Merge pull request #56460 from ClickHouse/keeper-map-backup-restore 
Backup/Restore for KeeperMap tables
 2023-11-18 19:27:36 +01:00
Ubuntu
16e7bd201b Add experimental flag for PostgresSQL table engine. Also add documentation on how to enable it 2023-11-18 18:22:33 +00:00
Alexey Milovidov
d56cbda185 Add metrics for the number of queued jobs, which is useful for the IO thread pool 2023-11-18 19:07:59 +01:00
Nikolay Degterinsky
a18b715591 Fix a bug 2023-11-18 15:44:45 +00:00
Raúl Marín
210a0ee045 Merge remote-tracking branch 'blessed/master' into backup_1 2023-11-17 18:50:24 +01:00
Sema Checherinda
29e64347b9 improve exception message 2023-11-17 17:37:14 +01:00
jsc0218
5b20118c66 Fix Typo 2023-11-17 14:55:36 +00:00
jsc0218
fb96f65904 Fix Style 2023-11-17 14:15:33 +00:00
Raúl Marín
3633e77a37 Refactor ZooKeeperWithFaultInjection 2023-11-17 12:57:32 +01:00
Sema Checherinda
86119dbc3f fix data race 2023-11-17 12:26:50 +01:00
Kseniia Sumarokova
3322f54713
Merge branch 'master' into remove-unused-code 2023-11-17 11:55:02 +01:00
Antonio Andelic
9bcedf3764 Cleanup 2023-11-17 10:27:19 +00:00
Antonio Andelic
803d9668e2 Merge branch 'master' into keeper-map-backup-restore 2023-11-17 10:11:59 +00:00
Ilya Golshtein
9eeae568e4 Merge remote-tracking branch 'origin/master' into merge_row_policy 
merge_row_policy: changes because of refactoring in master
 2023-11-17 09:19:56 +00:00
Aleksei Filatov
163b8a814e Fix clang-tidy issues and disable parallel execution of tests 2023-11-17 11:02:07 +03:00
jsc0218
bd5a715244 Fix Style 2023-11-17 03:26:36 +00:00
jsc0218
4c73b870b4 Fix the Error by Only Filter PostgreSQL 2023-11-17 03:01:25 +00:00
Alexey Milovidov
482d8ca9d8
Merge pull request #56102 from ClickHouse/coverage 
Granular code coverage with introspection
 2023-11-16 23:23:40 +01:00
Alexander Tokmakov
dfd606ef7e
Update StorageReplicatedMergeTree.cpp 2023-11-16 20:53:45 +01:00
Nikolai Kochetov
ca6de31ce5
Merge pull request #56695 from ClickHouse/table-function-input-in-subquery 
Fix duplicate usage of table function input().
 2023-11-16 20:48:10 +01:00
Kseniia Sumarokova
a28d541ba5
Merge branch 'master' into remove-unused-code 2023-11-16 17:17:51 +01:00
Raúl Marín
19931fee67 Prepare the introduction of more keeper faults 2023-11-16 15:46:39 +01:00
Sema Checherinda
ddca2c2187 server side waiting 2023-11-16 15:44:14 +01:00
Robert Schulze
ed052d76e6
Merge pull request #56141 from zvonand/zvonand-more-globs 
Performance enhancement for File, HDFS globs
 2023-11-16 14:09:22 +01:00
alesapin
ddcfecacc2
Merge pull request #56000 from evillique/fix-replica-already-exists-storage 
Fix `REPLICA_ALREADY_EXISTS` for ReplicatedMergeTree
 2023-11-16 12:51:50 +01:00
alesapin
437a911d7b
Merge pull request #56809 from ClickHouse/add_async_moves_execution 
Implement asynchronous execute of `ALTER TABLE ... MOVE ... TO DISK|VOLUME`
 2023-11-16 12:47:31 +01:00
Antonio Andelic
203f83c2c2 Merge branch 'master' into keeper-map-backup-restore 2023-11-16 11:31:11 +00:00
Aleksei Filatov
21a17f8334 Add cancellation hook for moving background operation 2023-11-16 13:37:51 +03:00
Andrey Zvonov
fc15b0c1e7
Merge branch 'ClickHouse:master' into zvonand-more-globs 2023-11-16 11:19:12 +01:00
Antonio Andelic
352eca7aa0
Merge pull request #56789 from ClickHouse/fix-no-user 
Try fix `No user in current context, it's a bug`
 2023-11-16 08:20:52 +01:00
Nikolay Degterinsky
7623153d38 Fix flattening existing Nested columns during ADD COLUMN 2023-11-16 07:01:49 +00:00
zvonand
4ff82528f1 small fixes after review 2023-11-16 00:01:36 +01:00
Alexander Tokmakov
c529e64768 Merge branch 'master' into rmt_check_shutdown_flags_in_retry_loops 2023-11-15 23:40:22 +01:00
zvonand
cb74f27b00 Merge branch 'master' of github.com:ClickHouse/ClickHouse into zvonand-more-globs 2023-11-15 21:56:51 +01:00
jsc0218
0d2acf9eec Able to Filter Engine When Scanning System Tables 2023-11-15 20:30:36 +00:00
Sema Checherinda
f3f8392051 upgrade replication protocol 2023-11-15 19:03:55 +01:00
avogar
28522d56b7 Support Iceberg metadata files for metastore tables 2023-11-15 17:45:07 +00:00
Alexander Sapin
3e26161eaa Implement asynchronous excute of ALTER TABLE ... MOVE ... TO DISK|VOLUME 2023-11-15 18:33:59 +01:00
Nikolay Degterinsky
57aa049fef Change table name to table UUID, fix test 2023-11-15 16:21:13 +00:00
Alexander Tokmakov
bf10989515
Fix unexpected parts handling (#56693) 
* fix unexpected parts handling

* Automatic style fix

* fix

---------

Co-authored-by: robot-clickhouse <robot-clickhouse@users.noreply.github.com>
 2023-11-15 16:54:47 +01:00
zvonand
a8aa206cfd Update after review 
expandSelector is now returning vector<string> directly
minor changes
 2023-11-15 16:37:24 +01:00
zvonand
70aa6e2672 Added a shortcut for no-wildcard globs in HDFS 2023-11-15 16:37:24 +01:00
zvonand
3e47a54a94 Added a shortcut for no-wildcard globs in file storage 2023-11-15 16:37:24 +01:00
Antonio Andelic
5c4e58d27d Try fix No user 2023-11-15 15:14:46 +00:00
Nikolai Kochetov
62378b4219 Fixing style. 2023-11-15 13:56:51 +00:00
Nikolai Kochetov
6ef5c4cafe
Merge pull request #56522 from kitaisreal/analyzer-support-explain-estimate 
Analyzer support EXPLAIN ESTIMATE
 2023-11-15 13:17:31 +01:00
Antonio Andelic
d567e718eb Merge branch 'master' into keeper-map-backup-restore 2023-11-15 11:51:27 +00:00
serxa
bd66ac483c Load queries for dashboard page from new system.dashboards table 2023-11-14 23:03:06 +00:00
Nikolai Kochetov
1356dc2eaa Fixing style. 2023-11-14 14:55:21 +00:00
Sema Checherinda
b9cc1580e0
Merge pull request #56367 from canhld94/ch_table_reinit_new_disk 
Adding new disk to storage configuration without restart
 2023-11-14 15:54:22 +01:00
Nikolai Kochetov
d5907e10de Fixing tests. 2023-11-14 12:51:25 +00:00
Ilya Golshtein
3e6a5d8208 merge_row_policy: some comments 2023-11-14 12:22:32 +00:00
Kseniia Sumarokova
0676c09d72
Merge branch 'master' into remove-unused-code 2023-11-14 11:59:53 +01:00
Sema Checherinda
8d36fd6e54 get rid off of client_with_long_timeout_ptr 2023-11-14 11:34:12 +01:00
Sema Checherinda
27fb25d056 alter the naming, fix client_with_long_timeout in s3 storage 2023-11-14 11:34:12 +01:00
Robert Schulze
2931bbfe62
Merge pull request #56665 from rschu1ze/refactor-ast-hash-calculation 
Small refactoring of AST hash calculation (follow-up to #56545)
 2023-11-14 11:24:30 +01:00
Antonio Andelic
29f54db917 Merge branch 'master' into keeper-map-backup-restore 2023-11-14 09:47:54 +00:00
Alexander Gololobov
80ca41f18d
Merge pull request #56420 from ClickHouse/system_replicas_with_many_tables 
Optimizations for system.replicas with many tables
 2023-11-14 10:23:48 +01:00
Duc Canh Le
3152cd25dc only check for exact column definitions in replicated merge tree 
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
 2023-11-14 07:41:45 +00:00
Duc Canh Le
57b160b6b9 compare type name in column description 
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
 2023-11-14 07:12:12 +00:00
Alexey Milovidov
3ef14f6098 Merge branch 'master' of github.com:ClickHouse/ClickHouse into coverage 2023-11-14 06:08:32 +01:00
Nikolai Kochetov
4004248c13 Fix duplicate usage of table function input(). 2023-11-13 19:01:36 +00:00
Alexey Milovidov
c81cc3f953
Merge pull request #56660 from ClickHouse/better-ccache-2 
Better use of build cache
 2023-11-13 19:28:33 +01:00
Alexander Gololobov
b49f50bf47 Set DB:Exception on unscheduled requests 2023-11-13 17:53:38 +01:00
Alexander Gololobov
6e861a6176 Handle query cancellation 2023-11-13 17:53:38 +01:00
Alexander Gololobov
1113b3041b Return exception from the job 2023-11-13 17:53:38 +01:00
Alexander Gololobov
c27913f09a Separate thread pools for request with and w/o zk 2023-11-13 17:53:38 +01:00
Alexander Gololobov
45f5d28299 Deduplicate concurrent requests in system.replicas 2023-11-13 17:53:38 +01:00
Nikolay Degterinsky
b8cba418e3
Merge pull request #56493 from evillique/fix-alter-column-alias 
Fix ALTER COLUMN with ALIAS
 2023-11-13 17:48:06 +01:00
Robert Schulze
d6df7d8403
Merge remote-tracking branch 'rschu1ze/master' into refactor-ast-hash-calculation 2023-11-13 15:54:21 +00:00
Antonio Andelic
9e91e4d671 Define BackupReferenceEntry 2023-11-13 14:43:02 +00:00
Sema Checherinda
7f3a082c0e
Merge pull request #56456 from ucasfl/fix-mysql 
Fix transfer query to MySQL compatible query
 2023-11-13 15:25:58 +01:00
kssenii
876b2cd7bc Remove unused code 2023-11-13 15:10:57 +01:00
Antonio Andelic
59480205d4 Merge branch 'master' into keeper-map-backup-restore 2023-11-13 12:21:26 +00:00
Alexey Milovidov
7a489059ac
Merge pull request #56350 from ClickHouse/system-symbols 
Remove garbage.
 2023-11-13 12:25:18 +01:00
Robert Schulze
50cd59a72b
Remove default value for argument 'ignore_aliases' from IAST::getTreeHash() 2023-11-13 10:27:38 +00:00
Alexey Milovidov
8e820185cb Merge branch 'master' of github.com:ClickHouse/ClickHouse into better-ccache-2 2023-11-13 11:17:18 +01:00
Alexey Milovidov
82c461031e Fix build 2023-11-13 10:09:23 +01:00
Kseniia Sumarokova
b7915e6983
Merge pull request #55782 from aiven-sal/aiven-sal/namedcoll_pr 
NamedCollections: prevent fields overriding in functions' calls
 2023-11-13 09:07:09 +01:00
Alexey Milovidov
58319f08be Merge branch 'master' into system-symbols 2023-11-11 04:12:37 +01:00
Alexey Milovidov
fcd45d47e4 Merge branch 'master' of github.com:ClickHouse/ClickHouse into remove-cpp-templates-2 2023-11-11 03:45:54 +01:00
Alexey Milovidov
c095b1e9ef
Merge pull request #56502 from amosbird/fix-56481 
Fix two cases of projection analysis.
 2023-11-11 02:29:04 +01:00
Alexey Milovidov
3bec4dce8e Merge branch 'master' into remove-cpp-templates-2 2023-11-11 00:50:32 +01:00
Maksim Kita
bf08f3ef5a
Revert "Inserting only non-duplicate chunks in MV" 2023-11-10 21:21:13 +03:00
Kseniia Sumarokova
2709ecd50d
Merge pull request #56457 from ClickHouse/fix-test-database-hdfs 
Fix test `02725_database_hdfs.sh`
 2023-11-10 11:40:18 +01:00
Kseniia Sumarokova
e48df94772
Merge pull request #56377 from ClickHouse/minor-improvements-for-s3-queue 
Minor improvements for S3Queue
 2023-11-10 11:28:09 +01:00
Antonio Andelic
5fea9f9dc6 Small fixes 2023-11-10 09:27:26 +00:00
Alexey Milovidov
e5df9de12b Merge branch 'remove-cpp-templates-2' into system-symbols 2023-11-10 05:28:45 +01:00
Alexey Milovidov
8c253b9e3e Remove C++ templates 2023-11-10 05:25:02 +01:00
Amos Bird
16578d97a3
Fix two cases of projection analysis. 
1. Fix projection analysis with partitions.
2. Apply alter conversions during projection analysis.
 2023-11-10 10:23:02 +08:00
Alexey Milovidov
29b75d034a Fix FreeBSD 2023-11-09 23:43:12 +01:00
Alexey Milovidov
2c220ba5ec Fix FreeBSD 2023-11-09 23:43:12 +01:00
Alexey Milovidov
84ada59e42 Fix FreeBSD 2023-11-09 23:43:12 +01:00
Alexey Milovidov
8f67894157 Add table system.symbols 2023-11-09 23:43:12 +01:00
Alexey Milovidov
cc39484f7f Merge branch 'master' into system-symbols 2023-11-09 23:05:55 +01:00
Maksim Kita
6fb7d44b62 Analyzer support EXPLAIN ESTIMATE 2023-11-09 19:43:14 +03:00
Salvatore Mesoraca
178f2bbee5
Fix format 2023-11-09 17:25:20 +01:00
Salvatore Mesoraca
1174689f8b
Refactoring 2023-11-09 17:25:20 +01:00
Salvatore Mesoraca
b8e478195f
Simplify code 2023-11-09 17:25:19 +01:00
Salvatore Mesoraca
52e49f4ce3
Add explanatory comments 2023-11-09 17:25:19 +01:00
Salvatore Mesoraca
45e4e122fe
Update src/Common/NamedCollections/NamedCollections.cpp 
Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>
 2023-11-09 17:25:19 +01:00
Salvatore Mesoraca
7d206cbc3c
NamedCollections: prevent fields overriding in functions call 
Add syntax in SQL and XML to mark specific fields to allow
override or not.
Also add a new setting to control the default behaviour when
overriding support is not specified.
 2023-11-09 17:25:16 +01:00
Antonio Andelic
f9895ab37b Small fixes and add test 2023-11-09 15:56:57 +00:00
Alexander Tokmakov
113add2556 rmt: check shutdown flags in retry loops 2023-11-09 16:41:01 +01:00
Nikolai Kochetov
c643069b0d
Merge pull request #54905 from nickitat/fix_key_analysis_merge_engine 
Fix key analysis (with set) for `Merge` engine
 2023-11-09 14:52:54 +01:00
Kseniia Sumarokova
49f9b6d340
Merge pull request #56260 from joelynch/delta-lake-repeated-add 
deltalake: Do not raise errors when processing add and remove actions
 2023-11-09 13:15:31 +01:00
Kseniia Sumarokova
0760e69e54
Merge pull request #56312 from ClickHouse/parallelize-backup-entries-collector 
Parallelize `BackupEntriesCollector`
 2023-11-09 13:07:48 +01:00
Kseniia Sumarokova
a9d8e4170f
Merge pull request #56387 from ClickHouse/fix-backup-of-mv-with-no-inner-table 
Allow backup of mat view with dropped inner table instead of failing the backup
 2023-11-09 13:02:41 +01:00
Nikolai Kochetov
f7e7089bc9
Merge pull request #56391 from ClickHouse/rocks-db-analyzer 
Fix rocksdb with analyzer.
 2023-11-09 12:48:18 +01:00
Nikolay Degterinsky
4ac3dcc417 Fix ALTER COLUMN with ALIAS 2023-11-09 07:07:41 +00:00
flynn
41cdd5dd1a Fix 2023-11-09 06:23:23 +00:00
Ilya Golshtein
9c5f9f7dcb merge_row_policy: style fix, cleanup 2023-11-09 00:51:57 +03:00
Ilya Golshtein
f1af83447e merge_row_policy: cleanup, traces removed 2023-11-08 23:07:08 +03:00
Alexander Gololobov
517a184d5a
Make IMergeTreeDataPart::getState() inlinable (#56461) 2023-11-08 20:30:45 +01:00
flynn
4e263956e0 Merge branch 'fix-mysql' of github.com:ucasfl/ClickHouse into fix-mysql 2023-11-08 16:22:16 +00:00
flynn
ca18674f1a update test 2023-11-08 16:21:16 +00:00
Alexey Milovidov
68d3f60ef7 Merge branch 'master' into system-symbols 2023-11-08 17:07:11 +01:00
Ilya Golshtein
6e189b17c0 merge_row_policy: processAliases introduced 2023-11-08 18:55:19 +03:00
flynn
d0ba561faa
Update src/Storages/transformQueryForExternalDatabase.cpp 
Co-authored-by: Sema Checherinda <104093494+CheSema@users.noreply.github.com>
 2023-11-08 23:20:12 +08:00
Antonio Andelic
4438c2f70a Remove unnecassary log 2023-11-08 14:36:39 +00:00
Antonio Andelic
1f000242a1 Merge branch 'master' into keeper-map-backup-restore 2023-11-08 13:16:47 +00:00
Antonio Andelic
18a5eeec38 Make on cluster backup/restore work 2023-11-08 13:16:38 +00:00
kssenii
a5b9ba51cd Better test 2023-11-08 12:55:14 +01:00
flynn
399d61b222 Fix transfer query to MySQL compatible query 2023-11-08 11:34:49 +00:00
kssenii
dfcc94fff0 Fix test 2023-11-08 12:29:40 +01:00
Antonio Andelic
86ba6ad1e8 Local backup and restore 2023-11-08 10:22:44 +00:00
Alexey Milovidov
ca83da14f2 Merge branch 'master' of github.com:ClickHouse/ClickHouse into system-symbols 2023-11-07 19:43:19 +01:00
slvrtrn
f663228404 Add information_schema.statistics 2023-11-07 18:59:45 +01:00
kssenii
15517d04df Fix 2023-11-07 18:02:44 +01:00
kssenii
70048236f3 Add ProfileEvents column to system.backups 2023-11-07 17:32:08 +01:00
kssenii
9dce1a52e1 Review fix 2023-11-07 16:31:41 +01:00
Alexey Milovidov
5169e4e54d Fix FreeBSD 2023-11-07 15:30:31 +01:00
Alexander Tokmakov
706f2dcf54
Add a metric for suspicious parts in ZooKeeper (#56395) 2023-11-07 12:55:57 +01:00
Nikolai Kochetov
c58c583043 Fix tests. 2023-11-07 11:43:16 +00:00
Nikita Mikhaylov
0555f8a68c
Fix crash in case of adding a column with type Object(JSON) (#56307) 2023-11-07 12:01:52 +01:00
serxa
19aa1375fc Merge branch 'master' into async-loader-integration 2023-11-07 10:25:38 +00:00
Nikita Mikhaylov
b696540a5a
Merge branch 'master' into rocks-db-analyzer 2023-11-07 00:44:49 +01:00
Vitaly Baranov
1f4697fbb7
Merge pull request #56339 from vitlibar/fix-using-table-shared-id-during-backup-and-improve-logs 
Fix using table shared id during backup and improve logs.
 2023-11-07 00:04:59 +01:00
Alexey Milovidov
9e6dee93f3 Fix FreeBSD 2023-11-06 23:26:11 +01:00
Alexander Tokmakov
cf6ab2e110
Merge pull request #56282 from ClickHouse/fix_unexpected_parts_after_truncate 
Fix unexpected parts after drop range
 2023-11-06 21:03:57 +01:00
Nikolai Kochetov
34fdb8a7c6 Fix rocksdb with analyzer. 2023-11-06 18:46:39 +00:00
kssenii
7883a74f6d Fix 2023-11-06 19:05:12 +01:00
Sema Checherinda
a5e17dc919
Merge pull request #55566 from CheSema/remote-blobs-naming 
feature: storage_metadata_write_full_object_key
 2023-11-06 16:41:49 +01:00
Alexey Milovidov
66b180d751 Fix FreeBSD 2023-11-06 16:31:02 +01:00
kssenii
f2c0434c4d Merge remote-tracking branch 'origin/master' into minor-improvements-for-s3-queue 2023-11-06 15:51:11 +01:00
kssenii
8db2aca832 Add a test 2023-11-06 15:48:04 +01:00
kssenii
da21413354 Better shutdown 2023-11-06 15:47:57 +01:00
kssenii
9952261a59 Add missing write to s3queue_log, pass cached FileStatus 2023-11-06 15:47:45 +01:00
Alexander Tokmakov
13599ff462 review fixes 2023-11-06 15:31:54 +01:00
Alexander Tokmakov
eba52b318d
Merge pull request #56272 from ClickHouse/rdb_fix_rare_logical_error 
Fix rare logical error in Replicated database
 2023-11-06 14:14:55 +01:00
Ilya Golshtein
d900f8a352 merge_row_policy: aliases slightly work 2023-11-06 15:18:25 +03:00
Sema Checherinda
52850d5a45
Update StorageSystemRemoteDataPaths.cpp 2023-11-06 11:45:17 +01:00
Kseniia Sumarokova
a050cdd897
Merge branch 'master' into delta-lake-repeated-add 2023-11-06 11:30:58 +01:00
Duc Canh Le
4c21ba7b6f tables auto initialize new disks without restart 
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
 2023-11-06 10:26:48 +00:00
Alexey Milovidov
99fe7a46fc Minor changes 2023-11-06 01:12:13 +01:00
Alexey Milovidov
8182d51978 Add table system.symbols 2023-11-06 01:11:53 +01:00
Vitaly Baranov
1e89e7a6b2 Fix using table shared id during backup and improve logs. 2023-11-05 09:38:08 +01:00
Igor Nikonov
9bdcba1b20
Merge branch 'master' into pr-progress-bar 2023-11-05 00:09:46 +01:00
Alexander Tokmakov
155bdfcf0d fix 2023-11-04 22:50:19 +01:00
Alexey Milovidov
cd94d02455 Merge branch 'master' into coverage 2023-11-04 21:49:59 +01:00
Arthur Passos
b6e205dcdf
Add ClickHouse setting to disable tunneling for HTTPS requests over HTTP proxy (#55033) 
* initial commit. integ tests passing, need to re-run unit & my own personal tests

* partial refactoring to remove Protocol::ANY

* improve naming

* remove all usages of ProxyConfiguration::Protocol::ANY

* fix ut

* blabla

* support url functions as well

* support for HTTPS requests over HTTP proxy with tunneling off

* remove gtestabc

* fix silly mistake

* ...

* remove usages of httpclientsession::proxyconfig in src/

* got you

* remove stale comment

* it seems like I need reasonable defaults

* fix ut

* add some comments

* remove no longer needed header

* matrix out

* add https over http proxy with no tunneling

* soem docs

* partial refactoring

* rename to use_tunneling_for_https_requests_over_http_proxy

* improve docs

* use shorter version

* remove useless test

* rename the setting

* update

* fix typo

* fix setting docs typo

* move ); up

* move ) up
 2023-11-04 13:47:52 -04:00
Alexander Tokmakov
9acd3707cb fix tests 2023-11-04 15:44:55 +01:00
Nikita Taranov
26d3a3a1e0
Add assertion that SizePredictor is set if preferred_block_size_bytes is set (#56302) 
* Test

* fix
 2023-11-03 23:13:18 +01:00
alesapin
f69796471c
Merge pull request #56303 from ClickHouse/fix_obscure_segfault 
Fix rare race condition related to Memory allocation failure
 2023-11-03 23:00:11 +01:00
Alexander Tokmakov
a44e27b0dc fix 2023-11-03 22:28:10 +01:00
Nikita Mikhaylov
000d6cd7e5
Merge pull request #56234 from evillique/fix-suggestions 
Fix client suggestions for user without grants
 2023-11-03 17:55:51 +01:00
Alexander Sapin
3b212a2593 Reserver in advance 2023-11-03 16:53:52 +01:00
Alexander Sapin
a0a0dd02d5 Fix typo everywhere 2023-11-03 16:40:50 +01:00
Alexander Sapin
c3dcb81a62 Fix race condition in case of memory errors 2023-11-03 16:00:16 +01:00
Alexander Sapin
a9ffb9808e Merge branch 'master' into fix_obscure_segfault 2023-11-03 12:20:52 +01:00
Kseniia Sumarokova
31ae602d37
Merge pull request #56131 from ClickHouse/backup-some-more-retries 
Add missing zookeeper retries in StorageReplicatedMergeTree::backupData
 2023-11-03 12:00:41 +01:00
Alexander Tokmakov
b50c4c9f3b fix 2023-11-03 11:24:10 +01:00
vdimir
86524fce0f
Merge pull request #56246 from ClickHouse/vdimir/analyzer_01019_alter_materialized_view 
Analyzer: fix 01019_alter_materialized_view_consistent
 2023-11-03 11:04:14 +01:00
vdimir
a70d0c5203
Merge pull request #56264 from azat/rocksdb/block-based-table-options 
Add ability to specify BlockBasedTableOptions for rocksdb
 2023-11-03 10:15:19 +01:00
vdimir
55a1baf029
Merge pull request #56022 from ClickHouse/vdimir/check_all_tables 
Add CHECK query for all the tables
 2023-11-03 10:10:42 +01:00
Alexander Tokmakov
480e58b427 fix unexpected parts after drop range 2023-11-03 01:47:50 +01:00
alesapin
2daff2d41e Fxi 2023-11-03 00:32:07 +01:00
Alexey Milovidov
b400a6682e
Merge pull request #56275 from ClickHouse/23.11-prepare 
Update version after release
 2023-11-02 20:20:03 +01:00
Alexey Milovidov
1348d4c696 Update autogenerated version to 23.11.1.1 and contributors 2023-11-02 20:18:15 +01:00
Alexander Tokmakov
e88fa10aa2
Merge pull request #56142 from ClickHouse/make_replicated_start_more_robust 
Better process broken parts on table start for replicated tables
 2023-11-02 20:11:22 +01:00
Alexander Tokmakov
0ce1560e5d retry if table replica being restarted 2023-11-02 18:30:32 +01:00
serxa
e97edf5285 Merge branch 'master' into async-loader-integration 2023-11-02 14:56:32 +00:00
Robert Schulze
99dd3ed060
Merge pull request #56226 from HarryLeeIBM/hlee-ftsearch-compress 
Add compression of posting lists for inverted index
 2023-11-02 15:56:30 +01:00
Azat Khuzhin
decf85baca Add ability to specify BlockBasedTableOptions for rocksdb 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-11-02 15:36:00 +01:00
Sema Checherinda
90b64bcdb9 backward compatibility and implementation feature storage_metadata_write_full_object_key 2023-11-02 15:20:55 +01:00
kssenii
d0bc77f084 Fxi 2023-11-02 14:45:15 +01:00
Joe Lynch
67e29d5eb9
Allow repeated adds and removes for delta lake files 2023-11-02 13:40:50 +01:00
alesapin
3ba12b6e3e
Merge pull request #56238 from evillique/remove-less-parts 
Do not remove part if `Too many open files` is thrown
 2023-11-02 12:33:02 +01:00
Alexey Milovidov
1267246785
Merge pull request #55642 from ClickHouse/morton-utils 
Indexing by space-filling curves
 2023-11-02 12:21:33 +01:00
alesapin
5c4915e264
Update src/Storages/MergeTree/MergeTreePartInfo.cpp 
Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
 2023-11-02 12:02:04 +01:00
alesapin
7917a2cb40
Update src/Storages/MergeTree/MergeTreePartInfo.cpp 
Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
 2023-11-02 12:01:58 +01:00
alesapin
800f2f097c
Update src/Storages/MergeTree/MergeTreePartInfo.cpp 
Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
 2023-11-02 12:01:51 +01:00
alesapin
81b93ef12b
Update src/Storages/MergeTree/MergeTreePartInfo.cpp 
Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
 2023-11-02 12:01:46 +01:00
vdimir
5703e0995b
Analyzer: fix 01019_alter_materialized_view_consistent 2023-11-02 10:27:22 +00:00
Nikolay Degterinsky
0d6a0d04ae Do not remove parts if 'Too many open files' is thrown 2023-11-02 05:01:53 +00:00
Nikolay Degterinsky
f93c032425 Fix suggestions for user without grants 2023-11-02 03:56:59 +00:00
Nikolay Degterinsky
1af52d3b00 Better 2023-11-02 00:42:57 +00:00
Igor Nikonov
823b62a55c
Merge pull request #56162 from ClickHouse/part-check-scheduling-monotonic-time 
Use monotonic clock for part check scheduling
 2023-11-01 23:15:27 +01:00
Azat Khuzhin
09c1e76982 Allow manual compaction of rocksdb via OPTIMIZE query 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-11-01 22:27:11 +01:00
HarryLeeIBM
013e4b40c6 Add compression of posting lists for inverted index 2023-11-01 14:14:38 -07:00
Igor Nikonov
447386edd9
Merge branch 'master' into fix-schedule-delayed-part-checks 2023-11-01 21:26:06 +01:00
Duc Canh Le
7d4c97e8f3
print broken part path before throw (#56181) 
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
 2023-11-01 19:11:23 +01:00
Alexander Sapin
1e0cab93f6 Satisfy tidy 2023-11-01 16:51:20 +01:00
Alexander Sapin
bfcd8b7de5 Better logging and tests 2023-11-01 16:50:02 +01:00
Azat Khuzhin
c25d6cd624
Rename directory monitor concept into background INSERT (#55978) 
* Limit log frequence for "Skipping send data over distributed table" message

After SYSTEM STOP DISTRIBUTED SENDS it will constantly print this
message.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Rename directory monitor concept into async INSERT

Rename the following query settings (with preserving backward
compatiblity, by keeping old name as an alias):
- distributed_directory_monitor_sleep_time_ms -> distributed_async_insert_sleep_time_ms
- distributed_directory_monitor_max_sleep_time_ms -> distributed_async_insert_max_sleep_time_ms
- distributed_directory_monitor_batch -> distributed_async_insert_batch_inserts
- distributed_directory_monitor_split_batch_on_failure -> distributed_async_insert_split_batch_on_failure

Rename the following table settings (with preserving backward
compatiblity, by keeping old name as an alias):
- monitor_batch_inserts -> async_insert_batch
- monitor_split_batch_on_failure -> async_insert_split_batch_on_failure
- directory_monitor_sleep_time_ms -> async_insert_sleep_time_ms
- directory_monitor_max_sleep_time_ms -> async_insert_max_sleep_time_ms

And also update all the references:

    $ gg -e directory_monitor_ -e monitor_ tests docs | cut -d: -f1 | sort -u | xargs sed -e 's/distributed_directory_monitor_sleep_time_ms/distributed_async_insert_sleep_time_ms/g' -e 's/distributed_directory_monitor_max_sleep_time_ms/distributed_async_insert_max_sleep_time_ms/g' -e 's/distributed_directory_monitor_batch_inserts/distributed_async_insert_batch/g' -e 's/distributed_directory_monitor_split_batch_on_failure/distributed_async_insert_split_batch_on_failure/g' -e 's/monitor_batch_inserts/async_insert_batch/g' -e 's/monitor_split_batch_on_failure/async_insert_split_batch_on_failure/g' -e 's/monitor_sleep_time_ms/async_insert_sleep_time_ms/g' -e 's/monitor_max_sleep_time_ms/async_insert_max_sleep_time_ms/g' -i

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Rename async_insert for Distributed into background_insert

This will avoid amigibuity between general async INSERT's and INSERT
into Distributed, which are indeed background, so new term express it
even better.

Mostly done with:

    $ git di HEAD^ --name-only | xargs sed -i -e 's/distributed_async_insert/distributed_background_insert/g' -e 's/async_insert_batch/background_insert_batch/g' -e 's/async_insert_split_batch_on_failure/background_insert_split_batch_on_failure/g' -e 's/async_insert_sleep_time_ms/background_insert_sleep_time_ms/g' -e 's/async_insert_max_sleep_time_ms/background_insert_max_sleep_time_ms/g'

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Mark 02417_opentelemetry_insert_on_distributed_table as long

CI: https://s3.amazonaws.com/clickhouse-test-reports/55978/7a6abb03a0b507e29e999cb7e04f246a119c6f28/stateless_tests_flaky_check__asan_.html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

---------

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-11-01 15:09:39 +01:00
Alexander Tokmakov
65f3cf81dc
Merge pull request #55987 from ClickHouse/fix_lost_part_count 
Avoid counting lost parts twice
 2023-11-01 15:07:27 +01:00
Kruglov Pavel
5533258091
Merge pull request #55477 from Avogar/streaming-errors 
Allow to save unparsed records and errors in RabbitMQ, NATS and FileLog engines
 2023-11-01 14:35:35 +01:00
vdimir
2ca1a14ac4
Merge pull request #55741 from ClickHouse/vdimir/fix_analyzer_materialized_distributed_join 
Analyzer: fix block stucture mismatch in matview with engine distributed
 2023-11-01 14:16:33 +01:00
Kseniia Sumarokova
4510014014
Update StorageReplicatedMergeTree.cpp 2023-11-01 14:05:12 +01:00
Kseniia Sumarokova
c300297153
Fix style check 2023-11-01 13:27:38 +01:00
Alexander Sapin
b4328ad8dc Fix stupid bug 2023-11-01 13:06:19 +01:00
kssenii
68fd273114 Better 2023-11-01 11:18:58 +01:00
Zhiguo Zhou
b66a9e8fd3 Optimize DB::combineFilters with AVX512_VBMI2 intrinsic 
The DB::combineFilters combines two UInt8 columns (filters) by
scanning the first column for non-zero elements and replacing them
with elements in the second column contiguously.

This function is optimized with Intel's AVX512_VBMI2 intrinsic,
_mm512_mask_expandloadu_epi8, which is designed for implementing
the exact operation when the bits in the writemask are set when
the corresponding packed integers in the first filter are non-zero,
and are not set otherwise. The dynamic dispatch ensures that this
optimization only applies to devices with AVX512_VBMI2 support.
 2023-11-01 17:14:40 +08:00
Zhiguo Zhou
1ae951efc2 Add unittest MergeTree.CombineFilters 
This commit adds a unittest for validating DB::combineFilters.
 2023-11-01 17:14:40 +08:00
jsc0218
f1c7665feb
Merge pull request #56121 from lithiumlabcompany/fix-auto-configuration-queue-for-librdkafka 
Correctly set queued.min.messages
 2023-10-31 19:59:04 -04:00
Michael Kolupaev
9c79c165b9
Re-fix 'Block structure mismatch' on concurrent ALTER and INSERTs in Buffer table (#56140) 
* Fix 'Block structure mismatch' on concurrent ALTER and INSERTs in Buffer table

* Fix the test
 2023-10-31 13:41:54 -07:00