thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-15 19:02:04 +00:00
Code Issues Packages Projects Releases Wiki Activity
111,211 Commits 9,326 Branches 1,783 Tags 2.1 GiB
13f29a7242
Commit Graph

12060 Commits

Author SHA1 Message Date
Konstantin Morozov
08fdf42cd8 Merge branch 'master' into indices-storage-memory/update-read 2023-03-10 11:51:06 +01:00
Antonio Andelic
a170a909a4 Add expiration window for S3 credentials 2023-03-10 10:06:32 +00:00
Dmitry Novik
a305c6e7ab Fix distributed GROUPING SETS and GROUPING function 2023-03-09 18:00:23 +00:00
Anton Popov
66a113ca61
Merge pull request #47384 from CurtizJ/fix-clear-column-with-sparse 
Fix `ALTER CLEAR COLUMN` with sparse columns
 2023-03-09 16:57:23 +01:00
vdimir
c9e48f37cb
Forbid insert select for the same StorageJoin 2023-03-09 12:28:44 +00:00
Antonio Andelic
a70ca31884 Merge branch 'master' into fix-url-progress-bar 2023-03-09 10:17:33 +00:00
Konstantin Morozov
255102292c refactoring: remove uselees ctor 2023-03-09 10:02:16 +01:00
Konstantin Morozov
4686eb3cdd refactoring: create new step only once 2023-03-09 09:57:23 +01:00
Konstantin Morozov
82efbf01aa fix empty pipe for read 2023-03-09 08:28:55 +01:00
Anton Popov
c3b9af96c4 fix ALTER CLEAR COLUMN with sparse columns 2023-03-08 17:19:04 +00:00
Antonio Andelic
a04b38db90
Merge pull request #45108 from ClickHouse/custom-key-parallel-replicas 
Add support for custom key in parallel replicas
 2023-03-08 17:46:11 +01:00
Alexander Tokmakov
38f9f08e1a
Merge pull request #47268 from myrrc/improvement/more-to-base-3 
use std::lerp, constexpr hex.h
 2023-03-08 19:07:34 +03:00
kssenii
0240ad4c68 Add spark to tests, rewrite tests, fix bug 2023-03-08 15:45:35 +01:00
Antonio Andelic
816a9c1099
Merge branch 'master' into custom-key-parallel-replicas 2023-03-08 11:06:30 +01:00
Vladimir C
2aec0c1cb0
Merge pull request #47309 from ClickHouse/vdimir/fix_live_view_bg_pool_exception 2023-03-08 10:06:17 +01:00
Robert Schulze
2a47326061
Merge pull request #47313 from ClickHouse/rs/doc-cleanup-kafka 
Docs: Small cleanups after Kafka fix #47138
 2023-03-08 09:40:47 +01:00
Alexey Milovidov
e8cdb0c8b1
Merge pull request #46839 from CurtizJ/fix-crash-in-skip-index 
Do not allow const and non-deterministic secondary indexes
 2023-03-08 03:03:09 +03:00
Alexey Milovidov
e2aff59a2d
Merge pull request #47106 from kssenii/allow-nested-disks-in-custom-disks 
Allow nested custom disks
 2023-03-08 03:00:55 +03:00
Mike Kot
9920a52c51 use std::lerp, constexpr hex.h 2023-03-07 22:50:17 +00:00
Robert Schulze
ff8fc0634a
Docs: Small cleanups after Kafka fix #47138 2023-03-07 19:50:42 +00:00
Robert Schulze
5b91631722
Merge pull request #47138 from AVMusorin/disallow-default-columns-kafka-engine 
Prohibit DEFAULT/EPHEMERAL/ALIAS/MATERIALIZED columns in KafkaEngine
 2023-03-07 20:36:46 +01:00
vdimir
4e79df448f
Catch exceptions in LiveViewPeriodicRefreshTask 2023-03-07 16:17:35 +00:00
alesapin
c10cb436f4 Fix build 2023-03-07 12:42:42 +01:00
alesapin
1db6b9414e
Update src/Storages/StorageReplicatedMergeTree.cpp 
Co-authored-by: Ilya Yatsishin <2159081+qoega@users.noreply.github.com>
 2023-03-07 12:39:17 +01:00
Antonio Andelic
35c15e6ef8 Merge branch 'master' into custom-key-parallel-replicas 2023-03-07 09:37:38 +00:00
alesapin
b5dffe7417 Fix bug in zero copy replica which can lead to dataloss 2023-03-06 19:57:09 +01:00
Anton Popov
1b0d0c6108 fix skip indexes 2023-03-06 15:29:13 +00:00
Han Fei
b7eef62458
Merge pull request #45491 from azat/dist/async-send-refactoring 
[RFC] Rewrite distributed sends to avoid using filesystem as a queue, use in-memory queue instead
 2023-03-06 12:32:33 +01:00
kssenii
8567e3976b Fix kafka 2023-03-06 12:30:25 +01:00
Vladimir C
1455e7602d
Merge pull request #47154 from azat/prewhere-columns-fix 2023-03-06 10:03:31 +01:00
Anton Popov
6f3e4d4137
Merge pull request #46118 from CurtizJ/fix-issues-with-sparse 
Randomize setting `ratio_of_defaults_for_sparse_serialization`
 2023-03-05 22:28:18 +01:00
kssenii
c06af1f1e7 Fix clang-tidy 2023-03-05 22:12:51 +01:00
kssenii
8f2d75cef8 Fix tests 2023-03-05 12:56:00 +01:00
Robert Schulze
10753481f9
Merge pull request #47242 from ClickHouse/rs/docs-index-support-has-hasall-hasany 
Docs: Update index support of has(), hasAny(), hasAll()
 2023-03-04 18:49:24 +01:00
Robert Schulze
11930ad685
Docs: Update index support of has(), hasAny(), hasAll() 2023-03-04 17:27:47 +00:00
Sema Checherinda
85178c9609
Merge pull request #47102 from CheSema/merge-mutate-do-thread-group 
do flushUntrackedMemory when context switches
 2023-03-03 20:23:25 +01:00
Konstantin Morozov
e0ca12dc66 fix failed CI 2023-03-03 18:16:14 +01:00
Anton Popov
9389cc29da better mutations of sparse columns 2023-03-03 16:52:01 +00:00
kssenii
f78da967b3 Fix test 2023-03-03 16:32:23 +01:00
AVMusorin
0f2ae72141
prohibit DEFAULT/EPHEMERAL/ALIAS in KafkaEngine 2023-03-03 16:25:42 +01:00
Antonio Andelic
737cf8e149 Better 2023-03-03 15:14:49 +00:00
Alexander Tokmakov
cadaf06829
Merge pull request #46681 from ClickHouse/fix_insert_cancellation_in_native_protocol 
Fix queries cancellation when a client dies
 2023-03-03 16:27:20 +03:00
Konstantin Bogdanov
1bbf5acd47
Pass headers from StorageURL to WriteBufferFromHTTP (#46996) 
* Pass headers from StorageURL to WriteBufferFromHTTP

* Add a test

* Lint

* `time.sleep(1)`

* Start echo server earlier

* Add proper handling for mock server start

* Automatic style fix

---------

Co-authored-by: robot-clickhouse <robot-clickhouse@users.noreply.github.com>
 2023-03-03 13:55:52 +01:00
Kseniia Sumarokova
386663953c
Merge branch 'master' into named-collections-finish 2023-03-03 12:23:38 +01:00
Konstantin Morozov
3cb82bfe07 Update read and with ReadFromMemoryStorage 2023-03-03 06:41:48 +01:00
kssenii
cd7cd0526b Fix tests 2023-03-02 21:21:04 +01:00
Maksim Kita
3cd88003dd
Merge pull request #47101 from kitaisreal/analyzer-planner-fixes-before-enable-by-default 
Analyzer Planner fixes before enable by default
 2023-03-02 22:16:39 +03:00
Sema Checherinda
7a9f4aab8e do not call adjustWithUntrackedMemory when untracked_memory is 0 2023-03-02 19:21:03 +01:00
kssenii
0c4bb3c80f Merge remote-tracking branch 'upstream/master' into named-collections-granular-access 2023-03-02 19:08:29 +01:00
Azat Khuzhin
167e4903a8 Fix concrete columns PREWHERE support 
This is the fix for the IStorage::supportedPrewhereColumns() API.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-03-02 18:51:23 +01:00
Alexander Tokmakov
c518f3690e Merge branch 'master' into fix_insert_cancellation_in_native_protocol 2023-03-02 18:24:08 +01:00
Anton Popov
fdf8a418f5 better interface 2023-03-02 14:46:26 +00:00
Antonio Andelic
01cf9c94f4 Merge branch 'master' into custom-key-parallel-replicas 2023-03-02 14:28:42 +00:00
Robert Schulze
740aeaba1f
Apply some CTAD 2023-03-02 13:36:47 +00:00
kssenii
2883861795 Merge remote-tracking branch 'upstream/master' into allow-nested-disks-in-custom-disks 2023-03-02 14:30:58 +01:00
Kruglov Pavel
ce8e49a9a0
Merge pull request #45693 from Avogar/fix-kafka-protobuf 
Fix formats parser resetting, test processing bad messages in kafka
 2023-03-02 11:50:45 +01:00
Alexey Milovidov
97c0f8a0e0
Merge pull request #47027 from amosbird/clang16 
Prepare to build with clang 16
 2023-03-02 02:10:03 +03:00
Anton Popov
c4a999ce29 Merge remote-tracking branch 'upstream/master' into HEAD 2023-03-01 19:33:16 +00:00
kssenii
f3e3b916c9 Allow nested custom disks 2023-03-01 19:55:59 +01:00
Maksim Kita
d39be3ac9c Fixed tests 2023-03-01 18:05:07 +01:00
Maksim Kita
51ee007e01 Fixed tests 2023-03-01 18:05:07 +01:00
kssenii
80d017629a Fix test 2023-03-01 17:01:34 +01:00
alesapin
ebba37b18f
Merge pull request #47010 from ClickHouse/zero_copy_fetch 
Fix bug in zero-copy replication disk choice during fetch
 2023-03-01 15:55:00 +01:00
alesapin
b7370865a7
Merge pull request #47032 from ClickHouse/fix_message_for_zero_copy 
Better exception message when move was unseccessful
 2023-03-01 15:26:53 +01:00
Alexander Tokmakov
a97e15e36f Merge branch 'master' into fix_insert_cancellation_in_native_protocol 2023-03-01 14:26:41 +01:00
Antonio Andelic
f540f7f6f9 Fix some tests 2023-03-01 12:45:00 +00:00
kssenii
4716273349 Fix test 2023-03-01 13:37:05 +01:00
alesapin
d72c310faa
Update src/Storages/MergeTree/MergeTreeData.cpp 
Co-authored-by: Yakov Olkhovskiy <99031427+yakov-olkhovskiy@users.noreply.github.com>
 2023-03-01 11:54:22 +01:00
alesapin
8ff77c89eb
Update src/Storages/MergeTree/MergeTreeData.cpp 
Co-authored-by: Yakov Olkhovskiy <99031427+yakov-olkhovskiy@users.noreply.github.com>
 2023-03-01 11:50:35 +01:00
alesapin
be5ef666ba
Update src/Storages/MergeTree/MergeTreeData.cpp 
Co-authored-by: Yakov Olkhovskiy <99031427+yakov-olkhovskiy@users.noreply.github.com>
 2023-03-01 11:50:29 +01:00
alesapin
5de8a8e776
Update src/Storages/MergeTree/MergeTreeData.cpp 
Co-authored-by: Yakov Olkhovskiy <99031427+yakov-olkhovskiy@users.noreply.github.com>
 2023-03-01 11:50:23 +01:00
Alexey Milovidov
fe446afb7d Increase max_replicated_merges_in_queue 2023-03-01 06:02:59 +01:00
Amos Bird
c0fb4ddc3e
Make thread analyzer happy 2023-03-01 11:07:36 +08:00
Azat Khuzhin
d06a4b50d6 Latest review fixes (variable naming: s/monitor/queue) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:48:07 +01:00
Azat Khuzhin
591fca57f3 Fix function names for opentelemetry spans in StorageDistributed 
Fixes: 02417_opentelemetry_insert_on_distributed_table
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
7063c20b3c Change noisy "Skipping send data over distributed table." message to test 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
16bfef3c8a Fix processing current_batch.txt on init 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
752d27d663 Fix lossing files during distributed batch send 
v2: do not suppress exceptions in case of errors
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
263c042c6a Fix opentelemetry for distributed batch sends 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
00115c6615 Rename readDistributedAsyncInsertHeader() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
a76d7b22c1 Use existing public methods of StorageDistributed in DistributedSink 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
e83699a8d3 Improve comment for DistributedAsyncInsertDirectoryQueue::initializeFilesFromDisk() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
e10fb142fd Fix race for distributed sends from disk 
Before it was initialized from disk only on startup, but if some INSERT
can create the object before, then, it will lead to the situation when
it will not be initialized.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
b5434eac3b Rename StorageDistributedDirectoryMonitor to DistributedAsyncInsertDirectoryQueue 
Since #44922 it is not a directory monitor anymore.

v2: Remove unused error codes
v3: Contains some header fixes due to conflicts with master
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
1c4659b8e7 Separate out Batch as DistributedAsyncInsertBatch (and also some helpers) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
33b13549ad Separate out DirectoryMonitorSource as DistributedAsyncInsertSource 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
325a7b2305 Separate out DistributedHeader as DistributedAsyncInsertHeader 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
22a39e29f7 Add a comment for StorageDistributedDirectoryMonitor::Batch::recovered 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
0c19a75a1c Add log message for batch restore 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
13a3e03f19 Introduce StorageDistributedDirectoryMonitor::Batch::{de,}serialize() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
f0a2efa630 Always manipulate with absolute file paths in DirectoryMonitor 
Otherwise on batch restore we can get the difference in file paths.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
ef1e642e05 Add log message to StorageDistributedDirectoryMonitor::addAndSchedule() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
16646c0923 Rename DirectoryMonitor::current_batch_file to current_file 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
3f892e52ab Revert "Revert "Merge pull request #44922 from azat/dist/async-INSERT-metrics"" 
This is the revert of revert since there will be follow up patches to
address the issues.

This reverts commit a55798626a.
 2023-02-28 22:33:36 +01:00
alesapin
c0f28ace51 Better 2023-02-28 17:52:21 +01:00
alesapin
abd6500ee3 Better exception message when move was unseccessful 2023-02-28 17:49:16 +01:00
pufit
29296cb004
Merge pull request #46550 from ClickHouse/46231-systerm-server-settings 
Implement `system.server_settings`
 2023-02-28 11:23:15 -05:00
Antonio Andelic
45dc5dc25d No progress bar if no size 2023-02-28 15:25:28 +00:00
Antonio Andelic
d8b72e91ef Merge branch 'master' into fix-url-progress-bar 2023-02-28 14:45:45 +00:00
alesapin
bb7571427b Fix bug in zero-copy replica disk choice during fetch 2023-02-28 14:48:46 +01:00
Kruglov Pavel
a06596bdba
Fix special build 2023-02-28 14:33:41 +01:00
alesapin
712a8309c5
Merge pull request #46976 from ClickHouse/decrease-log-level 
Decrease log level in "disks"
 2023-02-28 12:28:30 +01:00
kssenii
d4e6fc4546 Fix test 2023-02-27 21:43:45 +01:00
Nikolai Kochetov
738551dbf1
Merge pull request #46624 from CheSema/detached_mt 
add multitreading for StorageSystemDetachedParts
 2023-02-27 20:54:28 +01:00
Alexey Milovidov
29da7fc965 Decrease log level in "disks" 2023-02-27 17:48:38 +01:00
kssenii
ad88251ee7 Fix tests 2023-02-27 17:42:04 +01:00
Anton Popov
7b85b35534 fix skip indexes 2023-02-27 15:11:25 +00:00
Anton Popov
caa1d22759
Merge branch 'master' into fix-issues-with-sparse 2023-02-27 15:25:19 +01:00
Robert Schulze
6f0709032d
Merge pull request #46921 from ClickHouse/rs/inv-idx-checksum 
A better alternative to #46344
 2023-02-27 14:44:44 +01:00
Nikolai Kochetov
3392b96cc3
Merge branch 'master' into aggregate-projections-analysis-query-plan 2023-02-27 14:07:04 +01:00
kssenii
b19264cf9f Remove redundant 2023-02-27 12:32:13 +01:00
Robert Schulze
4c9b9b362c
Replace dynamic_cast by typeid_cast 2023-02-27 10:01:21 +00:00
Robert Schulze
cc0c0c6133
A better alternative to #46344 
The (experimental) inverted index writes/reads files different from the
standard files written by the other skip indexes. The original problem
was that with database engine "ordinary", DROP TABLE of a table with
inverted index finds unknown files in persistence and complains. The
same will happen with engine "atomic" but deferred. As a hotfix, the
error was silenced by explicitly adding the four files created in a
specific test to the deletion code.

This PR tries a cleaner solution where all needed files are provided via
the normal checksum structure. One drawback remains which is that the
affected files were written earlier and we don't have their checksums
available. Therefore, the inverted index is currently excluded from
CHECK TABLE.

Minimal repro:
  SET allow_experimental_inverted_index = 1;
  DROP TABLE IF EXISTS tab;
  CREATE TABLE tab(s String, INDEX af(s) TYPE inverted(2)) ENGINE = MergeTree() ORDER BY s;
  INSERT INTO tab VALUES ('Alick a01');
  CHECK TABLE tab;
  DROP TABLE IF EXISTS tab;

  run ./clickhouse-test with --db-engine Ordinary
 2023-02-26 21:06:38 +00:00
Alexey Milovidov
73aec86f5a Change exception message 2023-02-26 21:36:26 +01:00
Sema Checherinda
1666527dae do listing combined with getting size in one thread 2023-02-26 12:09:05 +01:00
Alexey Milovidov
d448a48f16
Merge pull request #46831 from azat/merge-default-prewhere 
Allow PREWHERE for Merge with different DEFAULT expression for column
 2023-02-26 10:02:34 +03:00
Robert Schulze
5e68d50b39
Merge pull request #46752 from ClickHouse/rs/periods-in-kafka-topics 
Allow configuration of Kafka topics with periods
 2023-02-26 07:49:28 +01:00
Alexey Milovidov
8f40bbd5b4
Revert "Fix several RENAME COLUMN bugs." 2023-02-26 05:47:05 +03:00
Alexey Milovidov
eb89ab471f
Merge pull request #45911 from ClickHouse/mutations_rename_hang 
Fix several `RENAME COLUMN` bugs.
 2023-02-26 01:56:40 +03:00
Alexey Milovidov
656af55831
Merge pull request #46384 from ClickHouse/fixes_for_993 
Fixes for 993
 2023-02-26 01:55:20 +03:00
Robert Schulze
fbba9e319d
Merge branch 'master' into rs/periods-in-kafka-topics 2023-02-25 21:45:20 +01:00
Robert Schulze
3618e37dd2
Reduce indentation by early out 2023-02-25 15:36:59 +00:00
Sema Checherinda
0c9643f1ed make listing in parrallel as well 2023-02-25 16:27:27 +01:00
Nikolai Kochetov
57b5f9e7d7 Fix case with no detached parts for table. 2023-02-25 13:33:20 +01:00
Sema Checherinda
e2bfa83429 cancel workers when exception 2023-02-25 13:33:20 +01:00
Sema Checherinda
08dc874a37 do not run support threads if no tasks left 2023-02-25 13:33:20 +01:00
Sema Checherinda
e5cbe4311e create limites count of support threads for a block 2023-02-25 13:33:20 +01:00
Sema Checherinda
ef0c1841af work with comments on review 2023-02-25 13:33:20 +01:00
Sema Checherinda
3b58249320 pack several files in one thread 2023-02-25 13:33:20 +01:00
Sema Checherinda
8abc1f0d5d fix build and style 2023-02-25 13:33:20 +01:00
Sema Checherinda
dce319e1c5 add multitreading for StorageSystemDetachedParts 2023-02-25 13:33:20 +01:00
Alexander Tokmakov
cad1e0b768 fix 2023-02-25 01:18:34 +01:00
kssenii
c2bcc4119f Better 2023-02-24 20:57:37 +01:00
kssenii
96b57c8a65 Better support for equal keys 2023-02-24 20:38:40 +01:00
Anton Popov
3a947ecddc do not allow const and non-deterministic secondary indexes 2023-02-24 19:17:44 +00:00
Alexander Tokmakov
ca1c793cac
Merge pull request #46820 from Algunenano/avoid_KeeperMultiException_system_errors 
Don't report ZK node exists to system.errors when a block was created concurrently by a different replica
 2023-02-24 19:42:32 +03:00
kssenii
6224ca9cd6 Fix 2023-02-24 17:08:05 +01:00
Robert Schulze
6402050125
Fixup comments 2023-02-24 16:59:38 +01:00
Azat Khuzhin
86e9b131c7 Allow PREWHERE for Merge with different DEFAULT expression for column 
It can be not that special to have Merge engine w/o default expression,
while MergeTree will have it. So improve PREWHERE for such kinds, and
also allow PREWHERE for DEFAULT and MATERIALIZED (since both are
physical columns on disk and this make sense).

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-24 16:33:49 +01:00
Antonio Andelic
56a126f7af Fix progress bar with URL 2023-02-24 14:49:14 +00:00
kssenii
f2a9109fe1 Merge remote-tracking branch 'upstream/master' into named-collections-granular-access 2023-02-24 15:20:46 +01:00
Alexander Gololobov
b43b98f7c0
Merge pull request #46785 from ClickHouse/fix_combined_prewhere_column 
Fix combined PREWHERE column accumulated from multiple steps
 2023-02-24 14:00:17 +01:00
Kruglov Pavel
2150b836b1
Merge branch 'master' into fix-kafka-protobuf 2023-02-24 12:48:07 +01:00
Raúl Marín
7ff8d2d3d5 Don't report ZK node exists to system.errors 2023-02-24 12:36:29 +01:00
pufit
2fbe75957d Add default column for system.settings and system.server_settings 2023-02-23 18:24:57 -05:00
Anton Popov
d926713cf5 Merge remote-tracking branch 'upstream/master' into HEAD 2023-02-23 23:04:22 +00:00
Anton Popov
5e5e802348 return back optimization for ALTER MODIFY Nullable 2023-02-23 22:55:47 +00:00
Mikhail f. Shiryaev
18a20268d8
Merge pull request #46792 from ClickHouse/23.3-prepare 
Update version after release
 2023-02-23 23:51:28 +01:00
Alexander Tokmakov
fe92fd8a61 Merge branch 'master' into fix_insert_cancellation_in_native_protocol 2023-02-23 22:35:31 +01:00
Robert Schulze
6906cafa99
Change to S3 cfg syntax 2023-02-23 20:04:41 +00:00
Antonio Andelic
1efba8995d Update version to 23.3.1.2537 2023-02-23 19:35:57 +00:00
Alexander Gololobov
47f51fe187
Merge pull request #46771 from ClickHouse/fix_prewhere_logging 
More concise logging at trace level for PREWHERE steps
 2023-02-23 20:20:30 +01:00
Alexander Gololobov
8b22373488 Apply filter at last step if it has been accumulated from multiple steps 2023-02-23 16:58:28 +01:00
Alexander Gololobov
cbcb1a7852 More concise logging at trace level 2023-02-23 13:31:56 +01:00
Kseniia Sumarokova
7e064ccbd4
Merge pull request #46667 from kssenii/hide-named-collections-under-setting 
Allow to hide only values from system.named_collections while making collection visible
 2023-02-23 11:44:50 +01:00
Kseniia Sumarokova
30a8eb0c2f
Merge pull request #46670 from kssenii/hide-disk-setting 
Hide arguments of custom disk merge tree setting
 2023-02-23 11:42:53 +01:00
Alexey Milovidov
c0bc549e77
Merge pull request #46709 from azat/bg-pool-no-query 
BackgroundSchedulePool should not have any query context
 2023-02-23 10:24:46 +03:00
Alexey Milovidov
e3f24b2b68
Merge pull request #46744 from ClickHouse/remove_unused_field 
Remove unused MergeTreeReadTask::remove_prewhere_column
 2023-02-23 08:02:54 +03:00
Robert Schulze
81bf43157f
Allow configuration of Kafka topics with periods 
The Kafka table engine allows global configuration and per-Kafka-topic
configuration. The latter uses syntax <kafka_TOPIC>, e.g. for topic
"football":

  <kafka_football>
      <retry_backoff_ms>250</retry_backoff_ms>
      <fetch_min_bytes>100000</fetch_min_bytes>
  </kafka_football>

Some users had to find out the hard way that such configuration doesn't
take effect if the topic name contains a period, e.g. "sports.football".
The reason is that ClickHouse configuration framework already uses
periods as level separators to descend the configuration hierarchy.
(Besides that, per-topic configuration at the same level as global
configuration could be considered ugly.)

Note that Kafka topics may contain characters "a-zA-Z0-9._-" (*) and
a tree-like topic organization using periods is quite common in
practice.

This PR deprecates the existing per-topic configuration syntax (but
continues to support it for backward compat) and introduces a new
per-topic configuration syntax below the global Kafka configuration of
the form:

<kafka>
   <topic name="football">
       <retry_backoff_ms>250</retry_backoff_ms>
       <fetch_min_bytes>100000</fetch_min_bytes>
   </topic>
</kafka>

The period restriction doesn't apply to XML attributes, so <topic
name="sports.football"> will work. Also, everything Kafka-related is
below <kafka>.

Considered but rejected alternatives:
- Extending Poco ConfigurationView with custom separators (e.g."/"
  instead of "."). Won't work easily because ConfigurationView only
  builds a path but defers descending the configuration tree to the
  normal configuration classes.
- Reloading the configuration file in StorageKafka (instead of reading
  the loaded file) but with a custom separator. This mode is supported
  by XML configuration. Too ugly and error-prone since the true
  configuration is composed from multiple configuration files.

(*) https://stackoverflow.com/a/37067544
 2023-02-22 20:35:09 +00:00
Alexander Tokmakov
e660c0838c fix multipart requests 2023-02-22 17:54:35 +01:00
Alexander Gololobov
67dcd9694c Remove unused MergeTreeReadTask::remove_prewhere_column 2023-02-22 17:49:22 +01:00
pufit
8e7533fa57
Merge pull request #46564 from AVMusorin/update-time-distribution-queue 
Added `last_exception_time` column into distribution_queue table
 2023-02-22 10:43:35 -05:00
kssenii
c4761d6cc6 Fix checks 2023-02-22 14:27:43 +01:00
kssenii
f0ca3ef207 Merge remote-tracking branch 'upstream/master' into named-collections-granular-access 2023-02-22 14:02:29 +01:00
kssenii
f518ddd96d Merge remote-tracking branch 'upstream/master' into named-collections-finish 2023-02-22 13:43:18 +01:00
Kruglov Pavel
3ba3fdbfa3
Merge pull request #46607 from kssenii/delay-loading-of-named-collections 
Do not load named collections on server startup (on first access instead)
 2023-02-22 13:22:34 +01:00
Azat Khuzhin
2ca47a6eb6 BackgroundSchedulePool should not have any query context 
BackgroundSchedulePool is used for some peridic jobs, not from the query
context, i.e. flush of Buffer table.

And for such jobs there cannot be any query context, and more
importantly it will not work correctly since that query_context will
eventually expires.

And this is the reason of this failures [1].

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/46668/015991bc5e20c787851050c2eaa13f0fef3aac00/stateless_tests_flaky_check__asan_.html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-22 10:50:51 +01:00
Alexander Tokmakov
592af6d652 fix incomplete interst through http 2023-02-22 02:34:03 +01:00
AVMusorin
0bf0fe488e
added last_exception_time column into distribution_queue table 2023-02-21 23:46:57 +01:00
Alexander Tokmakov
f5c232707d cancel insert queries correctly 2023-02-21 22:04:39 +01:00
Alexander Gololobov
b61bb56a5a
Merge pull request #46611 from ClickHouse/prewhere_progress 
Fix reporting progress when rows are filtered in PREWHERE
 2023-02-21 21:27:56 +01:00
kssenii
b15be14792 Fix table function remote 2023-02-21 18:55:43 +01:00
kssenii
9a7c71b78e Allow to hide only values from system.named_collections 2023-02-21 18:07:57 +01:00
Alexey Milovidov
848286eca7
Merge pull request #40945 from arthurpassos/add_select_final_mt_setting 
Add user setting to force select final at query level
 2023-02-21 19:58:45 +03:00
Nikolai Kochetov
c5f93eb108 Fix more tests. 2023-02-21 15:44:50 +00:00
kssenii
d84215d005 Fix style check 2023-02-21 16:20:57 +01:00
Kseniia Sumarokova
655e7f29f7
Merge pull request #46595 from kssenii/update-rabbit-producer 
Fix rabbitmq test
 2023-02-21 16:18:55 +01:00
Anton Popov
b66a5bdcf0 fix mutations with sparse columns 2023-02-21 15:13:29 +00:00
kssenii
b3ca976f5f Finish for streaming engines 2023-02-21 14:50:55 +01:00
Alexander Gololobov
5354b0d069
Merge pull request #46140 from ClickHouse/split_prewhere_dag 
Split PREWHERE DAG into multiple read steps
 2023-02-21 14:37:09 +01:00
kssenii
68e06ecb99 Replace for table function remote, and external storage 2023-02-21 14:33:37 +01:00
Kseniia Sumarokova
09a979f106
Merge branch 'master' into update-rabbit-producer 2023-02-21 11:50:30 +01:00
Vladimir C
a791250a4a
Merge pull request #38614 from bharatnc/ncb/profile-events-partlog 2023-02-21 11:06:44 +01:00
Kseniia Sumarokova
24d0490ef3
Merge branch 'master' into delay-loading-of-named-collections 2023-02-21 00:24:18 +01:00
Kruglov Pavel
828a06b511
Merge branch 'master' into fix-kafka-protobuf 2023-02-20 21:45:43 +01:00
Kruglov Pavel
b0424c1021
Merge pull request #46171 from Avogar/insert-null-as-default 
Use default of column type in `insert_null_as_default` if column DEFAULT values is not specified
 2023-02-20 21:45:02 +01:00
kssenii
a54b011670 Finish for mysql 2023-02-20 21:37:38 +01:00
Nikolai Kochetov
ee597b939f Merge branch 'master' into aggregate-projections-analysis-query-plan 2023-02-20 19:46:52 +00:00
Arthur Passos
4f2fcf763b
Merge branch 'master' into add_select_final_mt_setting 2023-02-20 16:22:14 -03:00
Kseniia Sumarokova
47604ad77e
Merge pull request #46106 from kssenii/return-materialized-postrgesql-tests-back 
Enable disabled MaterializedPostgreSQL tests back in CI
 2023-02-20 19:04:58 +01:00
Alexander Gololobov
c84a64d22f Return chunks with 0 rows from MergeTreeSource to report progress when rows are filtered in PREWHERE 2023-02-20 17:35:01 +01:00
kssenii
9d16205c8a Load named collections on first access 2023-02-20 16:33:45 +01:00
Nikolai Kochetov
84e442e620 Fixing build and style. 2023-02-20 14:21:20 +00:00
kssenii
80a0aa1327 Merge remote-tracking branch 'upstream/master' into return-materialized-postrgesql-tests-back 2023-02-20 14:37:39 +01:00
kssenii
361678ad73 rabbitmq-test-fix 2023-02-20 14:16:40 +01:00
Nikolai Kochetov
2a8460d962 Merge branch 'master' into aggregate-projections-analysis-query-plan 2023-02-20 13:54:05 +01:00
Alexey Milovidov
d8cda3dbb8 Remove PVS-Studio 2023-02-19 23:30:05 +01:00
Arthur Passos
3d9b23e5f3
Merge branch 'master' into add_select_final_mt_setting 2023-02-19 15:01:41 -03:00
Alexey Milovidov
5359dcc014
Merge pull request #46554 from ucasfl/iceberg-fix 
fix build without avro
 2023-02-19 14:35:29 +03:00
Alexander Gololobov
520b381675 Fixes for executing actions on block with unknown row count 2023-02-19 12:14:23 +01:00