alesapin
29e3e7cb96
Better if only 1 layer
2024-02-18 13:19:35 +01:00
alesapin
8f29320a73
Flush StorageBuffer into multiple threads if num_layers > 1
2024-02-18 13:15:24 +01:00
Alexey Milovidov
8d6e6d67ed
Merge branch 'fix-inconsistent-formatting-of-explain-in-subqueries' into lazy-primary-key-loading
2024-02-18 11:04:01 +01:00
Alexey Milovidov
232fe0dd1b
Fix inconsistent formatting of SELECT ... FROM (EXPLAIN ...)
2024-02-18 05:38:34 +01:00
Alexey Milovidov
c083498cf8
Apply review comments
2024-02-18 05:13:17 +01:00
Vitaly Baranov
3323d5ce81
Move threadPoolCallbackRunner to the "Common" folder.
2024-02-17 21:59:51 +01:00
Alexey Milovidov
0e944d400c
Merge pull request #60092 from ClickHouse/index-preparation
...
A small preparation for better handling of primary key in memory
2024-02-17 10:15:06 +01:00
Alexey Milovidov
c5107afb94
Merge branch 'master' into lazy-primary-key-loading
2024-02-17 10:14:35 +01:00
Alexey Milovidov
6cae75751b
Merge pull request #60049 from ClickHouse/less-memory-usage-primary-key
...
Maybe less memory usage for primary keys in memory
2024-02-17 08:58:02 +01:00
Alexey Milovidov
021e11e4fc
Add a setting for lazy loading
2024-02-17 06:42:53 +01:00
Alexey Milovidov
423ff759f1
Lazy loading of primary keys in memory
2024-02-17 06:35:03 +01:00
Alexey Milovidov
e9cf92245f
Move methods to .cpp
2024-02-17 06:20:44 +01:00
Alexey Milovidov
fce3a8cafc
A small preparation for better handling of primary key in memory
2024-02-17 06:16:56 +01:00
Alexey Milovidov
ac5bef7c74
A small preparation for better handling of primary key in memory
2024-02-17 06:16:01 +01:00
Alexey Milovidov
46c2f85314
Merge branch 'master' into less-memory-usage-primary-key
2024-02-17 04:55:59 +01:00
Alexey Milovidov
2f315e0eb5
Style check for abbreviations
2024-02-17 02:14:15 +01:00
Alexey Milovidov
731c484b3d
Revert "Implement system.dns_cache table ( #59856 )"
...
This reverts commit b5ef034697
.
2024-02-17 01:46:41 +01:00
Kirill Nikiforov
b5ef034697
Implement system.dns_cache table ( #59856 )
...
* system.dns_cache table WIP
* system.dns_cache table
* rollback unexpected contrib changes
* rollback unexpected changes
* add docs
* code style fixes
* optimize dns cache exporting
* stateful test for system.dns_cache table
* stateful test for system.dns_cache table
* cr fixes
* stateful test for system.dns_cache table
* stateful test for system.dns_cache table
* Update 02981_system_dns_cache_table.sql
* Update 02981_system_dns_cache_table.reference
---------
Co-authored-by: pufit <pufit@clickhouse.com>
2024-02-16 14:31:22 -05:00
Alexander Tokmakov
b513f1d456
fix build
2024-02-16 18:27:22 +01:00
Alexander Tokmakov
31479649fe
Merge pull request #59908 from azat/rmt-uuid-macro
...
[RFC] Allow uuid in replica_path if CREATE TABLE explicitly has it
2024-02-16 17:31:59 +01:00
Alexander Tokmakov
3813a97775
Merge pull request #59895 from azat/rmt-mutations-polling
...
Do not pull mutations if pulling replication log had been stopped
2024-02-16 17:24:30 +01:00
Alexander Tokmakov
03ff171922
Merge branch 'master' into modify-engine
2024-02-16 17:22:24 +01:00
Dmitry Novik
a3c34a7cc2
Merge pull request #54211 from ClickHouse/Support_parameterized_view_with_analyzer
...
Refactor parameterized view & add support for new analyzer
2024-02-16 16:13:49 +01:00
Igor Nikonov
b85a68790a
Cleanup: connection pool priority -> config priority
...
- names were creating confusion between config priority and balancing priority for a reader
2024-02-16 14:39:41 +00:00
Kruglov Pavel
4d6f167e0c
Merge pull request #59092 from Avogar/auto-format-detection
...
Try to detect file format automatically during schema inference if it's unknown
2024-02-16 14:32:18 +01:00
Alexander Tokmakov
228a29b93f
fix
2024-02-16 14:05:22 +01:00
alesapin
0a189e830c
Merge pull request #59942 from kitaisreal/storage-system-tables-add-metadata-version
...
StorageSystemTables add metadata_version
2024-02-16 13:17:22 +01:00
Raúl Marín
e168329ade
Revert "ReplicatedMergeTree invalid metadata_version fix"
2024-02-16 11:44:16 +01:00
Alexey Milovidov
2f36c9e965
Maybe less memory usage for primary keys in memory
2024-02-16 00:56:24 +01:00
Anton Popov
ef6b8275b1
Merge branch 'virtual-column-prewhere' of https://github.com/amosbird/ClickHouse into refactor-virtual-columns
2024-02-15 21:45:58 +00:00
Alexander Tokmakov
22c97be2a4
ability to detect undead ZooKeeper sessions
2024-02-15 22:17:31 +01:00
kssenii
2a783321d7
Fix deletion of s3 queue
2024-02-15 17:56:38 +01:00
Azat Khuzhin
f10fc95933
Fix INSERT into SQLite with single quote
...
Previously it leads to syntax error, due to incorrect escaping of single
quotes for SQLite, "\'" had been used instead of "''"
So set output_format_values_escape_quote_with_quote=true for SQLite to
fix this.
v2: prepare modified Context for writing on storage creation
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-02-15 12:47:29 +01:00
Robert Schulze
f8413481c4
Merge remote-tracking branch 'ClickHouse/master' into materialize_non_override_past_values
2024-02-15 10:29:00 +00:00
Alexey Milovidov
a98facee52
Merge pull request #59944 from ClickHouse/fewer_lambdas
...
Replace lambdas with pointers to members to simplify stacks
2024-02-15 06:02:35 +01:00
Alexey Milovidov
009d08561d
Merge pull request #59946 from kitaisreal/replicated-merge-tree-alter-update-metadata-version-in-single-transaction
...
ReplicatedMergeTree invalid metadata_version fix
2024-02-15 04:38:06 +01:00
Alexey Milovidov
681351fe1e
Merge pull request #59987 from nickitat/fix_data_race_storage_distributed
...
Fix data race in `StorageDistributed`
2024-02-15 02:55:08 +01:00
Alexey Milovidov
7a5e152d9e
Merge pull request #59967 from ClickHouse/dashboards-to-use-merge-tables
...
Use merge table in the dashboard; add Cloud overview to the system table
2024-02-14 21:56:37 +01:00
Nikita Taranov
5f87956fc8
impl
2024-02-14 16:22:14 +01:00
Dmitry Novik
8d2ad5383b
Fix execution name for constants
2024-02-14 15:19:08 +00:00
Maksim Kita
d4778d858d
Fixed tests
2024-02-14 18:05:29 +03:00
Maksim Kita
0cce386207
Added documentation
2024-02-14 18:05:29 +03:00
Maksim Kita
f6d211b44d
StorageSystemTables add metadata_version
2024-02-14 18:05:29 +03:00
Maksim Kita
be490291af
Updated implementation
2024-02-14 18:05:06 +03:00
Maksim Kita
dca627026c
ReplicatedMergeTree ALTER update metadata_version in single transaction
2024-02-14 18:05:06 +03:00
kssenii
8fe9ede207
Add a test
2024-02-14 15:25:43 +01:00
Dmitry Novik
617e73ce28
Merge remote-tracking branch 'origin/master' into analyzer-fix-test_select_table_name_from_merge_over_distributed
2024-02-14 13:32:14 +00:00
Nikolai Kochetov
03720d558e
Merge branch 'master' into allow-parallel-replicas-for-join-with-analyzer-2
2024-02-14 12:54:29 +01:00
Alexander Gololobov
dda7c5d2b7
Add comment with motivation
2024-02-14 11:41:07 +01:00
Alexey Milovidov
7b3e57e2d2
Use merge table in the dashboard
2024-02-14 02:37:42 +01:00
Alexey Milovidov
102e64cf1d
Merge branch 'master' of github.com:ClickHouse/ClickHouse into dashboards-to-use-merge-tables
2024-02-14 02:20:20 +01:00
Alexey Milovidov
27f115f581
Use merge table in the dashboard
2024-02-14 02:17:23 +01:00
Alexey Milovidov
7ee6b0bd87
Merge pull request #59941 from kitaisreal/register-storage-merge-tree-exception-message-fix
...
Register StorageMergeTree exception message fix
2024-02-14 01:31:00 +01:00
Alexey Milovidov
8b59977639
Merge pull request #59909 from azat/rmt-macros-cleanup
...
Petite cleanup around macros and ReplicatedMergeTree
2024-02-14 00:09:34 +01:00
Kruglov Pavel
1dbfeafb42
Merge branch 'master' into auto-format-detection
2024-02-13 19:08:33 +01:00
Alexander Gololobov
037bc043e6
Removed lambdas
2024-02-13 16:01:17 +01:00
Maksim Kita
4dc6ed3a39
Register StorageMergeTree exception message fix
2024-02-13 17:53:35 +03:00
Smita Kulkarni
d7056d642d
Addressed review comments
2024-02-13 13:53:40 +01:00
Smita Kulkarni
3526d7982c
Merge branch 'master' into Support_parameterized_view_with_analyzer
2024-02-13 13:08:35 +01:00
Kseniia Sumarokova
bb12fb45c4
Merge pull request #59873 from azat/file-log-shutdown-fix
...
Fix long shutdown of FileLog storage
2024-02-13 12:08:12 +01:00
Nikita Mikhaylov
4b85f6b39a
Revert "Less error prone interface of read buffers" ( #59911 )
2024-02-13 12:02:46 +01:00
Nikolai Kochetov
666b3d6664
Merge branch 'master' into allow-parallel-replicas-for-join-with-analyzer-2
2024-02-13 11:11:03 +01:00
Azat Khuzhin
58e5d7876b
Allow uuid in replica_path if CREATE TABLE explicitly has it
...
Right now the query with {uuid} macro without ON CLUSTER fails:
CREATE TABLE x UUID 'aaaaaaaa-1111-2222-3333-aaaaaaaaaaaa' (key Int) ENGINE = ReplicatedMergeTree('/tables/{database}/{uuid}', 'r1') ORDER BY tuple();
There is a workaround right now to use ATTACH instead of CREATE, but
ATTACH is not CREATE.
CREATE still useful for proper RESTORE without ON CLUSTER.
So this patch allows this syntax, but only if UUID had been explicitly
passed in the query. This looks safe.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-02-13 08:01:09 +01:00
Alexey Milovidov
4dc59051a1
Merge pull request #59897 from ClickHouse/s3queue-fix-uninitialized-value
...
s3queue: fix uninitialized value
2024-02-13 02:44:45 +01:00
Azat Khuzhin
c61ac1d3bc
Convert default_replica_path/default_replica_name into server settings
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-02-12 21:05:58 +01:00
kssenii
d008ee725f
Add a test
2024-02-12 20:23:21 +01:00
kssenii
0628ae6268
S3 queue fix uninitialized value
2024-02-12 17:13:30 +01:00
Azat Khuzhin
c7c05c9881
Do not pull mutations if pulling replication log had been stopped
...
Right now, mutations can be pulled even after:
SYSTEM STOP PULLING REPLICATION LOG
Since they pulled from two places:
- StorageReplicatedMergeTree::mutationsUpdatingTask()
- ReplicatedMergeTreeQueue::pullLogsToQueue()
And only the last one checks action blocker.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-02-12 16:28:08 +01:00
Nikolai Kochetov
fd2ee19f65
Follow up for #58554 . Cleanup.
2024-02-12 13:37:25 +00:00
Nikolai Kochetov
09d3b3c0d0
Merge pull request #58554 from ClickHouse/try-to-always-push-down-prewhere-from-query-plan
...
Simplify optimize-push-to-prewhere from query plan
2024-02-12 12:52:39 +01:00
Azat Khuzhin
67b73c3e49
Fix long shutdown of FileLog storage
...
Previously it was possible to wait up to
poll_directory_watch_events_backoff_max (default is 32000) on shutdown,
because it was not possible to stop poll of inotify.
Before (takes 3 seconds):
2024.02.12 11:27:55.058192 [ 10134 ] {} <Trace> StorageFileLog (file_log): Waiting for cleanup
2024.02.12 11:27:58.178021 [ 10271 ] {} <Trace> directory_watch: Execution took 7519 ms.
After:
2024.02.12 11:33:29.722403 [ 15866 ] {} <Trace> StorageFileLog (file_log): Waiting for cleanup
2024.02.12 11:33:29.722473 [ 15956 ] {} <Trace> directory_watch: Execution took 6399 ms.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-02-12 11:52:09 +01:00
Robert Schulze
06dc9ed074
Merge remote-tracking branch 'ClickHouse/master' into materialize_non_override_past_values
2024-02-11 15:48:55 +00:00
Alexander Gololobov
309db427e7
Do not reinitialize ZooKeeperWithFaultInjection on each chunk
2024-02-11 13:13:49 +01:00
Michael Kolupaev
5424f235a9
Fix StorageURL doing some of the query execution in one thread instead of max_threads
2024-02-11 02:44:18 +00:00
Anton Popov
b4449535c2
Merge pull request #59733 from CurtizJ/check-minmax-index-types
...
Validate types of arguments for `minmax` skipping index
2024-02-09 19:39:59 +01:00
Dmitry Novik
dd8680018f
Analyzer: WIP on test_merge_table_over_distributed/test.py::test_select_table_name_from_merge_over_distributed
2024-02-09 16:43:20 +01:00
Nikolai Kochetov
862c6cd79c
Merge pull request #57430 from ClickHouse/non-ready-set-ttl
...
Non ready set in TTL WHERE.
2024-02-09 16:31:26 +01:00
Igor Nikonov
effaaceb26
Merge pull request #59501 from ClickHouse/pr-better-replicas-failover-2
...
Parallel replicas: better initial replicas failover (2)
2024-02-09 15:50:56 +01:00
Dmitry Novik
2e6ca6200e
Merge pull request #59774 from ClickHouse/analyzer-merge-follow-up
...
Followup #50894
2024-02-09 15:05:19 +01:00
Vitaly Baranov
09e727d48b
Merge pull request #56988 from ClickHouse/Azure_backup
...
Backup & Restore support for AzureBlobStorage
2024-02-09 14:45:35 +01:00
Dmitry Novik
8655c11280
Fix typo
2024-02-09 11:45:15 +01:00
Igor Nikonov
84d64a6ed8
Merge remote-tracking branch 'origin/master' into pr-better-replicas-failover-2
2024-02-08 18:03:16 +00:00
kssenii
e55b60e05c
Fix
2024-02-08 17:43:16 +01:00
Dmitry Novik
5f75087178
Add comments
2024-02-08 15:48:20 +00:00
Nikolai Kochetov
73d2ff3933
Update MergeTask.cpp
2024-02-08 14:55:35 +01:00
Dmitry Novik
b8f74b5b05
Merge pull request #50894 from ClickHouse/storage-merge-aliases-analyzer
...
Analyzer: support aliases and distributed JOINs in StorageMerge
2024-02-08 11:22:00 +01:00
Igor Nikonov
3fb30a23d9
Merge remote-tracking branch 'origin/master' into pr-better-replicas-failover-2
2024-02-08 09:09:11 +00:00
Alexander Tokmakov
510f90edf8
Merge pull request #59688 from ClickHouse/revert_39507_again
...
Temporarily remove a feature that doesn't work
2024-02-08 02:27:55 +01:00
Anton Popov
9a4dbc843a
validate type of arguments for minmax secondary index
2024-02-07 17:52:17 +00:00
Dmitry Novik
813020a1a6
remove redundant logging
2024-02-07 16:44:09 +01:00
Dmitry Novik
ff4729ccc4
Improve replaceTableExpressionAndRemoveJoin function
2024-02-07 16:40:23 +01:00
Smita Kulkarni
e594608c42
Merge branch 'master' into Azure_backup
2024-02-07 13:48:46 +01:00
Nikolai Kochetov
d1902cdba0
Fix some tests.
2024-02-07 12:36:24 +01:00
Nikolai Kochetov
06d112135e
Simplify prewhere push down from query plan. Try to always use it.
2024-02-07 12:36:20 +01:00
Nikolai Kochetov
a3d1b3f502
Merge branch 'master' into non-ready-set-ttl
2024-02-07 11:29:04 +00:00
Nikolai Kochetov
c434748fc1
Merge branch 'master' into allow-parallel-replicas-for-join-with-analyzer-2
2024-02-07 12:25:11 +01:00
Raúl Marín
38c8e1b8cc
Merge pull request #59485 from Algunenano/strict_int8
...
Strict aliasing for Int8
2024-02-07 11:33:06 +01:00
Alexander Gololobov
4dcd806dfc
Merge pull request #59545 from ClickHouse/system_zookeeper_stream
...
Produce stream of chunks from system.zookeeper instead of accumulating the whole result
2024-02-07 10:17:38 +01:00
Alexander Tokmakov
77e4b0c8d9
Revert "Merge pull request #59122 from arthurpassos/bring_back_attach_partition_pr"
...
This reverts commit ac90637170
, reversing
changes made to fe2b5cde05
.
2024-02-07 02:52:43 +01:00
Igor Nikonov
0c00435cea
Merge remote-tracking branch 'origin/master' into pr-better-replicas-failover-2
2024-02-07 00:52:50 +00:00
Alexey Milovidov
01c8542462
Merge pull request #59606 from vitlibar/fix-distributed-table-with-const-sharding-key
...
Fix distributed table with a constant sharding key
2024-02-06 22:10:34 +01:00
Raúl Marín
ff63a1cf9f
Fix tidy
2024-02-06 20:34:12 +01:00
Dmitry Novik
d558728281
Merge branch 'master' into storage-merge-aliases-analyzer
2024-02-06 20:24:20 +01:00
Dmitry Novik
1abcf26df6
Fix filter expressions
2024-02-06 15:39:52 +00:00
Anton Popov
addbd78341
Merge pull request #59436 from nickitat/perf_introspection_for_inserts
...
More perf introspection for `INSERT`-s
2024-02-06 14:59:32 +01:00
Alexander Gololobov
ca4f46ab41
Check block size in bytes instead of rows
2024-02-06 14:27:15 +01:00
Alexander Gololobov
a26e8b3a9e
Fix for filters w/o path column
2024-02-05 22:48:48 +01:00
Vitaly Baranov
86ac7b14d0
Fix Distributed table engine with a constant sharding key.
2024-02-05 21:56:29 +01:00
Igor Nikonov
dab078f7d2
Profile events
2024-02-05 20:21:48 +00:00
Nikolai Kochetov
6b06fcf7fc
Fix more tests.
2024-02-05 17:54:15 +00:00
Nikolai Kochetov
15bf263153
Support non global in mode.
2024-02-05 17:05:21 +00:00
Nikolai Kochetov
8692e8f752
Add settings to squash external table blocks.
2024-02-05 17:05:21 +00:00
Nikolai Kochetov
1892318e46
Squash temporary tables.
2024-02-05 17:05:20 +00:00
Nikolai Kochetov
b60228af3f
Allow to send a chain of subqueries for parallel replicas with analyzer.
2024-02-05 17:05:20 +00:00
Kseniia Sumarokova
83515e3656
Merge pull request #59543 from nickolaj-jepsen/master
...
Add support for NATS credentials file
2024-02-05 12:20:26 +01:00
Nikita Mikhaylov
6ba35b5459
Make comments for system tables also available in local mode and enforce comments not to be empty ( #59493 )
...
Co-authored-by: Alexey Milovidov <milovidov@clickhouse.com>
2024-02-05 11:17:58 +01:00
alesapin
db046c1bce
Merge pull request #59517 from ClickHouse/remove_scarry_message
...
Remove a scary message if an error is retryable
2024-02-04 17:43:06 +01:00
MikhailBurdukov
b90a5b94a8
Move setting to mergetree
2024-02-04 11:24:26 +00:00
Smita Kulkarni
b0994c5fa7
Addressed comments, added test for named collection
2024-02-04 11:28:20 +01:00
Alexander Gololobov
82caaa744a
Produce stream of chunks instead of accumulating the whole result
2024-02-03 16:08:41 +01:00
Nickolaj Jepsen
e822ba3a47
Add support for NATS credentials file
2024-02-02 23:34:01 +01:00
Dmitry Novik
05a7d22c24
WIP on calculate aliases only once
2024-02-02 13:45:04 +00:00
Alexander Sapin
d5e3b71590
Remove scarry message if error is retryable
2024-02-02 14:07:37 +01:00
Raúl Marín
87d493533c
Revert "Poco Logger small refactoring"
2024-02-02 12:10:05 +01:00
Alexander Gololobov
095648d7bc
Merge pull request #59388 from ClickHouse/system_zookeeper_retries
...
Retry disconnects and expired sessions when reading system.zookeeper
2024-02-02 11:52:15 +01:00
serxa
1d31ac2a0c
add FORGET PARTITION
query to remove old parition nodes from ZooKeeper
2024-02-02 10:03:31 +00:00
Alexey Milovidov
15467f2efb
Merge pull request #59451 from ClickHouse/remove-unused-headers
...
CLion says these headers are unused
2024-02-01 21:47:17 +01:00
Nikita Taranov
d73abc17f5
better
2024-02-01 21:42:22 +01:00
Nikita Taranov
698d00dbfe
add to MutateTask
2024-02-01 20:43:43 +01:00
Nikita Taranov
76f2ae08b5
collect stats build time
2024-02-01 18:16:09 +01:00
Nikita Taranov
4795149863
print stats once
2024-02-01 17:16:42 +01:00
Kseniia Sumarokova
e029021696
Merge pull request #59446 from ClickHouse/s3-queue-allow-to-add-start-point
...
Allow to define a starting point for s3queue ordered mode at creation
2024-02-01 16:46:24 +01:00
Alexey Milovidov
f52b08070f
Merge branch 'master' into remove-unused-headers
2024-02-01 13:42:37 +01:00
kssenii
3a16427e00
Fix test
2024-02-01 12:35:01 +01:00
Maksim Kita
ecbea6f625
Updated implementation
2024-02-01 12:51:50 +03:00
Kseniia Sumarokova
b3418c506e
Update src/Storages/S3Queue/S3QueueSettings.h
...
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
2024-02-01 10:28:33 +01:00
Dmitry Novik
5b630ed745
Merge pull request #59399 from azat/build/format-fwd-decl
...
Forward declaration for PeekableReadBuffer
2024-02-01 09:15:07 +01:00
Alexey Milovidov
33f6940214
CLion says these headers are unused
2024-02-01 03:50:09 +01:00
kssenii
361b2f107b
Allow to define a starting point for s3queue ordered mode
2024-01-31 21:28:18 +01:00
Alexander Gololobov
5947e5b5a8
Use retry settings and fault injection
2024-01-31 16:48:46 +01:00
Dmitry Novik
572d47acc7
Remove logging
2024-01-31 14:43:06 +00:00
Dmitry Novik
0d21004218
WIP on StorageMerge
2024-01-31 13:50:15 +00:00
Kseniia Sumarokova
b20567a055
Merge pull request #59167 from ClickHouse/s3-queue-parallelize-ordered-mode
...
S3Queue: allow parallel & disrtibuted processing for ordered mode
2024-01-31 11:20:41 +01:00
Kseniia Sumarokova
caaafbcedb
Merge pull request #56864 from ClickHouse/broken-projections-better-handling
...
Do not consider data part as broken if projection is broken
2024-01-31 11:17:15 +01:00
Dmitry Novik
ac17ad1e66
Merge pull request #59073 from ClickHouse/fix-analyzer-resolve-distr-id
...
Analyzer: Do not resolve remote table id on initiator
2024-01-31 09:08:30 +01:00
Nikita Taranov
1be8b61c5b
add test
2024-01-30 21:52:32 +01:00
Dmitry Novik
722f3db738
Merge remote-tracking branch 'origin/master' into storage-merge-aliases-analyzer
2024-01-30 19:50:01 +00:00
Nikita Taranov
17ab2674f4
impl
2024-01-30 20:35:10 +01:00
Azat Khuzhin
010f1c6cb7
Forward declaration for PeekableReadBuffer
...
ReadHelpers.h is very common header and is the root cause of "recompile
everything".
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-01-30 19:24:19 +01:00
Alexander Gololobov
c348c4e828
Move ZooKeeperRetries.h to Common
2024-01-30 19:02:17 +01:00
Alexey Milovidov
4ddba907c8
Update autogenerated version to 24.2.1.1 and contributors
2024-01-30 18:49:28 +01:00
Alexander Gololobov
82c06ca294
Use ZooKeeperRetriesControl
2024-01-30 18:31:37 +01:00
Alexander Gololobov
023b8cbd53
Retry disconnects and expired sessions
2024-01-30 17:47:11 +01:00
kssenii
145f6a31cb
Fix
2024-01-30 16:29:40 +01:00
MikhailBurdukov
778efb8953
Merge branch 'master' into backoff_for_failed_mutations
2024-01-30 16:08:52 +03:00
MikhailBurdukov
5e845172da
Enable per query setting
2024-01-30 12:42:03 +00:00
Kruglov Pavel
3105960fd8
Merge pull request #59352 from Avogar/fix-bad-attached-rabbitmq
...
Fix lazy initialization in RabbitMQ
2024-01-30 12:52:07 +01:00
Kseniia Sumarokova
e16aaa8d81
Merge branch 'master' into broken-projections-better-handling
2024-01-30 10:54:22 +01:00
Dmitry Novik
5a34641e91
Merge remote-tracking branch 'origin/master' into fix-analyzer-resolve-distr-id
2024-01-30 07:17:15 +00:00
Alexey Milovidov
35ea956255
Merge pull request #59295 from ClickHouse/fix_block_number
...
Fix `_block_number` column with non-default serializations and mutation
2024-01-30 04:07:09 +01:00
Alexey Milovidov
ac90637170
Merge pull request #59122 from arthurpassos/bring_back_attach_partition_pr
...
Revert revert attach partition PR
2024-01-30 04:02:31 +01:00
Alexey Milovidov
de341a6d6e
Merge pull request #59351 from ClickHouse/fix-non-redy-set-system-tables
...
Fix not-ready set for system.tables
2024-01-30 03:59:18 +01:00
Alexey Milovidov
50532c485f
Merge pull request #59340 from CurtizJ/lower-memory-usage-vertical-merges
...
Lower memory usage in vertical merges
2024-01-30 03:42:28 +01:00
avogar
3a5ba56c36
Fix lazy initialization in RabbitMQ, fix possible deadlock on insert into unitialized queue engine
2024-01-29 20:09:09 +00:00
Robert Schulze
2cfcaf8ff4
Merge pull request #59040 from MochiXu/fix_remove_inverted_index_files
...
`DROP INDEX` of inverted index now removes all relevant files from persistence
2024-01-29 20:39:56 +01:00
kssenii
df7d3b8a40
Allocate current shard number from keeper
2024-01-29 19:36:16 +01:00
Arthur Passos
15d04d0a78
Merge branch 'master' into bring_back_attach_partition_pr
2024-01-29 15:16:49 -03:00
Arthur Passos
516daadeef
add static storage assert in datapartcloner
2024-01-29 15:16:00 -03:00
Nikolai Kochetov
d0143380d7
Fix not-ready set for system.tables
2024-01-29 17:53:59 +00:00
Nikolai Kochetov
5b2648929b
Merge pull request #59273 from kitaisreal/atomic-logger
...
Added AtomicLogger
2024-01-29 16:49:18 +01:00
avogar
377937d415
Merge branch 'master' of github.com:ClickHouse/ClickHouse into auto-format-detection
2024-01-29 15:45:18 +00:00
Anton Popov
d3cdc8826e
lower memory usage in vertical merges
2024-01-29 14:06:46 +00:00
kssenii
d0ea5392db
Fix storing iterated keys
2024-01-29 12:36:32 +01:00
kssenii
ef252d8376
Validate settings with keeper, add a test
2024-01-29 12:36:00 +01:00
Kruglov Pavel
6858d2f4ca
Merge pull request #58047 from Avogar/variant-data-type
...
Implement Variant data type
2024-01-29 11:36:08 +01:00
Smita Kulkarni
3183026521
Merge branch 'master' into Azure_backup
2024-01-29 11:08:13 +01:00
Kseniia Sumarokova
a29ab8f29c
Merge pull request #59170 from azat/disks/s3-plain-improvements
...
Small improvements for tables on write-once (s3_plain)/read-only (web) disks
2024-01-29 10:44:37 +01:00
Kseniia Sumarokova
0f88da5fd4
Merge pull request #59275 from ClickHouse/Avogar-patch-1
...
Fix abort in iceberg metadata on bad file paths
2024-01-29 10:35:34 +01:00
alesapin
b864b122fa
Merge branch 'master' into fix_block_number
2024-01-29 10:19:43 +01:00
Alexey Milovidov
2cb2bcfbc3
Merge pull request #59315 from ClickHouse/support-backups-for-compressed-memory-tables
...
Support backups for compressed in-memory tables
2024-01-29 01:50:41 +01:00
Alexey Milovidov
aec3f28ccb
Support backups for compressed in-memory tables
2024-01-28 23:06:50 +01:00
Arthur Passos
0bbfef884c
Merge branch 'ClickHouse:master' into bring_back_attach_partition_pr
2024-01-28 17:43:38 -03:00
Anton Popov
ef133bfd47
Merge remote-tracking branch 'upstream/master' into lower-memory-usage
2024-01-28 15:07:44 +00:00
avogar
92db86b829
Merge branch 'master' of github.com:ClickHouse/ClickHouse into variant-data-type
2024-01-28 14:33:49 +00:00
serxa
963e54cfd1
Merge branch 'master' into cpu-scheduler-and-cc
2024-01-28 13:09:58 +00:00
Maksim Kita
07750ee949
Added AtomicLogger
2024-01-28 13:55:44 +03:00
alesapin
7022ca9ece
Fix _block_number column with non-default serializations and mutation
2024-01-27 20:24:03 +01:00
Azat Khuzhin
4248ae509a
Support specifying settings for Distributed engine in config (like for MergeTree)
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-01-27 18:06:16 +01:00
Anton Popov
9d3b4e5308
lower memory usage
2024-01-27 16:40:11 +00:00
Azat Khuzhin
d94547d338
Use fresh ZooKeeper client on DROP (to have higher chances on success)
...
In case of DROP the client can be expired, and even though
StorageReplicatedMergeTree::drop() uses getZooKeeperIfTableShutDown(),
which creates new client if current is expired, it is not enough, since
current_zookeeper (cached client) will be used implicitly from
dropAllData().
This had been found by stress tests, that leads to DROP query hang [1].
[1]: https://s3.amazonaws.com/clickhouse-test-reports/59255/94eb33ef27a9ab7c4a99af40772ea287e67efcbf/stress_test__tsan_.html
<details>
<summary>stacktrace</summary>
2024.01.26 17:41:41.138577 [ 3319 ] {} <Error> DatabaseCatalog: Cannot drop table test_vzfk1xoc.alter_table1 (455a674c-161d-44d8-abc2-fd205bad1116). Will retry later.: Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):
0. ./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000001e06a5e3 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000f697c74 in /usr/bin/clickhouse
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000001adc298b in /usr/bin/clickhouse
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000001adfb418 in /usr/bin/clickhouse
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1343: Coordination::ZooKeeper::get(String const&, std::function<void (Coordination::GetResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000001adfd8e5 in /usr/bin/clickhouse
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001ad890bd in /usr/bin/clickhouse
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001ad88a36 in /usr/bin/clickhouse
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:580: zkutil::ZooKeeper::tryGetWatch(String const&, String&, Coordination::Stat*, std::function<void (Coordination::WatchResponse const&)>, Coordination::Error*) @ 0x000000001ad898bf in /usr/bin/clickhouse
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:570: zkutil::ZooKeeper::tryGet(String const&, String&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&, Coordination::Error*) @ 0x000000001ad89554 in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ZooKeeper/ZooKeeperWithFaultInjection.cpp:0: DB::ZooKeeperWithFaultInjection::tryGet(String const&, String&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&, Coordination::Error*) @ 0x000000001ae110dc in /usr/bin/clickhouse
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::StorageReplicatedMergeTree::unlockSharedDataByID(String, String const&, DB::MergeTreePartInfo const&, String const&, String const&, std::shared_ptr<DB::ZooKeeperWithFaultInjection> const&, DB::MergeTreeSettings const&, std::shared_ptr<Poco::Logger>, String const&, StrongTypedef<unsigned int, DB::MergeTreeDataFormatVersionTag>) @ 0x000000001975527e in /usr/bin/clickhouse
11. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::removeSharedDetachedPart(std::shared_ptr<DB::IDisk>, String const&, String const&, String const&, String const&, String const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<zkutil::ZooKeeper> const&) @ 0x000000001976c980 in /usr/bin/clickhouse
12. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:10171: DB::StorageReplicatedMergeTree::removeDetachedPart(std::shared_ptr<DB::IDisk>, String const&, String const&) @ 0x000000001976bdc6 in /usr/bin/clickhouse
13. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:2798: DB::MergeTreeData::dropAllData() @ 0x0000000019c643c8 in /usr/bin/clickhouse
14. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:1117: DB::StorageReplicatedMergeTree::drop() @ 0x0000000019664e0a in /usr/bin/clickhouse
15. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:0: DB::DatabaseCatalog::dropTableFinally(DB::DatabaseCatalog::TableMarkedAsDropped const&) @ 0x0000000017bf1fac in /usr/bin/clickhouse
16. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:0: DB::DatabaseCatalog::dropTableDataTask() @ 0x0000000017bf175e in /usr/bin/clickhouse
17. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000017bf94c2 in /usr/bin/clickhouse
18. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x00000000170bd0b8 in /usr/bin/clickhouse
19. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000170c06ce in /usr/bin/clickhouse
20. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000170c0feb in /usr/bin/clickhouse
21. ./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f7c6a2f in /usr/bin/clickhouse
22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f7cc8f2 in /usr/bin/clickhouse
23. ? @ 0x000000000733904f in /usr/bin/clickhouse
24. ? @ 0x00007f28b2319ac3
25. ? @ 0x00007f28b23ab850
</details>
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-01-27 16:04:18 +01:00
kssenii
0a21e6aa5d
Fix
2024-01-26 19:07:30 +01:00
avogar
5833641fa5
Merge branch 'master' of github.com:ClickHouse/ClickHouse into variant-data-type
2024-01-26 16:54:02 +00:00
Kruglov Pavel
8fcedddcda
Fix abort in iceberg metadata on bad file paths
2024-01-26 17:46:49 +01:00
Azat Khuzhin
90ab986be6
Revert "Prohibit CREATE on static storages (ATTACH should be used instead)"
...
CREATE is used by Replicated database even for ATTACH, and anyway ATTACH
creates format_version.txt as well (without previuos patch).
This reverts commit 323f1f320ebe7e588d443abf78efa2c60193e7a9.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-01-26 16:47:36 +01:00
Azat Khuzhin
7ba31bf55c
Do not create format_version.txt on write-once storages
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-01-26 16:47:36 +01:00
Azat Khuzhin
66a40bda0a
Revert "Prohibit DROP from static storages"
...
It is actually OK to DROP at least from read-only disk, as for the
write-once it should be OK as well, since CREATE is prohibited anyway,
so there should be no leftovers
CI: https://s3.amazonaws.com/clickhouse-test-reports/59170/c82050d1925439f0ede2b32acb5b1b8df4acae5d/stress_test__asan_.html
CI: https://s3.amazonaws.com/clickhouse-test-reports/59170/c82050d1925439f0ede2b32acb5b1b8df4acae5d/integration_tests__asan__[3_4].html
This reverts commit 082e62712c5e2f4c48d1abc6141ccfd1249d26b4.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-01-26 16:47:35 +01:00
Azat Khuzhin
aafc149c1e
Prohibit CREATE on static storages (ATTACH should be used instead)
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-01-26 16:47:35 +01:00
Azat Khuzhin
1ee723d50d
Prohibit DROP from static storages
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-01-26 16:47:35 +01:00
Azat Khuzhin
f3b4f28f0d
Disable mutable operations into {Replicated,}MergeTree with static storage
...
Mutable operations includes:
- INSERT
- OPTIMIZE
- parts manipulation
- ...
But does not include DROP, since you need an ability to DROP it.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-01-26 16:47:35 +01:00
Azat Khuzhin
6751e01cc3
Do not drop parts for static (ro/write-once) storages for Replicated engine
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-01-26 16:47:35 +01:00
kssenii
5cd757dd9a
Merge remote-tracking branch 'origin/master' into s3-queue-parallelize-ordered-mode
2024-01-26 16:33:22 +01:00
Kseniia Sumarokova
99653fc778
Merge branch 'master' into broken-projections-better-handling
2024-01-26 16:27:23 +01:00
kssenii
ee15b6e646
Fix
2024-01-26 16:26:59 +01:00
Kruglov Pavel
946c4e0495
Merge branch 'master' into auto-format-detection
2024-01-26 15:51:35 +01:00
Kruglov Pavel
692d37306e
Fix build
2024-01-26 15:50:16 +01:00
Arthur Passos
ce6877cbb6
Merge branch 'ClickHouse:master' into bring_back_attach_partition_pr
2024-01-26 10:57:11 -03:00
Kruglov Pavel
46a6b84a5a
Merge branch 'master' into auto-format-detection
2024-01-25 22:11:07 +01:00
Maksim Kita
5bb734a4bb
ActionsDAG buildFilterActionsDAG refactoring
2024-01-25 18:24:14 +03:00
Maksim Kita
2a327107b6
Updated implementation
2024-01-25 14:31:49 +03:00
Maksim Kita
42029f42e7
Table CREATE DROP Poco::Logger memory leak fix
2024-01-25 14:31:18 +03:00
Kseniia Sumarokova
951c874aa9
Merge pull request #59080 from ClickHouse/data-lakes-survive-restart
...
Allow server to start with broken data lake table and fix itself lazily
2024-01-25 11:50:16 +01:00
Nikita Taranov
1e3f6894a3
Merge pull request #59089 from nickitat/sort_res_ranges_before_sending_to_replica
...
Coordinator returns ranges for reading in sorted order
2024-01-24 21:04:53 +01:00
avogar
11f1ea50d7
Fix tests
2024-01-24 17:55:31 +00:00
kssenii
5ecf36f275
Fix style check
2024-01-24 17:19:52 +01:00
Dmitry Novik
95e2a3d6da
Merge pull request #58911 from ClickHouse/fix-index-hint-crash
...
Fix crash when indexHint() is used
2024-01-24 17:16:22 +01:00
kssenii
7f8f379d7f
Parallel & disrtibuted processing for ordered mode
2024-01-24 16:32:15 +01:00
Kruglov Pavel
89bafbcda6
Merge pull request #59133 from Avogar/allow-to-ignore-schema-evolution-in-iceberg
...
Allow to ignore schema evolution in Iceberg table engine under a setting
2024-01-24 13:19:09 +01:00
Smita Kulkarni
4d92eba316
Merge branch 'master' into Azure_backup
2024-01-24 11:13:57 +01:00
avogar
f05174e441
Fix style
2024-01-23 18:59:40 +00:00
avogar
617cc514b7
Try to detect file format automatically during schema inference if it's unknown
2024-01-23 18:59:39 +00:00
Arthur Passos
dba792dff0
Remove duplicate includ
2024-01-23 15:03:31 -03:00
Arthur Passos
e6b17377d0
Merge branch 'master' into bring_back_attach_partition_pr
2024-01-23 14:52:47 -03:00
avogar
be3b5dc45f
Allow to ignore schema evolution in Iceberg table engine under a setting
2024-01-23 17:45:41 +00:00
Nikolai Kochetov
d64b491b45
Fixing build
2024-01-23 17:00:21 +00:00
Nikolai Kochetov
a26f8f5425
Merge pull request #59076 from ClickHouse/revert-59059-revert-58838-allow-parallel-replicas-for-join-with-analyzer
...
Revert "Revert "Allow parallel replicas for JOIN with analyzer [part 1].""
2024-01-23 17:01:19 +01:00
Arthur Passos
5179891aef
remove static log
2024-01-23 11:43:08 -03:00
Arthur Passos
799a94081b
Revert "Revert "Allow to attach partition from table with different partition expression when destination partition expression doesn't re-partition""
...
This reverts commit a1c83e2f51
.
2024-01-23 11:40:42 -03:00
Robert Schulze
a1c83e2f51
Revert "Allow to attach partition from table with different partition expression when destination partition expression doesn't re-partition"
2024-01-23 15:26:04 +01:00
Anton Popov
bceda4cd29
Merge remote-tracking branch 'upstream/master' into HEAD
2024-01-23 14:03:34 +00:00
Anton Popov
4292ee7d51
Merge pull request #58886 from CurtizJ/better-read-buffers
...
Less error prone interface of read buffers
2024-01-23 14:59:27 +01:00
Alexander Gololobov
9a0e5d0aac
Merge pull request #58914 from ClickHouse/ot_spans_in_insert
...
Opentelemetry spans to analyze CPU and S3 bottlenecks on inserts
2024-01-23 13:53:13 +01:00
Nikolai Kochetov
fcbab407b8
Merge branch 'master' into revert-59059-revert-58838-allow-parallel-replicas-for-join-with-analyzer
2024-01-23 12:21:44 +01:00
kssenii
bca73d4b0a
Fix
2024-01-23 11:51:21 +01:00
kssenii
b76630c955
Merge remote-tracking branch 'origin/master' into broken-projections-better-handling
2024-01-23 11:34:58 +01:00
Robert Schulze
e67076ea2b
Merge pull request #55201 from canhld94/column_level_compress_block
...
Column-level compression block sizes
2024-01-23 11:22:03 +01:00
Robert Schulze
bfec324b28
Some fixups + test
2024-01-23 10:17:16 +00:00
Robert Schulze
81543c0a46
Merge remote-tracking branch 'rschu1ze/master' into fix_remove_inverted_index_files
2024-01-23 09:45:09 +00:00
Nikita Taranov
09e24ed6c5
Merge pull request #58068 from zhongyuankai/limit_materialized_view_size
...
A limit on the number of materialized views attached to a table.
2024-01-22 22:50:56 +01:00
Nikita Taranov
b161921825
impl
2024-01-22 22:33:23 +01:00
Max K
4932b1cad9
Merge branch 'master' into better-read-buffers
2024-01-22 21:52:55 +01:00
Arthur Passos
24b8bbe9fa
Allow to attach partition from table with different partition expression when destination partition expression doesn't re-partition ( #39507 )
...
* temp commit
* temp commit
* draft impl for feedback
* fix weird style changes
* fix weird style changes
* fix weird style changes
* fix weird style changes
* fix weird style changes
* aa
* aa
* Add integ tests and remove partition key restriction
* fix small incosistency in partition id
* style fix
* style fix
* style fix
* use existing DataPartStorageBuilder instead of new one
* Refactor part clone to make it more readable and maintainable
* Add MergeTreeDataPartCloner docs
* define ErrorCodes::BAD_ARGUMENTS
* Rebase
* camel case methods
* address some comments
* yet another rebase?
* Move from integ tests to stateless tests
* address more comments
* add finalize on min_max_idx files
* Add sync option to DistinctPartitionExpCloner
* just a temp test
* revert temp change
* Use echoOn to distinguish test queries
* remove comment
* fix build issue during rebase
* atempt to fix build after rebase
* finally fix build
* clear minmaxidx hyperrectangle before loading it
* Fix error on min_max files deletion where it was being assumed that partition expression contained all columns
* get it to the state it was previously
* add missing include
* getting functional?
* refactoring and renaming
* some more refactoring
* extern bad arguments
* try to fix style
* improvements and docs
* remove duplicate includes
* fix crash
* make tests more stable by ordering
* rebase once again..
* fix
* make ci happy?
* fix rebase issues
* docs
* rebase, but prolly needs to be improved
* refactor out from nasty inheritance to static methods
* fix style
* work around optional
* refactor & integrate some changes
* update column_type
* add tests by dencrane
* set utc
* fix ref file
* fix tests
* use MergeTree instead of SummingMergeTree
* mark MergeTreeDataPart::getBlock as const
* address a few comments
* compute module function name size at compile time
* simplify branching in getPartitionAstFieldsCount
* remove column_indexes argument
* merge getBlock with buildBlock
* add some const specifiers
* small adjustments
* remove no longer needed isNull check
* use std::min and max to update global min max idx
* add some assertions
* forward declare some symbols
* fix grammar
* forward decl
* try to fix build..
* remove IFunction forward decl
* Revert "use std::min and max to update global min max idx"
This reverts commit b2fe79dda7
.
* Revert "remove no longer needed isNull check"
This reverts commit 129db2610f
.
* Revert "Revert "remove no longer needed isNull check""
This reverts commit 9416087dd8
.
* Revert "Revert "use std::min and max to update global min max idx""
This reverts commit 20246d4416
.
* remove some comments
* partial use of MonotonicityCheckMatcher
* ranges
* remove KeyDescriptionMonotonicityChecker
* remove duplication of applyfunction
* move functions to anonymous namespace
* move functions to cpp
* Relax partition compatibility requirements by accepting subset, add tests from partitioned to unpartitioned
* updte reference file
* Support for partition by a, b, c to partition by a, b
* refactoring part 1
* refactoring part 2, use hyperrectangle, still not complete
* refactoring part 3, build hyperrectangle with intersection of source & destination min max columns
* Support attaching to table with partition expression of multiple expressions
* add tests
* rename method
* remove some code duplication
* draft impl for replicatedmergetree, need to dive deeper
* ship ref file
* fix impl for replicatedmergetree..
* forbid attach empty partition replicatedmergetree
* Add replicated merge tree integration tests
* add test missing files
* fix black
* do not check for monotonicity of empty partition
* add empty tests & fix replicated
* remove no longer needed buildBlockWithMinMaxINdexes
* remove column logic in buildHyperrectangle
* simplify implementation by using existing methods
* further simplify implementation
* move all MergeTreeDataPartClone private methods to .cpp file
* decrease decomposition
* use different namespaces
* reduce code duplication
* fix style
* address a few comments
* add chassert to assert arguments size on MonotonicityCheckVisitor
* remove deleteMinMaxFiles method
* remove useless checks from sanitycheck
* add tests for attach partition (not id)
* Remove sanityCheckASTPartition and bring back conditional getPartitionIDFromQuery
* remove empty block comment
* small fixes
* fix formatting
* add missing include
* remove duplicate iuncludes
* trigger ci
* reduce some code duplication
* use updated partition id on replicatedmergetree
* fix build
* fix build
* small refactor
* do not use insert increment on fetch part
* remove duplicate includes
* add one more integ test
* black
* black
* rely on partition exp instead of partition id on replicated part fetch to decide if it is a different partition exp
* add one more integ test
* add order by clause
* fix black
---------
Co-authored-by: Alexey Milovidov <milovidov@clickhouse.com>
2024-01-22 21:00:04 +01:00
kssenii
1aa8e62bb7
Fix
2024-01-22 18:26:44 +01:00
Nikolai Kochetov
65c5ea31a8
Revert "Revert "Allow parallel replicas for JOIN with analyzer [part 1].""
2024-01-22 17:09:34 +01:00
Dmitry Novik
4e5249275e
Do not resolve remote table id on initiator
2024-01-22 16:45:25 +01:00
Smita Kulkarni
2e7ce5b0e2
Updated settings ptr and fetching of client from Disk & ObjectStorage
2024-01-22 16:24:43 +01:00
Anton Popov
e1e7a9cd45
Merge branch 'master' into better-read-buffers
2024-01-22 15:50:48 +01:00
Alexander Tokmakov
c2202ff347
Skip unavailable replicas in parallel distributed insert select ( #58931 )
...
* Update StorageDistributed.cpp
* Update StorageDistributed.cpp
* Update StorageDistributed.cpp
* add a test
* Update 01099_parallel_distributed_insert_select.sql
2024-01-22 15:34:46 +01:00
mochi
0606a77267
Update src/Storages/MergeTree/MutateTask.cpp
...
Co-authored-by: Dmitry Novik <mrnovikd@gmail.com>
2024-01-22 22:02:50 +08:00
Alexander Tokmakov
4315d44517
Revert "Allow parallel replicas for JOIN with analyzer [part 1]."
2024-01-22 13:30:35 +01:00
Amos Bird
18d8469667
Remove another comment
2024-01-22 20:09:20 +08:00
Amos Bird
ce3242170b
Remove comment
2024-01-22 20:08:23 +08:00
Amos Bird
e0e2e35b9c
Fix some comments
2024-01-22 20:03:43 +08:00
Amos Bird
b2e6a0d69a
Style fix
2024-01-22 20:03:42 +08:00
Amos Bird
b7fbd9f0f2
Make it work
2024-01-22 20:03:41 +08:00
serxa
e6d1ba46df
Merge branch 'master' into cpu-scheduler-and-cc
2024-01-22 11:18:42 +00:00
Kseniia Sumarokova
aec76d6653
Merge pull request #58877 from ClickHouse/multiple-read-file-log-storage-in-mv
...
Multiple read file log storage in mv
2024-01-22 12:02:41 +01:00
MochiXu
02b178cc9c
fix drop inverted index
2024-01-22 11:10:58 +08:00
serxa
799b8d6356
support resource request canceling
2024-01-21 19:00:40 +00:00
Vitaly Baranov
01aafda27b
Merge pull request #58804 from vitlibar/allow-kill-query-for-backups
...
Allow KILL QUERY for backups
2024-01-20 23:48:18 +01:00
Igor Nikonov
4be068c73b
Merge pull request #58993 from ClickHouse/revert-58992-revert_flaky
...
Revive: Parallel replicas custom key: skip unavailable replicas
2024-01-20 19:44:56 +01:00
robot-clickhouse
273a06856a
Merge pull request #59025 from ClickHouse/cpu-scheduler
...
Split resource scheduler off `IO/` into `Common/Scheduler/`
2024-01-20 15:43:50 +01:00
Kseniia Sumarokova
5d67e26fc4
Merge pull request #58869 from ClickHouse/force-check-for-cached-custom-disk
...
Fix for custom cached disk creation
2024-01-20 14:47:36 +01:00
Kseniia Sumarokova
689e617c12
Merge pull request #58963 from ClickHouse/follow-up-disk-configuration
...
Follow up to #58357
2024-01-20 14:41:34 +01:00
serxa
824deab761
split resource scheduler off IO/
into Common/Scheduler/
2024-01-20 12:06:26 +00:00
Igor Nikonov
59025b2d3a
Merge remote-tracking branch 'origin/master' into revert-58992-revert_flaky
2024-01-19 23:38:24 +00:00
Kruglov Pavel
20379d34d5
Merge branch 'master' into variant-data-type
2024-01-19 23:17:42 +01:00
Michael Kolupaev
fd361273f0
Fix StorageURL forgetting headers on server restart
2024-01-19 11:35:12 -08:00
Alexey Milovidov
decdb62cef
Merge pull request #58800 from ZhiguoZh/20240115-combine-filter-bmi2
...
Optimize DB::combineFilters with BMI2/SSE intrinsics
2024-01-19 16:50:04 +01:00
Dmitry Novik
728276ab3f
Fix crash when indexHint() is used
2024-01-19 14:59:33 +00:00
zhongyuankai
cbb51659f6
Merge branch 'master' into limit_materialized_view_size
2024-01-19 22:00:12 +08:00
Igor Nikonov
131f71b50a
Revert "Revert flaky"
2024-01-19 13:51:30 +01:00
Raúl Marín
477489a05d
Revert "Merge pull request #57235 from ClickHouse/pr-custom-key-failover"
...
This reverts commit 8ba9b4a7ef
, reversing
changes made to 0f39245cb8
.
2024-01-19 12:46:05 +00:00
kssenii
2f34ef6ff8
Follow up
2024-01-19 11:19:59 +01:00
kssenii
48e489682b
Merge remote-tracking branch 'origin/master' into broken-projections-better-handling
2024-01-19 10:49:27 +01:00
Kseniia Sumarokova
1c74e7fa08
Merge pull request #58902 from ClickHouse/fs-cache-refactorings
...
fs cache: minor refactoring
2024-01-19 10:25:45 +01:00
MikhailBurdukov
ee5d8c0a27
Review
2024-01-19 09:04:02 +00:00
Robert Schulze
1bfeee1954
Some fixups
2024-01-18 20:49:50 +00:00
Robert Schulze
15700592f7
Merge remote-tracking branch 'ClickHouse/master' into column_level_compress_block
2024-01-18 19:12:57 +00:00
Kruglov Pavel
5444cde408
Merge branch 'master' into variant-data-type
2024-01-18 18:31:27 +01:00
avogar
f202d71371
Make alter with variant espansion as no-op, add tests for alters
2024-01-18 17:29:36 +00:00
MikhailBurdukov
97e0e5fc9c
typo
2024-01-18 15:43:43 +00:00
MikhailBurdukov
be58c1fe44
Fix style
2024-01-18 15:43:43 +00:00
MikhailBurdukov
f6fb20d6cf
Backoff policy for failed mutation.
2024-01-18 15:43:43 +00:00
Dmitry Novik
ba2dad98bb
Merge pull request #58913 from ClickHouse/fix-style
...
Style fix
2024-01-18 14:24:39 +01:00
Nikolai Kochetov
9db8b50753
Merge pull request #58838 from ClickHouse/allow-parallel-replicas-for-join-with-analyzer
...
Allow parallel replicas for JOIN with analyzer [part 1].
2024-01-18 14:01:08 +01:00
János Benjamin Antal
f34d1ae647
Use WithContext
2024-01-18 11:47:59 +00:00
Smita Kulkarni
e71977b724
Merge branch 'master' into Azure_backup
2024-01-18 08:29:01 +01:00
Duc Canh Le
22afcd46f4
add ALTER COLUMN MODIFY/RESET SETTING
...
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
2024-01-18 05:17:07 +00:00
Alexander Gololobov
0a42dbafe8
Add attributes
2024-01-17 17:46:15 +01:00
Anton Popov
72b0fed4ef
Merge pull request #58729 from CurtizJ/nested-size0
...
Fix reading of offsets subcolumn (`size0`) from `Nested`
2024-01-17 17:16:27 +01:00
Alexander Gololobov
7dda18c209
Spans in mergeBlock and finalizeImpl
2024-01-17 16:07:53 +01:00
Dmitry Novik
3a1c3638e8
Style fix
2024-01-17 15:04:50 +00:00
Anton Popov
245e1cbd5c
Merge remote-tracking branch 'upstream/master' into better-read-buffers
2024-01-17 13:46:48 +00:00
kssenii
c905937ef6
Refactoring
2024-01-17 13:22:59 +01:00
Dmitry Novik
a5e0be484a
Merge pull request #58875 from ClickHouse/fix-indexhint-rpn-builder
...
Fix RPN construction for indexHint
2024-01-17 12:24:28 +01:00
Smita Kulkarni
8d88f4cf87
Update setting is_parameterized_view & settings columns for view
2024-01-17 09:42:53 +01:00
Anton Popov
299293c088
better interface in read buffers
2024-01-16 23:26:20 +00:00
János Benjamin Antal
c9b6d53948
Unify naming
2024-01-16 16:16:08 +00:00
Dmitry Novik
80291ec3f2
Add comments
2024-01-16 16:59:04 +01:00
Dmitry Novik
c325f08e79
Fix RPN construction for indexHint
2024-01-16 15:34:45 +00:00
János Benjamin Antal
1d34f4a304
Merge pull request #58809 from ClickHouse/print-entries-to-be-processed-in-trace
...
Increase log level to trace to help debug `00993_system_parts_race_condition_drop_zookeeper`
2024-01-16 16:14:22 +01:00
Kruglov Pavel
74973c1b41
Merge pull request #58537 from sunny19930321/fix/hive-threadpool-query
...
Fix hive threadpool read ORC/Parquet/... Failed
2024-01-16 14:46:22 +01:00
kssenii
8decf5039a
Fix for custom cached disk
2024-01-16 14:41:12 +01:00
Nikita Taranov
7d03ccd8db
Merge pull request #58844 from nickitat/fixxxx
...
Fix reading from an invisible part in new (not released yet) parallel replicas coordinator
2024-01-16 14:09:21 +01:00
Nikolai Kochetov
534e086a5d
Update StorageMergeTree.cpp
2024-01-16 13:50:39 +01:00
Smita Kulkarni
76ba60aa24
Merge branch 'master' into Support_parameterized_view_with_analyzer
2024-01-16 11:26:41 +01:00
zhongyuankai
cca2fa6976
fix test
2024-01-16 15:39:33 +08:00
Zhiguo Zhou
7450c35189
Optimize DB::combineFilters with BMI2/SSE intrinsics
...
The DB::combineFilters combines two UInt8 columns (filters) by
scanning the first column for non-zero elements and replacing them
with elements in the second column contiguously.
As a follow-up of #56079 , the function is optimized with the BMI2
and SSE intrinsics so that the x86 devices without AVX512 support
could also accelerate the column combinations via data parallelism.
2024-01-16 15:33:27 +08:00
Nikita Taranov
b9132f7f69
add check in read pool
2024-01-16 00:57:10 +01:00
Nikita Taranov
90d30b6d11
impl
2024-01-15 23:56:53 +01:00
alesapin
f1b1096eb4
Fix isRetry
method
2024-01-15 20:10:21 +01:00
Nikolai Kochetov
733ceaa775
Allow parallel replicas for JOIN with analyzer.
2024-01-15 18:48:12 +00:00
Anton Popov
c8f3fad8de
fix clang-tidy
2024-01-15 18:25:15 +00:00
Anton Popov
134d389bea
Merge remote-tracking branch 'upstream/master' into HEAD
2024-01-15 18:23:25 +00:00
Anton Popov
b430f3094e
fix nested size0 for storage Log
2024-01-15 17:26:44 +00:00
Kseniia Sumarokova
7ded82e51f
Merge pull request #58297 from azat/qp-alter-part
...
Support query parameters in ALTER TABLE ... PART
2024-01-15 18:18:34 +01:00
János Benjamin Antal
b04e8b11ae
Delay reading from StorageFileLog by creating a query plan step
2024-01-15 11:39:43 +00:00
János Benjamin Antal
3e1d7bf685
Merge pull request #58477 from ClickHouse/multiple-read-kafka-storage-in-mv
...
Delay reading from StorageKafka to allow multiple reads in materialized views
2024-01-15 12:31:59 +01:00
Azat Khuzhin
48a700edba
Support query parameters in ALTER TABLE ... PART
...
But, to make this happen it is not enough to simply fix the parser, the
whole ASTs for ALTER should be fixed as well, since they store `ASTPtr`
instead of `IAST *`, and so because of `ReplaceQueryParameterVisitor`
does not work for ALTER query.
So this patch converts ASTPtr to `IAST *` for ASTAlterQuery and
ASTAlterCommand (plus it also fixes when the children for
ASTAlterCommand had not been updated sometimes).
And only after this it is possible to adjust the parser (see
places with `parser_string_and_substituion` in the parser code).
v2: also fix ASTAlterCommand::clone (add missing components) and ParserAlterCommand::parseImpl
v3: fix APPLY DELETED MASK
v4: fix mutations_execute_nondeterministic_on_initiator/mutations_execute_subqueries_on_initiator
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-01-15 14:16:54 +03:00
János Benjamin Antal
f4e8e47c7a
Increase log level to trace
2024-01-15 10:44:40 +00:00
Robert Schulze
9cfdff2ddb
Merge pull request #58284 from lingtaolf/Improvement/inverted_index_support_match
...
Utilize inverted index by function `match()`
2024-01-15 11:37:23 +01:00
zhongyuankai
826b29ef4d
Max materialized views size for table
2024-01-15 10:02:57 +08:00
Alexey Milovidov
a471a54987
Merge pull request #55528 from ClickHouse/insert-quorum-host-name-check
...
Insert quorum: check host node version in addition
2024-01-15 02:23:15 +01:00
Vitaly Baranov
3d7a17af08
Allow KILL QUERY to cancel backups/restores.
2024-01-15 02:22:35 +01:00
Alexander Tokmakov
5128607995
Merge pull request #58393 from jaymebrd/feat/add-from-modifier-sync-replica-lightweight
...
add `from srcreplicas` modifier for `SYSTEM SYNC REPLICA LIGHTWEIGHT`
2024-01-14 20:53:16 +01:00
Nikita Taranov
df0c6cc029
Fix bug in new (not released yet) parallel replicas coordinator ( #58722 )
2024-01-14 16:16:20 +01:00
Duc Canh Le
ac76eecb1e
Merge branch 'master' into column_level_compress_block
...
Resolve conflicts
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
2024-01-14 09:32:43 +00:00
Alexey Milovidov
65ac905032
Merge pull request #54366 from canhld94/final_no_copy
...
Implement vertical replacing FINAL
2024-01-14 04:21:07 +01:00
Alexey Milovidov
e92e3615e3
Merge branch 'master' into non-ready-set-ttl
2024-01-13 21:11:51 +01:00
Alexey Milovidov
12585ea0e4
Update TTLDescription.cpp
2024-01-13 23:10:27 +03:00
Alexey Milovidov
211c285a0b
Merge pull request #58657 from ClickHouse/analyzer-fix-index-set
...
Analyzer: Correctly handle constant set in index
2024-01-13 19:23:28 +01:00
Alexey Milovidov
d10a6e91af
Merge pull request #58770 from hdnpth/add-missing-includes
...
Add missing includes when _LIBCPP_REMOVE_TRANSITIVE_INCLUDES enabled
2024-01-13 19:22:49 +01:00
Alexey Milovidov
af8c8923cb
Merge branch 'master' into analyzer-fix-index-set
2024-01-13 04:15:10 +01:00
Alexey Milovidov
d112492c56
Remove some code
2024-01-13 03:48:04 +01:00
Artem Alperin
4139cb24ee
Add missing includes
2024-01-13 01:48:55 +03:00
Nikita Mikhaylov
8edab24189
Add comments to all columns of system tables ( #58356 )
2024-01-12 16:39:22 +01:00
Robert Schulze
8471a6c2fc
Merge remote-tracking branch 'ClickHouse/master' into materialize_non_override_past_values
2024-01-12 15:32:06 +00:00
Robert Schulze
c30736d415
Cosmetics
2024-01-12 15:31:15 +00:00
Robert Schulze
b183e1f048
Merge remote-tracking branch 'rschu1ze/master' into Improvement/inverted_index_support_match
2024-01-12 14:10:11 +00:00
Robert Schulze
d11ed921be
Some fixups
2024-01-12 14:05:19 +00:00
Anton Popov
8e88605a7c
Merge remote-tracking branch 'upstream/master' into HEAD
2024-01-12 03:55:15 +00:00
Konstantin Bogdanov
633ef126c7
Merge pull request #57550 from ClickHouse/nickitat-patch-15
...
Small change in log message in MergeTreeDataMergerMutator
2024-01-12 02:02:10 +03:00
Anton Popov
eb6647d4e8
fix reading of offsets subcolumn from Nested
2024-01-11 21:55:22 +00:00
Robert Schulze
790f5890a0
Merge pull request #58611 from rschu1ze/qc-isolation
...
Improve isolation of query cache entries under re-created users or role switches
2024-01-11 17:30:24 +01:00
Nikolai Kochetov
f83cf1a6b7
Merge pull request #58638 from amosbird/fix-58620
...
Fix broken partition key analysis when doing projection optimization
2024-01-11 14:17:48 +01:00
Nikita Mikhaylov
baf4e5e894
Added comments for system tables ( #48350 )
2024-01-11 12:50:00 +01:00
Duc Canh Le
6331d8a6f2
Merge branch 'master' into final_no_copy
...
Resolve conflicts
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
2024-01-11 02:55:14 +00:00
Igor Nikonov
8d4a395afa
Merge remote-tracking branch 'origin/master' into pr-custom-key-failover
2024-01-10 20:28:04 +00:00
Robert Schulze
52d75ab68f
Merge pull request #58458 from rschu1ze/replace-std_regexp-by-re2
...
Replace `std::regex` by re2
2024-01-10 17:46:54 +01:00
Kseniia Sumarokova
a6b5b7460b
Merge branch 'master' into broken-projections-better-handling
2024-01-10 16:46:18 +01:00