Raúl Marín
e168329ade
Revert "ReplicatedMergeTree invalid metadata_version fix"
2024-02-16 11:44:16 +01:00
Alexey Milovidov
2f36c9e965
Maybe less memory usage for primary keys in memory
2024-02-16 00:56:24 +01:00
Alexander Tokmakov
22c97be2a4
ability to detect undead ZooKeeper sessions
2024-02-15 22:17:31 +01:00
kssenii
2a783321d7
Fix deletion of s3 queue
2024-02-15 17:56:38 +01:00
Azat Khuzhin
f10fc95933
Fix INSERT into SQLite with single quote
...
Previously it leads to syntax error, due to incorrect escaping of single
quotes for SQLite, "\'" had been used instead of "''"
So set output_format_values_escape_quote_with_quote=true for SQLite to
fix this.
v2: prepare modified Context for writing on storage creation
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-02-15 12:47:29 +01:00
Robert Schulze
f8413481c4
Merge remote-tracking branch 'ClickHouse/master' into materialize_non_override_past_values
2024-02-15 10:29:00 +00:00
Alexey Milovidov
a98facee52
Merge pull request #59944 from ClickHouse/fewer_lambdas
...
Replace lambdas with pointers to members to simplify stacks
2024-02-15 06:02:35 +01:00
Alexey Milovidov
009d08561d
Merge pull request #59946 from kitaisreal/replicated-merge-tree-alter-update-metadata-version-in-single-transaction
...
ReplicatedMergeTree invalid metadata_version fix
2024-02-15 04:38:06 +01:00
Alexey Milovidov
681351fe1e
Merge pull request #59987 from nickitat/fix_data_race_storage_distributed
...
Fix data race in `StorageDistributed`
2024-02-15 02:55:08 +01:00
Alexey Milovidov
7a5e152d9e
Merge pull request #59967 from ClickHouse/dashboards-to-use-merge-tables
...
Use merge table in the dashboard; add Cloud overview to the system table
2024-02-14 21:56:37 +01:00
Nikita Taranov
5f87956fc8
impl
2024-02-14 16:22:14 +01:00
Maksim Kita
d4778d858d
Fixed tests
2024-02-14 18:05:29 +03:00
Maksim Kita
0cce386207
Added documentation
2024-02-14 18:05:29 +03:00
Maksim Kita
f6d211b44d
StorageSystemTables add metadata_version
2024-02-14 18:05:29 +03:00
Maksim Kita
be490291af
Updated implementation
2024-02-14 18:05:06 +03:00
Maksim Kita
dca627026c
ReplicatedMergeTree ALTER update metadata_version in single transaction
2024-02-14 18:05:06 +03:00
kssenii
8fe9ede207
Add a test
2024-02-14 15:25:43 +01:00
Nikolai Kochetov
03720d558e
Merge branch 'master' into allow-parallel-replicas-for-join-with-analyzer-2
2024-02-14 12:54:29 +01:00
Alexander Gololobov
dda7c5d2b7
Add comment with motivation
2024-02-14 11:41:07 +01:00
Alexey Milovidov
7b3e57e2d2
Use merge table in the dashboard
2024-02-14 02:37:42 +01:00
Alexey Milovidov
102e64cf1d
Merge branch 'master' of github.com:ClickHouse/ClickHouse into dashboards-to-use-merge-tables
2024-02-14 02:20:20 +01:00
Alexey Milovidov
27f115f581
Use merge table in the dashboard
2024-02-14 02:17:23 +01:00
Alexey Milovidov
7ee6b0bd87
Merge pull request #59941 from kitaisreal/register-storage-merge-tree-exception-message-fix
...
Register StorageMergeTree exception message fix
2024-02-14 01:31:00 +01:00
Alexey Milovidov
8b59977639
Merge pull request #59909 from azat/rmt-macros-cleanup
...
Petite cleanup around macros and ReplicatedMergeTree
2024-02-14 00:09:34 +01:00
Kruglov Pavel
1dbfeafb42
Merge branch 'master' into auto-format-detection
2024-02-13 19:08:33 +01:00
Alexander Gololobov
037bc043e6
Removed lambdas
2024-02-13 16:01:17 +01:00
Maksim Kita
4dc6ed3a39
Register StorageMergeTree exception message fix
2024-02-13 17:53:35 +03:00
Smita Kulkarni
d7056d642d
Addressed review comments
2024-02-13 13:53:40 +01:00
Smita Kulkarni
3526d7982c
Merge branch 'master' into Support_parameterized_view_with_analyzer
2024-02-13 13:08:35 +01:00
Kseniia Sumarokova
bb12fb45c4
Merge pull request #59873 from azat/file-log-shutdown-fix
...
Fix long shutdown of FileLog storage
2024-02-13 12:08:12 +01:00
Nikita Mikhaylov
4b85f6b39a
Revert "Less error prone interface of read buffers" ( #59911 )
2024-02-13 12:02:46 +01:00
Nikolai Kochetov
666b3d6664
Merge branch 'master' into allow-parallel-replicas-for-join-with-analyzer-2
2024-02-13 11:11:03 +01:00
Azat Khuzhin
58e5d7876b
Allow uuid in replica_path if CREATE TABLE explicitly has it
...
Right now the query with {uuid} macro without ON CLUSTER fails:
CREATE TABLE x UUID 'aaaaaaaa-1111-2222-3333-aaaaaaaaaaaa' (key Int) ENGINE = ReplicatedMergeTree('/tables/{database}/{uuid}', 'r1') ORDER BY tuple();
There is a workaround right now to use ATTACH instead of CREATE, but
ATTACH is not CREATE.
CREATE still useful for proper RESTORE without ON CLUSTER.
So this patch allows this syntax, but only if UUID had been explicitly
passed in the query. This looks safe.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-02-13 08:01:09 +01:00
Alexey Milovidov
4dc59051a1
Merge pull request #59897 from ClickHouse/s3queue-fix-uninitialized-value
...
s3queue: fix uninitialized value
2024-02-13 02:44:45 +01:00
Azat Khuzhin
c61ac1d3bc
Convert default_replica_path/default_replica_name into server settings
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-02-12 21:05:58 +01:00
kssenii
d008ee725f
Add a test
2024-02-12 20:23:21 +01:00
kssenii
0628ae6268
S3 queue fix uninitialized value
2024-02-12 17:13:30 +01:00
Azat Khuzhin
c7c05c9881
Do not pull mutations if pulling replication log had been stopped
...
Right now, mutations can be pulled even after:
SYSTEM STOP PULLING REPLICATION LOG
Since they pulled from two places:
- StorageReplicatedMergeTree::mutationsUpdatingTask()
- ReplicatedMergeTreeQueue::pullLogsToQueue()
And only the last one checks action blocker.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-02-12 16:28:08 +01:00
Nikolai Kochetov
fd2ee19f65
Follow up for #58554 . Cleanup.
2024-02-12 13:37:25 +00:00
Nikolai Kochetov
09d3b3c0d0
Merge pull request #58554 from ClickHouse/try-to-always-push-down-prewhere-from-query-plan
...
Simplify optimize-push-to-prewhere from query plan
2024-02-12 12:52:39 +01:00
Azat Khuzhin
67b73c3e49
Fix long shutdown of FileLog storage
...
Previously it was possible to wait up to
poll_directory_watch_events_backoff_max (default is 32000) on shutdown,
because it was not possible to stop poll of inotify.
Before (takes 3 seconds):
2024.02.12 11:27:55.058192 [ 10134 ] {} <Trace> StorageFileLog (file_log): Waiting for cleanup
2024.02.12 11:27:58.178021 [ 10271 ] {} <Trace> directory_watch: Execution took 7519 ms.
After:
2024.02.12 11:33:29.722403 [ 15866 ] {} <Trace> StorageFileLog (file_log): Waiting for cleanup
2024.02.12 11:33:29.722473 [ 15956 ] {} <Trace> directory_watch: Execution took 6399 ms.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-02-12 11:52:09 +01:00
Robert Schulze
06dc9ed074
Merge remote-tracking branch 'ClickHouse/master' into materialize_non_override_past_values
2024-02-11 15:48:55 +00:00
Alexander Gololobov
309db427e7
Do not reinitialize ZooKeeperWithFaultInjection on each chunk
2024-02-11 13:13:49 +01:00
Michael Kolupaev
5424f235a9
Fix StorageURL doing some of the query execution in one thread instead of max_threads
2024-02-11 02:44:18 +00:00
Anton Popov
b4449535c2
Merge pull request #59733 from CurtizJ/check-minmax-index-types
...
Validate types of arguments for `minmax` skipping index
2024-02-09 19:39:59 +01:00
Nikolai Kochetov
862c6cd79c
Merge pull request #57430 from ClickHouse/non-ready-set-ttl
...
Non ready set in TTL WHERE.
2024-02-09 16:31:26 +01:00
Igor Nikonov
effaaceb26
Merge pull request #59501 from ClickHouse/pr-better-replicas-failover-2
...
Parallel replicas: better initial replicas failover (2)
2024-02-09 15:50:56 +01:00
Dmitry Novik
2e6ca6200e
Merge pull request #59774 from ClickHouse/analyzer-merge-follow-up
...
Followup #50894
2024-02-09 15:05:19 +01:00
Vitaly Baranov
09e727d48b
Merge pull request #56988 from ClickHouse/Azure_backup
...
Backup & Restore support for AzureBlobStorage
2024-02-09 14:45:35 +01:00
Dmitry Novik
8655c11280
Fix typo
2024-02-09 11:45:15 +01:00
Igor Nikonov
84d64a6ed8
Merge remote-tracking branch 'origin/master' into pr-better-replicas-failover-2
2024-02-08 18:03:16 +00:00
kssenii
e55b60e05c
Fix
2024-02-08 17:43:16 +01:00
Dmitry Novik
5f75087178
Add comments
2024-02-08 15:48:20 +00:00
Nikolai Kochetov
73d2ff3933
Update MergeTask.cpp
2024-02-08 14:55:35 +01:00
Dmitry Novik
b8f74b5b05
Merge pull request #50894 from ClickHouse/storage-merge-aliases-analyzer
...
Analyzer: support aliases and distributed JOINs in StorageMerge
2024-02-08 11:22:00 +01:00
Igor Nikonov
3fb30a23d9
Merge remote-tracking branch 'origin/master' into pr-better-replicas-failover-2
2024-02-08 09:09:11 +00:00
Alexander Tokmakov
510f90edf8
Merge pull request #59688 from ClickHouse/revert_39507_again
...
Temporarily remove a feature that doesn't work
2024-02-08 02:27:55 +01:00
Anton Popov
9a4dbc843a
validate type of arguments for minmax secondary index
2024-02-07 17:52:17 +00:00
Dmitry Novik
813020a1a6
remove redundant logging
2024-02-07 16:44:09 +01:00
Dmitry Novik
ff4729ccc4
Improve replaceTableExpressionAndRemoveJoin function
2024-02-07 16:40:23 +01:00
Smita Kulkarni
e594608c42
Merge branch 'master' into Azure_backup
2024-02-07 13:48:46 +01:00
Nikolai Kochetov
d1902cdba0
Fix some tests.
2024-02-07 12:36:24 +01:00
Nikolai Kochetov
06d112135e
Simplify prewhere push down from query plan. Try to always use it.
2024-02-07 12:36:20 +01:00
Nikolai Kochetov
a3d1b3f502
Merge branch 'master' into non-ready-set-ttl
2024-02-07 11:29:04 +00:00
Nikolai Kochetov
c434748fc1
Merge branch 'master' into allow-parallel-replicas-for-join-with-analyzer-2
2024-02-07 12:25:11 +01:00
Raúl Marín
38c8e1b8cc
Merge pull request #59485 from Algunenano/strict_int8
...
Strict aliasing for Int8
2024-02-07 11:33:06 +01:00
Alexander Gololobov
4dcd806dfc
Merge pull request #59545 from ClickHouse/system_zookeeper_stream
...
Produce stream of chunks from system.zookeeper instead of accumulating the whole result
2024-02-07 10:17:38 +01:00
Alexander Tokmakov
77e4b0c8d9
Revert "Merge pull request #59122 from arthurpassos/bring_back_attach_partition_pr"
...
This reverts commit ac90637170
, reversing
changes made to fe2b5cde05
.
2024-02-07 02:52:43 +01:00
Igor Nikonov
0c00435cea
Merge remote-tracking branch 'origin/master' into pr-better-replicas-failover-2
2024-02-07 00:52:50 +00:00
Alexey Milovidov
01c8542462
Merge pull request #59606 from vitlibar/fix-distributed-table-with-const-sharding-key
...
Fix distributed table with a constant sharding key
2024-02-06 22:10:34 +01:00
Raúl Marín
ff63a1cf9f
Fix tidy
2024-02-06 20:34:12 +01:00
Dmitry Novik
d558728281
Merge branch 'master' into storage-merge-aliases-analyzer
2024-02-06 20:24:20 +01:00
Dmitry Novik
1abcf26df6
Fix filter expressions
2024-02-06 15:39:52 +00:00
Anton Popov
addbd78341
Merge pull request #59436 from nickitat/perf_introspection_for_inserts
...
More perf introspection for `INSERT`-s
2024-02-06 14:59:32 +01:00
Alexander Gololobov
ca4f46ab41
Check block size in bytes instead of rows
2024-02-06 14:27:15 +01:00
Alexander Gololobov
a26e8b3a9e
Fix for filters w/o path column
2024-02-05 22:48:48 +01:00
Vitaly Baranov
86ac7b14d0
Fix Distributed table engine with a constant sharding key.
2024-02-05 21:56:29 +01:00
Igor Nikonov
dab078f7d2
Profile events
2024-02-05 20:21:48 +00:00
Nikolai Kochetov
6b06fcf7fc
Fix more tests.
2024-02-05 17:54:15 +00:00
Nikolai Kochetov
15bf263153
Support non global in mode.
2024-02-05 17:05:21 +00:00
Nikolai Kochetov
8692e8f752
Add settings to squash external table blocks.
2024-02-05 17:05:21 +00:00
Nikolai Kochetov
1892318e46
Squash temporary tables.
2024-02-05 17:05:20 +00:00
Nikolai Kochetov
b60228af3f
Allow to send a chain of subqueries for parallel replicas with analyzer.
2024-02-05 17:05:20 +00:00
Kseniia Sumarokova
83515e3656
Merge pull request #59543 from nickolaj-jepsen/master
...
Add support for NATS credentials file
2024-02-05 12:20:26 +01:00
Nikita Mikhaylov
6ba35b5459
Make comments for system tables also available in local mode and enforce comments not to be empty ( #59493 )
...
Co-authored-by: Alexey Milovidov <milovidov@clickhouse.com>
2024-02-05 11:17:58 +01:00
alesapin
db046c1bce
Merge pull request #59517 from ClickHouse/remove_scarry_message
...
Remove a scary message if an error is retryable
2024-02-04 17:43:06 +01:00
MikhailBurdukov
b90a5b94a8
Move setting to mergetree
2024-02-04 11:24:26 +00:00
Smita Kulkarni
b0994c5fa7
Addressed comments, added test for named collection
2024-02-04 11:28:20 +01:00
Alexander Gololobov
82caaa744a
Produce stream of chunks instead of accumulating the whole result
2024-02-03 16:08:41 +01:00
Nickolaj Jepsen
e822ba3a47
Add support for NATS credentials file
2024-02-02 23:34:01 +01:00
Dmitry Novik
05a7d22c24
WIP on calculate aliases only once
2024-02-02 13:45:04 +00:00
Alexander Sapin
d5e3b71590
Remove scarry message if error is retryable
2024-02-02 14:07:37 +01:00
Raúl Marín
87d493533c
Revert "Poco Logger small refactoring"
2024-02-02 12:10:05 +01:00
Alexander Gololobov
095648d7bc
Merge pull request #59388 from ClickHouse/system_zookeeper_retries
...
Retry disconnects and expired sessions when reading system.zookeeper
2024-02-02 11:52:15 +01:00
serxa
1d31ac2a0c
add FORGET PARTITION
query to remove old parition nodes from ZooKeeper
2024-02-02 10:03:31 +00:00
Alexey Milovidov
15467f2efb
Merge pull request #59451 from ClickHouse/remove-unused-headers
...
CLion says these headers are unused
2024-02-01 21:47:17 +01:00
Nikita Taranov
d73abc17f5
better
2024-02-01 21:42:22 +01:00
Nikita Taranov
698d00dbfe
add to MutateTask
2024-02-01 20:43:43 +01:00
Nikita Taranov
76f2ae08b5
collect stats build time
2024-02-01 18:16:09 +01:00
Nikita Taranov
4795149863
print stats once
2024-02-01 17:16:42 +01:00
Kseniia Sumarokova
e029021696
Merge pull request #59446 from ClickHouse/s3-queue-allow-to-add-start-point
...
Allow to define a starting point for s3queue ordered mode at creation
2024-02-01 16:46:24 +01:00
Alexey Milovidov
f52b08070f
Merge branch 'master' into remove-unused-headers
2024-02-01 13:42:37 +01:00
kssenii
3a16427e00
Fix test
2024-02-01 12:35:01 +01:00
Maksim Kita
ecbea6f625
Updated implementation
2024-02-01 12:51:50 +03:00
Kseniia Sumarokova
b3418c506e
Update src/Storages/S3Queue/S3QueueSettings.h
...
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
2024-02-01 10:28:33 +01:00
Dmitry Novik
5b630ed745
Merge pull request #59399 from azat/build/format-fwd-decl
...
Forward declaration for PeekableReadBuffer
2024-02-01 09:15:07 +01:00
Alexey Milovidov
33f6940214
CLion says these headers are unused
2024-02-01 03:50:09 +01:00
kssenii
361b2f107b
Allow to define a starting point for s3queue ordered mode
2024-01-31 21:28:18 +01:00
Alexander Gololobov
5947e5b5a8
Use retry settings and fault injection
2024-01-31 16:48:46 +01:00
Dmitry Novik
572d47acc7
Remove logging
2024-01-31 14:43:06 +00:00
Dmitry Novik
0d21004218
WIP on StorageMerge
2024-01-31 13:50:15 +00:00
Kseniia Sumarokova
b20567a055
Merge pull request #59167 from ClickHouse/s3-queue-parallelize-ordered-mode
...
S3Queue: allow parallel & disrtibuted processing for ordered mode
2024-01-31 11:20:41 +01:00
Kseniia Sumarokova
caaafbcedb
Merge pull request #56864 from ClickHouse/broken-projections-better-handling
...
Do not consider data part as broken if projection is broken
2024-01-31 11:17:15 +01:00
Dmitry Novik
ac17ad1e66
Merge pull request #59073 from ClickHouse/fix-analyzer-resolve-distr-id
...
Analyzer: Do not resolve remote table id on initiator
2024-01-31 09:08:30 +01:00
Nikita Taranov
1be8b61c5b
add test
2024-01-30 21:52:32 +01:00
Dmitry Novik
722f3db738
Merge remote-tracking branch 'origin/master' into storage-merge-aliases-analyzer
2024-01-30 19:50:01 +00:00
Nikita Taranov
17ab2674f4
impl
2024-01-30 20:35:10 +01:00
Azat Khuzhin
010f1c6cb7
Forward declaration for PeekableReadBuffer
...
ReadHelpers.h is very common header and is the root cause of "recompile
everything".
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-01-30 19:24:19 +01:00
Alexander Gololobov
c348c4e828
Move ZooKeeperRetries.h to Common
2024-01-30 19:02:17 +01:00
Alexey Milovidov
4ddba907c8
Update autogenerated version to 24.2.1.1 and contributors
2024-01-30 18:49:28 +01:00
Alexander Gololobov
82c06ca294
Use ZooKeeperRetriesControl
2024-01-30 18:31:37 +01:00
Alexander Gololobov
023b8cbd53
Retry disconnects and expired sessions
2024-01-30 17:47:11 +01:00
kssenii
145f6a31cb
Fix
2024-01-30 16:29:40 +01:00
MikhailBurdukov
778efb8953
Merge branch 'master' into backoff_for_failed_mutations
2024-01-30 16:08:52 +03:00
MikhailBurdukov
5e845172da
Enable per query setting
2024-01-30 12:42:03 +00:00
Kruglov Pavel
3105960fd8
Merge pull request #59352 from Avogar/fix-bad-attached-rabbitmq
...
Fix lazy initialization in RabbitMQ
2024-01-30 12:52:07 +01:00
Kseniia Sumarokova
e16aaa8d81
Merge branch 'master' into broken-projections-better-handling
2024-01-30 10:54:22 +01:00
Dmitry Novik
5a34641e91
Merge remote-tracking branch 'origin/master' into fix-analyzer-resolve-distr-id
2024-01-30 07:17:15 +00:00
Alexey Milovidov
35ea956255
Merge pull request #59295 from ClickHouse/fix_block_number
...
Fix `_block_number` column with non-default serializations and mutation
2024-01-30 04:07:09 +01:00
Alexey Milovidov
ac90637170
Merge pull request #59122 from arthurpassos/bring_back_attach_partition_pr
...
Revert revert attach partition PR
2024-01-30 04:02:31 +01:00
Alexey Milovidov
de341a6d6e
Merge pull request #59351 from ClickHouse/fix-non-redy-set-system-tables
...
Fix not-ready set for system.tables
2024-01-30 03:59:18 +01:00
Alexey Milovidov
50532c485f
Merge pull request #59340 from CurtizJ/lower-memory-usage-vertical-merges
...
Lower memory usage in vertical merges
2024-01-30 03:42:28 +01:00
avogar
3a5ba56c36
Fix lazy initialization in RabbitMQ, fix possible deadlock on insert into unitialized queue engine
2024-01-29 20:09:09 +00:00
Robert Schulze
2cfcaf8ff4
Merge pull request #59040 from MochiXu/fix_remove_inverted_index_files
...
`DROP INDEX` of inverted index now removes all relevant files from persistence
2024-01-29 20:39:56 +01:00
kssenii
df7d3b8a40
Allocate current shard number from keeper
2024-01-29 19:36:16 +01:00
Arthur Passos
15d04d0a78
Merge branch 'master' into bring_back_attach_partition_pr
2024-01-29 15:16:49 -03:00
Arthur Passos
516daadeef
add static storage assert in datapartcloner
2024-01-29 15:16:00 -03:00
Nikolai Kochetov
d0143380d7
Fix not-ready set for system.tables
2024-01-29 17:53:59 +00:00
Nikolai Kochetov
5b2648929b
Merge pull request #59273 from kitaisreal/atomic-logger
...
Added AtomicLogger
2024-01-29 16:49:18 +01:00
avogar
377937d415
Merge branch 'master' of github.com:ClickHouse/ClickHouse into auto-format-detection
2024-01-29 15:45:18 +00:00
Anton Popov
d3cdc8826e
lower memory usage in vertical merges
2024-01-29 14:06:46 +00:00
kssenii
d0ea5392db
Fix storing iterated keys
2024-01-29 12:36:32 +01:00
kssenii
ef252d8376
Validate settings with keeper, add a test
2024-01-29 12:36:00 +01:00
Kruglov Pavel
6858d2f4ca
Merge pull request #58047 from Avogar/variant-data-type
...
Implement Variant data type
2024-01-29 11:36:08 +01:00
Smita Kulkarni
3183026521
Merge branch 'master' into Azure_backup
2024-01-29 11:08:13 +01:00
Kseniia Sumarokova
a29ab8f29c
Merge pull request #59170 from azat/disks/s3-plain-improvements
...
Small improvements for tables on write-once (s3_plain)/read-only (web) disks
2024-01-29 10:44:37 +01:00
Kseniia Sumarokova
0f88da5fd4
Merge pull request #59275 from ClickHouse/Avogar-patch-1
...
Fix abort in iceberg metadata on bad file paths
2024-01-29 10:35:34 +01:00
alesapin
b864b122fa
Merge branch 'master' into fix_block_number
2024-01-29 10:19:43 +01:00
Alexey Milovidov
2cb2bcfbc3
Merge pull request #59315 from ClickHouse/support-backups-for-compressed-memory-tables
...
Support backups for compressed in-memory tables
2024-01-29 01:50:41 +01:00
Alexey Milovidov
aec3f28ccb
Support backups for compressed in-memory tables
2024-01-28 23:06:50 +01:00